Advanced Helm Techniques: Utilizing Hooks for Efficient Deployments

Helm allows chart developer, an opportunity to perform operations at strategic points in a release lifecycle. It comes with 2 special prefix called pre and post.

Pre: It means "before" or "prior to" an event.
Post: It means "after" or "following" an event.

Annotation	Value
pre-install	Executes after templates are rendered, but before any resources are created in Kubernetes
post-install	Executes after all resources are loaded into Kubernetes
pre-delete	Executes on a deletion request before any resources are deleted from Kubernetes
post-delete	Executes on a deletion request after all of the release's resources have been deleted
pre-upgrade	Executes on an upgrade request after templates are rendered, but before any resources are updated
post-upgrade	Executes on an upgrade request after all resources have been upgraded
pre-rollback	Executes on a rollback request after templates are rendered, but before any resources are rolled back
post-rollback	Executes on a rollback request after all resources have been modified
test	Executes when the Helm test subcommand is invoked

Writing a Hook

Hooks are just Kubernetes manifest files with special annotations in the metadata section.

apiVersion: batch/v1
kind: Job
metadata:
  name: "{{ .Release.Name }}"
  labels:
    app.kubernetes.io/managed-by: {{ .Release.Service | quote }}
    app.kubernetes.io/instance: {{ .Release.Name | quote }}
    app.kubernetes.io/version: {{ .Chart.AppVersion }}
    helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "1"
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded, hook-failed
spec:
  template:
    metadata:
      name: "{{ .Release.Name }}"
      labels:
        app.kubernetes.io/managed-by: {{ .Release.Service | quote }}
        app.kubernetes.io/instance: {{ .Release.Name | quote }}
        helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
    spec:
      restartPolicy: Never
      containers:
      - name: post-install-job
        image: "alpine:3.3"
        command: ["/bin/sleep","{{ default "10" .Values.sleepyTime }}"]

Deep Dive for the template above

What makes this template a hook is the annotation below and one resource can implement multiple hooks:

annotations:
  "helm.sh/hook": post-install, post-upgrade

It is possible to define a weight for a hook which will help build a deterministic executing order. Hook weights can be positive or negative numbers but must be represented as strings. When Helm starts the execution cycle of hooks of a particular Kind it will sort those hooks in ascending order.

Weights are defined using the following annotation:

annotations:
  "helm.sh/hook-weight": "5"

Hook deletion policies It is possible to define policies that determine when to delete corresponding hook resources. Hook deletion policies are defined using the following annotation:

annotations:
  "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded

Annotation Value	Description
before-hook-creation	Delete the previous resource before a new hook is launched (default)
hook-succeeded	Delete the resource after the hook is successfully executed
hook-failed	Delete the resource if the hook failed during execution

Hands-on Experience

Create Two Tables (User and Sales) in a Database
Update the User Table and Add New Columns
Backup the Updated Data to S3
Rollback the Data to the state before Step 2 in case of an erroneous task by a developer.
Back Up the DB before deletion, the application is going to be discontinued

Hook	Task
pre-install	Create Two Tables (User and Sales) in a Database
pre-upgrade, post-upgrade	Update the User Table and Add New Columns ,Backup the Updated Data to S3
pre-rollback	Rollback the Data after an erroneous task by a developer
pre-delete ,post-delete	Back Up the DB before deletion, the application is going to be discontinued

STEP 1 ( pre-install,post-install )

import os
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = os.getenv("SQLALCHEMY_DATABASE_URI")
db = SQLAlchemy(app)
migrate = Migrate(app, db)

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(100))
    email = db.Column(db.String(100))

class Sales(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    user_id = db.Column(db.Integer, db.ForeignKey('user.id'), nullable=False)
    item = db.Column(db.String(100))
    price = db.Column(db.Numeric)

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip install Flask==2.0.1 Flask-SQLAlchemy==2.5.1 Flask-Migrate==3.1.0

COPY . .

CMD ["flask", "run", "--host=0.0.0.0", "--port=5000"]

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}
spec:
  replicas: 1
  selector:
    matchLabels:
      app: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app: {{ .Release.Name }}
    spec:
      containers:
        - name: flask-app
          image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
          ports:
            - containerPort: 5000
          env:
            - name: SQLALCHEMY_DATABASE_URI
              value: {{ .Values.database.uri }}

---
apiVersion: v1
kind: Service
metadata:
  name: {{ .Release.Name }}
spec:
  selector:
    app: {{ .Release.Name }}
  ports:
    - protocol: TCP
      port: 80
      targetPort: 5000

apiVersion: batch/v1
kind: Job
metadata:
  name: pre-upgrade-migrate
  annotations:
    "helm.sh/hook": pre-install, pre-upgrade
     "helm.sh/hook-weight": "1"
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
spec:
  template:
    spec:
      containers:
        - name: table-creation-migration
          image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
          command: ["/bin/sh"]
          args:
            - "-c"
            - |
              flask db init
              flask db migrate -m "initial migration"
              flask db upgrade
          env:
            - name: SQLALCHEMY_DATABASE_URI
              value: {{ .Values.database.uri }}
      restartPolicy: OnFailure

STEP 2 (Update the User Table and Add New Columns ,Backup the Updated Data to S3)

import os
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = os.getenv("SQLALCHEMY_DATABASE_URI")
db = SQLAlchemy(app)
migrate = Migrate(app, db)

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    # existing code .......
    # Adding new columns
    age = db.Column(db.Integer)
    address = db.Column(db.String(255))


class Sales(db.Model):
    # existing code .......

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

apiVersion: batch/v1
kind: Job
metadata:
  name: add-new-columns
  annotations:
    "helm.sh/hook": pre-upgrade
    "helm.sh/hook-delete-policy": hook-succeeded
spec:
  template:
    spec:
      containers:
        - name: pre-upgrade-migrate
          image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
          command: ["/bin/sh"]
          args:
            - "-c"
            - |
              flask db migrate -m "Add age and address to User model"
              flask db upgrade
          env:
            - name: SQLALCHEMY_DATABASE_URI
              value: {{ .Values.database.uri }}
      restartPolicy: OnFailure

apiVersion: batch/v1
kind: Job
metadata:
  name: backup-after-upgrade
  annotations:
    "helm.sh/hook": post-upgrade
    "helm.sh/hook-delete-policy": hook-succeeded
spec:
  template:
    spec:
      containers:
        - name: backup-before-update
          image: amazon/aws-cli
          env:
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: aws-secret
                  key: access-key-id
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: aws-secret
                  key: secret-access-key
            - name: POSTGRES_ADMIN_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: db-secret
                  key: password
            - name: BUCKET
              value: "your-s3-bucket"
          command: ["/bin/sh"]
          args:
            - "-c"
            - |
              export DB_NAME="<DB_NAME>$(date +%B%Y).dump"
              kubectl exec -it postgresql-0 -n <namespace> -- \
                env PGPASSWORD=$POSTGRES_ADMIN_PASSWORD \
                PGCONNECT_TIMEOUT=1800 pg_dump -U postgres <DB_NAME> > $DB_NAME
              python3 -m venv env && source env/bin/activate && \
              pip install --upgrade pip && pip install pyyaml awscli && \
              aws s3 cp $DB_NAME $BUCKET/$DB_NAME
              deactivate env && rm -rf env
      restartPolicy: OnFailure

STEP 3 (Rollback the Data after an erroneous task by a developer)

apiVersion: batch/v1
kind: Job
metadata:
  name: revert-back-to-previous-column
  annotations:
    "helm.sh/hook": pre-rollback
    "helm.sh/hook-delete-policy": hook-succeeded, hook-failed
spec:
  template:
    spec:
      containers:
        - name: pre-upgrade-migrate
          image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
          command: ["/bin/sh"]
          args:
            - "-c"
            - |
              flask db downgrade
          env:
            - name: SQLALCHEMY_DATABASE_URI
              value: {{ .Values.database.uri }}
      restartPolicy: OnFailure

STEP 4 (Back Up the DB before deletion, the application is going to be discontinued)

apiVersion: batch/v1
kind: Job
metadata:
  name: backup-after-upgrade
  annotations:
    "helm.sh/hook": pre-delete
    "helm.sh/hook-delete-policy": hook-succeeded
spec:
  template:
    spec:
      containers:
        - name: backup-before-update
          image: amazon/aws-cli
          env:
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: aws-secret
                  key: access-key-id
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: aws-secret
                  key: secret-access-key
            - name: POSTGRES_ADMIN_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: db-secret
                  key: password
            - name: BUCKET
              value: "your-s3-bucket"
          command: ["/bin/sh"]
          args:
            - "-c"
            - |
              export DB_NAME="<DB_NAME>$(date +%B%Y).dump"
              kubectl exec -it postgresql-0 -n <namespace> -- \
                env PGPASSWORD=$POSTGRES_ADMIN_PASSWORD \
                PGCONNECT_TIMEOUT=1800 pg_dump -U postgres <DB_NAME> > $DB_NAME
              python3 -m venv env && source env/bin/activate && \
              pip install --upgrade pip && pip install pyyaml awscli && \
              aws s3 cp $DB_NAME $BUCKET/$DB_NAME
              deactivate env && rm -rf env
      restartPolicy: OnFailure

apiVersion: batch/v1
kind: Job
metadata:
  name: "post-delete-drop-tables"
  annotations:
    "helm.sh/hook": post-delete
spec:
  template:
    spec:
      containers:
        - name: post-delete-drop-tables
          image: "your-docker-repo/your-image:latest"
          command: ["/bin/sh", "-c"]
          args:
            - |
              kubectl exec -it postgresql-0 -n infra -- \
                env PGPASSWORD="yourpassword" \
                psql -U postgres -d yourdatabase -c "DROP TABLE IF EXISTS User;"
              kubectl exec -it postgresql-0 -n infra -- \
                env PGPASSWORD="yourpassword" \
                psql -U postgres -d yourdatabase -c "DROP TABLE IF EXISTS Sales;"
      restartPolicy: Never

Advanced Helm Techniques: Utilizing Hooks for Efficient Deployments

Writing a Hook​

Deep Dive for the template above​

Hands-on Experience​

STEP 1 ( pre-install,post-install )​

STEP 2 (Update the User Table and Add New Columns ,Backup the Updated Data to S3)​

STEP 3 (Rollback the Data after an erroneous task by a developer)​

STEP 4 (Back Up the DB before deletion, the application is going to be discontinued)​