convoy docs
Search

Deletion

What convoy does when you delete a Release or a Cluster.

When you delete a Release, convoy deletes the objects of the Release from the target cluster before the API server deletes the Release. A finalizer holds the Release until this cleanup is done. spec.deletionPolicy and the delete-phase lifecycle hooks control the cleanup.

convoy adds the finalizer release.core.convoy.kumodo.io/prune-workloads at the first reconcile of each Release.

Deletion flow Pick a scenario, or change the policy, the hooks and the cluster. The table shows one pass of the Release finalizer.
StepWhat convoy doesSkipped whenIf the step fails
1Checks that the target cluster is reachablekeepWorkloads: true and no delete hooksWaits with blockOnClusterUnreachable: true. Else skips steps 2 to 4.
2Runs the preDelete hooksNo preDelete hooksBlocks with onFailure: Fail
3Deletes the objects in the ReleaseInventorykeepWorkloads: trueRetries
4Runs the postDelete hooksNo postDelete hooksContinues. convoy logs the failure.
5Deletes the ReleaseInventoryNeverRetries
6Deletes the renders of the Release from the render storeNo render storeRetries
7Clears the finalizerNeverRetries

The steps of the Release finalizer, in order.

Check the target cluster#

convoy resolves the Cluster only when it must delete objects or run delete hooks. It reads status.reachable of the Cluster. It does not connect to the cluster for this check.

State of the ClusterblockOnClusterUnreachable: trueblockOnClusterUnreachable: false
ReachableSteps 2 to 4 run.Steps 2 to 4 run.
Not reachableThe deletion waits. convoy retries.convoy skips steps 2 to 4.
Does not existconvoy skips steps 2 to 4.convoy skips steps 2 to 4.

When convoy skips steps 2 to 4, the objects stay on the cluster. convoy still deletes the ReleaseInventory, so it does not know about these objects after the deletion.

Delete hooks#

convoy runs the preDelete and postDelete hooks as Jobs or Pods on the target cluster. While a hook runs, convoy checks it again every 10 seconds. The observer also wakes the controller when a batch of hooks completes. See Lifecycle hooks.

  • preDelete. A failed hook with onFailure: Fail blocks the deletion. The finalizer stays, and convoy does not delete objects. A failed hook with onFailure: Continue does not block.
  • postDelete. These hooks run after convoy deleted the objects. A failed hook does not block the deletion, for each onFailure value.
releases/release-gitrepository-helm-in-cluster-with-deletion-lifecycle.yaml
apiVersion: core.convoy.kumodo.io/v1alpha1
kind: Release
metadata:
  name: podinfo-in-cluster-helm-with-deletion-lifecycle
  namespace: default
lifecycle:
  preDelete:
    - name: pre-sleep
      type: Job
      timeout: 30s
      onFailure: Continue
      cleanup: OnSuccess
      jobTemplate:
        metadata:
          labels:
            test-label: "true"
          annotations:
            test-annotation: "true"
        spec:
          template:
            spec:
              restartPolicy: Never
              containers:
                - name: job-worker
                  image: "busybox:1.38.0"
                  command: ["sh", "-c", "sleep 10"]
  postDelete:
    - name: post-sleep
      type: Job
      timeout: 30s
      onFailure: Continue
      cleanup: OnSuccess
      jobTemplate:
        metadata:
          labels:
            test-label: "true"
          annotations:
            test-annotation: "true"
        spec:
          template:
            spec:
              restartPolicy: Never
              containers:
                - name: job-worker
                  image: "busybox:1.38.0"
                  command: ["sh", "-c", "sleep 10"]
mutations:
  - type: syncWave
    waveStrategy:
      strategy: waitForHealthy
      wave: 1
    target:
      group: ""
      version: v1
      kind: Service
spec:
  clusterRef:
    name: dev-local
  source:
    type: gitRepository
    gitRepository:
      name: podinfo
      namespace: default
      ref:
        branch: master
  content:
    type: helm
    helm:
      path: charts/podinfo
      releaseName: podinfo-helm
      values:
        - type: inline
          inline:
            probes:
              readiness:
                initialDelaySeconds: 15
              liveness:
                initialDelaySeconds: 15
  syncPolicy:
    automatic: true
    prune:
      orphaned: false
      includeCRDs: false
      includeNamespaces: false
    interval: 1m
  healthAssessment:
    globalRuleSetSelector:
      matchLabels:
        convoy.kumodo.io/default: "true"
    deadline: 1m

Delete the objects#

With keepWorkloads: false (the default), convoy deletes each object that the ReleaseInventory records. An object that is already absent does not cause an error.

The inventory contains only the objects that convoy manages now. An object that a sync with prune.orphaned: false dropped from the inventory stays on the cluster. See Apply and prune.

With keepWorkloads: true, the objects stay on the cluster. The delete hooks still run.

Delete the convoy data#

After the objects and the hooks, convoy deletes its own data for the Release:

  1. The ReleaseInventory.
  2. All renders and the history of the Release in the render store, if a render store is configured.
  3. The finalizer. The API server then deletes the Release.

convoy does each step before it clears the finalizer. If a step fails, the finalizer stays and convoy retries. So no ReleaseInventory or stored render stays without its Release.

A failed step in the deletion flow retries with the default rate limiter of the controller. The retryBackoff policy does not apply. During the deletion, the observer stops the drift and health checks of the Release. It continues to watch the delete hooks.

Delete a Cluster#

convoy adds the finalizer cluster.core.convoy.kumodo.io/prune-releases to each Cluster. spec.deletionPolicy.pruneReleases controls what occurs to the Releases that target it:

  • false (the default). convoy deletes the Cluster at once. The Releases stay, and their Applied condition is False with reason ClusterNotReady.
  • true. convoy deletes each Release in the namespace of the Cluster that references it. The Cluster stays until the API server deletes all those Releases. Each Release runs its own deletion flow, with its own deletionPolicy.

With pruneReleases: true, the Cluster still exists while its Releases run their deletion flow. So the Releases can delete their objects from the cluster. See Cluster.

In the code

convoy v0.9.0