Status ownership
Which component writes each status field and each ReleaseInventory field, and the Server-Side Apply field manager that it uses.
Three loops in two processes write the status of a Release. Each field has one writer. Each
writer uses Server-Side Apply (SSA) with its own field manager and sends only the fields that it
owns. So one writer never overwrites the fields of another writer. The same rule applies to the
ReleaseInventory, which has four writers.
Release status#
All three writers send an apply patch to releases/<name>/status with force: true.
| Field manager | Process and loop | Conditions | Other fields |
|---|---|---|---|
convoy-release | controller-manager, Release controller | Applied, RolloutSucceeded, LifecycleSucceeded | observedGeneration, lastAttempted, lastSuccessful, lastHealthy, rollout, pin, waves, lifecycle, resolvedRetentionPolicy, reconciledAt, lastHandledSyncAt, lastHandledForceSyncAt |
convoy-observe | observer, drift loop | InSync | none |
convoy-health | observer, health loop | Healthy | healthSummary |
status.conditions is a list map with the key type. A writer that sends only its own
condition types does not change the other conditions. See
Status and conditions.
How the controller builds its patch#
The Release controller copies the full status from the informer cache, then changes it.
ownedStatusApplyConfig builds the patch field by field from this copy:
- It sends only the three conditions that it owns. The copy also contains
InSyncandHealthy, but the patch does not contain them. - It does not serialize the full
ReleaseStatusstruct. Some fields of that struct, for examplehealthSummary, have noomitempty. A full serialization would send zero values for fields of other writers. - It sends
pin,lifecycleandLifecycleSucceededonly when they have a value. When the field manager omits a field that it sent before, SSA clears the field. This is how an unpin clearsstatus.pin. - It always sends
resolvedRetentionPolicy.
status.waves[].healthy comes from the observer, but the controller writes it. The controller
reads the per-object health from the ReleaseInventory and calculates the health of each wave.
So status.waves has one writer.
When a writer skips the write#
- The controller compares the new status with the cached status (
statusChanged). It writes only when they are different. It setsreconciledAtonly when it writes. writeInSyncandwriteHealthskip the write when status, reason, message and observed generation of the condition do not change.writeHealthalso compareshealthSummary.writeInSyncandwriteHealthkeeplastTransitionTimewhile the condition status stays the same.
No Ready condition#
A Release has no Ready condition. A Ready condition would combine the signals of different
writers into one field. Then that field would have more than one writer. The four signals stay
separate:
Appliedtells what the controller did.InSynctells whether the live objects match the render.Healthytells whether the live objects pass the health rules.RolloutSucceededtells the result of the rollout.
A Release can have Applied=True, InSync=True and Healthy=False at the same time. See
Release conditions.
ReleaseInventory#
The ReleaseInventory has no status subresource. The writers send apply patches to the object
itself, with force: true. The inventory list is a list map with the keys group, kind,
namespace and name. The hooks list is a list map with the keys name and phase. So two
field managers can own different fields of the same entry. Each writer of an entry must send the
keys of the entry. Thus the keys have two owners.
| Field manager | Process | Owns |
|---|---|---|
convoy-release-inventory | controller-manager | The shard label. In each inventory entry: the keys, version, wave, uid, renderRef |
convoy-release-hooks | controller-manager | The shard label. In each hooks entry: the keys, weight, group, version, kind, workloadName, workloadNamespace, renderRef, createdAt, timeout |
convoy-health-inventory | observer, health loop | In each inventory entry: the keys, state, message, firstUnhealthy, assessedRenderRef |
convoy-hook-status | observer, health loop | In each hooks entry: the keys, state, message, finishedAt |
The controller uses two field managers on purpose. Put sends inventory without hooks, and
PutHooks sends hooks without inventory. With one field manager, each call would clear the
list that the other call wrote.
The health loop sends every hook entry that has a state, not only the entries that it polled in
this pass. If it omitted an entry, SSA would clear finishedAt from that entry. Then the hook
would look unfinished again.
Put writes the full membership. The health loop reads the inventory from its cache before it
writes. See ReleaseInventory.
Target clusters#
| Field manager | Process | Operation |
|---|---|---|
convoy | controller-manager | The apply of each render object, with force: true. The pre-apply dry run. The create of a lifecycle hook workload. |
convoy | observer, drift loop | A server-side apply dry run of each desired object, with force: true. A dry run does not store field ownership. |
The drift loop uses the same field manager as the apply. So the dry run shows exactly what a real apply would change. See Apply and prune and Drift.
Writes without SSA#
Some writes do not use SSA:
- The
Cluster,GitRepository,HelmRegistryandOCIRegistrycontrollers write status withUpdateStatus. They set no field manager. Each kind has one writer, so a field manager is not necessary. - The
ReleaseandClustercontrollers add and clear their finalizers withUpdateon the full object. - The apiserver sets the
convoy.kumodo.io/sync-requested-atannotation on a/sync,/pin,/mutationsor/lifecyclerequest. The/synchandler reads the Release and updates it in etcd. - The
Releasecontroller deletes theReleaseInventorywhen it finalizes a Release.
Status writes do not start a reconcile#
All convoy loops use the harness in internal/controller/base. On an informer update event,
shouldEnqueueOnUpdate enqueues the object only in these cases:
- The resource version did not change. This is a periodic resync.
metadata.generationchanged. The apiserver increments the generation of a Release only whenspecchanges.- The
Releasecontroller only: the annotationconvoy.kumodo.io/sync-requested-atchanged.
A status write changes the resource version but not the generation. So a status write of one loop does not wake the same loop or the other loops on the Release informer.
Two event handlers in the controller-manager react to status on purpose:
- A change of the
AppliedorHealthycondition status enqueues the Releases of a higher cluster wave on the same cluster. It does not enqueue the Release itself. - A change of
stateorassessedRenderRefin aReleaseInventoryentry enqueues the Release. A change in the number of entries also enqueues it. The controller ignores its own fieldsrenderRefandwave.
The observer has a similar handler on the ReleaseInventory. See Reconcile flow
and Health loop.
In the code
convoy v0.9.0
internal/controller/release.ownedStatusApplyConfiginternal/controller/release/controller.go:385internal/controller/release.(*Controller).syncHandlerinternal/controller/release/controller.go:199internal/observer.(*Controller).writeInSyncinternal/observer/status.go:49internal/observer.(*HealthController).writeHealthinternal/observer/healthstatus.go:58internal/inventory.(*apiStore).Putinternal/inventory/apistore.go:95internal/inventory.(*apiStore).PutHooksinternal/inventory/apistore.go:131internal/observer.(*HealthController).writeInventoryHealthinternal/observer/inventoryhealth.go:69internal/observer.(*HealthController).writeHookStatusinternal/observer/hooks.go:229internal/controller/base.(*Controller).shouldEnqueueOnUpdateinternal/controller/base/controller.go:159