GitOps guidance

Use this page for Argo CD or Flux-managed Bloodraven installs. It covers resource ownership, sync order, custom health checks, and upgrade sequencing.

CRD ownership

Pick one CRD owner:

Do not manage the same CRDs in both places.

Whichever owner you pick must apply the CRDs server-side. The MysqlFailoverGroup CRD exceeds the 256 KiB last-applied-configuration annotation limit of client-side apply. In Argo CD, set the ServerSideApply=true sync option on the application that owns the CRDs; with plain kubectl, use kubectl apply --server-side --force-conflicts.

Sync waves

Recommended order:

1. external-dns DNSEndpoint CRD and controller.
2. cert-manager CRDs and issuers when TLS is enabled.
3. Bloodraven CRDs.
4. Bloodraven operator Helm release.
5. Tenant namespaces and Secrets.
6. MysqlFailoverGroup resources.
7. Monitoring dashboards and alert rules.

Argo CD health checks

Argo CD does not know how to evaluate MysqlFailoverGroup or MysqlBackup custom resources by default. Without custom health checks, the resources show Unknown in the Argo CD UI.

Bloodraven ships Lua health scripts in charts/bloodraven/argocd/ that map custom resource (CR) status to Argo CD health:

The rows of each CR are evaluated top to bottom and the first match wins. The generation check sits below the MysqlFailoverGroup Degraded rows, so a hold, a Degraded condition, a pending recovery or a stalled update shows while a spec edit waits to be applied. The MysqlFailoverGroup script reads conditions only: a planned-failover hold carried only by the bloodraven.shipstream.io/planned-failover-terminal annotation (its status write failed) is not seen, so the group can show Healthy during it.

Patch argocd-cm directly

Extract the chart and apply the patch file:

helm pull bloodraven/bloodraven --untar
kubectl patch configmap argocd-cm -n argocd \
  --patch-file bloodraven/argocd/argocd-cm-patch.yaml
kubectl rollout restart deployment argocd-application-controller -n argocd

Use Argo CD Helm chart values

If you deploy Argo CD with its Helm chart, extract the Lua scripts from the Bloodraven chart:

helm pull bloodraven/bloodraven --untar

Then copy the contents of bloodraven/argocd/health-mysqlfailovergroup.lua and bloodraven/argocd/health-mysqlbackup.lua into your Argo CD values:

configs:
  cm:
    resource.customizations.health.shipstream.io_MysqlFailoverGroup: |
      # paste contents of bloodraven/argocd/health-mysqlfailovergroup.lua
    resource.customizations.health.shipstream.io_MysqlBackup: |
      # paste contents of bloodraven/argocd/health-mysqlbackup.lua

Use a Kustomize overlay

If you manage Argo CD with Kustomize, extract the patch from the chart and add it as a strategic merge patch:

helm pull bloodraven/bloodraven --untar
cp bloodraven/argocd/argocd-cm-patch.yaml .

patchesStrategicMerge:
  - argocd-cm-patch.yaml

Verify Argo CD health checks

Check that argocd-cm contains Bloodraven customizations:

kubectl get configmap argocd-cm -n argocd -o yaml | grep shipstream

After Argo CD refreshes the application, MysqlFailoverGroup and MysqlBackup resources should show Healthy, Progressing, or Degraded instead of Unknown.

Ignore status fields

GitOps tools should not try to reconcile .status.

Generated dashboards

If Grafana dashboard ConfigMaps are rendered by the Bloodraven Helm release, keep them in the operator application. If your monitoring stack owns dashboards centrally, set grafanaDashboards.enabled=false and import charts/bloodraven/dashboards/*.json in that stack.

Safe upgrade workflow

1. Read Upgrade Policy.
2. Apply CRD updates before controller image updates. This does not apply to v1.6.0 → v1.7.0, which has no in-place upgrade: never apply the v1.7.0 CRDs to a cluster that runs v1.6.0 (see Upgrading from v1.6.0 to v1.7.0).
3. Decide what the operator upgrade rolls before you apply it. A group that leaves spec.sidecar.image unset follows the sidecar image the operator release ships with, so an upgrade that ships a new sidecar starts an ordered update (followers first, a handoff of the primary role, then the old primary) in every such group on its own, as soon as the new operator reconciles it. A release that changes the rendered pod spec in other ways rolls pinned groups too, so read the release notes for such a change. To stage tenant rolls, pin spec.sidecar.image to the running release in each production group that must not move yet, and let Argo CD sync the pins, before step 4. Bloodraven tolerates a one-minor skew; a pinned group reports a SidecarVersionSkew Warning until you move it. See Sidecar ↔ operator skew.
4. Roll the operator.
5. Watch reconciliation on a non-production failover group that follows the new default.
6. Move the production groups to the new sidecar in maintenance windows, one group at a time: remove or bump each pin, then wait for Updating=False and Ready=True before the next.
7. Run make generate && make manifests before PRs that touch API fields.

Related pages

• Production Install
• Production Install Examples
• Production Hardening