Known limitations

This page is the short version of Bloodraven's current boundaries. Read it before writing production manifests so the failure and recovery model matches your expectations.

API maturity

• The CRD API version is shipstream.io/v1alpha1.
• Fields may still change before v1beta1 / v1.
• There is not yet a published CRD conversion-webhook or migration contract. Track this under the CRD version-migration wishlist item.

Replication and RPO

• Bloodraven uses asynchronous MySQL replication. Emergency failover can lose transactions that committed on the old primary but had not reached the promoted replica.
• The operator records promotion and divergent GTID sets so data loss is observable, but it does not merge divergent data automatically.
• Planned failover is the zero-RPO path: it fences the source, waits for the target's GTID_EXECUTED to cover the fenced source GTID, then promotes.
• If you require synchronous commit semantics or quorum-based zero RPO on primary loss, Bloodraven is the wrong tool; see Why not Group Replication?.
• Read-only sites are full MySQL replicas. Filtered or partial replication, replication proxying, and query routing through an operator-managed proxy are not implemented.
• Clone and reclone copy from the active primary at full speed. Bloodraven does not throttle clone traffic, so adding or rebuilding a large reader can consume primary network, storage, and CPU capacity. Schedule the operation and monitor the donor accordingly.

Operator availability

• Leader election is enabled by the chart, but the default deployment still runs one replica. Run more than one operator replica only after validating the deployment model in your cluster.
• Sidecars preserve safety while the operator is unavailable, but new failover decisions wait for an operator to run.
• If the primary fails while the operator is down, writes remain unavailable until the operator returns and completes failover.
• Anti-flap state (lastFailover, lastFailoverTarget) is persisted twice on every promotion: in status.lastFailover / status.lastFailoverTarget, and out of band in the failover group's own annotations (bloodraven.shipstream.io/last-failover, bloodraven.shipstream.io/last-failover-target). The two travel different API paths — the status subresource has its own RBAC rule and admission chain — so an outage on one does not take the other with it. A restarting operator rehydrates from whichever copy is newer, and both writes retry every poll until accepted. Losing the cooldown across a restart therefore requires both paths to be rejecting writes at once; while that holds, a subsequent site failure inside the intended cooldown window can promote earlier than configured (deterministic-simulation finding, CooldownViolated(restart+stateLost) class). GTID gates still bound the damage — any divergence this causes is fenced and reported. Alert on mysqlfailovergroups and mysqlfailovergroups/status write failures (the operator logs and retries both every poll) to close the window. A restart that had to fall back to the annotations logs restored lastFailover from out-of-band annotations at WARN, which is a direct signal that this group's status writes were failing. This guarantee starts with operator builds that understand those annotations. Rolling back to an older build makes the annotation copy invisible again; do not rely on cooldown continuity across such a rollback while status writes are failing.

Placement and shared nodes

• Taints and node discovery are scoped per failover group with spec.sites[].taintNodeSelector, so one physical node can advertise membership in multiple failover groups at the same site.
• Application workloads on shared nodes must tolerate other groups' readonly taints but not their own group's taint.

Backups and restore

• Backup and PITR support is present, including backup verification, but restore-duration metrics and restore-performance guidance are still missing.
• In-place restore exists for destructive rollback of a live group. Use it carefully: full-instance restore fences writes and reclones the peer after loading the dump.
• PVC loss can be recovered by recloning from the current primary, but committed transactions that only existed on the lost PVC are gone. See Operations.
• Total cluster loss (all nodes and PVCs destroyed) requires recovering into a separate Kubernetes cluster from the source backup archive. See Multi-cluster DR for the end-to-end runbook using spec.initFromBackup with optional PITR replay.
• read-only sites cannot be selected for scheduled, automatic, or source-overridden backups. Healthy dr-only followers remain eligible.

Dragonfly co-management

• Managed Dragonfly is optional and intended for cache/session continuity, not durable application state.
• Dragonfly pods use ephemeral storage unless you configure spec.dragonfly.snapshot for planned snapshot-restore maintenance. Bloodraven does not schedule Dragonfly backups as durable data backups.
• Emergency MySQL failover never blocks on Dragonfly. If Dragonfly sync or promotion fails, sessions/cache may be discarded while MySQL recovery completes.
• spec.tls applies to MySQL, not Dragonfly. Protect Dragonfly with NetworkPolicy, Dragonfly auth, and any external TLS/service-mesh controls your environment requires.

Network partitions

• The failure-mode matrix covers common partition classes, but a dedicated network-partition runbook with metrics/events for each asymmetric case is still missing.
• Playground partition tests must use pod-level NetworkPolicy or an equivalent mechanism; host-level iptables rules do not reliably block Kubernetes Service traffic in k3d.

Observability and tooling

• Grafana dashboards and metrics are shipped, but PrometheusRule alert examples are not yet packaged as first-class chart artifacts.
• The kubectl bloodraven plugin wraps supported day-2 operations, but it remains a thin API client; the operator performs the authoritative safety checks.

Data-at-rest encryption

• spec.encryptionAtRest uses MySQL Community's component_keyring_file. It is not Oracle "MySQL Enterprise TDE", and Oracle states that file-based keyrings are not intended as regulatory-compliance solutions. Do not present it as compliance-grade TDE.
• The live keyring is a Kubernetes Secret, so etcd becomes part of your key custody. API-server encryption at rest for Secrets (ideally KMS-backed) and tight Secret RBAC are prerequisites, not recommendations. Bloodraven cannot verify either.
• Memory-backed volumes can reach swap on kernels older than 6.3, where tmpfs noswap is unsupported. Disable or encrypt swap on worker nodes.
• Enabling encryption on a group that already holds data is refused by default: MySQL only encrypts what is written afterwards, so existing tablespaces stay plaintext. The supported adoption path is replica-first (bootstrap encrypted, load, planned-failover, wipe the old PVCs); an annotation exists to accept partial coverage knowingly.
• Master-key rotation is refused on the active primary. Rotate replicas, planned-failover, then rotate the ex-primary.
• Backup, restore, and backup-verification still stage plaintext in pod emptyDir volumes. Backup artifacts are separately protected by spec.backup.profiles[].encryption; the transient staging inside the Job's own pod is not covered by live MySQL encryption.
• The mysqldDir and pluginDir defaults match the official mysql:9.x images. Other images need those fields set explicitly, and a wrong value shows up as InnoDB refusing to start.

External dependencies

• DNS steering depends on external-dns consuming DNSEndpoint objects. Bloodraven updates the CR; DNS provider propagation time and TTLs are outside the operator's control.
• Production installs need real topology-aware persistent storage. Local path / hostPath storage is acceptable for playground use only.
• The unauthenticated auxiliary and sidecar HTTP surfaces assume a trusted pod network. Use NetworkPolicy before exposing those Services broadly.