The card you keep

Everything from here is reference. Nothing in this topic is new; all of it is spread across the six units behind you, which is exactly the problem — at 03:00 nobody reads a course. Print this page, or keep the tab open. It is the one screen the rest of the course was building toward.

Two rules for using it. Numbers marked shipped are CRD defaults; numbers marked playground are what the local group overrides them to. And anything with a date on it belongs to the version appendix rather than to this card — check there before you quote a version, an issue number or a documentation defect.

Timings and thresholds

The five condition reasons

There are exactly five, they come straight from EvalCrossSite, and Failover is not one of them.

Replication adds its own reasons to Degraded: ReplicationLagging, ReplicationBroken, ReplicationError, ReplicationSourceMismatch. All four skip role: read-only sites entirely.

Evaluation order (a row fires only if every row above declined): fence-first early return → TotalLoss → SplitBrain → failover → NoPrimary → degraded-with-peer-down → Healthy.

The nine promotion steps, with fatality

FailoverController.Execute, in order. Fatal means the promotion aborts; everything else logs and carries on.

Then, outside the sequence: the durable failover record and the counter are stamped before DNS. Node taints and source convergence are poll neighbours, not steps.

The four Service kinds

2 × len(sites) + 2 objects. Eight for a three-site group.

role="fenced" matches neither shared Service. A read-only site earns healthy=yes only when all five hold: converged source, replicating, non-nil lag, canonical direct source host, and lag within readOnlyMaxLagSeconds.

Metrics, and their label sets

Label sets are not uniform. Read them before you copy a selector between rules.

Four words for two concepts: namespace / mysql_namespace, and group / failover_group. That is not a typo in this table.

Annotations you apply by hand

Written by the operator, never by you: bloodraven.shipstream.io/last-failover and …/last-failover-target, RFC3339 at second precision, as a pair.

Roles, in one line each

• primary-candidate — the only promotable role. Counted in every tally.
• dr-only — never promoted, but counted in coreCount and the tallies. A full topology participant that cannot win.
• read-only — never promoted, invisible to the matrix, never tainted, refused as a backup source, and skipped by every replication condition. Not a spare.

Glossary

Active site — the one site status.activeSite names as the writable authority. One name, or empty; empty means authority is ambiguous, and every endpoint is shed.

Anti-flap cooldown — failoverCooldown. Suppresses automatic promotion and nothing else.

Core site — any site whose role is not read-only. What the matrix counts.

Divergent GTID set — GTID_SUBTRACT(old primary, new primary): the transactions the old primary holds that the new one never saw. Its cardinality is your lost-transaction count.

Fencing — SET GLOBAL super_read_only = ON. Blocks writes even from SUPER. Closes no sockets, so surviving sessions keep serving stale reads. At the Service layer, a separate thing: stamping the pod role=fenced, which matches neither shared selector.

GTID set — the transaction history a server has executed, as uuid:interval[:interval…], comma-separated across UUIDs. In MySQL 9.x a set may carry a user tag, uuid:tag:interval, and the tag is part of the identity — uuid:A:1-3 and uuid:B:1-3 are six transactions, not three.

Ordered update — the operator-driven rollout of a spec change: standby first, then a real failover, then the old active. Triggered by spec-hash drift; not cooldown-gated.

Promotability — exactly role == primary-candidate. Not earned by having the freshest data.

RPO / RTO — how much recently committed data you accept losing / how long you accept being unable to write. Bloodraven’s RPO on sudden primary loss is not zero, by design; its whole engineering budget goes to RTO.

Self-fence — the sidecar setting super_read_only=ON on its own MySQL without asking anyone. Rule #1: the known authoritative active site is somebody else. Rule #2: the operator and every peer have been silent past leaseTimeout.

Safety net — a different thing from self-fencing: a one-shot at sidecar startup that fences first and asks afterwards, and completes before the fencing monitor exists. safety net: in a log means a pod that has never been allowed to write; SELF-FENCED: means one that was writing and lost the argument.

Sealed — the steady-state keyring phase: the keyring file is projected read-only from the escrow Secret, so mysqld physically cannot add a key. Unsealed is mid-flight, and rotation re-enters it from Sealed, so the phase string alone is ambiguous — read it beside unsealReason.

Source convergence — the independent poll stage that repoints replicas at the current authority. Demands a direct source; a replica chained off another replica does not count as converged.

Split brain — more than one core site writable at once. sitePriorities does not prevent it and merges nothing; it is a standing decision about whose unreplicated writes you will discard.

Star topology — every replica replicates directly from the active site. Bloodraven does not accept chains.

super_read_only vs read_only — super_read_only also blocks users holding CONNECTION_ADMIN or SUPER, which plain read_only does not. Setting super_read_only=ON implicitly forces read_only=ON; clearing read_only implicitly clears super_read_only.

Where this leaves you

You have a card. Use it to stop looking things up, and use the version appendix to check anything on it that carries a date. What you can now say about a group you are handed is what the whole course was for: what it will do next, what it will refuse to do, what it will cost you when it does, and which of those numbers you measured yourself.

Mark as read