PVC Backups

PVC-backed backups are useful for labs, isolated clusters, and short-retention local copies. They are not a disaster-recovery substitute for off-cluster storage.
When to use PVC backups
Use PVC backups when:
- You are testing backup and restore behavior locally.
- Your cluster has a durable shared storage system.
- You need a short-lived staging copy before exporting elsewhere.
Avoid PVC backups when:
- Cluster loss must not lose backups.
- The StorageClass is node-local.
- The PVC cannot be mounted by backup, restore, and cleanup Jobs.
Operator-managed PVC
backup:
profiles:
- name: pvc-local
storage:
type: PVC
pvc:
provisioned:
storageClassName: fast-ssd
size: 500Gi
subPath: orders
retention:
keepLast: 7
schedules:
- name: nightly
profileName: pvc-local
schedule: "0 6 * * *"
timeZone: Etc/UTC
concurrencyPolicy: Forbid
pvc.provisioned makes the operator create and own the claim mysql-<group>-backup-<profile> with that class and size; without storageClassName the cluster's default StorageClass is used. Set exactly one of provisioned and claimName.
provisioned is applied only when the claim is created, so admission refuses changing it, adding it, or removing it for as long as the profile stays in spec.backup.profiles. That includes switching between provisioned and claimName, and switching the profile's storage to S3 and back. A size is compared by value, so rewriting 500Gi as 512000Mi is not a change. To grow the volume, expand the claim itself if its StorageClass allows it; the operator never changes an existing claim.
If a profile is removed and later added back under the same name with another class or a larger size, the claim the earlier profile created no longer matches. Each backup checks the claim against provisioned before its Job starts:
storageClassNameunset matches any class on the claim (the cluster fills in its default class).- A set
storageClassNamemust equal the claim's class. - The size is compared with the claim's request (
spec.resources.requests.storage), not with the provisioned capacity. A larger request, such as a claim expanded out of band, is accepted with a NormalBackupStorageExpansionAcceptedEvent. - Another class or a smaller request fails the backup with
Ready=False, reasonBackupStorageMismatch, and a Warning Event of the same name. No backup Job writes to the claim. The check covers backup Jobs only: whenspec.backup.pitr.profileNamenames this profile, the PITR archiver in the MySQL pods keeps writing binlogs to the claim.
To fix a mismatch, restore the profile's values, or replace the claim:
- If PITR archives to this profile, disable PITR (
spec.backup.pitr.enabled: false) or switchspec.backup.pitr.profileNameto another profile, and wait until every MySQL pod of the group has rolled, so no pod mounts the claim. - Keep or copy what the claim holds.
- Delete the claim. The operator provisions a new one from
provisioned.
mysql-<group>-backup-<profile> belongs to the group: deleting the group deletes it, and every backup on it, even a backup with deletionPolicy: Retain. To keep such backups, see Deleting a group without losing backups.
subPath is a directory inside the PVC; backups land under <subPath>/<backup-name>/. The API server rejects a subPath containing a .. segment, and backup and restore Jobs refuse one stored before that validation existed.
User-managed PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: orders-backup-store
namespace: orders
spec:
accessModes:
- ReadWriteOnce
storageClassName: fast-ssd
resources:
requests:
storage: 500Gi
---
apiVersion: shipstream.io/v1alpha1
kind: MysqlFailoverGroup
metadata:
name: orders
namespace: orders
spec:
backup:
profiles:
- name: pvc-local
storage:
type: PVC
pvc:
claimName: orders-backup-store
subPath: orders
On-demand backup
apiVersion: shipstream.io/v1alpha1
kind: MysqlBackup
metadata:
name: orders-pvc-manual-20260427
namespace: orders
annotations:
bloodraven.shipstream.io/triggered-by: manual
spec:
failoverGroupRef:
name: orders
profileName: pvc-local
kubectl apply -f orders-pvc-backup.yaml
kubectl get mysqlbackup orders-pvc-manual-20260427 -n orders -o wide
Restore from PVC
apiVersion: shipstream.io/v1alpha1
kind: MysqlFailoverGroup
metadata:
name: orders-restore
namespace: orders
spec:
initFromBackup:
source:
pvc:
claimName: orders-backup-store
subPath: orders/orders-pvc-manual-20260427
Include normal sites, credentials, and dns fields in the recovery failover group.
Sizing and retention
- Size the PVC for peak dump size, compression ratio uncertainty, and retention overlap.
- Keep at least two successful backups:
retention.keepLastof 2 or more. - Watch PVC usage; a full PVC usually causes backup Jobs to fail after doing expensive work.
Backups that never succeeded
A backup Job writes its backup stamp
(<backup>.bloodraven-backup.json beside the dump directory) before the
dump. When a backup that never succeeded (Failed, or deleted while Running)
is deleted, the finalizer stops its Job and runs artifact cleanup at the
planned location: it deletes the stamp and any partial dump, but only when
the stamp names that backup's UID. The cleanup is best effort; if it cannot
run or fails, the finalizer is released with an ArtifactCleanupSkipped
Warning naming what is left. The same happens when there is no stamp at the
location but the directory still exists (the cleanup Job exits 0 with
BLOODRAVEN_CLEANUP_SKIPPED): nothing is deleted, and an encrypted backup's
dump-key snapshot is kept, because those files may be its artifacts. A backup
re-created with the same name replaces
an unfinished stamp of an earlier backup of that name when nothing remains
under the location, and otherwise fails with backup stamp conflict naming
the first remaining file.
More generally, a backup never dumps into a location that already holds
files without its stamp, since abandoned cleanup deletes everything under a
location whose stamp names its backup. It fails with backup stamp conflict
naming the first file; delete the files, or use another backup name or
subPath.
Common failures
| Failure | Check |
|---|---|
| PVC full | kubectl describe pvc orders-backup-store -n orders, Job logs |
| Wrong access mode | Whether the backup Job can mount the claim on its scheduled node |
| Node-local storage | Restore Job scheduled to a different node cannot see data |
| Reclaim policy | Deleting a claim might delete the only backup copy |
Deleted MysqlBackup stuck Terminating | The ArtifactCleanup condition on the CR. Cleanup mounts the claim recorded in status.artifactStorage and runs from the operator image; see Artifact cleanup |
Restore fails with Read-only file system on load-progress | The restore Job was built by an operator that predates disabled load-progress tracking; upgrade it. The restore mounts the claim read-only by design |

