Backups
Nightly PostgreSQL backups to S3, and how to restore from one.
canette can back up its in-cluster PostgreSQL database to an S3-compatible bucket on a
schedule. This is opt-in and only supports the bundled database (postgres.enabled=true) —
if you're using externalDatabase, back it up through your provider instead.
How it works
When backup.enabled=true, the Helm chart deploys a CronJob in canette-system that,
on each run:
- Runs
pg_dumpagainst the same database the API and controller use (via the existingpostgres-credentialsSecret — no separate credentials to manage) - Gzip-compresses the dump
- Uploads it to
backup.s3.uriascanette-<UTC timestamp>.sql.gz
No dedicated backup image is built or published — the CronJob runs on the same
postgres.image already used for the database itself. An init container installs the AWS
CLI into a shared volume ahead of time so the main backup container can run fully
non-root (runAsNonRoot: true).
backup:
enabled: true
schedule: "0 3 * * *" # nightly at 03:00 UTC (default)
s3:
uri: s3://my-bucket/canette-backups
region: us-west-2
secretRef:
name: canette-backup-s3-credentials
accessKeyKey: access-key
secretKeyKey: secret-keyPrerequisites: the S3 bucket must exist before installing canette. The backup Job will not create it automatically.
Retention
The chart does not delete old backups — every run adds a new object, nothing is pruned. Configure a lifecycle rule on the bucket (or your provider's equivalent) to expire objects older than however long you want to retain backups for.
Excluding high-volume tables
By default the backup excludes row data (but keeps the schema) for build_logs and
scan_sboms — both are high-volume, regenerable, and not part of the platform's actual
desired/runtime state, so excluding them keeps backups small and fast without weakening
disaster recovery. Restoring a backup still leaves those tables intact and empty rather than
missing, so no schema errors on restore.
backup:
excludeTableData:
- "public.build_logs"
- "public.scan_sboms"Set to [] to include every table's data. Uses pg_dump --exclude-table-data, so entries
must be schema.table.
Credentials
Prefer backup.s3.secretRef over plain accessKey/secretKey values so credentials don't
end up in values.yaml or Helm release history:
kubectl create secret generic canette-backup-s3-credentials \
--namespace canette-system \
--from-literal=access-key=AKIA... \
--from-literal=secret-key=...backup:
s3:
secretRef:
name: canette-backup-s3-credentials
accessKeyKey: access-key
secretKeyKey: secret-keyIAM policy (attach to the IAM user, scoped to just the backup prefix):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:PutObject"],
"Resource": "arn:aws:s3:::my-bucket/canette-backups/*"
}
]
}S3-compatible providers
Any S3-compatible endpoint works by setting backup.s3.endpoint:
# MinIO
backup:
s3:
uri: s3://canette-backups/canette
region: us-east-1 # MinIO requires a region, any value works
endpoint: https://minio.example.com
accessKey: minioadmin
secretKey: minioadmin# Cloudflare R2
backup:
s3:
uri: s3://canette-backups/canette
region: auto
endpoint: https://<account-id>.r2.cloudflarestorage.com
secretRef:
name: canette-backup-s3-credentials
accessKeyKey: access-key
secretKeyKey: secret-keyLeave endpoint empty for AWS S3.
Restoring a backup
Download the dump and pipe it into psql against the running database:
aws s3 cp s3://my-bucket/canette-backups/canette-20260818T030000Z.sql.gz - | \
gunzip | \
kubectl exec -i -n canette-system deploy/postgres -- \
psql "postgresql://canette:<password>@localhost:5432/canette"This overwrites current data. pg_dump's default output re-creates rows on top of an
existing schema and will fail on conflicting primary keys rather than cleanly replacing
data. Restore into an empty database (or drop and recreate the canette database first)
unless you specifically want to merge.
Get the current password from the postgres-credentials Secret if needed:
kubectl get secret -n canette-system postgres-credentials -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -dManually triggering a backup
To run a backup outside the schedule, create a one-off Job from the CronJob:
kubectl create job -n canette-system manual-backup --from=cronjob/postgres-backup
kubectl logs -n canette-system job/manual-backup -fTroubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
CrashLoopBackOff on install-aws-cli init container | No egress to the Alpine package mirror | Check the pod's outbound network access; the init container needs internet access to run apk add |
AccessDenied from the backup container | IAM policy doesn't cover the bucket/prefix or credentials are wrong | Verify the IAM policy's Resource matches backup.s3.uri, and that the Secret keys match secretRef.accessKeyKey/secretKeyKey |
command not found: aws in the backup container logs | /tools volume from the init container wasn't mounted, or PATH wasn't set | Confirm both containers mount the shared tools emptyDir and the main container's PATH includes /tools/usr/bin |
| Job never runs | concurrencyPolicy: Forbid skips a run if the previous one is still active | Check kubectl get jobs -n canette-system for a stuck prior run |