Panel disaster recovery
How the panel backs ITSELF up (pg_dump + the boot secrets, encrypted offsite) and how to restore it into a fresh VM in under 30 minutes — the migrate-panel runbook.
The whole panel is one Postgres database plus four boot secrets. The panel backs itself up
nightly — a pg_dump of its own database, bundled with those secrets, shipped to an offsite storage
target through restic (the same engine, and the same kind of append-only target, your sites use).
If the VPS dies, you restore that snapshot into a fresh VM and the panel comes back; your nodes
reconnect on their own.
The canonical runbook lives in the repo
This page explains the mechanism. The step-by-step recovery procedure is
runbooks/migrate-panel.md at the repo root — follow it verbatim during a real recovery
or the quarterly drill. Target: a working panel with a reconnected node in under 30 minutes.
This is the DISASTER path, not the fresh path
Restoring from a backup (this page) and installing a brand-new panel are two flows that share the
same .env/compose mechanics. For a fresh install use the one-command
install.sh — it scripts the same Docker preflight,
.env collection, and docker compose pull && up -d that you run by hand here, and gates the box
on a correct .env the same way.
What the panel self-backup captures
A panel self-backup is a restic snapshot (tagged panel) of exactly two files:
panel.sql— apg_dumpof the entire panel database: nodes and their pinned SSH host-keys, sites, instances, deploys, probes, storage targets, the encrypted secrets/credentials columns, and the audit log.env.boot— the four boot secrets the restored panel needs:MASTER_KEY,BETTER_AUTH_SECRET,AGENT_GATEWAY_SECRET, andPOSTGRES_PASSWORD.
The boot secrets ride inside the snapshot because a pg_dump alone is useless without them — the
encrypted columns can only be decrypted with MASTER_KEY, sessions need BETTER_AUTH_SECRET, and
agents authenticate with AGENT_GATEWAY_SECRET.
The repository is encrypted
restic encrypts the repository client-side, with a password the panel holds, before any bytes
reach storage — so the snapshot sitting offsite is unreadable and the boot secrets inside it are
encrypted too. To restore it you need the restic repository password (and your
MASTER_KEY to decrypt the panel's stored credentials once the DB is back). Keep both
somewhere separate — a password manager or sealed envelope — because the backup is encrypted
with them; they're the things you carry to a recovery yourself.
What is not backed up, on purpose: the offsite bucket itself (it's the destination), Caddy's issued certs (re-issued from ACME on first boot at the same hostname), and node/site state (it lives on the nodes, which reconnect and re-report on their own).
Scheduling it
Set a panel-backup storage target in Backups → panel self-backup. A nightly job (02:30 by default) then ships a backup automatically; retention prunes older snapshots over the target's prune lane. You can also trigger a backup on demand. A backup is marked succeeded only once restic reports the snapshot landed — an untested backup is just a hope, which is why this also feeds the quarterly DR drill.
Restoring (the short version)
- Provision a fresh Ubuntu VM; install Docker + compose +
psql+restic. - Point
resticat the offsite repository with your repo password and storage credentials, andrestic restore latest --tag panel --target ./restoreto pull the latest panel snapshot. - From the restored bundle, drop
env.bootinto.env; bring updb;psql < panel.sql. docker compose pull && up -dbehind Caddy; repoint DNS to the new VM at the same hostname.- Watch the worker log — your nodes reconnect on their own, a reconcile runs, and probes resume.
The full, numbered procedure with the exact commands is runbooks/migrate-panel.md.
Custom hostnames API
The machine seam for a hosted app whose customers bring their own domains — register a hostname, SpipCP creates the DNS record when it controls the zone or hands back exact records to relay. Site-bound tokens, register/status/list/remove, and the polling recipe. For app authors.
Screenshot index
Every product screenshot in these docs, in one place — a contact sheet linking each image to the page it documents, and the one command that regenerates them all.