Skip to content

Backup / Restore Contract

Last resort creation note: no evergreen backup/restore standard existed in Engineering-Standard before this document. infrastructure/nas-operational-model.md covers GitLab↔NAS repository-taxonomy mirroring only; infrastructure/nas/NAS-CONVERGENCE-BEAD-HANDOFF.md is a dated (2026-08-02) Bead handoff plan, not a standing standard, and its own Bead #5 states plainly: "currently NO working scheduled backups... only ad-hoc July dumps exist." That gap is the evidence this document exists to close.

Authority

  • Canonical store: Oracle Beads/Dolt is the system of record for Bluefly's Gas City work graph. This document does not change that — see Gas City Adoption.
  • Backup feed: Oracle's native backup/sync/restore capability feeds a NAS-side copy at [NAS-ROOT]/.beads.
  • NAS-side ownership: HARBORMASTER owns NAS-side backup integrity, retention, and restore-proof for that feed. HARBORMASTER does not own the Oracle-side canonical store — it owns the copy and the proof that the copy is usable.

The Contract

A backup is not "done" because a copy exists. Required state, all four conditions, before a backup counts as complete:

BACKUP_EXISTS               = YES
BACKUP_CURRENT              = YES
CHECKSUM_OR_NATIVE_INTEGRITY = PASS
RESTORE_REHEARSAL           = PASS
  • BACKUP_EXISTS — a copy is present at the NAS backup location on a defined schedule, not only when someone remembers to run one manually.
  • BACKUP_CURRENT — the copy's timestamp is within the defined freshness window for its schedule; a stale backup that predates the schedule's own retention window is treated as BACKUP_CURRENT=NO, not as a degraded pass.
  • CHECKSUM_OR_NATIVE_INTEGRITY — the copy is verified byte-for-byte (checksum) or via the store's own native integrity check (e.g. Dolt's own verification, where the upstream tool provides one) — not merely "the copy job exited zero."
  • RESTORE_REHEARSAL — a restore from that backup has actually been performed into a non-production target and the restored store verified usable. A backup without a completed restore rehearsal is incomplete, full stop — an untested backup is a hypothesis about recoverability, not a proof of it.

Only when all four read YES/PASS does a given backup generation satisfy this contract. This is a recurring, scheduled state, not a one-time checkbox — each new backup generation must re-satisfy all four before the previous generation may be treated as fully superseded for retention purposes.

Relationship to the 2026-09-06 emergency export

An ad-hoc checksum-verified export of Beads state to NAS performed as incident-response evidence during tonight's convergence work is exactly that: emergency evidence, produced once, under pressure, to prove a specific point in time. It satisfies BACKUP_EXISTS and CHECKSUM_OR_NATIVE_INTEGRITY for that one snapshot. It does not satisfy BACKUP_CURRENT on an ongoing basis and has no RESTORE_REHEARSAL attached. It is not a substitute for this contract's ongoing schedule — treat it as a single verified data point, not as evidence the contract is already satisfied going forward.

Relationship to existing NAS material

  • nas-operational-model.md governs NAS repository-taxonomy mirroring and SMB/worktree execution constraints — a different concern from this contract. Where the two touch (the NAS is both a GitLab mirror target and a Beads backup target), this document is authoritative for backup/restore state specifically.
  • nas/NAS-CONVERGENCE-BEAD-HANDOFF.md Bead #5 ("Establish scheduled database and container-volume backups") is the dated plan that motivated this contract. That Bead's acceptance criteria should be read against the four conditions above, not against a weaker "a scheduled job exists" bar.

Upstream Dolt-native backup mechanism (context, not a substitute)

RETRIEVED 2026-09-09, github.com/gascityhall/gascity/blob/main/engdocs/contributors/dolt-maintenance.md (contributor runbook, not a published docs.gascity.com page): Gas City itself documents a native backup path for the managed Dolt store, distinct from the NAS-side feed above.

  • dolt backup sync writes to <city>/.beads/dolt-backups/current/, then atomically rotates current/ to success/<timestamp>/; retention keeps "the 3 newest successful snapshots and the most recent failed snapshot."
  • Each success/<timestamp>/ subdirectory is a complete, independently usable dolt backup target — restorable directly with dolt clone file://<path> dolt (documented emergency procedure: stop the controller, move the broken store aside, clone from the newest readable snapshot, falling back to the next-newest if the most recent is also unreadable).
  • Compaction runs CALL DOLT_GC() against the managed server, bounded by a documented default gc_timeout of 10 minutes, with a post-GC smoke test (SELECT COUNT(*) FROM issues).

Do not treat this as already satisfying BACKUP_EXISTS/RESTORE_REHEARSAL above. The same upstream document states its own snapshot-and-compaction steps are, as of that writing, "Observe-only mode — snapshot and compaction are not yet wired in production." That is an upstream-side caveat about the native mechanism's own maturity — separate from whether Bluefly has wired NAS-side scheduling around it. If/when this native mechanism is confirmed active in the gc release Bluefly runs, it is a candidate input to BACKUP_EXISTS/CHECKSUM_OR_NATIVE_INTEGRITY above; it does not substitute for RESTORE_REHEARSAL, which must still be proven against Bluefly's actual restore target.

Operator Decisions

  • Backup schedule/cadence and retention window (not yet specified in any Bead or standard reviewed for this document).
  • Restore-rehearsal cadence — how often RESTORE_REHEARSAL must be re-proven, not just proven once at setup.
  • Whether HARBORMASTER's machine identity and 1Password/GitLab principals are provisioned — see the Machine Identity Catalog.