Skip to content

Gas City Deployment Wiring

Scope: Bluefly integration wiring only. Upstream authority: Current docs.gascity.com. Gas City documentation is a predecessor/migration reference, not a second object model. Beads/bd docs live at beads.gascity.com.

This document does not define Gas City or Beads behavior. Upstream owns those contracts. Exact installed gc subcommands: cli-resources-reference.md. Operator HOW: blucity-operator-contract.md.

Upstream References

Bluefly-Owned Wiring

Bluefly owns repository, infrastructure, CI/CD, and deployment integration around upstream tools.

Current infrastructure repositories:

  • worktrees/gitlab_components
  • worktrees/iac
  • worktrees/blucity-packs
  • worktrees/agent-tailscale
  • worktrees/agent-docker

Current runtime host reference:

  • bluefly-platform.tailcf98b3.ts.net
  • 100.74.177.6
  • fd7a:115c:a1e0::8436:b108

Source vs Runtime Boundary

Per Gas City documentation, pack behavior is defined by pack.toml and conventional pack directories; deployment choices live in city.toml; machine-local bindings and runtime state live under .gc/.

Bluefly integration must preserve that boundary:

Artifact Owner Git posture
pack.toml and pack directories Gas City pack spec; Bluefly pack repo for local packs Commit when part of a source repository
city.toml Gas City deployment config; Bluefly wiring Commit only if it is the intended shared deployment source
.gc/ Gas City runtime Do not commit
.gt/ / ~/gt runtime state Gas City runtime Do not commit
.beads/config.yaml, .beads/metadata.json Beads git contract Tracked source of truth (proven 2026-09-02 across 7 repos; the in-repo .beads/.gitignore states this explicitly — commit both)
Rest of .beads/ (Dolt database, hooks, routes.jsonl, interactions.jsonl, formulas/ symlinks) Beads runtime Do not commit — host-generated, symlinks target absolute machine-local paths
GitLab CI components Bluefly / GitLab Commit in the owning repository
Terraform / cloud-init / Docker Compose / Tailscale config Bluefly infrastructure Commit in the owning infrastructure repository, excluding secrets and generated state

Official Model Applied To Bluefly

Upstream says Bluefly integration consequence
A shareable pack is portable behavior. A city is a root pack plus deployment config and machine-local runtime bindings. worktrees/blucity-packs owns shareable pack source. Host-local .gc/ state is not source and is not committed.
Standard pack behavior is expressed through pack.toml and conventional pack directories such as agents, formulas, orders, commands, doctor checks, overlays, skills, MCP, fragments, and assets. Bluefly pack repositories should conform to upstream pack layout instead of inventing Bluefly-only pack structure.
Durable pack imports are authored with source and optional version; registry handles are command handles, not durable authored imports. Bluefly-authored TOML must use durable import sources for committed configuration.
Credentials belong in Gas City credential storage, not literal source URLs. Bluefly integration uses the approved secret system and does not commit credentials into pack or city files.
Rigs are registered through the documented Gas City lifecycle. Source repositories stay in Git; runtime rig bindings are created by gc, not by hand-written runtime directories.
A repository existing on a host does not by itself require Gas City orchestration. A repository becomes a Rig only when agents execute directly against it, it needs its own Bead namespace, it needs rig-scoped session execution, it has independent source/release activity Gas City must route to directly, or it is otherwise a genuine dispatch target — not merely because the directory is present.
Tutorial 01: portable rig declaration in city.toml; machine-local path in .gc/site.toml. Installed gc 1.4.1 behaves that way. city.toml = portable rig identity. .gc/site.toml = machine-local path binding. Do not put path= in city.toml as the normal model. Leftover path= language on other upstream pages is a recorded disagreement, not a Bluefly topology.

Pack Repository Conformance

For Bluefly pack repositories, use the Gas City pack docs as the authority:

  • Keep shareable pack behavior in pack.toml plus conventional pack directories.
  • Use durable import source values and optional version constraints in authored TOML.
  • Do not commit registry handles as pack imports.
  • Do not manually create .gc/ runtime state.
  • Let gc create or repair runtime bindings when the CLI documents that lifecycle.

Execution authority

One owner per layer — duplicate implementations indicate convergence debt.

Layer Repository / project Role
Provision worktrees/iac Infrastructure, cloud-init, systemd, city path
Services worktrees/agent-docker Compose, tunnel snapshots, DDEV sidecars
CI orchestration worktrees/gitlab_components Deploy and receipt components
Network worktrees/agent-tailscale Tailnet routing
Runtime Gas City / Beads / OpenClaw (upstream). Gas City is predecessor pack / incomplete cutover, not a peer runtime. Execution primitives
Application Drupal (upstream) System of record

Removal candidates, preconditions, and gated sequence: infra-convergence-roadmap.md.

Deployment Boundary

Current docs.gascity.com does not prescribe a Bluefly production host topology. Gas City is a predecessor pack, not a second deployment authority.

Bluefly production deployment is therefore limited to integration wiring in the owning repositories:

  • GitLab components define pipeline composition.
  • iac defines infrastructure provisioning.
  • agent-tailscale defines network attachment.
  • agent-docker defines containerized supporting services.
  • blucity-packs defines Bluefly-owned Gas City packs.

Any claim about gc, gt, or bd behavior must cite the upstream documentation or official CLI output. If upstream documentation is silent, this document must remain silent.

Merge Authorization (operator, 2026-08-24, binding)

Agents merge feature → release/v0.1.x (and any other release branch) without asking the operator per instance — that authorization is standing, not per-MR. The single remaining approval gate is release/v0.1.x → main; only the operator approves that merge. This does not create merge capability where a session has none — a session must still hold a working merge-capable GitLab tool/credential to act on this authorization; the authorization removes the permission question, not the tool-capability question. Where the blocker is "no session holds a merge tool," that is a separate, still-open gap and should be reported as such rather than conflated with an authorization wait.

Provenance-or-Refusal (binding, 2026-08-24)

Five confirmed 2026-08-24 incidents (tracked as ubuntu-nqvf, ubuntu-qad3, ubuntu-bi8, ubuntu-6yz1, ubuntu-hmij) share one shape, independent of their surface differences (a stale Oracle checkout, a store resolved by cwd, a CI job that reports success while failing, an artifact republished from an old base): a read or write silently targets a different version or instance than the actor believes it is targeting, and returns a plausible, well-formed, wrong result with no error at any point. "Merged is not deployed" (above) is one instance of this shape, not the whole of it.

What unifies these is not staleness — every one of these systems had the information needed to say "you are not looking at what you think you are looking at" and did not say it. The binding remedy:

Every read that could be stale, ambiguous, or wrong-target either states its own provenance (the exact ref, store, or version it was read at) or refuses to answer.

This applies to agents and to source: an agent reporting file content from an /opt/bluefly/rigs/<name> checkout must state the local HEAD it read at, or read via the GitLab API at an explicit ref instead; a bd operation must fail loudly on an unresolved/ambiguous store rather than silently writing into or reading from the wrong one; a release job must fail its own status, not report success while its downstream step never fires; an artifact publish must diff against the current live version before writing, not assume a local copy is current.

"Read the API instead of the local checkout" is not a sufficient remedy on its own (Sentinel, 2026-08-24, a sixth confirmed instance): a GitLab MR diff endpoint served a stale pre-push diff — old SHAs, superseded TODO comments, no error, no staleness marker — moments after the branch head had already advanced via a legitimate push. Trusting it would have reported already-completed work as not done. The remote API itself can answer from an earlier state; hardening only the local-vs-remote axis leaves this exposed. The refusal token in every instance (deployed_sha, allow_failure_jobs_failed, local_head_sha+behind_count+dirty_file_count, resolved_database_name+config_path_used, element_id_set, diff_base_sha+head_sha) is a value the answering system already has at read time — the remedy is surfacing what a surface already knows, not building new provenance infrastructure (per the standing no-new-artifacts constraint, find the existing surface first).

Existence is not capability (Sentinel, 2026-08-24, a distinct failure mode from staleness, not a subset of it): confirming that an artifact/file/component exists is not evidence that it works for the purpose being asked about. A read that can only establish existence must not be rendered as an answer about capability — e.g. "this CI component file exists and is well-built" does not answer "does a working validation capability exist" when the component has zero consumers and its required input is absent (ubuntu-6ncb). Where provenance-or-refusal is about which version, this is about which question — a read answering "does X exist" must not silently stand in for "does X work here, for this."

Implementation is deliberately not specified here — provenance requirements differ per system (checkout vs. Dolt store vs. CI job vs. artifact tool) and belong in each owning repository's contract, not centralized as one mechanism. This section fixes the principle and the incident evidence; the per-system enforcement is separate, trackable work.

The claim vocabulary an agent uses to state a provenance-bearing result (OBSERVED / REPORTED / INFERRED / NOT VERIFIED) is not per-system in the same way — see standards/core/evidence-contract.md, which is canonical and central across the estate. Mechanisms differ by system; the vocabulary for stating a claim does not.

Integration Architecture & Decision Flow

1. Repository & Pack Governance

To avoid duplicate platform orchestration, all domain expertise and policy must be encapsulated as Gas City Packs and delegated to the Gas City runtime engine: - bluefly-repository-governance-pack: Collects local hook/config observations, verification rules, and convergence executors. - bluefly-drupal-pack: Contains database migrations, cache rebuilds, Drush formulas, and deployment receipts. - bluefly-oracle-pack: Manages provisioning, runtime deployment, repair, and doctor verification. - bluefly-receipts-pack: Houses verification logic, cryptographic signing, and audit logging.

2. Capability Decision Flow

Before implementing any new capability, engineers and agent workers must follow the Adoption Priority evaluation path:

                  Start New Capability
                           │
                           ▼
             Does Gas City have this primitive?
                   (Formula, Pack, Bead)
                           │
             ┌─────────────┴─────────────┐
             ▼ YES                       ▼ NO
      Do not implement.           Can it be a Pack?
      Adopt upstream.            ┌───────┴───────┐
                                 ▼ YES           ▼ NO
                            Build Pack.     Implement in blu.

If a capability can be implemented as a Pack, Order, or Formula, it is a policy violation to write it directly into the blu CLI or as custom shell script frameworks.

Deploy Boundary — No Unattended Auto-Deploy on Merge to Main

Rule: no repo may deploy to Oracle unattended on a merge to main. Confirmed necessary by a real incident (2026-08-23): deploy:oracle-blucity's rsync --delete job fired on every push to main, and because .beads/ was gitignored (absent from the CI checkout, therefore not in the rsync exclude list), it deleted the live work store on a routine merge.

Pattern, verified working, applied to two repos (blucity, agent-docker): 1. A release-train stamp-release component cuts a vMAJOR.MINOR.PATCH tag, human-gated (when: manual), main-only. 2. A deploy-oracle component fires only on that tag (only_on_tag: true), also human-gated, and deploys a pinned git archive --format=tar <tag> artifact over SSH — never a live rsync/git pull of a branch tip into a running host path. 3. environment: is wired with a real name: sub-key so every run is a tracked GitLab Deployment, not an opaque SSH step.

A deploy-boundary fix is not complete when it merges to a release branch — it is complete when it reaches the branch that actually fires in production. release/v0.1.x and main can drift for weeks with no merge between them; verify the fix landed on the branch the deploy runner actually watches.

A top-level rules: override REPLACES the component's rules — it does not extend them

Verified 2026-08-24 in agent-docker, where it silently disabled a production deploy for three days.

The oracle-deploy CI component generates a job named deploy:<job_name> (templates/oracle-deploy/template.yml:277) whose own rules auto-deploy on the configured branch. A consumer added this override to suppress the job on merge requests:

deploy:oracle-openclaw-gateway:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      when: never

Because a top-level rules: key replaces the component's rule set rather than appending to it, that block became the job's entire rule set. On a push to main no rule matched, so GitLab excluded the job from the pipeline altogether — not skipped, not manual: absent.

Proof is a job count, not a YAML reading: pipelines 2783978284 and 2778289334 (both main, both push) each contain 17 jobs and neither contains deploy:oracle-openclaw-gateway. After the fix (!307, !308) the same pipeline shape contains 19 jobs with that job present and status=manual.

Rule: when overriding a component-generated job's rules:, restate every branch case you still need, including the main case. Verify by listing the jobs a real pipeline generated — a rules change that looks correct in the file can still produce a pipeline that never creates the job.

Merged is not deployed, and deployed is not running

Three separate claims, each needing its own evidence:

Claim Evidence that settles it
The fix is in source MR state merged, plus the file read from origin/main at that SHA
The fix can run The job appears in the job list of a pipeline generated from that SHA
The fix ran That job's status and trace, with a start and end timestamp
The service recovered A runtime probe from inside the container, not container state

agent-docker!306 merged a correct config-seeding fix at 03:08 UTC and was reported as "source side closed, waiting for the deploy job to fire". The job it rides on could not be created, so the merge changed nothing at runtime for the next seven hours. A merge that lands on a job no pipeline generates is inert.

"Container is Up" is not a health signal

The OpenClaw gateway failed startup validation on a config conflict and — by design — stayed alive rather than exiting:

gateway startup failed: tailscale funnel requires gateway auth mode=password
(set gateway.auth.password or OPENCLAW_GATEWAY_PASSWORD).
Process will stay alive; fix the issue and restart.

It never bound its port, never crashed, never tripped a restart policy, and reported Up for two and a half days. FailingStreak reached 1254 by 03:16 UTC on 2026-08-24 and kept climbing after that.

Rule: the accepted proof that a service recovered is a request from inside the container to the service's own loopback address returning success, plus a FailingStreak that resets. Do not close a service incident on docker ps output.

Corollary to the boundary rule: manual means someone must play it

when: manual on main is the deploy-boundary rule working as intended — and it also means the deploy never happens on its own. A remediation is not finished when the fix merges and the job becomes creatable; the incident stays open until the job is played and the runtime probe passes. State that explicitly in the handoff, or the outage silently waits on nobody.

Rig Path Binding — Governing Model

Governing model (Tutorial 01 + installed gc 1.4.1):

city.toml       = portable rig identity / declaration
.gc/site.toml   = machine-local rig path binding

Do not put path= on [[rigs]] in city.toml as the normal current model.

Recorded disagreement (do not hide it): some leftover upstream pages, including gc-start-walkthrough, still say .gc/site.toml inference is gone and city.toml is the sole source of truth for rig locations. Contributor Pack v2 design notes still describe the path-move as mid-migration. Those pages do not override Tutorial 01 plus the installed binary. Escalate them as DOC_DRIFT.

A host still running gc 1.3.2 emits "unsupported pre-1.0 rig.path for rig %q; move it to .gc/site.toml" — the same split, an older release. Following leftover walkthrough text against either binary turns a diagnosable bind failure into a second, self-inflicted one. Always resolve which gc binary is running before acting.

Multiple Binaries in PATH Invalidate a Version Check Alone

Checking the deployed version is not sufficient when more than one bd or gc binary exists on the host. Resolve which binary runs, not just its reported version — which -a bd / which -a gc. A dev build or an unmerged feature-branch build earlier in PATH answers bd version coherently while reporting unreleased or off-branch behavior; the version string alone cannot distinguish it from a clean release.

When multiple binaries exist, run any comparison against a fully qualified released path. Do not reorder or prune PATH just to make the bare command resolve correctly. Treat upstream's own "multiple 'bd' binaries found in PATH" warning as invalidating any behavioral claim made with the bare command until re-verified against a known-clean binary. When a behavioral delta is found between builds, separate version drift from branch drift before attributing it to either — a build that is both off-branch and behind release confounds two distinct causes.

Beads (bd) Workspace Resolution — Verified Behavior

  • bd does not walk up the directory tree looking for a workspace. cd /tmp/nested/a/b && bd where fails exactly the same as cd /tmp && bd where. It resolves only from BEADS_DIR, or an explicitly configured workspace.
  • "No active beads workspace found" is the normal response to an unset BEADS_DIR — it does not mean the store is broken, missing, or that the host has no Beads capability.
  • The hint bd prints in that error (run 'bd init' to create a new database) is the one action that must not be taken reflexively. bd init is exactly the mechanism that manufactured multiple disconnected stub stores on Oracle during the 2026-08-23 incident (a directory with only a lone .beads/ and nothing else is the signature of an unwanted bd init/probe-created stub, not a real workspace).
  • An embedded Dolt store (bd dolt show → Mode: EMBEDDED, Remotes: (none)) has no shareable endpoint — it is single-writer, in-process, filesystem-local to one host. Normal rigs inherit the city's endpoint (gc.endpoint_origin: inherited_city). That city endpoint may be managed_city (gc owns Dolt lifecycle) or city_canonical (external Dolt; gc does not manage it). When the city endpoint is down, there is no substitute — do not point multiple agents at one host's embedded store as a workaround; that produces write contention and yet another disconnected ledger.

Supply-Chain: Pin CI Components, Never to a Branch

Rule: every GitLab CI component: reference must resolve to a tag or a 40-character commit SHA. Never @main, @release/*, or any branch name — branches are moving targets and this has already caused four confirmed production incidents in one estate: a pipeline dead for four months (component silently stopped existing), a dangling feature-branch ref that killed every pipeline in a repo, a component that resolved to an empty (0-byte) file at @main, and a component whose runtime scripts existed in no ref it pointed to.

Sequencing, in order — do not skip step 1: 1. Fix the propagating template first. If a shared .gitlab-ci.template.yml (or equivalent onboarding doc) exists specifically to be copied into new repos, and it hands out a branch-pinned example, every future onboarding reintroduces the defect no matter how many existing consumers get fixed. One observed repo's entire CI was that template's two lines, copied verbatim, branch-pin included. 2. Cut/confirm the target tag consumers should point at. 3. Move existing consumers to the tag. 4. Enforce it in CI going forward (a job that fails any component: ref that isn't a tag or 40-char SHA) so it cannot regress.

Do not repin a consumer that deliberately tracks @main because it needs inputs not yet promoted to a release branch — fix the promotion gap instead, or the repin breaks real functionality.

Measurement Discipline — Traps That Produce Wrong Findings

Verified the hard way, more than once, in the same investigation:

  • mtime is not creation/birth time. A later process touching a directory (a probe, a reconciler) resets mtime and can make old, real drift look like fresh noise, or vice versa. Use birth time (stat with the platform's actual birth-time field) when provenance matters.
  • ps ax | grep <pattern> always matches its own invocation. Any "is this secret/process running" check must explicitly exclude the searching shell/grep process, or it will report false positives every time.
  • Case sensitivity in stored-data queries is not guaranteed. A Dolt/SQL LIKE '%operator must%' can silently miss "Operator must" — wrap in LOWER() (or the store's equivalent) or state explicitly that the check is case-sensitive.
  • State your search bounds explicitly. The most common source of a wrong finding in this estate has been an unstated scope: an unmentioned maxdepth, a store that wasn't re-scoped after a topology change, a grep over raw text instead of enumerated lines. Always name what was searched and what was excluded, not just what was found.
  • A local checkout is a deployment artifact, not a source-of-truth view. Do not derive a source-code claim from /opt/bluefly/* or any other on-disk checkout without a fresh read from the actual GitLab API/forge — checkouts on Oracle have been observed up to 23 commits behind main within the same day.
  • A squash-merge produces a commit the source branch's own history never contains. Continuing to commit to that same branch afterward guarantees git sees two unrelated histories for logically identical content — the next merge attempt reports a conflict, not because anything actually changed twice, but because the two sides can't recognize their shared ancestry. Confirmed twice in one day (2026-08-24): once as a re-merge conflict on this repository (blucity-docs!194), once as wrong provenance SHAs on Forge's side (a squash-merged commit carrying the source commit's message but a different SHA). The resolution method is the actual finding, not the conflict itself: taking --theirs/--ours blindly on a conflicted file assumes one side is simply newer — but a target branch that received other real work in the meantime can hold content the source branch never saw, and a blind resolution deletes it while reporting success. Diff both sides against each other before committing any conflict resolution; verify by content, not by which side "should" win.