Bluefly Infrastructure as Code — Master Architecture¶
GitLab Ultimate Reference Implementation¶
Repo: blueflyio/agent-platform/iac
Version: 1.0.0
Oracle Host: bluefly-platform.tailcf98b3.ts.net
Updated: 2026-04-18
Philosophy¶
This repository is a living example of GitLab Ultimate used to its absolute maximum. Every GitLab feature that exists is mapped to a real infrastructure concern. No feature left unused. No manual SSH deploy. No orphaned .env file. Everything is code, everything has an audit trail, everything has a policy.
Four non-negotiable principles:
- GitLab is the control plane. No deploys happen outside a GitLab pipeline.
- No secrets in git. Vault / 1Password / GitLab CI variables only.
- Import before create. All existing live state is imported via open-source tooling before any
terraform apply. - Every GitLab Ultimate feature is exercised. If it exists, it's mapped below.
Repository Layout¶
iac/
├── .gitlab-ci.yml # Root pipeline — orchestrates all below
├── .gitlab/
│ ├── CODEOWNERS # GitLab Ultimate: Code Owners enforcement
│ ├── compliance/
│ │ ├── compliance-pipeline.yml # GitLab Ultimate: Compliance Pipelines
│ │ └── scan-policy.yml # GitLab Ultimate: Scan Execution Policies
│ ├── issue_templates/
│ │ ├── infra-change.md # GitLab: Issue templates
│ │ └── security-finding.md
│ └── merge_request_templates/
│ ├── terraform-change.md # GitLab: MR templates with checklists
│ └── security-review.md
│
├── terraform/
│ ├── oci/ # Oracle Cloud Infrastructure
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ ├── outputs.tf
│ │ ├── versions.tf
│ │ └── modules/
│ │ ├── compute/ # Oracle Compute instances
│ │ ├── networking/ # VCN, subnets, security lists
│ │ └── storage/ # Block volumes, object storage
│ ├── cloudflare/ # Cloudflare — tunnels, DNS, Zero Trust
│ │ ├── main.tf
│ │ ├── tunnels.tf
│ │ ├── dns.tf
│ │ ├── zero-trust.tf
│ │ └── variables.tf
│ ├── gitlab/ # GitLab groups, projects, CI vars, runners
│ │ ├── main.tf
│ │ ├── groups.tf
│ │ ├── projects.tf
│ │ ├── ci-variables.tf
│ │ └── runners.tf
│ └── tailscale/ # Tailscale ACLs and device policies
│ ├── main.tf
│ └── acls.tf
│
├── gitlab-components/ # GitLab CI/CD Component Catalog
│ ├── README.md
│ ├── templates/
│ │ ├── terraform-plan.yml # Component: terraform plan/apply
│ │ ├── docker-build.yml # Component: docker build with Chainguard
│ │ ├── security-scan.yml # Component: full security suite
│ │ ├── compliance-check.yml # Component: ContractPlane + Cedar
│ │ ├── caddy-deploy.yml # Component: Caddy/FrankenPHP deploy
│ │ └── k8s-deploy.yml # Component: kagent/k8s apply
│ └── catalog.yml # GitLab CI Catalog registration
│
├── environments/
│ ├── production/
│ │ ├── docker-compose.yml # Single source of truth for prod compose
│ │ └── .env.vault # Vault-encrypted env references (no secrets)
│ └── staging/
│ ├── docker-compose.yml
│ └── .env.vault
│
└── docs/
├── ARCHITECTURE.md # This file
├── STATE-IMPORT.md # How to import existing resources
├── RUNBOOKS.md # Day-2 operational procedures
└── GITLAB-FEATURES.md # Full GitLab Ultimate feature mapping
GitLab Ultimate Feature Map¶
Every feature below is actively used in this repository.
Source Control & Code Review¶
| Feature | Usage | Location |
|---|---|---|
| Protected branches | main requires 2 approvals, no force push |
GitLab settings |
| Code Owners | Security and Terraform files need specific approvers | .gitlab/CODEOWNERS |
| Merge Request Approvals | Minimum 2 approvals, security team for infra changes | MR settings |
| Required merge checks | Pipelines must pass, all threads resolved | Branch settings |
| Push rules | No secrets, signed commits required | Push rules settings |
| Repository mirroring | Mirrors to backup GitLab instance every 30min | Mirror settings |
CI/CD¶
| Feature | Usage | Location |
|---|---|---|
| CI/CD Components | Reusable terraform, docker, security components | gitlab-components/ |
| CI Inputs | Typed, validated inputs on all components | Component spec.inputs |
| Parent-child pipelines | Root triggers domain-specific sub-pipelines | .gitlab-ci.yml |
| Directed Acyclic Graphs (DAG) | needs: for parallel terraform plans |
Pipeline YAML |
| Merge trains | Serialized production deploys | MR settings |
| Environments & deployments | production / staging with deployment history | .gitlab-ci.yml |
| Review Apps | Staging preview URLs on every infra MR | review stage |
| Release management | Semantic releases with changelog on infra tags | Release jobs |
| GitLab Terraform state | Remote state in GitLab HTTP backend | versions.tf |
| GitLab Kubernetes Agent | kagent-based k8s management (no cluster cert exposure) | k8s/ |
| Runner fleet management | Tagged runners for OCI, Chainguard, arm64 | Runner config |
Security (The Big Ones — Ultimate Only)¶
| Feature | Usage | Location |
|---|---|---|
| SAST | Semgrep scanning on all Terraform + YAML | Auto-configured |
| IaC Scanning | Checkov/KICS on all Terraform files | Auto-configured |
| Secret Detection | Gitleaks on every push and MR | Auto-configured |
| Dependency Scanning | All npm/pip/composer lockfiles | Auto-configured |
| Container Scanning | All Docker images scanned pre-deploy | .gitlab-ci.yml |
| DAST | API scanning on staging endpoints | DAST config |
| Fuzz testing | API fuzz on contractplane endpoints | Fuzz config |
| License Compliance | OSS license gate on all dependencies | License policy |
| Security Policies (Scan Execution) | Force-run scans on all MRs — no bypass | .gitlab/compliance/ |
| Security Policies (Merge Request) | Block merge on critical/high vulns | Security policy |
| Vulnerability Management | Triage, assign, SLA tracking per vuln | Vulnerability UI |
| Security Dashboard | Group-level view across all 126 repos | Group security tab |
| Dependency Proxy | Pull-through cache for container images | Package registry |
Compliance¶
| Feature | Usage | Location |
|---|---|---|
| Compliance Pipelines | Injected into every pipeline — cannot be bypassed | .gitlab/compliance/ |
| Compliance Frameworks | NIST 800-53 / FedRAMP framework applied to group | Group compliance |
| Compliance Center | Dashboard of all violations across group | Group settings |
| Audit Events | Every action streamed to external SIEM | Audit settings |
| Audit Event Streaming | Forwarded to ContractPlane ledger endpoint | Audit streaming |
| Compliance reports | Framework adherence per project | Compliance UI |
Planning & Value Stream¶
| Feature | Usage | Location |
|---|---|---|
| Epics & Multi-level Epics | All infra work organized in epics | Group epics |
| Roadmaps | Q2 2026 infra migration visible as timeline | Group roadmap |
| DORA Metrics | Deployment frequency, lead time, CFR, MTTR | Analytics |
| Value Stream Analytics | End-to-end from issue to deploy | Group analytics |
| Requirements Management | NIST controls mapped as requirements | Requirements |
| Test Management | Infra test cases tracked with run history | Test cases |
| OKR Management | Platform OKRs visible to stakeholders | OKR tracking |
AI (Duo Enterprise)¶
| Feature | Usage | Location |
|---|---|---|
| Duo Code Suggestions | Active for all IaC engineers | Duo settings |
| Duo Chat | In-IDE context for Terraform/YAML | Duo settings |
| Duo Agent Platform | Autonomous pipeline remediation agent | Duo agents |
| Agentic SAST Resolution | Auto-generates MR to fix SAST findings | Vulnerability UI |
| Root Cause Analysis | Pipeline failure analysis | CI/CD |
| Vulnerability Explanation | AI explains each security finding | Security dashboard |
Package & Container Registry¶
| Feature | Usage | Location |
|---|---|---|
| Container Registry | All service images stored in GitLab | Registry |
| Terraform Module Registry | Shared OCI/Cloudflare modules | Registry |
| Helm Chart Registry | kagent and k8s chart storage | Registry |
| Dependency Proxy | Pull-through cache (DockerHub rate limit bypass) | Dependency proxy |
| Virtual Registry (beta) | Multi-upstream registry aggregation | Virtual registry |
Infrastructure State Import Plan¶
Before any terraform apply runs in CI, all existing live resources must be imported.
Use the following open-source import tools:
| Layer | Tool | Command | Target |
|---|---|---|---|
| Cloudflare tunnels, DNS, Zero Trust | cf-terraforming |
cf-terraforming generate --resource-type cloudflare_tunnel |
terraform/cloudflare/ |
| GitLab groups, projects, variables | gitlab-terraform + manual import |
terraform import gitlab_group.root <id> |
terraform/gitlab/ |
| OCI compute, networking | oci-terraformer / OCI Resource Discovery |
oci resource-manager stack ... |
terraform/oci/ |
| Tailscale ACLs | Tailscale provider + terraform import |
terraform import tailscale_acl.main acl |
terraform/tailscale/ |
| Docker Compose services | kompose (K8s conversion reference only) |
kompose convert |
environments/ |
All import commands are codified as one-time migration jobs in .gitlab/migrations/.
They run manually (never automatically) and produce state files committed to GitLab's HTTP backend.
Secret Management¶
Rule: No raw secrets anywhere in git or environment variables visible in logs.
| Secret Type | Storage | How consumed |
|---|---|---|
| OCI credentials | GitLab CI variable (masked, protected) | $OCI_TENANCY_OCID etc. |
| Cloudflare API token | GitLab CI variable (masked, protected) | $CLOUDFLARE_API_TOKEN |
| Tailscale auth key | GitLab CI variable (masked, protected) | $TAILSCALE_AUTH_KEY |
| Application secrets (DB passwords, API keys) | 1Password Connect | op run -- docker compose up |
| ContractPlane signing key | GitLab CI variable (masked, protected) | $CONTRACTPLANE_SIGNING_KEY |
| Chainguard pull token | GitLab CI variable (masked, protected) | $CHAINGUARD_TOKEN |
GitLab Secret Detection runs on every push. Any accidental commit of a real secret triggers: 1. Pipeline block 2. Security finding in vulnerability dashboard 3. Slack alert via webhook 4. Automatic MR close (via compliance pipeline)
Caddy / FrankenPHP — Canonical Proxy¶
Caddy is the only ingress layer on the Oracle host. No Nginx. No Apache at the host level. FrankenPHP runs as the PHP app server inside Drupal containers. Caddy reverse-proxies all services.
Service routing map:
contractplane.ai → contractplaneai container :3000
copaw.us → copaw-deploy container :8080
mcp.blueflyagents.com → agent-protocol :3010
gkg.blueflyagents.com → gkg service :3011
marketplace → marketplace container :3001
agentdash → agentdash-node :3002
drupal-fleet → frankenphp :80 (internal)
All Caddy config is in terraform/ as a GitLab CI-managed file — never edited by hand.
Chainguard Base Images¶
All Docker builds use Chainguard base images:
# All services use chainguard/node or chainguard/python as base
FROM cgr.dev/chainguard/node:latest AS base
FROM cgr.dev/chainguard/python:latest AS base
FROM cgr.dev/chainguard/php:latest AS base # Drupal/FrankenPHP
The docker-build CI component enforces this via a Chainguard image lint job.
Container Scanning runs on every built image before deploy.
Kubernetes Recovery Plan¶
K8s (k3s) is currently down on Oracle (127.0.0.1:6443 refused).
Recovery steps (Phase 0, Week 1):
1. SSH runner executes sudo systemctl restart k3s via a manual pipeline job
2. kubectl get nodes verification job confirms health
3. kagent manifests in agent-docker/k8s/kagent-*.yaml applied via GitLab K8s Agent
4. All future k8s operations go through GitLab Agent — no direct kubectl from CI
openclaw Ownership Investigation¶
~/.openclaw/ is owned by opc (OCI default user), not ubuntu.
This is a permissions drift — /opt/openclaw/ is the canonical Docker volume mount.
Resolution (Phase 0):
1. stat /opt/openclaw/ and stat ~/.openclaw/ via pipeline job
2. If ~/.openclaw/ is a stale config dir → remove or chown to ubuntu
3. If opc user is running openclaw → add ubuntu to same group, or migrate to Docker volume
4. Enforce: all service data in /opt/{service}/data/ — no home-dir data volumes
Phase Plan¶
Phase 0 — Triage & Baseline (Week 1)¶
- [ ] Fix k3s (manual pipeline job)
- [ ] Import Cloudflare state with
cf-terraforming - [ ] Import GitLab groups/projects with Terraform
- [ ] Import OCI resources with
oci-terraformer - [ ] Move
/opt/.env→ GitLab CI variables + 1Password - [ ] Fix unhealthy containers: contractplane-gateway, litellm-proxy, duadp-register
- [ ] Investigate openclaw
opcownership - [ ] Enable all GitLab Ultimate security scans on this repo
- [ ] Set up Compliance Pipelines + Scan Execution Policies
Phase 1 — Full Terraform Coverage (Week 2–3)¶
- [ ] All Cloudflare tunnels as Terraform
- [ ] All DNS records as Terraform
- [ ] All GitLab CI variables as Terraform (encrypted in state)
- [ ] OCI compute instances as Terraform
- [ ] Tailscale ACLs as Terraform
- [ ] Enable GitLab Terraform state HTTP backend for all workspaces
Phase 2 — CI Components & Catalog (Week 3–4)¶
- [ ] Publish all gitlab-components to CI Catalog
- [ ] All services consume components instead of copy-pasted YAML
- [ ] Enable merge trains on production environment
- [ ] DORA metrics dashboard live
Phase 3 — K8s & kagent (Week 4–5)¶
- [ ] k3s healthy and managed via GitLab K8s Agent
- [ ] kagent fleet deployed and reporting
- [ ] All agent workloads that fit in k8s migrated from Docker Compose
- [ ] Helm charts for all services in GitLab Helm Registry
Phase 4 — Full Duo Agent Platform (Week 5–6)¶
- [ ] Pipeline remediation agent active
- [ ] Agentic SAST resolution enabled
- [ ] ContractPlane Skills Contract governing all Duo agent skill invocations
- [ ] OSSA manifests for all GitLab Duo agents registered in DUADP
This document is the single source of truth for Bluefly infrastructure architecture. All changes go through MR → approval → CI → deploy. No exceptions.