Skip to content

AMCS Phase 1 Demo Runbook — OSSA-Governed Accessibility Reviewer

Version: 1.0
Date: 2026-05-21
Scope: Phase 1 proof — single agent, local Drupal, Cedar policy, ContractPlane evidence
Audience: Operator running the demo from a clean checkout

Prerequisites

  • DDEV installed and working (ddev version returns a valid result)
  • Drupal CMS 2.0 site (via ddev config --project-type=drupal --php-version=8.3)
  • LiteLLM running (docker-compose on NAS or local, port 4000)
  • ContractPlane dev service or mock endpoint
  • Cedar policy file: accessibility-reviewer.cedar
  • OSSA manifest: accessibility-reviewer.ossa.yaml

Environment Setup

Exact commands to prepare the demo environment:

# 1. Clone ContextControl.ai or use existing DDEV site
cd PROJECTS/ContextControl.ai

# 2. Start DDEV
ddev start

# 3. Enable required modules
ddev drush en -y ai ai_agents ai_agents_ossa tool cedar_policy contractplane_client mcp

# 4. Verify site health
ddev drush cr -y
ddev drush watchdog:show --severity=Error --count=20

[!IMPORTANT] If watchdog shows errors after module enable, resolve them before proceeding. A clean watchdog is required for a valid demo.

Step 1: Import OSSA Manifest

  1. Navigate to /admin/config/ai/ossa/import
  2. Upload accessibility-reviewer.ossa.yaml
  3. Expected: OssaAgent config entity created
  4. Verify:
ddev drush config:get ai_agents_ossa.agent.accessibility_reviewer

The output should show the agent's OSSA metadata including agent_id, capabilities, and allowed_tools.

Step 2: Validate Capabilities

  1. Check Tool API plugins derived from OSSA capabilities
  2. Verify:
ddev drush eval "print_r(\Drupal::service('plugin.manager.tool')->getDefinitions());"
  1. Expected: Tools listed matching OSSA capabilities:
  2. content_read
  3. wcag_check
  4. evidence_submit

Step 3: Bind Cedar Policy

  1. Navigate to /admin/config/ai/cedar-policy
  2. Import accessibility-reviewer.cedar
  3. Policy rules:
  4. ALLOW: read content + draft recommendation
  5. DENY: direct publish
  6. Verify: Policy entity created and bound to accessibility-reviewer agent
ddev drush config:get cedar_policy.policy.accessibility_reviewer

Step 4: Create Test Content

  1. Create a Basic Page node with known accessibility issues:
  2. Missing alt text on an image
  3. Low contrast text (e.g., light gray on white)
  4. Note the node ID for use in subsequent steps
ddev drush node:create --type=page --title="AMCS Demo - Accessibility Test Page"

Or create via the admin UI at /node/add/page.

Step 5: Run Accessibility Review

  1. Trigger agent execution via admin UI or Drush:
ddev drush ai-agents:execute accessibility_reviewer --target=node:<NID>
  1. Expected:
  2. Agent reads content
  3. Calls LiteLLM for analysis
  4. Produces accessibility report
  5. Verify: Report appears in agent output (stdout or admin UI results panel)

Step 6: Attempt Unauthorized Action

  1. Agent attempts to publish content directly (or call a tool outside its allowed_tools)
  2. Expected: Cedar policy DENIES the action
  3. Verify:
# Check watchdog for denial
ddev drush watchdog:show --type=cedar_policy --count=10

# Confirm denial event emitted
ddev drush watchdog:show --severity=Warning --count=10

The denial log entry should reference the Cedar policy ID and the denied action.

Step 7: Show Cedar Denial Evidence

  1. Check ContractPlane evidence for the denial event
  2. Expected: Evidence event with:
  3. policy_decision = 'deny'
  4. policy_id matching the Cedar policy
  5. Verify:
ddev drush sqlq "SELECT event_id, FROM_UNIXTIME(occurred_at), event_type, subject_id, integrity_hash FROM contractplane_evidence WHERE event_type = 'policy.denied' ORDER BY occurred_at DESC LIMIT 5;"

Step 8: Show Success Evidence

  1. Check ContractPlane evidence for successful tool calls
  2. Expected: Evidence events with:
  3. policy_decision = 'allow'
  4. tool_called
  5. token_usage
  6. trace_id
  7. Verify:
ddev drush sqlq "SELECT event_id, FROM_UNIXTIME(occurred_at), event_type, subject_id FROM contractplane_evidence WHERE event_type = 'tool.executed' ORDER BY occurred_at DESC LIMIT 10;"

Step 9: Export AMC Bundle

  1. Navigate to /admin/config/ai/ossa/export or use CLI
  2. Export accessibility-reviewer as AMC Bundle v2
  3. Expected: Directory with subdirectories:
  4. ossa/ — OSSA manifest
  5. policies/ — Cedar policy files
  6. drupal/ — Drupal config exports
  7. evidence/ — Evidence schema
  8. Verify:
ls -la accessibility-reviewer-v1.0.0/

Step 10: Validate Bundle

  1. Run bundle validation:
npx @tobilu/agent-buildkit validate-bundle ./accessibility-reviewer-v1.0.0/
  1. Expected: PASS — all schemas valid, no secrets, policy present, evidence schema present

[!NOTE] If agent-buildkit validate-bundle does not exist yet, use manual schema validation with ajv:

npx ajv validate -s ossa-agent.schema.json -d accessibility-reviewer-v1.0.0/ossa/accessibility-reviewer.ossa.yaml
npx ajv validate -s amc-bundle.schema.json -d accessibility-reviewer-v1.0.0/bundle.json

Step 11: Show Conformance Report (can be faked in Phase 1)

  1. Generate conformance report from evidence + bundle validation
  2. Expected: Bronze level (valid OSSA manifest + capability schema)

[!NOTE] This step can be a static report in Phase 1. Automated conformance scoring comes in Phase 2.

Example static report output:

AMCS Conformance Report
=======================
Agent:    accessibility-reviewer
Version:  1.0.0
Level:    Bronze
Criteria:
  ✅ Valid OSSA manifest
  ✅ Capability schema present
  ✅ Cedar policy bound
  ✅ Evidence schema present
  ⬜ DUADP registration (Phase 2)
  ⬜ Marketplace listing (Phase 2)

Step 12: Clean Checkout Reproducibility Check

  1. Start from scratch:
ddev delete -Oy
ddev config --project-type=drupal --php-version=8.3
ddev start
# Install from recipe or standard profile
ddev drush site:install --account-name=admin --account-pass=admin -y
  1. Run all steps (1–11) again
  2. Expected: Identical results

Success Criteria

  • [ ] OSSA manifest imports without error
  • [ ] Tool API plugins derive from OSSA capabilities
  • [ ] Cedar policy blocks unauthorized tool calls
  • [ ] Authorized tool calls succeed and produce results
  • [ ] ContractPlane evidence records both allow and deny events
  • [ ] Evidence events contain trace_id, agent_id, tool_called, policy_decision
  • [ ] AMC bundle exports and validates
  • [ ] Entire demo repeatable from clean checkout

Known Phase 1 Limitations

Limitation Status Target
Kagent external runtime NOT included in this demo Phase 1 Deliverable 2 Phase 1 D2
DUADP registration NOT included Deferred Phase 2
Marketplace listing NOT included Deferred Phase 2
Conformance report is static/manual Acceptable Automated in Phase 2
agent-buildkit validate-bundle CLI may not exist yet Manual validation acceptable Phase 1 stretch
ContractPlane may be a mock/dev endpoint Acceptable Production in Phase 2

[!WARNING] This runbook targets the Phase 1 proof of concept. Production hardening, DUADP registration, marketplace publishing, and automated conformance are explicitly out of scope.