Skip to content

ADR-0021 — RFP serving authority: registry.yaml vs LiteLLM routing

  • Status: Candidate — OPEN
  • Date: 2026-07-28
  • Related: ADR-0008 (upstream authority location)

Context

Two systems both claim authority over "what serves the RFP capability," with no cross-reference between them:

  1. agent-platform/models/orchestration's registry.yaml (the platform's self-declared "LLM Platform Model Registry") lists rfp: status: development, auto_deploy: false, endpoint: http://rfp.llm-platform.local. No code path in this repo (release_manager.py, registry_server.py) ever writes to this status field — promotion from development to production has no implemented mechanism; it would be a manual hand-edit.
  2. The live, in-production deployments/litellm/litellm-config.yaml (agent-docker) already defines named routes rfp-default / rfp-fast, in active use today, backed by anthropic/claude-sonnet-4-5-20250929 / claude-haiku-4-5-20251001 — not by the models/rfp repo's own trained model (TorchServe/vLLM/Triton serving stack, --served-model-name rfp-analyzer). The config's own comment states: "rfp-automation calls model: rfp-default; LiteLLM owns provider selection."

No issue, commit message, or document found anywhere (GitLab issues in the agent-platform/models group: zero open) records why RFP is served by Claude today, or whether/when the custom-trained model in models/rfp is meant to replace it.

Decision

Not established. This ADR records the divergence as evidence; it does not resolve it. Two candidate resolutions, presented without preference:

  • A — Explicit mapping. registry.yaml's rfp entry gains a field pointing at its current LiteLLM backend (litellm_route: rfp-default), making the Claude-backed interim state a declared, intentional fact instead of an unrecorded coincidence.
  • B — Declared separation. registry.yaml and litellm-config.yaml are formally scoped as unrelated systems — registry.yaml tracks the custom model's development lifecycle only; LiteLLM routing is a separate, independently-owned production concern with no obligation to converge.

Rule

None yet — this ADR is open pending an owner decision between A and B (or a third option not yet identified).

Consequences

Until resolved: anyone reading registry.yaml in isolation will incorrectly conclude RFP has no production backend (status: development), when a production backend (Claude, via LiteLLM) has existed and served traffic this entire time under a different name. This is the concrete, present-tense risk of leaving the divergence unrecorded.