Responses Proxy architecture

Responses Proxy runs a FastAPI backend, Redis live state, a telemetry worker, and SQL persistence. The Responses console lives in the Morph Cloud frontend and talks to the backend through a same-origin BFF that obtains Morph service delegation auth server-side.

Runtime components

Component	Responsibility
Frontend dashboard	Browser UI for providers, aliases, proxy keys, metrics, requests, decisions, and live state.
Frontend BFF	Same-origin server routes that validate org context, request Morph service delegation auth, sanitize headers, and proxy console calls.
FastAPI backend	Exposes health, Responses-compatible data-plane routes, admin routes, registry routes, and OpenAPI docs.
Provider registry	Stores ServiceProvider rows in SQL and validates provider settings.
Model registry	Stores client-facing aliases that resolve to providers and upstream model ids.
Provider adapters	Pass through Responses traffic or translate Responses to Chat Completions or Anthropic Messages.
Load balancer	Applies response affinity, session affinity, and weighted least-connections within the selected provider pool.
Redis state store	Tracks live resource health, counters, affinity, active resource ids, recent decisions, and telemetry queue depth.
Telemetry worker	Consumes Redis stream events, records raw telemetry, and idempotently replays durable domain rows.
SQL database	Stores providers, aliases, response state, sessions, requests, decisions, token usage, and raw telemetry events.

Request flow

For POST /v1/responses:

1. The proxy validates the per-org proxy key from Authorization: Bearer <key>.
2. The proxy resolves the key to its owning org, parses JSON, and reads model.
3. The model value must resolve to a registered alias in that org unless the request is a continuation.
4. The alias pins the provider pool and may rewrite the upstream model id.
5. Response affinity is checked, then session affinity.
6. If no affinity applies, the load balancer selects a healthy resource by weighted least-connections.
7. Incoming proxy-key authorization is stripped.
8. The selected provider credential is injected into the configured provider auth header.
9. The provider adapter forwards Responses traffic or translates it to the upstream protocol.
10. The response is returned as canonical Responses JSON or SSE.
11. Response id, status, latency, health, response state, and request telemetry are recorded.
12. Request and decision rows are persisted inline for dashboard freshness, while raw telemetry is also published to Redis for worker replay and auditing.

Create-time errors:

• empty or unroutable registry: 503
• missing or invalid proxy key: 401
• missing model: 400
• unknown alias: 404

Retryable upstream failures (429, 502, 503, 504, connection errors, and timeouts) can fail over to another healthy resource in the same org pool up to PROXY_UPSTREAM_MAX_ATTEMPTS. Non-429 4xx and known quota or balance failures are terminal. Streaming requests only fail over before bytes stream, and affinity-pinned continuations do not fail over.

Continuations

Continuations using previous_response_id or /v1/responses/resp_... route by response affinity and do not require a create-time alias. If a continuation also includes a registered alias, the alias still pins and rewrites the request.

For chat_completions and anthropic_messages, the proxy creates its own resp_* ids and stores reconstructed messages in SQL. This lets Codex continue conversations against stateless upstream APIs when the same SQL database is reused.

Streaming and WebSocket traffic

{"stream": true} returns a streaming response. Responses providers stream mostly unchanged. Chat Completions and Anthropic providers are converted into Responses SSE events before they reach the client. When an adapted upstream stream is active but not producing visible text or tool arguments, the proxy emits an SSE comment plus response.in_progress.

The proxy extracts response ids and usage, persists response state when needed, and records telemetry when the stream closes. Request and response body persistence is controlled separately by PROXY_CAPTURE_REQUEST_BODY, PROXY_CAPTURE_RESPONSE_BODY, and PROXY_MAX_CAPTURE_BYTES.

WS /v1/responses uses model alias query or header metadata to select the provider pool, opens the provider WebSocket URL, bridges messages, captures output up to the configured limit, and emits completion telemetry.

Adapter behavior

Supported routable adapters:

• responses
• chat_completions
• anthropic_messages

For non-Responses providers, function tools map to provider-native JSON tool or function calls. Codex custom tools are wrapped as upstream JSON tools with an input string, then converted back to Responses custom_tool_call items. Codex freeform apply_patch tools are treated as custom tools even when an upstream-compatible schema labels them as functions, and custom tool arguments tolerate keys such as input, patch, diff, content, cmd, and command. Namespace tool containers, including nested namespace/member shapes, are flattened into provider-native JSON tools.

OpenAI-hosted Responses tools that cannot run on Chat Completions or Anthropic providers, such as web_search, are omitted for compatibility.

Claude Code registered aliases use a separate Messages conversion pipeline. anthropic_messages remains native. responses maps Messages content directly into canonical Responses input. chat_completions first uses that canonical mapping, then reuses the Chat adapter for request and response conversion. Both translated paths convert canonical text and function calls back to Claude Messages JSON or SSE.

Alias capabilities.claude_code is the routing and feature boundary for this pipeline. It controls discovery, profile/protocol matching, portable image and document input, client tools, parallel tools, and an explicit hosted-tool allowlist. Only web_search is portable to Responses; Chat Completions has no hosted-tool mapping. Unsupported media sources, content blocks, and server tools fail before upstream dispatch.

Claude gateway bodies are limited by PROXY_CLAUDE_GATEWAY_MAX_BODY_BYTES before JSON parsing. Base64 media is redacted from captured request telemetry, even when body capture is enabled.

Health checks

Health checks call each provider resource base URL plus PROXY_HEALTH_CHECK_PATH or the provider health_path.

Status classification:

• 2xx: healthy
• 401 or 403: unhealthy
• 429 or >=500: degraded
• anything else: unhealthy

Non-Responses providers with health_path=null skip periodic probes and rely on /admin/providers/test plus /admin/models/test.

Trigger a manual check:

curl -X POST -H "$ADMIN_HEADER" "$RESPONSES_PROXY_BASE_URL/admin/health/check"

Persistence boundary

Use Redis for live decisions and SQL for durable routing, response state, and analytics. Redis counters, affinity, queue depth, active-resource ids, and live health can reset on deploy. SQL-backed providers, aliases, response states, sessions, requests, decisions, token usage, and telemetry events persist when the same database is reused.

Runtime performance

Normal data-plane requests use in-process routing snapshots rather than reading provider and alias rows from SQL. Proxy-key org resolution is cached in process for PROXY_PROXY_KEY_CACHE_TTL_SECONDS, which defaults to 30 seconds. The default upstream HTTP client is pooled across requests, so provider HTTP/TLS connections can be reused.

Resource selection reads the Redis resources:active set and then fetches the active resource hashes; it does not scan the whole Redis keyspace on every request. Health checks reuse one HTTP client per check cycle.

The worker is still important for raw telemetry events and replay after transient failures, but dashboard request and decision rows are written inline so the frontend does not depend on worker lag for fresh metrics.

Frontend boundary

The browser does not call the backend service directly for admin workflows. It calls Morph Cloud same-origin /api/responses/* routes. The BFF:

• obtains a Morph service delegation token for admin and registry calls
• binds calls to the active Morph organization
• validates active org context
• validates $SECRET provider credential references against org secrets
• exposes alias-level Claude Code profiles and compatibility toggles
• tests model aliases through either the Responses or Claude Code surface
• clamps and whitelists query params
• strips secret-shaped fields from admin responses before they reach the browser
• rate-limits registry, admin, key, and data-plane proxy routes
• rejects cross-origin mutations before forwarding
• streams SSE without adding a timeout