Responses Proxy configuration

Use this page when you operate or deploy Responses Proxy. If you only need to send requests through the hosted service, start with Getting started.

The hosted backend service is:

https://agent-responses-proxy.svc.cloud.morph.so/

Client SDKs and Codex should use:

https://agent-responses-proxy.svc.cloud.morph.so/v1

Production routing state comes from SQL provider and model alias rows. The service can boot with an empty registry, but data-plane create requests return 503 until a routable provider and alias exist.

Core variables

Variable	Default	Purpose
PROXY_REDIS_URL	redis://127.0.0.1:6379/0	Redis live state and telemetry stream.
PROXY_DATABASE_URL	unset	Preferred SQL URL; use Neon/Postgres in production.
SERVICE_DATABASE_URL	unset	Fallback SQL URL.
DATABASE_URL	unset	Fallback SQL URL.
PROXY_SQLITE_URL	sqlite:///./data/proxy.sqlite3	Local fallback.
PROXY_BOOTSTRAP_ORG_ID	unset	Required for legacy global-row migration; startup raises if legacy rows exist and this is unset.
PROXY_ADMIN_AUTH_MODE	morph_org_or_delegation	Morph-only admin auth mode: morph_org, morph_delegation, or morph_org_or_delegation.
PROXY_ADMIN_ALLOWED_ORG_ID	unset	Allowed Morph organization id for direct org-token auth.
PROXY_ADMIN_ALLOWED_ORG_SLUG	unset	Optional allowed Morph organization slug.
PROXY_ADMIN_ALLOWED_ROLES	org:admin,org:member	Comma-separated roles allowed to use admin APIs.
PROXY_MORPH_API_BASE_URL	https://cloud.morph.so/api	Morph API used to validate org membership.
PROXY_MORPH_SERVICE_API_BASE_URL	https://service.svc.cloud.morph.so/service	Morph service auth API used to introspect delegation tokens.
PROXY_ADMIN_DELEGATION_SERVICE_API_KEY	unset	Service-owner API key for delegation introspection.
PROXY_ADMIN_DELEGATION_SERVICE_NAME	agent-responses-proxy	Expected delegation service name.
PROXY_ADMIN_DELEGATION_REQUIRED_SCOPE	responses:admin	Required delegation scope for /admin/*.
PROXY_ADMIN_DELEGATION_AUDIENCE	unset	Optional delegation audience; defaults to the service name.
PROXY_ADMIN_AUTH_TIMEOUT_SECONDS	5.0	Timeout for Morph admin auth checks.
PROXY_ADMIN_AUTH_CACHE_TTL_SECONDS	60.0	TTL for cached admin auth decisions.
PROXY_SECRET_ENCRYPTION_KEY	unset	Required for inline provider credentials.
PROXY_ROUTING_REFRESH_INTERVAL_SECONDS	10.0	Multi-replica SQL refresh interval; 0 disables periodic refresh.
PROXY_AFFINITY_TTL_SECONDS	604800	Redis TTL for session and response affinity mappings.
PROXY_REQUEST_TIMEOUT_SECONDS	600.0	Upstream request timeout.
PROXY_UPSTREAM_MAX_ATTEMPTS	2	Max upstream attempts for retryable failover within the same org pool.
PROXY_MAX_CAPTURE_BYTES	2000000	Maximum captured request or response body bytes.
PROXY_CAPTURE_REQUEST_BODY	false	Persist captured request bodies when enabled.
PROXY_CAPTURE_RESPONSE_BODY	false	Persist captured response bodies when enabled.
PROXY_PROXY_KEY_CACHE_TTL_SECONDS	30.0	In-process positive cache TTL for data-plane proxy-key org resolution. Key revocation can take up to this long to affect a running instance. Invalid keys are cached for at most 5 seconds.
PROXY_ALLOW_HTTP_PROVIDER_URLS	false	Local-only allowance for HTTP provider URLs. Keep false in production.
PROXY_ALLOW_PRIVATE_PROVIDER_URLS	false	Local-only allowance for private-network provider URLs. Keep false in production.
PROXY_CREDENTIAL_REF_ENV_ALLOWLIST	unset	Extra env names allowed for credential_ref=env:NAME.
PROXY_ENV_CREDENTIAL_OWNER_ORG_IDS	unset	Comma-separated org ids allowed to resolve runtime env credentials beyond the bootstrap org.
PROXY_IN_PROCESS_WORKER	false	Run worker inside the API process.
PROXY_DASHBOARD_EVENT_INTERVAL_SECONDS	1.0	Minimum interval for /admin/events state snapshots.

When PROXY_CAPTURE_REQUEST_BODY=true, Claude gateway telemetry redacts base64 image and document data before persistence. URL paths, prompts, filenames, and other non-media request fields remain visible, but URL userinfo, query values, and fragments are removed.

Claude Code variables

Variable	Default	Purpose
PROXY_CLAUDE_GATEWAY_UPSTREAM_BASE_URL	https://api.anthropic.com/v1	Direct-mode Anthropic upstream for /v1/messages.
PROXY_CLAUDE_GATEWAY_PROXY_KEY_HEADER	x-responses-proxy-key	Header used for the org-scoped proxy key on Claude Code gateway routes. Must not be authorization or x-api-key.
PROXY_CLAUDE_GATEWAY_DIRECT_MODELS	unset	Comma-separated Anthropic model ids allowed for direct passthrough. Registered aliases do not require this.
PROXY_CLAUDE_GATEWAY_CHAT_COMPLETIONS_ENABLED	true	Enables Chat Completions aliases for Claude Code discovery and Messages routing. Set false as a deployment-wide rollback switch.
PROXY_CLAUDE_GATEWAY_MAX_BODY_BYTES	33554432	Maximum Claude gateway request body, including base64 media expansion. Must be positive; oversized requests return 413.
PROXY_ENABLE_CLAUDE_CODE_TOKEN_ESCROW	false	Enables Claude Code OAuth setup sessions and claude_code_oauth provider credential rows.
PROXY_CLAUDE_CODE_OAUTH_REFRESH_URL	unset	Deployment-owned refresh endpoint for stored Claude Code OAuth credentials.
PROXY_CLAUDE_CODE_OAUTH_CLIENT_ID	unset	Client id sent during server-side OAuth refresh when required.
PROXY_CLAUDE_CODE_OAUTH_REFRESH_SKEW_SECONDS	300	Refresh stored OAuth credentials this many seconds before expiry.
PROXY_CLAUDE_CODE_OAUTH_REFRESH_TIMEOUT_SECONDS	10.0	Timeout for OAuth refresh requests.

Health and telemetry variables

Variable	Default	Purpose
PROXY_HEALTH_CHECK_ENABLED	true	Start periodic provider health checks.
PROXY_HEALTH_CHECK_INTERVAL_SECONDS	30.0	Check interval.
PROXY_HEALTH_CHECK_TIMEOUT_SECONDS	10.0	Per-resource timeout.
PROXY_HEALTH_CHECK_PATH	models	Default relative provider health path.
PROXY_RESOURCE_COOLDOWN_SECONDS	30	Degraded cooldown after failures.
PROXY_TELEMETRY_STREAM	telemetry:events	Redis stream name.
PROXY_TELEMETRY_GROUP	responses-proxy-workers	Worker consumer group.
PROXY_TELEMETRY_CONSUMER	worker-1	Worker consumer prefix.

Provider configuration

Register providers through the Responses console/BFF or through the authenticated /admin/providers API. Providers, aliases, and proxy keys persist in SQL when the same PROXY_DATABASE_URL and PROXY_SECRET_ENCRYPTION_KEY are reused.

Preferred BYOS credentials live in /admin/providers/{provider_id}/credentials rows. If at least one enabled credential row resolves, routing uses those rows. If no enabled credential rows exist, the provider falls back to provider-level sources: env-sourced key slots, encrypted inline api_key, then credential_ref=env:NAME.

To load-balance multiple physical API keys behind one provider, register the provider once and add deployment env vars by provider id:

# provider id: openai
OPENAI_API_KEY=sk-primary
OPENAI_API_KEY_1=sk-secondary
OPENAI_API_KEY_3=sk-fourth

# provider id: kimi.k2
KIMI_K2_API_KEY=...

The deploy script can forward matching ^[A-Z0-9_]+_API_KEY(_N)?$ values through PROVIDER_API_KEYS_ENV. It logs only the forwarded count and blocks sensitive names such as MORPH_API_KEY, PROXY_*, DATABASE_*, and SERVICE_*.

Env-sourced keys are deployment-global. Runtime env credentials resolve only for PROXY_BOOTSTRAP_ORG_ID by default, or for org ids listed in PROXY_ENV_CREDENTIAL_OWNER_ORG_IDS. Use org-distinct provider ids when different orgs need separate physical credentials.

After provider or alias mutation, the current process refreshes routing immediately. Other replicas refresh from SQL within PROXY_ROUTING_REFRESH_INTERVAL_SECONDS.

Use POST /admin/routing/reload for an explicit reload.

Caching and hot paths

The data plane is designed to avoid Neon/Postgres reads on normal requests:

• provider resources and aliases are loaded into an in-process routing snapshot
• proxy-key-to-org resolution is cached for PROXY_PROXY_KEY_CACHE_TTL_SECONDS
• Redis stores live affinity, health, counters, and active resource ids
• the default upstream HTTP client reuses a pooled httpx client across requests

SQL remains the durable source of truth. Registry mutations refresh the current process immediately and other replicas on the configured refresh interval. Request and decision rows are persisted inline for dashboard freshness, and the telemetry worker still records raw Redis stream events and replays missed domain rows idempotently.

The proxy-key cache improves hot-path latency and reduces database reads. The tradeoff is revocation visibility: a revoked key may continue authenticating on a running instance until the positive cache entry expires. Set PROXY_PROXY_KEY_CACHE_TTL_SECONDS=0 only when immediate revocation matters more than request-path efficiency.

Local development

Run local development when you are changing the proxy service itself:

docker compose up -d redis
uv run uvicorn responses_proxy.app:create_app --factory --reload --host 127.0.0.1 --port 8000
uv run responses-proxy-worker

Set these before testing admin or provider registration flows:

PROXY_BOOTSTRAP_ORG_ID=organization_...
PROXY_ADMIN_AUTH_MODE=morph_org_or_delegation
PROXY_ADMIN_ALLOWED_ORG_ID=organization_...
PROXY_ADMIN_DELEGATION_SERVICE_API_KEY=...
PROXY_SECRET_ENCRYPTION_KEY=replace-with-stable-local-key

Admin auth is Morph-only. Then register a provider, alias, and proxy key through the authenticated admin API or through the frontend BFF before data-plane testing.

Docker service

Build and run the service container:

docker build -t agent-responses-proxy .
docker run --rm --env-file .env.service.local -p 8000:8000 agent-responses-proxy

The container starts Redis, the worker, and the API.

Morph service deployment

The deploy workflow is .github/workflows/deploy.yml in the proxy repo and calls scripts/deploy_service.py.

Required GitHub environment secrets:

• MORPH_API_KEY_PROD
• MORPH_API_KEY_STAGE for stage
• MORPH_BUILDER_SNAPSHOT_ID or MORPH_BASE_IMAGE_ID
• PROXY_DATABASE_URL
• PROXY_ADMIN_DELEGATION_SERVICE_API_KEY
• PROXY_ADMIN_SMOKE_MORPH_API_KEY
• PROXY_SECRET_ENCRYPTION_KEY

Optional GitHub environment secret:

• PROVIDER_API_KEYS_ENV: newline-separated provider key slots such as OPENAI_API_KEY=... and OPENAI_API_KEY_1=....

Required variables:

• PROXY_BOOTSTRAP_ORG_ID

Optional variables:

• PROXY_ADMIN_AUTH_MODE
• PROXY_ADMIN_ALLOWED_ORG_ID
• PROXY_ADMIN_ALLOWED_ORG_SLUG
• PROXY_ADMIN_ALLOWED_ROLES
• PROXY_MORPH_API_BASE_URL
• PROXY_MORPH_SERVICE_API_BASE_URL
• PROXY_ADMIN_DELEGATION_SERVICE_NAME
• PROXY_ADMIN_DELEGATION_REQUIRED_SCOPE
• PROXY_ADMIN_DELEGATION_AUDIENCE
• PROXY_ADMIN_AUTH_TIMEOUT_SECONDS
• PROXY_ADMIN_AUTH_CACHE_TTL_SECONDS
• PROXY_AFFINITY_TTL_SECONDS
• PROXY_ROUTING_REFRESH_INTERVAL_SECONDS
• PROXY_HEALTH_CHECK_ENABLED
• PROXY_HEALTH_CHECK_INTERVAL_SECONDS
• PROXY_HEALTH_CHECK_TIMEOUT_SECONDS
• PROXY_HEALTH_CHECK_PATH
• PROXY_CAPTURE_REQUEST_BODY
• PROXY_CAPTURE_RESPONSE_BODY
• PROXY_ALLOW_HTTP_PROVIDER_URLS
• PROXY_ALLOW_PRIVATE_PROVIDER_URLS
• PROXY_CREDENTIAL_REF_ENV_ALLOWLIST
• PROXY_ENV_CREDENTIAL_OWNER_ORG_IDS
• PROXY_PROXY_KEY_CACHE_TTL_SECONDS
• PROXY_CLAUDE_GATEWAY_CHAT_COMPLETIONS_ENABLED
• PROXY_CLAUDE_GATEWAY_MAX_BODY_BYTES
• PROXY_ENABLE_CLAUDE_CODE_TOKEN_ESCROW
• PROXY_CLAUDE_CODE_OAUTH_REFRESH_URL
• PROXY_CLAUDE_CODE_OAUTH_CLIENT_ID
• PROXY_CLAUDE_CODE_OAUTH_REFRESH_SKEW_SECONDS
• PROXY_CLAUDE_CODE_OAUTH_REFRESH_TIMEOUT_SECONDS

The deploy workflow forwards the Claude Code OAuth escrow and compatibility variables above. The gateway base URL, proxy-key header, and direct-model allowlist are runtime settings for custom service envs; defaults are normally used in production.

For staged Chat Completions rollout, deploy with PROXY_CLAUDE_GATEWAY_CHAT_COMPLETIONS_ENABLED=false, configure and test alias capabilities through POST /admin/models/test with surface="claude_code", then enable the flag. Turning it off removes Chat aliases from /v1/models and rejects their Messages traffic without affecting Responses or Anthropic aliases.

Validate deployment configuration without contacting Morph:

MORPH_API_KEY=... \
MORPH_BUILDER_SNAPSHOT_ID=... \
PROXY_DATABASE_URL=... \
PROXY_BOOTSTRAP_ORG_ID=organization_... \
PROXY_ADMIN_AUTH_MODE=morph_org_or_delegation \
PROXY_ADMIN_DELEGATION_SERVICE_API_KEY=... \
PROXY_ADMIN_ALLOWED_ORG_ID=organization_... \
PROXY_SECRET_ENCRYPTION_KEY=... \
uv run --extra deploy python scripts/deploy_service.py --dry-run

Delegation modes require PROXY_ADMIN_DELEGATION_SERVICE_API_KEY; organization modes require PROXY_ADMIN_ALLOWED_ORG_ID or PROXY_ADMIN_ALLOWED_ORG_SLUG.

Persistence

Use Neon/Postgres in production:

PROXY_DATABASE_URL=postgresql://user:password@host/database?sslmode=require

The app normalizes postgres:// and postgresql:// URLs to postgresql+psycopg://.

SQL persists:

• provider registry rows
• provider credential rows
• model alias rows
• proxy keys
• Claude Code OAuth setup sessions
• proxy-owned response state
• sessions and requests
• load-balancer decisions
• token usage
• raw telemetry events

Redis stores live counters, health, affinity, recent decisions, and queue depth. Redis can reset during container lifecycle.