mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
Yevhenii Shcherbina
63ec93a7ce
Implements https://linear.app/codercom/issue/AIGOV-213/add-bedrock-provider # AWS Bedrock mantle support in AI Gateway ## Summary Add support for the AWS Bedrock **mantle** endpoint (`bedrock-mantle.{region}.api.aws/anthropic/v1/messages`) to AI Gateway. Mantle serves Claude through the native Anthropic Messages API. We model it as a `protocol` field on the existing Bedrock provider settings (`invoke-model` default, or `mantle`) rather than as a new provider type, and we treat mantle as a pure passthrough: SigV4-sign and forward, no body translation. ## Background Claude on AWS Bedrock is reachable through two endpoints, each speaking exactly one wire protocol: 1. **InvokeModel** (existing): `bedrock-runtime.{region}.amazonaws.com`. Model id in the URL path, request translated into Bedrock's InvokeModel format, responses returned as a binary AWS eventstream. This is what AI Gateway already supported for Bedrock. 2. **Mantle** (this doc): `bedrock-mantle.{region}.api.aws/anthropic/v1/messages`. Native Anthropic Messages API: model in the body, plain SSE streaming. ## Why a `protocol` field, not a new provider type The alternative is to model mantle as its own `ai_provider_type` (`bedrock-mantle`) alongside `bedrock`. I chose the `protocol` field instead for two reasons: 1. Mantle reads more like a protocol of Bedrock than a separate provider. It is the same AWS account, credentials, region, and IAM, reached over a different wire protocol and host. One Bedrock provider with two protocols (`invoke-model` default and `mantle`) models that more organically than two provider types. 2. It avoids a database migration. The `protocol` field lives in the settings JSON blob (empty resolves to `invoke-model`, so existing providers are unaffected), whereas a new type means an enum value and the `ALTER TYPE ... ADD VALUE` migration that goes with it. ## Why passthrough, not translation The client already emits Bedrock-legal requests in mantle mode: ```sh export CLAUDE_CODE_USE_MANTLE=1 export CLAUDE_CODE_SKIP_MANTLE_AUTH=1 export ANTHROPIC_BEDROCK_MANTLE_BASE_URL=https://<coder>/api/v2/aibridge/<provider-name> ``` So the gateway just forwards the body and SigV4-signs it (service `bedrock-mantle`), and skips all the InvokeModel body-translation (model remap, thinking conversion, beta-flag allowlist, field stripping). This keeps the mantle path thin and avoids a second copy of translation logic to maintain. ## Consequences - Protocol-dependent fields: `model` / `small_fast_model` are used by InvokeModel but ignored by mantle (the client sends the model), and `base_url` is required for mantle but optional for InvokeModel. Validation is protocol-aware. - No central model control on mantle: because it is a passthrough, the operator cannot pin the model. - `region` and the `base_url` host must name the same region (the SigV4 scope must match the endpoint); a mismatch surfaces as `Credential should be scoped to a valid region`. ## Draft UI <img width="1100" height="579" alt="image" src="https://github.com/user-attachments/assets/37bab46d-8958-4a96-9f47-1fef3493e1b6" /> ## Follow-up PRs: - https://github.com/coder/coder/pull/27156