mirror of
https://github.com/coder/coder.git
synced 2026-10-16 14:48:56 +08:00
Yevhenii Shcherbina
b405d57638
Follow-up to #28877, which resolved Bedrock application inference profile (AIP) ARNs while constructing a provider in the gateway. Review found two consequences of putting an AWS call on that path: a transient failure dropped the provider from the reloaded snapshot until the next provider change or restart, and the initial reload blocked `coder server` startup for up to 30s per AIP-configured provider under an uncancelable context. Resolution now happens once, in coderd, when a provider is created or updated. The result is stored with the provider and travels to the gateway in the existing provider payload, so the gateway makes no Bedrock control-plane call at startup, on reload, or on a request. The resolved identifiers live in the provider's encrypted settings blob as `resolved_model` and `resolved_small_fast_model`. A table keyed by ARN was tried and dropped: the server always writes the resolution as part of the same provider write, so it needs no separate lifecycle, migration, or clearing rules. Resolution runs before the write, so a failed lookup rejects the save and stores nothing. Create resolves its own request, which carries the complete configuration. Update cannot: a `PATCH` supplies the model identifiers but inherits credentials from the stored row, so stored + patch is merged before the transaction to resolve against, then merged again inside it against the row being written. Both merges take the model identifiers from the patch, so they cannot disagree, and no AWS call happens inside a database transaction. Reporting the failure at save time means an operator learns immediately that an ARN is wrong, deleted, or that the identity lacks `bedrock:GetInferenceProfile`, instead of discovering it later through a Bedrock 400 on adaptive-only models. Providers configured with plain model IDs store no resolution and make no AWS call, because those identifiers are already model identities. Resolution uses the provider's own credential configuration: static access keys when set, otherwise the AWS default credential chain, and the role from `role_arn` when configured. An AIP ARN embeds the account ID, so resolving under a different principal cannot return another account's profile; it can only fail with a permission error, which surfaces as a rejected write. Using an AIP ARN now requires coderd to reach `bedrock.{region}.amazonaws.com`. Deployments where only the gateway has AWS egress cannot configure AIP ARNs, and they find out when they save. <details> <summary>Implementation plan</summary> ## Goal Resolve Bedrock AIP ARNs once, when a provider is created or updated, and persist the result. The gateway reads the resolved model from the provider payload and makes no AWS call on the boot, reload, or request paths. ## Identity coderd resolves for every credential mode. An AIP ARN carries the account ID, so resolving under a different principal either succeeds with the same answer or fails with a permission error. The risk is a false negative, not a wrong answer. - **Static access keys, with or without `role_arn`.** The settings fully determine the identity, so coderd reaches the same principal the gateway will. - **AWS default credential chain.** coderd resolves under its own environment, which may legitimately lack `bedrock:GetInferenceProfile` even where the gateway has it. The save then reports an actionable message. ## Behavior at write time 1. Skip when neither `model` nor `small_fast_model` is an AIP ARN. This is the common case and costs nothing. 2. Otherwise resolve both identifiers with a short deadline, before the provider write. 3. On success, store the resolved identifiers in the same provider write and publish normally. 4. On failure, surface the AWS error and store nothing. The remedy is to fix the ARN, configure static access keys, or grant `bedrock:GetInferenceProfile`. 5. Re-resolve on every save. Bedrock has no `UpdateInferenceProfile`, so an ARN means one model forever, but re-resolving proves this provider's own identity can read the profile. ## Gateway `ResolvedModelWithFallback` and `ResolvedSmallFastModelWithFallback` return the stored resolution when present and the configured identifier otherwise. The fallback covers plain model IDs and providers saved before this change, degrading to pre-#28877 behavior rather than dropping the provider from the pool. ## Out of scope - Normalizing directly configured system inference profile ARNs. - Provisioned throughput and prompt router identifiers. - Any gateway-side resolution, including a lazy fallback. - Displaying the resolved model in the UI. </details> Relates to https://linear.app/codercom/issue/AIGOV-488 Created by Coder Agents on behalf of @evgeniy-scherbina.