feat: resolve bedrock inference profiles when a provider is written (#29112)

Follow-up to #28877, which resolved Bedrock application inference
profile (AIP) ARNs while constructing a provider in the gateway. Review
found two consequences of putting an AWS call on that path: a transient
failure dropped the provider from the reloaded snapshot until the next
provider change or restart, and the initial reload blocked `coder
server` startup for up to 30s per AIP-configured provider under an
uncancelable context.

Resolution now happens once, in coderd, when a provider is created or
updated. The result is stored with the provider and travels to the
gateway in the existing provider payload, so the gateway makes no
Bedrock control-plane call at startup, on reload, or on a request.

The resolved identifiers live in the provider's encrypted settings blob
as `resolved_model` and `resolved_small_fast_model`. A table keyed by
ARN was tried and dropped: the server always writes the resolution as
part of the same provider write, so it needs no separate lifecycle,
migration, or clearing rules.

Resolution runs before the write, so a failed lookup rejects the save
and stores nothing. Create resolves its own request, which carries the
complete configuration. Update cannot: a `PATCH` supplies the model
identifiers but inherits credentials from the stored row, so stored +
patch is merged before the transaction to resolve against, then merged
again inside it against the row being written. Both merges take the
model identifiers from the patch, so they cannot disagree, and no AWS
call happens inside a database transaction.

Reporting the failure at save time means an operator learns immediately
that an ARN is wrong, deleted, or that the identity lacks
`bedrock:GetInferenceProfile`, instead of discovering it later through a
Bedrock 400 on adaptive-only models. Providers configured with plain
model IDs store no resolution and make no AWS call, because those
identifiers are already model identities.

Resolution uses the provider's own credential configuration: static
access keys when set, otherwise the AWS default credential chain, and
the role from `role_arn` when configured. An AIP ARN embeds the account
ID, so resolving under a different principal cannot return another
account's profile; it can only fail with a permission error, which
surfaces as a rejected write.

Using an AIP ARN now requires coderd to reach
`bedrock.{region}.amazonaws.com`. Deployments where only the gateway has
AWS egress cannot configure AIP ARNs, and they find out when they save.

<details>
<summary>Implementation plan</summary>

## Goal

Resolve Bedrock AIP ARNs once, when a provider is created or updated,
and persist the result. The gateway reads the resolved model from the
provider payload and makes no AWS call on the boot, reload, or request
paths.

## Identity

coderd resolves for every credential mode. An AIP ARN carries the
account ID, so resolving under a different principal either succeeds
with the same answer or fails with a permission error. The risk is a
false negative, not a wrong answer.

- **Static access keys, with or without `role_arn`.** The settings fully
determine the identity, so coderd reaches the same principal the gateway
will.
- **AWS default credential chain.** coderd resolves under its own
environment, which may legitimately lack `bedrock:GetInferenceProfile`
even where the gateway has it. The save then reports an actionable
message.

## Behavior at write time

1. Skip when neither `model` nor `small_fast_model` is an AIP ARN. This
is the common case and costs nothing.
2. Otherwise resolve both identifiers with a short deadline, before the
provider write.
3. On success, store the resolved identifiers in the same provider write
and publish normally.
4. On failure, surface the AWS error and store nothing. The remedy is to
fix the ARN, configure static access keys, or grant
`bedrock:GetInferenceProfile`.
5. Re-resolve on every save. Bedrock has no `UpdateInferenceProfile`, so
an ARN means one model forever, but re-resolving proves this provider's
own identity can read the profile.

## Gateway

`ResolvedModelWithFallback` and `ResolvedSmallFastModelWithFallback`
return the stored resolution when present and the configured identifier
otherwise. The fallback covers plain model IDs and providers saved
before this change, degrading to pre-#28877 behavior rather than
dropping the provider from the pool.

## Out of scope

- Normalizing directly configured system inference profile ARNs.
- Provisioned throughput and prompt router identifiers.
- Any gateway-side resolution, including a lazy fallback.
- Displaying the resolved model in the UI.

</details>

Relates to https://linear.app/codercom/issue/AIGOV-488

Created by Coder Agents on behalf of @evgeniy-scherbina.
This commit is contained in:
Yevhenii Shcherbina
2026-09-11 16:03:44 +00:00
committed by GitHub
parent 2eb018fd8c
commit b405d57638
17 changed files with 823 additions and 221 deletions
+20
View File
@@ -69,6 +69,12 @@ type AWSBedrock struct {
// Protocol selects the Bedrock wire protocol. The zero value behaves as
// BedrockProtocolInvokeModel.
Protocol BedrockProtocol
// ResolvedModel is the model ID behind Model, which differs from it only
// when Model is an application inference profile ARN. coderd resolves it
// when the provider is written, so the gateway never calls AWS for it.
ResolvedModel string
// ResolvedSmallFastModel is ResolvedModel for SmallFastModel.
ResolvedSmallFastModel string
}
// ResolvedProtocol returns the configured protocol, mapping the empty value to
@@ -81,6 +87,20 @@ func (c AWSBedrock) ResolvedProtocol() BedrockProtocol {
return c.Protocol
}
func (c AWSBedrock) ResolvedModelWithFallback() string {
if c.ResolvedModel != "" {
return c.ResolvedModel
}
return c.Model
}
func (c AWSBedrock) ResolvedSmallFastModelWithFallback() string {
if c.ResolvedSmallFastModel != "" {
return c.ResolvedSmallFastModel
}
return c.SmallFastModel
}
// Validate verifies protocol-specific Bedrock configuration.
func (c AWSBedrock) Validate() error {
switch c.ResolvedProtocol() {
+6 -15
View File
@@ -72,22 +72,13 @@ var bedrockSupportedBetaFlags = map[string]bool{
type BedrockRuntime struct {
Cfg aibconfig.AWSBedrock
Creds aws.CredentialsProvider
resolvedModel string
resolvedSmallFastModel string
}
// NewBedrockRuntime bundles the Bedrock config and credentials with the model
// IDs behind the configured identifiers. The resolved IDs differ from the
// configured ones only when those are application inference profile ARNs, which
// are opaque and must be resolved through AWS; every other identifier resolves
// to itself.
func NewBedrockRuntime(cfg aibconfig.AWSBedrock, creds aws.CredentialsProvider, resolvedModel, resolvedSmallFastModel string) *BedrockRuntime {
// NewBedrockRuntime bundles the Bedrock config and credentials.
func NewBedrockRuntime(cfg aibconfig.AWSBedrock, creds aws.CredentialsProvider) *BedrockRuntime {
return &BedrockRuntime{
Cfg: cfg,
Creds: creds,
resolvedModel: resolvedModel,
resolvedSmallFastModel: resolvedSmallFastModel,
Cfg: cfg,
Creds: creds,
}
}
@@ -108,13 +99,13 @@ func (b *BedrockRuntime) ConfiguredSmallFastModel() string {
// Model capabilities, usage records, pricing, and metrics all key off this
// rather than the configured identifier.
func (b *BedrockRuntime) ResolvedModel() string {
return b.resolvedModel
return b.Cfg.ResolvedModelWithFallback()
}
// ResolvedSmallFastModel is [BedrockRuntime.ResolvedModel] for the small/fast
// model.
func (b *BedrockRuntime) ResolvedSmallFastModel() string {
return b.resolvedSmallFastModel
return b.Cfg.ResolvedSmallFastModelWithFallback()
}
type interceptionBase struct {
@@ -179,7 +179,7 @@ func TestAWSBedrockValidation(t *testing.T) {
t.Parallel()
base := &interceptionBase{
bedrock: NewBedrockRuntime(tt.cfg, credentials.NewStaticCredentialsProvider("test-key", "test-secret", ""), "", ""),
bedrock: NewBedrockRuntime(tt.cfg, credentials.NewStaticCredentialsProvider("test-key", "test-secret", "")),
}
opts, err := base.withBedrockInvokeModelOptions(context.Background())
@@ -217,9 +217,11 @@ func TestModelForBedrockInvokeModel(t *testing.T) {
)
runtime := NewBedrockRuntime(config.AWSBedrock{
Model: profileARN,
SmallFastModel: smallFastProfileARN,
}, nil, "anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5")
Model: profileARN,
SmallFastModel: smallFastProfileARN,
ResolvedModel: "anthropic.claude-opus-4-8",
ResolvedSmallFastModel: "anthropic.claude-haiku-4-5",
}, nil)
tests := []struct {
name string
@@ -274,9 +276,11 @@ func TestSmallFastModelCapturedAtConstruction(t *testing.T) {
)
runtime := NewBedrockRuntime(config.AWSBedrock{
Model: profileARN,
SmallFastModel: smallFastProfileARN,
}, nil, "anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5")
Model: profileARN,
SmallFastModel: smallFastProfileARN,
ResolvedModel: "anthropic.claude-opus-4-8",
ResolvedSmallFastModel: "anthropic.claude-haiku-4-5",
}, nil)
const haikuPayload = `{"model":"claude-haiku-4-5","max_tokens":10000}`
const opusPayload = `{"model":"claude-opus-4-8","max_tokens":10000}`
@@ -336,7 +340,7 @@ func TestModelForPlainBedrockModelID(t *testing.T) {
bedrock: NewBedrockRuntime(config.AWSBedrock{
Model: "eu.anthropic.claude-opus-4-8",
SmallFastModel: "anthropic.claude-haiku-4-5",
}, nil, "eu.anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5"),
}, nil),
logger: slog.Make(),
}
@@ -958,18 +962,14 @@ func TestAugmentRequestForBedrock_AdaptiveThinking(t *testing.T) {
}
// Plain model IDs resolve to themselves; an application inference
// profile ARN resolves to the model behind it.
resolvedModel := tc.resolvedModel
if resolvedModel == "" {
resolvedModel = tc.bedrockModel
}
i := &interceptionBase{
reqPayload: mustMessagesPayload(t, tc.requestBody),
bedrock: NewBedrockRuntime(config.AWSBedrock{
Model: tc.bedrockModel,
SmallFastModel: "anthropic.claude-haiku-3-5",
}, nil, resolvedModel, "anthropic.claude-haiku-3-5"),
Model: tc.bedrockModel,
SmallFastModel: "anthropic.claude-haiku-3-5",
ResolvedModel: tc.resolvedModel,
ResolvedSmallFastModel: "anthropic.claude-haiku-3-5",
}, nil),
clientHeaders: clientHeaders,
logger: slog.Make(),
}
@@ -1322,7 +1322,7 @@ func TestBedrockMantleIsPassthrough(t *testing.T) {
Region: "us-east-1",
BaseURL: "https://bedrock-mantle.us-east-1.api.aws/anthropic",
Protocol: config.BedrockProtocolMantle,
}, credentials.NewStaticCredentialsProvider("test-key", "test-secret", ""), "", ""),
}, credentials.NewStaticCredentialsProvider("test-key", "test-secret", "")),
logger: slog.Make(),
}
@@ -1373,7 +1373,7 @@ func TestAWSMantleOptionsValidation(t *testing.T) {
t.Parallel()
base := &interceptionBase{
bedrock: NewBedrockRuntime(tt.cfg, credentials.NewStaticCredentialsProvider("test-key", "test-secret", ""), "", ""),
bedrock: NewBedrockRuntime(tt.cfg, credentials.NewStaticCredentialsProvider("test-key", "test-secret", "")),
}
opts, err := base.withBedrockMantleOptions(t.Context())
if tt.errorMsg != "" {
+1 -10
View File
@@ -83,16 +83,7 @@ func NewAnthropic(ctx context.Context, cfg config.Anthropic, bedrockCfg *config.
return nil, xerrors.Errorf("bedrock config: %w", err)
}
// Resolution only calls AWS for application inference profile ARNs, so
// deployments configured with plain model IDs need no extra permission.
resolveCtx, cancel := context.WithTimeout(ctx, inferenceProfileResolutionTimeout)
defer cancel()
model, smallFastModel, err := resolveBedrockModels(resolveCtx, runtimeCfg, awsCfg)
if err != nil {
return nil, xerrors.Errorf("resolve bedrock models: %w", err)
}
bedrock = messages.NewBedrockRuntime(runtimeCfg, awsCfg.Credentials, model, smallFastModel)
bedrock = messages.NewBedrockRuntime(runtimeCfg, awsCfg.Credentials)
}
return &Anthropic{
+32 -15
View File
@@ -88,25 +88,42 @@ func modelIDFromARN(modelARN string) (string, error) {
return model, nil
}
// resolveBedrockModels resolves the configured model identifiers to the model
// IDs used for capability detection, usage recording, and pricing. Identifiers
// that are not application inference profile ARNs are returned unchanged and
// cost no AWS call.
func resolveBedrockModels(ctx context.Context, cfg config.AWSBedrock, awsCfg aws.Config) (model, smallFastModel string, err error) {
resolveOne := func(configured string) (string, error) {
if !isApplicationInferenceProfileARN(configured) {
return configured, nil
// ResolveBedrockModels resolves the application inference profile ARNs among
// the configured model identifiers, returning what each ARN refers to. The
// result is empty when neither identifier is an ARN, which costs no AWS call.
//
// The identity comes from cfg, including any role assumed via config.AWSBedrock.RoleARN,
// so the required bedrock:GetInferenceProfile permission belongs to that identity.
func ResolveBedrockModels(ctx context.Context, cfg config.AWSBedrock) (map[string]string, error) {
resolved := make(map[string]string, 2)
var profiles []string
for _, configured := range []string{cfg.Model, cfg.SmallFastModel} {
if isApplicationInferenceProfileARN(configured) {
profiles = append(profiles, configured)
}
return resolveInferenceProfile(ctx, awsCfg, configured)
}
if len(profiles) == 0 {
return resolved, nil
}
model, err = resolveOne(cfg.Model)
awsCfg, err := buildBedrockCredentials(ctx, cfg)
if err != nil {
return "", "", xerrors.Errorf("resolve model: %w", err)
return nil, xerrors.Errorf("build bedrock credentials: %w", err)
}
smallFastModel, err = resolveOne(cfg.SmallFastModel)
if err != nil {
return "", "", xerrors.Errorf("resolve small fast model: %w", err)
resolveCtx, cancel := context.WithTimeout(ctx, inferenceProfileResolutionTimeout)
defer cancel()
for _, profileARN := range profiles {
if _, ok := resolved[profileARN]; ok {
continue
}
model, err := resolveInferenceProfile(resolveCtx, awsCfg, profileARN)
if err != nil {
return nil, err
}
resolved[profileARN] = model
}
return model, smallFastModel, nil
return resolved, nil
}
@@ -124,15 +124,19 @@ func TestModelIDFromARN(t *testing.T) {
}
}
// TestNewAnthropic_InferenceProfileResolution drives the Bedrock
// GetInferenceProfile path against a mock endpoint.
// TestResolveBedrockModels drives the Bedrock GetInferenceProfile path against
// a mock endpoint. Resolution runs where a provider is written, so this covers
// what coderd calls, not what the gateway does when serving.
// https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetInferenceProfile.html
// NOTE: no t.Parallel() because the subtests use t.Setenv.
func TestNewAnthropic_InferenceProfileResolution(t *testing.T) {
const profileARN = "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/46u2vhiyo6z5"
func TestResolveBedrockModels(t *testing.T) {
const (
profileARN = "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/46u2vhiyo6z5"
smallFastProfileARN = "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/8x1qk20fzp3r"
)
bedrockCfg := func(model, smallFastModel string) *config.AWSBedrock {
return &config.AWSBedrock{
bedrockCfg := func(model, smallFastModel string) config.AWSBedrock {
return config.AWSBedrock{
Region: "us-east-1",
AccessKey: "test-key",
AccessKeySecret: "test-secret",
@@ -155,24 +159,21 @@ func TestNewAnthropic_InferenceProfileResolution(t *testing.T) {
return srv.URL, &got
}
t.Run("resolved profile drives the model id", func(t *testing.T) {
t.Run("profile resolves to its model", func(t *testing.T) {
url, paths := mockBedrock(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"models":[{"modelArn":"arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-opus-4-8"}]}`))
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
p, err := NewAnthropic(context.Background(), config.Anthropic{}, bedrockCfg(profileARN, "anthropic.claude-haiku-4-5"))
resolved, err := ResolveBedrockModels(context.Background(), bedrockCfg(profileARN, "anthropic.claude-haiku-4-5"))
require.NoError(t, err)
require.Equal(t, "anthropic.claude-opus-4-8", p.bedrock.ResolvedModel())
// The profile stays the configured identifier so AWS attributes spend to it.
require.Equal(t, profileARN, p.bedrock.ConfiguredModel())
require.Equal(t, "anthropic.claude-haiku-4-5", p.bedrock.ResolvedSmallFastModel())
require.Equal(t, map[string]string{profileARN: "anthropic.claude-opus-4-8"}, resolved)
require.Len(t, *paths, 1, "only the profile ARN is resolved")
require.Contains(t, (*paths)[0], profileARN)
})
t.Run("failed resolution fails construction", func(t *testing.T) {
t.Run("failed resolution is an error", func(t *testing.T) {
url, _ := mockBedrock(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set("X-Amzn-Errortype", "AccessDeniedException")
@@ -181,50 +182,100 @@ func TestNewAnthropic_InferenceProfileResolution(t *testing.T) {
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
_, err := NewAnthropic(context.Background(), config.Anthropic{}, bedrockCfg(profileARN, "anthropic.claude-haiku-4-5"))
require.ErrorContains(t, err, "resolve bedrock models")
_, err := ResolveBedrockModels(context.Background(), bedrockCfg(profileARN, "anthropic.claude-haiku-4-5"))
require.ErrorContains(t, err, "GetInferenceProfile")
})
t.Run("profile without a model fails construction", func(t *testing.T) {
t.Run("profile without a model is an error", func(t *testing.T) {
url, _ := mockBedrock(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"models":[]}`))
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
_, err := NewAnthropic(context.Background(), config.Anthropic{}, bedrockCfg(profileARN, "anthropic.claude-haiku-4-5"))
_, err := ResolveBedrockModels(context.Background(), bedrockCfg(profileARN, "anthropic.claude-haiku-4-5"))
require.ErrorContains(t, err, "references no model")
})
t.Run("small fast profile resolves independently", func(t *testing.T) {
const smallFastProfileARN = "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/8x1qk20fzp3r"
url, paths := mockBedrock(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"models":[{"modelArn":"arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-haiku-4-5"}]}`))
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
p, err := NewAnthropic(context.Background(), config.Anthropic{}, bedrockCfg("eu.anthropic.claude-opus-4-8", smallFastProfileARN))
resolved, err := ResolveBedrockModels(context.Background(), bedrockCfg("eu.anthropic.claude-opus-4-8", smallFastProfileARN))
require.NoError(t, err)
require.Equal(t, "eu.anthropic.claude-opus-4-8", p.bedrock.ResolvedModel())
require.Equal(t, "anthropic.claude-haiku-4-5", p.bedrock.ResolvedSmallFastModel())
require.Equal(t, smallFastProfileARN, p.bedrock.ConfiguredSmallFastModel())
require.Equal(t, map[string]string{smallFastProfileARN: "anthropic.claude-haiku-4-5"}, resolved)
require.Len(t, *paths, 1, "only the small fast profile ARN is resolved")
require.Contains(t, (*paths)[0], smallFastProfileARN)
})
t.Run("plain model id needs no resolution", func(t *testing.T) {
t.Run("plain model ids need no resolution", func(t *testing.T) {
url, paths := mockBedrock(t, func(http.ResponseWriter, *http.Request) {
t.Error("Bedrock called for a plain model id")
t.Error("Bedrock called for plain model ids")
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
p, err := NewAnthropic(context.Background(), config.Anthropic{}, bedrockCfg("eu.anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5"))
resolved, err := ResolveBedrockModels(context.Background(), bedrockCfg("eu.anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5"))
require.NoError(t, err)
require.Equal(t, "eu.anthropic.claude-opus-4-8", p.bedrock.ResolvedModel())
require.Equal(t, "eu.anthropic.claude-opus-4-8", p.bedrock.ConfiguredModel())
require.Empty(t, resolved)
require.Empty(t, *paths)
})
}
// TestNewAnthropic_ServesStoredResolution covers what the gateway does with the
// resolution coderd stored.
func TestNewAnthropic_ServesStoredResolution(t *testing.T) {
t.Parallel()
const profileARN = "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/46u2vhiyo6z5"
bedrockCfg := func(mutate func(*config.AWSBedrock)) *config.AWSBedrock {
cfg := &config.AWSBedrock{
Region: "us-east-1",
AccessKey: "test-key",
AccessKeySecret: "test-secret",
Model: profileARN,
SmallFastModel: "anthropic.claude-haiku-4-5",
}
mutate(cfg)
return cfg
}
t.Run("stored resolution drives the model id", func(t *testing.T) {
t.Parallel()
p, err := NewAnthropic(context.Background(), config.Anthropic{}, bedrockCfg(func(cfg *config.AWSBedrock) {
cfg.ResolvedModel = "anthropic.claude-opus-4-8"
}))
require.NoError(t, err)
require.Equal(t, "anthropic.claude-opus-4-8", p.bedrock.ResolvedModel())
// The profile stays the configured identifier so AWS attributes spend to it.
require.Equal(t, profileARN, p.bedrock.ConfiguredModel())
require.Equal(t, "anthropic.claude-haiku-4-5", p.bedrock.ResolvedSmallFastModel())
})
t.Run("unresolved profile serves the configured identifier", func(t *testing.T) {
t.Parallel()
// A save whose profile lookup failed stores no resolution. The provider
// still serves, with the ARN as its own identity, which is wrong for
// capability detection and pricing but visible to the operator as the
// error their save returned.
p, err := NewAnthropic(context.Background(), config.Anthropic{}, bedrockCfg(func(*config.AWSBedrock) {}))
require.NoError(t, err)
require.Equal(t, profileARN, p.bedrock.ResolvedModel())
})
t.Run("plain model ids serve themselves", func(t *testing.T) {
t.Parallel()
p, err := NewAnthropic(context.Background(), config.Anthropic{}, bedrockCfg(func(cfg *config.AWSBedrock) {
cfg.Model = "eu.anthropic.claude-opus-4-8"
}))
require.NoError(t, err)
require.Equal(t, "eu.anthropic.claude-opus-4-8", p.bedrock.ResolvedModel())
require.Equal(t, "eu.anthropic.claude-opus-4-8", p.bedrock.ConfiguredModel())
})
}
+5 -31
View File
@@ -14,11 +14,11 @@ import (
"github.com/coder/coder/v2/aibridge/config"
"github.com/coder/coder/v2/aibridge/keypool"
"github.com/coder/coder/v2/coderd"
agplaibridge "github.com/coder/coder/v2/coderd/aibridge"
"github.com/coder/coder/v2/coderd/aibridged"
"github.com/coder/coder/v2/coderd/aibridged/proto"
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/tracing"
"github.com/coder/coder/v2/coderd/util/ptr"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/quartz"
)
@@ -215,6 +215,8 @@ func protoToProviderSpec(pp *proto.AIProvider) aiProviderSpec {
bedrock.RoleARN = b.GetRoleArn()
bedrock.ExternalID = b.GetExternalId()
bedrock.Protocol = codersdk.AIProviderBedrockProtocol(b.GetProtocol())
bedrock.ResolvedModel = b.GetResolvedModel()
bedrock.ResolvedSmallFastModel = b.GetResolvedSmallFastModel()
spec.Bedrock = new(bedrock)
}
return spec
@@ -307,10 +309,10 @@ func buildProvider(ctx context.Context, spec aiProviderSpec, cfg codersdk.AIBrid
// without populated Bedrock credentials it cannot make upstream
// calls, so refuse rather than falling back to an unsigned
// Anthropic client.
if spec.Bedrock == nil {
bedrock := agplaibridge.BedrockConfig(spec.BaseURL, spec.Bedrock)
if bedrock == nil {
return nil, xerrors.New("bedrock provider has no bedrock credentials configured")
}
bedrock := bedrockConfig(spec.BaseURL, spec.Bedrock)
return aibridge.NewBedrockProvider(ctx, aibridge.AnthropicConfig{
Name: spec.Name,
BaseURL: spec.BaseURL,
@@ -340,34 +342,6 @@ func buildAIProviderKeyPool(providerName string, keys []string, metrics *aibridg
return keypool.New(providerName, keys, quartz.NewReal(), metrics)
}
// bedrockConfig returns nil when the settings are absent or when the
// Bedrock fields are not actually configured. The provider's BaseURL is
// the generic upstream endpoint and is always non-empty, so it cannot
// serve as a Bedrock detection signal; gate on the settings alone via
// [codersdk.AIProviderBedrockSettings.IsConfigured].
func bedrockConfig(baseURL string, bedrock *codersdk.AIProviderBedrockSettings) *aibridge.AWSBedrockConfig {
if bedrock == nil {
return nil
}
bedrockSettings := *bedrock
if !bedrockSettings.IsConfigured() {
return nil
}
accessKey := ptr.NilToEmpty(bedrockSettings.AccessKey)
accessKeySecret := ptr.NilToEmpty(bedrockSettings.AccessKeySecret)
return &aibridge.AWSBedrockConfig{
BaseURL: baseURL,
Region: bedrockSettings.Region,
AccessKey: accessKey,
AccessKeySecret: accessKeySecret,
Model: bedrockSettings.Model,
SmallFastModel: bedrockSettings.SmallFastModel,
RoleARN: bedrockSettings.RoleARN,
ExternalID: bedrockSettings.ExternalID,
Protocol: config.BedrockProtocol(bedrockSettings.ResolvedProtocol()),
}
}
// circuitBreakerConfig returns nil when the breaker is disabled.
func circuitBreakerConfig(cfg codersdk.AIBridgeConfig) *config.CircuitBreaker {
if !cfg.CircuitBreakerEnabled.Value() {
+4 -4
View File
@@ -131,7 +131,7 @@ func TestBuildProviders(t *testing.T) {
Name: aibridge.ProviderAnthropic,
BaseUrl: "https://api.anthropic.com/",
}
assert.Nil(t, bedrockConfig(row.BaseUrl, codersdk.AIProviderSettings{}.Bedrock))
assert.Nil(t, agplaibridge.BedrockConfig(row.BaseUrl, codersdk.AIProviderSettings{}.Bedrock))
})
t.Run("NativeAnthropicCustomBaseURL", func(t *testing.T) {
@@ -141,7 +141,7 @@ func TestBuildProviders(t *testing.T) {
Name: "anthropic-proxy",
BaseUrl: "https://internal-proxy.example.com/anthropic/",
}
assert.Nil(t, bedrockConfig(row.BaseUrl, codersdk.AIProviderSettings{}.Bedrock))
assert.Nil(t, agplaibridge.BedrockConfig(row.BaseUrl, codersdk.AIProviderSettings{}.Bedrock))
})
t.Run("BedrockSettingsPresent", func(t *testing.T) {
@@ -166,7 +166,7 @@ func TestBuildProviders(t *testing.T) {
RoleARN: roleARN,
},
}
got := bedrockConfig(row.BaseUrl, settings.Bedrock)
got := agplaibridge.BedrockConfig(row.BaseUrl, settings.Bedrock)
require.NotNil(t, got)
assert.Equal(t, row.BaseUrl, got.BaseURL)
assert.Equal(t, "us-west-2", got.Region)
@@ -190,7 +190,7 @@ func TestBuildProviders(t *testing.T) {
settings := codersdk.AIProviderSettings{
Bedrock: &codersdk.AIProviderBedrockSettings{},
}
assert.Nil(t, bedrockConfig(row.BaseUrl, settings.Bedrock))
assert.Nil(t, agplaibridge.BedrockConfig(row.BaseUrl, settings.Bedrock))
})
}
+69 -15
View File
@@ -24,6 +24,8 @@ import (
"github.com/coder/coder/v2/coderd/httpapi"
"github.com/coder/coder/v2/coderd/httpmw"
coderpubsub "github.com/coder/coder/v2/coderd/pubsub"
"github.com/coder/coder/v2/coderd/rbac"
"github.com/coder/coder/v2/coderd/rbac/policy"
"github.com/coder/coder/v2/coderd/util/ptr"
"github.com/coder/coder/v2/codersdk"
)
@@ -160,6 +162,14 @@ func (api *API) aiProvidersCreate(rw http.ResponseWriter, r *http.Request) {
)
defer commitAudit()
// Provider configuration has side effects outside the database, notably
// the Bedrock profile lookup below, so the permission is checked before
// any of them rather than only by dbauthz on the write.
if !api.Authorize(r, policy.ActionCreate, rbac.ResourceAIProvider) {
httpapi.Forbidden(rw)
return
}
var req codersdk.CreateAIProviderRequest
if !httpapi.Read(ctx, rw, r, &req) {
return
@@ -186,6 +196,16 @@ func (api *API) aiProvidersCreate(rw http.ResponseWriter, r *http.Request) {
// Generate the server-owned external ID when the provider assumes a role.
ensureBedrockExternalID(&req.Settings)
// Resolve application inference profile ARNs before storing them, so an
// unresolvable profile is never written and the gateway never calls the
// Bedrock control plane.
resolved, err := resolveBedrockProfiles(ctx, req.Settings)
if err != nil {
api.writeAIProviderResolutionError(ctx, rw, err)
return
}
applyBedrockResolution(&req.Settings, resolved)
settings, err := encodeAIProviderSettings(req.Settings)
if err != nil {
api.Logger.Error(ctx, "encode AI provider settings", slog.Error(err))
@@ -288,6 +308,13 @@ func (api *API) aiProvidersUpdate(rw http.ResponseWriter, r *http.Request) {
)
defer commitAudit()
// Matches the create path: the Bedrock profile lookup below runs before
// dbauthz sees the write, so gate on the permission first.
if !api.Authorize(r, policy.ActionUpdate, rbac.ResourceAIProvider) {
httpapi.Forbidden(rw)
return
}
var req codersdk.UpdateAIProviderRequest
if !httpapi.Read(ctx, rw, r, &req) {
return
@@ -309,50 +336,59 @@ func (api *API) aiProvidersUpdate(rw http.ResponseWriter, r *http.Request) {
idOrName := chi.URLParam(r, "idOrName")
// Resolve outside the transaction, because it calls AWS. The merge is
// redone inside against the row that gets written; both merges take the
// model identifiers from the patch, so they cannot disagree on them.
var resolved map[string]string
if req.Settings != nil {
_, preview, err := lookupAndMergeSettings(ctx, api.Database, idOrName, req.Settings)
if err != nil {
writeAIProviderError(ctx, api.Logger, rw, err, "update AI provider", "Internal error updating AI provider.")
return
}
resolved, err = resolveBedrockProfiles(ctx, preview)
if err != nil {
api.writeAIProviderResolutionError(ctx, rw, err)
return
}
}
var (
updated database.AIProvider
keys []database.AIProviderKey
keyChanges aiProviderKeyChanges
)
err := api.Database.InTx(func(tx database.Store) error {
old, err := lookupAIProvider(ctx, tx, idOrName)
old, merged, err := lookupAndMergeSettings(ctx, tx, idOrName, req.Settings)
if err != nil {
return err
}
aReq.Old = old
// Decode the existing settings to merge with the patch. The dbcrypt
// wrapper has already decrypted the blob for us.
existing, err := db2sdk.AIProviderSettings(old.Settings)
if err != nil {
return xerrors.Errorf("decode existing settings: %w", err)
}
if req.Settings != nil {
if err := validateBedrockExternalIDUnchanged(existing, *req.Settings); err != nil {
if err := validateBedrockExternalIDUnchanged(merged, *req.Settings); err != nil {
return err
}
existing = mergeAIProviderSettings(existing, *req.Settings)
applyBedrockResolution(&merged, resolved)
}
// Bedrock settings are only meaningful for anthropic- or
// bedrock-typed providers; rejecting the mismatch keeps a
// misconfiguration from sitting silently in the encrypted
// blob.
if existing.Bedrock != nil &&
if merged.Bedrock != nil &&
old.Type != database.AIProviderTypeAnthropic &&
old.Type != database.AIProviderTypeBedrock {
return errAIProviderBedrockTypeMismatch
}
// Generate the server-owned external ID when the provider assumes a role
// and lacks one.
ensureBedrockExternalID(&existing)
settings, err := encodeAIProviderSettings(existing)
ensureBedrockExternalID(&merged)
settings, err := encodeAIProviderSettings(merged)
if err != nil {
return xerrors.Errorf("encode settings: %w", err)
}
// Reject keys against Bedrock providers (whether the existing
// row is Bedrock or the patch would make it so).
if req.APIKeys != nil && existing.Bedrock != nil && len(*req.APIKeys) > 0 {
if req.APIKeys != nil && merged.Bedrock != nil && len(*req.APIKeys) > 0 {
return errBedrockRejectsAPIKeys
}
@@ -842,6 +878,24 @@ func encodeAIProviderSettings(s codersdk.AIProviderSettings) (sql.NullString, er
return sql.NullString{String: string(out), Valid: true}, nil
}
// lookupAndMergeSettings loads a provider and merges patch onto its stored
// settings.
func lookupAndMergeSettings(ctx context.Context, db database.Store, idOrName string, patch *codersdk.AIProviderSettings) (database.AIProvider, codersdk.AIProviderSettings, error) {
old, err := lookupAIProvider(ctx, db, idOrName)
if err != nil {
return database.AIProvider{}, codersdk.AIProviderSettings{}, err
}
// The dbcrypt wrapper has already decrypted the blob for us.
settings, err := db2sdk.AIProviderSettings(old.Settings)
if err != nil {
return database.AIProvider{}, codersdk.AIProviderSettings{}, xerrors.Errorf("decode existing settings: %w", err)
}
if patch != nil {
settings = mergeAIProviderSettings(settings, *patch)
}
return old, settings, nil
}
// mergeAIProviderSettings overlays a patch onto an existing settings
// value. Write-only fields (Bedrock AccessKey and AccessKeySecret) use
// pointers so the patch can distinguish "omitted, keep existing" (nil)
+54
View File
@@ -0,0 +1,54 @@
package coderd
import (
"context"
"net/http"
"golang.org/x/xerrors"
"cdr.dev/slog/v3"
"github.com/coder/coder/v2/aibridge/provider"
agplaibridge "github.com/coder/coder/v2/coderd/aibridge"
"github.com/coder/coder/v2/coderd/httpapi"
"github.com/coder/coder/v2/codersdk"
)
// resolveBedrockProfiles asks AWS which model each application inference
// profile ARN in settings refers to. The result is empty when no identifier is
// an ARN, which costs no AWS call.
func resolveBedrockProfiles(ctx context.Context, settings codersdk.AIProviderSettings) (map[string]string, error) {
resolved := map[string]string{}
// BaseURL configures the runtime data-plane endpoint. GetInferenceProfile
// uses the Bedrock control-plane endpoint derived from Region instead.
cfg := agplaibridge.BedrockConfig("", settings.Bedrock)
if cfg == nil {
return resolved, nil
}
resolved, err := provider.ResolveBedrockModels(ctx, *cfg)
if err != nil {
return nil, xerrors.Errorf("resolve bedrock inference profile: %w", err)
}
return resolved, nil
}
// applyBedrockResolution records what the configured identifiers refer to. An
// identifier that is not an application inference profile ARN is its own
// identity and stores nothing, which also discards any value a client supplied.
func applyBedrockResolution(settings *codersdk.AIProviderSettings, resolved map[string]string) {
if settings.Bedrock == nil {
return
}
settings.Bedrock.ResolvedModel = resolved[settings.Bedrock.Model]
settings.Bedrock.ResolvedSmallFastModel = resolved[settings.Bedrock.SmallFastModel]
}
// writeAIProviderResolutionError reports a failed resolution. The write is
// rejected, because a stored ARN with no resolution would be served as its own
// identity and misshape every request made through it.
func (api *API) writeAIProviderResolutionError(ctx context.Context, rw http.ResponseWriter, err error) {
api.Logger.Warn(ctx, "resolve bedrock inference profile", slog.Error(err))
httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{
Message: "Could not resolve the Bedrock application inference profile. Check that the ARN is correct and that the AWS identity used by Coder is allowed bedrock:GetInferenceProfile.",
Detail: err.Error(),
})
}
+360
View File
@@ -0,0 +1,360 @@
package coderd_test
import (
"net/http"
"net/http/httptest"
"slices"
"strings"
"sync"
"testing"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/testutil"
)
const (
testProfileARN = "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/46u2vhiyo6z5"
testSmallFastProfileARN = "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/8x1qk20fzp3r"
)
func bedrockSettings(model, smallFastModel string) *codersdk.AIProviderSettings {
accessKey := "test-key"
accessKeySecret := "test-secret"
return &codersdk.AIProviderSettings{
Bedrock: &codersdk.AIProviderBedrockSettings{
Region: "us-east-1",
AccessKey: &accessKey,
AccessKeySecret: &accessKeySecret,
Model: model,
SmallFastModel: smallFastModel,
},
}
}
// mockBedrock serves the Bedrock control-plane API and records the profile
// lookups it receives. Callers point the AWS SDK at the returned URL.
func mockBedrock(t *testing.T, handler http.HandlerFunc) (url string, paths func() []string) {
t.Helper()
var (
mu sync.Mutex
got []string
)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
got = append(got, r.URL.Path)
mu.Unlock()
handler(w, r)
}))
t.Cleanup(srv.Close)
return srv.URL, func() []string {
mu.Lock()
defer mu.Unlock()
return slices.Clone(got)
}
}
func respondWithModel(modelARN string) http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"models":[{"modelArn":"` + modelARN + `"}]}`))
}
}
// TestAIProvidersBedrockProfileResolution drives provider writes against a mock
// Bedrock control plane, so the AWS SDK path runs for real.
// NOTE: no t.Parallel() because the subtests use t.Setenv.
func TestAIProvidersBedrockProfileResolution(t *testing.T) {
t.Run("CreateStoresResolvedModels", func(t *testing.T) {
url, paths := mockBedrock(t, func(w http.ResponseWriter, r *http.Request) {
modelARN := "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-opus-4-8"
if !strings.Contains(r.URL.Path, "46u2vhiyo6z5") {
modelARN = "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-haiku-4-5"
}
respondWithModel(modelARN)(w, r)
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
client := coderdtest.New(t, nil)
_ = coderdtest.CreateFirstUser(t, client)
ctx := testutil.Context(t, testutil.WaitLong)
//nolint:gocritic // Owner role is the audience for this endpoint.
created, err := client.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-profiles",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *bedrockSettings(testProfileARN, testSmallFastProfileARN),
})
require.NoError(t, err)
require.NotNil(t, created.Settings.Bedrock)
// The configured identifiers stay untouched: they remain the Bedrock
// invocation target, and AWS attributes spend to them.
require.Equal(t, testProfileARN, created.Settings.Bedrock.Model)
require.Equal(t, testSmallFastProfileARN, created.Settings.Bedrock.SmallFastModel)
require.Equal(t, "anthropic.claude-opus-4-8", created.Settings.Bedrock.ResolvedModel)
require.Equal(t, "anthropic.claude-haiku-4-5", created.Settings.Bedrock.ResolvedSmallFastModel)
require.Len(t, paths(), 2, "each profile is resolved once")
})
t.Run("CreateLeavesPlainModelIDsUnresolved", func(t *testing.T) {
url, paths := mockBedrock(t, func(http.ResponseWriter, *http.Request) {
t.Error("Bedrock called for plain model ids")
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
client := coderdtest.New(t, nil)
_ = coderdtest.CreateFirstUser(t, client)
ctx := testutil.Context(t, testutil.WaitLong)
//nolint:gocritic // Owner role is the audience for this endpoint.
created, err := client.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-plain",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *bedrockSettings("eu.anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5"),
})
require.NoError(t, err)
require.Empty(t, created.Settings.Bedrock.ResolvedModel)
require.Empty(t, created.Settings.Bedrock.ResolvedSmallFastModel)
require.Empty(t, paths())
})
t.Run("CreateIgnoresClientSuppliedResolution", func(t *testing.T) {
url, paths := mockBedrock(t, func(http.ResponseWriter, *http.Request) {
t.Error("Bedrock called for plain model ids")
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
client := coderdtest.New(t, nil)
_ = coderdtest.CreateFirstUser(t, client)
ctx := testutil.Context(t, testutil.WaitLong)
settings := bedrockSettings("eu.anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5")
settings.Bedrock.ResolvedModel = "anthropic.claude-opus-4-8"
//nolint:gocritic // Owner role is the audience for this endpoint.
created, err := client.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-spoofed",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *settings,
})
require.NoError(t, err)
require.Empty(t, created.Settings.Bedrock.ResolvedModel, "the server owns the resolution")
require.Empty(t, paths())
})
t.Run("CreateRejectsUnresolvableProfile", func(t *testing.T) {
url, _ := mockBedrock(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set("X-Amzn-Errortype", "AccessDeniedException")
w.WriteHeader(http.StatusForbidden)
_, _ = w.Write([]byte(`{"message":"not authorized to perform bedrock:GetInferenceProfile"}`))
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
client := coderdtest.New(t, nil)
_ = coderdtest.CreateFirstUser(t, client)
ctx := testutil.Context(t, testutil.WaitLong)
//nolint:gocritic // Owner role is the audience for this endpoint.
_, err := client.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-denied",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *bedrockSettings(testProfileARN, "anthropic.claude-haiku-4-5"),
})
var sdkErr *codersdk.Error
require.ErrorAs(t, err, &sdkErr)
require.Equal(t, http.StatusBadRequest, sdkErr.StatusCode())
require.Contains(t, sdkErr.Detail, "GetInferenceProfile")
// The write is rejected: a stored ARN with no resolution would be
// served as its own identity.
//nolint:gocritic // Owner role is the audience for this endpoint.
providers, err := client.AIProviders(ctx)
require.NoError(t, err)
require.Empty(t, providers)
})
t.Run("UpdateRejectsUnresolvableProfile", func(t *testing.T) {
var deny bool
url, _ := mockBedrock(t, func(w http.ResponseWriter, r *http.Request) {
if !deny {
respondWithModel("arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-opus-4-8")(w, r)
return
}
w.Header().Set("Content-Type", "application/json")
w.Header().Set("X-Amzn-Errortype", "AccessDeniedException")
w.WriteHeader(http.StatusForbidden)
_, _ = w.Write([]byte(`{"message":"not authorized to perform bedrock:GetInferenceProfile"}`))
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
client := coderdtest.New(t, nil)
_ = coderdtest.CreateFirstUser(t, client)
ctx := testutil.Context(t, testutil.WaitLong)
//nolint:gocritic // Owner role is the audience for this endpoint.
created, err := client.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-update-denied",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *bedrockSettings(testProfileARN, "anthropic.claude-haiku-4-5"),
})
require.NoError(t, err)
deny = true
//nolint:gocritic // Owner role is the audience for this endpoint.
_, err = client.UpdateAIProvider(ctx, created.ID.String(), codersdk.UpdateAIProviderRequest{
Settings: bedrockSettings(testSmallFastProfileARN, "anthropic.claude-haiku-4-5"),
})
var sdkErr *codersdk.Error
require.ErrorAs(t, err, &sdkErr)
require.Equal(t, http.StatusBadRequest, sdkErr.StatusCode())
// The stored provider still describes what it did before the failed
// update.
//nolint:gocritic // Owner role is the audience for this endpoint.
current, err := client.AIProvider(ctx, created.ID.String())
require.NoError(t, err)
require.Equal(t, testProfileARN, current.Settings.Bedrock.Model)
require.Equal(t, "anthropic.claude-opus-4-8", current.Settings.Bedrock.ResolvedModel)
})
t.Run("UpdateReresolvesChangedProfile", func(t *testing.T) {
url, _ := mockBedrock(t, respondWithModel("arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-opus-4-8"))
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
client := coderdtest.New(t, nil)
_ = coderdtest.CreateFirstUser(t, client)
ctx := testutil.Context(t, testutil.WaitLong)
//nolint:gocritic // Owner role is the audience for this endpoint.
created, err := client.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-update",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *bedrockSettings("eu.anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5"),
})
require.NoError(t, err)
require.Empty(t, created.Settings.Bedrock.ResolvedModel)
//nolint:gocritic // Owner role is the audience for this endpoint.
updated, err := client.UpdateAIProvider(ctx, created.ID.String(), codersdk.UpdateAIProviderRequest{
Settings: bedrockSettings(testProfileARN, "anthropic.claude-haiku-4-5"),
})
require.NoError(t, err)
require.Equal(t, testProfileARN, updated.Settings.Bedrock.Model)
require.Equal(t, "anthropic.claude-opus-4-8", updated.Settings.Bedrock.ResolvedModel)
require.Empty(t, updated.Settings.Bedrock.ResolvedSmallFastModel, "a plain model id is its own identity")
})
t.Run("UpdateToPlainModelIDClearsResolution", func(t *testing.T) {
url, paths := mockBedrock(t, respondWithModel("arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-opus-4-8"))
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
client := coderdtest.New(t, nil)
_ = coderdtest.CreateFirstUser(t, client)
ctx := testutil.Context(t, testutil.WaitLong)
//nolint:gocritic // Owner role is the audience for this endpoint.
created, err := client.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-replace",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *bedrockSettings(testProfileARN, "anthropic.claude-haiku-4-5"),
})
require.NoError(t, err)
require.Equal(t, "anthropic.claude-opus-4-8", created.Settings.Bedrock.ResolvedModel)
callsAfterCreate := len(paths())
//nolint:gocritic // Owner role is the audience for this endpoint.
updated, err := client.UpdateAIProvider(ctx, created.ID.String(), codersdk.UpdateAIProviderRequest{
Settings: bedrockSettings("eu.anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5"),
})
require.NoError(t, err)
require.Empty(t, updated.Settings.Bedrock.ResolvedModel)
require.Len(t, paths(), callsAfterCreate, "no profile is left to resolve")
})
t.Run("UpdateWithoutSettingsKeepsResolution", func(t *testing.T) {
url, paths := mockBedrock(t, respondWithModel("arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-opus-4-8"))
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
client := coderdtest.New(t, nil)
_ = coderdtest.CreateFirstUser(t, client)
ctx := testutil.Context(t, testutil.WaitLong)
//nolint:gocritic // Owner role is the audience for this endpoint.
created, err := client.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-keep",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *bedrockSettings(testProfileARN, "anthropic.claude-haiku-4-5"),
})
require.NoError(t, err)
callsAfterCreate := len(paths())
enabled := false
//nolint:gocritic // Owner role is the audience for this endpoint.
updated, err := client.UpdateAIProvider(ctx, created.ID.String(), codersdk.UpdateAIProviderRequest{
Enabled: &enabled,
})
require.NoError(t, err)
require.Equal(t, "anthropic.claude-opus-4-8", updated.Settings.Bedrock.ResolvedModel)
require.Len(t, paths(), callsAfterCreate, "an unrelated update does not call AWS")
})
t.Run("NonOwnerCannotDriveResolution", func(t *testing.T) {
url, _ := mockBedrock(t, func(http.ResponseWriter, *http.Request) {
t.Error("Bedrock called for an unauthorized request")
})
t.Setenv("AWS_ENDPOINT_URL_BEDROCK", url)
ownerClient := coderdtest.New(t, nil)
firstUser := coderdtest.CreateFirstUser(t, ownerClient)
ctx := testutil.Context(t, testutil.WaitLong)
//nolint:gocritic // Owner role is the audience for this endpoint.
created, err := ownerClient.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-owner-only",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *bedrockSettings("eu.anthropic.claude-opus-4-8", "anthropic.claude-haiku-4-5"),
})
require.NoError(t, err)
memberClient, _ := coderdtest.CreateAnotherUser(t, ownerClient, firstUser.OrganizationID)
_, err = memberClient.CreateAIProvider(ctx, codersdk.CreateAIProviderRequest{
Name: "bedrock-member",
Type: codersdk.AIProviderTypeBedrock,
BaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com",
Enabled: true,
Settings: *bedrockSettings(testProfileARN, "anthropic.claude-haiku-4-5"),
})
var sdkErr *codersdk.Error
require.ErrorAs(t, err, &sdkErr)
require.Equal(t, http.StatusForbidden, sdkErr.StatusCode())
_, err = memberClient.UpdateAIProvider(ctx, created.ID.String(), codersdk.UpdateAIProviderRequest{
Settings: bedrockSettings(testProfileARN, "anthropic.claude-haiku-4-5"),
})
require.ErrorAs(t, err, &sdkErr)
require.Equal(t, http.StatusForbidden, sdkErr.StatusCode())
})
}
+40
View File
@@ -0,0 +1,40 @@
package aibridge
import (
aibridgeconfig "github.com/coder/coder/v2/aibridge/config"
"github.com/coder/coder/v2/coderd/util/ptr"
"github.com/coder/coder/v2/codersdk"
)
// BedrockConfig maps stored provider settings onto the runtime Bedrock
// configuration. It is shared by the gateway, which serves requests with it,
// and by the provider write path, which resolves application inference profile
// ARNs with it.
//
// It returns nil when the settings are absent or when the Bedrock fields are
// not actually configured. The provider's BaseURL is the generic upstream
// endpoint and is always non-empty, so it cannot serve as a Bedrock detection
// signal; gate on the settings alone via
// [codersdk.AIProviderBedrockSettings.IsConfigured].
func BedrockConfig(baseURL string, bedrock *codersdk.AIProviderBedrockSettings) *aibridgeconfig.AWSBedrock {
if bedrock == nil {
return nil
}
settings := *bedrock
if !settings.IsConfigured() {
return nil
}
return &aibridgeconfig.AWSBedrock{
BaseURL: baseURL,
Region: settings.Region,
AccessKey: ptr.NilToEmpty(settings.AccessKey),
AccessKeySecret: ptr.NilToEmpty(settings.AccessKeySecret),
Model: settings.Model,
SmallFastModel: settings.SmallFastModel,
RoleARN: settings.RoleARN,
ExternalID: settings.ExternalID,
Protocol: aibridgeconfig.BedrockProtocol(settings.ResolvedProtocol()),
ResolvedModel: settings.ResolvedModel,
ResolvedSmallFastModel: settings.ResolvedSmallFastModel,
}
}
+103 -76
View File
@@ -1674,6 +1674,13 @@ type AIProviderKindBedrock struct {
// protocol selects the Bedrock wire protocol ("invoke-model" or "mantle").
// Empty falls back to invoke-model.
Protocol string `protobuf:"bytes,8,opt,name=protocol,proto3" json:"protocol,omitempty"`
// resolved_model is the model ID behind model, which differs from it only
// when model is an application inference profile ARN. coderd resolves it when
// the provider is written, so the gateway never calls the Bedrock control
// plane.
ResolvedModel string `protobuf:"bytes,9,opt,name=resolved_model,json=resolvedModel,proto3" json:"resolved_model,omitempty"`
// resolved_small_fast_model is resolved_model for small_fast_model.
ResolvedSmallFastModel string `protobuf:"bytes,10,opt,name=resolved_small_fast_model,json=resolvedSmallFastModel,proto3" json:"resolved_small_fast_model,omitempty"`
}
func (x *AIProviderKindBedrock) Reset() {
@@ -1764,6 +1771,20 @@ func (x *AIProviderKindBedrock) GetProtocol() string {
return ""
}
func (x *AIProviderKindBedrock) GetResolvedModel() string {
if x != nil {
return x.ResolvedModel
}
return ""
}
func (x *AIProviderKindBedrock) GetResolvedSmallFastModel() string {
if x != nil {
return x.ResolvedSmallFastModel
}
return ""
}
var File_coderd_aibridged_proto_aibridged_proto protoreflect.FileDescriptor
var file_coderd_aibridged_proto_aibridged_proto_rawDesc = []byte{
@@ -2060,7 +2081,7 @@ var file_coderd_aibridged_proto_aibridged_proto_rawDesc = []byte{
0x64, 0x72, 0x6f, 0x63, 0x6b, 0x18, 0x06, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1c, 0x2e, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x2e, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x4b, 0x69,
0x6e, 0x64, 0x42, 0x65, 0x64, 0x72, 0x6f, 0x63, 0x6b, 0x52, 0x07, 0x62, 0x65, 0x64, 0x72, 0x6f,
0x63, 0x6b, 0x22, 0x92, 0x02, 0x0a, 0x15, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65,
0x63, 0x6b, 0x22, 0xf4, 0x02, 0x0a, 0x15, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65,
0x72, 0x4b, 0x69, 0x6e, 0x64, 0x42, 0x65, 0x64, 0x72, 0x6f, 0x63, 0x6b, 0x12, 0x16, 0x0a, 0x06,
0x72, 0x65, 0x67, 0x69, 0x6f, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x72, 0x65,
0x67, 0x69, 0x6f, 0x6e, 0x12, 0x1d, 0x0a, 0x0a, 0x61, 0x63, 0x63, 0x65, 0x73, 0x73, 0x5f, 0x6b,
@@ -2077,83 +2098,89 @@ var file_coderd_aibridged_proto_aibridged_proto_rawDesc = []byte{
0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x5f, 0x69, 0x64, 0x18, 0x07, 0x20, 0x01, 0x28, 0x09, 0x52,
0x0a, 0x65, 0x78, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x49, 0x64, 0x12, 0x1a, 0x0a, 0x08, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x63, 0x6f, 0x6c, 0x18, 0x08, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x63, 0x6f, 0x6c, 0x32, 0xa9, 0x04, 0x0a, 0x08, 0x52, 0x65, 0x63, 0x6f,
0x72, 0x64, 0x65, 0x72, 0x12, 0x59, 0x0a, 0x12, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x49, 0x6e,
0x74, 0x65, 0x72, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x20, 0x2e, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x49, 0x6e, 0x74, 0x65, 0x72, 0x63, 0x65,
0x70, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x21, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x63, 0x6f, 0x6c, 0x12, 0x25, 0x0a, 0x0e, 0x72, 0x65, 0x73, 0x6f, 0x6c,
0x76, 0x65, 0x64, 0x5f, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x18, 0x09, 0x20, 0x01, 0x28, 0x09, 0x52,
0x0d, 0x72, 0x65, 0x73, 0x6f, 0x6c, 0x76, 0x65, 0x64, 0x4d, 0x6f, 0x64, 0x65, 0x6c, 0x12, 0x39,
0x0a, 0x19, 0x72, 0x65, 0x73, 0x6f, 0x6c, 0x76, 0x65, 0x64, 0x5f, 0x73, 0x6d, 0x61, 0x6c, 0x6c,
0x5f, 0x66, 0x61, 0x73, 0x74, 0x5f, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x18, 0x0a, 0x20, 0x01, 0x28,
0x09, 0x52, 0x16, 0x72, 0x65, 0x73, 0x6f, 0x6c, 0x76, 0x65, 0x64, 0x53, 0x6d, 0x61, 0x6c, 0x6c,
0x46, 0x61, 0x73, 0x74, 0x4d, 0x6f, 0x64, 0x65, 0x6c, 0x32, 0xa9, 0x04, 0x0a, 0x08, 0x52, 0x65,
0x63, 0x6f, 0x72, 0x64, 0x65, 0x72, 0x12, 0x59, 0x0a, 0x12, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64,
0x49, 0x6e, 0x74, 0x65, 0x72, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x20, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x49, 0x6e, 0x74, 0x65, 0x72,
0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12,
0x68, 0x0a, 0x17, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x49, 0x6e, 0x74, 0x65, 0x72, 0x63, 0x65,
0x70, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x6e, 0x64, 0x65, 0x64, 0x12, 0x25, 0x2e, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x49, 0x6e, 0x74, 0x65, 0x72, 0x63, 0x65,
0x70, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x6e, 0x64, 0x65, 0x64, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x1a, 0x26, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64,
0x49, 0x6e, 0x74, 0x65, 0x72, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x6e, 0x64, 0x65,
0x64, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x53, 0x0a, 0x10, 0x52, 0x65, 0x63,
0x6f, 0x72, 0x64, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x55, 0x73, 0x61, 0x67, 0x65, 0x12, 0x1e, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x6f, 0x6b, 0x65,
0x6e, 0x55, 0x73, 0x61, 0x67, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x6f, 0x6b, 0x65,
0x6e, 0x55, 0x73, 0x61, 0x67, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x56,
0x0a, 0x11, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x50, 0x72, 0x6f, 0x6d, 0x70, 0x74, 0x55, 0x73,
0x61, 0x67, 0x65, 0x12, 0x1f, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f,
0x72, 0x64, 0x50, 0x72, 0x6f, 0x6d, 0x70, 0x74, 0x55, 0x73, 0x61, 0x67, 0x65, 0x52, 0x65, 0x71,
0x75, 0x65, 0x73, 0x74, 0x1a, 0x20, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63,
0x6f, 0x72, 0x64, 0x50, 0x72, 0x6f, 0x6d, 0x70, 0x74, 0x55, 0x73, 0x61, 0x67, 0x65, 0x52, 0x65,
0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x50, 0x0a, 0x0f, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64,
0x54, 0x6f, 0x6f, 0x6c, 0x55, 0x73, 0x61, 0x67, 0x65, 0x12, 0x1d, 0x2e, 0x70, 0x72, 0x6f, 0x74,
0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x6f, 0x6f, 0x6c, 0x55, 0x73, 0x61, 0x67,
0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x6f, 0x6f, 0x6c, 0x55, 0x73, 0x61, 0x67, 0x65,
0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x59, 0x0a, 0x12, 0x52, 0x65, 0x63, 0x6f,
0x72, 0x64, 0x4d, 0x6f, 0x64, 0x65, 0x6c, 0x54, 0x68, 0x6f, 0x75, 0x67, 0x68, 0x74, 0x12, 0x20,
0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x4d, 0x6f, 0x64,
0x65, 0x6c, 0x54, 0x68, 0x6f, 0x75, 0x67, 0x68, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,
0x1a, 0x21, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x4d,
0x6f, 0x64, 0x65, 0x6c, 0x54, 0x68, 0x6f, 0x75, 0x67, 0x68, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f,
0x6e, 0x73, 0x65, 0x32, 0xeb, 0x01, 0x0a, 0x0f, 0x4d, 0x43, 0x50, 0x43, 0x6f, 0x6e, 0x66, 0x69,
0x67, 0x75, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x12, 0x5c, 0x0a, 0x13, 0x47, 0x65, 0x74, 0x4d, 0x43,
0x50, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x73, 0x12, 0x21,
0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x4d, 0x43, 0x50, 0x53, 0x65, 0x72,
0x76, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x1a, 0x22, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x4d, 0x43, 0x50,
0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x73, 0x52, 0x65, 0x73,
0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x7a, 0x0a, 0x1d, 0x47, 0x65, 0x74, 0x4d, 0x43, 0x50, 0x53,
0x65, 0x72, 0x76, 0x65, 0x72, 0x41, 0x63, 0x63, 0x65, 0x73, 0x73, 0x54, 0x6f, 0x6b, 0x65, 0x6e,
0x73, 0x42, 0x61, 0x74, 0x63, 0x68, 0x12, 0x2b, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47,
0x65, 0x74, 0x4d, 0x43, 0x50, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x41, 0x63, 0x63, 0x65, 0x73,
0x73, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x73, 0x42, 0x61, 0x74, 0x63, 0x68, 0x52, 0x65, 0x71, 0x75,
0x65, 0x73, 0x74, 0x1a, 0x2c, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x4d,
0x43, 0x50, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x41, 0x63, 0x63, 0x65, 0x73, 0x73, 0x54, 0x6f,
0x6b, 0x65, 0x6e, 0x73, 0x42, 0x61, 0x74, 0x63, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,
0x65, 0x32, 0xaa, 0x01, 0x0a, 0x0a, 0x41, 0x75, 0x74, 0x68, 0x6f, 0x72, 0x69, 0x7a, 0x65, 0x72,
0x12, 0x47, 0x0a, 0x0c, 0x49, 0x73, 0x41, 0x75, 0x74, 0x68, 0x6f, 0x72, 0x69, 0x7a, 0x65, 0x64,
0x12, 0x1a, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x49, 0x73, 0x41, 0x75, 0x74, 0x68, 0x6f,
0x72, 0x69, 0x7a, 0x65, 0x64, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1b, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x49, 0x73, 0x41, 0x75, 0x74, 0x68, 0x6f, 0x72, 0x69, 0x7a, 0x65,
0x64, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x53, 0x0a, 0x10, 0x49, 0x73, 0x42,
0x75, 0x64, 0x67, 0x65, 0x74, 0x45, 0x78, 0x63, 0x65, 0x65, 0x64, 0x65, 0x64, 0x12, 0x1e, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x49, 0x73, 0x42, 0x75, 0x64, 0x67, 0x65, 0x74, 0x45, 0x78,
0x63, 0x65, 0x65, 0x64, 0x65, 0x64, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x49, 0x73, 0x42, 0x75, 0x64, 0x67, 0x65, 0x74, 0x45, 0x78,
0x63, 0x65, 0x65, 0x64, 0x65, 0x64, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x32, 0xbc,
0x01, 0x0a, 0x14, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69,
0x67, 0x75, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x12, 0x4d, 0x0a, 0x0e, 0x47, 0x65, 0x74, 0x41, 0x49,
0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x12, 0x1c, 0x2e, 0x70, 0x72, 0x6f, 0x74,
0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x21,
0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x49, 0x6e, 0x74,
0x65, 0x72, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,
0x65, 0x12, 0x68, 0x0a, 0x17, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x49, 0x6e, 0x74, 0x65, 0x72,
0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x6e, 0x64, 0x65, 0x64, 0x12, 0x25, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x49, 0x6e, 0x74, 0x65, 0x72,
0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x6e, 0x64, 0x65, 0x64, 0x52, 0x65, 0x71, 0x75,
0x65, 0x73, 0x74, 0x1a, 0x26, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f,
0x72, 0x64, 0x49, 0x6e, 0x74, 0x65, 0x72, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x6e,
0x64, 0x65, 0x64, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x53, 0x0a, 0x10, 0x52,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x55, 0x73, 0x61, 0x67, 0x65, 0x12,
0x1e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x6f,
0x6b, 0x65, 0x6e, 0x55, 0x73, 0x61, 0x67, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
0x1f, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x6f,
0x6b, 0x65, 0x6e, 0x55, 0x73, 0x61, 0x67, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x12, 0x56, 0x0a, 0x11, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x50, 0x72, 0x6f, 0x6d, 0x70, 0x74,
0x55, 0x73, 0x61, 0x67, 0x65, 0x12, 0x1f, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65,
0x63, 0x6f, 0x72, 0x64, 0x50, 0x72, 0x6f, 0x6d, 0x70, 0x74, 0x55, 0x73, 0x61, 0x67, 0x65, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x20, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x50, 0x72, 0x6f, 0x6d, 0x70, 0x74, 0x55, 0x73, 0x61, 0x67, 0x65,
0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x50, 0x0a, 0x0f, 0x52, 0x65, 0x63, 0x6f,
0x72, 0x64, 0x54, 0x6f, 0x6f, 0x6c, 0x55, 0x73, 0x61, 0x67, 0x65, 0x12, 0x1d, 0x2e, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x6f, 0x6f, 0x6c, 0x55, 0x73,
0x61, 0x67, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x6f, 0x6f, 0x6c, 0x55, 0x73, 0x61,
0x67, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x59, 0x0a, 0x12, 0x52, 0x65,
0x63, 0x6f, 0x72, 0x64, 0x4d, 0x6f, 0x64, 0x65, 0x6c, 0x54, 0x68, 0x6f, 0x75, 0x67, 0x68, 0x74,
0x12, 0x20, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x4d,
0x6f, 0x64, 0x65, 0x6c, 0x54, 0x68, 0x6f, 0x75, 0x67, 0x68, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65,
0x73, 0x74, 0x1a, 0x21, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72,
0x64, 0x4d, 0x6f, 0x64, 0x65, 0x6c, 0x54, 0x68, 0x6f, 0x75, 0x67, 0x68, 0x74, 0x52, 0x65, 0x73,
0x70, 0x6f, 0x6e, 0x73, 0x65, 0x32, 0xeb, 0x01, 0x0a, 0x0f, 0x4d, 0x43, 0x50, 0x43, 0x6f, 0x6e,
0x66, 0x69, 0x67, 0x75, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x12, 0x5c, 0x0a, 0x13, 0x47, 0x65, 0x74,
0x4d, 0x43, 0x50, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x73,
0x12, 0x21, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x4d, 0x43, 0x50, 0x53,
0x65, 0x72, 0x76, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x73, 0x52, 0x65, 0x71, 0x75,
0x65, 0x73, 0x74, 0x1a, 0x22, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x4d,
0x43, 0x50, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x73, 0x52,
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x7a, 0x0a, 0x1d, 0x47, 0x65, 0x74, 0x4d, 0x43,
0x50, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x41, 0x63, 0x63, 0x65, 0x73, 0x73, 0x54, 0x6f, 0x6b,
0x65, 0x6e, 0x73, 0x42, 0x61, 0x74, 0x63, 0x68, 0x12, 0x2b, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x2e, 0x47, 0x65, 0x74, 0x4d, 0x43, 0x50, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x41, 0x63, 0x63,
0x65, 0x73, 0x73, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x73, 0x42, 0x61, 0x74, 0x63, 0x68, 0x52, 0x65,
0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2c, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65,
0x74, 0x4d, 0x43, 0x50, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x41, 0x63, 0x63, 0x65, 0x73, 0x73,
0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x73, 0x42, 0x61, 0x74, 0x63, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f,
0x6e, 0x73, 0x65, 0x32, 0xaa, 0x01, 0x0a, 0x0a, 0x41, 0x75, 0x74, 0x68, 0x6f, 0x72, 0x69, 0x7a,
0x65, 0x72, 0x12, 0x47, 0x0a, 0x0c, 0x49, 0x73, 0x41, 0x75, 0x74, 0x68, 0x6f, 0x72, 0x69, 0x7a,
0x65, 0x64, 0x12, 0x1a, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x49, 0x73, 0x41, 0x75, 0x74,
0x68, 0x6f, 0x72, 0x69, 0x7a, 0x65, 0x64, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1b,
0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x49, 0x73, 0x41, 0x75, 0x74, 0x68, 0x6f, 0x72, 0x69,
0x7a, 0x65, 0x64, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x53, 0x0a, 0x10, 0x49,
0x73, 0x42, 0x75, 0x64, 0x67, 0x65, 0x74, 0x45, 0x78, 0x63, 0x65, 0x65, 0x64, 0x65, 0x64, 0x12,
0x1e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x49, 0x73, 0x42, 0x75, 0x64, 0x67, 0x65, 0x74,
0x45, 0x78, 0x63, 0x65, 0x65, 0x64, 0x65, 0x64, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
0x1f, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x49, 0x73, 0x42, 0x75, 0x64, 0x67, 0x65, 0x74,
0x45, 0x78, 0x63, 0x65, 0x65, 0x64, 0x65, 0x64, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x32, 0xbc, 0x01, 0x0a, 0x14, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x43, 0x6f, 0x6e,
0x66, 0x69, 0x67, 0x75, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x12, 0x4d, 0x0a, 0x0e, 0x47, 0x65, 0x74,
0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x12, 0x1c, 0x2e, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65,
0x72, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d, 0x2e, 0x70, 0x72, 0x6f, 0x74,
0x6f, 0x2e, 0x47, 0x65, 0x74, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73,
0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e,
0x47, 0x65, 0x74, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x52, 0x65,
0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x55, 0x0a, 0x10, 0x57, 0x61, 0x74, 0x63, 0x68, 0x41,
0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x12, 0x1e, 0x2e, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x2e, 0x57, 0x61, 0x74, 0x63, 0x68, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64,
0x65, 0x72, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x2e, 0x57, 0x61, 0x74, 0x63, 0x68, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64,
0x65, 0x72, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x30, 0x01, 0x42, 0x32, 0x5a,
0x30, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x6f, 0x64, 0x65,
0x72, 0x2f, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2f, 0x76, 0x32, 0x2f, 0x63, 0x6f, 0x64, 0x65, 0x72,
0x64, 0x2f, 0x61, 0x69, 0x62, 0x72, 0x69, 0x64, 0x67, 0x65, 0x64, 0x2f, 0x70, 0x72, 0x6f, 0x74,
0x6f, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x55, 0x0a, 0x10, 0x57, 0x61, 0x74, 0x63,
0x68, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x12, 0x1e, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x57, 0x61, 0x74, 0x63, 0x68, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76,
0x69, 0x64, 0x65, 0x72, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x57, 0x61, 0x74, 0x63, 0x68, 0x41, 0x49, 0x50, 0x72, 0x6f, 0x76,
0x69, 0x64, 0x65, 0x72, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x30, 0x01, 0x42,
0x32, 0x5a, 0x30, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x6f,
0x64, 0x65, 0x72, 0x2f, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2f, 0x76, 0x32, 0x2f, 0x63, 0x6f, 0x64,
0x65, 0x72, 0x64, 0x2f, 0x61, 0x69, 0x62, 0x72, 0x69, 0x64, 0x67, 0x65, 0x64, 0x2f, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
+7
View File
@@ -236,4 +236,11 @@ message AIProviderKindBedrock {
// protocol selects the Bedrock wire protocol ("invoke-model" or "mantle").
// Empty falls back to invoke-model.
string protocol = 8;
// resolved_model is the model ID behind model, which differs from it only
// when model is an application inference profile ARN. coderd resolves it when
// the provider is written, so the gateway never calls the Bedrock control
// plane.
string resolved_model = 9;
// resolved_small_fast_model is resolved_model for small_fast_model.
string resolved_small_fast_model = 10;
}
+10 -8
View File
@@ -1191,14 +1191,16 @@ func aiProviderToProto(row database.AIProvider, keys []database.AIProviderKey) (
}
if settings.Bedrock != nil {
p.Bedrock = &proto.AIProviderKindBedrock{
Region: settings.Bedrock.Region,
AccessKey: ptr.NilToEmpty(settings.Bedrock.AccessKey),
AccessKeySecret: ptr.NilToEmpty(settings.Bedrock.AccessKeySecret),
Model: settings.Bedrock.Model,
SmallFastModel: settings.Bedrock.SmallFastModel,
RoleArn: settings.Bedrock.RoleARN,
ExternalId: settings.Bedrock.ExternalID,
Protocol: string(settings.Bedrock.Protocol),
Region: settings.Bedrock.Region,
AccessKey: ptr.NilToEmpty(settings.Bedrock.AccessKey),
AccessKeySecret: ptr.NilToEmpty(settings.Bedrock.AccessKeySecret),
Model: settings.Bedrock.Model,
SmallFastModel: settings.Bedrock.SmallFastModel,
RoleArn: settings.Bedrock.RoleARN,
ExternalId: settings.Bedrock.ExternalID,
Protocol: string(settings.Bedrock.Protocol),
ResolvedModel: settings.Bedrock.ResolvedModel,
ResolvedSmallFastModel: settings.Bedrock.ResolvedSmallFastModel,
}
}
+6
View File
@@ -61,6 +61,12 @@ type AIProviderBedrockSettings struct {
// AIProviderBedrockProtocolInvokeModel, so existing rows keep the legacy
// behavior.
Protocol AIProviderBedrockProtocol `json:"protocol,omitempty"`
// ResolvedModel and ResolvedSmallFastModel are the model IDs behind the
// configured identifiers, which differ from them only for application
// inference profile ARNs. The server resolves those through AWS when the
// provider is written and owns the values; a client cannot set them.
ResolvedModel string `json:"resolved_model,omitempty"`
ResolvedSmallFastModel string `json:"resolved_small_fast_model,omitempty"`
}
// ResolvedProtocol returns the configured protocol, mapping the empty value to
+8
View File
@@ -443,6 +443,14 @@ export interface AIProviderBedrockSettings {
* behavior.
*/
readonly protocol?: AIProviderBedrockProtocol;
/**
* ResolvedModel and ResolvedSmallFastModel are the model IDs behind the
* configured identifiers, which differ from them only for application
* inference profile ARNs. The server resolves those through AWS when the
* provider is written and owns the values; a client cannot set them.
*/
readonly resolved_model?: string;
readonly resolved_small_fast_model?: string;
}
// From codersdk/aiproviders_bedrock.go