feat: add automatic key failover for AI Bridge OpenAI (#24847)

## Description

Adds automatic key failover for centralized OpenAI provider, covering both chat completions and responses APIs. Same shape as the Anthropic PR: each upstream call walks the configured key pool, keys are marked **temporary** on 429 (with cooldown from `Retry-After`) and **permanent** on 401/403. Each agentic-loop iteration gets its own fresh walker so a tool-call continuation can fail over independently of the initial request.

BYOK is unchanged: BYOK requests run as a single attempt with no failover.

## Changes

- `config.OpenAI` carries a `KeyPool`. `Key` remains for BYOK Authorization Bearer set per interception.
- Chat completions blocking interceptor: walks the pool via `newChatCompletionWithKeyFailover`, marks keys on key-specific failures, returns on first success or non-failover error.
- Chat completions streaming interceptor: per-iteration walker. Pre-stream failures fail over to the next key; mid-stream errors are relayed as SSE events.
- Responses blocking interceptor: extracts `newResponseWithKeyFailover` parallel to chatcompletions.
- Responses streaming interceptor: per-iteration walker, retains the existing buffer-then-forward design.

## Related Issues

Related to: https://github.com/coder/internal/issues/1446
Related to: https://linear.app/codercom/issue/AIGOV-197/aibridge-automatic-key-failover-for-bridged-and-passthrough-routes

## Follow-up PRs

- Bedrock multi-key support.
- Refactor provider vs interceptor config separation.
- Record the actually-used key in the interception credential hint after failover.

> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
This commit is contained in:
Susana Ferreira
2026-05-07 15:35:46 +01:00
committed by GitHub
parent f1155ac4d7
commit b6dacb4a3c
20 changed files with 3112 additions and 102 deletions
+7 -4
View File
@@ -122,15 +122,18 @@ func buildProviders(cfg codersdk.AIBridgeConfig) ([]aibridge.Provider, error) {
}
switch p.Type {
case aibridge.ProviderOpenAI:
// TODO(ssncferreira): pass a keypool.Pool instead.
var key string
var pool *keypool.Pool
if len(p.Keys) > 0 {
key = p.Keys[0]
var err error
pool, err = keypool.New(p.Keys, quartz.NewReal())
if err != nil {
return nil, xerrors.Errorf("create openai key pool for provider %q: %w", name, err)
}
}
providers = append(providers, aibridge.NewOpenAIProvider(aibridge.OpenAIConfig{
Name: name,
BaseURL: p.BaseURL,
Key: key,
KeyPool: pool,
APIDumpDir: p.DumpDir,
CircuitBreaker: cbConfig,
SendActorHeaders: cfg.SendActorHeaders.Value(),