opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(codersdk): add circuit breaker configuration support for aibridge (#21546)

Browse Source 
## Summary

Add circuit breaker support for AI Bridge to protect against cascading
failures from upstream AI provider rate limits (HTTP 429, 503, and
Anthropic's 529 overloaded responses).

## Changes

- Add 5 new CLI options for circuit breaker configuration:
  - `--aibridge-circuit-breaker-enabled` (default: false)
  - `--aibridge-circuit-breaker-failure-threshold` (default: 5)
  - `--aibridge-circuit-breaker-interval` (default: 10s)
  - `--aibridge-circuit-breaker-timeout` (default: 30s)
  - `--aibridge-circuit-breaker-max-requests` (default: 3)
- Update aibridge dependency to include circuit breaker support
- Add tests for pool creation with circuit breaker providers

## Notes

- Circuit breaker is **disabled by default** for backward compatibility
- When enabled, applies to both OpenAI and Anthropic providers
- Uses sony/gobreaker internally via the aibridge library

## Testing

```
make test RUN=TestPoolWithCircuitBreakerProviders
```
This commit is contained in:
Kacper Sawicki
2026-01-20 14:59:29 +01:00
committed by GitHub
parent bfae5b03dc
commit ed679bb3da
12 changed files with 341 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
enterprise/cli/aibridged.go
+19 -5
View File
@@ -9,6 +9,7 @@ import (
"golang.org/x/xerrors"
"github.com/coder/aibridge"
"github.com/coder/aibridge/config"
"github.com/coder/coder/v2/coderd/tracing"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/enterprise/aibridged"
@@ -21,15 +22,28 @@ func newAIBridgeDaemon(coderAPI *coderd.API) (*aibridged.Server, error) {
logger := coderAPI.Logger.Named("aibridged")
// Setup supported providers.
// Build circuit breaker config if enabled.
var cbConfig *config.CircuitBreaker
if coderAPI.DeploymentValues.AI.BridgeConfig.CircuitBreakerEnabled.Value() {
cbConfig = &config.CircuitBreaker{
FailureThreshold: uint32(coderAPI.DeploymentValues.AI.BridgeConfig.CircuitBreakerFailureThreshold.Value()), //nolint:gosec // Validated by serpent.Validate in deployment options.
Interval: coderAPI.DeploymentValues.AI.BridgeConfig.CircuitBreakerInterval.Value(),
Timeout: coderAPI.DeploymentValues.AI.BridgeConfig.CircuitBreakerTimeout.Value(),
MaxRequests: uint32(coderAPI.DeploymentValues.AI.BridgeConfig.CircuitBreakerMaxRequests.Value()), //nolint:gosec // Validated by serpent.Validate in deployment options.
}
}
// Setup supported providers with circuit breaker config.
providers := []aibridge.Provider{
aibridge.NewOpenAIProvider(aibridge.OpenAIConfig{
BaseURL: coderAPI.DeploymentValues.AI.BridgeConfig.OpenAI.BaseURL.String(),
Key: coderAPI.DeploymentValues.AI.BridgeConfig.OpenAI.Key.String(),
BaseURL: coderAPI.DeploymentValues.AI.BridgeConfig.OpenAI.BaseURL.String(),
Key: coderAPI.DeploymentValues.AI.BridgeConfig.OpenAI.Key.String(),
CircuitBreaker: cbConfig,
}),
aibridge.NewAnthropicProvider(aibridge.AnthropicConfig{
BaseURL: coderAPI.DeploymentValues.AI.BridgeConfig.Anthropic.BaseURL.String(),
Key: coderAPI.DeploymentValues.AI.BridgeConfig.Anthropic.Key.String(),
BaseURL: coderAPI.DeploymentValues.AI.BridgeConfig.Anthropic.BaseURL.String(),
Key: coderAPI.DeploymentValues.AI.BridgeConfig.Anthropic.Key.String(),
CircuitBreaker: cbConfig,
}, getBedrockConfig(coderAPI.DeploymentValues.AI.BridgeConfig.Bedrock)),
}
enterprise/cli/testdata/coder_server_--help.golden
Vendored
+4
View File
@@ -122,6 +122,10 @@ AI BRIDGE OPTIONS:
See
https://docs.claude.com/en/docs/claude-code/settings#environment-variables.
--aibridge-circuit-breaker-enabled bool, $CODER_AIBRIDGE_CIRCUIT_BREAKER_ENABLED (default: false)
Enable the circuit breaker to protect against cascading failures from
upstream AI provider rate limits (429, 503, 529 overloaded).
--aibridge-retention duration, $CODER_AIBRIDGE_RETENTION (default: 60d)
Length of time to retain data such as interceptions and all related
records (token, prompt, tool use).