mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
feat(codersdk): add circuit breaker configuration support for aibridge (#21546)
## Summary Add circuit breaker support for AI Bridge to protect against cascading failures from upstream AI provider rate limits (HTTP 429, 503, and Anthropic's 529 overloaded responses). ## Changes - Add 5 new CLI options for circuit breaker configuration: - `--aibridge-circuit-breaker-enabled` (default: false) - `--aibridge-circuit-breaker-failure-threshold` (default: 5) - `--aibridge-circuit-breaker-interval` (default: 10s) - `--aibridge-circuit-breaker-timeout` (default: 30s) - `--aibridge-circuit-breaker-max-requests` (default: 3) - Update aibridge dependency to include circuit breaker support - Add tests for pool creation with circuit breaker providers ## Notes - Circuit breaker is **disabled by default** for backward compatibility - When enabled, applies to both OpenAI and Anthropic providers - Uses sony/gobreaker internally via the aibridge library ## Testing ``` make test RUN=TestPoolWithCircuitBreakerProviders ```
This commit is contained in:
+4
@@ -121,6 +121,10 @@ AI BRIDGE OPTIONS:
|
||||
See
|
||||
https://docs.claude.com/en/docs/claude-code/settings#environment-variables.
|
||||
|
||||
--aibridge-circuit-breaker-enabled bool, $CODER_AIBRIDGE_CIRCUIT_BREAKER_ENABLED (default: false)
|
||||
Enable the circuit breaker to protect against cascading failures from
|
||||
upstream AI provider rate limits (429, 503, 529 overloaded).
|
||||
|
||||
--aibridge-retention duration, $CODER_AIBRIDGE_RETENTION (default: 60d)
|
||||
Length of time to retain data such as interceptions and all related
|
||||
records (token, prompt, tool use).
|
||||
|
||||
+17
@@ -777,6 +777,23 @@ aibridge:
|
||||
# these records to external SIEM or observability systems.
|
||||
# (default: false, type: bool)
|
||||
structuredLogging: false
|
||||
# Enable the circuit breaker to protect against cascading failures from upstream
|
||||
# AI provider rate limits (429, 503, 529 overloaded).
|
||||
# (default: false, type: bool)
|
||||
circuitBreakerEnabled: false
|
||||
# Number of consecutive failures that triggers the circuit breaker to open.
|
||||
# (default: 5, type: int)
|
||||
circuitBreakerFailureThreshold: 5
|
||||
# Cyclic period of the closed state for clearing internal failure counts.
|
||||
# (default: 10s, type: duration)
|
||||
circuitBreakerInterval: 10s
|
||||
# How long the circuit breaker stays open before transitioning to half-open state.
|
||||
# (default: 30s, type: duration)
|
||||
circuitBreakerTimeout: 30s
|
||||
# Maximum number of requests allowed in half-open state before deciding to close
|
||||
# or re-open the circuit.
|
||||
# (default: 3, type: int)
|
||||
circuitBreakerMaxRequests: 3
|
||||
aibridgeproxy:
|
||||
# Enable the AI Bridge MITM Proxy for intercepting and decrypting AI provider
|
||||
# requests.
|
||||
|
||||
Reference in New Issue
Block a user