opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(codersdk): add circuit breaker configuration support for aibridge (#21546)

Browse Source 
## Summary

Add circuit breaker support for AI Bridge to protect against cascading
failures from upstream AI provider rate limits (HTTP 429, 503, and
Anthropic's 529 overloaded responses).

## Changes

- Add 5 new CLI options for circuit breaker configuration:
  - `--aibridge-circuit-breaker-enabled` (default: false)
  - `--aibridge-circuit-breaker-failure-threshold` (default: 5)
  - `--aibridge-circuit-breaker-interval` (default: 10s)
  - `--aibridge-circuit-breaker-timeout` (default: 30s)
  - `--aibridge-circuit-breaker-max-requests` (default: 3)
- Update aibridge dependency to include circuit breaker support
- Add tests for pool creation with circuit breaker providers

## Notes

- Circuit breaker is **disabled by default** for backward compatibility
- When enabled, applies to both OpenAI and Anthropic providers
- Uses sony/gobreaker internally via the aibridge library

## Testing

```
make test RUN=TestPoolWithCircuitBreakerProviders
```
This commit is contained in:
Kacper Sawicki
2026-01-20 14:59:29 +01:00
committed by GitHub
parent bfae5b03dc
commit ed679bb3da
12 changed files with 341 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reference/api/general.md
Generated
+5
View File
@@ -185,6 +185,11 @@ curl -X GET http://coder-server:8080/api/v2/deployment/config \
"region": "string",
"small_fast_model": "string"
},
"circuit_breaker_enabled": true,
"circuit_breaker_failure_threshold": 0,
"circuit_breaker_interval": 0,
"circuit_breaker_max_requests": 0,
"circuit_breaker_timeout": 0,
"enabled": true,
"inject_coder_mcp_tools": true,
"max_concurrency": 0,
docs/reference/api/schemas.md
Generated
+36 -11
View File
@@ -388,6 +388,11 @@
"region": "string",
"small_fast_model": "string"
},
"circuit_breaker_enabled": true,
"circuit_breaker_failure_threshold": 0,
"circuit_breaker_interval": 0,
"circuit_breaker_max_requests": 0,
"circuit_breaker_timeout": 0,
"enabled": true,
"inject_coder_mcp_tools": true,
"max_concurrency": 0,
@@ -403,17 +408,22 @@
### Properties
| Name | Type | Required | Restrictions | Description |
|--------------------------|----------------------------------------------------------------------|----------|--------------|-------------|
| `anthropic` | [codersdk.AIBridgeAnthropicConfig](#codersdkaibridgeanthropicconfig) | false | | |
| `bedrock` | [codersdk.AIBridgeBedrockConfig](#codersdkaibridgebedrockconfig) | false | | |
| `enabled` | boolean | false | | |
| `inject_coder_mcp_tools` | boolean | false | | |
| `max_concurrency` | integer | false | | |
| `openai` | [codersdk.AIBridgeOpenAIConfig](#codersdkaibridgeopenaiconfig) | false | | |
| `rate_limit` | integer | false | | |
| `retention` | integer | false | | |
| `structured_logging` | boolean | false | | |
| Name | Type | Required | Restrictions | Description |
|-------------------------------------|----------------------------------------------------------------------|----------|--------------|-----------------------------------------------------------------------------------------------------------------------|
| `anthropic` | [codersdk.AIBridgeAnthropicConfig](#codersdkaibridgeanthropicconfig) | false | | |
| `bedrock` | [codersdk.AIBridgeBedrockConfig](#codersdkaibridgebedrockconfig) | false | | |
| `circuit_breaker_enabled` | boolean | false | | Circuit breaker protects against cascading failures from upstream AI provider rate limits (429, 503, 529 overloaded). |
| `circuit_breaker_failure_threshold` | integer | false | | |
| `circuit_breaker_interval` | integer | false | | |
| `circuit_breaker_max_requests` | integer | false | | |
| `circuit_breaker_timeout` | integer | false | | |
| `enabled` | boolean | false | | |
| `inject_coder_mcp_tools` | boolean | false | | |
| `max_concurrency` | integer | false | | |
| `openai` | [codersdk.AIBridgeOpenAIConfig](#codersdkaibridgeopenaiconfig) | false | | |
| `rate_limit` | integer | false | | |
| `retention` | integer | false | | |
| `structured_logging` | boolean | false | | |
## codersdk.AIBridgeInterception
@@ -743,6 +753,11 @@
"region": "string",
"small_fast_model": "string"
},
"circuit_breaker_enabled": true,
"circuit_breaker_failure_threshold": 0,
"circuit_breaker_interval": 0,
"circuit_breaker_max_requests": 0,
"circuit_breaker_timeout": 0,
"enabled": true,
"inject_coder_mcp_tools": true,
"max_concurrency": 0,
@@ -2661,6 +2676,11 @@ CreateWorkspaceRequest provides options for creating a new workspace. Only one o
"region": "string",
"small_fast_model": "string"
},
"circuit_breaker_enabled": true,
"circuit_breaker_failure_threshold": 0,
"circuit_breaker_interval": 0,
"circuit_breaker_max_requests": 0,
"circuit_breaker_timeout": 0,
"enabled": true,
"inject_coder_mcp_tools": true,
"max_concurrency": 0,
@@ -3208,6 +3228,11 @@ CreateWorkspaceRequest provides options for creating a new workspace. Only one o
"region": "string",
"small_fast_model": "string"
},
"circuit_breaker_enabled": true,
"circuit_breaker_failure_threshold": 0,
"circuit_breaker_interval": 0,
"circuit_breaker_max_requests": 0,
"circuit_breaker_timeout": 0,
"enabled": true,
"inject_coder_mcp_tools": true,
"max_concurrency": 0,