mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
feat: add key pool failover metrics to aibridge (#25901)
## Description This PR adds Prometheus metrics for aibridge's API-key failover, giving visibility into key pool health and failover behavior per provider. The following metrics are introduced: - **`key_pool_state`** (gauge): number of keys currently in each state (`valid`, `temporary`, `permanent`) per provider, sampled at scrape time. - **`key_pool_state_transitions_total`** (counter): key state transitions during failover, labeled by `reason` (`rate_limited`, `unauthorized`, `forbidden`). - **`key_pool_exhaustions_total`** (counter): times a pool ran out of usable keys, labeled by `outcome` (`rate_limited`, `auth_failed`). - **`key_pool_failover_attempts`** (histogram): keys attempted before success or exhaustion (per interception for bridged requests, per request for passthrough). ## Changes - Moves `MarkKeyOnStatus` and key-pool error handling onto `*keypool.Pool`. - Attaches metrics to each provider's key pool at install time, on construction and on provider reload. - Adds a scrape-time state collector and a `KeyPools()` accessor on the bridge pool to feed it. - Tracks per-request key attempts in the bridged and passthrough failover paths. - Adds test coverage for the new metrics across the keypool unit tests, the bridged intercept failover tests, and the passthrough failover test. Closes https://github.com/coder/internal/issues/1447 Closes https://linear.app/codercom/issue/AIGOV-198/aibridge-key-failover-observability > [!NOTE] > Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
This commit is contained in:
@@ -17,6 +17,7 @@ import (
|
||||
|
||||
"cdr.dev/slog/v3"
|
||||
"github.com/coder/coder/v2/aibridge"
|
||||
"github.com/coder/coder/v2/aibridge/keypool"
|
||||
"github.com/coder/coder/v2/aibridge/mcp"
|
||||
"github.com/coder/coder/v2/aibridge/tracing"
|
||||
)
|
||||
@@ -148,6 +149,18 @@ func (p *CachedBridgePool) loadProviders() []aibridge.Provider {
|
||||
return nil
|
||||
}
|
||||
|
||||
// KeyPools returns the key pools of the current live providers.
|
||||
func (p *CachedBridgePool) KeyPools() []*keypool.Pool {
|
||||
providers := p.loadProviders()
|
||||
pools := make([]*keypool.Pool, 0, len(providers))
|
||||
for _, prov := range providers {
|
||||
if pool := prov.KeyPool(); pool != nil {
|
||||
pools = append(pools, pool)
|
||||
}
|
||||
}
|
||||
return pools
|
||||
}
|
||||
|
||||
// Acquire retrieves or creates a [*aibridge.RequestBridge] instance per given key.
|
||||
//
|
||||
// Each returned [*aibridge.RequestBridge] is safe for concurrent use.
|
||||
|
||||
Reference in New Issue
Block a user