mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
fix: remove 403 from key failover and cooldown on 401 (#27419)
## Problem When a key returned 401 or 403, the pool marked it permanently unavailable for the lifetime of that in-memory pool. This is bad UX: a transient auth failure or a briefly-misconfigured key could take a key out of rotation until the operator either restarted Coder or reconfigured the key (even re-saving the same working value). ## Changes - **403 removed from key failover**: it's a per-request authorization failure, not a key-level problem, so it's surfaced to the caller as-is without marking the key or failing over. - **401 now applies a temporary cooldown** (like 429) so the key recovers on its own instead of staying blocked. - When every key is in an auth-failure cooldown, the pool reports a `502` with no `Retry-After`, but the keys still recover automatically once the cooldown elapses. Closes https://linear.app/codercom/issue/AIGOV-421/ai-gateway-a-quarantined-centralized-key-never-recovers-without-a Closes https://linear.app/codercom/issue/AIGOV-533/403s-misclassifying-keys-as-permanently-down-in-ai-gateway > [!NOTE] > Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
This commit is contained in:
@@ -322,18 +322,18 @@ a maximum of **5 keys**.
|
||||
### Failover behavior
|
||||
|
||||
Every request starts with the first key in the list. If a key is rate-limited
|
||||
or returns an authentication error, AI Gateway automatically retries the request
|
||||
with the next available key.
|
||||
|
||||
> [!WARNING]
|
||||
> A key that fails with an authentication error (`401 Unauthorized` or
|
||||
> `403 Forbidden`) is permanently disabled and will not be used again until the
|
||||
> server is restarted or the provider configuration is reloaded.
|
||||
(`429 Too Many Requests`) or fails authentication (`401 Unauthorized`), AI
|
||||
Gateway puts that key on a temporary cooldown and retries the request with the
|
||||
next available key. Keys recover automatically when the cooldown elapses, so
|
||||
failover stays transparent to end users. Any other response, including a
|
||||
`403 Forbidden`, is returned to the caller unchanged.
|
||||
|
||||
If all keys in the pool are exhausted, AI Gateway returns:
|
||||
|
||||
- `429 Too Many Requests` when at least one key is rate-limited, with a `Retry-After` header set to the shortest cooldown across all keys.
|
||||
- `502 Bad Gateway` when every key has failed permanently.
|
||||
- `429 Too Many Requests` when at least one key is rate-limited, with a `Retry-After`
|
||||
header set to the shortest cooldown across all keys.
|
||||
- `502 Bad Gateway` when every key is in an authentication-failure cooldown.
|
||||
The keys still recover automatically once their cooldowns elapse, so no `Retry-After` is sent.
|
||||
|
||||
## Bring Your Own Key
|
||||
|
||||
|
||||
Reference in New Issue
Block a user