mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
feat: limit concurrent chat agents with pooled admission (#27902)
Limits concurrent chat generation on capped deployments to 5 root chats and 10 delegated subagent chats. The pools are deployment-wide and independent, so delegated work can continue while root capacity is full. The default caps live in AGPL code. Enterprise contributes only a licensing unlock, so unlicensed deployments stay capped and cannot fail open. Licensed deployments are uncapped while Agent Hours usage stays below an explicit hard limit. Deployments without a hard limit remain uncapped, and reaching the Agent Hours allocation only triggers warnings. Admission happens before a worker takes chat ownership. Capped deployments serialize admission across replicas with a transaction-scoped advisory lock and derive active and queued state from current ownership plus fresh runner heartbeats, rather than persisted queue markers or per-replica state. The acquisition query returns a bounded, pool-interleaved candidate set instead of ranking the whole backlog; a migration replaces the acquisition index with a pool-aware one. Refused chats stay running but unowned, and interrupt requests bypass admission so users can stop queued or over-cap chats. The single-chat API derives `queued_for_capacity` from live pool state; list endpoints do not report it. The UI polls that value every 5 seconds while a chat is running and shows a callout when the chat is waiting for capacity. Updates the administrator documentation and deployment-wide Prometheus gauges for active and queued agents. Replica-level values must be aggregated with `max`, not `sum`. > Mux updated this PR on Mike's behalf.
This commit is contained in:
@@ -273,6 +273,8 @@ type Options struct {
|
||||
// Set by enterprise for HA deployments. Nil uses chatd's local
|
||||
// in-process channel dialer.
|
||||
ChatStreamPartsDialer chatd.StreamPartsDialer
|
||||
// Nil keeps the default chat agent caps active.
|
||||
ChatAgentCapacityUnlock chatd.AgentCapacityUnlock
|
||||
// ChatProviderAPIKeys overrides deployment-derived provider keys.
|
||||
// Test harnesses use this to route chat models to local providers.
|
||||
ChatProviderAPIKeys *chatprovider.ProviderAPIKeys
|
||||
@@ -941,6 +943,7 @@ func New(options *Options) *API {
|
||||
HookDispatcher: hookDispatcher,
|
||||
UsageTracker: options.WorkspaceUsageTracker,
|
||||
PrometheusRegistry: options.PrometheusRegistry,
|
||||
AgentCapacityUnlock: options.ChatAgentCapacityUnlock,
|
||||
OIDCTokenSource: oidcMCPSrc,
|
||||
NotificationsEnqueuer: options.NotificationsEnqueuer,
|
||||
Auditor: &api.Auditor,
|
||||
|
||||
Reference in New Issue
Block a user