opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: limit concurrent chat agents with pooled admission (#27902)

Browse Source 
Limits concurrent chat generation on capped deployments to 5 root chats
and 10 delegated subagent chats. The pools are deployment-wide and
independent, so delegated work can continue while root capacity is full.

The default caps live in AGPL code. Enterprise contributes only a
licensing unlock, so unlicensed deployments stay capped and cannot fail
open. Licensed deployments are uncapped while Agent Hours usage stays
below an explicit hard limit. Deployments without a hard limit remain
uncapped, and reaching the Agent Hours allocation only triggers
warnings.

Admission happens before a worker takes chat ownership. Capped
deployments serialize admission across replicas with a
transaction-scoped advisory lock and derive active and queued state from
current ownership plus fresh runner heartbeats, rather than persisted
queue markers or per-replica state. The acquisition query returns a
bounded, pool-interleaved candidate set instead of ranking the whole
backlog; a migration replaces the acquisition index with a pool-aware
one. Refused chats stay running but unowned, and interrupt requests
bypass admission so users can stop queued or over-cap chats.

The single-chat API derives `queued_for_capacity` from live pool state;
list endpoints do not report it. The UI polls that value every 5 seconds
while a chat is running and shows a callout when the chat is waiting for
capacity.

Updates the administrator documentation and deployment-wide Prometheus
gauges for active and queued agents. Replica-level values must be
aggregated with `max`, not `sum`.

> Mux updated this PR on Mike's behalf.
This commit is contained in:
Michael Suchacz
2026-08-18 16:55:43 +02:00
committed by GitHub
parent cb0a9ebbbf
commit 119f2b1dd9
51 changed files with 2187 additions and 277 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/admin/integrations/prometheus.md
+2
View File
@@ -236,6 +236,8 @@ The `coder_ai_gateway_cost_control_*` metrics are exported only by `coderd`.
| `coderd_authz_prepare_authorize_duration_seconds` | histogram | Duration of the 'PrepareAuthorize' call in seconds. | |
| `coderd_build_info` | gauge | Describes the current build/version of the Coder server. Value is always 1. | `revision` `version` |
| `coderd_chat_auto_archive_records_archived_total` | counter | Total number of chats archived by the auto-archive job (counting both roots and cascaded children). | |
| `coderd_chatd_agents_active` | gauge | Deployment-wide number of chats holding a concurrent-agent capacity slot. Every replica reports the same database-derived value; aggregate with max, not sum. | `pool` |
| `coderd_chatd_agents_queued_for_capacity` | gauge | Deployment-wide number of chats waiting for a concurrent-agent capacity slot. Every replica reports the same database-derived value; aggregate with max, not sum. | `pool` |
| `coderd_chatd_chats` | gauge | Number of chats being processed, by state. | `state` |
| `coderd_chatd_compaction_total` | counter | Total compaction outcomes (only recorded when compaction was triggered or failed). | `model` `provider` `result` |
| `coderd_chatd_hook_context_size_bytes` | histogram | Lifecycle hook model context response size in bytes. | `event` |