opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: limit concurrent chat agents with pooled admission (#27902)

Browse Source 
Limits concurrent chat generation on capped deployments to 5 root chats
and 10 delegated subagent chats. The pools are deployment-wide and
independent, so delegated work can continue while root capacity is full.

The default caps live in AGPL code. Enterprise contributes only a
licensing unlock, so unlicensed deployments stay capped and cannot fail
open. Licensed deployments are uncapped while Agent Hours usage stays
below an explicit hard limit. Deployments without a hard limit remain
uncapped, and reaching the Agent Hours allocation only triggers
warnings.

Admission happens before a worker takes chat ownership. Capped
deployments serialize admission across replicas with a
transaction-scoped advisory lock and derive active and queued state from
current ownership plus fresh runner heartbeats, rather than persisted
queue markers or per-replica state. The acquisition query returns a
bounded, pool-interleaved candidate set instead of ranking the whole
backlog; a migration replaces the acquisition index with a pool-aware
one. Refused chats stay running but unowned, and interrupt requests
bypass admission so users can stop queued or over-cap chats.

The single-chat API derives `queued_for_capacity` from live pool state;
list endpoints do not report it. The UI polls that value every 5 seconds
while a chat is running and shows a callout when the chat is waiting for
capacity.

Updates the administrator documentation and deployment-wide Prometheus
gauges for active and queued agents. Replica-level values must be
aggregated with `max`, not `sum`.

> Mux updated this PR on Mike's behalf.
This commit is contained in:
Michael Suchacz
2026-08-18 16:55:43 +02:00
committed by GitHub
parent cb0a9ebbbf
commit 119f2b1dd9
51 changed files with 2187 additions and 277 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/admin/integrations/prometheus.md
+2
View File
@@ -236,6 +236,8 @@ The `coder_ai_gateway_cost_control_*` metrics are exported only by `coderd`.
| `coderd_authz_prepare_authorize_duration_seconds` | histogram | Duration of the 'PrepareAuthorize' call in seconds. | |
| `coderd_build_info` | gauge | Describes the current build/version of the Coder server. Value is always 1. | `revision` `version` |
| `coderd_chat_auto_archive_records_archived_total` | counter | Total number of chats archived by the auto-archive job (counting both roots and cascaded children). | |
| `coderd_chatd_agents_active` | gauge | Deployment-wide number of chats holding a concurrent-agent capacity slot. Every replica reports the same database-derived value; aggregate with max, not sum. | `pool` |
| `coderd_chatd_agents_queued_for_capacity` | gauge | Deployment-wide number of chats waiting for a concurrent-agent capacity slot. Every replica reports the same database-derived value; aggregate with max, not sum. | `pool` |
| `coderd_chatd_chats` | gauge | Number of chats being processed, by state. | `state` |
| `coderd_chatd_compaction_total` | counter | Total compaction outcomes (only recorded when compaction was triggered or failed). | `model` `provider` `result` |
| `coderd_chatd_hook_context_size_bytes` | histogram | Lifecycle hook model context response size in bytes. | `event` |
docs/ai-coder/agents/getting-started.md
+9
View File
@@ -221,6 +221,15 @@ token volume. Consider:
- Capping spend with [AI Gateway budgets](./platform-controls/spend-management.md).
- Monitoring provider dashboards for usage trends during the evaluation.
### Plan for concurrency limits
Community licenses run up to 5 agents at once.
Additional agents queue and start automatically when capacity frees.
Premium licenses with Agent Hours do not impose a concurrency limit unless the Agent Hours hard limit is reached.
If the Agent Hours allocation is exhausted without a configured hard limit, Coder warns about usage but does not impose a concurrency limit.
When the Agent Hours hard limit is reached, additional agents queue under the concurrency limit.
Refer to [Concurrent agents](./platform-controls/index.md#concurrent-agents) for details.
### Pilot with a small group
Identify 3–5 developers and a few concrete use cases for the initial rollout.
docs/ai-coder/agents/platform-controls/index.md
+16
View File
@@ -112,6 +112,22 @@ This setting is available under **Agents** > **Settings** >
days. When disabled, workspaces follow their template's autostop rules (or
none, if the template does not define any).
### Concurrent agents
Community licenses support up to 5 concurrently active agents.
Coder doesn't limit how long those agents can run or how many tasks they complete over time.
Additional agents queue until an agent session becomes available.
With concurrent agents, individuals and small teams can experiment with Coder Agents at no cost.
Queued agents show a banner in the chat and start automatically when capacity frees.
Subtasks delegated by an agent don't count toward this limit.
Those subtasks run in a separate pool of up to 10 concurrent subtasks.
Premium deployments can purchase Agent Hours with their Premium license.
Agent Hours are shared across the deployment, and agents can run concurrently unless the Agent Hours hard limit is reached.
If the Agent Hours allocation is exhausted without a configured hard limit, Coder warns about usage but does not impose a concurrency limit.
When the Agent Hours hard limit is reached, additional agents queue under the concurrency limit.
### Spend management
AI Gateway budgets cap each user's AI spend, including Coder Agents chats, over a monthly period.
docs/reference/api/chats.md
Generated
+15
View File
@@ -127,6 +127,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -224,6 +225,7 @@ Status Code **200**
| `» parent_chat_id` | string(uuid) | false | | |
| `» pin_order` | integer | false | | |
| `» plan_mode` | [codersdk.ChatPlanMode](schemas.md#codersdkchatplanmode) | false | | |
| `» queued_for_capacity` | boolean | false | | Queued for capacity reports that the chat is waiting for a concurrent agent slot. Single-chat reads derive it; list responses leave it false. |
| `» root_chat_id` | string(uuid) | false | | |
| `» shared` | boolean | false | | Shared is true when this chat's root chat has explicit user or group ACL entries. |
| `» status` | [codersdk.ChatStatus](schemas.md#codersdkchatstatus) | false | | |
@@ -404,6 +406,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -498,6 +501,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -749,6 +753,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -897,6 +902,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -991,6 +997,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -1176,6 +1183,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -1270,6 +1278,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -1505,6 +1514,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -1599,6 +1609,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -2520,6 +2531,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -2614,6 +2626,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -3122,6 +3135,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -3216,6 +3230,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
docs/reference/api/schemas.md
Generated
+4
View File
@@ -2280,6 +2280,7 @@ AuthorizationObject can represent a "set" of objects, such as: all workspaces in
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -2374,6 +2375,7 @@ AuthorizationObject can represent a "set" of objects, such as: all workspaces in
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -2416,6 +2418,7 @@ AuthorizationObject can represent a "set" of objects, such as: all workspaces in
| `parent_chat_id` | string | false | | |
| `pin_order` | integer | false | | |
| `plan_mode` | [codersdk.ChatPlanMode](#codersdkchatplanmode) | false | | |
| `queued_for_capacity` | boolean | false | | Queued for capacity reports that the chat is waiting for a concurrent agent slot. Single-chat reads derive it; list responses leave it false. |
| `root_chat_id` | string | false | | |
| `shared` | boolean | false | | Shared is true when this chat's root chat has explicit user or group ACL entries. |
| `status` | [codersdk.ChatStatus](#codersdkchatstatus) | false | | |
@@ -4203,6 +4206,7 @@ AuthorizationObject can represent a "set" of objects, such as: all workspaces in
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",