opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: limit concurrent chat agents with pooled admission (#27902)

Browse Source 
Limits concurrent chat generation on capped deployments to 5 root chats
and 10 delegated subagent chats. The pools are deployment-wide and
independent, so delegated work can continue while root capacity is full.

The default caps live in AGPL code. Enterprise contributes only a
licensing unlock, so unlicensed deployments stay capped and cannot fail
open. Licensed deployments are uncapped while Agent Hours usage stays
below an explicit hard limit. Deployments without a hard limit remain
uncapped, and reaching the Agent Hours allocation only triggers
warnings.

Admission happens before a worker takes chat ownership. Capped
deployments serialize admission across replicas with a
transaction-scoped advisory lock and derive active and queued state from
current ownership plus fresh runner heartbeats, rather than persisted
queue markers or per-replica state. The acquisition query returns a
bounded, pool-interleaved candidate set instead of ranking the whole
backlog; a migration replaces the acquisition index with a pool-aware
one. Refused chats stay running but unowned, and interrupt requests
bypass admission so users can stop queued or over-cap chats.

The single-chat API derives `queued_for_capacity` from live pool state;
list endpoints do not report it. The UI polls that value every 5 seconds
while a chat is running and shows a callout when the chat is waiting for
capacity.

Updates the administrator documentation and deployment-wide Prometheus
gauges for active and queued agents. Replica-level values must be
aggregated with `max`, not `sum`.

> Mux updated this PR on Mike's behalf.
This commit is contained in:
Michael Suchacz
2026-08-18 16:55:43 +02:00
committed by GitHub
parent cb0a9ebbbf
commit 119f2b1dd9
51 changed files with 2187 additions and 277 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reference/api/chats.md
Generated
+15
View File
@@ -127,6 +127,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -224,6 +225,7 @@ Status Code **200**
| `» parent_chat_id` | string(uuid) | false | | |
| `» pin_order` | integer | false | | |
| `» plan_mode` | [codersdk.ChatPlanMode](schemas.md#codersdkchatplanmode) | false | | |
| `» queued_for_capacity` | boolean | false | | Queued for capacity reports that the chat is waiting for a concurrent agent slot. Single-chat reads derive it; list responses leave it false. |
| `» root_chat_id` | string(uuid) | false | | |
| `» shared` | boolean | false | | Shared is true when this chat's root chat has explicit user or group ACL entries. |
| `» status` | [codersdk.ChatStatus](schemas.md#codersdkchatstatus) | false | | |
@@ -404,6 +406,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -498,6 +501,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -749,6 +753,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -897,6 +902,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -991,6 +997,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -1176,6 +1183,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -1270,6 +1278,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -1505,6 +1514,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -1599,6 +1609,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -2520,6 +2531,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -2614,6 +2626,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -3122,6 +3135,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -3216,6 +3230,7 @@ Experimental: this endpoint is subject to change.
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
docs/reference/api/schemas.md
Generated
+4
View File
@@ -2280,6 +2280,7 @@ AuthorizationObject can represent a "set" of objects, such as: all workspaces in
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -2374,6 +2375,7 @@ AuthorizationObject can represent a "set" of objects, such as: all workspaces in
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",
@@ -2416,6 +2418,7 @@ AuthorizationObject can represent a "set" of objects, such as: all workspaces in
| `parent_chat_id` | string | false | | |
| `pin_order` | integer | false | | |
| `plan_mode` | [codersdk.ChatPlanMode](#codersdkchatplanmode) | false | | |
| `queued_for_capacity` | boolean | false | | Queued for capacity reports that the chat is waiting for a concurrent agent slot. Single-chat reads derive it; list responses leave it false. |
| `root_chat_id` | string | false | | |
| `shared` | boolean | false | | Shared is true when this chat's root chat has explicit user or group ACL entries. |
| `status` | [codersdk.ChatStatus](#codersdkchatstatus) | false | | |
@@ -4203,6 +4206,7 @@ AuthorizationObject can represent a "set" of objects, such as: all workspaces in
"parent_chat_id": "c3609ee6-3b11-4a93-b9ae-e4fabcc99359",
"pin_order": 0,
"plan_mode": "plan",
"queued_for_capacity": true,
"root_chat_id": "2898031c-fdce-4e3e-8c53-4481dd42fcd7",
"shared": true,
"status": "waiting",