mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
feat: limit concurrent chat agents with pooled admission (#27902)
Limits concurrent chat generation on capped deployments to 5 root chats and 10 delegated subagent chats. The pools are deployment-wide and independent, so delegated work can continue while root capacity is full. The default caps live in AGPL code. Enterprise contributes only a licensing unlock, so unlicensed deployments stay capped and cannot fail open. Licensed deployments are uncapped while Agent Hours usage stays below an explicit hard limit. Deployments without a hard limit remain uncapped, and reaching the Agent Hours allocation only triggers warnings. Admission happens before a worker takes chat ownership. Capped deployments serialize admission across replicas with a transaction-scoped advisory lock and derive active and queued state from current ownership plus fresh runner heartbeats, rather than persisted queue markers or per-replica state. The acquisition query returns a bounded, pool-interleaved candidate set instead of ranking the whole backlog; a migration replaces the acquisition index with a pool-aware one. Refused chats stay running but unowned, and interrupt requests bypass admission so users can stop queued or over-cap chats. The single-chat API derives `queued_for_capacity` from live pool state; list endpoints do not report it. The UI polls that value every 5 seconds while a chat is running and shows a callout when the chat is waiting for capacity. Updates the administrator documentation and deployment-wide Prometheus gauges for active and queued agents. Replica-level values must be aggregated with `max`, not `sum`. > Mux updated this PR on Mike's behalf.
This commit is contained in:
@@ -221,6 +221,15 @@ token volume. Consider:
|
||||
- Capping spend with [AI Gateway budgets](./platform-controls/spend-management.md).
|
||||
- Monitoring provider dashboards for usage trends during the evaluation.
|
||||
|
||||
### Plan for concurrency limits
|
||||
|
||||
Community licenses run up to 5 agents at once.
|
||||
Additional agents queue and start automatically when capacity frees.
|
||||
Premium licenses with Agent Hours do not impose a concurrency limit unless the Agent Hours hard limit is reached.
|
||||
If the Agent Hours allocation is exhausted without a configured hard limit, Coder warns about usage but does not impose a concurrency limit.
|
||||
When the Agent Hours hard limit is reached, additional agents queue under the concurrency limit.
|
||||
Refer to [Concurrent agents](./platform-controls/index.md#concurrent-agents) for details.
|
||||
|
||||
### Pilot with a small group
|
||||
|
||||
Identify 3–5 developers and a few concrete use cases for the initial rollout.
|
||||
|
||||
@@ -112,6 +112,22 @@ This setting is available under **Agents** > **Settings** >
|
||||
days. When disabled, workspaces follow their template's autostop rules (or
|
||||
none, if the template does not define any).
|
||||
|
||||
### Concurrent agents
|
||||
|
||||
Community licenses support up to 5 concurrently active agents.
|
||||
Coder doesn't limit how long those agents can run or how many tasks they complete over time.
|
||||
Additional agents queue until an agent session becomes available.
|
||||
With concurrent agents, individuals and small teams can experiment with Coder Agents at no cost.
|
||||
|
||||
Queued agents show a banner in the chat and start automatically when capacity frees.
|
||||
Subtasks delegated by an agent don't count toward this limit.
|
||||
Those subtasks run in a separate pool of up to 10 concurrent subtasks.
|
||||
|
||||
Premium deployments can purchase Agent Hours with their Premium license.
|
||||
Agent Hours are shared across the deployment, and agents can run concurrently unless the Agent Hours hard limit is reached.
|
||||
If the Agent Hours allocation is exhausted without a configured hard limit, Coder warns about usage but does not impose a concurrency limit.
|
||||
When the Agent Hours hard limit is reached, additional agents queue under the concurrency limit.
|
||||
|
||||
### Spend management
|
||||
|
||||
AI Gateway budgets cap each user's AI spend, including Coder Agents chats, over a monthly period.
|
||||
|
||||
Reference in New Issue
Block a user