mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
Kyle Carberry
32217259b7
## Problem
Local tool results were persisted and replayed to the model verbatim,
with no size cap. A single oversized result, most often a multi-megabyte
response from an MCP tool, overflows the prompt on the next request.
Every retry rebuilds the same history and fails the same way, leaving
the chat wedged in `error`. Auto-compaction is reactive (token usage is
only known after a response), so it can't catch a single result that
blows the very next request.
## Fix
Cap every locally-executed tool result at its single choke point,
`executeSingleTool` in `chatloop`, so the cap covers built-in tools,
**global (deployment-pinned) MCP**, **workspace MCP**, and provider
runners uniformly. Because this runs before the result is published to
the live stream and before it is committed, the SSE preview, the
persisted message, and the model replay all see the same bounded output.
The budget is token-aware: a single tool result may use at most half the
model's context window (`~4 bytes/token`), with a `16KB` floor and a
`64KB` default when the window is unknown. Truncation keeps the head and
tail of the output and replaces the middle with a marker telling the
model how much was removed and to narrow its query; it is UTF-8 safe and
never exceeds the budget. Binary media `Data` is passed through
untouched (only the text payload is bounded).
A `coderd_chatd_tool_result_truncated_total{provider,model,tool_name}`
counter and a warning log record each truncation.
## Out of scope
- Provider-executed results (e.g. web search) arrive via the stream, not
`executeSingleTool`.
- Dynamic/external tool results submitted through the `/tool-results`
API are validated as JSON elsewhere.
- Cumulative growth across many results is still handled by context
compaction; this change only bounds any single result.
<details>
<summary>Implementation notes</summary>
- New `coderd/x/chatd/chatloop/tooltruncate.go`:
`toolResultByteBudget(contextLimitTokens)` and
`truncateToolResultText(text, maxBytes)` (pure, unit-tested).
- `chatloop.go`: added `ContextLimit` to `ExecuteLocalToolsOptions`;
threaded a computed byte budget through `executeTools` into
`executeSingleTool`, where `resp.Content` is capped for the text,
media-text, and error branches.
- `generation.go`: passes `ContextLimit: prepared.ContextLimitFallback`
(the model's configured context limit).
- `metrics.go`: new `ToolResultTruncatedTotal` counter +
`RecordToolResultTruncated`.
- Tunable knobs live as constants in `tooltruncate.go`
(`toolResultContextDivisor = 2`, `bytesPerTokenEstimate`,
`minToolResultBytes`, `defaultToolResultBytes`).
Verified: `go build ./coderd/x/chatd/...`, `go test
./coderd/x/chatd/chatloop/...`, and the `chatd` test binary compiles.
</details>
---
Resolves CODAGT-678
Generated by Coder Agents on behalf of @kylecarbs.
502 lines
36 KiB
Plaintext
502 lines
36 KiB
Plaintext
# HELP agent_boundary_log_proxy_batches_dropped_total Total number of boundary log batches dropped before reaching coderd. Reason: buffer_full = the agent's internal buffer is full, meaning boundary is producing logs faster than the agent can forward them to coderd; forward_failed = the agent failed to send the batch to coderd, potentially because coderd is unreachable or the connection was interrupted.
|
|
# TYPE agent_boundary_log_proxy_batches_dropped_total counter
|
|
agent_boundary_log_proxy_batches_dropped_total{reason=""} 0
|
|
# HELP agent_boundary_log_proxy_batches_forwarded_total Total number of boundary log batches successfully forwarded to coderd. Compare with batches_dropped_total to compute a drop rate.
|
|
# TYPE agent_boundary_log_proxy_batches_forwarded_total counter
|
|
agent_boundary_log_proxy_batches_forwarded_total 0
|
|
# HELP agent_boundary_log_proxy_logs_dropped_total Total number of individual boundary log entries dropped before reaching coderd. Reason: buffer_full = the agent's internal buffer is full; forward_failed = the agent failed to send the batch to coderd; boundary_channel_full = boundary's internal send channel overflowed, meaning boundary is generating logs faster than it can batch and send them; boundary_batch_full = boundary's outgoing batch buffer overflowed after a failed flush, meaning boundary could not write to the agent's socket.
|
|
# TYPE agent_boundary_log_proxy_logs_dropped_total counter
|
|
agent_boundary_log_proxy_logs_dropped_total{reason=""} 0
|
|
# HELP coder_derp_server_accepts_total Total DERP connections accepted.
|
|
# TYPE coder_derp_server_accepts_total counter
|
|
coder_derp_server_accepts_total 0
|
|
# HELP coder_derp_server_average_queue_duration_ms Average queue duration in milliseconds.
|
|
# TYPE coder_derp_server_average_queue_duration_ms gauge
|
|
coder_derp_server_average_queue_duration_ms 0
|
|
# HELP coder_derp_server_bytes_received_total Total bytes received.
|
|
# TYPE coder_derp_server_bytes_received_total counter
|
|
coder_derp_server_bytes_received_total 0
|
|
# HELP coder_derp_server_bytes_sent_total Total bytes sent.
|
|
# TYPE coder_derp_server_bytes_sent_total counter
|
|
coder_derp_server_bytes_sent_total 0
|
|
# HELP coder_derp_server_clients Total clients (local + remote).
|
|
# TYPE coder_derp_server_clients gauge
|
|
coder_derp_server_clients 0
|
|
# HELP coder_derp_server_clients_local Local clients.
|
|
# TYPE coder_derp_server_clients_local gauge
|
|
coder_derp_server_clients_local 0
|
|
# HELP coder_derp_server_clients_remote Remote (mesh) clients.
|
|
# TYPE coder_derp_server_clients_remote gauge
|
|
coder_derp_server_clients_remote 0
|
|
# HELP coder_derp_server_connections Current DERP connections.
|
|
# TYPE coder_derp_server_connections gauge
|
|
coder_derp_server_connections 0
|
|
# HELP coder_derp_server_got_ping_total Total pings received.
|
|
# TYPE coder_derp_server_got_ping_total counter
|
|
coder_derp_server_got_ping_total 0
|
|
# HELP coder_derp_server_home_connections Current home DERP connections.
|
|
# TYPE coder_derp_server_home_connections gauge
|
|
coder_derp_server_home_connections 0
|
|
# HELP coder_derp_server_home_moves_in_total Total home moves in.
|
|
# TYPE coder_derp_server_home_moves_in_total counter
|
|
coder_derp_server_home_moves_in_total 0
|
|
# HELP coder_derp_server_home_moves_out_total Total home moves out.
|
|
# TYPE coder_derp_server_home_moves_out_total counter
|
|
coder_derp_server_home_moves_out_total 0
|
|
# HELP coder_derp_server_packets_dropped_reason_total Packets dropped by reason.
|
|
# TYPE coder_derp_server_packets_dropped_reason_total counter
|
|
coder_derp_server_packets_dropped_reason_total{reason=""} 0
|
|
# HELP coder_derp_server_packets_dropped_total Total packets dropped.
|
|
# TYPE coder_derp_server_packets_dropped_total counter
|
|
coder_derp_server_packets_dropped_total 0
|
|
# HELP coder_derp_server_packets_dropped_type_total Packets dropped by type.
|
|
# TYPE coder_derp_server_packets_dropped_type_total counter
|
|
coder_derp_server_packets_dropped_type_total{type=""} 0
|
|
# HELP coder_derp_server_packets_forwarded_in_total Total packets forwarded in from mesh peers.
|
|
# TYPE coder_derp_server_packets_forwarded_in_total counter
|
|
coder_derp_server_packets_forwarded_in_total 0
|
|
# HELP coder_derp_server_packets_forwarded_out_total Total packets forwarded out to mesh peers.
|
|
# TYPE coder_derp_server_packets_forwarded_out_total counter
|
|
coder_derp_server_packets_forwarded_out_total 0
|
|
# HELP coder_derp_server_packets_received_kind_total Packets received by kind.
|
|
# TYPE coder_derp_server_packets_received_kind_total counter
|
|
coder_derp_server_packets_received_kind_total{kind=""} 0
|
|
# HELP coder_derp_server_packets_received_total Total packets received.
|
|
# TYPE coder_derp_server_packets_received_total counter
|
|
coder_derp_server_packets_received_total 0
|
|
# HELP coder_derp_server_packets_sent_total Total packets sent.
|
|
# TYPE coder_derp_server_packets_sent_total counter
|
|
coder_derp_server_packets_sent_total 0
|
|
# HELP coder_derp_server_peer_gone_disconnected_total Total peer gone (disconnected) frames sent.
|
|
# TYPE coder_derp_server_peer_gone_disconnected_total counter
|
|
coder_derp_server_peer_gone_disconnected_total 0
|
|
# HELP coder_derp_server_peer_gone_not_here_total Total peer gone (not here) frames sent.
|
|
# TYPE coder_derp_server_peer_gone_not_here_total counter
|
|
coder_derp_server_peer_gone_not_here_total 0
|
|
# HELP coder_derp_server_sent_pong_total Total pongs sent.
|
|
# TYPE coder_derp_server_sent_pong_total counter
|
|
coder_derp_server_sent_pong_total 0
|
|
# HELP coder_derp_server_unknown_frames_total Total unknown frames received.
|
|
# TYPE coder_derp_server_unknown_frames_total counter
|
|
coder_derp_server_unknown_frames_total 0
|
|
# HELP coder_derp_server_watchers Current watchers.
|
|
# TYPE coder_derp_server_watchers gauge
|
|
coder_derp_server_watchers 0
|
|
# HELP coder_nats_pubsub_connected Whether we are connected (1) or not connected (0) to nats
|
|
# TYPE coder_nats_pubsub_connected gauge
|
|
coder_nats_pubsub_connected 0
|
|
# HELP coder_nats_pubsub_current_events The current number of pubsub event channels listened for
|
|
# TYPE coder_nats_pubsub_current_events gauge
|
|
coder_nats_pubsub_current_events 0
|
|
# HELP coder_nats_pubsub_current_subscribers The current number of active pubsub subscribers
|
|
# TYPE coder_nats_pubsub_current_subscribers gauge
|
|
coder_nats_pubsub_current_subscribers 0
|
|
# HELP coder_nats_pubsub_disconnections_total Total number of times we disconnected unexpectedly from nats
|
|
# TYPE coder_nats_pubsub_disconnections_total counter
|
|
coder_nats_pubsub_disconnections_total 0
|
|
# HELP coder_nats_pubsub_latency_measure_errs_total The number of pubsub latency measurement failures
|
|
# TYPE coder_nats_pubsub_latency_measure_errs_total counter
|
|
coder_nats_pubsub_latency_measure_errs_total 0
|
|
# HELP coder_nats_pubsub_latency_measures_total The number of pubsub latency measurements
|
|
# TYPE coder_nats_pubsub_latency_measures_total counter
|
|
coder_nats_pubsub_latency_measures_total 0
|
|
# HELP coder_nats_pubsub_messages_total Total number of messages received from nats
|
|
# TYPE coder_nats_pubsub_messages_total counter
|
|
coder_nats_pubsub_messages_total{size=""} 0
|
|
# HELP coder_nats_pubsub_published_bytes_total Total number of bytes successfully published across all publishes
|
|
# TYPE coder_nats_pubsub_published_bytes_total counter
|
|
coder_nats_pubsub_published_bytes_total 0
|
|
# HELP coder_nats_pubsub_publishes_total Total number of calls to Publish
|
|
# TYPE coder_nats_pubsub_publishes_total counter
|
|
coder_nats_pubsub_publishes_total{success=""} 0
|
|
# HELP coder_nats_pubsub_receive_latency_seconds The time taken to receive a message from a pubsub event channel
|
|
# TYPE coder_nats_pubsub_receive_latency_seconds gauge
|
|
coder_nats_pubsub_receive_latency_seconds 0
|
|
# HELP coder_nats_pubsub_received_bytes_total Total number of bytes received across all messages
|
|
# TYPE coder_nats_pubsub_received_bytes_total counter
|
|
coder_nats_pubsub_received_bytes_total 0
|
|
# HELP coder_nats_pubsub_send_latency_seconds The time taken to send a message into a pubsub event channel
|
|
# TYPE coder_nats_pubsub_send_latency_seconds gauge
|
|
coder_nats_pubsub_send_latency_seconds 0
|
|
# HELP coder_nats_pubsub_subscribes_total Total number of calls to Subscribe/SubscribeWithErr
|
|
# TYPE coder_nats_pubsub_subscribes_total counter
|
|
coder_nats_pubsub_subscribes_total{success=""} 0
|
|
# HELP coder_pubsub_connected Whether we are connected (1) or not connected (0) to postgres
|
|
# TYPE coder_pubsub_connected gauge
|
|
coder_pubsub_connected 0
|
|
# HELP coder_pubsub_current_events The current number of pubsub event channels listened for
|
|
# TYPE coder_pubsub_current_events gauge
|
|
coder_pubsub_current_events 0
|
|
# HELP coder_pubsub_current_subscribers The current number of active pubsub subscribers
|
|
# TYPE coder_pubsub_current_subscribers gauge
|
|
coder_pubsub_current_subscribers 0
|
|
# HELP coder_pubsub_disconnections_total Total number of times we disconnected unexpectedly from postgres
|
|
# TYPE coder_pubsub_disconnections_total counter
|
|
coder_pubsub_disconnections_total 0
|
|
# HELP coder_pubsub_latency_measure_errs_total The number of pubsub latency measurement failures
|
|
# TYPE coder_pubsub_latency_measure_errs_total counter
|
|
coder_pubsub_latency_measure_errs_total 0
|
|
# HELP coder_pubsub_latency_measures_total The number of pubsub latency measurements
|
|
# TYPE coder_pubsub_latency_measures_total counter
|
|
coder_pubsub_latency_measures_total 0
|
|
# HELP coder_pubsub_messages_total Total number of messages received from postgres
|
|
# TYPE coder_pubsub_messages_total counter
|
|
coder_pubsub_messages_total{size=""} 0
|
|
# HELP coder_pubsub_published_bytes_total Total number of bytes successfully published across all publishes
|
|
# TYPE coder_pubsub_published_bytes_total counter
|
|
coder_pubsub_published_bytes_total 0
|
|
# HELP coder_pubsub_publishes_total Total number of calls to Publish
|
|
# TYPE coder_pubsub_publishes_total counter
|
|
coder_pubsub_publishes_total{success=""} 0
|
|
# HELP coder_pubsub_receive_latency_seconds The time taken to receive a message from a pubsub event channel
|
|
# TYPE coder_pubsub_receive_latency_seconds gauge
|
|
coder_pubsub_receive_latency_seconds 0
|
|
# HELP coder_pubsub_received_bytes_total Total number of bytes received across all messages
|
|
# TYPE coder_pubsub_received_bytes_total counter
|
|
coder_pubsub_received_bytes_total 0
|
|
# HELP coder_pubsub_send_latency_seconds The time taken to send a message into a pubsub event channel
|
|
# TYPE coder_pubsub_send_latency_seconds gauge
|
|
coder_pubsub_send_latency_seconds 0
|
|
# HELP coder_pubsub_subscribes_total Total number of calls to Subscribe/SubscribeWithErr
|
|
# TYPE coder_pubsub_subscribes_total counter
|
|
coder_pubsub_subscribes_total{success=""} 0
|
|
# HELP coder_servertailnet_connections_total Total number of TCP connections made to workspace agents.
|
|
# TYPE coder_servertailnet_connections_total counter
|
|
coder_servertailnet_connections_total{network=""} 0
|
|
# HELP coder_servertailnet_open_connections Total number of TCP connections currently open to workspace agents.
|
|
# TYPE coder_servertailnet_open_connections gauge
|
|
coder_servertailnet_open_connections{network=""} 0
|
|
# HELP coderd_agentapi_metadata_batch_size Total number of metadata entries in each batch, updated before flushes.
|
|
# TYPE coderd_agentapi_metadata_batch_size histogram
|
|
coderd_agentapi_metadata_batch_size 0
|
|
# HELP coderd_agentapi_metadata_batch_utilization Number of metadata keys per agent in each batch, updated before flushes.
|
|
# TYPE coderd_agentapi_metadata_batch_utilization histogram
|
|
coderd_agentapi_metadata_batch_utilization 0
|
|
# HELP coderd_agentapi_metadata_batches_total Total number of metadata batches flushed.
|
|
# TYPE coderd_agentapi_metadata_batches_total counter
|
|
coderd_agentapi_metadata_batches_total{reason=""} 0
|
|
# HELP coderd_agentapi_metadata_dropped_keys_total Total number of metadata keys dropped due to capacity limits.
|
|
# TYPE coderd_agentapi_metadata_dropped_keys_total counter
|
|
coderd_agentapi_metadata_dropped_keys_total 0
|
|
# HELP coderd_agentapi_metadata_flush_duration_seconds Time taken to flush metadata batch to database and pubsub.
|
|
# TYPE coderd_agentapi_metadata_flush_duration_seconds histogram
|
|
coderd_agentapi_metadata_flush_duration_seconds{reason=""} 0
|
|
# HELP coderd_agentapi_metadata_flushed_total Total number of unique metadatas flushed.
|
|
# TYPE coderd_agentapi_metadata_flushed_total counter
|
|
coderd_agentapi_metadata_flushed_total 0
|
|
# HELP coderd_agentapi_metadata_publish_errors_total Total number of metadata batch pubsub publish calls that have resulted in an error.
|
|
# TYPE coderd_agentapi_metadata_publish_errors_total counter
|
|
coderd_agentapi_metadata_publish_errors_total 0
|
|
# HELP coderd_agents_apps Agent applications with statuses.
|
|
# TYPE coderd_agents_apps gauge
|
|
coderd_agents_apps{agent_name="",username="",workspace_name="",app_name="",health=""} 0
|
|
# HELP coderd_agents_connection_latencies_seconds Agent connection latencies in seconds.
|
|
# TYPE coderd_agents_connection_latencies_seconds gauge
|
|
coderd_agents_connection_latencies_seconds{agent_name="",username="",workspace_name="",derp_region="",preferred=""} 0
|
|
# HELP coderd_agents_connections Agent connections with statuses.
|
|
# TYPE coderd_agents_connections gauge
|
|
coderd_agents_connections{agent_name="",username="",workspace_name="",status="",lifecycle_state="",tailnet_node=""} 0
|
|
# HELP coderd_agents_first_connection_seconds Duration from agent creation to first connection in seconds.
|
|
# TYPE coderd_agents_first_connection_seconds histogram
|
|
coderd_agents_first_connection_seconds{template_name="",agent_name=""} 0
|
|
# HELP coderd_agents_up The number of active agents per workspace.
|
|
# TYPE coderd_agents_up gauge
|
|
coderd_agents_up{username="",workspace_name="",template_name="",template_version=""} 0
|
|
# HELP coderd_agentstats_connection_count The number of established connections by agent
|
|
# TYPE coderd_agentstats_connection_count gauge
|
|
coderd_agentstats_connection_count 0
|
|
# HELP coderd_agentstats_connection_median_latency_seconds The median agent connection latency in seconds
|
|
# TYPE coderd_agentstats_connection_median_latency_seconds gauge
|
|
coderd_agentstats_connection_median_latency_seconds 0
|
|
# HELP coderd_agentstats_currently_reachable_peers The number of peers (e.g. clients) that are currently reachable over the encrypted network.
|
|
# TYPE coderd_agentstats_currently_reachable_peers gauge
|
|
coderd_agentstats_currently_reachable_peers{connection_type=""} 0
|
|
# HELP coderd_agentstats_rx_bytes Agent Rx bytes
|
|
# TYPE coderd_agentstats_rx_bytes gauge
|
|
coderd_agentstats_rx_bytes 0
|
|
# HELP coderd_agentstats_session_count_jetbrains The number of session established by JetBrains
|
|
# TYPE coderd_agentstats_session_count_jetbrains gauge
|
|
coderd_agentstats_session_count_jetbrains 0
|
|
# HELP coderd_agentstats_session_count_reconnecting_pty The number of session established by reconnecting PTY
|
|
# TYPE coderd_agentstats_session_count_reconnecting_pty gauge
|
|
coderd_agentstats_session_count_reconnecting_pty 0
|
|
# HELP coderd_agentstats_session_count_ssh The number of session established by SSH
|
|
# TYPE coderd_agentstats_session_count_ssh gauge
|
|
coderd_agentstats_session_count_ssh 0
|
|
# HELP coderd_agentstats_session_count_vscode The number of session established by VSCode
|
|
# TYPE coderd_agentstats_session_count_vscode gauge
|
|
coderd_agentstats_session_count_vscode 0
|
|
# HELP coderd_agentstats_startup_script_seconds Amount of time taken to run the startup script in seconds.
|
|
# TYPE coderd_agentstats_startup_script_seconds gauge
|
|
coderd_agentstats_startup_script_seconds{success=""} 0
|
|
# HELP coderd_agentstats_tx_bytes Agent Tx bytes
|
|
# TYPE coderd_agentstats_tx_bytes gauge
|
|
coderd_agentstats_tx_bytes 0
|
|
# HELP coderd_api_active_users_duration_hour The number of users that have been active within the last hour.
|
|
# TYPE coderd_api_active_users_duration_hour gauge
|
|
coderd_api_active_users_duration_hour 0
|
|
# HELP coderd_api_concurrent_requests The number of concurrent API requests.
|
|
# TYPE coderd_api_concurrent_requests gauge
|
|
coderd_api_concurrent_requests{method="",path=""} 0
|
|
# HELP coderd_api_concurrent_websockets The total number of concurrent API websockets.
|
|
# TYPE coderd_api_concurrent_websockets gauge
|
|
coderd_api_concurrent_websockets{path=""} 0
|
|
# HELP coderd_api_request_latencies_seconds Latency distribution of requests in seconds.
|
|
# TYPE coderd_api_request_latencies_seconds histogram
|
|
coderd_api_request_latencies_seconds{method="",path=""} 0
|
|
# HELP coderd_api_requests_processed_total The total number of processed API requests
|
|
# TYPE coderd_api_requests_processed_total counter
|
|
coderd_api_requests_processed_total{code="",method="",path=""} 0
|
|
# HELP coderd_api_total_user_count The total number of registered users, partitioned by status.
|
|
# TYPE coderd_api_total_user_count gauge
|
|
coderd_api_total_user_count{status=""} 0
|
|
# HELP coderd_api_websocket_durations_seconds Websocket duration distribution of requests in seconds.
|
|
# TYPE coderd_api_websocket_durations_seconds histogram
|
|
coderd_api_websocket_durations_seconds{path=""} 0
|
|
# HELP coderd_api_websocket_probes_total WebSocket liveness probe outcomes by route. Compare rate(...{result=\"ok\"}[1m]) against coderd_api_concurrent_websockets to detect unresponsive WebSocket connections.
|
|
# TYPE coderd_api_websocket_probes_total counter
|
|
coderd_api_websocket_probes_total{path="",result=""} 0
|
|
# HELP coderd_api_workspace_latest_build The current number of workspace builds by status for all non-deleted workspaces.
|
|
# TYPE coderd_api_workspace_latest_build gauge
|
|
coderd_api_workspace_latest_build{status=""} 0
|
|
# HELP coderd_authz_authorize_duration_seconds Duration of the 'Authorize' call in seconds. Only counts calls that succeed.
|
|
# TYPE coderd_authz_authorize_duration_seconds histogram
|
|
coderd_authz_authorize_duration_seconds{allowed=""} 0
|
|
# HELP coderd_authz_prepare_authorize_duration_seconds Duration of the 'PrepareAuthorize' call in seconds.
|
|
# TYPE coderd_authz_prepare_authorize_duration_seconds histogram
|
|
coderd_authz_prepare_authorize_duration_seconds 0
|
|
# HELP coderd_build_info Describes the current build/version of the Coder server. Value is always 1.
|
|
# TYPE coderd_build_info gauge
|
|
coderd_build_info{version="",revision=""} 0
|
|
# HELP coderd_chat_auto_archive_records_archived_total Total number of chats archived by the auto-archive job (counting both roots and cascaded children).
|
|
# TYPE coderd_chat_auto_archive_records_archived_total counter
|
|
coderd_chat_auto_archive_records_archived_total 0
|
|
# HELP coderd_chatd_chats Number of chats being processed, by state.
|
|
# TYPE coderd_chatd_chats gauge
|
|
coderd_chatd_chats{state=""} 0
|
|
# HELP coderd_chatd_compaction_total Total compaction outcomes (only recorded when compaction was triggered or failed).
|
|
# TYPE coderd_chatd_compaction_total counter
|
|
coderd_chatd_compaction_total{provider="",model="",result=""} 0
|
|
# HELP coderd_chatd_message_count Number of messages in the prompt per LLM request.
|
|
# TYPE coderd_chatd_message_count histogram
|
|
coderd_chatd_message_count{provider="",model=""} 0
|
|
# HELP coderd_chatd_prompt_size_bytes Estimated byte size of the prompt per LLM request.
|
|
# TYPE coderd_chatd_prompt_size_bytes histogram
|
|
coderd_chatd_prompt_size_bytes{provider="",model=""} 0
|
|
# HELP coderd_chatd_steps_total Total agentic loop steps across all chats.
|
|
# TYPE coderd_chatd_steps_total counter
|
|
coderd_chatd_steps_total{provider="",model=""} 0
|
|
# HELP coderd_chatd_stream_buffer_dropped_total Number of chat stream buffer events dropped due to the per-chat buffer cap.
|
|
# TYPE coderd_chatd_stream_buffer_dropped_total counter
|
|
coderd_chatd_stream_buffer_dropped_total 0
|
|
# HELP coderd_chatd_stream_retries_total Total LLM stream retries.
|
|
# TYPE coderd_chatd_stream_retries_total counter
|
|
coderd_chatd_stream_retries_total{provider="",model="",kind="",chain_broken=""} 0
|
|
# HELP coderd_chatd_tool_errors_total Total tool calls that returned an error result.
|
|
# TYPE coderd_chatd_tool_errors_total counter
|
|
coderd_chatd_tool_errors_total{provider="",model="",tool_name=""} 0
|
|
# HELP coderd_chatd_tool_result_size_bytes Size in bytes of each tool execution result.
|
|
# TYPE coderd_chatd_tool_result_size_bytes histogram
|
|
coderd_chatd_tool_result_size_bytes{provider="",model="",tool_name=""} 0
|
|
# HELP coderd_chatd_tool_result_truncated_total Total tool results truncated to fit the model context window.
|
|
# TYPE coderd_chatd_tool_result_truncated_total counter
|
|
coderd_chatd_tool_result_truncated_total{provider="",model="",tool_name=""} 0
|
|
# HELP coderd_chatd_ttft_seconds Time-to-first-token: wall time from LLM request to first streamed chunk.
|
|
# TYPE coderd_chatd_ttft_seconds histogram
|
|
coderd_chatd_ttft_seconds{provider="",model=""} 0
|
|
# HELP coderd_db_query_counts_total Total number of queries labelled by HTTP route, method, and query name.
|
|
# TYPE coderd_db_query_counts_total counter
|
|
coderd_db_query_counts_total{route="",method="",query=""} 0
|
|
# HELP coderd_db_query_latencies_seconds Latency distribution of queries in seconds.
|
|
# TYPE coderd_db_query_latencies_seconds histogram
|
|
coderd_db_query_latencies_seconds{query=""} 0
|
|
# HELP coderd_db_tx_duration_seconds Duration of transactions in seconds.
|
|
# TYPE coderd_db_tx_duration_seconds histogram
|
|
coderd_db_tx_duration_seconds{success="",tx_id=""} 0
|
|
# HELP coderd_db_tx_executions_count Total count of transactions executed. 'retries' is expected to be 0 for a successful transaction.
|
|
# TYPE coderd_db_tx_executions_count counter
|
|
coderd_db_tx_executions_count{success="",retries="",tx_id=""} 0
|
|
# HELP coderd_dbpurge_iteration_duration_seconds Duration of each dbpurge iteration in seconds.
|
|
# TYPE coderd_dbpurge_iteration_duration_seconds histogram
|
|
coderd_dbpurge_iteration_duration_seconds{success=""} 0
|
|
# HELP coderd_dbpurge_records_purged_total Total number of records purged by type.
|
|
# TYPE coderd_dbpurge_records_purged_total counter
|
|
coderd_dbpurge_records_purged_total{record_type=""} 0
|
|
# HELP coderd_experiments Indicates whether each experiment is enabled (1) or not (0)
|
|
# TYPE coderd_experiments gauge
|
|
coderd_experiments{experiment=""} 0
|
|
# HELP coderd_insights_applications_usage_seconds The application usage per template.
|
|
# TYPE coderd_insights_applications_usage_seconds gauge
|
|
coderd_insights_applications_usage_seconds{template_name="",application_name="",slug="",organization_name=""} 0
|
|
# HELP coderd_insights_parameters The parameter usage per template.
|
|
# TYPE coderd_insights_parameters gauge
|
|
coderd_insights_parameters{template_name="",parameter_name="",parameter_type="",parameter_value="",organization_name=""} 0
|
|
# HELP coderd_insights_templates_active_users The number of active users of the template.
|
|
# TYPE coderd_insights_templates_active_users gauge
|
|
coderd_insights_templates_active_users{template_name="",organization_name=""} 0
|
|
# HELP coderd_license_active_users The number of active users.
|
|
# TYPE coderd_license_active_users gauge
|
|
coderd_license_active_users 0
|
|
# HELP coderd_license_errors The number of active license errors.
|
|
# TYPE coderd_license_errors gauge
|
|
coderd_license_errors 0
|
|
# HELP coderd_license_limit_users The user seats limit based on the active Coder license.
|
|
# TYPE coderd_license_limit_users gauge
|
|
coderd_license_limit_users 0
|
|
# HELP coderd_license_user_limit_enabled Returns 1 if the current license enforces the user limit.
|
|
# TYPE coderd_license_user_limit_enabled gauge
|
|
coderd_license_user_limit_enabled 0
|
|
# HELP coderd_license_warnings The number of active license warnings.
|
|
# TYPE coderd_license_warnings gauge
|
|
coderd_license_warnings 0
|
|
# HELP coderd_lifecycle_autobuild_execution_duration_seconds Duration of each autobuild execution.
|
|
# TYPE coderd_lifecycle_autobuild_execution_duration_seconds histogram
|
|
coderd_lifecycle_autobuild_execution_duration_seconds 0
|
|
# HELP coderd_notifications_dispatcher_send_seconds The time taken to dispatch notifications.
|
|
# TYPE coderd_notifications_dispatcher_send_seconds histogram
|
|
coderd_notifications_dispatcher_send_seconds{method=""} 0
|
|
# HELP coderd_notifications_inflight_dispatches The number of dispatch attempts which are currently in progress.
|
|
# TYPE coderd_notifications_inflight_dispatches gauge
|
|
coderd_notifications_inflight_dispatches{method="",notification_template_id=""} 0
|
|
# HELP coderd_notifications_pending_updates The number of dispatch attempt results waiting to be flushed to the store.
|
|
# TYPE coderd_notifications_pending_updates gauge
|
|
coderd_notifications_pending_updates 0
|
|
# HELP coderd_notifications_queued_seconds The time elapsed between a notification being enqueued in the store and retrieved for dispatching (measures the latency of the notifications system). This should generally be within CODER_NOTIFICATIONS_FETCH_INTERVAL seconds; higher values for a sustained period indicates delayed processing and CODER_NOTIFICATIONS_LEASE_COUNT can be increased to accommodate this.
|
|
# TYPE coderd_notifications_queued_seconds histogram
|
|
coderd_notifications_queued_seconds{method=""} 0
|
|
# HELP coderd_notifications_retry_count The count of notification dispatch retry attempts.
|
|
# TYPE coderd_notifications_retry_count counter
|
|
coderd_notifications_retry_count{method="",notification_template_id=""} 0
|
|
# HELP coderd_notifications_synced_updates_total The number of dispatch attempt results flushed to the store.
|
|
# TYPE coderd_notifications_synced_updates_total counter
|
|
coderd_notifications_synced_updates_total 0
|
|
# HELP coderd_oauth2_external_requests_rate_limit The total number of allowed requests per interval.
|
|
# TYPE coderd_oauth2_external_requests_rate_limit gauge
|
|
coderd_oauth2_external_requests_rate_limit{name="",resource=""} 0
|
|
# HELP coderd_oauth2_external_requests_rate_limit_next_reset_unix Unix timestamp for when the next interval starts
|
|
# TYPE coderd_oauth2_external_requests_rate_limit_next_reset_unix gauge
|
|
coderd_oauth2_external_requests_rate_limit_next_reset_unix{name="",resource=""} 0
|
|
# HELP coderd_oauth2_external_requests_rate_limit_remaining The remaining number of allowed requests in this interval.
|
|
# TYPE coderd_oauth2_external_requests_rate_limit_remaining gauge
|
|
coderd_oauth2_external_requests_rate_limit_remaining{name="",resource=""} 0
|
|
# HELP coderd_oauth2_external_requests_rate_limit_reset_in_seconds Seconds until the next interval
|
|
# TYPE coderd_oauth2_external_requests_rate_limit_reset_in_seconds gauge
|
|
coderd_oauth2_external_requests_rate_limit_reset_in_seconds{name="",resource=""} 0
|
|
# HELP coderd_oauth2_external_requests_rate_limit_used The number of requests made in this interval.
|
|
# TYPE coderd_oauth2_external_requests_rate_limit_used gauge
|
|
coderd_oauth2_external_requests_rate_limit_used{name="",resource=""} 0
|
|
# HELP coderd_oauth2_external_requests_total The total number of api calls made to external oauth2 providers. 'status_code' will be 0 if the request failed with no response.
|
|
# TYPE coderd_oauth2_external_requests_total counter
|
|
coderd_oauth2_external_requests_total{name="",source="",status_code=""} 0
|
|
# HELP coderd_open_file_refs_current The count of file references currently open in the file cache. Multiple references can be held for the same file.
|
|
# TYPE coderd_open_file_refs_current gauge
|
|
coderd_open_file_refs_current 0
|
|
# HELP coderd_open_file_refs_total The total number of file references ever opened in the file cache. The 'hit' label indicates if the file was loaded from the cache.
|
|
# TYPE coderd_open_file_refs_total counter
|
|
coderd_open_file_refs_total{hit=""} 0
|
|
# HELP coderd_open_files_current The count of unique files currently open in the file cache.
|
|
# TYPE coderd_open_files_current gauge
|
|
coderd_open_files_current 0
|
|
# HELP coderd_open_files_size_bytes_current The current amount of memory of all files currently open in the file cache.
|
|
# TYPE coderd_open_files_size_bytes_current gauge
|
|
coderd_open_files_size_bytes_current 0
|
|
# HELP coderd_open_files_size_bytes_total The total amount of memory ever opened in the file cache. This number never decrements.
|
|
# TYPE coderd_open_files_size_bytes_total counter
|
|
coderd_open_files_size_bytes_total 0
|
|
# HELP coderd_open_files_total The total count of unique files ever opened in the file cache.
|
|
# TYPE coderd_open_files_total counter
|
|
coderd_open_files_total 0
|
|
# HELP coderd_prebuilds_reconciliation_duration_seconds Duration of each prebuilds reconciliation cycle.
|
|
# TYPE coderd_prebuilds_reconciliation_duration_seconds histogram
|
|
coderd_prebuilds_reconciliation_duration_seconds 0
|
|
# HELP coderd_prebuilt_workspace_claim_duration_seconds Time to claim a prebuilt workspace by organization, template, and preset.
|
|
# TYPE coderd_prebuilt_workspace_claim_duration_seconds histogram
|
|
coderd_prebuilt_workspace_claim_duration_seconds{organization_name="",template_name="",preset_name=""} 0
|
|
# HELP coderd_prebuilt_workspaces_claimed_total Total number of prebuilt workspaces which were claimed by users. Claiming refers to creating a workspace with a preset selected for which eligible prebuilt workspaces are available and one is reassigned to a user.
|
|
# TYPE coderd_prebuilt_workspaces_claimed_total counter
|
|
coderd_prebuilt_workspaces_claimed_total{template_name="",preset_name="",organization_name=""} 0
|
|
# HELP coderd_prebuilt_workspaces_created_total Total number of prebuilt workspaces that have been created to meet the desired instance count of each template preset.
|
|
# TYPE coderd_prebuilt_workspaces_created_total counter
|
|
coderd_prebuilt_workspaces_created_total{template_name="",preset_name="",organization_name=""} 0
|
|
# HELP coderd_prebuilt_workspaces_desired Target number of prebuilt workspaces that should be available for each template preset.
|
|
# TYPE coderd_prebuilt_workspaces_desired gauge
|
|
coderd_prebuilt_workspaces_desired{template_name="",preset_name="",organization_name=""} 0
|
|
# HELP coderd_prebuilt_workspaces_eligible Current number of prebuilt workspaces that are eligible to be claimed by users. These are workspaces that have completed their build process with their agent reporting 'ready' status.
|
|
# TYPE coderd_prebuilt_workspaces_eligible gauge
|
|
coderd_prebuilt_workspaces_eligible{template_name="",preset_name="",organization_name=""} 0
|
|
# HELP coderd_prebuilt_workspaces_failed_total Total number of prebuilt workspaces that failed to build.
|
|
# TYPE coderd_prebuilt_workspaces_failed_total counter
|
|
coderd_prebuilt_workspaces_failed_total{template_name="",preset_name="",organization_name=""} 0
|
|
# HELP coderd_prebuilt_workspaces_metrics_last_updated The unix timestamp when the metrics related to prebuilt workspaces were last updated; these metrics are cached.
|
|
# TYPE coderd_prebuilt_workspaces_metrics_last_updated gauge
|
|
coderd_prebuilt_workspaces_metrics_last_updated 0
|
|
# HELP coderd_prebuilt_workspaces_preset_hard_limited Indicates whether a given preset has reached the hard failure limit (1 = hard-limited). Metric is omitted otherwise.
|
|
# TYPE coderd_prebuilt_workspaces_preset_hard_limited gauge
|
|
coderd_prebuilt_workspaces_preset_hard_limited{template_name="",preset_name="",organization_name=""} 0
|
|
# HELP coderd_prebuilt_workspaces_preset_validation_failed Indicates whether a given preset has validation failures (1 = validation failed). Metric is omitted otherwise.
|
|
# TYPE coderd_prebuilt_workspaces_preset_validation_failed gauge
|
|
coderd_prebuilt_workspaces_preset_validation_failed{template_name="",preset_name="",organization_name=""} 0
|
|
# HELP coderd_prebuilt_workspaces_reconciliation_paused Indicates whether prebuilds reconciliation is currently paused (1 = paused, 0 = not paused).
|
|
# TYPE coderd_prebuilt_workspaces_reconciliation_paused gauge
|
|
coderd_prebuilt_workspaces_reconciliation_paused 0
|
|
# HELP coderd_prebuilt_workspaces_resource_replacements_total Total number of prebuilt workspaces whose resource(s) got replaced upon being claimed. In Terraform, drift on immutable attributes results in resource replacement. This represents a worst-case scenario for prebuilt workspaces because the pre-provisioned resource would have been recreated when claiming, thus obviating the point of pre-provisioning. See https://coder.com/docs/admin/templates/extending-templates/prebuilt-workspaces#preventing-resource-replacement
|
|
# TYPE coderd_prebuilt_workspaces_resource_replacements_total counter
|
|
coderd_prebuilt_workspaces_resource_replacements_total{template_name="",preset_name="",organization_name=""} 0
|
|
# HELP coderd_prebuilt_workspaces_running Current number of prebuilt workspaces that are in a running state. These workspaces have started successfully but may not yet be claimable by users (see coderd_prebuilt_workspaces_eligible).
|
|
# TYPE coderd_prebuilt_workspaces_running gauge
|
|
coderd_prebuilt_workspaces_running{template_name="",preset_name="",organization_name=""} 0
|
|
# HELP coderd_prometheusmetrics_agents_execution_seconds Histogram for duration of agents metrics collection in seconds.
|
|
# TYPE coderd_prometheusmetrics_agents_execution_seconds histogram
|
|
coderd_prometheusmetrics_agents_execution_seconds 0
|
|
# HELP coderd_prometheusmetrics_agentstats_execution_seconds Histogram for duration of agent stats metrics collection in seconds.
|
|
# TYPE coderd_prometheusmetrics_agentstats_execution_seconds histogram
|
|
coderd_prometheusmetrics_agentstats_execution_seconds 0
|
|
# HELP coderd_prometheusmetrics_metrics_aggregator_execution_cleanup_seconds Histogram for duration of metrics aggregator cleanup in seconds.
|
|
# TYPE coderd_prometheusmetrics_metrics_aggregator_execution_cleanup_seconds histogram
|
|
coderd_prometheusmetrics_metrics_aggregator_execution_cleanup_seconds 0
|
|
# HELP coderd_prometheusmetrics_metrics_aggregator_execution_update_seconds Histogram for duration of metrics aggregator update in seconds.
|
|
# TYPE coderd_prometheusmetrics_metrics_aggregator_execution_update_seconds histogram
|
|
coderd_prometheusmetrics_metrics_aggregator_execution_update_seconds 0
|
|
# HELP coderd_prometheusmetrics_metrics_aggregator_store_size The number of metrics stored in the aggregator
|
|
# TYPE coderd_prometheusmetrics_metrics_aggregator_store_size gauge
|
|
coderd_prometheusmetrics_metrics_aggregator_store_size 0
|
|
# HELP coderd_provisioner_job_queue_wait_seconds Time from job creation to acquisition by a provisioner daemon.
|
|
# TYPE coderd_provisioner_job_queue_wait_seconds histogram
|
|
coderd_provisioner_job_queue_wait_seconds{provisioner_type="",job_type="",transition="",build_reason=""} 0
|
|
# HELP coderd_provisionerd_job_timings_seconds The provisioner job time duration in seconds.
|
|
# TYPE coderd_provisionerd_job_timings_seconds histogram
|
|
coderd_provisionerd_job_timings_seconds{provisioner="",status=""} 0
|
|
# HELP coderd_provisionerd_jobs_current The number of currently running provisioner jobs.
|
|
# TYPE coderd_provisionerd_jobs_current gauge
|
|
coderd_provisionerd_jobs_current{provisioner=""} 0
|
|
# HELP coderd_provisionerd_num_daemons The number of provisioner daemons.
|
|
# TYPE coderd_provisionerd_num_daemons gauge
|
|
coderd_provisionerd_num_daemons 0
|
|
# HELP coderd_provisionerd_workspace_build_timings_seconds The time taken for a workspace to build.
|
|
# TYPE coderd_provisionerd_workspace_build_timings_seconds histogram
|
|
coderd_provisionerd_workspace_build_timings_seconds{template_name="",template_version="",workspace_transition="",status=""} 0
|
|
# HELP coderd_proxyhealth_health_check_duration_seconds Histogram for duration of proxy health collection in seconds.
|
|
# TYPE coderd_proxyhealth_health_check_duration_seconds histogram
|
|
coderd_proxyhealth_health_check_duration_seconds 0
|
|
# HELP coderd_proxyhealth_health_check_results This endpoint returns a number to indicate the health status. -3 (unknown), -2 (Unreachable), -1 (Unhealthy), 0 (Unregistered), 1 (Healthy)
|
|
# TYPE coderd_proxyhealth_health_check_results gauge
|
|
coderd_proxyhealth_health_check_results{proxy_id=""} 0
|
|
# HELP coderd_template_workspace_build_duration_seconds Duration from workspace build creation to agent ready, by template.
|
|
# TYPE coderd_template_workspace_build_duration_seconds histogram
|
|
coderd_template_workspace_build_duration_seconds{template_name="",organization_name="",transition="",status="",is_prebuild=""} 0
|
|
# HELP coderd_workspace_builds_enqueued_total Total number of workspace build enqueue attempts.
|
|
# TYPE coderd_workspace_builds_enqueued_total counter
|
|
coderd_workspace_builds_enqueued_total{provisioner_type="",build_reason="",transition="",status=""} 0
|
|
# HELP coderd_workspace_builds_total The number of workspaces started, updated, or deleted.
|
|
# TYPE coderd_workspace_builds_total counter
|
|
coderd_workspace_builds_total{workspace_owner="",workspace_name="",template_name="",template_version="",workspace_transition="",status=""} 0
|
|
# HELP coderd_workspace_creation_duration_seconds Time to create a workspace by organization, template, preset, and type (regular or prebuild).
|
|
# TYPE coderd_workspace_creation_duration_seconds histogram
|
|
coderd_workspace_creation_duration_seconds{organization_name="",template_name="",preset_name="",type=""} 0
|
|
# HELP coderd_workspace_creation_total Total regular (non-prebuilt) workspace creations by organization, template, and preset.
|
|
# TYPE coderd_workspace_creation_total counter
|
|
coderd_workspace_creation_total{organization_name="",template_name="",preset_name=""} 0
|
|
# HELP coderd_workspace_latest_build_status The current workspace statuses by template, transition, and owner for all non-deleted workspaces.
|
|
# TYPE coderd_workspace_latest_build_status gauge
|
|
coderd_workspace_latest_build_status{status="",template_name="",template_version="",workspace_owner="",workspace_transition=""} 0
|