opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: add Prometheus metrics for boundary log drop reporting (#22521)

Browse Source 
Add Prometheus metrics to the boundary log proxy for observability:
- batches_dropped_total (reason: buffer_full, forward_failed)
- logs_dropped_total (reason: buffer_full, forward_failed,
  boundary_channel_full, boundary_batch_full)
- batches_forwarded_total

Also add BoundaryStatus to the BoundaryMessage envelope so boundary
can report dropped log counts as a separate wire message. The agent
records these as Prometheus metrics, making boundary-side data loss
visible. Backwards compatibility for older versions of boundary is maintained.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Zach
2026-03-03 12:42:34 -07:00
committed by GitHub
co-authored by Claude Opus 4.6
parent 9b5573d7fa
commit 5b7377c375
9 changed files with 628 additions and 199 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/metricsdocgen/generated_metrics
+9
View File
@@ -1,3 +1,12 @@
# HELP agent_boundary_log_proxy_batches_dropped_total Total number of boundary log batches dropped before reaching coderd. Reason: buffer_full = the agent's internal buffer is full, meaning boundary is producing logs faster than the agent can forward them to coderd; forward_failed = the agent failed to send the batch to coderd, potentially because coderd is unreachable or the connection was interrupted.
# TYPE agent_boundary_log_proxy_batches_dropped_total counter
agent_boundary_log_proxy_batches_dropped_total{reason=""} 0
# HELP agent_boundary_log_proxy_batches_forwarded_total Total number of boundary log batches successfully forwarded to coderd. Compare with batches_dropped_total to compute a drop rate.
# TYPE agent_boundary_log_proxy_batches_forwarded_total counter
agent_boundary_log_proxy_batches_forwarded_total 0
# HELP agent_boundary_log_proxy_logs_dropped_total Total number of individual boundary log entries dropped before reaching coderd. Reason: buffer_full = the agent's internal buffer is full; forward_failed = the agent failed to send the batch to coderd; boundary_channel_full = boundary's internal send channel overflowed, meaning boundary is generating logs faster than it can batch and send them; boundary_batch_full = boundary's outgoing batch buffer overflowed after a failed flush, meaning boundary could not write to the agent's socket.
# TYPE agent_boundary_log_proxy_logs_dropped_total counter
agent_boundary_log_proxy_logs_dropped_total{reason=""} 0
# HELP coder_pubsub_connected Whether we are connected (1) or not connected (0) to postgres
# TYPE coder_pubsub_connected gauge
coder_pubsub_connected 0