fix: handle boundary usage across snapshots and flush races (#21805)

Previously there were two issues that could cause incorrect boundary
usage telemetry data.

1. Bad handling across snapshot intervals: After telemetry snapshot deleted
the DB row, the next flush would INSERT the stale cumulative data (which
included already-reported usage). This would then be overwritten by
subsequent UPDATE flushes, causing the delta between the last snapshot
and the reset to be lost (under-reporting usage). Additionally, if there
was no new usage after the reset, the tracker would carry over all usage
from the previous period into the next period (over-reporting usage).

2. Missed usage from a race condition: Track() calls between the first
mutex unlock and second mutex lock in FlushToDB() were lost. The data
wasn't included in the current flush (already snapshotted) and was wiped
by the subsequent reset. This is likely low impact to overall usage
numbers in the real world.

Fix by tracking unique workspace/user deltas separately from cumulative
values and always tracking delta allowed/denied requests. Deltas are used
for INSERT (fresh start after reset), cumulative for UPDATE (accurate unique
counts within a period). All counters reset atomically before the DB operation
so Track() calls during the operation are preserved for the next flush.
This commit is contained in:
Zach
2026-02-02 09:11:54 -07:00
committed by GitHub
parent 6b3d4377c3
commit 90aeea5649
6 changed files with 242 additions and 95 deletions
+4 -2
View File
@@ -40,8 +40,10 @@
// counters. When boundary logs are reported, Track() adds the IDs to the sets
// and increments request counters.
//
// FlushToDB() writes stats to the database, replacing all values with the current
// in-memory state. Stats accumulate in memory throughout the telemetry period.
// FlushToDB() writes stats to the database only when there's been new activity
// since the last flush. This prevents stale data from being written after a
// telemetry reset when no new usage occurred. Stats accumulate in memory
// throughout the telemetry period.
//
// A new period is detected when the upsert results in an INSERT (meaning
// telemetry deleted the replica's row). At that point, all in-memory stats are