mirror of
https://github.com/coder/coder.git
synced 2026-09-22 05:05:20 +08:00
Yevhenii Shcherbina
dae41eb711
Implements: https://linear.app/codercom/issue/AIGOV-448/use-decimal-for-cost-computation Follow-up to https://github.com/coder/coder/pull/26229 Follow-up to the AI Gateway cost-control work. Cost is computed per token category as `tokens × price / 1_000_000` in `int64`, then summed. This change makes an unrepresentable result a defined outcome instead of an accident of integer wrap-around. ## Motivation The intermediate `tokens × price` can exceed `int64`. Real usage cannot get there: at a $75/M model the product overflows at roughly 123 billion tokens in a single response, about six orders of magnitude above a maxed-out Opus request, so this is not a live incident. The problem is what happens if it ever does, because the sign of the wrapped value silently selects between two different failure modes, neither of which was chosen: 1. **Wraps positive.** A plausible-looking cost is stored, incremented into the user's daily spend, and enforced against their AI budget. No error, no signal, wrong number. 2. **Wraps negative.** The value violates `CHECK (cost_micros >= 0)`, the insert fails, the surrounding transaction rolls back, and `RecordTokenUsage` returns a Postgres constraint error that says nothing about overflow. The token usage record is lost entirely, along with its token counts. So the same class of bad input either corrupts budget accounting or discards an audit record, depending on arithmetic that nobody reasoned about. That is the undefined behaviour. ## Decision **An unrepresentable cost is treated as bad input, not a large bill.** Since real usage cannot produce one, it can only mean a wrong price row or implausible provider-reported token counts. In both cases the true cost is unknowable, so no number is stored. **Detect rather than avoid.** `computeCost` now evaluates in `decimal`, so nothing wraps, and range-checks the total against `[0, MaxInt64]` before converting back. Out of range returns `errCostOutOfRange`. Rejecting negatives in the same check also keeps them away from the non-negative column constraint, which would otherwise discard the record. **Log, do not block.** The error is swallowed at the call site: the record is written with token counts intact and `cost_micros` NULL, the spend update is skipped, and the condition is logged at ERROR. **Per-category truncation is unchanged.** Each category is still truncated independently rather than the total being rounded once, so a per-category breakdown recomputed from the snapshotted price columns sums exactly to the stored total. Every existing `computeCost` test case passes unmodified.