mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
feat: record cost on aibridge token usages (#26229)
Implements https://linear.app/codercom/issue/AIGOV-286/add-interception-cost-calculation-to-aibridge-token-usages Adds spend attribution to AI Gateway. After the upstream response, each token-usage record now captures the user's effective group, the per-token prices in effect at that moment, and a computed cost — so spend is recorded as an immutable, point-in-time snapshot. Concretely, `aibridge_token_usages` gains `effective_group_id`, `input_price_micros`, `output_price_micros`, `cache_read_price_micros`, `cache_write_price_micros`, and `cost_micros`. When a usage record is written, the effective group is resolved (per-user override, else the deployment budget policy), the `(provider, model)` price is looked up and snapshotted onto the row, and cost is computed from the provider-reported token counts. A model that isn't in the price table records its tokens with a `NULL` cost; any *other* resolution failure fails the write, so a `NULL` cost unambiguously means "model not priced" rather than "lookup errored." All values are stored in micro-units (1 unit = 1,000,000 micro-units; Phase 1 assumes USD, so 1 micro-unit = $0.000001). Prices are quoted per million tokens. This also grants the AI Bridge RBAC subject `read` on `ai_model_prices` (the per-interception price lookup needs it; it previously only had `update` for the startup seeder). ## Cost precision Cost is computed per token category as `tokens × price / 1_000_000` with integer division, then the four categories are summed. The division is done **per category** (not once over the summed numerator) on purpose: it keeps the per-category line items summing exactly to the stored total — no "the parts don't add up to the whole" in reporting). Integer division truncates sub-micro-unit fractions. For example, a cheap model at $0.10 per million tokens is a price of `100_000`; 9 tokens cost `9 × 100_000 / 1_000_000 = 900_000 / 1_000_000 = 0` (the true 0.9 micro-units floors to 0). At real list prices this rarely bites — $3/M input is a price of `3_000_000`, so even a single token is 3 micro-units. The per-record under-count is bounded below 1 micro-unit per category, so under $0.000004 total across the four categories, which is acceptable for list-price-based cost approximation. ## Overflow safety `cost_micros` is a `BIGINT` (int64), and the largest intermediate value is a single category's `tokens × price` before division. int64's ceiling is ≈ `9.223e18`. - At a steep $75/M model (price `75_000_000`), overflow would require ~123 billion tokens in one response: `123e9 × 75e6 = 9.225e18`, just over the limit. `122e9` stays under at `9.15e18`. - A realistically maxed-out Opus 4.8 response (≈1M input + 128K output at list prices) costs about $15, with a numerator around `1.5e13` — roughly six orders of magnitude below the ceiling. So overflow is unreachable from real token counts. ### Multi-currency support In the future, we may encounter issues with multi-currency support, especially when dealing with currencies that have very large exchange rates relative to USD, for example: IRR: ~1,300,000 IRR ≈ 1 USD VND: ~26,000 VND ≈ 1 USD For currencies with such large denominations, numeric overflow is technically possible, considering that we have only about six orders of magnitude of headroom before reaching the limit (see above). ## `effective_group_id` has no foreign key `effective_group_id` records the group a spend was attributed to, as an immutable historical fact. It is intentionally **not** a foreign key, so the record survives deletion of the group. Alternatives were considered and rejected: - **`ON DELETE SET NULL`** would mutate an "immutable" record — deleting a group silently erases that interception's attribution and under-counts the group's historical spend. - **`RESTRICT` / `NO ACTION`** would block group deletion entirely (groups are hard-deleted). - **`CASCADE`** would delete spend history when a group is deleted — the worst outcome for an audit record. There is also no insert-time check that the group still exists: the id comes from a budget that was just resolved, meaning it was valid at some point. ## Open question: group name snapshotting Should we also snapshot the group *name* onto each record? Two options: - **Denormalize it now** — readable in historical reports even after a group is deleted, but the snapshot can drift from the current name on rename, raising a "show point-in-time vs. current name" question. - **Postpone until needed** — it's a purely additive column later, and the name is display-only (not correctness-bearing like the price). The cost: names of groups deleted before the column is added can't be backfilled. Leaning toward postponing until a concrete reporting need settles the drift question.
This commit is contained in:
@@ -67,6 +67,13 @@ type store interface {
|
||||
UpdateAIBridgeInterceptionEnded(ctx context.Context, intcID database.UpdateAIBridgeInterceptionEndedParams) (database.AIBridgeInterception, error)
|
||||
GetAIBridgeInterceptionLineageByToolCallID(ctx context.Context, toolCallID string) (database.GetAIBridgeInterceptionLineageByToolCallIDRow, error)
|
||||
|
||||
// Cost-attribution queries, used to snapshot price and effective group on
|
||||
// each token usage record.
|
||||
GetAIBridgeInterceptionByID(ctx context.Context, id uuid.UUID) (database.AIBridgeInterception, error)
|
||||
GetAIModelPriceByProviderModel(ctx context.Context, arg database.GetAIModelPriceByProviderModelParams) (database.AIModelPrice, error)
|
||||
GetUserAIBudgetOverride(ctx context.Context, userID uuid.UUID) (database.UserAIBudgetOverride, error)
|
||||
GetHighestGroupAIBudgetByUser(ctx context.Context, userID uuid.UUID) (database.GetHighestGroupAIBudgetByUserRow, error)
|
||||
|
||||
// MCPConfigurator-related queries.
|
||||
GetExternalAuthLinksByUserID(ctx context.Context, userID uuid.UUID) ([]database.ExternalAuthLink, error)
|
||||
|
||||
@@ -87,6 +94,9 @@ type Server struct {
|
||||
coderMCPConfig *proto.MCPServerConfig // may be nil if not available
|
||||
structuredLogging bool
|
||||
aiSeatTracker aiseats.SeatTracker
|
||||
// budgetPolicy selects the effective group when a user belongs to multiple
|
||||
// budgeted groups, used for cost attribution on token usage records.
|
||||
budgetPolicy codersdk.AIBudgetPolicy
|
||||
}
|
||||
|
||||
func NewServer(lifecycleCtx context.Context, store store, logger slog.Logger, accessURL string,
|
||||
@@ -110,6 +120,7 @@ func NewServer(lifecycleCtx context.Context, store store, logger slog.Logger, ac
|
||||
externalAuthConfigs: eac,
|
||||
structuredLogging: bridgeCfg.StructuredLogging.Value(),
|
||||
aiSeatTracker: aiSeatTracker,
|
||||
budgetPolicy: codersdk.NewAIBudgetPolicyFromString(bridgeCfg.BudgetPolicy),
|
||||
}
|
||||
|
||||
if bridgeCfg.InjectCoderMCPTools {
|
||||
@@ -272,6 +283,21 @@ func (s *Server) RecordTokenUsage(ctx context.Context, in *proto.RecordTokenUsag
|
||||
s.logger.Warn(ctx, "failed to marshal aibridge metadata from proto to JSON", slog.F("metadata", in), slog.Error(err))
|
||||
}
|
||||
|
||||
// The interception is always recorded before any of its token usages,
|
||||
// so it must exist. It carries the provider, model, and initiator needed
|
||||
// for cost attribution.
|
||||
intc, err := s.store.GetAIBridgeInterceptionByID(ctx, intcID)
|
||||
if err != nil {
|
||||
return nil, xerrors.Errorf("get interception %q: %w", intcID, err)
|
||||
}
|
||||
|
||||
// Snapshot the effective group, per-token prices and compute cost. A
|
||||
// missing price row or unbudgeted user yields NULL columns.
|
||||
cost, err := s.resolveTokenUsageCost(ctx, intc, in)
|
||||
if err != nil {
|
||||
return nil, xerrors.Errorf("resolve token usage cost: %w", err)
|
||||
}
|
||||
|
||||
_, err = s.store.InsertAIBridgeTokenUsage(ctx, database.InsertAIBridgeTokenUsageParams{
|
||||
ID: uuid.New(),
|
||||
InterceptionID: intcID,
|
||||
@@ -282,6 +308,12 @@ func (s *Server) RecordTokenUsage(ctx context.Context, in *proto.RecordTokenUsag
|
||||
CacheWriteInputTokens: in.GetCacheWriteInputTokens(),
|
||||
Metadata: out,
|
||||
CreatedAt: in.GetCreatedAt().AsTime(),
|
||||
EffectiveGroupID: cost.effectiveGroupID,
|
||||
InputPriceMicros: cost.inputPriceMicros,
|
||||
OutputPriceMicros: cost.outputPriceMicros,
|
||||
CacheReadPriceMicros: cost.cacheReadPriceMicros,
|
||||
CacheWritePriceMicros: cost.cacheWritePriceMicros,
|
||||
CostMicros: cost.costMicros,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, xerrors.Errorf("insert token usage: %w", err)
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
"github.com/sqlc-dev/pqtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
@@ -29,13 +30,16 @@ import (
|
||||
"github.com/coder/coder/v2/coderd/aibridgedserver"
|
||||
agplaiseats "github.com/coder/coder/v2/coderd/aiseats"
|
||||
"github.com/coder/coder/v2/coderd/apikey"
|
||||
"github.com/coder/coder/v2/coderd/coderdtest"
|
||||
"github.com/coder/coder/v2/coderd/database"
|
||||
"github.com/coder/coder/v2/coderd/database/dbauthz"
|
||||
"github.com/coder/coder/v2/coderd/database/dbgen"
|
||||
"github.com/coder/coder/v2/coderd/database/dbmock"
|
||||
"github.com/coder/coder/v2/coderd/database/dbtestutil"
|
||||
"github.com/coder/coder/v2/coderd/database/dbtime"
|
||||
"github.com/coder/coder/v2/coderd/externalauth"
|
||||
codermcp "github.com/coder/coder/v2/coderd/mcp"
|
||||
"github.com/coder/coder/v2/coderd/rbac"
|
||||
"github.com/coder/coder/v2/coderd/util/ptr"
|
||||
"github.com/coder/coder/v2/codersdk"
|
||||
"github.com/coder/coder/v2/cryptorand"
|
||||
@@ -1172,7 +1176,7 @@ func TestRecordTokenUsage(t *testing.T) {
|
||||
},
|
||||
[]testRecordMethodCase[*proto.RecordTokenUsageRequest]{
|
||||
{
|
||||
name: "valid token usage",
|
||||
name: "valid token usage with null cost",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
@@ -1187,6 +1191,11 @@ func TestRecordTokenUsage(t *testing.T) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
// No budget configured and no price row: tokens recorded
|
||||
// with NULL cost, prices, and group attribution.
|
||||
intc := newTestInterception(interceptionID)
|
||||
expectTokenUsageCostLookups(db, intc, nil, nil, nil)
|
||||
|
||||
db.EXPECT().InsertAIBridgeTokenUsage(gomock.Any(), gomock.Cond(func(p database.InsertAIBridgeTokenUsageParams) bool {
|
||||
if !assert.NotEqual(t, uuid.Nil, p.ID, "ID") ||
|
||||
!assert.Equal(t, interceptionID, p.InterceptionID, "interception ID") ||
|
||||
@@ -1196,7 +1205,13 @@ func TestRecordTokenUsage(t *testing.T) {
|
||||
!assert.Equal(t, req.GetCacheReadInputTokens(), p.CacheReadInputTokens, "cache read input tokens") ||
|
||||
!assert.Equal(t, req.GetCacheWriteInputTokens(), p.CacheWriteInputTokens, "cache write input tokens") ||
|
||||
!assert.JSONEq(t, metadataJSON, string(p.Metadata), "metadata") ||
|
||||
!assert.WithinDuration(t, req.GetCreatedAt().AsTime(), p.CreatedAt, time.Second, "created at") {
|
||||
!assert.WithinDuration(t, req.GetCreatedAt().AsTime(), p.CreatedAt, time.Second, "created at") ||
|
||||
!assert.False(t, p.EffectiveGroupID.Valid, "effective group ID null") ||
|
||||
!assert.False(t, p.InputPriceMicros.Valid, "input price null") ||
|
||||
!assert.False(t, p.OutputPriceMicros.Valid, "output price null") ||
|
||||
!assert.False(t, p.CacheReadPriceMicros.Valid, "cache read price null") ||
|
||||
!assert.False(t, p.CacheWritePriceMicros.Valid, "cache write price null") ||
|
||||
!assert.False(t, p.CostMicros.Valid, "cost null") {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
@@ -1216,6 +1231,256 @@ func TestRecordTokenUsage(t *testing.T) {
|
||||
}, nil)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "valid token usage with cost",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
InputTokens: 100,
|
||||
OutputTokens: 200,
|
||||
CacheReadInputTokens: 50,
|
||||
CacheWriteInputTokens: 10,
|
||||
CreatedAt: timestamppb.Now(),
|
||||
},
|
||||
setupMocks: func(t *testing.T, db *dbmock.MockStore, req *proto.RecordTokenUsageRequest) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
intc := newTestInterception(interceptionID)
|
||||
groupID := uuid.New()
|
||||
group := &database.GetHighestGroupAIBudgetByUserRow{GroupID: groupID, SpendLimitMicros: 1_000_000_000}
|
||||
price := &database.AIModelPrice{
|
||||
Provider: intc.Provider,
|
||||
Model: intc.Model,
|
||||
InputPrice: sql.NullInt64{Int64: 3_000_000, Valid: true},
|
||||
OutputPrice: sql.NullInt64{Int64: 6_000_000, Valid: true},
|
||||
CacheReadPrice: sql.NullInt64{Int64: 300_000, Valid: true},
|
||||
CacheWritePrice: sql.NullInt64{Int64: 4_000_000, Valid: true},
|
||||
}
|
||||
// No override
|
||||
expectTokenUsageCostLookups(db, intc, nil, group, price)
|
||||
|
||||
// input 300 + output 1200 + cache read 15 + cache write 40.
|
||||
const wantCost int64 = 1555
|
||||
|
||||
db.EXPECT().InsertAIBridgeTokenUsage(gomock.Any(), gomock.Cond(func(p database.InsertAIBridgeTokenUsageParams) bool {
|
||||
if !assert.Equal(t, uuid.NullUUID{UUID: groupID, Valid: true}, p.EffectiveGroupID, "effective group ID") ||
|
||||
!assert.Equal(t, price.InputPrice, p.InputPriceMicros, "input price") ||
|
||||
!assert.Equal(t, price.OutputPrice, p.OutputPriceMicros, "output price") ||
|
||||
!assert.Equal(t, price.CacheReadPrice, p.CacheReadPriceMicros, "cache read price") ||
|
||||
!assert.Equal(t, price.CacheWritePrice, p.CacheWritePriceMicros, "cache write price") ||
|
||||
!assert.Equal(t, sql.NullInt64{Int64: wantCost, Valid: true}, p.CostMicros, "cost") {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
})).Return(database.AIBridgeTokenUsage{ID: uuid.New(), InterceptionID: interceptionID}, nil)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "valid token usage with user override",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
InputTokens: 100,
|
||||
CreatedAt: timestamppb.Now(),
|
||||
},
|
||||
setupMocks: func(t *testing.T, db *dbmock.MockStore, req *proto.RecordTokenUsageRequest) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
intc := newTestInterception(interceptionID)
|
||||
overrideGroupID := uuid.New()
|
||||
override := &database.UserAIBudgetOverride{
|
||||
UserID: intc.InitiatorID,
|
||||
GroupID: overrideGroupID,
|
||||
SpendLimitMicros: 1_500_000_000,
|
||||
}
|
||||
price := &database.AIModelPrice{
|
||||
Provider: intc.Provider,
|
||||
Model: intc.Model,
|
||||
InputPrice: sql.NullInt64{Int64: 3_000_000, Valid: true},
|
||||
}
|
||||
// No group
|
||||
expectTokenUsageCostLookups(db, intc, override, nil, price)
|
||||
|
||||
db.EXPECT().InsertAIBridgeTokenUsage(gomock.Any(), gomock.Cond(func(p database.InsertAIBridgeTokenUsageParams) bool {
|
||||
// Override group wins.
|
||||
if !assert.Equal(t, uuid.NullUUID{UUID: overrideGroupID, Valid: true}, p.EffectiveGroupID, "effective group ID") ||
|
||||
!assert.Equal(t, sql.NullInt64{Int64: 300, Valid: true}, p.CostMicros, "cost") {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
})).Return(database.AIBridgeTokenUsage{ID: uuid.New(), InterceptionID: interceptionID}, nil)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "valid token usage with budget but no price",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
InputTokens: 100,
|
||||
OutputTokens: 200,
|
||||
CacheReadInputTokens: 50,
|
||||
CacheWriteInputTokens: 10,
|
||||
CreatedAt: timestamppb.Now(),
|
||||
},
|
||||
setupMocks: func(t *testing.T, db *dbmock.MockStore, req *proto.RecordTokenUsageRequest) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
intc := newTestInterception(interceptionID)
|
||||
groupID := uuid.New()
|
||||
group := &database.GetHighestGroupAIBudgetByUserRow{GroupID: groupID, SpendLimitMicros: 1_000_000_000}
|
||||
// Budget resolves to a group, but the model has no price row.
|
||||
// The resolved group must survive the price lookup's early
|
||||
// return on sql.ErrNoRows, while prices and cost stay NULL.
|
||||
expectTokenUsageCostLookups(db, intc, nil, group, nil)
|
||||
|
||||
db.EXPECT().InsertAIBridgeTokenUsage(gomock.Any(), gomock.Cond(func(p database.InsertAIBridgeTokenUsageParams) bool {
|
||||
if !assert.Equal(t, uuid.NullUUID{UUID: groupID, Valid: true}, p.EffectiveGroupID, "effective group ID") ||
|
||||
!assert.False(t, p.InputPriceMicros.Valid, "input price null") ||
|
||||
!assert.False(t, p.OutputPriceMicros.Valid, "output price null") ||
|
||||
!assert.False(t, p.CacheReadPriceMicros.Valid, "cache read price null") ||
|
||||
!assert.False(t, p.CacheWritePriceMicros.Valid, "cache write price null") ||
|
||||
!assert.False(t, p.CostMicros.Valid, "cost null") {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
})).Return(database.AIBridgeTokenUsage{ID: uuid.New(), InterceptionID: interceptionID}, nil)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "valid token usage with price but no budget",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
InputTokens: 100,
|
||||
OutputTokens: 200,
|
||||
CacheReadInputTokens: 50,
|
||||
CacheWriteInputTokens: 10,
|
||||
CreatedAt: timestamppb.Now(),
|
||||
},
|
||||
setupMocks: func(t *testing.T, db *dbmock.MockStore, req *proto.RecordTokenUsageRequest) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
intc := newTestInterception(interceptionID)
|
||||
price := &database.AIModelPrice{
|
||||
Provider: intc.Provider,
|
||||
Model: intc.Model,
|
||||
InputPrice: sql.NullInt64{Int64: 3_000_000, Valid: true},
|
||||
OutputPrice: sql.NullInt64{Int64: 6_000_000, Valid: true},
|
||||
CacheReadPrice: sql.NullInt64{Int64: 300_000, Valid: true},
|
||||
CacheWritePrice: sql.NullInt64{Int64: 4_000_000, Valid: true},
|
||||
}
|
||||
// No budget configured, but the model is priced: cost is
|
||||
// computed independently of budget resolution, and the group
|
||||
// attribution stays NULL.
|
||||
expectTokenUsageCostLookups(db, intc, nil, nil, price)
|
||||
|
||||
// input 300 + output 1200 + cache read 15 + cache write 40.
|
||||
const wantCost int64 = 1555
|
||||
|
||||
db.EXPECT().InsertAIBridgeTokenUsage(gomock.Any(), gomock.Cond(func(p database.InsertAIBridgeTokenUsageParams) bool {
|
||||
if !assert.False(t, p.EffectiveGroupID.Valid, "effective group ID null") ||
|
||||
!assert.Equal(t, price.InputPrice, p.InputPriceMicros, "input price") ||
|
||||
!assert.Equal(t, price.OutputPrice, p.OutputPriceMicros, "output price") ||
|
||||
!assert.Equal(t, price.CacheReadPrice, p.CacheReadPriceMicros, "cache read price") ||
|
||||
!assert.Equal(t, price.CacheWritePrice, p.CacheWritePriceMicros, "cache write price") ||
|
||||
!assert.Equal(t, sql.NullInt64{Int64: wantCost, Valid: true}, p.CostMicros, "cost") {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
})).Return(database.AIBridgeTokenUsage{ID: uuid.New(), InterceptionID: interceptionID}, nil)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "valid token usage with zero prices",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
InputTokens: 100,
|
||||
OutputTokens: 200,
|
||||
CacheReadInputTokens: 50,
|
||||
CacheWriteInputTokens: 10,
|
||||
CreatedAt: timestamppb.Now(),
|
||||
},
|
||||
setupMocks: func(t *testing.T, db *dbmock.MockStore, req *proto.RecordTokenUsageRequest) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
intc := newTestInterception(interceptionID)
|
||||
// A model priced at zero is distinct from an unpriced model:
|
||||
// the price columns and cost are recorded as 0, not NULL.
|
||||
price := &database.AIModelPrice{
|
||||
Provider: intc.Provider,
|
||||
Model: intc.Model,
|
||||
InputPrice: sql.NullInt64{Int64: 0, Valid: true},
|
||||
OutputPrice: sql.NullInt64{Int64: 0, Valid: true},
|
||||
CacheReadPrice: sql.NullInt64{Int64: 0, Valid: true},
|
||||
CacheWritePrice: sql.NullInt64{Int64: 0, Valid: true},
|
||||
}
|
||||
expectTokenUsageCostLookups(db, intc, nil, nil, price)
|
||||
|
||||
db.EXPECT().InsertAIBridgeTokenUsage(gomock.Any(), gomock.Cond(func(p database.InsertAIBridgeTokenUsageParams) bool {
|
||||
zero := sql.NullInt64{Int64: 0, Valid: true}
|
||||
if !assert.Equal(t, zero, p.InputPriceMicros, "input price zero") ||
|
||||
!assert.Equal(t, zero, p.OutputPriceMicros, "output price zero") ||
|
||||
!assert.Equal(t, zero, p.CacheReadPriceMicros, "cache read price zero") ||
|
||||
!assert.Equal(t, zero, p.CacheWritePriceMicros, "cache write price zero") ||
|
||||
// Cost is 0 but recorded (Valid), not NULL.
|
||||
!assert.Equal(t, zero, p.CostMicros, "cost zero") {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
})).Return(database.AIBridgeTokenUsage{ID: uuid.New(), InterceptionID: interceptionID}, nil)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "valid token usage with all null prices",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
InputTokens: 100,
|
||||
OutputTokens: 200,
|
||||
CacheReadInputTokens: 50,
|
||||
CacheWriteInputTokens: 10,
|
||||
CreatedAt: timestamppb.Now(),
|
||||
},
|
||||
setupMocks: func(t *testing.T, db *dbmock.MockStore, req *proto.RecordTokenUsageRequest) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
intc := newTestInterception(interceptionID)
|
||||
// The price row exists but every price column is NULL. Each
|
||||
// category is treated as zero for cost, so the columns are
|
||||
// recorded as NULL while cost is recorded as 0 (not NULL):
|
||||
// cost's NULL-ness tracks price row presence, not the price
|
||||
// values.
|
||||
price := &database.AIModelPrice{
|
||||
Provider: intc.Provider,
|
||||
Model: intc.Model,
|
||||
InputPrice: sql.NullInt64{Valid: false},
|
||||
OutputPrice: sql.NullInt64{Valid: false},
|
||||
CacheReadPrice: sql.NullInt64{Valid: false},
|
||||
CacheWritePrice: sql.NullInt64{Valid: false},
|
||||
}
|
||||
expectTokenUsageCostLookups(db, intc, nil, nil, price)
|
||||
|
||||
db.EXPECT().InsertAIBridgeTokenUsage(gomock.Any(), gomock.Cond(func(p database.InsertAIBridgeTokenUsageParams) bool {
|
||||
if !assert.False(t, p.InputPriceMicros.Valid, "input price null") ||
|
||||
!assert.False(t, p.OutputPriceMicros.Valid, "output price null") ||
|
||||
!assert.False(t, p.CacheReadPriceMicros.Valid, "cache read price null") ||
|
||||
!assert.False(t, p.CacheWritePriceMicros.Valid, "cache write price null") ||
|
||||
// Cost is recorded as 0 (Valid), not NULL, because the
|
||||
// price row exists.
|
||||
!assert.Equal(t, sql.NullInt64{Int64: 0, Valid: true}, p.CostMicros, "cost zero") {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
})).Return(database.AIBridgeTokenUsage{ID: uuid.New(), InterceptionID: interceptionID}, nil)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "invalid interception ID",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
@@ -1228,7 +1493,7 @@ func TestRecordTokenUsage(t *testing.T) {
|
||||
expectedErr: "failed to parse interception_id",
|
||||
},
|
||||
{
|
||||
name: "database error",
|
||||
name: "interception lookup error",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
@@ -1237,6 +1502,56 @@ func TestRecordTokenUsage(t *testing.T) {
|
||||
CreatedAt: timestamppb.Now(),
|
||||
},
|
||||
setupMocks: func(t *testing.T, db *dbmock.MockStore, req *proto.RecordTokenUsageRequest) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
// An unexpected interception lookup error fails the record;
|
||||
// no token usage is inserted.
|
||||
db.EXPECT().GetAIBridgeInterceptionByID(gomock.Any(), interceptionID).
|
||||
Return(database.AIBridgeInterception{}, sql.ErrConnDone)
|
||||
},
|
||||
expectedErr: "get interception",
|
||||
},
|
||||
{
|
||||
name: "price lookup error",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
InputTokens: 100,
|
||||
OutputTokens: 200,
|
||||
CreatedAt: timestamppb.Now(),
|
||||
},
|
||||
setupMocks: func(t *testing.T, db *dbmock.MockStore, req *proto.RecordTokenUsageRequest) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
// An unexpected price lookup error (not sql.ErrNoRows) fails
|
||||
// the record.
|
||||
intc := newTestInterception(interceptionID)
|
||||
db.EXPECT().GetAIBridgeInterceptionByID(gomock.Any(), interceptionID).Return(intc, nil)
|
||||
db.EXPECT().GetUserAIBudgetOverride(gomock.Any(), intc.InitiatorID).
|
||||
Return(database.UserAIBudgetOverride{}, sql.ErrNoRows)
|
||||
db.EXPECT().GetHighestGroupAIBudgetByUser(gomock.Any(), intc.InitiatorID).
|
||||
Return(database.GetHighestGroupAIBudgetByUserRow{}, sql.ErrNoRows)
|
||||
db.EXPECT().GetAIModelPriceByProviderModel(gomock.Any(), gomock.Any()).
|
||||
Return(database.AIModelPrice{}, sql.ErrConnDone)
|
||||
},
|
||||
expectedErr: "resolve token usage cost",
|
||||
},
|
||||
{
|
||||
name: "insert error",
|
||||
request: &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: uuid.NewString(),
|
||||
MsgId: "msg_123",
|
||||
InputTokens: 100,
|
||||
OutputTokens: 200,
|
||||
CreatedAt: timestamppb.Now(),
|
||||
},
|
||||
setupMocks: func(t *testing.T, db *dbmock.MockStore, req *proto.RecordTokenUsageRequest) {
|
||||
interceptionID, err := uuid.Parse(req.GetInterceptionId())
|
||||
assert.NoError(t, err, "parse interception UUID")
|
||||
|
||||
expectTokenUsageCostLookups(db, newTestInterception(interceptionID), nil, nil, nil)
|
||||
db.EXPECT().InsertAIBridgeTokenUsage(gomock.Any(), gomock.Any()).Return(database.AIBridgeTokenUsage{}, sql.ErrConnDone)
|
||||
},
|
||||
expectedErr: "insert token usage",
|
||||
@@ -1245,6 +1560,128 @@ func TestRecordTokenUsage(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
// TestRecordTokenUsageAuthorized exercises RecordTokenUsage end-to-end against a
|
||||
// real database through the dbauthz layer as subjectAibridged. This catches missing
|
||||
// RBAC grants on the aibridged subject and verifies the cost columns round-trip to storage.
|
||||
func TestRecordTokenUsageAuthorized(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
ctx := testutil.Context(t, testutil.WaitLong)
|
||||
logger := testutil.Logger(t)
|
||||
|
||||
rawDB, _ := dbtestutil.NewDB(t)
|
||||
authzDB := dbauthz.New(rawDB, rbac.NewStrictAuthorizer(prometheus.NewRegistry()), logger, coderdtest.AccessControlStorePointer())
|
||||
|
||||
// Seed prerequisites via the raw (unauthorized) store. The user belongs to a
|
||||
// group with a budget, so the effective group resolves to that group.
|
||||
org := dbgen.Organization(t, rawDB, database.Organization{})
|
||||
user := dbgen.User(t, rawDB, database.User{})
|
||||
dbgen.OrganizationMember(t, rawDB, database.OrganizationMember{OrganizationID: org.ID, UserID: user.ID})
|
||||
group := dbgen.Group(t, rawDB, database.Group{OrganizationID: org.ID})
|
||||
dbgen.GroupMember(t, rawDB, database.GroupMemberTable{UserID: user.ID, GroupID: group.ID})
|
||||
|
||||
_, err := rawDB.UpsertGroupAIBudget(ctx, database.UpsertGroupAIBudgetParams{
|
||||
GroupID: group.ID,
|
||||
SpendLimitMicros: 1_000_000_000,
|
||||
})
|
||||
require.NoError(t, err, "upsert group AI budget")
|
||||
|
||||
const provider, model = "anthropic", "claude-sonnet-4-6"
|
||||
priceSeed, err := json.Marshal([]map[string]any{{
|
||||
"provider": provider,
|
||||
"model": model,
|
||||
"input_price": 3_000_000,
|
||||
"output_price": 6_000_000,
|
||||
"cache_read_price": 300_000,
|
||||
"cache_write_price": 4_000_000,
|
||||
}})
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, rawDB.UpsertAIModelPrices(ctx, priceSeed), "seed model prices")
|
||||
|
||||
intc := dbgen.AIBridgeInterception(t, rawDB, database.InsertAIBridgeInterceptionParams{
|
||||
InitiatorID: user.ID,
|
||||
Provider: provider,
|
||||
Model: model,
|
||||
}, nil)
|
||||
|
||||
// The server runs every store call as subjectAibridged via the authzDB.
|
||||
srv, err := aibridgedserver.NewServer(ctx, authzDB, logger, "/", codersdk.AIBridgeConfig{}, nil, requiredExperiments, agplaiseats.Noop{})
|
||||
require.NoError(t, err)
|
||||
|
||||
_, err = srv.RecordTokenUsage(ctx, &proto.RecordTokenUsageRequest{
|
||||
InterceptionId: intc.ID.String(),
|
||||
MsgId: "msg_e2e",
|
||||
InputTokens: 100,
|
||||
OutputTokens: 200,
|
||||
CacheReadInputTokens: 50,
|
||||
CacheWriteInputTokens: 10,
|
||||
CreatedAt: timestamppb.Now(),
|
||||
})
|
||||
require.NoError(t, err, "record token usage")
|
||||
|
||||
// Read the persisted row back via the raw store and verify the snapshot.
|
||||
usages, err := rawDB.GetAIBridgeTokenUsagesByInterceptionID(ctx, intc.ID)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, usages, 1)
|
||||
got := usages[0]
|
||||
|
||||
require.Equal(t, uuid.NullUUID{UUID: group.ID, Valid: true}, got.EffectiveGroupID, "effective group")
|
||||
require.Equal(t, sql.NullInt64{Int64: 3_000_000, Valid: true}, got.InputPriceMicros, "input price")
|
||||
require.Equal(t, sql.NullInt64{Int64: 6_000_000, Valid: true}, got.OutputPriceMicros, "output price")
|
||||
require.Equal(t, sql.NullInt64{Int64: 300_000, Valid: true}, got.CacheReadPriceMicros, "cache read price")
|
||||
require.Equal(t, sql.NullInt64{Int64: 4_000_000, Valid: true}, got.CacheWritePriceMicros, "cache write price")
|
||||
// input 300 + output 1200 + cache read 15 + cache write 40.
|
||||
require.Equal(t, sql.NullInt64{Int64: 1555, Valid: true}, got.CostMicros, "cost")
|
||||
}
|
||||
|
||||
// newTestInterception returns an interception with a fixed initiator, provider,
|
||||
// and model for cost-attribution test setup.
|
||||
func newTestInterception(id uuid.UUID) database.AIBridgeInterception {
|
||||
return database.AIBridgeInterception{
|
||||
ID: id,
|
||||
InitiatorID: uuid.New(),
|
||||
Provider: "anthropic",
|
||||
Model: "claude-sonnet-4-6",
|
||||
}
|
||||
}
|
||||
|
||||
// expectTokenUsageCostLookups mocks the store lookups made by resolveTokenUsageCost
|
||||
// (budget resolution and the price lookup). A nil override, group, or price makes that
|
||||
// lookup return sql.ErrNoRows. Budget resolution mirrors production code: a non-nil override
|
||||
// wins and skips the group lookup, so group is consulted only when override is nil.
|
||||
func expectTokenUsageCostLookups(
|
||||
db *dbmock.MockStore,
|
||||
intc database.AIBridgeInterception,
|
||||
override *database.UserAIBudgetOverride,
|
||||
group *database.GetHighestGroupAIBudgetByUserRow,
|
||||
price *database.AIModelPrice,
|
||||
) {
|
||||
db.EXPECT().GetAIBridgeInterceptionByID(gomock.Any(), intc.ID).Return(intc, nil)
|
||||
|
||||
if override != nil {
|
||||
db.EXPECT().GetUserAIBudgetOverride(gomock.Any(), intc.InitiatorID).Return(*override, nil)
|
||||
} else {
|
||||
db.EXPECT().GetUserAIBudgetOverride(gomock.Any(), intc.InitiatorID).
|
||||
Return(database.UserAIBudgetOverride{}, sql.ErrNoRows)
|
||||
if group != nil {
|
||||
db.EXPECT().GetHighestGroupAIBudgetByUser(gomock.Any(), intc.InitiatorID).Return(*group, nil)
|
||||
} else {
|
||||
db.EXPECT().GetHighestGroupAIBudgetByUser(gomock.Any(), intc.InitiatorID).
|
||||
Return(database.GetHighestGroupAIBudgetByUserRow{}, sql.ErrNoRows)
|
||||
}
|
||||
}
|
||||
|
||||
if price != nil {
|
||||
db.EXPECT().GetAIModelPriceByProviderModel(gomock.Any(), database.GetAIModelPriceByProviderModelParams{
|
||||
Provider: intc.Provider,
|
||||
Model: intc.Model,
|
||||
}).Return(*price, nil)
|
||||
} else {
|
||||
db.EXPECT().GetAIModelPriceByProviderModel(gomock.Any(), gomock.Any()).
|
||||
Return(database.AIModelPrice{}, sql.ErrNoRows)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordPromptUsage(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -1722,6 +2159,7 @@ func TestStructuredLogging(t *testing.T) {
|
||||
name: "RecordTokenUsage_logs_when_enabled",
|
||||
structuredLogging: true,
|
||||
setupMocks: func(db *dbmock.MockStore, intcID uuid.UUID) {
|
||||
expectTokenUsageCostLookups(db, newTestInterception(intcID), nil, nil, nil)
|
||||
db.EXPECT().InsertAIBridgeTokenUsage(gomock.Any(), gomock.Any()).Return(database.AIBridgeTokenUsage{
|
||||
ID: uuid.New(),
|
||||
InterceptionID: intcID,
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
package aibridgedserver
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"golang.org/x/xerrors"
|
||||
|
||||
"cdr.dev/slog/v3"
|
||||
"github.com/coder/coder/v2/coderd/aibridge/budget"
|
||||
"github.com/coder/coder/v2/coderd/aibridged/proto"
|
||||
"github.com/coder/coder/v2/coderd/database"
|
||||
)
|
||||
|
||||
// tokensPerMillion is the divisor for prices, which are quoted per million
|
||||
// tokens.
|
||||
const tokensPerMillion = 1_000_000
|
||||
|
||||
// tokenUsageCost holds the cost-attribution columns snapshotted onto a token
|
||||
// usage record. A field left unset (Valid == false) is recorded as SQL NULL; a
|
||||
// price or cost of 0 is recorded as 0, which is distinct from NULL.
|
||||
type tokenUsageCost struct {
|
||||
effectiveGroupID uuid.NullUUID
|
||||
inputPriceMicros sql.NullInt64
|
||||
outputPriceMicros sql.NullInt64
|
||||
cacheReadPriceMicros sql.NullInt64
|
||||
cacheWritePriceMicros sql.NullInt64
|
||||
costMicros sql.NullInt64
|
||||
}
|
||||
|
||||
// resolveTokenUsageCost resolves the effective group and per-token prices for an
|
||||
// interception and computes its cost. Two outcomes are expected and yield NULL
|
||||
// columns rather than an error: a user with no configured budget (yields a NULL
|
||||
// group) and a model absent from the price table (yields NULL prices and cost).
|
||||
// Any other error is returned. A NULL cost unambiguously means "model not priced".
|
||||
func (s *Server) resolveTokenUsageCost(ctx context.Context, intc database.AIBridgeInterception, in *proto.RecordTokenUsageRequest) (tokenUsageCost, error) {
|
||||
var result tokenUsageCost
|
||||
|
||||
// Resolve the effective group for attribution. This is independent of
|
||||
// whether the model is priced. ok is false when no budget is configured,
|
||||
// which leaves the group attribution NULL.
|
||||
effectiveBudget, ok, err := budget.ResolveUserAIBudget(ctx, s.store, intc.InitiatorID, s.budgetPolicy)
|
||||
if err != nil {
|
||||
return tokenUsageCost{}, xerrors.Errorf("resolve effective AI budget for user %q with policy %q: %w", intc.InitiatorID, s.budgetPolicy, err)
|
||||
}
|
||||
if ok {
|
||||
result.effectiveGroupID = uuid.NullUUID{UUID: effectiveBudget.GroupID, Valid: true}
|
||||
}
|
||||
|
||||
// Snapshot the price for this (provider, model) and compute cost.
|
||||
price, err := s.store.GetAIModelPriceByProviderModel(ctx, database.GetAIModelPriceByProviderModelParams{
|
||||
Provider: intc.Provider,
|
||||
Model: intc.Model,
|
||||
})
|
||||
switch {
|
||||
case errors.Is(err, sql.ErrNoRows):
|
||||
// Model not in the price table: record tokens but leave cost NULL.
|
||||
s.logger.Debug(ctx, "no price found for model, recording token usage with NULL cost",
|
||||
slog.F("provider", intc.Provider), slog.F("model", intc.Model))
|
||||
return result, nil
|
||||
case err != nil:
|
||||
return tokenUsageCost{}, xerrors.Errorf("look up model price for %s/%s: %w", intc.Provider, intc.Model, err)
|
||||
}
|
||||
|
||||
result.inputPriceMicros = price.InputPrice
|
||||
result.outputPriceMicros = price.OutputPrice
|
||||
result.cacheReadPriceMicros = price.CacheReadPrice
|
||||
result.cacheWritePriceMicros = price.CacheWritePrice
|
||||
result.costMicros = sql.NullInt64{
|
||||
Int64: computeCost(price,
|
||||
in.GetInputTokens(), in.GetOutputTokens(),
|
||||
in.GetCacheReadInputTokens(), in.GetCacheWriteInputTokens()),
|
||||
Valid: true,
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// computeCost returns the cost of an interception in micro-units, snapshotting
|
||||
// the per-token prices from the price table. Prices are expressed per million
|
||||
// tokens; a NULL price column is treated as zero (e.g. providers that do not
|
||||
// charge for cache writes).
|
||||
func computeCost(price database.AIModelPrice, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens int64) int64 {
|
||||
return tokenCost(inputTokens, price.InputPrice) +
|
||||
tokenCost(outputTokens, price.OutputPrice) +
|
||||
tokenCost(cacheReadTokens, price.CacheReadPrice) +
|
||||
tokenCost(cacheWriteTokens, price.CacheWritePrice)
|
||||
}
|
||||
|
||||
// tokenCost returns tokens * price / 1,000,000, treating a NULL price as zero.
|
||||
func tokenCost(tokens int64, pricePerMillion sql.NullInt64) int64 {
|
||||
if !pricePerMillion.Valid {
|
||||
return 0
|
||||
}
|
||||
return tokens * pricePerMillion.Int64 / tokensPerMillion
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
package aibridgedserver
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"testing"
|
||||
|
||||
"github.com/coder/coder/v2/coderd/database"
|
||||
)
|
||||
|
||||
func TestComputeCost(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
nullInt64 := func(v int64) sql.NullInt64 { return sql.NullInt64{Int64: v, Valid: true} }
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
price database.AIModelPrice
|
||||
inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens int64
|
||||
want int64
|
||||
}{
|
||||
{
|
||||
name: "all priced",
|
||||
price: database.AIModelPrice{
|
||||
InputPrice: nullInt64(3_000_000),
|
||||
OutputPrice: nullInt64(6_000_000),
|
||||
CacheReadPrice: nullInt64(300_000),
|
||||
CacheWritePrice: nullInt64(3_750_000),
|
||||
},
|
||||
inputTokens: 100,
|
||||
outputTokens: 200,
|
||||
cacheReadTokens: 50,
|
||||
cacheWriteTokens: 10,
|
||||
// 300 + 1200 + 15 + 37 (10*3_750_000/1e6 = 37, integer division).
|
||||
want: 1552,
|
||||
},
|
||||
{
|
||||
name: "null cache write price treated as zero",
|
||||
price: database.AIModelPrice{
|
||||
InputPrice: nullInt64(3_000_000),
|
||||
OutputPrice: nullInt64(6_000_000),
|
||||
CacheReadPrice: nullInt64(300_000),
|
||||
CacheWritePrice: sql.NullInt64{Valid: false},
|
||||
},
|
||||
inputTokens: 100,
|
||||
outputTokens: 200,
|
||||
cacheReadTokens: 50,
|
||||
cacheWriteTokens: 10,
|
||||
// 300 + 1200 + 15 + 0.
|
||||
want: 1515,
|
||||
},
|
||||
{
|
||||
name: "all prices null is zero cost",
|
||||
price: database.AIModelPrice{},
|
||||
inputTokens: 100,
|
||||
outputTokens: 200,
|
||||
cacheReadTokens: 50,
|
||||
cacheWriteTokens: 10,
|
||||
want: 0,
|
||||
},
|
||||
{
|
||||
name: "zero tokens is zero cost",
|
||||
price: database.AIModelPrice{
|
||||
InputPrice: nullInt64(3_000_000),
|
||||
OutputPrice: nullInt64(6_000_000),
|
||||
},
|
||||
want: 0,
|
||||
},
|
||||
{
|
||||
name: "integer division truncates",
|
||||
price: database.AIModelPrice{
|
||||
// 1 token at 1 micro-unit per million tokens rounds down to 0.
|
||||
InputPrice: nullInt64(1),
|
||||
},
|
||||
inputTokens: 1,
|
||||
want: 0,
|
||||
},
|
||||
{
|
||||
name: "price just below one micro-unit per token floors to zero",
|
||||
price: database.AIModelPrice{
|
||||
InputPrice: nullInt64(999_999),
|
||||
},
|
||||
inputTokens: 1, // 1 * 999_999 = 999_999, below 1_000_000
|
||||
want: 0,
|
||||
},
|
||||
{
|
||||
name: "sub-unit price summed across tokens still floors to zero",
|
||||
price: database.AIModelPrice{
|
||||
InputPrice: nullInt64(999),
|
||||
},
|
||||
inputTokens: 1000, // 1000 * 999 = 999_000, below 1_000_000
|
||||
want: 0,
|
||||
},
|
||||
{
|
||||
name: "sub-unit price crosses one micro-unit once the product reaches 1e6",
|
||||
price: database.AIModelPrice{
|
||||
InputPrice: nullInt64(999),
|
||||
},
|
||||
inputTokens: 1002, // 1002 * 999 = 1_000_998
|
||||
want: 1,
|
||||
},
|
||||
{
|
||||
// Stress the per-term numerator near the int64 ceiling. At a $75/M
|
||||
// model the overflow point is ~123e9 tokens (123e9 * 75e6 = 9.225e18,
|
||||
// just over int64 max 9.223e18); 122e9 stays just under.
|
||||
name: "large token count at a high price does not overflow",
|
||||
price: database.AIModelPrice{
|
||||
InputPrice: nullInt64(75_000_000), // $75 per 1M tokens
|
||||
},
|
||||
inputTokens: 122_000_000_000, // 122e9 * 75e6 = 9.15e18 < int64 max
|
||||
want: 9_150_000_000_000,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
got := computeCost(tt.price, tt.inputTokens, tt.outputTokens, tt.cacheReadTokens, tt.cacheWriteTokens)
|
||||
if got != tt.want {
|
||||
t.Fatalf("computeCost = %d, want %d", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user