opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: add experimental CLI to price unpriced AI models (#27926)

Browse Source 
## Description

AI Gateway computes the cost of an interception from `ai_model_prices`,
which is seeded on every server start from a price book embedded in the
binary. A model the price book does not cover records a NULL cost, so
its spend is invisible to cost reporting and is not enforced against
budgets. The only fix was to wait for a Coder release that added the
model.

This adds an experimental CLI, backed by an experimental HTTP endpoint,
for pricing those models. Models the price book already covers are
rejected, because the seeder re-applies the book on every start and
would overwrite an operator price. Support for custom pricing will be
handled in
https://linear.app/codercom/issue/AIGOV-589/extend-experimental-cli-command-to-set-custom-ai-model-prices.

## Commands

```
coder exp ai-model-prices list [--provider] [--model]
coder exp ai-model-prices update [file|-] [--provider] [--model] [--input-price] [--output-price] [--cache-read-price] [--cache-write-price] [--yes]
```

## Changes

- Add `GET` and `POST /api/experimental/ai/model-prices`, gated behind
the AI Bridge entitlement and the existing `ai_model_price` RBAC
resource.
- Add a `GetAIModelPrices` query with optional `provider` and `model`
filters applied in SQL.
- Validate the whole request before writing anything, so one bad entry
cannot leave the table half updated, and report every problem at once.
- Reject prices for models the embedded price book already covers,
through a new `prices.IsDefaultPriced`.
- Add the `coder exp ai-model-prices` command with `list` and `update`.
`update` accepts a JSON document or the single-model flags and prints a
plan, asking to confirm unless the document is piped in or `--yes` is
passed.
- Consolidate the supported provider list into
`coderd/aibridge/prices/providers` so the price generator and the server
share one definition.
- Add `codersdk` types and client methods for both endpoints, and bound
the request body at 1 MiB.
- Document the command in the AI Gateway cost controls page.

Closes
https://linear.app/codercom/issue/AIGOV-567/experimental-cli-command-to-set-prices-for-unpriced-ai-models

> [!NOTE]
> Initially generated by Claude Opus 5, modified and reviewed by
@ssncferreira
This commit is contained in:
Susana Ferreira
2026-08-13 15:00:36 +01:00
committed by GitHub
parent abe9c79605
commit 2d9b6eda8f
27 changed files with 2828 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ai-coder/ai-gateway/cost-controls.md
+40 -1
View File
@@ -212,7 +212,46 @@ Replace `<VERSION>` with your Coder minor version, for example `2.36`.
Monitor `coder_ai_gateway_cost_control_unpriced_token_usage_records_total`,
labeled by `provider` and `model`, to detect unpriced usage. Any non-zero value
means spend is under-counted. Because the price book ships with the release, a
newly launched model can remain unpriced until you upgrade Coder.
newly launched model is unpriced until you upgrade Coder or set a price for it
yourself.
### Set model prices
Use the experimental `coder exp ai-model-prices` command to set prices for
models the price book does not cover. It requires the AI Governance add-on and
the `ai_model_price:update` permission. Run
`coder exp ai-model-prices --help` for the full reference.
List the prices this deployment holds, optionally narrowed to one provider or
model:
```sh
coder exp ai-model-prices list --provider anthropic
```
Price a single model. Prices are micro-units per million tokens, so `3000000`
is $3.00 per million tokens. Use `null` for a price you do not have, and `0` to
declare a model free:
```sh
coder exp ai-model-prices update --provider anthropic --model my-model \
--input-price 3000000 --output-price 15000000 \
--cache-read-price null --cache-write-price null
```
Price several models at once from a JSON document in the same shape as the
price book:
```sh
coder exp ai-model-prices update prices.json
```
> [!IMPORTANT]
>
> - Prices are not retroactive. Usage recorded before you set a price stays
> unpriced, so past spend does not change.
> - You can only set prices for models the price book does not cover.
> - This command is experimental and can change without notice.
## Monitor spend
docs/reference/api/schemas.md
Generated
+75
View File
@@ -1138,6 +1138,58 @@ title: Schemas
| `last_heartbeat_at` | string | false | | |
| `name` | string | false | | |
## codersdk.AIModelPrice
```json
{
"cache_read_price": 0,
"cache_write_price": 0,
"created_at": "2019-08-24T14:15:22Z",
"input_price": 0,
"model": "string",
"output_price": 0,
"provider": "string",
"updated_at": "2019-08-24T14:15:22Z"
}
```
### Properties
| Name | Type | Required | Restrictions | Description |
|---------------------|---------|----------|--------------|-------------|
| `cache_read_price` | integer | false | | |
| `cache_write_price` | integer | false | | |
| `created_at` | string | false | | |
| `input_price` | integer | false | | |
| `model` | string | false | | |
| `output_price` | integer | false | | |
| `provider` | string | false | | |
| `updated_at` | string | false | | |
## codersdk.AIModelPriceUpsert
```json
{
"cache_read_price": 0,
"cache_write_price": 0,
"input_price": 0,
"model": "string",
"output_price": 0,
"provider": "string"
}
```
### Properties
| Name | Type | Required | Restrictions | Description |
|---------------------|---------|----------|--------------|-------------|
| `cache_read_price` | integer | false | | |
| `cache_write_price` | integer | false | | |
| `input_price` | integer | false | | |
| `model` | string | false | | |
| `output_price` | integer | false | | |
| `provider` | string | false | | |
## codersdk.AIProvider
```json
@@ -14464,6 +14516,29 @@ If the schedule is empty, the user will be updated to use the default schedule.|
|--------|--------|----------|--------------|-------------|
| `hash` | string | false | | |
## codersdk.UpsertAIModelPricesRequest
```json
{
"prices": [
{
"cache_read_price": 0,
"cache_write_price": 0,
"input_price": 0,
"model": "string",
"output_price": 0,
"provider": "string"
}
]
}
```
### Properties
| Name | Type | Required | Restrictions | Description |
|----------|---------------------------------------------------------------------|----------|--------------|-------------|
| `prices` | array of [codersdk.AIModelPriceUpsert](#codersdkaimodelpriceupsert) | false | | |
## codersdk.UpsertGroupAIBudgetRequest
```json