Yevhenii Shcherbina
4124d1137d
# Summary Implements https://linear.app/codercom/issue/AIGOV-282/add-ai-model-price-table-and-seed-generator This PR lays the groundwork for AI Bridge cost controls (per the AI Governance RFC). It adds the foundation needed for future cost tracking: a place to store per-model token prices, a way to keep those prices in sync with upstream pricing data, and a startup mechanism that ensures every deployment has prices loaded before AI Bridge starts processing requests. The price data comes from [models.dev](https://models.dev/), a community-maintained catalogue of AI provider pricing. A generator script fetches the latest prices, filters to Anthropic and OpenAI for now, and produces a seed file checked into the repository. On every server startup the seed is applied to the database, so new releases automatically pick up any price corrections that landed since the previous one. Existing rows are overwritten with the latest prices; rows for models no longer in the seed are left untouched. # Batching the AI model price seed: three approaches Context: at server startup we seed the `ai_model_prices` table from an embedded JSON price book (~70 rows today, will grow as we add providers, potentially 4000+). Each row is: ```text (provider, model, input_price, output_price, cache_read_price, cache_write_price) ``` Any of the four price columns can be: - `NULL` → “price unknown for this dimension” - explicit `0` → “free” The batch must be an UPSERT so re-running is idempotent and existing rows pick up new prices. We considered three implementations. --- ## Approach 1 — Per-row UPSERT in a Go loop ```go for _, row := range rows { if err := db.UpsertAIModelPrice(ctx, database.UpsertAIModelPriceParams{ Provider: row.Provider, Model: row.Model, InputPrice: nullInt64(row.InputPrice), // ... }); err != nil { return err } } ``` ### Pros - Trivial. - NULL handling falls out naturally from `sql.NullInt64`. ### Cons - `N` round-trips per seed. - With ~70 rows that means ~70 statement executions on every startup, even inside a transaction. - Doesn't scale gracefully as the price book grows, potentially 4000+. --- ## Approach 2 — `UNNEST` with parallel arrays Pass each column as a separate Go slice. Postgres unnests them in parallel into a virtual table, then `INSERT ... SELECT`. ```sql INSERT INTO ai_model_prices ( provider, model, input_price, output_price, cache_read_price, cache_write_price ) SELECT UNNEST(@providers::text[]), UNNEST(@models::text[]), NULLIF(UNNEST(@input_prices::bigint[]), -1), NULLIF(UNNEST(@output_prices::bigint[]), -1), NULLIF(UNNEST(@cache_read_prices::bigint[]), -1), NULLIF(UNNEST(@cache_write_prices::bigint[]), -1) ON CONFLICT (provider, model) DO UPDATE SET input_price = EXCLUDED.input_price, output_price = EXCLUDED.output_price, cache_read_price = EXCLUDED.cache_read_price, cache_write_price = EXCLUDED.cache_write_price, updated_at = NOW(); ``` Go side: flatten rows into six parallel slices. Use a sentinel (`-1`) for “missing”, since `lib/pq` can't encode `NULL` into a `bigint[]` element. ```go providers := make([]string, len(rows)) models := make([]string, len(rows)) inputs := make([]int64, len(rows)) outputs := make([]int64, len(rows)) cacheR := make([]int64, len(rows)) cacheW := make([]int64, len(rows)) for i, r := range rows { providers[i] = r.Provider models[i] = r.Model inputs[i] = -1 if r.InputPrice != nil { inputs[i] = *r.InputPrice } outputs[i] = -1 if r.OutputPrice != nil { outputs[i] = *r.OutputPrice } cacheR[i] = -1 if r.CacheReadPrice != nil { cacheR[i] = *r.CacheReadPrice } cacheW[i] = -1 if r.CacheWritePrice != nil { cacheW[i] = *r.CacheWritePrice } } return db.UpsertAIModelPrices(ctx, database.UpsertAIModelPricesParams{ Providers: providers, Models: models, InputPrices: inputs, OutputPrices: outputs, CacheReadPrices: cacheR, CacheWritePrices: cacheW, }) ``` ### Pros - Single round-trip. ### Cons - The generated `sqlc` params become plain `[]int64`, which can't represent `NULL`. --- ## Approach 3 — `jsonb_array_elements` over a single `@seed::jsonb` (chosen) Pass the raw seed JSON as one parameter; let Postgres expand and parse it. ```sql INSERT INTO ai_model_prices ( provider, model, input_price, output_price, cache_read_price, cache_write_price ) SELECT elem->>'provider', elem->>'model', (elem->>'input_price')::bigint, (elem->>'output_price')::bigint, (elem->>'cache_read_price')::bigint, (elem->>'cache_write_price')::bigint FROM jsonb_array_elements(@seed::jsonb) AS elem ON CONFLICT (provider, model) DO UPDATE SET input_price = EXCLUDED.input_price, output_price = EXCLUDED.output_price, cache_read_price = EXCLUDED.cache_read_price, cache_write_price = EXCLUDED.cache_write_price, updated_at = NOW(); ``` Go side reduces to: ```go return db.UpsertAIModelPrices(ctx, seedJSON) ``` ### Pros - Single round-trip. - NULLs fall out naturally: - `(elem->>'cache_write_price')::bigint` becomes `NULL` - no sentinels - The seed is already JSON: - Existing precedent: - `jsonb_array_elements` is already used elsewhere in the codebase ### Cons - Less type-safe at the SQL boundary than `UNNEST` - Slightly less standard than `UNNEST` - Readers need familiarity with: - `jsonb_array_elements` - `->>` extraction syntax - Postgres pays JSON parse cost - negligible at our scale --- --- # Decision We picked Approach 3. It collapses the round-trips like `UNNEST` does, but without: - nullable-array workarounds - sentinel values
About
Coder is a self-hosted platform for running AI coding agents and cloud development environments on infrastructure you control. It works with any cloud, IDE, OS, Git provider, and IDP.
Coder Workspaces
Coder Workspaces are cloud development environments defined with Terraform, connected through a secure Wireguard tunnel, and automatically shut down when not in use. Agents and developers share the same workspace infrastructure.
- Defined in Terraform: Templates describe the infrastructure for each workspace, from EC2 VMs and Kubernetes Pods to Docker containers.
- Any architecture and OS: Support ARM and x86-64 across Windows, Linux, and macOS from a single deployment.
- Managed by admins: Platform teams create and maintain templates that enforce approved images, resource limits, and security policies.
- Accessed from any IDE: Connect through VS Code, JetBrains, Cursor, a web terminal, remote desktop, or SSH.
- Automatic shutdown: Idle workspaces stop automatically to reduce cloud spend, and restart in seconds when needed.
Coder Agents
Coder Agents is a native AI coding agent built
into Coder. The agent loop runs in the Coder control plane on your
infrastructure, not in the workspace and not in a vendor's cloud. Developers
interact with agents through the web UI, the CLI (coder agents), or the REST
API for programmatic and CI-driven workflows.
- Self-hosted agent loop: The control plane handles planning, model calls, and tool dispatch. Workspaces have zero AI awareness.
- No API keys in workspaces: LLM credentials stay in the control plane.
- Any model: Anthropic, OpenAI, Google, Bedrock, or self-hosted endpoints. Switching is a configuration change.
- Governance and cost controls: Centralized model approval, per-user spend limits, and audit logging.
- Open source and inspectable: The full platform is available to audit and extend.
IDE support
You can use:
-
Any Web IDE, such as
- code-server
- JetBrains Projector
- Jupyter
- And others
-
Your existing remote development environment:
-
A file sync such as Mutagen
Why remote development
Provisioning consistent development environments for a large engineering team is difficult. Each developer has preferences for operating systems, editors, and toolchains, and ensuring a reliable build environment across all of them is a maintenance burden. A missed step during onboarding or an unsupported local configuration can cost hours of debugging.
Remote development solves this by moving the environment off the developer's machine and into managed infrastructure. The developer's laptop becomes a portal into the actual compute where work happens. If a device is lost or replaced, access is simply revoked; no source code or credentials are stored locally.
This approach provides:
- Speed: Server-grade hardware accelerates builds, tests, and large workloads without requiring expensive local machines.
- Consistency: Infrastructure tools such as Terraform, nix, Docker, and Dev Containers produce identical environments for every developer.
- Security: Source code stays on private servers. Users and groups are managed through SSO and RBAC.
- Compatibility: Workspaces share infrastructure configurations with staging and production, reducing configuration drift.
- Accessibility: Browser-based IDEs and remote IDE extensions let developers work from any device, including lightweight laptops, Chromebooks, and tablets.
Read more on the Coder blog, the Slack engineering blog, or from Alex Ellis at OpenFaaS.
Why Coder
The key difference between Coder and other platforms is that the entire system, agent loop, control plane, model routing, and workspace provisioning, runs on infrastructure you control.
For agents, this means platform teams can:
- Run the entire agent loop on their infrastructure, with no SaaS dependency for orchestration.
- Define MCP servers, skills, and system prompts centrally so every agent session starts with the same tools, policies, and context.
- Keep LLM credentials out of workspaces entirely.
- Tie every agent action to an authenticated user identity.
- Support air-gapped and restricted-network deployments with self-hosted models.
For workspaces, this means admins can:
- Support any architecture (ARM, x86-64) and operating system (Windows, Linux, macOS).
- Modify pod/container specs, such as adding disks, managing network policies, or setting/updating environment variables.
- Use VM or dedicated workspaces, developing with Kernel features (no container knowledge required).
- Enable persistent workspaces, which are like local machines, but faster and hosted by a cloud service.
Pricing
Coder is free and open source under the GNU Affero General Public License v3.0. All developer productivity features are included in the open source version. A Premium license is available for enhanced support and custom deployments.
How Coder works
Coder workspaces are represented with Terraform, but you do not need to know Terraform to get started. The Coder Registry provides production-ready templates for AWS EC2, Azure, Google Cloud, Kubernetes, and other providers.
Providers and compute environments
Workspaces can include more than just compute. Terraform can add storage buckets, secrets, sidecars, and other resources.
See the templates documentation for details.
What Coder is not
-
Coder is not an infrastructure as code (IaC) platform.
- Terraform is the first IaC provisioner in Coder, allowing Coder admins to define Terraform resources as Coder workspaces.
-
Coder is not a DevOps/CI platform.
- Coder workspaces can be configured to follow best practices for cloud-service-based workloads, but Coder is not responsible for how you define or deploy the software you write.
-
Coder is not an online IDE.
- Coder supports common editors, such as VS Code, vim, and JetBrains, all over HTTPS or SSH.
-
Coder is not a collaboration platform.
- You can use Git with your favorite Git platform and dedicated IDE extensions for pull requests, code reviews, and pair programming.
-
Coder is not a SaaS/fully-managed offering.
- Coder is a self-hosted solution. You must host Coder in a private data center or on a cloud service, such as AWS, Azure, or GCP.