mirror of
https://github.com/coder/coder.git
synced 2026-09-22 13:10:21 +08:00
Ethan
d0e67a74d5
Closes CODAGT-352
This adds the Coder Agents experiments (virtual desktop with computer
use, and the advisor) to telemetry, so they finally show up in each
deployment snapshot. Everything stays inside `coderd/telemetry/`.
## Shape received by the telemetry server
The experiments are reported as a single `agents_experiments` field on
the deployment record, alongside the other config-derived deployment
fields. Its value is one JSON blob with one top-level key per
experiment:
```json
{
"virtual_desktop": {
"enabled": false,
"computer_use": {"provider": "anthropic", "provider_source": "default"}
},
"advisor": {"enabled": true, "max_uses_per_run": 5, "max_output_tokens": 4096, "provider": "openai", "model": "gpt-5.2"}
}
```
When the advisor falls back to the chat model, either because no
override is set or because the configured override is inactive (its
config or provider was deleted or disabled), the provider and model
carry a sentinel instead:
```json
"advisor": {"enabled": true, "max_uses_per_run": 5, "max_output_tokens": 4096, "provider": "advisor_reuse_chat_model", "model": "advisor_reuse_chat_model"}
```
- `virtual_desktop.enabled` and `advisor.enabled` track the
`chat-virtual-desktop` and `chat-advisor` deployment experiments, not
the stored config. We ignore the stored advisor `enabled` flag on
purpose: since #26809 the runtime gates on the experiment, and the
stored flag ends up permanently true for any deployment that ever opened
the settings form.
- Computer use sits under `virtual_desktop` rather than as its own
top-level key because it isn't a separate experiment; the same
`chat-virtual-desktop` flag gates both the desktop and the computer-use
provider. `provider_source` says whether an admin picked the provider
(`configured`) or we fell back to the default (`default`).
- `advisor.provider` is the `ai_providers` type (e.g. `openai`,
`anthropic`, `azure`) and `advisor.model` is the configured model
string. Two sentinels stand in when there's no concrete value:
`advisor_reuse_chat_model` when the advisor has no active override and
falls back to the chat model (matching the runtime), and `unknown` when
we genuinely couldn't tell, e.g. a query failed or the stored config
wouldn't parse.
- `advisor.max_uses_per_run` and `advisor.max_output_tokens` are clamped
to 0 before reporting, matching how the API normalizes these values on
read.
## Why this shape
Putting the data on the deployment record keeps it next to the other
config-derived fields, and leaves `telemetry_items` as a faithful mirror
of the `telemetry_items` table rather than a place we inject synthetic
rows. Adding or removing an experiment is a one-line edit to the
`agentsExperiments` registry. The `agents_experiments` field itself
never changes; only the JSON inside it does. The field is `omitempty`,
so older Coder versions that don't emit it are distinguishable from a
real absence, and when an experiment isn't reported in a snapshot its
JSON path is simply missing, so queries can tell "not reported" apart
from a real `false`.
One key holding one JSON blob is also easier to query than many separate
fields. Because everything lives in one blob, a question like "of the
deployments running the desktop, how many changed the computer-use
provider?" is one query with no join:
```sql
SELECT
JSON_VALUE(agents_experiments, '$.virtual_desktop.computer_use.provider_source') AS src,
COUNT(*) AS deployments
FROM deployments
WHERE JSON_VALUE(agents_experiments, '$.virtual_desktop.enabled') = 'true'
GROUP BY src
```