Commit Graph
5 Commits
Author SHA1 Message Date
Jaayden Halko fa8ffe4eda feat: report agent runtime hours usage in entitlements (#27985)
Populate `FeatureAgentRuntimeHours.Actual` on every entitlements refresh
for licenses that grant the feature. A new
`GetTotalUsageHBAgentRuntimeV1` query sums `runtime_ms` over the
license's usage period, reading `usage_events` directly:
`hb_agent_runtime_v1` is exactly one row per hourly bucket
deployment-wide with `created_at` at the bucket start, enforced by the
unique partial index introduced in #27983.

The measurement reuses the shared `measureUsage` policy from #27984
through a new `AgentRuntimeMsFn` closure (usage publisher subject):
failures publish the stable
`LicenseAgentRuntimeUsageUnavailableErrorText` and log the cause. Usage
is floored to whole hours, matching the unit of the
`agent_runtime_hours_*` claims, and at most one warning is emitted per
refresh: reaching the allocation supersedes the advisory soft limit. The
dashboard renders the soft-limit advisory muted without a sales link and
treats the runtime usage-unavailable text as a diagnostic.

**Precise usage.** `Feature.ActualMs` (JSON `actual_ms`), set only for
`agent_runtime_hours`, carries the exact stored milliseconds backing the
floored `Actual` so clients can render fractional hours (e.g. `10.3`).
It has the same freshness as `Actual`; the whole-hour warning thresholds
are unchanged.

**Unlimited licenses.** A license minted with the unlimited (`-1`)
allocation decodes to an enabled feature with a nil `Limit` (#27984), so
the warning write-back now guards the allocation dereference: no
thresholds can exist for an unlimited license, so no runtime hours
warning is ever emitted, while `Actual` is still measured and published.
`Feature.Compare` is unchanged; for usage-period features the
issued-at/end dates decide first, so a metered feature outranks an
unlimited one only on an exact timestamp tie, an edge pinned by a
`TestFeatureComparison` case and documented on
`decodeAgentRuntimeHours`.

**Grandfathered premium licenses.** Premium licenses without
`agent_runtime_hours_*` claims are now granted the feature disabled with
a zero limit over the license term, identical to an explicit
`allocation: 0`: usage is measured and published for every Premium
deployment, and chatd's pooled admission (#27902) caps concurrent
agentic chats until a license with a positive allocation is added. The
default carries a fixed early `UsagePeriod.IssuedAt` (2026-08-01, the
same mechanism as the managed-agents default) so any license actually
carrying the claims outranks it in the `AddFeature` merge regardless of
the licenses' relative issue dates; the constant must stay earlier than
the earliest legitimately issued claim-bearing license. Zero allocations
(explicit or grandfathered) emit no deployment-wide warning banner:
those deployments are steered by the in-page upgrade CTA and the
concurrency cap. Enterprise licenses are unchanged.

Part 3 of a 3-PR stack splitting up #27796 (see there for review
history). Stack: #27983 → #27984 → this PR.

Closes CODAGT-852.
2026-08-18 12:40:33 +07:00
Jaayden Halko 54d5eb7ec2 feat: add hourly hb_agent_runtime_v1 usage events for Coder Agent runtime (#27312)
closes CODAGT-839
closes CODAGT-843
closes CODAGT-773

## Summary

Adds a new heartbeat usage event type, `hb_agent_runtime_v1`, measuring
the total agent-loop runtime of Coder Agents (chats) per UTC hour, plus
a reconciler that generates one event per hour with self-healing
backfill over a trailing 7-day window. Events flow to Tallyman through
the existing publisher unchanged. This measures the new Coder Agents
(the `chats` tables), not the deprecated Tasks counted by
`dc_managed_agents_v1`.

Independent of #27508, which fixes the dead ai-seats cron registration.
Both PRs carry the identical `usage_event` create permission hunk for
the usage-publisher subject (this feature's generator and the ai-seats
cron each need it for heartbeat inserts), so they can land in either
order and the overlap merges cleanly.

> [!WARNING]
> **Do not include this in a release until Tallyman accepts
`hb_agent_runtime_v1`.** The publisher marks permanently rejected events
as done-forever, and the generator then sees those buckets as complete
locally, so their usage would be silently and permanently lost.

## Details

Each event's payload is `{"runtime_ms": N}`: the sum of
`chat_messages.runtime_ms` for messages created in the hour bucket `[H,
H+1)`, across all chats (sub-agents, API-created, archived, and
soft-deleted messages included). Events use deterministic IDs
(`hb_agent_runtime_v1:<bucket start>`) with `created_at` set to the
bucket start, so concurrent replicas race safely via `ON CONFLICT (id)
DO NOTHING` without locking, and daily rollups attribute backfilled
hours to the correct day. Idle hours produce zero-valued events. A
bucket becomes eligible 5 minutes after it closes; hours missing for
longer than the 7-day window are forfeited, which can only undercount.

Note that this makes `usage_events.created_at` explicitly the *event
occurrence time* rather than the row insertion time; the two only
diverge for backfilled events. It already behaved as the occurrence
timestamp (it drives the daily rollup day and is shipped to
Tallyman/Metronome as the event timestamp), and the migration now
documents this with a `COMMENT ON COLUMN`, which also surfaces as a Go
doc comment on `UsageEvent.CreatedAt`.

The new `usage.Generator` runs unconditionally in enterprise builds; the
`publish_usage_data` license flag continues to gate egress only, so
air-gapped deployments still fill their local ledger. The
`aggregate_usage_event()` trigger sums `runtime_ms` per day into
`usage_events_daily` (unlike `hb_ai_seats_v1`, which takes the daily
max).

`InsertHeartbeatUsageEvent` now takes an explicit `createdAt` so
generators can backfill historical buckets; the cron passes
`clock.Now()` to preserve its existing behavior.

## Tallyman follow-up

<details>
<summary>Prompt for the Tallyman-repo agent</summary>

> **Task**: Add support for the new Coder usage event type
`hb_agent_runtime_v1` so Tallyman accepts, validates, and forwards it to
Metronome.
>
> **Background**: coder/coder PR (this PR) adds hourly heartbeat events
measuring Coder Agent runtime. Events arrive via the existing
`/api/v1/events/ingest` endpoint with: `event_type:
"hb_agent_runtime_v1"`, `event_data: {"runtime_ms": <int64 >= 0>}`,
deterministic `id` of the form `hb_agent_runtime_v1:2026-07-15_14:00:00`
(UTC hour bucket start), and `created_at` set to the bucket start (may
be up to ~8 days in the past due to backfill; within Metronome's 34-day
dedup window). Zero-value events are normal (idle hours).
>
> **Work**:
> 1. Update Tallyman's vendored/imported `coderd/usage/usagetypes` (or
equivalent) to the coder/coder commit that adds
`UsageEventTypeHBAgentRuntimeV1` and `HBAgentRuntime`.
> 2. Ensure ingestion validation accepts the type (`Valid()` switches)
and rejects negative `runtime_ms`.
> 3. Ensure Metronome forwarding maps the event with transaction ID
derived from the event `id` as for existing types, passing `runtime_ms`
through as the property for a SUM-aggregated billable metric ("Coder
Agent Hours" = `SUM(runtime_ms) / 3,600,000`).
> 4. Do NOT permanently reject unknown-but-well-formed future `hb_*`
types if avoidable; at minimum confirm current behavior for unknown
types (temporary vs permanent rejection) and report it.
> 5. Tests: ingest accept/validate, dedup by ID, Metronome payload
mapping.
>
> **Constraint**: this must be deployed to tallyman-prod **before** any
coder/coder release containing the event generator; coderd treats
permanent rejections as terminal per event.

</details>
2026-07-30 08:37:45 +01:00
Steven Masley 84de391f26 chore: add tallyman events for ai seat tracking (#22689)
AI seat tracking inserted as heartbeat into usage table.
2026-03-18 09:30:22 -05:00
Dean Sheather 39bf3ba628 chore: replace GetManagedAgentCount query with aggregate table (#19636)
- Removes GetManagedAgentCount query
- Adds new table `usage_events_daily` which stores aggregated usage
events by the type and UTC day
- Adds trigger to update the values in this table when a new row is
inserted into `usage_events`
- Adds a migration that adds `usage_events_daily` rows for existing data
in `usage_events`
- Adds tests for the trigger
- Adds tests for the backfill query in the migration

Since the `usage_events` table is unreleased currently, this migration
will do nothing on real deployments and will only affect preview
deployments such as dogfood.

Closes https://github.com/coder/internal/issues/943
2025-08-30 03:39:37 +10:00
Dean Sheather a25d85631b chore: add usage tracking package (#19095)
Not used in coderd yet, see stack.

Adds two new packages:
- `coderd/usage`: provides an interface for the "Collector" as well as a stub implementation for AGPL
- `enterprise/coderd/usage`: provides an interface for the "Publisher" as well as a Tallyman implementation

Relates to https://github.com/coder/internal/issues/814
2025-08-16 01:31:00 +10:00