mirror of
https://github.com/coder/coder.git
synced 2026-09-23 14:03:57 +08:00
closes CODAGT-839 closes CODAGT-843 closes CODAGT-773 ## Summary Adds a new heartbeat usage event type, `hb_agent_runtime_v1`, measuring the total agent-loop runtime of Coder Agents (chats) per UTC hour, plus a reconciler that generates one event per hour with self-healing backfill over a trailing 7-day window. Events flow to Tallyman through the existing publisher unchanged. This measures the new Coder Agents (the `chats` tables), not the deprecated Tasks counted by `dc_managed_agents_v1`. Independent of #27508, which fixes the dead ai-seats cron registration. Both PRs carry the identical `usage_event` create permission hunk for the usage-publisher subject (this feature's generator and the ai-seats cron each need it for heartbeat inserts), so they can land in either order and the overlap merges cleanly. > [!WARNING] > **Do not include this in a release until Tallyman accepts `hb_agent_runtime_v1`.** The publisher marks permanently rejected events as done-forever, and the generator then sees those buckets as complete locally, so their usage would be silently and permanently lost. ## Details Each event's payload is `{"runtime_ms": N}`: the sum of `chat_messages.runtime_ms` for messages created in the hour bucket `[H, H+1)`, across all chats (sub-agents, API-created, archived, and soft-deleted messages included). Events use deterministic IDs (`hb_agent_runtime_v1:<bucket start>`) with `created_at` set to the bucket start, so concurrent replicas race safely via `ON CONFLICT (id) DO NOTHING` without locking, and daily rollups attribute backfilled hours to the correct day. Idle hours produce zero-valued events. A bucket becomes eligible 5 minutes after it closes; hours missing for longer than the 7-day window are forfeited, which can only undercount. Note that this makes `usage_events.created_at` explicitly the *event occurrence time* rather than the row insertion time; the two only diverge for backfilled events. It already behaved as the occurrence timestamp (it drives the daily rollup day and is shipped to Tallyman/Metronome as the event timestamp), and the migration now documents this with a `COMMENT ON COLUMN`, which also surfaces as a Go doc comment on `UsageEvent.CreatedAt`. The new `usage.Generator` runs unconditionally in enterprise builds; the `publish_usage_data` license flag continues to gate egress only, so air-gapped deployments still fill their local ledger. The `aggregate_usage_event()` trigger sums `runtime_ms` per day into `usage_events_daily` (unlike `hb_ai_seats_v1`, which takes the daily max). `InsertHeartbeatUsageEvent` now takes an explicit `createdAt` so generators can backfill historical buckets; the cron passes `clock.Now()` to preserve its existing behavior. ## Tallyman follow-up <details> <summary>Prompt for the Tallyman-repo agent</summary> > **Task**: Add support for the new Coder usage event type `hb_agent_runtime_v1` so Tallyman accepts, validates, and forwards it to Metronome. > > **Background**: coder/coder PR (this PR) adds hourly heartbeat events measuring Coder Agent runtime. Events arrive via the existing `/api/v1/events/ingest` endpoint with: `event_type: "hb_agent_runtime_v1"`, `event_data: {"runtime_ms": <int64 >= 0>}`, deterministic `id` of the form `hb_agent_runtime_v1:2026-07-15_14:00:00` (UTC hour bucket start), and `created_at` set to the bucket start (may be up to ~8 days in the past due to backfill; within Metronome's 34-day dedup window). Zero-value events are normal (idle hours). > > **Work**: > 1. Update Tallyman's vendored/imported `coderd/usage/usagetypes` (or equivalent) to the coder/coder commit that adds `UsageEventTypeHBAgentRuntimeV1` and `HBAgentRuntime`. > 2. Ensure ingestion validation accepts the type (`Valid()` switches) and rejects negative `runtime_ms`. > 3. Ensure Metronome forwarding maps the event with transaction ID derived from the event `id` as for existing types, passing `runtime_ms` through as the property for a SUM-aggregated billable metric ("Coder Agent Hours" = `SUM(runtime_ms) / 3,600,000`). > 4. Do NOT permanently reject unknown-but-well-formed future `hb_*` types if avoidable; at minimum confirm current behavior for unknown types (temporary vs permanent rejection) and report it. > 5. Tests: ingest accept/validate, dedup by ID, Metronome payload mapping. > > **Constraint**: this must be deployed to tallyman-prod **before** any coder/coder release containing the event generator; coderd treats permanent rejections as terminal per event. </details>
56 lines
2.5 KiB
PL/PgSQL
56 lines
2.5 KiB
PL/PgSQL
ALTER TABLE usage_events
|
|
DROP CONSTRAINT usage_event_type_check,
|
|
ADD CONSTRAINT usage_event_type_check CHECK (event_type IN ('dc_managed_agents_v1', 'hb_ai_seats_v1', 'hb_agent_runtime_v1'));
|
|
|
|
COMMENT ON COLUMN usage_events.created_at IS 'The time the usage occurred, which is not necessarily the time the row was inserted. Events that measure a time bucket (e.g. hb_agent_runtime_v1) always set this to the bucket start, regardless of when the row was inserted. This timestamp determines the day used by the daily rollup trigger and is sent to the usage collector service as the event timestamp.';
|
|
|
|
-- Serves the usage generator's scan for missing hourly buckets.
|
|
CREATE INDEX idx_usage_events_agent_runtime
|
|
ON usage_events (event_type, created_at)
|
|
WHERE event_type = 'hb_agent_runtime_v1';
|
|
|
|
CREATE OR REPLACE FUNCTION aggregate_usage_event()
|
|
RETURNS TRIGGER AS $$
|
|
BEGIN
|
|
-- Check for supported event types and throw error for unknown types.
|
|
IF NEW.event_type NOT IN ('dc_managed_agents_v1', 'hb_ai_seats_v1', 'hb_agent_runtime_v1') THEN
|
|
RAISE EXCEPTION 'Unhandled usage event type in aggregate_usage_event: %', NEW.event_type;
|
|
END IF;
|
|
|
|
INSERT INTO usage_events_daily (day, event_type, usage_data)
|
|
VALUES (
|
|
date_trunc('day', NEW.created_at AT TIME ZONE 'UTC')::date,
|
|
NEW.event_type,
|
|
NEW.event_data
|
|
)
|
|
ON CONFLICT (day, event_type) DO UPDATE SET
|
|
usage_data = CASE
|
|
-- Handle simple counter events by summing the count.
|
|
WHEN NEW.event_type IN ('dc_managed_agents_v1') THEN
|
|
jsonb_build_object(
|
|
'count',
|
|
COALESCE((usage_events_daily.usage_data->>'count')::bigint, 0) +
|
|
COALESCE((NEW.event_data->>'count')::bigint, 0)
|
|
)
|
|
-- Heartbeat events: keep the max value seen that day
|
|
WHEN NEW.event_type IN ('hb_ai_seats_v1') THEN
|
|
jsonb_build_object(
|
|
'count',
|
|
GREATEST(
|
|
COALESCE((usage_events_daily.usage_data->>'count')::bigint, 0),
|
|
COALESCE((NEW.event_data->>'count')::bigint, 0)
|
|
)
|
|
)
|
|
-- Hourly runtime heartbeats: sum the runtime per day.
|
|
WHEN NEW.event_type IN ('hb_agent_runtime_v1') THEN
|
|
jsonb_build_object(
|
|
'runtime_ms',
|
|
COALESCE((usage_events_daily.usage_data->>'runtime_ms')::bigint, 0) +
|
|
COALESCE((NEW.event_data->>'runtime_ms')::bigint, 0)
|
|
)
|
|
END;
|
|
|
|
RETURN NEW;
|
|
END;
|
|
$$ LANGUAGE plpgsql;
|