Thomas Brugman dcc0d64a32 feat(opencode): show token throughput metrics (#12434) ...

* feat(opencode): record per-step token throughput metrics

Capture prompt-processing and text-generation tokens/sec on every
StepFinishPart. The metrics helper prefers provider-reported rates
from llama.cpp / vLLM timings and falls back to wall-clock computation.
A new kilocode tui usage route renders PP/TG inline.

chore(sdk): regenerate types for StepFinishPart.metrics
feat(tui): render PP/TG in sidebar usage panel

feat(vscode): per-message and aggregated token throughput display

Surface throughput on each AssistantMessage badge (behind the
showTokenThroughput toggle) and as a compact PP/TG row in the
expanded TaskHeader. Adds session helpers, i18n entries in 20 locales,
and StepFinishPart.metrics to extension/webview messages.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(vscode): wire token throughput toggle and drop unreachable provider branch

- Replace the dead sendThroughputSetting() private with the shared
  buildThroughputSettingMessage() helper and add validThroughputSetting
  to handleUpdateSetting so the showTokenThroughput setting has the same
  guard as the chat/indexing twins (fixes Knip regression).
- Bind the DisplayTab Switch to the local settings draft so the toggle
  flips on click instead of waiting for a Save round-trip (the user-facing
  kill switch for #6579).
- Narrow StepThroughputMetrics.source to "computed"; backend hard-codes
  computed metrics today because the upstream AI SDK drops provider
  timings. Drop the unused provider branches from AssistantMessage and
  TaskHeader so the rendering code has no dead paths.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(opencode): centralize token throughput labels and tighten type guard

- Replace inline PP/TG labels in the CLI sidebar with a throughputLabel
  constant in model-usage so a future i18n sweep is one file instead
  of every rendering site.
- Tighten isStepMetrics in sidebar-usage back to a real discriminator
  check after dropping the unreachable "provider" union member.
- Drop formatPP/formatTG exports from model-usage since callers already
  use the shared formatRateValue; mirror the swap in the TUI usage test.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* feat(token-throughput-v2): wire aggregation through DisplayProvider and test computed-only

- aggregateMetrics adopts the first non-empty computed sample per field
  across every step-finish in the session, replacing the dead provider-
  ranked last-wins strategy that shipped with the unreachable branch.
- Share the throughputVisible signal through DisplayProvider so every
  AssistantMessage and the TaskHeader row react to a single onMount
  requestThroughputSetting round-trip, instead of each message posting
  its own handshake.
- Drop the unused routes/session/usage.tsx TUI route (no remaining
  imports) and add the chat-layout badge/header pill styles it was
  gating on.
- Refresh session-utils tests to exercise only source: "computed"
  samples and follow the new first-wins rule.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): seed settings on hydration, drop dead provider branch

Seed the DisplayTab "Show Token Throughput" Switch on initial load by
mirroring the throughputSettingLoaded message into settings() (same
pattern as chat.shiftTabCyclesVariant). Without this, a persisted-true
setting renders unchecked on first open because the Switch was bound to
settings()["showTokenThroughput"] but no handler ever populated it.

Drop the dead data-source attributes on the per-message and task-header
throughput surfaces now that StepThroughputMetrics.source is narrowed
to "computed" only — the attribute was always the literal string.

Drop the unreachable chat.throughput.badge.provider and
.chat.throughput.badge.tooltip.provider i18n keys across all 20 locales.
The badges no longer branch on source === "provider" since the
provider-source branch is removed (the AI SDK adapter upstream strips
llama.cpp timings before they reach providerMetadata).

Co-Authored-By: Claude <noreply@anthropic.com>

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): drop PP display until llama.cpp wiring lands

PP (prompt-processing rate) has no signal in this build: the AI SDK
adapter upstream strips llama.cpp's `prompt_per_second` before it reaches
providerMetadata, and computeMetrics has nothing else to derive it from.
Ship the TG (text-generation) rate only — the UI no longer renders the
"PP –" placeholder that made the feature look broken.

CLI sidebar drops the PP row; per-message badge and aggregated header
pill both lose the "PP – ·" prefix. The wire shape keeps the optional
prompt field so the follow-up that wires the upstream metadataExtractor
can populate it without another schema bump.

The `throughputLabel` constant on the opencode side and the `formatPP`
helper on the webview side are removed; tests that fabricated prompt
values are pruned to match.

Co-Authored-By: Claude <noreply@anthropic.com>

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): integrate TG into Tokens row, plain-text style

The standalone blue pills read as loud for what is secondary session
info. Move the aggregated TG into the existing Tokens row as another
spanned value (alongside ↑ input, ↑ cache, ↓ output) and restyle the
per-message badge as plain text in descriptionForeground so both surfaces
match the tokens family.

TaskUsage now accepts a `throughput` prop and renders `TG <rate> t/s`
inline in the Summary component when the toggle is on. TaskHeader no
longer emits a standalone [data-slot="task-header-throughput"] element;
its [data-slot="task-header-throughput"] CSS rule is removed. The
throughputText / throughputTooltip memos and the unused formatTG
import are dropped — the values flow straight into TaskUsage.

Co-Authored-By: Claude <noreply@anthropic.com>

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): finish removing memory badge from AssistantMessage

The "Merge origin/main into feat/token-throughput-v2" resolution kept the
throughput branch's memory-badge code (already removed from main by
28d015f8fe), which broke the kilo-ui-contract test and the i18n-keys test.

Drop the dead code: `useMemory`/`MemoryMarkerMeta` imports, `mem`, the
`meta`/`recall`/`fmt`/`count`/`items`/`verbose` createMemos, the `tip`
function, and the `<Show when={mem.enabled() && recall()}>` block. The
file lands at 349 lines (down from 391), matching main + throughput only.

Verified locally:
- i18n-keys + kilo-ui-contract: 53 pass, 0 fail
- Full kilo-vscode suite: failures 138 → 136 (+2 from the two fixes)

Co-Authored-By: Claude <noreply@anthropic.com>

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): finish removing memory badge from AssistantMessage

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* formatting fixes

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): address Marius review comments

- Rename 'TG' to 'Generation speed' in en.ts and add a 'gauge' icon to
  packages/kilo-ui so the per-message badge and the Tokens row show
  '<icon> Generation speed <rate> t/s' instead of the cryptic 'TG <rate>'.
  Centralize the opencode sidebar label in throughputLabel.generation.
- Switch aggregateMetrics (both webview and CLI) to the latest non-empty
  step-finish snapshot so only the most recent assistant turn's generation
  rate is shown rather than a session-wide aggregate. Update tests and
  comments to match.
- Translate the throughput strings in no.ts to Norwegian; mirror the new
  key shape across the other locales (English fallback for untranslated
  strings).

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): close unclosed CSS block and apply prettier formatting

The throughput rebases left a duplicated .vscode-session-turn-diffs
selector and let three files drift from prettier's expectations. Fix
the CSS unclosed-block (which broke the Storybook preview build) and
re-run prettier --write on the touched files.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* ci: re-run after fixing CSS unclosed-block + prettier drift

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): address kilo-code-bot review

Kilo:
- CRITICAL: Remove the section === '' guard in KiloProvider.ts that
  blocked persistence of every top-level setting key. The throughput
  validator is now redundant, so drop it from throughput-settings.ts.
- WARNING: Reset samples in sidebar-usage.tsx when props.session_id
  changes via a keyed createEffect, so a session switch no longer
  blends step-finish metrics from the previous session and the array
  no longer grows without bound across long-lived plugin instances.
- WARNING: Pass {speed} to language.t for the TaskUsage throughput
  tooltip and reuse the shared formatTG helper instead of reformatting
  the value inline. Drop the dead [data-component='assistant-memory-badge']
  rule whose target component no longer exists in the tree.
- SUGGESTION: Drop redundant guards in model-usage.ts (undefined check
  after Number.isFinite, and the ?? 0 on an already-required number
  field). Use typeof === 'number' for the type narrowing.

* feat(token-throughput-v2): weighted per-turn rate, plain text footer

Address Marius's review of the throughput UI:

Calculation
- Persist per-step timing (start/end/elapsed) on step-start and
  step-finish parts in the session processor.
- Add wire schemas in core/src/v1/session.ts and packages/sdk/openapi.json
  so the new time field round-trips end-to-end.
- Replace the last-wins 'latest step rate' snapshot with a weighted
  aggregate: sum(output + reasoning tokens) / sum(active generation
  duration) across the turn's step-finish parts. Tool execution and
  idle waiting are excluded.
- The CLI sidebar (model-usage.ts) gains the same weighted semantics
  when timing is available, falling back to last-wins otherwise so
  older callers keep working.

Presentation
- Strip the per-message badge to plain muted text (no icon, no label,
  no border). The chip in the upstream action row reads as metadata.
- Move throughput out of the task header Tokens row so each turn owns
  its own value (no flicker across turns, single source of truth).
- Read the throughput memo from the full message parts in the data
  store rather than the chunked row slice, so step-finish in any
  chunk produces the badge.

i18n
- Replace chat.throughput.speed.{label,row,tooltip,tooltip.missing}
  with chat.throughput.tooltip and chat.throughput.tooltip.missing
  across all 19 locale files.

Tests
- Add messageThroughput and sessionThroughput describe blocks
  exercising the weighted aggregate across multiple steps.
- Cover weighted + fallback paths in the CLI aggregateMetrics tests.

* fix(token-throughput-v2): render t/s inline beside copy/feedback buttons

Move the throughput badge from a footer line below the assistant message
into the copy/feedback action row of the text part that carries the copy
button. This avoids the extra vertical space the footer consumed.

Also apply prettier formatting to drifted PR files (i18n line wraps,
TaskHeader/session-utils/test reflows).

* fix(token-throughput-v2): correct changeset package name to @kilocode/cli

* fix(token-throughput-v2): annotate step-start time field with kilocode_change

---------

Co-authored-by: Thomas Brugman <thomas@kilocode.ai>
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
Co-authored-by: marius-kilocode <marius@kilocode.ai>

2026-07-23 09:43:13 +00:00

1.3 MiB

Raw Blame History