feat(opencode): show token throughput metrics (#12434)

* feat(opencode): record per-step token throughput metrics

Capture prompt-processing and text-generation tokens/sec on every
StepFinishPart. The metrics helper prefers provider-reported rates
from llama.cpp / vLLM timings and falls back to wall-clock computation.
A new kilocode tui usage route renders PP/TG inline.

chore(sdk): regenerate types for StepFinishPart.metrics
feat(tui): render PP/TG in sidebar usage panel

feat(vscode): per-message and aggregated token throughput display

Surface throughput on each AssistantMessage badge (behind the
showTokenThroughput toggle) and as a compact PP/TG row in the
expanded TaskHeader. Adds session helpers, i18n entries in 20 locales,
and StepFinishPart.metrics to extension/webview messages.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(vscode): wire token throughput toggle and drop unreachable provider branch

- Replace the dead sendThroughputSetting() private with the shared
  buildThroughputSettingMessage() helper and add validThroughputSetting
  to handleUpdateSetting so the showTokenThroughput setting has the same
  guard as the chat/indexing twins (fixes Knip regression).
- Bind the DisplayTab Switch to the local settings draft so the toggle
  flips on click instead of waiting for a Save round-trip (the user-facing
  kill switch for #6579).
- Narrow StepThroughputMetrics.source to "computed"; backend hard-codes
  computed metrics today because the upstream AI SDK drops provider
  timings. Drop the unused provider branches from AssistantMessage and
  TaskHeader so the rendering code has no dead paths.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(opencode): centralize token throughput labels and tighten type guard

- Replace inline PP/TG labels in the CLI sidebar with a throughputLabel
  constant in model-usage so a future i18n sweep is one file instead
  of every rendering site.
- Tighten isStepMetrics in sidebar-usage back to a real discriminator
  check after dropping the unreachable "provider" union member.
- Drop formatPP/formatTG exports from model-usage since callers already
  use the shared formatRateValue; mirror the swap in the TUI usage test.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* feat(token-throughput-v2): wire aggregation through DisplayProvider and test computed-only

- aggregateMetrics adopts the first non-empty computed sample per field
  across every step-finish in the session, replacing the dead provider-
  ranked last-wins strategy that shipped with the unreachable branch.
- Share the throughputVisible signal through DisplayProvider so every
  AssistantMessage and the TaskHeader row react to a single onMount
  requestThroughputSetting round-trip, instead of each message posting
  its own handshake.
- Drop the unused routes/session/usage.tsx TUI route (no remaining
  imports) and add the chat-layout badge/header pill styles it was
  gating on.
- Refresh session-utils tests to exercise only source: "computed"
  samples and follow the new first-wins rule.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): seed settings on hydration, drop dead provider branch

Seed the DisplayTab "Show Token Throughput" Switch on initial load by
mirroring the throughputSettingLoaded message into settings() (same
pattern as chat.shiftTabCyclesVariant). Without this, a persisted-true
setting renders unchecked on first open because the Switch was bound to
settings()["showTokenThroughput"] but no handler ever populated it.

Drop the dead data-source attributes on the per-message and task-header
throughput surfaces now that StepThroughputMetrics.source is narrowed
to "computed" only — the attribute was always the literal string.

Drop the unreachable chat.throughput.badge.provider and
.chat.throughput.badge.tooltip.provider i18n keys across all 20 locales.
The badges no longer branch on source === "provider" since the
provider-source branch is removed (the AI SDK adapter upstream strips
llama.cpp timings before they reach providerMetadata).

Co-Authored-By: Claude <noreply@anthropic.com>

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): drop PP display until llama.cpp wiring lands

PP (prompt-processing rate) has no signal in this build: the AI SDK
adapter upstream strips llama.cpp's `prompt_per_second` before it reaches
providerMetadata, and computeMetrics has nothing else to derive it from.
Ship the TG (text-generation) rate only — the UI no longer renders the
"PP –" placeholder that made the feature look broken.

CLI sidebar drops the PP row; per-message badge and aggregated header
pill both lose the "PP – ·" prefix. The wire shape keeps the optional
prompt field so the follow-up that wires the upstream metadataExtractor
can populate it without another schema bump.

The `throughputLabel` constant on the opencode side and the `formatPP`
helper on the webview side are removed; tests that fabricated prompt
values are pruned to match.

Co-Authored-By: Claude <noreply@anthropic.com>

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): integrate TG into Tokens row, plain-text style

The standalone blue pills read as loud for what is secondary session
info. Move the aggregated TG into the existing Tokens row as another
spanned value (alongside ↑ input, ↑ cache, ↓ output) and restyle the
per-message badge as plain text in descriptionForeground so both surfaces
match the tokens family.

TaskUsage now accepts a `throughput` prop and renders `TG <rate> t/s`
inline in the Summary component when the toggle is on. TaskHeader no
longer emits a standalone [data-slot="task-header-throughput"] element;
its [data-slot="task-header-throughput"] CSS rule is removed. The
throughputText / throughputTooltip memos and the unused formatTG
import are dropped — the values flow straight into TaskUsage.

Co-Authored-By: Claude <noreply@anthropic.com>

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): finish removing memory badge from AssistantMessage

The "Merge origin/main into feat/token-throughput-v2" resolution kept the
throughput branch's memory-badge code (already removed from main by
28d015f8fe), which broke the kilo-ui-contract test and the i18n-keys test.

Drop the dead code: `useMemory`/`MemoryMarkerMeta` imports, `mem`, the
`meta`/`recall`/`fmt`/`count`/`items`/`verbose` createMemos, the `tip`
function, and the `<Show when={mem.enabled() && recall()}>` block. The
file lands at 349 lines (down from 391), matching main + throughput only.

Verified locally:
- i18n-keys + kilo-ui-contract: 53 pass, 0 fail
- Full kilo-vscode suite: failures 138 → 136 (+2 from the two fixes)

Co-Authored-By: Claude <noreply@anthropic.com>

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): finish removing memory badge from AssistantMessage

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* formatting fixes

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): address Marius review comments

- Rename 'TG' to 'Generation speed' in en.ts and add a 'gauge' icon to
  packages/kilo-ui so the per-message badge and the Tokens row show
  '<icon> Generation speed <rate> t/s' instead of the cryptic 'TG <rate>'.
  Centralize the opencode sidebar label in throughputLabel.generation.
- Switch aggregateMetrics (both webview and CLI) to the latest non-empty
  step-finish snapshot so only the most recent assistant turn's generation
  rate is shown rather than a session-wide aggregate. Update tests and
  comments to match.
- Translate the throughput strings in no.ts to Norwegian; mirror the new
  key shape across the other locales (English fallback for untranslated
  strings).

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): close unclosed CSS block and apply prettier formatting

The throughput rebases left a duplicated .vscode-session-turn-diffs
selector and let three files drift from prettier's expectations. Fix
the CSS unclosed-block (which broke the Storybook preview build) and
re-run prettier --write on the touched files.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* ci: re-run after fixing CSS unclosed-block + prettier drift

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>

* fix(token-throughput-v2): address kilo-code-bot review

Kilo:
- CRITICAL: Remove the section === '' guard in KiloProvider.ts that
  blocked persistence of every top-level setting key. The throughput
  validator is now redundant, so drop it from throughput-settings.ts.
- WARNING: Reset samples in sidebar-usage.tsx when props.session_id
  changes via a keyed createEffect, so a session switch no longer
  blends step-finish metrics from the previous session and the array
  no longer grows without bound across long-lived plugin instances.
- WARNING: Pass {speed} to language.t for the TaskUsage throughput
  tooltip and reuse the shared formatTG helper instead of reformatting
  the value inline. Drop the dead [data-component='assistant-memory-badge']
  rule whose target component no longer exists in the tree.
- SUGGESTION: Drop redundant guards in model-usage.ts (undefined check
  after Number.isFinite, and the ?? 0 on an already-required number
  field). Use typeof === 'number' for the type narrowing.

* feat(token-throughput-v2): weighted per-turn rate, plain text footer

Address Marius's review of the throughput UI:

Calculation
- Persist per-step timing (start/end/elapsed) on step-start and
  step-finish parts in the session processor.
- Add wire schemas in core/src/v1/session.ts and packages/sdk/openapi.json
  so the new time field round-trips end-to-end.
- Replace the last-wins 'latest step rate' snapshot with a weighted
  aggregate: sum(output + reasoning tokens) / sum(active generation
  duration) across the turn's step-finish parts. Tool execution and
  idle waiting are excluded.
- The CLI sidebar (model-usage.ts) gains the same weighted semantics
  when timing is available, falling back to last-wins otherwise so
  older callers keep working.

Presentation
- Strip the per-message badge to plain muted text (no icon, no label,
  no border). The chip in the upstream action row reads as metadata.
- Move throughput out of the task header Tokens row so each turn owns
  its own value (no flicker across turns, single source of truth).
- Read the throughput memo from the full message parts in the data
  store rather than the chunked row slice, so step-finish in any
  chunk produces the badge.

i18n
- Replace chat.throughput.speed.{label,row,tooltip,tooltip.missing}
  with chat.throughput.tooltip and chat.throughput.tooltip.missing
  across all 19 locale files.

Tests
- Add messageThroughput and sessionThroughput describe blocks
  exercising the weighted aggregate across multiple steps.
- Cover weighted + fallback paths in the CLI aggregateMetrics tests.

* fix(token-throughput-v2): render t/s inline beside copy/feedback buttons

Move the throughput badge from a footer line below the assistant message
into the copy/feedback action row of the text part that carries the copy
button. This avoids the extra vertical space the footer consumed.

Also apply prettier formatting to drifted PR files (i18n line wraps,
TaskHeader/session-utils/test reflows).

* fix(token-throughput-v2): correct changeset package name to @kilocode/cli

* fix(token-throughput-v2): annotate step-start time field with kilocode_change

---------

Co-authored-by: Thomas Brugman <thomas@kilocode.ai>
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
Co-authored-by: marius-kilocode <marius@kilocode.ai>
This commit is contained in:
Thomas Brugman
2026-07-23 11:43:13 +02:00
committed by GitHub
parent 8eeaa546ae
commit dcc0d64a32
48 changed files with 1284 additions and 14 deletions
+6
View File
@@ -0,0 +1,6 @@
---
"@kilocode/cli": minor
"@kilocode/sdk": minor
---
Show tokens-per-second text-generation throughput (TG) on each assistant message and in the usage sidebar, computed from step duration and tokens. The toggle "Show Token Throughput" in Display settings controls both surfaces. PP (prompt-processing) support lands in a follow-up once the upstream llama.cpp metadata wiring ships.
+10 -10
View File
@@ -844,22 +844,22 @@
},
},
"trustedDependencies": [
"web-tree-sitter",
"esbuild",
"tree-sitter-bash",
"protobufjs",
"web-tree-sitter",
"tree-sitter-bash",
],
"patchedDependencies": {
"@silvia-odwyer/photon-node@0.3.4": "patches/@silvia-odwyer%2Fphoton-node@0.3.4.patch",
"virtua@0.49.1": "patches/virtua@0.49.1.patch",
"mammoth@1.12.0": "patches/mammoth@1.12.0.patch",
"@ff-labs/fff-bun@0.9.4": "patches/@ff-labs%2Ffff-bun@0.9.4.patch",
"@standard-community/standard-openapi@0.2.9": "patches/@standard-community%2Fstandard-openapi@0.2.9.patch",
"@modelcontextprotocol/sdk@1.29.0": "patches/@modelcontextprotocol%2Fsdk@1.29.0.patch",
"@npmcli/agent@4.0.2": "patches/@npmcli%2Fagent@4.0.2.patch",
"@ai-sdk/google@3.0.73": "patches/@ai-sdk%2Fgoogle@3.0.73.patch",
"pacote@21.5.1": "patches/pacote@21.5.1.patch",
"virtua@0.49.1": "patches/virtua@0.49.1.patch",
"@ai-sdk/xai@3.0.102": "patches/@ai-sdk%2Fxai@3.0.102.patch",
"@modelcontextprotocol/sdk@1.29.0": "patches/@modelcontextprotocol%2Fsdk@1.29.0.patch",
"@ai-sdk/google@3.0.73": "patches/@ai-sdk%2Fgoogle@3.0.73.patch",
"@ff-labs/fff-bun@0.9.4": "patches/@ff-labs%2Ffff-bun@0.9.4.patch",
"pacote@21.5.1": "patches/pacote@21.5.1.patch",
"@standard-community/standard-openapi@0.2.9": "patches/@standard-community%2Fstandard-openapi@0.2.9.patch",
"@npmcli/agent@4.0.2": "patches/@npmcli%2Fagent@4.0.2.patch",
"@silvia-odwyer/photon-node@0.3.4": "patches/@silvia-odwyer%2Fphoton-node@0.3.4.patch",
},
"overrides": {
"@effect/platform-node-shared": "4.0.0-beta.74",
+29
View File
@@ -228,6 +228,16 @@ export const StepStartPart = Schema.Struct({
...partBase,
type: Schema.Literal("step-start"),
snapshot: Schema.optional(Schema.String),
// kilocode_change start - wall-clock timestamps captured at the processor
// and consumed by the webview's weighted throughput aggregator. Marked
// optional so older persisted sessions (and synthetic messages) without
// timing still decode cleanly.
time: Schema.optional(
Schema.Struct({
start: NonNegativeInt,
}),
),
// kilocode_change end
}).annotate({ identifier: "StepStartPart" })
export type StepStartPart = Types.DeepMutable<Schema.Schema.Type<typeof StepStartPart>>
@@ -243,6 +253,25 @@ export const StepFinishPart = Schema.Struct({
modelID: ModelV2.ID,
}),
),
metrics: Schema.optional(
Schema.Struct({
prompt: Schema.optional(Schema.Finite),
generation: Schema.optional(Schema.Finite),
source: Schema.Literals(["provider", "computed"]),
}),
),
// Wall-clock timestamps + active generation duration captured at the
// session processor. The webview's weighted throughput aggregator uses
// `time.elapsed` (active model-generation duration in milliseconds,
// excluding tool execution and idle waiting) to weight the per-turn
// rate. Optional so legacy persisted sessions keep decoding.
time: Schema.optional(
Schema.Struct({
start: NonNegativeInt,
end: NonNegativeInt,
elapsed: Schema.Finite,
}),
),
// kilocode_change end
cost: Schema.Finite,
tokens: Schema.Struct({
+4
View File
@@ -54,6 +54,10 @@ const icons: Record<string, { path: string; viewBox: string }> = {
viewBox: "0 0 24 24",
path: `<path d="M3 12a9 9 0 0 1 9-9 9.75 9.75 0 0 1 6.74 2.74L21 8" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round"/><path d="M21 3v5h-5" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round"/><path d="M21 12a9 9 0 0 1-9 9 9.75 9.75 0 0 1-6.74-2.74L3 16" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round"/><path d="M3 21v-5h5" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round"/>`,
},
gauge: {
viewBox: "0 0 24 24",
path: `<path d="M12 14L9 10M12 14L15 10M21 15C21 18.866 17.866 22 14 22H10C6.134 22 3 18.866 3 15V9C3 5.134 6.134 2 10 2H14C17.866 2 21 5.134 21 9V15Z" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>`,
},
}
type Name = keyof typeof icons
@@ -37,6 +37,14 @@
[data-component="icon-button"][data-icon="thumbs-down"]:hover [data-slot="icon-svg"] path {
fill: currentColor;
}
/* Throughput badge sits to the right of the copy/feedback buttons,
beside them rather than beneath the message. */
[data-slot="assistant-throughput-inline"] {
margin-left: 6px;
display: flex;
align-items: center;
}
}
}
@@ -157,6 +157,7 @@ export interface MessagePartProps {
animate?: boolean
working?: boolean
feedback?: MessageFeedbackControls
throughput?: JSX.Element
}
export type PartComponent = Component<MessagePartProps>
@@ -991,6 +992,7 @@ export function Part(props: MessagePartProps) {
animate={props.animate}
working={props.working}
feedback={props.feedback}
throughput={props.throughput}
/>
</Show>
)
@@ -1448,6 +1450,9 @@ PART_MAPPING["text"] = function TextPartDisplay(props) {
/>
</Tooltip>
</Show>
<Show when={props.throughput}>
{(el) => <span data-slot="assistant-throughput-inline">{el()}</span>}
</Show>
</div>
</Show>
<Show when={summary()}>
+5
View File
@@ -1129,6 +1129,11 @@
"default": true,
"description": "Show the task timeline graph in the chat header"
},
"kilo-code.new.showTokenThroughput": {
"type": "boolean",
"default": false,
"description": "Show tokens-per-second (prompt-processing / text-generation) badges on assistant messages and the task header"
},
"kilo-code.new.chat.shiftTabCyclesVariant": {
"type": "boolean",
"default": true,
+10
View File
@@ -172,6 +172,7 @@ import {
watchIndexingConfig,
} from "./kilo-provider/indexing-settings"
import { buildChatSettingsMessage, validChatSetting, watchChatConfig } from "./kilo-provider/chat-settings"
import { buildThroughputSettingMessage, watchThroughputConfig } from "./kilo-provider/throughput-settings"
let maxCost = 0
@@ -393,6 +394,7 @@ export class KiloProvider implements vscode.WebviewViewProvider, TelemetryProper
private autocompleteConfigDisposable: vscode.Disposable | null = null
private indexingConfigDisposable: vscode.Disposable | null = null
private chatConfigDisposable: vscode.Disposable | null = null
private throughputConfigDisposable: vscode.Disposable | null = null
private telemetryStateDisposable: vscode.Disposable | null = null
private viewStateDisposable: vscode.Disposable | null = null
private visibilityDisposable: vscode.Disposable | null = null
@@ -917,6 +919,8 @@ export class KiloProvider implements vscode.WebviewViewProvider, TelemetryProper
this.indexingConfigDisposable = watchIndexingConfig((msg) => this.postMessage(msg))
this.chatConfigDisposable?.dispose()
this.chatConfigDisposable = watchChatConfig((msg) => this.postMessage(msg))
this.throughputConfigDisposable?.dispose()
this.throughputConfigDisposable = watchThroughputConfig((msg) => this.postMessage(msg))
this.telemetryStateDisposable?.dispose()
this.telemetryStateDisposable = watchTelemetryState((msg) => this.postMessage(msg))
this.webviewMessageDisposable = webview.onDidReceiveMessage(async (message) => {
@@ -1351,6 +1355,9 @@ export class KiloProvider implements vscode.WebviewViewProvider, TelemetryProper
case "requestTimelineSetting":
this.sendTimelineSetting()
break
case "requestThroughputSetting":
this.postMessage(buildThroughputSettingMessage())
break
case "requestNotifications":
this.fetchAndSendNotifications().catch((e) =>
console.error("[Kilo New] fetchAndSendNotifications failed:", e),
@@ -1734,6 +1741,7 @@ export class KiloProvider implements vscode.WebviewViewProvider, TelemetryProper
this.postMessage({ type: "gitStatus", repo: this.cachedGitRepo })
this.sendNotificationSettings()
this.sendTimelineSetting()
this.postMessage(buildThroughputSettingMessage())
this.postMessage({ type: "extensionDataReady" })
if (this.cachedGitRepo) this.startStatsPolling()
@@ -3724,6 +3732,7 @@ export class KiloProvider implements vscode.WebviewViewProvider, TelemetryProper
this.sendBrowserSettings()
this.sendNotificationSettings()
this.sendTimelineSetting()
this.postMessage(buildThroughputSettingMessage())
this.sendWorkStyle()
await ModelState.reset(this.client, (msg) => this.postMessage(msg))
@@ -4518,6 +4527,7 @@ export class KiloProvider implements vscode.WebviewViewProvider, TelemetryProper
this.autocompleteConfigDisposable?.dispose()
this.indexingConfigDisposable?.dispose()
this.chatConfigDisposable?.dispose()
this.throughputConfigDisposable?.dispose()
this.telemetryStateDisposable?.dispose()
this.autoApproveBridge?.dispose()
this.visibleTaskStreams.clear()
@@ -0,0 +1,19 @@
import * as vscode from "vscode"
type Post = (msg: unknown) => void
export function buildThroughputSettingMessage() {
const config = vscode.workspace.getConfiguration("kilo-code.new")
return {
type: "throughputSettingLoaded" as const,
visible: config.get<boolean>("showTokenThroughput", false),
}
}
export function watchThroughputConfig(post: Post): vscode.Disposable {
return vscode.workspace.onDidChangeConfiguration((event) => {
if (event.affectsConfiguration("kilo-code.new.showTokenThroughput")) {
post(buildThroughputSettingMessage())
}
})
}
@@ -4,6 +4,12 @@ import {
calcTotalCost,
calcContextUsage,
calcTokenUsage,
aggregateMetrics,
latestMetrics,
messageMetrics,
messageThroughput,
sessionThroughput,
formatTG,
buildFamilyCosts,
buildFamilyParents,
buildFamilyParentsFromTools,
@@ -717,3 +723,263 @@ describe("collapseCostBreakdown", () => {
expect(shown).toBe(1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + 11)
})
})
// ── Throughput aggregation ─────────────────────────────────────────────
type StepFinishOverrides = {
metrics?: NonNullable<Part["metrics"]>
tokens?: { input: number; output: number; reasoning?: number; cache?: { read: number; write: number } }
time?: { start: number; end: number; elapsed: number }
}
function stepFinish(id: string, metricsOrOverrides?: NonNullable<Part["metrics"]> | StepFinishOverrides): Part {
// Older call sites pass only metrics directly. Keep that signature so
// the existing latestMetrics / messageMetrics tests stay readable.
if (
metricsOrOverrides &&
"metrics" in metricsOrOverrides === false &&
"tokens" in metricsOrOverrides === false &&
"time" in metricsOrOverrides === false
) {
return {
type: "step-finish",
id,
...(metricsOrOverrides ? { metrics: metricsOrOverrides } : {}),
}
}
const overrides = (metricsOrOverrides ?? {}) as StepFinishOverrides
return {
type: "step-finish",
id,
...(overrides.metrics ? { metrics: overrides.metrics } : {}),
...(overrides.tokens ? { tokens: overrides.tokens } : {}),
...(overrides.time ? { time: overrides.time } : {}),
}
}
describe("latestMetrics", () => {
it("returns undefined when no step-finish parts carry metrics", () => {
const parts: Part[] = [
{ type: "step-start", id: "s1" },
stepFinish("f1"),
{ type: "text", id: "t1", text: "hello" },
]
expect(latestMetrics(parts)).toBeUndefined()
})
it("picks the last non-empty generation rate across every step in the session", () => {
const parts: Part[] = [
stepFinish("f1", { prompt: 100, generation: 20, source: "computed" }),
{ type: "text", id: "t1", text: "mid" },
stepFinish("f2", { prompt: 412, generation: 38, source: "computed" }),
]
expect(latestMetrics(parts)).toEqual({ generation: 38, source: "computed" })
})
it("uses the latest computed value when earlier steps report lower rates", () => {
const parts: Part[] = [
stepFinish("f1", { prompt: 500, generation: 50, source: "computed" }),
stepFinish("f2", { generation: 30, source: "computed" }),
]
const result = latestMetrics(parts)
expect(result?.source).toBe("computed")
expect(result?.generation).toBe(30)
})
it("falls back to the only computed sample when no later one is present", () => {
const parts: Part[] = [stepFinish("f1", { generation: 12, source: "computed" }), stepFinish("f2")]
expect(latestMetrics(parts)).toEqual({ generation: 12, source: "computed" })
})
it("ignores non-step-finish parts even when they look like metrics", () => {
const parts: Part[] = [
{ type: "text", id: "t1", text: "noise" },
stepFinish("f1", { prompt: 200, generation: 22, source: "computed" }),
]
expect(latestMetrics(parts)).toEqual({ generation: 22, source: "computed" })
})
})
describe("aggregateMetrics", () => {
// Historical alias of latestMetrics — kept so external callers and tests
// that still use the original name keep working. Behaviour matches: the
// last non-empty step-finish generation rate wins.
it("matches latestMetrics for the same input", () => {
const parts: Part[] = [
stepFinish("f1", { generation: 25, source: "computed" }),
stepFinish("f2", { generation: 12, source: "computed" }),
]
expect(aggregateMetrics(parts)).toEqual(latestMetrics(parts))
})
})
describe("messageMetrics", () => {
it("picks the last non-empty generation rate within a single assistant message", () => {
// An assistant turn that runs reasoning + answer produces two step-finish
// parts; the badge surfaces the final step's generation rate so the
// user sees the rate for the most recent reasoning or text generation
// in that turn.
const parts: Part[] = [
stepFinish("f1", { generation: 25, source: "computed" }),
stepFinish("f2", { generation: 12, source: "computed" }),
]
expect(messageMetrics(parts)).toEqual({ generation: 12, source: "computed" })
})
it("matches latestMetrics behavior on the same input", () => {
const parts: Part[] = [
stepFinish("f1", { generation: 8, source: "computed" }),
stepFinish("f2", { prompt: 99, generation: 33, source: "computed" }),
]
expect(messageMetrics(parts)).toEqual(latestMetrics(parts))
})
it("returns undefined when no throughput metrics are present", () => {
expect(messageMetrics([])).toBeUndefined()
expect(messageMetrics([{ type: "text", id: "t1", text: "no metrics here" }])).toBeUndefined()
})
})
describe("throughput formatters", () => {
const locale = "en-US"
it("renders the value with a t/s suffix", () => {
expect(formatTG(412, locale)).toBe("412 t/s")
expect(formatTG(28.7, locale)).toBe("28.7 t/s")
})
it("falls back to dash for missing or bogus values", () => {
expect(formatTG(undefined, locale)).toBe("")
expect(formatTG(0, locale)).toBe("")
expect(formatTG(-5, locale)).toBe("")
expect(formatTG(Number.NaN, locale)).toBe("")
expect(formatTG(Number.POSITIVE_INFINITY, locale)).toBe("")
})
})
// Weighted throughput — the value rendered beneath each assistant message
// after the v2 refactor. Behaves like a per-turn weighted average: total
// generated tokens across step-finish parts divided by total active
// model-generation duration, excluding tool-only or untimed steps.
describe("messageThroughput", () => {
it("returns undefined when no step-finish parts carry timing", () => {
const parts: Part[] = [
{ type: "step-start", id: "s1" },
stepFinish("f1", { metrics: { generation: 100, source: "computed" } }),
]
expect(messageThroughput(parts)).toBeUndefined()
})
it("computes a single-step rate from tokens and elapsed ms", () => {
const parts: Part[] = [
stepFinish("f1", {
tokens: { input: 10, output: 200, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 0, end: 1000, elapsed: 1000 },
}),
]
// (200 + 0) * 1000 / 1000 = 200
expect(messageThroughput(parts)).toEqual({ generation: 200, source: "computed" })
})
it("weights multiple steps by their elapsed time rather than averaging rates", () => {
// Discriminating case: weighted = (300 * 1000 / 5000) = 60 t/s,
// last-wins = 50 t/s. Confirms the formula doesn't just take the final
// step's value.
const parts: Part[] = [
stepFinish("f1", {
tokens: { input: 10, output: 100, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 0, end: 1000, elapsed: 1000 },
}),
stepFinish("f2", {
tokens: { input: 10, output: 200, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 1000, end: 5000, elapsed: 4000 },
}),
]
expect(messageThroughput(parts)).toEqual({ generation: 60, source: "computed" })
})
it("includes reasoning tokens in the numerator", () => {
const parts: Part[] = [
stepFinish("f1", {
tokens: { input: 10, output: 100, reasoning: 200, cache: { read: 0, write: 0 } },
time: { start: 0, end: 1000, elapsed: 1000 },
}),
]
// (100 + 200) * 1000 / 1000 = 300
expect(messageThroughput(parts)).toEqual({ generation: 300, source: "computed" })
})
it("ignores step-finish parts without timing", () => {
const parts: Part[] = [
stepFinish("f1", {
tokens: { input: 10, output: 200, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 0, end: 1000, elapsed: 1000 },
}),
// No `time` field — older part shape, possibly replayed session.
stepFinish("f2", { metrics: { generation: 999, source: "computed" } }),
]
expect(messageThroughput(parts)).toEqual({ generation: 200, source: "computed" })
})
it("ignores tool-only steps that produced no output tokens", () => {
const parts: Part[] = [
stepFinish("f1", {
tokens: { input: 10, output: 0, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 0, end: 500, elapsed: 500 },
}),
stepFinish("f2", {
tokens: { input: 10, output: 100, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 500, end: 1500, elapsed: 1000 },
}),
]
expect(messageThroughput(parts)).toEqual({ generation: 100, source: "computed" })
})
it("returns undefined when only tool-only steps are present", () => {
const parts: Part[] = [
stepFinish("f1", {
tokens: { input: 10, output: 0, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 0, end: 500, elapsed: 500 },
}),
]
expect(messageThroughput(parts)).toBeUndefined()
})
it("returns undefined when timing is non-positive across all steps", () => {
const parts: Part[] = [
stepFinish("f1", {
tokens: { input: 10, output: 100, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 0, end: 0, elapsed: 0 },
}),
]
expect(messageThroughput(parts)).toBeUndefined()
})
})
describe("sessionThroughput", () => {
it("aggregates the same way as messageThroughput across a flat part array", () => {
const parts: Part[] = [
stepFinish("f1", {
tokens: { input: 10, output: 100, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 0, end: 1000, elapsed: 1000 },
}),
stepFinish("f2", {
tokens: { input: 10, output: 200, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 2000, end: 5000, elapsed: 3000 },
}),
// From the "next" message — still rolled up correctly.
stepFinish("f3", {
tokens: { input: 10, output: 500, reasoning: 0, cache: { read: 0, write: 0 } },
time: { start: 6000, end: 11000, elapsed: 5000 },
}),
]
// (800 * 1000) / 9000 = 88.888...
const result = sessionThroughput(parts)
expect(result?.source).toBe("computed")
expect(result?.generation).toBeCloseTo((800 * 1000) / 9000, 5)
})
it("returns undefined for empty input", () => {
expect(sessionThroughput([])).toBeUndefined()
})
})
@@ -7,7 +7,7 @@
* Active questions render inline via QuestionDock; permissions are in the bottom dock.
*/
import { Component, For, Show, createMemo } from "solid-js"
import { Component, For, Show, createMemo, type JSX } from "solid-js"
import { Dynamic } from "solid-js/web"
import { Part, PART_MAPPING, ToolRegistry } from "@kilocode/kilo-ui/message-part"
import type { MessageFeedbackControls } from "@kilocode/kilo-ui/message-part"
@@ -25,9 +25,11 @@ import { useLanguage } from "../../context/language"
import { useServer } from "../../context/server"
import { planDisplayPath } from "../../utils/plan-path"
import { isRenderable, UPSTREAM_SUPPRESSED_TOOLS } from "../../utils/transcript-parts"
import { messageThroughput, formatTG } from "../../context/session-utils"
import { color as timelineColor } from "../../utils/timeline/colors"
import type { Part as TimelinePart } from "../../types/messages"
import type { TimelineHighlight } from "../../utils/timeline/highlight"
import { Tooltip } from "@kilocode/kilo-ui/tooltip"
import { QuestionDock } from "./QuestionDock"
import { SuggestBar } from "./SuggestBar"
import { toolDefaultOpen } from "./tool-default-open"
@@ -164,14 +166,50 @@ function BashToolCard(props: { part: ToolPart; defaultOpen: boolean; forceOpen?:
)
}
/** Plain-text generation-speed value shown beside the copy/feedback buttons
* on an assistant message.
*
* Renders as muted metadata — no icon, no background, no border — so it
* reads as tertiary info rather than an interactive control. The
* description on hover explains that the value is a weighted generation
* rate across the turn's model-generation steps (output + reasoning
* tokens over active generation time).
*
* Visibility is gated by the same `kilo-code.new.showTokenThroughput`
* toggle that previously controlled the multi-row badge. The metric only
* renders when the message has at least one step-finish part carrying both
* a token count and elapsed timing.
*/
function ThroughputBadge(props: { metrics: { generation?: number } }) {
const language = useLanguage()
const speedText = createMemo(() => formatTG(props.metrics.generation, language.locale()))
const tooltip = createMemo(() => {
if (props.metrics.generation === undefined) {
return language.t("chat.throughput.tooltip.missing")
}
return language.t("chat.throughput.tooltip", { speed: speedText() })
})
return (
<Tooltip value={tooltip()} placement="top">
<span data-component="assistant-throughput">{speedText()}</span>
</Tooltip>
)
}
export const AssistantMessage: Component<AssistantMessageProps> = (props) => {
const data = useData()
const session = useSession()
const display = useDisplay()
const language = useLanguage()
const { config } = useConfig()
const open = createMemo(() => config().terminal_command_display !== "collapsed")
const edit = createMemo(() => config().code_edit_display === "expanded")
// Throughput toggle lives on the shared DisplayProvider so every
// AssistantMessage renders against the same signal without posting its
// own requestThroughputSetting round-trip on mount.
const throughputVisible = createMemo(() => display.throughputVisible())
const parts = createMemo(() => {
const stored = props.parts ?? data.store.part?.[props.message.id]
if (!stored) return []
@@ -182,6 +220,20 @@ export const AssistantMessage: Component<AssistantMessageProps> = (props) => {
return !!matchToolRequest(part, "question", session.questions())
})
})
// Pull the weighted generation rate across the turn's step-finish parts
// (output + reasoning tokens over active generation duration) so the badge
// represents the turn as a whole rather than whichever step happened to
// finish most recently. We intentionally read from the full message parts
// in the data store rather than `props.parts` — the parent chunks
// messages into rows of ~8 parts, and step-finish may land in a row
// different from the one currently rendered.
const throughput = createMemo(() =>
messageThroughput(
(data.store.part?.[props.message.id] as TimelinePart[] | undefined) ??
(props.parts as TimelinePart[] | undefined) ??
([] as TimelinePart[]),
),
)
return (
<>
<For each={parts()}>
@@ -216,6 +268,18 @@ export const AssistantMessage: Component<AssistantMessageProps> = (props) => {
return h?.msgId === props.message.id && h?.partId === part.id
})
// Throughput badge renders inside the copy/feedback action row of the
// text part that carries the copy button (the last text part of the
// message), pushed to the right of the buttons rather than below the
// message. Only built for that part so non-text parts skip the work.
const throughputEl = createMemo<JSX.Element | undefined>(() => {
if (!throughputVisible()) return undefined
const metrics = throughput()
if (!metrics) return undefined
if (part.id !== props.showAssistantCopyPartID) return undefined
return <ThroughputBadge metrics={metrics} />
})
return (
<Show
when={
@@ -260,6 +324,7 @@ export const AssistantMessage: Component<AssistantMessageProps> = (props) => {
forceOpenFile={forceOpen() ? props.forceOpenFile : undefined}
reasoningAutoCollapse={display.reasoningAutoCollapse()}
feedback={props.feedback}
throughput={throughputEl()}
animate={
part.type === "tool" &&
((part as unknown as ToolPart).state?.status === "pending" ||
@@ -91,6 +91,19 @@ const DisplayTab: Component = () => {
</Switch>
</SettingsRow>
<SettingsRow
title={language.t("settings.display.tokenThroughput.title")}
description={language.t("settings.display.tokenThroughput.description")}
>
<Switch
checked={Boolean(settings()["showTokenThroughput"] ?? false)}
onChange={(checked: boolean) => updateSetting("showTokenThroughput", checked)}
hideLabel
>
{language.t("settings.display.tokenThroughput.title")}
</Switch>
</SettingsRow>
<SettingsRow
title={language.t("settings.display.terminalCommand.title")}
description={language.t("settings.display.terminalCommand.description")}
@@ -108,6 +108,16 @@ export const ConfigProvider: ParentComponent = (props) => {
})
return
}
if (message.type === "throughputSettingLoaded") {
// Seed settings() so the DisplayTab Switch reflects persisted state on
// first open. DisplayProvider also reads this message to drive
// throughputVisible() for the per-message badge; both signals update
// from the same backend message without conflict.
mergeSettings({
showTokenThroughput: message.visible,
})
return
}
if (message.type === "configLoaded") {
// Skip if a save is in-flight — a stale configLoaded must not overwrite
// the optimistically-updated state while the write is being confirmed.
@@ -4,6 +4,7 @@ import {
createMemo,
createSignal,
onCleanup,
onMount,
useContext,
type Accessor,
type ParentComponent,
@@ -18,6 +19,10 @@ interface DisplayContextValue {
setReasoningAutoCollapse: (collapse: boolean) => void
fontSize: Accessor<number>
setFontSize: (size: number) => void
// Shared throughput toggle — the same signal backs the per-message badge in
// every AssistantMessage and the aggregated row in TaskHeader, so flipping
// the setting once updates both surfaces without round-trips.
throughputVisible: Accessor<boolean>
}
export const DisplayContext = createContext<DisplayContextValue>()
@@ -27,10 +32,16 @@ export const DisplayProvider: ParentComponent = (props) => {
const vscode = useVSCode()
const reasoningAutoCollapse = createMemo(() => config().auto_collapse_reasoning ?? false)
const [fontSize, setFontSizeSignal] = createSignal(readFontSize())
const [throughputVisible, setThroughputVisible] = createSignal(false)
// Request the throughput toggle once on mount; the extension posts back
// (and onDidChangeConfiguration forwards subsequent edits).
onMount(() => vscode.postMessage({ type: "requestThroughputSetting" }))
const unsubscribe = vscode.onMessage((message: ExtensionMessage) => {
if (message.type === "ready" && message.fontSize !== undefined) setFontSizeSignal(clampFontSize(message.fontSize))
if (message.type === "fontSizeChanged") setFontSizeSignal(clampFontSize(message.fontSize))
if (message.type === "throughputSettingLoaded") setThroughputVisible(Boolean(message.visible))
})
createEffect(() => {
@@ -50,6 +61,7 @@ export const DisplayProvider: ParentComponent = (props) => {
setFontSizeSignal(next)
vscode.postMessage({ type: "updateSetting", key: "fontSize", value: next })
},
throughputVisible,
}}
>
{props.children}
@@ -204,6 +204,106 @@ export function calcTokenUsage(
return undefined
}
/**
* Pick the throughput snapshot from the last step-finish part that carries a
* `metrics` block. We surface only the most recent assistant turn's rate so
* the figure reflects what the user is currently waiting on rather than a
* stale session-wide average — older turns scroll out of view and shouldn't
* keep pulling the displayed value down.
*
* PP (prompt-processing) is intentionally not surfaced here: the AI SDK
* adapter drops llama.cpp's `prompt_per_second` before providerMetadata
* reaches computeMetrics, so the wire shape stays `{ prompt?, generation? }`
* for future use but only `generation` is populated today. PP support lands
* when the upstream metadataExtractor wiring ships.
*
* Returns `undefined` when no step-finish part in the input carries metrics,
* which is the signal callers use to hide the throughput UI.
*/
export function latestMetrics(parts: readonly Part[]): { generation?: number; source: "computed" } | undefined {
let generation: number | undefined
for (const part of parts) {
if (part.type !== "step-finish") continue
const metrics = part.metrics
if (!metrics) continue
if (metrics.generation !== undefined) generation = metrics.generation
}
if (generation === undefined) return undefined
return { generation, source: "computed" }
}
/**
* Aggregate tokens-per-second throughput across the step-finish parts of a
* session. Kept as an alias of `latestMetrics` because the historical name
* still appears in tests and external callers — both now resolve to the same
* "last non-empty sample wins" snapshot semantics.
*/
export function aggregateMetrics(parts: readonly Part[]): { generation?: number; source: "computed" } | undefined {
return latestMetrics(parts)
}
/**
* Pick the throughput snapshot from a single assistant message's parts.
* Same selection strategy as `latestMetrics` so the per-message badge and
* the header row stay consistent.
*/
export function messageMetrics(parts: readonly Part[]): { generation?: number; source: "computed" } | undefined {
return latestMetrics(parts)
}
/**
* Weighted generation throughput for a single assistant message. Aggregates
* output + reasoning tokens across every step-finish part against the sum of
* their active model-generation durations, so the displayed value represents
* the turn rather than whichever step happened to finish last.
*
* Steps without `time.elapsed`, with non-positive `elapsed`, or with no
* generated tokens are skipped — tool-only steps, idempotent cache hits,
* and tool re-execution should not skew the figure.
*/
export function messageThroughput(parts: readonly Part[]): { generation?: number; source: "computed" } | undefined {
let generated = 0
let elapsedMs = 0
for (const part of parts) {
if (part.type !== "step-finish") continue
const time = part.time
if (!time || !Number.isFinite(time.elapsed) || time.elapsed <= 0) continue
const tokens = part.tokens
if (!tokens) continue
const stepGenerated = tokens.output + (tokens.reasoning ?? 0)
if (stepGenerated <= 0) continue
generated += stepGenerated
elapsedMs += time.elapsed
}
if (generated <= 0 || elapsedMs <= 0) return undefined
const generation = (generated * 1000) / elapsedMs
if (!Number.isFinite(generation) || generation <= 0) return undefined
return { generation, source: "computed" }
}
/**
* Weighted generation throughput across the flat array of parts from every
* message in a session. Same weighted semantics as `messageThroughput` —
* useful when a caller has already flattened parts across messages.
*/
export function sessionThroughput(parts: readonly Part[]): { generation?: number; source: "computed" } | undefined {
return messageThroughput(parts)
}
/**
* Format a text-generation rate for display. Shared by every rendering site
* so the same value reads the same in the per-message badge and the
* aggregated header row.
*/
function formatRateValue(value: number | undefined, locale: string): string {
if (!Number.isFinite(value) || value === undefined || value <= 0) return ""
return `${new Intl.NumberFormat(locale, { maximumFractionDigits: 1 }).format(value)} t/s`
}
export function formatTG(value: number | undefined, locale: string) {
return formatRateValue(value, locale)
}
/**
* Build a map of session ID → **own cost** for each session in the family
* that has non-zero own cost.
+9
View File
@@ -1684,6 +1684,15 @@ export const dict = {
"اختر ما إذا كانت الكتل التي تعرض تعديلات التعليمات البرمجية والفروقات تبدأ موسّعة أم مطوية.",
"settings.display.codeEdit.expanded": "موسّعة",
"settings.display.codeEdit.collapsed": "مطوية",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "النموذج الافتراضي",
"settings.providers.defaultModel.description": "النموذج الأساسي للمحادثات",
"settings.providers.smallModel.title": "نموذج صغير",
+9
View File
@@ -1735,6 +1735,15 @@ export const dict = {
"Escolha se os blocos que exibem edições de código e diferenças começam expandidos ou recolhidos.",
"settings.display.codeEdit.expanded": "Expandidos",
"settings.display.codeEdit.collapsed": "Recolhidos",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Modelo padrão",
"settings.providers.defaultModel.description": "Modelo principal para conversas",
"settings.providers.smallModel.title": "Modelo pequeno",
+9
View File
@@ -1727,6 +1727,15 @@ export const dict = {
"Odaberite da li će blokovi koji prikazuju izmjene koda i razlike u početku biti prošireni ili sažeti.",
"settings.display.codeEdit.expanded": "Prošireni",
"settings.display.codeEdit.collapsed": "Sažeti",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Zadani model",
"settings.providers.defaultModel.description": "Primarni model za razgovore",
"settings.providers.smallModel.title": "Mali model",
+9
View File
@@ -1718,6 +1718,15 @@ export const dict = {
"Vælg, om blokke, der viser koderedigeringer og forskelle, starter foldet ud eller sammen.",
"settings.display.codeEdit.expanded": "Foldet ud",
"settings.display.codeEdit.collapsed": "Foldet sammen",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Standardmodel",
"settings.providers.defaultModel.description": "Primær model til samtaler",
"settings.providers.smallModel.title": "Lille model",
@@ -1754,6 +1754,15 @@ export const dict = {
"Wählen Sie, ob Blöcke mit Codebearbeitungen und Unterschieden anfangs aus- oder eingeklappt sind.",
"settings.display.codeEdit.expanded": "Ausgeklappt",
"settings.display.codeEdit.collapsed": "Eingeklappt",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Standardmodell",
"settings.providers.defaultModel.description": "Primäres Modell für Gespräche",
"settings.providers.smallModel.title": "Kleines Modell",
@@ -1696,6 +1696,13 @@ export const dict = {
"settings.display.codeEdit.description": "Choose whether code edit and diff blocks start expanded or collapsed.",
"settings.display.codeEdit.expanded": "Expanded",
"settings.display.codeEdit.collapsed": "Collapsed",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Default Model",
"settings.providers.defaultModel.description": "Primary model for conversations",
+9
View File
@@ -1743,6 +1743,15 @@ export const dict = {
"Elige si los bloques de edición de código y de diferencias aparecen inicialmente expandidos o contraídos.",
"settings.display.codeEdit.expanded": "Expandidos",
"settings.display.codeEdit.collapsed": "Contraídos",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Modelo predeterminado",
"settings.providers.defaultModel.description": "Modelo principal para conversaciones",
"settings.providers.smallModel.title": "Modelo pequeño",
+9
View File
@@ -1764,6 +1764,15 @@ export const dict = {
"Choisissez si les blocs de modification du code et de différences sont initialement développés ou réduits.",
"settings.display.codeEdit.expanded": "Développés",
"settings.display.codeEdit.collapsed": "Réduits",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Modèle par défaut",
"settings.providers.defaultModel.description": "Modèle principal pour les conversations",
"settings.providers.smallModel.title": "Petit modèle",
+9
View File
@@ -1521,6 +1521,15 @@ export const dict = {
"Scegli se i blocchi delle modifiche al codice e delle differenze iniziano espansi o compressi.",
"settings.display.codeEdit.expanded": "Espansi",
"settings.display.codeEdit.collapsed": "Compressi",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Modello predefinito",
"settings.providers.defaultModel.description": "Modello principale per le conversazioni",
"settings.providers.smallModel.title": "Modello leggero",
+9
View File
@@ -1713,6 +1713,15 @@ export const dict = {
"コード編集ブロックと差分ブロックを最初から展開するか折りたたむかを選択します。",
"settings.display.codeEdit.expanded": "展開",
"settings.display.codeEdit.collapsed": "折りたたみ",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "デフォルトモデル",
"settings.providers.defaultModel.description": "会話のプライマリモデル",
"settings.providers.smallModel.title": "小型モデル",
+9
View File
@@ -1695,6 +1695,15 @@ export const dict = {
"settings.display.codeEdit.description": "코드 편집 블록과 차이점 블록을 처음부터 펼칠지 접을지 선택합니다.",
"settings.display.codeEdit.expanded": "펼침",
"settings.display.codeEdit.collapsed": "접힘",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "기본 모델",
"settings.providers.defaultModel.description": "대화의 기본 모델",
"settings.providers.smallModel.title": "소형 모델",
+8
View File
@@ -1696,6 +1696,14 @@ export const dict = {
"settings.display.codeEdit.expanded": "Uitgeklapt",
"settings.display.codeEdit.collapsed": "Ingeklapt",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Standaard Model",
"settings.providers.defaultModel.description": "Primair model voor gesprekken",
"settings.providers.smallModel.title": "Klein Model",
+9
View File
@@ -1716,6 +1716,15 @@ export const dict = {
"Velg om blokker for kodeendringer og forskjeller skal være utvidet eller skjult fra start.",
"settings.display.codeEdit.expanded": "Utvidet",
"settings.display.codeEdit.collapsed": "Skjult",
"settings.display.tokenThroughput.title": "Vis genereringshastighet",
"settings.display.tokenThroughput.description":
"Vis tekstgenereringshastighet (tokens/sek) på den siste assistentmeldingen og i oppgaveoverskriften. Skjult som standard for å holde chatten ryddig.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Standardmodell",
"settings.providers.defaultModel.description": "Primær modell for samtaler",
"settings.providers.smallModel.title": "Liten modell",
+9
View File
@@ -1726,6 +1726,15 @@ export const dict = {
"Wybierz, czy bloki edycji kodu i podglądy różnic mają być początkowo rozwinięte czy zwinięte.",
"settings.display.codeEdit.expanded": "Rozwinięte",
"settings.display.codeEdit.collapsed": "Zwinięte",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Domyślny model",
"settings.providers.defaultModel.description": "Główny model do rozmów",
"settings.providers.smallModel.title": "Mały model",
+9
View File
@@ -1724,6 +1724,15 @@ export const dict = {
"Выберите, будут ли блоки изменений кода и различий изначально развёрнуты или свёрнуты.",
"settings.display.codeEdit.expanded": "Развёрнуты",
"settings.display.codeEdit.collapsed": "Свёрнуты",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Модель по умолчанию",
"settings.providers.defaultModel.description": "Основная модель для разговоров",
"settings.providers.smallModel.title": "Малая модель",
+9
View File
@@ -1693,6 +1693,15 @@ export const dict = {
"settings.display.codeEdit.description": "เลือกว่าบล็อกการแก้ไขโค้ดและบล็อกแสดงความแตกต่างจะเริ่มต้นแบบขยายหรือยุบ",
"settings.display.codeEdit.expanded": "ขยาย",
"settings.display.codeEdit.collapsed": "ยุบ",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "โมเดลเริ่มต้น",
"settings.providers.defaultModel.description": "โมเดลหลักสำหรับบทสนทนา",
"settings.providers.smallModel.title": "โมเดลขนาดเล็ก",
+8
View File
@@ -1683,6 +1683,14 @@ export const dict = {
"settings.display.codeEdit.expanded": "Genişletilmiş",
"settings.display.codeEdit.collapsed": "Daraltılmış",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Varsayılan Model",
"settings.providers.defaultModel.description": "Sohbetler için birincil model",
"settings.providers.smallModel.title": "Küçük Model",
+8
View File
@@ -1680,6 +1680,14 @@ export const dict = {
"settings.display.codeEdit.expanded": "Розгорнуті",
"settings.display.codeEdit.collapsed": "Згорнуті",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "Модель за замовчуванням",
"settings.providers.defaultModel.description": "Основна модель для чатів",
"settings.providers.smallModel.title": "Мала модель",
+9
View File
@@ -1645,6 +1645,15 @@ export const dict = {
"settings.display.codeEdit.description": "选择代码编辑块和差异块的初始状态:展开或折叠。",
"settings.display.codeEdit.expanded": "展开",
"settings.display.codeEdit.collapsed": "折叠",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "默认模型",
"settings.providers.defaultModel.description": "对话的主要模型",
"settings.providers.smallModel.title": "小模型",
+9
View File
@@ -1609,6 +1609,15 @@ export const dict = {
"settings.display.codeEdit.description": "選擇程式碼編輯區塊與差異區塊的初始狀態:展開或收合。",
"settings.display.codeEdit.expanded": "展開",
"settings.display.codeEdit.collapsed": "收合",
"settings.display.tokenThroughput.title": "Show Token Throughput",
"settings.display.tokenThroughput.description":
"Display the text-generation rate (tokens/sec) on the latest assistant message and in the task header. Hidden by default to keep the chat uncluttered.",
"chat.throughput.tooltip":
"Average {{speed}} tokens/s for this turn. Includes output and reasoning tokens; excludes tool execution and waiting time.",
"chat.throughput.tooltip.missing": "Throughput metrics unavailable for this turn.",
"settings.providers.defaultModel.title": "預設模型",
"settings.providers.defaultModel.description": "對話的主要模型",
"settings.providers.smallModel.title": "小模型",
@@ -228,6 +228,18 @@
width: 100%;
}
[data-component="assistant-throughput"] {
/* Plain-text generation-speed value shown beside the copy/feedback buttons
* on an assistant message. Renders as muted metadata no icon, no
* background, no border so it reads as tertiary info that never
* competes with the action row for visual weight. */
color: var(--vscode-descriptionForeground);
font-family: var(--font-family-sans);
font-size: var(--kilo-font-size-11);
line-height: var(--line-height-normal);
font-variant-numeric: tabular-nums;
}
.vscode-session-turn-diffs {
width: 100%;
}
@@ -590,6 +590,11 @@ export interface TimelineSettingLoadedMessage {
visible: boolean
}
export interface ThroughputSettingLoadedMessage {
type: "throughputSettingLoaded"
visible: boolean
}
export interface WorkStyleLoadedMessage {
type: "workStyleLoaded"
style: WorkStyleState
@@ -1172,6 +1177,7 @@ export type ExtensionMessage =
| GlobalConfigLoadedMessage
| NotificationSettingsLoadedMessage
| TimelineSettingLoadedMessage
| ThroughputSettingLoadedMessage
| WorkStyleLoadedMessage
| WorkStyleAppliedMessage
| WorkStyleApplyFailedMessage
@@ -62,11 +62,38 @@ export interface ReasoningPart extends BasePart {
// Step parts from the backend
export interface StepStartPart extends BasePart {
type: "step-start"
// Wall-clock timestamps captured at the processor when the LLM stream
// emits `step-start`. Used by the webview to compute per-message
// throughput as a weighted aggregate of step durations.
time?: {
start: number
}
}
// Tokens-per-second throughput metrics reported by the backend on step-finish.
// Only `"computed"` is reachable today: llama.cpp surfaces
// prompt_per_second / predicted_per_second, but the upstream AI SDK drops
// them before the raw usage reaches our adapter. The `"provider"` literal is
// reserved for the follow-up that wires a metadataExtractor into the shared
// createOpenAICompatible call (see opencode/src/kilocode/session/metrics.ts).
export interface StepThroughputMetrics {
prompt?: number
generation?: number
source: "computed"
}
export interface StepFinishPart extends BasePart {
type: "step-finish"
reason?: string
// Wall-clock timestamps captured at the processor across the LLM step.
// `elapsed` is the active model-generation duration in milliseconds — it
// excludes tool execution and idle waiting — and is what the webview uses
// to weight the throughput aggregate.
time?: {
start: number
end: number
elapsed: number
}
model?: {
providerID: string
modelID: string
@@ -78,6 +105,7 @@ export interface StepFinishPart extends BasePart {
reasoning?: number
cache?: { read: number; write: number }
}
metrics?: StepThroughputMetrics
}
export interface CompactionPart extends BasePart {
@@ -433,6 +433,10 @@ export interface RequestTimelineSettingMessage {
type: "requestTimelineSetting"
}
export interface RequestThroughputSettingMessage {
type: "requestThroughputSetting"
}
export interface RequestWorkStyleMessage {
type: "requestWorkStyle"
}
@@ -1277,6 +1281,7 @@ export type WebviewMessage =
| ChatCompletionAcceptedMessage
| UpdateSettingRequest
| RequestTimelineSettingMessage
| RequestThroughputSettingMessage
| RequestWorkStyleMessage
| SetWorkStyleMessage
| ApplyWorkStyleMessage
@@ -1,8 +1,11 @@
import type { KilocodeSessionModelUsageResponse, Session } from "@kilocode/sdk/v2"
import type { KilocodeSessionModelUsageResponse, Session, StepFinishPart } from "@kilocode/sdk/v2"
export type SessionModelUsage = KilocodeSessionModelUsageResponse
export type UsageResult = { sessionID: string; data?: SessionModelUsage }
export type StepMetrics = NonNullable<StepFinishPart["metrics"]>
export type AggregatedMetrics = { generation?: number }
export function select(result: UsageResult | undefined, sessionID: string) {
if (result?.sessionID !== sessionID) return undefined
return result.data
@@ -53,6 +56,7 @@ const currency = new Intl.NumberFormat("en-US", {
style: "currency",
currency: "USD",
})
const throughput = new Intl.NumberFormat("en-US", { maximumFractionDigits: 1 })
export function formatCount(value: number) {
return count.format(value)
@@ -64,7 +68,82 @@ export function formatRate(tokens: SessionModelUsage["totals"]["tokens"]) {
return `${((tokens.cache.read / total) * 100).toFixed(1)}%`
}
export function formatRateValue(value: number | undefined) {
if (value === undefined || !Number.isFinite(value) || value <= 0) return "-"
return `${throughput.format(value)} t/s`
}
// Throughput label used by the sidebar / usage panel. Centralized here so a
// future i18n sweep only touches one file — the opencode CLI does not yet
// wire a translation layer, so today this is a literal English label.
// PP (prompt-processing) is intentionally omitted: llama.cpp's
// `prompt_per_second` is dropped upstream by the AI SDK adapter before it
// reaches providerMetadata, so the current build can only emit the
// generation rate. The PP row lands alongside generation speed once the
// upstream metadataExtractor wiring ships.
export const throughputLabel = {
generation: "Generation speed",
} as const
export function formatCost(input: number) {
const value = Math.max(0, Number.isFinite(input) ? input : 0)
return currency.format(value)
}
// Local aggregation of step-finish metrics for the sidebar/usage panel.
//
// When samples carry `elapsedMs` (kilocode_change: persisted on the
// step-finish part by the session processor) and matching `generated`
// counts, the figure is the *weighted* generation rate across the
// aggregated steps — total generated tokens over total active
// model-generation duration. That excludes tool execution and idle waiting
// so the value represents what the user paid for.
//
// When timing is missing or non-positive, the function falls back to the
// historical last-wins snapshot so older callers that haven't migrated to
// the new wire shape continue to surface a meaningful figure rather than
// silently dropping to `undefined`.
export function aggregateMetrics(
samples: ReadonlyArray<{
metrics?: StepMetrics
generated: number
elapsedMs?: number
output?: number
reasoning?: number
}>,
): AggregatedMetrics {
let generatedTotal = 0
let elapsedTotal = 0
let fallback: number | undefined
for (const sample of samples) {
const metrics = sample.metrics
if (!metrics) continue
const value = metrics.generation
if (typeof value !== "number" || !Number.isFinite(value)) continue
if (value <= 0) continue
if (sample.generated <= 0) continue
fallback = value
const elapsed = sample.elapsedMs
if (typeof elapsed !== "number" || !Number.isFinite(elapsed) || elapsed <= 0) continue
const tokens =
typeof sample.output === "number" && typeof sample.reasoning === "number"
? sample.output + sample.reasoning
: sample.generated
if (tokens <= 0) continue
generatedTotal += tokens
elapsedTotal += elapsed
}
if (generatedTotal > 0 && elapsedTotal > 0) {
const weighted = (generatedTotal * 1000) / elapsedTotal
if (Number.isFinite(weighted) && weighted > 0) {
return { generation: weighted }
}
}
return {
...(fallback !== undefined ? { generation: fallback } : {}),
}
}
export function hasMetrics(value: AggregatedMetrics | undefined): value is AggregatedMetrics {
return value !== undefined && value.generation !== undefined
}
@@ -1,29 +1,43 @@
import type { TuiPlugin, TuiPluginApi, TuiPluginModule } from "@kilocode/plugin/tui"
import { createMemo, createResource, createSignal, For, onCleanup, onMount, Show } from "solid-js"
import { createEffect, createMemo, createResource, createSignal, For, onCleanup, onMount, Show } from "solid-js"
import { useLocal } from "@tui/context/local"
import * as Model from "@tui/util/model"
import { Locale } from "@/util/locale"
import { RoutedModelMeta } from "@/kilocode/cli/cmd/tui/routes/session/routed-model-meta"
import { fmtAttemptCost, fmtScore } from "@/kilocode/components/model-info-panel-utils"
import {
aggregateMetrics,
failed,
formatCost,
formatCount,
formatRate,
formatRateValue,
groupModelsByProvider,
hasMetrics,
isSessionTreeMember,
select,
throughputLabel,
type StepMetrics,
type UsageResult,
} from "@/kilocode/plugins/model-usage"
import { ModelRow, UsageRow } from "@/kilocode/plugins/sidebar-usage-row"
const id = "internal:kilo-sidebar-usage"
type MetricSample = {
metrics?: StepMetrics
generated: number
elapsedMs?: number
output?: number
reasoning?: number
}
function View(props: { api: TuiPluginApi; session_id: string }) {
const [usageOpen, setUsageOpen] = createSignal(true)
const [modelsOpen, setModelsOpen] = createSignal(true)
const [benchOpen, setBenchOpen] = createSignal(true)
const [expanded, setExpanded] = createSignal(new Set<string>())
const [samples, setSamples] = createSignal<MetricSample[]>([])
const theme = () => props.api.theme.current
const local = useLocal()
const [result, { refetch }] = createResource(
@@ -38,6 +52,22 @@ function View(props: { api: TuiPluginApi; session_id: string }) {
const unavailable = createMemo(() => failed(result(), props.session_id))
const providers = createMemo(() => Model.index([...props.api.state.provider]))
const groups = createMemo(() => groupModelsByProvider(usage()?.models ?? [], props.api.state.provider))
const throughput = createMemo(() => aggregateMetrics(samples()))
// Reset accumulated samples whenever the sidebar is mounted against a new
// session. Without this guard, switching tabs in the TUI would blend
// step-finish metrics from the previous session into the new session's
// generation rate, and `samples` would grow without bound across
// long-lived plugin instances.
createEffect(
() => {
props.session_id
setSamples([])
},
() => {
setSamples([])
},
)
const bench = createMemo(() => {
const current = local.model.current()
if (!current) return undefined
@@ -59,8 +89,39 @@ function View(props: { api: TuiPluginApi; session_id: string }) {
const refresh = () => void refetch()
const related = (sessionID: string, info?: ReturnType<typeof props.api.state.session.get>) =>
isSessionTreeMember({ root: props.session_id, sessionID, info, get: props.api.state.session.get })
const recordSample = (
sessionID: string,
part: {
type?: string
metrics?: unknown
tokens?: unknown
// Loose time shape — different part kinds (e.g. retry) ship their own
// time fields; we only care about `elapsed` for step-finish weighting.
time?: { elapsed?: number; [k: string]: unknown }
},
) => {
if (part.type !== "step-finish") return
if (!related(sessionID)) return
const metrics = isStepMetrics(part.metrics) ? part.metrics : undefined
const generated = generatedTokens(part.tokens)
const elapsed = part.time?.elapsed
const { output, reasoning } = splitTokens(part.tokens)
setSamples((current) => [
...current,
{
...(metrics ? { metrics } : {}),
generated,
...(typeof elapsed === "number" && Number.isFinite(elapsed) && elapsed > 0
? { elapsedMs: elapsed }
: {}),
...(typeof output === "number" ? { output } : {}),
...(typeof reasoning === "number" ? { reasoning } : {}),
},
])
}
const offs = [
props.api.event.on("message.part.updated", (event) => {
recordSample(event.properties.sessionID, event.properties.part)
if (event.properties.part.type === "step-finish" && related(event.properties.sessionID)) refresh()
}),
props.api.event.on("message.part.removed", (event) => {
@@ -104,6 +165,9 @@ function View(props: { api: TuiPluginApi; session_id: string }) {
<Row label="Cache read" value={formatCount(data().totals.tokens.cache.read)} />
<Row label="Cache write" value={formatCount(data().totals.tokens.cache.write)} />
<Row label="Cache rate" value={formatRate(data().totals.tokens)} />
<Show when={hasMetrics(throughput())}>
<Row label={throughputLabel.generation} value={formatRateValue(throughput().generation)} />
</Show>
<Row label="Cost" value={formatCost(data().totals.cost)} />
</>
)}
@@ -202,6 +266,29 @@ function View(props: { api: TuiPluginApi; session_id: string }) {
)
}
function isStepMetrics(value: unknown): value is StepMetrics {
if (!value || typeof value !== "object") return false
const source = (value as { source?: unknown }).source
return source === "computed"
}
function generatedTokens(value: unknown): number {
if (!value || typeof value !== "object") return 0
const record = value as Record<string, unknown>
const output = typeof record.output === "number" ? record.output : 0
const reasoning = typeof record.reasoning === "number" ? record.reasoning : 0
return output + reasoning
}
function splitTokens(value: unknown): { output?: number; reasoning?: number } {
if (!value || typeof value !== "object") return {}
const record = value as Record<string, unknown>
const out: { output?: number; reasoning?: number } = {}
if (typeof record.output === "number") out.output = record.output
if (typeof record.reasoning === "number") out.reasoning = record.reasoning
return out
}
const tui: TuiPlugin = async (api) => {
api.slots.register({
order: 150,
@@ -0,0 +1,47 @@
// kilocode_change - new file
// Wire shape mirrors the SDK schema (packages/sdk/js/src/v2/gen/types.gen.ts
// StepFinishPart.metrics). `source` stays on the wire for backward
// compatibility with downstream consumers — see packages/kilo-vscode/
// webview-ui/src/context/session-utils.ts and AssistantMessage.tsx —
// but only the "computed" literal is reachable here because llama.cpp's
// `prompt_per_second` / `predicted_per_second` are dropped upstream by
// `@ai-sdk/openai-compatible` before the raw usage reaches our adapter.
// Follow-up: wire `metadataExtractor` into the shared
// `createOpenAICompatible` call so the provider source is reachable again.
export type TokenRates = {
prompt?: number
generation?: number
source: "computed"
}
export type ComputeInput = {
providerMetadata?: unknown
tokens: {
input: number
output: number
reasoning: number
cache: { read: number; write: number }
}
elapsedMs: number
}
// kilocode_change start - tokens/second throughput for #6579.
export function computeMetrics(input: ComputeInput): TokenRates | undefined {
if (!Number.isFinite(input.elapsedMs) || input.elapsedMs <= 0) return undefined
const generated = input.tokens.output + input.tokens.reasoning
if (generated <= 0) return undefined
const generation = (generated * 1000) / input.elapsedMs
if (!Number.isFinite(generation) || generation <= 0) return undefined
return { generation, source: "computed" }
}
const numberFormat = new Intl.NumberFormat("en-US", { maximumFractionDigits: 1 })
export function formatRate(value: number): string {
if (!Number.isFinite(value) || value <= 0) return "0 t/s"
return `${numberFormat.format(value)} t/s`
}
// kilocode_change end
@@ -13,6 +13,7 @@ import { EffectBridge } from "@/effect/bridge"
import type { LLMEvent, Usage } from "@opencode-ai/llm"
import type { ProviderV2 } from "@opencode-ai/core/provider"
import { SessionRetry } from "@/session/retry"
import { computeMetrics as computeMetricsHelper, type TokenRates } from "@/kilocode/session/metrics"
export type ReviewTelemetry = {
mode: "review"
@@ -131,6 +132,11 @@ export namespace KiloSessionProcessor {
}
}
/** Pure throughput helper re-exported for namespace symmetry. */
export const computeMetrics: typeof computeMetricsHelper = computeMetricsHelper
/** Returned shape for downstream consumers that prefer the namespace. */
export type Metrics = TokenRates
/**
* Effect-based offline handler for the retry schedule.
* Shows offline status, waits for network reconnection or user rejection.
+17 -1
View File
@@ -103,6 +103,7 @@ interface ProcessorContext extends Input {
reasoningMap: Record<string, SessionV1.ReasoningPart>
// kilocode_change start
stepStart: number
stepStartDate: number | undefined
step: { reasoning: boolean; text: boolean; tool: boolean }
// kilocode_change end
v2AssistantMessageID: SessionMessage.ID | undefined
@@ -158,6 +159,7 @@ export const layer = Layer.effect(
// kilocode_change start
telemetry: input.telemetry,
stepStart: 0,
stepStartDate: undefined,
step: { reasoning: false, text: false, tool: false },
// kilocode_change end
v2AssistantMessageID: undefined,
@@ -806,6 +808,7 @@ export const layer = Layer.effect(
case "step-start":
// kilocode_change start
ctx.stepStart = performance.now()
ctx.stepStartDate = Date.now()
ctx.step = { reasoning: false, text: false, tool: false }
if (!ctx.snapshot)
ctx.snapshot = yield* snapshot.track({
@@ -826,6 +829,7 @@ export const layer = Layer.effect(
sessionID: ctx.sessionID,
snapshot: ctx.snapshot,
type: "step-start",
time: { start: ctx.stepStartDate }, // kilocode_change
})
return
@@ -866,12 +870,22 @@ export const layer = Layer.effect(
// kilocode_change start - guard against finish-step without start-step:
// ctx.stepStart is 0 until `start-step` fires, which would feed a
// huge bogus `elapsed` into telemetry. Fall back to now().
const endDate = Date.now()
const elapsedMs = Math.round(
performance.now() - (ctx.stepStart || performance.now()),
)
const startDate = ctx.stepStartDate ?? (Number.isFinite(elapsedMs) ? endDate - elapsedMs : endDate)
const metrics = KiloSessionProcessor.computeMetrics({
providerMetadata: value.providerMetadata,
tokens: usage.tokens,
elapsedMs,
})
KiloSessionProcessor.trackStep({
sessionID: ctx.sessionID,
model: ctx.model,
tokens: usage.tokens,
cost: usage.cost,
elapsed: Math.round(performance.now() - (ctx.stepStart || performance.now())),
elapsed: elapsedMs,
telemetry: ctx.telemetry,
})
// kilocode_change end
@@ -903,7 +917,9 @@ export const layer = Layer.effect(
messageID: ctx.assistantMessage.id,
sessionID: ctx.assistantMessage.sessionID,
type: "step-finish",
time: { start: startDate, end: endDate, elapsed: elapsedMs }, // kilocode_change
...(model ? { model } : {}), // kilocode_change
...(metrics ? { metrics } : {}), // kilocode_change
tokens: usage.tokens,
cost: usage.cost,
})
@@ -0,0 +1,80 @@
// kilocode_change - new file
import { describe, expect, test } from "bun:test"
import { computeMetrics, formatRate } from "@/kilocode/session/metrics"
const tokens = {
input: 100,
output: 50,
reasoning: 0,
cache: { read: 0, write: 0 },
}
describe("kilocode.session.metrics.computeMetrics", () => {
test("derives generation rate from elapsed time", () => {
const metrics = computeMetrics({
tokens: { ...tokens, output: 100 },
elapsedMs: 1000,
})
expect(metrics?.source).toBe("computed")
expect(metrics?.generation).toBeCloseTo(100)
expect(metrics?.prompt).toBeUndefined()
})
test("returns undefined when there are no generation tokens", () => {
const metrics = computeMetrics({
tokens: { ...tokens, output: 0, reasoning: 0 },
elapsedMs: 2000,
})
expect(metrics).toBeUndefined()
})
test("guards against zero elapsed time", () => {
const metrics = computeMetrics({
tokens: { ...tokens, output: 50 },
elapsedMs: 0,
})
expect(metrics).toBeUndefined()
})
test("ignores providerMetadata until the upstream wiring lands (see #6579)", () => {
// llama.cpp surfaces prompt_per_second / predicted_per_second, but the
// upstream AI SDK drops them before the raw usage reaches our adapter.
// Until a metadataExtractor is wired into createOpenAICompatible, the
// provider source is unreachable — exercise the tolerance here.
const metrics = computeMetrics({
providerMetadata: {
llama: { prompt_per_second: 412.3, predicted_per_second: 28.7 },
},
tokens: { ...tokens, output: 100 },
elapsedMs: 2000,
})
expect(metrics?.source).toBe("computed")
expect(metrics?.generation).toBeCloseTo(50)
expect(metrics?.prompt).toBeUndefined()
})
test("tolerates missing providerMetadata", () => {
const metrics = computeMetrics({
tokens: { ...tokens, output: 200 },
elapsedMs: 4000,
})
expect(metrics?.source).toBe("computed")
expect(metrics?.generation).toBeCloseTo(50)
expect(metrics?.prompt).toBeUndefined()
})
})
describe("kilocode.session.metrics.formatRate", () => {
test.each([
[0, "0 t/s"],
[12, "12 t/s"],
[412.5, "412.5 t/s"],
[12345, "12,345 t/s"],
] as const)("formats %f as %s", (input, expected) => {
expect(formatRate(input)).toBe(expected)
})
test("returns zero string for negative inputs", () => {
expect(formatRate(-5)).toBe("0 t/s")
})
})
@@ -0,0 +1,118 @@
import { describe, expect, test } from "bun:test"
import {
aggregateMetrics,
formatRateValue,
hasMetrics,
throughputLabel,
} from "../../../src/kilocode/plugins/model-usage"
const step = (metrics: { generation?: number }) => ({
metrics: { source: "computed" as const, ...metrics },
generated: 0,
})
const weightedStep = (overrides: {
generation: number
output: number
reasoning?: number
elapsedMs: number
}) => ({
metrics: { generation: overrides.generation, source: "computed" as const },
generated: overrides.output + (overrides.reasoning ?? 0),
elapsedMs: overrides.elapsedMs,
output: overrides.output,
reasoning: overrides.reasoning ?? 0,
})
describe("kilocode.plugins.model-usage throughput helpers", () => {
test("formatRateValue renders positive values with grouping", () => {
expect(formatRateValue(412)).toBe("412 t/s")
expect(formatRateValue(412.5)).toBe("412.5 t/s")
expect(formatRateValue(12345)).toBe("12,345 t/s")
expect(formatRateValue(28.7)).toBe("28.7 t/s")
})
test("formatRateValue falls back to dash for missing or bogus values", () => {
expect(formatRateValue(undefined)).toBe("-")
expect(formatRateValue(0)).toBe("-")
expect(formatRateValue(-5)).toBe("-")
expect(formatRateValue(Number.NaN)).toBe("-")
expect(formatRateValue(Infinity)).toBe("-")
})
test("throughputLabel centralizes the generation-speed label so a future i18n sweep is one file", () => {
expect(throughputLabel.generation).toBe("Generation speed")
})
test("surfaces the most recent non-empty generation rate as the snapshot (fallback)", () => {
// Fallback path — used when callers don't pass timing on the wire.
// The weighted path is exercised by the dedicated tests below.
const aggregated = aggregateMetrics([
{ ...step({ generation: 20 }), generated: 100 },
{ ...step({ generation: 60 }), generated: 300 },
])
expect(aggregated.generation).toBe(60)
})
test("skips samples without metrics", () => {
const aggregated = aggregateMetrics([
{ metrics: undefined, generated: 100, elapsedMs: 1000 },
weightedStep({ generation: 40, output: 50, elapsedMs: 1000 }),
])
// weighted step contributes (50, 1000) → 50 t/s.
expect(aggregated.generation).toBe(50)
})
test("weights samples by elapsed time across steps", () => {
const aggregated = aggregateMetrics([
weightedStep({ generation: 100, output: 100, elapsedMs: 1000 }),
weightedStep({ generation: 50, output: 200, elapsedMs: 4000 }),
])
// totalGenerated=300, totalElapsedMs=5000 → 60 t/s
expect(aggregated.generation).toBe(60)
})
test("includes reasoning tokens in the weighted numerator", () => {
const aggregated = aggregateMetrics([
weightedStep({ generation: 200, output: 50, reasoning: 150, elapsedMs: 1000 }),
])
// (50 + 150) tokens / 1000 ms = 200 t/s
expect(aggregated.generation).toBe(200)
})
test("falls back to last-wins snapshot when no sample carries timing", () => {
const aggregated = aggregateMetrics([
{ ...step({ generation: 20 }), generated: 100 },
{ ...step({ generation: 60 }), generated: 300 },
])
expect(aggregated.generation).toBe(60)
})
test("skips zero-weight samples when picking the latest snapshot", () => {
const aggregated = aggregateMetrics([
{ ...step({ generation: 9999 }), generated: 0 },
{ ...step({ generation: 25 }), generated: 50 },
])
expect(aggregated.generation).toBe(25)
})
test("returns empty aggregate when nothing has metrics", () => {
expect(aggregateMetrics([])).toEqual({})
expect(aggregateMetrics([{ metrics: undefined, generated: 100 }])).toEqual({})
})
test("ignores bogus per-call values without poisoning the snapshot", () => {
const aggregated = aggregateMetrics([
{ ...step({ generation: -1 }), generated: 100 },
{ ...step({ generation: Number.POSITIVE_INFINITY }), generated: 100 },
{ ...step({ generation: 30 }), generated: 50 },
])
expect(aggregated.generation).toBe(30)
})
test("hasMetrics gates opportunistic rendering", () => {
expect(hasMetrics(undefined)).toBeFalse()
expect(hasMetrics({})).toBeFalse()
expect(hasMetrics({ generation: 12 })).toBeTrue()
})
})
+13
View File
@@ -766,6 +766,9 @@ export type StepStartPart = {
messageID: string
type: "step-start"
snapshot?: string
time?: {
start: number
}
}
export type StepFinishPart = {
@@ -779,6 +782,16 @@ export type StepFinishPart = {
providerID: string
modelID: string
}
metrics?: {
prompt?: number
generation?: number
source: "provider" | "computed"
}
time?: {
start: number
end: number
elapsed: number
}
cost: number
tokens: {
total?: number
+46
View File
@@ -26021,6 +26021,17 @@
},
"snapshot": {
"type": "string"
},
"time": {
"type": "object",
"properties": {
"start": {
"type": "number",
"minimum": 0
}
},
"required": ["start"],
"additionalProperties": false
}
},
"required": ["id", "sessionID", "messageID", "type"],
@@ -26064,6 +26075,41 @@
"required": ["providerID", "modelID"],
"additionalProperties": false
},
"metrics": {
"type": "object",
"properties": {
"prompt": {
"type": "number"
},
"generation": {
"type": "number"
},
"source": {
"type": "string",
"enum": ["provider", "computed"]
}
},
"required": ["source"],
"additionalProperties": false
},
"time": {
"type": "object",
"properties": {
"start": {
"type": "number",
"minimum": 0
},
"end": {
"type": "number",
"minimum": 0
},
"elapsed": {
"type": "number"
}
},
"required": ["start", "end", "elapsed"],
"additionalProperties": false
},
"cost": {
"type": "number"
},