Compare commits

...

42 Commits

Author SHA1 Message Date
candieduniverse 87bf6c3c0c Move latency observer into a non-dev tab in Settings 2026-03-17 10:40:04 -07:00
candieduniverse 526f2b437c Update latency observer backport guidance 2026-03-17 10:40:04 -07:00
candieduniverse 04442774df Reconcile latency observer Step 4 progress 2026-03-17 10:40:04 -07:00
candieduniverse 038ef177ed Validate observed latency scenarios in tests 2026-03-17 10:40:04 -07:00
candieduniverse a98124de38 Sync latency observer architecture checklist 2026-03-17 10:40:04 -07:00
candieduniverse 166a086f92 Show chunk timing capability in debug panel 2026-03-17 10:40:04 -07:00
candieduniverse 9d50aabaaa Mark latency observer docs items complete 2026-03-17 10:40:03 -07:00
candieduniverse f28e9c8b7f Track full-state chunk delivery timing 2026-03-17 10:40:03 -07:00
candieduniverse bfc4cbe4b6 Align latency observer checklist completions 2026-03-17 10:40:03 -07:00
candieduniverse 1a36031509 Sync chunk-to-webview checklist progress 2026-03-17 10:40:03 -07:00
candieduniverse ffd5ea3974 Track chunk-to-webview observer timing 2026-03-17 10:40:03 -07:00
candieduniverse b8fd91cb52 Refine latency observer portability notes 2026-03-17 10:40:03 -07:00
candieduniverse dcff01ba09 Document latency observer portability probe 2026-03-17 10:40:03 -07:00
candieduniverse 92b7a14098 Extend latency observer regression coverage 2026-03-17 10:40:03 -07:00
candieduniverse 8a01ac3f88 Sync latency observer lifecycle checklist 2026-03-17 10:40:03 -07:00
candieduniverse 2cf80da8bb Track first partial latency observer timing 2026-03-17 10:40:03 -07:00
candieduniverse ff547e8ab1 Add latency observer scenario launch action 2026-03-17 10:40:03 -07:00
candieduniverse d960bc5860 Update latency observer churn checklist 2026-03-17 10:40:03 -07:00
candieduniverse bf7117f4eb Add per-request latency observer churn summaries 2026-03-17 10:40:03 -07:00
candieduniverse 47e21aa137 Align latency observer checklist progress 2026-03-17 10:40:03 -07:00
candieduniverse e486bcf44e Refresh latency observer progress markers 2026-03-17 10:40:03 -07:00
candieduniverse 11083a70d3 Sync latency observer summary checklist 2026-03-17 10:40:03 -07:00
candieduniverse 726ddb5ccc Test latency observer stream adapters 2026-03-17 10:40:03 -07:00
candieduniverse 310538ed67 Track first full-state latency observer timing 2026-03-17 10:40:03 -07:00
candieduniverse cefa7dc566 Stabilize latency observer capability snapshots 2026-03-17 10:40:02 -07:00
candieduniverse 79a6f47bdc Mark latency observer overlap coverage complete 2026-03-17 10:40:02 -07:00
candieduniverse ca6e9e35ae Sync latency observer plan checklist 2026-03-17 10:40:02 -07:00
candieduniverse 51e1e7e2ef Test latency observer session reset 2026-03-17 10:40:02 -07:00
candieduniverse 386a59a568 Add latency observer session reset 2026-03-17 10:40:02 -07:00
candieduniverse 2747bbe288 Clarify latency observer portability guidance 2026-03-17 10:40:02 -07:00
candieduniverse fdf973487d Add export fallback coverage for latency observer 2026-03-17 10:40:02 -07:00
candieduniverse 73625029a3 Harden latency observer export test 2026-03-17 10:40:02 -07:00
candieduniverse 27fd9b9f16 Track latency observer task UI deltas 2026-03-17 10:40:02 -07:00
candieduniverse c92c5c5b24 Track latency observer state and partial payload bytes 2026-03-17 10:40:02 -07:00
candieduniverse b259a14b9c Document latency observer interpretation guidance 2026-03-17 10:40:02 -07:00
candieduniverse 7302a1a6ea Add latency observer scenario guidance 2026-03-17 10:40:02 -07:00
candieduniverse 2286050fed Add latency observer ping presets and capability UI 2026-03-17 10:40:02 -07:00
candieduniverse 3857e3927d Add latency observer session export 2026-03-17 10:40:02 -07:00
candieduniverse 76709a8aef Add richer latency observer session metrics 2026-03-17 10:40:02 -07:00
candieduniverse bc2bde95e3 Add task lifecycle latency observer hooks 2026-03-17 10:40:02 -07:00
candieduniverse 4205f6a969 Add initial latency observer metric contract and ping probe 2026-03-17 10:40:02 -07:00
candieduniverse e982b97324 Create implementation plan doc 2026-03-17 10:40:01 -07:00
24 changed files with 2771 additions and 1 deletions
@@ -0,0 +1,683 @@
# Local Latency Observer Plan for Remote-Workspace Comparison
This document describes a plan for building a **locally visible latency observer** for Cline that can be added to a branch derived from `main` **and** to a branch derived from `eve_troubleshooting-remote-workspaces`, so that behavior can be observed and compared under the same measurement mechanism.
The goal is not to recreate the old `arafatkatze/ping-pong-test` branch literally. That older work is useful as inspiration because it validated an important product-development idea:
> engineers need a fast, visible, low-friction way to observe latency behavior locally while iterating.
But the repository has changed substantially since then. The right move now is to design a measurement mechanism that fits the current architecture, is useful on both baseline and candidate branches, and helps engineers compare **user-perceived latency behavior**, not just transport RTT in isolation.
---
## What This Artifact Is
This is an **implementation plan / extraction plan** for a Staff+ level distributed systems and infrastructure engineer. It is not a greenfield “brainstorming” doc and it is not a narrow PR description.
Its purpose is to define a branch-portable observer mechanism that can:
- be added to `main`,
- be added to `eve_troubleshooting-remote-workspaces`,
- surface useful measurements locally,
- and support apples-to-apples comparison of baseline vs improved behavior.
The quality bar for this plan is:
- the mechanism must be easy to reason about,
- useful during iterative development,
- minimally invasive to product behavior,
- and explicit about what is being measured and what is not.
---
## Reference Inputs and Inspiration
This plan should be read alongside:
1. `docs/remote-workspace-latency-branch-analysis-report.md`
2. `docs/remote-workspace-latency-improvement-plan.md`
3. the current telemetry/validation tooling already present in this repo
4. the older inspiration branch `origin/arafatkatze/ping-pong-test`
The old branch appears to have included a visible `LatencyTester` UI in settings that measured gRPC ping/pong latency with variable payload sizes. That is a useful inspiration because it made latency locally visible and easy to probe. However, it is not enough by itself for the current task, because the current problem is broader than pure transport RTT.
Today we care about multiple layers of user-perceived latency, including:
- UI transport latency,
- snapshot / delta delivery behavior,
- chunk-to-visible-update timing,
- request-start latency,
- and hot-path churn during active execution.
So the modern observer should retain the spirit of “easy visible local latency testing” while measuring the parts of the system that matter most for the remote-workspace problem.
---
## Core Goal
Build a **single observer mechanism** that can be added to both baseline and candidate branches and used to compare their behavior in a way that is:
- visible to the developer locally,
- useful during manual iteration,
- scriptable enough to support repeatable experiments,
- and robust to architectural differences between `main` and `eve_troubleshooting-remote-workspaces`.
---
## Non-Goals
To keep the effort focused, this observer should **not** try to be all of the following at once:
- a benchmark harness for every product surface,
- a production telemetry dashboard,
- a load-testing framework,
- or a branch-specific debugging toy that only works on one side of the comparison.
Instead, it should be a **branch-portable latency observation layer** with a small visible UI and enough instrumentation hooks to answer the comparison questions we care about.
---
## Guiding Principles
Before implementation, keep these principles in mind:
- **Optimize for comparability over cleverness.** A simpler observer that works identically on both branches is better than a richer observer that only works on one.
- **Measure user-perceived boundaries, not just transport boundaries.** Ping/pong RTT is useful, but it is only one slice of the problem.
- **Separate observer plumbing from branch-specific latency improvements.** The observer should not depend on the candidate branchs optimizations in order to function.
- **Make local visibility first-class.** Developers should be able to see measurements in the UI during manual testing, not only in exported logs.
- **Be smart about backward compatibility.** If the observer can gracefully detect missing richer metrics on `main` and still provide useful output, that is preferable to forcing two divergent observer implementations.
---
## Recommended High-Level Shape
The best shape for this feature is a **three-layer observer**:
1. **Transport probe layer** — lightweight ping/pong style measurements with variable payload sizes.
2. **Task-execution latency layer** — timing measurements around request start, first visible update, state update frequency/size, and optional chunk-to-visible metrics where supported.
3. **Local observer UI layer** — a developer-facing panel or settings section that displays current values, rolling history, and recent logs in a human-comprehensible way.
This approach preserves the useful simplicity of the old `LatencyTester` while expanding it into something aligned with the current problem space.
---
## Why a Single Branch-Portable Observer Matters
If the observer mechanism differs substantially between baseline and candidate branches, then comparison quality degrades immediately. Engineers start asking questions like:
- is the difference real or an artifact of the measurement tool?
- does one branch surface richer metrics only because it has extra plumbing?
- are we comparing the same event boundaries?
The smart move is to design the observer so that:
- a **minimum shared metric set** works on both branches,
- richer metrics can appear opportunistically where supported,
- and the UI makes clear which metrics are available vs unavailable on the current branch.
That way a single observer branch can be rebased/cherry-picked onto both `main` and `eve_troubleshooting-remote-workspaces` with minimal divergence.
---
## Recommended Measurement Categories
## 1. Transport probe metrics
These are inspired most directly by the old ping-pong tester.
Measure:
- [x] round-trip UI service ping latency
- [x] effect of varying payload sizes
- [x] repeated sample min/max/avg/current
- [x] continuous test mode for drift/jitter observation
Why it matters:
- helps reveal raw extension-host ↔ webview transport overhead,
- useful in remote environments like Codespaces / SSH / remote containers,
- easy for a developer to understand immediately.
Limitations:
- does **not** directly measure task-execution UX,
- should be treated as a lower-level signal, not the only KPI.
## 2. Task lifecycle metrics
Measure:
- [x] task creation / initialization latency
- [x] request-start latency
- [x] time to first visible assistant update
- [x] time to first full-state update
- [x] time to first partial or delta update where available
Why it matters:
- these metrics align better with the users lived experience than transport RTT alone,
- they can show whether the candidate branch improves perceived responsiveness at important boundaries.
## 3. Hot-path churn metrics
Measure:
- [x] number of full-state pushes per request
- [x] total full-state payload bytes per request
- [x] number of partial-message events per request
- [x] number of partial-message events per request
- [x] total partial-message payload bytes per request
- [x] number of task UI deltas per request where supported
- [x] number of task UI deltas per request where supported
- [x] persistence flush counts where supported
Why it matters:
- these are the metrics most closely tied to the remote-workspace improvements described in the analysis report,
- they help explain *why* one branch feels faster, not just whether it does.
## 4. Comparison session metadata
Record:
- [x] branch / commit identity
- [x] local vs remote environment marker
- [x] selected scenario / payload size / cadence mode
- [x] timestamped session logs
Why it matters:
- makes manual comparisons auditable,
- prevents confusion when developers are switching between baseline and candidate runs.
---
## Branch-Portability Strategy
This is the most important design constraint.
The observer should be implemented so that it degrades gracefully:
### Minimum cross-branch baseline
These should work on both `main` and `eve_troubleshooting-remote-workspaces` with little or no branch-specific logic:
- [x] ping/pong RTT with configurable payload size
- [x] task initialization timing if a simple observer hook can be added
- [x] first state update timing
- [x] visible branch/commit/session labeling
- [x] local UI for logs and rolling stats
### Opportunistic richer metrics
These may only exist natively on `eve_troubleshooting-remote-workspaces` or may require additional light plumbing on `main`:
- [x] chunk-to-webview timing
- [x] full-state post counts / bytes
- [x] partial-message event counts
- [x] task UI delta counts
- [x] persistence flush metrics
### Recommended implementation rule
The observer UI should not fail if a metric is unavailable.
Instead it should display something like:
- “supported and active”
- “unsupported on this branch”
- or “observer hook not installed”
That makes the same branch usable on both baselines.
---
## Recommended Developer UX
The observer should be easy enough to use that engineers actually use it during iteration.
Recommended UI features:
- [x] a dedicated dev-facing section in Settings or a dev/debug panel
- [x] buttons for:
- [x] single ping
- [x] continuous ping
- [x] test all payload sizes
- [x] start observed task scenario
- [x] reset stats
- [x] visible stats cards / table for current/min/max/avg
- [x] recent logs panel
- [x] session metadata display (branch, commit, environment)
- [x] explicit note describing which metrics are branch-portable vs richer-on-candidate
The point is to make the observer *pleasant enough* that it becomes part of the engineering workflow rather than a script that only gets used once.
---
## Recommended Implementation Plan
## Step 1 — Define the minimum shared metric contract
### Goal
Create the smallest metric surface that can work on both `main` and `eve_troubleshooting-remote-workspaces`.
### Mental model
The observer succeeds if its core contract is branch-portable. Everything richer is an enhancement.
### Work
- [x] Define a shared observer metric model for:
- [x] ping RTT samples
- [x] task initialization / request-start samples
- [x] first-visible-update samples
- [x] optional richer counters
- [x] Mark each metric as either:
- [x] required/shared
- [x] optional/richer
### Detailed code changes
- Add a shared type module, e.g. `src/shared/LatencyObserver.ts` or similar, that defines:
- [x] sample types,
- [x] rolling stats shape,
- [x] branch capability flags,
- [x] session metadata shape.
### Tests
- [x] Unit test: metric aggregation shape is stable.
- [x] Unit test: missing optional metrics do not break the model.
---
## Step 2 — Reintroduce a modern ping/pong transport probe
### Goal
Port the useful idea from the old `LatencyTester` branch into the current architecture in a modernized form.
### Mental model
This is the simplest locally visible measurement and should serve as the “does the pipe feel slow?” test.
### Work
- [x] Add or confirm a simple UI-service ping endpoint that accepts payload size.
- [x] Measure round-trip latency in the webview using `performance.now()`.
- [x] Support multiple payload sizes and continuous testing.
### Detailed code changes
- Inspect whether the current codebase already has a UI ping path analogous to the old branchs `UiServiceClient.ping(...)` flow.
- If missing, add:
- [x] a lightweight request/response endpoint in the UI service layer,
- [x] optional payload-size expansion to simulate message size effects.
- In the webview, add a dev-facing component similar in spirit to the old `LatencyTester.tsx`, but keep it isolated behind a dev/debug visibility gate.
### Tests
- [x] Unit/integration test: ping returns successfully.
- [x] Test: payload size selection is reflected in the request.
- [x] Regression test: continuous mode handles pending request overlap safely.
---
## Step 3 — Add a task-observer hook layer that is intentionally branch-portable
### Goal
Create a small observer hook interface that can be invoked from task lifecycle boundaries on both branches.
### Mental model
Do **not** couple the observer directly to candidate-branch-only telemetry structures. Instead create a tiny observer API that can be wired into either branch with minimal intrusion.
### Work
- [x] Define a branch-portable observer service or callback layer.
- [x] Add hooks for:
- [x] task initialization start/end
- [x] request start
- [x] first visible update
- [x] request completion
- [x] Keep richer optional hooks for state-post counts, chunk-to-webview, etc.
### Detailed code changes
- Add a small service such as `LatencyObserverService` or similarly named utility in `src/services/` or `src/core/controller/`.
- It should:
- [x] record timestamps,
- [x] aggregate per-session/per-request values,
- [x] expose results to the UI layer,
- [x] not require the full candidate telemetry pipeline to exist.
### Tests
- [x] Unit test: initialization and request lifecycle timings aggregate correctly.
- [x] Unit test: optional hooks can be absent without crashing.
---
## Step 4 — Expose richer metrics when available, without making them required
### Goal
Allow the same observer branch to become more informative on `eve_troubleshooting-remote-workspaces` without breaking on `main`.
### Mental model
Think of this as capability detection, not branch forking.
### Work
- [x] Integrate existing richer latency metrics when present:
- [ ] `task.latency_metrics`
- [x] initialization telemetry
- [x] payload-size accounting
- [x] chunk-to-webview summaries
- [x] Surface unsupported metrics as unavailable instead of failing.
### Detailed code changes
- Where current richer telemetry exists (candidate branch or current branch after instrumentation work), add adapters that feed it into the observer UI.
- On `main`, either:
- [ ] wire minimal equivalents if easy,
- [x] or leave those fields explicitly unsupported.
### Tests
- [x] Unit test: richer metrics adapter populates observer state when data exists.
- [x] Unit test: missing richer metrics are displayed as unavailable.
Progress note:
- [x] Current branch now records full-state push counts plus total serialized full-state bytes.
- [x] Current branch now records partial-message event counts plus total serialized partial-message bytes.
- [x] Current branch now records task UI delta events from message-state changes.
---
## Step 5 — Build a visible local observer UI
### Goal
Create a developer-facing UI for running probes and reading results locally.
### Mental model
If the observer is only visible in logs, engineers will underuse it. The UI should make latency behavior tangible.
### Work
- [x] Add a dev/debug observer panel or settings section.
- [x] Show transport stats, lifecycle stats, optional richer metrics, and logs.
- [x] Display branch/session/environment metadata.
### Detailed code changes
- Add a component in `webview-ui/src/components/settings/` or a more appropriate dev/debug location.
- Prefer a design inspired by the old `LatencyTester`:
- [x] simple controls,
- [x] rolling stats,
- [x] recent logs,
- [x] easy reset.
- Add session identity fields:
- [x] git branch or commit label if available from backend,
- [x] environment marker (local vs remote),
- [x] capability flags for richer metrics.
### Tests
- [x] Component test: controls trigger expected actions.
- [x] Component test: unavailable metrics render intelligibly.
- [x] Regression test: UI remains hidden or low-noise outside intended dev/debug usage.
---
## Step 6 — Add scenario-driven observation, not just passive measurement
### Goal
Give developers a way to drive comparable scenarios, not just watch idle telemetry.
### Mental model
A latency observer becomes far more valuable when it can help the developer answer:
- how does branch A behave during a simple request?
- how does branch B behave during the same request?
- what happens under larger payloads or high-churn streaming?
### Work
- [x] Define a small set of recommended scenarios:
- [x] pure ping test
- [x] short assistant response
- [x] long streaming response
- [x] tool-heavy / high-churn scenario
- [x] large-file-write-adjacent scenario if feasible
- [x] Add UI affordances or documented steps for running those scenarios repeatedly.
### Detailed code changes
- This does not necessarily require the UI to generate tasks itself.
- At minimum, add a documented scenario matrix and a way for the observer to reset/session-label around a manual run.
- If practical, add a small “start known validation task” action in dev mode.
### Tests
- [x] Validation test: scenario runs produce observer output.
- [x] Regression test: observer reset cleanly separates sessions.
---
## Step 7 — Support export and comparison workflow
### Goal
Make it easy to compare baseline vs candidate observations after local runs.
### Mental model
Local visibility is great, but engineers also need artifacts they can compare side by side.
### Work
- [x] Add export of observer session data to JSON.
- [x] Include branch/commit/environment metadata in export.
- [x] Make exported structure easy to diff or post-process.
- [x] Add export of observer session data to JSON.
- [x] Include branch/commit/environment metadata in export.
- [x] Make exported structure easy to diff or post-process.
### Detailed code changes
- Add an export button or command that writes a session artifact.
- Keep schema intentionally simple:
- [x] session metadata,
- [x] ping samples and aggregates,
- [x] lifecycle measurements,
- [x] richer metrics if available,
- [x] recent logs or event markers.
### Tests
- [x] Unit test: exported schema is stable.
- [x] Regression test: export works even when optional metrics are unavailable.
---
## Step 8 — Make the observer safe for use on both `main` and candidate branches
### Goal
Minimize branch-specific surgery so the same observer work can be reused for baseline and future comparison runs.
### Mental model
The observer should behave like a thin compatibility layer, not a second product architecture.
### Work
- [x] Avoid relying on candidate-only classes for core functionality.
- [x] Gate richer behavior via capability detection.
- [x] Keep backend hooks shallow and intentionally placed.
### Detailed code changes
- Prefer hooks at stable abstraction boundaries:
- [x] UI service layer for ping/pong,
- [x] controller/task lifecycle boundaries for timing,
- [x] optional adapters for richer telemetry.
- Avoid deep invasive branch-specific assumptions where possible.
### Tests
- [ ] Manual validation on `main`.
- [ ] Manual validation on `eve_troubleshooting-remote-workspaces`.
- [ ] Confirm that the same observer branch can be adapted onto both with minimal or no code drift.
Portability note:
- [x] Initial portability probe completed against local `main` and `eve_troubleshooting-remote-workspaces` worktrees.
- [x] Direct cherry-pick of the current observer implementation does **not** apply cleanly yet.
- [x] Clean worktree inspection shows the target branches already contain some observer-related anchor points, including:
- [x] `src/core/controller/ui/subscribeToPartialMessage.ts`
- [x] `webview-ui/src/components/settings/sections/DebugSection.tsx`
- [x] However, the richer shared observer modules and tests on the current branch are still absent on the target branches, including:
- [x] `src/shared/LatencyObserver.ts`
- [x] `src/services/latency/LatencyObserverService.ts`
- [x] `src/services/latency/__tests__/LatencyObserverService.test.ts`
- [x] `webview-ui/src/components/settings/sections/DebugSection.spec.tsx`
- [x] So portability still requires an extraction/backport sequence rather than a pure cherry-pick, but the enabling UI/controller surfaces are partially present already.
Recommended extraction/backport sequence:
1. Add the shared observer type surface first:
- `src/shared/LatencyObserver.ts`
2. Add the observer service and unit coverage next:
- `src/services/latency/LatencyObserverService.ts`
- `src/services/latency/__tests__/LatencyObserverService.test.ts`
3. Wire controller/task integration points after the shared service exists:
- `src/core/controller/index.ts`
- `src/core/controller/state/subscribeToState.ts`
- `src/core/controller/ui/subscribeToPartialMessage.ts`
- `src/core/controller/ui/resetLatencyObserver.ts`
- `src/core/task/index.ts`
- `src/core/task/message-state.ts`
4. Add webview state plumbing and the debug panel once backend snapshots compile:
- `webview-ui/src/components/settings/sections/DebugSection.tsx`
- supporting grpc/webview state wiring on the target branch as needed
5. Backport test coverage last, once the target branch shape is stable:
- `src/test/latency-observer-streams.test.ts`
- `src/shared/__tests__/LatencyObserver.test.ts`
- `webview-ui/src/components/settings/sections/DebugSection.spec.tsx`
This sequence should minimize conflict churn because it introduces the shared contracts before the dependent task/controller and UI layers.
---
## Step 9 — Document how to interpret the measurements
### Goal
Prevent engineers from misusing the observer or over-interpreting low-level numbers.
### Mental model
Numbers without interpretation guidance lead to bad product decisions.
### Work
- [x] Document what ping RTT does and does not mean.
- [x] Document how to compare first-visible-update and state-churn metrics.
- [x] Document that the most important measurement is user-perceived latency, not just a single low-level counter.
### Detailed code changes
- Add an adjacent doc section or inline UI help text explaining:
- [x] transport RTT is a lower-level signal,
- [x] task lifecycle timings are closer to user perception,
- [x] churn metrics help explain cause,
- [x] branch comparison should use the same scenario and environment.
### Tests
- [x] No code-heavy tests needed; verify docs/UI copy is clear.
---
## Minimal Coherent Extraction Boundary
The smallest useful PR for this observer should probably include:
- a modernized ping/pong probe,
- the shared observer metric contract,
- a small local UI panel,
- a minimal task lifecycle timing hook layer,
- and an export path.
What should not be split apart if avoidable:
- observer metric model from the UI,
- ping probe from the visible stats surface,
- branch/session metadata from exported artifacts,
- and capability labeling from optional richer metrics.
---
## Common Failure Modes
Watch for these explicitly:
- building a tester that only measures transport RTT and not user-perceived task latency,
- building a tester that only works on one branch,
- tightly coupling the observer to candidate-only telemetry plumbing,
- surfacing metrics without clarifying whether they are unavailable vs zero,
- and creating a debug UI that is so hidden or awkward that engineers stop using it.
---
## Recommended Validation Workflow
Once implemented, a good manual workflow would be:
1. apply the observer branch to `main`
2. run a fixed set of scenarios and export results
3. apply the same observer branch to `eve_troubleshooting-remote-workspaces`
4. run the same scenarios in the same environment
5. compare:
- [ ] ping RTT behavior
- [ ] initialization and first-visible-update timing
- [ ] state push counts/bytes where available
- [ ] partial/delta event behavior where available
- [ ] subjective smoothness during the same manual workflow
That gives both a human and a machine-readable comparison story.
---
## Developer Checklist Summary
- [x] Define a branch-portable shared metric contract
- [x] Implement a modernized ping/pong transport probe
- [x] Add a shallow branch-portable task observer hook layer
- [x] Adapt richer metrics where available without making them required
- [x] Build a visible local observer UI
- [x] Support scenario-driven observation
- [x] Add export/comparison workflow support
- [ ] Validate portability on both `main` and `eve_troubleshooting-remote-workspaces`
- [x] Document how to interpret the results
---
## Final Mental Model Recap
- **The old ping-pong tester is inspiration, not a blueprint.**
- **The modern observer should measure both pipe latency and task-experience latency.**
- **A single branch-portable observer is much more valuable than two branch-specific testers.**
- **The best observer is one engineers will actually use while iterating.**
If implemented well, this mechanism will give the team a practical way to compare current behavior and future behavior using the same local visibility tool, which is exactly what the next stage of this work needs.
+6
View File
@@ -234,6 +234,12 @@ message ShowWebviewEvent {
// UiService provides methods for managing UI interactions
service UiService {
// Lightweight ping/pong endpoint for measuring local UI transport latency.
rpc pingLatencyProbe(BytesRequest) returns (Int64);
// Resets the backend latency observer session counters and logs.
rpc resetLatencyObserver(EmptyRequest) returns (Empty);
// Scrolls to a specific settings section in the settings view
rpc scrollToSettings(StringRequest) returns (KeyValuePair);
+12
View File
@@ -32,6 +32,7 @@ import { OcaAuthService } from "@/services/auth/oca/OcaAuthService"
import { LogoutReason } from "@/services/auth/types"
import { BannerService } from "@/services/banner/BannerService"
import { featureFlagsService } from "@/services/feature-flags"
import { getLatencyObserverService } from "@/services/latency/LatencyObserverService"
import { getDistinctId } from "@/services/logging/distinctId"
import { telemetryService } from "@/services/telemetry"
import { ClineExtensionContext } from "@/shared/cline"
@@ -40,6 +41,7 @@ import { ShowMessageType } from "@/shared/proto/host/window"
import { Logger } from "@/shared/services/Logger"
import { Session } from "@/shared/services/Session"
import { getLatestAnnouncementId } from "@/utils/announcements"
import { getCurrentGitBranch, getLatestGitCommitHash } from "@/utils/git"
import { getCwd, getDesktopDir } from "@/utils/path"
import { PromptRegistry } from "../prompts/system-prompt"
import {
@@ -279,6 +281,7 @@ export class Controller {
const cwd = this.workspaceManager?.getPrimaryRoot()?.path || (await getCwd(getDesktopDir()))
const taskId = historyItem?.id || Date.now().toString()
getLatencyObserverService().markTaskInitializationStart(taskId)
// Acquire task lock
let taskLockAcquired = false
@@ -911,6 +914,14 @@ export class Controller {
const version = ExtensionRegistryInfo.version
const clineConfig = ClineEnv.config()
const environment = clineConfig.environment
const workspaceCwd = this.workspaceManager?.getPrimaryRoot()?.path || (await getCwd(getDesktopDir()))
const [branch, commit] = await Promise.all([getCurrentGitBranch(workspaceCwd), getLatestGitCommitHash(workspaceCwd)])
getLatencyObserverService().setSessionMetadata({
branch: branch ?? undefined,
commit: commit ?? undefined,
environment,
platform,
})
const banners = BannerService.get().getActiveBanners() ?? []
const welcomeBanners = BannerService.get().getWelcomeBanners() ?? []
@@ -1000,6 +1011,7 @@ export class Controller {
banners,
welcomeBanners,
openAiCodexIsAuthenticated,
latencyObserver: getLatencyObserverService().getSnapshot(),
}
}
@@ -1,5 +1,6 @@
import { EmptyRequest } from "@shared/proto/cline/common"
import { State } from "@shared/proto/cline/state"
import { getLatencyObserverService } from "@/services/latency/LatencyObserverService"
import { telemetryService } from "@/services/telemetry"
import { ExtensionState } from "@/shared/ExtensionMessage"
import { Logger } from "@/shared/services/Logger"
@@ -40,14 +41,26 @@ export async function subscribeToState(
const initialStateJson = JSON.stringify(initialState)
recordStateSizeTelemetry(Buffer.byteLength(initialStateJson, "utf8"))
recordStateObserverMetrics(initialState)
try {
const startedAt = performance.now()
await responseStream(
{
stateJson: initialStateJson,
},
false, // Not the last message
)
const observer = getLatencyObserverService()
const endedAt = performance.now()
observer.setCapability("chunkToWebviewTiming", "supported")
observer.recordChunkToWebviewDelivery({
startedAt,
endedAt,
durationMs: Math.max(0, endedAt - startedAt),
label: "full-state-broadcast",
payloadBytes: Buffer.byteLength(initialStateJson, "utf8"),
})
} catch (error) {
Logger.error("Error sending initial state:", error)
activeStateSubscriptions.delete(responseStream)
@@ -68,15 +81,27 @@ export async function sendStateUpdate(state: ExtensionState): Promise<void> {
}
recordStateSizeTelemetry(Buffer.byteLength(stateJson, "utf8"))
recordStateObserverMetrics(state)
const promises = Array.from(activeStateSubscriptions).map(async (responseStream) => {
try {
const startedAt = performance.now()
await responseStream(
{
stateJson,
},
false, // Not the last message
)
const observer = getLatencyObserverService()
const endedAt = performance.now()
observer.setCapability("chunkToWebviewTiming", "supported")
observer.recordChunkToWebviewDelivery({
startedAt,
endedAt,
durationMs: Math.max(0, endedAt - startedAt),
label: "full-state-broadcast",
payloadBytes: Buffer.byteLength(stateJson, "utf8"),
})
} catch (error) {
Logger.error("Error sending state update:", error)
activeStateSubscriptions.delete(responseStream)
@@ -88,4 +113,19 @@ export async function sendStateUpdate(state: ExtensionState): Promise<void> {
function recordStateSizeTelemetry(sizeBytes: number): void {
telemetryService.captureGrpcResponseSize(sizeBytes, "cline.StateService", "subscribeToState")
const observer = getLatencyObserverService()
observer.incrementCounter("fullStatePushes")
observer.incrementCounter("fullStateBytes", sizeBytes)
observer.setCapability("fullStateMetrics", "supported")
}
function recordStateObserverMetrics(state: ExtensionState): void {
const taskId = state.currentTaskItem?.id
if (!taskId) {
return
}
const observer = getLatencyObserverService()
observer.recordFirstFullStateUpdate(taskId)
observer.setCapability("firstFullStateUpdate", "supported")
}
@@ -0,0 +1,6 @@
import { BytesRequest, Int64 } from "@shared/proto/cline/common"
import { Controller } from "../index"
export async function pingLatencyProbe(_controller: Controller, request: BytesRequest): Promise<Int64> {
return Int64.create({ value: request.value?.length ?? 0 })
}
@@ -0,0 +1,9 @@
import { Empty, EmptyRequest } from "@shared/proto/cline/common"
import { getLatencyObserverService } from "@/services/latency/LatencyObserverService"
import { Controller } from "../index"
export async function resetLatencyObserver(controller: Controller, _request: EmptyRequest): Promise<Empty> {
getLatencyObserverService().reset()
await controller.postStateToWebview()
return Empty.create({})
}
@@ -1,5 +1,6 @@
import { EmptyRequest } from "@shared/proto/cline/common"
import { ClineMessage } from "@shared/proto/cline/ui"
import { getLatencyObserverService } from "@/services/latency/LatencyObserverService"
import { Logger } from "@/shared/services/Logger"
import { getRequestRegistry, StreamingResponseHandler } from "../grpc-handler"
import { Controller } from "../index"
@@ -55,6 +56,14 @@ export function registerPartialMessageCallback(callback: PartialMessageCallback)
* @param partialMessage The ClineMessage to send
*/
export async function sendPartialMessageEvent(partialMessage: ClineMessage): Promise<void> {
const observer = getLatencyObserverService()
const payloadBytes = Buffer.byteLength(JSON.stringify(partialMessage), "utf8")
const startedAt = performance.now()
observer.incrementCounter("partialMessageEvents")
observer.incrementCounter("partialMessageBytes", payloadBytes)
observer.setCapability("partialMessageMetrics", "supported")
observer.setCapability("chunkToWebviewTiming", "supported")
// Send to gRPC stream subscribers
const streamPromises = Array.from(activePartialMessageSubscriptions).map(async (responseStream) => {
try {
@@ -79,4 +88,12 @@ export async function sendPartialMessageEvent(partialMessage: ClineMessage): Pro
}
await Promise.all(streamPromises)
const endedAt = performance.now()
observer.recordChunkToWebviewDelivery({
startedAt,
endedAt,
durationMs: Math.max(0, endedAt - startedAt),
label: "partial-message-broadcast",
payloadBytes,
})
}
+1
View File
@@ -59,6 +59,7 @@ export class TaskState {
// Focus Chain / Todo List Management
apiRequestCount = 0
currentLatencyObserverRequestId?: string
apiRequestsSinceLastTodoUpdate = 0
currentFocusChainChecklist: string | null = null
todoListWasUpdatedByUser = false
+15
View File
@@ -51,6 +51,7 @@ import { BrowserSession } from "@services/browser/BrowserSession"
import { UrlContentFetcher } from "@services/browser/UrlContentFetcher"
import { featureFlagsService } from "@services/feature-flags"
import { listFiles } from "@services/glob/list-files"
import { getLatencyObserverService } from "@services/latency/LatencyObserverService"
import { McpHub } from "@services/mcp/McpHub"
import { ApiConfiguration } from "@shared/api"
import { findLast, findLastIndex } from "@shared/array"
@@ -606,6 +607,7 @@ export class Task {
// await this.saveClineMessagesAndUpdateHistory()
// await this.postStateToWebview()
const protoMessage = convertClineMessageToProto(lastMessage)
getLatencyObserverService().recordFirstPartialMessageUpdate(this.taskId)
await sendPartialMessageEvent(protoMessage)
throw new Error("Current ask promise was ignored 1")
}
@@ -648,6 +650,7 @@ export class Task {
})
// await this.postStateToWebview()
const protoMessage = convertClineMessageToProto(lastMessage)
getLatencyObserverService().recordFirstPartialMessageUpdate(this.taskId)
await sendPartialMessageEvent(protoMessage)
} else {
// this is a new partial=false message, so add it like normal
@@ -787,6 +790,7 @@ export class Task {
})
const protoMessage = convertClineMessageToProto(lastMessage)
getLatencyObserverService().recordFirstPartialMessageUpdate(this.taskId)
await sendPartialMessageEvent(protoMessage)
return undefined
}
@@ -821,6 +825,7 @@ export class Task {
// await this.postStateToWebview()
const protoMessage = convertClineMessageToProto(lastMessage)
getLatencyObserverService().recordFirstPartialMessageUpdate(this.taskId)
await sendPartialMessageEvent(protoMessage) // more performant than an entire postStateToWebview
return undefined
}
@@ -840,6 +845,9 @@ export class Task {
return sayTs
}
// this is a new non-partial message, so add it like normal
if (type === "text" || type === "reasoning") {
getLatencyObserverService().recordFirstVisibleUpdate(this.taskId, type)
}
const sayTs = Date.now()
this.taskState.lastMessageTs = sayTs
await this.messageStateHandler.addToClineMessages({
@@ -2350,6 +2358,8 @@ export class Task {
// Save checkpoint if this is the first API request
const isFirstRequest = this.messageStateHandler.getClineMessages().filter((m) => m.say === "api_req_started").length === 0
this.taskState.currentLatencyObserverRequestId = `${this.taskId}:request-${this.taskState.apiRequestCount}`
getLatencyObserverService().markRequestStart(this.taskId, this.taskState.currentLatencyObserverRequestId)
// Initialize checkpointManager first if enabled and it's the first request
if (
@@ -2546,6 +2556,7 @@ export class Task {
durationMs,
this.stateManager.getGlobalSettingsKey("enableCheckpointsSetting"),
)
getLatencyObserverService().recordTaskInitializationEnd(this.taskId)
}
// since we sent off a placeholder api_req_started message to update the webview while waiting to actually start the API request (to load potential details for example), we need to update the text of that message
@@ -2730,6 +2741,7 @@ export class Task {
})
const completedReasoning = this.messageStateHandler.getClineMessages()[pendingReasoningIndex]
if (completedReasoning) {
getLatencyObserverService().recordFirstPartialMessageUpdate(this.taskId)
await sendPartialMessageEvent(convertClineMessageToProto(completedReasoning))
}
return true
@@ -3209,6 +3221,8 @@ export class Task {
} catch (_error) {
// this should never happen since the only thing that can throw an error is the attemptApiRequest, which is wrapped in a try catch that sends an ask where if noButtonClicked, will clear current task and destroy this instance. However to avoid unhandled promise rejection, we will end this loop which will end execution of this instance (see startTask)
return true // needs to be true so parent loop knows to end task
} finally {
getLatencyObserverService().completeRequest(this.taskId)
}
}
@@ -3369,6 +3383,7 @@ export class Task {
lastMessage.partial = false
await this.messageStateHandler.saveClineMessagesAndUpdateHistory()
const protoMessage = convertClineMessageToProto(lastMessage)
getLatencyObserverService().recordFirstPartialMessageUpdate(this.taskId)
await sendPartialMessageEvent(protoMessage)
}
+7
View File
@@ -2,6 +2,7 @@ import CheckpointTracker from "@integrations/checkpoints/CheckpointTracker"
import { EventEmitter } from "events"
import getFolderSize from "get-folder-size"
import Mutex from "p-mutex"
import { getLatencyObserverService } from "@/services/latency/LatencyObserverService"
import { findLastIndex } from "@/shared/array"
import { combineApiRequests } from "@/shared/combineApiRequests"
import { combineCommandSequences } from "@/shared/combineCommandSequences"
@@ -74,6 +75,9 @@ export class MessageStateHandler extends EventEmitter<MessageStateHandlerEvents>
* Emit a clineMessagesChanged event with the change details
*/
private emitClineMessagesChanged(change: ClineMessageChange): void {
const observer = getLatencyObserverService()
observer.incrementCounter("taskUiDeltaEvents")
observer.setCapability("taskUiDeltaMetrics", "supported")
this.emit("clineMessagesChanged", change)
}
@@ -119,6 +123,9 @@ export class MessageStateHandler extends EventEmitter<MessageStateHandlerEvents>
*/
private async saveClineMessagesAndUpdateHistoryInternal(): Promise<void> {
try {
const observer = getLatencyObserverService()
observer.incrementCounter("persistenceFlushes")
observer.setCapability("persistenceMetrics", "supported")
await saveClineMessages(this.taskId, this.clineMessages)
// combined as they are in ChatView
@@ -5,6 +5,7 @@ import { getHooksEnabledSafe } from "@core/hooks/hooks-utils"
import { formatResponse } from "@core/prompts/responses"
import { processFilesIntoText } from "@integrations/misc/extract-text"
import { showSystemNotification } from "@integrations/notifications"
import { getLatencyObserverService } from "@services/latency/LatencyObserverService"
import { telemetryService } from "@services/telemetry"
import { findLastIndex } from "@shared/array"
import { COMPLETION_RESULT_CHANGES_FLAG } from "@shared/ExtensionMessage"
@@ -216,6 +217,7 @@ export class AttemptCompletionHandler implements IToolHandler, IPartialBlockHand
// Run TaskComplete hook BEFORE presenting the "Start New Task" button
// At this point we know: task is complete, checkpoint saved, result shown to user
getLatencyObserverService().markTaskComplete(config.taskId)
await this.runTaskCompleteHook(config, block)
await this.runNotificationHook(config, {
event: "task_complete",
@@ -0,0 +1,326 @@
import {
createRollingLatencyStats,
DEFAULT_LATENCY_OBSERVER_CAPABILITIES,
type LatencyObserverCapabilities,
type LatencyObserverLogEntry,
type LatencyObserverRequestCounterSummary,
type LatencyObserverSessionMetadata,
type LatencyObserverStateSnapshot,
type LatencySample,
} from "@/shared/LatencyObserver"
type ActiveRequest = {
taskId: string
requestId: string
startedAt: number
firstVisibleRecorded: boolean
firstFullStateRecorded: boolean
firstPartialMessageRecorded: boolean
counterBaseline: Record<
| "fullStatePushes"
| "fullStateBytes"
| "partialMessageEvents"
| "partialMessageBytes"
| "taskUiDeltaEvents"
| "persistenceFlushes",
number
>
}
export class LatencyObserverService {
private sessionStartedAt = Date.now()
private currentObservedTaskId: string | undefined
private sessionMetadata: LatencyObserverSessionMetadata = {
startedAt: this.sessionStartedAt,
}
private capabilities: LatencyObserverCapabilities = {
...DEFAULT_LATENCY_OBSERVER_CAPABILITIES,
taskInitialization: "supported",
requestStart: "supported",
firstVisibleUpdate: "supported",
firstFullStateUpdate: "supported",
firstPartialMessageUpdate: "supported",
chunkToWebviewTiming: "supported",
}
private readonly taskInitializationStarts = new Map<string, number>()
private readonly activeRequests = new Map<string, ActiveRequest>()
private readonly transportSamples: LatencySample[] = []
private readonly taskInitializationSamples: LatencySample[] = []
private readonly requestStartSamples: LatencySample[] = []
private readonly firstVisibleUpdateSamples: LatencySample[] = []
private readonly firstFullStateUpdateSamples: LatencySample[] = []
private readonly firstPartialMessageUpdateSamples: LatencySample[] = []
private readonly chunkToWebviewSamples: LatencySample[] = []
private readonly requestCounterSummaries: LatencyObserverRequestCounterSummary[] = []
private readonly logs: LatencyObserverLogEntry[] = []
private optionalCounters: Record<
| "fullStatePushes"
| "fullStateBytes"
| "partialMessageEvents"
| "partialMessageBytes"
| "taskUiDeltaEvents"
| "persistenceFlushes",
number
> = {
fullStatePushes: 0,
fullStateBytes: 0,
partialMessageEvents: 0,
partialMessageBytes: 0,
taskUiDeltaEvents: 0,
persistenceFlushes: 0,
}
setSessionMetadata(metadata: Partial<Omit<LatencyObserverSessionMetadata, "startedAt">>): void {
this.sessionMetadata = {
...this.sessionMetadata,
...metadata,
startedAt: this.sessionStartedAt,
}
}
setCapability<K extends keyof LatencyObserverCapabilities>(key: K, value: LatencyObserverCapabilities[K]): void {
this.capabilities[key] = value
}
markTaskInitializationStart(taskId: string, startedAt = performance.now()): void {
if (this.currentObservedTaskId !== taskId) {
this.resetForTask(taskId)
}
this.pushLog(`task_start`, taskId)
this.taskInitializationStarts.set(taskId, startedAt)
this.pushLog(`task initialization started`, taskId)
}
markTaskComplete(taskId: string): void {
if (this.currentObservedTaskId !== taskId) {
return
}
this.pushLog(`task_complete`, taskId)
}
recordTaskInitializationEnd(taskId: string, endedAt = performance.now()): void {
const startedAt = this.taskInitializationStarts.get(taskId)
if (startedAt === undefined) {
return
}
this.taskInitializationSamples.push({
startedAt,
endedAt,
durationMs: Math.max(0, endedAt - startedAt),
requestId: taskId,
})
this.taskInitializationStarts.delete(taskId)
this.pushLog(`task initialization completed`, taskId)
}
markRequestStart(taskId: string, requestId: string, startedAt = performance.now()): void {
this.activeRequests.set(taskId, {
taskId,
requestId,
startedAt,
firstVisibleRecorded: false,
firstFullStateRecorded: false,
firstPartialMessageRecorded: false,
counterBaseline: { ...this.optionalCounters },
})
this.requestStartSamples.push({
startedAt,
endedAt: startedAt,
durationMs: 0,
requestId,
})
this.pushLog(`request started`, taskId, requestId)
}
recordFirstVisibleUpdate(taskId: string, source: string, endedAt = performance.now()): void {
const activeRequest = this.activeRequests.get(taskId)
if (!activeRequest || activeRequest.firstVisibleRecorded) {
return
}
activeRequest.firstVisibleRecorded = true
this.firstVisibleUpdateSamples.push({
startedAt: activeRequest.startedAt,
endedAt,
durationMs: Math.max(0, endedAt - activeRequest.startedAt),
label: source,
requestId: activeRequest.requestId,
})
this.pushLog(`first visible update (${source})`, taskId, activeRequest.requestId)
}
recordFirstFullStateUpdate(taskId: string, endedAt = performance.now()): void {
const activeRequest = this.activeRequests.get(taskId)
if (!activeRequest || activeRequest.firstFullStateRecorded) {
return
}
activeRequest.firstFullStateRecorded = true
this.firstFullStateUpdateSamples.push({
startedAt: activeRequest.startedAt,
endedAt,
durationMs: Math.max(0, endedAt - activeRequest.startedAt),
requestId: activeRequest.requestId,
label: "full-state",
})
this.pushLog(`first full-state update`, taskId, activeRequest.requestId)
}
recordFirstPartialMessageUpdate(taskId: string, endedAt = performance.now()): void {
const activeRequest = this.activeRequests.get(taskId)
if (!activeRequest || activeRequest.firstPartialMessageRecorded) {
return
}
activeRequest.firstPartialMessageRecorded = true
this.firstPartialMessageUpdateSamples.push({
startedAt: activeRequest.startedAt,
endedAt,
durationMs: Math.max(0, endedAt - activeRequest.startedAt),
requestId: activeRequest.requestId,
label: "partial-message",
})
this.pushLog(`first partial-message update`, taskId, activeRequest.requestId)
}
recordChunkToWebviewDelivery(sample: LatencySample): void {
this.chunkToWebviewSamples.push(sample)
if (this.chunkToWebviewSamples.length > 200) {
this.chunkToWebviewSamples.shift()
}
}
completeRequest(taskId: string): void {
const activeRequest = this.activeRequests.get(taskId)
if (!activeRequest) {
return
}
this.requestCounterSummaries.push({
requestId: activeRequest.requestId,
taskId: activeRequest.taskId,
startedAt: activeRequest.startedAt,
completedAt: performance.now(),
fullStatePushes: this.optionalCounters.fullStatePushes - activeRequest.counterBaseline.fullStatePushes,
fullStateBytes: this.optionalCounters.fullStateBytes - activeRequest.counterBaseline.fullStateBytes,
partialMessageEvents: this.optionalCounters.partialMessageEvents - activeRequest.counterBaseline.partialMessageEvents,
partialMessageBytes: this.optionalCounters.partialMessageBytes - activeRequest.counterBaseline.partialMessageBytes,
taskUiDeltaEvents: this.optionalCounters.taskUiDeltaEvents - activeRequest.counterBaseline.taskUiDeltaEvents,
persistenceFlushes: this.optionalCounters.persistenceFlushes - activeRequest.counterBaseline.persistenceFlushes,
})
if (this.requestCounterSummaries.length > 50) {
this.requestCounterSummaries.shift()
}
this.activeRequests.delete(taskId)
this.pushLog(`request completed`, taskId, activeRequest.requestId)
}
recordTransportSample(sample: LatencySample): void {
this.transportSamples.push(sample)
}
incrementCounter(counter: keyof typeof this.optionalCounters, amount = 1): void {
this.optionalCounters[counter] += amount
}
getSnapshot(): LatencyObserverStateSnapshot {
return {
session: { ...this.sessionMetadata },
capabilities: { ...this.capabilities },
transport: {
support: this.capabilities.transportProbe,
samples: [...this.transportSamples],
stats: createRollingLatencyStats(this.transportSamples),
},
taskInitialization: {
support: this.capabilities.taskInitialization,
samples: [...this.taskInitializationSamples],
stats: createRollingLatencyStats(this.taskInitializationSamples),
},
requestStart: {
support: this.capabilities.requestStart,
samples: [...this.requestStartSamples],
stats: createRollingLatencyStats(this.requestStartSamples),
},
firstVisibleUpdate: {
support: this.capabilities.firstVisibleUpdate,
samples: [...this.firstVisibleUpdateSamples],
stats: createRollingLatencyStats(this.firstVisibleUpdateSamples),
},
firstFullStateUpdate: {
support: this.capabilities.firstFullStateUpdate,
samples: [...this.firstFullStateUpdateSamples],
stats: createRollingLatencyStats(this.firstFullStateUpdateSamples),
},
firstPartialMessageUpdate: {
support: this.capabilities.firstPartialMessageUpdate,
samples: [...this.firstPartialMessageUpdateSamples],
stats: createRollingLatencyStats(this.firstPartialMessageUpdateSamples),
},
chunkToWebview: {
support: this.capabilities.chunkToWebviewTiming,
samples: [...this.chunkToWebviewSamples],
stats: createRollingLatencyStats(this.chunkToWebviewSamples),
},
requestCounterSummaries: [...this.requestCounterSummaries],
logs: [...this.logs],
optionalCounters: { ...this.optionalCounters },
}
}
reset(): void {
const { branch, commit, environment, platform, label } = this.sessionMetadata
this.sessionStartedAt = Date.now()
this.currentObservedTaskId = undefined
this.sessionMetadata = {
startedAt: this.sessionStartedAt,
branch,
commit,
environment,
platform,
label,
}
this.taskInitializationStarts.clear()
this.activeRequests.clear()
this.transportSamples.length = 0
this.taskInitializationSamples.length = 0
this.requestStartSamples.length = 0
this.firstVisibleUpdateSamples.length = 0
this.firstFullStateUpdateSamples.length = 0
this.firstPartialMessageUpdateSamples.length = 0
this.chunkToWebviewSamples.length = 0
this.requestCounterSummaries.length = 0
this.logs.length = 0
this.optionalCounters = {
fullStatePushes: 0,
fullStateBytes: 0,
partialMessageEvents: 0,
partialMessageBytes: 0,
taskUiDeltaEvents: 0,
persistenceFlushes: 0,
}
}
private resetForTask(taskId: string): void {
this.reset()
this.currentObservedTaskId = taskId
}
private pushLog(message: string, taskId?: string, requestId?: string): void {
this.logs.push({
ts: Date.now(),
message,
taskId,
requestId,
})
}
}
let latencyObserverService: LatencyObserverService | undefined
export function getLatencyObserverService(): LatencyObserverService {
latencyObserverService ??= new LatencyObserverService()
return latencyObserverService
}
@@ -0,0 +1,212 @@
import { strict as assert } from "assert"
import { describe, it } from "mocha"
import { LatencyObserverService } from "@/services/latency/LatencyObserverService"
describe("LatencyObserverService", () => {
it("records task initialization and first visible update lifecycle", () => {
const service = new LatencyObserverService()
service.markTaskInitializationStart("task-1", 10)
service.recordTaskInitializationEnd("task-1", 25)
service.markRequestStart("task-1", "task-1:req-1", 30)
service.recordFirstVisibleUpdate("task-1", "text", 42)
service.recordFirstFullStateUpdate("task-1", 45)
service.completeRequest("task-1")
const snapshot = service.getSnapshot()
assert.equal(snapshot.taskInitialization.stats.count, 1)
assert.equal(snapshot.taskInitialization.stats.lastMs, 15)
assert.equal(snapshot.firstVisibleUpdate.stats.count, 1)
assert.equal(snapshot.firstVisibleUpdate.stats.lastMs, 12)
assert.equal(snapshot.firstFullStateUpdate.stats.count, 1)
assert.equal(snapshot.firstFullStateUpdate.stats.lastMs, 15)
assert.equal(snapshot.logs.length >= 4, true)
})
it("records first full-state update only once per request", () => {
const service = new LatencyObserverService()
service.markRequestStart("task-fs", "task-fs:req-1", 100)
service.recordFirstFullStateUpdate("task-fs", 125)
service.recordFirstFullStateUpdate("task-fs", 140)
const snapshot = service.getSnapshot()
assert.equal(snapshot.firstFullStateUpdate.stats.count, 1)
assert.equal(snapshot.firstFullStateUpdate.samples[0].durationMs, 25)
})
it("records first partial-message update only once per request", () => {
const service = new LatencyObserverService()
service.markRequestStart("task-pm", "task-pm:req-1", 100)
service.recordFirstPartialMessageUpdate("task-pm", 118)
service.recordFirstPartialMessageUpdate("task-pm", 140)
const snapshot = service.getSnapshot()
assert.equal(snapshot.firstPartialMessageUpdate.stats.count, 1)
assert.equal(snapshot.firstPartialMessageUpdate.samples[0].durationMs, 18)
})
it("records chunk-to-webview delivery summaries", () => {
const service = new LatencyObserverService()
service.recordChunkToWebviewDelivery({
startedAt: 200,
endedAt: 212,
durationMs: 12,
label: "partial-message-broadcast",
payloadBytes: 64,
})
const snapshot = service.getSnapshot()
assert.equal(snapshot.chunkToWebview.stats.count, 1)
assert.equal(snapshot.chunkToWebview.stats.lastMs, 12)
assert.equal(snapshot.chunkToWebview.samples[0].payloadBytes, 64)
})
it("records first visible update only once per request", () => {
const service = new LatencyObserverService()
service.markRequestStart("task-2", "task-2:req-1", 100)
service.recordFirstVisibleUpdate("task-2", "text", 140)
service.recordFirstVisibleUpdate("task-2", "reasoning", 150)
const snapshot = service.getSnapshot()
assert.equal(snapshot.firstVisibleUpdate.stats.count, 1)
assert.equal(snapshot.firstVisibleUpdate.samples[0].label, "text")
})
it("tracks optional counters and session metadata", () => {
const service = new LatencyObserverService()
service.setSessionMetadata({ branch: "feature/latency", commit: "abc123", environment: "production", platform: "darwin" })
service.incrementCounter("fullStatePushes")
service.incrementCounter("fullStateBytes", 512)
service.incrementCounter("partialMessageEvents", 2)
service.incrementCounter("partialMessageBytes", 128)
const snapshot = service.getSnapshot()
assert.equal(snapshot.session.branch, "feature/latency")
assert.equal(snapshot.session.commit, "abc123")
assert.equal(snapshot.optionalCounters?.fullStatePushes, 1)
assert.equal(snapshot.optionalCounters?.fullStateBytes, 512)
assert.equal(snapshot.optionalCounters?.partialMessageEvents, 2)
assert.equal(snapshot.optionalCounters?.partialMessageBytes, 128)
})
it("captures per-request richer counter summaries on completion", () => {
const service = new LatencyObserverService()
service.markRequestStart("task-4", "task-4:req-1", 50)
service.incrementCounter("fullStatePushes", 2)
service.incrementCounter("fullStateBytes", 300)
service.incrementCounter("partialMessageEvents", 3)
service.incrementCounter("partialMessageBytes", 120)
service.incrementCounter("taskUiDeltaEvents", 4)
service.incrementCounter("persistenceFlushes", 1)
service.completeRequest("task-4")
const snapshot = service.getSnapshot()
assert.equal(snapshot.requestCounterSummaries.length, 1)
assert.deepEqual(snapshot.requestCounterSummaries[0], {
requestId: "task-4:req-1",
taskId: "task-4",
startedAt: 50,
completedAt: snapshot.requestCounterSummaries[0].completedAt,
fullStatePushes: 2,
fullStateBytes: 300,
partialMessageEvents: 3,
partialMessageBytes: 120,
taskUiDeltaEvents: 4,
persistenceFlushes: 1,
})
})
it("uses capability support values in metric snapshots", () => {
const service = new LatencyObserverService()
service.setCapability("taskInitialization", "hook-not-installed")
service.setCapability("requestStart", "unsupported")
service.setCapability("firstVisibleUpdate", "hook-not-installed")
service.setCapability("firstFullStateUpdate", "unsupported")
service.setCapability("firstPartialMessageUpdate", "hook-not-installed")
service.setCapability("chunkToWebviewTiming", "supported")
const snapshot = service.getSnapshot()
assert.equal(snapshot.taskInitialization.support, "hook-not-installed")
assert.equal(snapshot.requestStart.support, "unsupported")
assert.equal(snapshot.firstVisibleUpdate.support, "hook-not-installed")
assert.equal(snapshot.firstFullStateUpdate.support, "unsupported")
assert.equal(snapshot.firstPartialMessageUpdate.support, "hook-not-installed")
assert.equal(snapshot.chunkToWebview.support, "supported")
})
it("reset clears recorded samples, counters, and logs for a fresh session", () => {
const service = new LatencyObserverService()
service.setSessionMetadata({ branch: "feature/latency", commit: "abc123", environment: "production" })
const startedAtBeforeReset = service.getSnapshot().session.startedAt
service.markTaskInitializationStart("task-3", 1)
service.recordTaskInitializationEnd("task-3", 5)
service.markRequestStart("task-3", "task-3:req-1", 6)
service.recordFirstVisibleUpdate("task-3", "text", 10)
service.incrementCounter("fullStatePushes", 2)
service.completeRequest("task-3")
service.reset()
const snapshot = service.getSnapshot()
assert.equal(snapshot.session.branch, "feature/latency")
assert.equal(snapshot.session.commit, "abc123")
assert.equal(snapshot.session.environment, "production")
assert.equal(snapshot.session.startedAt >= startedAtBeforeReset, true)
assert.equal(snapshot.taskInitialization.stats.count, 0)
assert.equal(snapshot.requestStart.stats.count, 0)
assert.equal(snapshot.firstVisibleUpdate.stats.count, 0)
assert.equal(snapshot.firstFullStateUpdate.stats.count, 0)
assert.equal(snapshot.firstPartialMessageUpdate.stats.count, 0)
assert.equal(snapshot.chunkToWebview.stats.count, 0)
assert.equal(snapshot.requestCounterSummaries.length, 0)
assert.equal(snapshot.logs.length, 0)
assert.equal(snapshot.optionalCounters?.fullStatePushes, 0)
})
it("automatically starts a fresh observer session when a new task begins", () => {
const service = new LatencyObserverService()
service.markTaskInitializationStart("task-1", 10)
service.recordTaskInitializationEnd("task-1", 25)
service.markRequestStart("task-1", "task-1:req-1", 30)
service.recordFirstVisibleUpdate("task-1", "text", 40)
service.incrementCounter("fullStatePushes", 2)
const beforeNewTask = service.getSnapshot()
assert.equal(beforeNewTask.taskInitialization.stats.count, 1)
assert.equal(beforeNewTask.firstVisibleUpdate.stats.count, 1)
assert.equal(beforeNewTask.optionalCounters?.fullStatePushes, 2)
service.markTaskInitializationStart("task-2", 100)
const afterNewTask = service.getSnapshot()
assert.equal(afterNewTask.taskInitialization.stats.count, 0)
assert.equal(afterNewTask.firstVisibleUpdate.stats.count, 0)
assert.equal(afterNewTask.requestStart.stats.count, 0)
assert.equal(afterNewTask.optionalCounters?.fullStatePushes, 0)
assert.equal(afterNewTask.logs.length, 2)
assert.equal(afterNewTask.logs[0].message, "task_start")
assert.equal(afterNewTask.logs[1].message, "task initialization started")
})
it("records explicit task_start and task_complete markers", () => {
const service = new LatencyObserverService()
service.markTaskInitializationStart("task-1", 10)
service.markTaskComplete("task-1")
const snapshot = service.getSnapshot()
assert.equal(snapshot.logs[0].message, "task_start")
assert.equal(
snapshot.logs.some((entry) => entry.message === "task_complete"),
true,
)
})
})
+2
View File
@@ -11,6 +11,7 @@ import { BannerCardData } from "./cline/banner"
import { ClineRulesToggles } from "./cline-rules"
import { FocusChainSettings } from "./FocusChainSettings"
import { HistoryItem } from "./HistoryItem"
import type { LatencyObserverStateSnapshot } from "./LatencyObserver"
import { McpDisplayMode } from "./McpDisplayMode"
import { ClineMessageModelInfo } from "./messages"
import { OnboardingModelGroup } from "./proto/cline/state"
@@ -111,6 +112,7 @@ export interface ExtensionState {
banners?: BannerCardData[]
welcomeBanners?: BannerCardData[]
openAiCodexIsAuthenticated?: boolean
latencyObserver?: LatencyObserverStateSnapshot
}
export interface ClineMessage {
+146
View File
@@ -0,0 +1,146 @@
export type LatencyObserverMetricSupport = "supported" | "unsupported" | "hook-not-installed"
export interface LatencySample {
startedAt: number
endedAt: number
durationMs: number
label?: string
payloadBytes?: number
requestId?: string
metadata?: Record<string, string | number | boolean | null | undefined>
}
export interface RollingLatencyStats {
count: number
minMs: number | null
maxMs: number | null
avgMs: number | null
lastMs: number | null
totalMs: number
}
export interface LatencyObserverSessionMetadata {
branch?: string
commit?: string
environment?: string
platform?: string
startedAt: number
label?: string
}
export interface LatencyObserverCapabilities {
transportProbe: LatencyObserverMetricSupport
taskInitialization: LatencyObserverMetricSupport
requestStart: LatencyObserverMetricSupport
firstVisibleUpdate: LatencyObserverMetricSupport
firstFullStateUpdate: LatencyObserverMetricSupport
firstPartialMessageUpdate: LatencyObserverMetricSupport
chunkToWebviewTiming: LatencyObserverMetricSupport
fullStateMetrics: LatencyObserverMetricSupport
partialMessageMetrics: LatencyObserverMetricSupport
taskUiDeltaMetrics: LatencyObserverMetricSupport
persistenceMetrics: LatencyObserverMetricSupport
}
export interface LatencyObserverMetricSet {
transportSamples: LatencySample[]
requestStartSamples: LatencySample[]
taskInitializationSamples: LatencySample[]
firstVisibleUpdateSamples: LatencySample[]
firstFullStateUpdateSamples: LatencySample[]
firstPartialMessageUpdateSamples: LatencySample[]
chunkToWebviewSamples: LatencySample[]
capabilities: LatencyObserverCapabilities
session: LatencyObserverSessionMetadata
optionalCounters?: Partial<
Record<
| "fullStatePushes"
| "fullStateBytes"
| "partialMessageEvents"
| "partialMessageBytes"
| "taskUiDeltaEvents"
| "persistenceFlushes",
number
>
>
}
export interface LatencyObserverMetricSnapshot {
support: LatencyObserverMetricSupport
samples: LatencySample[]
stats: RollingLatencyStats
}
export interface LatencyObserverRequestCounterSummary {
requestId: string
taskId?: string
startedAt: number
completedAt: number
fullStatePushes: number
fullStateBytes: number
partialMessageEvents: number
partialMessageBytes: number
taskUiDeltaEvents: number
persistenceFlushes: number
}
export interface LatencyObserverLogEntry {
ts: number
message: string
taskId?: string
requestId?: string
}
export interface LatencyObserverStateSnapshot {
session: LatencyObserverSessionMetadata
capabilities: LatencyObserverCapabilities
transport: LatencyObserverMetricSnapshot
taskInitialization: LatencyObserverMetricSnapshot
requestStart: LatencyObserverMetricSnapshot
firstVisibleUpdate: LatencyObserverMetricSnapshot
firstFullStateUpdate: LatencyObserverMetricSnapshot
firstPartialMessageUpdate: LatencyObserverMetricSnapshot
chunkToWebview: LatencyObserverMetricSnapshot
requestCounterSummaries: LatencyObserverRequestCounterSummary[]
logs: LatencyObserverLogEntry[]
optionalCounters?: LatencyObserverMetricSet["optionalCounters"]
}
export const DEFAULT_LATENCY_OBSERVER_CAPABILITIES: LatencyObserverCapabilities = {
transportProbe: "supported",
taskInitialization: "unsupported",
requestStart: "unsupported",
firstVisibleUpdate: "unsupported",
firstFullStateUpdate: "unsupported",
firstPartialMessageUpdate: "unsupported",
chunkToWebviewTiming: "unsupported",
fullStateMetrics: "unsupported",
partialMessageMetrics: "unsupported",
taskUiDeltaMetrics: "unsupported",
persistenceMetrics: "unsupported",
}
export function createRollingLatencyStats(samples: readonly LatencySample[]): RollingLatencyStats {
if (samples.length === 0) {
return {
count: 0,
minMs: null,
maxMs: null,
avgMs: null,
lastMs: null,
totalMs: 0,
}
}
const durations = samples.map((sample) => sample.durationMs)
const totalMs = durations.reduce((sum, duration) => sum + duration, 0)
return {
count: samples.length,
minMs: Math.min(...durations),
maxMs: Math.max(...durations),
avgMs: totalMs / samples.length,
lastMs: samples.at(-1)?.durationMs ?? null,
totalMs,
}
}
@@ -0,0 +1,52 @@
import { strict as assert } from "assert"
import { describe, it } from "mocha"
import {
createRollingLatencyStats,
DEFAULT_LATENCY_OBSERVER_CAPABILITIES,
type LatencyObserverMetricSet,
} from "@/shared/LatencyObserver"
describe("LatencyObserver", () => {
it("aggregates rolling latency stats consistently", () => {
const stats = createRollingLatencyStats([
{ startedAt: 0, endedAt: 5, durationMs: 5 },
{ startedAt: 10, endedAt: 18, durationMs: 8 },
{ startedAt: 20, endedAt: 26, durationMs: 6 },
])
assert.deepStrictEqual(stats, {
count: 3,
minMs: 5,
maxMs: 8,
avgMs: 19 / 3,
lastMs: 6,
totalMs: 19,
})
})
it("supports missing optional metrics without breaking the shared model", () => {
const metricSet: LatencyObserverMetricSet = {
transportSamples: [],
requestStartSamples: [],
taskInitializationSamples: [],
firstVisibleUpdateSamples: [],
firstFullStateUpdateSamples: [],
firstPartialMessageUpdateSamples: [],
chunkToWebviewSamples: [],
capabilities: DEFAULT_LATENCY_OBSERVER_CAPABILITIES,
session: {
startedAt: 1,
},
}
assert.equal(metricSet.optionalCounters, undefined)
assert.deepStrictEqual(createRollingLatencyStats(metricSet.transportSamples), {
count: 0,
minMs: null,
maxMs: null,
avgMs: null,
lastMs: null,
totalMs: 0,
})
})
})
+109
View File
@@ -0,0 +1,109 @@
import type { State as ProtoState } from "@shared/proto/cline/state"
import type { ClineMessage as ProtoClineMessage } from "@shared/proto/cline/ui"
import { strict as assert } from "assert"
import { describe, it } from "mocha"
import { subscribeToState } from "@/core/controller/state/subscribeToState"
import { registerPartialMessageCallback, sendPartialMessageEvent } from "@/core/controller/ui/subscribeToPartialMessage"
import { getLatencyObserverService } from "@/services/latency/LatencyObserverService"
import type { ExtensionState } from "@/shared/ExtensionMessage"
describe("Latency observer state and partial-message adapters", () => {
it("records full-state observer metrics when state updates are emitted", async () => {
const observer = getLatencyObserverService()
observer.reset()
observer.markRequestStart("task-1", "task-1:req-1", 100)
const state = {
version: "test",
isNewUser: false,
welcomeViewCompleted: true,
onboardingModels: undefined,
autoApprovalSettings: { enabled: false, actions: {}, version: 1 },
browserSettings: {
viewport: undefined,
userAgent: undefined,
javaScriptEnabled: true,
},
mode: "act",
clineMessages: [],
currentTaskItem: {
id: "task-1",
ts: Date.now(),
task: "latency test",
tokensIn: 0,
tokensOut: 0,
totalCost: 0,
} as ExtensionState["currentTaskItem"],
mcpDisplayMode: "rich",
planActSeparateModelsSetting: false,
platform: "darwin",
shouldShowAnnouncement: false,
taskHistory: [],
telemetrySetting: "unset",
shellIntegrationTimeout: 4000,
terminalOutputLineLimit: 500,
maxConsecutiveMistakes: 3,
distinctId: "test",
globalClineRulesToggles: {},
localClineRulesToggles: {},
localWorkflowToggles: {},
globalWorkflowToggles: {},
localCursorRulesToggles: {},
localWindsurfRulesToggles: {},
localAgentsRulesToggles: {},
focusChainSettings: { enabled: false, remindClineInterval: 6 },
favoritedModelIds: [],
workspaceRoots: [],
primaryRootIndex: 0,
isMultiRootWorkspace: false,
multiRootSetting: { user: false, featureFlag: false },
lastDismissedInfoBannerVersion: 0,
lastDismissedModelBannerVersion: 0,
lastDismissedCliBannerVersion: 0,
} as unknown as ExtensionState
const sentStates: ProtoState[] = []
await subscribeToState({ getStateToPostToWebview: async () => state } as any, {} as any, async (message: ProtoState) => {
sentStates.push(message)
})
assert.equal(sentStates.length, 1)
const snapshot = observer.getSnapshot()
assert.equal(snapshot.capabilities.fullStateMetrics, "supported")
assert.equal(snapshot.capabilities.firstFullStateUpdate, "supported")
assert.equal(snapshot.capabilities.chunkToWebviewTiming, "supported")
assert.equal(snapshot.optionalCounters?.fullStatePushes, 1)
assert.equal(snapshot.firstFullStateUpdate.stats.count, 1)
assert.equal(snapshot.chunkToWebview.stats.count, 1)
})
it("records partial-message observer metrics when partial events are emitted", async () => {
const observer = getLatencyObserverService()
observer.reset()
observer.markRequestStart("task-2", "task-2:req-1", 100)
const received: ProtoClineMessage[] = []
const unregister = registerPartialMessageCallback((message) => {
received.push(message)
})
const partialMessage = {
ts: Date.now(),
type: "say",
say: "text",
text: "partial",
} as unknown as ProtoClineMessage
await sendPartialMessageEvent(partialMessage)
unregister()
assert.equal(received.length, 1)
const snapshot = observer.getSnapshot()
assert.equal(snapshot.capabilities.partialMessageMetrics, "supported")
assert.equal(snapshot.capabilities.chunkToWebviewTiming, "supported")
assert.equal(snapshot.optionalCounters?.partialMessageEvents, 1)
assert.equal(snapshot.firstPartialMessageUpdate.stats.count, 0)
assert.equal(snapshot.chunkToWebview.stats.count, 1)
assert.equal((snapshot.optionalCounters?.partialMessageBytes ?? 0) > 0, true)
})
})
+16
View File
@@ -3,6 +3,7 @@ import "should"
import should from "should"
import { MessageStateHandler } from "../core/task/message-state"
import { TaskState } from "../core/task/TaskState"
import { getLatencyObserverService } from "../services/latency/LatencyObserverService"
import { ClineMessage } from "../shared/ExtensionMessage"
/**
@@ -58,6 +59,21 @@ describe("MessageStateHandler Mutex Protection", () => {
handler.getClineMessages().should.deepEqual(testMessages)
})
it("tracks task UI delta events when message state changes", async () => {
const observer = getLatencyObserverService()
observer.reset()
const handler = createTestHandler()
handler.setClineMessages([createTestMessage("initial")])
await handler.addToClineMessages(createTestMessage("added"))
await handler.updateClineMessage(0, { text: "updated" })
await handler.deleteClineMessage(1)
const snapshot = observer.getSnapshot()
should.equal(snapshot.capabilities.taskUiDeltaMetrics, "supported")
should.equal(snapshot.optionalCounters?.taskUiDeltaEvents, 4)
})
/**
* CRITICAL TEST: Verify that addToClineMessages is atomic
* This test simulates the race condition that can occur when multiple
+21
View File
@@ -278,6 +278,27 @@ export async function getLatestGitCommitHash(cwd: string): Promise<string | null
}
}
export async function getCurrentGitBranch(cwd: string): Promise<string | null> {
try {
const isInstalled = await checkGitInstalled()
if (!isInstalled) {
return null
}
const isRepo = await checkGitRepo(cwd)
if (!isRepo) {
return null
}
const { stdout } = await execAsync("git rev-parse --abbrev-ref HEAD", { cwd })
const branch = stdout.trim()
return branch.length > 0 ? branch : null
} catch (error) {
Logger.error("Error getting current git branch:", error)
return null
}
}
function truncateOutput(content: string): string {
if (!GIT_OUTPUT_LINE_LIMIT) {
return content
@@ -0,0 +1,114 @@
import { afterEach, describe, expect, it, vi } from "vitest"
vi.mock("lucide-react", () => {
const Icon = () => null
return {
CheckCheck: Icon,
FlaskConical: Icon,
Gauge: Icon,
HardDriveDownload: Icon,
Info: Icon,
SlidersHorizontal: Icon,
SquareMousePointer: Icon,
SquareTerminal: Icon,
Wrench: Icon,
}
})
vi.mock("react-use", () => ({
useEvent: vi.fn(),
}))
vi.mock("@/components/ui/tooltip", () => ({
Tooltip: ({ children }: { children: React.ReactNode }) => children,
TooltipContent: ({ children }: { children: React.ReactNode }) => children,
TooltipTrigger: ({ children }: { children: React.ReactNode }) => children,
}))
vi.mock("@/context/ClineAuthContext", () => ({
useClineAuth: () => ({ activeOrganization: null }),
}))
vi.mock("@/context/ExtensionStateContext", () => ({
useExtensionState: () => ({
version: "test",
environment: "local",
settingsInitialModelTab: undefined,
}),
}))
vi.mock("@/lib/utils", () => ({
cn: (...args: unknown[]) => args.filter(Boolean).join(" "),
}))
vi.mock("@/services/grpc-client", () => ({
StateServiceClient: {
resetState: vi.fn(),
},
}))
vi.mock("../account/helpers", () => ({
isAdminOrOwner: () => false,
}))
vi.mock("../common/Tab", () => ({
Tab: ({ children }: { children: React.ReactNode }) => children,
TabContent: ({ children }: { children: React.ReactNode }) => children,
TabList: ({ children }: { children: React.ReactNode }) => children,
TabTrigger: ({ children }: { children: React.ReactNode }) => children,
}))
vi.mock("../common/ViewHeader", () => ({
default: () => null,
}))
vi.mock("./SectionHeader", () => ({
default: ({ children }: { children: React.ReactNode }) => children,
}))
vi.mock("./sections/AboutSection", () => ({ default: () => null }))
vi.mock("./sections/ApiConfigurationSection", () => ({ default: () => null }))
vi.mock("./sections/BrowserSettingsSection", () => ({ default: () => null }))
vi.mock("./sections/DebugSection", () => ({ default: () => null }))
vi.mock("./sections/FeatureSettingsSection", () => ({ default: () => null }))
vi.mock("./sections/GeneralSettingsSection", () => ({ default: () => null }))
vi.mock("./sections/LatencyObserverSection", () => ({ default: () => null }))
vi.mock("./sections/RemoteConfigSection", () => ({ RemoteConfigSection: () => null }))
vi.mock("./sections/TerminalSettingsSection", () => ({ default: () => null }))
describe("SETTINGS_TABS debug visibility", () => {
afterEach(() => {
vi.unstubAllEnvs()
vi.resetModules()
})
it("hides the debug tab outside dev mode", async () => {
vi.stubEnv("IS_DEV", "")
const { SETTINGS_TABS } = await import("./SettingsView")
const debugTab = SETTINGS_TABS.find((tab) => tab.id === "debug")
expect(debugTab).toBeDefined()
expect(debugTab?.hidden?.({ activeOrganization: null })).toBe(true)
})
it("shows the debug tab in dev mode", async () => {
vi.stubEnv("IS_DEV", '"true"')
const { SETTINGS_TABS } = await import("./SettingsView")
const debugTab = SETTINGS_TABS.find((tab) => tab.id === "debug")
expect(debugTab).toBeDefined()
expect(debugTab?.hidden?.({ activeOrganization: null })).toBe(false)
})
})
describe("SETTINGS_TABS latency visibility", () => {
it("always shows the latency tab", async () => {
const { SETTINGS_TABS } = await import("./SettingsView")
const latencyTab = SETTINGS_TABS.find((tab) => tab.id === "latency")
expect(latencyTab).toBeDefined()
expect(latencyTab?.hidden?.({ activeOrganization: null })).toBeUndefined()
})
})
@@ -4,6 +4,7 @@ import { UserOrganization } from "@shared/proto/index.cline"
import {
CheckCheck,
FlaskConical,
Gauge,
HardDriveDownload,
Info,
type LucideIcon,
@@ -29,13 +30,23 @@ import BrowserSettingsSection from "./sections/BrowserSettingsSection"
import DebugSection from "./sections/DebugSection"
import FeatureSettingsSection from "./sections/FeatureSettingsSection"
import GeneralSettingsSection from "./sections/GeneralSettingsSection"
import LatencyObserverSection from "./sections/LatencyObserverSection"
import { RemoteConfigSection } from "./sections/RemoteConfigSection"
import TerminalSettingsSection from "./sections/TerminalSettingsSection"
const IS_DEV = process.env.IS_DEV
// Tab definitions
type SettingsTabID = "api-config" | "features" | "browser" | "terminal" | "general" | "about" | "debug" | "remote-config"
type SettingsTabID =
| "api-config"
| "features"
| "browser"
| "terminal"
| "general"
| "latency"
| "about"
| "debug"
| "remote-config"
interface SettingsTab {
id: SettingsTabID
name: string
@@ -81,6 +92,13 @@ export const SETTINGS_TABS: SettingsTab[] = [
headerText: "General Settings",
icon: Wrench,
},
{
id: "latency",
name: "Latency",
tooltipText: "Latency Observer",
headerText: "Latency Observer",
icon: Gauge,
},
{
id: "remote-config",
name: "Remote Config",
@@ -139,6 +157,7 @@ const SettingsView = ({ onDone, targetSection }: SettingsViewProps) => {
features: FeatureSettingsSection,
browser: BrowserSettingsSection,
terminal: TerminalSettingsSection,
latency: LatencyObserverSection,
"remote-config": RemoteConfigSection,
about: AboutSection,
debug: DebugSection,
@@ -10,6 +10,7 @@ interface DebugSectionProps {
const DebugSection = ({ onResetState, renderSectionHeader }: DebugSectionProps) => {
const { setShowWelcome } = useExtensionState()
return (
<div>
{renderSectionHeader("debug")}
@@ -0,0 +1,346 @@
import { fireEvent, render, screen, waitFor } from "@testing-library/react"
import { beforeEach, describe, expect, it, vi } from "vitest"
import LatencyObserverSection from "./LatencyObserverSection"
const grpcClientMocks = vi.hoisted(() => ({
pingLatencyProbe: vi.fn(),
newTask: vi.fn(),
}))
type TransportMetric = {
support: string
samples: unknown[]
stats: {
count: number
minMs: number | null
maxMs: number | null
avgMs: number | null
lastMs: number | null
totalMs: number
}
}
type MockLatencyObserverState = {
session: { startedAt: number; branch: string; commit: string; environment: string }
capabilities: {
transportProbe: string
taskInitialization: string
requestStart: string
firstVisibleUpdate: string
firstFullStateUpdate: string
firstPartialMessageUpdate: string
chunkToWebviewTiming: string
fullStateMetrics: string
partialMessageMetrics: string
taskUiDeltaMetrics: string
persistenceMetrics: string
}
transport: TransportMetric
taskInitialization: TransportMetric
requestStart: TransportMetric
firstVisibleUpdate: TransportMetric
firstFullStateUpdate: TransportMetric
firstPartialMessageUpdate: TransportMetric
chunkToWebview: TransportMetric
logs: unknown[]
requestCounterSummaries: Array<{
requestId: string
taskId?: string
startedAt: number
completedAt: number
fullStatePushes: number
fullStateBytes: number
partialMessageEvents: number
partialMessageBytes: number
taskUiDeltaEvents: number
persistenceFlushes: number
}>
optionalCounters?: {
fullStatePushes: number
fullStateBytes: number
partialMessageEvents: number
partialMessageBytes: number
taskUiDeltaEvents: number
persistenceFlushes: number
}
}
const makeLatencyObserverState = (): MockLatencyObserverState => ({
session: { startedAt: 1, branch: "main", commit: "abcdef123456", environment: "production" },
capabilities: {
transportProbe: "supported",
taskInitialization: "supported",
requestStart: "supported",
firstVisibleUpdate: "supported",
firstFullStateUpdate: "supported",
firstPartialMessageUpdate: "supported",
chunkToWebviewTiming: "supported",
fullStateMetrics: "supported",
partialMessageMetrics: "supported",
taskUiDeltaMetrics: "unsupported",
persistenceMetrics: "supported",
},
transport: {
support: "supported",
samples: [],
stats: { count: 0, minMs: null, maxMs: null, avgMs: null, lastMs: null, totalMs: 0 },
},
taskInitialization: {
support: "supported",
samples: [],
stats: { count: 1, minMs: 10, maxMs: 10, avgMs: 10, lastMs: 10, totalMs: 10 },
},
requestStart: {
support: "supported",
samples: [],
stats: { count: 1, minMs: 0, maxMs: 0, avgMs: 0, lastMs: 0, totalMs: 0 },
},
firstVisibleUpdate: {
support: "supported",
samples: [],
stats: { count: 1, minMs: 12, maxMs: 12, avgMs: 12, lastMs: 12, totalMs: 12 },
},
firstFullStateUpdate: {
support: "supported",
samples: [],
stats: { count: 1, minMs: 14, maxMs: 14, avgMs: 14, lastMs: 14, totalMs: 14 },
},
firstPartialMessageUpdate: {
support: "supported",
samples: [],
stats: { count: 1, minMs: 16, maxMs: 16, avgMs: 16, lastMs: 16, totalMs: 16 },
},
chunkToWebview: {
support: "supported",
samples: [],
stats: { count: 1, minMs: 9, maxMs: 9, avgMs: 9, lastMs: 9, totalMs: 9 },
},
logs: [],
requestCounterSummaries: [
{
requestId: "task-1:req-1",
taskId: "task-1",
startedAt: 10,
completedAt: 24,
fullStatePushes: 2,
fullStateBytes: 512,
partialMessageEvents: 3,
partialMessageBytes: 256,
taskUiDeltaEvents: 4,
persistenceFlushes: 1,
},
],
optionalCounters: {
fullStatePushes: 3,
fullStateBytes: 1024,
partialMessageEvents: 4,
partialMessageBytes: 256,
taskUiDeltaEvents: 7,
persistenceFlushes: 2,
},
})
const extensionStateMock = vi.hoisted(() => ({
latencyObserver: null as unknown as MockLatencyObserverState,
}))
vi.mock("@/services/grpc-client", () => ({
UiServiceClient: {
pingLatencyProbe: grpcClientMocks.pingLatencyProbe,
},
TaskServiceClient: {
newTask: grpcClientMocks.newTask,
},
}))
vi.mock("@/components/ui/select", () => {
const React = require("react") as typeof import("react")
const SelectContext = React.createContext<{ onValueChange?: (value: string) => void } | null>(null)
return {
Select: ({ children, onValueChange }: { children: React.ReactNode; onValueChange?: (value: string) => void }) => (
<SelectContext.Provider value={{ onValueChange }}>{children}</SelectContext.Provider>
),
SelectTrigger: ({ children, className }: { children: React.ReactNode; className?: string }) => (
<div className={className} role="combobox">
{children}
</div>
),
SelectValue: () => null,
SelectContent: ({ children }: { children: React.ReactNode }) => <div>{children}</div>,
SelectItem: ({ children, value }: { children: React.ReactNode; value: string }) => {
const context = React.useContext(SelectContext)
return (
<button onClick={() => context?.onValueChange?.(value)} role="option" type="button">
{children}
</button>
)
},
}
})
vi.mock("@/context/ExtensionStateContext", () => ({
useExtensionState: () => extensionStateMock,
}))
describe("LatencyObserverSection", () => {
beforeEach(() => {
vi.clearAllMocks()
extensionStateMock.latencyObserver = makeLatencyObserverState()
})
it("runs the latency probe and renders rolling stats", async () => {
grpcClientMocks.pingLatencyProbe.mockResolvedValue({ value: 64 })
const createObjectURL = vi.fn((_: Blob) => "blob:test")
const revokeObjectURL = vi.fn()
const click = vi.fn()
const originalCreateElement = document.createElement.bind(document)
const createElementSpy = vi.spyOn(document, "createElement").mockImplementation(((tagName: string) => {
if (tagName === "a") {
return { click, href: "", download: "" } as unknown as HTMLAnchorElement
}
return originalCreateElement(tagName)
}) as typeof document.createElement)
vi.stubGlobal("URL", { createObjectURL, revokeObjectURL })
render(<LatencyObserverSection renderSectionHeader={() => null} />)
fireEvent.change(screen.getByLabelText("Ping payload bytes"), { target: { value: "64" } })
fireEvent.click(screen.getByText("Run Ping Probe"))
await waitFor(() => expect(grpcClientMocks.pingLatencyProbe).toHaveBeenCalledTimes(1))
expect(screen.getByText(/Samples: 1/)).toBeTruthy()
expect(screen.getByText(/Payload: 64 bytes/)).toBeTruthy()
expect(screen.getByText(/First full-state avg: 14.00 ms/)).toBeTruthy()
expect(screen.getByText(/First partial avg: 16.00 ms/)).toBeTruthy()
expect(screen.getByText(/Chunk→webview avg: 9.00 ms/)).toBeTruthy()
expect(screen.getByText(/Chunk→webview timing: Supported/)).toBeTruthy()
expect(screen.getByText(/Req partial bytes: 256/)).toBeTruthy()
fireEvent.click(screen.getByText("Export Session JSON"))
expect(createObjectURL).toHaveBeenCalledTimes(1)
expect(click).toHaveBeenCalledTimes(1)
expect(revokeObjectURL).toHaveBeenCalledTimes(1)
createElementSpy.mockRestore()
})
it("exports a stable session even when optional metrics are unavailable", () => {
extensionStateMock.latencyObserver = {
...makeLatencyObserverState(),
capabilities: {
...makeLatencyObserverState().capabilities,
fullStateMetrics: "unsupported",
partialMessageMetrics: "unsupported",
taskUiDeltaMetrics: "hook-not-installed",
},
optionalCounters: undefined,
}
const createObjectURL = vi.fn((_: Blob) => "blob:test")
const revokeObjectURL = vi.fn()
const click = vi.fn()
const originalCreateElement = document.createElement.bind(document)
const createElementSpy = vi.spyOn(document, "createElement").mockImplementation(((tagName: string) => {
if (tagName === "a") {
return { click, href: "", download: "" } as unknown as HTMLAnchorElement
}
return originalCreateElement(tagName)
}) as typeof document.createElement)
vi.stubGlobal("URL", { createObjectURL, revokeObjectURL })
render(<LatencyObserverSection renderSectionHeader={() => null} />)
fireEvent.click(screen.getByText("Export Session JSON"))
expect(screen.getByText(/Full-state metrics: Unsupported on this branch/)).toBeTruthy()
expect(screen.getByText(/Task UI delta metrics: Observer hook not installed/)).toBeTruthy()
expect(createObjectURL).toHaveBeenCalledTimes(1)
expect(click).toHaveBeenCalledTimes(1)
expect(revokeObjectURL).toHaveBeenCalledTimes(1)
createElementSpy.mockRestore()
})
it("exports a stable latency observer schema with scenario and transport stats", () => {
const OriginalBlob = globalThis.Blob
class FakeBlob {
public readonly parts: unknown[]
public readonly type: string
constructor(parts: unknown[], options?: { type?: string }) {
this.parts = parts
this.type = options?.type ?? ""
}
}
vi.stubGlobal("Blob", FakeBlob as unknown as typeof Blob)
const createObjectURL = vi.fn((_: Blob) => "blob:test")
const revokeObjectURL = vi.fn()
const click = vi.fn()
const originalCreateElement = document.createElement.bind(document)
const createElementSpy = vi.spyOn(document, "createElement").mockImplementation(((tagName: string) => {
if (tagName === "a") {
return { click, href: "", download: "" } as unknown as HTMLAnchorElement
}
return originalCreateElement(tagName)
}) as typeof document.createElement)
vi.stubGlobal("URL", { createObjectURL, revokeObjectURL })
render(<LatencyObserverSection renderSectionHeader={() => null} />)
fireEvent.click(screen.getByText("Export Session JSON"))
const exportedBlob = createObjectURL.mock.calls[0][0] as unknown as FakeBlob
const exportedJson = JSON.parse(String(exportedBlob.parts[0]))
expect(exportedJson).toMatchObject({
session: { branch: "main", commit: "abcdef123456", environment: "production" },
observationScenario: { id: "ping-only", label: "Pure ping test" },
transport: { support: "supported", stats: { count: 0, totalMs: 0 } },
})
vi.stubGlobal("Blob", OriginalBlob)
createElementSpy.mockRestore()
})
it("runs payload presets and prevents overlapping manual pings while active", async () => {
let resolvePing: (() => void) | undefined
grpcClientMocks.pingLatencyProbe.mockImplementation(
() =>
new Promise((resolve) => {
resolvePing = () => resolve({ value: 0 })
}),
)
render(<LatencyObserverSection renderSectionHeader={() => null} />)
fireEvent.click(screen.getByText("Run Ping Probe"))
await waitFor(() => expect(grpcClientMocks.pingLatencyProbe).toHaveBeenCalledTimes(1))
resolvePing?.()
await waitFor(() => expect(screen.getByText("Run Ping Probe")).toBeTruthy())
grpcClientMocks.pingLatencyProbe.mockResolvedValue({ value: 0 })
fireEvent.click(screen.getByText("Test Payload Presets"))
await waitFor(() => expect(grpcClientMocks.pingLatencyProbe).toHaveBeenCalledTimes(5))
})
it("starts an observed task scenario using the selected scenario template", async () => {
grpcClientMocks.newTask.mockResolvedValue({ value: "task-1" })
render(<LatencyObserverSection renderSectionHeader={() => null} />)
fireEvent.click(screen.getByRole("button", { name: "Start Observed Task Scenario" }))
await waitFor(() => expect(grpcClientMocks.newTask).toHaveBeenCalledTimes(1))
})
it("starts the selected large-file scenario with the plan document attached", async () => {
grpcClientMocks.newTask.mockResolvedValue({ value: "task-2" })
render(<LatencyObserverSection renderSectionHeader={() => null} />)
fireEvent.mouseDown(screen.getByRole("combobox"))
fireEvent.click(screen.getByRole("option", { name: "Large-file-write adjacent" }))
fireEvent.click(screen.getByRole("button", { name: "Start Observed Task Scenario" }))
await waitFor(() => expect(grpcClientMocks.newTask).toHaveBeenCalledTimes(1))
expect(grpcClientMocks.newTask).toHaveBeenCalledWith({
text: "Latency observer large-file scenario. Read the latency observer plan document and summarize the sections most relevant to payload size and export behavior.",
images: [],
files: ["docs/remote-workspace-local-latency-observer-plan.md"],
})
})
it("explains that the latest task becomes the observer session", () => {
render(<LatencyObserverSection renderSectionHeader={() => null} />)
expect(screen.getByText(/most recently started task/i)).toBeTruthy()
expect(screen.getByText(/You no longer need to manually reset the observer/i)).toBeTruthy()
})
})
@@ -0,0 +1,608 @@
import { createRollingLatencyStats, type LatencySample } from "@shared/LatencyObserver"
import { NewTaskRequest } from "@shared/proto/cline/task"
import { useEffect, useMemo, useRef, useState } from "react"
import { Button } from "@/components/ui/button"
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select"
import { useExtensionState } from "@/context/ExtensionStateContext"
import { TaskServiceClient, UiServiceClient } from "@/services/grpc-client"
import Section from "../Section"
const PAYLOAD_PRESETS = [0, 64, 1024, 16_384] as const
const OBSERVATION_SCENARIOS = [
{
id: "ping-only",
label: "Pure ping test",
description: "Use ping presets and continuous ping to compare transport RTT and jitter only.",
},
{
id: "short-response",
label: "Short assistant response",
description: "Run a brief question and compare request start, first visible update, and completion smoothness.",
},
{
id: "long-streaming",
label: "Long streaming response",
description: "Use a longer prompt to compare first visible update timing and sustained streaming behavior.",
},
{
id: "tool-heavy",
label: "Tool-heavy / high churn",
description: "Run a task that triggers multiple tool calls to compare state pushes, partials, and persistence churn.",
},
{
id: "large-file",
label: "Large-file-write adjacent",
description: "Exercise file-heavy workflows to compare payload size effects and observer counters under load.",
},
] as const
const OBSERVATION_SCENARIO_PROMPTS: Record<(typeof OBSERVATION_SCENARIOS)[number]["id"], { prompt: string; files?: string[] }> = {
"ping-only": {
prompt: "Latency observer ping-only scenario. Do not perform any tools. Respond with a short acknowledgement so transport RTT can be compared without task churn.",
},
"short-response": {
prompt: "Latency observer short-response scenario. Answer in 2 concise sentences describing what you are measuring for responsiveness.",
},
"long-streaming": {
prompt: "Latency observer long-streaming scenario. Produce a longer multi-paragraph explanation of how to compare perceived latency across branches, with enough content to stream for several seconds.",
},
"tool-heavy": {
prompt: "Latency observer tool-heavy scenario. Inspect the repository by listing relevant files, then summarize likely hot paths for state churn and partial updates.",
},
"large-file": {
prompt: "Latency observer large-file scenario. Read the latency observer plan document and summarize the sections most relevant to payload size and export behavior.",
files: ["docs/remote-workspace-local-latency-observer-plan.md"],
},
}
const capabilityLabel: Record<string, string> = {
supported: "Supported",
unsupported: "Unsupported on this branch",
"hook-not-installed": "Observer hook not installed",
}
function formatMs(value: number | null | undefined): string {
return value == null ? "-" : `${value.toFixed(2)} ms`
}
function formatDurationFromSample(sample?: LatencySample): string {
return formatMs(sample?.durationMs)
}
function formatSampleCount(samples?: readonly unknown[]): string {
return `${samples?.length ?? 0} sample${(samples?.length ?? 0) === 1 ? "" : "s"}`
}
function formatWallClockTime(timestamp: number | null | undefined): string {
if (timestamp == null) {
return "--:--:--"
}
return new Date(timestamp).toLocaleTimeString([], {
hour: "2-digit",
minute: "2-digit",
second: "2-digit",
})
}
function formatSinceTaskStart(timestamp: number | null | undefined, taskStartedAt: number | null | undefined): string {
if (timestamp == null || taskStartedAt == null) {
return "t+ -"
}
return `t+${(Math.max(0, timestamp - taskStartedAt) / 1000).toFixed(3)} s`
}
interface LatencyObserverSectionProps {
renderSectionHeader: (tabId: string) => JSX.Element | null
}
function containScrollWithinElement(event: React.WheelEvent<HTMLDivElement>) {
const container = event.currentTarget
const { scrollTop, scrollHeight, clientHeight } = container
const canScroll = scrollHeight > clientHeight
if (!canScroll) {
return
}
const deltaY = event.deltaY
const scrollingDown = deltaY > 0
const scrollingUp = deltaY < 0
const atTop = scrollTop <= 0
const atBottom = Math.ceil(scrollTop + clientHeight) >= scrollHeight
if ((scrollingUp && atTop) || (scrollingDown && atBottom)) {
event.preventDefault()
}
event.stopPropagation()
}
const LatencyObserverSection = ({ renderSectionHeader }: LatencyObserverSectionProps) => {
const { latencyObserver } = useExtensionState()
const [payloadBytes, setPayloadBytes] = useState(0)
const [samples, setSamples] = useState<LatencySample[]>([])
const [isPinging, setIsPinging] = useState(false)
const [isContinuousPinging, setIsContinuousPinging] = useState(false)
const [isRunningPayloadSweep, setIsRunningPayloadSweep] = useState(false)
const [isStartingScenario, setIsStartingScenario] = useState(false)
const [pingError, setPingError] = useState<string | null>(null)
const [selectedScenarioId, setSelectedScenarioId] = useState<(typeof OBSERVATION_SCENARIOS)[number]["id"]>("ping-only")
const continuousPingEnabledRef = useRef(false)
const selectedScenario =
OBSERVATION_SCENARIOS.find((scenario) => scenario.id === selectedScenarioId) ?? OBSERVATION_SCENARIOS[0]
const transportSamples = latencyObserver?.transport.samples ?? []
const latestRequestSummary = latencyObserver?.requestCounterSummaries.at(-1)
const latestTaskInitializationSample = latencyObserver?.taskInitialization.samples.at(-1)
const latestFirstVisibleSample = latencyObserver?.firstVisibleUpdate.samples.at(-1)
const latestFirstFullStateSample = latencyObserver?.firstFullStateUpdate.samples.at(-1)
const latestFirstPartialSample = latencyObserver?.firstPartialMessageUpdate.samples.at(-1)
const latestChunkToWebviewSample = latencyObserver?.chunkToWebview.samples.at(-1)
const effectiveTransportSamples = useMemo(() => {
if (transportSamples.length > 0) {
return transportSamples
}
return samples
}, [samples, transportSamples])
const stats = useMemo(() => createRollingLatencyStats(effectiveTransportSamples), [effectiveTransportSamples])
const latestRequestElapsedMs =
latestRequestSummary != null ? Math.max(0, latestRequestSummary.completedAt - latestRequestSummary.startedAt) : null
const timelineEntries = latencyObserver?.logs ?? []
const perceivedTimelineCards = [
{
title: "Task initialized",
value: formatDurationFromSample(latestTaskInitializationSample),
subtitle: "Measured from task start",
detail: formatSampleCount(latencyObserver?.taskInitialization.samples),
},
{
title: "First visible assistant update",
value: formatDurationFromSample(latestFirstVisibleSample),
subtitle: "Measured from request start",
detail: formatSampleCount(latencyObserver?.firstVisibleUpdate.samples),
},
{
title: "First full-state update",
value: formatDurationFromSample(latestFirstFullStateSample),
subtitle: "Measured from request start",
detail: formatSampleCount(latencyObserver?.firstFullStateUpdate.samples),
},
{
title: "First partial-message update",
value: formatDurationFromSample(latestFirstPartialSample),
subtitle: "Measured from request start",
detail: formatSampleCount(latencyObserver?.firstPartialMessageUpdate.samples),
},
{
title: "Latest request end-to-end",
value: formatMs(latestRequestElapsedMs),
subtitle: "Measured from request start to completion",
detail: latestRequestSummary?.requestId ?? "No completed request yet",
},
{
title: "Chunk → webview hop",
value: formatDurationFromSample(latestChunkToWebviewSample),
subtitle: "Per-event delivery duration",
detail: formatSampleCount(latencyObserver?.chunkToWebview.samples),
},
]
const bottleneckHints = [
latestFirstVisibleSample &&
latestFirstFullStateSample &&
latestFirstVisibleSample.durationMs - latestFirstFullStateSample.durationMs > 150
? "First full-state arrives noticeably before the first visible assistant update, suggesting rendering / content-assembly delay after transport delivery."
: null,
latestFirstPartialSample &&
latestFirstVisibleSample &&
latestFirstVisibleSample.durationMs - latestFirstPartialSample.durationMs > 150
? "Partial-message events are arriving before users see content, which points to UI-visible processing delay rather than raw backend arrival time."
: null,
latestChunkToWebviewSample && latestChunkToWebviewSample.durationMs > 75
? "Chunk→webview delivery is elevated, which may indicate remote transport / extension-host delivery overhead becoming user-visible."
: null,
latestRequestSummary && latestRequestSummary.fullStatePushes + latestRequestSummary.partialMessageEvents > 25
? "The latest request generated a high number of UI updates; compare this against local runs to see whether remote workspaces amplify visible churn."
: null,
].filter(Boolean) as string[]
const runPing = async (nextPayloadBytes = payloadBytes) => {
setIsPinging(true)
setPingError(null)
const startedAt = performance.now()
try {
await UiServiceClient.pingLatencyProbe({ value: new Uint8Array(nextPayloadBytes) })
const endedAt = performance.now()
setSamples((current) => [
...current,
{
startedAt,
endedAt,
durationMs: endedAt - startedAt,
payloadBytes: nextPayloadBytes,
},
])
} catch (error) {
setPingError(error instanceof Error ? error.message : String(error))
} finally {
setIsPinging(false)
}
}
const runPayloadSweep = async () => {
setIsRunningPayloadSweep(true)
setPingError(null)
try {
for (const preset of PAYLOAD_PRESETS) {
await runPing(preset)
}
} finally {
setIsRunningPayloadSweep(false)
}
}
const toggleContinuousPing = async () => {
if (isContinuousPinging) {
continuousPingEnabledRef.current = false
setIsContinuousPinging(false)
return
}
continuousPingEnabledRef.current = true
setIsContinuousPinging(true)
setPingError(null)
try {
while (continuousPingEnabledRef.current) {
await runPing(payloadBytes)
if (!continuousPingEnabledRef.current) {
break
}
await new Promise((resolve) => setTimeout(resolve, 500))
}
} finally {
continuousPingEnabledRef.current = false
setIsContinuousPinging(false)
}
}
useEffect(() => {
return () => {
continuousPingEnabledRef.current = false
}
}, [])
const exportLatencyObserverSession = () => {
if (!latencyObserver) {
return
}
const exportSnapshot = {
...latencyObserver,
observationScenario: {
id: selectedScenario.id,
label: selectedScenario.label,
description: selectedScenario.description,
},
transport: {
...latencyObserver.transport,
samples: effectiveTransportSamples,
stats,
},
}
const blob = new Blob([JSON.stringify(exportSnapshot, null, 2)], { type: "application/json" })
const url = URL.createObjectURL(blob)
const anchor = document.createElement("a")
const branchLabel = latencyObserver.session.branch ?? "unknown-branch"
const commitLabel = latencyObserver.session.commit?.slice(0, 8) ?? "unknown-commit"
anchor.href = url
anchor.download = `latency-observer-${branchLabel}-${commitLabel}.json`
anchor.click()
URL.revokeObjectURL(url)
}
const startObservedScenario = async () => {
const scenarioConfig = OBSERVATION_SCENARIO_PROMPTS[selectedScenario.id]
setIsStartingScenario(true)
setPingError(null)
try {
await TaskServiceClient.newTask(
NewTaskRequest.create({
text: scenarioConfig.prompt,
images: [],
files: scenarioConfig.files ?? [],
}),
)
} catch (error) {
setPingError(error instanceof Error ? error.message : String(error))
} finally {
setIsStartingScenario(false)
}
}
return (
<div>
{renderSectionHeader("latency")}
<Section>
<div className="flex flex-col gap-3">
<h4 className="m-0 text-sm font-medium">Latency Observer</h4>
<p className="m-0 text-xs text-(--vscode-descriptionForeground)">
Use this panel to compare transport RTT and task-lifecycle responsiveness across branches.
</p>
<div className="flex flex-col gap-1 text-xs text-(--vscode-descriptionForeground)">
<span>Observation scenario</span>
<Select
onValueChange={(value) =>
setSelectedScenarioId(value as (typeof OBSERVATION_SCENARIOS)[number]["id"])
}
value={selectedScenarioId}>
<SelectTrigger className="w-full">
<SelectValue />
</SelectTrigger>
<SelectContent>
{OBSERVATION_SCENARIOS.map((scenario) => (
<SelectItem key={scenario.id} value={scenario.id}>
{scenario.label}
</SelectItem>
))}
</SelectContent>
</Select>
<p className="m-0">{selectedScenario.description}</p>
</div>
<div className="rounded border border-[var(--vscode-panel-border)] p-2 text-xs text-(--vscode-descriptionForeground)">
<div className="font-medium text-foreground">How to interpret these numbers</div>
<ul className="mb-0 mt-2 list-disc pl-4">
<li>Ping RTT is a lower-level transport signal, not a full UX measurement.</li>
<li>
Task initialization and first visible update are better proxies for user-perceived responsiveness.
</li>
<li>
State push, partial-message, and persistence counters help explain why one branch feels faster.
</li>
<li>Compare the same scenario, payload, and environment across branches for meaningful results.</li>
</ul>
</div>
<div className="rounded border border-[var(--vscode-panel-border)] p-2 text-xs text-(--vscode-descriptionForeground)">
<div className="font-medium text-foreground">Branch-portable vs richer metrics</div>
<p className="mb-0 mt-2">
Transport RTT, task initialization, request count, first visible update, logs, and export data are
intended to stay branch-portable. Full-state bytes, partial-message bytes, task UI deltas, and future
chunk timing are richer metrics that may vary by branch capability.
</p>
</div>
<div className="rounded border border-[var(--vscode-panel-border)] p-3 text-xs">
<div className="font-medium text-foreground">What the user perceives as the task progresses</div>
<p className="mb-0 mt-2 text-(--vscode-descriptionForeground)">
Use these cards to distinguish between cumulative, from-start timings and per-event timings. The first
group helps answer how long did the human wait before seeing something?, while the latter helps
answer how long does this step itself take once it begins?.
</p>
<div className="mt-3 grid grid-cols-1 gap-2 md:grid-cols-2 xl:grid-cols-3">
{perceivedTimelineCards.map((card) => (
<div
className="rounded border border-[var(--vscode-panel-border)] bg-[var(--vscode-editor-background)] p-3"
key={card.title}>
<div className="text-[11px] uppercase tracking-wide text-(--vscode-descriptionForeground)">
{card.title}
</div>
<div className="mt-1 text-base font-medium text-foreground">{card.value}</div>
<div className="mt-1 text-[11px] text-(--vscode-descriptionForeground)">{card.subtitle}</div>
<div className="mt-2 text-[11px] text-(--vscode-descriptionForeground)">{card.detail}</div>
</div>
))}
</div>
</div>
<div className="rounded border border-[var(--vscode-panel-border)] p-3 text-xs">
<div className="font-medium text-foreground">Bottleneck hints to compare local vs remote workspaces</div>
<ul className="mb-0 mt-2 list-disc pl-4 text-(--vscode-descriptionForeground)">
<li>
If <span className="text-foreground">first visible assistant update</span> grows much more than
transport RTT, the human is feeling bottlenecks beyond the pipe.
</li>
<li>
If <span className="text-foreground">first full-state / first partial-message</span> are early but
visible content is late, the latency is likely in presentation or state handling rather than
request dispatch.
</li>
<li>
If <span className="text-foreground">chunk webview</span> rises in remote workspaces, the
delivery hop itself is becoming user-visible.
</li>
<li>
If <span className="text-foreground">state bytes / partial bytes / UI deltas</span> spike, remote
workspaces may feel slower because more UI work is happening while the user waits.
</li>
{bottleneckHints.length === 0 ? (
<li>
No strong bottleneck hint yet run a scenario and compare the first-visible timeline against
transport RTT.
</li>
) : (
bottleneckHints.map((hint) => <li key={hint}>{hint}</li>)
)}
</ul>
</div>
<div className="rounded border border-[var(--vscode-panel-border)] p-3 text-xs text-(--vscode-descriptionForeground)">
<div className="font-medium text-foreground">Observer session semantics</div>
<p className="mb-0 mt-2">
The latency observer now treats the{" "}
<span className="text-foreground">most recently started task</span> as the active observer session.
Starting a new task automatically clears prior observer data so the timeline stays aligned with what
the user is currently experiencing.
</p>
<p className="mb-0 mt-2">
Use <span className="text-foreground">Start Observed Task Scenario</span> or any new Cline task to
begin a fresh observer session. You no longer need to manually reset the observer to correlate task
start with user-perceived events.
</p>
</div>
{latencyObserver && (
<div className="rounded border border-[var(--vscode-panel-border)] p-3 text-xs">
<div className="font-medium text-foreground">Observer event timeline</div>
<p className="mb-0 mt-2 text-(--vscode-descriptionForeground)">
Wall clock time, elapsed time since task start, and the event marker are shown together so you can
follow perceived progress without scrolling to the bottom of the tab.
</p>
<div
className="mt-3 max-h-56 overflow-auto overscroll-contain rounded border border-[var(--vscode-panel-border)] p-2"
onWheelCapture={containScrollWithinElement}
style={{ overscrollBehavior: "contain" }}>
{timelineEntries.length === 0
? "No observer events yet."
: timelineEntries.map((entry) => (
<div
className="grid grid-cols-[auto_auto_1fr] gap-x-2 py-0.5"
key={`${entry.ts}-${entry.message}`}>
<span className="text-(--vscode-descriptionForeground)">
{formatWallClockTime(entry.ts)}
</span>
<span className="text-(--vscode-descriptionForeground)">
{formatSinceTaskStart(entry.ts, latencyObserver.session.startedAt)}
</span>
<span>{entry.message}</span>
</div>
))}
</div>
</div>
)}
<label className="flex flex-col gap-1 text-xs text-(--vscode-descriptionForeground)">
<span>Ping payload bytes</span>
<input
aria-label="Ping payload bytes"
className="bg-input text-foreground border border-input rounded px-2 py-1"
min={0}
onChange={(event) => setPayloadBytes(Math.max(0, Number(event.target.value) || 0))}
type="number"
value={payloadBytes}
/>
</label>
<div className="grid grid-cols-1 gap-2 md:grid-cols-2 xl:grid-cols-3">
<Button
className="w-full justify-center"
disabled={isPinging || isContinuousPinging || isRunningPayloadSweep}
onClick={() => runPing()}
variant="secondary">
{isPinging ? "Pinging..." : "Run Ping Probe"}
</Button>
<Button
className="w-full justify-center"
disabled={isRunningPayloadSweep}
onClick={toggleContinuousPing}
variant="secondary">
{isContinuousPinging ? "Stop Continuous Ping" : "Start Continuous Ping"}
</Button>
<Button
className="w-full justify-center"
disabled={isStartingScenario}
onClick={startObservedScenario}
variant="secondary">
{isStartingScenario ? "Starting Scenario..." : "Start Observed Task Scenario"}
</Button>
<Button
className="w-full justify-center"
disabled={isPinging || isContinuousPinging || isRunningPayloadSweep}
onClick={runPayloadSweep}
variant="ghost">
Test Payload Presets
</Button>
<Button className="w-full justify-center" onClick={() => setSamples([])} variant="ghost">
Reset Stats
</Button>
<Button
className="w-full justify-center md:col-span-2 xl:col-span-2"
disabled={!latencyObserver}
onClick={exportLatencyObserverSession}
variant="ghost">
Export Session JSON
</Button>
</div>
<div className="grid grid-cols-2 gap-2 text-xs">
<div>Samples: {stats.count}</div>
<div>Last: {stats.lastMs?.toFixed(2) ?? "-"} ms</div>
<div>Min: {stats.minMs?.toFixed(2) ?? "-"} ms</div>
<div>Max: {stats.maxMs?.toFixed(2) ?? "-"} ms</div>
<div>Avg: {stats.avgMs?.toFixed(2) ?? "-"} ms</div>
<div>Payload: {payloadBytes} bytes</div>
</div>
<div className="flex flex-wrap gap-2 text-xs text-(--vscode-descriptionForeground)">
{PAYLOAD_PRESETS.map((preset) => (
<button
className="rounded border border-[var(--vscode-panel-border)] px-2 py-1"
key={preset}
onClick={() => setPayloadBytes(preset)}
type="button">
{preset.toLocaleString()} B
</button>
))}
</div>
{pingError && <p className="m-0 text-xs text-[var(--vscode-errorForeground)]">{pingError}</p>}
{latencyObserver && (
<div className="flex flex-col gap-2 border-t border-[var(--vscode-panel-border)] pt-3 text-xs">
<div>Scenario: {selectedScenario.label}</div>
<div>Branch: {latencyObserver.session.branch ?? "unknown"}</div>
<div>Commit: {latencyObserver.session.commit?.slice(0, 8) ?? "unknown"}</div>
<div>Environment: {String(latencyObserver.session.environment ?? "unknown")}</div>
<div>Transport probe: {capabilityLabel[latencyObserver.capabilities.transportProbe]}</div>
<div>Full-state metrics: {capabilityLabel[latencyObserver.capabilities.fullStateMetrics]}</div>
<div>
Partial-message metrics: {capabilityLabel[latencyObserver.capabilities.partialMessageMetrics]}
</div>
<div>Chunkwebview timing: {capabilityLabel[latencyObserver.capabilities.chunkToWebviewTiming]}</div>
<div>Task UI delta metrics: {capabilityLabel[latencyObserver.capabilities.taskUiDeltaMetrics]}</div>
<div>Persistence metrics: {capabilityLabel[latencyObserver.capabilities.persistenceMetrics]}</div>
<div>Task init avg: {latencyObserver.taskInitialization.stats.avgMs?.toFixed(2) ?? "-"} ms</div>
<div>First visible avg: {latencyObserver.firstVisibleUpdate.stats.avgMs?.toFixed(2) ?? "-"} ms</div>
<div>
First full-state avg: {latencyObserver.firstFullStateUpdate.stats.avgMs?.toFixed(2) ?? "-"} ms
</div>
<div>
First partial avg: {latencyObserver.firstPartialMessageUpdate.stats.avgMs?.toFixed(2) ?? "-"} ms
</div>
<div>Chunkwebview avg: {latencyObserver.chunkToWebview.stats.avgMs?.toFixed(2) ?? "-"} ms</div>
<div>Observed requests: {latencyObserver.requestStart.stats.count}</div>
<div>State pushes: {latencyObserver.optionalCounters?.fullStatePushes ?? 0}</div>
<div>State bytes: {latencyObserver.optionalCounters?.fullStateBytes ?? 0}</div>
<div>Partial events: {latencyObserver.optionalCounters?.partialMessageEvents ?? 0}</div>
<div>Partial bytes: {latencyObserver.optionalCounters?.partialMessageBytes ?? 0}</div>
<div>Task UI deltas: {latencyObserver.optionalCounters?.taskUiDeltaEvents ?? 0}</div>
<div>Persistence flushes: {latencyObserver.optionalCounters?.persistenceFlushes ?? 0}</div>
{latestRequestSummary && (
<>
<div className="font-medium text-foreground">Latest request churn</div>
<div>Req state pushes: {latestRequestSummary.fullStatePushes}</div>
<div>Req state bytes: {latestRequestSummary.fullStateBytes}</div>
<div>Req partial events: {latestRequestSummary.partialMessageEvents}</div>
<div>Req partial bytes: {latestRequestSummary.partialMessageBytes}</div>
<div>Req UI deltas: {latestRequestSummary.taskUiDeltaEvents}</div>
<div>Req persistence flushes: {latestRequestSummary.persistenceFlushes}</div>
</>
)}
</div>
)}
</div>
</Section>
</div>
)
}
export default LatencyObserverSection