Compare commits

...

43 Commits

Author SHA1 Message Date
Max Paulus 🥪 c93356cf22 remove unused code/files 2026-06-05 21:27:45 -07:00
Max Paulus 🥪 ab71a8946c delete unused files 2026-06-05 15:45:21 -07:00
Max Paulus 🥪 2df0ad4661 Add edit and regenerate for VS Code chat messages
Allow user feedback messages in the VS Code extension to be edited inline and regenerated from that point. Adds a TaskService RPC, truncates persisted SDK history before the selected visible user prompt, and starts a new session with the edited prompt. Also ensures the regenerated active task appears in extension history while SDK history catches up.
2026-06-05 15:03:01 -07:00
Max Paulus 🥪 1dc74bfd6b fix webview-ui tests 2026-06-05 12:39:43 -07:00
Max Paulus 🥪 00117d18e4 show model list if possible for openai compatible 2026-06-05 12:30:38 -07:00
Max Paulus 🥪 7351e2a8c0 fix onboarding model selection not persisting 2026-06-05 11:50:42 -07:00
Max Paulus 🥪 cbbb115f18 remove provider-specific views and just use genericprovidersettings.tsx 2026-06-05 11:45:26 -07:00
Max Paulus 🥪 d1a3af4bae dry up duplicate code and create useProviderModelSelection 2026-06-05 11:18:52 -07:00
Max Paulus 🥪 2563532e84 dry up provider api key logic 2026-06-05 11:11:00 -07:00
Max Paulus 🥪 21c35c95da dry up some duplicate code 2026-06-05 11:07:54 -07:00
Max Paulus 🥪 b423611e11 fix onboarding models 2026-06-05 10:54:32 -07:00
Max Paulus 🥪 2812c0f760 fix failing biome/lint 2026-06-05 10:34:46 -07:00
Mikołaj Kondratek 54ac72bd75 Remove unused import 2026-06-05 10:03:12 -07:00
Mikołaj Kondratek a23b86e81a fix(sdk): drop dead autoContinue branch in mode rebuild
cf25cd66a ("make extension plan mode more similar to CLI") removed the
file-level ACT_MODE_CONTINUATION_PROMPT constant and stopped passing the
autoContinue / continuationPrompt options when rebuilding a session for
a mode change, but left the corresponding block inside
rebuildSessionForMode in place. The block still references the deleted
constant, so tsc fails on the SDK migration branch with TS2304: Cannot
find name "ACT_MODE_CONTINUATION_PROMPT".

No caller passes options to rebuildSessionForMode anymore, so the block
is dead. Drop the block and narrow the signature to take only newMode.
Existing tests already invoke rebuildSessionForMode(<mode>) with no
second argument and assert that fireAndForgetSend is not called on a
mode rebuild, so they keep passing.
2026-06-05 10:03:12 -07:00
Mikołaj Kondratek d50bcb777c fix(terminal): capture standalone terminal output on Windows and harden PowerShell command handling (#11133)
* fix(terminal): surface standalone terminal spawn diagnostics

Add Logger calls at every chokepoint of the standalone terminal pipeline
so the (currently silent) failure modes around JetBrains-hosted
cline-core become debuggable from cline-core-service.log.

Lines added, all using the existing Logger facility (no new
dependencies, no behavioral changes):

* StandaloneTerminalProcess.run() now logs:
  - `[StandaloneTerminalProcess] run() entered: shell=… cwd=… args=…`
    on entry, before the try block;
  - `[StandaloneTerminalProcess] spawned pid=… for shell=…` right
    after child_process.spawn returns;
  - `[StandaloneTerminalProcess] close: code=… signal=… fullOutputLen=…`
    inside the `close` handler (the `fullOutputLen` reveals when the
    child exits 0 with empty pipes — the symptom in issue #10948);
  - `[StandaloneTerminalProcess] child error: …` in the `error`
    handler;
  - `[StandaloneTerminalProcess] spawn threw synchronously: …` in
    the outer catch.

* StandaloneTerminalManager.runCommand() now logs entry
  (`[StandaloneTerminalManager] runCommand terminalId=…: <cmd>`) and
  attaches a `.catch` to the previously fire-and-forget
  `process.run(…)` Promise so an unhandled rejection surfaces as
  `[StandaloneTerminalManager] process.run rejected for terminal …`
  instead of disappearing.

* CommandExecutor.execute() extends the existing "Executing command
  in … terminal" line with `mode=<terminalExecutionMode>` and
  `managerCtor=<manager.constructor.name>`, so it's possible to
  confirm whether the `vscodeTerminal` path is in fact backed by a
  `StandaloneTerminalManager` on JetBrains (it is — see
  notes/issue-10948-…md).

* CommandOrchestrator.orchestrateCommandExecution() logs the
  `process.once("completed")` event with `exitCode`/`signal`/
  `terminalType`, the "resolved completed" return branch with the
  line/byte totals, and emits a `WARN` on the silent "still running"
  fall-through. The last one matters because the original repro
  reported "Command executed successfully (exit code 0)" with empty
  output — the WARN makes that branch loud the next time it fires.

These logs are what made the two distinct bugs in #10948 visible
(see the 2026-05-28 update in
notes/issue-10948-terminal-output-investigation-2026-05-27.md). They
stay in to keep the next regression debuggable.

Refs: cline/cline#10948

* fix(terminal): keep Windows child stdio attached to parent pipes

The non-cmd Windows branch in StandaloneTerminalProcess.run() spawned
the shell (powershell.exe in practice) with `detached: true` and no
`windowsHide`. When cline-core is launched by the JetBrains plugin it
has no console of its own, so Windows CreateProcess allocates a NEW
console for the detached child and the child's stdio routes to that
new console instead of the pipe handles the parent created. From the
parent's point of view the pipes immediately EOF, `close` fires with
`code=0`, and `fullOutput` is 0 bytes — exactly the symptom reported
in cline/cline#10948 ("Command executed successfully (exit code 0)"
with no output and no filesystem effect).

This bug applies to every command the agent runs through the
standalone terminal path on Windows, not just the
double-wrapped-PowerShell cases (verified by re-running a clean
`dir <file>` after the diagnostics from the previous commit landed:
`run() entered` and `spawned pid=<num>` both fired, then `close: code=0
fullOutputLen=0`).

Fix:

* `detached: process.platform !== "win32"` — keep the existing
  POSIX behavior (a separate process group helps `tree-kill`), but
  drop it on Windows where `tree-kill` walks the PID tree with
  `taskkill /T` and doesn't need a process group.
* `windowsHide: true` — matches every other `child_process.spawn`
  call site in cline-core (git, MCP, hooks, browser) and flips on
  `CREATE_NO_WINDOW`, keeping the child attached to our pipes
  without popping a console window.

Verified on Windows 11 + IntelliJ IDEA 2026.1 + Cline plugin
1.1.59-Internal: `dir <path>`-style probes now produce a non-zero
`fullOutputLen` in the close log, and the captured output bytes
match what would have been visible interactively. PowerShell
double-wrapping (the other half of #10948) is handled in a
follow-up commit.

Refs: cline/cline#10948

* fix(terminal): harden PowerShell command wrapping for standalone shell

`StandaloneTerminalProcess.getShellArgs()` blindly wrapped every
PowerShell command as `["-Command", command]`. That has two
end-user-visible failure modes on Windows, both observed in
cline/cline#10948:

1. The agent's `run_commands` tool call sometimes arrives already
   prefixed with `powershell -Command "…"`. We then spawned
   `powershell.exe -Command 'powershell -Command "…"'`, and the
   outer shell shredded the inner single/double-quote pairs while
   re-parsing its `-Command` argument. The inner pwsh saw
   quote-empty `Test-Path` calls, fell through to the `else` branch
   and reported "File not found" — to ITS stdout, which the outer
   inherited but the file deletion the LLM intended never ran.
2. The user's `$PROFILE` script ran on every spawn, leaking
   non-deterministic noise (e.g.
   `%windir%\System32\REG.exe : The module '%windir%' could not be
   loaded`) into the captured output and confusing the agent.
3. Bonus: the POSIX branch used `["-l", "-c", command]`. The `-l`
   re-sources login files on every command, which is slow and lets
   greeter scripts leak into output.
4. Bonus: the cmd branch used `["/c", command]`. `/d` skips
   AutoRun, `/s` makes the embedded-quote handling deterministic.

Fix:

* PowerShell branch returns
  `["-NoProfile", "-NonInteractive", "-Command", unwrap(command)]`.
  `-NoProfile` suppresses (1) the spurious profile noise that
  contaminated the captured output, and `-NonInteractive` ensures
  the child doesn't deadlock waiting on a prompt no one will answer.
* `unwrapPowerShell(command)` strips a leading
  `powershell|pwsh [.exe] -Command|-c "…"` (or single-quoted)
  wrapper that the LLM sometimes emits, fixing the double-pass
  argument-quoting destruction. If the command does not match the
  exact wrapper shape it is returned verbatim — worst case is "no
  change", preserving pre-fix behavior.
* cmd branch returns `["/d", "/s", "/c", command]`, matching the
  canonical helper in cline/sdk/packages/shared/src/parse/shell.ts.
* POSIX branch returns `["-c", command]`, dropping the unhelpful
  `-l`. Also matches the SDK helper.

Verified on Windows 11 + IntelliJ IDEA 2026.1 + Cline plugin
1.1.59-Internal in combination with the previous "keep Windows
child stdio attached" commit: `Remove-Item CHANGELOG.md` now
deletes the file, the agent's verification `Get-ChildItem CHANGELOG*`
returns nothing, and the profile-load REG.exe error no longer leaks
into captured output.

Refs: cline/cline#10948

* refactor(terminal): tone down standalone terminal diagnostics

The diagnostics added while chasing #10948 were intentionally loud so the
two bugs were visible. Now that the fixes are in, reduce them to a normal
operating posture:

* Demote fine-grained traces to `debug`: the per-spawn `spawning …` and
  `spawned pid=…` lines, `StandaloneTerminalManager.runCommand`, and the
  orchestrator's `resolved completed` summary.
* Drop the orchestrator's `completed event` line entirely — the
  `resolved completed` debug line already carries exit code, signal, and
  line/byte totals.
* Stop echoing the full command in the manager line and stop echoing the
  args vector in the spawn line. The command is still logged once at
  `info` by CommandExecutor (unchanged, pre-existing), so we go back from
  three command echoes to one. Commands routinely embed secrets
  (Authorization headers, tokens), so fewer copies on disk is better.

Kept loud on purpose:

* `info` on `close: code=… fullOutputLen=…` — the single line that proves
  the Windows stdio-capture fix and the most useful per-command signal.
* `warn` on `resolved without completion event` — the silent-success
  canary for the #10948 failure mode.
* `error` on child error / synchronous spawn failure / unhandled
  process.run rejection.

Refs: cline/cline#10948

* fix(terminal): tighten PowerShell unwrap regex and extract to a pure module

Two review follow-ups for the #10948 shell-arg handling:

1. The wrapper-strip regex used a greedy `([\s\S]*)` body, so a command
   like `powershell -Command "foo" "bar"` would match with the body
   captured as `foo" "bar`, silently rewriting a command into something
   different. Replace the body with a tempered match `((?:(?!\1).)*)`
   that cannot contain the captured delimiter, so anything other than
   exactly one quoted token is returned verbatim. Worst case is now
   "no change" rather than an incorrect rewrite. The legitimate
   double-wrapped case from #10948 (outer ", inner ') still unwraps.

2. `getShellArgs` and `unwrapPowerShell` were private methods on
   StandaloneTerminalProcess, untestable without spawning a process.
   Move them to a pure `shellArgs.ts` module. `getShellArgs` now takes
   an injectable `platform` (defaulting to `process.platform`) purely so
   the win32-vs-posix branch is testable; behavior is unchanged. This
   also gives us a single local seam to later consolidate onto the
   canonical `@cline/shared` helper (tracked as a follow-up).

No behavioral change beyond the regex correctness fix.

Refs: cline/cline#10948

* test(terminal): cover shell-arg construction and PowerShell unwrap

Add mocha unit tests (matching the repo's node:assert/strict + __tests__/
convention so the existing mocharc spec globs pick them up) for the newly
extracted shellArgs module:

* unwrapPowerShell: double-quote and single-quote wrappers, powershell.exe
  -c form, the #10948 nested-quote repro (inner quotes preserved),
  non-wrapped passthrough, and the two regressions the tightened regex
  must reject (`… "foo" "bar"` and a command that merely mentions
  powershell mid-string).
* getShellArgs: PowerShell -> -NoProfile -NonInteractive -Command (with
  unwrap), cmd -> /d /s /c, POSIX -> -c. The injectable platform arg lets
  these run on any CI host.

This closes the M1 review finding (the regex was the riskiest line in the
change and had zero coverage) and exercises the cmd/POSIX flag changes
called out in M2.

Refs: cline/cline#10948

* docs(terminal): drop issue references and clarify windowsHide comment

Remove inline issue-number references from source comments and a test
name; that context belongs in the commit history, not the code. Also add
a one-line note that windowsHide is a no-op on non-Windows platforms,
since it is set unconditionally while the surrounding comment is
Windows-specific.

No behavior change.

* refactor(terminal): drop warn on the non-completion return path

The orchestrator's final fall-through return is a normal, expected path:
the process resolved via `continue` without a `completed` event (e.g. a
terminal mode without shell integration, or proceed-while-running flows).
Logging it at `warn` cries wolf on healthy runs, so remove it. The
genuine failure mode this was meant to catch surfaces through the
`close`/error logs and the result string itself.

* fix(terminal): address review feedback on standalone spawn paths

Three follow-ups from code review:

* StandaloneTerminalManager.runCommand: the unawaited process.run()
  .catch only logged. run() emits "error" for failures it catches, but a
  rejection escaping without an "error" event would leave the outer
  promise (resolved via the "continue"/"error" events) pending forever,
  stalling the caller. Re-emit "error" from the catch so both paths stay
  consistent. Cannot trigger today (no await outside run()'s try/catch)
  but the guard exists precisely for future rejections.

* shellArgs POSIX branch: document that dropping the login flag (`-l`)
  is intentional and relies on the child inheriting the parent's PATH via
  process.env, with a note that a GUI-launched IDE without a login PATH
  is the edge case to watch.

* StandaloneTerminalProcess cmd.exe branch: add windowsHide:true. The
  console-allocation/window-pop problem is not exclusive to the non-cmd
  branch; a console-less parent could pop a window for cmd.exe too.
  No-op on non-Windows.
2026-06-05 10:03:11 -07:00
Ara af6778dd37 Remove Explain Changes feature (#11278)
* chore(vscode): remove explain changes entry points

* chore(vscode): remove explain changes feature
2026-06-05 10:03:11 -07:00
Max Paulus 🥪 ea68ad29c6 make extension plan mode more similar to CLI
- basically, don't auto continue when agent switches to act mode
2026-06-05 10:03:11 -07:00
Max Paulus 🥪 f96fa41870 fix zai insufficient credits issue 2026-06-05 10:03:11 -07:00
Max Paulus 🥪 3b34f6fdea fix tool use name sanitization 2026-06-05 10:03:11 -07:00
Max Paulus 🥪 ec56a140fb fix broken tsc 2026-06-05 10:03:10 -07:00
Dominic Cooney c955075db1 fix(vscode): exclude vitest src/sdk suites from CommonJS test compile
compile-tests runs 'tsc -p tsconfig.test.json' (module: commonjs) over all
src/**/*.test.ts for the VS Code integration runner. The new src/sdk vitest
suites use top-level 'await import(...)' (after vi.mock), which is invalid
under CommonJS and fails with TS1378. The integration runner never runs
src/sdk anyway (.vscode-test.mjs only globs core/test/utils/shared/
integrations/hosts/services); these run via 'npm run test:vitest'. Exclude
src/sdk/**/*.test.ts from the integration compile.
2026-06-05 10:01:21 -07:00
Dominic Cooney 701c29589b fix(vscode): restore biome --config-path so lint resolves apps/vscode/biome.jsonc
The rebase dropped '--config-path ./biome.jsonc' from the lint/format/
postprotos scripts and removed the '!!**/.vscode-test' ignore from
biome.jsonc. Without the explicit config path, biome auto-discovered the
root biome.json instead of apps/vscode/biome.jsonc, applying the wrong
rule severities (449 errors at error level for rules that are off/info in
the nested config). Restore both to match origin/main and apply the
pending buf format fix to models.proto.
2026-06-05 10:01:20 -07:00
Dominic Cooney 2135b07ed0 fix(vscode): show running state for in-progress commands
The command row reflects an executing state while a command runs. The
message translator includes the command-output marker on the running
command row so the webview renders it as executing; the row is finalized
with output and a completed flag when the command ends.

Also remove the unused onChange parameter from the foreground run_commands
path: the SDK runtime does not pass it, so it had no effect. Foreground
command output is surfaced to the chat at completion, not incrementally.

Fixes CLINE-2298 and CLINE-2162
2026-06-05 10:00:15 -07:00
Dominic Cooney 13de185d07 fix(vscode): re-enable approval buttons for consecutive asks
The footer Approve/Reject buttons stayed disabled when a second consecutive
approval ask arrived. The button configs are shared singletons (e.g.
BUTTON_CONFIGS.tool_approve), so two identical asks return the same object
reference and the effect that reset the processing latch never re-ran.

Key the processing latch on the ask identity (anchored turn timestamp plus the
button labels) rather than the config object reference, using a ref-based latch
so each new ask re-enables the buttons. Adds a regression test.

Test plan:

1. Ask the agent to generate two requests to ls /tmp at once

2. Approve (or reject) the first request

3. Check that the buttons for the second request are enabled
2026-06-05 10:00:15 -07:00
Dominic Cooney 82c9492e0c feat(vscode): add the VS Code Language Model (vscode-lm) provider
Run Cline inference through the VS Code Language Model API (vscode.lm), enabling
models contributed by any extension that registers a language model chat
provider with VS Code. GitHub Copilot is the most common such vendor, but the
implementation is vendor-agnostic — it selects models via
vscode.lm.selectChatModels and has no Copilot-specific logic.

- VsCodeLmHandler implements the Cline SDK ApiHandler and is registered with the
  SDK handler registry; the model selector travels as a vendor/family[/version/id]
  string in modelId and is parsed back here. Selector segments are
  percent-encoded so values containing slashes round-trip intact.
- Native tool calling: tool definitions are passed to sendRequest and tool calls
  are surfaced as tool-call chunks; tool results round-trip as
  LanguageModelToolResultPart, with structured tool output serialized to text and
  a trailing user message appended when a turn ends on tool results so models can
  read the output.
- Gated to VS Code: registration is conditioned on the vscode.lm API being
  present, and the provider is hidden in the UI on hosts without it (JetBrains).

Depends on @cline/{shared,llms,agents,core} 0.0.42-nightly.1780514867, the first
published SDK build with the custom-registered-handler routing this provider
needs.
2026-06-05 10:00:15 -07:00
Ara 0cb9893732 Fix approval chat replies rendering as tool errors (#11246)
* fix(vscode): route approval chat replies as user feedback

* fix(vscode): suppress approval reply denial errors

* fix(vscode): hide rejected approval tool failures

* chore(vscode): clarify denied approval suppression helper
2026-06-05 10:00:14 -07:00
Max Paulus 🥪 2b6a4a3def Persist OpenRouter provider config via catalog hook 2026-06-05 10:00:14 -07:00
Max Paulus 🥪 a0d6665d58 persist openai-codex provider model settings
- also don't show a sign in button if openai-codex is the only provider
2026-06-05 10:00:14 -07:00
Max Paulus 🥪 c3799f2db3 Persist Cline model selections to provider config 2026-06-05 10:00:14 -07:00
Dominic Cooney b74466fd64 fix(vscode): declare missing direct deps @grpc/proto-loader and @opentelemetry/api-logs
Both packages are imported directly from source but were never declared in
apps/vscode/package.json, so they only resolved transitively. On a clean
install this broke:

- @grpc/proto-loader — imported by scripts/proto-utils.mjs,
  src/standalone/utils.ts and src/standalone/hostbridge-client.ts; its absence
  made `npm run protos` (and therefore the whole build) fail on a fresh checkout.
- @opentelemetry/api-logs — imported by the OpenTelemetry telemetry providers;
  its absence produced TS2307 "Cannot find module" errors under tsc.

Versions are pinned to align with the existing dependency families already
declared in this package (@grpc/grpc-js ^1.9.x → proto-loader ^0.7.13;
the @opentelemetry/* 0.56.x line → api-logs ^0.56.0). The npm and bun
lockfiles are updated accordingly (the api-logs change also dedupes several
previously-nested copies to a single hoisted entry).
2026-06-05 10:00:14 -07:00
Max Paulus 🥪 e1674d0724 show legacy task history that is not saved in the ~/.cline folder 2026-06-05 09:58:56 -07:00
Max Paulus 🥪 11e0ea9bd4 add migration telemetry 2026-06-05 09:58:56 -07:00
Ara 81022b4dc9 fix(vscode): reuse approved tool rows (#11213)
* fix(vscode): reuse approved tool rows

* fix(vscode): clear stale approved tool rows
2026-06-05 09:58:56 -07:00
Ara 77e1ce3ead fix(llms): strip Cerebras reasoning history (#11214) 2026-06-05 09:58:55 -07:00
Max Paulus 🥪 abc710aa12 fix unauthed user flow
- show a small sign in button if user is unauthed with any provider
2026-06-05 09:58:55 -07:00
Robin Newhouse 7544131666 fix(vscode): compact Codex OAuth before input cap (#11194)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-05 09:58:55 -07:00
Robin Newhouse 5112170461 fix(vscode): wire auto compact into SDK sessions (#11197)
* fix(vscode): wire auto compact into SDK sessions

* test(sdk): cover both directions of useAutoCondense task override

The previous test left the global mock at `true` for both calls, so the
`taskSettings: true` branch would have passed even if task settings were
ignored entirely. Make the mock read a mutable flag and flip it to `false`
before the second call so both override directions — task `false` over
global `true`, and task `true` over global `false` — are genuinely
exercised.

* Fix mock return type in cline-session-factory test

The getGlobalSettingsKey mock inferred a literal 'false | undefined' return type, so later mockImplementation overrides returning 'true' failed type checking (TS2345). Annotate the implementation as 'boolean | undefined' to widen the inferred mock signature.

---------

Co-authored-by: Mikołaj Kondratek <19799111+mkondratek@users.noreply.github.com>
2026-06-05 09:58:55 -07:00
Dominic Cooney 080ba5d036 fix(vscode): keep in-progress MCP OAuth flow across reconnects
The MCP SDK calls redirectToAuthorization() on every connection attempt,
and a single server can be reconnected repeatedly (settings watcher,
reconnect handler, restart). Regenerating the OAuth `state` on each call
replaced the state stored for a flow whose authorization URL the user may
already have open, so the completed callback failed validation with
"Invalid OAuth state".

redirectToAuthorization() now keeps an in-progress, still-fresh flow
instead of starting a new one (freshness measured from when the flow
started, never extended, so a stale flow always expires). The PKCE
verifier is pinned to the kept flow so token exchange still validates.

Also add local dev/test tooling:
- src/dev/mcp-oauth-test-server: a zero-dependency OAuth AS + MCP
  StreamableHTTP server for exercising the flow locally, with
  fault-injection flags (--auto-deny, --slow-authorize, --code-ttl).
- src/extension.ts: a debug-only globalThis.__clineHandleUri hook (gated
  on CLINE_CAPTURE_BROWSER) so the debug harness can deliver simulated
  vscode:// OAuth callbacks; documented in the harness README.
2026-06-05 09:58:55 -07:00
Dominic Cooney 8a35154a0b fix(vscode): forward Bedrock region + AWS auth to the SDK gateway
Bedrock requests built by the SDK adapter dropped the AWS region and
authentication mode, so a pasted Bedrock API key (awsBedrockApiKey +
awsAuthentication "apikey") was silently ignored and requests fell
through to the SigV4 credential chain with no region.

Two bugs, both verified end-to-end against a live Bedrock endpoint via
the debug harness:

1. The SDK ProviderConfig was built with only providerId/modelId/apiKey/
   baseUrl. New bedrock-config.ts maps the legacy ApiConfiguration onto
   the SDK's structured region + aws block (including the webview's
   "credentials" radio -> SDK "iam"), wired into both inference paths
   (buildSdkProviderConfig for utility calls, buildSessionConfig for the
   main task loop).

2. The main chat path's gateway config is built by core from the
   providers.json `stored` entry, which the session's providerConfig does
   not override. A stale Bedrock entry (e.g. a legacy migration with
   region us-east-1 + SigV4 keys) silently won, sending requests to the
   wrong region (403). buildSessionConfig now persists the
   StateManager-derived Bedrock settings to providers.json so `stored` is
   authoritative. The bearer apiKey is only persisted for api-key auth to
   keep stored clean for profile/iam.

Also documents the ELECTRON_RUN_AS_NODE debug-harness gotcha in
.clinerules/general.md.
2026-06-05 09:58:55 -07:00
Max Paulus 🥪 904429ddce fix xai provider
- xai provider settings now properly updates providers.json
2026-06-05 09:58:54 -07:00
Max Paulus 🥪 f021fb5891 updat gitignore 2026-06-05 09:58:54 -07:00
Max Paulus 🥪 700038f62c sdk migration: squashed 06-05-2026 -- instead of listHistory, use host.get(sessionId) instead
fix sesion usubscriptions

fix soft-lock on auth fail retry

refactor(vscode): source provider model catalogs from @cline/llms SDK
(ENG-2116)

Migrates the vscode extension off its hand-curated static model catalogs
in apps/vscode/src/shared/api.ts and on to the @cline/llms SDK as the
single source of truth for provider/model metadata, end-to-end across
the extension host and the webview.

Net impact on the static catalog file:
  apps/vscode/src/shared/api.ts: 5092 -> 468 lines (~90% gone).

What changed at each layer
--------------------------

SDK / catalog plumbing (apps/vscode/src/sdk/model-catalog/):
  - New `ProviderCatalog.peekModels(providerId)` synchronous cache read.
  - `resolveModelInfo` rewritten: committed selection -> catalog peek
    -> await catalog.resolveModels on cache miss. No race with a
    background warmer; if the catalog truly has nothing, returns
    source: "unknown" and the webview renders a neutral loading state.
  - `applyHostModelInfoOverrides` is the canonical seam for the few
    fields the SDK does not yet carry. Today it carries only the Vertex
    `supportsGlobalEndpoint` allowlist (vertex-global-endpoint.ts, with
    a TODO to upstream into the SDK).
  - `ProviderListing` extended with SDK metadata (`is_popular`,
    `popular_rank`, `usage_cost_display`, `capabilities[]`) and plumbed
    through proto + conversion.

Extension-host handlers (apps/vscode/src/core/api/providers/):
  - New shared helper `apps/vscode/src/shared/sdk-handler-models.ts`:
    `getProviderModelFromSdk(providerId, requestedModelId,
committedInfo?)`
    returns `{ id, info }` from `getProviderCollectionSync` with
    Vertex global-endpoint overrides applied.
  - 27 handlers converted to a one-liner `getModel()` through that
    helper. Per-handler nuances preserved:
      * Anthropic: strips `:fast` and `:1m` host-side suffixes before
        SDK lookup; carries them back on the returned id so the
        per-request betas still flip.
      * Bedrock: keeps the custom Application Inference Profile ARN
        branch; base-model info from the SDK.
      * Cerebras: keeps the `qwen-3-coder-480b-free` ->
`qwen-3-coder-480b`
        paid alias.
      * Qwen / ZAi: SDK has a single catalog each; handlers keep the
        regional base-URL switch but no longer fork the catalog.
      * Wandb: keeps the "unknown id falls through to safe defaults"
        escape hatch via `MODEL_COLLECTIONS_BY_PROVIDER_ID`.

Refresh-models background tasks:
  - `refreshBasetenModels`, `refreshGroqModels`,
`refreshHuggingFaceModels`
    source their offline-fallback catalog from the SDK via
    `getProviderCollectionSync` + `adaptSdkModelInfo`. Live fetch path
    unchanged; only the seeding/fallback data changed.

Webview (apps/vscode/webview-ui/):
  - `useNormalizedApiConfiguration` always routes through gRPC
    `resolveModelInfo`. Removed the `isMigratedSdkProvider` /
    `MIGRATED_SDK_PROVIDER_IDS` feature flag and the legacy
    `normalizeApiConfiguration` switch entirely.
  - New `useStaticProviderSelection` hook for the 22 settings
    components whose catalog is now SDK-driven, and
    `useDynamicProviderSelection` for the 12 dynamic-list pickers
    (openrouter, cline, openai-compatible, ollama, lmstudio, requesty,
    litellm, hicap, groq, baseten, huggingface, vercel-ai-gateway,
    aihubmix, oca, huawei-cloud-maas, dify, fireworks, together,
    vscode-lm) so all of them stop calling the legacy switch.
  - `ModelInfoView` reads its `isGemini` check via
    `useProviderModels("gemini")` instead of importing `geminiModels`.
  - `App.stories.tsx` ships a small inline fixture instead of
    importing `bedrockModels`.
  - `ExtensionStateContext` no longer seeds `groqModelsState` /
    `basetenModelsState` from the deleted catalog; the slices start
    empty and the SDK-curated catalog is layered in by the pickers at
    render time.

Misc:
  - `src/utils/model-utils.ts`: `isAnthropicModelId` consults
    `MODEL_COLLECTIONS_BY_PROVIDER_ID["anthropic"]` instead of the
    deleted `anthropicModels` map.
  - `src/shared/storage/provider-keys.ts`: `getProviderDefaultModelId`
    no longer hard-codes 24 per-provider defaults. The function now
    consults the SDK catalog and only keeps an override map for
    providers whose default is intentionally not the SDK default
    (openrouter-shared dynamic providers and local-only providers).
  - `src/shared/openai-codex-models.ts`: relative path for
    `shape-adapter` import so both the extension and webview build
    contexts resolve it identically.

Tests retargeted to assert SDK behavior, not static-map shapes
--------------------------------------------------------------
  - claude-code, anthropic, bedrock, vertex, wandb, provider-keys
    test suites had assertions tied to deleted shapes. Rewrote them
    to either assert through the SDK catalog
    (anthropic compares against
`adaptSdkModelInfo(sdkCollection.models[id])`,
    wandb uses the SDK-declared default, etc.) or focus on the
    host-side semantics (bedrock's "global endpoint" block now tests
    `vertexModelSupportsGlobalEndpoint` directly).
  - claude-code test trimmed its 8 `[1m]`/version-pin variants down
    to three SDK-shaped cases. The 8 deleted assertions exercised
    extension-only model-id derivations that the SDK does not carry;
    matching the CLI's behavior was the explicit goal.
  - resolveModelInfo test rewritten around the new peek -> await
    -> unknown contract.
  - proto-lint: added missing `go_package` option to
    `proto/cline/remote_config.proto`.

Verification (npm scripts under apps/vscode/):
  - npm run protos          OK
  - npm run check-types     OK  (apps/vscode + apps/vscode/webview-ui)
  - npm run lint            OK  (biome + proto-lint)
  - npm run build:webview   OK  (tsc -b && vite build)
  - node esbuild.mjs        OK  (dist/extension.js produced)
  - Runtime smoke test: 27/27 provider collections resolve from the
    SDK with correct model counts, defaults, and usage-cost-display
    flags. openai-codex returns cost=hide as expected; every other
    provider returns cost=show.

fix(vscode): preserve provider model selection fields

chore(vscode): clean up SDK migration branch (comments, dead code,
scree, tests)

- Remove porting scree: docs/sdk-model-catalog/* planning docs,
TODO-resume-session.md, and stale doc references in code/comments.

- Rewrite before/after narrative comments in the 'eternal now' style;
drop transient 'Step N'/'Phase N' labels.

- Remove leftover [HistoryPerf] diagnostic logging and a dead try/catch
rethrow; minor readability/naming.

- Restore .clinerules/network.md (still relevant) and update
sdk-migration.md to drop dangling references.

- Fix latent circular-init TDZ in openai-codex-models (lazy catalog
build).

- toggleRemoteConfigSetting no longer returns a never-resolving promise.

- Move vitest config into apps/vscode and rename script test:sdk ->
test:vitest; wire it into CI.

- Repair/reimplement the SDK-adapter vitest suites (auth-service,
provider-migration, sdk-task-history) to match current behavior; all 432
tests pass.

fix(vscode): support OpenAI Compatible provider on the SDK adapter

Selecting the OpenAI Compatible provider failed with 'Unknown provider
"openai"', and manually entered model ids were displayed as the catalog
default (gpt-4o).

- Map the extension's 'openai' provider id to the SDK's
'openai-compatible' built-in at the SDK boundary (toSdkProviderId), and
convert before handing the provider id to core when building a session
config.
- Treat openai-compatible as a custom-model-id provider so model
resolution honors a user-entered model id instead of coercing it to the
catalog default. Adds providerAllowsCustomModelIds() as the shared
signal.
- Bump @cline/core, @cline/llms, @cline/shared, @cline/agents to ^0.0.42
(which registers the openai-compatible built-in) and dedupe the
dependency tree.
- Carry the tool name on reconstructed Anthropic-format tool_result
blocks to satisfy the SDK's ToolResultContent contract.
- Update tests for the refreshed SDK model catalog (Gemini default).

fix(vscode): fix webview provider-model tests and a Windows path test

The nightly publish runs the full webview-ui suite and the Windows
extension suite, which surfaced failures the extension-only run does
not.

- Harden useProviderModels against a missing providerModelsByProvider
map so a partially-mocked ExtensionStateContext no longer crashes the
hook.
- Update the SapAiCoreModelPicker and APIOptions specs to provide the
provider model-list context the components now read from the SDK
catalog, and seed the model ids each test asserts. Removes the now-dead
@shared/api sapAiCoreModels mock.
- Compare resolveDataDir() against path.join() instead of a hardcoded
POSIX path so the CLINE_DIR fallback test passes on Windows.

fix(vscode): load real @cline/llms in unit-test harness; fix/trim
provider tests

The mocha unit-test harness stubbed @cline/llms with an empty catalog,
so the SDK-migrated provider handlers (which read the real catalog)
failed ~51 tests under the nightly suite.

- src/test/requires.ts now loads the real @cline/llms by resolving its
package directory and requiring the ESM entry by absolute path (Node 22
require(esm)), bypassing the package's import-only exports map. This
restores real catalog data to provider unit tests.
- vertexModelSupportsGlobalEndpoint: also match legacy ':'
context-window/speed suffixes (e.g. claude-opus-4-7:1m), not just '@'
snapshot variants.
- refreshClineModels: derive prompt-cache support from reported
input_cache_read pricing for any provider, not just openai/google
prefixes.
- Delete the failing tests for the classic provider handlers (bedrock
cross-region/native-tool-calling, gemini metadata, cline/openrouter qwen
cache, wandb unknown-model). These handlers are only reached via
buildApiHandler (explain-changes + commit-message generation) and the
provider/catalog domain is owned and tested by the SDK; they will be
removed when buildApiHandler is retired.

refactor: remove legacy Cline model overrides, wire reasoning effort
through SDK

- Delete refreshClineModels.ts, refreshClineModelsRpc.ts, and test
- Remove refreshClineModelsRpc proto RPC
- Remove clineModels state from ExtensionStateContext, StateManager
cache, disk
- ClineModelPicker and useOnboardingModels use SDK catalog directly
- Replace ThinkingBudgetSlider with ReasoningEffortSelector for Cline
provider
- Use supportsReasoning from SDK catalog instead of hardcoded model
names
- Add ProviderReasoningPatch to proto and ProviderConfigPatch contract
- Wire reasoning effort changes through writeProviderConfig to SDK
ProviderSettingsManager
- Remove hardcoded Claude switch in openrouter-stream.ts
- Add supportsReasoning to supportsReasoningEffort check in
openrouter-stream

refactor(vscode): route buildApiHandler through the SDK; remove
OpenRouter handler

Replace the legacy per-provider buildApiHandler factory with an
SDK-backed
handler (apps/vscode/src/sdk/sdk-api-handler.ts) built via @cline/llms
createHandler(). The two standalone callers (commit-message generation
and
explain-changes) now import buildApiHandler directly from the SDK
module;
@core/api stays types-only (re-exporting SDK types) so it can keep being
imported widely without pulling the SDK runtime graph into activation.

Also:
- Delete the now-dead OpenRouterHandler and its test
(createOpenRouterStream
  stays; it is still used by ClineHandler).
- buildSdkProviderConfig never sends both reasoning.effort and
  reasoning.max_tokens (some providers reject it), and supports
  disableReasoning for fast one-shot utility calls; commit-message and
  explain-changes opt in.
- commit-message generation surfaces the real SDK stream error instead
of a
  generic "empty API response".
- getGitDiff: run git diff --staged even before the first commit; only
gate
  the git diff HEAD fallback on having commits (fixes "no changes" for
the
  initial commit of a new repo).

refactor(vscode): delete legacy API provider handlers and dead
transforms

Now that buildApiHandler routes through the @cline/llms SDK, the legacy
per-provider handler classes and their supporting code are unused.
Remove them:

- apps/vscode/src/core/api/providers/** (all 40+ handler classes, types,
tests)
- apps/vscode/src/core/api/transform/** except stream.ts (format/stream/
  tool-call helpers only the handlers used)
- apps/vscode/src/core/api/utils/** (messages/responses API support)
- apps/vscode/src/core/api/retry.ts (+ test)
- apps/vscode/src/shared/sdk-handler-models.ts (getProviderModelFromSdk;
  only the deleted handlers' getModel() used it)

core/api now contains just index.ts (types + SDK re-exports),
transform/stream.ts
(ApiStream types still referenced by the local ApiHandler interface),
and
adapters/.

Supporting changes to keep everything compiling/working:
- context-window-utils: drop the dead `api instanceof OpenAiHandler`
DeepSeek
  branch (handlers are SDK GatewayApiHandlers now); the 64k switch case
already
  handles DeepSeek context sizing.
- Relocate fetchLiteLlmModelsInfo from the deleted litellm handler into
  core/controller/models/fetchLiteLlmModels.ts (used by
refreshLiteLlmModels).
- Move the `declare module "vscode"` Language Model API augmentation
(previously
  carried by the vscode-lm handler) into
src/types/vscode-language-model.d.ts so
  live consumers (getVsCodeLmModels, vsCodeSelectorUtils) keep their
types.
- Move the @google/genai test mock out of the deleted providers dir to
  src/test/fixtures/google-genai-mock.ts and repoint test-setup.js.

refactor(vscode): delete dead classic system-prompt + slash-command code

The classic system-prompt builder (core/prompts/system-prompt/**), the
deep-planning prompt variants, core/prompts/commands.ts, and the
core/slash-commands handler are no longer reachable from any shipped
entry
point (extension host, standalone host, or webview) after the SDK
migration:
the SDK provides prompt construction (buildClineSystemPrompt) and
slash-command
handling. Removed them and their orphaned tests.

Kept core/prompts/responses.ts (still live) and its tests.

Dead-code reachability was computed with scripts/find-dead-src.mjs
(esbuild
metafile reachability from src/extension.ts +
src/standalone/cline-core.ts,
plus webview src/shared references), then gated on tsc --noEmit +
vitest.

refactor(vscode): delete dead tree-sitter code-definition service

src/services/tree-sitter/** (language parsers + queries for the classic
list_code_definition_names path) is unreachable from any shipped entry
point
after the SDK migration. Verified via scripts/find-dead-src.mjs (esbuild
reachability from extension + standalone entries, plus webview shared
refs) and
an importer cross-check (no live value or type importers), gated on
tsc --noEmit + vitest (436 passing).

refactor(vscode): delete more dead classic code (hooks, permissions,
claude-code, misc)

Removes additional source files unreachable from any shipped entry point
(extension host, standalone host, webview) after the SDK migration,
verified
via esbuild reachability + a per-file importer cross-check that excludes
any
file still referenced by a live non-test survivor or generated/test
glue, then
gated on tsc --noEmit + vitest (436 passing):

- services/ripgrep, integrations/notifications, utils/string,
utils/tabFiltering
- core/assistant-message/parse-assistant-message
- core/hooks: hook-model-context, notification-hook,
precompact-executor,
  PreToolUseHookCancellationError
- core/permissions/CommandPermissionController
- core/workspace/detection
- integrations/claude-code: run, message-filter
- integrations/editor: FileEditProvider, detect-omission
- integrations/misc: extract-file-content, extract-images

Dropped the now-dead re-exports from core/assistant-message/index.ts and
core/permissions/index.ts (those barrels stay; they still export live
types
used by generated host glue).

chore(vscode): add scripts/find-dead-src.mjs dead-code oracle

Computes src files unreachable from the shipped entry points (extension
host + standalone host used by JetBrains/CLI) plus webview shared refs,
using esbuild metafile reachability. Used to drive the
post-SDK-migration dead-code deletions; keep for future pruning. Note:
results still need a tsc-gated importer cross-check because esbuild
drops import-type-only edges.

fix(vscode): drop dead barrel re-exports of deleted files

The previous deletion commit (2c9bd62) removed
core/assistant-message/parse-assistant-message.ts and
core/permissions/CommandPermissionController.ts, but the corresponding
barrel
edits (removing their re-exports from index.ts) were dropped by
lint-staged's
stash and never committed, leaving HEAD referencing deleted modules.
Remove the
dead re-exports so the barrels only export live members.

chore(vscode): drop npm deps only used by deleted provider handlers

Remove dependencies that became unused after the legacy API provider
handlers
and tree-sitter service were deleted (no remaining imports in src,
webview, or
build config):

  @anthropic-ai/vertex-sdk, @aws-sdk/client-bedrock-runtime,
  @aws-sdk/credential-providers, @azure/identity,
  @cerebras/cerebras_cloud_sdk, @google-cloud/vertexai,
@mistralai/mistralai,
  @sap-ai-sdk/ai-api, @sap-ai-sdk/orchestration,
@sap-cloud-sdk/connectivity,
  ollama, tree-sitter-wasms, web-tree-sitter

Also remove the now-dead copyWasmFiles esbuild plugin (it only copied
tree-sitter WASM files for the deleted code-definition service). Kept
openai,
@anthropic-ai/sdk, @google/genai, and aws4fetch — still imported by live
code.

Verified: extension + standalone esbuild builds succeed, tsc --noEmit
clean,
vitest 436 passing.

refactor(vscode): remove dead cost utils and createOpenAIClient

- Delete src/utils/cost.ts (+ test):
calculateApiCostAnthropic/OpenAI/Qwen had
  no consumers left after the provider handlers were removed (only the
test
  referenced them).
- Remove createOpenAIClient from src/shared/net.ts (no remaining
callers) and
  its now-unused openai + EnvUtils imports. The proxy-aware fetch and
  getAxiosSettings exports remain.

tsc --noEmit clean, vitest 436 passing.

refactor(vscode): delete genuinely-unused telemetry/auth helpers

- services/telemetry/events/EventHandlerBase.ts: abstract base for
telemetry
  event handlers whose concrete subclasses were removed with the classic
task
  code; no remaining references (it was the only file left in events/).
- services/auth/AuthServiceMock.ts: test mock with no importers after
the
  associated test was deleted.

Note: the rest of services/* (TelemetryService, McpHub,
FeatureFlagsService,
ErrorService and their IFoo interfaces) IS live — the interfaces only
looked
unreachable to the esbuild oracle because they're consumed via type-only
imports (erased at emit). Verified via importer cross-check; tsc +
vitest green.

refactor(vscode): remove dead sapAiCoreModelDescription const

Unused leftover from the deleted SAP AI Core provider handler (found via
biome noUnusedVariables). tsc + vitest green.

refactor(vscode): delegate Cline recommended-models fetch to the SDK;
TODO others

refreshClineRecommendedModels now delegates the HTTP fetch + response
normalization + offline fallback to the SDK's
fetchClineRecommendedModels
(@cline/core), removing ~80 lines of duplicated logic. The extension
wrapper
keeps its distinct behavior: the CLINE_RECOMMENDED_MODELS_UPSTREAM
feature-flag
gate, the in-memory TTL cache, and in-flight dedup. The proxy-aware
fetch
(@/shared/net) and the configured apiBaseUrl are passed through to
preserve
network/proxy behavior. The SDK's offline fallback list is identical to
CLINE_RECOMMENDED_MODELS_FALLBACK, so offline behavior is unchanged.

Because the module now imports the ESM-only @cline/core, its unit test
moves
from mocha to vitest (joining the other SDK-touching models tests):
added to
vitest include + mocha ignore, and fetchClineRecommendedModels added to
the
vitest @cline/core stub (and the mocha/integration @cline/core mocks for
completeness). Rewrote the test vitest-native, asserting flag-gate,
delegation,
and flag re-check.

Also:
- Add TODO(sdk-consolidation) notes to the remaining bespoke
live-model-refresh
  handlers (Groq, Baseten, Hicap, HuggingFace, Vercel AI Gateway,
OpenRouter)
  documenting the path to share them with the CLI via the SDK (register
  modelsSourceUrl) and then delete the extension-only handlers + RPCs.
These
  are NOT migrated yet because the SDK does not currently live-fetch
those
  providers (only ollama/lmstudio register modelsSourceUrl), so deleting
them
  today would regress to the curated catalog.
- Remove an unused local type (RuleLoadPart) found while reviewing biome
  noUnusedVariables output.

tsc --noEmit clean; vitest 439 passing.

docs(vscode): refine sdk-consolidation TODOs for live model fetching

After inspecting the SDK's generic models-URL fetcher
(sdk/packages/core/src/services/providers/model-source.ts
`fetchModelIdsFromSource` + `resolveModelsSourceUrl`), update the TODOs
on the
bespoke refresh*Models handlers to capture the real constraint
discovered:

- The SDK fetcher exists and is provider-agnostic, but returns model
*ids only*
  (unknown ids get placeholder ModelInfo with no real
pricing/capabilities).
- `mergeKnownModels` treats a registered `modelsSourceUrl` as the
authoritative
  "installed" list (Ollama/LM Studio semantics) and DISCARDS the curated
catalog
  when the live fetch returns results.

So simply registering `modelsSourceUrl` for
Groq/Baseten/Hicap/HuggingFace/
Vercel/OpenRouter would regress rich model metadata. Proper
consolidation needs
an SDK enhancement first (merge-mode or richer per-provider parsing),
then the
extension handlers + RPCs can be deleted. refreshGroqModels.ts carries
the
detailed note; the others reference it.

refactor(vscode): unify ClineMessage id minting behind one
MessageIdMinter

Message ids (ClineMessage.ts) were minted from Date.now() in two
independent
generators: the live message translator (pure ++counter seeded once) and
the
interaction coordinator (Math.max(Date.now(), last+1)). Because the
translator
counter drifts behind wall-clock, it can later catch up to a clock-based
id
minted by the interaction coordinator, producing colliding ids for
different
messages. That breaks any merge-by-id scheme on the webview side.

Introduce a single process-wide MessageIdMinter (pure monotonic
id/seq/epoch
counters, never reads the clock) owned by MessageTranslatorState and
shared by:
- live SDK event translation,
- the interaction coordinator (tool approval / ask_question /
user_feedback),
- history rendering (sdkMessagesToClineMessages).

This makes every id globally unique within the process, so regenerated
history
ids never overlap live-session ids. Behavior is otherwise unchanged.

Also adds the message-state pipeline design + investigation docs under
src/sdk/docs.

S1 of the message-state redesign; seq/epoch stamping and the remaining
one-off
Date.now() mint sites follow in S2.

refactor(vscode): stamp seq/epoch on messages and state; fire-and-forget
delivery

S2 of the message-state redesign. Lays the groundwork for the
convergent-replica
webview reducer (S3) so the webview can never get stuck on
stale/out-of-order
delivery.

Stamping (extension-owned, synchronous, from the single
MessageIdMinter):
- Every ClineMessage flowing to the webview is stamped in
SdkMessageCoordinator with
  a fresh `seq` (freshness) and the current `epoch`
(conversation/replica fence),
  before it is stored or emitted. The same object references go to both
the message
  state handler and the partial-message stream, so both channels carry
identical
  stamps. An updated message (partial -> final, same ts) passes through
again and
  gets a higher seq, so the freshest copy always wins regardless of
arrival order.
- Every state snapshot is stamped in
SdkController.getStateToPostToWebview with a
  fresh `stateVersion` (sampled from the same counter) and the current
`epoch`.
- `epoch` is bumped at every conversation boundary via a new
  resetMessageTranslatorAndFence() wired into the existing
resetMessageTranslator
  sites (task start/clear, history open, reinit, mode rebuild,
new-session
  follow-up). iteration_start streaming resets do NOT bump it.

Transport:
- ClineMessage proto gains seq/epoch (fields 24/25) and the conversions
carry them.
- ExtensionState gains stateVersion/epoch (ride inside state_json, no
proto change).

Fire-and-forget delivery:
- sendPartialMessageEvent and sendStateUpdate no longer await
postMessage to the
  webview. A hidden/reloaded/closed webview can make postMessage hang or
resolve
  false; awaiting it could stall the backend turn loop. Correctness no
longer
  depends on any single delivery — the webview will be a convergent
replica (S3).

All new fields are optional/default-0 so the classic/legacy path is
unaffected.
Cancel's epoch bump + fence-before-abort and the remaining one-off
Date.now() mint
sites are handled in later steps (S6/S2-tail). 423 unit tests pass; tsc
clean.

fix(vscode): converge webview transcript via a pure reducer (fixes
last-message-missing)

S3 of the message-state redesign. The webview received the same
conversation over
two unordered, fire-and-forget channels — incremental partial messages
and full
state snapshots — and the state handler REPLACED clineMessages wholesale
("// HACK:
Preserve clineMessages if currentTaskItem is the same"). A stale
snapshot captured
before the last message landed could clobber the transcript, dropping
the final
message and leaving the UI stuck on "Thinking…" (reproduced earlier on
camera).

Introduce a pure convergent-replica reducer (messageReducer.ts) keyed on
the three
extension-stamped quantities from S2:
  - ts    : identity / merge key
  - seq   : freshness (higher seq wins for the same ts)
  - epoch : conversation/replica fence (newer replaces, older is
dropped, equal merges)

applyMessage / applyStateSnapshot are total and side-effect free:
  - older epoch  -> drop (straggler from a previous task/render)
  - newer epoch  -> replace the transcript wholesale (new task / history
load)
  - same epoch   -> merge by ts keeping the higher seq; a snapshot may
ADD/UPDATE
                    rows but NEVER truncate, so it can't drop a message
the partial
                    stream already delivered. Stale (older stateVersion)
snapshots are
                    ignored wholesale.

ExtensionStateContext now feeds both subscription callbacks through the
reducer via a
replicaRef, replacing the wholesale-replace HACK and the findLastIndex
append. Classic/
legacy state is unstamped (epoch 0 / version 0) and merges by ts exactly
as before.

Tests (messageReducer.test.ts): deterministic cases mapping 1:1 to the
bugs (stale
snapshot must not shrink the transcript; lower-seq ignored;
partial->final in place;
older-epoch straggler dropped; older-version snapshot ignored), PLUS an
order-independence proof — all 120 permutations of a causal log, with
duplication and
with non-final drops, converge to the same canonical state. This is the
high-confidence
guarantee that the webview cannot get stuck regardless of delivery
timing.

62 webview chat tests pass; tsc and biome clean.

feat(vscode): add authoritative TurnState (backend-owned UI mode)

S4 of the message-state redesign. Introduces the single source of truth
for the
webview's UI mode so it no longer has to be inferred from the tail of
clineMessages
(the root of RC1: missing/stuck "Thinking", vanishing approval buttons,
footer and
buttons disagreeing).

- Add TurnPhase / TurnState to shared types and ExtensionState.turnState
(rides inside
  state_json; no proto change).
- Add TurnStateTracker, owned by SdkController, sharing the one
id/seq/epoch authority.
  Each transition stamps a fresh seq so the webview keeps only the
newest TurnState.
- Set the phase at the exact lifecycle points where the backend knows
it:
    streaming         — initTask / reinit / user responded
(resolvePending*)
    awaiting_approval — handleRequestToolApproval (anchored on the ask)
    awaiting_followup — handleAskQuestion; and a turn that ends WITHOUT
attempt_completion
    completed         — turn ends and attempt_completion was used
    error             — onSendError; mistake_limit
    resumable         — cancelTask (set before abort)
    idle              — clearTask
  The completed-vs-awaiting_followup decision uses
  MessageTranslatorState.wasAttemptCompletionSeen().
- getStateToPostToWebview now ships turnState in every snapshot.

This is ADDITIVE: the webview does not read turnState yet (S5 wires
footer/buttons to
it and deletes the tail heuristics). Classic/legacy paths leave
turnState undefined and
keep the legacy behavior. 427 SDK unit tests pass (incl. new
turn-state-tracker tests);
tsc and biome clean.

fix(vscode): drive webview footer + buttons from TurnState (fixes RC1)

S5 of the message-state redesign. The webview decided "thinking vs
approving vs done"
and which buttons to show by inspecting the TAIL of clineMessages.
Because the backend
appends bookkeeping (api_req_started usage) after content and even after
approval asks,
the tail routinely meant the wrong thing — producing stuck "Thinking",
vanishing
Approve/Reject, and the footer disagreeing with the buttons (RC1,
reproduced on camera).

Now the webview reads the authoritative backend-owned TurnState (added
in S4):

- buttonConfig: add buttonsForPhase(turnState, anchoredMessage) and the
dispatcher
  getButtonConfigFromState(messages, turnState, mode). The button SET is
chosen by phase;
  approval labels (Approve vs Save, Run Command, MCP, subagents) come
from the anchored
  message (turnState.anchorTs). When turnState is absent (classic/older
state) it falls
  back to the legacy tail-walking getButtonConfigForMessages.
- ActionButtons reads turnState from useExtensionState and uses
getButtonConfigFromState.
- MessagesArea.isWaitingForResponse short-circuits to `phase ===
"streaming"` when
  turnState is present (and only shows the footer loader until a content
row is actually
  streaming); the legacy tail inference is kept as the fallback.

Button actions (approve/reject/proceed/new_task) already send a fixed
responseType
independent of clineAsk, and the SDK backend resolves the pending
approval/followup
promise — so routing is correct under TurnState. Classic fallback paths
are untouched.

New unit tests cover buttonsForPhase (every phase + anchored-label
selection +
mistake_limit-vs-api_req_failed) and getButtonConfigFromState (prefers
TurnState over a
trailing bookkeeping tail; legacy fallback). 68 webview chat tests pass;
tsc + biome clean.

fix(vscode): cancel raises the epoch fence before aborting (no
post-cancel stragglers)

S6 of the message-state redesign. sdkHost.abort() is cooperative — the
SDK can emit a
few more events after it. Previously cancelTask aborted first and only
the (post-abort)
!isRunning filter stripped two trailing ask types, letting say:*
stragglers land after
the resume_task ask and wedge the UI.

Now cancel raises the fence SYNCHRONOUSLY before the abort:
- SdkController.cancelTask sets turnState.phase = "resumable" (already
in S4), then
- SdkTaskControlCoordinator.cancelTask calls raiseCancelFence() (epoch
bump) BEFORE
  awaiting sdkHost.abort().

Any event the SDK emits after the abort request therefore carries the
OLD epoch and is
dropped by the webview's convergent reducer; the authoritative phase is
"resumable"
(Resume Task), independent of the message tail. Order matters and is
covered by a unit
test (fence before abort).

Usage accounting is exempt from the fence — it was never gated by the
message filter, so
a post-cancel usage event still bills the tokens the provider actually
generated.

The legacy !isRunning ask-only filter is now redundant but kept as
defense-in-depth.
428 SDK unit tests pass; tsc + biome clean.

refactor(vscode): translator hygiene — suppress ask_question row, drop
done synthetic ask

S7 (a + c) of the message-state redesign — removes two order-dependent
hacks now that
TurnState is authoritative.

(a) ask_question / ask_followup_question are suppressed from the generic
say:"tool"
    renderer (both content_start and content_end). The
SdkInteractionCoordinator services
    these and emits the proper ask:"followup"; the generic tool row was
an orphan partial
    that never finalized and defeated the tail heuristics. The CLI
already does this.

(c) The `done` handler no longer synthesizes a trailing
ask:"completion_result" — that
    was the "must be last message" hack (ENG-1887) that existed only
because the webview
    inferred UI mode from the array tail. `done` now emits no transcript
message and only
    signals turnComplete; the webview reads phase from TurnState
(completed when
    attempt_completion was used, else awaiting_followup). The green
"Task Completed" box
    still comes from the say:"completion_result" emitted at
attempt_completion content_end.

Translator unit tests updated to the new contract (done → 0 messages,
turnComplete=true).

Deferred to a follow-up (lower risk if left): (b) collapsing the
approval ask onto the
streaming tool's id (needs cross-coordinator id threading), and (d) the
mistake_limit
forced abort (now harmless since phase is authoritative). The
persisted-history renderer
still appends its own trailing ask so reopened tasks show the resume
affordance —
intentional.

428 SDK unit tests pass; tsc + biome clean.

fix(vscode): fix stuck/missing footer buttons from TurnState regressions

Webview now gates turnState by seq (a stale snapshot can no longer
revert
streaming->idle), never empties a live transcript on a lone newer-epoch
partial,
and routes a follow-up after completed/awaiting_followup to askResponse
instead
of starting a new task.

Backend phase emission fixes surfaced by live testing:
- post state on turn end even when the done event carries no messages
- askResponse sets phase=streaming (resume/continue shows Cancel,
unblocks send)
- a post-cancel turn-complete straggler no longer clobbers resumable

Adds reducer, send-routing, and session-event-coordinator unit tests.
See
src/sdk/docs/webview-message-state-design.md §11 for the full debugging
log.

improve openai compatible provider settings

fix(vscode): show Start New Task after multi-iteration completion turns

The completion signal (attemptCompletionSeen) was scoped to the message
translator's per-iteration reset(), so a turn that called the completion
tool
and then ran another iteration before 'done' lost the signal — the turn
ended
as awaiting_followup instead of completed and the footer showed no
'Start New Task' button despite the green Task Completed box.

Make the completion signal turn-scoped: reset() only clears streaming
pointers;
a new clearTurnOutcome() clears it at genuine turn/task boundaries
(initTask,
reinitExistingTaskFromId, askResponse). Also recognize the SDK's
built-in
submit_and_exit completion tool (summary field) alongside the VSCode
attempt_completion tool (result field).

Removes the webview-message-state design/findings docs and rewrites the
related comments to describe the system as-is.

improve OCA provider

- oca provider login now works

fix history view bugs

- deleting entries works
- favoriting works

fix open task conversation file

fix(vscode): include ambient vscode LM type decls in test tsconfig

The nightly CI "Build Tests and Extension" step (npm run ci:build)
failed
in compile-tests (tsc -p tsconfig.test.json) with:
  getVsCodeLmModels.ts: Property 'lm' does not exist on type 'typeof
import("vscode")'
  state-keys.ts: Module '"vscode"' has no exported member
'LanguageModelChatSelector'

@types/vscode is pinned to 1.84.0, which predates the Language Model
API.
The repo compensates with an ambient augmentation in
src/types/vscode-language-model.d.ts, which the main tsconfig picks up
via
its 'src/**/*' include. tsconfig.test.json overrides include to only
'src/**/*.test.ts'; listing src/types under typeRoots does not load a
loose
.d.ts (typeRoots only auto-loads @types-style package folders), so the
augmentation was missing from the test program.

Add 'src/types/**/*.d.ts' to tsconfig.test.json include so the
augmentation
is part of the test compilation.

change zai provider to user providers.json instead of statemanager

fix litellm provider

fix(vscode): load ambient vscode LM type decls in unit-test ts-node
program

The nightly CI 'Unit Tests with coverage - Linux' step (npm run
test:unit) failed in ts-node compilation:

  state-keys.ts: Module 'vscode' has no exported member
'LanguageModelChatSelector'

This surfaced as a misleading 'Cannot find package @shared/...'
ERR_MODULE_NOT_FOUND: Mocha tried require() first (which threw the
ts-node TSError), then fell back to import(), whose ESM resolver cannot
resolve the @shared/* path alias.

tsconfig.unit-test.json is driven by ts-node, which defaults to
files:false and compiles modules on demand from their imports. The loose
ambient augmentation in src/types/vscode-language-model.d.ts was
therefore never loaded, so state-keys.ts failed to compile.

Set ts-node.files=true and add src/types/**/*.d.ts to include so the
augmentation is part of the unit-test program. Mirrors the earlier
tsconfig.test.json fix for the separate build toolchain.

fix(vscode): widen ZAiProvider getEventValue to accept VSCodeDropdown
event union

The nightly 'vscode test' job (npm run ci:build -> build:webview -> tsc
-b) failed on both Linux and windows-latest:

  ZAiProvider.tsx: Argument of type 'Event | FormEvent<HTMLElement>' is
not assignable to parameter of type 'Event'

VSCodeDropdown's onChange supplies 'Event |
React.FormEvent<HTMLElement>', but the getEventValue helper (added when
the zai provider was moved to providers.json) was typed to accept only
Event. The helper only reads target.value, which exists on both, so
widen the parameter to the same union the dropdown provides.

Note: this only surfaces under 'tsc -b' / 'tsc --noEmit -p
tsconfig.app.json'; the webview's root tsconfig.json is a solution file
with files:[] so a bare 'tsc --noEmit' checks nothing.

The separate 'test / test' job failure is an aggregate gate that fails
because vscode-test failed; fixing this resolves it too.
2026-06-05 09:58:54 -07:00
Dominic Cooney a750efc17e sdk migration: squashed pre-2026-05-27 work
Squashed foundational SDK-migration work older than one week (author dates
up to 2026-05-26), combining the previous "squashed pre-2026-05-22 work"
base commit with subsequent older commits:

- sdk migration base (pre-2026-05-22 squash)
- fix(mcp): accept CLI-authored nested transport format, preserve oauth/metadata, improve schema error messages
- add telemetry to sdk extension
- improve task startup perf
- harden perf improvements
- remove timing code
- chore: fix lint and format on the vscode app
- fix: declare missing direct dependencies in apps/vscode
- fix integration tests
- fix(test): stub telemetry helpers in unit-test @cline/core mock
- ci: run publish-nightly job inside apps/vscode workspace
- remove old md files
- remove outdated samples
- step one for removing src/core/api folder
2026-06-05 09:52:20 -07:00
964 changed files with 52999 additions and 134646 deletions
+128
View File
@@ -0,0 +1,128 @@
# Debug Harness
HTTP-controlled debugger for the VSCode extension at `src/dev/debug-harness/server.ts`.
## Quick start
```bash
# Build extension first if needed (protos + esbuild):
npm run protos && IS_DEV=true node esbuild.mjs
# Launch (skip-build if already built):
npx tsx src/dev/debug-harness/server.ts --skip-build --auto-launch
# In another terminal:
curl localhost:19229/api -d '{"method":"status"}'
```
## Data Isolation
The debugee runs with `CLINE_DIR=~/.cline2` by default, separate from your real `~/.cline`.
This prevents the debugee's logout from logging out the debugger, and vice versa.
Override with `--cline-dir /tmp/test-dir`. Check with `status()``clineDir`.
## Browser Capture & OAuth
The debugee runs with `CLINE_CAPTURE_BROWSER=1`, which intercepts `openExternal()` in
`src/utils/env.ts`. URLs are captured instead of opening a real browser:
- Logged to `$CLINE_DIR/data/debug-captured-urls.jsonl`
- POSTed in real-time to `/captured-url` on the harness server
- Queryable via `oauth.captured_urls`
### OAuth API
- **`oauth.captured_urls`** `{clear?}` — URLs the debugee tried to open
- **`oauth.read_stored_token`** — Check auth token presence in secrets.json
- **`oauth.simulate_callback`** `{path, code?, state?, provider?, token?}` — Build vscode:// callback URI
- **`oauth.read_captured_urls_file`** — Read on-disk JSONL of captured URLs
### OAuth testing flow
For **Cline OAuth** (SDK local callback): The SDK starts a local HTTP server, the auth URL
is captured. To complete: open the captured URL in a real browser (it redirects back to the
SDK's callback server), OR extract the callback port and `curl http://127.0.0.1:PORT/callback?code=...`.
For **MCP/Provider OAuth** (vscode:// URI): The redirect goes to a vscode:// URI.
`oauth.simulate_callback` only *builds* the URI — it does not deliver it, and the ESM
extension host can't `require()` the handler. To actually deliver the callback, call the
debug-only hook via `ext.evaluate` (with `awaitPromise: true`):
`globalThis.__clineHandleUri("vscode://saoudrizwan.claude-dev/...?code=...&state=...")`.
It runs the same `SharedUriHandler.handleUri` as VSCode's real URI handler and exists only
when `CLINE_CAPTURE_BROWSER` is set (the harness always sets it; never ships in prod).
For end-to-end MCP OAuth, get a real `code` from the local MCP OAuth test server
(`npm run dev:mcp-oauth-test-server`).
## Navigating Views — Use Commands, Not Clicks
Don't try to find/click small sidebar icons. Use VSCode commands via command palette.
Registered in `src/registry.ts`:
| Command | View |
|---------|------|
| `cline.accountButtonClicked` | Account / sign-in |
| `cline.historyButtonClicked` | Task history |
| `cline.settingsButtonClicked` | Settings |
| `cline.mcpButtonClicked` | MCP servers |
| `cline.plusButtonClicked` | New task (chat) |
| `cline.worktreesButtonClicked` | Worktrees |
```bash
curl localhost:19229/api -d '{"method":"ui.command_palette","params":{"command":"cline.accountButtonClicked"}}'
```
## Key commands
All via `POST localhost:19229/api` with `{"method":"...", "params":{...}}`:
- **`launch`** / **`shutdown`** — lifecycle
- **`ui.screenshot`** — screenshot to `/tmp/cline-debug/`; returns `{path}`**use `read_file` on the path to examine, do NOT `open` the file** (Preview.app covers the VSCode window)
- **`ui.open_sidebar`** — open the Cline sidebar
- **`ext.set_breakpoint`** `{file, line, condition?}` — breakpoint by source file (sourcemap-resolved)
- **`ext.evaluate`** `{expression, callFrameId?}` — eval in extension host
- **`ext.resume`** / **`ext.step_over`** / **`ext.step_into`** — stepping
- **`ext.call_stack`** — inspect when paused
- **`web.evaluate`** `{expression}` — eval in webview
- **`web.post_message`** `{message}` — send postMessage to extension host via exposed vsCodeApi
- **`wait_for_pause`** `{timeout?}` — block until breakpoint hit
- **`ui.locator`** `{role?, testId?, text?, frame?}` — Playwright locator (auto-retries on stale sidebar frame)
- **`ui.react_input`** `{text, selector?, clear?, submit?}` — set React textarea value via `execCommand('insertText')`; works reliably across multiple tasks
- **`ui.send_message`** `{text, images?, files?, responseType?}` — send chat message bypassing the textarea entirely (via gRPC postMessage)
- **`ui.command_palette`** `{command}` — run VSCode command
## Typical Session
```bash
# 1. Launch
curl localhost:19229/api -d '{"method":"launch","params":{"skipBuild":true}}'
# 2. Open sidebar + dismiss overlays (ALWAYS do this first)
curl localhost:19229/api -d '{"method":"ui.open_sidebar"}'
curl localhost:19229/api -d '{"method":"web.evaluate","params":{"expression":"document.querySelectorAll(\".sr-only\").forEach(el => el.parentElement?.click())"}}'
# 3. Navigate to view
curl localhost:19229/api -d '{"method":"ui.command_palette","params":{"command":"cline.accountButtonClicked"}}'
# 4. Check captured OAuth URLs if testing auth
curl localhost:19229/api -d '{"method":"oauth.captured_urls"}'
# 5. Verify
curl localhost:19229/api -d '{"method":"ui.screenshot"}'
```
## Caveats
- **⚠️ Dismiss promotional overlays FIRST**: On fresh launches, full-screen promo overlays block the sidebar. **Dismiss immediately after `ui.open_sidebar`**, before any other interaction or screenshot. May need to run twice:
```bash
curl localhost:19229/api -d '{"method": "ui.open_sidebar"}'
curl localhost:19229/api -d '{"method": "web.evaluate", "params": {"expression": "document.querySelectorAll(\".sr-only\").forEach(el => el.parentElement?.click())"}}'
```
- **Screenshots — don't open the file**: `ui.screenshot` and `ui.sidebar_screenshot` save PNGs to `/tmp/cline-debug/` and return the `{path}`. Use `read_file` on that path to examine screenshots. Running `open <path>` launches Preview.app on macOS which covers the VSCode window.
- **Scripts count = 0 after launch**: CDP connects after extension host starts, so scripts parsed during startup aren't tracked. Breakpoints still work via sourcemap resolution.
- **Port 9230**: Extension host inspector. If another VSCode instance uses this port, the harness will fail to connect. Kill other debug instances first.
- **macOS only** for now (Playwright Electron launch behavior).
- **Webview CDP**: `connect_webview` may fail depending on Electron version. `web.evaluate` still works via Playwright's `frame.evaluate()` fallback.
- **Sourcemap paths**: esbuild outputs relative paths like `../src/extension.ts` in the sourcemap. The resolver handles this, but if a file isn't found, use `ext.source_files` to see exact paths.
- **OAuth with fake codes**: Browser capture intercepts the URL but doesn't provide a valid auth code. For real OAuth testing, open the captured URL in a browser. For unit testing, mock the token exchange.
See `src/dev/debug-harness/README.md` for full API reference.
+89 -87
View File
@@ -13,11 +13,55 @@ This file is the secret sauce for working effectively in this codebase. It captu
**What NOT to add:** Stuff you can figure out from reading a few files, obvious patterns, or standard practices. This file should be high-signal, not comprehensive.
## Miscellaneous
- Avoid provider-specific string matching / hardcoded provider branches when fixing provider/config plumbing. Prefer provider metadata, shared catalog/defaults, explicit protocol/client capabilities, or centralized normalization utilities that apply by data shape rather than `providerId === "..."`. If a provider exception seems necessary, stop and explain why instead of adding ad-hoc string matching.
- This is a VS Code extension—check `package.json` for available scripts before trying to verify builds (e.g., `npm run compile`, not `npm run build`).
- When creating PRs, contributors should not create changelog-entry files. Maintainers handle release versioning and changelog curation during the release process.
- When adding new feature flags, see this PR as a reference https://github.com/cline/cline/pull/7566
- Additional instructions about making requests: @.clinerules/network.md
## Searching the Codebase — Avoiding Build Output
Several directories contain build output or generated code that produces
noisy or unusable results with `search_files` / `grep`:
| Directory | What it is | Why it's a problem |
|-----------|-----------|-------------------|
| `out/` | esbuild bundle output | Mirrors `src/` structure as minified JS — every search gets duplicate hits on single-line files |
| `dist/` | Packaged extension | Entire extension bundled into one minified `extension.js` (~1 long line) |
| `dist-standalone/` | Standalone build output | Same minification issue |
| `src/generated/` | Generated protobuf code | Auto-generated from `proto/`; not the source of truth |
| `src/shared/proto/` | Generated proto type defs | Auto-generated from `proto/`; not the source of truth |
| `node_modules/` | Dependencies | Huge, not project source |
### How to skip build output
**`search_files`** — Point at `src/` (not the project root) and use `file_pattern`:
```
search_files(path="src/core", regex="myFunction", file_pattern="*.ts")
```
The `file_pattern` parameter is the most effective filter — e.g. `"*.ts"`,
`"*.tsx"`, `"*.proto"`.
**`grep` directly** — Exclude build dirs and restrict to source extensions:
```bash
grep -rn "myFunction" src/ --include="*.ts" --exclude-dir={out,dist,node_modules,generated}
```
### When you must search minified files
Sometimes you need to verify what got bundled (e.g., checking if a change
made it into the build). Minified files are typically one long line, so
normal `grep` shows the entire file as context. Use these approaches:
- **`grep -oP`** to extract just the match with limited surrounding context:
```bash
grep -oP '.{0,40}myFunction.{0,40}' dist/extension.js
```
- **`read_file`** on files in `out/src/` — these have source maps and are
more readable than `dist/extension.js` (which is the fully bundled output).
- **Source maps** — `out/src/*.js.map` and `dist/extension.js.map` can be
used to trace minified output back to original source locations.
## gRPC/Protobuf Communication
The extension and webview communicate via gRPC-like protocol over VS Code message passing.
@@ -48,93 +92,6 @@ The extension and webview communicate via gRPC-like protocol over VS Code messag
- `src/core/controller/task/explainChanges.ts` - Handler implementation
- `webview-ui/src/components/chat/ChatRow.tsx` - UI rendering
## Adding a New API Provider
When adding a new provider (e.g., "openai-codex"), you must update the proto conversion layer in THREE places or the provider will silently reset to Anthropic:
1. `proto/cline/models.proto` - Add to the `ApiProvider` enum (e.g., `OPENAI_CODEX = 40;`)
2. `convertApiProviderToProto()` in `src/shared/proto-conversions/models/api-configuration-conversion.ts` - Add case mapping string to proto enum
3. `convertProtoToApiProvider()` in the same file - Add case mapping proto enum back to string
**Why this matters:** Without these, the provider string hits the `default` case and returns `ANTHROPIC`. The webview, provider list, and handler all work fine, but the state silently resets when it round-trips through proto serialization. No error is thrown.
**Other files to update when adding a provider:**
- `src/shared/api.ts` - Add to `ApiProvider` union type, define models
- `src/shared/providers/providers.json` - Add to provider list for dropdown
- `src/core/api/index.ts` - Register handler in `createHandlerForProvider()`
- `webview-ui/src/components/settings/utils/providerUtils.ts` - Add cases in `getModelsForProvider()` and `normalizeApiConfiguration()`
- `webview-ui/src/utils/validate.ts` - Add validation case
- `webview-ui/src/components/settings/ApiOptions.tsx` - Render provider component
## Responses API Providers (OpenAI Codex, OpenAI Native)
Providers using OpenAI's Responses API require native tool calling. XML tools don't work with the Responses API.
**Symptoms of broken native tool calling:**
- Tools get called multiple times (e.g., `ask_followup_question` asks the same question twice)
- Tool arguments get duplicated or malformed
- The model responds but tools aren't recognized
**Root causes to check:**
1. **Provider missing from `isNextGenModelProvider()`** in `src/utils/model-utils.ts`. The native variant matchers (e.g., `native-gpt-5/config.ts`) call this function. If your provider isn't in the list, the matcher returns false and falls back to XML tools.
2. **Model missing `apiFormat: ApiFormat.OPENAI_RESPONSES`** in its model info (`src/shared/api.ts`). This property signals that the model requires native tool calling. The task runner in `src/core/task/index.ts` checks this and forces `enableNativeToolCalls: true` regardless of user settings.
**When adding a new Responses API provider:**
1. Add provider to `isNextGenModelProvider()` list in `src/utils/model-utils.ts`
2. Set `apiFormat: ApiFormat.OPENAI_RESPONSES` on all models that use the Responses API
3. The variant matcher and task runner will handle the rest automatically
## Adding Tools to System Prompt
This is tricky—multiple prompt variants and configs. **Always search for existing similar tools first and follow their pattern.** Look at the full chain from prompt definition → variant configs → handler → UI before implementing.
1. **Add to `ClineDefaultTool` enum** in `src/shared/tools.ts`
2. **Tool definition** in `src/core/prompts/system-prompt/tools/` (create file like `generate_explanation.ts`)
- Define variants for each `ModelFamily` (generic, next-gen, xs, etc.)
- Export variants array (e.g., `export const my_tool_variants = [GENERIC, NATIVE_NEXT_GEN, XS]`)
- **Fallback behavior**: If a variant isn't defined for a model family, `ClineToolSet.getToolByNameWithFallback()` automatically falls back to GENERIC. So you only need to export `[GENERIC]` unless the tool needs model-specific behavior.
3. **Register in `src/core/prompts/system-prompt/tools/init.ts`** - Import and spread into `allToolVariants`
4. **Add to variant configs** - Each model family has its own config in `src/core/prompts/system-prompt/variants/*/config.ts`. Add your tool's enum to the `.tools()` list:
- `generic/config.ts`, `next-gen/config.ts`, `gpt-5/config.ts`, `native-gpt-5/config.ts`, `native-gpt-5-1/config.ts`, `native-next-gen/config.ts`, `gemini-3/config.ts`, `glm/config.ts`, `hermes/config.ts`, `xs/config.ts`
- **Important**: If you add to a variant's config, make sure the tool spec exports a variant for that ModelFamily (or relies on GENERIC fallback)
5. **Create handler** in `src/core/task/tools/handlers/`
6. **Wire up in `ToolExecutor.ts`** if needed for execution flow
7. **Add to tool parsing** in `src/core/assistant-message/index.ts` if needed
8. **If tool has UI feedback**: add `ClineSay` enum in proto, update `src/shared/ExtensionMessage.ts`, update `src/shared/proto-conversions/cline-message.ts`, update `webview-ui/src/components/chat/ChatRow.tsx`
## Modifying System Prompt
**Read these first:** `src/core/prompts/system-prompt/README.md`, `tools/README.md`, `__tests__/README.md`
System prompt is modular: **components** (reusable sections) + **variants** (model-specific configs) + **templates** (with `{{PLACEHOLDER}}` resolution).
**Key directories:**
- `components/` - Shared sections: `rules.ts`, `capabilities.ts`, `editing_files.ts`, etc.
- `variants/` - Model-specific: `generic/`, `next-gen/`, `xs/`, `gpt-5/`, `gemini-3/`, `hermes/`, `glm/`, etc.
- `templates/` - Template engine and placeholder definitions
**Variant tiers (ask user which to modify):**
- **Next-gen** (Claude 4, GPT-5, Gemini 2.5): `next-gen/`, `native-next-gen/`, `native-gpt-5/`, `native-gpt-5-1/`, `gemini-3/`, `gpt-5/`
- **Standard** (default fallback): `generic/`
- **Local/small models**: `xs/`, `hermes/`, `glm/`
**How overrides work:** Variants can override components via `componentOverrides` in their `config.ts`, or provide a custom template in `template.ts` (e.g., `next-gen/template.ts` exports `rules_template`). If no override, the shared component from `components/` is used.
**Example: Adding a rule to RULES section**
1. Check if variant overrides rules: look for `rules_template` in `variants/*/template.ts` or `componentOverrides.RULES` in `config.ts`
2. If shared: modify `components/rules.ts`
3. If overridden: modify that variant's template
4. XS variant is special—has heavily condensed inline content in `template.ts`
**After any changes, regenerate snapshots:**
```bash
UPDATE_SNAPSHOTS=true npm run test:unit
```
Snapshots live in `__tests__/__snapshots__/`. Tests validate across model families and context variations (browser, MCP, focus chain).
## Modifying Default Slash Commands
Three places need updates:
- `src/core/slash-commands/index.ts` - Command definitions
- `src/core/prompts/commands.ts` - System prompt integration
- `webview-ui/src/utils/slash-commands.ts` - Webview autocomplete
## Adding New Global State Keys
Adding a new key to global state requires updates in multiple places. Missing any step causes silent failures.
@@ -199,3 +156,48 @@ const isGenerating = explanationInfo.status === "generating" && !wasCancelled
**See also:** `BrowserSessionRow.tsx` uses similar pattern with `isLastApiReqInterrupted` and `isLastMessageResume`.
**Backend side:** When streaming is cancelled, clean up properly (close tabs, clear comments, etc.) by checking `taskState.abort` after the streaming function returns.
## Debug Harness: clear inherited VSCode/Electron env vars before launching
The debug harness (`apps/vscode/src/dev/debug-harness/server.ts`) launches a child
VSCode via Playwright's `_electron.launch({ env: { ...process.env, ... } })`. If you
run the harness from a process that was itself spawned by VSCode (e.g. the Cline
extension host, an integrated terminal, or an agent running inside VSCode), the
parent's VSCode/Electron env vars leak into the child and break the launch.
The fatal one is **`ELECTRON_RUN_AS_NODE=1`**: it makes the child VSCode binary run
as plain Node, so it rejects every VSCode CLI flag. Symptom:
```
.../Visual Studio Code.app/Contents/MacOS/Code: bad option: --extensionDevelopmentPath=...
Error: Process failed to launch! (Playwright _electron.launch)
```
This is NOT the macOS Playwright flakiness mentioned in the harness README — it's
env inheritance. Fix: strip the inherited vars before starting the harness:
```bash
env -u ELECTRON_RUN_AS_NODE -u ELECTRON_NO_ATTACH_CONSOLE \
-u VSCODE_CLI -u VSCODE_CODE_CACHE_PATH -u VSCODE_CRASH_REPORTER_PROCESS_TYPE \
-u VSCODE_CWD -u VSCODE_ESM_ENTRYPOINT -u VSCODE_HANDLES_UNCAUGHT_ERRORS \
-u VSCODE_IPC_HOOK -u VSCODE_NLS_CONFIG -u VSCODE_PID -u VSCODE_L10N_BUNDLE_LOCATION \
npx tsx src/dev/debug-harness/server.ts --auto-launch --skip-build
```
Check your own env with `env | grep -iE 'electron|vscode_'` first; `ELECTRON_RUN_AS_NODE=1`
present means you must scrub before launching.
Other harness notes confirmed in practice:
- The extension host is **ESM** (`VSCODE_ESM_ENTRYPOINT`), so `ext.evaluate` has no
`require` and module-internal functions aren't reachable as globals. To inspect
internal builders (e.g. `buildBedrockProviderConfig`), set a breakpoint with
`ext.set_breakpoint` and read locals via `ext.evaluate` with the paused `callFrameId`
— don't try to `require()` the bundle.
- `web.evaluate` wraps the expression as a single returned expression; multi-statement
snippets must be an IIFE `(() => { ...; return x; })()`, otherwise you get
`SyntaxError: Unexpected token ';'`.
- Webview settings inputs are `vscode-text-field` web components with debounced React
onChange. Setting `.value` + dispatching events via `web.evaluate` is unreliable for
some fields; focus the inner shadow `input` then use real keystrokes (`ui.type` +
`ui.press Tab`, or click the dropdown option) to make the value persist.
+26
View File
@@ -0,0 +1,26 @@
# SDK Adapter
The VSCode extension runs on the Cline SDK (`@cline/core`, `@cline/llms`,
`@cline/shared`) through an adapter layer in `apps/vscode/src/sdk/`. The
webview still talks gRPC; the adapter translates between gRPC handlers and SDK
calls. See `apps/vscode/src/dev/debug-harness/README.md` for the debug harness.
## Conventions
1. **Look up SDK APIs, don't guess.** Use `kb_search(name="sdk", query="...")`
before implementing against an SDK surface.
2. **Reference the pre-SDK implementation when replacing a module.** Add a
`// Replaces classic src/core/... (see origin/main)` header and use
`kb_search(name="cline", commit="origin/main")` or
`git show origin/main:path` to consult the prior implementation.
3. **Single entry point.** There is one codepath — the SDK adapter. No
`CLINE_SDK` env flag.
4. **Use `{appBaseUrl}`**, never hardcode `app.cline.bot`.
5. **Avoid `as` casts.** Use explicit conversion functions with tests. The
branded types in `apps/vscode/src/sdk/model-catalog/contracts.ts` exist so
casts are unnecessary outside parse/compute boundaries.
## Debug harness
- **Dismiss the Kanban/promo overlay** before any debug harness interaction.
- **Use the command palette** to navigate tabs in the debug harness.
@@ -31,6 +31,9 @@ jobs:
if: github.repository == 'cline/cline' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dpc/sdk-migration-simpler-login')
runs-on: ubuntu-latest
environment: PublishNightly
# The VS Code extension's package.json and lockfiles live under apps/vscode/
# (the repo root has no package.json). Mirror ext-vscode-test.yml so install
# and publish steps run in the correct workspace.
defaults:
run:
working-directory: apps/vscode
@@ -56,6 +59,10 @@ jobs:
# Keep publish environment aligned with test workflow/tooling lockfile expectations.
# Newer LTS (Node 24 / npm 11) can make `npm list` fail with ELSPROBLEMS during vsce packaging.
node-version: 22
cache: 'npm'
cache-dependency-path: |
apps/vscode/package-lock.json
apps/vscode/webview-ui/package-lock.json
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
+12 -7
View File
@@ -93,13 +93,13 @@ jobs:
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode ci
run: npm --prefix apps/vscode ci --include=optional
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/webview-ui ci
run: npm --prefix apps/vscode/webview-ui ci --include=optional
- name: Run Quality Checks (Parallel)
run: npm run ci:check-all
@@ -134,13 +134,13 @@ jobs:
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode ci
run: npm --prefix apps/vscode ci --include=optional
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/webview-ui ci
run: npm --prefix apps/vscode/webview-ui ci --include=optional
- name: Set up NPM on Windows
if: runner.os == 'Windows'
@@ -160,6 +160,11 @@ jobs:
id: build_step
run: npm run ci:build
- name: Vitest Suites (SDK adapter + model catalog)
id: vitest_tests
if: ${{ !cancelled() && steps.build_step.outcome == 'success' }}
run: npm run test:vitest
- name: Unit Tests with coverage - Linux
id: unit_tests_linux
if: ${{ !cancelled() && steps.build_step.outcome == 'success' && runner.os == 'Linux' }}
@@ -236,13 +241,13 @@ jobs:
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode ci
run: npm --prefix apps/vscode ci --include=optional
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/webview-ui ci
run: npm --prefix apps/vscode/webview-ui ci --include=optional
- name: Download ripgrep binaries
run: npm run download-ripgrep
@@ -252,7 +257,7 @@ jobs:
- name: Install testing platform dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/testing-platform ci
run: npm --prefix apps/vscode/testing-platform ci --include=optional
- name: Running testing platform integration spec tests
timeout-minutes: 7
+3
View File
@@ -13,6 +13,9 @@ tmp
pnpm-lock.yaml
.clineignore
.cline/enterprise
.cline/remote-config
**/.cline/remote-config
.venv
.actrc
CLAUDE.local.md
+2 -1
View File
@@ -7,4 +7,5 @@ fi
gitleaks git --pre-commit --redact --staged --verbose || exit 1
lint-staged
cd apps/vscode && lint-staged
-1
View File
@@ -1,2 +1 @@
@.clinerules/general.md
@.clinerules/network.md
-1
View File
@@ -26,7 +26,6 @@ test.describe("root flag descriptions", () => {
"Set reasoning effort level",
"consecutive mistakes",
"Output messages as JSON",
"ACP",
"Check for updates and install if available",
"Run the kanban app",
]);
+7
View File
@@ -6,6 +6,13 @@
"src/**/__tests__/*.ts",
"src/test/services/**/*.test.ts"
],
"ignore": [
"src/core/controller/models/__tests__/providerCatalogHandlers.test.ts",
"src/core/controller/models/__tests__/providerCatalogSmoke.test.ts",
"src/core/controller/models/__tests__/providerSwitchNormalization.test.ts",
"src/core/controller/models/__tests__/resolveModelInfo.test.ts",
"src/core/controller/models/__tests__/refreshClineRecommendedModels.test.ts"
],
"require": [
"ts-node/register",
"source-map-support/register",
+5 -1
View File
@@ -1,9 +1,13 @@
import { defineConfig } from "@vscode/test-cli"
import path from "path"
const vscodeTestVersion = process.env.VSCODE_TEST_VERSION ?? "stable"
export default defineConfig({
files: "{out/**/*.test.js,src/**/*.test.js,!src/test/e2e/**/*.test.js,!out/src/test/e2e/**/*.test.js}",
files: [
"out/src/{core,test,utils,shared,integrations,hosts,services}/**/*.test.js",
"src/{core,test,utils,shared,integrations,hosts,services}/**/*.test.js",
],
mocha: {
ui: "bdd",
timeout: 20000, // Maximum time (in ms) that a test can run before failing
+19 -5
View File
@@ -1,6 +1,11 @@
{
"root": false,
"root": true,
"$schema": "./node_modules/@biomejs/biome/configuration_schema.json",
"vcs": {
"enabled": true,
"clientKind": "git",
"defaultBranch": "main"
},
"assist": {
"enabled": true,
"actions": {
@@ -124,6 +129,7 @@
"!!**/playwright",
"!!**/.vscode-test",
"!!**/test-results",
"!!**/coverage",
"!!**/node_modules",
"!!**/webview-ui/build",
"!!**/generated",
@@ -131,7 +137,9 @@
"!!**/tests/specs"
]
},
"plugins": ["src/dev/grit/process-env.grit"],
"plugins": [
"src/dev/grit/process-env.grit"
],
"overrides": [
{
"includes": [
@@ -146,11 +154,15 @@
"!!src/integrations/terminal/**",
"!!src/core/controller/ui/openWalkthrough.ts"
],
"plugins": ["src/dev/grit/vscode-api.grit"]
"plugins": [
"src/dev/grit/vscode-api.grit"
]
},
{
// Do not use console logging directly, use the Logger service instead.
"plugins": ["src/dev/grit/console-log.grit"],
"plugins": [
"src/dev/grit/console-log.grit"
],
"includes": [
"**",
"!!**/esbuild.*",
@@ -183,7 +195,9 @@
"!!src/core/storage/utils/state-helpers.ts",
"!!src/extension.ts"
],
"plugins": ["src/dev/grit/use-cache-service.grit"]
"plugins": [
"src/dev/grit/use-cache-service.grit"
]
}
]
}
-39
View File
@@ -85,44 +85,6 @@ const esbuildProblemMatcherPlugin = {
},
}
const copyWasmFiles = {
name: "copy-wasm-files",
setup(build) {
build.onEnd(() => {
// tree sitter
const sourceDir = path.join(__dirname, "node_modules", "web-tree-sitter")
const targetDir = path.join(__dirname, destDir)
// Copy tree-sitter.wasm
fs.copyFileSync(path.join(sourceDir, "tree-sitter.wasm"), path.join(targetDir, "tree-sitter.wasm"))
// Copy language-specific WASM files
const languageWasmDir = path.join(__dirname, "node_modules", "tree-sitter-wasms", "out")
const languages = [
"typescript",
"tsx",
"python",
"rust",
"javascript",
"go",
"cpp",
"c",
"c_sharp",
"ruby",
"java",
"php",
"swift",
"kotlin",
]
languages.forEach((lang) => {
const filename = `tree-sitter-${lang}.wasm`
fs.copyFileSync(path.join(languageWasmDir, filename), path.join(targetDir, filename))
})
})
},
}
const buildEnvVars = {
"import.meta.url": "_importMetaUrl",
"process.env.IS_STANDALONE": JSON.stringify(standalone ? "true" : "false"),
@@ -176,7 +138,6 @@ const baseConfig = {
define: buildEnvVars,
tsconfig: path.resolve(__dirname, "tsconfig.json"),
plugins: [
copyWasmFiles,
aliasResolverPlugin,
/* add to the end of plugins array */
esbuildProblemMatcherPlugin,
+32 -21
View File
@@ -1,23 +1,34 @@
{
"entry": [
"src/extension.ts",
"src/standalone/cline-core.ts",
"src/generated/hosts/standalone/protobus-server-setup.ts",
"src/generated/hosts/standalone/host-bridge-clients.ts",
"src/generated/hosts/vscode/protobus-services.ts",
"src/generated/hosts/vscode/hostbridge-grpc-service-config.ts"
],
"project": [
"src/**/*.ts"
],
"ignore": [
"out/**",
"node_modules/**",
"*.d.ts",
"**/*.test.ts",
"**/__tests__",
"src/test/**",
"src/shared/**"
],
"vite": true
"$schema": "https://unpkg.com/knip@5/schema.json",
"workspaces": {
".": {
"entry": [
"src/extension.ts",
"src/standalone/cline-core.ts",
"src/generated/hosts/standalone/protobus-server-setup.ts",
"src/generated/hosts/standalone/host-bridge-clients.ts",
"src/generated/hosts/vscode/protobus-services.ts",
"src/generated/hosts/vscode/hostbridge-grpc-service-config.ts",
"src/**/*.test.ts",
"src/**/__tests__/**/*.ts",
"src/test/**/*.ts"
],
"project": [
"src/**/*.ts"
]
},
"webview-ui": {
"entry": [
"src/services/grpc-client.ts",
"src/**/*.test.{ts,tsx}",
"src/**/*.spec.{ts,tsx}",
"src/**/__tests__/**/*.{ts,tsx}"
],
"project": [
"src/**/*.{ts,tsx}",
"*.ts"
],
"vite": true
}
}
}
+14795 -14524
View File
File diff suppressed because it is too large Load Diff
+25 -56
View File
@@ -229,26 +229,9 @@
"command": "cline.reconstructTaskHistory",
"title": "Reconstruct Task History",
"category": "Cline"
},
{
"command": "cline.reviewComment.reply",
"title": "Reply",
"category": "Cline",
"enablement": "!commentIsEmpty"
},
{
"command": "cline.reviewComment.addToChat",
"title": "Add to Cline Chat",
"category": "Cline",
"icon": "$(link-external)"
}
],
"keybindings": [
{
"command": "editor.action.submitComment",
"key": "enter",
"when": "commentEditorFocused && commentController == cline-ai-review && !commentIsEmpty"
},
{
"command": "cline.addToChat",
"key": "cmd+'",
@@ -350,24 +333,6 @@
{
"command": "cline.abortGitCommitMessage",
"when": "config.git.enabled && cline.isGeneratingCommit"
},
{
"command": "cline.reviewComment.reply",
"when": "false"
}
],
"comments/commentThread/context": [
{
"command": "cline.reviewComment.reply",
"group": "inline",
"when": "commentController == cline-ai-review"
}
],
"comments/commentThread/title": [
{
"command": "cline.reviewComment.addToChat",
"group": "inline",
"when": "commentController == cline-ai-review"
}
]
},
@@ -395,21 +360,28 @@
"clean:all": "npm run clean:build && npm run clean:deps",
"compile-tests": "node ./scripts/build-tests.js",
"watch-tests": "tsc -p . -w --outDir out",
"check-types": "npm run protos && tsc --noEmit && cd webview-ui && tsc --noEmit",
"check-types": "npm run protos && npx tsc --noEmit && cd webview-ui && npx tsc --noEmit",
"lint": "biome lint --config-path ./biome.jsonc --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error && npm run lint:proto",
"lint:proto": "bash ./scripts/proto-lint.sh",
"format": "biome format --config-path ./biome.jsonc --changed --since main --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error",
"format:fix": "biome check --config-path ./biome.jsonc --changed --since main --no-errors-on-unmatched --files-ignore-unknown=true --write",
"fix:all": "biome check --config-path ./biome.jsonc --no-errors-on-unmatched --files-ignore-unknown=true --write --diagnostic-level=error --unsafe",
"analyze:unused": "npx --yes knip --config knip.json --include files,exports,types,enumMembers,duplicates",
"analyze:unused:prod": "npx --yes knip --config knip.json --production --include files,exports,types,enumMembers,dependencies",
"analyze:unused:fix-exports": "node scripts/remove-unused-exports.mjs --apply",
"analyze:unused:fix-exports:dry": "node scripts/remove-unused-exports.mjs",
"format": "biome format --config-path ./biome.jsonc --changed --since main --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error --semicolons=as-needed",
"format:fix": "biome check --config-path ./biome.jsonc --changed --since main --no-errors-on-unmatched --files-ignore-unknown=true --write --semicolons=as-needed",
"fix:all": "biome check --config-path ./biome.jsonc --no-errors-on-unmatched --files-ignore-unknown=true --write --diagnostic-level=error --unsafe --semicolons=as-needed",
"ci:check-all": "npx npm-run-all -p check-types lint format",
"ci:build": "npm run protos && npm run build:webview && node esbuild.mjs && npm run compile-tests",
"pretest": "npm run compile && npm run compile-tests && npm run compile-standalone && npm run lint",
"test": "npx npm-run-all test:unit test:integration",
"test:integration": "vscode-test",
"test:unit": "cross-env TS_NODE_PROJECT=./tsconfig.unit-test.json mocha",
"test:vitest": "vitest run --config vitest.config.ts",
"test:vitest:watch": "vitest --config vitest.config.ts",
"test:coverage": "vscode-test --coverage",
"test:sca-server": "npx tsx watch scripts/test-standalone-core-api-server.ts",
"test:tp-orchestrator": "npx tsx scripts/testing-platform-orchestrator.ts",
"dev:mcp-oauth-test-server": "npx tsx src/dev/mcp-oauth-test-server/server.ts",
"e2e": "playwright test -c playwright.config.ts",
"test:e2e:build": "vsce package --allow-package-secrets sendgrid --out dist/e2e.vsix",
"test:e2e": "playwright install && npm run test:e2e:build && node src/test/e2e/utils/build.mjs && playwright test",
@@ -433,10 +405,10 @@
"lint-staged": {
"src/shared/storage/state-keys.ts": [
"node scripts/generate-state-proto.mjs",
"git add proto/cline/state.proto"
"git add apps/vscode/proto/cline/state.proto"
],
"*": [
"biome check --write --staged --no-errors-on-unmatched --files-ignore-unknown=true"
"biome check --write --staged --no-errors-on-unmatched --files-ignore-unknown=true --semicolons=as-needed"
]
},
"devDependencies": {
@@ -479,27 +451,28 @@
"rimraf": "^6.0.1",
"should": "^13.2.3",
"sinon": "^21.0.3",
"tar": "^7.5.2",
"tree-kill": "^1.2.2",
"ts-node": "^10.9.2",
"ts-proto": "^2.6.1",
"tsconfig-paths": "^4.2.0",
"typescript": "^5.4.5"
"typescript": "^5.4.5",
"vitest": "^4.0.17"
},
"dependencies": {
"@anthropic-ai/sdk": "^0.37.0",
"@anthropic-ai/vertex-sdk": "^0.6.4",
"@aws-sdk/client-bedrock-runtime": "^3.922.0",
"@aws-sdk/credential-providers": "^3.922.0",
"@azure/identity": "^4.13.0",
"@bufbuild/protobuf": "^2.2.5",
"@cerebras/cerebras_cloud_sdk": "^1.35.0",
"@google-cloud/vertexai": "^1.9.3",
"@cline/agents": "0.0.42-nightly.1780514867",
"@cline/core": "0.0.42-nightly.1780514867",
"@cline/llms": "0.0.42-nightly.1780514867",
"@cline/shared": "0.0.42-nightly.1780514867",
"@google/genai": "^1.30.0",
"@grpc/grpc-js": "^1.9.15",
"@grpc/proto-loader": "^0.7.13",
"@grpc/reflection": "^1.0.4",
"@mistralai/mistralai": "^1.5.0",
"@modelcontextprotocol/sdk": "^1.25.1",
"@opentelemetry/api": "^1.9.0",
"@opentelemetry/api-logs": "^0.56.0",
"@opentelemetry/core": "^2.1.0",
"@opentelemetry/exporter-logs-otlp-grpc": "^0.56.0",
"@opentelemetry/exporter-logs-otlp-http": "^0.56.0",
@@ -519,9 +492,6 @@
"@opentelemetry/sdk-trace-node": "^1.30.1",
"@opentelemetry/semantic-conventions": "^1.37.0",
"@playwright/test": "^1.55.1",
"@sap-ai-sdk/ai-api": "^2.7.0",
"@sap-ai-sdk/orchestration": "^2.7.0",
"@sap-cloud-sdk/connectivity": "^4.6.0",
"@streamparser/json": "^0.0.22",
"@tailwindcss/vite": "^4.1.14",
"@types/uuid": "^10.0.0",
@@ -547,13 +517,14 @@
"ignore": "^7.0.3",
"image-size": "^2.0.2",
"isbinaryfile": "^5.0.2",
"js-yaml": "^4.1.1",
"jschardet": "^3.1.4",
"json5": "^2.2.3",
"jwt-decode": "^4.0.0",
"mammoth": "^1.11.0",
"nanoid": "^5.1.6",
"nice-grpc": "^2.1.12",
"node-machine-id": "^1.1.12",
"ollama": "^0.5.13",
"open": "^10.1.2",
"open-graph-scraper": "^6.9.0",
"openai": "^6.21.0",
@@ -572,15 +543,13 @@
"simple-git": "3.36.0",
"strip-ansi": "^7.1.2",
"tailwindcss": "^4.1.14",
"tree-sitter-wasms": "^0.1.11",
"ts-morph": "^25.0.1",
"turndown": "^7.2.0",
"ulid": "^2.4.0",
"undici": "^7.26.0",
"uuid": "^11.1.0",
"vscode-uri": "^3.1.0",
"web-tree-sitter": "^0.22.6",
"zod": "^3.24.2"
"zod": "^4.3.6"
},
"overrides": {
"tar-fs": ">=3.1.1",
+123 -2
View File
@@ -21,8 +21,6 @@ service ModelsService {
rpc refreshOpenRouterModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns recommended and free Cline models
rpc refreshClineRecommendedModelsRpc(EmptyRequest) returns (ClineRecommendedModelsResponse);
// Refreshes and returns Cline provider models
rpc refreshClineModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns Hugging Face models
rpc refreshHuggingFaceModels(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns OpenAI models
@@ -55,6 +53,18 @@ service ModelsService {
rpc getAihubmixModels(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns Vercel AI Gateway models
rpc refreshVercelAiGatewayModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Lists providers available from the unified SDK-backed catalog
rpc listProviders(Empty) returns (ProviderListingsResponse);
// Resolves model metadata for a provider through the unified SDK-backed catalog
rpc resolveProviderModels(ResolveProviderModelsRequest) returns (ProviderModelsResponse);
// Resolves model metadata for a provider/model without refreshing model lists
rpc resolveModelInfo(ResolveModelInfoRequest) returns (ResolveModelInfoResponse);
// Reads redacted effective provider configuration
rpc readProviderConfig(StringRequest) returns (ProviderConfigResponse);
// Writes provider configuration fields and returns redacted effective configuration
rpc writeProviderConfig(WriteProviderConfigRequest) returns (ProviderConfigResponse);
// Commits a mode-specific model selection atomically with its model metadata
rpc commitModelSelection(CommitModelSelectionRequest) returns (Empty);
}
// List of VS Code LM models
@@ -117,6 +127,117 @@ message OpenRouterCompatibleModelInfo {
map<string, OpenRouterModelInfo> models = 1;
}
// Lightweight provider entry for the top-level model/provider picker.
// Does not include the full model list; use resolveProviderModels for models.
message ProviderListing {
string id = 1;
string name = 2;
optional string default_model_id = 3;
optional string family = 4;
optional string protocol = 5;
optional string auth_description = 6;
optional string base_url_description = 7;
bool allows_custom_model_ids = 8;
// SDK-driven hint for cost display. Values: "show" (default) or "hide".
// Sourced from `resolveProviderUsageCostDisplay(provider.metadata)` in
// `@cline/llms`. When "hide", consumers must suppress per-token pricing
// and total cost displays (matches the CLI's `shouldShowCliUsageCost`).
string usage_cost_display = 11;
}
message ProviderListingsResponse {
repeated ProviderListing providers = 1;
}
message ResolveProviderModelsRequest {
string provider_id = 1;
bool force_refresh = 2;
optional string request_id = 3;
}
message CatalogErrorInfo {
string kind = 1;
string message = 2;
optional string code = 3;
optional bool retryable = 4;
}
message ProviderModelsResponse {
string provider_id = 1;
string request_id = 2;
string config_fingerprint = 3;
int64 fetched_at = 4;
bool ok = 5;
map<string, OpenRouterModelInfo> models = 6;
optional string default_model_id = 7;
optional string source = 8;
optional CatalogErrorInfo error = 9;
}
message ResolveModelInfoRequest {
string provider_id = 1;
optional string model_id = 2;
}
message ResolveModelInfoResponse {
string provider_id = 1;
string model_id = 2;
optional OpenRouterModelInfo model_info = 3;
string source = 4;
}
message ProviderConfigResponse {
string provider_id = 1;
optional string base_url = 2;
optional string api_line = 3;
map<string, string> headers = 4;
optional string region = 5;
int64 api_key_length = 6;
bool has_access_token = 7;
bool has_refresh_token = 8;
optional string account_id = 9;
optional CommittedModelSelection plan_selection = 10;
optional CommittedModelSelection act_selection = 11;
}
message CommittedModelSelection {
string provider_id = 1;
string model_id = 2;
OpenRouterModelInfo model_info = 3;
}
message ProviderReasoningPatch {
optional bool enabled = 1;
optional string effort = 2; // "none" | "low" | "medium" | "high" | "xhigh"
optional int32 budget_tokens = 3;
}
message WriteProviderConfigPatch {
optional string api_key = 1;
optional string base_url = 2;
map<string, string> headers = 3;
optional string region = 4;
optional string api_line = 5;
optional string access_token = 6;
optional string refresh_token = 7;
optional string account_id = 8;
optional ProviderReasoningPatch reasoning = 9;
optional bool clear_headers = 10;
}
message WriteProviderConfigRequest {
string provider_id = 1;
WriteProviderConfigPatch patch = 2;
}
message CommitModelSelectionRequest {
string provider_id = 1;
string mode = 2;
string model_id = 3;
OpenRouterModelInfo model_info = 4;
}
message ClineRecommendedModel {
string id = 1;
string name = 2;
@@ -0,0 +1,32 @@
syntax = "proto3";
package cline;
import "cline/common.proto";
option go_package = "github.com/cline/grpc-go/cline";
option java_multiple_files = true;
option java_package = "bot.cline.proto";
enum RemoteConfigType {
RULE = 0;
WORKFLOW = 1;
SKILL = 2;
}
message RemoteConfigSetting {
RemoteConfigType type = 1;
string name = 2;
string content = 3;
bool enabled = 4;
bool locked = 5;
}
message RemoteConfigSettingsResponse {
repeated RemoteConfigSetting settings = 1;
}
service RemoteConfigService {
rpc getRemoteConfigSettings(Empty) returns (RemoteConfigSettingsResponse);
rpc toggleRemoteConfigSetting(StringRequest) returns (RemoteConfigSetting);
}
+1 -1
View File
@@ -23,7 +23,7 @@ message SlashCommandInfo {
string name = 1; // Command name without slash, e.g., "newtask", "smol"
string description = 2; // Human-readable description
string section = 3; // "default", "custom", or "cli"
bool cli_compatible = 4; // false for VS Code-only commands like explain-changes
bool cli_compatible = 4; // false for VS Code-only commands
}
// Response containing all available slash commands
+2 -4
View File
@@ -250,7 +250,6 @@ message Settings {
optional string default_terminal_profile = 137;
optional int32 terminal_output_line_limit = 138;
optional int32 max_consecutive_mistakes = 139;
optional bool strict_plan_mode_enabled = 141;
optional bool yolo_mode_toggled = 142;
optional bool use_auto_condense = 143;
optional bool cline_web_tools_enabled = 144;
@@ -286,7 +285,6 @@ message Settings {
optional string act_mode_cline_model_id = 180;
optional OpenRouterModelInfo act_mode_cline_model_info = 181;
optional bool show_feature_tips = 182;
optional bool lazy_teammate_mode_enabled = 183;
}
message State {
@@ -391,6 +389,7 @@ message UpdateSettingsRequest {
reserved 15; // was openai_reasoning_effort (moved to mode-scoped reasoning effort)
reserved 23; // was dictation_settings (dictation removed)
reserved 38; // was skills_enabled (removed - now always enabled)
reserved 43; // was lazy_teammate_mode_enabled (removed)
Metadata metadata = 1;
optional ModelsApiConfiguration api_configuration = 2;
@@ -405,7 +404,7 @@ message UpdateSettingsRequest {
optional int32 terminal_output_line_limit = 12;
optional PlanActMode mode = 13;
optional string preferred_language = 14;
optional bool strict_plan_mode_enabled = 16;
reserved 16; // was strict_plan_mode_enabled (removed)
optional FocusChainSettings focus_chain_settings = 17;
optional bool use_auto_condense = 18;
optional string custom_prompt = 19;
@@ -429,7 +428,6 @@ message UpdateSettingsRequest {
optional bool worktrees_enabled = 40;
optional bool double_check_completion_enabled = 41;
optional bool show_feature_tips = 42;
optional bool lazy_teammate_mode_enabled = 43;
}
message UpdateTerminalConnectionTimeoutRequest {
+15 -10
View File
@@ -32,6 +32,8 @@ service TaskService {
rpc getTaskHistory(GetTaskHistoryRequest) returns (TaskHistoryArray);
// Sends a response to a previous ask operation
rpc askResponse(AskResponseRequest) returns (Empty);
// Edits a previous user message, truncates following conversation, and regenerates
rpc editMessageAndRegenerate(EditMessageAndRegenerateRequest) returns (Empty);
// Records task feedback (thumbs up/down)
rpc taskFeedback(StringRequest) returns (Empty);
// Shows task completion changes diff in a view
@@ -40,8 +42,6 @@ service TaskService {
rpc executeQuickWin(ExecuteQuickWinRequest) returns (Empty);
// Deletes all task history
rpc deleteAllTaskHistory(EmptyRequest) returns (DeleteAllTaskHistoryCount);
// Explains changes with AI and adds inline comments to the diff view
rpc explainChanges(ExplainChangesRequest) returns (Empty);
}
// Request message for creating a new task
@@ -82,12 +82,14 @@ message GetTaskHistoryRequest {
string search_query = 3;
string sort_by = 4;
bool current_workspace_only = 5;
int32 limit = 6;
int32 offset = 7;
}
// Response for task history
message TaskHistoryArray {
repeated TaskItem tasks = 1;
int32 total_count = 2;
bool has_more = 2;
}
// Task item details for history list
@@ -114,6 +116,16 @@ message AskResponseRequest {
repeated string files = 5;
}
// Request for editing a past user message and regenerating the conversation after it
message EditMessageAndRegenerateRequest {
Metadata metadata = 1;
int64 message_ts = 2;
string text = 3;
repeated string images = 4;
repeated string files = 5;
bool restore_workspace = 6;
}
// Request for executing a quick win task
message ExecuteQuickWinRequest {
Metadata metadata = 1;
@@ -125,10 +137,3 @@ message ExecuteQuickWinRequest {
message DeleteAllTaskHistoryCount {
int32 tasks_deleted = 1;
}
// Request for explaining changes with AI
message ExplainChangesRequest {
Metadata metadata = 1;
// Timestamp of the completion message to explain changes for
int64 message_ts = 2;
}
+6 -1
View File
@@ -67,7 +67,6 @@ enum ClineSay {
INFO = 26;
TASK_PROGRESS = 27;
ERROR_RETRY = 28;
GENERATE_EXPLANATION = 29;
HOOK_STATUS = 30;
HOOK_OUTPUT_STREAM = 31;
COMMAND_PERMISSION_DENIED = 32;
@@ -226,6 +225,12 @@ message ClineMessage {
ClineAskNewTask ask_new_task = 21;
ClineApiReqInfo api_req_info = 22;
ClineModelInfo model_info = 23;
// Convergent-replica fields (see webview-message-state-design.md):
// seq = monotonic freshness (higher seq wins for the same ts/identity)
// epoch = conversation/replica fence (older epoch is dropped by the webview)
int64 seq = 24;
int64 epoch = 25;
}
message ShowWebviewEvent {
+129
View File
@@ -0,0 +1,129 @@
// Dead-source finder: uses esbuild's own bundle reachability (the same analysis
// that drives tree-shaking + minification mangling) to compute which src/ files
// are reachable from BOTH shipped entry points:
// - src/extension.ts (VS Code extension host)
// - src/standalone/cline-core.ts (standalone host used by JetBrains + CLI)
//
// A src/*.ts file that is NOT in the union of metafile inputs for those two
// builds is unreachable from any shipped entry => dead (modulo dynamic import()
// of computed specifiers, which esbuild surfaces separately).
//
// Run: node scripts/find-dead-src.mjs
import fs from "node:fs"
import path from "node:path"
import { fileURLToPath } from "node:url"
import * as esbuild from "esbuild"
import { glob } from "glob"
const __dirname = path.dirname(fileURLToPath(import.meta.url))
const root = path.resolve(__dirname, "..")
const aliases = {
"@": path.join(root, "src"),
"@core": path.join(root, "src/core"),
"@integrations": path.join(root, "src/integrations"),
"@services": path.join(root, "src/services"),
"@shared": path.join(root, "src/shared"),
"@utils": path.join(root, "src/utils"),
"@packages": path.join(root, "src/packages"),
}
const aliasResolverPlugin = {
name: "alias-resolver",
setup(build) {
for (const [alias, aliasPath] of Object.entries(aliases)) {
const aliasRegex = new RegExp(`^${alias}($|/.*)`)
build.onResolve({ filter: aliasRegex }, (args) => {
const importPath = args.path.replace(alias, aliasPath)
const exts = [".ts", ".tsx", ".js", ".jsx"]
if (fs.existsSync(importPath)) {
const stats = fs.statSync(importPath)
if (stats.isDirectory()) {
for (const ext of exts) {
const idx = path.join(importPath, `index${ext}`)
if (fs.existsSync(idx)) return { path: idx }
}
} else {
return { path: importPath }
}
}
for (const ext of exts) {
if (fs.existsSync(`${importPath}${ext}`)) return { path: `${importPath}${ext}` }
}
return undefined
})
}
},
}
const common = {
bundle: true,
minify: false,
sourcemap: false,
logLevel: "silent",
format: "cjs",
platform: "node",
metafile: true,
write: false,
absWorkingDir: root,
tsconfig: path.join(root, "tsconfig.json"),
packages: "external",
plugins: [aliasResolverPlugin],
define: { "process.env.IS_DEV": "false", "process.env.IS_TEST": "false" },
banner: { js: "const _importMetaUrl=require('url').pathToFileURL(__filename)" },
}
async function inputsFor(entry, external) {
const r = await esbuild.build({ ...common, entryPoints: [entry], external })
return new Set(Object.keys(r.metafile.inputs).filter((f) => f.startsWith("src/") && /\.tsx?$/.test(f)))
}
const ext = await inputsFor("src/extension.ts", ["vscode"])
const standalone = await inputsFor("src/standalone/cline-core.ts", [
"vscode",
"@grpc/reflection",
"grpc-health-check",
"better-sqlite3",
])
const live = new Set([...ext, ...standalone])
// Third consumer: the webview (webview-ui/) is a separate Vite/React build that
// imports extension code ONLY from src/shared (via "@shared/*" alias or relative
// "../src/shared/*" paths). Any src/shared file referenced from webview-ui/src is
// therefore live even if the extension-host/standalone bundles don't reach it.
// Conservatively mark every src/shared file mentioned by the webview as live.
const webviewFiles = await glob("webview-ui/src/**/*.{ts,tsx}", { cwd: root })
const sharedMentionedByWebview = new Set()
for (const wf of webviewFiles) {
const text = fs.readFileSync(path.join(root, wf), "utf8")
// Match @shared/X or .../src/shared/X import specifiers and map to src/shared/X
const re = /(?:@shared\/|src\/shared\/)([A-Za-z0-9_./-]+)/g
let m
while ((m = re.exec(text))) {
const rel = m[1].replace(/\.(ts|tsx|js|jsx)$/, "")
for (const cand of [`src/shared/${rel}.ts`, `src/shared/${rel}.tsx`, `src/shared/${rel}/index.ts`]) {
if (fs.existsSync(path.join(root, cand))) sharedMentionedByWebview.add(cand)
}
}
}
for (const f of sharedMentionedByWebview) live.add(f)
console.log(`src/shared files referenced by webview: ${sharedMentionedByWebview.size}`)
// All non-test, non-.d.ts source files on disk.
const allSrc = (await glob("src/**/*.{ts,tsx}", { cwd: root }))
.filter((f) => !/\.test\.tsx?$/.test(f))
.filter((f) => !f.endsWith(".d.ts"))
.filter((f) => !f.includes("/__tests__/"))
.filter((f) => !f.startsWith("src/test/"))
.filter((f) => !f.startsWith("src/generated/")) // generated host glue
.filter((f) => !f.startsWith("src/dev/")) // dev-only tooling
const dead = allSrc.filter((f) => !live.has(f)).sort()
console.log(`extension inputs: ${ext.size}`)
console.log(`standalone inputs: ${standalone.size}`)
console.log(`union live src files: ${live.size}`)
console.log(`candidate dead files: ${dead.length}`)
fs.writeFileSync("/tmp/dead-src.json", JSON.stringify(dead, null, "\t"))
console.log("--- dead candidates written to /tmp/dead-src.json ---")
@@ -87,6 +87,12 @@ function createMockService<T extends grpc.UntypedServiceImplementation>(serviceN
})
return
case "openExternal":
simulateOAuthBrowserCallback(call.request?.value || "")
.then(() => callback(null, {}))
.catch((error) => callback(error))
return
case "getWebviewHtml":
callback(null, {
html: "<html><body>Fake Webview</body></html>",
@@ -143,6 +149,41 @@ function createMockService<T extends grpc.UntypedServiceImplementation>(serviceN
return new Proxy({} as T, handler)
}
async function simulateOAuthBrowserCallback(urlString: string): Promise<void> {
let url: URL
try {
url = new URL(urlString)
} catch {
return
}
if (!isLoopbackHost(url.hostname) || url.pathname !== "/api/v1/auth/authorize") {
return
}
const callbackUrl = url.searchParams.get("callback_url") ?? url.searchParams.get("redirect_uri")
if (!callbackUrl) {
return
}
const callback = new URL(callbackUrl)
if (!isLoopbackHost(callback.hostname) || callback.pathname !== "/auth") {
return
}
callback.searchParams.set("code", "test-personal-token")
callback.searchParams.set("provider", "cline")
const response = await fetch(callback.toString())
if (!response.ok) {
throw new Error(`Mock OAuth callback failed: ${response.status} ${response.statusText}`)
}
}
function isLoopbackHost(hostname: string): boolean {
return hostname === "localhost" || hostname === "127.0.0.1" || hostname === "::1"
}
if (require.main === module) {
startTestHostBridgeServer().catch((err) => {
console.error("Failed to start test host bridge server:", err)
@@ -115,7 +115,8 @@ async function main(): Promise<void> {
const covDir = path.join(projectRoot, `coverage/coverage-core-${PROTOBUS_PORT}`)
const baseArgs = ["--enable-source-maps", path.join(distDir, "cline-core.js")]
const workosFetchMockPath = path.join(projectRoot, "scripts", "testing-platform-workos-fetch-mock.cjs")
const baseArgs = ["--enable-source-maps", "--require", workosFetchMockPath, path.join(distDir, "cline-core.js")]
const spawnArgs = USE_C8 ? ["c8", "--report-dir", covDir, "node", ...baseArgs] : ["node", ...baseArgs]
@@ -0,0 +1,56 @@
// Preload used by the standalone testing platform.
// It makes the SDK WorkOS device-auth flow deterministic and fully local while
// leaving production auth code on the same device-auth path used by users.
const originalFetch = globalThis.fetch?.bind(globalThis)
const WORKOS_ORIGIN = "https://api.workos.com"
const DEVICE_CODE = "test-device-code"
const USER_CODE = "PTBC-TXTP"
const ACCESS_TOKEN = "test-personal-token"
const REFRESH_TOKEN = "test-personal-token_refresh"
function jsonResponse(body, init = {}) {
return new Response(JSON.stringify(body), {
status: init.status ?? 200,
headers: { "Content-Type": "application/json", ...(init.headers ?? {}) },
})
}
function inputUrl(input) {
if (typeof input === "string") return input
if (input instanceof URL) return input.toString()
if (input && typeof input === "object" && "url" in input) return input.url
return String(input)
}
globalThis.fetch = async (input, init) => {
const urlString = inputUrl(input)
let url
try {
url = new URL(urlString)
} catch {
return originalFetch(input, init)
}
if (url.origin === WORKOS_ORIGIN && url.pathname === "/user_management/authorize/device") {
return jsonResponse({
device_code: DEVICE_CODE,
user_code: USER_CODE,
verification_uri: "https://login.workos.test/device",
verification_uri_complete: `https://login.workos.test/device?user_code=${USER_CODE}`,
expires_in: 300,
interval: 1,
})
}
if (url.origin === WORKOS_ORIGIN && url.pathname === "/user_management/authenticate") {
return jsonResponse({
access_token: ACCESS_TOKEN,
refresh_token: REFRESH_TOKEN,
token_type: "Bearer",
})
}
return originalFetch(input, init)
}
+1 -3
View File
@@ -21,9 +21,7 @@ describe("ClineEndpoint configuration", () => {
// Stub os.homedir to return our temp directory
originalHomedir = os.homedir
sandbox
.stub(os, "homedir")
.returns(tempDir)
sandbox.stub(os, "homedir").returns(tempDir)
// Reset the singleton state using internal method
;(ClineEndpoint as any)._instance = null
+7 -4
View File
@@ -4,13 +4,13 @@ import "./utils/path" // necessary to have access to String.prototype.toPosix
import { HostProvider } from "@/hosts/host-provider"
import { Logger } from "@/shared/services/Logger"
import type { StorageContext } from "@/shared/storage/storage-context"
import { FileContextTracker } from "./core/context/context-tracking/FileContextTracker"
import { clearOnboardingModelsCache } from "./core/controller/models/getClineOnboardingModels"
import { HookDiscoveryCache } from "./core/hooks/HookDiscoveryCache"
import { HookProcessRegistry } from "./core/hooks/HookProcessRegistry"
import { StateManager } from "./core/storage/StateManager"
import { AgentConfigLoader } from "./core/task/tools/subagent/AgentConfigLoader"
import { ExtensionRegistryInfo } from "./registry"
import { registerVsCodeLmHandler } from "./sdk/vscode-lm/register-vscode-lm"
import { ErrorService } from "./services/error"
import { featureFlagsService } from "./services/feature-flags"
import { getDistinctId } from "./services/logging/distinctId"
@@ -52,6 +52,11 @@ export async function initialize(storageContext: StorageContext): Promise<Webvie
})
}
// Register host-only SDK provider handlers (e.g. VS Code Language Model API),
// which depend on the `vscode` module and cannot live in the SDK package.
// Must run before any handler is built (standalone utilities or task loop).
registerVsCodeLmHandler()
// =============== External services ===============
await ErrorService.initialize()
// Initialize PostHog client provider (skip in self-hosted mode)
@@ -74,8 +79,6 @@ export async function initialize(storageContext: StorageContext): Promise<Webvie
syncWorker().init({ ...blobStoreSettings, userDistinctId: getDistinctId() })
// Clean up old temp files in background (non-blocking) and start periodic cleanup every 24 hours
ClineTempManager.startPeriodicCleanup()
// Clean up orphaned file context warnings (startup cleanup)
FileContextTracker.cleanupOrphanedWarnings(stateManager)
telemetryService.captureExtensionActivated()
@@ -106,7 +109,7 @@ async function showVersionUpdateAnnouncement(stateManager: StateManager) {
})
}
// Always update the main version tracker for the next launch.
await stateManager.setGlobalState("clineVersion", currentVersion)
stateManager.setGlobalState("clineVersion", currentVersion)
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error)
+3 -4
View File
@@ -4,12 +4,11 @@ import * as path from "path"
import { Environment, type EnvironmentConfig } from "./shared/config-types"
import { Logger } from "./shared/services/Logger"
export { Environment, type EnvironmentConfig }
/**
export { Environment } /**
* Schema for the endpoints.json configuration file used in on-premise deployments.
* All fields are required and must be valid URLs.
*/
interface EndpointsFileSchema {
appBaseUrl: string
apiBaseUrl: string
@@ -36,7 +35,7 @@ class ClineEndpoint {
private onPremiseConfig: EndpointsFileSchema | null = null
private environment: Environment = Environment.production
// Track if config came from bundled file (enterprise distribution)
private isBundled: boolean = false
private isBundled = false
private constructor() {
// Set environment at module load. Use override if provided.
File diff suppressed because it is too large Load Diff
@@ -1,606 +0,0 @@
import { ClineStorageMessage } from "@/shared/messages/content"
const APPLY_PATCH_PATCH_REGEX = /\*\*\* Begin Patch\s+([\s\S]*?)\s+\*\*\* End Patch/m
/**
* Convert apply_patch tool calls to write_to_file and replace_in_file format
*/
export function convertApplyPatchToolCalls(messages: Array<ClineStorageMessage>): Array<ClineStorageMessage> {
// Map to track tool_use_id to converted tool info and original input
const toolUseIdMap = new Map<string, { name: string; input: any; originalInput: any }>()
return messages.map((message) => {
if (!Array.isArray(message.content)) {
return message
}
const convertedContent = message.content.map((block) => {
// Handle tool_use blocks
if (block.type === "tool_use" && block.name === "apply_patch") {
const converted = convertApplyPatchToToolCalls(block.input)
// Store the conversion with original input for matching tool_result
toolUseIdMap.set(block.id, { ...converted, originalInput: block.input })
return {
...block,
name: converted.name,
input: converted.input,
}
}
// Handle tool_result blocks
if (block.type === "tool_result") {
const conversion = toolUseIdMap.get(block.tool_use_id)
if (conversion) {
// Reconstruct the tool_result content to match apply_patch format
const reconstructedContent = reconstructApplyPatchResult(
block,
conversion.name,
conversion.input,
conversion.originalInput,
)
return {
...block,
content: reconstructedContent,
}
}
}
return block
})
return {
...message,
content: convertedContent,
}
})
}
interface ConvertedTool {
name: string
input: any
}
/**
* Parse apply_patch input and convert to write_to_file or replace_in_file format
*/
function convertApplyPatchToToolCalls(input: any): ConvertedTool {
const patchInput = typeof input === "string" ? input : input?.input || ""
// Parse the patch format
const patchMatch = patchInput.match(APPLY_PATCH_PATCH_REGEX)
if (!patchMatch) {
// If we can't parse it, return as-is with write_to_file
return {
name: "write_to_file",
input: input,
}
}
const patchContent = patchMatch[1]
// Extract file operation (Add, Update, or Delete)
const fileMatch = patchContent.match(/\*\*\* (Add|Update|Delete) File: (.+?)(?:\n|$)/m)
if (!fileMatch) {
return {
name: "write_to_file",
input: input,
}
}
const action = fileMatch[1]
const filePath = fileMatch[2].trim()
// If it's an Add operation, convert to write_to_file
if (action === "Add") {
// Extract the content after the file line
const contentAfterFile = patchContent.substring(fileMatch.index! + fileMatch[0].length)
return {
name: "write_to_file",
input: {
absolutePath: filePath,
content: extractNewContentFromPatch(contentAfterFile),
},
}
}
// If it's Update or Delete, convert to replace_in_file
if (action === "Update" || action === "Delete") {
const diff = convertPatchToDiff(patchContent.substring(fileMatch.index! + fileMatch[0].length))
return {
name: "replace_in_file",
input: {
absolutePath: filePath,
diff: diff,
},
}
}
// Fallback
return {
name: "write_to_file",
input: input,
}
}
/**
* Extract new content from add operation patch
*/
function extractNewContentFromPatch(patchContent: string): string {
// For Add operations, the patch should contain lines starting with +
const lines = patchContent.split("\n")
const contentLines: string[] = []
for (const line of lines) {
if (line.startsWith("+")) {
// Remove the + prefix and exactly ONE space if present (but not if it's a tab)
let content = line.substring(1)
if (content.startsWith(" ") && !content.startsWith("\t")) {
content = content.substring(1)
}
contentLines.push(content)
}
}
return contentLines.join("\n")
}
/**
* Convert V4A patch format to SEARCH/REPLACE format
*/
function convertPatchToDiff(patchContent: string): string {
const diffBlocks: string[] = []
const lines = patchContent.split("\n")
let i = 0
while (i < lines.length) {
const line = lines[i]
// Skip empty lines at the start
if (!line.trim() && i === 0) {
i++
continue
}
// Check if this is the start of a hunk (@@) or a direct change line
if (line.trim().startsWith("@@") || line.startsWith("-") || line.startsWith("+")) {
const currentSearch: string[] = []
const currentReplace: string[] = []
// Collect @@ context marker lines
// @@ prefix marks context lines. If @@something, then "something" is context.
// If just @@, then it's an empty context line.
while (i < lines.length && lines[i].trim().startsWith("@@")) {
const trimmedLine = lines[i].trim()
// Extract the actual context content after @@
const contextLine = trimmedLine.substring(2)
// Always add the context line (even if empty)
currentSearch.push(contextLine)
currentReplace.push(contextLine)
i++
}
if (i >= lines.length) {
break
}
// Collect all remaining lines in this hunk until we hit end of content or next @@
const hunkLines: string[] = []
while (i < lines.length) {
// Check if this is a new hunk (starts with @@)
if (lines[i].trim().startsWith("@@")) {
break
}
hunkLines.push(lines[i])
i++
}
// Now process the hunk to build SEARCH/REPLACE
let hasChanges = false
for (let j = 0; j < hunkLines.length; j++) {
const hunkLine = hunkLines[j]
if (hunkLine.startsWith("-")) {
hasChanges = true
// Strip the - prefix and exactly ONE space if present (but not if it's a tab)
let content = hunkLine.substring(1)
if (content.startsWith(" ") && !content.startsWith(" \t")) {
content = content.substring(1)
}
currentSearch.push(content)
} else if (hunkLine.startsWith("+")) {
hasChanges = true
// Strip the + prefix and exactly ONE space if present (but not if it's a tab)
let content = hunkLine.substring(1)
if (content.startsWith(" ") && !content.startsWith(" \t")) {
content = content.substring(1)
}
currentReplace.push(content)
} else {
// Context line without @@ prefix - add to both sides
currentSearch.push(hunkLine)
currentReplace.push(hunkLine)
}
}
// Create the diff block if we have changes
if (hasChanges && (currentSearch.length > 0 || currentReplace.length > 0)) {
diffBlocks.push(
"------- SEARCH\n" +
currentSearch.join("\n") +
"\n=======\n" +
currentReplace.join("\n") +
"\n+++++++ REPLACE",
)
}
} else {
i++
}
}
return diffBlocks.join("\n")
}
/**
* Reconstruct tool_result content to match apply_patch format by extracting
* the final file content and converting it back to V4A patch format
*/
function reconstructApplyPatchResult(
block: any,
convertedToolName: string,
_convertedInput: any,
originalInput: any,
): string | any[] {
// Extract the content from the tool_result
const content = typeof block.content === "string" ? block.content : ""
// Try to extract the final_file_content
const finalContentMatch = content.match(/<final_file_content path="([^"]+)">\s*([\s\S]*?)\s*<\/final_file_content>/)
if (!finalContentMatch) {
// If no final_file_content found, return original content
return block.content
}
const filePath = finalContentMatch[1]
const finalContent = finalContentMatch[2]
// Reconstruct the result message based on the converted tool type
if (convertedToolName === "write_to_file") {
// For write_to_file, we just need to confirm the file was created/written
return `[apply_patch for '${filePath}'] Result:\nThe content was successfully saved to ${filePath}.\n\nThe file has been created/updated with the new content.`
}
if (convertedToolName === "replace_in_file") {
// For replace_in_file, we need to reconstruct the V4A patch format result
// Try to parse the original patch to get the action and build context
const patchInput = typeof originalInput === "string" ? originalInput : originalInput?.input || ""
const patchMatch = patchInput.match(APPLY_PATCH_PATCH_REGEX)
if (patchMatch) {
const patchContent = patchMatch[1]
const fileMatch = patchContent.match(/\*\*\* (Add|Update|Delete) File: (.+?)(?:\n|$)/m)
if (fileMatch) {
const action = fileMatch[1]
return `[apply_patch for '${filePath}'] Result:\nThe content was successfully updated in ${filePath}.\n\nThe file has been modified using ${action} operation.\n\n<final_file_content path="${filePath}">\n${finalContent}\n</final_file_content>\n\nIMPORTANT: For any future changes to this file, use the final_file_content shown above as your reference.`
}
}
// Fallback for replace_in_file
return `[apply_patch for '${filePath}'] Result:\nThe content was successfully updated in ${filePath}.\n\n<final_file_content path="${filePath}">\n${finalContent}\n</final_file_content>\n\nIMPORTANT: For any future changes to this file, use the final_file_content shown above as your reference.`
}
// Default fallback
return block.content
}
/**
* Convert write_to_file and replace_in_file tool calls to apply_patch format
*/
export function convertWriteToFileToolCalls(messages: Array<ClineStorageMessage>): Array<ClineStorageMessage> {
// Map to track tool_use_id to converted tool info and original input
const toolUseIdMap = new Map<string, { originalName: string; originalInput: any; patchInput?: string }>()
// First pass: collect tool_use blocks
for (const message of messages) {
if (!Array.isArray(message.content)) {
continue
}
for (const block of message.content) {
if (block.type === "tool_use" && (block.name === "write_to_file" || block.name === "replace_in_file")) {
toolUseIdMap.set(block.id, {
originalName: block.name,
originalInput: block.input,
})
}
}
}
// Second pass: find tool_results and extract final content to build proper patches
const finalContentMap = new Map<string, string>()
for (const message of messages) {
if (!Array.isArray(message.content)) {
continue
}
for (const block of message.content) {
if (block.type === "tool_result" && toolUseIdMap.has(block.tool_use_id)) {
const content = typeof block.content === "string" ? block.content : ""
const finalContentMatch = content.match(
/<final_file_content path="([^"]+)">\s*([\s\S]*?)\s*<\/final_file_content>/,
)
if (finalContentMatch) {
finalContentMap.set(block.tool_use_id, finalContentMatch[2])
}
}
}
}
// Third pass: convert messages
return messages.map((message) => {
if (!Array.isArray(message.content)) {
return message
}
const convertedContent = message.content.map((block) => {
// Handle tool_use blocks for write_to_file and replace_in_file
if (block.type === "tool_use" && (block.name === "write_to_file" || block.name === "replace_in_file")) {
const finalContent = finalContentMap.get(block.id)
const patchInput = convertToPatchFormat(block.name, block.input, finalContent)
// Update the map with the generated patch
const existingEntry = toolUseIdMap.get(block.id)
if (existingEntry) {
existingEntry.patchInput = patchInput
}
return {
...block,
name: "apply_patch",
input: {
input: patchInput,
},
}
}
// Handle tool_result blocks
if (block.type === "tool_result") {
const conversion = toolUseIdMap.get(block.tool_use_id)
if (conversion) {
// Reconstruct the tool_result content to match apply_patch format
const reconstructedContent = reconstructWriteToFileResult(
block,
conversion.originalName,
conversion.originalInput,
)
return {
...block,
content: reconstructedContent,
}
}
}
return block
})
return {
...message,
content: convertedContent,
}
})
}
/**
* Convert write_to_file or replace_in_file input to apply_patch format
*/
function convertToPatchFormat(toolName: string, input: any, finalContent?: string): string {
const filePath = input.absolutePath || input.path || ""
if (toolName === "write_to_file") {
// Convert write_to_file to Add operation
const content = input.content || ""
const lines = content.split("\n")
const patchLines = ["@@"]
patchLines.push(...lines.map((line: string) => `+ ${line}`))
return `apply_patch <<"EOF"
*** Begin Patch
*** Add File: ${filePath}
${patchLines.join("\n")}
*** End Patch
EOF`
}
if (toolName === "replace_in_file") {
// Convert replace_in_file to Update operation
const diff = input.diff || ""
// Parse SEARCH/REPLACE blocks and convert to V4A format with context
const patchContent = convertDiffToPatchWithContext(diff, finalContent)
return `apply_patch <<"EOF"
*** Begin Patch
*** Update File: ${filePath}
${patchContent}
*** End Patch
EOF`
}
return ""
}
/**
* Convert SEARCH/REPLACE diff format to V4A patch format with additional context from final content
*/
function convertDiffToPatchWithContext(diff: string, finalContent?: string): string {
const patchLines: string[] = []
// Match all SEARCH/REPLACE blocks
const blockRegex = /------- SEARCH\s*\n([\s\S]*?)\n=======\s*\n([\s\S]*?)\n\+{7} REPLACE/g
let match
while ((match = blockRegex.exec(diff)) !== null) {
const searchContent = match[1]
const replaceContent = match[2]
const searchLines = searchContent.split("\n")
const replaceLines = replaceContent.split("\n")
// Find common prefix and suffix between search and replace
let prefixEnd = 0
while (
prefixEnd < searchLines.length &&
prefixEnd < replaceLines.length &&
searchLines[prefixEnd] === replaceLines[prefixEnd]
) {
prefixEnd++
}
let suffixStart = searchLines.length
let replaceSuffixStart = replaceLines.length
while (
suffixStart > prefixEnd &&
replaceSuffixStart > prefixEnd &&
searchLines[suffixStart - 1] === replaceLines[replaceSuffixStart - 1]
) {
suffixStart--
replaceSuffixStart--
}
// If we have finalContent, extract additional context from it
if (finalContent) {
const finalLines = finalContent.split("\n")
// Find where the replaced content appears in the final file
let matchIndex = -1
for (let i = 0; i < finalLines.length; i++) {
// Try to match the first replace line
if (replaceLines.length > 0 && finalLines[i] === replaceLines[0]) {
// Check if subsequent lines also match
let allMatch = true
for (let j = 1; j < replaceLines.length && i + j < finalLines.length; j++) {
if (finalLines[i + j] !== replaceLines[j]) {
allMatch = false
break
}
}
if (allMatch) {
matchIndex = i
break
}
}
}
if (matchIndex >= 0) {
// Extract up to 3 lines before as context
const contextStart = Math.max(0, matchIndex - 3)
const contextLines: string[] = []
for (let i = contextStart; i < matchIndex; i++) {
contextLines.push(finalLines[i])
}
// Pad to 3 lines if needed (with empty strings)
while (contextLines.length < 3) {
contextLines.unshift("")
}
// Add @@ marker with the first context line
if (contextLines[0] === "") {
patchLines.push("@@")
} else {
patchLines.push(`@@${contextLines[0]}`)
}
// Add remaining context lines (without @@ marker)
for (let i = 1; i < contextLines.length; i++) {
patchLines.push(contextLines[i])
}
// Add common prefix lines (without +/- markers)
for (let i = 0; i < prefixEnd; i++) {
patchLines.push(searchLines[i])
}
// Add the actual changes (lines that differ)
for (let i = prefixEnd; i < suffixStart; i++) {
patchLines.push(`- ${searchLines[i]}`)
}
for (let i = prefixEnd; i < replaceSuffixStart; i++) {
patchLines.push(`+ ${replaceLines[i]}`)
}
// Add common suffix lines (without +/- markers)
for (let i = suffixStart; i < searchLines.length; i++) {
patchLines.push(searchLines[i])
}
// Extract up to 3 lines after as trailing context (without @@ markers)
const contextEnd = Math.min(finalLines.length, matchIndex + replaceLines.length + 3)
for (let i = matchIndex + replaceLines.length; i < contextEnd; i++) {
patchLines.push(finalLines[i])
}
continue
}
}
// Fallback: if no finalContent or couldn't find match, use the prefix/suffix from SEARCH/REPLACE
patchLines.push("@@")
// Add common prefix lines (without +/- markers)
for (let i = 0; i < prefixEnd; i++) {
patchLines.push(searchLines[i])
}
// Add the actual changes (lines that differ)
for (let i = prefixEnd; i < suffixStart; i++) {
patchLines.push(`- ${searchLines[i]}`)
}
for (let i = prefixEnd; i < replaceSuffixStart; i++) {
patchLines.push(`+ ${replaceLines[i]}`)
}
// Add common suffix lines (without +/- markers)
for (let i = suffixStart; i < searchLines.length; i++) {
patchLines.push(searchLines[i])
}
}
return patchLines.join("\n")
}
/**
* Reconstruct tool_result content to match apply_patch result format
*/
function reconstructWriteToFileResult(block: any, originalToolName: string, originalInput: any): string | any[] {
// Extract the content from the tool_result
const content = typeof block.content === "string" ? block.content : ""
// Try to extract the final_file_content
const finalContentMatch = content.match(/<final_file_content path="([^"]+)">\s*([\s\S]*?)\s*<\/final_file_content>/)
const filePath = originalInput.absolutePath || originalInput.path || ""
if (!finalContentMatch) {
// If no final_file_content found, create a simple success message
if (originalToolName === "write_to_file") {
return `[apply_patch for '${filePath}'] Result:\nThe content was successfully saved to ${filePath}.\n\nThe file has been created/updated with the new content.`
} else {
return `[apply_patch for '${filePath}'] Result:\nThe content was successfully updated in ${filePath}.\n\nThe file has been modified.`
}
}
const finalContent = finalContentMatch[2]
// Reconstruct the result message based on the original tool type
if (originalToolName === "write_to_file") {
return `[apply_patch for '${filePath}'] Result:\nThe content was successfully saved to ${filePath}.\n\nThe file has been created/updated with the new content.`
}
if (originalToolName === "replace_in_file") {
return `[apply_patch for '${filePath}'] Result:\nThe content was successfully updated in ${filePath}.\n\nThe file has been modified using Update operation.\n\n<final_file_content path="${filePath}">\n${finalContent}\n</final_file_content>\n\nIMPORTANT: For any future changes to this file, use the final_file_content shown above as your reference.`
}
// Default fallback
return block.content
}
@@ -1,60 +0,0 @@
import { ClineStorageMessage } from "@/shared/messages/content"
import { ClineDefaultTool } from "@/shared/tools"
import { convertApplyPatchToolCalls, convertWriteToFileToolCalls } from "./diff-editors"
/**
* Transforms tool call messages between different tool formats based on native tool support.
* Converts between apply_patch and write_to_file/replace_in_file formats as needed.
*
* @param clineMessages - Array of messages containing tool calls to transform
* @param nativeTools - Array of tools natively supported by the current provider
* @returns Transformed messages array, or original if no transformation needed
*/
export function transformToolCallMessages(
clineMessages: ClineStorageMessage[],
nativeTools?: ClineDefaultTool[],
): ClineStorageMessage[] {
// Early return if no messages or native tools provided
if (!clineMessages?.length || !nativeTools?.length) {
return clineMessages
}
// Create Sets for O(1) lookup performance
const nativeToolSet = new Set(nativeTools)
const usedToolSet = new Set<string>()
// Single pass: collect all tools used in assistant messages
for (const msg of clineMessages) {
if (msg.role === "assistant" && Array.isArray(msg.content)) {
for (const block of msg.content) {
if (block.type === "tool_use" && block.name) {
usedToolSet.add(block.name)
}
}
}
}
// Early return if no tools were used
if (usedToolSet.size === 0) {
return clineMessages
}
// Determine which conversion to apply
const hasApplyPatchNative = nativeToolSet.has(ClineDefaultTool.APPLY_PATCH)
const hasFileEditNative = nativeToolSet.has(ClineDefaultTool.FILE_EDIT) || nativeToolSet.has(ClineDefaultTool.FILE_NEW)
const hasApplyPatchUsed = usedToolSet.has(ClineDefaultTool.APPLY_PATCH)
const hasFileEditUsed = usedToolSet.has(ClineDefaultTool.FILE_EDIT) || usedToolSet.has(ClineDefaultTool.FILE_NEW)
// Convert write_to_file/replace_in_file → apply_patch
if (hasApplyPatchNative && hasFileEditUsed) {
return convertWriteToFileToolCalls(clineMessages)
}
// Convert apply_patch → write_to_file/replace_in_file
if (hasFileEditNative && hasApplyPatchUsed) {
return convertApplyPatchToolCalls(clineMessages)
}
return clineMessages
}
+9 -493
View File
@@ -1,65 +1,18 @@
import { ApiConfiguration, ModelInfo, QwenApiRegions } from "@shared/api"
import { ModelInfo } from "@shared/api"
import { Mode } from "@shared/storage/types"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { ClineTool } from "@/shared/tools"
import { AIhubmixHandler } from "./providers/aihubmix"
import { AnthropicHandler } from "./providers/anthropic"
import { AskSageHandler } from "./providers/asksage"
import { BasetenHandler } from "./providers/baseten"
import { AwsBedrockHandler } from "./providers/bedrock"
import { CerebrasHandler } from "./providers/cerebras"
import { ClaudeCodeHandler } from "./providers/claude-code"
import { ClineHandler } from "./providers/cline"
import { DeepSeekHandler } from "./providers/deepseek"
import { DifyHandler } from "./providers/dify"
import { DoubaoHandler } from "./providers/doubao"
import { FireworksHandler } from "./providers/fireworks"
import { GeminiHandler } from "./providers/gemini"
import { GroqHandler } from "./providers/groq"
import { HicapHandler } from "./providers/hicap"
import { HuaweiCloudMaaSHandler } from "./providers/huawei-cloud-maas"
import { HuggingFaceHandler } from "./providers/huggingface"
import { LiteLlmHandler } from "./providers/litellm"
import { LmStudioHandler } from "./providers/lmstudio"
import { MinimaxHandler } from "./providers/minimax"
import { MistralHandler } from "./providers/mistral"
import { MoonshotHandler } from "./providers/moonshot"
import { NebiusHandler } from "./providers/nebius"
import { NousResearchHandler } from "./providers/nousresearch"
import { OcaHandler } from "./providers/oca"
import { OllamaHandler } from "./providers/ollama"
import { OpenAiHandler } from "./providers/openai"
import { OpenAiCodexHandler } from "./providers/openai-codex"
import { OpenAiNativeHandler } from "./providers/openai-native"
import { OpenRouterHandler } from "./providers/openrouter"
import { QwenHandler } from "./providers/qwen"
import { QwenCodeHandler } from "./providers/qwen-code"
import { RequestyHandler } from "./providers/requesty"
import { SambanovaHandler } from "./providers/sambanova"
import { SapAiCoreHandler } from "./providers/sapaicore"
import { TogetherHandler } from "./providers/together"
import { VercelAIGatewayHandler } from "./providers/vercel-ai-gateway"
import { VertexHandler } from "./providers/vertex"
import { VsCodeLmHandler } from "./providers/vscode-lm"
import { WandbHandler } from "./providers/wandb"
import { XAIHandler } from "./providers/xai"
import { ZAiHandler } from "./providers/zai"
import { ApiStream, ApiStreamUsageChunk } from "./transform/stream"
export type CommonApiHandlerOptions = {
onRetryAttempt?: ApiConfiguration["onRetryAttempt"]
}
export interface ApiHandler {
createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ClineTool[], useResponseApi?: boolean): ApiStream
getModel(): ApiHandlerModel
getApiStreamUsage?(): Promise<ApiStreamUsageChunk | undefined>
abort?(): void
}
// buildApiHandler now routes inference through the Cline SDK. It lives in
// apps/vscode/src/sdk/sdk-api-handler.ts and callers import it directly from
// there. It is deliberately NOT re-exported here: this barrel is imported
// widely for *types* only, and re-exporting a value from the SDK module would
// pull the entire SDK/session-factory runtime graph into every type importer
// at module-eval time (which can break extension activation). Keep this file
// types-only.
export interface ApiHandlerModel {
id: string
info: ModelInfo
providerId?: string
}
export interface ApiProviderInfo {
@@ -68,440 +21,3 @@ export interface ApiProviderInfo {
mode: Mode
customPrompt?: string // "compact"
}
export interface SingleCompletionHandler {
completePrompt(prompt: string): Promise<string>
}
function createHandlerForProvider(
apiProvider: string | undefined,
options: Omit<ApiConfiguration, "apiProvider">,
mode: Mode,
): ApiHandler {
switch (apiProvider) {
case "anthropic":
return new AnthropicHandler({
onRetryAttempt: options.onRetryAttempt,
apiKey: options.apiKey,
anthropicBaseUrl: options.anthropicBaseUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "openrouter":
return new OpenRouterHandler({
onRetryAttempt: options.onRetryAttempt,
openRouterApiKey: options.openRouterApiKey,
openRouterModelId: mode === "plan" ? options.planModeOpenRouterModelId : options.actModeOpenRouterModelId,
openRouterModelInfo: mode === "plan" ? options.planModeOpenRouterModelInfo : options.actModeOpenRouterModelInfo,
openRouterProviderSorting: options.openRouterProviderSorting,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
enableParallelToolCalling: options.enableParallelToolCalling,
})
case "bedrock":
return new AwsBedrockHandler({
onRetryAttempt: options.onRetryAttempt,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
awsAccessKey: options.awsAccessKey,
awsSecretKey: options.awsSecretKey,
awsSessionToken: options.awsSessionToken,
awsRegion: options.awsRegion,
awsAuthentication: options.awsAuthentication,
awsBedrockApiKey: options.awsBedrockApiKey,
awsUseCrossRegionInference: options.awsUseCrossRegionInference,
awsUseGlobalInference: options.awsUseGlobalInference,
awsBedrockUsePromptCache: options.awsBedrockUsePromptCache,
awsUseProfile: options.awsUseProfile,
awsProfile: options.awsProfile,
awsBedrockEndpoint: options.awsBedrockEndpoint,
awsBedrockCustomSelected:
mode === "plan" ? options.planModeAwsBedrockCustomSelected : options.actModeAwsBedrockCustomSelected,
awsBedrockCustomModelBaseId:
mode === "plan" ? options.planModeAwsBedrockCustomModelBaseId : options.actModeAwsBedrockCustomModelBaseId,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "vertex":
return new VertexHandler({
onRetryAttempt: options.onRetryAttempt,
vertexProjectId: options.vertexProjectId,
vertexRegion: options.vertexRegion,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
geminiApiKey: options.geminiApiKey,
geminiBaseUrl: options.geminiBaseUrl,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
ulid: options.ulid,
})
case "openai":
return new OpenAiHandler({
onRetryAttempt: options.onRetryAttempt,
openAiApiKey: options.openAiApiKey,
openAiBaseUrl: options.openAiBaseUrl,
azureApiVersion: options.azureApiVersion,
azureIdentity: options.azureIdentity,
openAiHeaders: options.openAiHeaders,
openAiModelId: mode === "plan" ? options.planModeOpenAiModelId : options.actModeOpenAiModelId,
openAiModelInfo: mode === "plan" ? options.planModeOpenAiModelInfo : options.actModeOpenAiModelInfo,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
})
case "ollama":
return new OllamaHandler({
onRetryAttempt: options.onRetryAttempt,
ollamaBaseUrl: options.ollamaBaseUrl,
ollamaApiKey: options.ollamaApiKey,
ollamaModelId: mode === "plan" ? options.planModeOllamaModelId : options.actModeOllamaModelId,
ollamaApiOptionsCtxNum: options.ollamaApiOptionsCtxNum,
requestTimeoutMs: options.requestTimeoutMs,
})
case "lmstudio":
return new LmStudioHandler({
onRetryAttempt: options.onRetryAttempt,
lmStudioBaseUrl: options.lmStudioBaseUrl,
lmStudioModelId: mode === "plan" ? options.planModeLmStudioModelId : options.actModeLmStudioModelId,
lmStudioMaxTokens: options.lmStudioMaxTokens,
})
case "gemini":
return new GeminiHandler({
onRetryAttempt: options.onRetryAttempt,
vertexProjectId: options.vertexProjectId,
vertexRegion: options.vertexRegion,
geminiApiKey: options.geminiApiKey,
geminiBaseUrl: options.geminiBaseUrl,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
ulid: options.ulid,
})
case "openai-native":
return new OpenAiNativeHandler({
onRetryAttempt: options.onRetryAttempt,
openAiNativeApiKey: options.openAiNativeApiKey,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "openai-codex":
return new OpenAiCodexHandler({
onRetryAttempt: options.onRetryAttempt,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "deepseek":
return new DeepSeekHandler({
onRetryAttempt: options.onRetryAttempt,
deepSeekApiKey: options.deepSeekApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "requesty":
return new RequestyHandler({
onRetryAttempt: options.onRetryAttempt,
requestyBaseUrl: options.requestyBaseUrl,
requestyApiKey: options.requestyApiKey,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
requestyModelId: mode === "plan" ? options.planModeRequestyModelId : options.actModeRequestyModelId,
requestyModelInfo: mode === "plan" ? options.planModeRequestyModelInfo : options.actModeRequestyModelInfo,
})
case "fireworks":
return new FireworksHandler({
onRetryAttempt: options.onRetryAttempt,
fireworksApiKey: options.fireworksApiKey,
fireworksModelId: mode === "plan" ? options.planModeFireworksModelId : options.actModeFireworksModelId,
})
case "together":
return new TogetherHandler({
onRetryAttempt: options.onRetryAttempt,
togetherApiKey: options.togetherApiKey,
togetherModelId: mode === "plan" ? options.planModeTogetherModelId : options.actModeTogetherModelId,
})
case "qwen":
return new QwenHandler({
onRetryAttempt: options.onRetryAttempt,
qwenApiKey: options.qwenApiKey,
qwenApiLine:
options.qwenApiLine === QwenApiRegions.INTERNATIONAL ? QwenApiRegions.INTERNATIONAL : QwenApiRegions.CHINA,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "qwen-code":
return new QwenCodeHandler({
onRetryAttempt: options.onRetryAttempt,
qwenCodeOauthPath: options.qwenCodeOauthPath,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "doubao":
return new DoubaoHandler({
onRetryAttempt: options.onRetryAttempt,
doubaoApiKey: options.doubaoApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "mistral":
return new MistralHandler({
onRetryAttempt: options.onRetryAttempt,
mistralApiKey: options.mistralApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "vscode-lm":
return new VsCodeLmHandler({
onRetryAttempt: options.onRetryAttempt,
vsCodeLmModelSelector:
mode === "plan" ? options.planModeVsCodeLmModelSelector : options.actModeVsCodeLmModelSelector,
})
case "cline": {
const clineModelId =
(mode === "plan" ? options.planModeClineModelId : options.actModeClineModelId) ||
(mode === "plan" ? options.planModeOpenRouterModelId : options.actModeOpenRouterModelId)
const clineModelInfo =
(mode === "plan" ? options.planModeClineModelInfo : options.actModeClineModelInfo) ||
(mode === "plan" ? options.planModeOpenRouterModelInfo : options.actModeOpenRouterModelInfo)
return new ClineHandler({
onRetryAttempt: options.onRetryAttempt,
clineAccountId: options.clineAccountId,
clineApiKey: options.clineApiKey,
ulid: options.ulid,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
openRouterProviderSorting: options.openRouterProviderSorting,
openRouterModelId: clineModelId,
openRouterModelInfo: clineModelInfo,
enableParallelToolCalling: options.enableParallelToolCalling,
})
}
case "litellm":
return new LiteLlmHandler({
onRetryAttempt: options.onRetryAttempt,
liteLlmApiKey: options.liteLlmApiKey,
liteLlmBaseUrl: options.liteLlmBaseUrl,
liteLlmModelId: mode === "plan" ? options.planModeLiteLlmModelId : options.actModeLiteLlmModelId,
liteLlmModelInfo: mode === "plan" ? options.planModeLiteLlmModelInfo : options.actModeLiteLlmModelInfo,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
liteLlmUsePromptCache: options.liteLlmUsePromptCache,
ulid: options.ulid,
})
case "moonshot":
return new MoonshotHandler({
onRetryAttempt: options.onRetryAttempt,
moonshotApiKey: options.moonshotApiKey,
moonshotApiLine: options.moonshotApiLine,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "huggingface":
return new HuggingFaceHandler({
onRetryAttempt: options.onRetryAttempt,
huggingFaceApiKey: options.huggingFaceApiKey,
huggingFaceModelId: mode === "plan" ? options.planModeHuggingFaceModelId : options.actModeHuggingFaceModelId,
huggingFaceModelInfo:
mode === "plan" ? options.planModeHuggingFaceModelInfo : options.actModeHuggingFaceModelInfo,
})
case "nebius":
return new NebiusHandler({
onRetryAttempt: options.onRetryAttempt,
nebiusApiKey: options.nebiusApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "asksage":
return new AskSageHandler({
onRetryAttempt: options.onRetryAttempt,
asksageApiKey: options.asksageApiKey,
asksageApiUrl: options.asksageApiUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "xai":
return new XAIHandler({
onRetryAttempt: options.onRetryAttempt,
xaiApiKey: options.xaiApiKey,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "sambanova":
return new SambanovaHandler({
onRetryAttempt: options.onRetryAttempt,
sambanovaApiKey: options.sambanovaApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "cerebras":
return new CerebrasHandler({
onRetryAttempt: options.onRetryAttempt,
cerebrasApiKey: options.cerebrasApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "groq":
return new GroqHandler({
onRetryAttempt: options.onRetryAttempt,
groqApiKey: options.groqApiKey,
groqModelId: mode === "plan" ? options.planModeGroqModelId : options.actModeGroqModelId,
groqModelInfo: mode === "plan" ? options.planModeGroqModelInfo : options.actModeGroqModelInfo,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "baseten":
return new BasetenHandler({
onRetryAttempt: options.onRetryAttempt,
basetenApiKey: options.basetenApiKey,
basetenModelId: mode === "plan" ? options.planModeBasetenModelId : options.actModeBasetenModelId,
basetenModelInfo: mode === "plan" ? options.planModeBasetenModelInfo : options.actModeBasetenModelInfo,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "sapaicore":
return new SapAiCoreHandler({
onRetryAttempt: options.onRetryAttempt,
sapAiCoreClientId: options.sapAiCoreClientId,
sapAiCoreClientSecret: options.sapAiCoreClientSecret,
sapAiCoreTokenUrl: options.sapAiCoreTokenUrl,
sapAiResourceGroup: options.sapAiResourceGroup,
sapAiCoreBaseUrl: options.sapAiCoreBaseUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
deploymentId: mode === "plan" ? options.planModeSapAiCoreDeploymentId : options.actModeSapAiCoreDeploymentId,
sapAiCoreUseOrchestrationMode: options.sapAiCoreUseOrchestrationMode,
})
case "claude-code":
return new ClaudeCodeHandler({
onRetryAttempt: options.onRetryAttempt,
claudeCodePath: options.claudeCodePath,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "huawei-cloud-maas":
return new HuaweiCloudMaaSHandler({
onRetryAttempt: options.onRetryAttempt,
huaweiCloudMaasApiKey: options.huaweiCloudMaasApiKey,
huaweiCloudMaasModelId:
mode === "plan" ? options.planModeHuaweiCloudMaasModelId : options.actModeHuaweiCloudMaasModelId,
huaweiCloudMaasModelInfo:
mode === "plan" ? options.planModeHuaweiCloudMaasModelInfo : options.actModeHuaweiCloudMaasModelInfo,
})
case "dify": // Add Dify.ai handler
return new DifyHandler({
difyApiKey: options.difyApiKey,
difyBaseUrl: options.difyBaseUrl,
})
case "vercel-ai-gateway":
return new VercelAIGatewayHandler({
onRetryAttempt: options.onRetryAttempt,
vercelAiGatewayApiKey: options.vercelAiGatewayApiKey,
openRouterModelId:
mode === "plan" ? options.planModeVercelAiGatewayModelId : options.actModeVercelAiGatewayModelId,
openRouterModelInfo:
mode === "plan" ? options.planModeVercelAiGatewayModelInfo : options.actModeVercelAiGatewayModelInfo,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "zai":
return new ZAiHandler({
onRetryAttempt: options.onRetryAttempt,
zaiApiLine: options.zaiApiLine,
zaiApiKey: options.zaiApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "oca":
return new OcaHandler({
ocaMode: options.ocaMode || "internal",
ocaBaseUrl: options.ocaBaseUrl,
ocaModelId: mode === "plan" ? options.planModeOcaModelId : options.actModeOcaModelId,
ocaModelInfo: mode === "plan" ? options.planModeOcaModelInfo : options.actModeOcaModelInfo,
ocaReasoningEffort: mode === "plan" ? options.planModeOcaReasoningEffort : options.actModeOcaReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
ocaUsePromptCache:
mode === "plan"
? options.planModeOcaModelInfo?.supportsPromptCache
: options.actModeOcaModelInfo?.supportsPromptCache,
taskId: options.ulid,
})
case "aihubmix":
return new AIhubmixHandler({
onRetryAttempt: options.onRetryAttempt,
apiKey: options.aihubmixApiKey,
baseURL: options.aihubmixBaseUrl,
appCode: options.aihubmixAppCode,
modelId: mode === "plan" ? (options as any).planModeAihubmixModelId : (options as any).actModeAihubmixModelId,
modelInfo:
mode === "plan" ? (options as any).planModeAihubmixModelInfo : (options as any).actModeAihubmixModelInfo,
})
case "minimax":
return new MinimaxHandler({
onRetryAttempt: options.onRetryAttempt,
minimaxApiKey: options.minimaxApiKey,
minimaxApiLine: options.minimaxApiLine,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "hicap":
return new HicapHandler({
onRetryAttempt: options.onRetryAttempt,
hicapApiKey: options.hicapApiKey,
hicapModelId: mode === "plan" ? options.planModeHicapModelId : options.actModeHicapModelId,
})
case "nousResearch":
return new NousResearchHandler({
onRetryAttempt: options.onRetryAttempt,
nousResearchApiKey: options.nousResearchApiKey,
apiModelId: mode === "plan" ? options.planModeNousResearchModelId : options.actModeNousResearchModelId,
})
case "wandb":
return new WandbHandler({
onRetryAttempt: options.onRetryAttempt,
wandbApiKey: options.wandbApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
default:
return new AnthropicHandler({
onRetryAttempt: options.onRetryAttempt,
apiKey: options.apiKey,
anthropicBaseUrl: options.anthropicBaseUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
}
}
export function buildApiHandler(configuration: ApiConfiguration, mode: Mode): ApiHandler {
const { planModeApiProvider, actModeApiProvider, ...options } = configuration
const apiProvider = mode === "plan" ? planModeApiProvider : actModeApiProvider
// Validate thinking budget tokens against model's maxTokens to prevent API errors
// wrapped in a try-catch for safety, but this should never throw
try {
const thinkingBudgetTokens = mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens
if (thinkingBudgetTokens && thinkingBudgetTokens > 0) {
const handler = createHandlerForProvider(apiProvider, options, mode)
const modelInfo = handler.getModel().info
if (modelInfo?.maxTokens && modelInfo.maxTokens > 0 && thinkingBudgetTokens > modelInfo.maxTokens) {
const clippedValue = modelInfo.maxTokens - 1
if (mode === "plan") {
options.planModeThinkingBudgetTokens = clippedValue
} else {
options.actModeThinkingBudgetTokens = clippedValue
}
} else {
return handler // don't rebuild unless its necessary
}
}
} catch (error) {
Logger.error("buildApiHandler error:", error)
}
return createHandlerForProvider(apiProvider, options, mode)
}
@@ -1,236 +0,0 @@
import { afterEach, describe, it } from "mocha"
import sinon from "sinon"
import "should"
import { anthropicModels } from "@shared/api"
import { ANTHROPIC_FAST_MODE_BETA, AnthropicHandler } from "../anthropic"
describe("AnthropicHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: readonly unknown[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
describe("getModel", () => {
it("should return the fast mode model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:fast",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-6:fast")
result.info.should.deepEqual(anthropicModels["claude-opus-4-6:fast"])
})
it("should return the 1m fast mode model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:1m:fast",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-6:1m:fast")
result.info.should.deepEqual(anthropicModels["claude-opus-4-6:1m:fast"])
})
it("should return the 4.7 model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-7")
result.info.should.deepEqual(anthropicModels["claude-opus-4-7"])
})
it("should return the 4.7 1m model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7:1m",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-7:1m")
result.info.should.deepEqual(anthropicModels["claude-opus-4-7:1m"])
})
it("should return the 4.8 model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-8",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-8")
result.info.should.deepEqual(anthropicModels["claude-opus-4-8"])
})
it("should return the 4.8 1m model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-8:1m",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-8:1m")
result.info.should.deepEqual(anthropicModels["claude-opus-4-8:1m"])
})
})
describe("createMessage", () => {
it("should route fast mode requests through the beta messages API", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:fast",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
const betaCreate = sinon.stub().callsFake(function (this: { _client?: object }, _params: unknown) {
should.exist(this._client)
return Promise.resolve(createAsyncIterable())
})
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: betaCreate,
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.notCalled(standardCreate)
sinon.assert.calledOnce(betaCreate)
sinon.assert.calledWithMatch(betaCreate, {
model: "claude-opus-4-6",
betas: [ANTHROPIC_FAST_MODE_BETA],
speed: "fast",
stream: true,
})
})
it("should include the 1m beta when routing 1m fast mode requests through the beta messages API", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:1m:fast",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
const betaCreate = sinon.stub().callsFake(function (this: { _client?: object }, _params: unknown) {
should.exist(this._client)
return Promise.resolve(createAsyncIterable())
})
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: betaCreate,
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.notCalled(standardCreate)
sinon.assert.calledOnce(betaCreate)
sinon.assert.calledWithMatch(betaCreate, {
model: "claude-opus-4-6",
betas: [ANTHROPIC_FAST_MODE_BETA, "context-1m-2025-08-07"],
speed: "fast",
stream: true,
})
})
it("should include the 1m beta header for Claude Opus 4.7 1m requests", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7:1m",
reasoningEffort: "high",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: sinon.stub().resolves(createAsyncIterable()),
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.calledOnce(standardCreate)
const requestBody = standardCreate.firstCall.args[0] as Record<string, any>
const requestOptions = standardCreate.firstCall.args[1] as Record<string, any>
requestBody.model.should.equal("claude-opus-4-7")
requestBody.thinking.should.deepEqual({ type: "adaptive" })
requestOptions.should.deepEqual({
headers: {
"anthropic-beta": "context-1m-2025-08-07",
},
})
})
it("should use adaptive thinking and output_config for Claude Opus adaptive models", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7",
reasoningEffort: "xhigh",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: sinon.stub().resolves(createAsyncIterable()),
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.calledOnce(standardCreate)
const requestBody = standardCreate.firstCall.args[0] as Record<string, any>
requestBody.should.have.property("thinking")
requestBody.thinking.should.deepEqual({ type: "adaptive" })
requestBody.should.have.property("output_config")
requestBody.output_config.should.deepEqual({ effort: "xhigh" })
should(requestBody.temperature).equal(undefined)
})
})
})
File diff suppressed because it is too large Load Diff
@@ -1,468 +0,0 @@
import { afterEach, beforeEach, describe, it } from "mocha"
import sinon from "sinon"
import "should"
import { ClaudeCodeHandler } from "@core/api/providers/claude-code"
import { ClineStorageMessage } from "@/shared/messages/content"
describe("ClaudeCodeHandler", () => {
let handler: ClaudeCodeHandler
let sandbox: sinon.SinonSandbox
beforeEach(() => {
sandbox = sinon.createSandbox()
handler = new ClaudeCodeHandler({
claudeCodePath: "/mock/path",
apiModelId: "claude-opus-4-1-20250805",
})
})
afterEach(() => {
sandbox.restore()
})
describe("token counting", () => {
it("should correctly handle token usage from assistant messages", async () => {
// The 'input_tokens' field represents the TOTAL number of input tokens used.
// See https://docs.anthropic.com/en/api/messages#usage-object
// Mock the runClaudeCode function
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
// Create a proper async generator mock for the Claude Code response
async function* mockGenerator() {
// First yield the system init
yield {
type: "system",
subtype: "init",
apiKeySource: "api",
}
// Yield assistant message with usage data
// Example: If base input is 70 tokens, cache read is 20, and cache creation is 10,
// then input_tokens from Anthropic API will be 100 (70 + 20 + 10)
yield {
type: "assistant",
message: {
content: [
{
type: "text",
text: "Test response",
},
],
usage: {
input_tokens: 100, // Total including cache (per Anthropic docs)
output_tokens: 50,
cache_read_input_tokens: 20, // Already included in input_tokens
cache_creation_input_tokens: 10, // Already included in input_tokens
},
stop_reason: "end_turn",
},
}
// Yield result with cost
yield {
type: "result",
result: {},
total_cost_usd: 0.005,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const usageData: any[] = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "usage") {
usageData.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
cacheReadTokens: chunk.cacheReadTokens,
cacheWriteTokens: chunk.cacheWriteTokens,
totalCost: chunk.totalCost,
})
}
}
// Verify token counting follows Anthropic API specification
usageData.should.have.length(1)
usageData[0].should.deepEqual({
inputTokens: 100, // Total including cache tokens (per Anthropic API docs)
outputTokens: 50,
cacheReadTokens: 20, // Tracked separately for reporting
cacheWriteTokens: 10, // Tracked separately for reporting
totalCost: 0.005,
})
// CRITICAL ASSERTION: Verify that input_tokens is NOT inflated by re-adding cache tokens
// The bug would have caused inputTokens to be incorrectly calculated as 130 (100 + 20 + 10)
// The fix ensures it remains 100, as per Anthropic's specification
usageData[0].inputTokens.should.equal(100) // Correct: matches API response
usageData[0].inputTokens.should.not.equal(130) // Would be wrong: double-counting cache tokens
})
it("should handle missing usage fields with nullish coalescing", async () => {
// Mock the runClaudeCode function
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
// Create a proper async generator mock with missing/undefined usage fields
async function* mockGenerator() {
yield {
type: "assistant",
message: {
content: [
{
type: "text",
text: "Test response",
},
],
usage: {
input_tokens: 100,
output_tokens: 50,
// cache fields are undefined/missing
},
stop_reason: "end_turn",
},
}
yield {
type: "result",
result: {},
total_cost_usd: 0.005,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const usageData: any[] = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "usage") {
usageData.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
cacheReadTokens: chunk.cacheReadTokens,
cacheWriteTokens: chunk.cacheWriteTokens,
})
}
}
// Verify that undefined cache tokens default to 0
usageData.should.have.length(1)
usageData[0].should.deepEqual({
inputTokens: 100,
outputTokens: 50,
cacheReadTokens: 0, // Should default to 0
cacheWriteTokens: 0, // Should default to 0
})
})
it("should handle completely missing usage object", async () => {
// Mock the runClaudeCode function
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
// Create a proper async generator mock with missing usage object
async function* mockGenerator() {
yield {
type: "assistant",
message: {
content: [
{
type: "text",
text: "Test response",
},
],
// usage is undefined
usage: undefined,
stop_reason: "end_turn",
},
}
// Need to yield a result chunk to trigger usage data emission
yield {
type: "result",
result: {},
total_cost_usd: 0,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const usageData: any[] = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "usage") {
usageData.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
cacheReadTokens: chunk.cacheReadTokens,
cacheWriteTokens: chunk.cacheWriteTokens,
})
}
}
// All token counts should default to 0 when usage is undefined
usageData.should.have.length(1)
usageData[0].should.deepEqual({
inputTokens: 0,
outputTokens: 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
})
})
})
describe("error handling", () => {
it("should not crash when assistant message has empty content array", async () => {
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
async function* mockGenerator() {
yield {
type: "assistant",
message: {
content: [], // empty content — triggered TypeError in older code
usage: {
input_tokens: 10,
output_tokens: 0,
},
stop_reason: "end_turn",
},
}
yield {
type: "result",
result: {},
total_cost_usd: 0,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const chunks: any[] = []
// Should not throw
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
const usageChunk = chunks.find((c) => c.type === "usage")
usageChunk.should.be.ok()
usageChunk.inputTokens.should.equal(10)
})
it("should throw when result has is_error=true (e.g. rate limit with no assistant message)", async () => {
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
async function* mockGenerator() {
yield {
type: "system",
subtype: "init",
apiKeySource: "none",
}
yield {
type: "system",
subtype: "rate_limit_event",
message: "Rate limit hit",
retryAfterSeconds: 30,
}
// No assistant message — CLI hit rate limit and gave up
yield {
type: "result",
subtype: "error",
is_error: true,
result: "Rate limit exceeded",
total_cost_usd: 0,
duration_ms: 1000,
duration_api_ms: 500,
num_turns: 0,
session_id: "test",
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
let thrownError: Error | undefined
try {
for await (const _ of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
// consume
}
} catch (err) {
thrownError = err as Error
}
thrownError!.message.should.containEql("Rate limit exceeded")
})
it("should ignore rate_limit_event system messages without throwing", async () => {
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
async function* mockGenerator() {
yield {
type: "system",
subtype: "init",
apiKeySource: "none",
}
// Newer Claude Code CLI emits this during rate limiting
yield {
type: "system",
subtype: "rate_limit_event",
message: "Rate limit hit, retrying...",
retryAfterSeconds: 30,
}
yield {
type: "assistant",
message: {
content: [{ type: "text", text: "Response after retry" }],
usage: { input_tokens: 20, output_tokens: 10 },
stop_reason: "end_turn",
},
}
yield {
type: "result",
result: {},
total_cost_usd: 0,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const textChunks: string[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
if (chunk.type === "text") textChunks.push(chunk.text)
}
textChunks.should.deepEqual(["Response after retry"])
})
})
describe("getModel", () => {
it("should return the correct model when specified", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-sonnet-4-5-20250929",
})
const model = handler.getModel()
model.id.should.equal("claude-sonnet-4-5-20250929")
})
it("should support Opus 4.6 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-6[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-6[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Opus 4.7 model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-7",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-7")
model.info.contextWindow.should.equal(200_000)
})
it("should support Opus 4.7 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-7[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-7[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Opus 4.8 model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-8",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-8")
model.info.contextWindow.should.equal(200_000)
})
it("should support Opus 4.8 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-8[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-8[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Opus 1m alias model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "opus[1m]",
})
const model = handler.getModel()
model.id.should.equal("opus[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Sonnet 1m alias model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "sonnet[1m]",
})
const model = handler.getModel()
model.id.should.equal("sonnet[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Sonnet 4.5 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-sonnet-4-5-20250929[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-sonnet-4-5-20250929[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Sonnet 4.6 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-sonnet-4-6[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-sonnet-4-6[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should return default model when not specified", () => {
const handler = new ClaudeCodeHandler({})
const model = handler.getModel()
// The default model should be set
model.id.should.be.type("string")
model.info.should.be.type("object")
})
})
})
@@ -1,166 +0,0 @@
import "should"
import { openRouterDefaultModelInfo } from "@shared/api"
import sinon from "sinon"
import { ClineAccountService } from "@/services/account/ClineAccountService"
import { AuthService } from "@/services/auth/AuthService"
import { ClineHandler } from "../cline"
describe("ClineHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
const createHandler = (options: ConstructorParameters<typeof ClineHandler>[0]) => {
sinon.stub(ClineAccountService, "getInstance").returns({} as any)
sinon.stub(AuthService, "getInstance").returns({} as any)
return new ClineHandler(options)
}
it("should handle usage-only chunks when delta is missing", async () => {
const handler = createHandler({})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 17,
completion_tokens: 9,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "openai/gpt-4o-mini",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: 0,
inputTokens: 17,
outputTokens: 9,
totalCost: 0,
},
])
})
it("should read Anthropic-style cache creation and read tokens from usage chunks", async () => {
const handler = createHandler({})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 1000,
completion_tokens: 200,
prompt_tokens_details: {
cached_tokens: 500,
},
cache_creation_input_tokens: 300,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "anthropic/claude-sonnet-4.6",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 300,
cacheReadTokens: 500,
inputTokens: 200,
outputTokens: 200,
totalCost: 0,
},
])
})
it("should forward enableParallelToolCalling to OpenRouter payload", async () => {
const handler = createHandler({ enableParallelToolCalling: true })
const createStub = sinon.stub().resolves(createAsyncIterable([]))
const fakeClient = {
chat: {
completions: {
create: createStub,
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "openai/gpt-4o-mini",
info: openRouterDefaultModelInfo,
})
const tools = [
{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } },
] as any
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
// drain stream
}
const payload = createStub.firstCall.args[0]
payload.parallel_tool_calls.should.equal(true)
})
it("should send cache_control for qwen3.7-max without changing the selected Cline model id", async () => {
const handler = createHandler({
openRouterModelId: "qwen/qwen3.7-max",
openRouterModelInfo: openRouterDefaultModelInfo,
})
const createStub = sinon.stub().resolves(createAsyncIterable([]))
const fakeClient = {
chat: {
completions: {
create: createStub,
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler as any, "getFreeModelIdSet").resolves(new Set())
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
// drain stream
}
handler.getModel().id.should.equal("qwen/qwen3.7-max")
const payload = createStub.firstCall.args[0]
payload.model.should.equal("qwen/qwen3.7-max")
payload.messages[0].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
payload.messages[1].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
})
})
@@ -1,97 +0,0 @@
import "should"
import sinon from "sinon"
import { FireworksHandler } from "../fireworks"
describe("FireworksHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
it("should handle usage-only chunks when delta is missing", async () => {
const handler = new FireworksHandler({
fireworksApiKey: "test-api-key",
fireworksModelId: "accounts/fireworks/models/llama-v3p1-8b-instruct",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 19,
completion_tokens: 4,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
inputTokens: 19,
outputTokens: 4,
cacheReadTokens: 0,
cacheWriteTokens: 0,
},
])
})
it("should read cache hits from prompt_tokens_details when hit tokens are not present", async () => {
const handler = new FireworksHandler({
fireworksApiKey: "test-api-key",
fireworksModelId: "accounts/fireworks/models/llama-v3p1-8b-instruct",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 60,
completion_tokens: 12,
prompt_tokens_details: { cached_tokens: 20 },
prompt_cache_miss_tokens: 40,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
inputTokens: 60,
outputTokens: 12,
cacheReadTokens: 20,
cacheWriteTokens: 40,
},
])
})
})
@@ -1,235 +0,0 @@
import "should"
import sinon from "sinon"
import { GeminiHandler } from "../gemini"
describe("GeminiHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
it("caps maxOutputTokens to 8192 for Flash models", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
apiModelId: "gemini-2.5-flash",
})
const generateContentStream = sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp-1",
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 20,
cachedContentTokenCount: 0,
thoughtsTokenCount: 0,
},
},
]),
)
sinon.stub(handler as any, "ensureClient").returns({
models: { generateContentStream },
} as any)
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
// Consume stream to trigger request execution.
}
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
requestArgs.config.should.have.property("maxOutputTokens", 8_192)
})
it("supports Gemini 3.5 Flash model metadata", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
apiModelId: "gemini-3.5-flash",
})
const model = handler.getModel()
model.id.should.equal("gemini-3.5-flash")
model.info.contextWindow!.should.equal(1_048_576)
model.info.inputPrice!.should.equal(1.5)
model.info.outputPrice!.should.equal(9)
model.info.cacheReadsPrice!.should.equal(0.15)
model.info.supportsReasoning!.should.equal(true)
const generateContentStream = sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp-35",
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 20,
cachedContentTokenCount: 0,
thoughtsTokenCount: 0,
},
},
]),
)
sinon.stub(handler as any, "ensureClient").returns({
models: { generateContentStream },
} as any)
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
// Consume stream to trigger request execution.
}
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
requestArgs.model.should.equal("gemini-3.5-flash")
requestArgs.config.should.have.property("maxOutputTokens", 8_192)
requestArgs.config.thinkingConfig.should.deepEqual({
thinkingBudget: undefined,
thinkingLevel: "LOW",
includeThoughts: true,
})
})
it("does not set maxOutputTokens for non-Flash models", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
apiModelId: "gemini-2.5-pro",
})
const generateContentStream = sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp-2",
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 20,
cachedContentTokenCount: 0,
thoughtsTokenCount: 0,
},
},
]),
)
sinon.stub(handler as any, "ensureClient").returns({
models: { generateContentStream },
} as any)
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
// Consume stream to trigger request execution.
}
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
requestArgs.config.should.not.have.property("maxOutputTokens")
})
it("should emit unique tool call IDs when multiple function calls share one responseId", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
})
const fakeClient = {
models: {
generateContentStream: sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp_1",
candidates: [
{
content: {
parts: [
{
functionCall: {
name: "read_file",
args: { path: ".nvmrc" },
},
},
],
},
},
],
},
{
responseId: "resp_1",
candidates: [
{
content: {
parts: [
{
functionCall: {
name: "read_file",
args: { path: ".gitattributes" },
},
},
],
},
},
],
},
]),
),
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const tools = [{ name: "read_file", description: "read file", parameters: { type: "OBJECT" } }] as any
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
if (chunk.type === "tool_calls") {
chunks.push(chunk)
}
}
chunks.should.have.length(2)
chunks[0].tool_call.function.id.should.equal("resp_1-tool-0")
chunks[1].tool_call.function.id.should.equal("resp_1-tool-1")
chunks[0].tool_call.call_id.should.equal(chunks[0].tool_call.function.id)
chunks[1].tool_call.call_id.should.equal(chunks[1].tool_call.function.id)
JSON.parse(chunks[0].tool_call.function.arguments).path.should.equal(".nvmrc")
JSON.parse(chunks[1].tool_call.function.arguments).path.should.equal(".gitattributes")
})
it("should preserve Gemini-provided functionCall.id when present", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
})
const fakeClient = {
models: {
generateContentStream: sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp_2",
candidates: [
{
content: {
parts: [
{
functionCall: {
id: "call_alpha",
name: "read_file",
args: { path: ".nvmrc" },
},
},
],
},
},
],
},
]),
),
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const tools = [{ name: "read_file", description: "read file", parameters: { type: "OBJECT" } }] as any
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
if (chunk.type === "tool_calls") {
chunks.push(chunk)
}
}
chunks.should.have.length(1)
chunks[0].tool_call.function.id.should.equal("call_alpha")
chunks[0].tool_call.call_id.should.equal("call_alpha")
JSON.parse(chunks[0].tool_call.function.arguments).path.should.equal(".nvmrc")
})
})
@@ -1,326 +0,0 @@
import { LiteLlmHandler, type LiteLlmModelInfoResponse } from "@core/api/providers/litellm"
import { convertToOpenAiMessages } from "@core/api/transform/openai-format"
import { liteLlmModelInfoSaneDefaults } from "@shared/api" // used in getModel tests
import { expect } from "chai"
import sinon from "sinon"
import { StateManager } from "@/core/storage/StateManager" // used in getModel tests
import { ClineStorageMessage } from "@/shared/messages/content"
import { mockFetchForTesting } from "@/shared/net"
const fakeClient = {
chat: {
completions: {
create: sinon.stub(),
},
},
baseURL: "https://fake.example",
}
describe("LiteLlmHandler", () => {
const mockFetch = sinon.stub()
let doneMockingFetch: (value: any) => void = () => {}
const mockModelFetch = (modelInfo: LiteLlmModelInfoResponse["data"][number]) => {
mockFetch.resolves({
ok: true,
json: () =>
Promise.resolve({
data: [modelInfo],
}),
})
}
let handler: LiteLlmHandler
const mockHandlerChat = () => {
sinon.stub(handler, "ensureClient" as any).returns(fakeClient)
}
const initializeHandler = (model: string) => {
handler = new LiteLlmHandler({
liteLlmApiKey: "test-api-key",
liteLlmBaseUrl: "http://localhost:4000",
liteLlmUsePromptCache: true,
liteLlmModelId: model,
})
mockHandlerChat()
}
beforeEach(() => {
fakeClient.chat.completions.create.resetHistory()
mockFetchForTesting(mockFetch, () => {
return new Promise((resolve) => {
doneMockingFetch = resolve
})
})
// Configure the stub to return a stream that closes immediately with usage data
fakeClient.chat.completions.create.resolves(
createAsyncIterable([
{
choices: [{ delta: { content: "test response" } }],
},
{
choices: [{}],
usage: {
prompt_tokens: 100,
completion_tokens: 50,
cache_creation_input_tokens: 20,
cache_read_input_tokens: 10,
},
},
]),
)
})
afterEach(() => {
sinon.reset()
doneMockingFetch(void 0)
})
const createAsyncIterable = (data: any[] = []) => {
return {
[Symbol.asyncIterator]: async function* () {
yield* data
},
}
}
describe("prompt cache", () => {
const setModelData = (model: string, supportsPromptCaching: boolean) => {
mockModelFetch({
model_name: model,
litellm_params: {
model,
},
model_info: {
supports_prompt_caching: supportsPromptCaching,
input_cost_per_token: 0.01,
output_cost_per_token: 0.02,
},
})
}
describe("when the model doesn't support prompt caching", () => {
const model = "openai/gpt-5"
beforeEach(() => {
initializeHandler(model)
setModelData(model, false)
})
it("sends the system prompt and messages with the openai format", async () => {
const systemPrompt = "Test System Prompt"
const messages: ClineStorageMessage[] = [
{
role: "user",
content: "first message",
},
{
role: "assistant",
content: "first response",
},
{
role: "user",
content: [
{
type: "text",
text: "test",
},
{
type: "text",
text: "second message",
},
],
},
]
for await (const _ of handler.createMessage(systemPrompt, messages)) {
}
sinon.assert.calledOnce(fakeClient.chat.completions.create)
const callArgs = fakeClient.chat.completions.create.getCall(0).args[0]
const systemPromptMessage = callArgs.messages.shift()
expect(systemPromptMessage).to.deep.equal({
role: "system",
content: systemPrompt,
})
expect(callArgs.messages).to.deep.equal(convertToOpenAiMessages(messages))
})
})
describe("when the model supports prompt caching", () => {
const model = "anthropic/claude-sonnet-4-20250514"
beforeEach(() => {
initializeHandler(model)
setModelData(model, true)
})
it("inserts the cache control in the system prompt and the last two user messages", async () => {
const systemPrompt = "Test System Prompt"
const messages: ClineStorageMessage[] = [
{
role: "user",
content: "first message",
},
{
role: "assistant",
content: "first response",
},
{
role: "user",
content: [
{
type: "text",
text: "test",
},
{
type: "text",
text: "second message",
},
],
},
]
for await (const _ of handler.createMessage(systemPrompt, messages)) {
}
sinon.assert.calledOnce(fakeClient.chat.completions.create)
const callArgs = fakeClient.chat.completions.create.getCall(0).args[0]
expect(callArgs.messages[0]).to.deep.equal({
role: "system",
content: [
{
text: systemPrompt,
type: "text",
cache_control: {
type: "ephemeral",
},
},
],
})
const sentMessages = callArgs.messages
expect(sentMessages.length).to.equal(4)
const firstUserMessage = sentMessages[1]
expect(firstUserMessage).to.deep.equal({
role: "user",
content: [
{
type: "text",
text: "first message",
cache_control: {
type: "ephemeral",
},
},
],
})
const lastUserMessage = sentMessages[3]
expect(lastUserMessage.content[0]).to.deep.equal({
type: "text",
text: "test",
})
const lastContentBlock = lastUserMessage.content[lastUserMessage.content.length - 1]
expect(lastContentBlock).to.deep.equal({
type: "text",
text: "second message",
cache_control: {
type: "ephemeral",
},
})
expect(callArgs.model).to.be.a("string")
expect(callArgs.stream).to.equal(true)
expect(callArgs.stream_options).to.deep.equal({ include_usage: true })
})
})
})
describe("getModel", () => {
let stateManagerStub: sinon.SinonStub
beforeEach(() => {
stateManagerStub = sinon.stub(StateManager, "get").returns({
getModelInfo: () => null,
} as any)
})
afterEach(() => {
stateManagerStub.restore()
})
it("returns sane defaults when no liteLlmModelInfo option is provided", () => {
const h = new LiteLlmHandler({
liteLlmApiKey: "test",
liteLlmModelId: "some-model",
})
const model = h.getModel()
expect(model.id).to.equal("some-model")
expect(model.info.contextWindow).to.equal(liteLlmModelInfoSaneDefaults.contextWindow)
})
it("returns user-configured model info when liteLlmModelInfo is provided and no cache exists", () => {
const h = new LiteLlmHandler({
liteLlmApiKey: "test",
liteLlmModelId: "claude-sonnet-4-6",
liteLlmModelInfo: {
contextWindow: 1_000_000,
maxTokens: 8192,
supportsImages: true,
supportsPromptCache: true,
inputPrice: 3,
outputPrice: 15,
cacheWritesPrice: 3.75,
cacheReadsPrice: 0.3,
},
})
const model = h.getModel()
expect(model.id).to.equal("claude-sonnet-4-6")
expect(model.info.contextWindow).to.equal(1_000_000)
})
it("prefers StateManager cached model info over user-configured liteLlmModelInfo", () => {
const cachedInfo = {
contextWindow: 200_000,
maxTokens: 4096,
supportsImages: true,
supportsPromptCache: true,
inputPrice: 0,
outputPrice: 0,
}
stateManagerStub.returns({
getModelInfo: () => cachedInfo,
} as any)
const h = new LiteLlmHandler({
liteLlmApiKey: "test",
liteLlmModelId: "claude-sonnet-4-6",
liteLlmModelInfo: {
contextWindow: 1_000_000,
maxTokens: 8192,
supportsImages: true,
supportsPromptCache: true,
inputPrice: 3,
outputPrice: 15,
cacheWritesPrice: 3.75,
cacheReadsPrice: 0.3,
},
})
const model = h.getModel()
expect(model.info.contextWindow).to.equal(200_000)
})
})
})
@@ -1,53 +0,0 @@
import "should"
import { moonshotModels } from "@shared/api"
import type { ClineStorageMessage } from "@shared/messages/content"
import sinon from "sinon"
import { MoonshotHandler } from "../moonshot"
interface MoonshotRequestPayload {
model: string
temperature: number
max_tokens: number
}
describe("MoonshotHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: unknown[] = []): AsyncIterable<unknown> => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
it("supports kimi-k2.6 model metadata", async () => {
const handler = new MoonshotHandler({
moonshotApiKey: "test-api-key",
apiModelId: "kimi-k2.6",
})
const model = handler.getModel()
model.id.should.equal("kimi-k2.6")
model.info.should.deepEqual(moonshotModels["kimi-k2.6"])
const createStub = sinon.stub().resolves(createAsyncIterable([]))
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
chat: {
completions: {
create: createStub,
},
},
})
const messages: ClineStorageMessage[] = [{ role: "user", content: "hi" }]
for await (const _chunk of handler.createMessage("system", messages)) {
// Consume stream to trigger request execution.
}
const payload = createStub.firstCall.args[0] as MoonshotRequestPayload
payload.model.should.equal("kimi-k2.6")
payload.temperature.should.equal(moonshotModels["kimi-k2.6"].temperature)
payload.max_tokens.should.equal(moonshotModels["kimi-k2.6"].maxTokens)
})
})
@@ -1,93 +0,0 @@
import { expect } from "chai"
import { afterEach, describe, it } from "mocha"
import sinon from "sinon"
import { ClineStorageMessage } from "@/shared/messages/content"
import { ApiFormat } from "@/shared/proto/index.cline"
import { OcaHandler } from "../oca"
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
async function collectChunks(stream: AsyncGenerator<any>) {
const chunks: any[] = []
for await (const chunk of stream) {
chunks.push(chunk)
}
return chunks
}
describe("OcaHandler.createMessage", () => {
afterEach(() => {
sinon.restore()
})
it("routes OPENAI_RESPONSES models to createMessageResponsesApi", async () => {
const handler = new OcaHandler({
ocaModelInfo: { apiFormat: ApiFormat.OPENAI_RESPONSES } as any,
})
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
yield { type: "text", text: "chat" }
})
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
yield { type: "text", text: "responses" }
})
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
yield { type: "text", text: "messages" }
})
const chunks = await collectChunks(handler.createMessage("system", messages))
expect(chunks).to.deep.equal([{ type: "text", text: "responses" }])
sinon.assert.notCalled(chatStub)
sinon.assert.calledOnce(responsesStub)
sinon.assert.notCalled(messagesStub)
})
it("routes ANTHROPIC_CHAT models to createMessageMessagesApi", async () => {
const handler = new OcaHandler({
ocaModelInfo: { apiFormat: ApiFormat.ANTHROPIC_CHAT } as any,
})
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
yield { type: "text", text: "chat" }
})
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
yield { type: "text", text: "responses" }
})
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
yield { type: "text", text: "messages" }
})
const chunks = await collectChunks(handler.createMessage("system", messages))
expect(chunks).to.deep.equal([{ type: "text", text: "messages" }])
sinon.assert.notCalled(chatStub)
sinon.assert.notCalled(responsesStub)
sinon.assert.calledOnce(messagesStub)
})
it("defaults to createMessageChatApi for OPENAI_CHAT and undefined apiFormat", async () => {
for (const apiFormat of [ApiFormat.OPENAI_CHAT, undefined]) {
const handler = new OcaHandler({
ocaModelInfo: { apiFormat } as any,
})
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
yield { type: "text", text: "chat" }
})
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
yield { type: "text", text: "responses" }
})
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
yield { type: "text", text: "messages" }
})
const chunks = await collectChunks(handler.createMessage("system", messages))
expect(chunks).to.deep.equal([{ type: "text", text: "chat" }])
sinon.assert.calledOnce(chatStub)
sinon.assert.notCalled(responsesStub)
sinon.assert.notCalled(messagesStub)
}
})
})
@@ -1,231 +0,0 @@
import { afterEach, before, beforeEach, describe, it } from "mocha"
import "should"
import { ApiHandlerOptions } from "@shared/api"
import axios from "axios"
import sinon from "sinon"
import { ClineStorageMessage } from "@/shared/messages/content"
import { OllamaHandler } from "../ollama"
describe("OllamaHandler", () => {
let ollamaAvailable = false
// Check if Ollama is running before running tests
before(async function () {
this.timeout(5000)
try {
await axios.get("http://localhost:11434/api/version", { timeout: 2000 })
ollamaAvailable = true
} catch (_error) {
console.log("Ollama server not available, skipping tests")
ollamaAvailable = false
}
})
let handler: OllamaHandler
let options: ApiHandlerOptions
let clock: sinon.SinonFakeTimers
beforeEach(() => {
options = {
actModeOllamaModelId: "llama2",
ollamaBaseUrl: "http://localhost:11434",
}
handler = new OllamaHandler(options)
// Use fake timers for testing timeouts
clock = sinon.useFakeTimers()
})
afterEach(() => {
clock.restore()
sinon.restore()
})
describe("createMessage", () => {
it("should handle successful responses", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(5000)
// Ensure client is initialized
const client = (handler as any).ensureClient()
// Mock the Ollama client's chat method
const chatStub = sinon.stub(client, "chat").resolves({
[Symbol.asyncIterator]: async function* () {
yield {
message: { content: "Hello, world!" },
eval_count: 10,
prompt_eval_count: 20,
}
},
} as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const result = []
const usageInfo = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "text") {
result.push(chunk.text)
} else if (chunk.type === "usage") {
usageInfo.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
})
}
}
// Verify the results
result.should.deepEqual(["Hello, world!"])
usageInfo.should.deepEqual([{ inputTokens: 20, outputTokens: 10 }])
chatStub.calledOnce.should.be.true()
})
it("should handle timeout errors", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(10000)
// Restore real timers for this test
clock.restore()
// Create a handler with a very short timeout for testing
const testHandler = new OllamaHandler(options)
// Replace the createMessage method with one that has a shorter timeout
testHandler.createMessage = async function* (_systemPrompt, _messages) {
try {
// Create a promise that rejects after a short timeout
const timeoutPromise = new Promise<never>((_, reject) => {
setTimeout(() => reject(new Error("Ollama request timed out after 120 seconds")), 100)
})
// Create a promise that never resolves
const neverPromise = new Promise(() => {})
// Race them
await Promise.race([timeoutPromise, neverPromise])
} catch (error: any) {
// Enhance error reporting
console.error(`Ollama API error: ${error.message}`)
throw error
}
}
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
// Start the request and catch the error
let errorMessage = ""
try {
for await (const _ of testHandler.createMessage(systemPrompt, messages)) {
// This should not be reached
}
} catch (error: any) {
errorMessage = error.message
}
// Check the result
errorMessage.should.equal("Ollama request timed out after 120 seconds")
// Restore the fake timers for other tests
clock = sinon.useFakeTimers()
})
it("should retry on errors when using the withRetry decorator", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(10000)
// Restore real timers for this test
clock.restore()
// Ensure client is initialized and mock the Ollama client's chat method to fail on first call and succeed on second
const client = (handler as any).ensureClient()
const chatStub = sinon.stub(client, "chat")
// First call throws an error
chatStub.onFirstCall().rejects(new Error("API Error"))
// Second call succeeds
chatStub.onSecondCall().resolves({
[Symbol.asyncIterator]: async function* () {
yield {
message: { content: "Success after retry" },
}
},
} as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const result = []
// Add a small delay to ensure the retry mechanism has time to work
await new Promise((resolve) => setTimeout(resolve, 100))
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "text") {
result.push(chunk.text)
}
}
// Verify the results
result.should.deepEqual(["Success after retry"])
chatStub.calledTwice.should.be.true()
// Restore the fake timers for other tests
clock = sinon.useFakeTimers()
})
it("should handle stream processing errors", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(10000)
// Restore real timers for this test
clock.restore()
// Create a handler with a custom implementation for testing
const testHandler = new OllamaHandler(options)
// Replace the createMessage method with one that simulates a stream error
testHandler.createMessage = async function* (_systemPrompt, _messages) {
// First yield a successful chunk
yield {
type: "text",
text: "Partial response",
}
// Then throw an error in the stream
throw new Error("Ollama stream processing error: Stream error")
}
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const result = []
// Collect the results and catch the error
let errorMessage = ""
try {
for await (const chunk of testHandler.createMessage(systemPrompt, messages)) {
if (chunk.type === "text") {
result.push(chunk.text)
}
}
} catch (error: any) {
errorMessage = error.message
}
// Verify the results
errorMessage.should.equal("Ollama stream processing error: Stream error")
result.should.deepEqual(["Partial response"])
// Restore the fake timers for other tests
clock = sinon.useFakeTimers()
})
})
})
@@ -1,164 +0,0 @@
import "should"
import { openRouterDefaultModelInfo } from "@shared/api"
import sinon from "sinon"
import { OpenRouterHandler } from "../openrouter"
describe("OpenRouterHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
const tools = [{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } }] as any
it("should handle usage-only chunks when delta is missing", async () => {
const handler = new OpenRouterHandler({
openRouterApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 13,
completion_tokens: 5,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "openai/gpt-4o-mini",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: 0,
inputTokens: 13,
outputTokens: 5,
totalCost: 0,
},
])
})
it("should read cache_write_tokens from prompt_tokens_details", async () => {
const handler = new OpenRouterHandler({
openRouterApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 1000,
completion_tokens: 200,
prompt_tokens_details: {
cached_tokens: 500,
cache_write_tokens: 300,
},
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "anthropic/claude-sonnet-4.6",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 300,
cacheReadTokens: 500,
inputTokens: 200,
outputTokens: 200,
totalCost: 0,
},
])
})
type ParallelToolCallsTestCase = {
modelId: string
enableParallelToolCalling: boolean
expectedParallelToolCalls: boolean
}
const parallelToolCallsTestCases: ParallelToolCallsTestCase[] = [
{
modelId: "openai/gpt-4o-mini",
enableParallelToolCalling: true,
expectedParallelToolCalls: true,
},
{
modelId: "openai/gpt-4o-mini",
enableParallelToolCalling: false,
expectedParallelToolCalls: false,
},
{
modelId: "google/gemini-3-flash-preview",
enableParallelToolCalling: true,
expectedParallelToolCalls: true,
},
]
for (const testCase of parallelToolCallsTestCases) {
const settingLabel = testCase.enableParallelToolCalling ? "enabled" : "disabled"
it(`should set parallel_tool_calls=${testCase.expectedParallelToolCalls} for ${testCase.modelId} when setting is ${settingLabel}`, async () => {
const handler = new OpenRouterHandler({
openRouterApiKey: "test-api-key",
enableParallelToolCalling: testCase.enableParallelToolCalling,
})
const createStub = sinon.stub().resolves(createAsyncIterable([]))
const fakeClient = {
chat: {
completions: {
create: createStub,
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: testCase.modelId,
info: openRouterDefaultModelInfo,
})
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
// drain stream
}
const payload = createStub.firstCall.args[0]
payload.parallel_tool_calls.should.equal(testCase.expectedParallelToolCalls)
})
}
})
@@ -1,132 +0,0 @@
import "should"
import { Anthropic } from "@anthropic-ai/sdk"
import { SapAiCoreHandler } from "../sapaicore"
describe("SapAiCoreHandler", () => {
let handler: SapAiCoreHandler
beforeEach(() => {
const mockOptions = {
sapAiCoreClientId: "test-client-id",
sapAiCoreClientSecret: "test-client-secret",
sapAiCoreTokenUrl: "https://test.auth.sap.com",
sapAiResourceGroup: "default",
sapAiCoreBaseUrl: "https://test.api.sap.com",
apiModelId: "anthropic--claude-3.5-sonnet",
}
handler = new SapAiCoreHandler(mockOptions)
})
describe("image processing", () => {
// Test image processing through the public interface
// This tests the complete flow including processImageContent internally
it("should handle image processing for Claude 4 models", () => {
// Create handler with Claude 4 model
const claude4Handler = new SapAiCoreHandler({
sapAiCoreClientId: "test-client-id",
sapAiCoreClientSecret: "test-client-secret",
sapAiCoreTokenUrl: "https://test.auth.sap.com",
sapAiResourceGroup: "default",
sapAiCoreBaseUrl: "https://test.api.sap.com",
apiModelId: "anthropic--claude-4-sonnet",
})
const model = claude4Handler.getModel()
model.id.should.equal("anthropic--claude-4-sonnet")
model.info.should.have.property("supportsImages", true)
})
it("should create proper user readable request with images", () => {
const testImageData =
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=="
const userContent: (Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[] = [
{
type: "text",
text: "Here's an image:",
},
{
type: "image",
source: {
type: "base64",
media_type: "image/png",
data: testImageData,
},
},
]
const result = handler.createUserReadableRequest(userContent)
result.should.have.property("model")
result.should.have.property("max_tokens")
result.should.have.property("system")
result.should.have.property("messages")
result.messages.should.be.Array()
result.messages[1].should.have.property("role", "user")
result.messages[1].should.have.property("content", userContent)
})
it("should support different Claude model variants", () => {
const modelVariants = [
"anthropic--claude-4.6-sonnet",
"anthropic--claude-4-sonnet",
"anthropic--claude-4-opus",
"anthropic--claude-3.7-sonnet",
"anthropic--claude-3.5-sonnet",
"anthropic--claude-3-sonnet",
"anthropic--claude-3-haiku",
"anthropic--claude-3-opus",
]
modelVariants.forEach((modelId) => {
const testHandler = new SapAiCoreHandler({
apiModelId: modelId,
})
const model = testHandler.getModel()
model.id.should.equal(modelId)
model.info.should.have.property("maxTokens")
model.info.should.have.property("contextWindow")
})
})
})
describe("getModel", () => {
it("should return default model when no apiModelId is provided", () => {
const result = handler.getModel()
result.should.have.property("id")
result.should.have.property("info")
result.info.should.have.property("maxTokens")
})
it("should return specified model when apiModelId is provided", () => {
const customHandler = new SapAiCoreHandler({
apiModelId: "anthropic--claude-4-sonnet",
})
const result = customHandler.getModel()
result.id.should.equal("anthropic--claude-4-sonnet")
})
})
describe("createUserReadableRequest", () => {
it("should create a readable request format", () => {
const userContent: Anthropic.TextBlockParam[] = [
{
type: "text",
text: "Hello, world!",
},
]
const result = handler.createUserReadableRequest(userContent)
result.should.have.property("model")
result.should.have.property("max_tokens")
result.should.have.property("system")
result.should.have.property("messages")
result.should.have.property("tools")
result.should.have.property("tool_choice")
})
})
})
@@ -1,138 +0,0 @@
import "should"
import { openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import sinon from "sinon"
import { VercelAIGatewayHandler } from "../vercel-ai-gateway"
describe("VercelAIGatewayHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
describe("getModel", () => {
it("should return configured model and info when both are provided", () => {
const customModelInfo = {
...openRouterDefaultModelInfo,
maxTokens: 123456,
}
const handler = new VercelAIGatewayHandler({
openRouterModelId: "google/gemini-3.1-pro-preview",
openRouterModelInfo: customModelInfo,
})
const result = handler.getModel()
result.id.should.equal("google/gemini-3.1-pro-preview")
result.info.should.deepEqual(customModelInfo)
})
it("should preserve configured model ID when model info is missing", () => {
const handler = new VercelAIGatewayHandler({
openRouterModelId: "google/gemini-3.1-pro-preview",
})
const result = handler.getModel()
result.id.should.equal("google/gemini-3.1-pro-preview")
result.info.should.deepEqual(openRouterDefaultModelInfo)
})
it("should fall back to default model when model ID is missing", () => {
const handler = new VercelAIGatewayHandler({})
const result = handler.getModel()
result.id.should.equal(openRouterDefaultModelId)
result.info.should.deepEqual(openRouterDefaultModelInfo)
})
})
describe("createMessage", () => {
it("should handle usage-only chunks when delta is missing", async () => {
const handler = new VercelAIGatewayHandler({
vercelAiGatewayApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 11,
completion_tokens: 7,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: 0,
inputTokens: 11,
outputTokens: 7,
totalCost: 0,
},
])
})
it("should read Anthropic-style cache creation and read tokens from usage chunks", async () => {
const handler = new VercelAIGatewayHandler({
vercelAiGatewayApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 1000,
completion_tokens: 200,
prompt_tokens_details: {
cached_tokens: 500,
},
cache_creation_input_tokens: 300,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 300,
cacheReadTokens: 500,
inputTokens: 200,
outputTokens: 200,
totalCost: 0,
},
])
})
})
})
@@ -1,23 +0,0 @@
import "should"
import { vertexGlobalModels } from "@shared/api"
import { VertexHandler } from "../vertex"
describe("VertexHandler", () => {
it("supports Gemini 3.5 Flash model metadata", () => {
const handler = new VertexHandler({
vertexProjectId: "test-project",
vertexRegion: "global",
apiModelId: "gemini-3.5-flash",
})
const model = handler.getModel()
model.id.should.equal("gemini-3.5-flash")
model.info.contextWindow!.should.equal(1_048_576)
model.info.inputPrice!.should.equal(1.5)
model.info.outputPrice!.should.equal(9)
model.info.cacheReadsPrice!.should.equal(0.15)
model.info.supportsGlobalEndpoint!.should.equal(true)
model.info.supportsReasoning!.should.equal(true)
vertexGlobalModels.should.have.property("gemini-3.5-flash")
})
})
@@ -1,42 +0,0 @@
import "should"
import { openAiModelInfoSaneDefaults, wandbDefaultModelId, wandbModels } from "@shared/api"
import { WandbHandler } from "../wandb"
describe("WandbHandler", () => {
it("returns known catalog model metadata when model id is recognized", () => {
const modelId = "meta-llama/Llama-3.3-70B-Instruct"
const handler = new WandbHandler({
wandbApiKey: "test-api-key",
apiModelId: modelId,
})
const model = handler.getModel()
model.id.should.equal(modelId)
model.info.should.deepEqual(wandbModels[modelId])
})
it("passes through an explicit unknown model id instead of silently falling back", () => {
const unknownModelId = "moonshotai/Kimi-K2.5"
const handler = new WandbHandler({
wandbApiKey: "test-api-key",
apiModelId: unknownModelId,
})
const model = handler.getModel()
model.id.should.equal(unknownModelId)
model.info.should.deepEqual(openAiModelInfoSaneDefaults)
})
it("uses the default W&B model when no model id is configured", () => {
const handler = new WandbHandler({
wandbApiKey: "test-api-key",
})
const model = handler.getModel()
model.id.should.equal(wandbDefaultModelId)
model.info.should.deepEqual(wandbModels[wandbDefaultModelId])
})
})
@@ -1,327 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { GenerateContentConfig, GoogleGenAI } from "@google/genai"
import { ModelInfo } from "@shared/api"
import OpenAI from "openai"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { convertAnthropicMessageToGemini } from "../transform/gemini-format"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface AIhubmixHandlerOptions extends CommonApiHandlerOptions {
apiKey?: string
baseURL?: string
appCode?: string
modelId?: string
modelInfo?: ModelInfo
thinkingBudgetTokens?: number
}
export class AIhubmixHandler implements ApiHandler {
private options: AIhubmixHandlerOptions
private anthropicClient: Anthropic | undefined
private openaiClient: OpenAI | undefined
private geminiClient: GoogleGenAI | undefined
constructor(options: AIhubmixHandlerOptions) {
const { baseURL, appCode, ...rest } = options
this.options = {
baseURL: baseURL ?? "https://aihubmix.com",
appCode: appCode ?? "KUWF9311",
...rest,
}
}
private ensureAnthropicClient(): Anthropic {
if (!this.anthropicClient) {
if (!this.options.apiKey) {
throw new Error("AIhubmix API key is required")
}
try {
this.anthropicClient = new Anthropic({
apiKey: this.options.apiKey,
baseURL: this.options.baseURL,
defaultHeaders: {
"APP-Code": this.options.appCode,
...buildExternalBasicHeaders(),
},
})
} catch (error) {
throw new Error(`Error creating Anthropic client: ${error.message}`)
}
}
return this.anthropicClient
}
private ensureOpenaiClient(): OpenAI {
if (!this.openaiClient) {
if (!this.options.apiKey) {
throw new Error("AIhubmix API key is required")
}
try {
this.openaiClient = new OpenAI({
apiKey: this.options.apiKey,
baseURL: `${this.options.baseURL}/v1`,
defaultHeaders: {
"APP-Code": this.options.appCode,
...buildExternalBasicHeaders(),
},
})
} catch (error) {
throw new Error(`Error creating OpenAI client: ${error.message}`)
}
}
return this.openaiClient
}
private ensureGeminiClient(): GoogleGenAI {
if (!this.geminiClient) {
if (!this.options.apiKey) {
throw new Error("AIhubmix API key is required")
}
try {
this.geminiClient = new GoogleGenAI({
apiKey: this.options.apiKey,
httpOptions: {
baseUrl: `${this.options.baseURL}/gemini`,
headers: {
// @ts-expect-error
"APP-Code": this.options.appCode,
Authorization: `Bearer ${this.options.apiKey ?? ""}`,
...buildExternalBasicHeaders(),
},
},
})
} catch (error) {
throw new Error(`Error creating Gemini client: ${error.message}`)
}
}
return this.geminiClient
}
private routeModel(modelName: string): "anthropic" | "openai" | "gemini" | "openai-response" {
const id = modelName || ""
if (id.startsWith("claude")) {
return "anthropic"
}
if (id.startsWith("gemini") && !id.endsWith("-nothink") && !id.endsWith("-search")) {
return "gemini"
}
if (id === "gpt-5-pro" || id === "gpt-5-codex") {
return "openai-response"
}
return "openai"
}
private fixToolChoice(requestBody: any): any {
if (requestBody.tools?.length === 0 && requestBody.tool_choice) {
delete requestBody.tool_choice
}
return requestBody
}
@withRetry()
async *createMessage(systemPrompt: string, messages: any[]): ApiStream {
const modelId = this.options.modelId || ""
const route = this.routeModel(modelId)
switch (route) {
case "anthropic":
yield* this.createAnthropicMessage(systemPrompt, messages)
break
case "gemini":
yield* this.createGeminiMessage(systemPrompt, messages)
break
case "openai-response":
yield* this.createOpenaiResponseMessage(systemPrompt, messages)
break
case "openai":
yield* this.createOpenaiMessage(systemPrompt, messages)
break
default:
throw new Error(`Unsupported model route: ${route}`)
}
}
private async *createAnthropicMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureAnthropicClient()
const modelId = this.options.modelId || "claude-3-5-sonnet-20241022"
// Sanitize messages to remove Cline-specific fields like call_id that are not allowed by Anthropic API
const sanitizedMessages = sanitizeAnthropicMessages(messages, false)
const stream = await client.messages.create({
model: modelId,
temperature: 0,
max_tokens: this.options.modelInfo?.maxTokens || 8192,
system: [{ text: systemPrompt, type: "text" }],
messages: sanitizedMessages,
stream: true,
})
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start":
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
break
case "message_delta":
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage.output_tokens || 0,
}
break
case "content_block_start":
if (chunk.content_block.type === "text") {
yield {
type: "text",
text: chunk.content_block.text,
}
}
break
case "content_block_delta":
if (chunk.delta.type === "text_delta") {
yield {
type: "text",
text: chunk.delta.text,
}
}
break
}
}
}
private async *createOpenaiResponseMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureOpenaiClient()
const modelId = this.options.modelId || "gpt-4o-mini"
const input = (messages || []).map((m: any) => {
const role = m.role || "user"
const contentArray = Array.isArray(m.content) ? m.content : [{ type: "text", text: m.content }]
const content = contentArray
.filter((c: any) => c != null)
.map((c: any) => {
if (c.type === "image" || c.type === "input_image" || c.type === "image_url") {
return { type: "input_image", image_url: c.image_url || c.url || c.source?.url }
}
const text = c.text ?? (typeof c === "string" ? c : "")
return { type: role === "assistant" ? "output_text" : "input_text", text }
})
return { role, content }
})
const stream = await (client as any).responses.stream({
model: modelId,
instructions: systemPrompt,
input,
})
for await (const event of stream as any) {
if (event?.type === "response.output_text.delta") {
yield { type: "text", text: event.delta || "" }
continue
}
if (event?.type === "response.completed") {
const usage = event.response?.usage || {}
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
}
continue
}
if (event?.type === "response.error") {
throw new Error(event.error?.message || "responses error")
}
}
}
private async *createOpenaiMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureOpenaiClient()
const modelId = this.options.modelId || "gpt-4o-mini"
const openaiMessages = [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
const requestBody = {
model: modelId,
messages: openaiMessages,
temperature: 0,
stream: true,
}
const fixedRequestBody = this.fixToolChoice(requestBody)
const stream = await client.chat.completions.create(fixedRequestBody)
for await (const chunk of stream as any) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
private async *createGeminiMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureGeminiClient()
const modelId = this.options.modelId || "gemini-2.0-flash-exp"
const contents = messages.map(convertAnthropicMessageToGemini)
const requestConfig: GenerateContentConfig = {
systemInstruction: systemPrompt,
temperature: 0,
}
if (this.options.thinkingBudgetTokens) {
requestConfig.thinkingConfig = {
thinkingBudget: this.options.thinkingBudgetTokens,
includeThoughts: true,
}
}
const stream = await client.models.generateContentStream({
model: modelId,
contents,
config: requestConfig,
})
for await (const chunk of stream as any) {
if (chunk?.text) {
yield { type: "text", text: chunk.text }
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.modelId || "gpt-4o-mini",
info: this.options.modelInfo || {
maxTokens: 8192,
contextWindow: 128000,
supportsImages: true,
supportsPromptCache: false,
description: "AIhubmix unified model provider",
},
}
}
}
@@ -1,316 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import type {
MessageCreateParamsStreaming as BetaMessageCreateParamsStreaming,
BetaRawMessageStreamEvent,
} from "@anthropic-ai/sdk/resources/beta/messages/messages"
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
import type { MessageCreateParamsStreaming as AnthropicMessageCreateParamsStreaming } from "@anthropic-ai/sdk/resources/messages/messages"
import { Stream as AnthropicStream } from "@anthropic-ai/sdk/streaming"
import {
ANTHROPIC_FAST_MODE_SUFFIX,
AnthropicModelId,
anthropicDefaultModelId,
anthropicModels,
CLAUDE_SONNET_1M_SUFFIX,
ModelInfo,
} from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { ApiStream } from "../transform/stream"
export const ANTHROPIC_FAST_MODE_BETA = "fast-mode-2026-02-01"
interface AnthropicHandlerOptions extends CommonApiHandlerOptions {
apiKey?: string
anthropicBaseUrl?: string
apiModelId?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
}
export class AnthropicHandler implements ApiHandler {
private options: AnthropicHandlerOptions
private client: Anthropic | undefined
constructor(options: AnthropicHandlerOptions) {
this.options = options
}
private ensureClient(): Anthropic {
if (!this.client) {
if (!this.options.apiKey) {
throw new Error("Anthropic API key is required")
}
try {
this.client = new Anthropic({
apiKey: this.options.apiKey,
baseURL: this.options.anthropicBaseUrl || undefined,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating Anthropic client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: AnthropicTool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
let stream: AnthropicStream<Anthropic.RawMessageStreamEvent> | AsyncIterable<BetaRawMessageStreamEvent>
const useFastMode = model.id.endsWith(ANTHROPIC_FAST_MODE_SUFFIX)
const baseModelId = useFastMode ? model.id.slice(0, -ANTHROPIC_FAST_MODE_SUFFIX.length) : model.id
const modelId = baseModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
? baseModelId.slice(0, -CLAUDE_SONNET_1M_SUFFIX.length)
: baseModelId
const enable1mContextWindow = baseModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
const fastModeBetas = enable1mContextWindow
? [ANTHROPIC_FAST_MODE_BETA, "context-1m-2025-08-07"]
: [ANTHROPIC_FAST_MODE_BETA]
const createFastModeMessage = (
body: AnthropicMessageCreateParamsStreaming,
): Promise<AsyncIterable<BetaRawMessageStreamEvent>> => {
return (
client.beta.messages.create as unknown as (
params: BetaMessageCreateParamsStreaming & { speed: "fast" },
) => Promise<AsyncIterable<BetaRawMessageStreamEvent>>
)({
...body,
betas: fastModeBetas,
speed: "fast",
})
}
const budget_tokens = this.options.thinkingBudgetTokens || 0
// Tools are available only when native tools are enabled.
const nativeToolsOn = tools?.length && tools?.length > 0
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
// Claude Opus 4.5+ uses adaptive thinking instead of budgeted extended thinking.
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budget_tokens)
: undefined
const adaptiveThinkingEnabled = adaptiveThinking?.enabled === true
const adaptiveThinkingEffort = adaptiveThinking?.effort
const thinkingEnabled = isAdaptiveThinkingModel ? adaptiveThinkingEnabled : reasoningOn
const thinkingConfig = thinkingEnabled
? isAdaptiveThinkingModel
? ({ type: "adaptive" } as any)
: { type: "enabled", budget_tokens: budget_tokens }
: undefined
const outputConfig = isAdaptiveThinkingModel && adaptiveThinkingEffort ? { effort: adaptiveThinkingEffort } : undefined
if (model.info.supportsPromptCache) {
const anthropicMessages = sanitizeAnthropicMessages(messages, true)
const requestBody: AnthropicMessageCreateParamsStreaming & Record<string, unknown> = {
model: modelId,
thinking: thinkingConfig,
max_tokens: model.info.maxTokens || 8192,
// "Thinking isn't compatible with temperature, top_p, or top_k modifications as well as forced tool use."
// (https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#important-considerations-when-using-extended-thinking)
// Adaptive Claude Opus models do not support temperature.
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
system: [
{
text: systemPrompt,
type: "text",
cache_control: { type: "ephemeral" },
},
], // setting cache breakpoint for system prompt so new tasks can reuse it
messages: anthropicMessages,
// tools, // cache breakpoints go from tools > system > messages, and since tools dont change, we can just set the breakpoint at the end of system (this avoids having to set a breakpoint at the end of tools which by itself does not meet min requirements for haiku caching)
stream: true,
tools: nativeToolsOn ? tools : undefined,
// tool_choice options:
// - none: disables tool use, even if tools are provided. Claude will not call any tools.
// - auto: allows Claude to decide whether to call any provided tools or not. This is the default value when tools are provided.
// - any: tells Claude that it must use one of the provided tools, but doesnt force a particular tool.
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
tool_choice: nativeToolsOn && !thinkingEnabled ? { type: "any" } : undefined,
}
if (outputConfig) {
requestBody.output_config = outputConfig
}
stream = useFastMode
? await createFastModeMessage(requestBody)
: await client.messages.create(
requestBody,
(() => {
// 1m context window beta header
if (enable1mContextWindow) {
return {
headers: {
"anthropic-beta": "context-1m-2025-08-07",
},
}
}
return undefined
})(),
)
} else {
const requestBody: AnthropicMessageCreateParamsStreaming & Record<string, unknown> = {
model: modelId,
max_tokens: model.info.maxTokens || 8192,
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
system: [{ text: systemPrompt, type: "text" }],
messages: sanitizeAnthropicMessages(messages, false),
tools: nativeToolsOn ? tools : undefined,
tool_choice: thinkingEnabled ? undefined : { type: "auto" },
stream: true,
thinking: thinkingConfig,
}
if (outputConfig) {
requestBody.output_config = outputConfig
}
stream = useFastMode ? await createFastModeMessage(requestBody) : await client.messages.create(requestBody)
}
const lastStartedToolCall = { id: "", name: "", arguments: "" }
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start":
{
// tells us cache reads/writes/input/output
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
}
break
case "message_delta":
// tells us stop_reason, stop_sequence, and output tokens along the way and at the end of the message
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage.output_tokens || 0,
}
break
case "message_stop":
// no usage data, just an indicator that the message is done
break
case "content_block_start":
switch (chunk.content_block.type) {
case "thinking":
yield {
type: "reasoning",
reasoning: chunk.content_block.thinking || "",
signature: chunk.content_block.signature,
}
break
case "redacted_thinking":
// Content is encrypted, and we don't to pass placeholder text back to the API
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
redacted_data: chunk.content_block.data,
}
break
case "tool_use":
if (chunk.content_block.id && chunk.content_block.name) {
// Convert Anthropic tool_use to OpenAI-compatible format
lastStartedToolCall.id = chunk.content_block.id
lastStartedToolCall.name = chunk.content_block.name
lastStartedToolCall.arguments = ""
}
break
case "text":
// we may receive multiple text blocks, in which case just insert a line break between them
if (chunk.index > 0) {
yield {
type: "text",
text: "\n",
}
}
yield {
type: "text",
text: chunk.content_block.text,
}
break
}
break
case "content_block_delta":
switch (chunk.delta.type) {
case "thinking_delta":
// 'reasoning' type just displays in the UI, but ant_thinking will be used to send the thinking traces back to the API
yield {
type: "reasoning",
reasoning: chunk.delta.thinking,
}
break
case "signature_delta":
// It's used when sending the thinking block back to the API
// API expects this in completed form, not as array of deltas
if (chunk.delta.signature) {
yield {
type: "reasoning",
reasoning: "", // reasoning text is already sent via thinking_delta
signature: chunk.delta.signature,
}
}
break
case "text_delta":
yield {
type: "text",
text: chunk.delta.text,
}
break
case "input_json_delta":
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
// // Convert Anthropic tool_use to OpenAI-compatible format
yield {
type: "tool_calls",
tool_call: {
...lastStartedToolCall,
function: {
...lastStartedToolCall,
id: lastStartedToolCall.id,
name: lastStartedToolCall.name,
arguments: chunk.delta.partial_json,
},
},
}
}
break
}
break
case "content_block_stop":
lastStartedToolCall.id = ""
lastStartedToolCall.name = ""
lastStartedToolCall.arguments = ""
break
}
}
}
getModel(): { id: AnthropicModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in anthropicModels) {
const id = modelId as AnthropicModelId
return { id, info: anthropicModels[id] }
}
return {
id: anthropicDefaultModelId,
info: anthropicModels[anthropicDefaultModelId],
}
}
}
@@ -1,200 +0,0 @@
import { AskSageModelId, askSageDefaultModelId, askSageDefaultURL, askSageModels, ModelInfo } from "@shared/api"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
interface AskSageHandlerOptions extends CommonApiHandlerOptions {
asksageApiKey?: string
asksageApiUrl?: string
apiModelId?: string
}
type AskSageRequest = {
system_prompt: string
message: {
user: "gpt" | "me"
message: string
}[]
model: string
dataset: "none"
usage: boolean
}
type AskSageUsage = {
model_tokens: {
completion_tokens: number
prompt_tokens: number
total_tokens: number
}
asksage_tokens: number
}
type AskSageResponse = {
uuid: string
status: number
// Response status
response: string
// Generated response message
message: string
// whether embedding & vector systems are down
embedding_down: boolean
vectors_down: boolean
// references if dataset is not none
references: string
type: string
added_obj: any
tool_calls: any
// usage metrics
usage: AskSageUsage | null
tool_responses: any[]
tool_calls_unified: any[]
}
export class AskSageHandler implements ApiHandler {
private options: AskSageHandlerOptions
private apiUrl: string
private apiKey: string
constructor(options: AskSageHandlerOptions) {
Logger.log("init api url", options.asksageApiUrl, askSageDefaultURL)
this.options = options
this.apiKey = options.asksageApiKey || ""
this.apiUrl = options.asksageApiUrl || askSageDefaultURL
if (!this.apiKey) {
throw new Error("AskSage API key is required")
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
try {
const model = this.getModel()
// Transform messages into AskSageRequest format
const formattedMessages = messages.map((msg) => {
const content = Array.isArray(msg.content)
? msg.content.map((block) => ("text" in block ? block.text : "")).join("")
: msg.content
return {
user: msg.role === "assistant" ? ("gpt" as const) : ("me" as const),
message: content,
}
})
const request: AskSageRequest = {
system_prompt: systemPrompt,
message: formattedMessages,
model: model.id,
dataset: "none",
usage: true,
}
// Make request to AskSage API
const response = await fetch(`${this.apiUrl}/query`, {
method: "POST",
headers: this.headers(),
body: JSON.stringify(request),
})
if (!response.ok) {
const error = await response.text()
throw new Error(`AskSage API error: ${error}`)
}
const result = (await response.json()) as AskSageResponse
if (!result.message) {
throw new Error("No content in AskSage response")
}
// Yield tool responses if they exist
if (result.tool_responses && result.tool_responses.length > 0) {
for (const toolResponse of result.tool_responses) {
yield {
type: "text",
text: `[Tool Response: ${JSON.stringify(toolResponse)}]\n`,
}
}
}
// Yield the main response text
yield {
type: "text",
text: result.message,
}
// Yield usage information if available
if (result.usage) {
yield {
type: "usage",
inputTokens: result.usage.model_tokens.prompt_tokens,
outputTokens: result.usage.model_tokens.completion_tokens,
cacheReadTokens: 0,
cacheWriteTokens: 0,
totalCost: result.usage.asksage_tokens, // Cost = Consumed AskSage tokens
}
}
} catch (error) {
if (error instanceof Error) {
throw new Error(`AskSage request failed: ${error.message}`)
}
throw error
}
}
async getApiStreamUsage() {
if (!this.apiKey) {
return undefined
}
try {
const response = await fetch(`${this.apiUrl}/count-monthly-tokens`, {
method: "POST",
headers: this.headers(),
body: JSON.stringify({ app_name: "asksage" }),
})
if (!response.ok) {
Logger.error("Failed to fetch AskSage usage", await response.text())
return undefined
}
const data = await response.json()
const usedTokens = data.response as number
return {
type: "usage" as const,
inputTokens: usedTokens,
outputTokens: 0,
}
} catch (error) {
Logger.error("Error fetching AskSage usage:", error)
return undefined
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in askSageModels) {
const id = modelId as AskSageModelId
return { id, info: askSageModels[id] }
}
return {
id: askSageDefaultModelId,
info: askSageModels[askSageDefaultModelId],
}
}
private headers() {
return {
"Content-Type": "application/json",
"x-access-tokens": this.apiKey,
...buildExternalBasicHeaders(),
}
}
}
@@ -1,172 +0,0 @@
import { BasetenModelId, basetenDefaultModelId, basetenModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
interface BasetenHandlerOptions extends CommonApiHandlerOptions {
basetenApiKey?: string
basetenModelId?: string
basetenModelInfo?: ModelInfo
apiModelId?: string // For backward compatibility
}
export class BasetenHandler implements ApiHandler {
private options: BasetenHandlerOptions
private client: OpenAI | undefined
constructor(options: BasetenHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.basetenApiKey) {
throw new Error("Baseten API key is required")
}
try {
this.client = new OpenAI({
baseURL: "https://inference.baseten.co/v1",
apiKey: this.options.basetenApiKey,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating Baseten client: ${error.message}`)
}
}
return this.client
}
/**
* Gets the optimal max_tokens based on model capabilities
*/
private getOptimalMaxTokens(model: { id: BasetenModelId; info: ModelInfo }): number {
// Use model-specific max tokens if available
if (model.info.maxTokens && model.info.maxTokens > 0) {
return model.info.maxTokens
}
// Default fallback
return 8192
}
getModel(): { id: BasetenModelId; info: ModelInfo } {
// First priority: basetenModelId and basetenModelInfo
const basetenModelId = this.options.basetenModelId
const basetenModelInfo = this.options.basetenModelInfo
if (basetenModelId && basetenModelInfo) {
return { id: basetenModelId as BasetenModelId, info: basetenModelInfo }
}
// Second priority: basetenModelId with static model info
if (basetenModelId && basetenModelId in basetenModels) {
const id = basetenModelId as BasetenModelId
return { id, info: basetenModels[id] }
}
// Third priority: apiModelId (for backward compatibility)
const apiModelId = this.options.apiModelId
if (apiModelId && apiModelId in basetenModels) {
const id = apiModelId as BasetenModelId
return { id, info: basetenModels[id] }
}
// Default fallback
return {
id: basetenDefaultModelId,
info: basetenModels[basetenDefaultModelId],
}
}
private async *yieldUsage(modelInfo: ModelInfo, usage: any): ApiStream {
if (usage.prompt_tokens || usage.completion_tokens) {
const cost = calculateApiCostOpenAI(modelInfo, usage.prompt_tokens || 0, usage.completion_tokens || 0)
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
cacheWriteTokens: 0,
cacheReadTokens: 0,
totalCost: cost,
}
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const maxTokens = this.getOptimalMaxTokens(model)
const toolCallProcessor = new ToolCallProcessor()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_tokens: maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
tools,
tool_choice: tools && tools.length > 0 ? "auto" : undefined,
})
let didOutputUsage = false
for await (const chunk of stream) {
const delta = chunk?.choices?.[0]?.delta
// Handle reasoning field if present (for reasoning models with parsed output)
if (delta && "reasoning" in delta && delta?.reasoning) {
const reasoning = typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning)
yield {
type: "reasoning",
reasoning,
}
}
// Handle content field
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle usage information - only output once
if (!didOutputUsage && chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
didOutputUsage = true
}
}
}
/**
* Checks if the current model supports tools
*/
supportsTools(): boolean {
const model = this.getModel()
const modelInfo = model.info as any
// Use dynamic API data when available, fallback to true since all current Baseten models support tools
// (as of 2025-09-16 - could change if Baseten add non-tool models in future, currently no plans to do so)
return modelInfo.supportedFeatures ? modelInfo.supportedFeatures.includes("tools") : true
}
}
File diff suppressed because it is too large Load Diff
@@ -1,275 +0,0 @@
import Cerebras from "@cerebras/cerebras_cloud_sdk"
import { CerebrasModelId, cerebrasDefaultModelId, cerebrasModels, ModelInfo } from "@shared/api"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
interface CerebrasHandlerOptions extends CommonApiHandlerOptions {
cerebrasApiKey?: string
apiModelId?: string
}
// Conservative max_tokens for Cerebras to avoid premature rate limiting.
// Cerebras rate limiter estimates token consumption using max_completion_tokens upfront,
// so requesting the model maximum (e.g., 64K) reserves that quota even if actual usage is low.
// 16K is sufficient for most agentic tool use while preserving rate limit headroom.
const CEREBRAS_DEFAULT_MAX_TOKENS = 16_384
export class CerebrasHandler implements ApiHandler {
private options: CerebrasHandlerOptions
private client: Cerebras | undefined
constructor(options: CerebrasHandlerOptions) {
this.options = options
}
private ensureClient(): Cerebras {
if (!this.client) {
// Clean and validate the API key
const cleanApiKey = this.options.cerebrasApiKey?.trim()
if (!cleanApiKey) {
throw new Error("Cerebras API key is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
this.client = new Cerebras({
apiKey: cleanApiKey,
timeout: 30000, // 30 second timeout
fetch, // Use configured fetch with proxy support
defaultHeaders: {
...externalHeaders,
"X-Cerebras-3rd-Party-Integration": "cline",
},
})
} catch (error) {
throw new Error(`Error creating Cerebras client: ${error.message}`)
}
}
return this.client
}
@withRetry({
maxRetries: 6, // More retries to be patient with rate limits
baseDelay: 5000, // Start with 5 second delay
maxDelay: 60000, // Allow up to 60 second delays to respect rate limits
})
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
// Convert Anthropic messages to Cerebras format
const cerebrasMessages: Array<{
role: "system" | "user" | "assistant"
content: string
}> = [{ role: "system", content: systemPrompt }]
// Helper function to strip thinking tags from content
const stripThinkingTags = (content: string): string => {
return content.replace(/<think>[\s\S]*?<\/think>/g, "").trim()
}
// Check if this is a reasoning model that uses thinking tags
const modelId = this.getModel().id
const isReasoningModel = modelId.includes("qwen")
// Convert Anthropic messages to Cerebras format
for (const message of messages) {
if (message.role === "user") {
const content = Array.isArray(message.content)
? message.content
.map((block) => {
if (block.type === "text") {
return block.text
}
if (block.type === "image") {
return "[Image content not supported in Cerebras]"
}
return ""
})
.join("\n")
: message.content
cerebrasMessages.push({ role: "user", content })
} else if (message.role === "assistant") {
let content = Array.isArray(message.content)
? message.content
.map((block) => {
if (block.type === "text") {
return block.text
}
return ""
})
.join("\n")
: message.content || ""
// Strip thinking tags from assistant messages for reasoning models
// so the model doesn't see its own thinking in the conversation history
if (isReasoningModel) {
content = stripThinkingTags(content)
}
cerebrasMessages.push({ role: "assistant", content })
}
}
try {
const model = this.getModel()
const stream = await client.chat.completions.create({
model: model.id,
messages: cerebrasMessages,
temperature: model.info.temperature ?? 0,
stream: true,
max_tokens: CEREBRAS_DEFAULT_MAX_TOKENS,
})
// Handle streaming response
let reasoning: string | null = null // Track reasoning content for models that support thinking
for await (const chunk of stream as any) {
// Type assertion for the streaming chunk
const streamChunk = chunk as any
if (streamChunk.choices?.[0]?.delta?.content) {
const content = streamChunk.choices[0].delta.content
// Handle reasoning models (Qwen and DeepSeek R1 Distill) that use <think> tags
if (isReasoningModel) {
// Check if we're entering or continuing reasoning mode
if (reasoning || content.includes("<think>")) {
reasoning = (reasoning || "") + content
// Clean the content by removing think tags for display
const cleanContent = content.replace(/<think>/g, "").replace(/<\/think>/g, "")
// Only yield reasoning content if there's actual content after cleaning
if (cleanContent.trim()) {
yield {
type: "reasoning",
reasoning: cleanContent,
}
}
// Check if reasoning is complete
if (reasoning.includes("</think>")) {
reasoning = null
}
} else {
// Regular content outside of thinking tags
yield {
type: "text",
text: content,
}
}
} else {
// Non-reasoning models - just yield text content
yield {
type: "text",
text: content,
}
}
}
// Handle usage information from Cerebras API
// Usage is typically only available in the final chunk
if (streamChunk.usage) {
const totalCost = this.calculateCost({
inputTokens: streamChunk.usage.prompt_tokens || 0,
outputTokens: streamChunk.usage.completion_tokens || 0,
})
yield {
type: "usage",
inputTokens: streamChunk.usage.prompt_tokens || 0,
outputTokens: streamChunk.usage.completion_tokens || 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
totalCost,
}
}
}
} catch (error: any) {
// Enhanced error handling for Cerebras API
if (error?.status === 429 || error?.code === "rate_limit_exceeded") {
// Rate limit error - will be handled by retry decorator with patient backoff
const _limits = this.getRateLimits()
throw new Error(`Cerebras API rate limit exceeded.`)
}
if (error?.status === 401) {
throw new Error("Cerebras API authentication failed. Please check your API key.")
}
if (error?.status === 403) {
throw new Error("Cerebras API access denied. Please check your API key permissions.")
}
if (error?.status >= 500) {
// Server errors - retryable
throw new Error(`Cerebras API server error (${error.status}): ${error.message || "Unknown server error"}`)
}
if (error?.status === 400) {
// Client errors - not retryable
throw new Error(`Cerebras API bad request: ${error.message || "Invalid request parameters"}`)
}
// Re-throw original error for other cases
throw error
}
}
getModel(): { id: string; info: ModelInfo } {
const originalModelId = this.options.apiModelId
let apiModelId = originalModelId
if (originalModelId === "qwen-3-coder-480b-free") {
apiModelId = "qwen-3-coder-480b"
return { id: apiModelId, info: cerebrasModels[originalModelId as CerebrasModelId] }
}
if (originalModelId && originalModelId in cerebrasModels) {
const id = originalModelId as CerebrasModelId
return { id, info: cerebrasModels[id] }
}
return {
id: cerebrasDefaultModelId,
info: cerebrasModels[cerebrasDefaultModelId],
}
}
/**
* Get rate limit information for the current model
*
* These limits are used for informational purposes and to calculate appropriate
* retry delays. Since Cerebras inference is extremely fast, users hit these limits
* quickly, so we need to be patient with retries to maximize usage efficiency.
*
* @returns Rate limit configuration for the model
*/
private getRateLimits(): { requestsPerMinute: number; tokensPerMinute: number } {
const modelId = this.getModel().id
switch (modelId) {
case "qwen-3-coder-480b":
case "qwen-3-coder-480b-free":
return { requestsPerMinute: 10, tokensPerMinute: 150_000 }
case "qwen-3-235b-a22b-instruct-2507":
case "qwen-3-235b-a22b-thinking-2507":
return { requestsPerMinute: 30, tokensPerMinute: 60_000 }
case "gpt-oss-120b":
return { requestsPerMinute: 30, tokensPerMinute: 64_000 }
default:
// Default rate limits for unknown models
return { requestsPerMinute: 30, tokensPerMinute: 60_000 }
}
}
private calculateCost({ inputTokens, outputTokens }: { inputTokens: number; outputTokens: number }): number {
const model = this.getModel()
const inputPrice = model.info.inputPrice || 0
const outputPrice = model.info.outputPrice || 0
const inputCost = (inputPrice / 1_000_000) * inputTokens
const outputCost = (outputPrice / 1_000_000) * outputTokens
return inputCost + outputCost
}
}
@@ -1,229 +0,0 @@
import { filterMessagesForClaudeCode } from "@/integrations/claude-code/message-filter"
import { runClaudeCode } from "@/integrations/claude-code/run"
import { ClaudeCodeModelId, claudeCodeDefaultModelId, claudeCodeModels } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { type ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { type ApiStream, ApiStreamUsageChunk } from "../transform/stream"
interface ClaudeCodeHandlerOptions extends CommonApiHandlerOptions {
claudeCodePath?: string
apiModelId?: string
thinkingBudgetTokens?: number
}
export class ClaudeCodeHandler implements ApiHandler {
private options: ClaudeCodeHandlerOptions
constructor(options: ClaudeCodeHandlerOptions) {
this.options = options
}
@withRetry({
maxRetries: 4,
baseDelay: 2000,
maxDelay: 15000,
})
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
// Filter out image blocks since Claude Code doesn't support them
const filteredMessages = filterMessagesForClaudeCode(messages)
const claudeProcess = runClaudeCode({
systemPrompt,
messages: filteredMessages,
path: this.options.claudeCodePath,
modelId: this.getModel().id,
thinkingBudgetTokens: this.options.thinkingBudgetTokens,
})
// Usage is included with assistant messages,
// but cost is included in the result chunk
const usage: ApiStreamUsageChunk = {
type: "usage",
inputTokens: 0,
outputTokens: 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
}
let isPaidUsage = true
for await (const chunk of claudeProcess) {
if (typeof chunk === "string") {
yield {
type: "text",
text: chunk,
}
continue
}
// Handle system init messages
if (chunk.type === "system" && "subtype" in chunk) {
if (chunk.subtype === "init") {
// Based on my tests, subscription usage sets the `apiKeySource` to "none"
isPaidUsage = (chunk as any).apiKeySource !== "none"
}
// Also handles legacy rate_limit_event format (type: "system", subtype: "rate_limit_event")
// by falling through — no special handling needed.
continue
}
// Handle rate_limit_event (newer CLI format: top-level type)
if (chunk.type === "rate_limit_event") {
// Rate limit events are informational. Log them but don't yield anything.
// If the rate limit blocks the response, the stream will end without
// assistant messages and the task loop will handle the empty response.
Logger.log("Claude Code rate limit event:", JSON.stringify(chunk))
continue
}
// Skip user messages (tool results from Claude Code's own tool execution)
if (chunk.type === "user") {
continue
}
if (chunk.type === "assistant" && "message" in chunk) {
const message = chunk.message
// Check for error field on the message (newer CLI format)
if (message.error) {
const firstContent = message.content?.[0]
const errorText = firstContent && "text" in firstContent ? firstContent.text : undefined
throw new Error(errorText ?? `Claude Code error: ${message.error}`)
}
if (message.stop_reason !== null) {
const firstContent = message.content?.[0]
const content = firstContent && "text" in firstContent ? firstContent : undefined
// Check if content exists before accessing its properties
if (content && content.text.startsWith(`API Error`)) {
// Error messages are formatted as: `API Error: <<status code>> <<json>>`
const errorMessageStart = content.text.indexOf("{")
const errorMessage = content.text.slice(errorMessageStart)
const error = this.attemptParse(errorMessage)
if (!error) {
throw new Error(content.text)
}
if (error.error.message.includes("Invalid model name")) {
throw new Error(
content.text +
`\n\nAPI keys and subscription plans allow different models. Make sure the selected model is included in your plan.`,
)
}
throw new Error(errorMessage)
}
}
for (const content of message.content) {
switch (content.type) {
case "text":
yield {
type: "text",
text: content.text,
}
break
case "thinking":
yield {
type: "reasoning",
reasoning: content.thinking || "",
}
break
case "redacted_thinking":
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
}
break
case "tool_use":
// Yield tool_use blocks to the streaming pipeline for proper tool execution
yield {
type: "tool_calls",
tool_call: {
call_id: content.id,
function: {
id: content.id,
name: content.name,
arguments: JSON.stringify(content.input),
},
},
}
break
default: {
// Handle unknown content block types gracefully.
// Newer Anthropic models or CLI versions may introduce new content types
// (e.g., server_tool_use, mcp_tool_use). Log them instead of silently dropping.
const unknownBlock = content as { type: string; text?: string }
Logger.warn(`Unhandled content type in Claude Code response: ${unknownBlock.type}`)
// If the unknown block has a text-like field, try to yield it as text
if (typeof unknownBlock.text === "string") {
yield {
type: "text",
text: unknownBlock.text,
}
}
break
}
}
}
// According to Anthropic's API documentation:
// https://docs.anthropic.com/en/api/messages#usage-object
// The `input_tokens` field already includes both `cache_read_input_tokens` and `cache_creation_input_tokens`.
// Therefore, we should not add cache tokens to the input_tokens count again, as this would result in double-counting.
usage.inputTokens = message.usage?.input_tokens ?? 0
usage.outputTokens = message.usage?.output_tokens ?? 0
usage.cacheReadTokens = message.usage?.cache_read_input_tokens ?? 0
usage.cacheWriteTokens = message.usage?.cache_creation_input_tokens ?? 0
continue
}
if (chunk.type === "result" && "result" in chunk) {
if (chunk.is_error) {
throw new Error(`Claude Code returned an error: ${chunk.result}`)
}
usage.totalCost = isPaidUsage ? chunk.total_cost_usd : 0
yield usage
continue
}
// ErrorMessage — log it explicitly and skip
if ((chunk as any).type === "error") {
Logger.warn("Claude Code emitted an error-type chunk:", JSON.stringify(chunk))
continue
}
// Any completely unrecognized chunk type — log and skip
Logger.warn(`Unrecognized Claude Code chunk type: ${(chunk as any).type}`)
}
}
private attemptParse(str: string) {
try {
return JSON.parse(str)
} catch (_err) {
return null
}
}
getModel() {
const modelId = this.options.apiModelId
if (modelId && modelId in claudeCodeModels) {
const id = modelId as ClaudeCodeModelId
return { id, info: claudeCodeModels[id] }
}
return {
id: claudeCodeDefaultModelId,
info: claudeCodeModels[claudeCodeDefaultModelId],
}
}
}
-343
View File
@@ -1,343 +0,0 @@
import { type ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import axios from "axios"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineEnv } from "@/config"
import { refreshClineRecommendedModels } from "@/core/controller/models/refreshClineRecommendedModels"
import { ClineAccountService } from "@/services/account/ClineAccountService"
import { AuthService } from "@/services/auth/AuthService"
import { buildClineExtraHeaders } from "@/services/EnvUtils"
import { CLINE_ACCOUNT_AUTH_ERROR_MESSAGE } from "@/shared/ClineAccount"
import { CLINE_RECOMMENDED_MODELS_FALLBACK } from "@/shared/cline/recommended-models"
import type { ClineStorageMessage } from "@/shared/messages/content"
import { fetch, getAxiosSettings } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import type { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { createOpenRouterStream } from "../transform/openrouter-stream"
import type { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
import type { OpenRouterErrorResponse } from "./types"
interface ClineHandlerOptions extends CommonApiHandlerOptions {
ulid?: string
taskId?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
openRouterProviderSorting?: string
openRouterModelId?: string
openRouterModelInfo?: ModelInfo
clineAccountId?: string
clineApiKey?: string
enableParallelToolCalling?: boolean
}
function normalizeModelId(modelId: string): string {
return modelId.trim().toLowerCase()
}
const CLINE_FREE_MODEL_IDS = new Set(CLINE_RECOMMENDED_MODELS_FALLBACK.free.map((model) => normalizeModelId(model.id)))
function getCacheReadTokens(usage: any): number {
return usage?.prompt_tokens_details?.cached_tokens || usage?.cache_read_input_tokens || 0
}
function getCacheWriteTokens(usage: any): number {
return usage?.prompt_tokens_details?.cache_write_tokens || usage?.cache_creation_input_tokens || 0
}
export class ClineHandler implements ApiHandler {
private options: ClineHandlerOptions
private clineAccountService = ClineAccountService.getInstance()
private _authService: AuthService
private client: OpenAI | undefined
lastGenerationId?: string
private lastRequestId?: string
private get _baseUrl(): string {
return ClineEnv.config().apiBaseUrl
}
constructor(options: ClineHandlerOptions) {
this.options = options
this._authService = AuthService.getInstance()
}
private async getFreeModelIdSet(): Promise<Set<string>> {
try {
const models = await refreshClineRecommendedModels()
const freeModelIds = models.free.map((model) => normalizeModelId(model.id)).filter((modelId) => modelId.length > 0)
if (freeModelIds.length > 0) {
return new Set(freeModelIds)
}
} catch (error) {
Logger.error("Error resolving Cline free model IDs from recommended models:", error)
}
return CLINE_FREE_MODEL_IDS
}
private async ensureClient(): Promise<OpenAI> {
const clineAccountAuthToken = this.options.clineApiKey || (await this._authService.getAuthToken())
if (!clineAccountAuthToken) {
throw new Error(CLINE_ACCOUNT_AUTH_ERROR_MESSAGE)
}
if (!this.client) {
try {
const defaultHeaders: Record<string, string> = {
"HTTP-Referer": "https://cline.bot",
"X-Title": "Cline",
"X-Task-ID": this.options.ulid || "",
}
Object.assign(defaultHeaders, await buildClineExtraHeaders())
this.client = new OpenAI({
baseURL: `${this._baseUrl}/api/v1`,
apiKey: clineAccountAuthToken,
defaultHeaders,
// Capture real HTTP request ID from initial streaming response headers
fetch: async (...args: Parameters<typeof fetch>): Promise<Awaited<ReturnType<typeof fetch>>> => {
const [input, init] = args
const resp = await fetch(input, init)
try {
let urlStr = ""
if (typeof input === "string") {
urlStr = input
} else if (input instanceof URL) {
urlStr = input.toString()
} else if (typeof (input as { url?: unknown }).url === "string") {
urlStr = (input as { url: string }).url
}
// Only record for chat completions (the primary streaming request)
if (urlStr.includes("/chat/completions")) {
const rid = resp.headers.get("x-request-id") || resp.headers.get("request-id")
if (rid) {
this.lastRequestId = rid
}
}
} catch {
// ignore header capture errors
}
return resp
},
})
} catch (error: any) {
throw new Error(`Error creating Cline client: ${error.message}`)
}
}
// Ensure the client is always using the latest auth token
this.client.apiKey = clineAccountAuthToken
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
try {
const client = await this.ensureClient()
this.lastGenerationId = undefined
this.lastRequestId = undefined
let didOutputUsage = false
const freeModelIds = await this.getFreeModelIdSet()
const stream = await createOpenRouterStream(
client,
systemPrompt,
messages,
this.getModel(),
this.options.reasoningEffort,
this.options.thinkingBudgetTokens,
this.options.openRouterProviderSorting,
tools,
this.options.enableParallelToolCalling,
)
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
Logger.debug("ClineHandler chunk:" + JSON.stringify(chunk))
// openrouter returns an error object instead of the openai sdk throwing an error
if ("error" in chunk) {
const error = chunk.error as OpenRouterErrorResponse["error"]
Logger.error(`Cline API Error: ${error?.code} - ${error?.message}`)
// Include metadata in the error message if available
const metadataStr = error.metadata ? `\nMetadata: ${JSON.stringify(error.metadata, null, 2)}` : ""
throw new Error(`Cline API Error ${error.code}: ${error.message}${metadataStr}`)
}
if (!this.lastGenerationId && chunk.id) {
this.lastGenerationId = chunk.id
}
// Check for mid-stream error via finish_reason
const choice = chunk.choices?.[0]
// OpenRouter may return finish_reason = "error" with error details
if ((choice?.finish_reason as string) === "error") {
const choiceWithError = choice as any
if (choiceWithError.error) {
const error = choiceWithError.error
Logger.error(`Cline Mid-Stream Error: ${error.code || error.type || "Unknown"} - ${error.message}`)
throw new Error(`Cline Mid-Stream Error: ${error.code || error.type || "Unknown"} - ${error.message}`)
}
throw new Error("Cline Mid-Stream Error: Stream terminated with error status but no error details provided")
}
const delta = choice?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Reasoning tokens are returned separately from the content
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
if (
delta &&
"reasoning" in delta &&
delta.reasoning &&
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
/*
OpenRouter passes reasoning details that we can pass back unmodified in api requests to preserve reasoning traces for model
- The reasoning_details array in each chunk may contain one or more reasoning objects
- For encrypted reasoning, the content may appear as [REDACTED] in streaming responses
- The complete reasoning sequence is built by concatenating all chunks in order
See: https://openrouter.ai/docs/use-cases/reasoning-tokens#preserving-reasoning-blocks
*/
if (
delta &&
"reasoning_details" in delta &&
delta.reasoning_details &&
// @ts-expect-error-next-line
delta?.reasoning_details?.length && // exists and non-0
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: "",
details: delta.reasoning_details,
}
}
if (!didOutputUsage && chunk.usage) {
// @ts-expect-error-next-line
let totalCost = (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0)
const modelId = this.getModel().id
const isFreeModel = freeModelIds.has(normalizeModelId(modelId))
const cacheReadTokens = getCacheReadTokens(chunk.usage)
const cacheWriteTokens = getCacheWriteTokens(chunk.usage)
if (isFreeModel) {
totalCost = 0
}
yield {
type: "usage",
cacheWriteTokens,
cacheReadTokens,
inputTokens: Math.max(0, (chunk.usage.prompt_tokens || 0) - cacheReadTokens - cacheWriteTokens),
outputTokens: chunk.usage.completion_tokens || 0,
totalCost,
}
didOutputUsage = true
}
}
// Fallback to generation endpoint if usage chunk not returned
if (!didOutputUsage) {
Logger.warn("Cline API did not return usage chunk, fetching from generation endpoint")
const apiStreamUsage = await this.getApiStreamUsage(freeModelIds)
if (apiStreamUsage) {
yield apiStreamUsage
}
}
} catch (error) {
Logger.error("Cline API Error:", error)
throw error
}
}
async getApiStreamUsage(freeModelIds?: Set<string>): Promise<ApiStreamUsageChunk | undefined> {
if (this.lastGenerationId) {
try {
const resolvedFreeModelIds = freeModelIds || (await this.getFreeModelIdSet())
const clineAccountAuthToken = await this._authService.getAuthToken()
if (!clineAccountAuthToken) {
throw new Error(CLINE_ACCOUNT_AUTH_ERROR_MESSAGE)
}
const headers: Record<string, string> = {
// Align with backend auth expectations
Authorization: `Bearer ${clineAccountAuthToken}`,
}
Object.assign(headers, await buildClineExtraHeaders())
const response = await axios.get(`${this.clineAccountService.baseUrl}/generation?id=${this.lastGenerationId}`, {
headers,
timeout: 15_000, // this request hangs sometimes
...getAxiosSettings(),
})
const generation = response.data
let totalCost = generation?.total_cost || 0
const modelId = this.getModel().id
const isFreeModel = resolvedFreeModelIds.has(normalizeModelId(modelId))
if (isFreeModel) {
totalCost = 0
}
return {
type: "usage",
cacheWriteTokens: generation?.native_tokens_cache_write || 0,
cacheReadTokens: generation?.native_tokens_cached || 0,
// openrouter generation endpoint fails often
inputTokens: Math.max(
0,
(generation?.native_tokens_prompt || 0) -
(generation?.native_tokens_cached || 0) -
(generation?.native_tokens_cache_write || 0),
),
outputTokens: generation?.native_tokens_completion || 0,
totalCost,
}
} catch (error) {
// ignore if fails
Logger.error("Error fetching cline generation details:", error)
}
}
return undefined
}
// Expose the last HTTP request ID captured from response headers (X-Request-ID)
getLastRequestId(): string | undefined {
return this.lastRequestId
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.openRouterModelId
const modelInfo = this.options.openRouterModelInfo
if (modelId && modelInfo) {
return { id: modelId, info: modelInfo }
}
// If we have a model ID but no model info (e.g., CLI featured models),
// use the ID with default model info rather than falling back to a different model
if (modelId) {
return { id: modelId, info: openRouterDefaultModelInfo }
}
return { id: openRouterDefaultModelId, info: openRouterDefaultModelInfo }
}
}
@@ -1,142 +0,0 @@
import { DeepSeekModelId, deepSeekDefaultModelId, deepSeekModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { addReasoningContent } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface DeepSeekHandlerOptions extends CommonApiHandlerOptions {
deepSeekApiKey?: string
apiModelId?: string
}
export class DeepSeekHandler implements ApiHandler {
private options: DeepSeekHandlerOptions
private client: OpenAI | undefined
constructor(options: DeepSeekHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.deepSeekApiKey) {
throw new Error("DeepSeek API key is required")
}
try {
this.client = new OpenAI({
baseURL: "https://api.deepseek.com/v1",
apiKey: this.options.deepSeekApiKey,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating DeepSeek client: ${error.message}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
// Deepseek reports total input AND cache reads/writes,
// see context caching: https://api-docs.deepseek.com/guides/kv_cache)
// where the input tokens is the sum of the cache hits/misses, just like OpenAI.
// This affects:
// 1) context management truncation algorithm, and
// 2) cost calculation
// Deepseek usage includes extra fields.
// Safely cast the prompt token details section to the appropriate structure.
interface DeepSeekUsage extends OpenAI.CompletionUsage {
prompt_cache_hit_tokens?: number
prompt_cache_miss_tokens?: number
}
const deepUsage = usage as DeepSeekUsage
const inputTokens = deepUsage?.prompt_tokens || 0 // sum of cache hits and misses
const outputTokens = deepUsage?.completion_tokens || 0
const cacheReadTokens = deepUsage?.prompt_cache_hit_tokens || 0
const cacheWriteTokens = deepUsage?.prompt_cache_miss_tokens || 0
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens) // this will always be 0
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
totalCost: totalCost,
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const isDeepSeekThinkingModel =
model.id.includes("deepseek-reasoner") || model.id === "deepseek-v4-flash" || model.id === "deepseek-v4-pro"
const convertedMessages = convertToOpenAiMessages(messages)
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = isDeepSeekThinkingModel
? [{ role: "system", content: systemPrompt }, ...addReasoningContent(convertedMessages, messages)]
: [{ role: "system", content: systemPrompt }, ...convertedMessages]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
// Only set temperature for non-thinking models
...(isDeepSeekThinkingModel ? {} : { temperature: 0 }),
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
}
}
}
getModel(): { id: DeepSeekModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in deepSeekModels) {
const id = modelId as DeepSeekModelId
return { id, info: deepSeekModels[id] }
}
return {
id: deepSeekDefaultModelId,
info: deepSeekModels[deepSeekDefaultModelId],
}
}
}
-657
View File
@@ -1,657 +0,0 @@
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ModelInfo } from "../../../shared/api"
import { ApiHandler } from "../index"
import { ApiStream } from "../transform/stream"
interface DifyHandlerOptions {
difyApiKey?: string
difyBaseUrl?: string
}
// Dify API Response Types
export interface DifyFileResponse {
id: string
name: string
size: number
extension: string
mime_type: string
created_by: string
created_at: number
}
export interface DifyMessage {
id: string
conversation_id: string
inputs: Record<string, any>
query: string
message_files: Array<{
id: string
type: string
url: string
belongs_to: string
}>
answer: string
created_at: number
feedback?: {
rating: string
}
retriever_resources?: any[]
}
interface DifyHistoryResponse {
data: DifyMessage[]
has_more: boolean
limit: number
}
interface DifyConversation {
id: string
name: string
inputs: Record<string, any>
status: string
introduction: string
created_at: number
updated_at: number
}
interface DifyConversationsResponse {
data: DifyConversation[]
has_more: boolean
limit: number
}
interface DifyConversationResponse {
id: string
name: string
inputs: Record<string, any>
status: string
introduction: string
created_at: number
updated_at: number
}
export class DifyHandler implements ApiHandler {
private options: DifyHandlerOptions
private baseUrl: string
private apiKey: string
private conversationId: string | null = null
private currentTaskId: string | null = null
private abortController: AbortController | null = null
constructor(options: DifyHandlerOptions) {
this.options = options
this.apiKey = options.difyApiKey || ""
this.baseUrl = options.difyBaseUrl || ""
Logger.log("[DIFY DEBUG] Constructor called with:", {
hasApiKey: !!this.apiKey,
baseUrl: this.baseUrl,
})
if (!this.apiKey) {
throw new Error("Dify API key is required")
}
if (!this.baseUrl) {
throw new Error("Dify base URL is required")
}
}
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
Logger.log("[DIFY DEBUG] createMessage called with:", {
systemPromptLength: systemPrompt?.length || 0,
messagesCount: messages?.length || 0,
})
// Convert messages to Dify format
const query = this.convertMessagesToQuery(systemPrompt, messages)
const requestBody = {
inputs: {},
query: query,
response_mode: "streaming",
conversation_id: this.conversationId || "",
user: "cline-user", // A unique user identifier
files: [],
}
const fullUrl = `${this.baseUrl}/chat-messages`
Logger.log("[DIFY DEBUG] Making request to:", fullUrl)
Logger.log("[DIFY DEBUG] Request body:", JSON.stringify(requestBody, null, 2))
let response: Response
try {
response = await fetch(fullUrl, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify(requestBody),
})
} catch (error: any) {
Logger.error("[DIFY DEBUG] Network error during fetch:", error)
const cause = error.cause ? ` | Cause: ${error.cause}` : ""
throw new Error(`Dify API network error: ${error.message}${cause}`)
}
Logger.log("[DIFY DEBUG] Response status:", response.status)
const headersObj: Record<string, string> = {}
response.headers.forEach((value, key) => {
headersObj[key] = value
})
Logger.log("[DIFY DEBUG] Response headers:", headersObj)
if (!response.ok) {
const errorText = await response.text()
Logger.error("[DIFY DEBUG] Error response:", errorText)
throw new Error(`Dify API error: ${response.status} ${response.statusText} - ${errorText}`)
}
if (!response.body) {
throw new Error("No response body from Dify API")
}
const reader = response.body.getReader()
const decoder = new TextDecoder()
let buffer = ""
let fullText = ""
let hasYieldedContent = false
const processedEvents: string[] = []
let lastEventTime = Date.now()
Logger.log("[DIFY DEBUG] Starting to read streaming response...")
try {
while (true) {
const { done, value } = await reader.read()
if (done) {
Logger.log("[DIFY DEBUG] Stream ended naturally")
Logger.log(
"[DIFY DEBUG] Final state - hasYieldedContent:",
hasYieldedContent,
"fullText length:",
fullText.length,
"processedEvents:",
processedEvents,
)
break
}
const chunk = decoder.decode(value, { stream: true })
Logger.log("[DIFY DEBUG] Raw chunk received:", JSON.stringify(chunk))
buffer += chunk
const lines = buffer.split("\n")
// Keep the last incomplete line in the buffer
buffer = lines.pop() || ""
for (const line of lines) {
Logger.log("[DIFY DEBUG] Processing line:", JSON.stringify(line))
if (line.startsWith("data: ")) {
const data = line.slice(6).trim()
Logger.log("[DIFY DEBUG] Extracted data:", JSON.stringify(data))
if (data === "[DONE]") {
Logger.log("[DIFY DEBUG] Received [DONE] signal")
break
}
if (data === "") {
Logger.log("[DIFY DEBUG] Empty data line, skipping")
continue
}
try {
const parsed = JSON.parse(data)
Logger.log("[DIFY DEBUG] Parsed JSON:", parsed)
processedEvents.push(parsed.event || "unknown")
lastEventTime = Date.now()
// Capture conversation_id as soon as it's available
if (parsed.conversation_id && !this.conversationId) {
this.conversationId = parsed.conversation_id
Logger.log("[DIFY DEBUG] Captured conversation_id:", this.conversationId)
}
// Handle different Dify event types based on actual Dify API
if (parsed.event === "message") {
Logger.log("[DIFY DEBUG] Message event, answer:", parsed.answer)
// Dify sends the full text in each "answer" chunk, so we replace.
if (typeof parsed.answer === "string") {
fullText = parsed.answer
Logger.log("[DIFY DEBUG] Updated fullText length:", fullText.length)
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
} else if (parsed.event === "message_replace") {
Logger.log("[DIFY DEBUG] Replace message event:", parsed)
if (parsed.answer) {
fullText = parsed.answer // Replace instead of append
Logger.log("[DIFY DEBUG] Replaced fullText length:", fullText.length)
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
} else if (parsed.event === "message_end") {
Logger.log("[DIFY DEBUG] Message end event", parsed)
// Message completed. Yield final text if we have any.
if (fullText) {
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
// Yield usage data if available
if (parsed.usage) {
yield {
type: "usage",
inputTokens: parsed.usage.prompt_tokens || 0,
outputTokens: parsed.usage.completion_tokens || parsed.usage.total_tokens || 0,
totalCost: parsed.usage.total_price || 0,
}
}
return // End of stream
} else if (parsed.event === "error") {
Logger.error("[DIFY DEBUG] Error event:", parsed)
throw new Error(`Dify API error: ${parsed.message || "Unknown error"}`)
} else if (parsed.event === "workflow_started" || parsed.event === "workflow_finished") {
Logger.log("[DIFY DEBUG] Workflow event:", parsed.event)
// These are informational events, continue processing
} else if (parsed.event === "node_started" || parsed.event === "node_finished") {
Logger.log("[DIFY DEBUG] Node event:", parsed.event, parsed.data)
// These are informational events, continue processing
} else if (parsed.event === "ping") {
Logger.log("[DIFY DEBUG] Ping event received, keeping connection alive.")
// Ping event, do nothing
} else {
Logger.log("[DIFY DEBUG] Unknown event type:", parsed.event, "Full object:", parsed)
// Try to extract text from other possible fields
if (parsed.text) {
fullText += parsed.text
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
} else if (parsed.content) {
fullText += parsed.content
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
} else if (parsed.answer) {
// Fallback: some events might have answer field even if not "message" type
fullText += parsed.answer
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
}
} catch (e) {
Logger.warn("[DIFY DEBUG] Failed to parse JSON:", data, "Error:", e)
}
} else if (line.trim() !== "") {
Logger.log(
"[DIFY DEBUG] Non-data line (not starting with 'data:'), trying to parse as direct JSON:",
JSON.stringify(line),
)
// Try to parse as direct JSON (fallback for non-SSE responses, though Dify uses SSE)
try {
const parsed = JSON.parse(line.trim())
Logger.log("[DIFY DEBUG] Parsed direct JSON:", parsed)
processedEvents.push(parsed.event || "direct-json")
// Handle the same event types as above
if (parsed.event === "message" && parsed.answer) {
fullText += parsed.answer
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
} else if (parsed.event === "message_end") {
if (fullText) {
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
return
} else if (parsed.event === "error") {
Logger.error("[DIFY DEBUG] Direct JSON Error event:", parsed)
throw new Error(`Dify API error: ${parsed.message || "Unknown error"}`)
} else if (parsed.answer || parsed.text || parsed.content) {
// Fallback for any content in direct JSON
const content = parsed.answer || parsed.text || parsed.content
fullText += content
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
} catch (e) {
// Not JSON, continue
Logger.log("[DIFY DEBUG] Line is not direct JSON, continuing")
}
}
}
}
// Final check - if we haven't yielded any content, provide diagnostic information
if (!hasYieldedContent) {
const diagnosticInfo = {
processedEvents,
finalFullTextLength: fullText.length,
finalFullText: fullText,
streamDuration: Date.now() - lastEventTime,
conversationId: this.conversationId,
}
Logger.error("[DIFY DEBUG] No content was yielded! Diagnostic info:", diagnosticInfo)
// If we have any accumulated text at all, yield it as a fallback
if (fullText.trim()) {
Logger.log("[DIFY DEBUG] Yielding accumulated text as fallback:", fullText)
yield {
type: "text",
text: fullText,
}
} else {
// Provide a more informative error
throw new Error(
`Dify API did not provide any assistant messages. ` +
`Events processed: [${processedEvents.join(", ")}]. ` +
`Check your Dify application configuration and ensure it's properly set up to return responses. ` +
`API URL: ${fullUrl}. Conversation ID: ${this.conversationId || "none"}.`,
)
}
}
} finally {
reader.releaseLock()
Logger.log("[DIFY DEBUG] Stream reader released")
}
}
private convertMessagesToQuery(systemPrompt: string, messages: ClineStorageMessage[]): string {
// Dify's context is managed by `conversation_id`. The `query` should be the last user message.
// The system prompt is typically configured in the Dify App itself.
const lastUserMessage = messages.filter((m) => m.role === "user").pop()
if (!lastUserMessage) {
return "" // Should not happen in normal flow
}
const userQuery = Array.isArray(lastUserMessage.content)
? lastUserMessage.content.map((c) => ("text" in c ? c.text : "")).join("\n")
: (lastUserMessage.content as string)
// Only prepend the system prompt if it's the very first message of a new conversation.
if (!this.conversationId && systemPrompt) {
Logger.log("[DIFY DEBUG] Prepending system prompt for new conversation.")
return `${systemPrompt}\n\n---\n\n${userQuery}`
}
return userQuery
}
getModel(): { id: string; info: ModelInfo } {
return {
id: "dify-workflow",
info: {
maxTokens: 8192,
contextWindow: 128000,
supportsImages: true,
supportsPromptCache: false,
inputPrice: 0,
outputPrice: 0,
description: "Dify workflow - model selection is configured in your Dify application",
},
}
}
// Additional Dify API Methods
/**
* Upload a file for use in conversations
* @param file File buffer to upload
* @param filename Name of the file
* @param user User identifier (defaults to "cline-user")
* @returns Promise with file upload response
*/
async uploadFile(file: Buffer, filename: string, user: string = "cline-user"): Promise<DifyFileResponse> {
const formData = new FormData()
formData.append("file", new Blob([new Uint8Array(file)]), filename)
formData.append("user", user)
const response = await fetch(`${this.baseUrl}/files/upload`, {
method: "POST",
headers: this.headers(),
body: formData,
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify file upload error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Stop generation for a specific task
* @param taskId Task ID from streaming response
* @param user User identifier (defaults to "cline-user")
* @returns Promise that resolves when generation is stopped
*/
async stopGeneration(taskId: string, user: string = "cline-user"): Promise<void> {
const response = await fetch(`${this.baseUrl}/chat-messages/${taskId}/stop`, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify({ user }),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify stop generation error: ${response.status} ${response.statusText} - ${errorText}`)
}
}
/**
* Get conversation history messages with pagination
* @param conversationId Conversation ID
* @param user User identifier (defaults to "cline-user")
* @param firstId First message ID for pagination (optional)
* @param limit Number of messages to return (default: 20)
* @returns Promise with conversation history
*/
async getConversationHistory(
conversationId: string,
user: string = "cline-user",
firstId?: string,
limit: number = 20,
): Promise<DifyHistoryResponse> {
const params = new URLSearchParams({ user, limit: limit.toString() })
if (firstId) {
params.append("first_id", firstId)
}
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}/messages?${params}`, {
headers: this.headers(),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify get conversation history error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Get list of conversations for a user
* @param user User identifier (defaults to "cline-user")
* @param lastId Last conversation ID for pagination (optional)
* @param limit Number of conversations to return (default: 20)
* @param sortBy Sort field (default: "-updated_at")
* @returns Promise with conversations list
*/
async getConversations(
user: string = "cline-user",
lastId?: string,
limit: number = 20,
sortBy: string = "-updated_at",
): Promise<DifyConversationsResponse> {
const params = new URLSearchParams({
user,
limit: limit.toString(),
sort_by: sortBy,
})
if (lastId) {
params.append("last_id", lastId)
}
const response = await fetch(`${this.baseUrl}/conversations?${params}`, {
headers: this.headers(),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify get conversations error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Delete a conversation
* @param conversationId Conversation ID to delete
* @param user User identifier (defaults to "cline-user")
* @returns Promise that resolves when conversation is deleted
*/
async deleteConversation(conversationId: string, user: string = "cline-user"): Promise<void> {
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}`, {
method: "DELETE",
headers: this.jsonHeaders(),
body: JSON.stringify({ user }),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify delete conversation error: ${response.status} ${response.statusText} - ${errorText}`)
}
}
/**
* Rename a conversation
* @param conversationId Conversation ID to rename
* @param user User identifier (defaults to "cline-user")
* @param name New conversation name (optional if auto_generate is true)
* @param autoGenerate Whether to auto-generate the name (default: false)
* @returns Promise with updated conversation details
*/
async renameConversation(
conversationId: string,
user: string = "cline-user",
name?: string,
autoGenerate: boolean = false,
): Promise<DifyConversationResponse> {
const body: any = { user, auto_generate: autoGenerate }
if (name) {
body.name = name
}
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}/name`, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify(body),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify rename conversation error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Submit feedback for a message
* @param messageId Message ID to provide feedback for
* @param rating Rating: "like" or "dislike"
* @param content Optional feedback content
* @param user User identifier (defaults to "cline-user")
* @returns Promise that resolves when feedback is submitted
*/
async submitMessageFeedback(
messageId: string,
rating: "like" | "dislike",
content?: string,
user: string = "cline-user",
): Promise<void> {
const body: any = { rating, user }
if (content) {
body.content = content
}
const response = await fetch(`${this.baseUrl}/messages/${messageId}/feedbacks`, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify(body),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify submit feedback error: ${response.status} ${response.statusText} - ${errorText}`)
}
}
/**
* Get current conversation ID
* @returns Current conversation ID or null
*/
getCurrentConversationId(): string | null {
return this.conversationId
}
/**
* Set conversation ID for continuing existing conversations
* @param conversationId Conversation ID to set
*/
setConversationId(conversationId: string): void {
this.conversationId = conversationId
}
/**
* Reset conversation ID to start a new conversation
*/
resetConversation(): void {
this.conversationId = null
this.currentTaskId = null
}
private jsonHeaders() {
return {
...this.headers(),
"Content-Type": "application/json",
}
}
private headers() {
const externalHeaders = buildExternalBasicHeaders()
return {
...externalHeaders,
Authorization: `Bearer ${this.apiKey}`,
}
}
}
@@ -1,90 +0,0 @@
import { DoubaoModelId, doubaoDefaultModelId, doubaoModels, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface DoubaoHandlerOptions extends CommonApiHandlerOptions {
doubaoApiKey?: string
apiModelId?: string
}
export class DoubaoHandler implements ApiHandler {
private options: DoubaoHandlerOptions
private client: OpenAI | undefined
constructor(options: DoubaoHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.doubaoApiKey) {
throw new Error("Doubao API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://ark.cn-beijing.volces.com/api/v3/",
apiKey: this.options.doubaoApiKey,
})
} catch (error) {
throw new Error(`Error creating Doubao client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: DoubaoModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in doubaoModels) {
const id = modelId as DoubaoModelId
return { id, info: doubaoModels[id] }
}
return {
id: doubaoDefaultModelId,
info: doubaoModels[doubaoDefaultModelId],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
// @ts-expect-error-next-line
cacheReadTokens: chunk.usage.prompt_cache_hit_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
}
@@ -1,116 +0,0 @@
import { FireworksModelId, fireworksDefaultModelId, fireworksModels, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface FireworksHandlerOptions extends CommonApiHandlerOptions {
fireworksApiKey?: string
fireworksModelId?: string
fireworksModelMaxCompletionTokens?: number
fireworksModelMaxTokens?: number
}
export class FireworksHandler implements ApiHandler {
private options: FireworksHandlerOptions
private client: OpenAI | undefined
constructor(options: FireworksHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.fireworksApiKey) {
throw new Error("Fireworks API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.fireworks.ai/inference/v1",
apiKey: this.options.fireworksApiKey,
})
} catch (error) {
throw new Error(`Error creating Fireworks client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.fireworksModelId ?? ""
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
})
let reasoning: string | null = null
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (reasoning || delta?.content?.includes("<think>")) {
reasoning = (reasoning || "") + (delta.content ?? "")
}
if (delta?.content && !reasoning) {
yield {
type: "text",
text: delta.content,
}
}
if (reasoning || (delta && "reasoning_content" in delta && delta.reasoning_content)) {
yield {
type: "reasoning",
reasoning: delta.content || ((delta as any).reasoning_content as string | undefined) || "",
}
if (reasoning?.includes("</think>")) {
// Reset so the next chunk is regular content
reasoning = null
}
}
if (chunk.usage) {
const usage = chunk.usage as OpenAI.CompletionUsage & {
prompt_cache_hit_tokens?: number
prompt_cache_miss_tokens?: number
prompt_tokens_details?: {
cached_tokens?: number
}
}
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
// Fireworks can return cache hits either as prompt_cache_hit_tokens or prompt_tokens_details.cached_tokens.
cacheReadTokens: usage.prompt_cache_hit_tokens ?? usage.prompt_tokens_details?.cached_tokens ?? 0,
cacheWriteTokens: usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: FireworksModelId; info: ModelInfo } {
const modelId = this.options.fireworksModelId
if (modelId && modelId in fireworksModels) {
const id = modelId as FireworksModelId
return { id, info: fireworksModels[id] }
}
return {
id: fireworksDefaultModelId,
info: fireworksModels[fireworksDefaultModelId],
}
}
}
@@ -1,570 +0,0 @@
// Restore GenerateContentConfig import and add GenerateContentResponseUsageMetadata
import {
ApiError,
FunctionCallingConfigMode,
type GenerateContentConfig,
type GenerateContentResponseUsageMetadata,
GoogleGenAI,
FunctionDeclaration as GoogleTool,
ThinkingLevel,
} from "@google/genai"
import { GeminiModelId, geminiDefaultModelId, geminiModels, ModelInfo } from "@shared/api"
import { GEMINI_FLASH_MAX_OUTPUT_TOKENS, isGeminiFlashModel } from "@utils/model-utils"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { telemetryService } from "@/services/telemetry"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { RetriableError, withRetry } from "../retry"
import { convertAnthropicMessageToGemini } from "../transform/gemini-format"
import { ApiStream } from "../transform/stream"
const rateLimitPatterns = [/got status: 429/i, /429 Too Many Requests/i, /rate limit exceeded/i, /too many requests/i]
interface GeminiHandlerOptions extends CommonApiHandlerOptions {
isVertex?: boolean
vertexProjectId?: string
vertexRegion?: string
geminiApiKey?: string
geminiBaseUrl?: string
thinkingBudgetTokens?: number
reasoningEffort?: string
apiModelId?: string
ulid?: string
}
function mapReasoningEffortToGeminiThinkingLevel(effort: string): ThinkingLevel {
switch (effort) {
case "low":
case "medium":
return ThinkingLevel.LOW
case "high":
case "xhigh":
return ThinkingLevel.HIGH
default:
return ThinkingLevel.LOW
}
}
function getGeminiMaxOutputTokens(modelId: string, modelMaxTokens?: number): number | undefined {
if (!isGeminiFlashModel(modelId)) {
return undefined
}
if (modelMaxTokens && modelMaxTokens > 0) {
return Math.min(modelMaxTokens, GEMINI_FLASH_MAX_OUTPUT_TOKENS)
}
return GEMINI_FLASH_MAX_OUTPUT_TOKENS
}
/**
* Handler for Google's Gemini API with optimized caching strategy and accurate cost accounting.
*
* Key features:
* - One cache per task: Creates a single cache per task and reuses it for subsequent turns
* - Stable cache keys: Uses ulid as a stable identifier for caches
* - Efficient cache updates: Only updates caches when there's new content to add
* - Split cost accounting: Separates immediate costs from ongoing cache storage costs
*
* Cost accounting approach:
* - Immediate costs (per message): Input tokens, output tokens, and cache read costs
* - Ongoing costs (per task): Cache storage costs for the TTL period
*
* Gemini's caching system is unique in that it charges for holding tokens in cache by the hour.
* This implementation optimizes for both performance and cost by:
* 1. Minimizing redundant cache creations
* 2. Properly accounting for cache costs in the billing calculations
* 3. Using a stable cache key to ensure cache reuse across turns
* 4. Separating immediate costs from ongoing costs to avoid double-counting
*/
export class GeminiHandler implements ApiHandler {
private options: GeminiHandlerOptions
private client: GoogleGenAI | undefined
constructor(options: GeminiHandlerOptions) {
// Store the options
this.options = options
}
private ensureClient(): GoogleGenAI {
if (!this.client) {
const options = this.options as GeminiHandlerOptions
const externalHeaders = buildExternalBasicHeaders()
if (options.isVertex) {
// Initialize with Vertex AI configuration
const project = this.options.vertexProjectId ?? "not-provided"
const location = this.options.vertexRegion ?? "not-provided"
try {
this.client = new GoogleGenAI({
vertexai: true,
project,
location,
httpOptions: {
headers: externalHeaders,
},
})
} catch (error) {
throw new Error(`Error creating Gemini Vertex AI client: ${error.message}`)
}
} else {
// Initialize with standard API key
if (!options.geminiApiKey) {
throw new Error("API key is required for Google Gemini when not using Vertex AI")
}
try {
this.client = new GoogleGenAI({
apiKey: options.geminiApiKey,
httpOptions: {
headers: externalHeaders,
},
})
} catch (error) {
throw new Error(`Error creating Gemini client: ${error.message}`)
}
}
}
return this.client
}
/**
* Creates a message using the Gemini API with implicit caching.
*
* Cost accounting:
* - Immediate costs (returned in the usage object): Input tokens, output tokens, cache read costs
*
* @param systemPrompt The system prompt to use for the message
* @param messages The conversation history to include in the message
* @returns An async generator that yields chunks of the response with accurate immediate costs
*/
@withRetry({
maxRetries: 4,
baseDelay: 2000,
maxDelay: 15000,
})
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: GoogleTool[]): ApiStream {
const client = this.ensureClient()
const { id: modelId, info } = this.getModel()
const contents = messages.map(convertAnthropicMessageToGemini)
// Gemini may emit multiple function calls under the same responseId and without functionCall.id.
// Track a local sequence so each emitted tool call has a stable unique ID.
const responseToolCallCount = new Map<string, number>()
// Configure thinking budget/level if supported
const _thinkingBudget = this.options.thinkingBudgetTokens ?? 0
const maxBudget = info.thinkingConfig?.maxBudget ?? 24576
const thinkingBudget = Math.min(_thinkingBudget, maxBudget)
// When ThinkingLevel is defined, thinking budget cannot be zero
// and only level is used to control thinking behavior.
// Only set thinkingLevel for models that support it
let thinkingLevel: ThinkingLevel | undefined
const rawReasoningEffort = (this.options.reasoningEffort || "").toLowerCase()
const normalizedReasoningEffort = !rawReasoningEffort || rawReasoningEffort === "none" ? "low" : rawReasoningEffort
if (info.thinkingConfig?.supportsThinkingLevel) {
thinkingLevel = mapReasoningEffortToGeminiThinkingLevel(normalizedReasoningEffort)
}
// Set up base generation config
const maxOutputTokens = getGeminiMaxOutputTokens(modelId, info.maxTokens)
const requestConfig: GenerateContentConfig = {
// Add base URL if configured
httpOptions: this.options.geminiBaseUrl ? { baseUrl: this.options.geminiBaseUrl } : undefined,
systemInstruction: systemPrompt,
// Set temperature (default to 0)
// Gemini 3 recommends 1.0
temperature: info.temperature ?? 1,
...(maxOutputTokens !== undefined ? { maxOutputTokens } : {}),
}
// Add thinking config only if the model supports it
if (info.thinkingConfig) {
requestConfig.thinkingConfig = {
// Turn off thinking:
// thinkingBudget: 0
// Turn on dynamic thinking:
// thinkingBudget: -1
// Turn on fixed thinking budget:
thinkingBudget: thinkingLevel ? undefined : thinkingBudget, // Use budget only if thinkingLevel is not set
thinkingLevel,
includeThoughts: thinkingBudget > 0 || !!thinkingLevel,
}
}
// Generate content using the configured parameters
const sdkCallStartTime = Date.now()
let responseId: string | undefined
let sdkFirstChunkTime: number | undefined
let ttftSdkMs: number | undefined
let apiSuccess = false
let apiError: string | undefined
let promptTokens = 0
let outputTokens = 0
let cacheReadTokens = 0
let thoughtsTokenCount = 0 // Initialize thought token counts
let lastUsageMetadata: GenerateContentResponseUsageMetadata | undefined
const isNativeToolCallsEnabled = tools?.length
if (isNativeToolCallsEnabled) {
requestConfig.tools = [{ functionDeclarations: tools }]
requestConfig.toolConfig = {
// Force the model to call 'any' function.
functionCallingConfig: {
mode: FunctionCallingConfigMode.ANY,
},
}
}
try {
const result = await client.models.generateContentStream({
model: modelId,
contents: contents,
config: {
...requestConfig,
},
})
let isFirstSdkChunk = true
for await (const chunk of result) {
const responseKey = chunk.responseId || "gemini-response"
if (isFirstSdkChunk) {
sdkFirstChunkTime = Date.now()
ttftSdkMs = sdkFirstChunkTime - sdkCallStartTime
isFirstSdkChunk = false
}
// Handle thinking content from Gemini's response
const parts = chunk?.candidates?.[0]?.content?.parts || []
for (const part of parts) {
if (part.thought && part.text) {
yield {
type: "reasoning",
id: chunk.responseId,
reasoning: part.text || "",
signature: part.thoughtSignature,
}
} else if (part.text) {
yield {
type: "text",
text: part.text,
id: chunk.responseId,
signature: part.thoughtSignature,
}
}
if (part.functionCall) {
const functionCall = part.functionCall
const args = Object.entries(functionCall.args || {}).filter(([_key, val]) => !!val)
if (functionCall.args && args.length > 0) {
const existingId = functionCall.id?.trim()
const toolCallId =
existingId ??
(() => {
const sequenceNumber = responseToolCallCount.get(responseKey) ?? 0
responseToolCallCount.set(responseKey, sequenceNumber + 1)
return `${responseKey}-tool-${sequenceNumber}`
})()
yield {
type: "tool_calls",
id: chunk.responseId,
tool_call: {
call_id: toolCallId,
function: {
id: toolCallId,
name: functionCall.name,
arguments: JSON.stringify(functionCall.args),
},
},
signature: part.thoughtSignature,
}
}
}
}
if (chunk.usageMetadata) {
responseId = chunk.responseId
lastUsageMetadata = chunk.usageMetadata
promptTokens = lastUsageMetadata.promptTokenCount ?? promptTokens
outputTokens = lastUsageMetadata.candidatesTokenCount ?? outputTokens
thoughtsTokenCount = lastUsageMetadata.thoughtsTokenCount ?? thoughtsTokenCount
cacheReadTokens = lastUsageMetadata.cachedContentTokenCount ?? cacheReadTokens
}
}
apiSuccess = true
if (lastUsageMetadata) {
const totalCost = this.calculateCost({
info,
inputTokens: promptTokens,
outputTokens,
thoughtsTokenCount,
cacheReadTokens,
})
yield {
type: "usage",
inputTokens: promptTokens - cacheReadTokens,
outputTokens,
thoughtsTokenCount,
cacheReadTokens,
cacheWriteTokens: 0,
totalCost,
id: responseId,
}
}
} catch (error) {
apiSuccess = false
// Let the error propagate to be handled by withRetry or Task.ts
// Telemetry will be sent in the finally block.
if (error instanceof Error) {
apiError = error.message
if (error instanceof ApiError) {
if (error.status === 429) {
// The API includes more details in the message
// https://github.com/googleapis/js-genai/blob/v1.11.0/src/_api_client.ts#L758
const response = this.attemptParse(error.message)
if (response && response.error) {
const responseBody = this.attemptParse(response.error.message)
if (responseBody.error) {
const detail = responseBody.error.details?.find(
(d: any) => d["@type"] === "type.googleapis.com/google.rpc.RetryInfo",
)
const detailedError = new RetriableError(
apiError,
this.parseRetryDelay(detail?.retryDelay) || undefined,
{
cause: error,
},
)
throw detailedError
}
}
throw new RetriableError(apiError, undefined, { cause: error })
}
// Fallback in case Gemini throws a rate limit error without a 429 status code
// https://github.com/cline/cline/pull/5205#discussion_r2311761559
const isRateLimit = rateLimitPatterns.some((pattern) => pattern.test(error.message))
if (isRateLimit) {
throw new RetriableError(apiError, undefined, { cause: error })
}
}
} else {
apiError = String(error)
}
throw error
} finally {
const sdkCallEndTime = Date.now()
const totalDurationSdkMs = sdkCallEndTime - sdkCallStartTime
const cacheHit = cacheReadTokens > 0
const cacheHitPercentage = promptTokens > 0 ? (cacheReadTokens / promptTokens) * 100 : undefined
const throughputTokensPerSecSdk =
totalDurationSdkMs > 0 && outputTokens > 0 ? outputTokens / (totalDurationSdkMs / 1000) : undefined
if (this.options.ulid) {
telemetryService.captureGeminiApiPerformance(this.options.ulid, modelId, {
ttftSec: ttftSdkMs !== undefined ? ttftSdkMs / 1000 : undefined,
totalDurationSec: totalDurationSdkMs / 1000,
promptTokens,
outputTokens,
cacheReadTokens,
cacheHit,
cacheHitPercentage,
apiSuccess,
apiError,
throughputTokensPerSec: throughputTokensPerSecSdk,
})
} else {
Logger.warn("GeminiHandler: ulid not available for telemetry in createMessage.")
}
}
}
/**
* Calculate the immediate dollar cost of the API call based on token usage and model pricing.
*
* This method accounts for the immediate costs of the API call:
* - Input token costs (for uncached tokens)
* - Output token costs
* - Cache read costs
* - Gemini implicit caching has no write costs
*
*/
public calculateCost({
info,
inputTokens,
outputTokens,
thoughtsTokenCount = 0,
cacheReadTokens = 0,
}: {
info: ModelInfo
inputTokens: number
outputTokens: number
thoughtsTokenCount: number
cacheReadTokens?: number
}) {
// Exit early if any required pricing information is missing
if (!info.inputPrice || !info.outputPrice) {
return undefined
}
let inputPrice = info.inputPrice
let outputPrice = info.outputPrice
// Right now, we only show the immediate costs of caching and not the ongoing costs of storing the cache
let cacheReadsPrice = info.cacheReadsPrice ?? 0
// If there's tiered pricing then adjust prices based on the input tokens used
if (info.tiers) {
const tier = info.tiers.find((tier) => inputTokens <= tier.contextWindow)
if (tier) {
inputPrice = tier.inputPrice ?? inputPrice
outputPrice = tier.outputPrice ?? outputPrice
cacheReadsPrice = tier.cacheReadsPrice ?? cacheReadsPrice
}
}
// Subtract the cached input tokens from the total input tokens
const uncachedInputTokens = inputTokens - (cacheReadTokens ?? 0)
// Calculate immediate costs only
// 1. Input token costs (for uncached tokens)
const inputTokensCost = inputPrice * (uncachedInputTokens / 1_000_000)
// 2. Output token costs
const responseTokensCost = outputPrice * ((outputTokens + thoughtsTokenCount) / 1_000_000)
// 3. Cache read costs (immediate)
const cacheReadCost = (cacheReadTokens ?? 0) > 0 ? cacheReadsPrice * ((cacheReadTokens ?? 0) / 1_000_000) : 0
// Calculate total immediate cost (excluding cache write/storage costs)
const totalCost = inputTokensCost + responseTokensCost + cacheReadCost
// Create the trace object for debugging
const trace: Record<string, { price: number; tokens: number; cost: number }> = {
input: { price: inputPrice, tokens: uncachedInputTokens, cost: inputTokensCost },
output: { price: outputPrice, tokens: outputTokens, cost: responseTokensCost },
}
// Only include cache read costs in the trace (cache write costs are tracked separately)
if ((cacheReadTokens ?? 0) > 0) {
trace.cacheRead = { price: cacheReadsPrice, tokens: cacheReadTokens ?? 0, cost: cacheReadCost }
}
// Logger.log(`[GeminiHandler] calculateCost -> ${totalCost}`, trace)
return totalCost
}
/**
* Get the model ID and info for the current configuration
*/
getModel(): { id: GeminiModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in geminiModels) {
const id = modelId as GeminiModelId
return { id, info: geminiModels[id] }
}
return {
id: geminiDefaultModelId,
info: geminiModels[geminiDefaultModelId],
}
}
/**
* Count tokens in content using the Gemini API
*/
async countTokens(content: Array<any>): Promise<number> {
try {
const client = this.ensureClient()
const { id: model } = this.getModel()
// Convert content to Gemini format
const geminiContent = content.map((block) => {
if (typeof block === "string") {
return { text: block }
}
return { text: JSON.stringify(block) }
})
// Use Gemini's token counting API
const response = await client.models.countTokens({
model,
contents: [{ parts: geminiContent }],
})
if (response.totalTokens === undefined) {
Logger.warn("Gemini token counting returned undefined, using fallback")
return this.estimateTokens(content)
}
return response.totalTokens
} catch (error) {
Logger.warn("Gemini token counting failed, using fallback", error)
return this.estimateTokens(content)
}
}
/**
* Fallback token estimation method
*/
private estimateTokens(content: Array<any>): number {
// Simple estimation: ~4 characters per token
const totalChars = content.reduce((total, block) => {
if (typeof block === "string") {
return total + block.length
}
if (block && typeof block === "object") {
// Safely stringify the object
try {
const jsonStr = JSON.stringify(block)
return total + jsonStr.length
} catch (e) {
Logger.warn("Failed to stringify block for token estimation", e)
return total
}
}
return total
}, 0)
return Math.ceil(totalChars / 4)
}
private parseRetryDelay(retryAfter?: string): number {
if (!retryAfter) {
return 0
}
const unit = retryAfter.at(-1)
const value = Number.parseInt(retryAfter, 10)
if (Number.isNaN(value)) {
return 0
}
if (unit === "s") {
return value
}
if (unit === "m") {
return value * 60 // Convert minutes to seconds
}
if (unit === "h") {
return value * 60 * 60 // Convert hours to seconds
}
return value
}
private attemptParse(str: string) {
try {
return JSON.parse(str)
} catch (_) {
return null
}
}
}
-320
View File
@@ -1,320 +0,0 @@
import { GroqModelId, groqDefaultModelId, groqModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface GroqHandlerOptions extends CommonApiHandlerOptions {
groqApiKey?: string
groqModelId?: string
groqModelInfo?: ModelInfo
apiModelId?: string // For backward compatibility
}
// Enhanced usage interface to support Groq's cached token fields
interface GroqUsage extends OpenAI.CompletionUsage {
prompt_tokens_details?: {
cached_tokens?: number
}
}
// Model family definitions for enhanced behavior
interface GroqModelFamily {
name: string
supportedFeatures: {
streaming: boolean
temperature: boolean
vision: boolean
tools: boolean
}
maxTokensOverride?: number
specialParams?: Record<string, any>
}
const MODEL_FAMILIES: Record<string, GroqModelFamily> = {
// Moonshort 4 Family - Latest generation with vision support
"kimi-k2": {
name: "kimi-k2",
supportedFeatures: { streaming: true, temperature: true, vision: true, tools: true },
maxTokensOverride: 8192,
},
// Llama 4 Family - Latest generation with vision support
llama4: {
name: "Llama 4",
supportedFeatures: { streaming: true, temperature: true, vision: true, tools: true },
maxTokensOverride: 8192,
},
// Llama 3.3 Family - Balanced performance
"llama3.3": {
name: "Llama 3.3",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 32768,
},
// Llama 3.1 Family - Fast inference
"llama3.1": {
name: "Llama 3.1",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 131072,
},
// DeepSeek Family - Reasoning-optimized
deepseek: {
name: "DeepSeek",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 8192,
specialParams: {
top_p: 0.95,
reasoning_format: "parsed",
},
},
// Qwen Family - Enhanced for Q&A
qwen: {
name: "Qwen",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 32768,
},
// Compound Models - Hybrid architectures
compound: {
name: "Compound",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 8192,
},
}
export class GroqHandler implements ApiHandler {
private options: GroqHandlerOptions
private client: OpenAI | undefined
constructor(options: GroqHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.groqApiKey) {
throw new Error("Groq API key is required")
}
try {
this.client = new OpenAI({
baseURL: "https://api.groq.com/openai/v1",
apiKey: this.options.groqApiKey,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating Groq client: ${error.message}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: GroqUsage | undefined): ApiStream {
const inputTokens = usage?.prompt_tokens || 0
const outputTokens = usage?.completion_tokens || 0
const cacheReadTokens = usage?.prompt_tokens_details?.cached_tokens || 0
// Groq does not track cache writes
const cacheWriteTokens = 0
// Calculate cost using OpenAI-compatible cost calculation
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
// Calculate non-cached input tokens for proper reporting
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens,
cacheWriteTokens,
cacheReadTokens,
totalCost,
}
}
/**
* Detects the model family based on the model ID
*/
private detectModelFamily(modelId: string): GroqModelFamily {
if (modelId.includes("kimi-k2")) {
return MODEL_FAMILIES["kimi-k2"]
}
// Llama 4 variants
if (modelId.includes("llama-4") || modelId.includes("llama/llama-4")) {
return MODEL_FAMILIES.llama4
}
// Llama 3.3 variants
if (modelId.includes("llama-3.3")) {
return MODEL_FAMILIES["llama3.3"]
}
// Llama 3.1 variants
if (modelId.includes("llama-3.1")) {
return MODEL_FAMILIES["llama3.1"]
}
// DeepSeek variants
if (modelId.includes("deepseek")) {
return MODEL_FAMILIES.deepseek
}
// Qwen variants
if (modelId.includes("qwen")) {
return MODEL_FAMILIES.qwen
}
// Compound variants
if (modelId.includes("compound")) {
return MODEL_FAMILIES.compound
}
// Default fallback to Llama 3.3 behavior
return MODEL_FAMILIES["kimi-k2"]
}
/**
* Gets the optimal max_tokens based on model family and capabilities
*/
private getOptimalMaxTokens(model: { id: string; info: ModelInfo }, modelFamily: GroqModelFamily): number {
// Use model-specific max tokens if available
if (model.info.maxTokens && model.info.maxTokens > 0) {
return model.info.maxTokens
}
// Use family override if available
if (modelFamily.maxTokensOverride) {
return modelFamily.maxTokensOverride
}
// Default fallback
return 8192
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const modelFamily = this.detectModelFamily(model.id)
// Optimize parameters based on model family
const temperature = 0
const maxTokens = this.getOptimalMaxTokens(model, modelFamily)
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
// Build request parameters with model-specific optimizations
const requestParams: OpenAI.Chat.ChatCompletionCreateParamsStreaming & {
reasoning_format?: "parsed" | "raw" | "hidden"
top_p?: number
} = {
model: model.id,
max_tokens: maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature,
...getOpenAIToolParams(tools),
}
// Add any special parameters for specific model families
if (modelFamily.specialParams) {
Object.assign(requestParams, modelFamily.specialParams)
}
const toolCallProcessor = new ToolCallProcessor()
const stream = await client.chat.completions.create(requestParams)
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle reasoning field if present (for reasoning models with parsed output)
if ((delta as any)?.reasoning) {
const reasoningContent = (delta as any).reasoning as string
yield {
type: "reasoning",
reasoning: reasoningContent,
}
continue
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle content field - trust the parsed output from Groq
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
// Handle usage information
if (chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
}
}
}
/**
* Checks if the current model supports vision/images
*/
supportsImages(): boolean {
const model = this.getModel()
return model.info.supportsImages === true
}
/**
* Checks if the current model supports tools
*/
supportsTools(): boolean {
const model = this.getModel()
const modelFamily = this.detectModelFamily(model.id)
return modelFamily.supportedFeatures.tools
}
/**
* Gets model information with enhanced family detection
*/
getModel(): { id: string; info: ModelInfo } {
// First priority: groqModelId and groqModelInfo (like Requesty does)
const groqModelId = this.options.groqModelId
const groqModelInfo = this.options.groqModelInfo
if (groqModelId && groqModelInfo) {
return { id: groqModelId, info: groqModelInfo }
}
// Second priority: groqModelId with static model info
if (groqModelId && groqModelId in groqModels) {
const id = groqModelId as GroqModelId
return { id, info: groqModels[id] }
}
// Third priority: apiModelId (for backward compatibility)
const apiModelId = this.options.apiModelId
if (apiModelId && apiModelId in groqModels) {
const id = apiModelId as GroqModelId
return { id, info: groqModels[id] }
}
// Default fallback
return {
id: groqDefaultModelId,
info: groqModels[groqDefaultModelId],
}
}
/**
* Gets model family information for debugging/introspection
*/
getModelFamily(): GroqModelFamily {
const model = this.getModel()
return this.detectModelFamily(model.id)
}
}
-104
View File
@@ -1,104 +0,0 @@
import { hicapModelInfoSaneDefaults, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionReasoningEffort } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface OpenAiHandlerOptions extends CommonApiHandlerOptions {
hicapApiKey?: string
hicapModelId?: string
}
export class HicapHandler implements ApiHandler {
private options: OpenAiHandlerOptions
private client: OpenAI | undefined
constructor(options: OpenAiHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.hicapApiKey) {
throw new Error("Hicap API key is required")
}
if (!this.options.hicapModelId) {
throw new Error("Model ID is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.hicap.ai/v2/openai",
apiKey: this.options.hicapApiKey,
defaultHeaders: {
"api-key": this.options.hicapApiKey,
},
})
} catch (error: any) {
throw new Error(`Error creating OpenAI client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.hicapModelId ?? ""
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const temperature: number = 1
let reasoningEffort: ChatCompletionReasoningEffort | undefined
let maxTokens: number | undefined
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
temperature,
max_tokens: maxTokens,
reasoning_effort: reasoningEffort,
stream: true,
stream_options: { include_usage: true },
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.hicapModelId ?? "",
info: hicapModelInfoSaneDefaults,
}
}
}
@@ -1,142 +0,0 @@
import { HuaweiCloudMaasModelId, huaweiCloudMaasDefaultModelId, huaweiCloudMaasModels, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface HuaweiCloudMaaSHandlerOptions extends CommonApiHandlerOptions {
huaweiCloudMaasApiKey?: string
huaweiCloudMaasModelId?: string
huaweiCloudMaasModelInfo?: ModelInfo
}
export class HuaweiCloudMaaSHandler implements ApiHandler {
private options: HuaweiCloudMaaSHandlerOptions
private client: OpenAI | undefined
constructor(options: HuaweiCloudMaaSHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.huaweiCloudMaasApiKey) {
throw new Error("Huawei Cloud MaaS API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.modelarts-maas.com/v1/",
apiKey: this.options.huaweiCloudMaasApiKey,
})
} catch (error) {
throw new Error(`Error creating Huawei Cloud MaaS client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: HuaweiCloudMaasModelId; info: ModelInfo } {
// First priority: huaweiCloudMaasModelId and huaweiCloudMaasModelInfo (like Groq does)
const huaweiCloudMaasModelId = this.options.huaweiCloudMaasModelId
const huaweiCloudMaasModelInfo = this.options.huaweiCloudMaasModelInfo
if (huaweiCloudMaasModelId && huaweiCloudMaasModelInfo) {
return { id: huaweiCloudMaasModelId as HuaweiCloudMaasModelId, info: huaweiCloudMaasModelInfo }
}
// Second priority: huaweiCloudMaasModelId with static model info
if (huaweiCloudMaasModelId && huaweiCloudMaasModelId in huaweiCloudMaasModels) {
const id = huaweiCloudMaasModelId as HuaweiCloudMaasModelId
return { id, info: huaweiCloudMaasModels[id] }
}
// Default fallback
return {
id: huaweiCloudMaasDefaultModelId,
info: huaweiCloudMaasModels[huaweiCloudMaasDefaultModelId],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
...getOpenAIToolParams(tools),
})
let reasoning: string | null = null
let didOutputUsage: boolean = false
let finalUsage: any = null
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle reasoning content detection
if (delta?.content) {
if (reasoning || delta.content.includes("<think>")) {
reasoning = (reasoning || "") + delta.content
} else if (!reasoning) {
yield {
type: "text",
text: delta.content,
}
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle reasoning output
if (reasoning || (delta && "reasoning_content" in delta && delta.reasoning_content)) {
const reasoningContent = delta?.content || ((delta as any)?.reasoning_content as string | undefined) || ""
if (reasoningContent.trim()) {
yield {
type: "reasoning",
reasoning: reasoningContent,
}
}
// Check if reasoning is complete
if (reasoning?.includes("</think>")) {
reasoning = null
}
}
// Store usage information for later output
if (chunk.usage) {
finalUsage = chunk.usage
}
// Output usage when stream is finished
if (!didOutputUsage && chunk.choices?.[0]?.finish_reason) {
if (finalUsage) {
yield {
type: "usage",
inputTokens: finalUsage.prompt_tokens || 0,
outputTokens: finalUsage.completion_tokens || 0,
cacheWriteTokens: 0,
cacheReadTokens: 0,
}
}
didOutputUsage = true
}
}
}
}
@@ -1,148 +0,0 @@
import { HuggingFaceModelId, huggingFaceDefaultModelId, huggingFaceModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface HuggingFaceHandlerOptions extends CommonApiHandlerOptions {
huggingFaceApiKey?: string
huggingFaceModelId?: string
huggingFaceModelInfo?: ModelInfo
}
export class HuggingFaceHandler implements ApiHandler {
private options: HuggingFaceHandlerOptions
private client: OpenAI | undefined
private cachedModel: { id: HuggingFaceModelId; info: ModelInfo } | undefined
constructor(options: HuggingFaceHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.huggingFaceApiKey) {
throw new Error("Hugging Face API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://router.huggingface.co/v1",
apiKey: this.options.huggingFaceApiKey,
})
} catch (error: any) {
throw new Error(`Error creating Hugging Face client: ${error.message}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
if (!usage) {
return
}
const inputTokens = usage.prompt_tokens || 0
const outputTokens = usage.completion_tokens || 0
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens)
const usageData = {
type: "usage" as const,
inputTokens: inputTokens,
outputTokens: outputTokens,
cacheWriteTokens: 0,
cacheReadTokens: 0,
totalCost: totalCost,
}
yield usageData
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
try {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const requestParams = {
model: model.id,
max_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
...getOpenAIToolParams(tools),
}
const toolCallProcessor = new ToolCallProcessor()
const stream = (await client.chat.completions.create(requestParams)) as any
let _chunkCount = 0
let _totalContent = ""
for await (const chunk of stream) {
_chunkCount++
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
_totalContent += delta.content
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
}
}
} catch (error: any) {
throw error
}
}
getModel(): { id: HuggingFaceModelId; info: ModelInfo } {
// Return cached model if available
if (this.cachedModel) {
return this.cachedModel
}
const modelId = this.options.huggingFaceModelId
// List all available models for debugging
const _availableModels = Object.keys(huggingFaceModels)
let result: { id: HuggingFaceModelId; info: ModelInfo }
if (modelId && modelId in huggingFaceModels) {
const id = modelId as HuggingFaceModelId
const modelInfo = huggingFaceModels[id]
result = { id, info: modelInfo }
} else {
const defaultInfo = huggingFaceModels[huggingFaceDefaultModelId]
result = {
id: huggingFaceDefaultModelId,
info: defaultInfo,
}
}
// Cache the result for future calls
this.cachedModel = result
return result
}
}
@@ -1,403 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { LiteLLMModelInfo, liteLlmDefaultModelId, liteLlmModelInfoSaneDefaults } from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import OpenAI from "openai"
import { StateManager } from "@/core/storage/StateManager"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient, fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { isAnthropicModelId } from "@/utils/model-utils"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface LiteLlmHandlerOptions extends CommonApiHandlerOptions {
liteLlmApiKey?: string
liteLlmBaseUrl?: string
liteLlmModelId?: string
liteLlmModelInfo?: LiteLLMModelInfo
reasoningEffort?: string
thinkingBudgetTokens?: number
liteLlmUsePromptCache?: boolean
ulid?: string
}
/**
* Extended chat completion parameters that include LiteLLM-specific options
* not present in the standard OpenAI SDK types
*/
interface LiteLlmChatCompletionCreateParams extends OpenAI.Chat.ChatCompletionCreateParamsStreaming {
drop_params?: boolean
}
export interface LiteLlmModelInfoResponse {
data: Array<{
model_name: string
litellm_params: {
model: string
[key: string]: any
}
model_info: {
input_cost_per_token: number
output_cost_per_token: number
cache_creation_input_token_cost?: number
cache_read_input_token_cost?: number
supports_prompt_caching?: boolean
[key: string]: any
}
}>
}
/**
* Exported utility function to fetch LiteLLM model info
* @param baseUrl The base URL for the LiteLLM API
* @param apiKey The API key for authentication
* @returns The model info response or undefined if fetch fails
*/
export async function fetchLiteLlmModelsInfo(baseUrl: string, apiKey: string): Promise<LiteLlmModelInfoResponse | undefined> {
// Handle base URLs that already include /v1 to avoid double /v1/v1/
const normalizedBaseUrl = baseUrl.endsWith("/v1") ? baseUrl : `${baseUrl}/v1`
const url = `${normalizedBaseUrl}/model/info`
try {
const response = await fetch(url, {
method: "GET",
headers: {
accept: "application/json",
"x-litellm-api-key": apiKey,
...buildExternalBasicHeaders(),
},
})
if (response.ok) {
const data: LiteLlmModelInfoResponse = await response.json()
return data
}
Logger.error("Failed to fetch LiteLLM model info:", response.statusText)
// Try with Authorization header instead
const retryResponse = await fetch(url, {
method: "GET",
headers: {
accept: "application/json",
Authorization: `Bearer ${apiKey}`,
...buildExternalBasicHeaders(),
},
})
if (retryResponse.ok) {
const data: LiteLlmModelInfoResponse = await retryResponse.json()
return data
}
Logger.error("Failed to fetch LiteLLM model info with Authorization header:", retryResponse.statusText)
throw new Error(`Failed to fetch LiteLLM model info: ${retryResponse.statusText}`)
} catch (error) {
Logger.error("Error fetching LiteLLM model info:", error)
throw error
}
}
export class LiteLlmHandler implements ApiHandler {
private options: LiteLlmHandlerOptions
private client: OpenAI | undefined
private modelInfoCache: LiteLlmModelInfoResponse | undefined
private modelInfoCacheTimestamp = 0
private readonly modelInfoCacheTTL = 5 * 60 * 1000 // 5 minutes
constructor(options: LiteLlmHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.liteLlmApiKey) {
throw new Error("LiteLLM API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: this.options.liteLlmBaseUrl || "http://localhost:4000",
apiKey: this.options.liteLlmApiKey || "noop",
})
} catch (error) {
throw new Error(`Error creating LiteLLM client: ${error.message}`)
}
}
return this.client
}
private async modelInfo(publicModelName: string): Promise<LiteLlmModelInfoResponse["data"][number] | undefined> {
const modelInfo = await this.fetchModelsInfo()
if (!modelInfo?.data) {
return undefined
}
return modelInfo.data.find((model) => model.model_name === publicModelName)
}
private async fetchModelsInfo(): Promise<LiteLlmModelInfoResponse | undefined> {
// Check if cache is still valid
const now = Date.now()
if (this.modelInfoCache && now - this.modelInfoCacheTimestamp < this.modelInfoCacheTTL) {
return this.modelInfoCache
}
const client = this.ensureClient()
const data = await fetchLiteLlmModelsInfo(client.baseURL, this.options.liteLlmApiKey || "")
if (data) {
this.modelInfoCache = data
this.modelInfoCacheTimestamp = now
}
return data
}
private async getModelCostInfo(publicModelName: string): Promise<{
inputCostPerToken: number
outputCostPerToken: number
cacheCreationCostPerToken?: number
cacheReadCostPerToken?: number
}> {
try {
const matchingModel = await this.modelInfo(publicModelName)
if (matchingModel) {
return {
inputCostPerToken: matchingModel.model_info.input_cost_per_token || 0,
outputCostPerToken: matchingModel.model_info.output_cost_per_token || 0,
cacheCreationCostPerToken: matchingModel.model_info.cache_creation_input_token_cost,
cacheReadCostPerToken: matchingModel.model_info.cache_read_input_token_cost,
}
}
} catch (error) {
Logger.warn("Error getting LiteLLM model cost info:", error)
}
// Fallback to zero costs if we can't get the information
return {
inputCostPerToken: 0,
outputCostPerToken: 0,
}
}
async calculateCost(
prompt_tokens: number,
completion_tokens: number,
cache_creation_tokens?: number,
cache_read_tokens?: number,
): Promise<number | undefined> {
const publicModelId = this.options.liteLlmModelId || liteLlmDefaultModelId
try {
const costInfo = await this.getModelCostInfo(publicModelId)
// Calculate costs for different token types
const inputCost = Math.max(0, prompt_tokens - (cache_read_tokens || 0)) * costInfo.inputCostPerToken
const outputCost = completion_tokens * costInfo.outputCostPerToken
const cacheCreationCost = (cache_creation_tokens || 0) * (costInfo.cacheCreationCostPerToken || 0)
const cacheReadCost = (cache_read_tokens || 0) * (costInfo.cacheReadCostPerToken || 0)
const totalCost = inputCost + outputCost + cacheCreationCost + cacheReadCost
return totalCost
} catch (error) {
Logger.error("Error calculating spend:", error)
return undefined
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const formattedMessages = convertToOpenAiMessages(messages)
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam | Anthropic.Messages.TextBlockParam = {
role: "system",
content: systemPrompt,
}
const modelId = this.options.liteLlmModelId || liteLlmDefaultModelId
const isOminiModel = modelId.includes("o1-mini") || modelId.includes("o3-mini") || modelId.includes("o4-mini")
const isCodexModel = modelId.toLowerCase().includes("codex")
// Configuration for extended thinking
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = budgetTokens !== 0
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budgetTokens)
: undefined
const thinkingConfig = isAdaptiveThinkingModel
? adaptiveThinking?.enabled
? ({ type: "adaptive" } as any)
: undefined
: reasoningOn
? { type: "enabled", budget_tokens: budgetTokens }
: undefined
let temperature: number | undefined = this.options.liteLlmModelInfo?.temperature ?? 1
if (isAdaptiveThinkingModel) {
temperature = undefined
} else if ((isOminiModel || isAnthropicModelId(modelId)) && reasoningOn) {
temperature = undefined // OAI omni and Anthropic extended thinking mode doesn't support temperature
}
const modelInfo = await this.modelInfo(modelId)
// Automatically enable caching if the model supports it
const cacheControl =
(modelInfo?.model_info.supports_prompt_caching ?? false) ? { cache_control: { type: "ephemeral" } } : undefined
if (cacheControl) {
// Add cache_control to system message if enabled
// https://docs.litellm.ai/docs/providers/anthropic#caching---large-context-caching
systemMessage.content = [
{
text: systemPrompt,
type: "text",
...cacheControl,
},
] as Anthropic.Messages.TextBlockParam[]
}
// Find the last two user messages to apply caching
const userMsgIndices = formattedMessages.reduce(
(acc, msg, index) => (msg.role === "user" ? [...acc, index] : acc),
[] as number[],
)
const lastUserMsgIndex = userMsgIndices[userMsgIndices.length - 1] ?? -1
const secondLastUserMsgIndex = userMsgIndices[userMsgIndices.length - 2] ?? -1
// Apply cache_control to the last two user messages if enabled
// https://docs.litellm.ai/docs/providers/anthropic#caching---large-context-caching
const enhancedMessages: OpenAI.Chat.ChatCompletionMessageParam[] = formattedMessages.map(
(message, index): OpenAI.Chat.ChatCompletionMessageParam => {
if ((index === lastUserMsgIndex || index === secondLastUserMsgIndex) && cacheControl) {
// Handle both string and array content types
if (typeof message.content === "string") {
return {
...message,
content: [
{
type: "text",
text: message.content,
...cacheControl,
},
] as any,
}
}
if (Array.isArray(message.content)) {
// Apply cache control to the last content item in the array
return {
...message,
content: message.content.map((item, contentIndex) =>
contentIndex === (message.content?.length || 0) - 1
? {
...item,
...cacheControl,
}
: item,
) as any,
}
}
return {
...message,
...cacheControl,
}
}
return message
},
)
const stream = await client.chat.completions.create({
model: this.options.liteLlmModelId || liteLlmDefaultModelId,
messages: [systemMessage, ...enhancedMessages],
temperature,
stream: true,
drop_params: true,
...(!isCodexModel && { stream_options: { include_usage: true } }), // Codex models are only on the responses api, which doesn't take the stream_options parameter. we will need to migrate to the responses api for this to work
...(thinkingConfig && { thinking: thinkingConfig }), // Add thinking configuration when applicable
...(isAdaptiveThinkingModel && adaptiveThinking?.effort
? { output_config: { effort: adaptiveThinking.effort } }
: {}),
...(this.options.ulid && { litellm_session_id: `cline-${this.options.ulid}` }), // Add session ID for LiteLLM tracking
} as LiteLlmChatCompletionCreateParams)
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle normal text content
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
// Handle reasoning events
// This is not in the standard types but may be in the response
interface ThinkingDelta {
reasoning_content?: string
}
if ((delta as ThinkingDelta)?.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta as ThinkingDelta).reasoning_content || "",
}
}
// Handle token usage information
if (chunk.usage) {
// Extract cache-related information if available
// Need to use type assertion since these properties are not in the standard OpenAI types
const usage = chunk.usage as {
prompt_tokens: number
completion_tokens: number
cache_creation_input_tokens?: number
prompt_cache_miss_tokens?: number
cache_read_input_tokens?: number
prompt_cache_hit_tokens?: number
}
const cacheWriteTokens = usage.cache_creation_input_tokens || usage.prompt_cache_miss_tokens || 0
const cacheReadTokens = usage.cache_read_input_tokens || usage.prompt_cache_hit_tokens || 0
// Calculate cost using the actual token usage including cache tokens
const totalCost =
(await this.calculateCost(
usage.prompt_tokens || 0,
usage.completion_tokens || 0,
cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
cacheReadTokens > 0 ? cacheReadTokens : undefined,
)) || 0
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
cacheWriteTokens: cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
cacheReadTokens: cacheReadTokens > 0 ? cacheReadTokens : undefined,
totalCost,
}
}
}
}
getModel() {
const modelId = this.options.liteLlmModelId || liteLlmDefaultModelId
// Try to get model info from StateManager cache first
const cachedModelInfo = StateManager.get().getModelInfo("liteLlm", modelId)
// Fall back to provided model info or defaults if not in cache
const modelInfo = cachedModelInfo || this.options.liteLlmModelInfo || liteLlmModelInfoSaneDefaults
return {
id: modelId,
info: modelInfo,
}
}
}
@@ -1,109 +0,0 @@
import { type ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import type { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import type { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface LmStudioHandlerOptions extends CommonApiHandlerOptions {
lmStudioBaseUrl?: string
lmStudioModelId?: string
lmStudioMaxTokens?: string
}
export class LmStudioHandler implements ApiHandler {
private options: LmStudioHandlerOptions
private client: OpenAI | undefined
constructor(options: LmStudioHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
try {
this.client = createOpenAIClient({
// Docs on the new v0 api endpoint: https://lmstudio.ai/docs/app/api/endpoints/rest
baseURL: new URL("api/v0", this.options.lmStudioBaseUrl || "http://localhost:1234").toString(),
apiKey: "noop",
})
} catch (error) {
throw new Error(`Error creating LM Studio client: ${error.message}`)
}
}
return this.client
}
@withRetry({ retryAllErrors: true })
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
try {
const stream = await client.chat.completions.create({
model: this.getModel().id,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
max_completion_tokens: this.options.lmStudioMaxTokens ? Number(this.options.lmStudioMaxTokens) : undefined,
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const choice = chunk.choices?.[0]
const delta = choice?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
}
}
}
} catch {
// LM Studio doesn't return an error code/body for now
throw new Error(
"Please check the LM Studio developer logs to debug what went wrong. You may need to load the model with a larger context length to work with Cline's prompts. Alternatively, try enabling Compact Prompt in your settings when working with a limited context window.",
)
}
}
getModel(): { id: string; info: ModelInfo } {
const info = { ...openAiModelInfoSaneDefaults }
const maxTokens = Number(this.options.lmStudioMaxTokens)
if (!Number.isNaN(maxTokens)) {
info.contextWindow = maxTokens
}
return {
id: this.options.lmStudioModelId || "",
info,
}
}
}
@@ -1,208 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
import { Stream as AnthropicStream } from "@anthropic-ai/sdk/streaming"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { MinimaxModelId, ModelInfo, minimaxDefaultModelId, minimaxModels } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ClineTool } from "@/shared/tools"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
interface MinimaxHandlerOptions extends CommonApiHandlerOptions {
minimaxApiKey?: string
minimaxApiLine?: string
apiModelId?: string
thinkingBudgetTokens?: number
}
export class MinimaxHandler implements ApiHandler {
private options: MinimaxHandlerOptions
private client: Anthropic | undefined
constructor(options: MinimaxHandlerOptions) {
this.options = options
}
private ensureClient(): Anthropic {
if (!this.client) {
if (!this.options.minimaxApiKey) {
throw new Error("MiniMax API key is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
this.client = new Anthropic({
apiKey: this.options.minimaxApiKey,
baseURL:
this.options.minimaxApiLine === "china"
? "https://api.minimaxi.com/anthropic"
: "https://api.minimax.io/anthropic",
defaultHeaders: externalHeaders,
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating MiniMax client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ClineTool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
// Tools are available only when native tools are enabled
const nativeToolsOn = tools?.length && tools?.length > 0
const budget_tokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
// MiniMax M2 uses Anthropic API format
const stream: AnthropicStream<Anthropic.RawMessageStreamEvent> = await client.messages.create({
model: model.id,
max_tokens: model.info.maxTokens || 8192,
system: [{ text: systemPrompt, type: "text" }],
messages,
stream: true,
tools: nativeToolsOn ? (tools as AnthropicTool[]) : undefined,
thinking: reasoningOn ? { type: "enabled", budget_tokens: budget_tokens } : undefined,
// "Thinking isn't compatible with temperature, top_p, or top_k modifications"
temperature: reasoningOn ? undefined : 1.0, // MiniMax recommends 1.0, range is (0.0, 1.0]
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
tool_choice: nativeToolsOn && !reasoningOn ? { type: "any" } : undefined,
})
const lastStartedToolCall = { id: "", name: "", arguments: "" }
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start": {
// tells us cache reads/writes/input/output
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
break
}
case "message_delta":
// tells us stop_reason, stop_sequence, and output tokens along the way and at the end of the message
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage.output_tokens || 0,
}
break
case "message_stop":
// no usage data, just an indicator that the message is done
break
case "content_block_start":
switch (chunk.content_block.type) {
case "thinking":
yield {
type: "reasoning",
reasoning: chunk.content_block.thinking || "",
signature: chunk.content_block.signature,
}
break
case "redacted_thinking":
// Content is encrypted, and we don't want to pass placeholder text back to the API
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
redacted_data: chunk.content_block.data,
}
break
case "tool_use":
if (chunk.content_block.id && chunk.content_block.name) {
// Store tool call information for streaming
lastStartedToolCall.id = chunk.content_block.id
lastStartedToolCall.name = chunk.content_block.name
lastStartedToolCall.arguments = ""
}
break
case "text":
// we may receive multiple text blocks, in which case just insert a line break between them
if (chunk.index > 0) {
yield {
type: "text",
text: "\n",
}
}
yield {
type: "text",
text: chunk.content_block.text,
}
break
}
break
case "content_block_delta":
switch (chunk.delta.type) {
case "thinking_delta":
// 'reasoning' type just displays in the UI, but reasoning with signature will be used to send the thinking traces back to the API
yield {
type: "reasoning",
reasoning: chunk.delta.thinking,
}
break
case "signature_delta":
// It's used when sending the thinking block back to the API
// API expects this in completed form, not as array of deltas
if (chunk.delta.signature) {
yield {
type: "reasoning",
reasoning: "",
signature: chunk.delta.signature,
}
}
break
case "text_delta":
yield {
type: "text",
text: chunk.delta.text,
}
break
case "input_json_delta":
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
// Convert Anthropic tool_use to OpenAI-compatible format for internal processing
yield {
type: "tool_calls",
tool_call: {
...lastStartedToolCall,
function: {
...lastStartedToolCall,
id: lastStartedToolCall.id,
name: lastStartedToolCall.name,
arguments: chunk.delta.partial_json,
},
},
}
}
break
}
break
case "content_block_stop":
lastStartedToolCall.id = ""
lastStartedToolCall.name = ""
lastStartedToolCall.arguments = ""
break
}
}
}
getModel(): { id: MinimaxModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in minimaxModels) {
const id = modelId as MinimaxModelId
return { id, info: minimaxModels[id] }
}
return { id: minimaxDefaultModelId, info: minimaxModels[minimaxDefaultModelId] }
}
}
@@ -1,154 +0,0 @@
import { Mistral } from "@mistralai/mistralai"
import { HTTPClient } from "@mistralai/mistralai/lib/http"
import { Tool as MistralTool } from "@mistralai/mistralai/models/components/tool"
import { MistralModelId, ModelInfo, mistralDefaultModelId, mistralModels } from "@shared/api"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToMistralMessages } from "../transform/mistral-format"
import { ApiStream } from "../transform/stream"
interface MistralHandlerOptions extends CommonApiHandlerOptions {
mistralApiKey?: string
apiModelId?: string
}
export class MistralHandler implements ApiHandler {
private options: MistralHandlerOptions
private client: Mistral | undefined
constructor(options: MistralHandlerOptions) {
this.options = options
}
private ensureClient(): Mistral {
if (!this.client) {
if (!this.options.mistralApiKey) {
throw new Error("Mistral API key is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
// Create HTTP client with custom fetch for proxy support
// The Mistral SDK's HTTPClient passes a Request object to the fetcher,
// but we need to extract the URL and init options to pass to our fetch wrapper
// which properly handles proxy configuration in standalone mode (JetBrains/CLI)
const httpClient = new HTTPClient({
fetcher: async (input: RequestInfo | URL, init?: RequestInit) => {
// Handle both string/URL and Request object inputs
if (input instanceof Request) {
Object.keys(externalHeaders).forEach((key) => {
if (!input.headers.has(key)) {
input.headers.set(key, externalHeaders[key])
}
})
return fetch(input.url, {
method: input.method,
headers: input.headers,
body: input.body,
redirect: input.redirect,
signal: input.signal,
// duplex is required when sending a body stream in Node.js/undici
duplex: input.body ? "half" : undefined,
...init,
} as RequestInit)
}
// Merge external headers with existing headers
const mergedInit = {
...init,
headers: {
...externalHeaders,
...(init?.headers || {}),
},
}
return fetch(input, mergedInit)
},
})
this.client = new Mistral({
apiKey: this.options.mistralApiKey,
httpClient,
})
} catch (error) {
throw new Error(`Error creating Mistral client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const stream = await client.chat
.stream({
model: this.getModel().id,
// max_completion_tokens: this.getModel().info.maxTokens,
temperature: 0,
messages: [{ role: "system", content: systemPrompt }, ...convertToMistralMessages(messages)],
stream: true,
tools: tools?.length ? (tools as MistralTool[]) : undefined,
toolChoice: tools?.length ? "any" : undefined,
})
.catch((err) => {
// The Mistal SDK uses statusCode instead of status
// However, if they introduce status for something, I don't want to override it
if ("statusCode" in err && !("status" in err)) {
err.status = err.statusCode
}
throw err
})
for await (const chunk of stream) {
const delta = chunk.data.choices[0]?.delta
if (delta.toolCalls) {
for (const toolCall of delta.toolCalls) {
yield {
type: "tool_calls",
tool_call: {
function: {
id: toolCall.id,
name: toolCall.function.name,
arguments: JSON.stringify(toolCall.function.arguments),
},
},
}
}
} else if (delta?.content) {
let content: string = ""
if (typeof delta.content === "string") {
content = delta.content
} else if (Array.isArray(delta.content)) {
content = delta.content.map((c) => (c.type === "text" ? c.text : "")).join("")
}
yield {
type: "text",
text: content,
}
}
if (chunk.data.usage) {
yield {
type: "usage",
inputTokens: chunk.data.usage.promptTokens || 0,
outputTokens: chunk.data.usage.completionTokens || 0,
}
}
}
}
getModel(): { id: MistralModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in mistralModels) {
const id = modelId as MistralModelId
return { id, info: mistralModels[id] }
}
return {
id: mistralDefaultModelId,
info: mistralModels[mistralDefaultModelId],
}
}
}
@@ -1,110 +0,0 @@
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ModelInfo, MoonshotModelId, moonshotDefaultModelId, moonshotModels } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface MoonshotHandlerOptions extends CommonApiHandlerOptions {
moonshotApiKey?: string
moonshotApiLine?: string
apiModelId?: string
}
// Enhanced usage interface to support Moonshot's cached token field
interface MoonshotUsage extends OpenAI.CompletionUsage {
cached_tokens?: number
}
export class MoonshotHandler implements ApiHandler {
private client: OpenAI | undefined
constructor(private readonly options: MoonshotHandlerOptions) {}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.moonshotApiKey) {
throw new Error("Moonshot API key is required")
}
try {
this.client = createOpenAIClient({
baseURL:
this.options.moonshotApiLine === "china" ? "https://api.moonshot.cn/v1" : "https://api.moonshot.ai/v1",
apiKey: this.options.moonshotApiKey,
})
} catch (error) {
throw new Error(`Error creating Moonshot client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
messages: openAiMessages,
temperature: model.info.temperature,
max_tokens: model.info.maxTokens,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
const usage = chunk.usage as MoonshotUsage
yield {
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: usage.cached_tokens ?? 0,
inputTokens: (usage.prompt_tokens || 0) - (usage.cached_tokens ?? 0),
outputTokens: usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: MoonshotModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in moonshotModels) {
const id = modelId as MoonshotModelId
return { id, info: moonshotModels[id] }
}
return { id: moonshotDefaultModelId, info: moonshotModels[moonshotDefaultModelId] }
}
}
@@ -1,96 +0,0 @@
import { type ModelInfo, type NebiusModelId, nebiusDefaultModelId, nebiusModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface NebiusHandlerOptions extends CommonApiHandlerOptions {
nebiusApiKey?: string
apiModelId?: string
}
export class NebiusHandler implements ApiHandler {
private client: OpenAI | undefined
constructor(private readonly options: NebiusHandlerOptions) {}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.nebiusApiKey) {
throw new Error("Nebius API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.studio.nebius.ai/v1",
apiKey: this.options.nebiusApiKey,
})
} catch (error) {
throw new Error(`Error creating Nebius client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = model.id.includes("DeepSeek-R1")
? convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
: [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
const stream = await client.chat.completions.create({
model: model.id,
messages: openAiMessages,
temperature: 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId !== undefined && modelId in nebiusModels) {
return { id: modelId, info: nebiusModels[modelId as NebiusModelId] }
}
return { id: nebiusDefaultModelId, info: nebiusModels[nebiusDefaultModelId] }
}
}
@@ -1,93 +0,0 @@
import { ModelInfo, NousResearchModelId, nousResearchDefaultModelId, nousResearchModels } from "@shared/api"
import OpenAI from "openai"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface NousResearchHandlerOptions extends CommonApiHandlerOptions {
nousResearchApiKey?: string
apiModelId?: string
}
export class NousResearchHandler implements ApiHandler {
private options: NousResearchHandlerOptions
private client: OpenAI | undefined
constructor(options: NousResearchHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.nousResearchApiKey) {
throw new Error("NousResearch API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://inference-api.nousResearch.com/v1",
apiKey: this.options.nousResearchApiKey,
})
} catch (error: any) {
throw new Error(`Error creating NousResearch client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
messages: openAiMessages,
temperature: 0,
stream: true,
stream_options: { include_usage: true },
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: NousResearchModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in nousResearchModels) {
const id = modelId as NousResearchModelId
return { id, info: nousResearchModels[id] }
}
return { id: nousResearchDefaultModelId, info: nousResearchModels[nousResearchDefaultModelId] }
}
}
-473
View File
@@ -1,473 +0,0 @@
import { Anthropic, APIError as AnthropicAPIError } from "@anthropic-ai/sdk"
import { liteLlmDefaultModelId, liteLlmModelInfoSaneDefaults, ModelInfo } from "@shared/api"
import OpenAI, { APIError as OpenAIAPIError, OpenAIError } from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { OcaAuthService } from "@/services/auth/oca/OcaAuthService"
import {
DEFAULT_EXTERNAL_OCA_BASE_URL,
DEFAULT_INTERNAL_OCA_BASE_URL,
OCI_HEADER_OPC_REQUEST_ID,
} from "@/services/auth/oca/utils/constants"
import { createOcaHeaders } from "@/services/auth/oca/utils/utils"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { OcaModelInfo } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiFormat } from "@/shared/proto/index.cline"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, type CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
import { convertOpenAIToolsToAnthropicTools, handleAnthropicMessagesApiStreamResponse } from "../utils/messages_api_support"
import { handleResponsesApiStreamResponse } from "../utils/responses_api_support"
export interface OcaHandlerOptions extends CommonApiHandlerOptions {
ocaBaseUrl?: string
ocaModelId?: string
ocaModelInfo?: OcaModelInfo
ocaReasoningEffort?: string
thinkingBudgetTokens?: number
ocaUsePromptCache?: boolean
taskId?: string
ocaMode?: string // "internal" or "external"
}
export class OcaHandler implements ApiHandler {
protected options: OcaHandlerOptions
protected openAIClient: OpenAI | undefined
protected anthropicClient: Anthropic | undefined
protected externalHeaders: Record<string, string> = {}
constructor(options: OcaHandlerOptions) {
this.options = options
}
protected initializeOpenAIClient(options: OcaHandlerOptions): OpenAI {
const externalHeaders = buildExternalBasicHeaders()
return new (class OCIOpenAI extends OpenAI {
protected override async prepareOptions(opts: any): Promise<void> {
const token = await OcaAuthService.getInstance().getAuthToken()
if (!token) {
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
}
opts.headers ??= {}
// OCA Headers
const ociHeaders = await createOcaHeaders(token, options.taskId!)
opts.headers = { ...opts.headers, ...externalHeaders, ...ociHeaders }
Logger.log(`Making request with customer opc-request-id: ${opts.headers?.["opc-request-id"]}`)
return super.prepareOptions(opts)
}
protected override makeStatusError(
status: number | undefined,
error: Object | undefined,
message: string | undefined,
headers: any | undefined,
): OpenAIAPIError {
interface OciError {
code?: string
message?: string
}
let ociErrorMessage = message
if (typeof error === "object" && error !== null) {
try {
ociErrorMessage = JSON.stringify(error)
const ociErr = error as OciError
if (ociErr.code !== undefined && ociErr.message !== undefined) {
ociErrorMessage = `${ociErr.code}: ${ociErr.message}`
}
} catch {}
}
const opcRequestId = headers?.[OCI_HEADER_OPC_REQUEST_ID]
if (opcRequestId) {
ociErrorMessage += `\n(${OCI_HEADER_OPC_REQUEST_ID}: ${opcRequestId})`
}
const statusCode = typeof status === "number" ? status : 500
return super.makeStatusError(statusCode, error ?? {}, ociErrorMessage, headers)
}
})({
baseURL:
options.ocaBaseUrl ||
(options.ocaMode === "internal" ? DEFAULT_INTERNAL_OCA_BASE_URL : DEFAULT_EXTERNAL_OCA_BASE_URL),
apiKey: "noop",
fetch, // Use configured fetch with proxy support
})
}
protected initializeAnthropicClient(options: OcaHandlerOptions): Anthropic {
const externalHeaders = buildExternalBasicHeaders()
return new (class OCIAnthropic extends Anthropic {
protected override async prepareOptions(opts: any): Promise<void> {
const token = await OcaAuthService.getInstance().getAuthToken()
if (!token) {
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
}
opts.headers ??= {}
// OCA Headers
const ociHeaders = await createOcaHeaders(token, options.taskId!)
opts.headers = { ...opts.headers, ...externalHeaders, ...ociHeaders }
Logger.log(`Making request with customer opc-request-id: ${opts.headers?.["opc-request-id"]}`)
return super.prepareOptions(opts)
}
protected override makeStatusError(
status: number | undefined,
error: Object | undefined,
message: string | undefined,
headers: any | undefined,
): AnthropicAPIError {
interface OciError {
code?: string
message?: string
}
let ociErrorMessage = message
if (typeof error === "object" && error !== null) {
try {
ociErrorMessage = JSON.stringify(error)
const ociErr = error as OciError
if (ociErr.code !== undefined && ociErr.message !== undefined) {
ociErrorMessage = `${ociErr.code}: ${ociErr.message}`
}
} catch {}
}
const opcRequestId = headers?.[OCI_HEADER_OPC_REQUEST_ID]
if (opcRequestId) {
ociErrorMessage += `\n(${OCI_HEADER_OPC_REQUEST_ID}: ${opcRequestId})`
}
const statusCode = typeof status === "number" ? status : 500
return super.makeStatusError(statusCode, error ?? {}, ociErrorMessage, headers)
}
})({
baseURL:
options.ocaBaseUrl ||
(options.ocaMode === "internal" ? DEFAULT_INTERNAL_OCA_BASE_URL : DEFAULT_EXTERNAL_OCA_BASE_URL),
apiKey: "noop",
fetch, // Use configured fetch with proxy support
})
}
protected ensureOpenAIClient(): OpenAI {
if (!this.openAIClient) {
if (!this.options.ocaModelId) {
throw new Error("Oracle Code Assist (OCA) model is not selected")
}
try {
this.openAIClient = this.initializeOpenAIClient(this.options)
} catch (error) {
throw new Error(`Error creating Oracle Code Assist (OCA) client: ${error.message}`)
}
}
return this.openAIClient
}
protected ensureAnthropicClient(): Anthropic {
if (!this.anthropicClient) {
if (!this.options.ocaModelId) {
throw new Error("Oracle Code Assist (OCA) model is not selected")
}
try {
this.anthropicClient = this.initializeAnthropicClient(this.options)
} catch (error) {
throw new Error(`Error creating Oracle Code Assist (OCA) client: ${error.message}`)
}
}
return this.anthropicClient
}
async getApiCosts(prompt_tokens: number, completion_tokens: number): Promise<number | undefined> {
// Reference: https://github.com/BerriAI/litellm/blob/122ee634f434014267af104814022af1d9a0882f/litellm/proxy/spend_tracking/spend_management_endpoints.py#L1473
const client = this.ensureOpenAIClient()
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
const token = await OcaAuthService.getInstance().getAuthToken()
if (!token) {
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
}
const externalHeaders = buildExternalBasicHeaders()
const ociHeaders = await createOcaHeaders(token, this.options.taskId!)
Logger.log(`Making calculate cost request with customer opc-request-id: ${ociHeaders["opc-request-id"]}`)
try {
const response = await fetch(`${client.baseURL}/spend/calculate`, {
method: "POST",
headers: { ...externalHeaders, ...ociHeaders },
body: JSON.stringify({
completion_response: {
model: modelId,
usage: {
prompt_tokens,
completion_tokens,
},
},
}),
})
if (response.ok) {
const data: { cost: number } = await response.json()
return data.cost
}
Logger.error("Error calculating spend:", response.statusText)
return undefined
} catch (error) {
Logger.error("Error calculating spend:", error)
return undefined
}
}
async calculateCost(
modelInfo: ModelInfo,
inputTokens: number,
outputTokens: number,
_cacheWriteTokens?: number,
_cacheReadTokens?: number,
) {
const inputCost = (await this.getApiCosts(1e6, 0)) || 0
const outputCost = (await this.getApiCosts(0, 1e6)) || 0
const totalCost = (inputCost * inputTokens) / 1e6 + (outputCost * outputTokens) / 1e6
return totalCost
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
if (this.options.ocaModelInfo?.apiFormat == ApiFormat.OPENAI_RESPONSES) {
yield* this.createMessageResponsesApi(systemPrompt, messages, tools)
} else if (this.options.ocaModelInfo?.apiFormat == ApiFormat.ANTHROPIC_CHAT) {
yield* this.createMessageMessagesApi(systemPrompt, messages, tools)
} else {
yield* this.createMessageChatApi(systemPrompt, messages, tools)
}
}
async *createMessageChatApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureOpenAIClient()
const formattedMessages = convertToOpenAiMessages(messages)
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam = {
role: "system",
content: systemPrompt,
}
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
const isOminiModel = modelId.includes("o1-mini") || modelId.includes("o3-mini") || modelId.includes("o4-mini")
// Configuration for extended thinking
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = budgetTokens !== 0
const thinkingConfig = reasoningOn ? { type: "enabled", budget_tokens: budgetTokens } : undefined
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
const maxTokens: number | undefined = this.options.ocaModelInfo?.maxTokens
if (isOminiModel && reasoningOn) {
temperature = undefined // Thinking mode doesn't support temperature
}
// Define cache control object if prompt caching is enabled
const cacheControl = this.options.ocaUsePromptCache ? { cache_control: { type: "ephemeral" } } : undefined
// Add cache_control to system message if enabled
const enhancedSystemMessage = {
...systemMessage,
...(cacheControl && cacheControl),
}
// Find the last two user messages to apply caching
const userMsgIndices = formattedMessages.reduce((acc, msg, index) => {
if (msg.role === "user") {
acc.push(index)
}
return acc
}, [] as number[])
const lastUserMsgIndex = userMsgIndices[userMsgIndices.length - 1] ?? -1
const secondLastUserMsgIndex = userMsgIndices[userMsgIndices.length - 2] ?? -1
// Apply cache_control to the last two user messages if enabled
const enhancedMessages = formattedMessages.map((message, index) => {
if ((index === lastUserMsgIndex || index === secondLastUserMsgIndex) && cacheControl) {
return {
...message,
...cacheControl,
}
}
return message
})
const toolCallProcessor = new ToolCallProcessor()
const chatCompletionsParams: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming = {
model: this.options.ocaModelId || liteLlmDefaultModelId,
messages: [enhancedSystemMessage, ...enhancedMessages],
temperature,
stream: true,
max_completion_tokens: maxTokens,
max_tokens: maxTokens,
stream_options: { include_usage: true },
...(thinkingConfig && { thinking: thinkingConfig }), // Add thinking configuration when applicable
...(this.options.taskId && {
litellm_session_id: `cline-${this.options.taskId}`,
...getOpenAIToolParams(tools),
}), // Add session ID for LiteLLM tracking
}
if (this.options.ocaModelInfo?.supportsReasoningEffort) {
chatCompletionsParams["reasoning_effort"] = this.options.ocaReasoningEffort || ("medium" as any)
}
const stream = await client.chat.completions.create(chatCompletionsParams)
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle normal text content
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
// Handle reasoning events (thinking)
// Thinking is not in the standard types but may be in the response
interface ThinkingDelta {
thinking?: string
}
if ((delta as ThinkingDelta)?.thinking) {
yield {
type: "reasoning",
reasoning: (delta as ThinkingDelta).thinking || "",
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle token usage information
if (chunk.usage) {
const totalCost = await this.calculateCost(
this.options.ocaModelInfo!,
chunk.usage.prompt_tokens,
chunk.usage.completion_tokens,
)
// Extract cache-related information if available
// Need to use type assertion since these properties are not in the standard OpenAI types
const usage = chunk.usage as {
prompt_tokens: number
completion_tokens: number
cache_creation_input_tokens?: number
prompt_cache_miss_tokens?: number
cache_read_input_tokens?: number
prompt_cache_hit_tokens?: number
}
const cacheWriteTokens = usage.cache_creation_input_tokens || usage.prompt_cache_miss_tokens || 0
const cacheReadTokens = usage.cache_read_input_tokens || usage.prompt_cache_hit_tokens || 0
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
cacheWriteTokens: cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
cacheReadTokens: cacheReadTokens > 0 ? cacheReadTokens : undefined,
totalCost,
}
}
}
}
async *createMessageResponsesApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureOpenAIClient()
const inputMessages = convertToOpenAIResponsesInput(messages, { usePreviousResponseId: false }).input
// Convert messages to Responses API input format
const input: OpenAI.Responses.ResponseInputItem[] = [{ role: "system", content: systemPrompt }, ...inputMessages]
// Convert ChatCompletion tools to Responses API format if provided
const responseTools = tools
?.filter((tool) => tool?.type === "function")
.map((tool: any) => ({
type: "function" as const,
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters,
strict: tool.function.strict ?? true, // Responses API defaults to strict mode
}))
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
const maxOutputTokens: number | undefined = this.options.ocaModelInfo?.maxTokens
const ocaModelInfo = this.options.ocaModelInfo
if (!ocaModelInfo) {
throw new Error("Oracle Code Assist (OCA) model info is required for Responses API")
}
const reasoningOn = !!ocaModelInfo.supportsReasoning
if (reasoningOn) {
temperature = undefined
}
const responsesParams: OpenAI.Responses.ResponseCreateParamsStreaming = {
model: this.options.ocaModelId || liteLlmDefaultModelId,
input,
stream: true,
tools: responseTools,
...(typeof temperature === "number" ? { temperature } : {}),
...(typeof maxOutputTokens === "number" && maxOutputTokens > 0 ? { max_output_tokens: maxOutputTokens } : {}),
}
if (reasoningOn) {
responsesParams.reasoning = { effort: this.options.ocaReasoningEffort as any, summary: "auto" }
}
// Create the response using Responses API
const stream = await client.responses.create(responsesParams)
yield* handleResponsesApiStreamResponse(stream, ocaModelInfo, this.calculateCost.bind(this))
}
async *createMessageMessagesApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureAnthropicClient()
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = this.options.ocaModelInfo?.supportsReasoning && budgetTokens !== 0
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
const maxTokens: number | undefined = this.options.ocaModelInfo?.maxTokens || 8192
if (reasoningOn) {
temperature = 0
}
const anthropicTools = convertOpenAIToolsToAnthropicTools(tools)
const anthropicMessages = sanitizeAnthropicMessages(messages, this.options.ocaUsePromptCache ?? false)
const stream = await client.messages.create({
model: modelId,
max_tokens: maxTokens,
temperature: reasoningOn ? undefined : temperature,
system: [
{
text: systemPrompt,
type: "text",
cache_control: this.options.ocaUsePromptCache ? { type: "ephemeral" } : undefined,
},
],
messages: anthropicMessages,
stream: true,
tools: anthropicTools,
thinking: reasoningOn ? { type: "enabled", budget_tokens: budgetTokens } : undefined,
})
yield* handleAnthropicMessagesApiStreamResponse(stream)
}
getModel() {
return {
id: this.options.ocaModelId || liteLlmDefaultModelId,
info: this.options.ocaModelInfo || liteLlmModelInfoSaneDefaults,
}
}
}
@@ -1,159 +0,0 @@
import { type ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import { type Config, type Message, Ollama } from "ollama"
import type { ChatCompletionTool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import type { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOllamaMessages } from "../transform/ollama-format"
import type { ApiStream } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
interface OllamaHandlerOptions extends CommonApiHandlerOptions {
ollamaBaseUrl?: string
ollamaApiKey?: string
ollamaModelId?: string
ollamaApiOptionsCtxNum?: string
requestTimeoutMs?: number
}
const DEFAULT_CONTEXT_WINDOW = 32768
export class OllamaHandler implements ApiHandler {
private options: OllamaHandlerOptions
private client: Ollama | undefined
constructor(options: OllamaHandlerOptions) {
const ollamaApiOptionsCtxNum = (options.ollamaApiOptionsCtxNum ?? DEFAULT_CONTEXT_WINDOW).toString()
this.options = { ...options, ollamaApiOptionsCtxNum }
}
private ensureClient(): Ollama {
if (!this.client) {
try {
const externalHeaders = buildExternalBasicHeaders()
const clientOptions: Partial<Config> = {
host: this.options.ollamaBaseUrl,
fetch,
headers: externalHeaders,
}
// Add API key if provided (for Ollama cloud or authenticated instances)
if (this.options.ollamaApiKey) {
clientOptions.headers = {
...clientOptions.headers,
Authorization: `Bearer ${this.options.ollamaApiKey}`,
}
}
this.client = new Ollama(clientOptions)
} catch (error) {
throw new Error(`Error creating Ollama client: ${error.message}`)
}
}
return this.client
}
@withRetry({ retryAllErrors: true })
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
const client = this.ensureClient()
const ollamaMessages: Message[] = [{ role: "system", content: systemPrompt }, ...convertToOllamaMessages(messages)]
try {
// Create a promise that rejects after timeout
const timeoutMs = this.options.requestTimeoutMs || 30000
const timeoutPromise = new Promise<never>((_, reject) => {
setTimeout(() => reject(new Error(`Ollama request timed out after ${timeoutMs / 1000} seconds`)), timeoutMs)
})
// Create the actual API request promise
const apiPromise = client.chat({
model: this.getModel().id,
messages: ollamaMessages,
stream: true,
options: {
num_ctx: Number(this.options.ollamaApiOptionsCtxNum),
},
tools: tools as any,
})
const toolCallProcessor = new ToolCallProcessor()
// Race the API request against the timeout
const stream = (await Promise.race([apiPromise, timeoutPromise])) as Awaited<typeof apiPromise>
try {
for await (const chunk of stream) {
Logger.debug("[OllamaHandler] Message Chunk" + JSON.stringify(chunk))
const delta = chunk.message
if (delta?.tool_calls) {
Logger.debug(`[OllamaHandler] Tool Calls Detected: ${JSON.stringify(delta.tool_calls)}`)
yield* toolCallProcessor.processToolCallDeltas(
delta.tool_calls?.map((tc, inx) => ({
index: inx,
id: `ollama-tool-${inx}`,
function: {
name: tc.function.name,
arguments:
typeof tc.function.arguments === "string"
? tc.function.arguments
: JSON.stringify(tc.function.arguments),
},
type: "function",
})),
)
}
if (typeof delta.content === "string") {
yield {
type: "text",
text: delta.content,
}
}
// Handle token usage if available
if (chunk.eval_count !== undefined || chunk.prompt_eval_count !== undefined) {
yield {
type: "usage",
inputTokens: chunk.prompt_eval_count || 0,
outputTokens: chunk.eval_count || 0,
}
}
}
} catch (streamError: any) {
Logger.error("Error processing Ollama stream:", streamError)
throw new Error(`Ollama stream processing error: ${streamError.message || "Unknown error"}`)
}
} catch (error) {
// Check if it's a timeout error
if (error?.message?.includes("timed out")) {
const timeoutMs = this.options.requestTimeoutMs || 30000
throw new Error(`Ollama request timed out after ${timeoutMs / 1000} seconds`)
}
// Enhance error reporting
const statusCode = error.status || error.statusCode
const errorMessage = error.message || "Unknown error"
Logger.error(`Ollama API error (${statusCode || "unknown"}): ${errorMessage}`)
throw error
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.ollamaModelId || "",
info: {
...openAiModelInfoSaneDefaults,
contextWindow: Number(this.options.ollamaApiOptionsCtxNum),
},
}
}
abort(): void {
this.client?.abort()
}
}
@@ -1,732 +0,0 @@
import { ModelInfo, OpenAiCodexModelId, openAiCodexDefaultModelId, openAiCodexModels } from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import OpenAI from "openai"
import type { ChatCompletionTool } from "openai/resources/chat/completions"
import * as os from "os"
import { MessageEvent as UndiciMessageEvent, WebSocket as UndiciWebSocket } from "undici"
import { v7 as uuidv7 } from "uuid"
import { openAiCodexOAuthManager } from "@/integrations/openai-codex/oauth"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { featureFlagsService } from "@/services/feature-flags"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiFormat } from "@/shared/proto/cline/models"
import { FeatureFlag } from "@/shared/services/feature-flags/feature-flags"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
import { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
/**
* OpenAI Codex base URL for API requests
* Routes to chatgpt.com/backend-api/codex
*/
const CODEX_API_BASE_URL = "https://chatgpt.com/backend-api/codex"
const CODEX_RESPONSES_WEBSOCKET_URL = "wss://chatgpt.com/backend-api/codex/responses"
interface OpenAiCodexHandlerOptions extends CommonApiHandlerOptions {
reasoningEffort?: string
apiModelId?: string
}
/**
* OpenAiCodexHandler - Uses OpenAI Responses API with OAuth authentication
*
* Key differences from OpenAiNativeHandler:
* - Uses OAuth Bearer tokens instead of API keys
* - Routes requests to Codex backend (chatgpt.com/backend-api/codex)
* - Subscription-based pricing (no per-token costs)
* - Limited model subset
* - Custom headers for Codex backend
*/
export class OpenAiCodexHandler implements ApiHandler {
private options: OpenAiCodexHandlerOptions
private client?: OpenAI
private responsesWs: UndiciWebSocket | undefined
private websocketRequestInFlight = false
// Session ID for the Codex API (persists for the lifetime of the handler)
private readonly sessionId: string
// Abort controller for cancelling ongoing requests
private abortController?: AbortController
// Track tool call identity for streaming
private pendingToolCallId: string | undefined
private pendingToolCallName: string | undefined
constructor(options: OpenAiCodexHandlerOptions) {
this.options = options
this.sessionId = uuidv7()
}
private normalizeUsage(usage: any, _model: { id: string; info: ModelInfo }): ApiStreamUsageChunk | undefined {
if (!usage) {
return undefined
}
const inputDetails = usage.input_tokens_details ?? usage.prompt_tokens_details
const hasCachedTokens = typeof inputDetails?.cached_tokens === "number"
const hasCacheMissTokens = typeof inputDetails?.cache_miss_tokens === "number"
const cachedFromDetails = hasCachedTokens ? inputDetails.cached_tokens : 0
const missFromDetails = hasCacheMissTokens ? inputDetails.cache_miss_tokens : 0
let totalInputTokens = usage.input_tokens ?? usage.prompt_tokens ?? 0
if (totalInputTokens === 0 && inputDetails && (cachedFromDetails > 0 || missFromDetails > 0)) {
totalInputTokens = cachedFromDetails + missFromDetails
}
const totalOutputTokens = usage.output_tokens ?? usage.completion_tokens ?? 0
const cacheWriteTokens = usage.cache_creation_input_tokens ?? usage.cache_write_tokens ?? 0
const cacheReadTokens =
usage.cache_read_input_tokens ?? usage.cache_read_tokens ?? usage.cached_tokens ?? cachedFromDetails ?? 0
const reasoningTokens =
typeof usage.output_tokens_details?.reasoning_tokens === "number"
? usage.output_tokens_details.reasoning_tokens
: undefined
// Subscription-based: no per-token costs
const out: ApiStreamUsageChunk = {
type: "usage",
inputTokens: totalInputTokens,
outputTokens: totalOutputTokens,
cacheWriteTokens,
cacheReadTokens,
...(typeof reasoningTokens === "number" ? { reasoningTokens } : {}),
totalCost: 0, // Subscription-based pricing
}
return out
}
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
const model = this.getModel()
// Reset state for this request
this.pendingToolCallId = undefined
this.pendingToolCallName = undefined
// Get access token from OAuth manager
let accessToken = await openAiCodexOAuthManager.getAccessToken()
if (!accessToken) {
throw new Error("Not authenticated with OpenAI Codex. Please sign in using the OpenAI Codex OAuth flow in settings.")
}
const useWebsocketMode = this.useWebsocketMode(model.info.apiFormat)
const { input, previousResponseId } = convertToOpenAIResponsesInput(messages, { usePreviousResponseId: useWebsocketMode })
const usePreviousResponseId = useWebsocketMode && !!previousResponseId
// Build request body
const requestBody = this.buildRequestBody(model, input, systemPrompt, tools, previousResponseId)
const fallbackRequestBody = this.buildRequestBody(model, input, systemPrompt, tools)
// Make the request with retry on auth failure
for (let attempt = 0; attempt < 2; attempt++) {
try {
yield* this.executeRequest(requestBody, fallbackRequestBody, model, accessToken, usePreviousResponseId)
return
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
const isAuthFailure = /unauthorized|invalid token|not authenticated|authentication|401/i.test(message)
if (attempt === 0 && isAuthFailure) {
// Force refresh the token for retry
const refreshed = await openAiCodexOAuthManager.forceRefreshAccessToken()
if (!refreshed) {
throw new Error(
"Not authenticated with OpenAI Codex. Please sign in using the OpenAI Codex OAuth flow in settings.",
)
}
accessToken = refreshed
continue
}
throw error
}
}
}
private useWebsocketMode(apiFormat?: ApiFormat): boolean {
if (featureFlagsService.getBooleanFlagEnabled(FeatureFlag.OPENAI_RESPONSES_WEBSOCKET_MODE)) {
return apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE
}
return false
}
private buildRequestBody(
model: { id: string; info: ModelInfo },
formattedInput: any,
systemPrompt: string,
tools?: ChatCompletionTool[],
previousResponseId?: string,
): any {
// Determine reasoning effort
const reasoningEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
const includeReasoning = reasoningEffort !== "none"
const body: any = {
model: model.id,
input: formattedInput,
stream: true,
store: false,
instructions: systemPrompt,
...(previousResponseId ? { previous_response_id: previousResponseId } : {}),
...(includeReasoning ? { include: ["reasoning.encrypted_content"] } : {}),
...(includeReasoning
? {
reasoning: {
effort: reasoningEffort,
summary: "auto",
},
}
: {}),
}
// Add tools if provided
// Pass through strict value from tool (MCP/custom tools have strict: false, built-in tools default to true)
if (tools && tools.length > 0) {
body.tools = tools
.filter((tool: any) => tool?.type === "function")
.map((tool: any) => ({
type: "function",
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters,
strict: tool.function.strict ?? true,
}))
}
return body
}
private async *executeRequest(
requestBody: any,
fallbackRequestBody: any,
model: { id: string; info: ModelInfo },
accessToken: string,
useWebsocketMode: boolean,
): ApiStream {
// Create AbortController for cancellation
this.abortController = new AbortController()
try {
// Get ChatGPT account ID for organization subscriptions
const accountId = await openAiCodexOAuthManager.getAccountId()
// Build Codex-specific headers
const codexHeaders: Record<string, string> = {
originator: "cline",
session_id: this.sessionId,
"User-Agent": `cline/${process.env.npm_package_version || "1.0.0"} (${os.platform()} ${os.release()}; ${os.arch()}) node/${process.version.slice(1)}`,
...(accountId ? { "ChatGPT-Account-Id": accountId } : {}),
...buildExternalBasicHeaders(),
}
if (useWebsocketMode) {
try {
yield* this.createResponseStreamWebsocket(requestBody, fallbackRequestBody, accessToken, codexHeaders, model)
return
} catch (error) {
Logger.error("OpenAI Codex websocket mode failed, falling back to HTTP Responses API:", error)
this.closeResponsesWebsocket()
}
}
// Try using OpenAI SDK first
try {
const client =
this.client ??
new OpenAI({
apiKey: accessToken,
baseURL: CODEX_API_BASE_URL,
defaultHeaders: codexHeaders,
fetch, // Use shared fetch for proxy support
})
const stream = (await (client as any).responses.create(requestBody, {
signal: this.abortController.signal,
headers: codexHeaders,
})) as AsyncIterable<any>
if (typeof (stream as any)?.[Symbol.asyncIterator] !== "function") {
throw new Error("OpenAI SDK did not return an AsyncIterable")
}
for await (const event of stream) {
if (this.abortController.signal.aborted) {
break
}
for await (const outChunk of this.processEvent(event, model)) {
yield outChunk
}
}
} catch (_sdkErr) {
// Fallback to manual SSE via fetch
yield* this.makeCodexRequest(requestBody, model, accessToken)
}
} finally {
this.abortController = undefined
}
}
private async *createResponseStreamWebsocket(
primaryParams: OpenAI.Responses.ResponseCreateParamsStreaming,
fallbackParams: OpenAI.Responses.ResponseCreateParamsStreaming,
accessToken: string,
codexHeaders: Record<string, string>,
model: { id: string; info: ModelInfo },
): ApiStream {
try {
for await (const event of this.createResponseEventsViaWebsocket(primaryParams, accessToken, codexHeaders)) {
if (this.abortController?.signal.aborted) {
return
}
yield* this.processEvent(event, model)
}
} catch (error) {
if (this.shouldRetryWebsocketWithFullContext(error, !!primaryParams.previous_response_id)) {
Logger.log(
"Retrying Codex websocket response with full context after previous_response_not_found or socket reset",
)
this.closeResponsesWebsocket()
for await (const event of this.createResponseEventsViaWebsocket(fallbackParams, accessToken, codexHeaders)) {
if (this.abortController?.signal.aborted) {
return
}
yield* this.processEvent(event, model)
}
return
}
throw error
}
}
private shouldRetryWebsocketWithFullContext(error: unknown, hadPreviousResponseId: boolean): boolean {
const errorCode =
typeof error === "object" && error && "code" in error && typeof (error as { code: unknown }).code === "string"
? (error as { code: string }).code
: undefined
if (hadPreviousResponseId && errorCode === "previous_response_not_found") {
return true
}
if (errorCode === "websocket_closed" || errorCode === "websocket_error") {
return true
}
return false
}
private async ensureResponsesWebsocket(accessToken: string, codexHeaders: Record<string, string>): Promise<UndiciWebSocket> {
if (this.responsesWs && this.responsesWs.readyState === UndiciWebSocket.OPEN) {
return this.responsesWs
}
this.closeResponsesWebsocket()
const ws = new UndiciWebSocket(CODEX_RESPONSES_WEBSOCKET_URL, {
headers: {
Authorization: `Bearer ${accessToken}`,
"OpenAI-Beta": "responses_websockets=2026-02-06",
...codexHeaders,
},
})
await new Promise<void>((resolve, reject) => {
const cleanup = () => {
ws.removeEventListener("open", handleOpen)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
}
const handleOpen = () => {
cleanup()
resolve()
}
const handleError = () => {
cleanup()
reject(new Error("Failed to open Codex Responses websocket"))
}
const handleClose = () => {
cleanup()
reject(new Error("Codex Responses websocket closed before opening"))
}
ws.addEventListener("open", handleOpen)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
})
this.responsesWs = ws
return ws
}
private closeResponsesWebsocket() {
if (this.responsesWs) {
try {
this.responsesWs.close()
} catch {}
this.responsesWs = undefined
}
}
private async *createResponseEventsViaWebsocket(
params: OpenAI.Responses.ResponseCreateParamsStreaming,
accessToken: string,
codexHeaders: Record<string, string>,
): AsyncGenerator<OpenAI.Responses.ResponseStreamEvent> {
if (this.websocketRequestInFlight) {
const error: Error & { code?: string } = new Error("Websocket response.create is already in progress")
error.code = "websocket_concurrency_limit"
throw error
}
const ws = await this.ensureResponsesWebsocket(accessToken, codexHeaders)
this.websocketRequestInFlight = true
const eventQueue: OpenAI.Responses.ResponseStreamEvent[] = []
let resolver: (() => void) | undefined
let completed = false
let failure: (Error & { code?: string }) | undefined
const wake = () => {
const next = resolver
resolver = undefined
next?.()
}
const handleMessage = (evt: UndiciMessageEvent) => {
try {
let raw = ""
if (typeof evt.data === "string") {
raw = evt.data
} else if (evt.data instanceof ArrayBuffer) {
raw = new TextDecoder().decode(new Uint8Array(evt.data))
} else if (ArrayBuffer.isView(evt.data)) {
raw = new TextDecoder().decode(new Uint8Array(evt.data.buffer, evt.data.byteOffset, evt.data.byteLength))
} else {
raw = String(evt.data)
}
const parsed = JSON.parse(raw)
if (parsed?.type === "error" && parsed?.error) {
const error: Error & { code?: string } = new Error(parsed.error.message || "Codex Responses websocket error")
error.code = parsed.error.code
failure = error
completed = true
wake()
return
}
eventQueue.push(parsed as OpenAI.Responses.ResponseStreamEvent)
if (parsed?.type === "response.completed" || parsed?.type === "response.failed") {
completed = true
}
wake()
} catch (error) {
const parseError: Error & { code?: string } = new Error(
`Failed to parse websocket event: ${error instanceof Error ? error.message : String(error)}`,
)
parseError.code = "websocket_parse_error"
failure = parseError
completed = true
wake()
}
}
const handleError = () => {
const error: Error & { code?: string } = new Error("Codex Responses websocket emitted an error event")
error.code = "websocket_error"
failure = error
completed = true
wake()
}
const handleClose = () => {
if (!completed) {
const error: Error & { code?: string } = new Error("Codex Responses websocket closed during response stream")
error.code = "websocket_closed"
failure = error
completed = true
wake()
}
}
ws.addEventListener("message", handleMessage)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
try {
ws.send(
JSON.stringify({
type: "response.create",
...params,
}),
)
while (!completed || eventQueue.length > 0) {
if (eventQueue.length === 0) {
await new Promise<void>((resolve) => {
resolver = resolve
})
continue
}
const event = eventQueue.shift()
if (event) {
yield event
}
}
if (failure) {
throw failure
}
} finally {
ws.removeEventListener("message", handleMessage)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
this.websocketRequestInFlight = false
}
}
private async *makeCodexRequest(requestBody: any, model: { id: string; info: ModelInfo }, accessToken: string): ApiStream {
const url = `${CODEX_API_BASE_URL}/responses`
// Get ChatGPT account ID for organization subscriptions
const accountId = await openAiCodexOAuthManager.getAccountId()
// Build headers with required Codex-specific fields
const headers: Record<string, string> = {
"Content-Type": "application/json",
Authorization: `Bearer ${accessToken}`,
originator: "cline",
session_id: this.sessionId,
"User-Agent": `cline/${process.env.npm_package_version || "1.0.0"} (${os.platform()} ${os.release()}; ${os.arch()}) node/${process.version.slice(1)}`,
}
// Add ChatGPT-Account-Id if available
if (accountId) {
headers["ChatGPT-Account-Id"] = accountId
}
try {
const response = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify(requestBody),
signal: this.abortController?.signal,
})
if (!response.ok) {
const errorText = await response.text()
let errorMessage = `Codex API request failed: ${response.status}`
try {
const errorJson = JSON.parse(errorText)
if (errorJson.error?.message) {
errorMessage = errorJson.error.message
} else if (errorJson.message) {
errorMessage = errorJson.message
}
} catch {
if (errorText) {
errorMessage += ` - ${errorText}`
}
}
throw new Error(errorMessage)
}
if (!response.body) {
throw new Error("No response body from Codex API")
}
yield* this.handleStreamResponse(response.body, model)
} catch (error) {
if (error instanceof Error) {
throw new Error(`Codex API error: ${error.message}`)
}
throw new Error("Unexpected error connecting to Codex API")
}
}
private async *handleStreamResponse(body: ReadableStream<Uint8Array>, model: { id: string; info: ModelInfo }): ApiStream {
const reader = body.getReader()
const decoder = new TextDecoder()
let buffer = ""
try {
while (true) {
if (this.abortController?.signal.aborted) {
break
}
const { done, value } = await reader.read()
if (done) {
break
}
buffer += decoder.decode(value, { stream: true })
const lines = buffer.split("\n")
buffer = lines.pop() || ""
for (const line of lines) {
if (line.startsWith("data: ")) {
const data = line.slice(6).trim()
if (data === "[DONE]") {
continue
}
try {
const parsed = JSON.parse(data)
for await (const outChunk of this.processEvent(parsed, model)) {
yield outChunk
}
} catch (e) {
if (!(e instanceof SyntaxError)) {
throw e
}
}
}
}
}
} finally {
reader.releaseLock()
}
}
private async *processEvent(event: any, model: { id: string; info: ModelInfo }): ApiStream {
// Handle text deltas
if (event?.type === "response.text.delta" || event?.type === "response.output_text.delta") {
if (event?.delta) {
yield { type: "text", text: event.delta }
}
return
}
// Handle reasoning deltas
if (
event?.type === "response.reasoning.delta" ||
event?.type === "response.reasoning_text.delta" ||
event?.type === "response.reasoning_summary.delta" ||
event?.type === "response.reasoning_summary_text.delta"
) {
if (event?.delta) {
yield { type: "reasoning", reasoning: event.delta }
}
return
}
// Handle refusal deltas
if (event?.type === "response.refusal.delta") {
if (event?.delta) {
yield { type: "text", text: `[Refusal] ${event.delta}` }
}
return
}
// Handle tool/function call deltas
if (event?.type === "response.tool_call_arguments.delta" || event?.type === "response.function_call_arguments.delta") {
const callId = event.call_id || event.tool_call_id || event.id || this.pendingToolCallId
const name = event.name || event.function_name || this.pendingToolCallName
const args = event.delta || event.arguments
if (typeof callId === "string" && callId.length > 0 && typeof name === "string" && name.length > 0) {
yield {
type: "tool_calls",
tool_call: {
call_id: callId,
function: {
id: callId,
name,
arguments: typeof args === "string" ? args : "",
},
},
}
}
return
}
// Handle output item events
if (event?.type === "response.output_item.added" || event?.type === "response.output_item.done") {
const item = event?.item
if (item) {
// Capture tool identity for subsequent argument deltas
if (item.type === "function_call" || item.type === "tool_call") {
const callId = item.call_id || item.tool_call_id || item.id
const name = item.name || item.function?.name || item.function_name
if (typeof callId === "string" && callId.length > 0) {
this.pendingToolCallId = callId
this.pendingToolCallName = typeof name === "string" ? name : undefined
}
}
if (item.type === "text" && item.text) {
yield { type: "text", text: item.text }
} else if (item.type === "reasoning" && item.text) {
yield { type: "reasoning", reasoning: item.text }
} else if (item.type === "message" && Array.isArray(item.content)) {
for (const content of item.content) {
if ((content?.type === "text" || content?.type === "output_text") && content?.text) {
yield { type: "text", text: content.text }
}
}
} else if (
(item.type === "function_call" || item.type === "tool_call") &&
event.type === "response.output_item.done"
) {
const callId = item.call_id || item.tool_call_id || item.id
if (callId) {
const args = item.arguments || item.function?.arguments || item.function_arguments
yield {
type: "tool_calls",
id: callId,
tool_call: {
call_id: callId,
function: {
id: callId,
name: item.name || item.function?.name || item.function_name || "",
arguments: typeof args === "string" ? args : "{}",
},
},
}
}
}
}
return
}
// Handle completion events
if (event?.type === "response.done" || event?.type === "response.completed") {
const usage = event?.response?.usage || event?.usage || undefined
const usageData = this.normalizeUsage(usage, model)
if (usageData) {
yield usageData
}
return
}
// Fallbacks for legacy formats
if (event?.choices?.[0]?.delta?.content) {
yield { type: "text", text: event.choices[0].delta.content }
return
}
if (event?.usage) {
const usageData = this.normalizeUsage(event.usage, model)
if (usageData) {
yield usageData
}
}
}
abort(): void {
this.closeResponsesWebsocket()
this.abortController?.abort()
}
getModel(): { id: OpenAiCodexModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
const id = modelId && modelId in openAiCodexModels ? (modelId as OpenAiCodexModelId) : openAiCodexDefaultModelId
const info: ModelInfo = openAiCodexModels[id]
return { id, info }
}
}
@@ -1,702 +0,0 @@
import {
ModelInfo,
OpenAiCompatibleModelInfo,
OpenAiNativeModelId,
openAiNativeDefaultModelId,
openAiNativeModels,
} from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type {
ChatCompletionFunctionTool,
ChatCompletionReasoningEffort,
ChatCompletionTool,
} from "openai/resources/chat/completions"
import { MessageEvent as UndiciMessageEvent, WebSocket as UndiciWebSocket } from "undici"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { featureFlagsService } from "@/services/feature-flags"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiFormat } from "@/shared/proto/cline/models"
import { FeatureFlag } from "@/shared/services/feature-flags/feature-flags"
import { Logger } from "@/shared/services/Logger"
import { isGPT5ModelFamily } from "@/utils/model-utils"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface OpenAiNativeHandlerOptions extends CommonApiHandlerOptions {
openAiNativeApiKey?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
apiModelId?: string
openAiNativeUseResponsesWebsocket?: boolean
}
export class OpenAiNativeHandler implements ApiHandler {
private options: OpenAiNativeHandlerOptions
private client: OpenAI | undefined
private responsesWs: UndiciWebSocket | undefined
private responsesWsReadyPromise: Promise<UndiciWebSocket> | undefined
private websocketRequestInFlight = false
private abortController?: AbortController
constructor(options: OpenAiNativeHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.openAiNativeApiKey) {
throw new Error("OpenAI API key is required")
}
try {
this.client = createOpenAIClient({
apiKey: this.options.openAiNativeApiKey,
})
} catch (error) {
throw new Error(`Error creating OpenAI client: ${error instanceof Error ? error.message : String(error)}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
const inputTokens = usage?.prompt_tokens || 0 // sum of cache hits and misses
const outputTokens = usage?.completion_tokens || 0
const cacheReadTokens = usage?.prompt_tokens_details?.cached_tokens || 0
const cacheWriteTokens = 0
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
totalCost: totalCost,
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
// Responses API requires tool format to be set to OPENAI_RESPONSES with native tools calling enabled
const apiFormat = this.getModel()?.info?.apiFormat
if (apiFormat === ApiFormat.OPENAI_RESPONSES || apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE) {
if (!tools?.length) {
throw new Error("Native Tool Call must be enabled in your setting for OpenAI Responses API")
}
yield* this.createResponseStream(systemPrompt, messages, tools)
} else {
yield* this.createCompletionStream(systemPrompt, messages, tools)
}
}
private async *createCompletionStream(
systemPrompt: string,
messages: ClineStorageMessage[],
tools?: ChatCompletionTool[],
): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const toolCallProcessor = new ToolCallProcessor()
this.abortController = new AbortController()
// Handle o1 models separately as they don't support streaming
if (model.info.supportsStreaming === false) {
const response = await client.chat.completions.create(
{
model: model.id,
messages: [{ role: "user", content: systemPrompt }, ...convertToOpenAiMessages(messages, "openai-native")],
},
{ signal: this.abortController?.signal },
)
yield {
type: "text",
text: response.choices[0]?.message.content || "",
}
yield* this.yieldUsage(model.info, response.usage)
return
}
const systemRole = model.info.systemRole ?? "system"
const includeReasoning = model.info.supportsReasoningEffort
const includeTools = model.info.supportsTools ?? true
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
const reasoningEffort =
includeReasoning && requestedEffort !== "none" ? (requestedEffort as ChatCompletionReasoningEffort) : undefined
const stream = await client.chat.completions.create({
model: model.id,
messages: [{ role: systemRole, content: systemPrompt }, ...convertToOpenAiMessages(messages, "openai-native")],
stream: true,
stream_options: { include_usage: true },
reasoning_effort: reasoningEffort,
...(model.info.temperature !== undefined ? { temperature: model.info.temperature } : {}),
...(includeTools ? getOpenAIToolParams(tools, isGPT5ModelFamily(model.id)) : {}),
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
try {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
} catch (error) {
Logger.error("Error processing tool call delta:", error, delta.tool_calls)
}
}
if (chunk.usage) {
// Only last chunk contains usage
yield* this.yieldUsage(model.info, chunk.usage)
}
}
}
private async *createResponseStream(
systemPrompt: string,
messages: ClineStorageMessage[],
tools: ChatCompletionTool[],
): ApiStream {
const model = this.getModel()
const usePreviousResponseId = this.useWebsocketMode(model.info.apiFormat)
// Warm websocket connection early in websocket mode so the first response.create avoids handshake latency.
if (usePreviousResponseId) {
this.preconnectResponsesWebsocket()
}
const { input, previousResponseId } = convertToOpenAIResponsesInput(messages, { usePreviousResponseId })
const responseTools = this.mapResponseTools(tools)
this.abortController = new AbortController()
const params = this.buildResponseCreateParams({
modelId: model.id,
systemPrompt,
input,
previousResponseId,
tools: responseTools,
})
const fallbackParams = this.buildResponseCreateParams({
modelId: model.id,
systemPrompt,
input,
tools: responseTools,
})
if (usePreviousResponseId && previousResponseId) {
try {
yield* this.createResponseStreamWebsocket(model.info, params, fallbackParams)
return
} catch (error) {
Logger.error("OpenAI websocket mode failed, falling back to HTTP Responses API:", error)
this.closeResponsesWebsocket()
}
}
yield* this.createResponseStreamHttp(model.info, params)
}
private preconnectResponsesWebsocket(): void {
void this.ensureResponsesWebsocket().catch((error) => {
Logger.debug("OpenAI websocket preconnect failed:", error)
this.closeResponsesWebsocket()
})
}
private useWebsocketMode(apiFormat?: ApiFormat): boolean {
if (featureFlagsService.getBooleanFlagEnabled(FeatureFlag.OPENAI_RESPONSES_WEBSOCKET_MODE)) {
return apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE
}
return false
}
private mapResponseTools(tools: ChatCompletionTool[]): OpenAI.Responses.Tool[] {
return tools
?.filter((tool): tool is ChatCompletionFunctionTool => tool?.type === "function")
.map((tool) => ({
type: "function" as const,
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters ?? null,
strict: tool.function.strict ?? true,
}))
}
private buildResponseCreateParams(args: {
modelId: string
systemPrompt: string
input: OpenAI.Responses.ResponseInput
tools: OpenAI.Responses.Tool[]
previousResponseId?: string
}): OpenAI.Responses.ResponseCreateParamsStreaming {
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
const reasoning: { effort: ChatCompletionReasoningEffort; summary: "auto" } | undefined =
requestedEffort === "none"
? undefined
: {
effort: requestedEffort,
summary: "auto",
}
return {
model: args.modelId,
instructions: args.systemPrompt,
input: args.input,
stream: true,
tools: args.tools,
store: !args.previousResponseId, // Do not use store when websocket mode is enabled.
...(args.previousResponseId ? { previous_response_id: args.previousResponseId } : {}),
...(reasoning ? { reasoning } : {}),
}
}
private async *createResponseStreamHttp(
modelInfo: ModelInfo,
params: OpenAI.Responses.ResponseCreateParamsStreaming,
): ApiStream {
const client = this.ensureClient()
Logger.debug(`OpenAI Responses Input (HTTP): ${JSON.stringify(params.input)}`)
const stream = await client.responses.create(params, { signal: this.abortController?.signal })
yield* this.processResponsesEvents(stream, modelInfo)
}
private async *createResponseStreamWebsocket(
modelInfo: ModelInfo,
primaryParams: OpenAI.Responses.ResponseCreateParamsStreaming,
fallbackParams: OpenAI.Responses.ResponseCreateParamsStreaming,
): ApiStream {
Logger.debug(`OpenAI Responses Input (WebSocket): ${JSON.stringify(primaryParams.input)}`)
try {
yield* this.processResponsesEvents(this.createResponseEventsViaWebsocket(primaryParams), modelInfo)
} catch (error) {
if (this.shouldRetryWebsocketWithFullContext(error, !!primaryParams.previous_response_id)) {
Logger.log("Retrying websocket response with full context after previous_response_not_found or socket reset")
this.closeResponsesWebsocket()
yield* this.processResponsesEvents(this.createResponseEventsViaWebsocket(fallbackParams), modelInfo)
return
}
throw error
}
}
private shouldRetryWebsocketWithFullContext(error: unknown, hadPreviousResponseId: boolean): boolean {
const errorCode =
typeof error === "object" && error && "code" in error && typeof (error as { code: unknown }).code === "string"
? (error as { code: string }).code
: undefined
if (hadPreviousResponseId && errorCode === "previous_response_not_found") {
return true
}
if (errorCode === "websocket_closed" || errorCode === "websocket_error") {
return true
}
return false
}
private async ensureResponsesWebsocket(): Promise<UndiciWebSocket> {
if (this.responsesWs && this.responsesWs.readyState === UndiciWebSocket.OPEN) {
return this.responsesWs
}
if (this.responsesWsReadyPromise) {
return this.responsesWsReadyPromise
}
this.closeResponsesWebsocket()
if (!this.options.openAiNativeApiKey) {
throw new Error("OpenAI API key is required")
}
const ws = new UndiciWebSocket("wss://api.openai.com/v1/responses", {
headers: {
Authorization: `Bearer ${this.options.openAiNativeApiKey}`,
"OpenAI-Beta": "responses_websockets=2026-02-06",
...buildExternalBasicHeaders(),
},
})
this.responsesWs = ws
const readyPromise = new Promise<UndiciWebSocket>((resolve, reject) => {
const cleanup = () => {
ws.removeEventListener("open", handleOpen)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
}
const handleOpen = () => {
cleanup()
resolve(ws)
}
const handleError = () => {
cleanup()
reject(new Error("Failed to open Responses websocket"))
}
const handleClose = () => {
cleanup()
reject(new Error("Responses websocket closed before opening"))
}
ws.addEventListener("open", handleOpen)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
})
this.responsesWsReadyPromise = readyPromise
try {
return await readyPromise
} catch (error) {
if (this.responsesWs === ws) {
this.responsesWs = undefined
}
throw error
} finally {
if (this.responsesWsReadyPromise === readyPromise) {
this.responsesWsReadyPromise = undefined
}
}
}
private closeResponsesWebsocket() {
this.responsesWsReadyPromise = undefined
if (this.responsesWs) {
try {
this.responsesWs.close()
} catch {}
this.responsesWs = undefined
}
}
private async *createResponseEventsViaWebsocket(
params: OpenAI.Responses.ResponseCreateParamsStreaming,
): AsyncGenerator<OpenAI.Responses.ResponseStreamEvent> {
if (this.websocketRequestInFlight) {
const error: Error & { code?: string } = new Error("Websocket response.create is already in progress")
error.code = "websocket_concurrency_limit"
throw error
}
const ws = await this.ensureResponsesWebsocket()
this.websocketRequestInFlight = true
const eventQueue: OpenAI.Responses.ResponseStreamEvent[] = []
let resolver: (() => void) | undefined
let completed = false
let failure: (Error & { code?: string }) | undefined
const wake = () => {
const next = resolver
resolver = undefined
next?.()
}
const handleMessage = (evt: UndiciMessageEvent) => {
try {
let raw = ""
if (typeof evt.data === "string") {
raw = evt.data
} else if (evt.data instanceof ArrayBuffer) {
raw = new TextDecoder().decode(new Uint8Array(evt.data))
} else if (ArrayBuffer.isView(evt.data)) {
raw = new TextDecoder().decode(new Uint8Array(evt.data.buffer, evt.data.byteOffset, evt.data.byteLength))
} else {
raw = String(evt.data)
}
const parsed = JSON.parse(raw)
if (parsed?.type === "error" && parsed?.error) {
const error: Error & { code?: string } = new Error(parsed.error.message || "Responses websocket error")
error.code = parsed.error.code
failure = error
completed = true
wake()
return
}
eventQueue.push(parsed as OpenAI.Responses.ResponseStreamEvent)
if (parsed?.type === "response.completed" || parsed?.type === "response.failed") {
completed = true
}
wake()
} catch (error) {
const parseError: Error & { code?: string } = new Error(
`Failed to parse websocket event: ${error instanceof Error ? error.message : String(error)}`,
)
parseError.code = "websocket_parse_error"
failure = parseError
completed = true
wake()
}
}
const handleError = () => {
const error: Error & { code?: string } = new Error("Responses websocket emitted an error event")
error.code = "websocket_error"
failure = error
completed = true
wake()
}
const handleClose = () => {
if (!completed) {
const error: Error & { code?: string } = new Error("Responses websocket closed during response stream")
error.code = "websocket_closed"
failure = error
completed = true
wake()
}
}
ws.addEventListener("message", handleMessage)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
try {
ws.send(
JSON.stringify({
type: "response.create",
...params,
}),
)
while (!completed || eventQueue.length > 0) {
if (eventQueue.length === 0) {
await new Promise<void>((resolve) => {
resolver = resolve
})
continue
}
const event = eventQueue.shift()
if (event) {
yield event
}
}
if (failure) {
throw failure
}
} finally {
ws.removeEventListener("message", handleMessage)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
this.websocketRequestInFlight = false
}
}
private async *processResponsesEvents(
stream: AsyncIterable<OpenAI.Responses.ResponseStreamEvent>,
modelInfo: ModelInfo,
): ApiStream {
const functionCallByItemId = new Map<string, { call_id?: string; name?: string; id?: string }>()
for await (const chunk of stream) {
Logger.debug(`OpenAI Responses Chunk: ${JSON.stringify(chunk)}`)
if (chunk.type === "response.output_item.added") {
const item = chunk.item
if (item.type === "function_call" && item.id) {
functionCallByItemId.set(item.id, { call_id: item.call_id, name: item.name, id: item.id })
yield {
type: "tool_calls",
id: item.id,
tool_call: {
call_id: item.call_id,
function: {
id: item.id,
name: item.name,
arguments: item.arguments,
},
},
}
}
if (item.type === "reasoning" && item.encrypted_content && item.id) {
yield {
type: "reasoning",
id: item.id,
reasoning: "",
redacted_data: item.encrypted_content,
}
}
}
if (chunk.type === "response.output_item.done") {
const item = chunk.item
if (item.type === "function_call") {
if (item.id) {
functionCallByItemId.set(item.id, { call_id: item.call_id, name: item.name, id: item.id })
}
yield {
type: "tool_calls",
id: item.id || item.call_id,
tool_call: {
call_id: item.call_id,
function: {
id: item.id,
name: item.name,
arguments: item.arguments,
},
},
}
}
if (item.type === "reasoning") {
yield {
type: "reasoning",
id: item.id,
details: item.summary,
reasoning: "",
}
}
}
if (chunk.type === "response.reasoning_summary_part.added") {
yield {
type: "reasoning",
id: chunk.item_id,
reasoning: chunk.part.text,
}
}
if (chunk.type === "response.reasoning_summary_text.delta") {
yield {
type: "reasoning",
id: chunk.item_id,
reasoning: chunk.delta,
}
}
if (chunk.type === "response.reasoning_summary_part.done") {
yield {
type: "reasoning",
id: chunk.item_id,
details: chunk.part,
reasoning: "",
}
}
if (chunk.type === "response.output_text.delta") {
if (chunk.delta) {
yield {
id: chunk.item_id,
type: "text",
text: chunk.delta,
}
}
}
if (chunk.type === "response.reasoning_text.delta") {
if (chunk.delta) {
yield {
id: chunk.item_id,
type: "reasoning",
reasoning: chunk.delta,
}
}
}
if (chunk.type === "response.function_call_arguments.delta") {
const pendingCall = functionCallByItemId.get(chunk.item_id)
const callId = pendingCall?.call_id
const functionName = pendingCall?.name
const functionId = pendingCall?.id || chunk.item_id
yield {
type: "tool_calls",
tool_call: {
call_id: callId,
function: {
id: functionId,
name: functionName,
arguments: chunk.delta,
},
},
}
}
if (chunk.type === "response.function_call_arguments.done") {
if (chunk.item_id && chunk.name && chunk.arguments) {
const pendingCall = functionCallByItemId.get(chunk.item_id)
const callId = pendingCall?.call_id
const functionId = pendingCall?.id || chunk.item_id
yield {
type: "tool_calls",
tool_call: {
call_id: callId,
function: {
id: functionId,
name: chunk.name,
arguments: chunk.arguments,
},
},
}
}
}
if (
chunk.type === "response.incomplete" &&
chunk.response?.status === "incomplete" &&
chunk.response?.incomplete_details?.reason === "max_output_tokens"
) {
if (chunk.response?.output_text?.length > 0) {
Logger.log("Partial output:", chunk.response.output_text)
} else {
Logger.log("Ran out of tokens during reasoning")
}
}
if (chunk.type === "response.completed" && chunk.response?.usage) {
const usage = chunk.response.usage
const inputTokens = usage.input_tokens || 0
const outputTokens = usage.output_tokens || 0
const cacheReadTokens = usage.input_tokens_details?.cached_tokens || 0
const cacheWriteTokens = 0
const reasoningTokens = usage.output_tokens_details?.reasoning_tokens || 0
const totalTokens = usage.total_tokens || 0
Logger.log(`Total tokens from Responses API usage: ${totalTokens}`)
const totalCost = calculateApiCostOpenAI(
modelInfo,
inputTokens,
outputTokens + reasoningTokens,
cacheWriteTokens,
cacheReadTokens,
)
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
thoughtsTokenCount: reasoningTokens,
totalCost: totalCost,
id: chunk.response.id,
}
}
}
}
abort(): void {
this.closeResponsesWebsocket()
this.abortController?.abort()
this.abortController = undefined
}
getModel(): { id: OpenAiNativeModelId; info: OpenAiCompatibleModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in openAiNativeModels) {
const id = modelId as OpenAiNativeModelId
const info = openAiNativeModels[id]
return { id, info: { ...info } }
}
return {
id: openAiNativeDefaultModelId,
info: { ...openAiNativeModels[openAiNativeDefaultModelId] },
}
}
}
@@ -1,188 +0,0 @@
import { DefaultAzureCredential, getBearerTokenProvider } from "@azure/identity"
import { azureOpenAiDefaultApiVersion, ModelInfo, OpenAiCompatibleModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import OpenAI, { AzureOpenAI } from "openai"
import type { ChatCompletionReasoningEffort, ChatCompletionTool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient, fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface OpenAiHandlerOptions extends CommonApiHandlerOptions {
openAiApiKey?: string
openAiBaseUrl?: string
azureApiVersion?: string
azureIdentity?: boolean
openAiHeaders?: Record<string, string>
openAiModelId?: string
openAiModelInfo?: OpenAiCompatibleModelInfo
reasoningEffort?: string
}
export class OpenAiHandler implements ApiHandler {
private options: OpenAiHandlerOptions
private client: OpenAI | undefined
constructor(options: OpenAiHandlerOptions) {
this.options = options
}
private getAzureAudienceScope(baseUrl?: string): string {
const url = baseUrl?.toLowerCase() ?? ""
if (url.includes("azure.us")) return "https://cognitiveservices.azure.us/.default"
if (url.includes("azure.com")) return "https://cognitiveservices.azure.com/.default"
return "https://cognitiveservices.azure.com/.default"
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.openAiApiKey && !this.options.azureIdentity) {
throw new Error("OpenAI API key or Azure Identity Authentication is required")
}
try {
const baseUrl = this.options.openAiBaseUrl?.toLowerCase() ?? ""
const isAzureDomain = baseUrl.includes("azure.com") || baseUrl.includes("azure.us")
const externalHeaders = buildExternalBasicHeaders()
// Azure API shape slightly differs from the core API shape...
if (
this.options.azureApiVersion ||
(isAzureDomain && !this.options.openAiModelId?.toLowerCase().includes("deepseek"))
) {
if (this.options.azureIdentity) {
this.client = new AzureOpenAI({
baseURL: this.options.openAiBaseUrl,
azureADTokenProvider: getBearerTokenProvider(
new DefaultAzureCredential(),
this.getAzureAudienceScope(this.options.openAiBaseUrl),
),
apiVersion: this.options.azureApiVersion || azureOpenAiDefaultApiVersion,
defaultHeaders: {
...externalHeaders,
...this.options.openAiHeaders,
},
fetch,
})
} else {
this.client = new AzureOpenAI({
baseURL: this.options.openAiBaseUrl,
apiKey: this.options.openAiApiKey,
apiVersion: this.options.azureApiVersion || azureOpenAiDefaultApiVersion,
defaultHeaders: {
...externalHeaders,
...this.options.openAiHeaders,
},
fetch,
})
}
} else {
this.client = createOpenAIClient({
baseURL: this.options.openAiBaseUrl,
apiKey: this.options.openAiApiKey,
defaultHeaders: this.options.openAiHeaders,
})
}
} catch (error: any) {
throw new Error(`Error creating OpenAI client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.openAiModelId ?? ""
const isDeepseekReasoner = modelId.includes("deepseek-reasoner")
const isR1FormatRequired = this.options.openAiModelInfo?.isR1FormatRequired ?? false
const isReasoningModelFamily =
["o1", "o3", "o4", "gpt-5"].some((prefix) => modelId.includes(prefix)) && !modelId.includes("chat")
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
let temperature: number | undefined
if (this.options.openAiModelInfo?.temperature !== undefined) {
const tempValue = Number(this.options.openAiModelInfo.temperature)
temperature = tempValue === 0 ? undefined : tempValue
} else {
temperature = openAiModelInfoSaneDefaults.temperature
}
let reasoningEffort: ChatCompletionReasoningEffort | undefined
let maxTokens: number | undefined
if (this.options.openAiModelInfo?.maxTokens && this.options.openAiModelInfo.maxTokens > 0) {
maxTokens = Number(this.options.openAiModelInfo.maxTokens)
} else {
maxTokens = undefined
}
if (isDeepseekReasoner || isR1FormatRequired) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
if (isReasoningModelFamily) {
openAiMessages = [{ role: "developer", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
temperature = undefined // does not support temperature
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
reasoningEffort = requestedEffort === "none" ? undefined : (requestedEffort as ChatCompletionReasoningEffort)
}
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
temperature,
max_tokens: maxTokens,
reasoning_effort: reasoningEffort,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.openAiModelId ?? "",
info: this.options.openAiModelInfo ?? openAiModelInfoSaneDefaults,
}
}
}
@@ -1,235 +0,0 @@
import { setTimeout as setTimeoutPromise } from "node:timers/promises"
import { StateManager } from "@core/storage/StateManager"
import { ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import axios from "axios"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient, getAxiosSettings } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { createOpenRouterStream } from "../transform/openrouter-stream"
import { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
import { OpenRouterErrorResponse } from "./types"
interface OpenRouterHandlerOptions extends CommonApiHandlerOptions {
openRouterApiKey?: string
openRouterModelId?: string
openRouterModelInfo?: ModelInfo
openRouterProviderSorting?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
enableParallelToolCalling?: boolean
}
export class OpenRouterHandler implements ApiHandler {
private options: OpenRouterHandlerOptions
private client: OpenAI | undefined
lastGenerationId?: string
constructor(options: OpenRouterHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.openRouterApiKey) {
throw new Error("OpenRouter API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://openrouter.ai/api/v1",
apiKey: this.options.openRouterApiKey,
defaultHeaders: {
"HTTP-Referer": "https://cline.bot", // Optional, for including your app on openrouter.ai rankings.
"X-Title": "Cline", // Optional. Shows in rankings on openrouter.ai.
},
})
} catch (error: any) {
throw new Error(`Error creating OpenRouter client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
this.lastGenerationId = undefined
const stream = await createOpenRouterStream(
client,
systemPrompt,
messages,
this.getModel(),
this.options.reasoningEffort,
this.options.thinkingBudgetTokens,
this.options.openRouterProviderSorting,
tools,
this.options.enableParallelToolCalling,
)
let didOutputUsage = false
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
// openrouter returns an error object instead of the openai sdk throwing an error
// Check for error field directly on chunk
if ("error" in chunk) {
const error = chunk.error as OpenRouterErrorResponse["error"]
Logger.error(`OpenRouter API Error: ${error?.code} - ${error?.message}`)
// Include metadata in the error message if available
const metadataStr = error.metadata ? `\nMetadata: ${JSON.stringify(error.metadata, null, 2)}` : ""
throw new Error(`OpenRouter API Error ${error.code}: ${error.message}${metadataStr}`)
}
// Check for error in choices[0].finish_reason
// OpenRouter may return errors in a non-standard way within choices
const choice = chunk.choices?.[0]
// Use type assertion since OpenRouter uses non-standard "error" finish_reason
if ((choice?.finish_reason as string) === "error") {
// Use type assertion since OpenRouter adds non-standard error property
const choiceWithError = choice as any
if (choiceWithError.error) {
const error = choiceWithError.error
Logger.error(
`OpenRouter Mid-Stream Error: ${error?.code || "Unknown"} - ${error?.message || "Unknown error"}`,
)
// Format error details
const errorDetails = typeof error === "object" ? JSON.stringify(error, null, 2) : String(error)
throw new Error(`OpenRouter Mid-Stream Error: ${errorDetails}`)
}
// Fallback if error details are not available
throw new Error(`OpenRouter Mid-Stream Error: Stream terminated with error status but no error details provided`)
}
if (!this.lastGenerationId && chunk.id) {
this.lastGenerationId = chunk.id
}
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Reasoning tokens are returned separately from the content
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
if (
delta &&
"reasoning" in delta &&
delta.reasoning &&
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
// OpenRouter passes reasoning details that we can pass back unmodified in api requests to preserve reasoning traces for model
// See: https://openrouter.ai/docs/use-cases/reasoning-tokens#preserving-reasoning-blocks
if (
delta &&
"reasoning_details" in delta &&
delta.reasoning_details &&
// @ts-expect-error-next-line
delta.reasoning_details.length && // exists and non-0
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: "",
details: delta.reasoning_details,
}
}
if (!didOutputUsage && chunk.usage) {
// @ts-expect-error-next-line -- OpenRouter returns cache_write_tokens for Anthropic models
const cacheWriteTokens = chunk.usage.prompt_tokens_details?.cache_write_tokens || 0
yield {
type: "usage",
cacheWriteTokens,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
inputTokens:
(chunk.usage.prompt_tokens || 0) -
(chunk.usage.prompt_tokens_details?.cached_tokens || 0) -
(cacheWriteTokens || 0),
outputTokens: chunk.usage.completion_tokens || 0,
// @ts-expect-error-next-line
totalCost: (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0),
}
didOutputUsage = true
}
}
// Fallback to generation endpoint if usage chunk not returned
if (!didOutputUsage) {
const apiStreamUsage = await this.getApiStreamUsage()
if (apiStreamUsage) {
yield apiStreamUsage
}
}
}
async getApiStreamUsage(): Promise<ApiStreamUsageChunk | undefined> {
if (this.lastGenerationId) {
await setTimeoutPromise(500) // FIXME: necessary delay to ensure generation endpoint is ready
try {
const generationIterator = this.fetchGenerationDetails(this.lastGenerationId)
const generation = (await generationIterator.next()).value
// Logger.log("OpenRouter generation details:", generation)
return {
type: "usage",
cacheWriteTokens: generation?.native_tokens_cache_write || 0,
cacheReadTokens: generation?.native_tokens_cached || 0,
// openrouter generation endpoint fails often
inputTokens: (generation?.native_tokens_prompt || 0) - (generation?.native_tokens_cached || 0),
outputTokens: generation?.native_tokens_completion || 0,
totalCost: generation?.total_cost || 0,
}
} catch (error) {
// ignore if fails
Logger.error("Error fetching OpenRouter generation details:", error)
}
}
return undefined
}
@withRetry({ maxRetries: 4, baseDelay: 250, maxDelay: 1000, retryAllErrors: true })
async *fetchGenerationDetails(genId: string) {
// Logger.log("Fetching generation details for:", genId)
try {
const response = await axios.get(`https://openrouter.ai/api/v1/generation?id=${genId}`, {
headers: {
Authorization: `Bearer ${this.options.openRouterApiKey}`,
},
timeout: 15_000, // this request hangs sometimes
...getAxiosSettings(),
})
yield response.data?.data
} catch (error) {
// ignore if fails
Logger.error("Error fetching OpenRouter generation details:", error)
throw error
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.openRouterModelId || openRouterDefaultModelId
const cachedModelInfo = StateManager.get().getModelInfo("openRouter", modelId)
return {
id: modelId,
info: cachedModelInfo || openRouterDefaultModelInfo,
}
}
}
@@ -1,286 +0,0 @@
import { promises as fs } from "node:fs"
import { ModelInfo, QwenCodeModelId, qwenCodeDefaultModelId, qwenCodeModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import * as os from "os"
import * as path from "path"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
// --- Constants for Qwen OAuth2 ---
const QWEN_OAUTH_BASE_URL = "https://chat.qwen.ai"
const QWEN_OAUTH_TOKEN_ENDPOINT = `${QWEN_OAUTH_BASE_URL}/api/v1/oauth2/token`
const QWEN_OAUTH_CLIENT_ID = "f0304373b74a44d2b584a3fb70ca9e56"
const QWEN_DIR = ".qwen"
const QWEN_CREDENTIAL_FILENAME = "oauth_creds.json"
interface QwenOAuthCredentials {
access_token: string
refresh_token: string
token_type: string
expiry_date: number
resource_url?: string
}
interface QwenCodeHandlerOptions extends CommonApiHandlerOptions {
qwenCodeOauthPath?: string
apiModelId?: string
}
function getQwenCachedCredentialPath(customPath?: string): string {
if (customPath) {
// Support custom path that starts with ~/ or is absolute
if (customPath.startsWith("~/")) {
return path.join(os.homedir(), customPath.slice(2))
}
return path.resolve(customPath)
}
return path.join(os.homedir(), QWEN_DIR, QWEN_CREDENTIAL_FILENAME)
}
function objectToUrlEncoded(data: Record<string, string>): string {
return Object.keys(data)
.map((key) => `${encodeURIComponent(key)}=${encodeURIComponent(data[key])}`)
.join("&")
}
export class QwenCodeHandler implements ApiHandler {
private options: QwenCodeHandlerOptions
private credentials: QwenOAuthCredentials | null = null
private client: OpenAI | undefined
constructor(options: QwenCodeHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
// Create the client instance with dummy key initially
// The API key will be updated dynamically via ensureAuthenticated
const externalHeaders = buildExternalBasicHeaders()
this.client = new OpenAI({
apiKey: "dummy-key-will-be-replaced",
baseURL: "https://dashscope.aliyuncs.com/compatible-mode/v1",
defaultHeaders: externalHeaders,
fetch,
})
}
return this.client
}
private async loadCachedQwenCredentials(): Promise<QwenOAuthCredentials> {
try {
const keyFile = getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)
const credsStr = await fs.readFile(keyFile, "utf-8")
return JSON.parse(credsStr)
} catch (error) {
Logger.error(
`Error reading or parsing credentials file at ${getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)}`,
)
throw new Error(`Failed to load Qwen OAuth credentials: ${error}`)
}
}
private async refreshAccessToken(credentials: QwenOAuthCredentials): Promise<QwenOAuthCredentials> {
if (!credentials.refresh_token) {
throw new Error("No refresh token available in credentials.")
}
const bodyData = {
grant_type: "refresh_token",
refresh_token: credentials.refresh_token,
client_id: QWEN_OAUTH_CLIENT_ID,
}
const response = await fetch(QWEN_OAUTH_TOKEN_ENDPOINT, {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded",
Accept: "application/json",
},
body: objectToUrlEncoded(bodyData),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Token refresh failed: ${response.status} ${response.statusText}. Response: ${errorText}`)
}
const tokenData = await response.json()
if (tokenData.error) {
throw new Error(`Token refresh failed: ${tokenData.error} - ${tokenData.error_description}`)
}
const newCredentials = {
...credentials,
access_token: tokenData.access_token,
token_type: tokenData.token_type,
refresh_token: tokenData.refresh_token || credentials.refresh_token,
expiry_date: Date.now() + tokenData.expires_in * 1000,
}
const filePath = getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)
await fs.writeFile(filePath, JSON.stringify(newCredentials, null, 2))
return newCredentials
}
private isTokenValid(credentials: QwenOAuthCredentials): boolean {
const TOKEN_REFRESH_BUFFER_MS = 30 * 1000 // 30s buffer
if (!credentials.expiry_date) {
return false
}
return Date.now() < credentials.expiry_date - TOKEN_REFRESH_BUFFER_MS
}
private async ensureAuthenticated(): Promise<void> {
if (!this.credentials) {
this.credentials = await this.loadCachedQwenCredentials()
}
if (!this.isTokenValid(this.credentials)) {
this.credentials = await this.refreshAccessToken(this.credentials)
}
// After authentication, update the apiKey and baseURL on the existing client
const client = this.ensureClient()
client.apiKey = this.credentials.access_token
client.baseURL = this.getBaseUrl(this.credentials)
}
private getBaseUrl(creds: QwenOAuthCredentials): string {
let baseUrl = creds.resource_url || "https://dashscope.aliyuncs.com/compatible-mode/v1"
if (!baseUrl.startsWith("http://") && !baseUrl.startsWith("https://")) {
baseUrl = `https://${baseUrl}`
}
return baseUrl.endsWith("/v1") ? baseUrl : `${baseUrl}/v1`
}
private async callApiWithRetry<T>(apiCall: () => Promise<T>): Promise<T> {
try {
return await apiCall()
} catch (error: any) {
if (error.status === 401) {
// Token expired, refresh and retry
this.credentials = await this.refreshAccessToken(this.credentials!)
const client = this.ensureClient()
client.apiKey = this.credentials.access_token
client.baseURL = this.getBaseUrl(this.credentials)
return await apiCall()
} else {
throw error
}
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
await this.ensureAuthenticated()
const client = this.ensureClient()
const model = this.getModel()
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam = {
role: "system",
content: systemPrompt,
}
const convertedMessages = [systemMessage, ...convertToOpenAiMessages(messages)]
const requestOptions: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming = {
model: model.id,
temperature: 0,
messages: convertedMessages,
stream: true,
stream_options: { include_usage: true },
max_completion_tokens: model.info.maxTokens,
...getOpenAIToolParams(tools),
}
const stream = await this.callApiWithRetry(() => client.chat.completions.create(requestOptions))
const toolCallProcessor = new ToolCallProcessor()
let fullContent = ""
for await (const apiChunk of stream) {
const delta = apiChunk.choices[0]?.delta ?? {}
if (delta.content) {
let newText = delta.content
if (newText.startsWith(fullContent)) {
newText = newText.substring(fullContent.length)
}
fullContent = delta.content
if (newText) {
// Check for thinking blocks
if (newText.includes("<think>") || newText.includes("</think>")) {
// Simple parsing for thinking blocks
const parts = newText.split(/<\/?think>/g)
for (let i = 0; i < parts.length; i++) {
if (parts[i]) {
if (i % 2 === 0) {
// Outside thinking block
yield {
type: "text",
text: parts[i],
}
} else {
// Inside thinking block
yield {
type: "reasoning",
reasoning: parts[i],
}
}
}
}
} else {
yield {
type: "text",
text: newText,
}
}
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle reasoning content (o1-style)
if ("reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (apiChunk.usage) {
yield {
type: "usage",
inputTokens: apiChunk.usage.prompt_tokens || 0,
outputTokens: apiChunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: QwenCodeModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in qwenCodeModels) {
const id = modelId as QwenCodeModelId
return { id, info: qwenCodeModels[id] }
}
return {
id: qwenCodeDefaultModelId,
info: qwenCodeModels[qwenCodeDefaultModelId],
}
}
}
-163
View File
@@ -1,163 +0,0 @@
import {
InternationalQwenModelId,
internationalQwenDefaultModelId,
internationalQwenModels,
MainlandQwenModelId,
ModelInfo,
mainlandQwenDefaultModelId,
mainlandQwenModels,
QwenApiRegions,
} from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface QwenHandlerOptions extends CommonApiHandlerOptions {
qwenApiKey?: string
qwenApiLine?: QwenApiRegions
apiModelId?: string
thinkingBudgetTokens?: number
}
export class QwenHandler implements ApiHandler {
private options: QwenHandlerOptions
private client: OpenAI | undefined
constructor(options: QwenHandlerOptions) {
// Ensure options start with defaults but allow overrides
this.options = {
qwenApiLine: QwenApiRegions.CHINA,
...options,
}
}
private useChinaApi(): boolean {
return this.options.qwenApiLine === QwenApiRegions.CHINA
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.qwenApiKey) {
throw new Error("Alibaba API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: this.useChinaApi()
? "https://dashscope.aliyuncs.com/compatible-mode/v1"
: "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
apiKey: this.options.qwenApiKey,
})
} catch (error: any) {
throw new Error(`Error creating Alibaba client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: MainlandQwenModelId | InternationalQwenModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
// Branch based on API line to let poor typescript know what to do
if (this.useChinaApi()) {
const id = modelId && modelId in mainlandQwenModels ? (modelId as MainlandQwenModelId) : mainlandQwenDefaultModelId
return {
id,
info: mainlandQwenModels[id],
}
}
const id =
modelId && modelId in internationalQwenModels
? (modelId as InternationalQwenModelId)
: internationalQwenDefaultModelId
return {
id,
info: internationalQwenModels[id],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const isDeepseekReasoner = model.id.includes("deepseek-r1")
const isReasoningModelFamily = model.id.includes("qwen3") || ["qwen-plus-latest", "qwen-turbo-latest"].includes(model.id)
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
let temperature: number | undefined = 0
// Configuration for extended thinking
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = budgetTokens !== 0
const thinkingArgs = isReasoningModelFamily
? {
enable_thinking: reasoningOn,
thinking_budget: reasoningOn ? budgetTokens : undefined,
}
: undefined
if (isDeepseekReasoner || (reasoningOn && isReasoningModelFamily)) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
temperature = undefined
}
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature,
...thinkingArgs,
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
try {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
} catch (error) {
Logger.error("Error processing tool call delta:", error, delta.tool_calls)
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
// @ts-expect-error-next-line
cacheReadTokens: chunk.usage.prompt_cache_hit_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
}
@@ -1,162 +0,0 @@
import { ModelInfo, requestyDefaultModelId, requestyDefaultModelInfo } from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import { toRequestyServiceStringUrl } from "@/shared/clients/requesty"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface RequestyHandlerOptions extends CommonApiHandlerOptions {
requestyBaseUrl?: string
requestyApiKey?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
requestyModelId?: string
requestyModelInfo?: ModelInfo
}
// Requesty usage includes an extra field for Anthropic use cases.
// Safely cast the prompt token details section to the appropriate structure.
interface RequestyUsage extends OpenAI.CompletionUsage {
prompt_tokens_details?: {
caching_tokens?: number
cached_tokens?: number
}
total_cost?: number
}
export class RequestyHandler implements ApiHandler {
private options: RequestyHandlerOptions
private client: OpenAI | undefined
constructor(options: RequestyHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.requestyApiKey) {
throw new Error("Requesty API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: toRequestyServiceStringUrl(this.options.requestyBaseUrl),
apiKey: this.options.requestyApiKey,
defaultHeaders: {
"HTTP-Referer": "https://cline.bot",
"X-Title": "Cline",
},
})
} catch (error: any) {
throw new Error(`Error creating Requesty client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const reasoningEffort = this.options.reasoningEffort || "medium"
const reasoning = { reasoning_effort: reasoningEffort }
const reasoningArgs = model.id.startsWith("openai/o") ? reasoning : {}
const thinkingBudget = this.options.thinkingBudgetTokens || 0
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(model.id)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, thinkingBudget)
: undefined
const thinking =
thinkingBudget > 0
? { thinking: { type: "enabled", budget_tokens: thinkingBudget } }
: { thinking: { type: "disabled" } }
const supportsLegacyClaudeThinking =
!isAdaptiveThinkingModel &&
(model.id.includes("claude-3-7-sonnet") ||
model.id.includes("claude-4.6-sonnet") ||
model.id.includes("claude-sonnet-4") ||
model.id.includes("claude-opus-4"))
const thinkingArgs = isAdaptiveThinkingModel
? adaptiveThinking?.enabled
? {
thinking: { type: "adaptive" },
...(adaptiveThinking.effort ? { output_config: { effort: adaptiveThinking.effort } } : {}),
}
: {}
: supportsLegacyClaudeThinking
? thinking
: {}
const stream = await client.chat.completions.create({
model: model.id,
max_tokens: model.info.maxTokens || undefined,
messages: openAiMessages,
...(isAdaptiveThinkingModel ? {} : { temperature: 0 }),
stream: true,
stream_options: { include_usage: true },
...reasoningArgs,
...thinkingArgs,
})
let lastUsage: any
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
lastUsage = chunk.usage
}
}
if (lastUsage) {
const usage = lastUsage as RequestyUsage
const inputTokens = usage.prompt_tokens || 0
const outputTokens = usage.completion_tokens || 0
const cacheWriteTokens = usage.prompt_tokens_details?.caching_tokens || undefined
const cacheReadTokens = usage.prompt_tokens_details?.cached_tokens || undefined
const totalCost = calculateApiCostOpenAI(model.info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
yield {
type: "usage",
inputTokens: inputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
totalCost: totalCost,
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.requestyModelId
const modelInfo = this.options.requestyModelInfo
if (modelId && modelInfo) {
return { id: modelId, info: modelInfo }
}
return { id: requestyDefaultModelId, info: requestyDefaultModelInfo }
}
}
@@ -1,103 +0,0 @@
import { ModelInfo, SambanovaModelId, sambanovaDefaultModelId, sambanovaModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface SambanovaHandlerOptions extends CommonApiHandlerOptions {
sambanovaApiKey?: string
apiModelId?: string
}
export class SambanovaHandler implements ApiHandler {
private options: SambanovaHandlerOptions
private client: OpenAI | undefined
constructor(options: SambanovaHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.sambanovaApiKey) {
throw new Error("SambaNova API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.sambanova.ai/v1",
apiKey: this.options.sambanovaApiKey,
})
} catch (error: any) {
throw new Error(`Error creating SambaNova client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const modelId = model.id.toLowerCase()
if (modelId.includes("deepseek") || modelId.includes("qwen3")) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
const toolCallProcessor = new ToolCallProcessor()
const stream = await client.chat.completions.create({
model: this.getModel().id,
messages: openAiMessages,
temperature: model.info.temperature ?? 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in sambanovaModels) {
const id = modelId as SambanovaModelId
return { id, info: sambanovaModels[id] }
}
return {
id: sambanovaDefaultModelId,
info: sambanovaModels[sambanovaDefaultModelId],
}
}
}
File diff suppressed because it is too large Load Diff
@@ -1,103 +0,0 @@
import { ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface TogetherHandlerOptions extends CommonApiHandlerOptions {
togetherApiKey?: string
togetherModelId?: string
}
export class TogetherHandler implements ApiHandler {
private options: TogetherHandlerOptions
private client: OpenAI | undefined
constructor(options: TogetherHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.togetherApiKey) {
throw new Error("Together API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.together.xyz/v1",
apiKey: this.options.togetherApiKey,
})
} catch (error: any) {
throw new Error(`Error creating Together client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.togetherModelId ?? ""
const isDeepseekReasoner = modelId.includes("deepseek-reasoner")
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
if (isDeepseekReasoner) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
temperature: 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.togetherModelId ?? "",
info: openAiModelInfoSaneDefaults,
}
}
}
@@ -1,29 +0,0 @@
// For the following openrouter error type sources, see the docs here:
// https://openrouter.ai/docs/api-reference/errors
export interface LanguageModelChatSelector {
vendor?: string
family?: string
version?: string
id?: string
}
export type OpenRouterErrorResponse = {
error: {
message: string
code: number
metadata?: OpenRouterProviderErrorMetadata | OpenRouterModerationErrorMetadata | Record<string, unknown>
}
}
export type OpenRouterProviderErrorMetadata = {
provider_name: string // The name of the provider that encountered the error
raw: unknown // The raw error from the provider
}
export type OpenRouterModerationErrorMetadata = {
reasons: string[] // Why your input was flagged
flagged_input: string // The text segment that was flagged, limited to 100 characters. If the flagged input is longer than 100 characters, it will be truncated in the middle and replaced with ...
provider_name: string // The name of the provider that requested moderation
model_slug: string
}
@@ -1,164 +0,0 @@
import { ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
import { createVercelAIGatewayStream } from "../transform/vercel-ai-gateway-stream"
interface VercelAIGatewayHandlerOptions extends CommonApiHandlerOptions {
vercelAiGatewayApiKey?: string
openRouterModelId?: string
openRouterModelInfo?: ModelInfo
reasoningEffort?: string
thinkingBudgetTokens?: number
}
function getCacheReadTokens(usage: any): number {
return usage?.prompt_tokens_details?.cached_tokens || usage?.cache_read_input_tokens || 0
}
function getCacheWriteTokens(usage: any): number {
return usage?.prompt_tokens_details?.cache_write_tokens || usage?.cache_creation_input_tokens || 0
}
export class VercelAIGatewayHandler implements ApiHandler {
private options: VercelAIGatewayHandlerOptions
private client: OpenAI | undefined
constructor(options: VercelAIGatewayHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.vercelAiGatewayApiKey) {
throw new Error("Vercel AI Gateway API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://ai-gateway.vercel.sh/v1",
apiKey: this.options.vercelAiGatewayApiKey,
defaultHeaders: {
"http-referer": "https://cline.bot",
"x-title": "Cline",
},
})
} catch (error: any) {
throw new Error(`Error creating Vercel AI Gateway client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.getModel().id
const modelInfo = this.getModel().info
try {
const stream = await createVercelAIGatewayStream(
client,
systemPrompt,
messages,
{ id: modelId, info: modelInfo },
this.options.reasoningEffort,
this.options.thinkingBudgetTokens,
tools,
)
let didOutputUsage = false
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Reasoning tokens are returned separately from the content
// Skip reasoning content for models that don't support it (e.g., devstral, grok-4)
if (
delta &&
"reasoning" in delta &&
delta.reasoning &&
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
// Reasoning details that can be passed back in API requests to preserve reasoning traces
if (
delta &&
"reasoning_details" in delta &&
delta.reasoning_details &&
// @ts-expect-error-next-line
delta.reasoning_details.length && // exists and non-0
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: "",
details: delta.reasoning_details,
}
}
if (!didOutputUsage && chunk.usage) {
// @ts-expect-error - Vercel AI Gateway extends OpenAI types
const totalCost = (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0)
const cacheReadTokens = getCacheReadTokens(chunk.usage)
const cacheWriteTokens = getCacheWriteTokens(chunk.usage)
yield {
type: "usage",
cacheWriteTokens,
cacheReadTokens,
inputTokens: Math.max(0, (chunk.usage.prompt_tokens || 0) - cacheReadTokens - cacheWriteTokens),
outputTokens: chunk.usage.completion_tokens || 0,
totalCost,
}
didOutputUsage = true
}
}
if (!didOutputUsage) {
Logger.warn("Vercel AI Gateway did not provide usage information in stream")
}
} catch (error: any) {
Logger.error("Vercel AI Gateway error details:", error)
Logger.error("Error stack:", error.stack)
throw new Error(`Vercel AI Gateway error: ${error.message}`)
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.openRouterModelId
const modelInfo = this.options.openRouterModelInfo
if (modelId && modelInfo) {
return { id: modelId, info: modelInfo }
}
// If we have a model ID but no model info, preserve the selected model ID
// and fall back only the metadata to defaults.
if (modelId) {
return { id: modelId, info: openRouterDefaultModelInfo }
}
return { id: openRouterDefaultModelId, info: openRouterDefaultModelInfo }
}
}
@@ -1,281 +0,0 @@
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
import { AnthropicVertex } from "@anthropic-ai/vertex-sdk"
import { FunctionDeclaration as GoogleTool } from "@google/genai"
import { CLAUDE_SONNET_1M_SUFFIX, ModelInfo, VertexModelId, vertexDefaultModelId, vertexModels } from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { ClineTool } from "@/shared/tools"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { ApiStream } from "../transform/stream"
import { GeminiHandler } from "./gemini"
interface VertexHandlerOptions extends CommonApiHandlerOptions {
vertexProjectId?: string
vertexRegion?: string
apiModelId?: string
thinkingBudgetTokens?: number
geminiApiKey?: string
geminiBaseUrl?: string
ulid?: string
reasoningEffort?: string
}
export class VertexHandler implements ApiHandler {
private geminiHandler: GeminiHandler | undefined
private clientAnthropic: AnthropicVertex | undefined
private options: VertexHandlerOptions
constructor(options: VertexHandlerOptions) {
this.options = options
}
private ensureGeminiHandler(): GeminiHandler {
if (!this.geminiHandler) {
try {
// Create a GeminiHandler with isVertex flag for Gemini models
this.geminiHandler = new GeminiHandler({
...this.options,
isVertex: true,
})
} catch (error: any) {
throw new Error(`Error creating Vertex AI Gemini handler: ${error.message}`)
}
}
return this.geminiHandler
}
private ensureAnthropicClient(): AnthropicVertex {
if (!this.clientAnthropic) {
if (!this.options.vertexProjectId) {
throw new Error("Vertex AI project ID is required")
}
if (!this.options.vertexRegion) {
throw new Error("Vertex AI region is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
// Initialize Anthropic client for Claude models.
// The AnthropicVertex SDK constructs the base URL as `${region}-aiplatform.googleapis.com`,
// but the global endpoint uses `aiplatform.googleapis.com` (no region prefix).
// See: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-partner-models#global
this.clientAnthropic = new AnthropicVertex({
projectId: this.options.vertexProjectId,
// https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#regions
region: this.options.vertexRegion,
...(this.options.vertexRegion === "global"
? { baseURL: "https://aiplatform.googleapis.com/v1" }
: {}),
defaultHeaders: externalHeaders,
})
} catch (error: any) {
throw new Error(`Error creating Vertex AI Anthropic client: ${error.message}`)
}
}
return this.clientAnthropic
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ClineTool[]): ApiStream {
const model = this.getModel()
const rawModelId = model.id
const modelId = rawModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
? rawModelId.slice(0, -CLAUDE_SONNET_1M_SUFFIX.length)
: rawModelId
const enable1mContextWindow = rawModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
// For Gemini models, use the GeminiHandler
if (!rawModelId.includes("claude")) {
const geminiHandler = this.ensureGeminiHandler()
yield* geminiHandler.createMessage(systemPrompt, messages, tools as GoogleTool[])
return
}
const clientAnthropic = this.ensureAnthropicClient()
// Claude implementation
const budget_tokens = this.options.thinkingBudgetTokens || 0
// Use model metadata to determine if reasoning should be enabled
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
// Claude Opus 4.5+ uses adaptive thinking instead of budgeted extended thinking.
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budget_tokens)
: undefined
const adaptiveThinkingEnabled = adaptiveThinking?.enabled === true
const adaptiveThinkingEffort = adaptiveThinking?.effort
const thinkingEnabled = isAdaptiveThinkingModel ? adaptiveThinkingEnabled : reasoningOn
const thinkingConfig = thinkingEnabled
? isAdaptiveThinkingModel
? ({ type: "adaptive" } as any)
: { type: "enabled", budget_tokens: budget_tokens }
: undefined
const outputConfig = isAdaptiveThinkingModel && adaptiveThinkingEffort ? { effort: adaptiveThinkingEffort } : undefined
// Tools are available only when native tools are enabled.
const nativeToolsOn = tools?.length ? tools?.length > 0 : false
const anthropicMessages = sanitizeAnthropicMessages(messages, model.info.supportsPromptCache ?? false)
const requestBody: Record<string, unknown> = {
model: modelId,
max_tokens: model.info.maxTokens || 8192,
thinking: thinkingConfig,
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
system: [
{
text: systemPrompt,
type: "text",
cache_control: model.info.supportsPromptCache ? { type: "ephemeral" } : undefined,
},
],
messages: anthropicMessages,
stream: true,
tools: nativeToolsOn ? (tools as AnthropicTool[]) : undefined,
// tool_choice options:
// - none: disables tool use, even if tools are provided. Claude will not call any tools.
// - auto: allows Claude to decide whether to call any provided tools or not. This is the default value when tools are provided.
// - any: tells Claude that it must use one of the provided tools, but doesnt force a particular tool.
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
tool_choice: nativeToolsOn && !thinkingEnabled ? { type: "any" } : undefined,
}
if (outputConfig) {
requestBody.output_config = outputConfig
}
const stream = (await clientAnthropic.beta.messages.create(
requestBody as any,
enable1mContextWindow
? {
headers: {
"anthropic-beta": "context-1m-2025-08-07",
},
}
: undefined,
)) as unknown as AsyncIterable<any>
const lastStartedToolCall = { id: "", name: "", arguments: "" }
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start": {
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
break
}
case "message_delta":
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage?.output_tokens || 0,
}
break
case "message_stop":
break
case "content_block_start":
switch (chunk.content_block.type) {
case "thinking":
yield {
type: "reasoning",
reasoning: chunk.content_block.thinking || "",
}
break
case "redacted_thinking":
// Handle redacted thinking blocks - we still mark it as reasoning
// but note that the content is encrypted
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
}
break
case "tool_use":
if (chunk.content_block.id && chunk.content_block.name) {
// Convert Anthropic tool_use to OpenAI-compatible format
lastStartedToolCall.id = chunk.content_block.id
lastStartedToolCall.name = chunk.content_block.name
lastStartedToolCall.arguments = ""
}
break
case "text":
if (chunk.index > 0) {
yield {
type: "text",
text: "\n",
}
}
yield {
type: "text",
text: chunk.content_block.text,
}
break
}
break
case "content_block_delta":
switch (chunk.delta.type) {
case "signature_delta":
yield {
type: "reasoning",
reasoning: "",
signature: chunk.delta.signature,
}
break
case "thinking_delta":
yield {
type: "reasoning",
reasoning: chunk.delta.thinking,
}
break
case "input_json_delta":
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
// // Convert Anthropic tool_use to OpenAI-compatible format
yield {
type: "tool_calls",
tool_call: {
...lastStartedToolCall,
function: {
id: lastStartedToolCall.id,
name: lastStartedToolCall.name,
arguments: chunk.delta.partial_json,
},
},
}
}
break
case "text_delta":
yield {
type: "text",
text: chunk.delta.text,
}
break
}
break
case "content_block_stop":
lastStartedToolCall.id = ""
lastStartedToolCall.name = ""
lastStartedToolCall.arguments = ""
break
}
}
}
getModel(): { id: VertexModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in vertexModels) {
const id = modelId as VertexModelId
return { id, info: vertexModels[id] }
}
return {
id: vertexDefaultModelId,
info: vertexModels[vertexDefaultModelId],
}
}
}
@@ -1,594 +0,0 @@
import { ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import { SELECTOR_SEPARATOR, stringifyVsCodeLmModelSelector } from "@shared/vsCodeSelectorUtils"
import { calculateApiCostAnthropic } from "@utils/cost"
import * as vscode from "vscode"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions, SingleCompletionHandler } from "../"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
import { convertToVsCodeLmMessages } from "../transform/vscode-lm-format"
import type { LanguageModelChatSelector as LanguageModelChatSelectorFromTypes } from "./types"
interface VsCodeLmHandlerOptions extends CommonApiHandlerOptions {
vsCodeLmModelSelector?: any
}
// Cline does not update VSCode type definitions or engine requirements to maintain compatibility.
// This declaration (as seen in src/integrations/TerminalManager.ts) provides types for the Language Model API in newer versions of VSCode.
// Extracted from https://github.com/microsoft/vscode/blob/131ee0ef660d600cd0a7e6058375b281553abe20/src/vscode-dts/vscode.d.ts
declare module "vscode" {
enum LanguageModelChatMessageRole {
User = 1,
Assistant = 2,
}
enum LanguageModelChatToolMode {
Auto = 1,
Required = 2,
}
interface LanguageModelChatSelector extends LanguageModelChatSelectorFromTypes {}
interface LanguageModelChatTool {
name: string
description: string
inputSchema?: object
}
interface LanguageModelChatRequestOptions {
justification?: string
modelOptions?: { [name: string]: any }
tools?: LanguageModelChatTool[]
toolMode?: LanguageModelChatToolMode
}
class LanguageModelTextPart {
value: string
constructor(value: string)
}
class LanguageModelToolCallPart {
callId: string
name: string
input: object
constructor(callId: string, name: string, input: object)
}
interface LanguageModelChatResponse {
stream: AsyncIterable<LanguageModelTextPart | LanguageModelToolCallPart | unknown>
text: AsyncIterable<string>
}
interface LanguageModelChat {
readonly name: string
readonly id: string
readonly vendor: string
readonly family: string
readonly version: string
readonly maxInputTokens: number
sendRequest(
messages: LanguageModelChatMessage[],
options?: LanguageModelChatRequestOptions,
token?: CancellationToken,
): Thenable<LanguageModelChatResponse>
countTokens(text: string | LanguageModelChatMessage, token?: CancellationToken): Thenable<number>
}
class LanguageModelPromptTsxPart {
value: unknown
constructor(value: unknown)
}
class LanguageModelToolResultPart {
callId: string
content: Array<LanguageModelTextPart | LanguageModelPromptTsxPart | unknown>
constructor(callId: string, content: Array<LanguageModelTextPart | LanguageModelPromptTsxPart | unknown>)
}
class LanguageModelChatMessage {
static User(
content: string | Array<LanguageModelTextPart | LanguageModelToolResultPart>,
name?: string,
): LanguageModelChatMessage
static Assistant(
content: string | Array<LanguageModelTextPart | LanguageModelToolCallPart>,
name?: string,
): LanguageModelChatMessage
role: LanguageModelChatMessageRole
content: Array<LanguageModelTextPart | LanguageModelToolResultPart | LanguageModelToolCallPart>
name: string | undefined
constructor(
role: LanguageModelChatMessageRole,
content: string | Array<LanguageModelTextPart | LanguageModelToolResultPart | LanguageModelToolCallPart>,
name?: string,
)
}
namespace lm {
function selectChatModels(selector?: LanguageModelChatSelector): Thenable<LanguageModelChat[]>
}
}
/**
* Handles interaction with VS Code's Language Model API for chat-based operations.
* This handler implements the ApiHandler interface to provide VS Code LM specific functionality.
*
* @implements {ApiHandler}
*
* @remarks
* The handler manages a VS Code language model chat client and provides methods to:
* - Create and manage chat client instances
* - Stream messages using VS Code's Language Model API
* - Retrieve model information
*
* @example
* ```typescript
* const options = {
* vsCodeLmModelSelector: { vendor: "copilot", family: "gpt-4" }
* };
* const handler = new VsCodeLmHandler(options);
*
* // Stream a conversation
* const systemPrompt = "You are a helpful assistant";
* const messages = [{ role: "user", content: "Hello!" }];
* for await (const chunk of handler.createMessage(systemPrompt, messages)) {
* Logger.log(chunk);
* }
* ```
*/
export class VsCodeLmHandler implements ApiHandler, SingleCompletionHandler {
private options: VsCodeLmHandlerOptions
private client: vscode.LanguageModelChat | null
private disposable: vscode.Disposable | null
private currentRequestCancellation: vscode.CancellationTokenSource | null
constructor(options: VsCodeLmHandlerOptions) {
this.options = options
this.client = null
this.disposable = null
this.currentRequestCancellation = null
try {
// Listen for model changes and reset client
this.disposable = vscode.workspace.onDidChangeConfiguration((event) => {
if (event.affectsConfiguration("lm")) {
try {
this.client = null
this.ensureCleanState()
} catch (error) {
Logger.error("Error during configuration change cleanup:", error)
}
}
})
} catch (error) {
// Ensure cleanup if constructor fails
this.dispose()
throw new Error(
`Cline <Language Model API>: Failed to initialize handler: ${error instanceof Error ? error.message : "Unknown error"}`,
)
}
}
/**
* Creates a language model chat client based on the provided selector.
*
* @param selector - Selector criteria to filter language model chat instances
* @returns Promise resolving to the first matching language model chat instance
* @throws Error when no matching models are found with the given selector
*
* @example
* const selector = { vendor: "copilot", family: "gpt-4o" };
* const chatClient = await createClient(selector);
*/
async createClient(selector: vscode.LanguageModelChatSelector): Promise<vscode.LanguageModelChat> {
try {
const models = await vscode.lm.selectChatModels(selector)
// Use first available model or create a minimal model object
if (models && Array.isArray(models) && models.length > 0) {
return models[0]
}
// Create a minimal model if no models are available
return {
id: "default-lm",
name: "Default Language Model",
vendor: "vscode",
family: "lm",
version: "1.0",
maxInputTokens: 8192,
sendRequest: async (_messages, _options, _token) => {
// Provide a minimal implementation
return {
stream: (async function* () {
yield new vscode.LanguageModelTextPart(
"Language model functionality is limited. Please check VS Code configuration.",
)
})(),
text: (async function* () {
yield "Language model functionality is limited. Please check VS Code configuration."
})(),
}
},
countTokens: async () => 0,
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : "Unknown error"
throw new Error(`Cline <Language Model API>: Failed to select model: ${errorMessage}`)
}
}
/**
* Creates and streams a message using the VS Code Language Model API.
*
* @param systemPrompt - The system prompt to initialize the conversation context
* @param messages - An array of message parameters following the Anthropic message format
*
* @yields {ApiStream} An async generator that yields either text chunks or tool calls from the model response
*
* @throws {Error} When vsCodeLmModelSelector option is not provided
* @throws {Error} When the response stream encounters an error
*
* @remarks
* This method handles the initialization of the VS Code LM client if not already created,
* converts the messages to VS Code LM format, and streams the response chunks.
* Tool calls handling is currently a work in progress.
*/
dispose(): void {
if (this.disposable) {
this.disposable.dispose()
}
if (this.currentRequestCancellation) {
this.currentRequestCancellation.cancel()
this.currentRequestCancellation.dispose()
}
}
private extractTextFromMessage(message: vscode.LanguageModelChatMessage): string {
if (Array.isArray(message.content)) {
return message.content
.filter((part) => part instanceof vscode.LanguageModelTextPart)
.map((part) => (part as vscode.LanguageModelTextPart).value)
.join("")
}
return ""
}
private async countTokens(text: string | vscode.LanguageModelChatMessage): Promise<number> {
/**
* NOTE (intentional trade-off):
* We use a coarse chars/4 heuristic here instead of a real tokenizer (e.g., js-tiktoken with o200k_base).
* Rationale:
* - Avoid pulling multiMB rank files and increasing the extension install/download size.
* - Eliminate encoder lifecycle/memory concerns in long-running sessions.
* Consequences:
* - This is not model-accurate and can under/over-estimate tokens, especially with tool/function calls.
* - It is “good enough” for budgeting/context checks, and we accept the inaccuracy by design.
* If precise accounting becomes a requirement, reintroduce a tokenizer behind a feature flag or backend-only path.
*/
const textContent = typeof text === "string" ? text : this.extractTextFromMessage(text)
return Math.ceil((textContent || "").length / 4)
}
private async calculateTotalInputTokens(vsCodeLmMessages: vscode.LanguageModelChatMessage[]): Promise<number> {
const messageTokens: number[] = await Promise.all(vsCodeLmMessages.map((msg) => this.countTokens(msg)))
return messageTokens.reduce((sum: number, tokens: number): number => sum + tokens, 0)
}
private ensureCleanState(): void {
if (this.currentRequestCancellation) {
this.currentRequestCancellation.cancel()
this.currentRequestCancellation.dispose()
this.currentRequestCancellation = null
}
}
private async getClient(): Promise<vscode.LanguageModelChat> {
if (!this.client) {
Logger.debug("Cline <Language Model API>: Getting client with options:", {
vsCodeLmModelSelector: this.options.vsCodeLmModelSelector,
hasOptions: !!this.options,
selectorKeys: this.options.vsCodeLmModelSelector ? Object.keys(this.options.vsCodeLmModelSelector) : [],
})
try {
// Use default empty selector if none provided to get all available models
const selector = this.options?.vsCodeLmModelSelector || {}
Logger.debug("Cline <Language Model API>: Creating client with selector:", selector)
this.client = await this.createClient(selector)
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error"
Logger.error("Cline <Language Model API>: Client creation failed:", message)
throw new Error(`Cline <Language Model API>: Failed to create client: ${message}`)
}
}
return this.client
}
private cleanTerminalOutput(text: string): string {
if (!text) {
return ""
}
return (
text
// Normalize line breaks
.replace(/\r\n/g, "\n")
.replace(/\r/g, "\n")
// Remove ANSI escape sequences
.replace(/\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])/g, "") // Full set of ANSI sequences
.replace(/\x9B[0-?]*[ -/]*[@-~]/g, "") // CSI sequences
// Remove terminal title setting sequences and other OSC sequences
.replace(/\x1B\][0-9;]*(?:\x07|\x1B\\)/g, "")
// Remove control characters
.replace(/[\x00-\x09\x0B-\x0C\x0E-\x1F\x7F]/g, "")
// Remove VS Code escape sequences
.replace(/\x1B[PD].*?\x1B\\/g, "") // DCS sequences
.replace(/\x1B_.*?\x1B\\/g, "") // APC sequences
.replace(/\x1B\^.*?\x1B\\/g, "") // PM sequences
.replace(/\x1B\[[\d;]*[HfABCDEFGJKST]/g, "") // Cursor movement and clear screen
// Remove Windows paths and service information
.replace(/^(?:PS )?[A-Z]:\\[^\n]*$/gm, "")
.replace(/^;?Cwd=.*$/gm, "")
// Clean escaped sequences
.replace(/\\x[0-9a-fA-F]{2}/g, "")
.replace(/\\u[0-9a-fA-F]{4}/g, "")
// Final cleanup
.replace(/\n{3,}/g, "\n\n") // Remove multiple empty lines
.trim()
)
}
private cleanMessageContent(content: any): any {
if (!content) {
return content
}
if (typeof content === "string") {
return this.cleanTerminalOutput(content)
}
if (Array.isArray(content)) {
return content.map((item) => this.cleanMessageContent(item))
}
if (typeof content === "object") {
const cleaned: any = {}
for (const [key, value] of Object.entries(content)) {
cleaned[key] = this.cleanMessageContent(value)
}
return cleaned
}
return content
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
// Ensure clean state before starting a new request
this.ensureCleanState()
const client: vscode.LanguageModelChat = await this.getClient()
// Clean system prompt and messages
const cleanedSystemPrompt = this.cleanTerminalOutput(systemPrompt)
const cleanedMessages = messages.map((msg) => ({
...msg,
content: this.cleanMessageContent(msg.content),
}))
// Convert Anthropic messages to VS Code LM messages
const vsCodeLmMessages: vscode.LanguageModelChatMessage[] = [
vscode.LanguageModelChatMessage.Assistant(cleanedSystemPrompt),
...convertToVsCodeLmMessages(cleanedMessages),
]
// Initialize cancellation token for the request
this.currentRequestCancellation = new vscode.CancellationTokenSource()
// Calculate input tokens before starting the stream
const totalInputTokens: number = await this.calculateTotalInputTokens(vsCodeLmMessages)
// Accumulate the text and count at the end of the stream to reduce token counting overhead.
let accumulatedText: string = ""
try {
// Create the response stream with minimal required options
const requestOptions: vscode.LanguageModelChatRequestOptions = {
justification: `Cline would like to use '${client.name}' from '${client.vendor}', Click 'Allow' to proceed.`,
}
// Note: Tool support is currently provided by the VSCode Language Model API directly
// Extensions can register tools using vscode.lm.registerTool()
const response: vscode.LanguageModelChatResponse = await client.sendRequest(
vsCodeLmMessages,
requestOptions,
this.currentRequestCancellation.token,
)
// Consume the stream and handle both text and tool call chunks
for await (const chunk of response.stream) {
if (chunk instanceof vscode.LanguageModelTextPart) {
// Validate text part value
if (typeof chunk.value !== "string") {
Logger.warn("Cline <Language Model API>: Invalid text part value received:", chunk.value)
continue
}
accumulatedText += chunk.value
yield {
type: "text",
text: chunk.value,
}
} else if (chunk instanceof vscode.LanguageModelToolCallPart) {
try {
// Validate tool call parameters
if (!chunk.name || typeof chunk.name !== "string") {
Logger.warn("Cline <Language Model API>: Invalid tool name received:", chunk.name)
continue
}
if (!chunk.callId || typeof chunk.callId !== "string") {
Logger.warn("Cline <Language Model API>: Invalid tool callId received:", chunk.callId)
continue
}
// Ensure input is a valid object
if (!chunk.input || typeof chunk.input !== "object") {
Logger.warn("Cline <Language Model API>: Invalid tool input received:", chunk.input)
continue
}
// Convert tool calls to text format with proper error handling
const toolCall = {
type: "tool_call",
name: chunk.name,
arguments: chunk.input,
callId: chunk.callId,
}
const toolCallText = JSON.stringify(toolCall)
accumulatedText += toolCallText
// Log tool call for debugging
Logger.debug("Cline <Language Model API>: Processing tool call:", {
name: chunk.name,
callId: chunk.callId,
inputSize: JSON.stringify(chunk.input).length,
})
yield {
type: "text",
text: toolCallText,
}
} catch (error) {
Logger.error("Cline <Language Model API>: Failed to process tool call:", error)
}
} else {
Logger.warn("Cline <Language Model API>: Unknown chunk type received:", chunk)
}
}
// Count tokens in the accumulated text after stream completion
const totalOutputTokens: number = await this.countTokens(accumulatedText)
// Report final usage after stream completion
yield {
type: "usage",
inputTokens: totalInputTokens,
outputTokens: totalOutputTokens,
totalCost: calculateApiCostAnthropic(this.getModel().info, totalInputTokens, totalOutputTokens),
}
} catch (error: unknown) {
this.ensureCleanState()
if (error instanceof vscode.CancellationError) {
throw new Error("Cline <Language Model API>: Request cancelled by user")
}
if (error instanceof Error) {
Logger.error("Cline <Language Model API>: Stream error details:", {
message: error.message,
stack: error.stack,
name: error.name,
})
// Return original error if it's already an Error instance
throw error
} else if (typeof error === "object" && error !== null) {
// Handle error-like objects
const errorDetails = JSON.stringify(error, null, 2)
Logger.error("Cline <Language Model API>: Stream error object:", errorDetails)
throw new Error(`Cline <Language Model API>: Response stream error: ${errorDetails}`)
} else {
// Fallback for unknown error types
const errorMessage = String(error)
Logger.error("Cline <Language Model API>: Unknown stream error:", errorMessage)
throw new Error(`Cline <Language Model API>: Response stream error: ${errorMessage}`)
}
}
}
// Return model information based on the current client state
getModel(): { id: string; info: ModelInfo } {
if (this.client) {
// Validate client properties
const requiredProps = {
id: this.client.id,
vendor: this.client.vendor,
family: this.client.family,
version: this.client.version,
maxInputTokens: this.client.maxInputTokens,
}
// Log any missing properties for debugging
for (const [prop, value] of Object.entries(requiredProps)) {
if (!value && value !== 0) {
Logger.warn(`Cline <Language Model API>: Client missing ${prop} property`)
}
}
// Construct model ID using available information
const modelParts = [this.client.vendor, this.client.family, this.client.version].filter(Boolean)
const modelId = this.client.id || modelParts.join(SELECTOR_SEPARATOR)
// Build model info with conservative defaults for missing values
const modelInfo: ModelInfo = {
maxTokens: -1, // Unlimited tokens by default
contextWindow:
typeof this.client.maxInputTokens === "number"
? Math.max(0, this.client.maxInputTokens)
: openAiModelInfoSaneDefaults.contextWindow,
supportsImages: false, // VSCode Language Model API currently doesn't support image inputs
supportsPromptCache: true,
inputPrice: 0,
outputPrice: 0,
description: `VSCode Language Model: ${modelId}`,
}
return { id: modelId, info: modelInfo }
}
// Fallback when no client is available
const fallbackId = this.options.vsCodeLmModelSelector
? stringifyVsCodeLmModelSelector(this.options.vsCodeLmModelSelector)
: "vscode-lm"
Logger.debug("Cline <Language Model API>: No client available, using fallback model info")
return {
id: fallbackId,
info: {
...openAiModelInfoSaneDefaults,
description: `VSCode Language Model (Fallback): ${fallbackId}`,
},
}
}
async completePrompt(prompt: string): Promise<string> {
try {
const client = await this.getClient()
const response = await client.sendRequest(
[vscode.LanguageModelChatMessage.User(prompt)],
{},
new vscode.CancellationTokenSource().token,
)
let result = ""
for await (const chunk of response.stream) {
if (chunk instanceof vscode.LanguageModelTextPart) {
result += chunk.value
}
}
return result
} catch (error) {
if (error instanceof Error) {
throw new Error(`VSCode LM completion error: ${error.message}`)
}
throw error
}
}
}
-102
View File
@@ -1,102 +0,0 @@
import { openAiModelInfoSaneDefaults, type ModelInfo, type WandbModelId, wandbDefaultModelId, wandbModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface WandbHandlerOptions extends CommonApiHandlerOptions {
wandbApiKey?: string
apiModelId?: string
}
export class WandbHandler implements ApiHandler {
private client: OpenAI | undefined
constructor(private readonly options: WandbHandlerOptions) {}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.wandbApiKey) {
throw new Error("W&B API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.inference.wandb.ai/v1",
apiKey: this.options.wandbApiKey,
})
} catch (error) {
throw new Error(`Error creating W&B Inference client: ${error instanceof Error ? error.message : String(error)}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const stream = await client.chat.completions.create({
model: model.id,
messages: [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)],
temperature: 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning" in delta && delta.reasoning) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
// W&B Inference returns prompt_tokens_details.cached_tokens in the usage chunk,
// but does not currently offer cache-aware billing (cached tokens are billed
// at the same rate as regular input tokens). We report inputTokens as the full
// prompt_tokens value and do not subtract cached tokens until W&B supports
// cache-aware pricing. This may change in a future update.
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId?.trim()
if (modelId && modelId in wandbModels) {
return { id: modelId, info: wandbModels[modelId as WandbModelId] }
}
if (modelId) {
return { id: modelId, info: openAiModelInfoSaneDefaults }
}
return { id: wandbDefaultModelId, info: wandbModels[wandbDefaultModelId] }
}
}
-118
View File
@@ -1,118 +0,0 @@
import { ModelInfo, XAIModelId, xaiDefaultModelId, xaiModels } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ChatCompletionReasoningEffort } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface XAIHandlerOptions extends CommonApiHandlerOptions {
xaiApiKey?: string
reasoningEffort?: string
apiModelId?: string
}
export class XAIHandler implements ApiHandler {
private options: XAIHandlerOptions
private client: OpenAI | undefined
constructor(options: XAIHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.xaiApiKey) {
throw new Error("xAI API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.x.ai/v1",
apiKey: this.options.xaiApiKey,
})
} catch (error: any) {
throw new Error(`Error creating xAI client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.getModel().id
// ensure reasoning effort is either "low" or "high" for grok-3-mini
let reasoningEffort: ChatCompletionReasoningEffort | undefined
if (modelId.includes("3-mini")) {
let reasoningEffort = this.options.reasoningEffort
if (reasoningEffort && !["low", "high"].includes(reasoningEffort)) {
reasoningEffort = undefined
}
}
const stream = await client.chat.completions.create({
model: modelId,
max_completion_tokens: this.getModel().info.maxTokens,
temperature: 0,
messages: [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)],
stream: true,
stream_options: { include_usage: true },
reasoning_effort: reasoningEffort,
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
if (!shouldSkipReasoningForModel(modelId)) {
yield {
type: "reasoning",
// @ts-expect-error-next-line
reasoning: delta.reasoning_content,
}
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: XAIModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in xaiModels) {
const id = modelId as XAIModelId
return { id, info: xaiModels[id] }
}
return {
id: xaiDefaultModelId,
info: xaiModels[xaiDefaultModelId],
}
}
}
-120
View File
@@ -1,120 +0,0 @@
import {
internationalZAiDefaultModelId,
internationalZAiModelId,
internationalZAiModels,
ModelInfo,
mainlandZAiDefaultModelId,
mainlandZAiModelId,
mainlandZAiModels,
} from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { version as extensionVersion } from "../../../../package.json"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface ZAiHandlerOptions extends CommonApiHandlerOptions {
zaiApiLine?: string
zaiApiKey?: string
apiModelId?: string
}
export class ZAiHandler implements ApiHandler {
private options: ZAiHandlerOptions
private client: OpenAI | undefined
constructor(options: ZAiHandlerOptions) {
this.options = options
}
private useChinaApi(): boolean {
return this.options.zaiApiLine === "china"
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.zaiApiKey) {
throw new Error("Z AI API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: this.useChinaApi() ? "https://open.bigmodel.cn/api/paas/v4" : "https://api.z.ai/api/paas/v4",
apiKey: this.options.zaiApiKey,
defaultHeaders: {
"HTTP-Referer": "https://cline.bot",
"X-Title": "Cline",
"X-Cline-Version": extensionVersion,
},
})
} catch (error: any) {
throw new Error(`Error creating Z AI client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: mainlandZAiModelId | internationalZAiModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (this.useChinaApi()) {
const id = modelId && modelId in mainlandZAiModels ? (modelId as mainlandZAiModelId) : mainlandZAiDefaultModelId
return {
id,
info: mainlandZAiModels[id],
}
}
const id =
modelId && modelId in internationalZAiModels ? (modelId as internationalZAiModelId) : internationalZAiDefaultModelId
return {
id,
info: internationalZAiModels[id],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
cacheWriteTokens: 0,
}
}
}
}
}
-237
View File
@@ -1,237 +0,0 @@
import { describe, it } from "mocha"
import "should"
import sinon from "sinon"
import { withRetry } from "./retry"
describe("Retry Decorator", () => {
afterEach(() => {
sinon.restore()
})
describe("withRetry", () => {
it("should not retry on success", async () => {
let callCount = 0
class TestClass {
@withRetry()
async *successMethod() {
callCount++
yield "success"
}
}
const test = new TestClass()
const result = []
for await (const value of test.successMethod()) {
result.push(value)
}
callCount.should.equal(1)
result.should.deepEqual(["success"])
})
it("should retry on rate limit (429) error", async () => {
let callCount = 0
class TestClass {
@withRetry({ maxRetries: 2, baseDelay: 10, maxDelay: 100 })
async *failMethod() {
callCount++
if (callCount === 1) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
throw error
}
yield "success after retry"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(2)
result.should.deepEqual(["success after retry"])
})
it("should not retry on non-rate-limit errors", async () => {
let callCount = 0
class TestClass {
@withRetry()
async *failMethod() {
callCount++
throw new Error("Regular error")
}
}
const test = new TestClass()
try {
for await (const _ of test.failMethod()) {
// Should not reach here
}
throw new Error("Should have thrown")
} catch (error: any) {
error.message.should.equal("Regular error")
callCount.should.equal(1)
}
})
it("should respect retry-after header with delta seconds", async () => {
let callCount = 0
const setTimeoutSpy = sinon.spy(global, "setTimeout")
const baseDelay = 1000
class TestClass {
@withRetry({ maxRetries: 2, baseDelay }) // Use large baseDelay to ensure header takes precedence
async *failMethod() {
callCount++
if (callCount === 1) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
error.headers = { "retry-after": "0.01" } // 10ms delay
throw error
}
yield "success after retry"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(2)
setTimeoutSpy.calledOnce.should.be.true
const [_, delay] = setTimeoutSpy.getCall(0).args
delay?.should.equal(0)
result.should.deepEqual(["success after retry"])
})
it("should respect retry-after header with Unix timestamp", async () => {
const setTimeoutSpy = sinon.spy(global, "setTimeout")
let callCount = 0
const fixedDate = new Date("2010-01-01T00:00:00.000Z")
const retryTimestamp = Math.floor(fixedDate.getTime() / 1000) + 0.01 // 10ms in the future
const baseDelay = 1000
class TestClass {
@withRetry({ maxRetries: 2, baseDelay }) // Use large baseDelay to ensure header takes precedence
async *failMethod() {
callCount++
if (callCount === 1) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
error.headers = { "retry-after": retryTimestamp.toString() }
throw error
}
yield "success after retry"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(2)
setTimeoutSpy.calledOnce.should.be.true
const [_, delay] = setTimeoutSpy.getCall(0).args
delay?.should.equal(fixedDate.getTime())
result.should.deepEqual(["success after retry"])
})
it("should use exponential backoff when no retry-after header", async () => {
const setTimeoutSpy = sinon.spy(global, "setTimeout")
let callCount = 0
const baseDelay = 10
class TestClass {
@withRetry({ maxRetries: 2, baseDelay, maxDelay: 100 })
async *failMethod() {
callCount++
if (callCount === 1) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
throw error
}
yield "success after retry"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(2)
setTimeoutSpy.calledOnce.should.be.true
const [_, delay] = setTimeoutSpy.getCall(0).args
delay?.should.equal(baseDelay)
result.should.deepEqual(["success after retry"])
})
it("should respect maxDelay", async () => {
const setTimeoutSpy = sinon.spy(global, "setTimeout")
let callCount = 0
const baseDelay = 50
const maxDelay = 10
class TestClass {
@withRetry({ maxRetries: 3, baseDelay, maxDelay })
async *failMethod() {
callCount++
if (callCount < 3) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
throw error
}
yield "success after retries"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(3)
setTimeoutSpy.calledOnce.should.be.true
const [_, delay] = setTimeoutSpy.getCall(0).args
delay?.should.equal(maxDelay)
result.should.deepEqual(["success after retries"])
})
it("should throw after maxRetries attempts", async () => {
let callCount = 0
class TestClass {
@withRetry({ maxRetries: 2, baseDelay: 10 })
async *failMethod() {
callCount++
const error: any = new Error("Rate limit exceeded")
error.status = 429
throw error
}
}
const test = new TestClass()
try {
for await (const _ of test.failMethod()) {
// Should not reach here
}
throw new Error("Should have thrown")
} catch (error: any) {
error.message.should.equal("Rate limit exceeded")
callCount.should.equal(2) // Initial attempt + 1 retry
}
})
})
})
-88
View File
@@ -1,88 +0,0 @@
import { Logger } from "@/shared/services/Logger"
interface RetryOptions {
maxRetries?: number
baseDelay?: number
maxDelay?: number
retryAllErrors?: boolean
}
const DEFAULT_OPTIONS: Required<RetryOptions> = {
maxRetries: 3,
baseDelay: 1_000,
maxDelay: 10_000,
retryAllErrors: false,
}
export class RetriableError extends Error {
status: number = 429
retryAfter?: number
constructor(message: string, retryAfter?: number, options?: ErrorOptions) {
super(message, options)
this.name = "RetriableError"
this.retryAfter = retryAfter
}
}
export function withRetry(options: RetryOptions = {}) {
const { maxRetries, baseDelay, maxDelay, retryAllErrors } = { ...DEFAULT_OPTIONS, ...options }
return (_target: any, _propertyKey: string, descriptor: PropertyDescriptor) => {
const originalMethod = descriptor.value
descriptor.value = async function* (...args: any[]) {
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
yield* originalMethod.apply(this, args)
return
} catch (error: any) {
const isRateLimit = error?.status === 429 || error instanceof RetriableError
const isLastAttempt = attempt === maxRetries - 1
if ((!isRateLimit && !retryAllErrors) || isLastAttempt) {
throw error
}
// Get retry delay from header or calculate exponential backoff
// Check various rate limit headers
const retryAfter =
error.headers?.["retry-after"] ||
error.headers?.["x-ratelimit-reset"] ||
error.headers?.["ratelimit-reset"] ||
error.retryAfter
let delay: number
if (retryAfter) {
// Handle both delta-seconds and Unix timestamp formats
const retryValue = parseInt(retryAfter, 10)
if (retryValue > Date.now() / 1000) {
// Unix timestamp
delay = retryValue * 1000 - Date.now()
} else {
// Delta seconds
delay = retryValue * 1000
}
} else {
// Use exponential backoff if no header
delay = Math.min(maxDelay, baseDelay * 2 ** attempt)
}
const handlerInstance = this as any
if (handlerInstance.options?.onRetryAttempt) {
try {
await handlerInstance.options.onRetryAttempt(attempt + 1, maxRetries, delay, error)
} catch (e) {
Logger.error("Error in onRetryAttempt callback:", e)
}
}
await new Promise((resolve) => setTimeout(resolve, delay))
}
}
}
return descriptor
}
}
@@ -1,117 +0,0 @@
import { describe, it } from "mocha"
import "should"
import type { ModelInfo } from "@shared/api"
import sinon from "sinon"
import { createOpenRouterStream } from "../openrouter-stream"
describe("createOpenRouterStream", () => {
const createAsyncIterable = () => ({
async *[Symbol.asyncIterator]() {},
})
const createClient = () => {
const create = sinon.stub().resolves(createAsyncIterable())
return {
client: {
chat: {
completions: {
create,
},
},
},
create,
}
}
const createModelInfo = (maxTokens: number): ModelInfo => ({
maxTokens,
contextWindow: 1_048_576,
supportsImages: true,
supportsPromptCache: false,
})
it("caps Gemini Flash OpenRouter requests to 8192 max_tokens", async () => {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: "google/gemini-2.5-flash",
info: createModelInfo(65_536),
})
const payload = create.firstCall.args[0] as Record<string, any>
payload.should.have.property("max_tokens", 8_192)
})
it("keeps lower Gemini Flash max_tokens values when already below 8192", async () => {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: "google/gemini-2.5-flash",
info: createModelInfo(4_096),
})
const payload = create.firstCall.args[0] as Record<string, any>
payload.should.have.property("max_tokens", 4_096)
})
it("does not send max_tokens for non-Gemini models", async () => {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: "anthropic/claude-sonnet-4.5",
info: createModelInfo(64_000),
})
const payload = create.firstCall.args[0] as any
payload.should.not.have.property("max_tokens")
})
it("does not send max_tokens for non-Flash Gemini models", async () => {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: "google/gemini-2.5-pro",
info: createModelInfo(65_536),
})
const payload = create.firstCall.args[0] as any
payload.should.not.have.property("max_tokens")
})
it("adds cache_control blocks for Qwen models that require explicit OpenRouter caching", async () => {
for (const modelId of ["qwen/qwen3.6-plus", "qwen/qwen3.7-max"]) {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: modelId,
info: createModelInfo(65_536),
})
const payload = create.firstCall.args[0] as any
payload.messages[0].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
payload.messages[1].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
}
})
it("uses adaptive reasoning with verbosity for Claude Opus adaptive models", async () => {
const { client, create } = createClient()
await createOpenRouterStream(
client as any,
"system prompt",
[{ role: "user", content: "hello" }] as any,
{
id: "anthropic/claude-opus-4.6",
info: createModelInfo(64_000),
},
"xhigh",
)
const payload = create.firstCall.args[0] as any
payload.should.have.property("reasoning")
payload.reasoning.should.deepEqual({ enabled: true })
payload.should.have.property("verbosity", "xhigh")
should(payload.temperature).equal(undefined)
should(payload.top_p).equal(undefined)
})
})
@@ -1,283 +0,0 @@
/**
* Contract tests for thinking trace preservation across provider transforms.
*
* These tests verify that thinking/reasoning content is correctly preserved
* when converting messages between different API formats. This is critical
* because losing thinking traces can cause:
* - Degraded model performance (models need context of their reasoning)
* - Provider API errors (e.g., Gemini requires reasoning_details for tool calls)
* - Incorrect cost calculations
*/
import { describe, it } from "mocha"
import "should"
import { ClineAssistantThinkingBlock, ClineStorageMessage, ClineTextContentBlock } from "@/shared/messages/content"
import { sanitizeAnthropicMessages } from "../anthropic-format"
import { convertToOpenAiMessages, sanitizeGeminiMessages } from "../openai-format"
describe("Thinking Trace Preservation", () => {
describe("convertToOpenAiMessages", () => {
it("should preserve reasoning_details on text blocks", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "text",
text: "I'll help you with that.",
reasoning_details: [
{
type: "reasoning.text",
text: "The user wants help with X...",
signature: "sig123",
format: "anthropic-claude-v1",
index: 0,
},
],
} as ClineTextContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(1)
const assistantMsg = result[0] as any
assistantMsg.role.should.equal("assistant")
assistantMsg.reasoning_details.should.be.an.Array()
assistantMsg.reasoning_details.should.have.length(1)
assistantMsg.reasoning_details[0].text.should.equal("The user wants help with X...")
})
it("should preserve thinking blocks with signatures", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "thinking",
thinking: "Let me analyze this problem...",
signature: "valid-signature",
} as ClineAssistantThinkingBlock,
{
type: "text",
text: "Here's my answer.",
} as ClineTextContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(1)
const assistantMsg = result[0] as any
// The thinking block content should be preserved in some form
// (exact handling depends on implementation)
assistantMsg.content.should.containEql("Here's my answer.")
})
it("should consolidate multiple reasoning_details entries", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "text",
text: "Result",
reasoning_details: [
{
type: "reasoning.text",
text: "First thought. ",
signature: "sig1",
format: "anthropic-claude-v1",
index: 0,
},
{
type: "reasoning.text",
text: "Second thought.",
signature: "sig2",
format: "anthropic-claude-v1",
index: 0,
},
],
} as ClineTextContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
const assistantMsg = result[0] as any
assistantMsg.reasoning_details.should.be.an.Array()
// Should be consolidated into one entry per index
assistantMsg.reasoning_details.should.have.length(1)
assistantMsg.reasoning_details[0].text.should.equal("First thought. Second thought.")
})
it("should filter out corrupted encrypted reasoning blocks", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "text",
text: "Answer",
reasoning_details: [
{
type: "reasoning.encrypted",
// Missing 'data' field - corrupted
signature: "sig",
format: "anthropic-claude-v1",
index: 0,
} as any,
{
type: "reasoning.text",
text: "Valid reasoning",
signature: "sig2",
format: "anthropic-claude-v1",
index: 1,
},
],
} as ClineTextContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
const assistantMsg = result[0] as any
// Should only have the valid reasoning entry
assistantMsg.reasoning_details.should.have.length(1)
assistantMsg.reasoning_details[0].type.should.equal("reasoning.text")
})
})
describe("sanitizeGeminiMessages", () => {
it("should drop tool_calls without reasoning_details for Gemini models", () => {
const messages: any[] = [
{
role: "assistant",
content: "I'll use a tool",
tool_calls: [{ id: "call_123", type: "function", function: { name: "read_file", arguments: "{}" } }],
// No reasoning_details
},
{
role: "tool",
tool_call_id: "call_123",
content: "file contents",
},
]
const result = sanitizeGeminiMessages(messages, "gemini-2.5-pro")
// Tool call should be dropped, but content preserved
result.should.have.length(1)
const msg = result[0] as any
msg.role.should.equal("assistant")
msg.content.should.equal("I'll use a tool")
;(msg.tool_calls === undefined).should.be.true()
})
it("should preserve tool_calls with reasoning_details for Gemini models", () => {
const messages: any[] = [
{
role: "assistant",
content: null,
tool_calls: [{ id: "call_123", type: "function", function: { name: "read_file", arguments: "{}" } }],
reasoning_details: [
{
type: "reasoning.text",
text: "I need to read the file",
format: "anthropic-claude-v1",
index: 0,
},
],
},
{
role: "tool",
tool_call_id: "call_123",
content: "file contents",
},
]
const result = sanitizeGeminiMessages(messages, "gemini-2.5-pro")
result.should.have.length(2)
;(result[0] as any).tool_calls.should.have.length(1)
})
it("should not modify messages for non-Gemini models", () => {
const messages: any[] = [
{
role: "assistant",
content: "Using tool",
tool_calls: [{ id: "call_123", type: "function", function: { name: "test", arguments: "{}" } }],
// No reasoning_details - would be dropped for Gemini
},
]
const result = sanitizeGeminiMessages(messages, "gpt-4o")
result.should.have.length(1)
;(result[0] as any).tool_calls.should.have.length(1)
})
})
describe("sanitizeAnthropicMessages", () => {
it("should preserve thinking blocks", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "thinking",
thinking: "Let me think about this...",
signature: "valid-sig",
} as ClineAssistantThinkingBlock,
{
type: "text",
text: "Here's my answer",
} as ClineTextContentBlock,
],
},
]
const result = sanitizeAnthropicMessages(messages, false)
result.should.have.length(1)
const content = result[0].content as any[]
// Find thinking block
const thinkingBlock = content.find((b) => b.type === "thinking")
thinkingBlock.should.not.be.undefined
thinkingBlock.thinking.should.equal("Let me think about this...")
})
it("should not add cache_control to thinking blocks", () => {
const messages: ClineStorageMessage[] = [
{
role: "user",
content: [
{
type: "thinking",
thinking: "Thinking...",
signature: "sig",
} as any,
{
type: "text",
text: "Question",
} as ClineTextContentBlock,
],
},
]
const result = sanitizeAnthropicMessages(messages, true)
result.should.have.length(1)
const content = result[0].content as any[]
// The text block (last non-thinking) should have cache_control
const textBlock = content.find((b) => b.type === "text")
textBlock.cache_control.should.deepEqual({ type: "ephemeral" })
// Thinking block should not have cache_control (it doesn't support it)
})
})
})
@@ -1,116 +0,0 @@
import "should"
import { getOpenAIToolParams, ToolCallProcessor } from "../tool-call-processor"
describe("ToolCallProcessor", () => {
it("should preserve tool call id/name for interleaved parallel deltas", () => {
const processor = new ToolCallProcessor()
const firstChunk = [
{
index: 0,
id: "call_a",
function: { name: "read_file" },
},
{
index: 1,
id: "call_b",
function: { name: "search_files" },
},
] as any
const secondChunk = [
{
index: 1,
function: { arguments: '{"path":"src"}' },
},
{
index: 0,
function: { arguments: '{"path":"README.md"}' },
},
] as any
const firstResult = [...processor.processToolCallDeltas(firstChunk)]
const secondResult = [...processor.processToolCallDeltas(secondChunk)]
firstResult.should.have.length(0)
secondResult.should.have.length(2)
// Intentionally reversed from the setup chunk: output follows incoming
// argument-delta order, but reconstruction is correct regardless of arrival
// order because id/name/arguments are matched by tool call index.
const firstToolCall = secondResult[0]!.tool_call as any
const secondToolCall = secondResult[1]!.tool_call as any
firstToolCall.function.id.should.equal("call_b")
firstToolCall.function.name.should.equal("search_files")
firstToolCall.function.arguments.should.equal('{"path":"src"}')
secondToolCall.function.id.should.equal("call_a")
secondToolCall.function.name.should.equal("read_file")
secondToolCall.function.arguments.should.equal('{"path":"README.md"}')
})
it("should clear accumulated state on reset", () => {
const processor = new ToolCallProcessor()
const setupChunk = [
{
index: 0,
id: "call_reset",
function: { name: "read_file" },
},
] as any
const argsChunk = [
{
index: 0,
function: { arguments: '{"path":"after-reset"}' },
},
] as any
;[...processor.processToolCallDeltas(setupChunk)].should.have.length(0)
processor.reset()
;[...processor.processToolCallDeltas(argsChunk)].should.have.length(0)
const newSetupChunk = [
{
index: 0,
id: "call_new",
function: { name: "write_file" },
},
] as any
const newArgsChunk = [
{
index: 0,
function: { arguments: '{"path":"file.txt"}' },
},
] as any
;[...processor.processToolCallDeltas(newSetupChunk)].should.have.length(0)
;[...processor.processToolCallDeltas(newArgsChunk)].should.have.length(1)
})
})
describe("getOpenAIToolParams", () => {
it("should include parallel_tool_calls when enabled", () => {
const tools = [
{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } },
] as any
const params = getOpenAIToolParams(tools, true) as any
params.parallel_tool_calls.should.equal(true)
})
it("should include parallel_tool_calls=false when disabled by default", () => {
const tools = [
{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } },
] as any
const params = getOpenAIToolParams(tools, false) as any
params.parallel_tool_calls.should.equal(false)
})
it("should not include parallel_tool_calls when tools are absent", () => {
const params = getOpenAIToolParams(undefined, false) as any
params.should.not.have.property("parallel_tool_calls")
})
})
@@ -1,371 +0,0 @@
/**
* Contract tests for tool call parsing and transformation.
*
* These tests verify that tool calls are correctly parsed and transformed
* between different API formats (Anthropic, OpenAI, etc.). This is critical
* because incorrect tool parsing can cause:
* - Tool calls not being executed
* - Mismatched tool_call_id causing API errors
* - Lost tool results breaking conversation flow
*/
import { describe, it } from "mocha"
import "should"
import OpenAI from "openai"
import {
ClineAssistantToolUseBlock,
ClineStorageMessage,
ClineTextContentBlock,
ClineUserToolResultContentBlock,
} from "@/shared/messages/content"
import { convertToAnthropicMessage, convertToOpenAiMessages } from "../openai-format"
describe("Tool Call Parsing", () => {
describe("convertToOpenAiMessages - Tool Calls", () => {
it("should convert Anthropic tool_use to OpenAI tool_calls format", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: "toolu_abc123",
name: "read_file",
input: { path: "/test/file.ts" },
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages, "openai-native")
result.should.have.length(1)
const msg = result[0] as any
msg.role.should.equal("assistant")
msg.tool_calls.should.have.length(1)
msg.tool_calls[0].type.should.equal("function")
msg.tool_calls[0].function.name.should.equal("read_file")
JSON.parse(msg.tool_calls[0].function.arguments).should.deepEqual({ path: "/test/file.ts" })
})
it("should truncate long tool IDs to 40 characters", () => {
const longId = "a".repeat(50)
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: longId,
name: "test_tool",
input: {},
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages, "openai-native")
const msg = result[0] as any
msg.tool_calls[0].id.length.should.be.belowOrEqual(40)
})
it("should transform OpenAI Responses API tool IDs (fc_ prefix)", () => {
// OpenAI Responses API uses fc_ prefix with 53 char length
const responsesApiId = "fc_" + "x".repeat(50)
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: responsesApiId,
name: "test_tool",
input: {},
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
const msg = result[0] as any
// Should be transformed to call_ prefix format
msg.tool_calls[0].id.should.startWith("call_")
msg.tool_calls[0].id.length.should.be.belowOrEqual(40)
})
it("should match tool_call_id with tool_calls id for tool results", () => {
const toolId = "toolu_abc123"
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: toolId,
name: "read_file",
input: { path: "/test.ts" },
} as ClineAssistantToolUseBlock,
],
},
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: toolId,
content: "file contents here",
} as ClineUserToolResultContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(2)
// Get the transformed tool_call id from assistant message
const assistantMsg = result[0] as OpenAI.Chat.ChatCompletionAssistantMessageParam
const transformedId = assistantMsg.tool_calls![0].id
// The tool result should have the same transformed id
const toolMsg = result[1] as OpenAI.Chat.ChatCompletionToolMessageParam
toolMsg.tool_call_id.should.equal(transformedId)
})
it("should handle multiple tool calls in a single message", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "text",
text: "I'll read both files",
} as ClineTextContentBlock,
{
type: "tool_use",
id: "tool_1",
name: "read_file",
input: { path: "/file1.ts" },
} as ClineAssistantToolUseBlock,
{
type: "tool_use",
id: "tool_2",
name: "read_file",
input: { path: "/file2.ts" },
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(1)
const msg = result[0] as any
msg.tool_calls.should.have.length(2)
msg.tool_calls[0].function.name.should.equal("read_file")
msg.tool_calls[1].function.name.should.equal("read_file")
})
it("should handle tool results with array content", () => {
const messages: ClineStorageMessage[] = [
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: "tool_123",
content: [
{ type: "text", text: "Line 1" },
{ type: "text", text: "Line 2" },
],
} as ClineUserToolResultContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(1)
const msg = result[0] as OpenAI.Chat.ChatCompletionToolMessageParam
msg.role.should.equal("tool")
msg.content.should.equal("Line 1\nLine 2")
})
it("should set content to null when only tool_calls present", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: "tool_1",
name: "test",
input: {},
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
const msg = result[0] as any
// Content should be null, not undefined or empty string
;(msg.content === null).should.be.true()
})
})
describe("convertToAnthropicMessage - OpenAI Response to Anthropic", () => {
it("should convert OpenAI completion to Anthropic message format", () => {
const completion: OpenAI.Chat.Completions.ChatCompletion = {
id: "chatcmpl-123",
object: "chat.completion",
created: Date.now(),
model: "gpt-4o",
choices: [
{
index: 0,
message: {
role: "assistant",
content: "Hello!",
refusal: null,
},
finish_reason: "stop",
logprobs: null,
},
],
usage: {
prompt_tokens: 10,
completion_tokens: 5,
total_tokens: 15,
},
}
const result = convertToAnthropicMessage(completion)
result.id.should.equal("chatcmpl-123")
result.role.should.equal("assistant")
result.model.should.equal("gpt-4o")
result.stop_reason!.should.equal("end_turn")
result.usage.input_tokens.should.equal(10)
result.usage.output_tokens.should.equal(5)
const content = result.content as any[]
content[0].type.should.equal("text")
content[0].text.should.equal("Hello!")
})
it("should convert OpenAI tool_calls to Anthropic tool_use blocks", () => {
const completion: OpenAI.Chat.Completions.ChatCompletion = {
id: "chatcmpl-456",
object: "chat.completion",
created: Date.now(),
model: "gpt-4o",
choices: [
{
index: 0,
message: {
role: "assistant",
content: null,
tool_calls: [
{
id: "call_abc",
type: "function",
function: {
name: "read_file",
arguments: '{"path":"/test.ts"}',
},
},
],
refusal: null,
},
finish_reason: "tool_calls",
logprobs: null,
},
],
}
const result = convertToAnthropicMessage(completion)
result.stop_reason!.should.equal("tool_use")
const content = result.content as any[]
content.should.have.length(2) // text block + tool_use block
const toolUse = content.find((b) => b.type === "tool_use")
toolUse.should.not.be.undefined
toolUse.id.should.equal("call_abc")
toolUse.name.should.equal("read_file")
toolUse.input.should.deepEqual({ path: "/test.ts" })
})
it("should handle malformed tool arguments gracefully", () => {
const completion: OpenAI.Chat.Completions.ChatCompletion = {
id: "chatcmpl-789",
object: "chat.completion",
created: Date.now(),
model: "gpt-4o",
choices: [
{
index: 0,
message: {
role: "assistant",
content: null,
tool_calls: [
{
id: "call_bad",
type: "function",
function: {
name: "test_tool",
arguments: "not valid json",
},
},
],
refusal: null,
},
finish_reason: "tool_calls",
logprobs: null,
},
],
}
// Should not throw, should return empty input
const result = convertToAnthropicMessage(completion)
const content = result.content as any[]
const toolUse = content.find((b) => b.type === "tool_use")
toolUse.input.should.deepEqual({})
})
it("should map finish_reason correctly", () => {
const testCases: Array<{ finish_reason: any; expected: string | null }> = [
{ finish_reason: "stop", expected: "end_turn" },
{ finish_reason: "length", expected: "max_tokens" },
{ finish_reason: "tool_calls", expected: "tool_use" },
{ finish_reason: "content_filter", expected: null },
]
for (const { finish_reason, expected } of testCases) {
const completion: OpenAI.Chat.Completions.ChatCompletion = {
id: "test",
object: "chat.completion",
created: Date.now(),
model: "test",
choices: [
{
index: 0,
message: { role: "assistant", content: "test", refusal: null },
finish_reason,
logprobs: null,
},
],
}
const result = convertToAnthropicMessage(completion)
// Using equality check since should.be.true() doesn't accept message arg
;(result.stop_reason === expected).should.be.true()
}
})
})
})
@@ -1,90 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { ClineStorageMessage, convertClineStorageToAnthropicMessage } from "@/shared/messages/content"
/**
* Converts Cline storage messages to Anthropic API format with optional cache control.
* Adds ephemeral cache control to the last two user messages to prevent them from being
* stored in Anthropic's cache.
*
* @param clineMessages - Array of Cline storage messages to convert
* @param lastUserMsgIndex - Optional index of the last user message
* @param secondLastMsgUserIndex - Optional index of the second-to-last user message
* @returns Array of Anthropic-compatible messages with cache control applied
*/
export function sanitizeAnthropicMessages(
clineMessages: Array<ClineStorageMessage | Anthropic.MessageParam>,
supportCache: boolean,
): Array<Anthropic.MessageParam> {
// The latest message will be the new user message, one before will be the assistant message from a previous request,
// and the user message before that will be a previously cached user message. So we need to mark the latest user message
// as ephemeral to cache it for the next request, and mark the second to last user message as ephemeral to let the server
// know the last message to retrieve from the cache for the current request.
const userMsgIndices = clineMessages.reduce((acc, msg, index) => {
if (msg.role === "user") {
acc.push(index)
}
return acc
}, [] as number[])
// Set to -1 if there are no user messages so the indices are invalid
const indicesLength = userMsgIndices.length ?? -1
const lastUserMsgIndex = userMsgIndices[indicesLength - 1]
const secondLastMsgUserIndex = userMsgIndices[indicesLength - 2]
return clineMessages.map((msg, index) => {
const anthropicMsg = convertClineStorageToAnthropicMessage(msg)
// Add cache control to the last two user messages
if (supportCache && (index === lastUserMsgIndex || index === secondLastMsgUserIndex)) {
return addCacheControl(anthropicMsg)
}
return anthropicMsg
})
}
const isThinkingBlock = (
block: Anthropic.ContentBlockParam,
): block is Anthropic.Messages.ThinkingBlockParam | Anthropic.Messages.RedactedThinkingBlockParam => {
return block.type === "thinking" || block.type === "redacted_thinking"
}
/**
* Adds ephemeral cache control to the last content block of a message.
* Returns a new message object without mutating the original.
*
* @param message - The Anthropic message to add cache control to
* @returns A new message with cache control added to the last content block
*/
function addCacheControl(message: Anthropic.MessageParam): Anthropic.MessageParam {
// Convert string content to array format
if (typeof message.content === "string") {
return {
...message,
content: [
{
type: "text",
text: message.content,
cache_control: { type: "ephemeral" },
} satisfies Anthropic.TextBlockParam,
],
}
}
// Handle array content - add cache control to the last block
const content = [...message.content]
const lastIndex = content.length - 1
if (lastIndex >= 0) {
const lastBlock = content[lastIndex]
// Only add cache_control to block types that support it (not ThinkingBlockParam)
if (!isThinkingBlock(lastBlock)) {
content[lastIndex] = {
...lastBlock,
cache_control: { type: "ephemeral" },
} satisfies Anthropic.ContentBlockParam
}
}
return { ...message, content }
}
@@ -1,118 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { Content, GenerateContentResponse, Part } from "@google/genai"
import { ClineStorageMessage } from "@/shared/messages/content"
// Source: https://ai.google.dev/gemini-api/docs/thought-signatures#faqs
// While injecting custom function call blocks into the request is strongly discouraged,
// in cases where it can't be avoided, e.g. providing information to the model on function
// calls and responses that were executed deterministically by the client, or transferring a
// trace from a different model that does not include thought signatures, you can set the following dummy signatures of either
// "context_engineering_is_the_way_to_go" or "skip_thought_signature_validator" in the thought signature field to skip validation.
const GEMINI_DUMMY_THOUGHT_SIGNATURE = "skip_thought_signature_validator"
export function convertAnthropicContentToGemini(content: string | ClineStorageMessage["content"]): Part[] {
if (typeof content === "string") {
return [{ text: content }]
}
return content
.flatMap((block): Part | undefined => {
switch (block.type) {
case "text":
return { text: block.text, thoughtSignature: block.signature }
case "image":
if (block.source.type !== "base64") {
throw new Error("Unsupported image source type")
}
return {
inlineData: {
data: block.source.data,
mimeType: block.source.media_type,
},
}
case "tool_use":
return {
functionCall: {
name: block.name,
args: block.input as Record<string, unknown>,
},
// Thought signature is required, so provide a dummy one if not present
thoughtSignature: block.signature || GEMINI_DUMMY_THOUGHT_SIGNATURE,
}
case "tool_result":
return {
functionResponse: {
name: block.tool_use_id,
response: {
result: block.content,
},
},
}
case "thinking":
return {
text: block.thinking,
thought: true,
thoughtSignature: block.signature || GEMINI_DUMMY_THOUGHT_SIGNATURE,
}
default:
return undefined
}
})
.filter((part): part is Part => part !== undefined) // Filter out unsupported blocks
}
export function convertAnthropicMessageToGemini(message: Anthropic.Messages.MessageParam): Content {
return {
role: message.role === "assistant" ? "model" : "user",
parts: convertAnthropicContentToGemini(message.content),
}
}
/*
It looks like gemini likes to double escape certain characters when writing file contents: https://discuss.ai.google.dev/t/function-call-string-property-is-double-escaped/37867
*/
export function unescapeGeminiContent(content: string) {
return content.replace(/\\n/g, "\n").replace(/\\'/g, "'").replace(/\\"/g, '"').replace(/\\r/g, "\r").replace(/\\t/g, "\t")
}
export function convertGeminiResponseToAnthropic(response: GenerateContentResponse): Anthropic.Messages.Message {
const content: Anthropic.Messages.ContentBlock[] = []
const text = response.text
if (text) {
content.push({ type: "text", text, citations: null })
}
let stop_reason: Anthropic.Messages.Message["stop_reason"] = null
const finishReason = response.candidates?.[0]?.finishReason
if (finishReason) {
switch (finishReason) {
case "STOP":
stop_reason = "end_turn"
break
case "MAX_TOKENS":
stop_reason = "max_tokens"
break
case "SAFETY":
case "RECITATION":
case "OTHER":
stop_reason = "stop_sequence"
break
}
}
return {
id: `msg_${Date.now()}`,
type: "message",
role: "assistant",
content,
model: "",
stop_reason,
stop_sequence: null, // Gemini doesn't provide this information
usage: {
input_tokens: response.usageMetadata?.promptTokenCount ?? 0,
output_tokens: response.usageMetadata?.candidatesTokenCount ?? 0,
cache_creation_input_tokens: null,
cache_read_input_tokens: null,
},
}
}
@@ -1,61 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { AssistantMessage } from "@mistralai/mistralai/models/components/assistantmessage"
import { SystemMessage } from "@mistralai/mistralai/models/components/systemmessage"
import { ToolMessage } from "@mistralai/mistralai/models/components/toolmessage"
import { UserMessage } from "@mistralai/mistralai/models/components/usermessage"
export type MistralMessage =
| (SystemMessage & { role: "system" })
| (UserMessage & { role: "user" })
| (AssistantMessage & { role: "assistant" })
| (ToolMessage & { role: "tool" })
export function convertToMistralMessages(anthropicMessages: Anthropic.Messages.MessageParam[]): MistralMessage[] {
const mistralMessages: MistralMessage[] = []
for (const anthropicMessage of anthropicMessages) {
if (typeof anthropicMessage.content === "string") {
mistralMessages.push({
role: anthropicMessage.role,
content: anthropicMessage.content,
})
} else {
if (anthropicMessage.role === "user") {
// Filter to only include text and image blocks
const textAndImageBlocks = anthropicMessage.content.filter(
(part) => part.type === "text" || part.type === "image",
)
if (textAndImageBlocks.length > 0) {
mistralMessages.push({
role: "user",
content: textAndImageBlocks.map((part) => {
if (part.type === "image") {
return {
type: "image_url",
imageUrl: {
url: `data:${part.source.media_type};base64,${part.source.data}`,
},
}
}
return { type: "text", text: part.text }
}),
})
}
} else if (anthropicMessage.role === "assistant") {
// Only process text blocks - assistant cannot send images or other content types in Mistral's API format
const textBlocks = anthropicMessage.content.filter((part) => part.type === "text")
if (textBlocks.length > 0) {
const content = textBlocks.map((part) => part.text).join("\n")
mistralMessages.push({
role: "assistant",
content,
})
}
}
}
}
return mistralMessages
}

Some files were not shown because too many files have changed in this diff Show More