Compare commits

...
Author SHA1 Message Date
Arafatkatze a6cd094460 fix: show running state for auto-approved commands 2026-06-03 15:00:59 -07:00
Dominic Cooney 39f5e564f6 fix(vscode): declare missing direct deps @grpc/proto-loader and @opentelemetry/api-logs
Both packages are imported directly from source but were never declared in
apps/vscode/package.json, so they only resolved transitively. On a clean
install this broke:

- @grpc/proto-loader — imported by scripts/proto-utils.mjs,
  src/standalone/utils.ts and src/standalone/hostbridge-client.ts; its absence
  made `npm run protos` (and therefore the whole build) fail on a fresh checkout.
- @opentelemetry/api-logs — imported by the OpenTelemetry telemetry providers;
  its absence produced TS2307 "Cannot find module" errors under tsc.

Versions are pinned to align with the existing dependency families already
declared in this package (@grpc/grpc-js ^1.9.x → proto-loader ^0.7.13;
the @opentelemetry/* 0.56.x line → api-logs ^0.56.0). The npm and bun
lockfiles are updated accordingly (the api-logs change also dedupes several
previously-nested copies to a single hoisted entry).
2026-06-03 10:43:52 -07:00
Max Paulus 🥪 3f2fe65c19 show legacy task history that is not saved in the ~/.cline folder 2026-06-03 10:13:30 -07:00
Max Paulus 🥪 ede87d82f7 add migration telemetry 2026-06-03 10:12:20 -07:00
Ara e6bb1a14ec fix(vscode): reuse approved tool rows (#11213)
* fix(vscode): reuse approved tool rows

* fix(vscode): clear stale approved tool rows
2026-06-03 09:56:40 -07:00
Ara 42ab1b94a2 fix(llms): strip Cerebras reasoning history (#11214) 2026-06-03 09:56:40 -07:00
Max Paulus 🥪 97d8a33db0 fix unauthed user flow
- show a small sign in button if user is unauthed with any provider
2026-06-03 09:56:40 -07:00
Robin NewhouseandCursor 4961bf2898 fix(vscode): compact Codex OAuth before input cap (#11194)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-03 09:56:40 -07:00
Robin NewhouseandMikołaj Kondratek 47f3654b70 fix(vscode): wire auto compact into SDK sessions (#11197)
* fix(vscode): wire auto compact into SDK sessions

* test(sdk): cover both directions of useAutoCondense task override

The previous test left the global mock at `true` for both calls, so the
`taskSettings: true` branch would have passed even if task settings were
ignored entirely. Make the mock read a mutable flag and flip it to `false`
before the second call so both override directions — task `false` over
global `true`, and task `true` over global `false` — are genuinely
exercised.

* Fix mock return type in cline-session-factory test

The getGlobalSettingsKey mock inferred a literal 'false | undefined' return type, so later mockImplementation overrides returning 'true' failed type checking (TS2345). Annotate the implementation as 'boolean | undefined' to widen the inferred mock signature.

---------

Co-authored-by: Mikołaj Kondratek <19799111+mkondratek@users.noreply.github.com>
2026-06-03 09:56:40 -07:00
Dominic Cooney 7b4a0bf40a fix(vscode): keep in-progress MCP OAuth flow across reconnects
The MCP SDK calls redirectToAuthorization() on every connection attempt,
and a single server can be reconnected repeatedly (settings watcher,
reconnect handler, restart). Regenerating the OAuth `state` on each call
replaced the state stored for a flow whose authorization URL the user may
already have open, so the completed callback failed validation with
"Invalid OAuth state".

redirectToAuthorization() now keeps an in-progress, still-fresh flow
instead of starting a new one (freshness measured from when the flow
started, never extended, so a stale flow always expires). The PKCE
verifier is pinned to the kept flow so token exchange still validates.

Also add local dev/test tooling:
- src/dev/mcp-oauth-test-server: a zero-dependency OAuth AS + MCP
  StreamableHTTP server for exercising the flow locally, with
  fault-injection flags (--auto-deny, --slow-authorize, --code-ttl).
- src/extension.ts: a debug-only globalThis.__clineHandleUri hook (gated
  on CLINE_CAPTURE_BROWSER) so the debug harness can deliver simulated
  vscode:// OAuth callbacks; documented in the harness README.
2026-06-03 09:56:40 -07:00
Dominic Cooney cf814e1479 fix(vscode): forward Bedrock region + AWS auth to the SDK gateway
Bedrock requests built by the SDK adapter dropped the AWS region and
authentication mode, so a pasted Bedrock API key (awsBedrockApiKey +
awsAuthentication "apikey") was silently ignored and requests fell
through to the SigV4 credential chain with no region.

Two bugs, both verified end-to-end against a live Bedrock endpoint via
the debug harness:

1. The SDK ProviderConfig was built with only providerId/modelId/apiKey/
   baseUrl. New bedrock-config.ts maps the legacy ApiConfiguration onto
   the SDK's structured region + aws block (including the webview's
   "credentials" radio -> SDK "iam"), wired into both inference paths
   (buildSdkProviderConfig for utility calls, buildSessionConfig for the
   main task loop).

2. The main chat path's gateway config is built by core from the
   providers.json `stored` entry, which the session's providerConfig does
   not override. A stale Bedrock entry (e.g. a legacy migration with
   region us-east-1 + SigV4 keys) silently won, sending requests to the
   wrong region (403). buildSessionConfig now persists the
   StateManager-derived Bedrock settings to providers.json so `stored` is
   authoritative. The bearer apiKey is only persisted for api-key auth to
   keep stored clean for profile/iam.

Also documents the ELECTRON_RUN_AS_NODE debug-harness gotcha in
.clinerules/general.md.
2026-06-03 09:56:40 -07:00
Max Paulus 🥪 40e64baa0a fix xai provider
- xai provider settings now properly updates providers.json
2026-06-03 09:56:39 -07:00
Max Paulus 🥪 540b9234dc updat gitignore 2026-06-03 09:56:39 -07:00
Dominic Cooney 52828aab71 fix(vscode): widen ZAiProvider getEventValue to accept VSCodeDropdown event union
The nightly 'vscode test' job (npm run ci:build -> build:webview -> tsc -b) failed on both Linux and windows-latest:

  ZAiProvider.tsx: Argument of type 'Event | FormEvent<HTMLElement>' is not assignable to parameter of type 'Event'

VSCodeDropdown's onChange supplies 'Event | React.FormEvent<HTMLElement>', but the getEventValue helper (added when the zai provider was moved to providers.json) was typed to accept only Event. The helper only reads target.value, which exists on both, so widen the parameter to the same union the dropdown provides.

Note: this only surfaces under 'tsc -b' / 'tsc --noEmit -p tsconfig.app.json'; the webview's root tsconfig.json is a solution file with files:[] so a bare 'tsc --noEmit' checks nothing.

The separate 'test / test' job failure is an aggregate gate that fails because vscode-test failed; fixing this resolves it too.
2026-06-03 09:56:39 -07:00
Dominic Cooney 33dafb193b fix(vscode): load ambient vscode LM type decls in unit-test ts-node program
The nightly CI 'Unit Tests with coverage - Linux' step (npm run test:unit) failed in ts-node compilation:

  state-keys.ts: Module 'vscode' has no exported member 'LanguageModelChatSelector'

This surfaced as a misleading 'Cannot find package @shared/...' ERR_MODULE_NOT_FOUND: Mocha tried require() first (which threw the ts-node TSError), then fell back to import(), whose ESM resolver cannot resolve the @shared/* path alias.

tsconfig.unit-test.json is driven by ts-node, which defaults to files:false and compiles modules on demand from their imports. The loose ambient augmentation in src/types/vscode-language-model.d.ts was therefore never loaded, so state-keys.ts failed to compile.

Set ts-node.files=true and add src/types/**/*.d.ts to include so the augmentation is part of the unit-test program. Mirrors the earlier tsconfig.test.json fix for the separate build toolchain.
2026-06-03 09:56:39 -07:00
Max Paulus 🥪 643d945d65 fix litellm provider 2026-06-03 09:56:39 -07:00
Max Paulus 🥪 f3a215cd0e change zai provider to user providers.json instead of statemanager 2026-06-03 09:56:39 -07:00
Dominic Cooney 4fce10248e fix(vscode): include ambient vscode LM type decls in test tsconfig
The nightly CI "Build Tests and Extension" step (npm run ci:build) failed
in compile-tests (tsc -p tsconfig.test.json) with:
  getVsCodeLmModels.ts: Property 'lm' does not exist on type 'typeof import("vscode")'
  state-keys.ts: Module '"vscode"' has no exported member 'LanguageModelChatSelector'

@types/vscode is pinned to 1.84.0, which predates the Language Model API.
The repo compensates with an ambient augmentation in
src/types/vscode-language-model.d.ts, which the main tsconfig picks up via
its 'src/**/*' include. tsconfig.test.json overrides include to only
'src/**/*.test.ts'; listing src/types under typeRoots does not load a loose
.d.ts (typeRoots only auto-loads @types-style package folders), so the
augmentation was missing from the test program.

Add 'src/types/**/*.d.ts' to tsconfig.test.json include so the augmentation
is part of the test compilation.
2026-06-03 09:56:39 -07:00
Max Paulus 🥪 154f9e0e11 fix open task conversation file 2026-06-03 09:56:39 -07:00
Max Paulus 🥪 c041089a6d fix history view bugs
- deleting entries works
- favoriting works
2026-06-03 09:56:39 -07:00
Max Paulus 🥪 62f11cd1d3 improve OCA provider
- oca provider login now works
2026-06-03 09:56:38 -07:00
Dominic Cooney ea38d1049d fix(vscode): show Start New Task after multi-iteration completion turns
The completion signal (attemptCompletionSeen) was scoped to the message
translator's per-iteration reset(), so a turn that called the completion tool
and then ran another iteration before 'done' lost the signal — the turn ended
as awaiting_followup instead of completed and the footer showed no
'Start New Task' button despite the green Task Completed box.

Make the completion signal turn-scoped: reset() only clears streaming pointers;
a new clearTurnOutcome() clears it at genuine turn/task boundaries (initTask,
reinitExistingTaskFromId, askResponse). Also recognize the SDK's built-in
submit_and_exit completion tool (summary field) alongside the VSCode
attempt_completion tool (result field).

Removes the webview-message-state design/findings docs and rewrites the
related comments to describe the system as-is.
2026-06-03 09:56:38 -07:00
Max Paulus 🥪 cfb4ef49be improve openai compatible provider settings 2026-06-03 09:56:38 -07:00
Dominic Cooney a94d1b2c08 fix(vscode): fix stuck/missing footer buttons from TurnState regressions
Webview now gates turnState by seq (a stale snapshot can no longer revert
streaming->idle), never empties a live transcript on a lone newer-epoch partial,
and routes a follow-up after completed/awaiting_followup to askResponse instead
of starting a new task.

Backend phase emission fixes surfaced by live testing:
- post state on turn end even when the done event carries no messages
- askResponse sets phase=streaming (resume/continue shows Cancel, unblocks send)
- a post-cancel turn-complete straggler no longer clobbers resumable

Adds reducer, send-routing, and session-event-coordinator unit tests. See
src/sdk/docs/webview-message-state-design.md §11 for the full debugging log.
2026-06-03 09:56:38 -07:00
Dominic Cooney 5685c2fa36 refactor(vscode): translator hygiene — suppress ask_question row, drop done synthetic ask
S7 (a + c) of the message-state redesign — removes two order-dependent hacks now that
TurnState is authoritative.

(a) ask_question / ask_followup_question are suppressed from the generic say:"tool"
    renderer (both content_start and content_end). The SdkInteractionCoordinator services
    these and emits the proper ask:"followup"; the generic tool row was an orphan partial
    that never finalized and defeated the tail heuristics. The CLI already does this.

(c) The `done` handler no longer synthesizes a trailing ask:"completion_result" — that
    was the "must be last message" hack (ENG-1887) that existed only because the webview
    inferred UI mode from the array tail. `done` now emits no transcript message and only
    signals turnComplete; the webview reads phase from TurnState (completed when
    attempt_completion was used, else awaiting_followup). The green "Task Completed" box
    still comes from the say:"completion_result" emitted at attempt_completion content_end.

Translator unit tests updated to the new contract (done → 0 messages, turnComplete=true).

Deferred to a follow-up (lower risk if left): (b) collapsing the approval ask onto the
streaming tool's id (needs cross-coordinator id threading), and (d) the mistake_limit
forced abort (now harmless since phase is authoritative). The persisted-history renderer
still appends its own trailing ask so reopened tasks show the resume affordance —
intentional.

428 SDK unit tests pass; tsc + biome clean.
2026-06-03 09:56:38 -07:00
Dominic Cooney ba28c556b4 fix(vscode): cancel raises the epoch fence before aborting (no post-cancel stragglers)
S6 of the message-state redesign. sdkHost.abort() is cooperative — the SDK can emit a
few more events after it. Previously cancelTask aborted first and only the (post-abort)
!isRunning filter stripped two trailing ask types, letting say:* stragglers land after
the resume_task ask and wedge the UI.

Now cancel raises the fence SYNCHRONOUSLY before the abort:
- SdkController.cancelTask sets turnState.phase = "resumable" (already in S4), then
- SdkTaskControlCoordinator.cancelTask calls raiseCancelFence() (epoch bump) BEFORE
  awaiting sdkHost.abort().

Any event the SDK emits after the abort request therefore carries the OLD epoch and is
dropped by the webview's convergent reducer; the authoritative phase is "resumable"
(Resume Task), independent of the message tail. Order matters and is covered by a unit
test (fence before abort).

Usage accounting is exempt from the fence — it was never gated by the message filter, so
a post-cancel usage event still bills the tokens the provider actually generated.

The legacy !isRunning ask-only filter is now redundant but kept as defense-in-depth.
428 SDK unit tests pass; tsc + biome clean.
2026-06-03 09:56:38 -07:00
Dominic Cooney 0fbcbc45f5 fix(vscode): drive webview footer + buttons from TurnState (fixes RC1)
S5 of the message-state redesign. The webview decided "thinking vs approving vs done"
and which buttons to show by inspecting the TAIL of clineMessages. Because the backend
appends bookkeeping (api_req_started usage) after content and even after approval asks,
the tail routinely meant the wrong thing — producing stuck "Thinking", vanishing
Approve/Reject, and the footer disagreeing with the buttons (RC1, reproduced on camera).

Now the webview reads the authoritative backend-owned TurnState (added in S4):

- buttonConfig: add buttonsForPhase(turnState, anchoredMessage) and the dispatcher
  getButtonConfigFromState(messages, turnState, mode). The button SET is chosen by phase;
  approval labels (Approve vs Save, Run Command, MCP, subagents) come from the anchored
  message (turnState.anchorTs). When turnState is absent (classic/older state) it falls
  back to the legacy tail-walking getButtonConfigForMessages.
- ActionButtons reads turnState from useExtensionState and uses getButtonConfigFromState.
- MessagesArea.isWaitingForResponse short-circuits to `phase === "streaming"` when
  turnState is present (and only shows the footer loader until a content row is actually
  streaming); the legacy tail inference is kept as the fallback.

Button actions (approve/reject/proceed/new_task) already send a fixed responseType
independent of clineAsk, and the SDK backend resolves the pending approval/followup
promise — so routing is correct under TurnState. Classic fallback paths are untouched.

New unit tests cover buttonsForPhase (every phase + anchored-label selection +
mistake_limit-vs-api_req_failed) and getButtonConfigFromState (prefers TurnState over a
trailing bookkeeping tail; legacy fallback). 68 webview chat tests pass; tsc + biome clean.
2026-06-03 09:56:38 -07:00
Dominic Cooney b37d8e466b feat(vscode): add authoritative TurnState (backend-owned UI mode)
S4 of the message-state redesign. Introduces the single source of truth for the
webview's UI mode so it no longer has to be inferred from the tail of clineMessages
(the root of RC1: missing/stuck "Thinking", vanishing approval buttons, footer and
buttons disagreeing).

- Add TurnPhase / TurnState to shared types and ExtensionState.turnState (rides inside
  state_json; no proto change).
- Add TurnStateTracker, owned by SdkController, sharing the one id/seq/epoch authority.
  Each transition stamps a fresh seq so the webview keeps only the newest TurnState.
- Set the phase at the exact lifecycle points where the backend knows it:
    streaming         — initTask / reinit / user responded (resolvePending*)
    awaiting_approval — handleRequestToolApproval (anchored on the ask)
    awaiting_followup — handleAskQuestion; and a turn that ends WITHOUT attempt_completion
    completed         — turn ends and attempt_completion was used
    error             — onSendError; mistake_limit
    resumable         — cancelTask (set before abort)
    idle              — clearTask
  The completed-vs-awaiting_followup decision uses
  MessageTranslatorState.wasAttemptCompletionSeen().
- getStateToPostToWebview now ships turnState in every snapshot.

This is ADDITIVE: the webview does not read turnState yet (S5 wires footer/buttons to
it and deletes the tail heuristics). Classic/legacy paths leave turnState undefined and
keep the legacy behavior. 427 SDK unit tests pass (incl. new turn-state-tracker tests);
tsc and biome clean.
2026-06-03 09:56:37 -07:00
Dominic Cooney 482ae279f8 fix(vscode): converge webview transcript via a pure reducer (fixes last-message-missing)
S3 of the message-state redesign. The webview received the same conversation over
two unordered, fire-and-forget channels — incremental partial messages and full
state snapshots — and the state handler REPLACED clineMessages wholesale ("// HACK:
Preserve clineMessages if currentTaskItem is the same"). A stale snapshot captured
before the last message landed could clobber the transcript, dropping the final
message and leaving the UI stuck on "Thinking…" (reproduced earlier on camera).

Introduce a pure convergent-replica reducer (messageReducer.ts) keyed on the three
extension-stamped quantities from S2:
  - ts    : identity / merge key
  - seq   : freshness (higher seq wins for the same ts)
  - epoch : conversation/replica fence (newer replaces, older is dropped, equal merges)

applyMessage / applyStateSnapshot are total and side-effect free:
  - older epoch  -> drop (straggler from a previous task/render)
  - newer epoch  -> replace the transcript wholesale (new task / history load)
  - same epoch   -> merge by ts keeping the higher seq; a snapshot may ADD/UPDATE
                    rows but NEVER truncate, so it can't drop a message the partial
                    stream already delivered. Stale (older stateVersion) snapshots are
                    ignored wholesale.

ExtensionStateContext now feeds both subscription callbacks through the reducer via a
replicaRef, replacing the wholesale-replace HACK and the findLastIndex append. Classic/
legacy state is unstamped (epoch 0 / version 0) and merges by ts exactly as before.

Tests (messageReducer.test.ts): deterministic cases mapping 1:1 to the bugs (stale
snapshot must not shrink the transcript; lower-seq ignored; partial->final in place;
older-epoch straggler dropped; older-version snapshot ignored), PLUS an
order-independence proof — all 120 permutations of a causal log, with duplication and
with non-final drops, converge to the same canonical state. This is the high-confidence
guarantee that the webview cannot get stuck regardless of delivery timing.

62 webview chat tests pass; tsc and biome clean.
2026-06-03 09:56:37 -07:00
Dominic Cooney ddb16f7dc6 refactor(vscode): stamp seq/epoch on messages and state; fire-and-forget delivery
S2 of the message-state redesign. Lays the groundwork for the convergent-replica
webview reducer (S3) so the webview can never get stuck on stale/out-of-order
delivery.

Stamping (extension-owned, synchronous, from the single MessageIdMinter):
- Every ClineMessage flowing to the webview is stamped in SdkMessageCoordinator with
  a fresh `seq` (freshness) and the current `epoch` (conversation/replica fence),
  before it is stored or emitted. The same object references go to both the message
  state handler and the partial-message stream, so both channels carry identical
  stamps. An updated message (partial -> final, same ts) passes through again and
  gets a higher seq, so the freshest copy always wins regardless of arrival order.
- Every state snapshot is stamped in SdkController.getStateToPostToWebview with a
  fresh `stateVersion` (sampled from the same counter) and the current `epoch`.
- `epoch` is bumped at every conversation boundary via a new
  resetMessageTranslatorAndFence() wired into the existing resetMessageTranslator
  sites (task start/clear, history open, reinit, mode rebuild, new-session
  follow-up). iteration_start streaming resets do NOT bump it.

Transport:
- ClineMessage proto gains seq/epoch (fields 24/25) and the conversions carry them.
- ExtensionState gains stateVersion/epoch (ride inside state_json, no proto change).

Fire-and-forget delivery:
- sendPartialMessageEvent and sendStateUpdate no longer await postMessage to the
  webview. A hidden/reloaded/closed webview can make postMessage hang or resolve
  false; awaiting it could stall the backend turn loop. Correctness no longer
  depends on any single delivery — the webview will be a convergent replica (S3).

All new fields are optional/default-0 so the classic/legacy path is unaffected.
Cancel's epoch bump + fence-before-abort and the remaining one-off Date.now() mint
sites are handled in later steps (S6/S2-tail). 423 unit tests pass; tsc clean.
2026-06-03 09:56:37 -07:00
Dominic Cooney 4aace9e226 refactor(vscode): unify ClineMessage id minting behind one MessageIdMinter
Message ids (ClineMessage.ts) were minted from Date.now() in two independent
generators: the live message translator (pure ++counter seeded once) and the
interaction coordinator (Math.max(Date.now(), last+1)). Because the translator
counter drifts behind wall-clock, it can later catch up to a clock-based id
minted by the interaction coordinator, producing colliding ids for different
messages. That breaks any merge-by-id scheme on the webview side.

Introduce a single process-wide MessageIdMinter (pure monotonic id/seq/epoch
counters, never reads the clock) owned by MessageTranslatorState and shared by:
- live SDK event translation,
- the interaction coordinator (tool approval / ask_question / user_feedback),
- history rendering (sdkMessagesToClineMessages).

This makes every id globally unique within the process, so regenerated history
ids never overlap live-session ids. Behavior is otherwise unchanged.

Also adds the message-state pipeline design + investigation docs under
src/sdk/docs.

S1 of the message-state redesign; seq/epoch stamping and the remaining one-off
Date.now() mint sites follow in S2.
2026-06-03 09:56:37 -07:00
Dominic Cooney 437f7eb745 docs(vscode): refine sdk-consolidation TODOs for live model fetching
After inspecting the SDK's generic models-URL fetcher
(sdk/packages/core/src/services/providers/model-source.ts
`fetchModelIdsFromSource` + `resolveModelsSourceUrl`), update the TODOs on the
bespoke refresh*Models handlers to capture the real constraint discovered:

- The SDK fetcher exists and is provider-agnostic, but returns model *ids only*
  (unknown ids get placeholder ModelInfo with no real pricing/capabilities).
- `mergeKnownModels` treats a registered `modelsSourceUrl` as the authoritative
  "installed" list (Ollama/LM Studio semantics) and DISCARDS the curated catalog
  when the live fetch returns results.

So simply registering `modelsSourceUrl` for Groq/Baseten/Hicap/HuggingFace/
Vercel/OpenRouter would regress rich model metadata. Proper consolidation needs
an SDK enhancement first (merge-mode or richer per-provider parsing), then the
extension handlers + RPCs can be deleted. refreshGroqModels.ts carries the
detailed note; the others reference it.
2026-06-03 09:56:37 -07:00
Dominic Cooney 1107df80d3 refactor(vscode): delegate Cline recommended-models fetch to the SDK; TODO others
refreshClineRecommendedModels now delegates the HTTP fetch + response
normalization + offline fallback to the SDK's fetchClineRecommendedModels
(@cline/core), removing ~80 lines of duplicated logic. The extension wrapper
keeps its distinct behavior: the CLINE_RECOMMENDED_MODELS_UPSTREAM feature-flag
gate, the in-memory TTL cache, and in-flight dedup. The proxy-aware fetch
(@/shared/net) and the configured apiBaseUrl are passed through to preserve
network/proxy behavior. The SDK's offline fallback list is identical to
CLINE_RECOMMENDED_MODELS_FALLBACK, so offline behavior is unchanged.

Because the module now imports the ESM-only @cline/core, its unit test moves
from mocha to vitest (joining the other SDK-touching models tests): added to
vitest include + mocha ignore, and fetchClineRecommendedModels added to the
vitest @cline/core stub (and the mocha/integration @cline/core mocks for
completeness). Rewrote the test vitest-native, asserting flag-gate, delegation,
and flag re-check.

Also:
- Add TODO(sdk-consolidation) notes to the remaining bespoke live-model-refresh
  handlers (Groq, Baseten, Hicap, HuggingFace, Vercel AI Gateway, OpenRouter)
  documenting the path to share them with the CLI via the SDK (register
  modelsSourceUrl) and then delete the extension-only handlers + RPCs. These
  are NOT migrated yet because the SDK does not currently live-fetch those
  providers (only ollama/lmstudio register modelsSourceUrl), so deleting them
  today would regress to the curated catalog.
- Remove an unused local type (RuleLoadPart) found while reviewing biome
  noUnusedVariables output.

tsc --noEmit clean; vitest 439 passing.
2026-06-03 09:56:37 -07:00
Dominic Cooney a1a88c4258 refactor(vscode): remove dead sapAiCoreModelDescription const
Unused leftover from the deleted SAP AI Core provider handler (found via biome noUnusedVariables). tsc + vitest green.
2026-06-03 09:56:37 -07:00
Dominic Cooney 5320885770 refactor(vscode): delete genuinely-unused telemetry/auth helpers
- services/telemetry/events/EventHandlerBase.ts: abstract base for telemetry
  event handlers whose concrete subclasses were removed with the classic task
  code; no remaining references (it was the only file left in events/).
- services/auth/AuthServiceMock.ts: test mock with no importers after the
  associated test was deleted.

Note: the rest of services/* (TelemetryService, McpHub, FeatureFlagsService,
ErrorService and their IFoo interfaces) IS live — the interfaces only looked
unreachable to the esbuild oracle because they're consumed via type-only
imports (erased at emit). Verified via importer cross-check; tsc + vitest green.
2026-06-03 09:56:37 -07:00
Dominic Cooney 6fcbd039fa refactor(vscode): remove dead cost utils and createOpenAIClient
- Delete src/utils/cost.ts (+ test): calculateApiCostAnthropic/OpenAI/Qwen had
  no consumers left after the provider handlers were removed (only the test
  referenced them).
- Remove createOpenAIClient from src/shared/net.ts (no remaining callers) and
  its now-unused openai + EnvUtils imports. The proxy-aware fetch and
  getAxiosSettings exports remain.

tsc --noEmit clean, vitest 436 passing.
2026-06-03 09:56:36 -07:00
Dominic Cooney 79ffd2f5fb chore(vscode): drop npm deps only used by deleted provider handlers
Remove dependencies that became unused after the legacy API provider handlers
and tree-sitter service were deleted (no remaining imports in src, webview, or
build config):

  @anthropic-ai/vertex-sdk, @aws-sdk/client-bedrock-runtime,
  @aws-sdk/credential-providers, @azure/identity,
  @cerebras/cerebras_cloud_sdk, @google-cloud/vertexai, @mistralai/mistralai,
  @sap-ai-sdk/ai-api, @sap-ai-sdk/orchestration, @sap-cloud-sdk/connectivity,
  ollama, tree-sitter-wasms, web-tree-sitter

Also remove the now-dead copyWasmFiles esbuild plugin (it only copied
tree-sitter WASM files for the deleted code-definition service). Kept openai,
@anthropic-ai/sdk, @google/genai, and aws4fetch — still imported by live code.

Verified: extension + standalone esbuild builds succeed, tsc --noEmit clean,
vitest 436 passing.
2026-06-03 09:56:36 -07:00
Dominic Cooney 9a83fcb4fa fix(vscode): drop dead barrel re-exports of deleted files
The previous deletion commit (2c9bd62) removed
core/assistant-message/parse-assistant-message.ts and
core/permissions/CommandPermissionController.ts, but the corresponding barrel
edits (removing their re-exports from index.ts) were dropped by lint-staged's
stash and never committed, leaving HEAD referencing deleted modules. Remove the
dead re-exports so the barrels only export live members.
2026-06-03 09:56:36 -07:00
Dominic Cooney 0ad9de2317 chore(vscode): add scripts/find-dead-src.mjs dead-code oracle
Computes src files unreachable from the shipped entry points (extension host + standalone host used by JetBrains/CLI) plus webview shared refs, using esbuild metafile reachability. Used to drive the post-SDK-migration dead-code deletions; keep for future pruning. Note: results still need a tsc-gated importer cross-check because esbuild drops import-type-only edges.
2026-06-03 09:56:36 -07:00
Dominic Cooney 1bba06ae6f refactor(vscode): delete more dead classic code (hooks, permissions, claude-code, misc)
Removes additional source files unreachable from any shipped entry point
(extension host, standalone host, webview) after the SDK migration, verified
via esbuild reachability + a per-file importer cross-check that excludes any
file still referenced by a live non-test survivor or generated/test glue, then
gated on tsc --noEmit + vitest (436 passing):

- services/ripgrep, integrations/notifications, utils/string, utils/tabFiltering
- core/assistant-message/parse-assistant-message
- core/hooks: hook-model-context, notification-hook, precompact-executor,
  PreToolUseHookCancellationError
- core/permissions/CommandPermissionController
- core/workspace/detection
- integrations/claude-code: run, message-filter
- integrations/editor: FileEditProvider, detect-omission
- integrations/misc: extract-file-content, extract-images

Dropped the now-dead re-exports from core/assistant-message/index.ts and
core/permissions/index.ts (those barrels stay; they still export live types
used by generated host glue).
2026-06-03 09:56:36 -07:00
Dominic Cooney 5575f681f2 refactor(vscode): delete dead tree-sitter code-definition service
src/services/tree-sitter/** (language parsers + queries for the classic
list_code_definition_names path) is unreachable from any shipped entry point
after the SDK migration. Verified via scripts/find-dead-src.mjs (esbuild
reachability from extension + standalone entries, plus webview shared refs) and
an importer cross-check (no live value or type importers), gated on
tsc --noEmit + vitest (436 passing).
2026-06-03 09:56:36 -07:00
Dominic Cooney 4922935564 refactor(vscode): delete dead classic system-prompt + slash-command code
The classic system-prompt builder (core/prompts/system-prompt/**), the
deep-planning prompt variants, core/prompts/commands.ts, and the
core/slash-commands handler are no longer reachable from any shipped entry
point (extension host, standalone host, or webview) after the SDK migration:
the SDK provides prompt construction (buildClineSystemPrompt) and slash-command
handling. Removed them and their orphaned tests.

Kept core/prompts/responses.ts (still live) and its tests.

Dead-code reachability was computed with scripts/find-dead-src.mjs (esbuild
metafile reachability from src/extension.ts + src/standalone/cline-core.ts,
plus webview src/shared references), then gated on tsc --noEmit + vitest.
2026-06-03 09:56:36 -07:00
Dominic Cooney 7e39120191 refactor(vscode): delete legacy API provider handlers and dead transforms
Now that buildApiHandler routes through the @cline/llms SDK, the legacy
per-provider handler classes and their supporting code are unused. Remove them:

- apps/vscode/src/core/api/providers/** (all 40+ handler classes, types, tests)
- apps/vscode/src/core/api/transform/** except stream.ts (format/stream/
  tool-call helpers only the handlers used)
- apps/vscode/src/core/api/utils/** (messages/responses API support)
- apps/vscode/src/core/api/retry.ts (+ test)
- apps/vscode/src/shared/sdk-handler-models.ts (getProviderModelFromSdk;
  only the deleted handlers' getModel() used it)

core/api now contains just index.ts (types + SDK re-exports), transform/stream.ts
(ApiStream types still referenced by the local ApiHandler interface), and
adapters/.

Supporting changes to keep everything compiling/working:
- context-window-utils: drop the dead `api instanceof OpenAiHandler` DeepSeek
  branch (handlers are SDK GatewayApiHandlers now); the 64k switch case already
  handles DeepSeek context sizing.
- Relocate fetchLiteLlmModelsInfo from the deleted litellm handler into
  core/controller/models/fetchLiteLlmModels.ts (used by refreshLiteLlmModels).
- Move the `declare module "vscode"` Language Model API augmentation (previously
  carried by the vscode-lm handler) into src/types/vscode-language-model.d.ts so
  live consumers (getVsCodeLmModels, vsCodeSelectorUtils) keep their types.
- Move the @google/genai test mock out of the deleted providers dir to
  src/test/fixtures/google-genai-mock.ts and repoint test-setup.js.
2026-06-03 09:56:36 -07:00
Dominic Cooney 8c36159c43 refactor(vscode): route buildApiHandler through the SDK; remove OpenRouter handler
Replace the legacy per-provider buildApiHandler factory with an SDK-backed
handler (apps/vscode/src/sdk/sdk-api-handler.ts) built via @cline/llms
createHandler(). The two standalone callers (commit-message generation and
explain-changes) now import buildApiHandler directly from the SDK module;
@core/api stays types-only (re-exporting SDK types) so it can keep being
imported widely without pulling the SDK runtime graph into activation.

Also:
- Delete the now-dead OpenRouterHandler and its test (createOpenRouterStream
  stays; it is still used by ClineHandler).
- buildSdkProviderConfig never sends both reasoning.effort and
  reasoning.max_tokens (some providers reject it), and supports
  disableReasoning for fast one-shot utility calls; commit-message and
  explain-changes opt in.
- commit-message generation surfaces the real SDK stream error instead of a
  generic "empty API response".
- getGitDiff: run git diff --staged even before the first commit; only gate
  the git diff HEAD fallback on having commits (fixes "no changes" for the
  initial commit of a new repo).
2026-06-03 09:56:35 -07:00
Dominic Cooney a193f19468 refactor: remove legacy Cline model overrides, wire reasoning effort through SDK
- Delete refreshClineModels.ts, refreshClineModelsRpc.ts, and test
- Remove refreshClineModelsRpc proto RPC
- Remove clineModels state from ExtensionStateContext, StateManager cache, disk
- ClineModelPicker and useOnboardingModels use SDK catalog directly
- Replace ThinkingBudgetSlider with ReasoningEffortSelector for Cline provider
- Use supportsReasoning from SDK catalog instead of hardcoded model names
- Add ProviderReasoningPatch to proto and ProviderConfigPatch contract
- Wire reasoning effort changes through writeProviderConfig to SDK ProviderSettingsManager
- Remove hardcoded Claude switch in openrouter-stream.ts
- Add supportsReasoning to supportsReasoningEffort check in openrouter-stream
2026-06-03 09:56:35 -07:00
Dominic Cooney 855d31c86f fix(vscode): load real @cline/llms in unit-test harness; fix/trim provider tests
The mocha unit-test harness stubbed @cline/llms with an empty catalog, so the SDK-migrated provider handlers (which read the real catalog) failed ~51 tests under the nightly suite.

- src/test/requires.ts now loads the real @cline/llms by resolving its package directory and requiring the ESM entry by absolute path (Node 22 require(esm)), bypassing the package's import-only exports map. This restores real catalog data to provider unit tests.
- vertexModelSupportsGlobalEndpoint: also match legacy ':' context-window/speed suffixes (e.g. claude-opus-4-7:1m), not just '@' snapshot variants.
- refreshClineModels: derive prompt-cache support from reported input_cache_read pricing for any provider, not just openai/google prefixes.
- Delete the failing tests for the classic provider handlers (bedrock cross-region/native-tool-calling, gemini metadata, cline/openrouter qwen cache, wandb unknown-model). These handlers are only reached via buildApiHandler (explain-changes + commit-message generation) and the provider/catalog domain is owned and tested by the SDK; they will be removed when buildApiHandler is retired.
2026-06-03 09:56:35 -07:00
Dominic Cooney 1acacda3f5 fix(vscode): fix webview provider-model tests and a Windows path test
The nightly publish runs the full webview-ui suite and the Windows extension suite, which surfaced failures the extension-only run does not.

- Harden useProviderModels against a missing providerModelsByProvider map so a partially-mocked ExtensionStateContext no longer crashes the hook.
- Update the SapAiCoreModelPicker and APIOptions specs to provide the provider model-list context the components now read from the SDK catalog, and seed the model ids each test asserts. Removes the now-dead @shared/api sapAiCoreModels mock.
- Compare resolveDataDir() against path.join() instead of a hardcoded POSIX path so the CLINE_DIR fallback test passes on Windows.
2026-06-03 09:56:35 -07:00
Dominic Cooney 6f2f159f7e fix(vscode): support OpenAI Compatible provider on the SDK adapter
Selecting the OpenAI Compatible provider failed with 'Unknown provider "openai"', and manually entered model ids were displayed as the catalog default (gpt-4o).

- Map the extension's 'openai' provider id to the SDK's 'openai-compatible' built-in at the SDK boundary (toSdkProviderId), and convert before handing the provider id to core when building a session config.
- Treat openai-compatible as a custom-model-id provider so model resolution honors a user-entered model id instead of coercing it to the catalog default. Adds providerAllowsCustomModelIds() as the shared signal.
- Bump @cline/core, @cline/llms, @cline/shared, @cline/agents to ^0.0.42 (which registers the openai-compatible built-in) and dedupe the dependency tree.
- Carry the tool name on reconstructed Anthropic-format tool_result blocks to satisfy the SDK's ToolResultContent contract.
- Update tests for the refreshed SDK model catalog (Gemini default).
2026-06-03 09:56:35 -07:00
Dominic Cooney 96da30d8c7 chore(vscode): clean up SDK migration branch (comments, dead code, scree, tests)
- Remove porting scree: docs/sdk-model-catalog/* planning docs, TODO-resume-session.md, and stale doc references in code/comments.

- Rewrite before/after narrative comments in the 'eternal now' style; drop transient 'Step N'/'Phase N' labels.

- Remove leftover [HistoryPerf] diagnostic logging and a dead try/catch rethrow; minor readability/naming.

- Restore .clinerules/network.md (still relevant) and update sdk-migration.md to drop dangling references.

- Fix latent circular-init TDZ in openai-codex-models (lazy catalog build).

- toggleRemoteConfigSetting no longer returns a never-resolving promise.

- Move vitest config into apps/vscode and rename script test:sdk -> test:vitest; wire it into CI.

- Repair/reimplement the SDK-adapter vitest suites (auth-service, provider-migration, sdk-task-history) to match current behavior; all 432 tests pass.
2026-06-03 09:56:35 -07:00
Dominic Cooney 7a0d48c2e4 fix(vscode): preserve provider model selection fields 2026-06-03 09:56:35 -07:00
Dominic Cooney 18a29b7563 refactor(vscode): source provider model catalogs from @cline/llms SDK (ENG-2116)
Migrates the vscode extension off its hand-curated static model catalogs
in apps/vscode/src/shared/api.ts and on to the @cline/llms SDK as the
single source of truth for provider/model metadata, end-to-end across
the extension host and the webview.

Net impact on the static catalog file:
  apps/vscode/src/shared/api.ts: 5092 -> 468 lines (~90% gone).

What changed at each layer
--------------------------

SDK / catalog plumbing (apps/vscode/src/sdk/model-catalog/):
  - New `ProviderCatalog.peekModels(providerId)` synchronous cache read.
  - `resolveModelInfo` rewritten: committed selection -> catalog peek
    -> await catalog.resolveModels on cache miss. No race with a
    background warmer; if the catalog truly has nothing, returns
    source: "unknown" and the webview renders a neutral loading state.
  - `applyHostModelInfoOverrides` is the canonical seam for the few
    fields the SDK does not yet carry. Today it carries only the Vertex
    `supportsGlobalEndpoint` allowlist (vertex-global-endpoint.ts, with
    a TODO to upstream into the SDK).
  - `ProviderListing` extended with SDK metadata (`is_popular`,
    `popular_rank`, `usage_cost_display`, `capabilities[]`) and plumbed
    through proto + conversion.

Extension-host handlers (apps/vscode/src/core/api/providers/):
  - New shared helper `apps/vscode/src/shared/sdk-handler-models.ts`:
    `getProviderModelFromSdk(providerId, requestedModelId, committedInfo?)`
    returns `{ id, info }` from `getProviderCollectionSync` with
    Vertex global-endpoint overrides applied.
  - 27 handlers converted to a one-liner `getModel()` through that
    helper. Per-handler nuances preserved:
      * Anthropic: strips `:fast` and `:1m` host-side suffixes before
        SDK lookup; carries them back on the returned id so the
        per-request betas still flip.
      * Bedrock: keeps the custom Application Inference Profile ARN
        branch; base-model info from the SDK.
      * Cerebras: keeps the `qwen-3-coder-480b-free` -> `qwen-3-coder-480b`
        paid alias.
      * Qwen / ZAi: SDK has a single catalog each; handlers keep the
        regional base-URL switch but no longer fork the catalog.
      * Wandb: keeps the "unknown id falls through to safe defaults"
        escape hatch via `MODEL_COLLECTIONS_BY_PROVIDER_ID`.

Refresh-models background tasks:
  - `refreshBasetenModels`, `refreshGroqModels`, `refreshHuggingFaceModels`
    source their offline-fallback catalog from the SDK via
    `getProviderCollectionSync` + `adaptSdkModelInfo`. Live fetch path
    unchanged; only the seeding/fallback data changed.

Webview (apps/vscode/webview-ui/):
  - `useNormalizedApiConfiguration` always routes through gRPC
    `resolveModelInfo`. Removed the `isMigratedSdkProvider` /
    `MIGRATED_SDK_PROVIDER_IDS` feature flag and the legacy
    `normalizeApiConfiguration` switch entirely.
  - New `useStaticProviderSelection` hook for the 22 settings
    components whose catalog is now SDK-driven, and
    `useDynamicProviderSelection` for the 12 dynamic-list pickers
    (openrouter, cline, openai-compatible, ollama, lmstudio, requesty,
    litellm, hicap, groq, baseten, huggingface, vercel-ai-gateway,
    aihubmix, oca, huawei-cloud-maas, dify, fireworks, together,
    vscode-lm) so all of them stop calling the legacy switch.
  - `ModelInfoView` reads its `isGemini` check via
    `useProviderModels("gemini")` instead of importing `geminiModels`.
  - `App.stories.tsx` ships a small inline fixture instead of
    importing `bedrockModels`.
  - `ExtensionStateContext` no longer seeds `groqModelsState` /
    `basetenModelsState` from the deleted catalog; the slices start
    empty and the SDK-curated catalog is layered in by the pickers at
    render time.

Misc:
  - `src/utils/model-utils.ts`: `isAnthropicModelId` consults
    `MODEL_COLLECTIONS_BY_PROVIDER_ID["anthropic"]` instead of the
    deleted `anthropicModels` map.
  - `src/shared/storage/provider-keys.ts`: `getProviderDefaultModelId`
    no longer hard-codes 24 per-provider defaults. The function now
    consults the SDK catalog and only keeps an override map for
    providers whose default is intentionally not the SDK default
    (openrouter-shared dynamic providers and local-only providers).
  - `src/shared/openai-codex-models.ts`: relative path for
    `shape-adapter` import so both the extension and webview build
    contexts resolve it identically.

Tests retargeted to assert SDK behavior, not static-map shapes
--------------------------------------------------------------
  - claude-code, anthropic, bedrock, vertex, wandb, provider-keys
    test suites had assertions tied to deleted shapes. Rewrote them
    to either assert through the SDK catalog
    (anthropic compares against `adaptSdkModelInfo(sdkCollection.models[id])`,
    wandb uses the SDK-declared default, etc.) or focus on the
    host-side semantics (bedrock's "global endpoint" block now tests
    `vertexModelSupportsGlobalEndpoint` directly).
  - claude-code test trimmed its 8 `[1m]`/version-pin variants down
    to three SDK-shaped cases. The 8 deleted assertions exercised
    extension-only model-id derivations that the SDK does not carry;
    matching the CLI's behavior was the explicit goal.
  - resolveModelInfo test rewritten around the new peek -> await
    -> unknown contract.
  - proto-lint: added missing `go_package` option to
    `proto/cline/remote_config.proto`.

Verification (npm scripts under apps/vscode/):
  - npm run protos          OK
  - npm run check-types     OK  (apps/vscode + apps/vscode/webview-ui)
  - npm run lint            OK  (biome + proto-lint)
  - npm run build:webview   OK  (tsc -b && vite build)
  - node esbuild.mjs        OK  (dist/extension.js produced)
  - Runtime smoke test: 27/27 provider collections resolve from the
    SDK with correct model counts, defaults, and usage-cost-display
    flags. openai-codex returns cost=hide as expected; every other
    provider returns cost=show.
2026-06-03 09:56:34 -07:00
Max Paulus 🥪 3d8a849f03 fix soft-lock on auth fail retry 2026-06-03 09:55:38 -07:00
Max Paulus 🥪 e7e0e2b559 fix sesion usubscriptions 2026-06-03 09:55:13 -07:00
Max Paulus 🥪 695492a97b instead of listHistory, use host.get(sessionId) instead 2026-06-03 09:55:13 -07:00
Dominic Cooney 7460d460ac sdk migration: squashed pre-2026-05-27 work
Squashed foundational SDK-migration work older than one week (author dates
up to 2026-05-26), combining the previous "squashed pre-2026-05-22 work"
base commit with subsequent older commits:

- sdk migration base (pre-2026-05-22 squash)
- fix(mcp): accept CLI-authored nested transport format, preserve oauth/metadata, improve schema error messages
- add telemetry to sdk extension
- improve task startup perf
- harden perf improvements
- remove timing code
- chore: fix lint and format on the vscode app
- fix: declare missing direct dependencies in apps/vscode
- fix integration tests
- fix(test): stub telemetry helpers in unit-test @cline/core mock
- ci: run publish-nightly job inside apps/vscode workspace
- remove old md files
- remove outdated samples
- step one for removing src/core/api folder
2026-06-03 09:55:13 -07:00
745 changed files with 52138 additions and 113741 deletions
+128
View File
@@ -0,0 +1,128 @@
# Debug Harness
HTTP-controlled debugger for the VSCode extension at `src/dev/debug-harness/server.ts`.
## Quick start
```bash
# Build extension first if needed (protos + esbuild):
npm run protos && IS_DEV=true node esbuild.mjs
# Launch (skip-build if already built):
npx tsx src/dev/debug-harness/server.ts --skip-build --auto-launch
# In another terminal:
curl localhost:19229/api -d '{"method":"status"}'
```
## Data Isolation
The debugee runs with `CLINE_DIR=~/.cline2` by default, separate from your real `~/.cline`.
This prevents the debugee's logout from logging out the debugger, and vice versa.
Override with `--cline-dir /tmp/test-dir`. Check with `status()``clineDir`.
## Browser Capture & OAuth
The debugee runs with `CLINE_CAPTURE_BROWSER=1`, which intercepts `openExternal()` in
`src/utils/env.ts`. URLs are captured instead of opening a real browser:
- Logged to `$CLINE_DIR/data/debug-captured-urls.jsonl`
- POSTed in real-time to `/captured-url` on the harness server
- Queryable via `oauth.captured_urls`
### OAuth API
- **`oauth.captured_urls`** `{clear?}` — URLs the debugee tried to open
- **`oauth.read_stored_token`** — Check auth token presence in secrets.json
- **`oauth.simulate_callback`** `{path, code?, state?, provider?, token?}` — Build vscode:// callback URI
- **`oauth.read_captured_urls_file`** — Read on-disk JSONL of captured URLs
### OAuth testing flow
For **Cline OAuth** (SDK local callback): The SDK starts a local HTTP server, the auth URL
is captured. To complete: open the captured URL in a real browser (it redirects back to the
SDK's callback server), OR extract the callback port and `curl http://127.0.0.1:PORT/callback?code=...`.
For **MCP/Provider OAuth** (vscode:// URI): The redirect goes to a vscode:// URI.
`oauth.simulate_callback` only *builds* the URI — it does not deliver it, and the ESM
extension host can't `require()` the handler. To actually deliver the callback, call the
debug-only hook via `ext.evaluate` (with `awaitPromise: true`):
`globalThis.__clineHandleUri("vscode://saoudrizwan.claude-dev/...?code=...&state=...")`.
It runs the same `SharedUriHandler.handleUri` as VSCode's real URI handler and exists only
when `CLINE_CAPTURE_BROWSER` is set (the harness always sets it; never ships in prod).
For end-to-end MCP OAuth, get a real `code` from the local MCP OAuth test server
(`npm run dev:mcp-oauth-test-server`).
## Navigating Views — Use Commands, Not Clicks
Don't try to find/click small sidebar icons. Use VSCode commands via command palette.
Registered in `src/registry.ts`:
| Command | View |
|---------|------|
| `cline.accountButtonClicked` | Account / sign-in |
| `cline.historyButtonClicked` | Task history |
| `cline.settingsButtonClicked` | Settings |
| `cline.mcpButtonClicked` | MCP servers |
| `cline.plusButtonClicked` | New task (chat) |
| `cline.worktreesButtonClicked` | Worktrees |
```bash
curl localhost:19229/api -d '{"method":"ui.command_palette","params":{"command":"cline.accountButtonClicked"}}'
```
## Key commands
All via `POST localhost:19229/api` with `{"method":"...", "params":{...}}`:
- **`launch`** / **`shutdown`** — lifecycle
- **`ui.screenshot`** — screenshot to `/tmp/cline-debug/`; returns `{path}`**use `read_file` on the path to examine, do NOT `open` the file** (Preview.app covers the VSCode window)
- **`ui.open_sidebar`** — open the Cline sidebar
- **`ext.set_breakpoint`** `{file, line, condition?}` — breakpoint by source file (sourcemap-resolved)
- **`ext.evaluate`** `{expression, callFrameId?}` — eval in extension host
- **`ext.resume`** / **`ext.step_over`** / **`ext.step_into`** — stepping
- **`ext.call_stack`** — inspect when paused
- **`web.evaluate`** `{expression}` — eval in webview
- **`web.post_message`** `{message}` — send postMessage to extension host via exposed vsCodeApi
- **`wait_for_pause`** `{timeout?}` — block until breakpoint hit
- **`ui.locator`** `{role?, testId?, text?, frame?}` — Playwright locator (auto-retries on stale sidebar frame)
- **`ui.react_input`** `{text, selector?, clear?, submit?}` — set React textarea value via `execCommand('insertText')`; works reliably across multiple tasks
- **`ui.send_message`** `{text, images?, files?, responseType?}` — send chat message bypassing the textarea entirely (via gRPC postMessage)
- **`ui.command_palette`** `{command}` — run VSCode command
## Typical Session
```bash
# 1. Launch
curl localhost:19229/api -d '{"method":"launch","params":{"skipBuild":true}}'
# 2. Open sidebar + dismiss overlays (ALWAYS do this first)
curl localhost:19229/api -d '{"method":"ui.open_sidebar"}'
curl localhost:19229/api -d '{"method":"web.evaluate","params":{"expression":"document.querySelectorAll(\".sr-only\").forEach(el => el.parentElement?.click())"}}'
# 3. Navigate to view
curl localhost:19229/api -d '{"method":"ui.command_palette","params":{"command":"cline.accountButtonClicked"}}'
# 4. Check captured OAuth URLs if testing auth
curl localhost:19229/api -d '{"method":"oauth.captured_urls"}'
# 5. Verify
curl localhost:19229/api -d '{"method":"ui.screenshot"}'
```
## Caveats
- **⚠️ Dismiss promotional overlays FIRST**: On fresh launches, full-screen promo overlays block the sidebar. **Dismiss immediately after `ui.open_sidebar`**, before any other interaction or screenshot. May need to run twice:
```bash
curl localhost:19229/api -d '{"method": "ui.open_sidebar"}'
curl localhost:19229/api -d '{"method": "web.evaluate", "params": {"expression": "document.querySelectorAll(\".sr-only\").forEach(el => el.parentElement?.click())"}}'
```
- **Screenshots — don't open the file**: `ui.screenshot` and `ui.sidebar_screenshot` save PNGs to `/tmp/cline-debug/` and return the `{path}`. Use `read_file` on that path to examine screenshots. Running `open <path>` launches Preview.app on macOS which covers the VSCode window.
- **Scripts count = 0 after launch**: CDP connects after extension host starts, so scripts parsed during startup aren't tracked. Breakpoints still work via sourcemap resolution.
- **Port 9230**: Extension host inspector. If another VSCode instance uses this port, the harness will fail to connect. Kill other debug instances first.
- **macOS only** for now (Playwright Electron launch behavior).
- **Webview CDP**: `connect_webview` may fail depending on Electron version. `web.evaluate` still works via Playwright's `frame.evaluate()` fallback.
- **Sourcemap paths**: esbuild outputs relative paths like `../src/extension.ts` in the sourcemap. The resolver handles this, but if a file isn't found, use `ext.source_files` to see exact paths.
- **OAuth with fake codes**: Browser capture intercepts the URL but doesn't provide a valid auth code. For real OAuth testing, open the captured URL in a browser. For unit testing, mock the token exchange.
See `src/dev/debug-harness/README.md` for full API reference.
+89 -87
View File
@@ -13,11 +13,55 @@ This file is the secret sauce for working effectively in this codebase. It captu
**What NOT to add:** Stuff you can figure out from reading a few files, obvious patterns, or standard practices. This file should be high-signal, not comprehensive.
## Miscellaneous
- Avoid provider-specific string matching / hardcoded provider branches when fixing provider/config plumbing. Prefer provider metadata, shared catalog/defaults, explicit protocol/client capabilities, or centralized normalization utilities that apply by data shape rather than `providerId === "..."`. If a provider exception seems necessary, stop and explain why instead of adding ad-hoc string matching.
- This is a VS Code extension—check `package.json` for available scripts before trying to verify builds (e.g., `npm run compile`, not `npm run build`).
- When creating PRs, contributors should not create changelog-entry files. Maintainers handle release versioning and changelog curation during the release process.
- When adding new feature flags, see this PR as a reference https://github.com/cline/cline/pull/7566
- Additional instructions about making requests: @.clinerules/network.md
## Searching the Codebase — Avoiding Build Output
Several directories contain build output or generated code that produces
noisy or unusable results with `search_files` / `grep`:
| Directory | What it is | Why it's a problem |
|-----------|-----------|-------------------|
| `out/` | esbuild bundle output | Mirrors `src/` structure as minified JS — every search gets duplicate hits on single-line files |
| `dist/` | Packaged extension | Entire extension bundled into one minified `extension.js` (~1 long line) |
| `dist-standalone/` | Standalone build output | Same minification issue |
| `src/generated/` | Generated protobuf code | Auto-generated from `proto/`; not the source of truth |
| `src/shared/proto/` | Generated proto type defs | Auto-generated from `proto/`; not the source of truth |
| `node_modules/` | Dependencies | Huge, not project source |
### How to skip build output
**`search_files`** — Point at `src/` (not the project root) and use `file_pattern`:
```
search_files(path="src/core", regex="myFunction", file_pattern="*.ts")
```
The `file_pattern` parameter is the most effective filter — e.g. `"*.ts"`,
`"*.tsx"`, `"*.proto"`.
**`grep` directly** — Exclude build dirs and restrict to source extensions:
```bash
grep -rn "myFunction" src/ --include="*.ts" --exclude-dir={out,dist,node_modules,generated}
```
### When you must search minified files
Sometimes you need to verify what got bundled (e.g., checking if a change
made it into the build). Minified files are typically one long line, so
normal `grep` shows the entire file as context. Use these approaches:
- **`grep -oP`** to extract just the match with limited surrounding context:
```bash
grep -oP '.{0,40}myFunction.{0,40}' dist/extension.js
```
- **`read_file`** on files in `out/src/` — these have source maps and are
more readable than `dist/extension.js` (which is the fully bundled output).
- **Source maps** — `out/src/*.js.map` and `dist/extension.js.map` can be
used to trace minified output back to original source locations.
## gRPC/Protobuf Communication
The extension and webview communicate via gRPC-like protocol over VS Code message passing.
@@ -48,93 +92,6 @@ The extension and webview communicate via gRPC-like protocol over VS Code messag
- `src/core/controller/task/explainChanges.ts` - Handler implementation
- `webview-ui/src/components/chat/ChatRow.tsx` - UI rendering
## Adding a New API Provider
When adding a new provider (e.g., "openai-codex"), you must update the proto conversion layer in THREE places or the provider will silently reset to Anthropic:
1. `proto/cline/models.proto` - Add to the `ApiProvider` enum (e.g., `OPENAI_CODEX = 40;`)
2. `convertApiProviderToProto()` in `src/shared/proto-conversions/models/api-configuration-conversion.ts` - Add case mapping string to proto enum
3. `convertProtoToApiProvider()` in the same file - Add case mapping proto enum back to string
**Why this matters:** Without these, the provider string hits the `default` case and returns `ANTHROPIC`. The webview, provider list, and handler all work fine, but the state silently resets when it round-trips through proto serialization. No error is thrown.
**Other files to update when adding a provider:**
- `src/shared/api.ts` - Add to `ApiProvider` union type, define models
- `src/shared/providers/providers.json` - Add to provider list for dropdown
- `src/core/api/index.ts` - Register handler in `createHandlerForProvider()`
- `webview-ui/src/components/settings/utils/providerUtils.ts` - Add cases in `getModelsForProvider()` and `normalizeApiConfiguration()`
- `webview-ui/src/utils/validate.ts` - Add validation case
- `webview-ui/src/components/settings/ApiOptions.tsx` - Render provider component
## Responses API Providers (OpenAI Codex, OpenAI Native)
Providers using OpenAI's Responses API require native tool calling. XML tools don't work with the Responses API.
**Symptoms of broken native tool calling:**
- Tools get called multiple times (e.g., `ask_followup_question` asks the same question twice)
- Tool arguments get duplicated or malformed
- The model responds but tools aren't recognized
**Root causes to check:**
1. **Provider missing from `isNextGenModelProvider()`** in `src/utils/model-utils.ts`. The native variant matchers (e.g., `native-gpt-5/config.ts`) call this function. If your provider isn't in the list, the matcher returns false and falls back to XML tools.
2. **Model missing `apiFormat: ApiFormat.OPENAI_RESPONSES`** in its model info (`src/shared/api.ts`). This property signals that the model requires native tool calling. The task runner in `src/core/task/index.ts` checks this and forces `enableNativeToolCalls: true` regardless of user settings.
**When adding a new Responses API provider:**
1. Add provider to `isNextGenModelProvider()` list in `src/utils/model-utils.ts`
2. Set `apiFormat: ApiFormat.OPENAI_RESPONSES` on all models that use the Responses API
3. The variant matcher and task runner will handle the rest automatically
## Adding Tools to System Prompt
This is tricky—multiple prompt variants and configs. **Always search for existing similar tools first and follow their pattern.** Look at the full chain from prompt definition → variant configs → handler → UI before implementing.
1. **Add to `ClineDefaultTool` enum** in `src/shared/tools.ts`
2. **Tool definition** in `src/core/prompts/system-prompt/tools/` (create file like `generate_explanation.ts`)
- Define variants for each `ModelFamily` (generic, next-gen, xs, etc.)
- Export variants array (e.g., `export const my_tool_variants = [GENERIC, NATIVE_NEXT_GEN, XS]`)
- **Fallback behavior**: If a variant isn't defined for a model family, `ClineToolSet.getToolByNameWithFallback()` automatically falls back to GENERIC. So you only need to export `[GENERIC]` unless the tool needs model-specific behavior.
3. **Register in `src/core/prompts/system-prompt/tools/init.ts`** - Import and spread into `allToolVariants`
4. **Add to variant configs** - Each model family has its own config in `src/core/prompts/system-prompt/variants/*/config.ts`. Add your tool's enum to the `.tools()` list:
- `generic/config.ts`, `next-gen/config.ts`, `gpt-5/config.ts`, `native-gpt-5/config.ts`, `native-gpt-5-1/config.ts`, `native-next-gen/config.ts`, `gemini-3/config.ts`, `glm/config.ts`, `hermes/config.ts`, `xs/config.ts`
- **Important**: If you add to a variant's config, make sure the tool spec exports a variant for that ModelFamily (or relies on GENERIC fallback)
5. **Create handler** in `src/core/task/tools/handlers/`
6. **Wire up in `ToolExecutor.ts`** if needed for execution flow
7. **Add to tool parsing** in `src/core/assistant-message/index.ts` if needed
8. **If tool has UI feedback**: add `ClineSay` enum in proto, update `src/shared/ExtensionMessage.ts`, update `src/shared/proto-conversions/cline-message.ts`, update `webview-ui/src/components/chat/ChatRow.tsx`
## Modifying System Prompt
**Read these first:** `src/core/prompts/system-prompt/README.md`, `tools/README.md`, `__tests__/README.md`
System prompt is modular: **components** (reusable sections) + **variants** (model-specific configs) + **templates** (with `{{PLACEHOLDER}}` resolution).
**Key directories:**
- `components/` - Shared sections: `rules.ts`, `capabilities.ts`, `editing_files.ts`, etc.
- `variants/` - Model-specific: `generic/`, `next-gen/`, `xs/`, `gpt-5/`, `gemini-3/`, `hermes/`, `glm/`, etc.
- `templates/` - Template engine and placeholder definitions
**Variant tiers (ask user which to modify):**
- **Next-gen** (Claude 4, GPT-5, Gemini 2.5): `next-gen/`, `native-next-gen/`, `native-gpt-5/`, `native-gpt-5-1/`, `gemini-3/`, `gpt-5/`
- **Standard** (default fallback): `generic/`
- **Local/small models**: `xs/`, `hermes/`, `glm/`
**How overrides work:** Variants can override components via `componentOverrides` in their `config.ts`, or provide a custom template in `template.ts` (e.g., `next-gen/template.ts` exports `rules_template`). If no override, the shared component from `components/` is used.
**Example: Adding a rule to RULES section**
1. Check if variant overrides rules: look for `rules_template` in `variants/*/template.ts` or `componentOverrides.RULES` in `config.ts`
2. If shared: modify `components/rules.ts`
3. If overridden: modify that variant's template
4. XS variant is special—has heavily condensed inline content in `template.ts`
**After any changes, regenerate snapshots:**
```bash
UPDATE_SNAPSHOTS=true npm run test:unit
```
Snapshots live in `__tests__/__snapshots__/`. Tests validate across model families and context variations (browser, MCP, focus chain).
## Modifying Default Slash Commands
Three places need updates:
- `src/core/slash-commands/index.ts` - Command definitions
- `src/core/prompts/commands.ts` - System prompt integration
- `webview-ui/src/utils/slash-commands.ts` - Webview autocomplete
## Adding New Global State Keys
Adding a new key to global state requires updates in multiple places. Missing any step causes silent failures.
@@ -203,3 +160,48 @@ const isGenerating = explanationInfo.status === "generating" && !wasCancelled
**See also:** `BrowserSessionRow.tsx` uses similar pattern with `isLastApiReqInterrupted` and `isLastMessageResume`.
**Backend side:** When streaming is cancelled, clean up properly (close tabs, clear comments, etc.) by checking `taskState.abort` after the streaming function returns.
## Debug Harness: clear inherited VSCode/Electron env vars before launching
The debug harness (`apps/vscode/src/dev/debug-harness/server.ts`) launches a child
VSCode via Playwright's `_electron.launch({ env: { ...process.env, ... } })`. If you
run the harness from a process that was itself spawned by VSCode (e.g. the Cline
extension host, an integrated terminal, or an agent running inside VSCode), the
parent's VSCode/Electron env vars leak into the child and break the launch.
The fatal one is **`ELECTRON_RUN_AS_NODE=1`**: it makes the child VSCode binary run
as plain Node, so it rejects every VSCode CLI flag. Symptom:
```
.../Visual Studio Code.app/Contents/MacOS/Code: bad option: --extensionDevelopmentPath=...
Error: Process failed to launch! (Playwright _electron.launch)
```
This is NOT the macOS Playwright flakiness mentioned in the harness README — it's
env inheritance. Fix: strip the inherited vars before starting the harness:
```bash
env -u ELECTRON_RUN_AS_NODE -u ELECTRON_NO_ATTACH_CONSOLE \
-u VSCODE_CLI -u VSCODE_CODE_CACHE_PATH -u VSCODE_CRASH_REPORTER_PROCESS_TYPE \
-u VSCODE_CWD -u VSCODE_ESM_ENTRYPOINT -u VSCODE_HANDLES_UNCAUGHT_ERRORS \
-u VSCODE_IPC_HOOK -u VSCODE_NLS_CONFIG -u VSCODE_PID -u VSCODE_L10N_BUNDLE_LOCATION \
npx tsx src/dev/debug-harness/server.ts --auto-launch --skip-build
```
Check your own env with `env | grep -iE 'electron|vscode_'` first; `ELECTRON_RUN_AS_NODE=1`
present means you must scrub before launching.
Other harness notes confirmed in practice:
- The extension host is **ESM** (`VSCODE_ESM_ENTRYPOINT`), so `ext.evaluate` has no
`require` and module-internal functions aren't reachable as globals. To inspect
internal builders (e.g. `buildBedrockProviderConfig`), set a breakpoint with
`ext.set_breakpoint` and read locals via `ext.evaluate` with the paused `callFrameId`
— don't try to `require()` the bundle.
- `web.evaluate` wraps the expression as a single returned expression; multi-statement
snippets must be an IIFE `(() => { ...; return x; })()`, otherwise you get
`SyntaxError: Unexpected token ';'`.
- Webview settings inputs are `vscode-text-field` web components with debounced React
onChange. Setting `.value` + dispatching events via `web.evaluate` is unreliable for
some fields; focus the inner shadow `input` then use real keystrokes (`ui.type` +
`ui.press Tab`, or click the dropdown option) to make the value persist.
+26
View File
@@ -0,0 +1,26 @@
# SDK Adapter
The VSCode extension runs on the Cline SDK (`@cline/core`, `@cline/llms`,
`@cline/shared`) through an adapter layer in `apps/vscode/src/sdk/`. The
webview still talks gRPC; the adapter translates between gRPC handlers and SDK
calls. See `apps/vscode/src/dev/debug-harness/README.md` for the debug harness.
## Conventions
1. **Look up SDK APIs, don't guess.** Use `kb_search(name="sdk", query="...")`
before implementing against an SDK surface.
2. **Reference the pre-SDK implementation when replacing a module.** Add a
`// Replaces classic src/core/... (see origin/main)` header and use
`kb_search(name="cline", commit="origin/main")` or
`git show origin/main:path` to consult the prior implementation.
3. **Single entry point.** There is one codepath — the SDK adapter. No
`CLINE_SDK` env flag.
4. **Use `{appBaseUrl}`**, never hardcode `app.cline.bot`.
5. **Avoid `as` casts.** Use explicit conversion functions with tests. The
branded types in `apps/vscode/src/sdk/model-catalog/contracts.ts` exist so
casts are unnecessary outside parse/compute boundaries.
## Debug harness
- **Dismiss the Kanban/promo overlay** before any debug harness interaction.
- **Use the command palette** to navigate tabs in the debug harness.
@@ -31,6 +31,12 @@ jobs:
if: github.repository == 'cline/cline' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dpc/sdk-migration-simpler-login')
runs-on: ubuntu-latest
environment: PublishNightly
# The VS Code extension's package.json and lockfiles live under apps/vscode/
# (the repo root has no package.json). Mirror ext-vscode-test.yml so install
# and publish steps run in the correct workspace.
defaults:
run:
working-directory: apps/vscode
steps:
- name: Checkout selected branch
@@ -53,6 +59,10 @@ jobs:
# Keep publish environment aligned with test workflow/tooling lockfile expectations.
# Newer LTS (Node 24 / npm 11) can make `npm list` fail with ELSPROBLEMS during vsce packaging.
node-version: 22
cache: 'npm'
cache-dependency-path: |
apps/vscode/package-lock.json
apps/vscode/webview-ui/package-lock.json
- name: Install root dependencies
run: npm ci --include=optional
+12 -7
View File
@@ -92,12 +92,12 @@ jobs:
apps/vscode/webview-ui/package-lock.json
- name: Install root dependencies
run: npm ci
run: npm ci --include=optional
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
run: cd webview-ui && npm ci
run: cd webview-ui && npm ci --include=optional
- name: Run Quality Checks (Parallel)
run: npm run ci:check-all
@@ -131,12 +131,12 @@ jobs:
apps/vscode/webview-ui/package-lock.json
- name: Install root dependencies
run: npm ci
run: npm ci --include=optional
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
run: cd webview-ui && npm ci
run: cd webview-ui && npm ci --include=optional
- name: Set up NPM on Windows
if: runner.os == 'Windows'
@@ -155,6 +155,11 @@ jobs:
id: build_step
run: npm run ci:build
- name: Vitest Suites (SDK adapter + model catalog)
id: vitest_tests
if: ${{ !cancelled() && steps.build_step.outcome == 'success' }}
run: npm run test:vitest
- name: Unit Tests with coverage - Linux
id: unit_tests_linux
if: ${{ !cancelled() && steps.build_step.outcome == 'success' && runner.os == 'Linux' }}
@@ -230,12 +235,12 @@ jobs:
apps/vscode/testing-platform/package-lock.json
- name: Install root dependencies
run: npm ci
run: npm ci --include=optional
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
run: cd webview-ui && npm ci
run: cd webview-ui && npm ci --include=optional
- name: Download ripgrep binaries
run: npm run download-ripgrep
@@ -244,7 +249,7 @@ jobs:
run: npm run compile-standalone
- name: Install testing platform dependencies
run: cd testing-platform && npm ci
run: cd testing-platform && npm ci --include=optional
- name: Running testing platform integration spec tests
timeout-minutes: 7
+3
View File
@@ -13,6 +13,9 @@ tmp
pnpm-lock.yaml
.clineignore
.cline/enterprise
.cline/remote-config
**/.cline/remote-config
.venv
.actrc
CLAUDE.local.md
+2 -1
View File
@@ -7,4 +7,5 @@ fi
gitleaks git --pre-commit --redact --staged --verbose || exit 1
lint-staged
cd apps/vscode && lint-staged
-1
View File
@@ -1,2 +1 @@
@.clinerules/general.md
@.clinerules/network.md
-1
View File
@@ -26,7 +26,6 @@ test.describe("root flag descriptions", () => {
"Set reasoning effort level",
"consecutive mistakes",
"Output messages as JSON",
"ACP",
"Check for updates and install if available",
"Run the kanban app",
]);
+7
View File
@@ -6,6 +6,13 @@
"src/**/__tests__/*.ts",
"src/test/services/**/*.test.ts"
],
"ignore": [
"src/core/controller/models/__tests__/providerCatalogHandlers.test.ts",
"src/core/controller/models/__tests__/providerCatalogSmoke.test.ts",
"src/core/controller/models/__tests__/providerSwitchNormalization.test.ts",
"src/core/controller/models/__tests__/resolveModelInfo.test.ts",
"src/core/controller/models/__tests__/refreshClineRecommendedModels.test.ts"
],
"require": [
"ts-node/register",
"source-map-support/register",
+5 -1
View File
@@ -1,9 +1,13 @@
import { defineConfig } from "@vscode/test-cli"
import path from "path"
const vscodeTestVersion = process.env.VSCODE_TEST_VERSION ?? "stable"
export default defineConfig({
files: "{out/**/*.test.js,src/**/*.test.js,!src/test/e2e/**/*.test.js,!out/src/test/e2e/**/*.test.js}",
files: [
"out/src/{core,test,utils,shared,integrations,hosts,services}/**/*.test.js",
"src/{core,test,utils,shared,integrations,hosts,services}/**/*.test.js",
],
mocha: {
ui: "bdd",
timeout: 20000, // Maximum time (in ms) that a test can run before failing
+1 -39
View File
@@ -21,6 +21,7 @@ const aliasResolverPlugin = {
const aliases = {
"@": path.resolve(__dirname, "src"),
"@core": path.resolve(__dirname, "src/core"),
"@generated": path.resolve(__dirname, "src/generated"),
"@integrations": path.resolve(__dirname, "src/integrations"),
"@services": path.resolve(__dirname, "src/services"),
"@shared": path.resolve(__dirname, "src/shared"),
@@ -85,44 +86,6 @@ const esbuildProblemMatcherPlugin = {
},
}
const copyWasmFiles = {
name: "copy-wasm-files",
setup(build) {
build.onEnd(() => {
// tree sitter
const sourceDir = path.join(__dirname, "node_modules", "web-tree-sitter")
const targetDir = path.join(__dirname, destDir)
// Copy tree-sitter.wasm
fs.copyFileSync(path.join(sourceDir, "tree-sitter.wasm"), path.join(targetDir, "tree-sitter.wasm"))
// Copy language-specific WASM files
const languageWasmDir = path.join(__dirname, "node_modules", "tree-sitter-wasms", "out")
const languages = [
"typescript",
"tsx",
"python",
"rust",
"javascript",
"go",
"cpp",
"c",
"c_sharp",
"ruby",
"java",
"php",
"swift",
"kotlin",
]
languages.forEach((lang) => {
const filename = `tree-sitter-${lang}.wasm`
fs.copyFileSync(path.join(languageWasmDir, filename), path.join(targetDir, filename))
})
})
},
}
const buildEnvVars = {
"import.meta.url": "_importMetaUrl",
"process.env.IS_STANDALONE": JSON.stringify(standalone ? "true" : "false"),
@@ -176,7 +139,6 @@ const baseConfig = {
define: buildEnvVars,
tsconfig: path.resolve(__dirname, "tsconfig.json"),
plugins: [
copyWasmFiles,
aliasResolverPlugin,
/* add to the end of plugins array */
esbuildProblemMatcherPlugin,
+14843 -14965
View File
File diff suppressed because it is too large Load Diff
+15 -15
View File
@@ -410,9 +410,12 @@
"test": "npx npm-run-all test:unit test:integration",
"test:integration": "vscode-test",
"test:unit": "cross-env TS_NODE_PROJECT=./tsconfig.unit-test.json mocha",
"test:vitest": "vitest run --config vitest.config.ts",
"test:vitest:watch": "vitest --config vitest.config.ts",
"test:coverage": "vscode-test --coverage",
"test:sca-server": "npx tsx watch scripts/test-standalone-core-api-server.ts",
"test:tp-orchestrator": "npx tsx scripts/testing-platform-orchestrator.ts",
"dev:mcp-oauth-test-server": "npx tsx src/dev/mcp-oauth-test-server/server.ts",
"e2e": "playwright test -c playwright.config.ts",
"test:e2e:build": "vsce package --allow-package-secrets sendgrid --out dist/e2e.vsix",
"test:e2e": "playwright install && npm run test:e2e:build && node src/test/e2e/utils/build.mjs && playwright test",
@@ -482,27 +485,28 @@
"rimraf": "^6.0.1",
"should": "^13.2.3",
"sinon": "^21.0.3",
"tar": "^7.5.2",
"tree-kill": "^1.2.2",
"ts-node": "^10.9.2",
"ts-proto": "^2.6.1",
"tsconfig-paths": "^4.2.0",
"typescript": "^5.4.5"
"typescript": "^5.4.5",
"vitest": "^4.0.17"
},
"dependencies": {
"@anthropic-ai/sdk": "^0.37.0",
"@anthropic-ai/vertex-sdk": "^0.6.4",
"@aws-sdk/client-bedrock-runtime": "^3.922.0",
"@aws-sdk/credential-providers": "^3.922.0",
"@azure/identity": "^4.13.0",
"@bufbuild/protobuf": "^2.2.5",
"@cerebras/cerebras_cloud_sdk": "^1.35.0",
"@google-cloud/vertexai": "^1.9.3",
"@cline/agents": "^0.0.42",
"@cline/core": "^0.0.42",
"@cline/llms": "^0.0.42",
"@cline/shared": "^0.0.42",
"@google/genai": "^1.30.0",
"@grpc/grpc-js": "^1.9.15",
"@grpc/proto-loader": "^0.7.13",
"@grpc/reflection": "^1.0.4",
"@mistralai/mistralai": "^1.5.0",
"@modelcontextprotocol/sdk": "^1.25.1",
"@opentelemetry/api": "^1.9.0",
"@opentelemetry/api-logs": "^0.56.0",
"@opentelemetry/core": "^2.1.0",
"@opentelemetry/exporter-logs-otlp-grpc": "^0.56.0",
"@opentelemetry/exporter-logs-otlp-http": "^0.56.0",
@@ -522,9 +526,6 @@
"@opentelemetry/sdk-trace-node": "^1.30.1",
"@opentelemetry/semantic-conventions": "^1.37.0",
"@playwright/test": "^1.55.1",
"@sap-ai-sdk/ai-api": "^2.7.0",
"@sap-ai-sdk/orchestration": "^2.7.0",
"@sap-cloud-sdk/connectivity": "^4.6.0",
"@streamparser/json": "^0.0.22",
"@tailwindcss/vite": "^4.1.14",
"@types/uuid": "^10.0.0",
@@ -550,13 +551,14 @@
"ignore": "^7.0.3",
"image-size": "^2.0.2",
"isbinaryfile": "^5.0.2",
"js-yaml": "^4.1.1",
"jschardet": "^3.1.4",
"json5": "^2.2.3",
"jwt-decode": "^4.0.0",
"mammoth": "^1.11.0",
"nanoid": "^5.1.6",
"nice-grpc": "^2.1.12",
"node-machine-id": "^1.1.12",
"ollama": "^0.5.13",
"open": "^10.1.2",
"open-graph-scraper": "^6.9.0",
"openai": "^6.21.0",
@@ -575,15 +577,13 @@
"simple-git": "3.36.0",
"strip-ansi": "^7.1.2",
"tailwindcss": "^4.1.14",
"tree-sitter-wasms": "^0.1.11",
"ts-morph": "^25.0.1",
"turndown": "^7.2.0",
"ulid": "^2.4.0",
"undici": "^7.26.0",
"uuid": "^11.1.0",
"vscode-uri": "^3.1.0",
"web-tree-sitter": "^0.22.6",
"zod": "^3.24.2"
"zod": "^4.3.6"
},
"overrides": {
"tar-fs": ">=3.1.1",
+123 -2
View File
@@ -21,8 +21,6 @@ service ModelsService {
rpc refreshOpenRouterModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns recommended and free Cline models
rpc refreshClineRecommendedModelsRpc(EmptyRequest) returns (ClineRecommendedModelsResponse);
// Refreshes and returns Cline provider models
rpc refreshClineModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns Hugging Face models
rpc refreshHuggingFaceModels(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns OpenAI models
@@ -55,6 +53,18 @@ service ModelsService {
rpc getAihubmixModels(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns Vercel AI Gateway models
rpc refreshVercelAiGatewayModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Lists providers available from the unified SDK-backed catalog
rpc listProviders(Empty) returns (ProviderListingsResponse);
// Resolves model metadata for a provider through the unified SDK-backed catalog
rpc resolveProviderModels(ResolveProviderModelsRequest) returns (ProviderModelsResponse);
// Resolves model metadata for a provider/model without refreshing model lists
rpc resolveModelInfo(ResolveModelInfoRequest) returns (ResolveModelInfoResponse);
// Reads redacted effective provider configuration
rpc readProviderConfig(StringRequest) returns (ProviderConfigResponse);
// Writes provider configuration fields and returns redacted effective configuration
rpc writeProviderConfig(WriteProviderConfigRequest) returns (ProviderConfigResponse);
// Commits a mode-specific model selection atomically with its model metadata
rpc commitModelSelection(CommitModelSelectionRequest) returns (Empty);
}
// List of VS Code LM models
@@ -117,6 +127,117 @@ message OpenRouterCompatibleModelInfo {
map<string, OpenRouterModelInfo> models = 1;
}
// Lightweight provider entry for the top-level model/provider picker.
// Does not include the full model list; use resolveProviderModels for models.
message ProviderListing {
string id = 1;
string name = 2;
optional string default_model_id = 3;
optional string family = 4;
optional string protocol = 5;
optional string auth_description = 6;
optional string base_url_description = 7;
bool allows_custom_model_ids = 8;
// SDK-driven hint for cost display. Values: "show" (default) or "hide".
// Sourced from `resolveProviderUsageCostDisplay(provider.metadata)` in
// `@cline/llms`. When "hide", consumers must suppress per-token pricing
// and total cost displays (matches the CLI's `shouldShowCliUsageCost`).
string usage_cost_display = 11;
}
message ProviderListingsResponse {
repeated ProviderListing providers = 1;
}
message ResolveProviderModelsRequest {
string provider_id = 1;
bool force_refresh = 2;
optional string request_id = 3;
}
message CatalogErrorInfo {
string kind = 1;
string message = 2;
optional string code = 3;
optional bool retryable = 4;
}
message ProviderModelsResponse {
string provider_id = 1;
string request_id = 2;
string config_fingerprint = 3;
int64 fetched_at = 4;
bool ok = 5;
map<string, OpenRouterModelInfo> models = 6;
optional string default_model_id = 7;
optional string source = 8;
optional CatalogErrorInfo error = 9;
}
message ResolveModelInfoRequest {
string provider_id = 1;
optional string model_id = 2;
}
message ResolveModelInfoResponse {
string provider_id = 1;
string model_id = 2;
optional OpenRouterModelInfo model_info = 3;
string source = 4;
}
message ProviderConfigResponse {
string provider_id = 1;
optional string base_url = 2;
optional string api_line = 3;
map<string, string> headers = 4;
optional string region = 5;
int64 api_key_length = 6;
bool has_access_token = 7;
bool has_refresh_token = 8;
optional string account_id = 9;
optional CommittedModelSelection plan_selection = 10;
optional CommittedModelSelection act_selection = 11;
}
message CommittedModelSelection {
string provider_id = 1;
string model_id = 2;
OpenRouterModelInfo model_info = 3;
}
message ProviderReasoningPatch {
optional bool enabled = 1;
optional string effort = 2; // "none" | "low" | "medium" | "high" | "xhigh"
optional int32 budget_tokens = 3;
}
message WriteProviderConfigPatch {
optional string api_key = 1;
optional string base_url = 2;
map<string, string> headers = 3;
optional string region = 4;
optional string api_line = 5;
optional string access_token = 6;
optional string refresh_token = 7;
optional string account_id = 8;
optional ProviderReasoningPatch reasoning = 9;
optional bool clear_headers = 10;
}
message WriteProviderConfigRequest {
string provider_id = 1;
WriteProviderConfigPatch patch = 2;
}
message CommitModelSelectionRequest {
string provider_id = 1;
string mode = 2;
string model_id = 3;
OpenRouterModelInfo model_info = 4;
}
message ClineRecommendedModel {
string id = 1;
string name = 2;
@@ -0,0 +1,32 @@
syntax = "proto3";
package cline;
import "cline/common.proto";
option go_package = "github.com/cline/grpc-go/cline";
option java_multiple_files = true;
option java_package = "bot.cline.proto";
enum RemoteConfigType {
RULE = 0;
WORKFLOW = 1;
SKILL = 2;
}
message RemoteConfigSetting {
RemoteConfigType type = 1;
string name = 2;
string content = 3;
bool enabled = 4;
bool locked = 5;
}
message RemoteConfigSettingsResponse {
repeated RemoteConfigSetting settings = 1;
}
service RemoteConfigService {
rpc getRemoteConfigSettings(Empty) returns (RemoteConfigSettingsResponse);
rpc toggleRemoteConfigSetting(StringRequest) returns (RemoteConfigSetting);
}
+2 -4
View File
@@ -250,7 +250,6 @@ message Settings {
optional string default_terminal_profile = 137;
optional int32 terminal_output_line_limit = 138;
optional int32 max_consecutive_mistakes = 139;
optional bool strict_plan_mode_enabled = 141;
optional bool yolo_mode_toggled = 142;
optional bool use_auto_condense = 143;
optional bool cline_web_tools_enabled = 144;
@@ -286,7 +285,6 @@ message Settings {
optional string act_mode_cline_model_id = 180;
optional OpenRouterModelInfo act_mode_cline_model_info = 181;
optional bool show_feature_tips = 182;
optional bool lazy_teammate_mode_enabled = 183;
}
message State {
@@ -391,6 +389,7 @@ message UpdateSettingsRequest {
reserved 15; // was openai_reasoning_effort (moved to mode-scoped reasoning effort)
reserved 23; // was dictation_settings (dictation removed)
reserved 38; // was skills_enabled (removed - now always enabled)
reserved 43; // was lazy_teammate_mode_enabled (removed)
Metadata metadata = 1;
optional ModelsApiConfiguration api_configuration = 2;
@@ -405,7 +404,7 @@ message UpdateSettingsRequest {
optional int32 terminal_output_line_limit = 12;
optional PlanActMode mode = 13;
optional string preferred_language = 14;
optional bool strict_plan_mode_enabled = 16;
reserved 16; // was strict_plan_mode_enabled (removed)
optional FocusChainSettings focus_chain_settings = 17;
optional bool use_auto_condense = 18;
optional string custom_prompt = 19;
@@ -429,7 +428,6 @@ message UpdateSettingsRequest {
optional bool worktrees_enabled = 40;
optional bool double_check_completion_enabled = 41;
optional bool show_feature_tips = 42;
optional bool lazy_teammate_mode_enabled = 43;
}
message UpdateTerminalConnectionTimeoutRequest {
+3 -1
View File
@@ -82,12 +82,14 @@ message GetTaskHistoryRequest {
string search_query = 3;
string sort_by = 4;
bool current_workspace_only = 5;
int32 limit = 6;
int32 offset = 7;
}
// Response for task history
message TaskHistoryArray {
repeated TaskItem tasks = 1;
int32 total_count = 2;
bool has_more = 2;
}
// Task item details for history list
+6
View File
@@ -226,6 +226,12 @@ message ClineMessage {
ClineAskNewTask ask_new_task = 21;
ClineApiReqInfo api_req_info = 22;
ClineModelInfo model_info = 23;
// Convergent-replica fields (see webview-message-state-design.md):
// seq = monotonic freshness (higher seq wins for the same ts/identity)
// epoch = conversation/replica fence (older epoch is dropped by the webview)
int64 seq = 24;
int64 epoch = 25;
}
message ShowWebviewEvent {
+129
View File
@@ -0,0 +1,129 @@
// Dead-source finder: uses esbuild's own bundle reachability (the same analysis
// that drives tree-shaking + minification mangling) to compute which src/ files
// are reachable from BOTH shipped entry points:
// - src/extension.ts (VS Code extension host)
// - src/standalone/cline-core.ts (standalone host used by JetBrains + CLI)
//
// A src/*.ts file that is NOT in the union of metafile inputs for those two
// builds is unreachable from any shipped entry => dead (modulo dynamic import()
// of computed specifiers, which esbuild surfaces separately).
//
// Run: node scripts/find-dead-src.mjs
import fs from "node:fs"
import path from "node:path"
import { fileURLToPath } from "node:url"
import * as esbuild from "esbuild"
import { glob } from "glob"
const __dirname = path.dirname(fileURLToPath(import.meta.url))
const root = path.resolve(__dirname, "..")
const aliases = {
"@": path.join(root, "src"),
"@core": path.join(root, "src/core"),
"@integrations": path.join(root, "src/integrations"),
"@services": path.join(root, "src/services"),
"@shared": path.join(root, "src/shared"),
"@utils": path.join(root, "src/utils"),
"@packages": path.join(root, "src/packages"),
}
const aliasResolverPlugin = {
name: "alias-resolver",
setup(build) {
for (const [alias, aliasPath] of Object.entries(aliases)) {
const aliasRegex = new RegExp(`^${alias}($|/.*)`)
build.onResolve({ filter: aliasRegex }, (args) => {
const importPath = args.path.replace(alias, aliasPath)
const exts = [".ts", ".tsx", ".js", ".jsx"]
if (fs.existsSync(importPath)) {
const stats = fs.statSync(importPath)
if (stats.isDirectory()) {
for (const ext of exts) {
const idx = path.join(importPath, `index${ext}`)
if (fs.existsSync(idx)) return { path: idx }
}
} else {
return { path: importPath }
}
}
for (const ext of exts) {
if (fs.existsSync(`${importPath}${ext}`)) return { path: `${importPath}${ext}` }
}
return undefined
})
}
},
}
const common = {
bundle: true,
minify: false,
sourcemap: false,
logLevel: "silent",
format: "cjs",
platform: "node",
metafile: true,
write: false,
absWorkingDir: root,
tsconfig: path.join(root, "tsconfig.json"),
packages: "external",
plugins: [aliasResolverPlugin],
define: { "process.env.IS_DEV": "false", "process.env.IS_TEST": "false" },
banner: { js: "const _importMetaUrl=require('url').pathToFileURL(__filename)" },
}
async function inputsFor(entry, external) {
const r = await esbuild.build({ ...common, entryPoints: [entry], external })
return new Set(Object.keys(r.metafile.inputs).filter((f) => f.startsWith("src/") && /\.tsx?$/.test(f)))
}
const ext = await inputsFor("src/extension.ts", ["vscode"])
const standalone = await inputsFor("src/standalone/cline-core.ts", [
"vscode",
"@grpc/reflection",
"grpc-health-check",
"better-sqlite3",
])
const live = new Set([...ext, ...standalone])
// Third consumer: the webview (webview-ui/) is a separate Vite/React build that
// imports extension code ONLY from src/shared (via "@shared/*" alias or relative
// "../src/shared/*" paths). Any src/shared file referenced from webview-ui/src is
// therefore live even if the extension-host/standalone bundles don't reach it.
// Conservatively mark every src/shared file mentioned by the webview as live.
const webviewFiles = await glob("webview-ui/src/**/*.{ts,tsx}", { cwd: root })
const sharedMentionedByWebview = new Set()
for (const wf of webviewFiles) {
const text = fs.readFileSync(path.join(root, wf), "utf8")
// Match @shared/X or .../src/shared/X import specifiers and map to src/shared/X
const re = /(?:@shared\/|src\/shared\/)([A-Za-z0-9_./-]+)/g
let m
while ((m = re.exec(text))) {
const rel = m[1].replace(/\.(ts|tsx|js|jsx)$/, "")
for (const cand of [`src/shared/${rel}.ts`, `src/shared/${rel}.tsx`, `src/shared/${rel}/index.ts`]) {
if (fs.existsSync(path.join(root, cand))) sharedMentionedByWebview.add(cand)
}
}
}
for (const f of sharedMentionedByWebview) live.add(f)
console.log(`src/shared files referenced by webview: ${sharedMentionedByWebview.size}`)
// All non-test, non-.d.ts source files on disk.
const allSrc = (await glob("src/**/*.{ts,tsx}", { cwd: root }))
.filter((f) => !/\.test\.tsx?$/.test(f))
.filter((f) => !f.endsWith(".d.ts"))
.filter((f) => !f.includes("/__tests__/"))
.filter((f) => !f.startsWith("src/test/"))
.filter((f) => !f.startsWith("src/generated/")) // generated host glue
.filter((f) => !f.startsWith("src/dev/")) // dev-only tooling
const dead = allSrc.filter((f) => !live.has(f)).sort()
console.log(`extension inputs: ${ext.size}`)
console.log(`standalone inputs: ${standalone.size}`)
console.log(`union live src files: ${live.size}`)
console.log(`candidate dead files: ${dead.length}`)
fs.writeFileSync("/tmp/dead-src.json", JSON.stringify(dead, null, "\t"))
console.log("--- dead candidates written to /tmp/dead-src.json ---")
@@ -87,6 +87,12 @@ function createMockService<T extends grpc.UntypedServiceImplementation>(serviceN
})
return
case "openExternal":
simulateOAuthBrowserCallback(call.request?.value || "")
.then(() => callback(null, {}))
.catch((error) => callback(error))
return
case "getWebviewHtml":
callback(null, {
html: "<html><body>Fake Webview</body></html>",
@@ -143,6 +149,41 @@ function createMockService<T extends grpc.UntypedServiceImplementation>(serviceN
return new Proxy({} as T, handler)
}
async function simulateOAuthBrowserCallback(urlString: string): Promise<void> {
let url: URL
try {
url = new URL(urlString)
} catch {
return
}
if (!isLoopbackHost(url.hostname) || url.pathname !== "/api/v1/auth/authorize") {
return
}
const callbackUrl = url.searchParams.get("callback_url") ?? url.searchParams.get("redirect_uri")
if (!callbackUrl) {
return
}
const callback = new URL(callbackUrl)
if (!isLoopbackHost(callback.hostname) || callback.pathname !== "/auth") {
return
}
callback.searchParams.set("code", "test-personal-token")
callback.searchParams.set("provider", "cline")
const response = await fetch(callback.toString())
if (!response.ok) {
throw new Error(`Mock OAuth callback failed: ${response.status} ${response.statusText}`)
}
}
function isLoopbackHost(hostname: string): boolean {
return hostname === "localhost" || hostname === "127.0.0.1" || hostname === "::1"
}
if (require.main === module) {
startTestHostBridgeServer().catch((err) => {
console.error("Failed to start test host bridge server:", err)
@@ -115,7 +115,8 @@ async function main(): Promise<void> {
const covDir = path.join(projectRoot, `coverage/coverage-core-${PROTOBUS_PORT}`)
const baseArgs = ["--enable-source-maps", path.join(distDir, "cline-core.js")]
const workosFetchMockPath = path.join(projectRoot, "scripts", "testing-platform-workos-fetch-mock.cjs")
const baseArgs = ["--enable-source-maps", "--require", workosFetchMockPath, path.join(distDir, "cline-core.js")]
const spawnArgs = USE_C8 ? ["c8", "--report-dir", covDir, "node", ...baseArgs] : ["node", ...baseArgs]
@@ -0,0 +1,56 @@
// Preload used by the standalone testing platform.
// It makes the SDK WorkOS device-auth flow deterministic and fully local while
// leaving production auth code on the same device-auth path used by users.
const originalFetch = globalThis.fetch?.bind(globalThis)
const WORKOS_ORIGIN = "https://api.workos.com"
const DEVICE_CODE = "test-device-code"
const USER_CODE = "PTBC-TXTP"
const ACCESS_TOKEN = "test-personal-token"
const REFRESH_TOKEN = "test-personal-token_refresh"
function jsonResponse(body, init = {}) {
return new Response(JSON.stringify(body), {
status: init.status ?? 200,
headers: { "Content-Type": "application/json", ...(init.headers ?? {}) },
})
}
function inputUrl(input) {
if (typeof input === "string") return input
if (input instanceof URL) return input.toString()
if (input && typeof input === "object" && "url" in input) return input.url
return String(input)
}
globalThis.fetch = async (input, init) => {
const urlString = inputUrl(input)
let url
try {
url = new URL(urlString)
} catch {
return originalFetch(input, init)
}
if (url.origin === WORKOS_ORIGIN && url.pathname === "/user_management/authorize/device") {
return jsonResponse({
device_code: DEVICE_CODE,
user_code: USER_CODE,
verification_uri: "https://login.workos.test/device",
verification_uri_complete: `https://login.workos.test/device?user_code=${USER_CODE}`,
expires_in: 300,
interval: 1,
})
}
if (url.origin === WORKOS_ORIGIN && url.pathname === "/user_management/authenticate") {
return jsonResponse({
access_token: ACCESS_TOKEN,
refresh_token: REFRESH_TOKEN,
token_type: "Bearer",
})
}
return originalFetch(input, init)
}
+1 -3
View File
@@ -21,9 +21,7 @@ describe("ClineEndpoint configuration", () => {
// Stub os.homedir to return our temp directory
originalHomedir = os.homedir
sandbox
.stub(os, "homedir")
.returns(tempDir)
sandbox.stub(os, "homedir").returns(tempDir)
// Reset the singleton state using internal method
;(ClineEndpoint as any)._instance = null
+1 -4
View File
@@ -4,7 +4,6 @@ import "./utils/path" // necessary to have access to String.prototype.toPosix
import { HostProvider } from "@/hosts/host-provider"
import { Logger } from "@/shared/services/Logger"
import type { StorageContext } from "@/shared/storage/storage-context"
import { FileContextTracker } from "./core/context/context-tracking/FileContextTracker"
import { clearOnboardingModelsCache } from "./core/controller/models/getClineOnboardingModels"
import { HookDiscoveryCache } from "./core/hooks/HookDiscoveryCache"
import { HookProcessRegistry } from "./core/hooks/HookProcessRegistry"
@@ -74,8 +73,6 @@ export async function initialize(storageContext: StorageContext): Promise<Webvie
syncWorker().init({ ...blobStoreSettings, userDistinctId: getDistinctId() })
// Clean up old temp files in background (non-blocking) and start periodic cleanup every 24 hours
ClineTempManager.startPeriodicCleanup()
// Clean up orphaned file context warnings (startup cleanup)
FileContextTracker.cleanupOrphanedWarnings(stateManager)
telemetryService.captureExtensionActivated()
@@ -106,7 +103,7 @@ async function showVersionUpdateAnnouncement(stateManager: StateManager) {
})
}
// Always update the main version tracker for the next launch.
await stateManager.setGlobalState("clineVersion", currentVersion)
stateManager.setGlobalState("clineVersion", currentVersion)
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error)
+17 -477
View File
@@ -1,52 +1,24 @@
import { ApiConfiguration, ModelInfo, QwenApiRegions } from "@shared/api"
import { type ApiHandler as SdkApiHandler, type ApiStreamChunk as SdkApiStreamChunk } from "@cline/llms"
import { ApiConfiguration, ModelInfo } from "@shared/api"
import { Mode } from "@shared/storage/types"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { ClineTool } from "@/shared/tools"
import { AIhubmixHandler } from "./providers/aihubmix"
import { AnthropicHandler } from "./providers/anthropic"
import { AskSageHandler } from "./providers/asksage"
import { BasetenHandler } from "./providers/baseten"
import { AwsBedrockHandler } from "./providers/bedrock"
import { CerebrasHandler } from "./providers/cerebras"
import { ClaudeCodeHandler } from "./providers/claude-code"
import { ClineHandler } from "./providers/cline"
import { DeepSeekHandler } from "./providers/deepseek"
import { DifyHandler } from "./providers/dify"
import { DoubaoHandler } from "./providers/doubao"
import { FireworksHandler } from "./providers/fireworks"
import { GeminiHandler } from "./providers/gemini"
import { GroqHandler } from "./providers/groq"
import { HicapHandler } from "./providers/hicap"
import { HuaweiCloudMaaSHandler } from "./providers/huawei-cloud-maas"
import { HuggingFaceHandler } from "./providers/huggingface"
import { LiteLlmHandler } from "./providers/litellm"
import { LmStudioHandler } from "./providers/lmstudio"
import { MinimaxHandler } from "./providers/minimax"
import { MistralHandler } from "./providers/mistral"
import { MoonshotHandler } from "./providers/moonshot"
import { NebiusHandler } from "./providers/nebius"
import { NousResearchHandler } from "./providers/nousresearch"
import { OcaHandler } from "./providers/oca"
import { OllamaHandler } from "./providers/ollama"
import { OpenAiHandler } from "./providers/openai"
import { OpenAiCodexHandler } from "./providers/openai-codex"
import { OpenAiNativeHandler } from "./providers/openai-native"
import { OpenRouterHandler } from "./providers/openrouter"
import { QwenHandler } from "./providers/qwen"
import { QwenCodeHandler } from "./providers/qwen-code"
import { RequestyHandler } from "./providers/requesty"
import { SambanovaHandler } from "./providers/sambanova"
import { SapAiCoreHandler } from "./providers/sapaicore"
import { TogetherHandler } from "./providers/together"
import { VercelAIGatewayHandler } from "./providers/vercel-ai-gateway"
import { VertexHandler } from "./providers/vertex"
import { VsCodeLmHandler } from "./providers/vscode-lm"
import { WandbHandler } from "./providers/wandb"
import { XAIHandler } from "./providers/xai"
import { ZAiHandler } from "./providers/zai"
import { ApiStream, ApiStreamUsageChunk } from "./transform/stream"
// buildApiHandler now routes inference through the Cline SDK. It lives in
// apps/vscode/src/sdk/sdk-api-handler.ts and callers import it directly from
// there. It is deliberately NOT re-exported here: this barrel is imported
// widely for *types* only, and re-exporting a value from the SDK module would
// pull the entire SDK/session-factory runtime graph into every type importer
// at module-eval time (which can break extension activation). Keep this file
// types-only.
// Re-export the SDK inference contracts so callers can depend on the SDK types
// through the existing @core/api entry point. These are the canonical handler
// and stream types going forward; the local interfaces below remain for the
// classic provider classes until they are removed.
export type { SdkApiHandler, SdkApiStreamChunk }
export type CommonApiHandlerOptions = {
onRetryAttempt?: ApiConfiguration["onRetryAttempt"]
}
@@ -60,6 +32,7 @@ export interface ApiHandler {
export interface ApiHandlerModel {
id: string
info: ModelInfo
providerId?: string
}
export interface ApiProviderInfo {
@@ -72,436 +45,3 @@ export interface ApiProviderInfo {
export interface SingleCompletionHandler {
completePrompt(prompt: string): Promise<string>
}
function createHandlerForProvider(
apiProvider: string | undefined,
options: Omit<ApiConfiguration, "apiProvider">,
mode: Mode,
): ApiHandler {
switch (apiProvider) {
case "anthropic":
return new AnthropicHandler({
onRetryAttempt: options.onRetryAttempt,
apiKey: options.apiKey,
anthropicBaseUrl: options.anthropicBaseUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "openrouter":
return new OpenRouterHandler({
onRetryAttempt: options.onRetryAttempt,
openRouterApiKey: options.openRouterApiKey,
openRouterModelId: mode === "plan" ? options.planModeOpenRouterModelId : options.actModeOpenRouterModelId,
openRouterModelInfo: mode === "plan" ? options.planModeOpenRouterModelInfo : options.actModeOpenRouterModelInfo,
openRouterProviderSorting: options.openRouterProviderSorting,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
enableParallelToolCalling: options.enableParallelToolCalling,
})
case "bedrock":
return new AwsBedrockHandler({
onRetryAttempt: options.onRetryAttempt,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
awsAccessKey: options.awsAccessKey,
awsSecretKey: options.awsSecretKey,
awsSessionToken: options.awsSessionToken,
awsRegion: options.awsRegion,
awsAuthentication: options.awsAuthentication,
awsBedrockApiKey: options.awsBedrockApiKey,
awsUseCrossRegionInference: options.awsUseCrossRegionInference,
awsUseGlobalInference: options.awsUseGlobalInference,
awsBedrockUsePromptCache: options.awsBedrockUsePromptCache,
awsUseProfile: options.awsUseProfile,
awsProfile: options.awsProfile,
awsBedrockEndpoint: options.awsBedrockEndpoint,
awsBedrockCustomSelected:
mode === "plan" ? options.planModeAwsBedrockCustomSelected : options.actModeAwsBedrockCustomSelected,
awsBedrockCustomModelBaseId:
mode === "plan" ? options.planModeAwsBedrockCustomModelBaseId : options.actModeAwsBedrockCustomModelBaseId,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "vertex":
return new VertexHandler({
onRetryAttempt: options.onRetryAttempt,
vertexProjectId: options.vertexProjectId,
vertexRegion: options.vertexRegion,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
geminiApiKey: options.geminiApiKey,
geminiBaseUrl: options.geminiBaseUrl,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
ulid: options.ulid,
})
case "openai":
return new OpenAiHandler({
onRetryAttempt: options.onRetryAttempt,
openAiApiKey: options.openAiApiKey,
openAiBaseUrl: options.openAiBaseUrl,
azureApiVersion: options.azureApiVersion,
azureIdentity: options.azureIdentity,
openAiHeaders: options.openAiHeaders,
openAiModelId: mode === "plan" ? options.planModeOpenAiModelId : options.actModeOpenAiModelId,
openAiModelInfo: mode === "plan" ? options.planModeOpenAiModelInfo : options.actModeOpenAiModelInfo,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
})
case "ollama":
return new OllamaHandler({
onRetryAttempt: options.onRetryAttempt,
ollamaBaseUrl: options.ollamaBaseUrl,
ollamaApiKey: options.ollamaApiKey,
ollamaModelId: mode === "plan" ? options.planModeOllamaModelId : options.actModeOllamaModelId,
ollamaApiOptionsCtxNum: options.ollamaApiOptionsCtxNum,
requestTimeoutMs: options.requestTimeoutMs,
})
case "lmstudio":
return new LmStudioHandler({
onRetryAttempt: options.onRetryAttempt,
lmStudioBaseUrl: options.lmStudioBaseUrl,
lmStudioModelId: mode === "plan" ? options.planModeLmStudioModelId : options.actModeLmStudioModelId,
lmStudioMaxTokens: options.lmStudioMaxTokens,
})
case "gemini":
return new GeminiHandler({
onRetryAttempt: options.onRetryAttempt,
vertexProjectId: options.vertexProjectId,
vertexRegion: options.vertexRegion,
geminiApiKey: options.geminiApiKey,
geminiBaseUrl: options.geminiBaseUrl,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
ulid: options.ulid,
})
case "openai-native":
return new OpenAiNativeHandler({
onRetryAttempt: options.onRetryAttempt,
openAiNativeApiKey: options.openAiNativeApiKey,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "openai-codex":
return new OpenAiCodexHandler({
onRetryAttempt: options.onRetryAttempt,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "deepseek":
return new DeepSeekHandler({
onRetryAttempt: options.onRetryAttempt,
deepSeekApiKey: options.deepSeekApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "requesty":
return new RequestyHandler({
onRetryAttempt: options.onRetryAttempt,
requestyBaseUrl: options.requestyBaseUrl,
requestyApiKey: options.requestyApiKey,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
requestyModelId: mode === "plan" ? options.planModeRequestyModelId : options.actModeRequestyModelId,
requestyModelInfo: mode === "plan" ? options.planModeRequestyModelInfo : options.actModeRequestyModelInfo,
})
case "fireworks":
return new FireworksHandler({
onRetryAttempt: options.onRetryAttempt,
fireworksApiKey: options.fireworksApiKey,
fireworksModelId: mode === "plan" ? options.planModeFireworksModelId : options.actModeFireworksModelId,
})
case "together":
return new TogetherHandler({
onRetryAttempt: options.onRetryAttempt,
togetherApiKey: options.togetherApiKey,
togetherModelId: mode === "plan" ? options.planModeTogetherModelId : options.actModeTogetherModelId,
})
case "qwen":
return new QwenHandler({
onRetryAttempt: options.onRetryAttempt,
qwenApiKey: options.qwenApiKey,
qwenApiLine:
options.qwenApiLine === QwenApiRegions.INTERNATIONAL ? QwenApiRegions.INTERNATIONAL : QwenApiRegions.CHINA,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "qwen-code":
return new QwenCodeHandler({
onRetryAttempt: options.onRetryAttempt,
qwenCodeOauthPath: options.qwenCodeOauthPath,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "doubao":
return new DoubaoHandler({
onRetryAttempt: options.onRetryAttempt,
doubaoApiKey: options.doubaoApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "mistral":
return new MistralHandler({
onRetryAttempt: options.onRetryAttempt,
mistralApiKey: options.mistralApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "vscode-lm":
return new VsCodeLmHandler({
onRetryAttempt: options.onRetryAttempt,
vsCodeLmModelSelector:
mode === "plan" ? options.planModeVsCodeLmModelSelector : options.actModeVsCodeLmModelSelector,
})
case "cline": {
const clineModelId =
(mode === "plan" ? options.planModeClineModelId : options.actModeClineModelId) ||
(mode === "plan" ? options.planModeOpenRouterModelId : options.actModeOpenRouterModelId)
const clineModelInfo =
(mode === "plan" ? options.planModeClineModelInfo : options.actModeClineModelInfo) ||
(mode === "plan" ? options.planModeOpenRouterModelInfo : options.actModeOpenRouterModelInfo)
return new ClineHandler({
onRetryAttempt: options.onRetryAttempt,
clineAccountId: options.clineAccountId,
clineApiKey: options.clineApiKey,
ulid: options.ulid,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
openRouterProviderSorting: options.openRouterProviderSorting,
openRouterModelId: clineModelId,
openRouterModelInfo: clineModelInfo,
enableParallelToolCalling: options.enableParallelToolCalling,
})
}
case "litellm":
return new LiteLlmHandler({
onRetryAttempt: options.onRetryAttempt,
liteLlmApiKey: options.liteLlmApiKey,
liteLlmBaseUrl: options.liteLlmBaseUrl,
liteLlmModelId: mode === "plan" ? options.planModeLiteLlmModelId : options.actModeLiteLlmModelId,
liteLlmModelInfo: mode === "plan" ? options.planModeLiteLlmModelInfo : options.actModeLiteLlmModelInfo,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
liteLlmUsePromptCache: options.liteLlmUsePromptCache,
ulid: options.ulid,
})
case "moonshot":
return new MoonshotHandler({
onRetryAttempt: options.onRetryAttempt,
moonshotApiKey: options.moonshotApiKey,
moonshotApiLine: options.moonshotApiLine,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "huggingface":
return new HuggingFaceHandler({
onRetryAttempt: options.onRetryAttempt,
huggingFaceApiKey: options.huggingFaceApiKey,
huggingFaceModelId: mode === "plan" ? options.planModeHuggingFaceModelId : options.actModeHuggingFaceModelId,
huggingFaceModelInfo:
mode === "plan" ? options.planModeHuggingFaceModelInfo : options.actModeHuggingFaceModelInfo,
})
case "nebius":
return new NebiusHandler({
onRetryAttempt: options.onRetryAttempt,
nebiusApiKey: options.nebiusApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "asksage":
return new AskSageHandler({
onRetryAttempt: options.onRetryAttempt,
asksageApiKey: options.asksageApiKey,
asksageApiUrl: options.asksageApiUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "xai":
return new XAIHandler({
onRetryAttempt: options.onRetryAttempt,
xaiApiKey: options.xaiApiKey,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "sambanova":
return new SambanovaHandler({
onRetryAttempt: options.onRetryAttempt,
sambanovaApiKey: options.sambanovaApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "cerebras":
return new CerebrasHandler({
onRetryAttempt: options.onRetryAttempt,
cerebrasApiKey: options.cerebrasApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "groq":
return new GroqHandler({
onRetryAttempt: options.onRetryAttempt,
groqApiKey: options.groqApiKey,
groqModelId: mode === "plan" ? options.planModeGroqModelId : options.actModeGroqModelId,
groqModelInfo: mode === "plan" ? options.planModeGroqModelInfo : options.actModeGroqModelInfo,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "baseten":
return new BasetenHandler({
onRetryAttempt: options.onRetryAttempt,
basetenApiKey: options.basetenApiKey,
basetenModelId: mode === "plan" ? options.planModeBasetenModelId : options.actModeBasetenModelId,
basetenModelInfo: mode === "plan" ? options.planModeBasetenModelInfo : options.actModeBasetenModelInfo,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "sapaicore":
return new SapAiCoreHandler({
onRetryAttempt: options.onRetryAttempt,
sapAiCoreClientId: options.sapAiCoreClientId,
sapAiCoreClientSecret: options.sapAiCoreClientSecret,
sapAiCoreTokenUrl: options.sapAiCoreTokenUrl,
sapAiResourceGroup: options.sapAiResourceGroup,
sapAiCoreBaseUrl: options.sapAiCoreBaseUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
deploymentId: mode === "plan" ? options.planModeSapAiCoreDeploymentId : options.actModeSapAiCoreDeploymentId,
sapAiCoreUseOrchestrationMode: options.sapAiCoreUseOrchestrationMode,
})
case "claude-code":
return new ClaudeCodeHandler({
onRetryAttempt: options.onRetryAttempt,
claudeCodePath: options.claudeCodePath,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "huawei-cloud-maas":
return new HuaweiCloudMaaSHandler({
onRetryAttempt: options.onRetryAttempt,
huaweiCloudMaasApiKey: options.huaweiCloudMaasApiKey,
huaweiCloudMaasModelId:
mode === "plan" ? options.planModeHuaweiCloudMaasModelId : options.actModeHuaweiCloudMaasModelId,
huaweiCloudMaasModelInfo:
mode === "plan" ? options.planModeHuaweiCloudMaasModelInfo : options.actModeHuaweiCloudMaasModelInfo,
})
case "dify": // Add Dify.ai handler
return new DifyHandler({
difyApiKey: options.difyApiKey,
difyBaseUrl: options.difyBaseUrl,
})
case "vercel-ai-gateway":
return new VercelAIGatewayHandler({
onRetryAttempt: options.onRetryAttempt,
vercelAiGatewayApiKey: options.vercelAiGatewayApiKey,
openRouterModelId:
mode === "plan" ? options.planModeVercelAiGatewayModelId : options.actModeVercelAiGatewayModelId,
openRouterModelInfo:
mode === "plan" ? options.planModeVercelAiGatewayModelInfo : options.actModeVercelAiGatewayModelInfo,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "zai":
return new ZAiHandler({
onRetryAttempt: options.onRetryAttempt,
zaiApiLine: options.zaiApiLine,
zaiApiKey: options.zaiApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "oca":
return new OcaHandler({
ocaMode: options.ocaMode || "internal",
ocaBaseUrl: options.ocaBaseUrl,
ocaModelId: mode === "plan" ? options.planModeOcaModelId : options.actModeOcaModelId,
ocaModelInfo: mode === "plan" ? options.planModeOcaModelInfo : options.actModeOcaModelInfo,
ocaReasoningEffort: mode === "plan" ? options.planModeOcaReasoningEffort : options.actModeOcaReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
ocaUsePromptCache:
mode === "plan"
? options.planModeOcaModelInfo?.supportsPromptCache
: options.actModeOcaModelInfo?.supportsPromptCache,
taskId: options.ulid,
})
case "aihubmix":
return new AIhubmixHandler({
onRetryAttempt: options.onRetryAttempt,
apiKey: options.aihubmixApiKey,
baseURL: options.aihubmixBaseUrl,
appCode: options.aihubmixAppCode,
modelId: mode === "plan" ? (options as any).planModeAihubmixModelId : (options as any).actModeAihubmixModelId,
modelInfo:
mode === "plan" ? (options as any).planModeAihubmixModelInfo : (options as any).actModeAihubmixModelInfo,
})
case "minimax":
return new MinimaxHandler({
onRetryAttempt: options.onRetryAttempt,
minimaxApiKey: options.minimaxApiKey,
minimaxApiLine: options.minimaxApiLine,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "hicap":
return new HicapHandler({
onRetryAttempt: options.onRetryAttempt,
hicapApiKey: options.hicapApiKey,
hicapModelId: mode === "plan" ? options.planModeHicapModelId : options.actModeHicapModelId,
})
case "nousResearch":
return new NousResearchHandler({
onRetryAttempt: options.onRetryAttempt,
nousResearchApiKey: options.nousResearchApiKey,
apiModelId: mode === "plan" ? options.planModeNousResearchModelId : options.actModeNousResearchModelId,
})
case "wandb":
return new WandbHandler({
onRetryAttempt: options.onRetryAttempt,
wandbApiKey: options.wandbApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
default:
return new AnthropicHandler({
onRetryAttempt: options.onRetryAttempt,
apiKey: options.apiKey,
anthropicBaseUrl: options.anthropicBaseUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
}
}
export function buildApiHandler(configuration: ApiConfiguration, mode: Mode): ApiHandler {
const { planModeApiProvider, actModeApiProvider, ...options } = configuration
const apiProvider = mode === "plan" ? planModeApiProvider : actModeApiProvider
// Validate thinking budget tokens against model's maxTokens to prevent API errors
// wrapped in a try-catch for safety, but this should never throw
try {
const thinkingBudgetTokens = mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens
if (thinkingBudgetTokens && thinkingBudgetTokens > 0) {
const handler = createHandlerForProvider(apiProvider, options, mode)
const modelInfo = handler.getModel().info
if (modelInfo?.maxTokens && modelInfo.maxTokens > 0 && thinkingBudgetTokens > modelInfo.maxTokens) {
const clippedValue = modelInfo.maxTokens - 1
if (mode === "plan") {
options.planModeThinkingBudgetTokens = clippedValue
} else {
options.actModeThinkingBudgetTokens = clippedValue
}
} else {
return handler // don't rebuild unless its necessary
}
}
} catch (error) {
Logger.error("buildApiHandler error:", error)
}
return createHandlerForProvider(apiProvider, options, mode)
}
@@ -1,236 +0,0 @@
import { afterEach, describe, it } from "mocha"
import sinon from "sinon"
import "should"
import { anthropicModels } from "@shared/api"
import { ANTHROPIC_FAST_MODE_BETA, AnthropicHandler } from "../anthropic"
describe("AnthropicHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: readonly unknown[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
describe("getModel", () => {
it("should return the fast mode model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:fast",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-6:fast")
result.info.should.deepEqual(anthropicModels["claude-opus-4-6:fast"])
})
it("should return the 1m fast mode model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:1m:fast",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-6:1m:fast")
result.info.should.deepEqual(anthropicModels["claude-opus-4-6:1m:fast"])
})
it("should return the 4.7 model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-7")
result.info.should.deepEqual(anthropicModels["claude-opus-4-7"])
})
it("should return the 4.7 1m model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7:1m",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-7:1m")
result.info.should.deepEqual(anthropicModels["claude-opus-4-7:1m"])
})
it("should return the 4.8 model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-8",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-8")
result.info.should.deepEqual(anthropicModels["claude-opus-4-8"])
})
it("should return the 4.8 1m model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-8:1m",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-8:1m")
result.info.should.deepEqual(anthropicModels["claude-opus-4-8:1m"])
})
})
describe("createMessage", () => {
it("should route fast mode requests through the beta messages API", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:fast",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
const betaCreate = sinon.stub().callsFake(function (this: { _client?: object }, _params: unknown) {
should.exist(this._client)
return Promise.resolve(createAsyncIterable())
})
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: betaCreate,
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.notCalled(standardCreate)
sinon.assert.calledOnce(betaCreate)
sinon.assert.calledWithMatch(betaCreate, {
model: "claude-opus-4-6",
betas: [ANTHROPIC_FAST_MODE_BETA],
speed: "fast",
stream: true,
})
})
it("should include the 1m beta when routing 1m fast mode requests through the beta messages API", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:1m:fast",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
const betaCreate = sinon.stub().callsFake(function (this: { _client?: object }, _params: unknown) {
should.exist(this._client)
return Promise.resolve(createAsyncIterable())
})
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: betaCreate,
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.notCalled(standardCreate)
sinon.assert.calledOnce(betaCreate)
sinon.assert.calledWithMatch(betaCreate, {
model: "claude-opus-4-6",
betas: [ANTHROPIC_FAST_MODE_BETA, "context-1m-2025-08-07"],
speed: "fast",
stream: true,
})
})
it("should include the 1m beta header for Claude Opus 4.7 1m requests", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7:1m",
reasoningEffort: "high",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: sinon.stub().resolves(createAsyncIterable()),
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.calledOnce(standardCreate)
const requestBody = standardCreate.firstCall.args[0] as Record<string, any>
const requestOptions = standardCreate.firstCall.args[1] as Record<string, any>
requestBody.model.should.equal("claude-opus-4-7")
requestBody.thinking.should.deepEqual({ type: "adaptive" })
requestOptions.should.deepEqual({
headers: {
"anthropic-beta": "context-1m-2025-08-07",
},
})
})
it("should use adaptive thinking and output_config for Claude Opus adaptive models", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7",
reasoningEffort: "xhigh",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: sinon.stub().resolves(createAsyncIterable()),
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.calledOnce(standardCreate)
const requestBody = standardCreate.firstCall.args[0] as Record<string, any>
requestBody.should.have.property("thinking")
requestBody.thinking.should.deepEqual({ type: "adaptive" })
requestBody.should.have.property("output_config")
requestBody.output_config.should.deepEqual({ effort: "xhigh" })
should(requestBody.temperature).equal(undefined)
})
})
})
File diff suppressed because it is too large Load Diff
@@ -1,468 +0,0 @@
import { afterEach, beforeEach, describe, it } from "mocha"
import sinon from "sinon"
import "should"
import { ClaudeCodeHandler } from "@core/api/providers/claude-code"
import { ClineStorageMessage } from "@/shared/messages/content"
describe("ClaudeCodeHandler", () => {
let handler: ClaudeCodeHandler
let sandbox: sinon.SinonSandbox
beforeEach(() => {
sandbox = sinon.createSandbox()
handler = new ClaudeCodeHandler({
claudeCodePath: "/mock/path",
apiModelId: "claude-opus-4-1-20250805",
})
})
afterEach(() => {
sandbox.restore()
})
describe("token counting", () => {
it("should correctly handle token usage from assistant messages", async () => {
// The 'input_tokens' field represents the TOTAL number of input tokens used.
// See https://docs.anthropic.com/en/api/messages#usage-object
// Mock the runClaudeCode function
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
// Create a proper async generator mock for the Claude Code response
async function* mockGenerator() {
// First yield the system init
yield {
type: "system",
subtype: "init",
apiKeySource: "api",
}
// Yield assistant message with usage data
// Example: If base input is 70 tokens, cache read is 20, and cache creation is 10,
// then input_tokens from Anthropic API will be 100 (70 + 20 + 10)
yield {
type: "assistant",
message: {
content: [
{
type: "text",
text: "Test response",
},
],
usage: {
input_tokens: 100, // Total including cache (per Anthropic docs)
output_tokens: 50,
cache_read_input_tokens: 20, // Already included in input_tokens
cache_creation_input_tokens: 10, // Already included in input_tokens
},
stop_reason: "end_turn",
},
}
// Yield result with cost
yield {
type: "result",
result: {},
total_cost_usd: 0.005,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const usageData: any[] = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "usage") {
usageData.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
cacheReadTokens: chunk.cacheReadTokens,
cacheWriteTokens: chunk.cacheWriteTokens,
totalCost: chunk.totalCost,
})
}
}
// Verify token counting follows Anthropic API specification
usageData.should.have.length(1)
usageData[0].should.deepEqual({
inputTokens: 100, // Total including cache tokens (per Anthropic API docs)
outputTokens: 50,
cacheReadTokens: 20, // Tracked separately for reporting
cacheWriteTokens: 10, // Tracked separately for reporting
totalCost: 0.005,
})
// CRITICAL ASSERTION: Verify that input_tokens is NOT inflated by re-adding cache tokens
// The bug would have caused inputTokens to be incorrectly calculated as 130 (100 + 20 + 10)
// The fix ensures it remains 100, as per Anthropic's specification
usageData[0].inputTokens.should.equal(100) // Correct: matches API response
usageData[0].inputTokens.should.not.equal(130) // Would be wrong: double-counting cache tokens
})
it("should handle missing usage fields with nullish coalescing", async () => {
// Mock the runClaudeCode function
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
// Create a proper async generator mock with missing/undefined usage fields
async function* mockGenerator() {
yield {
type: "assistant",
message: {
content: [
{
type: "text",
text: "Test response",
},
],
usage: {
input_tokens: 100,
output_tokens: 50,
// cache fields are undefined/missing
},
stop_reason: "end_turn",
},
}
yield {
type: "result",
result: {},
total_cost_usd: 0.005,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const usageData: any[] = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "usage") {
usageData.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
cacheReadTokens: chunk.cacheReadTokens,
cacheWriteTokens: chunk.cacheWriteTokens,
})
}
}
// Verify that undefined cache tokens default to 0
usageData.should.have.length(1)
usageData[0].should.deepEqual({
inputTokens: 100,
outputTokens: 50,
cacheReadTokens: 0, // Should default to 0
cacheWriteTokens: 0, // Should default to 0
})
})
it("should handle completely missing usage object", async () => {
// Mock the runClaudeCode function
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
// Create a proper async generator mock with missing usage object
async function* mockGenerator() {
yield {
type: "assistant",
message: {
content: [
{
type: "text",
text: "Test response",
},
],
// usage is undefined
usage: undefined,
stop_reason: "end_turn",
},
}
// Need to yield a result chunk to trigger usage data emission
yield {
type: "result",
result: {},
total_cost_usd: 0,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const usageData: any[] = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "usage") {
usageData.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
cacheReadTokens: chunk.cacheReadTokens,
cacheWriteTokens: chunk.cacheWriteTokens,
})
}
}
// All token counts should default to 0 when usage is undefined
usageData.should.have.length(1)
usageData[0].should.deepEqual({
inputTokens: 0,
outputTokens: 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
})
})
})
describe("error handling", () => {
it("should not crash when assistant message has empty content array", async () => {
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
async function* mockGenerator() {
yield {
type: "assistant",
message: {
content: [], // empty content — triggered TypeError in older code
usage: {
input_tokens: 10,
output_tokens: 0,
},
stop_reason: "end_turn",
},
}
yield {
type: "result",
result: {},
total_cost_usd: 0,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const chunks: any[] = []
// Should not throw
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
const usageChunk = chunks.find((c) => c.type === "usage")
usageChunk.should.be.ok()
usageChunk.inputTokens.should.equal(10)
})
it("should throw when result has is_error=true (e.g. rate limit with no assistant message)", async () => {
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
async function* mockGenerator() {
yield {
type: "system",
subtype: "init",
apiKeySource: "none",
}
yield {
type: "system",
subtype: "rate_limit_event",
message: "Rate limit hit",
retryAfterSeconds: 30,
}
// No assistant message — CLI hit rate limit and gave up
yield {
type: "result",
subtype: "error",
is_error: true,
result: "Rate limit exceeded",
total_cost_usd: 0,
duration_ms: 1000,
duration_api_ms: 500,
num_turns: 0,
session_id: "test",
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
let thrownError: Error | undefined
try {
for await (const _ of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
// consume
}
} catch (err) {
thrownError = err as Error
}
thrownError!.message.should.containEql("Rate limit exceeded")
})
it("should ignore rate_limit_event system messages without throwing", async () => {
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
async function* mockGenerator() {
yield {
type: "system",
subtype: "init",
apiKeySource: "none",
}
// Newer Claude Code CLI emits this during rate limiting
yield {
type: "system",
subtype: "rate_limit_event",
message: "Rate limit hit, retrying...",
retryAfterSeconds: 30,
}
yield {
type: "assistant",
message: {
content: [{ type: "text", text: "Response after retry" }],
usage: { input_tokens: 20, output_tokens: 10 },
stop_reason: "end_turn",
},
}
yield {
type: "result",
result: {},
total_cost_usd: 0,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const textChunks: string[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
if (chunk.type === "text") textChunks.push(chunk.text)
}
textChunks.should.deepEqual(["Response after retry"])
})
})
describe("getModel", () => {
it("should return the correct model when specified", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-sonnet-4-5-20250929",
})
const model = handler.getModel()
model.id.should.equal("claude-sonnet-4-5-20250929")
})
it("should support Opus 4.6 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-6[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-6[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Opus 4.7 model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-7",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-7")
model.info.contextWindow.should.equal(200_000)
})
it("should support Opus 4.7 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-7[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-7[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Opus 4.8 model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-8",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-8")
model.info.contextWindow.should.equal(200_000)
})
it("should support Opus 4.8 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-8[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-8[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Opus 1m alias model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "opus[1m]",
})
const model = handler.getModel()
model.id.should.equal("opus[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Sonnet 1m alias model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "sonnet[1m]",
})
const model = handler.getModel()
model.id.should.equal("sonnet[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Sonnet 4.5 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-sonnet-4-5-20250929[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-sonnet-4-5-20250929[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Sonnet 4.6 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-sonnet-4-6[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-sonnet-4-6[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should return default model when not specified", () => {
const handler = new ClaudeCodeHandler({})
const model = handler.getModel()
// The default model should be set
model.id.should.be.type("string")
model.info.should.be.type("object")
})
})
})
@@ -1,166 +0,0 @@
import "should"
import { openRouterDefaultModelInfo } from "@shared/api"
import sinon from "sinon"
import { ClineAccountService } from "@/services/account/ClineAccountService"
import { AuthService } from "@/services/auth/AuthService"
import { ClineHandler } from "../cline"
describe("ClineHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
const createHandler = (options: ConstructorParameters<typeof ClineHandler>[0]) => {
sinon.stub(ClineAccountService, "getInstance").returns({} as any)
sinon.stub(AuthService, "getInstance").returns({} as any)
return new ClineHandler(options)
}
it("should handle usage-only chunks when delta is missing", async () => {
const handler = createHandler({})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 17,
completion_tokens: 9,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "openai/gpt-4o-mini",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: 0,
inputTokens: 17,
outputTokens: 9,
totalCost: 0,
},
])
})
it("should read Anthropic-style cache creation and read tokens from usage chunks", async () => {
const handler = createHandler({})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 1000,
completion_tokens: 200,
prompt_tokens_details: {
cached_tokens: 500,
},
cache_creation_input_tokens: 300,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "anthropic/claude-sonnet-4.6",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 300,
cacheReadTokens: 500,
inputTokens: 200,
outputTokens: 200,
totalCost: 0,
},
])
})
it("should forward enableParallelToolCalling to OpenRouter payload", async () => {
const handler = createHandler({ enableParallelToolCalling: true })
const createStub = sinon.stub().resolves(createAsyncIterable([]))
const fakeClient = {
chat: {
completions: {
create: createStub,
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "openai/gpt-4o-mini",
info: openRouterDefaultModelInfo,
})
const tools = [
{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } },
] as any
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
// drain stream
}
const payload = createStub.firstCall.args[0]
payload.parallel_tool_calls.should.equal(true)
})
it("should send cache_control for qwen3.7-max without changing the selected Cline model id", async () => {
const handler = createHandler({
openRouterModelId: "qwen/qwen3.7-max",
openRouterModelInfo: openRouterDefaultModelInfo,
})
const createStub = sinon.stub().resolves(createAsyncIterable([]))
const fakeClient = {
chat: {
completions: {
create: createStub,
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler as any, "getFreeModelIdSet").resolves(new Set())
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
// drain stream
}
handler.getModel().id.should.equal("qwen/qwen3.7-max")
const payload = createStub.firstCall.args[0]
payload.model.should.equal("qwen/qwen3.7-max")
payload.messages[0].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
payload.messages[1].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
})
})
@@ -1,97 +0,0 @@
import "should"
import sinon from "sinon"
import { FireworksHandler } from "../fireworks"
describe("FireworksHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
it("should handle usage-only chunks when delta is missing", async () => {
const handler = new FireworksHandler({
fireworksApiKey: "test-api-key",
fireworksModelId: "accounts/fireworks/models/llama-v3p1-8b-instruct",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 19,
completion_tokens: 4,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
inputTokens: 19,
outputTokens: 4,
cacheReadTokens: 0,
cacheWriteTokens: 0,
},
])
})
it("should read cache hits from prompt_tokens_details when hit tokens are not present", async () => {
const handler = new FireworksHandler({
fireworksApiKey: "test-api-key",
fireworksModelId: "accounts/fireworks/models/llama-v3p1-8b-instruct",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 60,
completion_tokens: 12,
prompt_tokens_details: { cached_tokens: 20 },
prompt_cache_miss_tokens: 40,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
inputTokens: 60,
outputTokens: 12,
cacheReadTokens: 20,
cacheWriteTokens: 40,
},
])
})
})
@@ -1,235 +0,0 @@
import "should"
import sinon from "sinon"
import { GeminiHandler } from "../gemini"
describe("GeminiHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
it("caps maxOutputTokens to 8192 for Flash models", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
apiModelId: "gemini-2.5-flash",
})
const generateContentStream = sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp-1",
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 20,
cachedContentTokenCount: 0,
thoughtsTokenCount: 0,
},
},
]),
)
sinon.stub(handler as any, "ensureClient").returns({
models: { generateContentStream },
} as any)
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
// Consume stream to trigger request execution.
}
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
requestArgs.config.should.have.property("maxOutputTokens", 8_192)
})
it("supports Gemini 3.5 Flash model metadata", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
apiModelId: "gemini-3.5-flash",
})
const model = handler.getModel()
model.id.should.equal("gemini-3.5-flash")
model.info.contextWindow!.should.equal(1_048_576)
model.info.inputPrice!.should.equal(1.5)
model.info.outputPrice!.should.equal(9)
model.info.cacheReadsPrice!.should.equal(0.15)
model.info.supportsReasoning!.should.equal(true)
const generateContentStream = sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp-35",
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 20,
cachedContentTokenCount: 0,
thoughtsTokenCount: 0,
},
},
]),
)
sinon.stub(handler as any, "ensureClient").returns({
models: { generateContentStream },
} as any)
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
// Consume stream to trigger request execution.
}
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
requestArgs.model.should.equal("gemini-3.5-flash")
requestArgs.config.should.have.property("maxOutputTokens", 8_192)
requestArgs.config.thinkingConfig.should.deepEqual({
thinkingBudget: undefined,
thinkingLevel: "LOW",
includeThoughts: true,
})
})
it("does not set maxOutputTokens for non-Flash models", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
apiModelId: "gemini-2.5-pro",
})
const generateContentStream = sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp-2",
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 20,
cachedContentTokenCount: 0,
thoughtsTokenCount: 0,
},
},
]),
)
sinon.stub(handler as any, "ensureClient").returns({
models: { generateContentStream },
} as any)
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
// Consume stream to trigger request execution.
}
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
requestArgs.config.should.not.have.property("maxOutputTokens")
})
it("should emit unique tool call IDs when multiple function calls share one responseId", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
})
const fakeClient = {
models: {
generateContentStream: sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp_1",
candidates: [
{
content: {
parts: [
{
functionCall: {
name: "read_file",
args: { path: ".nvmrc" },
},
},
],
},
},
],
},
{
responseId: "resp_1",
candidates: [
{
content: {
parts: [
{
functionCall: {
name: "read_file",
args: { path: ".gitattributes" },
},
},
],
},
},
],
},
]),
),
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const tools = [{ name: "read_file", description: "read file", parameters: { type: "OBJECT" } }] as any
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
if (chunk.type === "tool_calls") {
chunks.push(chunk)
}
}
chunks.should.have.length(2)
chunks[0].tool_call.function.id.should.equal("resp_1-tool-0")
chunks[1].tool_call.function.id.should.equal("resp_1-tool-1")
chunks[0].tool_call.call_id.should.equal(chunks[0].tool_call.function.id)
chunks[1].tool_call.call_id.should.equal(chunks[1].tool_call.function.id)
JSON.parse(chunks[0].tool_call.function.arguments).path.should.equal(".nvmrc")
JSON.parse(chunks[1].tool_call.function.arguments).path.should.equal(".gitattributes")
})
it("should preserve Gemini-provided functionCall.id when present", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
})
const fakeClient = {
models: {
generateContentStream: sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp_2",
candidates: [
{
content: {
parts: [
{
functionCall: {
id: "call_alpha",
name: "read_file",
args: { path: ".nvmrc" },
},
},
],
},
},
],
},
]),
),
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const tools = [{ name: "read_file", description: "read file", parameters: { type: "OBJECT" } }] as any
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
if (chunk.type === "tool_calls") {
chunks.push(chunk)
}
}
chunks.should.have.length(1)
chunks[0].tool_call.function.id.should.equal("call_alpha")
chunks[0].tool_call.call_id.should.equal("call_alpha")
JSON.parse(chunks[0].tool_call.function.arguments).path.should.equal(".nvmrc")
})
})
@@ -1,326 +0,0 @@
import { LiteLlmHandler, type LiteLlmModelInfoResponse } from "@core/api/providers/litellm"
import { convertToOpenAiMessages } from "@core/api/transform/openai-format"
import { liteLlmModelInfoSaneDefaults } from "@shared/api" // used in getModel tests
import { expect } from "chai"
import sinon from "sinon"
import { StateManager } from "@/core/storage/StateManager" // used in getModel tests
import { ClineStorageMessage } from "@/shared/messages/content"
import { mockFetchForTesting } from "@/shared/net"
const fakeClient = {
chat: {
completions: {
create: sinon.stub(),
},
},
baseURL: "https://fake.example",
}
describe("LiteLlmHandler", () => {
const mockFetch = sinon.stub()
let doneMockingFetch: (value: any) => void = () => {}
const mockModelFetch = (modelInfo: LiteLlmModelInfoResponse["data"][number]) => {
mockFetch.resolves({
ok: true,
json: () =>
Promise.resolve({
data: [modelInfo],
}),
})
}
let handler: LiteLlmHandler
const mockHandlerChat = () => {
sinon.stub(handler, "ensureClient" as any).returns(fakeClient)
}
const initializeHandler = (model: string) => {
handler = new LiteLlmHandler({
liteLlmApiKey: "test-api-key",
liteLlmBaseUrl: "http://localhost:4000",
liteLlmUsePromptCache: true,
liteLlmModelId: model,
})
mockHandlerChat()
}
beforeEach(() => {
fakeClient.chat.completions.create.resetHistory()
mockFetchForTesting(mockFetch, () => {
return new Promise((resolve) => {
doneMockingFetch = resolve
})
})
// Configure the stub to return a stream that closes immediately with usage data
fakeClient.chat.completions.create.resolves(
createAsyncIterable([
{
choices: [{ delta: { content: "test response" } }],
},
{
choices: [{}],
usage: {
prompt_tokens: 100,
completion_tokens: 50,
cache_creation_input_tokens: 20,
cache_read_input_tokens: 10,
},
},
]),
)
})
afterEach(() => {
sinon.reset()
doneMockingFetch(void 0)
})
const createAsyncIterable = (data: any[] = []) => {
return {
[Symbol.asyncIterator]: async function* () {
yield* data
},
}
}
describe("prompt cache", () => {
const setModelData = (model: string, supportsPromptCaching: boolean) => {
mockModelFetch({
model_name: model,
litellm_params: {
model,
},
model_info: {
supports_prompt_caching: supportsPromptCaching,
input_cost_per_token: 0.01,
output_cost_per_token: 0.02,
},
})
}
describe("when the model doesn't support prompt caching", () => {
const model = "openai/gpt-5"
beforeEach(() => {
initializeHandler(model)
setModelData(model, false)
})
it("sends the system prompt and messages with the openai format", async () => {
const systemPrompt = "Test System Prompt"
const messages: ClineStorageMessage[] = [
{
role: "user",
content: "first message",
},
{
role: "assistant",
content: "first response",
},
{
role: "user",
content: [
{
type: "text",
text: "test",
},
{
type: "text",
text: "second message",
},
],
},
]
for await (const _ of handler.createMessage(systemPrompt, messages)) {
}
sinon.assert.calledOnce(fakeClient.chat.completions.create)
const callArgs = fakeClient.chat.completions.create.getCall(0).args[0]
const systemPromptMessage = callArgs.messages.shift()
expect(systemPromptMessage).to.deep.equal({
role: "system",
content: systemPrompt,
})
expect(callArgs.messages).to.deep.equal(convertToOpenAiMessages(messages))
})
})
describe("when the model supports prompt caching", () => {
const model = "anthropic/claude-sonnet-4-20250514"
beforeEach(() => {
initializeHandler(model)
setModelData(model, true)
})
it("inserts the cache control in the system prompt and the last two user messages", async () => {
const systemPrompt = "Test System Prompt"
const messages: ClineStorageMessage[] = [
{
role: "user",
content: "first message",
},
{
role: "assistant",
content: "first response",
},
{
role: "user",
content: [
{
type: "text",
text: "test",
},
{
type: "text",
text: "second message",
},
],
},
]
for await (const _ of handler.createMessage(systemPrompt, messages)) {
}
sinon.assert.calledOnce(fakeClient.chat.completions.create)
const callArgs = fakeClient.chat.completions.create.getCall(0).args[0]
expect(callArgs.messages[0]).to.deep.equal({
role: "system",
content: [
{
text: systemPrompt,
type: "text",
cache_control: {
type: "ephemeral",
},
},
],
})
const sentMessages = callArgs.messages
expect(sentMessages.length).to.equal(4)
const firstUserMessage = sentMessages[1]
expect(firstUserMessage).to.deep.equal({
role: "user",
content: [
{
type: "text",
text: "first message",
cache_control: {
type: "ephemeral",
},
},
],
})
const lastUserMessage = sentMessages[3]
expect(lastUserMessage.content[0]).to.deep.equal({
type: "text",
text: "test",
})
const lastContentBlock = lastUserMessage.content[lastUserMessage.content.length - 1]
expect(lastContentBlock).to.deep.equal({
type: "text",
text: "second message",
cache_control: {
type: "ephemeral",
},
})
expect(callArgs.model).to.be.a("string")
expect(callArgs.stream).to.equal(true)
expect(callArgs.stream_options).to.deep.equal({ include_usage: true })
})
})
})
describe("getModel", () => {
let stateManagerStub: sinon.SinonStub
beforeEach(() => {
stateManagerStub = sinon.stub(StateManager, "get").returns({
getModelInfo: () => null,
} as any)
})
afterEach(() => {
stateManagerStub.restore()
})
it("returns sane defaults when no liteLlmModelInfo option is provided", () => {
const h = new LiteLlmHandler({
liteLlmApiKey: "test",
liteLlmModelId: "some-model",
})
const model = h.getModel()
expect(model.id).to.equal("some-model")
expect(model.info.contextWindow).to.equal(liteLlmModelInfoSaneDefaults.contextWindow)
})
it("returns user-configured model info when liteLlmModelInfo is provided and no cache exists", () => {
const h = new LiteLlmHandler({
liteLlmApiKey: "test",
liteLlmModelId: "claude-sonnet-4-6",
liteLlmModelInfo: {
contextWindow: 1_000_000,
maxTokens: 8192,
supportsImages: true,
supportsPromptCache: true,
inputPrice: 3,
outputPrice: 15,
cacheWritesPrice: 3.75,
cacheReadsPrice: 0.3,
},
})
const model = h.getModel()
expect(model.id).to.equal("claude-sonnet-4-6")
expect(model.info.contextWindow).to.equal(1_000_000)
})
it("prefers StateManager cached model info over user-configured liteLlmModelInfo", () => {
const cachedInfo = {
contextWindow: 200_000,
maxTokens: 4096,
supportsImages: true,
supportsPromptCache: true,
inputPrice: 0,
outputPrice: 0,
}
stateManagerStub.returns({
getModelInfo: () => cachedInfo,
} as any)
const h = new LiteLlmHandler({
liteLlmApiKey: "test",
liteLlmModelId: "claude-sonnet-4-6",
liteLlmModelInfo: {
contextWindow: 1_000_000,
maxTokens: 8192,
supportsImages: true,
supportsPromptCache: true,
inputPrice: 3,
outputPrice: 15,
cacheWritesPrice: 3.75,
cacheReadsPrice: 0.3,
},
})
const model = h.getModel()
expect(model.info.contextWindow).to.equal(200_000)
})
})
})
@@ -1,53 +0,0 @@
import "should"
import { moonshotModels } from "@shared/api"
import type { ClineStorageMessage } from "@shared/messages/content"
import sinon from "sinon"
import { MoonshotHandler } from "../moonshot"
interface MoonshotRequestPayload {
model: string
temperature: number
max_tokens: number
}
describe("MoonshotHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: unknown[] = []): AsyncIterable<unknown> => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
it("supports kimi-k2.6 model metadata", async () => {
const handler = new MoonshotHandler({
moonshotApiKey: "test-api-key",
apiModelId: "kimi-k2.6",
})
const model = handler.getModel()
model.id.should.equal("kimi-k2.6")
model.info.should.deepEqual(moonshotModels["kimi-k2.6"])
const createStub = sinon.stub().resolves(createAsyncIterable([]))
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
chat: {
completions: {
create: createStub,
},
},
})
const messages: ClineStorageMessage[] = [{ role: "user", content: "hi" }]
for await (const _chunk of handler.createMessage("system", messages)) {
// Consume stream to trigger request execution.
}
const payload = createStub.firstCall.args[0] as MoonshotRequestPayload
payload.model.should.equal("kimi-k2.6")
payload.temperature.should.equal(moonshotModels["kimi-k2.6"].temperature)
payload.max_tokens.should.equal(moonshotModels["kimi-k2.6"].maxTokens)
})
})
@@ -1,93 +0,0 @@
import { expect } from "chai"
import { afterEach, describe, it } from "mocha"
import sinon from "sinon"
import { ClineStorageMessage } from "@/shared/messages/content"
import { ApiFormat } from "@/shared/proto/index.cline"
import { OcaHandler } from "../oca"
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
async function collectChunks(stream: AsyncGenerator<any>) {
const chunks: any[] = []
for await (const chunk of stream) {
chunks.push(chunk)
}
return chunks
}
describe("OcaHandler.createMessage", () => {
afterEach(() => {
sinon.restore()
})
it("routes OPENAI_RESPONSES models to createMessageResponsesApi", async () => {
const handler = new OcaHandler({
ocaModelInfo: { apiFormat: ApiFormat.OPENAI_RESPONSES } as any,
})
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
yield { type: "text", text: "chat" }
})
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
yield { type: "text", text: "responses" }
})
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
yield { type: "text", text: "messages" }
})
const chunks = await collectChunks(handler.createMessage("system", messages))
expect(chunks).to.deep.equal([{ type: "text", text: "responses" }])
sinon.assert.notCalled(chatStub)
sinon.assert.calledOnce(responsesStub)
sinon.assert.notCalled(messagesStub)
})
it("routes ANTHROPIC_CHAT models to createMessageMessagesApi", async () => {
const handler = new OcaHandler({
ocaModelInfo: { apiFormat: ApiFormat.ANTHROPIC_CHAT } as any,
})
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
yield { type: "text", text: "chat" }
})
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
yield { type: "text", text: "responses" }
})
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
yield { type: "text", text: "messages" }
})
const chunks = await collectChunks(handler.createMessage("system", messages))
expect(chunks).to.deep.equal([{ type: "text", text: "messages" }])
sinon.assert.notCalled(chatStub)
sinon.assert.notCalled(responsesStub)
sinon.assert.calledOnce(messagesStub)
})
it("defaults to createMessageChatApi for OPENAI_CHAT and undefined apiFormat", async () => {
for (const apiFormat of [ApiFormat.OPENAI_CHAT, undefined]) {
const handler = new OcaHandler({
ocaModelInfo: { apiFormat } as any,
})
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
yield { type: "text", text: "chat" }
})
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
yield { type: "text", text: "responses" }
})
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
yield { type: "text", text: "messages" }
})
const chunks = await collectChunks(handler.createMessage("system", messages))
expect(chunks).to.deep.equal([{ type: "text", text: "chat" }])
sinon.assert.calledOnce(chatStub)
sinon.assert.notCalled(responsesStub)
sinon.assert.notCalled(messagesStub)
}
})
})
@@ -1,231 +0,0 @@
import { afterEach, before, beforeEach, describe, it } from "mocha"
import "should"
import { ApiHandlerOptions } from "@shared/api"
import axios from "axios"
import sinon from "sinon"
import { ClineStorageMessage } from "@/shared/messages/content"
import { OllamaHandler } from "../ollama"
describe("OllamaHandler", () => {
let ollamaAvailable = false
// Check if Ollama is running before running tests
before(async function () {
this.timeout(5000)
try {
await axios.get("http://localhost:11434/api/version", { timeout: 2000 })
ollamaAvailable = true
} catch (_error) {
console.log("Ollama server not available, skipping tests")
ollamaAvailable = false
}
})
let handler: OllamaHandler
let options: ApiHandlerOptions
let clock: sinon.SinonFakeTimers
beforeEach(() => {
options = {
actModeOllamaModelId: "llama2",
ollamaBaseUrl: "http://localhost:11434",
}
handler = new OllamaHandler(options)
// Use fake timers for testing timeouts
clock = sinon.useFakeTimers()
})
afterEach(() => {
clock.restore()
sinon.restore()
})
describe("createMessage", () => {
it("should handle successful responses", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(5000)
// Ensure client is initialized
const client = (handler as any).ensureClient()
// Mock the Ollama client's chat method
const chatStub = sinon.stub(client, "chat").resolves({
[Symbol.asyncIterator]: async function* () {
yield {
message: { content: "Hello, world!" },
eval_count: 10,
prompt_eval_count: 20,
}
},
} as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const result = []
const usageInfo = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "text") {
result.push(chunk.text)
} else if (chunk.type === "usage") {
usageInfo.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
})
}
}
// Verify the results
result.should.deepEqual(["Hello, world!"])
usageInfo.should.deepEqual([{ inputTokens: 20, outputTokens: 10 }])
chatStub.calledOnce.should.be.true()
})
it("should handle timeout errors", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(10000)
// Restore real timers for this test
clock.restore()
// Create a handler with a very short timeout for testing
const testHandler = new OllamaHandler(options)
// Replace the createMessage method with one that has a shorter timeout
testHandler.createMessage = async function* (_systemPrompt, _messages) {
try {
// Create a promise that rejects after a short timeout
const timeoutPromise = new Promise<never>((_, reject) => {
setTimeout(() => reject(new Error("Ollama request timed out after 120 seconds")), 100)
})
// Create a promise that never resolves
const neverPromise = new Promise(() => {})
// Race them
await Promise.race([timeoutPromise, neverPromise])
} catch (error: any) {
// Enhance error reporting
console.error(`Ollama API error: ${error.message}`)
throw error
}
}
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
// Start the request and catch the error
let errorMessage = ""
try {
for await (const _ of testHandler.createMessage(systemPrompt, messages)) {
// This should not be reached
}
} catch (error: any) {
errorMessage = error.message
}
// Check the result
errorMessage.should.equal("Ollama request timed out after 120 seconds")
// Restore the fake timers for other tests
clock = sinon.useFakeTimers()
})
it("should retry on errors when using the withRetry decorator", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(10000)
// Restore real timers for this test
clock.restore()
// Ensure client is initialized and mock the Ollama client's chat method to fail on first call and succeed on second
const client = (handler as any).ensureClient()
const chatStub = sinon.stub(client, "chat")
// First call throws an error
chatStub.onFirstCall().rejects(new Error("API Error"))
// Second call succeeds
chatStub.onSecondCall().resolves({
[Symbol.asyncIterator]: async function* () {
yield {
message: { content: "Success after retry" },
}
},
} as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const result = []
// Add a small delay to ensure the retry mechanism has time to work
await new Promise((resolve) => setTimeout(resolve, 100))
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "text") {
result.push(chunk.text)
}
}
// Verify the results
result.should.deepEqual(["Success after retry"])
chatStub.calledTwice.should.be.true()
// Restore the fake timers for other tests
clock = sinon.useFakeTimers()
})
it("should handle stream processing errors", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(10000)
// Restore real timers for this test
clock.restore()
// Create a handler with a custom implementation for testing
const testHandler = new OllamaHandler(options)
// Replace the createMessage method with one that simulates a stream error
testHandler.createMessage = async function* (_systemPrompt, _messages) {
// First yield a successful chunk
yield {
type: "text",
text: "Partial response",
}
// Then throw an error in the stream
throw new Error("Ollama stream processing error: Stream error")
}
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const result = []
// Collect the results and catch the error
let errorMessage = ""
try {
for await (const chunk of testHandler.createMessage(systemPrompt, messages)) {
if (chunk.type === "text") {
result.push(chunk.text)
}
}
} catch (error: any) {
errorMessage = error.message
}
// Verify the results
errorMessage.should.equal("Ollama stream processing error: Stream error")
result.should.deepEqual(["Partial response"])
// Restore the fake timers for other tests
clock = sinon.useFakeTimers()
})
})
})
@@ -1,164 +0,0 @@
import "should"
import { openRouterDefaultModelInfo } from "@shared/api"
import sinon from "sinon"
import { OpenRouterHandler } from "../openrouter"
describe("OpenRouterHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
const tools = [{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } }] as any
it("should handle usage-only chunks when delta is missing", async () => {
const handler = new OpenRouterHandler({
openRouterApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 13,
completion_tokens: 5,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "openai/gpt-4o-mini",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: 0,
inputTokens: 13,
outputTokens: 5,
totalCost: 0,
},
])
})
it("should read cache_write_tokens from prompt_tokens_details", async () => {
const handler = new OpenRouterHandler({
openRouterApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 1000,
completion_tokens: 200,
prompt_tokens_details: {
cached_tokens: 500,
cache_write_tokens: 300,
},
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "anthropic/claude-sonnet-4.6",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 300,
cacheReadTokens: 500,
inputTokens: 200,
outputTokens: 200,
totalCost: 0,
},
])
})
type ParallelToolCallsTestCase = {
modelId: string
enableParallelToolCalling: boolean
expectedParallelToolCalls: boolean
}
const parallelToolCallsTestCases: ParallelToolCallsTestCase[] = [
{
modelId: "openai/gpt-4o-mini",
enableParallelToolCalling: true,
expectedParallelToolCalls: true,
},
{
modelId: "openai/gpt-4o-mini",
enableParallelToolCalling: false,
expectedParallelToolCalls: false,
},
{
modelId: "google/gemini-3-flash-preview",
enableParallelToolCalling: true,
expectedParallelToolCalls: true,
},
]
for (const testCase of parallelToolCallsTestCases) {
const settingLabel = testCase.enableParallelToolCalling ? "enabled" : "disabled"
it(`should set parallel_tool_calls=${testCase.expectedParallelToolCalls} for ${testCase.modelId} when setting is ${settingLabel}`, async () => {
const handler = new OpenRouterHandler({
openRouterApiKey: "test-api-key",
enableParallelToolCalling: testCase.enableParallelToolCalling,
})
const createStub = sinon.stub().resolves(createAsyncIterable([]))
const fakeClient = {
chat: {
completions: {
create: createStub,
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: testCase.modelId,
info: openRouterDefaultModelInfo,
})
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
// drain stream
}
const payload = createStub.firstCall.args[0]
payload.parallel_tool_calls.should.equal(testCase.expectedParallelToolCalls)
})
}
})
@@ -1,132 +0,0 @@
import "should"
import { Anthropic } from "@anthropic-ai/sdk"
import { SapAiCoreHandler } from "../sapaicore"
describe("SapAiCoreHandler", () => {
let handler: SapAiCoreHandler
beforeEach(() => {
const mockOptions = {
sapAiCoreClientId: "test-client-id",
sapAiCoreClientSecret: "test-client-secret",
sapAiCoreTokenUrl: "https://test.auth.sap.com",
sapAiResourceGroup: "default",
sapAiCoreBaseUrl: "https://test.api.sap.com",
apiModelId: "anthropic--claude-3.5-sonnet",
}
handler = new SapAiCoreHandler(mockOptions)
})
describe("image processing", () => {
// Test image processing through the public interface
// This tests the complete flow including processImageContent internally
it("should handle image processing for Claude 4 models", () => {
// Create handler with Claude 4 model
const claude4Handler = new SapAiCoreHandler({
sapAiCoreClientId: "test-client-id",
sapAiCoreClientSecret: "test-client-secret",
sapAiCoreTokenUrl: "https://test.auth.sap.com",
sapAiResourceGroup: "default",
sapAiCoreBaseUrl: "https://test.api.sap.com",
apiModelId: "anthropic--claude-4-sonnet",
})
const model = claude4Handler.getModel()
model.id.should.equal("anthropic--claude-4-sonnet")
model.info.should.have.property("supportsImages", true)
})
it("should create proper user readable request with images", () => {
const testImageData =
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=="
const userContent: (Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[] = [
{
type: "text",
text: "Here's an image:",
},
{
type: "image",
source: {
type: "base64",
media_type: "image/png",
data: testImageData,
},
},
]
const result = handler.createUserReadableRequest(userContent)
result.should.have.property("model")
result.should.have.property("max_tokens")
result.should.have.property("system")
result.should.have.property("messages")
result.messages.should.be.Array()
result.messages[1].should.have.property("role", "user")
result.messages[1].should.have.property("content", userContent)
})
it("should support different Claude model variants", () => {
const modelVariants = [
"anthropic--claude-4.6-sonnet",
"anthropic--claude-4-sonnet",
"anthropic--claude-4-opus",
"anthropic--claude-3.7-sonnet",
"anthropic--claude-3.5-sonnet",
"anthropic--claude-3-sonnet",
"anthropic--claude-3-haiku",
"anthropic--claude-3-opus",
]
modelVariants.forEach((modelId) => {
const testHandler = new SapAiCoreHandler({
apiModelId: modelId,
})
const model = testHandler.getModel()
model.id.should.equal(modelId)
model.info.should.have.property("maxTokens")
model.info.should.have.property("contextWindow")
})
})
})
describe("getModel", () => {
it("should return default model when no apiModelId is provided", () => {
const result = handler.getModel()
result.should.have.property("id")
result.should.have.property("info")
result.info.should.have.property("maxTokens")
})
it("should return specified model when apiModelId is provided", () => {
const customHandler = new SapAiCoreHandler({
apiModelId: "anthropic--claude-4-sonnet",
})
const result = customHandler.getModel()
result.id.should.equal("anthropic--claude-4-sonnet")
})
})
describe("createUserReadableRequest", () => {
it("should create a readable request format", () => {
const userContent: Anthropic.TextBlockParam[] = [
{
type: "text",
text: "Hello, world!",
},
]
const result = handler.createUserReadableRequest(userContent)
result.should.have.property("model")
result.should.have.property("max_tokens")
result.should.have.property("system")
result.should.have.property("messages")
result.should.have.property("tools")
result.should.have.property("tool_choice")
})
})
})
@@ -1,138 +0,0 @@
import "should"
import { openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import sinon from "sinon"
import { VercelAIGatewayHandler } from "../vercel-ai-gateway"
describe("VercelAIGatewayHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
describe("getModel", () => {
it("should return configured model and info when both are provided", () => {
const customModelInfo = {
...openRouterDefaultModelInfo,
maxTokens: 123456,
}
const handler = new VercelAIGatewayHandler({
openRouterModelId: "google/gemini-3.1-pro-preview",
openRouterModelInfo: customModelInfo,
})
const result = handler.getModel()
result.id.should.equal("google/gemini-3.1-pro-preview")
result.info.should.deepEqual(customModelInfo)
})
it("should preserve configured model ID when model info is missing", () => {
const handler = new VercelAIGatewayHandler({
openRouterModelId: "google/gemini-3.1-pro-preview",
})
const result = handler.getModel()
result.id.should.equal("google/gemini-3.1-pro-preview")
result.info.should.deepEqual(openRouterDefaultModelInfo)
})
it("should fall back to default model when model ID is missing", () => {
const handler = new VercelAIGatewayHandler({})
const result = handler.getModel()
result.id.should.equal(openRouterDefaultModelId)
result.info.should.deepEqual(openRouterDefaultModelInfo)
})
})
describe("createMessage", () => {
it("should handle usage-only chunks when delta is missing", async () => {
const handler = new VercelAIGatewayHandler({
vercelAiGatewayApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 11,
completion_tokens: 7,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: 0,
inputTokens: 11,
outputTokens: 7,
totalCost: 0,
},
])
})
it("should read Anthropic-style cache creation and read tokens from usage chunks", async () => {
const handler = new VercelAIGatewayHandler({
vercelAiGatewayApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 1000,
completion_tokens: 200,
prompt_tokens_details: {
cached_tokens: 500,
},
cache_creation_input_tokens: 300,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 300,
cacheReadTokens: 500,
inputTokens: 200,
outputTokens: 200,
totalCost: 0,
},
])
})
})
})
@@ -1,23 +0,0 @@
import "should"
import { vertexGlobalModels } from "@shared/api"
import { VertexHandler } from "../vertex"
describe("VertexHandler", () => {
it("supports Gemini 3.5 Flash model metadata", () => {
const handler = new VertexHandler({
vertexProjectId: "test-project",
vertexRegion: "global",
apiModelId: "gemini-3.5-flash",
})
const model = handler.getModel()
model.id.should.equal("gemini-3.5-flash")
model.info.contextWindow!.should.equal(1_048_576)
model.info.inputPrice!.should.equal(1.5)
model.info.outputPrice!.should.equal(9)
model.info.cacheReadsPrice!.should.equal(0.15)
model.info.supportsGlobalEndpoint!.should.equal(true)
model.info.supportsReasoning!.should.equal(true)
vertexGlobalModels.should.have.property("gemini-3.5-flash")
})
})
@@ -1,42 +0,0 @@
import "should"
import { openAiModelInfoSaneDefaults, wandbDefaultModelId, wandbModels } from "@shared/api"
import { WandbHandler } from "../wandb"
describe("WandbHandler", () => {
it("returns known catalog model metadata when model id is recognized", () => {
const modelId = "meta-llama/Llama-3.3-70B-Instruct"
const handler = new WandbHandler({
wandbApiKey: "test-api-key",
apiModelId: modelId,
})
const model = handler.getModel()
model.id.should.equal(modelId)
model.info.should.deepEqual(wandbModels[modelId])
})
it("passes through an explicit unknown model id instead of silently falling back", () => {
const unknownModelId = "moonshotai/Kimi-K2.5"
const handler = new WandbHandler({
wandbApiKey: "test-api-key",
apiModelId: unknownModelId,
})
const model = handler.getModel()
model.id.should.equal(unknownModelId)
model.info.should.deepEqual(openAiModelInfoSaneDefaults)
})
it("uses the default W&B model when no model id is configured", () => {
const handler = new WandbHandler({
wandbApiKey: "test-api-key",
})
const model = handler.getModel()
model.id.should.equal(wandbDefaultModelId)
model.info.should.deepEqual(wandbModels[wandbDefaultModelId])
})
})
@@ -1,327 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { GenerateContentConfig, GoogleGenAI } from "@google/genai"
import { ModelInfo } from "@shared/api"
import OpenAI from "openai"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { convertAnthropicMessageToGemini } from "../transform/gemini-format"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface AIhubmixHandlerOptions extends CommonApiHandlerOptions {
apiKey?: string
baseURL?: string
appCode?: string
modelId?: string
modelInfo?: ModelInfo
thinkingBudgetTokens?: number
}
export class AIhubmixHandler implements ApiHandler {
private options: AIhubmixHandlerOptions
private anthropicClient: Anthropic | undefined
private openaiClient: OpenAI | undefined
private geminiClient: GoogleGenAI | undefined
constructor(options: AIhubmixHandlerOptions) {
const { baseURL, appCode, ...rest } = options
this.options = {
baseURL: baseURL ?? "https://aihubmix.com",
appCode: appCode ?? "KUWF9311",
...rest,
}
}
private ensureAnthropicClient(): Anthropic {
if (!this.anthropicClient) {
if (!this.options.apiKey) {
throw new Error("AIhubmix API key is required")
}
try {
this.anthropicClient = new Anthropic({
apiKey: this.options.apiKey,
baseURL: this.options.baseURL,
defaultHeaders: {
"APP-Code": this.options.appCode,
...buildExternalBasicHeaders(),
},
})
} catch (error) {
throw new Error(`Error creating Anthropic client: ${error.message}`)
}
}
return this.anthropicClient
}
private ensureOpenaiClient(): OpenAI {
if (!this.openaiClient) {
if (!this.options.apiKey) {
throw new Error("AIhubmix API key is required")
}
try {
this.openaiClient = new OpenAI({
apiKey: this.options.apiKey,
baseURL: `${this.options.baseURL}/v1`,
defaultHeaders: {
"APP-Code": this.options.appCode,
...buildExternalBasicHeaders(),
},
})
} catch (error) {
throw new Error(`Error creating OpenAI client: ${error.message}`)
}
}
return this.openaiClient
}
private ensureGeminiClient(): GoogleGenAI {
if (!this.geminiClient) {
if (!this.options.apiKey) {
throw new Error("AIhubmix API key is required")
}
try {
this.geminiClient = new GoogleGenAI({
apiKey: this.options.apiKey,
httpOptions: {
baseUrl: `${this.options.baseURL}/gemini`,
headers: {
// @ts-expect-error
"APP-Code": this.options.appCode,
Authorization: `Bearer ${this.options.apiKey ?? ""}`,
...buildExternalBasicHeaders(),
},
},
})
} catch (error) {
throw new Error(`Error creating Gemini client: ${error.message}`)
}
}
return this.geminiClient
}
private routeModel(modelName: string): "anthropic" | "openai" | "gemini" | "openai-response" {
const id = modelName || ""
if (id.startsWith("claude")) {
return "anthropic"
}
if (id.startsWith("gemini") && !id.endsWith("-nothink") && !id.endsWith("-search")) {
return "gemini"
}
if (id === "gpt-5-pro" || id === "gpt-5-codex") {
return "openai-response"
}
return "openai"
}
private fixToolChoice(requestBody: any): any {
if (requestBody.tools?.length === 0 && requestBody.tool_choice) {
delete requestBody.tool_choice
}
return requestBody
}
@withRetry()
async *createMessage(systemPrompt: string, messages: any[]): ApiStream {
const modelId = this.options.modelId || ""
const route = this.routeModel(modelId)
switch (route) {
case "anthropic":
yield* this.createAnthropicMessage(systemPrompt, messages)
break
case "gemini":
yield* this.createGeminiMessage(systemPrompt, messages)
break
case "openai-response":
yield* this.createOpenaiResponseMessage(systemPrompt, messages)
break
case "openai":
yield* this.createOpenaiMessage(systemPrompt, messages)
break
default:
throw new Error(`Unsupported model route: ${route}`)
}
}
private async *createAnthropicMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureAnthropicClient()
const modelId = this.options.modelId || "claude-3-5-sonnet-20241022"
// Sanitize messages to remove Cline-specific fields like call_id that are not allowed by Anthropic API
const sanitizedMessages = sanitizeAnthropicMessages(messages, false)
const stream = await client.messages.create({
model: modelId,
temperature: 0,
max_tokens: this.options.modelInfo?.maxTokens || 8192,
system: [{ text: systemPrompt, type: "text" }],
messages: sanitizedMessages,
stream: true,
})
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start":
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
break
case "message_delta":
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage.output_tokens || 0,
}
break
case "content_block_start":
if (chunk.content_block.type === "text") {
yield {
type: "text",
text: chunk.content_block.text,
}
}
break
case "content_block_delta":
if (chunk.delta.type === "text_delta") {
yield {
type: "text",
text: chunk.delta.text,
}
}
break
}
}
}
private async *createOpenaiResponseMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureOpenaiClient()
const modelId = this.options.modelId || "gpt-4o-mini"
const input = (messages || []).map((m: any) => {
const role = m.role || "user"
const contentArray = Array.isArray(m.content) ? m.content : [{ type: "text", text: m.content }]
const content = contentArray
.filter((c: any) => c != null)
.map((c: any) => {
if (c.type === "image" || c.type === "input_image" || c.type === "image_url") {
return { type: "input_image", image_url: c.image_url || c.url || c.source?.url }
}
const text = c.text ?? (typeof c === "string" ? c : "")
return { type: role === "assistant" ? "output_text" : "input_text", text }
})
return { role, content }
})
const stream = await (client as any).responses.stream({
model: modelId,
instructions: systemPrompt,
input,
})
for await (const event of stream as any) {
if (event?.type === "response.output_text.delta") {
yield { type: "text", text: event.delta || "" }
continue
}
if (event?.type === "response.completed") {
const usage = event.response?.usage || {}
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
}
continue
}
if (event?.type === "response.error") {
throw new Error(event.error?.message || "responses error")
}
}
}
private async *createOpenaiMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureOpenaiClient()
const modelId = this.options.modelId || "gpt-4o-mini"
const openaiMessages = [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
const requestBody = {
model: modelId,
messages: openaiMessages,
temperature: 0,
stream: true,
}
const fixedRequestBody = this.fixToolChoice(requestBody)
const stream = await client.chat.completions.create(fixedRequestBody)
for await (const chunk of stream as any) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
private async *createGeminiMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureGeminiClient()
const modelId = this.options.modelId || "gemini-2.0-flash-exp"
const contents = messages.map(convertAnthropicMessageToGemini)
const requestConfig: GenerateContentConfig = {
systemInstruction: systemPrompt,
temperature: 0,
}
if (this.options.thinkingBudgetTokens) {
requestConfig.thinkingConfig = {
thinkingBudget: this.options.thinkingBudgetTokens,
includeThoughts: true,
}
}
const stream = await client.models.generateContentStream({
model: modelId,
contents,
config: requestConfig,
})
for await (const chunk of stream as any) {
if (chunk?.text) {
yield { type: "text", text: chunk.text }
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.modelId || "gpt-4o-mini",
info: this.options.modelInfo || {
maxTokens: 8192,
contextWindow: 128000,
supportsImages: true,
supportsPromptCache: false,
description: "AIhubmix unified model provider",
},
}
}
}
@@ -1,316 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import type {
MessageCreateParamsStreaming as BetaMessageCreateParamsStreaming,
BetaRawMessageStreamEvent,
} from "@anthropic-ai/sdk/resources/beta/messages/messages"
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
import type { MessageCreateParamsStreaming as AnthropicMessageCreateParamsStreaming } from "@anthropic-ai/sdk/resources/messages/messages"
import { Stream as AnthropicStream } from "@anthropic-ai/sdk/streaming"
import {
ANTHROPIC_FAST_MODE_SUFFIX,
AnthropicModelId,
anthropicDefaultModelId,
anthropicModels,
CLAUDE_SONNET_1M_SUFFIX,
ModelInfo,
} from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { ApiStream } from "../transform/stream"
export const ANTHROPIC_FAST_MODE_BETA = "fast-mode-2026-02-01"
interface AnthropicHandlerOptions extends CommonApiHandlerOptions {
apiKey?: string
anthropicBaseUrl?: string
apiModelId?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
}
export class AnthropicHandler implements ApiHandler {
private options: AnthropicHandlerOptions
private client: Anthropic | undefined
constructor(options: AnthropicHandlerOptions) {
this.options = options
}
private ensureClient(): Anthropic {
if (!this.client) {
if (!this.options.apiKey) {
throw new Error("Anthropic API key is required")
}
try {
this.client = new Anthropic({
apiKey: this.options.apiKey,
baseURL: this.options.anthropicBaseUrl || undefined,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating Anthropic client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: AnthropicTool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
let stream: AnthropicStream<Anthropic.RawMessageStreamEvent> | AsyncIterable<BetaRawMessageStreamEvent>
const useFastMode = model.id.endsWith(ANTHROPIC_FAST_MODE_SUFFIX)
const baseModelId = useFastMode ? model.id.slice(0, -ANTHROPIC_FAST_MODE_SUFFIX.length) : model.id
const modelId = baseModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
? baseModelId.slice(0, -CLAUDE_SONNET_1M_SUFFIX.length)
: baseModelId
const enable1mContextWindow = baseModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
const fastModeBetas = enable1mContextWindow
? [ANTHROPIC_FAST_MODE_BETA, "context-1m-2025-08-07"]
: [ANTHROPIC_FAST_MODE_BETA]
const createFastModeMessage = (
body: AnthropicMessageCreateParamsStreaming,
): Promise<AsyncIterable<BetaRawMessageStreamEvent>> => {
return (
client.beta.messages.create as unknown as (
params: BetaMessageCreateParamsStreaming & { speed: "fast" },
) => Promise<AsyncIterable<BetaRawMessageStreamEvent>>
)({
...body,
betas: fastModeBetas,
speed: "fast",
})
}
const budget_tokens = this.options.thinkingBudgetTokens || 0
// Tools are available only when native tools are enabled.
const nativeToolsOn = tools?.length && tools?.length > 0
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
// Claude Opus 4.5+ uses adaptive thinking instead of budgeted extended thinking.
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budget_tokens)
: undefined
const adaptiveThinkingEnabled = adaptiveThinking?.enabled === true
const adaptiveThinkingEffort = adaptiveThinking?.effort
const thinkingEnabled = isAdaptiveThinkingModel ? adaptiveThinkingEnabled : reasoningOn
const thinkingConfig = thinkingEnabled
? isAdaptiveThinkingModel
? ({ type: "adaptive" } as any)
: { type: "enabled", budget_tokens: budget_tokens }
: undefined
const outputConfig = isAdaptiveThinkingModel && adaptiveThinkingEffort ? { effort: adaptiveThinkingEffort } : undefined
if (model.info.supportsPromptCache) {
const anthropicMessages = sanitizeAnthropicMessages(messages, true)
const requestBody: AnthropicMessageCreateParamsStreaming & Record<string, unknown> = {
model: modelId,
thinking: thinkingConfig,
max_tokens: model.info.maxTokens || 8192,
// "Thinking isn't compatible with temperature, top_p, or top_k modifications as well as forced tool use."
// (https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#important-considerations-when-using-extended-thinking)
// Adaptive Claude Opus models do not support temperature.
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
system: [
{
text: systemPrompt,
type: "text",
cache_control: { type: "ephemeral" },
},
], // setting cache breakpoint for system prompt so new tasks can reuse it
messages: anthropicMessages,
// tools, // cache breakpoints go from tools > system > messages, and since tools dont change, we can just set the breakpoint at the end of system (this avoids having to set a breakpoint at the end of tools which by itself does not meet min requirements for haiku caching)
stream: true,
tools: nativeToolsOn ? tools : undefined,
// tool_choice options:
// - none: disables tool use, even if tools are provided. Claude will not call any tools.
// - auto: allows Claude to decide whether to call any provided tools or not. This is the default value when tools are provided.
// - any: tells Claude that it must use one of the provided tools, but doesnt force a particular tool.
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
tool_choice: nativeToolsOn && !thinkingEnabled ? { type: "any" } : undefined,
}
if (outputConfig) {
requestBody.output_config = outputConfig
}
stream = useFastMode
? await createFastModeMessage(requestBody)
: await client.messages.create(
requestBody,
(() => {
// 1m context window beta header
if (enable1mContextWindow) {
return {
headers: {
"anthropic-beta": "context-1m-2025-08-07",
},
}
}
return undefined
})(),
)
} else {
const requestBody: AnthropicMessageCreateParamsStreaming & Record<string, unknown> = {
model: modelId,
max_tokens: model.info.maxTokens || 8192,
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
system: [{ text: systemPrompt, type: "text" }],
messages: sanitizeAnthropicMessages(messages, false),
tools: nativeToolsOn ? tools : undefined,
tool_choice: thinkingEnabled ? undefined : { type: "auto" },
stream: true,
thinking: thinkingConfig,
}
if (outputConfig) {
requestBody.output_config = outputConfig
}
stream = useFastMode ? await createFastModeMessage(requestBody) : await client.messages.create(requestBody)
}
const lastStartedToolCall = { id: "", name: "", arguments: "" }
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start":
{
// tells us cache reads/writes/input/output
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
}
break
case "message_delta":
// tells us stop_reason, stop_sequence, and output tokens along the way and at the end of the message
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage.output_tokens || 0,
}
break
case "message_stop":
// no usage data, just an indicator that the message is done
break
case "content_block_start":
switch (chunk.content_block.type) {
case "thinking":
yield {
type: "reasoning",
reasoning: chunk.content_block.thinking || "",
signature: chunk.content_block.signature,
}
break
case "redacted_thinking":
// Content is encrypted, and we don't to pass placeholder text back to the API
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
redacted_data: chunk.content_block.data,
}
break
case "tool_use":
if (chunk.content_block.id && chunk.content_block.name) {
// Convert Anthropic tool_use to OpenAI-compatible format
lastStartedToolCall.id = chunk.content_block.id
lastStartedToolCall.name = chunk.content_block.name
lastStartedToolCall.arguments = ""
}
break
case "text":
// we may receive multiple text blocks, in which case just insert a line break between them
if (chunk.index > 0) {
yield {
type: "text",
text: "\n",
}
}
yield {
type: "text",
text: chunk.content_block.text,
}
break
}
break
case "content_block_delta":
switch (chunk.delta.type) {
case "thinking_delta":
// 'reasoning' type just displays in the UI, but ant_thinking will be used to send the thinking traces back to the API
yield {
type: "reasoning",
reasoning: chunk.delta.thinking,
}
break
case "signature_delta":
// It's used when sending the thinking block back to the API
// API expects this in completed form, not as array of deltas
if (chunk.delta.signature) {
yield {
type: "reasoning",
reasoning: "", // reasoning text is already sent via thinking_delta
signature: chunk.delta.signature,
}
}
break
case "text_delta":
yield {
type: "text",
text: chunk.delta.text,
}
break
case "input_json_delta":
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
// // Convert Anthropic tool_use to OpenAI-compatible format
yield {
type: "tool_calls",
tool_call: {
...lastStartedToolCall,
function: {
...lastStartedToolCall,
id: lastStartedToolCall.id,
name: lastStartedToolCall.name,
arguments: chunk.delta.partial_json,
},
},
}
}
break
}
break
case "content_block_stop":
lastStartedToolCall.id = ""
lastStartedToolCall.name = ""
lastStartedToolCall.arguments = ""
break
}
}
}
getModel(): { id: AnthropicModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in anthropicModels) {
const id = modelId as AnthropicModelId
return { id, info: anthropicModels[id] }
}
return {
id: anthropicDefaultModelId,
info: anthropicModels[anthropicDefaultModelId],
}
}
}
@@ -1,200 +0,0 @@
import { AskSageModelId, askSageDefaultModelId, askSageDefaultURL, askSageModels, ModelInfo } from "@shared/api"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
interface AskSageHandlerOptions extends CommonApiHandlerOptions {
asksageApiKey?: string
asksageApiUrl?: string
apiModelId?: string
}
type AskSageRequest = {
system_prompt: string
message: {
user: "gpt" | "me"
message: string
}[]
model: string
dataset: "none"
usage: boolean
}
type AskSageUsage = {
model_tokens: {
completion_tokens: number
prompt_tokens: number
total_tokens: number
}
asksage_tokens: number
}
type AskSageResponse = {
uuid: string
status: number
// Response status
response: string
// Generated response message
message: string
// whether embedding & vector systems are down
embedding_down: boolean
vectors_down: boolean
// references if dataset is not none
references: string
type: string
added_obj: any
tool_calls: any
// usage metrics
usage: AskSageUsage | null
tool_responses: any[]
tool_calls_unified: any[]
}
export class AskSageHandler implements ApiHandler {
private options: AskSageHandlerOptions
private apiUrl: string
private apiKey: string
constructor(options: AskSageHandlerOptions) {
Logger.log("init api url", options.asksageApiUrl, askSageDefaultURL)
this.options = options
this.apiKey = options.asksageApiKey || ""
this.apiUrl = options.asksageApiUrl || askSageDefaultURL
if (!this.apiKey) {
throw new Error("AskSage API key is required")
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
try {
const model = this.getModel()
// Transform messages into AskSageRequest format
const formattedMessages = messages.map((msg) => {
const content = Array.isArray(msg.content)
? msg.content.map((block) => ("text" in block ? block.text : "")).join("")
: msg.content
return {
user: msg.role === "assistant" ? ("gpt" as const) : ("me" as const),
message: content,
}
})
const request: AskSageRequest = {
system_prompt: systemPrompt,
message: formattedMessages,
model: model.id,
dataset: "none",
usage: true,
}
// Make request to AskSage API
const response = await fetch(`${this.apiUrl}/query`, {
method: "POST",
headers: this.headers(),
body: JSON.stringify(request),
})
if (!response.ok) {
const error = await response.text()
throw new Error(`AskSage API error: ${error}`)
}
const result = (await response.json()) as AskSageResponse
if (!result.message) {
throw new Error("No content in AskSage response")
}
// Yield tool responses if they exist
if (result.tool_responses && result.tool_responses.length > 0) {
for (const toolResponse of result.tool_responses) {
yield {
type: "text",
text: `[Tool Response: ${JSON.stringify(toolResponse)}]\n`,
}
}
}
// Yield the main response text
yield {
type: "text",
text: result.message,
}
// Yield usage information if available
if (result.usage) {
yield {
type: "usage",
inputTokens: result.usage.model_tokens.prompt_tokens,
outputTokens: result.usage.model_tokens.completion_tokens,
cacheReadTokens: 0,
cacheWriteTokens: 0,
totalCost: result.usage.asksage_tokens, // Cost = Consumed AskSage tokens
}
}
} catch (error) {
if (error instanceof Error) {
throw new Error(`AskSage request failed: ${error.message}`)
}
throw error
}
}
async getApiStreamUsage() {
if (!this.apiKey) {
return undefined
}
try {
const response = await fetch(`${this.apiUrl}/count-monthly-tokens`, {
method: "POST",
headers: this.headers(),
body: JSON.stringify({ app_name: "asksage" }),
})
if (!response.ok) {
Logger.error("Failed to fetch AskSage usage", await response.text())
return undefined
}
const data = await response.json()
const usedTokens = data.response as number
return {
type: "usage" as const,
inputTokens: usedTokens,
outputTokens: 0,
}
} catch (error) {
Logger.error("Error fetching AskSage usage:", error)
return undefined
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in askSageModels) {
const id = modelId as AskSageModelId
return { id, info: askSageModels[id] }
}
return {
id: askSageDefaultModelId,
info: askSageModels[askSageDefaultModelId],
}
}
private headers() {
return {
"Content-Type": "application/json",
"x-access-tokens": this.apiKey,
...buildExternalBasicHeaders(),
}
}
}
@@ -1,172 +0,0 @@
import { BasetenModelId, basetenDefaultModelId, basetenModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
interface BasetenHandlerOptions extends CommonApiHandlerOptions {
basetenApiKey?: string
basetenModelId?: string
basetenModelInfo?: ModelInfo
apiModelId?: string // For backward compatibility
}
export class BasetenHandler implements ApiHandler {
private options: BasetenHandlerOptions
private client: OpenAI | undefined
constructor(options: BasetenHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.basetenApiKey) {
throw new Error("Baseten API key is required")
}
try {
this.client = new OpenAI({
baseURL: "https://inference.baseten.co/v1",
apiKey: this.options.basetenApiKey,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating Baseten client: ${error.message}`)
}
}
return this.client
}
/**
* Gets the optimal max_tokens based on model capabilities
*/
private getOptimalMaxTokens(model: { id: BasetenModelId; info: ModelInfo }): number {
// Use model-specific max tokens if available
if (model.info.maxTokens && model.info.maxTokens > 0) {
return model.info.maxTokens
}
// Default fallback
return 8192
}
getModel(): { id: BasetenModelId; info: ModelInfo } {
// First priority: basetenModelId and basetenModelInfo
const basetenModelId = this.options.basetenModelId
const basetenModelInfo = this.options.basetenModelInfo
if (basetenModelId && basetenModelInfo) {
return { id: basetenModelId as BasetenModelId, info: basetenModelInfo }
}
// Second priority: basetenModelId with static model info
if (basetenModelId && basetenModelId in basetenModels) {
const id = basetenModelId as BasetenModelId
return { id, info: basetenModels[id] }
}
// Third priority: apiModelId (for backward compatibility)
const apiModelId = this.options.apiModelId
if (apiModelId && apiModelId in basetenModels) {
const id = apiModelId as BasetenModelId
return { id, info: basetenModels[id] }
}
// Default fallback
return {
id: basetenDefaultModelId,
info: basetenModels[basetenDefaultModelId],
}
}
private async *yieldUsage(modelInfo: ModelInfo, usage: any): ApiStream {
if (usage.prompt_tokens || usage.completion_tokens) {
const cost = calculateApiCostOpenAI(modelInfo, usage.prompt_tokens || 0, usage.completion_tokens || 0)
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
cacheWriteTokens: 0,
cacheReadTokens: 0,
totalCost: cost,
}
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const maxTokens = this.getOptimalMaxTokens(model)
const toolCallProcessor = new ToolCallProcessor()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_tokens: maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
tools,
tool_choice: tools && tools.length > 0 ? "auto" : undefined,
})
let didOutputUsage = false
for await (const chunk of stream) {
const delta = chunk?.choices?.[0]?.delta
// Handle reasoning field if present (for reasoning models with parsed output)
if (delta && "reasoning" in delta && delta?.reasoning) {
const reasoning = typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning)
yield {
type: "reasoning",
reasoning,
}
}
// Handle content field
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle usage information - only output once
if (!didOutputUsage && chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
didOutputUsage = true
}
}
}
/**
* Checks if the current model supports tools
*/
supportsTools(): boolean {
const model = this.getModel()
const modelInfo = model.info as any
// Use dynamic API data when available, fallback to true since all current Baseten models support tools
// (as of 2025-09-16 - could change if Baseten add non-tool models in future, currently no plans to do so)
return modelInfo.supportedFeatures ? modelInfo.supportedFeatures.includes("tools") : true
}
}
File diff suppressed because it is too large Load Diff
@@ -1,275 +0,0 @@
import Cerebras from "@cerebras/cerebras_cloud_sdk"
import { CerebrasModelId, cerebrasDefaultModelId, cerebrasModels, ModelInfo } from "@shared/api"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
interface CerebrasHandlerOptions extends CommonApiHandlerOptions {
cerebrasApiKey?: string
apiModelId?: string
}
// Conservative max_tokens for Cerebras to avoid premature rate limiting.
// Cerebras rate limiter estimates token consumption using max_completion_tokens upfront,
// so requesting the model maximum (e.g., 64K) reserves that quota even if actual usage is low.
// 16K is sufficient for most agentic tool use while preserving rate limit headroom.
const CEREBRAS_DEFAULT_MAX_TOKENS = 16_384
export class CerebrasHandler implements ApiHandler {
private options: CerebrasHandlerOptions
private client: Cerebras | undefined
constructor(options: CerebrasHandlerOptions) {
this.options = options
}
private ensureClient(): Cerebras {
if (!this.client) {
// Clean and validate the API key
const cleanApiKey = this.options.cerebrasApiKey?.trim()
if (!cleanApiKey) {
throw new Error("Cerebras API key is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
this.client = new Cerebras({
apiKey: cleanApiKey,
timeout: 30000, // 30 second timeout
fetch, // Use configured fetch with proxy support
defaultHeaders: {
...externalHeaders,
"X-Cerebras-3rd-Party-Integration": "cline",
},
})
} catch (error) {
throw new Error(`Error creating Cerebras client: ${error.message}`)
}
}
return this.client
}
@withRetry({
maxRetries: 6, // More retries to be patient with rate limits
baseDelay: 5000, // Start with 5 second delay
maxDelay: 60000, // Allow up to 60 second delays to respect rate limits
})
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
// Convert Anthropic messages to Cerebras format
const cerebrasMessages: Array<{
role: "system" | "user" | "assistant"
content: string
}> = [{ role: "system", content: systemPrompt }]
// Helper function to strip thinking tags from content
const stripThinkingTags = (content: string): string => {
return content.replace(/<think>[\s\S]*?<\/think>/g, "").trim()
}
// Check if this is a reasoning model that uses thinking tags
const modelId = this.getModel().id
const isReasoningModel = modelId.includes("qwen")
// Convert Anthropic messages to Cerebras format
for (const message of messages) {
if (message.role === "user") {
const content = Array.isArray(message.content)
? message.content
.map((block) => {
if (block.type === "text") {
return block.text
}
if (block.type === "image") {
return "[Image content not supported in Cerebras]"
}
return ""
})
.join("\n")
: message.content
cerebrasMessages.push({ role: "user", content })
} else if (message.role === "assistant") {
let content = Array.isArray(message.content)
? message.content
.map((block) => {
if (block.type === "text") {
return block.text
}
return ""
})
.join("\n")
: message.content || ""
// Strip thinking tags from assistant messages for reasoning models
// so the model doesn't see its own thinking in the conversation history
if (isReasoningModel) {
content = stripThinkingTags(content)
}
cerebrasMessages.push({ role: "assistant", content })
}
}
try {
const model = this.getModel()
const stream = await client.chat.completions.create({
model: model.id,
messages: cerebrasMessages,
temperature: model.info.temperature ?? 0,
stream: true,
max_tokens: CEREBRAS_DEFAULT_MAX_TOKENS,
})
// Handle streaming response
let reasoning: string | null = null // Track reasoning content for models that support thinking
for await (const chunk of stream as any) {
// Type assertion for the streaming chunk
const streamChunk = chunk as any
if (streamChunk.choices?.[0]?.delta?.content) {
const content = streamChunk.choices[0].delta.content
// Handle reasoning models (Qwen and DeepSeek R1 Distill) that use <think> tags
if (isReasoningModel) {
// Check if we're entering or continuing reasoning mode
if (reasoning || content.includes("<think>")) {
reasoning = (reasoning || "") + content
// Clean the content by removing think tags for display
const cleanContent = content.replace(/<think>/g, "").replace(/<\/think>/g, "")
// Only yield reasoning content if there's actual content after cleaning
if (cleanContent.trim()) {
yield {
type: "reasoning",
reasoning: cleanContent,
}
}
// Check if reasoning is complete
if (reasoning.includes("</think>")) {
reasoning = null
}
} else {
// Regular content outside of thinking tags
yield {
type: "text",
text: content,
}
}
} else {
// Non-reasoning models - just yield text content
yield {
type: "text",
text: content,
}
}
}
// Handle usage information from Cerebras API
// Usage is typically only available in the final chunk
if (streamChunk.usage) {
const totalCost = this.calculateCost({
inputTokens: streamChunk.usage.prompt_tokens || 0,
outputTokens: streamChunk.usage.completion_tokens || 0,
})
yield {
type: "usage",
inputTokens: streamChunk.usage.prompt_tokens || 0,
outputTokens: streamChunk.usage.completion_tokens || 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
totalCost,
}
}
}
} catch (error: any) {
// Enhanced error handling for Cerebras API
if (error?.status === 429 || error?.code === "rate_limit_exceeded") {
// Rate limit error - will be handled by retry decorator with patient backoff
const _limits = this.getRateLimits()
throw new Error(`Cerebras API rate limit exceeded.`)
}
if (error?.status === 401) {
throw new Error("Cerebras API authentication failed. Please check your API key.")
}
if (error?.status === 403) {
throw new Error("Cerebras API access denied. Please check your API key permissions.")
}
if (error?.status >= 500) {
// Server errors - retryable
throw new Error(`Cerebras API server error (${error.status}): ${error.message || "Unknown server error"}`)
}
if (error?.status === 400) {
// Client errors - not retryable
throw new Error(`Cerebras API bad request: ${error.message || "Invalid request parameters"}`)
}
// Re-throw original error for other cases
throw error
}
}
getModel(): { id: string; info: ModelInfo } {
const originalModelId = this.options.apiModelId
let apiModelId = originalModelId
if (originalModelId === "qwen-3-coder-480b-free") {
apiModelId = "qwen-3-coder-480b"
return { id: apiModelId, info: cerebrasModels[originalModelId as CerebrasModelId] }
}
if (originalModelId && originalModelId in cerebrasModels) {
const id = originalModelId as CerebrasModelId
return { id, info: cerebrasModels[id] }
}
return {
id: cerebrasDefaultModelId,
info: cerebrasModels[cerebrasDefaultModelId],
}
}
/**
* Get rate limit information for the current model
*
* These limits are used for informational purposes and to calculate appropriate
* retry delays. Since Cerebras inference is extremely fast, users hit these limits
* quickly, so we need to be patient with retries to maximize usage efficiency.
*
* @returns Rate limit configuration for the model
*/
private getRateLimits(): { requestsPerMinute: number; tokensPerMinute: number } {
const modelId = this.getModel().id
switch (modelId) {
case "qwen-3-coder-480b":
case "qwen-3-coder-480b-free":
return { requestsPerMinute: 10, tokensPerMinute: 150_000 }
case "qwen-3-235b-a22b-instruct-2507":
case "qwen-3-235b-a22b-thinking-2507":
return { requestsPerMinute: 30, tokensPerMinute: 60_000 }
case "gpt-oss-120b":
return { requestsPerMinute: 30, tokensPerMinute: 64_000 }
default:
// Default rate limits for unknown models
return { requestsPerMinute: 30, tokensPerMinute: 60_000 }
}
}
private calculateCost({ inputTokens, outputTokens }: { inputTokens: number; outputTokens: number }): number {
const model = this.getModel()
const inputPrice = model.info.inputPrice || 0
const outputPrice = model.info.outputPrice || 0
const inputCost = (inputPrice / 1_000_000) * inputTokens
const outputCost = (outputPrice / 1_000_000) * outputTokens
return inputCost + outputCost
}
}
@@ -1,229 +0,0 @@
import { filterMessagesForClaudeCode } from "@/integrations/claude-code/message-filter"
import { runClaudeCode } from "@/integrations/claude-code/run"
import { ClaudeCodeModelId, claudeCodeDefaultModelId, claudeCodeModels } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { type ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { type ApiStream, ApiStreamUsageChunk } from "../transform/stream"
interface ClaudeCodeHandlerOptions extends CommonApiHandlerOptions {
claudeCodePath?: string
apiModelId?: string
thinkingBudgetTokens?: number
}
export class ClaudeCodeHandler implements ApiHandler {
private options: ClaudeCodeHandlerOptions
constructor(options: ClaudeCodeHandlerOptions) {
this.options = options
}
@withRetry({
maxRetries: 4,
baseDelay: 2000,
maxDelay: 15000,
})
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
// Filter out image blocks since Claude Code doesn't support them
const filteredMessages = filterMessagesForClaudeCode(messages)
const claudeProcess = runClaudeCode({
systemPrompt,
messages: filteredMessages,
path: this.options.claudeCodePath,
modelId: this.getModel().id,
thinkingBudgetTokens: this.options.thinkingBudgetTokens,
})
// Usage is included with assistant messages,
// but cost is included in the result chunk
const usage: ApiStreamUsageChunk = {
type: "usage",
inputTokens: 0,
outputTokens: 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
}
let isPaidUsage = true
for await (const chunk of claudeProcess) {
if (typeof chunk === "string") {
yield {
type: "text",
text: chunk,
}
continue
}
// Handle system init messages
if (chunk.type === "system" && "subtype" in chunk) {
if (chunk.subtype === "init") {
// Based on my tests, subscription usage sets the `apiKeySource` to "none"
isPaidUsage = (chunk as any).apiKeySource !== "none"
}
// Also handles legacy rate_limit_event format (type: "system", subtype: "rate_limit_event")
// by falling through — no special handling needed.
continue
}
// Handle rate_limit_event (newer CLI format: top-level type)
if (chunk.type === "rate_limit_event") {
// Rate limit events are informational. Log them but don't yield anything.
// If the rate limit blocks the response, the stream will end without
// assistant messages and the task loop will handle the empty response.
Logger.log("Claude Code rate limit event:", JSON.stringify(chunk))
continue
}
// Skip user messages (tool results from Claude Code's own tool execution)
if (chunk.type === "user") {
continue
}
if (chunk.type === "assistant" && "message" in chunk) {
const message = chunk.message
// Check for error field on the message (newer CLI format)
if (message.error) {
const firstContent = message.content?.[0]
const errorText = firstContent && "text" in firstContent ? firstContent.text : undefined
throw new Error(errorText ?? `Claude Code error: ${message.error}`)
}
if (message.stop_reason !== null) {
const firstContent = message.content?.[0]
const content = firstContent && "text" in firstContent ? firstContent : undefined
// Check if content exists before accessing its properties
if (content && content.text.startsWith(`API Error`)) {
// Error messages are formatted as: `API Error: <<status code>> <<json>>`
const errorMessageStart = content.text.indexOf("{")
const errorMessage = content.text.slice(errorMessageStart)
const error = this.attemptParse(errorMessage)
if (!error) {
throw new Error(content.text)
}
if (error.error.message.includes("Invalid model name")) {
throw new Error(
content.text +
`\n\nAPI keys and subscription plans allow different models. Make sure the selected model is included in your plan.`,
)
}
throw new Error(errorMessage)
}
}
for (const content of message.content) {
switch (content.type) {
case "text":
yield {
type: "text",
text: content.text,
}
break
case "thinking":
yield {
type: "reasoning",
reasoning: content.thinking || "",
}
break
case "redacted_thinking":
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
}
break
case "tool_use":
// Yield tool_use blocks to the streaming pipeline for proper tool execution
yield {
type: "tool_calls",
tool_call: {
call_id: content.id,
function: {
id: content.id,
name: content.name,
arguments: JSON.stringify(content.input),
},
},
}
break
default: {
// Handle unknown content block types gracefully.
// Newer Anthropic models or CLI versions may introduce new content types
// (e.g., server_tool_use, mcp_tool_use). Log them instead of silently dropping.
const unknownBlock = content as { type: string; text?: string }
Logger.warn(`Unhandled content type in Claude Code response: ${unknownBlock.type}`)
// If the unknown block has a text-like field, try to yield it as text
if (typeof unknownBlock.text === "string") {
yield {
type: "text",
text: unknownBlock.text,
}
}
break
}
}
}
// According to Anthropic's API documentation:
// https://docs.anthropic.com/en/api/messages#usage-object
// The `input_tokens` field already includes both `cache_read_input_tokens` and `cache_creation_input_tokens`.
// Therefore, we should not add cache tokens to the input_tokens count again, as this would result in double-counting.
usage.inputTokens = message.usage?.input_tokens ?? 0
usage.outputTokens = message.usage?.output_tokens ?? 0
usage.cacheReadTokens = message.usage?.cache_read_input_tokens ?? 0
usage.cacheWriteTokens = message.usage?.cache_creation_input_tokens ?? 0
continue
}
if (chunk.type === "result" && "result" in chunk) {
if (chunk.is_error) {
throw new Error(`Claude Code returned an error: ${chunk.result}`)
}
usage.totalCost = isPaidUsage ? chunk.total_cost_usd : 0
yield usage
continue
}
// ErrorMessage — log it explicitly and skip
if ((chunk as any).type === "error") {
Logger.warn("Claude Code emitted an error-type chunk:", JSON.stringify(chunk))
continue
}
// Any completely unrecognized chunk type — log and skip
Logger.warn(`Unrecognized Claude Code chunk type: ${(chunk as any).type}`)
}
}
private attemptParse(str: string) {
try {
return JSON.parse(str)
} catch (_err) {
return null
}
}
getModel() {
const modelId = this.options.apiModelId
if (modelId && modelId in claudeCodeModels) {
const id = modelId as ClaudeCodeModelId
return { id, info: claudeCodeModels[id] }
}
return {
id: claudeCodeDefaultModelId,
info: claudeCodeModels[claudeCodeDefaultModelId],
}
}
}
-343
View File
@@ -1,343 +0,0 @@
import { type ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import axios from "axios"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineEnv } from "@/config"
import { refreshClineRecommendedModels } from "@/core/controller/models/refreshClineRecommendedModels"
import { ClineAccountService } from "@/services/account/ClineAccountService"
import { AuthService } from "@/services/auth/AuthService"
import { buildClineExtraHeaders } from "@/services/EnvUtils"
import { CLINE_ACCOUNT_AUTH_ERROR_MESSAGE } from "@/shared/ClineAccount"
import { CLINE_RECOMMENDED_MODELS_FALLBACK } from "@/shared/cline/recommended-models"
import type { ClineStorageMessage } from "@/shared/messages/content"
import { fetch, getAxiosSettings } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import type { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { createOpenRouterStream } from "../transform/openrouter-stream"
import type { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
import type { OpenRouterErrorResponse } from "./types"
interface ClineHandlerOptions extends CommonApiHandlerOptions {
ulid?: string
taskId?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
openRouterProviderSorting?: string
openRouterModelId?: string
openRouterModelInfo?: ModelInfo
clineAccountId?: string
clineApiKey?: string
enableParallelToolCalling?: boolean
}
function normalizeModelId(modelId: string): string {
return modelId.trim().toLowerCase()
}
const CLINE_FREE_MODEL_IDS = new Set(CLINE_RECOMMENDED_MODELS_FALLBACK.free.map((model) => normalizeModelId(model.id)))
function getCacheReadTokens(usage: any): number {
return usage?.prompt_tokens_details?.cached_tokens || usage?.cache_read_input_tokens || 0
}
function getCacheWriteTokens(usage: any): number {
return usage?.prompt_tokens_details?.cache_write_tokens || usage?.cache_creation_input_tokens || 0
}
export class ClineHandler implements ApiHandler {
private options: ClineHandlerOptions
private clineAccountService = ClineAccountService.getInstance()
private _authService: AuthService
private client: OpenAI | undefined
lastGenerationId?: string
private lastRequestId?: string
private get _baseUrl(): string {
return ClineEnv.config().apiBaseUrl
}
constructor(options: ClineHandlerOptions) {
this.options = options
this._authService = AuthService.getInstance()
}
private async getFreeModelIdSet(): Promise<Set<string>> {
try {
const models = await refreshClineRecommendedModels()
const freeModelIds = models.free.map((model) => normalizeModelId(model.id)).filter((modelId) => modelId.length > 0)
if (freeModelIds.length > 0) {
return new Set(freeModelIds)
}
} catch (error) {
Logger.error("Error resolving Cline free model IDs from recommended models:", error)
}
return CLINE_FREE_MODEL_IDS
}
private async ensureClient(): Promise<OpenAI> {
const clineAccountAuthToken = this.options.clineApiKey || (await this._authService.getAuthToken())
if (!clineAccountAuthToken) {
throw new Error(CLINE_ACCOUNT_AUTH_ERROR_MESSAGE)
}
if (!this.client) {
try {
const defaultHeaders: Record<string, string> = {
"HTTP-Referer": "https://cline.bot",
"X-Title": "Cline",
"X-Task-ID": this.options.ulid || "",
}
Object.assign(defaultHeaders, await buildClineExtraHeaders())
this.client = new OpenAI({
baseURL: `${this._baseUrl}/api/v1`,
apiKey: clineAccountAuthToken,
defaultHeaders,
// Capture real HTTP request ID from initial streaming response headers
fetch: async (...args: Parameters<typeof fetch>): Promise<Awaited<ReturnType<typeof fetch>>> => {
const [input, init] = args
const resp = await fetch(input, init)
try {
let urlStr = ""
if (typeof input === "string") {
urlStr = input
} else if (input instanceof URL) {
urlStr = input.toString()
} else if (typeof (input as { url?: unknown }).url === "string") {
urlStr = (input as { url: string }).url
}
// Only record for chat completions (the primary streaming request)
if (urlStr.includes("/chat/completions")) {
const rid = resp.headers.get("x-request-id") || resp.headers.get("request-id")
if (rid) {
this.lastRequestId = rid
}
}
} catch {
// ignore header capture errors
}
return resp
},
})
} catch (error: any) {
throw new Error(`Error creating Cline client: ${error.message}`)
}
}
// Ensure the client is always using the latest auth token
this.client.apiKey = clineAccountAuthToken
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
try {
const client = await this.ensureClient()
this.lastGenerationId = undefined
this.lastRequestId = undefined
let didOutputUsage = false
const freeModelIds = await this.getFreeModelIdSet()
const stream = await createOpenRouterStream(
client,
systemPrompt,
messages,
this.getModel(),
this.options.reasoningEffort,
this.options.thinkingBudgetTokens,
this.options.openRouterProviderSorting,
tools,
this.options.enableParallelToolCalling,
)
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
Logger.debug("ClineHandler chunk:" + JSON.stringify(chunk))
// openrouter returns an error object instead of the openai sdk throwing an error
if ("error" in chunk) {
const error = chunk.error as OpenRouterErrorResponse["error"]
Logger.error(`Cline API Error: ${error?.code} - ${error?.message}`)
// Include metadata in the error message if available
const metadataStr = error.metadata ? `\nMetadata: ${JSON.stringify(error.metadata, null, 2)}` : ""
throw new Error(`Cline API Error ${error.code}: ${error.message}${metadataStr}`)
}
if (!this.lastGenerationId && chunk.id) {
this.lastGenerationId = chunk.id
}
// Check for mid-stream error via finish_reason
const choice = chunk.choices?.[0]
// OpenRouter may return finish_reason = "error" with error details
if ((choice?.finish_reason as string) === "error") {
const choiceWithError = choice as any
if (choiceWithError.error) {
const error = choiceWithError.error
Logger.error(`Cline Mid-Stream Error: ${error.code || error.type || "Unknown"} - ${error.message}`)
throw new Error(`Cline Mid-Stream Error: ${error.code || error.type || "Unknown"} - ${error.message}`)
}
throw new Error("Cline Mid-Stream Error: Stream terminated with error status but no error details provided")
}
const delta = choice?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Reasoning tokens are returned separately from the content
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
if (
delta &&
"reasoning" in delta &&
delta.reasoning &&
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
/*
OpenRouter passes reasoning details that we can pass back unmodified in api requests to preserve reasoning traces for model
- The reasoning_details array in each chunk may contain one or more reasoning objects
- For encrypted reasoning, the content may appear as [REDACTED] in streaming responses
- The complete reasoning sequence is built by concatenating all chunks in order
See: https://openrouter.ai/docs/use-cases/reasoning-tokens#preserving-reasoning-blocks
*/
if (
delta &&
"reasoning_details" in delta &&
delta.reasoning_details &&
// @ts-expect-error-next-line
delta?.reasoning_details?.length && // exists and non-0
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: "",
details: delta.reasoning_details,
}
}
if (!didOutputUsage && chunk.usage) {
// @ts-expect-error-next-line
let totalCost = (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0)
const modelId = this.getModel().id
const isFreeModel = freeModelIds.has(normalizeModelId(modelId))
const cacheReadTokens = getCacheReadTokens(chunk.usage)
const cacheWriteTokens = getCacheWriteTokens(chunk.usage)
if (isFreeModel) {
totalCost = 0
}
yield {
type: "usage",
cacheWriteTokens,
cacheReadTokens,
inputTokens: Math.max(0, (chunk.usage.prompt_tokens || 0) - cacheReadTokens - cacheWriteTokens),
outputTokens: chunk.usage.completion_tokens || 0,
totalCost,
}
didOutputUsage = true
}
}
// Fallback to generation endpoint if usage chunk not returned
if (!didOutputUsage) {
Logger.warn("Cline API did not return usage chunk, fetching from generation endpoint")
const apiStreamUsage = await this.getApiStreamUsage(freeModelIds)
if (apiStreamUsage) {
yield apiStreamUsage
}
}
} catch (error) {
Logger.error("Cline API Error:", error)
throw error
}
}
async getApiStreamUsage(freeModelIds?: Set<string>): Promise<ApiStreamUsageChunk | undefined> {
if (this.lastGenerationId) {
try {
const resolvedFreeModelIds = freeModelIds || (await this.getFreeModelIdSet())
const clineAccountAuthToken = await this._authService.getAuthToken()
if (!clineAccountAuthToken) {
throw new Error(CLINE_ACCOUNT_AUTH_ERROR_MESSAGE)
}
const headers: Record<string, string> = {
// Align with backend auth expectations
Authorization: `Bearer ${clineAccountAuthToken}`,
}
Object.assign(headers, await buildClineExtraHeaders())
const response = await axios.get(`${this.clineAccountService.baseUrl}/generation?id=${this.lastGenerationId}`, {
headers,
timeout: 15_000, // this request hangs sometimes
...getAxiosSettings(),
})
const generation = response.data
let totalCost = generation?.total_cost || 0
const modelId = this.getModel().id
const isFreeModel = resolvedFreeModelIds.has(normalizeModelId(modelId))
if (isFreeModel) {
totalCost = 0
}
return {
type: "usage",
cacheWriteTokens: generation?.native_tokens_cache_write || 0,
cacheReadTokens: generation?.native_tokens_cached || 0,
// openrouter generation endpoint fails often
inputTokens: Math.max(
0,
(generation?.native_tokens_prompt || 0) -
(generation?.native_tokens_cached || 0) -
(generation?.native_tokens_cache_write || 0),
),
outputTokens: generation?.native_tokens_completion || 0,
totalCost,
}
} catch (error) {
// ignore if fails
Logger.error("Error fetching cline generation details:", error)
}
}
return undefined
}
// Expose the last HTTP request ID captured from response headers (X-Request-ID)
getLastRequestId(): string | undefined {
return this.lastRequestId
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.openRouterModelId
const modelInfo = this.options.openRouterModelInfo
if (modelId && modelInfo) {
return { id: modelId, info: modelInfo }
}
// If we have a model ID but no model info (e.g., CLI featured models),
// use the ID with default model info rather than falling back to a different model
if (modelId) {
return { id: modelId, info: openRouterDefaultModelInfo }
}
return { id: openRouterDefaultModelId, info: openRouterDefaultModelInfo }
}
}
@@ -1,142 +0,0 @@
import { DeepSeekModelId, deepSeekDefaultModelId, deepSeekModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { addReasoningContent } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface DeepSeekHandlerOptions extends CommonApiHandlerOptions {
deepSeekApiKey?: string
apiModelId?: string
}
export class DeepSeekHandler implements ApiHandler {
private options: DeepSeekHandlerOptions
private client: OpenAI | undefined
constructor(options: DeepSeekHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.deepSeekApiKey) {
throw new Error("DeepSeek API key is required")
}
try {
this.client = new OpenAI({
baseURL: "https://api.deepseek.com/v1",
apiKey: this.options.deepSeekApiKey,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating DeepSeek client: ${error.message}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
// Deepseek reports total input AND cache reads/writes,
// see context caching: https://api-docs.deepseek.com/guides/kv_cache)
// where the input tokens is the sum of the cache hits/misses, just like OpenAI.
// This affects:
// 1) context management truncation algorithm, and
// 2) cost calculation
// Deepseek usage includes extra fields.
// Safely cast the prompt token details section to the appropriate structure.
interface DeepSeekUsage extends OpenAI.CompletionUsage {
prompt_cache_hit_tokens?: number
prompt_cache_miss_tokens?: number
}
const deepUsage = usage as DeepSeekUsage
const inputTokens = deepUsage?.prompt_tokens || 0 // sum of cache hits and misses
const outputTokens = deepUsage?.completion_tokens || 0
const cacheReadTokens = deepUsage?.prompt_cache_hit_tokens || 0
const cacheWriteTokens = deepUsage?.prompt_cache_miss_tokens || 0
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens) // this will always be 0
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
totalCost: totalCost,
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const isDeepSeekThinkingModel =
model.id.includes("deepseek-reasoner") || model.id === "deepseek-v4-flash" || model.id === "deepseek-v4-pro"
const convertedMessages = convertToOpenAiMessages(messages)
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = isDeepSeekThinkingModel
? [{ role: "system", content: systemPrompt }, ...addReasoningContent(convertedMessages, messages)]
: [{ role: "system", content: systemPrompt }, ...convertedMessages]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
// Only set temperature for non-thinking models
...(isDeepSeekThinkingModel ? {} : { temperature: 0 }),
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
}
}
}
getModel(): { id: DeepSeekModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in deepSeekModels) {
const id = modelId as DeepSeekModelId
return { id, info: deepSeekModels[id] }
}
return {
id: deepSeekDefaultModelId,
info: deepSeekModels[deepSeekDefaultModelId],
}
}
}
-657
View File
@@ -1,657 +0,0 @@
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ModelInfo } from "../../../shared/api"
import { ApiHandler } from "../index"
import { ApiStream } from "../transform/stream"
interface DifyHandlerOptions {
difyApiKey?: string
difyBaseUrl?: string
}
// Dify API Response Types
export interface DifyFileResponse {
id: string
name: string
size: number
extension: string
mime_type: string
created_by: string
created_at: number
}
export interface DifyMessage {
id: string
conversation_id: string
inputs: Record<string, any>
query: string
message_files: Array<{
id: string
type: string
url: string
belongs_to: string
}>
answer: string
created_at: number
feedback?: {
rating: string
}
retriever_resources?: any[]
}
interface DifyHistoryResponse {
data: DifyMessage[]
has_more: boolean
limit: number
}
interface DifyConversation {
id: string
name: string
inputs: Record<string, any>
status: string
introduction: string
created_at: number
updated_at: number
}
interface DifyConversationsResponse {
data: DifyConversation[]
has_more: boolean
limit: number
}
interface DifyConversationResponse {
id: string
name: string
inputs: Record<string, any>
status: string
introduction: string
created_at: number
updated_at: number
}
export class DifyHandler implements ApiHandler {
private options: DifyHandlerOptions
private baseUrl: string
private apiKey: string
private conversationId: string | null = null
private currentTaskId: string | null = null
private abortController: AbortController | null = null
constructor(options: DifyHandlerOptions) {
this.options = options
this.apiKey = options.difyApiKey || ""
this.baseUrl = options.difyBaseUrl || ""
Logger.log("[DIFY DEBUG] Constructor called with:", {
hasApiKey: !!this.apiKey,
baseUrl: this.baseUrl,
})
if (!this.apiKey) {
throw new Error("Dify API key is required")
}
if (!this.baseUrl) {
throw new Error("Dify base URL is required")
}
}
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
Logger.log("[DIFY DEBUG] createMessage called with:", {
systemPromptLength: systemPrompt?.length || 0,
messagesCount: messages?.length || 0,
})
// Convert messages to Dify format
const query = this.convertMessagesToQuery(systemPrompt, messages)
const requestBody = {
inputs: {},
query: query,
response_mode: "streaming",
conversation_id: this.conversationId || "",
user: "cline-user", // A unique user identifier
files: [],
}
const fullUrl = `${this.baseUrl}/chat-messages`
Logger.log("[DIFY DEBUG] Making request to:", fullUrl)
Logger.log("[DIFY DEBUG] Request body:", JSON.stringify(requestBody, null, 2))
let response: Response
try {
response = await fetch(fullUrl, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify(requestBody),
})
} catch (error: any) {
Logger.error("[DIFY DEBUG] Network error during fetch:", error)
const cause = error.cause ? ` | Cause: ${error.cause}` : ""
throw new Error(`Dify API network error: ${error.message}${cause}`)
}
Logger.log("[DIFY DEBUG] Response status:", response.status)
const headersObj: Record<string, string> = {}
response.headers.forEach((value, key) => {
headersObj[key] = value
})
Logger.log("[DIFY DEBUG] Response headers:", headersObj)
if (!response.ok) {
const errorText = await response.text()
Logger.error("[DIFY DEBUG] Error response:", errorText)
throw new Error(`Dify API error: ${response.status} ${response.statusText} - ${errorText}`)
}
if (!response.body) {
throw new Error("No response body from Dify API")
}
const reader = response.body.getReader()
const decoder = new TextDecoder()
let buffer = ""
let fullText = ""
let hasYieldedContent = false
const processedEvents: string[] = []
let lastEventTime = Date.now()
Logger.log("[DIFY DEBUG] Starting to read streaming response...")
try {
while (true) {
const { done, value } = await reader.read()
if (done) {
Logger.log("[DIFY DEBUG] Stream ended naturally")
Logger.log(
"[DIFY DEBUG] Final state - hasYieldedContent:",
hasYieldedContent,
"fullText length:",
fullText.length,
"processedEvents:",
processedEvents,
)
break
}
const chunk = decoder.decode(value, { stream: true })
Logger.log("[DIFY DEBUG] Raw chunk received:", JSON.stringify(chunk))
buffer += chunk
const lines = buffer.split("\n")
// Keep the last incomplete line in the buffer
buffer = lines.pop() || ""
for (const line of lines) {
Logger.log("[DIFY DEBUG] Processing line:", JSON.stringify(line))
if (line.startsWith("data: ")) {
const data = line.slice(6).trim()
Logger.log("[DIFY DEBUG] Extracted data:", JSON.stringify(data))
if (data === "[DONE]") {
Logger.log("[DIFY DEBUG] Received [DONE] signal")
break
}
if (data === "") {
Logger.log("[DIFY DEBUG] Empty data line, skipping")
continue
}
try {
const parsed = JSON.parse(data)
Logger.log("[DIFY DEBUG] Parsed JSON:", parsed)
processedEvents.push(parsed.event || "unknown")
lastEventTime = Date.now()
// Capture conversation_id as soon as it's available
if (parsed.conversation_id && !this.conversationId) {
this.conversationId = parsed.conversation_id
Logger.log("[DIFY DEBUG] Captured conversation_id:", this.conversationId)
}
// Handle different Dify event types based on actual Dify API
if (parsed.event === "message") {
Logger.log("[DIFY DEBUG] Message event, answer:", parsed.answer)
// Dify sends the full text in each "answer" chunk, so we replace.
if (typeof parsed.answer === "string") {
fullText = parsed.answer
Logger.log("[DIFY DEBUG] Updated fullText length:", fullText.length)
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
} else if (parsed.event === "message_replace") {
Logger.log("[DIFY DEBUG] Replace message event:", parsed)
if (parsed.answer) {
fullText = parsed.answer // Replace instead of append
Logger.log("[DIFY DEBUG] Replaced fullText length:", fullText.length)
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
} else if (parsed.event === "message_end") {
Logger.log("[DIFY DEBUG] Message end event", parsed)
// Message completed. Yield final text if we have any.
if (fullText) {
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
// Yield usage data if available
if (parsed.usage) {
yield {
type: "usage",
inputTokens: parsed.usage.prompt_tokens || 0,
outputTokens: parsed.usage.completion_tokens || parsed.usage.total_tokens || 0,
totalCost: parsed.usage.total_price || 0,
}
}
return // End of stream
} else if (parsed.event === "error") {
Logger.error("[DIFY DEBUG] Error event:", parsed)
throw new Error(`Dify API error: ${parsed.message || "Unknown error"}`)
} else if (parsed.event === "workflow_started" || parsed.event === "workflow_finished") {
Logger.log("[DIFY DEBUG] Workflow event:", parsed.event)
// These are informational events, continue processing
} else if (parsed.event === "node_started" || parsed.event === "node_finished") {
Logger.log("[DIFY DEBUG] Node event:", parsed.event, parsed.data)
// These are informational events, continue processing
} else if (parsed.event === "ping") {
Logger.log("[DIFY DEBUG] Ping event received, keeping connection alive.")
// Ping event, do nothing
} else {
Logger.log("[DIFY DEBUG] Unknown event type:", parsed.event, "Full object:", parsed)
// Try to extract text from other possible fields
if (parsed.text) {
fullText += parsed.text
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
} else if (parsed.content) {
fullText += parsed.content
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
} else if (parsed.answer) {
// Fallback: some events might have answer field even if not "message" type
fullText += parsed.answer
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
}
} catch (e) {
Logger.warn("[DIFY DEBUG] Failed to parse JSON:", data, "Error:", e)
}
} else if (line.trim() !== "") {
Logger.log(
"[DIFY DEBUG] Non-data line (not starting with 'data:'), trying to parse as direct JSON:",
JSON.stringify(line),
)
// Try to parse as direct JSON (fallback for non-SSE responses, though Dify uses SSE)
try {
const parsed = JSON.parse(line.trim())
Logger.log("[DIFY DEBUG] Parsed direct JSON:", parsed)
processedEvents.push(parsed.event || "direct-json")
// Handle the same event types as above
if (parsed.event === "message" && parsed.answer) {
fullText += parsed.answer
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
} else if (parsed.event === "message_end") {
if (fullText) {
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
return
} else if (parsed.event === "error") {
Logger.error("[DIFY DEBUG] Direct JSON Error event:", parsed)
throw new Error(`Dify API error: ${parsed.message || "Unknown error"}`)
} else if (parsed.answer || parsed.text || parsed.content) {
// Fallback for any content in direct JSON
const content = parsed.answer || parsed.text || parsed.content
fullText += content
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
} catch (e) {
// Not JSON, continue
Logger.log("[DIFY DEBUG] Line is not direct JSON, continuing")
}
}
}
}
// Final check - if we haven't yielded any content, provide diagnostic information
if (!hasYieldedContent) {
const diagnosticInfo = {
processedEvents,
finalFullTextLength: fullText.length,
finalFullText: fullText,
streamDuration: Date.now() - lastEventTime,
conversationId: this.conversationId,
}
Logger.error("[DIFY DEBUG] No content was yielded! Diagnostic info:", diagnosticInfo)
// If we have any accumulated text at all, yield it as a fallback
if (fullText.trim()) {
Logger.log("[DIFY DEBUG] Yielding accumulated text as fallback:", fullText)
yield {
type: "text",
text: fullText,
}
} else {
// Provide a more informative error
throw new Error(
`Dify API did not provide any assistant messages. ` +
`Events processed: [${processedEvents.join(", ")}]. ` +
`Check your Dify application configuration and ensure it's properly set up to return responses. ` +
`API URL: ${fullUrl}. Conversation ID: ${this.conversationId || "none"}.`,
)
}
}
} finally {
reader.releaseLock()
Logger.log("[DIFY DEBUG] Stream reader released")
}
}
private convertMessagesToQuery(systemPrompt: string, messages: ClineStorageMessage[]): string {
// Dify's context is managed by `conversation_id`. The `query` should be the last user message.
// The system prompt is typically configured in the Dify App itself.
const lastUserMessage = messages.filter((m) => m.role === "user").pop()
if (!lastUserMessage) {
return "" // Should not happen in normal flow
}
const userQuery = Array.isArray(lastUserMessage.content)
? lastUserMessage.content.map((c) => ("text" in c ? c.text : "")).join("\n")
: (lastUserMessage.content as string)
// Only prepend the system prompt if it's the very first message of a new conversation.
if (!this.conversationId && systemPrompt) {
Logger.log("[DIFY DEBUG] Prepending system prompt for new conversation.")
return `${systemPrompt}\n\n---\n\n${userQuery}`
}
return userQuery
}
getModel(): { id: string; info: ModelInfo } {
return {
id: "dify-workflow",
info: {
maxTokens: 8192,
contextWindow: 128000,
supportsImages: true,
supportsPromptCache: false,
inputPrice: 0,
outputPrice: 0,
description: "Dify workflow - model selection is configured in your Dify application",
},
}
}
// Additional Dify API Methods
/**
* Upload a file for use in conversations
* @param file File buffer to upload
* @param filename Name of the file
* @param user User identifier (defaults to "cline-user")
* @returns Promise with file upload response
*/
async uploadFile(file: Buffer, filename: string, user: string = "cline-user"): Promise<DifyFileResponse> {
const formData = new FormData()
formData.append("file", new Blob([new Uint8Array(file)]), filename)
formData.append("user", user)
const response = await fetch(`${this.baseUrl}/files/upload`, {
method: "POST",
headers: this.headers(),
body: formData,
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify file upload error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Stop generation for a specific task
* @param taskId Task ID from streaming response
* @param user User identifier (defaults to "cline-user")
* @returns Promise that resolves when generation is stopped
*/
async stopGeneration(taskId: string, user: string = "cline-user"): Promise<void> {
const response = await fetch(`${this.baseUrl}/chat-messages/${taskId}/stop`, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify({ user }),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify stop generation error: ${response.status} ${response.statusText} - ${errorText}`)
}
}
/**
* Get conversation history messages with pagination
* @param conversationId Conversation ID
* @param user User identifier (defaults to "cline-user")
* @param firstId First message ID for pagination (optional)
* @param limit Number of messages to return (default: 20)
* @returns Promise with conversation history
*/
async getConversationHistory(
conversationId: string,
user: string = "cline-user",
firstId?: string,
limit: number = 20,
): Promise<DifyHistoryResponse> {
const params = new URLSearchParams({ user, limit: limit.toString() })
if (firstId) {
params.append("first_id", firstId)
}
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}/messages?${params}`, {
headers: this.headers(),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify get conversation history error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Get list of conversations for a user
* @param user User identifier (defaults to "cline-user")
* @param lastId Last conversation ID for pagination (optional)
* @param limit Number of conversations to return (default: 20)
* @param sortBy Sort field (default: "-updated_at")
* @returns Promise with conversations list
*/
async getConversations(
user: string = "cline-user",
lastId?: string,
limit: number = 20,
sortBy: string = "-updated_at",
): Promise<DifyConversationsResponse> {
const params = new URLSearchParams({
user,
limit: limit.toString(),
sort_by: sortBy,
})
if (lastId) {
params.append("last_id", lastId)
}
const response = await fetch(`${this.baseUrl}/conversations?${params}`, {
headers: this.headers(),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify get conversations error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Delete a conversation
* @param conversationId Conversation ID to delete
* @param user User identifier (defaults to "cline-user")
* @returns Promise that resolves when conversation is deleted
*/
async deleteConversation(conversationId: string, user: string = "cline-user"): Promise<void> {
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}`, {
method: "DELETE",
headers: this.jsonHeaders(),
body: JSON.stringify({ user }),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify delete conversation error: ${response.status} ${response.statusText} - ${errorText}`)
}
}
/**
* Rename a conversation
* @param conversationId Conversation ID to rename
* @param user User identifier (defaults to "cline-user")
* @param name New conversation name (optional if auto_generate is true)
* @param autoGenerate Whether to auto-generate the name (default: false)
* @returns Promise with updated conversation details
*/
async renameConversation(
conversationId: string,
user: string = "cline-user",
name?: string,
autoGenerate: boolean = false,
): Promise<DifyConversationResponse> {
const body: any = { user, auto_generate: autoGenerate }
if (name) {
body.name = name
}
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}/name`, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify(body),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify rename conversation error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Submit feedback for a message
* @param messageId Message ID to provide feedback for
* @param rating Rating: "like" or "dislike"
* @param content Optional feedback content
* @param user User identifier (defaults to "cline-user")
* @returns Promise that resolves when feedback is submitted
*/
async submitMessageFeedback(
messageId: string,
rating: "like" | "dislike",
content?: string,
user: string = "cline-user",
): Promise<void> {
const body: any = { rating, user }
if (content) {
body.content = content
}
const response = await fetch(`${this.baseUrl}/messages/${messageId}/feedbacks`, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify(body),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify submit feedback error: ${response.status} ${response.statusText} - ${errorText}`)
}
}
/**
* Get current conversation ID
* @returns Current conversation ID or null
*/
getCurrentConversationId(): string | null {
return this.conversationId
}
/**
* Set conversation ID for continuing existing conversations
* @param conversationId Conversation ID to set
*/
setConversationId(conversationId: string): void {
this.conversationId = conversationId
}
/**
* Reset conversation ID to start a new conversation
*/
resetConversation(): void {
this.conversationId = null
this.currentTaskId = null
}
private jsonHeaders() {
return {
...this.headers(),
"Content-Type": "application/json",
}
}
private headers() {
const externalHeaders = buildExternalBasicHeaders()
return {
...externalHeaders,
Authorization: `Bearer ${this.apiKey}`,
}
}
}
@@ -1,90 +0,0 @@
import { DoubaoModelId, doubaoDefaultModelId, doubaoModels, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface DoubaoHandlerOptions extends CommonApiHandlerOptions {
doubaoApiKey?: string
apiModelId?: string
}
export class DoubaoHandler implements ApiHandler {
private options: DoubaoHandlerOptions
private client: OpenAI | undefined
constructor(options: DoubaoHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.doubaoApiKey) {
throw new Error("Doubao API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://ark.cn-beijing.volces.com/api/v3/",
apiKey: this.options.doubaoApiKey,
})
} catch (error) {
throw new Error(`Error creating Doubao client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: DoubaoModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in doubaoModels) {
const id = modelId as DoubaoModelId
return { id, info: doubaoModels[id] }
}
return {
id: doubaoDefaultModelId,
info: doubaoModels[doubaoDefaultModelId],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
// @ts-expect-error-next-line
cacheReadTokens: chunk.usage.prompt_cache_hit_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
}
@@ -1,116 +0,0 @@
import { FireworksModelId, fireworksDefaultModelId, fireworksModels, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface FireworksHandlerOptions extends CommonApiHandlerOptions {
fireworksApiKey?: string
fireworksModelId?: string
fireworksModelMaxCompletionTokens?: number
fireworksModelMaxTokens?: number
}
export class FireworksHandler implements ApiHandler {
private options: FireworksHandlerOptions
private client: OpenAI | undefined
constructor(options: FireworksHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.fireworksApiKey) {
throw new Error("Fireworks API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.fireworks.ai/inference/v1",
apiKey: this.options.fireworksApiKey,
})
} catch (error) {
throw new Error(`Error creating Fireworks client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.fireworksModelId ?? ""
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
})
let reasoning: string | null = null
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (reasoning || delta?.content?.includes("<think>")) {
reasoning = (reasoning || "") + (delta.content ?? "")
}
if (delta?.content && !reasoning) {
yield {
type: "text",
text: delta.content,
}
}
if (reasoning || (delta && "reasoning_content" in delta && delta.reasoning_content)) {
yield {
type: "reasoning",
reasoning: delta.content || ((delta as any).reasoning_content as string | undefined) || "",
}
if (reasoning?.includes("</think>")) {
// Reset so the next chunk is regular content
reasoning = null
}
}
if (chunk.usage) {
const usage = chunk.usage as OpenAI.CompletionUsage & {
prompt_cache_hit_tokens?: number
prompt_cache_miss_tokens?: number
prompt_tokens_details?: {
cached_tokens?: number
}
}
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
// Fireworks can return cache hits either as prompt_cache_hit_tokens or prompt_tokens_details.cached_tokens.
cacheReadTokens: usage.prompt_cache_hit_tokens ?? usage.prompt_tokens_details?.cached_tokens ?? 0,
cacheWriteTokens: usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: FireworksModelId; info: ModelInfo } {
const modelId = this.options.fireworksModelId
if (modelId && modelId in fireworksModels) {
const id = modelId as FireworksModelId
return { id, info: fireworksModels[id] }
}
return {
id: fireworksDefaultModelId,
info: fireworksModels[fireworksDefaultModelId],
}
}
}
@@ -1,570 +0,0 @@
// Restore GenerateContentConfig import and add GenerateContentResponseUsageMetadata
import {
ApiError,
FunctionCallingConfigMode,
type GenerateContentConfig,
type GenerateContentResponseUsageMetadata,
GoogleGenAI,
FunctionDeclaration as GoogleTool,
ThinkingLevel,
} from "@google/genai"
import { GeminiModelId, geminiDefaultModelId, geminiModels, ModelInfo } from "@shared/api"
import { GEMINI_FLASH_MAX_OUTPUT_TOKENS, isGeminiFlashModel } from "@utils/model-utils"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { telemetryService } from "@/services/telemetry"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { RetriableError, withRetry } from "../retry"
import { convertAnthropicMessageToGemini } from "../transform/gemini-format"
import { ApiStream } from "../transform/stream"
const rateLimitPatterns = [/got status: 429/i, /429 Too Many Requests/i, /rate limit exceeded/i, /too many requests/i]
interface GeminiHandlerOptions extends CommonApiHandlerOptions {
isVertex?: boolean
vertexProjectId?: string
vertexRegion?: string
geminiApiKey?: string
geminiBaseUrl?: string
thinkingBudgetTokens?: number
reasoningEffort?: string
apiModelId?: string
ulid?: string
}
function mapReasoningEffortToGeminiThinkingLevel(effort: string): ThinkingLevel {
switch (effort) {
case "low":
case "medium":
return ThinkingLevel.LOW
case "high":
case "xhigh":
return ThinkingLevel.HIGH
default:
return ThinkingLevel.LOW
}
}
function getGeminiMaxOutputTokens(modelId: string, modelMaxTokens?: number): number | undefined {
if (!isGeminiFlashModel(modelId)) {
return undefined
}
if (modelMaxTokens && modelMaxTokens > 0) {
return Math.min(modelMaxTokens, GEMINI_FLASH_MAX_OUTPUT_TOKENS)
}
return GEMINI_FLASH_MAX_OUTPUT_TOKENS
}
/**
* Handler for Google's Gemini API with optimized caching strategy and accurate cost accounting.
*
* Key features:
* - One cache per task: Creates a single cache per task and reuses it for subsequent turns
* - Stable cache keys: Uses ulid as a stable identifier for caches
* - Efficient cache updates: Only updates caches when there's new content to add
* - Split cost accounting: Separates immediate costs from ongoing cache storage costs
*
* Cost accounting approach:
* - Immediate costs (per message): Input tokens, output tokens, and cache read costs
* - Ongoing costs (per task): Cache storage costs for the TTL period
*
* Gemini's caching system is unique in that it charges for holding tokens in cache by the hour.
* This implementation optimizes for both performance and cost by:
* 1. Minimizing redundant cache creations
* 2. Properly accounting for cache costs in the billing calculations
* 3. Using a stable cache key to ensure cache reuse across turns
* 4. Separating immediate costs from ongoing costs to avoid double-counting
*/
export class GeminiHandler implements ApiHandler {
private options: GeminiHandlerOptions
private client: GoogleGenAI | undefined
constructor(options: GeminiHandlerOptions) {
// Store the options
this.options = options
}
private ensureClient(): GoogleGenAI {
if (!this.client) {
const options = this.options as GeminiHandlerOptions
const externalHeaders = buildExternalBasicHeaders()
if (options.isVertex) {
// Initialize with Vertex AI configuration
const project = this.options.vertexProjectId ?? "not-provided"
const location = this.options.vertexRegion ?? "not-provided"
try {
this.client = new GoogleGenAI({
vertexai: true,
project,
location,
httpOptions: {
headers: externalHeaders,
},
})
} catch (error) {
throw new Error(`Error creating Gemini Vertex AI client: ${error.message}`)
}
} else {
// Initialize with standard API key
if (!options.geminiApiKey) {
throw new Error("API key is required for Google Gemini when not using Vertex AI")
}
try {
this.client = new GoogleGenAI({
apiKey: options.geminiApiKey,
httpOptions: {
headers: externalHeaders,
},
})
} catch (error) {
throw new Error(`Error creating Gemini client: ${error.message}`)
}
}
}
return this.client
}
/**
* Creates a message using the Gemini API with implicit caching.
*
* Cost accounting:
* - Immediate costs (returned in the usage object): Input tokens, output tokens, cache read costs
*
* @param systemPrompt The system prompt to use for the message
* @param messages The conversation history to include in the message
* @returns An async generator that yields chunks of the response with accurate immediate costs
*/
@withRetry({
maxRetries: 4,
baseDelay: 2000,
maxDelay: 15000,
})
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: GoogleTool[]): ApiStream {
const client = this.ensureClient()
const { id: modelId, info } = this.getModel()
const contents = messages.map(convertAnthropicMessageToGemini)
// Gemini may emit multiple function calls under the same responseId and without functionCall.id.
// Track a local sequence so each emitted tool call has a stable unique ID.
const responseToolCallCount = new Map<string, number>()
// Configure thinking budget/level if supported
const _thinkingBudget = this.options.thinkingBudgetTokens ?? 0
const maxBudget = info.thinkingConfig?.maxBudget ?? 24576
const thinkingBudget = Math.min(_thinkingBudget, maxBudget)
// When ThinkingLevel is defined, thinking budget cannot be zero
// and only level is used to control thinking behavior.
// Only set thinkingLevel for models that support it
let thinkingLevel: ThinkingLevel | undefined
const rawReasoningEffort = (this.options.reasoningEffort || "").toLowerCase()
const normalizedReasoningEffort = !rawReasoningEffort || rawReasoningEffort === "none" ? "low" : rawReasoningEffort
if (info.thinkingConfig?.supportsThinkingLevel) {
thinkingLevel = mapReasoningEffortToGeminiThinkingLevel(normalizedReasoningEffort)
}
// Set up base generation config
const maxOutputTokens = getGeminiMaxOutputTokens(modelId, info.maxTokens)
const requestConfig: GenerateContentConfig = {
// Add base URL if configured
httpOptions: this.options.geminiBaseUrl ? { baseUrl: this.options.geminiBaseUrl } : undefined,
systemInstruction: systemPrompt,
// Set temperature (default to 0)
// Gemini 3 recommends 1.0
temperature: info.temperature ?? 1,
...(maxOutputTokens !== undefined ? { maxOutputTokens } : {}),
}
// Add thinking config only if the model supports it
if (info.thinkingConfig) {
requestConfig.thinkingConfig = {
// Turn off thinking:
// thinkingBudget: 0
// Turn on dynamic thinking:
// thinkingBudget: -1
// Turn on fixed thinking budget:
thinkingBudget: thinkingLevel ? undefined : thinkingBudget, // Use budget only if thinkingLevel is not set
thinkingLevel,
includeThoughts: thinkingBudget > 0 || !!thinkingLevel,
}
}
// Generate content using the configured parameters
const sdkCallStartTime = Date.now()
let responseId: string | undefined
let sdkFirstChunkTime: number | undefined
let ttftSdkMs: number | undefined
let apiSuccess = false
let apiError: string | undefined
let promptTokens = 0
let outputTokens = 0
let cacheReadTokens = 0
let thoughtsTokenCount = 0 // Initialize thought token counts
let lastUsageMetadata: GenerateContentResponseUsageMetadata | undefined
const isNativeToolCallsEnabled = tools?.length
if (isNativeToolCallsEnabled) {
requestConfig.tools = [{ functionDeclarations: tools }]
requestConfig.toolConfig = {
// Force the model to call 'any' function.
functionCallingConfig: {
mode: FunctionCallingConfigMode.ANY,
},
}
}
try {
const result = await client.models.generateContentStream({
model: modelId,
contents: contents,
config: {
...requestConfig,
},
})
let isFirstSdkChunk = true
for await (const chunk of result) {
const responseKey = chunk.responseId || "gemini-response"
if (isFirstSdkChunk) {
sdkFirstChunkTime = Date.now()
ttftSdkMs = sdkFirstChunkTime - sdkCallStartTime
isFirstSdkChunk = false
}
// Handle thinking content from Gemini's response
const parts = chunk?.candidates?.[0]?.content?.parts || []
for (const part of parts) {
if (part.thought && part.text) {
yield {
type: "reasoning",
id: chunk.responseId,
reasoning: part.text || "",
signature: part.thoughtSignature,
}
} else if (part.text) {
yield {
type: "text",
text: part.text,
id: chunk.responseId,
signature: part.thoughtSignature,
}
}
if (part.functionCall) {
const functionCall = part.functionCall
const args = Object.entries(functionCall.args || {}).filter(([_key, val]) => !!val)
if (functionCall.args && args.length > 0) {
const existingId = functionCall.id?.trim()
const toolCallId =
existingId ??
(() => {
const sequenceNumber = responseToolCallCount.get(responseKey) ?? 0
responseToolCallCount.set(responseKey, sequenceNumber + 1)
return `${responseKey}-tool-${sequenceNumber}`
})()
yield {
type: "tool_calls",
id: chunk.responseId,
tool_call: {
call_id: toolCallId,
function: {
id: toolCallId,
name: functionCall.name,
arguments: JSON.stringify(functionCall.args),
},
},
signature: part.thoughtSignature,
}
}
}
}
if (chunk.usageMetadata) {
responseId = chunk.responseId
lastUsageMetadata = chunk.usageMetadata
promptTokens = lastUsageMetadata.promptTokenCount ?? promptTokens
outputTokens = lastUsageMetadata.candidatesTokenCount ?? outputTokens
thoughtsTokenCount = lastUsageMetadata.thoughtsTokenCount ?? thoughtsTokenCount
cacheReadTokens = lastUsageMetadata.cachedContentTokenCount ?? cacheReadTokens
}
}
apiSuccess = true
if (lastUsageMetadata) {
const totalCost = this.calculateCost({
info,
inputTokens: promptTokens,
outputTokens,
thoughtsTokenCount,
cacheReadTokens,
})
yield {
type: "usage",
inputTokens: promptTokens - cacheReadTokens,
outputTokens,
thoughtsTokenCount,
cacheReadTokens,
cacheWriteTokens: 0,
totalCost,
id: responseId,
}
}
} catch (error) {
apiSuccess = false
// Let the error propagate to be handled by withRetry or Task.ts
// Telemetry will be sent in the finally block.
if (error instanceof Error) {
apiError = error.message
if (error instanceof ApiError) {
if (error.status === 429) {
// The API includes more details in the message
// https://github.com/googleapis/js-genai/blob/v1.11.0/src/_api_client.ts#L758
const response = this.attemptParse(error.message)
if (response && response.error) {
const responseBody = this.attemptParse(response.error.message)
if (responseBody.error) {
const detail = responseBody.error.details?.find(
(d: any) => d["@type"] === "type.googleapis.com/google.rpc.RetryInfo",
)
const detailedError = new RetriableError(
apiError,
this.parseRetryDelay(detail?.retryDelay) || undefined,
{
cause: error,
},
)
throw detailedError
}
}
throw new RetriableError(apiError, undefined, { cause: error })
}
// Fallback in case Gemini throws a rate limit error without a 429 status code
// https://github.com/cline/cline/pull/5205#discussion_r2311761559
const isRateLimit = rateLimitPatterns.some((pattern) => pattern.test(error.message))
if (isRateLimit) {
throw new RetriableError(apiError, undefined, { cause: error })
}
}
} else {
apiError = String(error)
}
throw error
} finally {
const sdkCallEndTime = Date.now()
const totalDurationSdkMs = sdkCallEndTime - sdkCallStartTime
const cacheHit = cacheReadTokens > 0
const cacheHitPercentage = promptTokens > 0 ? (cacheReadTokens / promptTokens) * 100 : undefined
const throughputTokensPerSecSdk =
totalDurationSdkMs > 0 && outputTokens > 0 ? outputTokens / (totalDurationSdkMs / 1000) : undefined
if (this.options.ulid) {
telemetryService.captureGeminiApiPerformance(this.options.ulid, modelId, {
ttftSec: ttftSdkMs !== undefined ? ttftSdkMs / 1000 : undefined,
totalDurationSec: totalDurationSdkMs / 1000,
promptTokens,
outputTokens,
cacheReadTokens,
cacheHit,
cacheHitPercentage,
apiSuccess,
apiError,
throughputTokensPerSec: throughputTokensPerSecSdk,
})
} else {
Logger.warn("GeminiHandler: ulid not available for telemetry in createMessage.")
}
}
}
/**
* Calculate the immediate dollar cost of the API call based on token usage and model pricing.
*
* This method accounts for the immediate costs of the API call:
* - Input token costs (for uncached tokens)
* - Output token costs
* - Cache read costs
* - Gemini implicit caching has no write costs
*
*/
public calculateCost({
info,
inputTokens,
outputTokens,
thoughtsTokenCount = 0,
cacheReadTokens = 0,
}: {
info: ModelInfo
inputTokens: number
outputTokens: number
thoughtsTokenCount: number
cacheReadTokens?: number
}) {
// Exit early if any required pricing information is missing
if (!info.inputPrice || !info.outputPrice) {
return undefined
}
let inputPrice = info.inputPrice
let outputPrice = info.outputPrice
// Right now, we only show the immediate costs of caching and not the ongoing costs of storing the cache
let cacheReadsPrice = info.cacheReadsPrice ?? 0
// If there's tiered pricing then adjust prices based on the input tokens used
if (info.tiers) {
const tier = info.tiers.find((tier) => inputTokens <= tier.contextWindow)
if (tier) {
inputPrice = tier.inputPrice ?? inputPrice
outputPrice = tier.outputPrice ?? outputPrice
cacheReadsPrice = tier.cacheReadsPrice ?? cacheReadsPrice
}
}
// Subtract the cached input tokens from the total input tokens
const uncachedInputTokens = inputTokens - (cacheReadTokens ?? 0)
// Calculate immediate costs only
// 1. Input token costs (for uncached tokens)
const inputTokensCost = inputPrice * (uncachedInputTokens / 1_000_000)
// 2. Output token costs
const responseTokensCost = outputPrice * ((outputTokens + thoughtsTokenCount) / 1_000_000)
// 3. Cache read costs (immediate)
const cacheReadCost = (cacheReadTokens ?? 0) > 0 ? cacheReadsPrice * ((cacheReadTokens ?? 0) / 1_000_000) : 0
// Calculate total immediate cost (excluding cache write/storage costs)
const totalCost = inputTokensCost + responseTokensCost + cacheReadCost
// Create the trace object for debugging
const trace: Record<string, { price: number; tokens: number; cost: number }> = {
input: { price: inputPrice, tokens: uncachedInputTokens, cost: inputTokensCost },
output: { price: outputPrice, tokens: outputTokens, cost: responseTokensCost },
}
// Only include cache read costs in the trace (cache write costs are tracked separately)
if ((cacheReadTokens ?? 0) > 0) {
trace.cacheRead = { price: cacheReadsPrice, tokens: cacheReadTokens ?? 0, cost: cacheReadCost }
}
// Logger.log(`[GeminiHandler] calculateCost -> ${totalCost}`, trace)
return totalCost
}
/**
* Get the model ID and info for the current configuration
*/
getModel(): { id: GeminiModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in geminiModels) {
const id = modelId as GeminiModelId
return { id, info: geminiModels[id] }
}
return {
id: geminiDefaultModelId,
info: geminiModels[geminiDefaultModelId],
}
}
/**
* Count tokens in content using the Gemini API
*/
async countTokens(content: Array<any>): Promise<number> {
try {
const client = this.ensureClient()
const { id: model } = this.getModel()
// Convert content to Gemini format
const geminiContent = content.map((block) => {
if (typeof block === "string") {
return { text: block }
}
return { text: JSON.stringify(block) }
})
// Use Gemini's token counting API
const response = await client.models.countTokens({
model,
contents: [{ parts: geminiContent }],
})
if (response.totalTokens === undefined) {
Logger.warn("Gemini token counting returned undefined, using fallback")
return this.estimateTokens(content)
}
return response.totalTokens
} catch (error) {
Logger.warn("Gemini token counting failed, using fallback", error)
return this.estimateTokens(content)
}
}
/**
* Fallback token estimation method
*/
private estimateTokens(content: Array<any>): number {
// Simple estimation: ~4 characters per token
const totalChars = content.reduce((total, block) => {
if (typeof block === "string") {
return total + block.length
}
if (block && typeof block === "object") {
// Safely stringify the object
try {
const jsonStr = JSON.stringify(block)
return total + jsonStr.length
} catch (e) {
Logger.warn("Failed to stringify block for token estimation", e)
return total
}
}
return total
}, 0)
return Math.ceil(totalChars / 4)
}
private parseRetryDelay(retryAfter?: string): number {
if (!retryAfter) {
return 0
}
const unit = retryAfter.at(-1)
const value = Number.parseInt(retryAfter, 10)
if (Number.isNaN(value)) {
return 0
}
if (unit === "s") {
return value
}
if (unit === "m") {
return value * 60 // Convert minutes to seconds
}
if (unit === "h") {
return value * 60 * 60 // Convert hours to seconds
}
return value
}
private attemptParse(str: string) {
try {
return JSON.parse(str)
} catch (_) {
return null
}
}
}
-320
View File
@@ -1,320 +0,0 @@
import { GroqModelId, groqDefaultModelId, groqModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface GroqHandlerOptions extends CommonApiHandlerOptions {
groqApiKey?: string
groqModelId?: string
groqModelInfo?: ModelInfo
apiModelId?: string // For backward compatibility
}
// Enhanced usage interface to support Groq's cached token fields
interface GroqUsage extends OpenAI.CompletionUsage {
prompt_tokens_details?: {
cached_tokens?: number
}
}
// Model family definitions for enhanced behavior
interface GroqModelFamily {
name: string
supportedFeatures: {
streaming: boolean
temperature: boolean
vision: boolean
tools: boolean
}
maxTokensOverride?: number
specialParams?: Record<string, any>
}
const MODEL_FAMILIES: Record<string, GroqModelFamily> = {
// Moonshort 4 Family - Latest generation with vision support
"kimi-k2": {
name: "kimi-k2",
supportedFeatures: { streaming: true, temperature: true, vision: true, tools: true },
maxTokensOverride: 8192,
},
// Llama 4 Family - Latest generation with vision support
llama4: {
name: "Llama 4",
supportedFeatures: { streaming: true, temperature: true, vision: true, tools: true },
maxTokensOverride: 8192,
},
// Llama 3.3 Family - Balanced performance
"llama3.3": {
name: "Llama 3.3",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 32768,
},
// Llama 3.1 Family - Fast inference
"llama3.1": {
name: "Llama 3.1",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 131072,
},
// DeepSeek Family - Reasoning-optimized
deepseek: {
name: "DeepSeek",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 8192,
specialParams: {
top_p: 0.95,
reasoning_format: "parsed",
},
},
// Qwen Family - Enhanced for Q&A
qwen: {
name: "Qwen",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 32768,
},
// Compound Models - Hybrid architectures
compound: {
name: "Compound",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 8192,
},
}
export class GroqHandler implements ApiHandler {
private options: GroqHandlerOptions
private client: OpenAI | undefined
constructor(options: GroqHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.groqApiKey) {
throw new Error("Groq API key is required")
}
try {
this.client = new OpenAI({
baseURL: "https://api.groq.com/openai/v1",
apiKey: this.options.groqApiKey,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating Groq client: ${error.message}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: GroqUsage | undefined): ApiStream {
const inputTokens = usage?.prompt_tokens || 0
const outputTokens = usage?.completion_tokens || 0
const cacheReadTokens = usage?.prompt_tokens_details?.cached_tokens || 0
// Groq does not track cache writes
const cacheWriteTokens = 0
// Calculate cost using OpenAI-compatible cost calculation
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
// Calculate non-cached input tokens for proper reporting
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens,
cacheWriteTokens,
cacheReadTokens,
totalCost,
}
}
/**
* Detects the model family based on the model ID
*/
private detectModelFamily(modelId: string): GroqModelFamily {
if (modelId.includes("kimi-k2")) {
return MODEL_FAMILIES["kimi-k2"]
}
// Llama 4 variants
if (modelId.includes("llama-4") || modelId.includes("llama/llama-4")) {
return MODEL_FAMILIES.llama4
}
// Llama 3.3 variants
if (modelId.includes("llama-3.3")) {
return MODEL_FAMILIES["llama3.3"]
}
// Llama 3.1 variants
if (modelId.includes("llama-3.1")) {
return MODEL_FAMILIES["llama3.1"]
}
// DeepSeek variants
if (modelId.includes("deepseek")) {
return MODEL_FAMILIES.deepseek
}
// Qwen variants
if (modelId.includes("qwen")) {
return MODEL_FAMILIES.qwen
}
// Compound variants
if (modelId.includes("compound")) {
return MODEL_FAMILIES.compound
}
// Default fallback to Llama 3.3 behavior
return MODEL_FAMILIES["kimi-k2"]
}
/**
* Gets the optimal max_tokens based on model family and capabilities
*/
private getOptimalMaxTokens(model: { id: string; info: ModelInfo }, modelFamily: GroqModelFamily): number {
// Use model-specific max tokens if available
if (model.info.maxTokens && model.info.maxTokens > 0) {
return model.info.maxTokens
}
// Use family override if available
if (modelFamily.maxTokensOverride) {
return modelFamily.maxTokensOverride
}
// Default fallback
return 8192
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const modelFamily = this.detectModelFamily(model.id)
// Optimize parameters based on model family
const temperature = 0
const maxTokens = this.getOptimalMaxTokens(model, modelFamily)
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
// Build request parameters with model-specific optimizations
const requestParams: OpenAI.Chat.ChatCompletionCreateParamsStreaming & {
reasoning_format?: "parsed" | "raw" | "hidden"
top_p?: number
} = {
model: model.id,
max_tokens: maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature,
...getOpenAIToolParams(tools),
}
// Add any special parameters for specific model families
if (modelFamily.specialParams) {
Object.assign(requestParams, modelFamily.specialParams)
}
const toolCallProcessor = new ToolCallProcessor()
const stream = await client.chat.completions.create(requestParams)
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle reasoning field if present (for reasoning models with parsed output)
if ((delta as any)?.reasoning) {
const reasoningContent = (delta as any).reasoning as string
yield {
type: "reasoning",
reasoning: reasoningContent,
}
continue
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle content field - trust the parsed output from Groq
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
// Handle usage information
if (chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
}
}
}
/**
* Checks if the current model supports vision/images
*/
supportsImages(): boolean {
const model = this.getModel()
return model.info.supportsImages === true
}
/**
* Checks if the current model supports tools
*/
supportsTools(): boolean {
const model = this.getModel()
const modelFamily = this.detectModelFamily(model.id)
return modelFamily.supportedFeatures.tools
}
/**
* Gets model information with enhanced family detection
*/
getModel(): { id: string; info: ModelInfo } {
// First priority: groqModelId and groqModelInfo (like Requesty does)
const groqModelId = this.options.groqModelId
const groqModelInfo = this.options.groqModelInfo
if (groqModelId && groqModelInfo) {
return { id: groqModelId, info: groqModelInfo }
}
// Second priority: groqModelId with static model info
if (groqModelId && groqModelId in groqModels) {
const id = groqModelId as GroqModelId
return { id, info: groqModels[id] }
}
// Third priority: apiModelId (for backward compatibility)
const apiModelId = this.options.apiModelId
if (apiModelId && apiModelId in groqModels) {
const id = apiModelId as GroqModelId
return { id, info: groqModels[id] }
}
// Default fallback
return {
id: groqDefaultModelId,
info: groqModels[groqDefaultModelId],
}
}
/**
* Gets model family information for debugging/introspection
*/
getModelFamily(): GroqModelFamily {
const model = this.getModel()
return this.detectModelFamily(model.id)
}
}
-104
View File
@@ -1,104 +0,0 @@
import { hicapModelInfoSaneDefaults, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionReasoningEffort } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface OpenAiHandlerOptions extends CommonApiHandlerOptions {
hicapApiKey?: string
hicapModelId?: string
}
export class HicapHandler implements ApiHandler {
private options: OpenAiHandlerOptions
private client: OpenAI | undefined
constructor(options: OpenAiHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.hicapApiKey) {
throw new Error("Hicap API key is required")
}
if (!this.options.hicapModelId) {
throw new Error("Model ID is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.hicap.ai/v2/openai",
apiKey: this.options.hicapApiKey,
defaultHeaders: {
"api-key": this.options.hicapApiKey,
},
})
} catch (error: any) {
throw new Error(`Error creating OpenAI client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.hicapModelId ?? ""
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const temperature: number = 1
let reasoningEffort: ChatCompletionReasoningEffort | undefined
let maxTokens: number | undefined
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
temperature,
max_tokens: maxTokens,
reasoning_effort: reasoningEffort,
stream: true,
stream_options: { include_usage: true },
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.hicapModelId ?? "",
info: hicapModelInfoSaneDefaults,
}
}
}
@@ -1,142 +0,0 @@
import { HuaweiCloudMaasModelId, huaweiCloudMaasDefaultModelId, huaweiCloudMaasModels, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface HuaweiCloudMaaSHandlerOptions extends CommonApiHandlerOptions {
huaweiCloudMaasApiKey?: string
huaweiCloudMaasModelId?: string
huaweiCloudMaasModelInfo?: ModelInfo
}
export class HuaweiCloudMaaSHandler implements ApiHandler {
private options: HuaweiCloudMaaSHandlerOptions
private client: OpenAI | undefined
constructor(options: HuaweiCloudMaaSHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.huaweiCloudMaasApiKey) {
throw new Error("Huawei Cloud MaaS API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.modelarts-maas.com/v1/",
apiKey: this.options.huaweiCloudMaasApiKey,
})
} catch (error) {
throw new Error(`Error creating Huawei Cloud MaaS client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: HuaweiCloudMaasModelId; info: ModelInfo } {
// First priority: huaweiCloudMaasModelId and huaweiCloudMaasModelInfo (like Groq does)
const huaweiCloudMaasModelId = this.options.huaweiCloudMaasModelId
const huaweiCloudMaasModelInfo = this.options.huaweiCloudMaasModelInfo
if (huaweiCloudMaasModelId && huaweiCloudMaasModelInfo) {
return { id: huaweiCloudMaasModelId as HuaweiCloudMaasModelId, info: huaweiCloudMaasModelInfo }
}
// Second priority: huaweiCloudMaasModelId with static model info
if (huaweiCloudMaasModelId && huaweiCloudMaasModelId in huaweiCloudMaasModels) {
const id = huaweiCloudMaasModelId as HuaweiCloudMaasModelId
return { id, info: huaweiCloudMaasModels[id] }
}
// Default fallback
return {
id: huaweiCloudMaasDefaultModelId,
info: huaweiCloudMaasModels[huaweiCloudMaasDefaultModelId],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
...getOpenAIToolParams(tools),
})
let reasoning: string | null = null
let didOutputUsage: boolean = false
let finalUsage: any = null
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle reasoning content detection
if (delta?.content) {
if (reasoning || delta.content.includes("<think>")) {
reasoning = (reasoning || "") + delta.content
} else if (!reasoning) {
yield {
type: "text",
text: delta.content,
}
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle reasoning output
if (reasoning || (delta && "reasoning_content" in delta && delta.reasoning_content)) {
const reasoningContent = delta?.content || ((delta as any)?.reasoning_content as string | undefined) || ""
if (reasoningContent.trim()) {
yield {
type: "reasoning",
reasoning: reasoningContent,
}
}
// Check if reasoning is complete
if (reasoning?.includes("</think>")) {
reasoning = null
}
}
// Store usage information for later output
if (chunk.usage) {
finalUsage = chunk.usage
}
// Output usage when stream is finished
if (!didOutputUsage && chunk.choices?.[0]?.finish_reason) {
if (finalUsage) {
yield {
type: "usage",
inputTokens: finalUsage.prompt_tokens || 0,
outputTokens: finalUsage.completion_tokens || 0,
cacheWriteTokens: 0,
cacheReadTokens: 0,
}
}
didOutputUsage = true
}
}
}
}
@@ -1,148 +0,0 @@
import { HuggingFaceModelId, huggingFaceDefaultModelId, huggingFaceModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface HuggingFaceHandlerOptions extends CommonApiHandlerOptions {
huggingFaceApiKey?: string
huggingFaceModelId?: string
huggingFaceModelInfo?: ModelInfo
}
export class HuggingFaceHandler implements ApiHandler {
private options: HuggingFaceHandlerOptions
private client: OpenAI | undefined
private cachedModel: { id: HuggingFaceModelId; info: ModelInfo } | undefined
constructor(options: HuggingFaceHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.huggingFaceApiKey) {
throw new Error("Hugging Face API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://router.huggingface.co/v1",
apiKey: this.options.huggingFaceApiKey,
})
} catch (error: any) {
throw new Error(`Error creating Hugging Face client: ${error.message}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
if (!usage) {
return
}
const inputTokens = usage.prompt_tokens || 0
const outputTokens = usage.completion_tokens || 0
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens)
const usageData = {
type: "usage" as const,
inputTokens: inputTokens,
outputTokens: outputTokens,
cacheWriteTokens: 0,
cacheReadTokens: 0,
totalCost: totalCost,
}
yield usageData
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
try {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const requestParams = {
model: model.id,
max_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
...getOpenAIToolParams(tools),
}
const toolCallProcessor = new ToolCallProcessor()
const stream = (await client.chat.completions.create(requestParams)) as any
let _chunkCount = 0
let _totalContent = ""
for await (const chunk of stream) {
_chunkCount++
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
_totalContent += delta.content
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
}
}
} catch (error: any) {
throw error
}
}
getModel(): { id: HuggingFaceModelId; info: ModelInfo } {
// Return cached model if available
if (this.cachedModel) {
return this.cachedModel
}
const modelId = this.options.huggingFaceModelId
// List all available models for debugging
const _availableModels = Object.keys(huggingFaceModels)
let result: { id: HuggingFaceModelId; info: ModelInfo }
if (modelId && modelId in huggingFaceModels) {
const id = modelId as HuggingFaceModelId
const modelInfo = huggingFaceModels[id]
result = { id, info: modelInfo }
} else {
const defaultInfo = huggingFaceModels[huggingFaceDefaultModelId]
result = {
id: huggingFaceDefaultModelId,
info: defaultInfo,
}
}
// Cache the result for future calls
this.cachedModel = result
return result
}
}
@@ -1,403 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { LiteLLMModelInfo, liteLlmDefaultModelId, liteLlmModelInfoSaneDefaults } from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import OpenAI from "openai"
import { StateManager } from "@/core/storage/StateManager"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient, fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { isAnthropicModelId } from "@/utils/model-utils"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface LiteLlmHandlerOptions extends CommonApiHandlerOptions {
liteLlmApiKey?: string
liteLlmBaseUrl?: string
liteLlmModelId?: string
liteLlmModelInfo?: LiteLLMModelInfo
reasoningEffort?: string
thinkingBudgetTokens?: number
liteLlmUsePromptCache?: boolean
ulid?: string
}
/**
* Extended chat completion parameters that include LiteLLM-specific options
* not present in the standard OpenAI SDK types
*/
interface LiteLlmChatCompletionCreateParams extends OpenAI.Chat.ChatCompletionCreateParamsStreaming {
drop_params?: boolean
}
export interface LiteLlmModelInfoResponse {
data: Array<{
model_name: string
litellm_params: {
model: string
[key: string]: any
}
model_info: {
input_cost_per_token: number
output_cost_per_token: number
cache_creation_input_token_cost?: number
cache_read_input_token_cost?: number
supports_prompt_caching?: boolean
[key: string]: any
}
}>
}
/**
* Exported utility function to fetch LiteLLM model info
* @param baseUrl The base URL for the LiteLLM API
* @param apiKey The API key for authentication
* @returns The model info response or undefined if fetch fails
*/
export async function fetchLiteLlmModelsInfo(baseUrl: string, apiKey: string): Promise<LiteLlmModelInfoResponse | undefined> {
// Handle base URLs that already include /v1 to avoid double /v1/v1/
const normalizedBaseUrl = baseUrl.endsWith("/v1") ? baseUrl : `${baseUrl}/v1`
const url = `${normalizedBaseUrl}/model/info`
try {
const response = await fetch(url, {
method: "GET",
headers: {
accept: "application/json",
"x-litellm-api-key": apiKey,
...buildExternalBasicHeaders(),
},
})
if (response.ok) {
const data: LiteLlmModelInfoResponse = await response.json()
return data
}
Logger.error("Failed to fetch LiteLLM model info:", response.statusText)
// Try with Authorization header instead
const retryResponse = await fetch(url, {
method: "GET",
headers: {
accept: "application/json",
Authorization: `Bearer ${apiKey}`,
...buildExternalBasicHeaders(),
},
})
if (retryResponse.ok) {
const data: LiteLlmModelInfoResponse = await retryResponse.json()
return data
}
Logger.error("Failed to fetch LiteLLM model info with Authorization header:", retryResponse.statusText)
throw new Error(`Failed to fetch LiteLLM model info: ${retryResponse.statusText}`)
} catch (error) {
Logger.error("Error fetching LiteLLM model info:", error)
throw error
}
}
export class LiteLlmHandler implements ApiHandler {
private options: LiteLlmHandlerOptions
private client: OpenAI | undefined
private modelInfoCache: LiteLlmModelInfoResponse | undefined
private modelInfoCacheTimestamp = 0
private readonly modelInfoCacheTTL = 5 * 60 * 1000 // 5 minutes
constructor(options: LiteLlmHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.liteLlmApiKey) {
throw new Error("LiteLLM API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: this.options.liteLlmBaseUrl || "http://localhost:4000",
apiKey: this.options.liteLlmApiKey || "noop",
})
} catch (error) {
throw new Error(`Error creating LiteLLM client: ${error.message}`)
}
}
return this.client
}
private async modelInfo(publicModelName: string): Promise<LiteLlmModelInfoResponse["data"][number] | undefined> {
const modelInfo = await this.fetchModelsInfo()
if (!modelInfo?.data) {
return undefined
}
return modelInfo.data.find((model) => model.model_name === publicModelName)
}
private async fetchModelsInfo(): Promise<LiteLlmModelInfoResponse | undefined> {
// Check if cache is still valid
const now = Date.now()
if (this.modelInfoCache && now - this.modelInfoCacheTimestamp < this.modelInfoCacheTTL) {
return this.modelInfoCache
}
const client = this.ensureClient()
const data = await fetchLiteLlmModelsInfo(client.baseURL, this.options.liteLlmApiKey || "")
if (data) {
this.modelInfoCache = data
this.modelInfoCacheTimestamp = now
}
return data
}
private async getModelCostInfo(publicModelName: string): Promise<{
inputCostPerToken: number
outputCostPerToken: number
cacheCreationCostPerToken?: number
cacheReadCostPerToken?: number
}> {
try {
const matchingModel = await this.modelInfo(publicModelName)
if (matchingModel) {
return {
inputCostPerToken: matchingModel.model_info.input_cost_per_token || 0,
outputCostPerToken: matchingModel.model_info.output_cost_per_token || 0,
cacheCreationCostPerToken: matchingModel.model_info.cache_creation_input_token_cost,
cacheReadCostPerToken: matchingModel.model_info.cache_read_input_token_cost,
}
}
} catch (error) {
Logger.warn("Error getting LiteLLM model cost info:", error)
}
// Fallback to zero costs if we can't get the information
return {
inputCostPerToken: 0,
outputCostPerToken: 0,
}
}
async calculateCost(
prompt_tokens: number,
completion_tokens: number,
cache_creation_tokens?: number,
cache_read_tokens?: number,
): Promise<number | undefined> {
const publicModelId = this.options.liteLlmModelId || liteLlmDefaultModelId
try {
const costInfo = await this.getModelCostInfo(publicModelId)
// Calculate costs for different token types
const inputCost = Math.max(0, prompt_tokens - (cache_read_tokens || 0)) * costInfo.inputCostPerToken
const outputCost = completion_tokens * costInfo.outputCostPerToken
const cacheCreationCost = (cache_creation_tokens || 0) * (costInfo.cacheCreationCostPerToken || 0)
const cacheReadCost = (cache_read_tokens || 0) * (costInfo.cacheReadCostPerToken || 0)
const totalCost = inputCost + outputCost + cacheCreationCost + cacheReadCost
return totalCost
} catch (error) {
Logger.error("Error calculating spend:", error)
return undefined
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const formattedMessages = convertToOpenAiMessages(messages)
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam | Anthropic.Messages.TextBlockParam = {
role: "system",
content: systemPrompt,
}
const modelId = this.options.liteLlmModelId || liteLlmDefaultModelId
const isOminiModel = modelId.includes("o1-mini") || modelId.includes("o3-mini") || modelId.includes("o4-mini")
const isCodexModel = modelId.toLowerCase().includes("codex")
// Configuration for extended thinking
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = budgetTokens !== 0
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budgetTokens)
: undefined
const thinkingConfig = isAdaptiveThinkingModel
? adaptiveThinking?.enabled
? ({ type: "adaptive" } as any)
: undefined
: reasoningOn
? { type: "enabled", budget_tokens: budgetTokens }
: undefined
let temperature: number | undefined = this.options.liteLlmModelInfo?.temperature ?? 1
if (isAdaptiveThinkingModel) {
temperature = undefined
} else if ((isOminiModel || isAnthropicModelId(modelId)) && reasoningOn) {
temperature = undefined // OAI omni and Anthropic extended thinking mode doesn't support temperature
}
const modelInfo = await this.modelInfo(modelId)
// Automatically enable caching if the model supports it
const cacheControl =
(modelInfo?.model_info.supports_prompt_caching ?? false) ? { cache_control: { type: "ephemeral" } } : undefined
if (cacheControl) {
// Add cache_control to system message if enabled
// https://docs.litellm.ai/docs/providers/anthropic#caching---large-context-caching
systemMessage.content = [
{
text: systemPrompt,
type: "text",
...cacheControl,
},
] as Anthropic.Messages.TextBlockParam[]
}
// Find the last two user messages to apply caching
const userMsgIndices = formattedMessages.reduce(
(acc, msg, index) => (msg.role === "user" ? [...acc, index] : acc),
[] as number[],
)
const lastUserMsgIndex = userMsgIndices[userMsgIndices.length - 1] ?? -1
const secondLastUserMsgIndex = userMsgIndices[userMsgIndices.length - 2] ?? -1
// Apply cache_control to the last two user messages if enabled
// https://docs.litellm.ai/docs/providers/anthropic#caching---large-context-caching
const enhancedMessages: OpenAI.Chat.ChatCompletionMessageParam[] = formattedMessages.map(
(message, index): OpenAI.Chat.ChatCompletionMessageParam => {
if ((index === lastUserMsgIndex || index === secondLastUserMsgIndex) && cacheControl) {
// Handle both string and array content types
if (typeof message.content === "string") {
return {
...message,
content: [
{
type: "text",
text: message.content,
...cacheControl,
},
] as any,
}
}
if (Array.isArray(message.content)) {
// Apply cache control to the last content item in the array
return {
...message,
content: message.content.map((item, contentIndex) =>
contentIndex === (message.content?.length || 0) - 1
? {
...item,
...cacheControl,
}
: item,
) as any,
}
}
return {
...message,
...cacheControl,
}
}
return message
},
)
const stream = await client.chat.completions.create({
model: this.options.liteLlmModelId || liteLlmDefaultModelId,
messages: [systemMessage, ...enhancedMessages],
temperature,
stream: true,
drop_params: true,
...(!isCodexModel && { stream_options: { include_usage: true } }), // Codex models are only on the responses api, which doesn't take the stream_options parameter. we will need to migrate to the responses api for this to work
...(thinkingConfig && { thinking: thinkingConfig }), // Add thinking configuration when applicable
...(isAdaptiveThinkingModel && adaptiveThinking?.effort
? { output_config: { effort: adaptiveThinking.effort } }
: {}),
...(this.options.ulid && { litellm_session_id: `cline-${this.options.ulid}` }), // Add session ID for LiteLLM tracking
} as LiteLlmChatCompletionCreateParams)
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle normal text content
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
// Handle reasoning events
// This is not in the standard types but may be in the response
interface ThinkingDelta {
reasoning_content?: string
}
if ((delta as ThinkingDelta)?.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta as ThinkingDelta).reasoning_content || "",
}
}
// Handle token usage information
if (chunk.usage) {
// Extract cache-related information if available
// Need to use type assertion since these properties are not in the standard OpenAI types
const usage = chunk.usage as {
prompt_tokens: number
completion_tokens: number
cache_creation_input_tokens?: number
prompt_cache_miss_tokens?: number
cache_read_input_tokens?: number
prompt_cache_hit_tokens?: number
}
const cacheWriteTokens = usage.cache_creation_input_tokens || usage.prompt_cache_miss_tokens || 0
const cacheReadTokens = usage.cache_read_input_tokens || usage.prompt_cache_hit_tokens || 0
// Calculate cost using the actual token usage including cache tokens
const totalCost =
(await this.calculateCost(
usage.prompt_tokens || 0,
usage.completion_tokens || 0,
cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
cacheReadTokens > 0 ? cacheReadTokens : undefined,
)) || 0
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
cacheWriteTokens: cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
cacheReadTokens: cacheReadTokens > 0 ? cacheReadTokens : undefined,
totalCost,
}
}
}
}
getModel() {
const modelId = this.options.liteLlmModelId || liteLlmDefaultModelId
// Try to get model info from StateManager cache first
const cachedModelInfo = StateManager.get().getModelInfo("liteLlm", modelId)
// Fall back to provided model info or defaults if not in cache
const modelInfo = cachedModelInfo || this.options.liteLlmModelInfo || liteLlmModelInfoSaneDefaults
return {
id: modelId,
info: modelInfo,
}
}
}
@@ -1,109 +0,0 @@
import { type ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import type { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import type { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface LmStudioHandlerOptions extends CommonApiHandlerOptions {
lmStudioBaseUrl?: string
lmStudioModelId?: string
lmStudioMaxTokens?: string
}
export class LmStudioHandler implements ApiHandler {
private options: LmStudioHandlerOptions
private client: OpenAI | undefined
constructor(options: LmStudioHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
try {
this.client = createOpenAIClient({
// Docs on the new v0 api endpoint: https://lmstudio.ai/docs/app/api/endpoints/rest
baseURL: new URL("api/v0", this.options.lmStudioBaseUrl || "http://localhost:1234").toString(),
apiKey: "noop",
})
} catch (error) {
throw new Error(`Error creating LM Studio client: ${error.message}`)
}
}
return this.client
}
@withRetry({ retryAllErrors: true })
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
try {
const stream = await client.chat.completions.create({
model: this.getModel().id,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
max_completion_tokens: this.options.lmStudioMaxTokens ? Number(this.options.lmStudioMaxTokens) : undefined,
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const choice = chunk.choices?.[0]
const delta = choice?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
}
}
}
} catch {
// LM Studio doesn't return an error code/body for now
throw new Error(
"Please check the LM Studio developer logs to debug what went wrong. You may need to load the model with a larger context length to work with Cline's prompts. Alternatively, try enabling Compact Prompt in your settings when working with a limited context window.",
)
}
}
getModel(): { id: string; info: ModelInfo } {
const info = { ...openAiModelInfoSaneDefaults }
const maxTokens = Number(this.options.lmStudioMaxTokens)
if (!Number.isNaN(maxTokens)) {
info.contextWindow = maxTokens
}
return {
id: this.options.lmStudioModelId || "",
info,
}
}
}
@@ -1,208 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
import { Stream as AnthropicStream } from "@anthropic-ai/sdk/streaming"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { MinimaxModelId, ModelInfo, minimaxDefaultModelId, minimaxModels } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ClineTool } from "@/shared/tools"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
interface MinimaxHandlerOptions extends CommonApiHandlerOptions {
minimaxApiKey?: string
minimaxApiLine?: string
apiModelId?: string
thinkingBudgetTokens?: number
}
export class MinimaxHandler implements ApiHandler {
private options: MinimaxHandlerOptions
private client: Anthropic | undefined
constructor(options: MinimaxHandlerOptions) {
this.options = options
}
private ensureClient(): Anthropic {
if (!this.client) {
if (!this.options.minimaxApiKey) {
throw new Error("MiniMax API key is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
this.client = new Anthropic({
apiKey: this.options.minimaxApiKey,
baseURL:
this.options.minimaxApiLine === "china"
? "https://api.minimaxi.com/anthropic"
: "https://api.minimax.io/anthropic",
defaultHeaders: externalHeaders,
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating MiniMax client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ClineTool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
// Tools are available only when native tools are enabled
const nativeToolsOn = tools?.length && tools?.length > 0
const budget_tokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
// MiniMax M2 uses Anthropic API format
const stream: AnthropicStream<Anthropic.RawMessageStreamEvent> = await client.messages.create({
model: model.id,
max_tokens: model.info.maxTokens || 8192,
system: [{ text: systemPrompt, type: "text" }],
messages,
stream: true,
tools: nativeToolsOn ? (tools as AnthropicTool[]) : undefined,
thinking: reasoningOn ? { type: "enabled", budget_tokens: budget_tokens } : undefined,
// "Thinking isn't compatible with temperature, top_p, or top_k modifications"
temperature: reasoningOn ? undefined : 1.0, // MiniMax recommends 1.0, range is (0.0, 1.0]
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
tool_choice: nativeToolsOn && !reasoningOn ? { type: "any" } : undefined,
})
const lastStartedToolCall = { id: "", name: "", arguments: "" }
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start": {
// tells us cache reads/writes/input/output
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
break
}
case "message_delta":
// tells us stop_reason, stop_sequence, and output tokens along the way and at the end of the message
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage.output_tokens || 0,
}
break
case "message_stop":
// no usage data, just an indicator that the message is done
break
case "content_block_start":
switch (chunk.content_block.type) {
case "thinking":
yield {
type: "reasoning",
reasoning: chunk.content_block.thinking || "",
signature: chunk.content_block.signature,
}
break
case "redacted_thinking":
// Content is encrypted, and we don't want to pass placeholder text back to the API
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
redacted_data: chunk.content_block.data,
}
break
case "tool_use":
if (chunk.content_block.id && chunk.content_block.name) {
// Store tool call information for streaming
lastStartedToolCall.id = chunk.content_block.id
lastStartedToolCall.name = chunk.content_block.name
lastStartedToolCall.arguments = ""
}
break
case "text":
// we may receive multiple text blocks, in which case just insert a line break between them
if (chunk.index > 0) {
yield {
type: "text",
text: "\n",
}
}
yield {
type: "text",
text: chunk.content_block.text,
}
break
}
break
case "content_block_delta":
switch (chunk.delta.type) {
case "thinking_delta":
// 'reasoning' type just displays in the UI, but reasoning with signature will be used to send the thinking traces back to the API
yield {
type: "reasoning",
reasoning: chunk.delta.thinking,
}
break
case "signature_delta":
// It's used when sending the thinking block back to the API
// API expects this in completed form, not as array of deltas
if (chunk.delta.signature) {
yield {
type: "reasoning",
reasoning: "",
signature: chunk.delta.signature,
}
}
break
case "text_delta":
yield {
type: "text",
text: chunk.delta.text,
}
break
case "input_json_delta":
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
// Convert Anthropic tool_use to OpenAI-compatible format for internal processing
yield {
type: "tool_calls",
tool_call: {
...lastStartedToolCall,
function: {
...lastStartedToolCall,
id: lastStartedToolCall.id,
name: lastStartedToolCall.name,
arguments: chunk.delta.partial_json,
},
},
}
}
break
}
break
case "content_block_stop":
lastStartedToolCall.id = ""
lastStartedToolCall.name = ""
lastStartedToolCall.arguments = ""
break
}
}
}
getModel(): { id: MinimaxModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in minimaxModels) {
const id = modelId as MinimaxModelId
return { id, info: minimaxModels[id] }
}
return { id: minimaxDefaultModelId, info: minimaxModels[minimaxDefaultModelId] }
}
}
@@ -1,154 +0,0 @@
import { Mistral } from "@mistralai/mistralai"
import { HTTPClient } from "@mistralai/mistralai/lib/http"
import { Tool as MistralTool } from "@mistralai/mistralai/models/components/tool"
import { MistralModelId, ModelInfo, mistralDefaultModelId, mistralModels } from "@shared/api"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToMistralMessages } from "../transform/mistral-format"
import { ApiStream } from "../transform/stream"
interface MistralHandlerOptions extends CommonApiHandlerOptions {
mistralApiKey?: string
apiModelId?: string
}
export class MistralHandler implements ApiHandler {
private options: MistralHandlerOptions
private client: Mistral | undefined
constructor(options: MistralHandlerOptions) {
this.options = options
}
private ensureClient(): Mistral {
if (!this.client) {
if (!this.options.mistralApiKey) {
throw new Error("Mistral API key is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
// Create HTTP client with custom fetch for proxy support
// The Mistral SDK's HTTPClient passes a Request object to the fetcher,
// but we need to extract the URL and init options to pass to our fetch wrapper
// which properly handles proxy configuration in standalone mode (JetBrains/CLI)
const httpClient = new HTTPClient({
fetcher: async (input: RequestInfo | URL, init?: RequestInit) => {
// Handle both string/URL and Request object inputs
if (input instanceof Request) {
Object.keys(externalHeaders).forEach((key) => {
if (!input.headers.has(key)) {
input.headers.set(key, externalHeaders[key])
}
})
return fetch(input.url, {
method: input.method,
headers: input.headers,
body: input.body,
redirect: input.redirect,
signal: input.signal,
// duplex is required when sending a body stream in Node.js/undici
duplex: input.body ? "half" : undefined,
...init,
} as RequestInit)
}
// Merge external headers with existing headers
const mergedInit = {
...init,
headers: {
...externalHeaders,
...(init?.headers || {}),
},
}
return fetch(input, mergedInit)
},
})
this.client = new Mistral({
apiKey: this.options.mistralApiKey,
httpClient,
})
} catch (error) {
throw new Error(`Error creating Mistral client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const stream = await client.chat
.stream({
model: this.getModel().id,
// max_completion_tokens: this.getModel().info.maxTokens,
temperature: 0,
messages: [{ role: "system", content: systemPrompt }, ...convertToMistralMessages(messages)],
stream: true,
tools: tools?.length ? (tools as MistralTool[]) : undefined,
toolChoice: tools?.length ? "any" : undefined,
})
.catch((err) => {
// The Mistal SDK uses statusCode instead of status
// However, if they introduce status for something, I don't want to override it
if ("statusCode" in err && !("status" in err)) {
err.status = err.statusCode
}
throw err
})
for await (const chunk of stream) {
const delta = chunk.data.choices[0]?.delta
if (delta.toolCalls) {
for (const toolCall of delta.toolCalls) {
yield {
type: "tool_calls",
tool_call: {
function: {
id: toolCall.id,
name: toolCall.function.name,
arguments: JSON.stringify(toolCall.function.arguments),
},
},
}
}
} else if (delta?.content) {
let content: string = ""
if (typeof delta.content === "string") {
content = delta.content
} else if (Array.isArray(delta.content)) {
content = delta.content.map((c) => (c.type === "text" ? c.text : "")).join("")
}
yield {
type: "text",
text: content,
}
}
if (chunk.data.usage) {
yield {
type: "usage",
inputTokens: chunk.data.usage.promptTokens || 0,
outputTokens: chunk.data.usage.completionTokens || 0,
}
}
}
}
getModel(): { id: MistralModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in mistralModels) {
const id = modelId as MistralModelId
return { id, info: mistralModels[id] }
}
return {
id: mistralDefaultModelId,
info: mistralModels[mistralDefaultModelId],
}
}
}
@@ -1,110 +0,0 @@
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ModelInfo, MoonshotModelId, moonshotDefaultModelId, moonshotModels } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface MoonshotHandlerOptions extends CommonApiHandlerOptions {
moonshotApiKey?: string
moonshotApiLine?: string
apiModelId?: string
}
// Enhanced usage interface to support Moonshot's cached token field
interface MoonshotUsage extends OpenAI.CompletionUsage {
cached_tokens?: number
}
export class MoonshotHandler implements ApiHandler {
private client: OpenAI | undefined
constructor(private readonly options: MoonshotHandlerOptions) {}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.moonshotApiKey) {
throw new Error("Moonshot API key is required")
}
try {
this.client = createOpenAIClient({
baseURL:
this.options.moonshotApiLine === "china" ? "https://api.moonshot.cn/v1" : "https://api.moonshot.ai/v1",
apiKey: this.options.moonshotApiKey,
})
} catch (error) {
throw new Error(`Error creating Moonshot client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
messages: openAiMessages,
temperature: model.info.temperature,
max_tokens: model.info.maxTokens,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
const usage = chunk.usage as MoonshotUsage
yield {
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: usage.cached_tokens ?? 0,
inputTokens: (usage.prompt_tokens || 0) - (usage.cached_tokens ?? 0),
outputTokens: usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: MoonshotModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in moonshotModels) {
const id = modelId as MoonshotModelId
return { id, info: moonshotModels[id] }
}
return { id: moonshotDefaultModelId, info: moonshotModels[moonshotDefaultModelId] }
}
}
@@ -1,96 +0,0 @@
import { type ModelInfo, type NebiusModelId, nebiusDefaultModelId, nebiusModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface NebiusHandlerOptions extends CommonApiHandlerOptions {
nebiusApiKey?: string
apiModelId?: string
}
export class NebiusHandler implements ApiHandler {
private client: OpenAI | undefined
constructor(private readonly options: NebiusHandlerOptions) {}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.nebiusApiKey) {
throw new Error("Nebius API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.studio.nebius.ai/v1",
apiKey: this.options.nebiusApiKey,
})
} catch (error) {
throw new Error(`Error creating Nebius client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = model.id.includes("DeepSeek-R1")
? convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
: [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
const stream = await client.chat.completions.create({
model: model.id,
messages: openAiMessages,
temperature: 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId !== undefined && modelId in nebiusModels) {
return { id: modelId, info: nebiusModels[modelId as NebiusModelId] }
}
return { id: nebiusDefaultModelId, info: nebiusModels[nebiusDefaultModelId] }
}
}
@@ -1,93 +0,0 @@
import { ModelInfo, NousResearchModelId, nousResearchDefaultModelId, nousResearchModels } from "@shared/api"
import OpenAI from "openai"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface NousResearchHandlerOptions extends CommonApiHandlerOptions {
nousResearchApiKey?: string
apiModelId?: string
}
export class NousResearchHandler implements ApiHandler {
private options: NousResearchHandlerOptions
private client: OpenAI | undefined
constructor(options: NousResearchHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.nousResearchApiKey) {
throw new Error("NousResearch API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://inference-api.nousResearch.com/v1",
apiKey: this.options.nousResearchApiKey,
})
} catch (error: any) {
throw new Error(`Error creating NousResearch client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
messages: openAiMessages,
temperature: 0,
stream: true,
stream_options: { include_usage: true },
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: NousResearchModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in nousResearchModels) {
const id = modelId as NousResearchModelId
return { id, info: nousResearchModels[id] }
}
return { id: nousResearchDefaultModelId, info: nousResearchModels[nousResearchDefaultModelId] }
}
}
-473
View File
@@ -1,473 +0,0 @@
import { Anthropic, APIError as AnthropicAPIError } from "@anthropic-ai/sdk"
import { liteLlmDefaultModelId, liteLlmModelInfoSaneDefaults, ModelInfo } from "@shared/api"
import OpenAI, { APIError as OpenAIAPIError, OpenAIError } from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { OcaAuthService } from "@/services/auth/oca/OcaAuthService"
import {
DEFAULT_EXTERNAL_OCA_BASE_URL,
DEFAULT_INTERNAL_OCA_BASE_URL,
OCI_HEADER_OPC_REQUEST_ID,
} from "@/services/auth/oca/utils/constants"
import { createOcaHeaders } from "@/services/auth/oca/utils/utils"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { OcaModelInfo } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiFormat } from "@/shared/proto/index.cline"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, type CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
import { convertOpenAIToolsToAnthropicTools, handleAnthropicMessagesApiStreamResponse } from "../utils/messages_api_support"
import { handleResponsesApiStreamResponse } from "../utils/responses_api_support"
export interface OcaHandlerOptions extends CommonApiHandlerOptions {
ocaBaseUrl?: string
ocaModelId?: string
ocaModelInfo?: OcaModelInfo
ocaReasoningEffort?: string
thinkingBudgetTokens?: number
ocaUsePromptCache?: boolean
taskId?: string
ocaMode?: string // "internal" or "external"
}
export class OcaHandler implements ApiHandler {
protected options: OcaHandlerOptions
protected openAIClient: OpenAI | undefined
protected anthropicClient: Anthropic | undefined
protected externalHeaders: Record<string, string> = {}
constructor(options: OcaHandlerOptions) {
this.options = options
}
protected initializeOpenAIClient(options: OcaHandlerOptions): OpenAI {
const externalHeaders = buildExternalBasicHeaders()
return new (class OCIOpenAI extends OpenAI {
protected override async prepareOptions(opts: any): Promise<void> {
const token = await OcaAuthService.getInstance().getAuthToken()
if (!token) {
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
}
opts.headers ??= {}
// OCA Headers
const ociHeaders = await createOcaHeaders(token, options.taskId!)
opts.headers = { ...opts.headers, ...externalHeaders, ...ociHeaders }
Logger.log(`Making request with customer opc-request-id: ${opts.headers?.["opc-request-id"]}`)
return super.prepareOptions(opts)
}
protected override makeStatusError(
status: number | undefined,
error: Object | undefined,
message: string | undefined,
headers: any | undefined,
): OpenAIAPIError {
interface OciError {
code?: string
message?: string
}
let ociErrorMessage = message
if (typeof error === "object" && error !== null) {
try {
ociErrorMessage = JSON.stringify(error)
const ociErr = error as OciError
if (ociErr.code !== undefined && ociErr.message !== undefined) {
ociErrorMessage = `${ociErr.code}: ${ociErr.message}`
}
} catch {}
}
const opcRequestId = headers?.[OCI_HEADER_OPC_REQUEST_ID]
if (opcRequestId) {
ociErrorMessage += `\n(${OCI_HEADER_OPC_REQUEST_ID}: ${opcRequestId})`
}
const statusCode = typeof status === "number" ? status : 500
return super.makeStatusError(statusCode, error ?? {}, ociErrorMessage, headers)
}
})({
baseURL:
options.ocaBaseUrl ||
(options.ocaMode === "internal" ? DEFAULT_INTERNAL_OCA_BASE_URL : DEFAULT_EXTERNAL_OCA_BASE_URL),
apiKey: "noop",
fetch, // Use configured fetch with proxy support
})
}
protected initializeAnthropicClient(options: OcaHandlerOptions): Anthropic {
const externalHeaders = buildExternalBasicHeaders()
return new (class OCIAnthropic extends Anthropic {
protected override async prepareOptions(opts: any): Promise<void> {
const token = await OcaAuthService.getInstance().getAuthToken()
if (!token) {
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
}
opts.headers ??= {}
// OCA Headers
const ociHeaders = await createOcaHeaders(token, options.taskId!)
opts.headers = { ...opts.headers, ...externalHeaders, ...ociHeaders }
Logger.log(`Making request with customer opc-request-id: ${opts.headers?.["opc-request-id"]}`)
return super.prepareOptions(opts)
}
protected override makeStatusError(
status: number | undefined,
error: Object | undefined,
message: string | undefined,
headers: any | undefined,
): AnthropicAPIError {
interface OciError {
code?: string
message?: string
}
let ociErrorMessage = message
if (typeof error === "object" && error !== null) {
try {
ociErrorMessage = JSON.stringify(error)
const ociErr = error as OciError
if (ociErr.code !== undefined && ociErr.message !== undefined) {
ociErrorMessage = `${ociErr.code}: ${ociErr.message}`
}
} catch {}
}
const opcRequestId = headers?.[OCI_HEADER_OPC_REQUEST_ID]
if (opcRequestId) {
ociErrorMessage += `\n(${OCI_HEADER_OPC_REQUEST_ID}: ${opcRequestId})`
}
const statusCode = typeof status === "number" ? status : 500
return super.makeStatusError(statusCode, error ?? {}, ociErrorMessage, headers)
}
})({
baseURL:
options.ocaBaseUrl ||
(options.ocaMode === "internal" ? DEFAULT_INTERNAL_OCA_BASE_URL : DEFAULT_EXTERNAL_OCA_BASE_URL),
apiKey: "noop",
fetch, // Use configured fetch with proxy support
})
}
protected ensureOpenAIClient(): OpenAI {
if (!this.openAIClient) {
if (!this.options.ocaModelId) {
throw new Error("Oracle Code Assist (OCA) model is not selected")
}
try {
this.openAIClient = this.initializeOpenAIClient(this.options)
} catch (error) {
throw new Error(`Error creating Oracle Code Assist (OCA) client: ${error.message}`)
}
}
return this.openAIClient
}
protected ensureAnthropicClient(): Anthropic {
if (!this.anthropicClient) {
if (!this.options.ocaModelId) {
throw new Error("Oracle Code Assist (OCA) model is not selected")
}
try {
this.anthropicClient = this.initializeAnthropicClient(this.options)
} catch (error) {
throw new Error(`Error creating Oracle Code Assist (OCA) client: ${error.message}`)
}
}
return this.anthropicClient
}
async getApiCosts(prompt_tokens: number, completion_tokens: number): Promise<number | undefined> {
// Reference: https://github.com/BerriAI/litellm/blob/122ee634f434014267af104814022af1d9a0882f/litellm/proxy/spend_tracking/spend_management_endpoints.py#L1473
const client = this.ensureOpenAIClient()
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
const token = await OcaAuthService.getInstance().getAuthToken()
if (!token) {
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
}
const externalHeaders = buildExternalBasicHeaders()
const ociHeaders = await createOcaHeaders(token, this.options.taskId!)
Logger.log(`Making calculate cost request with customer opc-request-id: ${ociHeaders["opc-request-id"]}`)
try {
const response = await fetch(`${client.baseURL}/spend/calculate`, {
method: "POST",
headers: { ...externalHeaders, ...ociHeaders },
body: JSON.stringify({
completion_response: {
model: modelId,
usage: {
prompt_tokens,
completion_tokens,
},
},
}),
})
if (response.ok) {
const data: { cost: number } = await response.json()
return data.cost
}
Logger.error("Error calculating spend:", response.statusText)
return undefined
} catch (error) {
Logger.error("Error calculating spend:", error)
return undefined
}
}
async calculateCost(
modelInfo: ModelInfo,
inputTokens: number,
outputTokens: number,
_cacheWriteTokens?: number,
_cacheReadTokens?: number,
) {
const inputCost = (await this.getApiCosts(1e6, 0)) || 0
const outputCost = (await this.getApiCosts(0, 1e6)) || 0
const totalCost = (inputCost * inputTokens) / 1e6 + (outputCost * outputTokens) / 1e6
return totalCost
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
if (this.options.ocaModelInfo?.apiFormat == ApiFormat.OPENAI_RESPONSES) {
yield* this.createMessageResponsesApi(systemPrompt, messages, tools)
} else if (this.options.ocaModelInfo?.apiFormat == ApiFormat.ANTHROPIC_CHAT) {
yield* this.createMessageMessagesApi(systemPrompt, messages, tools)
} else {
yield* this.createMessageChatApi(systemPrompt, messages, tools)
}
}
async *createMessageChatApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureOpenAIClient()
const formattedMessages = convertToOpenAiMessages(messages)
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam = {
role: "system",
content: systemPrompt,
}
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
const isOminiModel = modelId.includes("o1-mini") || modelId.includes("o3-mini") || modelId.includes("o4-mini")
// Configuration for extended thinking
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = budgetTokens !== 0
const thinkingConfig = reasoningOn ? { type: "enabled", budget_tokens: budgetTokens } : undefined
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
const maxTokens: number | undefined = this.options.ocaModelInfo?.maxTokens
if (isOminiModel && reasoningOn) {
temperature = undefined // Thinking mode doesn't support temperature
}
// Define cache control object if prompt caching is enabled
const cacheControl = this.options.ocaUsePromptCache ? { cache_control: { type: "ephemeral" } } : undefined
// Add cache_control to system message if enabled
const enhancedSystemMessage = {
...systemMessage,
...(cacheControl && cacheControl),
}
// Find the last two user messages to apply caching
const userMsgIndices = formattedMessages.reduce((acc, msg, index) => {
if (msg.role === "user") {
acc.push(index)
}
return acc
}, [] as number[])
const lastUserMsgIndex = userMsgIndices[userMsgIndices.length - 1] ?? -1
const secondLastUserMsgIndex = userMsgIndices[userMsgIndices.length - 2] ?? -1
// Apply cache_control to the last two user messages if enabled
const enhancedMessages = formattedMessages.map((message, index) => {
if ((index === lastUserMsgIndex || index === secondLastUserMsgIndex) && cacheControl) {
return {
...message,
...cacheControl,
}
}
return message
})
const toolCallProcessor = new ToolCallProcessor()
const chatCompletionsParams: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming = {
model: this.options.ocaModelId || liteLlmDefaultModelId,
messages: [enhancedSystemMessage, ...enhancedMessages],
temperature,
stream: true,
max_completion_tokens: maxTokens,
max_tokens: maxTokens,
stream_options: { include_usage: true },
...(thinkingConfig && { thinking: thinkingConfig }), // Add thinking configuration when applicable
...(this.options.taskId && {
litellm_session_id: `cline-${this.options.taskId}`,
...getOpenAIToolParams(tools),
}), // Add session ID for LiteLLM tracking
}
if (this.options.ocaModelInfo?.supportsReasoningEffort) {
chatCompletionsParams["reasoning_effort"] = this.options.ocaReasoningEffort || ("medium" as any)
}
const stream = await client.chat.completions.create(chatCompletionsParams)
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle normal text content
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
// Handle reasoning events (thinking)
// Thinking is not in the standard types but may be in the response
interface ThinkingDelta {
thinking?: string
}
if ((delta as ThinkingDelta)?.thinking) {
yield {
type: "reasoning",
reasoning: (delta as ThinkingDelta).thinking || "",
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle token usage information
if (chunk.usage) {
const totalCost = await this.calculateCost(
this.options.ocaModelInfo!,
chunk.usage.prompt_tokens,
chunk.usage.completion_tokens,
)
// Extract cache-related information if available
// Need to use type assertion since these properties are not in the standard OpenAI types
const usage = chunk.usage as {
prompt_tokens: number
completion_tokens: number
cache_creation_input_tokens?: number
prompt_cache_miss_tokens?: number
cache_read_input_tokens?: number
prompt_cache_hit_tokens?: number
}
const cacheWriteTokens = usage.cache_creation_input_tokens || usage.prompt_cache_miss_tokens || 0
const cacheReadTokens = usage.cache_read_input_tokens || usage.prompt_cache_hit_tokens || 0
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
cacheWriteTokens: cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
cacheReadTokens: cacheReadTokens > 0 ? cacheReadTokens : undefined,
totalCost,
}
}
}
}
async *createMessageResponsesApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureOpenAIClient()
const inputMessages = convertToOpenAIResponsesInput(messages, { usePreviousResponseId: false }).input
// Convert messages to Responses API input format
const input: OpenAI.Responses.ResponseInputItem[] = [{ role: "system", content: systemPrompt }, ...inputMessages]
// Convert ChatCompletion tools to Responses API format if provided
const responseTools = tools
?.filter((tool) => tool?.type === "function")
.map((tool: any) => ({
type: "function" as const,
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters,
strict: tool.function.strict ?? true, // Responses API defaults to strict mode
}))
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
const maxOutputTokens: number | undefined = this.options.ocaModelInfo?.maxTokens
const ocaModelInfo = this.options.ocaModelInfo
if (!ocaModelInfo) {
throw new Error("Oracle Code Assist (OCA) model info is required for Responses API")
}
const reasoningOn = !!ocaModelInfo.supportsReasoning
if (reasoningOn) {
temperature = undefined
}
const responsesParams: OpenAI.Responses.ResponseCreateParamsStreaming = {
model: this.options.ocaModelId || liteLlmDefaultModelId,
input,
stream: true,
tools: responseTools,
...(typeof temperature === "number" ? { temperature } : {}),
...(typeof maxOutputTokens === "number" && maxOutputTokens > 0 ? { max_output_tokens: maxOutputTokens } : {}),
}
if (reasoningOn) {
responsesParams.reasoning = { effort: this.options.ocaReasoningEffort as any, summary: "auto" }
}
// Create the response using Responses API
const stream = await client.responses.create(responsesParams)
yield* handleResponsesApiStreamResponse(stream, ocaModelInfo, this.calculateCost.bind(this))
}
async *createMessageMessagesApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureAnthropicClient()
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = this.options.ocaModelInfo?.supportsReasoning && budgetTokens !== 0
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
const maxTokens: number | undefined = this.options.ocaModelInfo?.maxTokens || 8192
if (reasoningOn) {
temperature = 0
}
const anthropicTools = convertOpenAIToolsToAnthropicTools(tools)
const anthropicMessages = sanitizeAnthropicMessages(messages, this.options.ocaUsePromptCache ?? false)
const stream = await client.messages.create({
model: modelId,
max_tokens: maxTokens,
temperature: reasoningOn ? undefined : temperature,
system: [
{
text: systemPrompt,
type: "text",
cache_control: this.options.ocaUsePromptCache ? { type: "ephemeral" } : undefined,
},
],
messages: anthropicMessages,
stream: true,
tools: anthropicTools,
thinking: reasoningOn ? { type: "enabled", budget_tokens: budgetTokens } : undefined,
})
yield* handleAnthropicMessagesApiStreamResponse(stream)
}
getModel() {
return {
id: this.options.ocaModelId || liteLlmDefaultModelId,
info: this.options.ocaModelInfo || liteLlmModelInfoSaneDefaults,
}
}
}
@@ -1,159 +0,0 @@
import { type ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import { type Config, type Message, Ollama } from "ollama"
import type { ChatCompletionTool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import type { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOllamaMessages } from "../transform/ollama-format"
import type { ApiStream } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
interface OllamaHandlerOptions extends CommonApiHandlerOptions {
ollamaBaseUrl?: string
ollamaApiKey?: string
ollamaModelId?: string
ollamaApiOptionsCtxNum?: string
requestTimeoutMs?: number
}
const DEFAULT_CONTEXT_WINDOW = 32768
export class OllamaHandler implements ApiHandler {
private options: OllamaHandlerOptions
private client: Ollama | undefined
constructor(options: OllamaHandlerOptions) {
const ollamaApiOptionsCtxNum = (options.ollamaApiOptionsCtxNum ?? DEFAULT_CONTEXT_WINDOW).toString()
this.options = { ...options, ollamaApiOptionsCtxNum }
}
private ensureClient(): Ollama {
if (!this.client) {
try {
const externalHeaders = buildExternalBasicHeaders()
const clientOptions: Partial<Config> = {
host: this.options.ollamaBaseUrl,
fetch,
headers: externalHeaders,
}
// Add API key if provided (for Ollama cloud or authenticated instances)
if (this.options.ollamaApiKey) {
clientOptions.headers = {
...clientOptions.headers,
Authorization: `Bearer ${this.options.ollamaApiKey}`,
}
}
this.client = new Ollama(clientOptions)
} catch (error) {
throw new Error(`Error creating Ollama client: ${error.message}`)
}
}
return this.client
}
@withRetry({ retryAllErrors: true })
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
const client = this.ensureClient()
const ollamaMessages: Message[] = [{ role: "system", content: systemPrompt }, ...convertToOllamaMessages(messages)]
try {
// Create a promise that rejects after timeout
const timeoutMs = this.options.requestTimeoutMs || 30000
const timeoutPromise = new Promise<never>((_, reject) => {
setTimeout(() => reject(new Error(`Ollama request timed out after ${timeoutMs / 1000} seconds`)), timeoutMs)
})
// Create the actual API request promise
const apiPromise = client.chat({
model: this.getModel().id,
messages: ollamaMessages,
stream: true,
options: {
num_ctx: Number(this.options.ollamaApiOptionsCtxNum),
},
tools: tools as any,
})
const toolCallProcessor = new ToolCallProcessor()
// Race the API request against the timeout
const stream = (await Promise.race([apiPromise, timeoutPromise])) as Awaited<typeof apiPromise>
try {
for await (const chunk of stream) {
Logger.debug("[OllamaHandler] Message Chunk" + JSON.stringify(chunk))
const delta = chunk.message
if (delta?.tool_calls) {
Logger.debug(`[OllamaHandler] Tool Calls Detected: ${JSON.stringify(delta.tool_calls)}`)
yield* toolCallProcessor.processToolCallDeltas(
delta.tool_calls?.map((tc, inx) => ({
index: inx,
id: `ollama-tool-${inx}`,
function: {
name: tc.function.name,
arguments:
typeof tc.function.arguments === "string"
? tc.function.arguments
: JSON.stringify(tc.function.arguments),
},
type: "function",
})),
)
}
if (typeof delta.content === "string") {
yield {
type: "text",
text: delta.content,
}
}
// Handle token usage if available
if (chunk.eval_count !== undefined || chunk.prompt_eval_count !== undefined) {
yield {
type: "usage",
inputTokens: chunk.prompt_eval_count || 0,
outputTokens: chunk.eval_count || 0,
}
}
}
} catch (streamError: any) {
Logger.error("Error processing Ollama stream:", streamError)
throw new Error(`Ollama stream processing error: ${streamError.message || "Unknown error"}`)
}
} catch (error) {
// Check if it's a timeout error
if (error?.message?.includes("timed out")) {
const timeoutMs = this.options.requestTimeoutMs || 30000
throw new Error(`Ollama request timed out after ${timeoutMs / 1000} seconds`)
}
// Enhance error reporting
const statusCode = error.status || error.statusCode
const errorMessage = error.message || "Unknown error"
Logger.error(`Ollama API error (${statusCode || "unknown"}): ${errorMessage}`)
throw error
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.ollamaModelId || "",
info: {
...openAiModelInfoSaneDefaults,
contextWindow: Number(this.options.ollamaApiOptionsCtxNum),
},
}
}
abort(): void {
this.client?.abort()
}
}
@@ -1,732 +0,0 @@
import { ModelInfo, OpenAiCodexModelId, openAiCodexDefaultModelId, openAiCodexModels } from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import OpenAI from "openai"
import type { ChatCompletionTool } from "openai/resources/chat/completions"
import * as os from "os"
import { MessageEvent as UndiciMessageEvent, WebSocket as UndiciWebSocket } from "undici"
import { v7 as uuidv7 } from "uuid"
import { openAiCodexOAuthManager } from "@/integrations/openai-codex/oauth"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { featureFlagsService } from "@/services/feature-flags"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiFormat } from "@/shared/proto/cline/models"
import { FeatureFlag } from "@/shared/services/feature-flags/feature-flags"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
import { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
/**
* OpenAI Codex base URL for API requests
* Routes to chatgpt.com/backend-api/codex
*/
const CODEX_API_BASE_URL = "https://chatgpt.com/backend-api/codex"
const CODEX_RESPONSES_WEBSOCKET_URL = "wss://chatgpt.com/backend-api/codex/responses"
interface OpenAiCodexHandlerOptions extends CommonApiHandlerOptions {
reasoningEffort?: string
apiModelId?: string
}
/**
* OpenAiCodexHandler - Uses OpenAI Responses API with OAuth authentication
*
* Key differences from OpenAiNativeHandler:
* - Uses OAuth Bearer tokens instead of API keys
* - Routes requests to Codex backend (chatgpt.com/backend-api/codex)
* - Subscription-based pricing (no per-token costs)
* - Limited model subset
* - Custom headers for Codex backend
*/
export class OpenAiCodexHandler implements ApiHandler {
private options: OpenAiCodexHandlerOptions
private client?: OpenAI
private responsesWs: UndiciWebSocket | undefined
private websocketRequestInFlight = false
// Session ID for the Codex API (persists for the lifetime of the handler)
private readonly sessionId: string
// Abort controller for cancelling ongoing requests
private abortController?: AbortController
// Track tool call identity for streaming
private pendingToolCallId: string | undefined
private pendingToolCallName: string | undefined
constructor(options: OpenAiCodexHandlerOptions) {
this.options = options
this.sessionId = uuidv7()
}
private normalizeUsage(usage: any, _model: { id: string; info: ModelInfo }): ApiStreamUsageChunk | undefined {
if (!usage) {
return undefined
}
const inputDetails = usage.input_tokens_details ?? usage.prompt_tokens_details
const hasCachedTokens = typeof inputDetails?.cached_tokens === "number"
const hasCacheMissTokens = typeof inputDetails?.cache_miss_tokens === "number"
const cachedFromDetails = hasCachedTokens ? inputDetails.cached_tokens : 0
const missFromDetails = hasCacheMissTokens ? inputDetails.cache_miss_tokens : 0
let totalInputTokens = usage.input_tokens ?? usage.prompt_tokens ?? 0
if (totalInputTokens === 0 && inputDetails && (cachedFromDetails > 0 || missFromDetails > 0)) {
totalInputTokens = cachedFromDetails + missFromDetails
}
const totalOutputTokens = usage.output_tokens ?? usage.completion_tokens ?? 0
const cacheWriteTokens = usage.cache_creation_input_tokens ?? usage.cache_write_tokens ?? 0
const cacheReadTokens =
usage.cache_read_input_tokens ?? usage.cache_read_tokens ?? usage.cached_tokens ?? cachedFromDetails ?? 0
const reasoningTokens =
typeof usage.output_tokens_details?.reasoning_tokens === "number"
? usage.output_tokens_details.reasoning_tokens
: undefined
// Subscription-based: no per-token costs
const out: ApiStreamUsageChunk = {
type: "usage",
inputTokens: totalInputTokens,
outputTokens: totalOutputTokens,
cacheWriteTokens,
cacheReadTokens,
...(typeof reasoningTokens === "number" ? { reasoningTokens } : {}),
totalCost: 0, // Subscription-based pricing
}
return out
}
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
const model = this.getModel()
// Reset state for this request
this.pendingToolCallId = undefined
this.pendingToolCallName = undefined
// Get access token from OAuth manager
let accessToken = await openAiCodexOAuthManager.getAccessToken()
if (!accessToken) {
throw new Error("Not authenticated with OpenAI Codex. Please sign in using the OpenAI Codex OAuth flow in settings.")
}
const useWebsocketMode = this.useWebsocketMode(model.info.apiFormat)
const { input, previousResponseId } = convertToOpenAIResponsesInput(messages, { usePreviousResponseId: useWebsocketMode })
const usePreviousResponseId = useWebsocketMode && !!previousResponseId
// Build request body
const requestBody = this.buildRequestBody(model, input, systemPrompt, tools, previousResponseId)
const fallbackRequestBody = this.buildRequestBody(model, input, systemPrompt, tools)
// Make the request with retry on auth failure
for (let attempt = 0; attempt < 2; attempt++) {
try {
yield* this.executeRequest(requestBody, fallbackRequestBody, model, accessToken, usePreviousResponseId)
return
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
const isAuthFailure = /unauthorized|invalid token|not authenticated|authentication|401/i.test(message)
if (attempt === 0 && isAuthFailure) {
// Force refresh the token for retry
const refreshed = await openAiCodexOAuthManager.forceRefreshAccessToken()
if (!refreshed) {
throw new Error(
"Not authenticated with OpenAI Codex. Please sign in using the OpenAI Codex OAuth flow in settings.",
)
}
accessToken = refreshed
continue
}
throw error
}
}
}
private useWebsocketMode(apiFormat?: ApiFormat): boolean {
if (featureFlagsService.getBooleanFlagEnabled(FeatureFlag.OPENAI_RESPONSES_WEBSOCKET_MODE)) {
return apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE
}
return false
}
private buildRequestBody(
model: { id: string; info: ModelInfo },
formattedInput: any,
systemPrompt: string,
tools?: ChatCompletionTool[],
previousResponseId?: string,
): any {
// Determine reasoning effort
const reasoningEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
const includeReasoning = reasoningEffort !== "none"
const body: any = {
model: model.id,
input: formattedInput,
stream: true,
store: false,
instructions: systemPrompt,
...(previousResponseId ? { previous_response_id: previousResponseId } : {}),
...(includeReasoning ? { include: ["reasoning.encrypted_content"] } : {}),
...(includeReasoning
? {
reasoning: {
effort: reasoningEffort,
summary: "auto",
},
}
: {}),
}
// Add tools if provided
// Pass through strict value from tool (MCP/custom tools have strict: false, built-in tools default to true)
if (tools && tools.length > 0) {
body.tools = tools
.filter((tool: any) => tool?.type === "function")
.map((tool: any) => ({
type: "function",
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters,
strict: tool.function.strict ?? true,
}))
}
return body
}
private async *executeRequest(
requestBody: any,
fallbackRequestBody: any,
model: { id: string; info: ModelInfo },
accessToken: string,
useWebsocketMode: boolean,
): ApiStream {
// Create AbortController for cancellation
this.abortController = new AbortController()
try {
// Get ChatGPT account ID for organization subscriptions
const accountId = await openAiCodexOAuthManager.getAccountId()
// Build Codex-specific headers
const codexHeaders: Record<string, string> = {
originator: "cline",
session_id: this.sessionId,
"User-Agent": `cline/${process.env.npm_package_version || "1.0.0"} (${os.platform()} ${os.release()}; ${os.arch()}) node/${process.version.slice(1)}`,
...(accountId ? { "ChatGPT-Account-Id": accountId } : {}),
...buildExternalBasicHeaders(),
}
if (useWebsocketMode) {
try {
yield* this.createResponseStreamWebsocket(requestBody, fallbackRequestBody, accessToken, codexHeaders, model)
return
} catch (error) {
Logger.error("OpenAI Codex websocket mode failed, falling back to HTTP Responses API:", error)
this.closeResponsesWebsocket()
}
}
// Try using OpenAI SDK first
try {
const client =
this.client ??
new OpenAI({
apiKey: accessToken,
baseURL: CODEX_API_BASE_URL,
defaultHeaders: codexHeaders,
fetch, // Use shared fetch for proxy support
})
const stream = (await (client as any).responses.create(requestBody, {
signal: this.abortController.signal,
headers: codexHeaders,
})) as AsyncIterable<any>
if (typeof (stream as any)?.[Symbol.asyncIterator] !== "function") {
throw new Error("OpenAI SDK did not return an AsyncIterable")
}
for await (const event of stream) {
if (this.abortController.signal.aborted) {
break
}
for await (const outChunk of this.processEvent(event, model)) {
yield outChunk
}
}
} catch (_sdkErr) {
// Fallback to manual SSE via fetch
yield* this.makeCodexRequest(requestBody, model, accessToken)
}
} finally {
this.abortController = undefined
}
}
private async *createResponseStreamWebsocket(
primaryParams: OpenAI.Responses.ResponseCreateParamsStreaming,
fallbackParams: OpenAI.Responses.ResponseCreateParamsStreaming,
accessToken: string,
codexHeaders: Record<string, string>,
model: { id: string; info: ModelInfo },
): ApiStream {
try {
for await (const event of this.createResponseEventsViaWebsocket(primaryParams, accessToken, codexHeaders)) {
if (this.abortController?.signal.aborted) {
return
}
yield* this.processEvent(event, model)
}
} catch (error) {
if (this.shouldRetryWebsocketWithFullContext(error, !!primaryParams.previous_response_id)) {
Logger.log(
"Retrying Codex websocket response with full context after previous_response_not_found or socket reset",
)
this.closeResponsesWebsocket()
for await (const event of this.createResponseEventsViaWebsocket(fallbackParams, accessToken, codexHeaders)) {
if (this.abortController?.signal.aborted) {
return
}
yield* this.processEvent(event, model)
}
return
}
throw error
}
}
private shouldRetryWebsocketWithFullContext(error: unknown, hadPreviousResponseId: boolean): boolean {
const errorCode =
typeof error === "object" && error && "code" in error && typeof (error as { code: unknown }).code === "string"
? (error as { code: string }).code
: undefined
if (hadPreviousResponseId && errorCode === "previous_response_not_found") {
return true
}
if (errorCode === "websocket_closed" || errorCode === "websocket_error") {
return true
}
return false
}
private async ensureResponsesWebsocket(accessToken: string, codexHeaders: Record<string, string>): Promise<UndiciWebSocket> {
if (this.responsesWs && this.responsesWs.readyState === UndiciWebSocket.OPEN) {
return this.responsesWs
}
this.closeResponsesWebsocket()
const ws = new UndiciWebSocket(CODEX_RESPONSES_WEBSOCKET_URL, {
headers: {
Authorization: `Bearer ${accessToken}`,
"OpenAI-Beta": "responses_websockets=2026-02-06",
...codexHeaders,
},
})
await new Promise<void>((resolve, reject) => {
const cleanup = () => {
ws.removeEventListener("open", handleOpen)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
}
const handleOpen = () => {
cleanup()
resolve()
}
const handleError = () => {
cleanup()
reject(new Error("Failed to open Codex Responses websocket"))
}
const handleClose = () => {
cleanup()
reject(new Error("Codex Responses websocket closed before opening"))
}
ws.addEventListener("open", handleOpen)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
})
this.responsesWs = ws
return ws
}
private closeResponsesWebsocket() {
if (this.responsesWs) {
try {
this.responsesWs.close()
} catch {}
this.responsesWs = undefined
}
}
private async *createResponseEventsViaWebsocket(
params: OpenAI.Responses.ResponseCreateParamsStreaming,
accessToken: string,
codexHeaders: Record<string, string>,
): AsyncGenerator<OpenAI.Responses.ResponseStreamEvent> {
if (this.websocketRequestInFlight) {
const error: Error & { code?: string } = new Error("Websocket response.create is already in progress")
error.code = "websocket_concurrency_limit"
throw error
}
const ws = await this.ensureResponsesWebsocket(accessToken, codexHeaders)
this.websocketRequestInFlight = true
const eventQueue: OpenAI.Responses.ResponseStreamEvent[] = []
let resolver: (() => void) | undefined
let completed = false
let failure: (Error & { code?: string }) | undefined
const wake = () => {
const next = resolver
resolver = undefined
next?.()
}
const handleMessage = (evt: UndiciMessageEvent) => {
try {
let raw = ""
if (typeof evt.data === "string") {
raw = evt.data
} else if (evt.data instanceof ArrayBuffer) {
raw = new TextDecoder().decode(new Uint8Array(evt.data))
} else if (ArrayBuffer.isView(evt.data)) {
raw = new TextDecoder().decode(new Uint8Array(evt.data.buffer, evt.data.byteOffset, evt.data.byteLength))
} else {
raw = String(evt.data)
}
const parsed = JSON.parse(raw)
if (parsed?.type === "error" && parsed?.error) {
const error: Error & { code?: string } = new Error(parsed.error.message || "Codex Responses websocket error")
error.code = parsed.error.code
failure = error
completed = true
wake()
return
}
eventQueue.push(parsed as OpenAI.Responses.ResponseStreamEvent)
if (parsed?.type === "response.completed" || parsed?.type === "response.failed") {
completed = true
}
wake()
} catch (error) {
const parseError: Error & { code?: string } = new Error(
`Failed to parse websocket event: ${error instanceof Error ? error.message : String(error)}`,
)
parseError.code = "websocket_parse_error"
failure = parseError
completed = true
wake()
}
}
const handleError = () => {
const error: Error & { code?: string } = new Error("Codex Responses websocket emitted an error event")
error.code = "websocket_error"
failure = error
completed = true
wake()
}
const handleClose = () => {
if (!completed) {
const error: Error & { code?: string } = new Error("Codex Responses websocket closed during response stream")
error.code = "websocket_closed"
failure = error
completed = true
wake()
}
}
ws.addEventListener("message", handleMessage)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
try {
ws.send(
JSON.stringify({
type: "response.create",
...params,
}),
)
while (!completed || eventQueue.length > 0) {
if (eventQueue.length === 0) {
await new Promise<void>((resolve) => {
resolver = resolve
})
continue
}
const event = eventQueue.shift()
if (event) {
yield event
}
}
if (failure) {
throw failure
}
} finally {
ws.removeEventListener("message", handleMessage)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
this.websocketRequestInFlight = false
}
}
private async *makeCodexRequest(requestBody: any, model: { id: string; info: ModelInfo }, accessToken: string): ApiStream {
const url = `${CODEX_API_BASE_URL}/responses`
// Get ChatGPT account ID for organization subscriptions
const accountId = await openAiCodexOAuthManager.getAccountId()
// Build headers with required Codex-specific fields
const headers: Record<string, string> = {
"Content-Type": "application/json",
Authorization: `Bearer ${accessToken}`,
originator: "cline",
session_id: this.sessionId,
"User-Agent": `cline/${process.env.npm_package_version || "1.0.0"} (${os.platform()} ${os.release()}; ${os.arch()}) node/${process.version.slice(1)}`,
}
// Add ChatGPT-Account-Id if available
if (accountId) {
headers["ChatGPT-Account-Id"] = accountId
}
try {
const response = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify(requestBody),
signal: this.abortController?.signal,
})
if (!response.ok) {
const errorText = await response.text()
let errorMessage = `Codex API request failed: ${response.status}`
try {
const errorJson = JSON.parse(errorText)
if (errorJson.error?.message) {
errorMessage = errorJson.error.message
} else if (errorJson.message) {
errorMessage = errorJson.message
}
} catch {
if (errorText) {
errorMessage += ` - ${errorText}`
}
}
throw new Error(errorMessage)
}
if (!response.body) {
throw new Error("No response body from Codex API")
}
yield* this.handleStreamResponse(response.body, model)
} catch (error) {
if (error instanceof Error) {
throw new Error(`Codex API error: ${error.message}`)
}
throw new Error("Unexpected error connecting to Codex API")
}
}
private async *handleStreamResponse(body: ReadableStream<Uint8Array>, model: { id: string; info: ModelInfo }): ApiStream {
const reader = body.getReader()
const decoder = new TextDecoder()
let buffer = ""
try {
while (true) {
if (this.abortController?.signal.aborted) {
break
}
const { done, value } = await reader.read()
if (done) {
break
}
buffer += decoder.decode(value, { stream: true })
const lines = buffer.split("\n")
buffer = lines.pop() || ""
for (const line of lines) {
if (line.startsWith("data: ")) {
const data = line.slice(6).trim()
if (data === "[DONE]") {
continue
}
try {
const parsed = JSON.parse(data)
for await (const outChunk of this.processEvent(parsed, model)) {
yield outChunk
}
} catch (e) {
if (!(e instanceof SyntaxError)) {
throw e
}
}
}
}
}
} finally {
reader.releaseLock()
}
}
private async *processEvent(event: any, model: { id: string; info: ModelInfo }): ApiStream {
// Handle text deltas
if (event?.type === "response.text.delta" || event?.type === "response.output_text.delta") {
if (event?.delta) {
yield { type: "text", text: event.delta }
}
return
}
// Handle reasoning deltas
if (
event?.type === "response.reasoning.delta" ||
event?.type === "response.reasoning_text.delta" ||
event?.type === "response.reasoning_summary.delta" ||
event?.type === "response.reasoning_summary_text.delta"
) {
if (event?.delta) {
yield { type: "reasoning", reasoning: event.delta }
}
return
}
// Handle refusal deltas
if (event?.type === "response.refusal.delta") {
if (event?.delta) {
yield { type: "text", text: `[Refusal] ${event.delta}` }
}
return
}
// Handle tool/function call deltas
if (event?.type === "response.tool_call_arguments.delta" || event?.type === "response.function_call_arguments.delta") {
const callId = event.call_id || event.tool_call_id || event.id || this.pendingToolCallId
const name = event.name || event.function_name || this.pendingToolCallName
const args = event.delta || event.arguments
if (typeof callId === "string" && callId.length > 0 && typeof name === "string" && name.length > 0) {
yield {
type: "tool_calls",
tool_call: {
call_id: callId,
function: {
id: callId,
name,
arguments: typeof args === "string" ? args : "",
},
},
}
}
return
}
// Handle output item events
if (event?.type === "response.output_item.added" || event?.type === "response.output_item.done") {
const item = event?.item
if (item) {
// Capture tool identity for subsequent argument deltas
if (item.type === "function_call" || item.type === "tool_call") {
const callId = item.call_id || item.tool_call_id || item.id
const name = item.name || item.function?.name || item.function_name
if (typeof callId === "string" && callId.length > 0) {
this.pendingToolCallId = callId
this.pendingToolCallName = typeof name === "string" ? name : undefined
}
}
if (item.type === "text" && item.text) {
yield { type: "text", text: item.text }
} else if (item.type === "reasoning" && item.text) {
yield { type: "reasoning", reasoning: item.text }
} else if (item.type === "message" && Array.isArray(item.content)) {
for (const content of item.content) {
if ((content?.type === "text" || content?.type === "output_text") && content?.text) {
yield { type: "text", text: content.text }
}
}
} else if (
(item.type === "function_call" || item.type === "tool_call") &&
event.type === "response.output_item.done"
) {
const callId = item.call_id || item.tool_call_id || item.id
if (callId) {
const args = item.arguments || item.function?.arguments || item.function_arguments
yield {
type: "tool_calls",
id: callId,
tool_call: {
call_id: callId,
function: {
id: callId,
name: item.name || item.function?.name || item.function_name || "",
arguments: typeof args === "string" ? args : "{}",
},
},
}
}
}
}
return
}
// Handle completion events
if (event?.type === "response.done" || event?.type === "response.completed") {
const usage = event?.response?.usage || event?.usage || undefined
const usageData = this.normalizeUsage(usage, model)
if (usageData) {
yield usageData
}
return
}
// Fallbacks for legacy formats
if (event?.choices?.[0]?.delta?.content) {
yield { type: "text", text: event.choices[0].delta.content }
return
}
if (event?.usage) {
const usageData = this.normalizeUsage(event.usage, model)
if (usageData) {
yield usageData
}
}
}
abort(): void {
this.closeResponsesWebsocket()
this.abortController?.abort()
}
getModel(): { id: OpenAiCodexModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
const id = modelId && modelId in openAiCodexModels ? (modelId as OpenAiCodexModelId) : openAiCodexDefaultModelId
const info: ModelInfo = openAiCodexModels[id]
return { id, info }
}
}
@@ -1,702 +0,0 @@
import {
ModelInfo,
OpenAiCompatibleModelInfo,
OpenAiNativeModelId,
openAiNativeDefaultModelId,
openAiNativeModels,
} from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type {
ChatCompletionFunctionTool,
ChatCompletionReasoningEffort,
ChatCompletionTool,
} from "openai/resources/chat/completions"
import { MessageEvent as UndiciMessageEvent, WebSocket as UndiciWebSocket } from "undici"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { featureFlagsService } from "@/services/feature-flags"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiFormat } from "@/shared/proto/cline/models"
import { FeatureFlag } from "@/shared/services/feature-flags/feature-flags"
import { Logger } from "@/shared/services/Logger"
import { isGPT5ModelFamily } from "@/utils/model-utils"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface OpenAiNativeHandlerOptions extends CommonApiHandlerOptions {
openAiNativeApiKey?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
apiModelId?: string
openAiNativeUseResponsesWebsocket?: boolean
}
export class OpenAiNativeHandler implements ApiHandler {
private options: OpenAiNativeHandlerOptions
private client: OpenAI | undefined
private responsesWs: UndiciWebSocket | undefined
private responsesWsReadyPromise: Promise<UndiciWebSocket> | undefined
private websocketRequestInFlight = false
private abortController?: AbortController
constructor(options: OpenAiNativeHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.openAiNativeApiKey) {
throw new Error("OpenAI API key is required")
}
try {
this.client = createOpenAIClient({
apiKey: this.options.openAiNativeApiKey,
})
} catch (error) {
throw new Error(`Error creating OpenAI client: ${error instanceof Error ? error.message : String(error)}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
const inputTokens = usage?.prompt_tokens || 0 // sum of cache hits and misses
const outputTokens = usage?.completion_tokens || 0
const cacheReadTokens = usage?.prompt_tokens_details?.cached_tokens || 0
const cacheWriteTokens = 0
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
totalCost: totalCost,
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
// Responses API requires tool format to be set to OPENAI_RESPONSES with native tools calling enabled
const apiFormat = this.getModel()?.info?.apiFormat
if (apiFormat === ApiFormat.OPENAI_RESPONSES || apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE) {
if (!tools?.length) {
throw new Error("Native Tool Call must be enabled in your setting for OpenAI Responses API")
}
yield* this.createResponseStream(systemPrompt, messages, tools)
} else {
yield* this.createCompletionStream(systemPrompt, messages, tools)
}
}
private async *createCompletionStream(
systemPrompt: string,
messages: ClineStorageMessage[],
tools?: ChatCompletionTool[],
): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const toolCallProcessor = new ToolCallProcessor()
this.abortController = new AbortController()
// Handle o1 models separately as they don't support streaming
if (model.info.supportsStreaming === false) {
const response = await client.chat.completions.create(
{
model: model.id,
messages: [{ role: "user", content: systemPrompt }, ...convertToOpenAiMessages(messages, "openai-native")],
},
{ signal: this.abortController?.signal },
)
yield {
type: "text",
text: response.choices[0]?.message.content || "",
}
yield* this.yieldUsage(model.info, response.usage)
return
}
const systemRole = model.info.systemRole ?? "system"
const includeReasoning = model.info.supportsReasoningEffort
const includeTools = model.info.supportsTools ?? true
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
const reasoningEffort =
includeReasoning && requestedEffort !== "none" ? (requestedEffort as ChatCompletionReasoningEffort) : undefined
const stream = await client.chat.completions.create({
model: model.id,
messages: [{ role: systemRole, content: systemPrompt }, ...convertToOpenAiMessages(messages, "openai-native")],
stream: true,
stream_options: { include_usage: true },
reasoning_effort: reasoningEffort,
...(model.info.temperature !== undefined ? { temperature: model.info.temperature } : {}),
...(includeTools ? getOpenAIToolParams(tools, isGPT5ModelFamily(model.id)) : {}),
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
try {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
} catch (error) {
Logger.error("Error processing tool call delta:", error, delta.tool_calls)
}
}
if (chunk.usage) {
// Only last chunk contains usage
yield* this.yieldUsage(model.info, chunk.usage)
}
}
}
private async *createResponseStream(
systemPrompt: string,
messages: ClineStorageMessage[],
tools: ChatCompletionTool[],
): ApiStream {
const model = this.getModel()
const usePreviousResponseId = this.useWebsocketMode(model.info.apiFormat)
// Warm websocket connection early in websocket mode so the first response.create avoids handshake latency.
if (usePreviousResponseId) {
this.preconnectResponsesWebsocket()
}
const { input, previousResponseId } = convertToOpenAIResponsesInput(messages, { usePreviousResponseId })
const responseTools = this.mapResponseTools(tools)
this.abortController = new AbortController()
const params = this.buildResponseCreateParams({
modelId: model.id,
systemPrompt,
input,
previousResponseId,
tools: responseTools,
})
const fallbackParams = this.buildResponseCreateParams({
modelId: model.id,
systemPrompt,
input,
tools: responseTools,
})
if (usePreviousResponseId && previousResponseId) {
try {
yield* this.createResponseStreamWebsocket(model.info, params, fallbackParams)
return
} catch (error) {
Logger.error("OpenAI websocket mode failed, falling back to HTTP Responses API:", error)
this.closeResponsesWebsocket()
}
}
yield* this.createResponseStreamHttp(model.info, params)
}
private preconnectResponsesWebsocket(): void {
void this.ensureResponsesWebsocket().catch((error) => {
Logger.debug("OpenAI websocket preconnect failed:", error)
this.closeResponsesWebsocket()
})
}
private useWebsocketMode(apiFormat?: ApiFormat): boolean {
if (featureFlagsService.getBooleanFlagEnabled(FeatureFlag.OPENAI_RESPONSES_WEBSOCKET_MODE)) {
return apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE
}
return false
}
private mapResponseTools(tools: ChatCompletionTool[]): OpenAI.Responses.Tool[] {
return tools
?.filter((tool): tool is ChatCompletionFunctionTool => tool?.type === "function")
.map((tool) => ({
type: "function" as const,
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters ?? null,
strict: tool.function.strict ?? true,
}))
}
private buildResponseCreateParams(args: {
modelId: string
systemPrompt: string
input: OpenAI.Responses.ResponseInput
tools: OpenAI.Responses.Tool[]
previousResponseId?: string
}): OpenAI.Responses.ResponseCreateParamsStreaming {
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
const reasoning: { effort: ChatCompletionReasoningEffort; summary: "auto" } | undefined =
requestedEffort === "none"
? undefined
: {
effort: requestedEffort,
summary: "auto",
}
return {
model: args.modelId,
instructions: args.systemPrompt,
input: args.input,
stream: true,
tools: args.tools,
store: !args.previousResponseId, // Do not use store when websocket mode is enabled.
...(args.previousResponseId ? { previous_response_id: args.previousResponseId } : {}),
...(reasoning ? { reasoning } : {}),
}
}
private async *createResponseStreamHttp(
modelInfo: ModelInfo,
params: OpenAI.Responses.ResponseCreateParamsStreaming,
): ApiStream {
const client = this.ensureClient()
Logger.debug(`OpenAI Responses Input (HTTP): ${JSON.stringify(params.input)}`)
const stream = await client.responses.create(params, { signal: this.abortController?.signal })
yield* this.processResponsesEvents(stream, modelInfo)
}
private async *createResponseStreamWebsocket(
modelInfo: ModelInfo,
primaryParams: OpenAI.Responses.ResponseCreateParamsStreaming,
fallbackParams: OpenAI.Responses.ResponseCreateParamsStreaming,
): ApiStream {
Logger.debug(`OpenAI Responses Input (WebSocket): ${JSON.stringify(primaryParams.input)}`)
try {
yield* this.processResponsesEvents(this.createResponseEventsViaWebsocket(primaryParams), modelInfo)
} catch (error) {
if (this.shouldRetryWebsocketWithFullContext(error, !!primaryParams.previous_response_id)) {
Logger.log("Retrying websocket response with full context after previous_response_not_found or socket reset")
this.closeResponsesWebsocket()
yield* this.processResponsesEvents(this.createResponseEventsViaWebsocket(fallbackParams), modelInfo)
return
}
throw error
}
}
private shouldRetryWebsocketWithFullContext(error: unknown, hadPreviousResponseId: boolean): boolean {
const errorCode =
typeof error === "object" && error && "code" in error && typeof (error as { code: unknown }).code === "string"
? (error as { code: string }).code
: undefined
if (hadPreviousResponseId && errorCode === "previous_response_not_found") {
return true
}
if (errorCode === "websocket_closed" || errorCode === "websocket_error") {
return true
}
return false
}
private async ensureResponsesWebsocket(): Promise<UndiciWebSocket> {
if (this.responsesWs && this.responsesWs.readyState === UndiciWebSocket.OPEN) {
return this.responsesWs
}
if (this.responsesWsReadyPromise) {
return this.responsesWsReadyPromise
}
this.closeResponsesWebsocket()
if (!this.options.openAiNativeApiKey) {
throw new Error("OpenAI API key is required")
}
const ws = new UndiciWebSocket("wss://api.openai.com/v1/responses", {
headers: {
Authorization: `Bearer ${this.options.openAiNativeApiKey}`,
"OpenAI-Beta": "responses_websockets=2026-02-06",
...buildExternalBasicHeaders(),
},
})
this.responsesWs = ws
const readyPromise = new Promise<UndiciWebSocket>((resolve, reject) => {
const cleanup = () => {
ws.removeEventListener("open", handleOpen)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
}
const handleOpen = () => {
cleanup()
resolve(ws)
}
const handleError = () => {
cleanup()
reject(new Error("Failed to open Responses websocket"))
}
const handleClose = () => {
cleanup()
reject(new Error("Responses websocket closed before opening"))
}
ws.addEventListener("open", handleOpen)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
})
this.responsesWsReadyPromise = readyPromise
try {
return await readyPromise
} catch (error) {
if (this.responsesWs === ws) {
this.responsesWs = undefined
}
throw error
} finally {
if (this.responsesWsReadyPromise === readyPromise) {
this.responsesWsReadyPromise = undefined
}
}
}
private closeResponsesWebsocket() {
this.responsesWsReadyPromise = undefined
if (this.responsesWs) {
try {
this.responsesWs.close()
} catch {}
this.responsesWs = undefined
}
}
private async *createResponseEventsViaWebsocket(
params: OpenAI.Responses.ResponseCreateParamsStreaming,
): AsyncGenerator<OpenAI.Responses.ResponseStreamEvent> {
if (this.websocketRequestInFlight) {
const error: Error & { code?: string } = new Error("Websocket response.create is already in progress")
error.code = "websocket_concurrency_limit"
throw error
}
const ws = await this.ensureResponsesWebsocket()
this.websocketRequestInFlight = true
const eventQueue: OpenAI.Responses.ResponseStreamEvent[] = []
let resolver: (() => void) | undefined
let completed = false
let failure: (Error & { code?: string }) | undefined
const wake = () => {
const next = resolver
resolver = undefined
next?.()
}
const handleMessage = (evt: UndiciMessageEvent) => {
try {
let raw = ""
if (typeof evt.data === "string") {
raw = evt.data
} else if (evt.data instanceof ArrayBuffer) {
raw = new TextDecoder().decode(new Uint8Array(evt.data))
} else if (ArrayBuffer.isView(evt.data)) {
raw = new TextDecoder().decode(new Uint8Array(evt.data.buffer, evt.data.byteOffset, evt.data.byteLength))
} else {
raw = String(evt.data)
}
const parsed = JSON.parse(raw)
if (parsed?.type === "error" && parsed?.error) {
const error: Error & { code?: string } = new Error(parsed.error.message || "Responses websocket error")
error.code = parsed.error.code
failure = error
completed = true
wake()
return
}
eventQueue.push(parsed as OpenAI.Responses.ResponseStreamEvent)
if (parsed?.type === "response.completed" || parsed?.type === "response.failed") {
completed = true
}
wake()
} catch (error) {
const parseError: Error & { code?: string } = new Error(
`Failed to parse websocket event: ${error instanceof Error ? error.message : String(error)}`,
)
parseError.code = "websocket_parse_error"
failure = parseError
completed = true
wake()
}
}
const handleError = () => {
const error: Error & { code?: string } = new Error("Responses websocket emitted an error event")
error.code = "websocket_error"
failure = error
completed = true
wake()
}
const handleClose = () => {
if (!completed) {
const error: Error & { code?: string } = new Error("Responses websocket closed during response stream")
error.code = "websocket_closed"
failure = error
completed = true
wake()
}
}
ws.addEventListener("message", handleMessage)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
try {
ws.send(
JSON.stringify({
type: "response.create",
...params,
}),
)
while (!completed || eventQueue.length > 0) {
if (eventQueue.length === 0) {
await new Promise<void>((resolve) => {
resolver = resolve
})
continue
}
const event = eventQueue.shift()
if (event) {
yield event
}
}
if (failure) {
throw failure
}
} finally {
ws.removeEventListener("message", handleMessage)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
this.websocketRequestInFlight = false
}
}
private async *processResponsesEvents(
stream: AsyncIterable<OpenAI.Responses.ResponseStreamEvent>,
modelInfo: ModelInfo,
): ApiStream {
const functionCallByItemId = new Map<string, { call_id?: string; name?: string; id?: string }>()
for await (const chunk of stream) {
Logger.debug(`OpenAI Responses Chunk: ${JSON.stringify(chunk)}`)
if (chunk.type === "response.output_item.added") {
const item = chunk.item
if (item.type === "function_call" && item.id) {
functionCallByItemId.set(item.id, { call_id: item.call_id, name: item.name, id: item.id })
yield {
type: "tool_calls",
id: item.id,
tool_call: {
call_id: item.call_id,
function: {
id: item.id,
name: item.name,
arguments: item.arguments,
},
},
}
}
if (item.type === "reasoning" && item.encrypted_content && item.id) {
yield {
type: "reasoning",
id: item.id,
reasoning: "",
redacted_data: item.encrypted_content,
}
}
}
if (chunk.type === "response.output_item.done") {
const item = chunk.item
if (item.type === "function_call") {
if (item.id) {
functionCallByItemId.set(item.id, { call_id: item.call_id, name: item.name, id: item.id })
}
yield {
type: "tool_calls",
id: item.id || item.call_id,
tool_call: {
call_id: item.call_id,
function: {
id: item.id,
name: item.name,
arguments: item.arguments,
},
},
}
}
if (item.type === "reasoning") {
yield {
type: "reasoning",
id: item.id,
details: item.summary,
reasoning: "",
}
}
}
if (chunk.type === "response.reasoning_summary_part.added") {
yield {
type: "reasoning",
id: chunk.item_id,
reasoning: chunk.part.text,
}
}
if (chunk.type === "response.reasoning_summary_text.delta") {
yield {
type: "reasoning",
id: chunk.item_id,
reasoning: chunk.delta,
}
}
if (chunk.type === "response.reasoning_summary_part.done") {
yield {
type: "reasoning",
id: chunk.item_id,
details: chunk.part,
reasoning: "",
}
}
if (chunk.type === "response.output_text.delta") {
if (chunk.delta) {
yield {
id: chunk.item_id,
type: "text",
text: chunk.delta,
}
}
}
if (chunk.type === "response.reasoning_text.delta") {
if (chunk.delta) {
yield {
id: chunk.item_id,
type: "reasoning",
reasoning: chunk.delta,
}
}
}
if (chunk.type === "response.function_call_arguments.delta") {
const pendingCall = functionCallByItemId.get(chunk.item_id)
const callId = pendingCall?.call_id
const functionName = pendingCall?.name
const functionId = pendingCall?.id || chunk.item_id
yield {
type: "tool_calls",
tool_call: {
call_id: callId,
function: {
id: functionId,
name: functionName,
arguments: chunk.delta,
},
},
}
}
if (chunk.type === "response.function_call_arguments.done") {
if (chunk.item_id && chunk.name && chunk.arguments) {
const pendingCall = functionCallByItemId.get(chunk.item_id)
const callId = pendingCall?.call_id
const functionId = pendingCall?.id || chunk.item_id
yield {
type: "tool_calls",
tool_call: {
call_id: callId,
function: {
id: functionId,
name: chunk.name,
arguments: chunk.arguments,
},
},
}
}
}
if (
chunk.type === "response.incomplete" &&
chunk.response?.status === "incomplete" &&
chunk.response?.incomplete_details?.reason === "max_output_tokens"
) {
if (chunk.response?.output_text?.length > 0) {
Logger.log("Partial output:", chunk.response.output_text)
} else {
Logger.log("Ran out of tokens during reasoning")
}
}
if (chunk.type === "response.completed" && chunk.response?.usage) {
const usage = chunk.response.usage
const inputTokens = usage.input_tokens || 0
const outputTokens = usage.output_tokens || 0
const cacheReadTokens = usage.input_tokens_details?.cached_tokens || 0
const cacheWriteTokens = 0
const reasoningTokens = usage.output_tokens_details?.reasoning_tokens || 0
const totalTokens = usage.total_tokens || 0
Logger.log(`Total tokens from Responses API usage: ${totalTokens}`)
const totalCost = calculateApiCostOpenAI(
modelInfo,
inputTokens,
outputTokens + reasoningTokens,
cacheWriteTokens,
cacheReadTokens,
)
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
thoughtsTokenCount: reasoningTokens,
totalCost: totalCost,
id: chunk.response.id,
}
}
}
}
abort(): void {
this.closeResponsesWebsocket()
this.abortController?.abort()
this.abortController = undefined
}
getModel(): { id: OpenAiNativeModelId; info: OpenAiCompatibleModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in openAiNativeModels) {
const id = modelId as OpenAiNativeModelId
const info = openAiNativeModels[id]
return { id, info: { ...info } }
}
return {
id: openAiNativeDefaultModelId,
info: { ...openAiNativeModels[openAiNativeDefaultModelId] },
}
}
}
@@ -1,188 +0,0 @@
import { DefaultAzureCredential, getBearerTokenProvider } from "@azure/identity"
import { azureOpenAiDefaultApiVersion, ModelInfo, OpenAiCompatibleModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import OpenAI, { AzureOpenAI } from "openai"
import type { ChatCompletionReasoningEffort, ChatCompletionTool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient, fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface OpenAiHandlerOptions extends CommonApiHandlerOptions {
openAiApiKey?: string
openAiBaseUrl?: string
azureApiVersion?: string
azureIdentity?: boolean
openAiHeaders?: Record<string, string>
openAiModelId?: string
openAiModelInfo?: OpenAiCompatibleModelInfo
reasoningEffort?: string
}
export class OpenAiHandler implements ApiHandler {
private options: OpenAiHandlerOptions
private client: OpenAI | undefined
constructor(options: OpenAiHandlerOptions) {
this.options = options
}
private getAzureAudienceScope(baseUrl?: string): string {
const url = baseUrl?.toLowerCase() ?? ""
if (url.includes("azure.us")) return "https://cognitiveservices.azure.us/.default"
if (url.includes("azure.com")) return "https://cognitiveservices.azure.com/.default"
return "https://cognitiveservices.azure.com/.default"
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.openAiApiKey && !this.options.azureIdentity) {
throw new Error("OpenAI API key or Azure Identity Authentication is required")
}
try {
const baseUrl = this.options.openAiBaseUrl?.toLowerCase() ?? ""
const isAzureDomain = baseUrl.includes("azure.com") || baseUrl.includes("azure.us")
const externalHeaders = buildExternalBasicHeaders()
// Azure API shape slightly differs from the core API shape...
if (
this.options.azureApiVersion ||
(isAzureDomain && !this.options.openAiModelId?.toLowerCase().includes("deepseek"))
) {
if (this.options.azureIdentity) {
this.client = new AzureOpenAI({
baseURL: this.options.openAiBaseUrl,
azureADTokenProvider: getBearerTokenProvider(
new DefaultAzureCredential(),
this.getAzureAudienceScope(this.options.openAiBaseUrl),
),
apiVersion: this.options.azureApiVersion || azureOpenAiDefaultApiVersion,
defaultHeaders: {
...externalHeaders,
...this.options.openAiHeaders,
},
fetch,
})
} else {
this.client = new AzureOpenAI({
baseURL: this.options.openAiBaseUrl,
apiKey: this.options.openAiApiKey,
apiVersion: this.options.azureApiVersion || azureOpenAiDefaultApiVersion,
defaultHeaders: {
...externalHeaders,
...this.options.openAiHeaders,
},
fetch,
})
}
} else {
this.client = createOpenAIClient({
baseURL: this.options.openAiBaseUrl,
apiKey: this.options.openAiApiKey,
defaultHeaders: this.options.openAiHeaders,
})
}
} catch (error: any) {
throw new Error(`Error creating OpenAI client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.openAiModelId ?? ""
const isDeepseekReasoner = modelId.includes("deepseek-reasoner")
const isR1FormatRequired = this.options.openAiModelInfo?.isR1FormatRequired ?? false
const isReasoningModelFamily =
["o1", "o3", "o4", "gpt-5"].some((prefix) => modelId.includes(prefix)) && !modelId.includes("chat")
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
let temperature: number | undefined
if (this.options.openAiModelInfo?.temperature !== undefined) {
const tempValue = Number(this.options.openAiModelInfo.temperature)
temperature = tempValue === 0 ? undefined : tempValue
} else {
temperature = openAiModelInfoSaneDefaults.temperature
}
let reasoningEffort: ChatCompletionReasoningEffort | undefined
let maxTokens: number | undefined
if (this.options.openAiModelInfo?.maxTokens && this.options.openAiModelInfo.maxTokens > 0) {
maxTokens = Number(this.options.openAiModelInfo.maxTokens)
} else {
maxTokens = undefined
}
if (isDeepseekReasoner || isR1FormatRequired) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
if (isReasoningModelFamily) {
openAiMessages = [{ role: "developer", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
temperature = undefined // does not support temperature
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
reasoningEffort = requestedEffort === "none" ? undefined : (requestedEffort as ChatCompletionReasoningEffort)
}
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
temperature,
max_tokens: maxTokens,
reasoning_effort: reasoningEffort,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.openAiModelId ?? "",
info: this.options.openAiModelInfo ?? openAiModelInfoSaneDefaults,
}
}
}
@@ -1,235 +0,0 @@
import { setTimeout as setTimeoutPromise } from "node:timers/promises"
import { StateManager } from "@core/storage/StateManager"
import { ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import axios from "axios"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient, getAxiosSettings } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { createOpenRouterStream } from "../transform/openrouter-stream"
import { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
import { OpenRouterErrorResponse } from "./types"
interface OpenRouterHandlerOptions extends CommonApiHandlerOptions {
openRouterApiKey?: string
openRouterModelId?: string
openRouterModelInfo?: ModelInfo
openRouterProviderSorting?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
enableParallelToolCalling?: boolean
}
export class OpenRouterHandler implements ApiHandler {
private options: OpenRouterHandlerOptions
private client: OpenAI | undefined
lastGenerationId?: string
constructor(options: OpenRouterHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.openRouterApiKey) {
throw new Error("OpenRouter API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://openrouter.ai/api/v1",
apiKey: this.options.openRouterApiKey,
defaultHeaders: {
"HTTP-Referer": "https://cline.bot", // Optional, for including your app on openrouter.ai rankings.
"X-Title": "Cline", // Optional. Shows in rankings on openrouter.ai.
},
})
} catch (error: any) {
throw new Error(`Error creating OpenRouter client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
this.lastGenerationId = undefined
const stream = await createOpenRouterStream(
client,
systemPrompt,
messages,
this.getModel(),
this.options.reasoningEffort,
this.options.thinkingBudgetTokens,
this.options.openRouterProviderSorting,
tools,
this.options.enableParallelToolCalling,
)
let didOutputUsage = false
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
// openrouter returns an error object instead of the openai sdk throwing an error
// Check for error field directly on chunk
if ("error" in chunk) {
const error = chunk.error as OpenRouterErrorResponse["error"]
Logger.error(`OpenRouter API Error: ${error?.code} - ${error?.message}`)
// Include metadata in the error message if available
const metadataStr = error.metadata ? `\nMetadata: ${JSON.stringify(error.metadata, null, 2)}` : ""
throw new Error(`OpenRouter API Error ${error.code}: ${error.message}${metadataStr}`)
}
// Check for error in choices[0].finish_reason
// OpenRouter may return errors in a non-standard way within choices
const choice = chunk.choices?.[0]
// Use type assertion since OpenRouter uses non-standard "error" finish_reason
if ((choice?.finish_reason as string) === "error") {
// Use type assertion since OpenRouter adds non-standard error property
const choiceWithError = choice as any
if (choiceWithError.error) {
const error = choiceWithError.error
Logger.error(
`OpenRouter Mid-Stream Error: ${error?.code || "Unknown"} - ${error?.message || "Unknown error"}`,
)
// Format error details
const errorDetails = typeof error === "object" ? JSON.stringify(error, null, 2) : String(error)
throw new Error(`OpenRouter Mid-Stream Error: ${errorDetails}`)
}
// Fallback if error details are not available
throw new Error(`OpenRouter Mid-Stream Error: Stream terminated with error status but no error details provided`)
}
if (!this.lastGenerationId && chunk.id) {
this.lastGenerationId = chunk.id
}
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Reasoning tokens are returned separately from the content
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
if (
delta &&
"reasoning" in delta &&
delta.reasoning &&
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
// OpenRouter passes reasoning details that we can pass back unmodified in api requests to preserve reasoning traces for model
// See: https://openrouter.ai/docs/use-cases/reasoning-tokens#preserving-reasoning-blocks
if (
delta &&
"reasoning_details" in delta &&
delta.reasoning_details &&
// @ts-expect-error-next-line
delta.reasoning_details.length && // exists and non-0
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: "",
details: delta.reasoning_details,
}
}
if (!didOutputUsage && chunk.usage) {
// @ts-expect-error-next-line -- OpenRouter returns cache_write_tokens for Anthropic models
const cacheWriteTokens = chunk.usage.prompt_tokens_details?.cache_write_tokens || 0
yield {
type: "usage",
cacheWriteTokens,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
inputTokens:
(chunk.usage.prompt_tokens || 0) -
(chunk.usage.prompt_tokens_details?.cached_tokens || 0) -
(cacheWriteTokens || 0),
outputTokens: chunk.usage.completion_tokens || 0,
// @ts-expect-error-next-line
totalCost: (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0),
}
didOutputUsage = true
}
}
// Fallback to generation endpoint if usage chunk not returned
if (!didOutputUsage) {
const apiStreamUsage = await this.getApiStreamUsage()
if (apiStreamUsage) {
yield apiStreamUsage
}
}
}
async getApiStreamUsage(): Promise<ApiStreamUsageChunk | undefined> {
if (this.lastGenerationId) {
await setTimeoutPromise(500) // FIXME: necessary delay to ensure generation endpoint is ready
try {
const generationIterator = this.fetchGenerationDetails(this.lastGenerationId)
const generation = (await generationIterator.next()).value
// Logger.log("OpenRouter generation details:", generation)
return {
type: "usage",
cacheWriteTokens: generation?.native_tokens_cache_write || 0,
cacheReadTokens: generation?.native_tokens_cached || 0,
// openrouter generation endpoint fails often
inputTokens: (generation?.native_tokens_prompt || 0) - (generation?.native_tokens_cached || 0),
outputTokens: generation?.native_tokens_completion || 0,
totalCost: generation?.total_cost || 0,
}
} catch (error) {
// ignore if fails
Logger.error("Error fetching OpenRouter generation details:", error)
}
}
return undefined
}
@withRetry({ maxRetries: 4, baseDelay: 250, maxDelay: 1000, retryAllErrors: true })
async *fetchGenerationDetails(genId: string) {
// Logger.log("Fetching generation details for:", genId)
try {
const response = await axios.get(`https://openrouter.ai/api/v1/generation?id=${genId}`, {
headers: {
Authorization: `Bearer ${this.options.openRouterApiKey}`,
},
timeout: 15_000, // this request hangs sometimes
...getAxiosSettings(),
})
yield response.data?.data
} catch (error) {
// ignore if fails
Logger.error("Error fetching OpenRouter generation details:", error)
throw error
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.openRouterModelId || openRouterDefaultModelId
const cachedModelInfo = StateManager.get().getModelInfo("openRouter", modelId)
return {
id: modelId,
info: cachedModelInfo || openRouterDefaultModelInfo,
}
}
}
@@ -1,286 +0,0 @@
import { promises as fs } from "node:fs"
import { ModelInfo, QwenCodeModelId, qwenCodeDefaultModelId, qwenCodeModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import * as os from "os"
import * as path from "path"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
// --- Constants for Qwen OAuth2 ---
const QWEN_OAUTH_BASE_URL = "https://chat.qwen.ai"
const QWEN_OAUTH_TOKEN_ENDPOINT = `${QWEN_OAUTH_BASE_URL}/api/v1/oauth2/token`
const QWEN_OAUTH_CLIENT_ID = "f0304373b74a44d2b584a3fb70ca9e56"
const QWEN_DIR = ".qwen"
const QWEN_CREDENTIAL_FILENAME = "oauth_creds.json"
interface QwenOAuthCredentials {
access_token: string
refresh_token: string
token_type: string
expiry_date: number
resource_url?: string
}
interface QwenCodeHandlerOptions extends CommonApiHandlerOptions {
qwenCodeOauthPath?: string
apiModelId?: string
}
function getQwenCachedCredentialPath(customPath?: string): string {
if (customPath) {
// Support custom path that starts with ~/ or is absolute
if (customPath.startsWith("~/")) {
return path.join(os.homedir(), customPath.slice(2))
}
return path.resolve(customPath)
}
return path.join(os.homedir(), QWEN_DIR, QWEN_CREDENTIAL_FILENAME)
}
function objectToUrlEncoded(data: Record<string, string>): string {
return Object.keys(data)
.map((key) => `${encodeURIComponent(key)}=${encodeURIComponent(data[key])}`)
.join("&")
}
export class QwenCodeHandler implements ApiHandler {
private options: QwenCodeHandlerOptions
private credentials: QwenOAuthCredentials | null = null
private client: OpenAI | undefined
constructor(options: QwenCodeHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
// Create the client instance with dummy key initially
// The API key will be updated dynamically via ensureAuthenticated
const externalHeaders = buildExternalBasicHeaders()
this.client = new OpenAI({
apiKey: "dummy-key-will-be-replaced",
baseURL: "https://dashscope.aliyuncs.com/compatible-mode/v1",
defaultHeaders: externalHeaders,
fetch,
})
}
return this.client
}
private async loadCachedQwenCredentials(): Promise<QwenOAuthCredentials> {
try {
const keyFile = getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)
const credsStr = await fs.readFile(keyFile, "utf-8")
return JSON.parse(credsStr)
} catch (error) {
Logger.error(
`Error reading or parsing credentials file at ${getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)}`,
)
throw new Error(`Failed to load Qwen OAuth credentials: ${error}`)
}
}
private async refreshAccessToken(credentials: QwenOAuthCredentials): Promise<QwenOAuthCredentials> {
if (!credentials.refresh_token) {
throw new Error("No refresh token available in credentials.")
}
const bodyData = {
grant_type: "refresh_token",
refresh_token: credentials.refresh_token,
client_id: QWEN_OAUTH_CLIENT_ID,
}
const response = await fetch(QWEN_OAUTH_TOKEN_ENDPOINT, {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded",
Accept: "application/json",
},
body: objectToUrlEncoded(bodyData),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Token refresh failed: ${response.status} ${response.statusText}. Response: ${errorText}`)
}
const tokenData = await response.json()
if (tokenData.error) {
throw new Error(`Token refresh failed: ${tokenData.error} - ${tokenData.error_description}`)
}
const newCredentials = {
...credentials,
access_token: tokenData.access_token,
token_type: tokenData.token_type,
refresh_token: tokenData.refresh_token || credentials.refresh_token,
expiry_date: Date.now() + tokenData.expires_in * 1000,
}
const filePath = getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)
await fs.writeFile(filePath, JSON.stringify(newCredentials, null, 2))
return newCredentials
}
private isTokenValid(credentials: QwenOAuthCredentials): boolean {
const TOKEN_REFRESH_BUFFER_MS = 30 * 1000 // 30s buffer
if (!credentials.expiry_date) {
return false
}
return Date.now() < credentials.expiry_date - TOKEN_REFRESH_BUFFER_MS
}
private async ensureAuthenticated(): Promise<void> {
if (!this.credentials) {
this.credentials = await this.loadCachedQwenCredentials()
}
if (!this.isTokenValid(this.credentials)) {
this.credentials = await this.refreshAccessToken(this.credentials)
}
// After authentication, update the apiKey and baseURL on the existing client
const client = this.ensureClient()
client.apiKey = this.credentials.access_token
client.baseURL = this.getBaseUrl(this.credentials)
}
private getBaseUrl(creds: QwenOAuthCredentials): string {
let baseUrl = creds.resource_url || "https://dashscope.aliyuncs.com/compatible-mode/v1"
if (!baseUrl.startsWith("http://") && !baseUrl.startsWith("https://")) {
baseUrl = `https://${baseUrl}`
}
return baseUrl.endsWith("/v1") ? baseUrl : `${baseUrl}/v1`
}
private async callApiWithRetry<T>(apiCall: () => Promise<T>): Promise<T> {
try {
return await apiCall()
} catch (error: any) {
if (error.status === 401) {
// Token expired, refresh and retry
this.credentials = await this.refreshAccessToken(this.credentials!)
const client = this.ensureClient()
client.apiKey = this.credentials.access_token
client.baseURL = this.getBaseUrl(this.credentials)
return await apiCall()
} else {
throw error
}
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
await this.ensureAuthenticated()
const client = this.ensureClient()
const model = this.getModel()
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam = {
role: "system",
content: systemPrompt,
}
const convertedMessages = [systemMessage, ...convertToOpenAiMessages(messages)]
const requestOptions: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming = {
model: model.id,
temperature: 0,
messages: convertedMessages,
stream: true,
stream_options: { include_usage: true },
max_completion_tokens: model.info.maxTokens,
...getOpenAIToolParams(tools),
}
const stream = await this.callApiWithRetry(() => client.chat.completions.create(requestOptions))
const toolCallProcessor = new ToolCallProcessor()
let fullContent = ""
for await (const apiChunk of stream) {
const delta = apiChunk.choices[0]?.delta ?? {}
if (delta.content) {
let newText = delta.content
if (newText.startsWith(fullContent)) {
newText = newText.substring(fullContent.length)
}
fullContent = delta.content
if (newText) {
// Check for thinking blocks
if (newText.includes("<think>") || newText.includes("</think>")) {
// Simple parsing for thinking blocks
const parts = newText.split(/<\/?think>/g)
for (let i = 0; i < parts.length; i++) {
if (parts[i]) {
if (i % 2 === 0) {
// Outside thinking block
yield {
type: "text",
text: parts[i],
}
} else {
// Inside thinking block
yield {
type: "reasoning",
reasoning: parts[i],
}
}
}
}
} else {
yield {
type: "text",
text: newText,
}
}
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle reasoning content (o1-style)
if ("reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (apiChunk.usage) {
yield {
type: "usage",
inputTokens: apiChunk.usage.prompt_tokens || 0,
outputTokens: apiChunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: QwenCodeModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in qwenCodeModels) {
const id = modelId as QwenCodeModelId
return { id, info: qwenCodeModels[id] }
}
return {
id: qwenCodeDefaultModelId,
info: qwenCodeModels[qwenCodeDefaultModelId],
}
}
}
-163
View File
@@ -1,163 +0,0 @@
import {
InternationalQwenModelId,
internationalQwenDefaultModelId,
internationalQwenModels,
MainlandQwenModelId,
ModelInfo,
mainlandQwenDefaultModelId,
mainlandQwenModels,
QwenApiRegions,
} from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface QwenHandlerOptions extends CommonApiHandlerOptions {
qwenApiKey?: string
qwenApiLine?: QwenApiRegions
apiModelId?: string
thinkingBudgetTokens?: number
}
export class QwenHandler implements ApiHandler {
private options: QwenHandlerOptions
private client: OpenAI | undefined
constructor(options: QwenHandlerOptions) {
// Ensure options start with defaults but allow overrides
this.options = {
qwenApiLine: QwenApiRegions.CHINA,
...options,
}
}
private useChinaApi(): boolean {
return this.options.qwenApiLine === QwenApiRegions.CHINA
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.qwenApiKey) {
throw new Error("Alibaba API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: this.useChinaApi()
? "https://dashscope.aliyuncs.com/compatible-mode/v1"
: "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
apiKey: this.options.qwenApiKey,
})
} catch (error: any) {
throw new Error(`Error creating Alibaba client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: MainlandQwenModelId | InternationalQwenModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
// Branch based on API line to let poor typescript know what to do
if (this.useChinaApi()) {
const id = modelId && modelId in mainlandQwenModels ? (modelId as MainlandQwenModelId) : mainlandQwenDefaultModelId
return {
id,
info: mainlandQwenModels[id],
}
}
const id =
modelId && modelId in internationalQwenModels
? (modelId as InternationalQwenModelId)
: internationalQwenDefaultModelId
return {
id,
info: internationalQwenModels[id],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const isDeepseekReasoner = model.id.includes("deepseek-r1")
const isReasoningModelFamily = model.id.includes("qwen3") || ["qwen-plus-latest", "qwen-turbo-latest"].includes(model.id)
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
let temperature: number | undefined = 0
// Configuration for extended thinking
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = budgetTokens !== 0
const thinkingArgs = isReasoningModelFamily
? {
enable_thinking: reasoningOn,
thinking_budget: reasoningOn ? budgetTokens : undefined,
}
: undefined
if (isDeepseekReasoner || (reasoningOn && isReasoningModelFamily)) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
temperature = undefined
}
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature,
...thinkingArgs,
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
try {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
} catch (error) {
Logger.error("Error processing tool call delta:", error, delta.tool_calls)
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
// @ts-expect-error-next-line
cacheReadTokens: chunk.usage.prompt_cache_hit_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
}
@@ -1,162 +0,0 @@
import { ModelInfo, requestyDefaultModelId, requestyDefaultModelInfo } from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import { toRequestyServiceStringUrl } from "@/shared/clients/requesty"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface RequestyHandlerOptions extends CommonApiHandlerOptions {
requestyBaseUrl?: string
requestyApiKey?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
requestyModelId?: string
requestyModelInfo?: ModelInfo
}
// Requesty usage includes an extra field for Anthropic use cases.
// Safely cast the prompt token details section to the appropriate structure.
interface RequestyUsage extends OpenAI.CompletionUsage {
prompt_tokens_details?: {
caching_tokens?: number
cached_tokens?: number
}
total_cost?: number
}
export class RequestyHandler implements ApiHandler {
private options: RequestyHandlerOptions
private client: OpenAI | undefined
constructor(options: RequestyHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.requestyApiKey) {
throw new Error("Requesty API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: toRequestyServiceStringUrl(this.options.requestyBaseUrl),
apiKey: this.options.requestyApiKey,
defaultHeaders: {
"HTTP-Referer": "https://cline.bot",
"X-Title": "Cline",
},
})
} catch (error: any) {
throw new Error(`Error creating Requesty client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const reasoningEffort = this.options.reasoningEffort || "medium"
const reasoning = { reasoning_effort: reasoningEffort }
const reasoningArgs = model.id.startsWith("openai/o") ? reasoning : {}
const thinkingBudget = this.options.thinkingBudgetTokens || 0
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(model.id)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, thinkingBudget)
: undefined
const thinking =
thinkingBudget > 0
? { thinking: { type: "enabled", budget_tokens: thinkingBudget } }
: { thinking: { type: "disabled" } }
const supportsLegacyClaudeThinking =
!isAdaptiveThinkingModel &&
(model.id.includes("claude-3-7-sonnet") ||
model.id.includes("claude-4.6-sonnet") ||
model.id.includes("claude-sonnet-4") ||
model.id.includes("claude-opus-4"))
const thinkingArgs = isAdaptiveThinkingModel
? adaptiveThinking?.enabled
? {
thinking: { type: "adaptive" },
...(adaptiveThinking.effort ? { output_config: { effort: adaptiveThinking.effort } } : {}),
}
: {}
: supportsLegacyClaudeThinking
? thinking
: {}
const stream = await client.chat.completions.create({
model: model.id,
max_tokens: model.info.maxTokens || undefined,
messages: openAiMessages,
...(isAdaptiveThinkingModel ? {} : { temperature: 0 }),
stream: true,
stream_options: { include_usage: true },
...reasoningArgs,
...thinkingArgs,
})
let lastUsage: any
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
lastUsage = chunk.usage
}
}
if (lastUsage) {
const usage = lastUsage as RequestyUsage
const inputTokens = usage.prompt_tokens || 0
const outputTokens = usage.completion_tokens || 0
const cacheWriteTokens = usage.prompt_tokens_details?.caching_tokens || undefined
const cacheReadTokens = usage.prompt_tokens_details?.cached_tokens || undefined
const totalCost = calculateApiCostOpenAI(model.info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
yield {
type: "usage",
inputTokens: inputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
totalCost: totalCost,
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.requestyModelId
const modelInfo = this.options.requestyModelInfo
if (modelId && modelInfo) {
return { id: modelId, info: modelInfo }
}
return { id: requestyDefaultModelId, info: requestyDefaultModelInfo }
}
}
@@ -1,103 +0,0 @@
import { ModelInfo, SambanovaModelId, sambanovaDefaultModelId, sambanovaModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface SambanovaHandlerOptions extends CommonApiHandlerOptions {
sambanovaApiKey?: string
apiModelId?: string
}
export class SambanovaHandler implements ApiHandler {
private options: SambanovaHandlerOptions
private client: OpenAI | undefined
constructor(options: SambanovaHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.sambanovaApiKey) {
throw new Error("SambaNova API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.sambanova.ai/v1",
apiKey: this.options.sambanovaApiKey,
})
} catch (error: any) {
throw new Error(`Error creating SambaNova client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const modelId = model.id.toLowerCase()
if (modelId.includes("deepseek") || modelId.includes("qwen3")) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
const toolCallProcessor = new ToolCallProcessor()
const stream = await client.chat.completions.create({
model: this.getModel().id,
messages: openAiMessages,
temperature: model.info.temperature ?? 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in sambanovaModels) {
const id = modelId as SambanovaModelId
return { id, info: sambanovaModels[id] }
}
return {
id: sambanovaDefaultModelId,
info: sambanovaModels[sambanovaDefaultModelId],
}
}
}
File diff suppressed because it is too large Load Diff
@@ -1,103 +0,0 @@
import { ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface TogetherHandlerOptions extends CommonApiHandlerOptions {
togetherApiKey?: string
togetherModelId?: string
}
export class TogetherHandler implements ApiHandler {
private options: TogetherHandlerOptions
private client: OpenAI | undefined
constructor(options: TogetherHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.togetherApiKey) {
throw new Error("Together API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.together.xyz/v1",
apiKey: this.options.togetherApiKey,
})
} catch (error: any) {
throw new Error(`Error creating Together client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.togetherModelId ?? ""
const isDeepseekReasoner = modelId.includes("deepseek-reasoner")
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
if (isDeepseekReasoner) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
temperature: 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.togetherModelId ?? "",
info: openAiModelInfoSaneDefaults,
}
}
}
@@ -1,29 +0,0 @@
// For the following openrouter error type sources, see the docs here:
// https://openrouter.ai/docs/api-reference/errors
export interface LanguageModelChatSelector {
vendor?: string
family?: string
version?: string
id?: string
}
export type OpenRouterErrorResponse = {
error: {
message: string
code: number
metadata?: OpenRouterProviderErrorMetadata | OpenRouterModerationErrorMetadata | Record<string, unknown>
}
}
export type OpenRouterProviderErrorMetadata = {
provider_name: string // The name of the provider that encountered the error
raw: unknown // The raw error from the provider
}
export type OpenRouterModerationErrorMetadata = {
reasons: string[] // Why your input was flagged
flagged_input: string // The text segment that was flagged, limited to 100 characters. If the flagged input is longer than 100 characters, it will be truncated in the middle and replaced with ...
provider_name: string // The name of the provider that requested moderation
model_slug: string
}
@@ -1,164 +0,0 @@
import { ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
import { createVercelAIGatewayStream } from "../transform/vercel-ai-gateway-stream"
interface VercelAIGatewayHandlerOptions extends CommonApiHandlerOptions {
vercelAiGatewayApiKey?: string
openRouterModelId?: string
openRouterModelInfo?: ModelInfo
reasoningEffort?: string
thinkingBudgetTokens?: number
}
function getCacheReadTokens(usage: any): number {
return usage?.prompt_tokens_details?.cached_tokens || usage?.cache_read_input_tokens || 0
}
function getCacheWriteTokens(usage: any): number {
return usage?.prompt_tokens_details?.cache_write_tokens || usage?.cache_creation_input_tokens || 0
}
export class VercelAIGatewayHandler implements ApiHandler {
private options: VercelAIGatewayHandlerOptions
private client: OpenAI | undefined
constructor(options: VercelAIGatewayHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.vercelAiGatewayApiKey) {
throw new Error("Vercel AI Gateway API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://ai-gateway.vercel.sh/v1",
apiKey: this.options.vercelAiGatewayApiKey,
defaultHeaders: {
"http-referer": "https://cline.bot",
"x-title": "Cline",
},
})
} catch (error: any) {
throw new Error(`Error creating Vercel AI Gateway client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.getModel().id
const modelInfo = this.getModel().info
try {
const stream = await createVercelAIGatewayStream(
client,
systemPrompt,
messages,
{ id: modelId, info: modelInfo },
this.options.reasoningEffort,
this.options.thinkingBudgetTokens,
tools,
)
let didOutputUsage = false
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Reasoning tokens are returned separately from the content
// Skip reasoning content for models that don't support it (e.g., devstral, grok-4)
if (
delta &&
"reasoning" in delta &&
delta.reasoning &&
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
// Reasoning details that can be passed back in API requests to preserve reasoning traces
if (
delta &&
"reasoning_details" in delta &&
delta.reasoning_details &&
// @ts-expect-error-next-line
delta.reasoning_details.length && // exists and non-0
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: "",
details: delta.reasoning_details,
}
}
if (!didOutputUsage && chunk.usage) {
// @ts-expect-error - Vercel AI Gateway extends OpenAI types
const totalCost = (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0)
const cacheReadTokens = getCacheReadTokens(chunk.usage)
const cacheWriteTokens = getCacheWriteTokens(chunk.usage)
yield {
type: "usage",
cacheWriteTokens,
cacheReadTokens,
inputTokens: Math.max(0, (chunk.usage.prompt_tokens || 0) - cacheReadTokens - cacheWriteTokens),
outputTokens: chunk.usage.completion_tokens || 0,
totalCost,
}
didOutputUsage = true
}
}
if (!didOutputUsage) {
Logger.warn("Vercel AI Gateway did not provide usage information in stream")
}
} catch (error: any) {
Logger.error("Vercel AI Gateway error details:", error)
Logger.error("Error stack:", error.stack)
throw new Error(`Vercel AI Gateway error: ${error.message}`)
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.openRouterModelId
const modelInfo = this.options.openRouterModelInfo
if (modelId && modelInfo) {
return { id: modelId, info: modelInfo }
}
// If we have a model ID but no model info, preserve the selected model ID
// and fall back only the metadata to defaults.
if (modelId) {
return { id: modelId, info: openRouterDefaultModelInfo }
}
return { id: openRouterDefaultModelId, info: openRouterDefaultModelInfo }
}
}
@@ -1,281 +0,0 @@
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
import { AnthropicVertex } from "@anthropic-ai/vertex-sdk"
import { FunctionDeclaration as GoogleTool } from "@google/genai"
import { CLAUDE_SONNET_1M_SUFFIX, ModelInfo, VertexModelId, vertexDefaultModelId, vertexModels } from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { ClineTool } from "@/shared/tools"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { ApiStream } from "../transform/stream"
import { GeminiHandler } from "./gemini"
interface VertexHandlerOptions extends CommonApiHandlerOptions {
vertexProjectId?: string
vertexRegion?: string
apiModelId?: string
thinkingBudgetTokens?: number
geminiApiKey?: string
geminiBaseUrl?: string
ulid?: string
reasoningEffort?: string
}
export class VertexHandler implements ApiHandler {
private geminiHandler: GeminiHandler | undefined
private clientAnthropic: AnthropicVertex | undefined
private options: VertexHandlerOptions
constructor(options: VertexHandlerOptions) {
this.options = options
}
private ensureGeminiHandler(): GeminiHandler {
if (!this.geminiHandler) {
try {
// Create a GeminiHandler with isVertex flag for Gemini models
this.geminiHandler = new GeminiHandler({
...this.options,
isVertex: true,
})
} catch (error: any) {
throw new Error(`Error creating Vertex AI Gemini handler: ${error.message}`)
}
}
return this.geminiHandler
}
private ensureAnthropicClient(): AnthropicVertex {
if (!this.clientAnthropic) {
if (!this.options.vertexProjectId) {
throw new Error("Vertex AI project ID is required")
}
if (!this.options.vertexRegion) {
throw new Error("Vertex AI region is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
// Initialize Anthropic client for Claude models.
// The AnthropicVertex SDK constructs the base URL as `${region}-aiplatform.googleapis.com`,
// but the global endpoint uses `aiplatform.googleapis.com` (no region prefix).
// See: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-partner-models#global
this.clientAnthropic = new AnthropicVertex({
projectId: this.options.vertexProjectId,
// https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#regions
region: this.options.vertexRegion,
...(this.options.vertexRegion === "global"
? { baseURL: "https://aiplatform.googleapis.com/v1" }
: {}),
defaultHeaders: externalHeaders,
})
} catch (error: any) {
throw new Error(`Error creating Vertex AI Anthropic client: ${error.message}`)
}
}
return this.clientAnthropic
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ClineTool[]): ApiStream {
const model = this.getModel()
const rawModelId = model.id
const modelId = rawModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
? rawModelId.slice(0, -CLAUDE_SONNET_1M_SUFFIX.length)
: rawModelId
const enable1mContextWindow = rawModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
// For Gemini models, use the GeminiHandler
if (!rawModelId.includes("claude")) {
const geminiHandler = this.ensureGeminiHandler()
yield* geminiHandler.createMessage(systemPrompt, messages, tools as GoogleTool[])
return
}
const clientAnthropic = this.ensureAnthropicClient()
// Claude implementation
const budget_tokens = this.options.thinkingBudgetTokens || 0
// Use model metadata to determine if reasoning should be enabled
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
// Claude Opus 4.5+ uses adaptive thinking instead of budgeted extended thinking.
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budget_tokens)
: undefined
const adaptiveThinkingEnabled = adaptiveThinking?.enabled === true
const adaptiveThinkingEffort = adaptiveThinking?.effort
const thinkingEnabled = isAdaptiveThinkingModel ? adaptiveThinkingEnabled : reasoningOn
const thinkingConfig = thinkingEnabled
? isAdaptiveThinkingModel
? ({ type: "adaptive" } as any)
: { type: "enabled", budget_tokens: budget_tokens }
: undefined
const outputConfig = isAdaptiveThinkingModel && adaptiveThinkingEffort ? { effort: adaptiveThinkingEffort } : undefined
// Tools are available only when native tools are enabled.
const nativeToolsOn = tools?.length ? tools?.length > 0 : false
const anthropicMessages = sanitizeAnthropicMessages(messages, model.info.supportsPromptCache ?? false)
const requestBody: Record<string, unknown> = {
model: modelId,
max_tokens: model.info.maxTokens || 8192,
thinking: thinkingConfig,
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
system: [
{
text: systemPrompt,
type: "text",
cache_control: model.info.supportsPromptCache ? { type: "ephemeral" } : undefined,
},
],
messages: anthropicMessages,
stream: true,
tools: nativeToolsOn ? (tools as AnthropicTool[]) : undefined,
// tool_choice options:
// - none: disables tool use, even if tools are provided. Claude will not call any tools.
// - auto: allows Claude to decide whether to call any provided tools or not. This is the default value when tools are provided.
// - any: tells Claude that it must use one of the provided tools, but doesnt force a particular tool.
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
tool_choice: nativeToolsOn && !thinkingEnabled ? { type: "any" } : undefined,
}
if (outputConfig) {
requestBody.output_config = outputConfig
}
const stream = (await clientAnthropic.beta.messages.create(
requestBody as any,
enable1mContextWindow
? {
headers: {
"anthropic-beta": "context-1m-2025-08-07",
},
}
: undefined,
)) as unknown as AsyncIterable<any>
const lastStartedToolCall = { id: "", name: "", arguments: "" }
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start": {
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
break
}
case "message_delta":
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage?.output_tokens || 0,
}
break
case "message_stop":
break
case "content_block_start":
switch (chunk.content_block.type) {
case "thinking":
yield {
type: "reasoning",
reasoning: chunk.content_block.thinking || "",
}
break
case "redacted_thinking":
// Handle redacted thinking blocks - we still mark it as reasoning
// but note that the content is encrypted
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
}
break
case "tool_use":
if (chunk.content_block.id && chunk.content_block.name) {
// Convert Anthropic tool_use to OpenAI-compatible format
lastStartedToolCall.id = chunk.content_block.id
lastStartedToolCall.name = chunk.content_block.name
lastStartedToolCall.arguments = ""
}
break
case "text":
if (chunk.index > 0) {
yield {
type: "text",
text: "\n",
}
}
yield {
type: "text",
text: chunk.content_block.text,
}
break
}
break
case "content_block_delta":
switch (chunk.delta.type) {
case "signature_delta":
yield {
type: "reasoning",
reasoning: "",
signature: chunk.delta.signature,
}
break
case "thinking_delta":
yield {
type: "reasoning",
reasoning: chunk.delta.thinking,
}
break
case "input_json_delta":
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
// // Convert Anthropic tool_use to OpenAI-compatible format
yield {
type: "tool_calls",
tool_call: {
...lastStartedToolCall,
function: {
id: lastStartedToolCall.id,
name: lastStartedToolCall.name,
arguments: chunk.delta.partial_json,
},
},
}
}
break
case "text_delta":
yield {
type: "text",
text: chunk.delta.text,
}
break
}
break
case "content_block_stop":
lastStartedToolCall.id = ""
lastStartedToolCall.name = ""
lastStartedToolCall.arguments = ""
break
}
}
}
getModel(): { id: VertexModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in vertexModels) {
const id = modelId as VertexModelId
return { id, info: vertexModels[id] }
}
return {
id: vertexDefaultModelId,
info: vertexModels[vertexDefaultModelId],
}
}
}
@@ -1,594 +0,0 @@
import { ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import { SELECTOR_SEPARATOR, stringifyVsCodeLmModelSelector } from "@shared/vsCodeSelectorUtils"
import { calculateApiCostAnthropic } from "@utils/cost"
import * as vscode from "vscode"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions, SingleCompletionHandler } from "../"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
import { convertToVsCodeLmMessages } from "../transform/vscode-lm-format"
import type { LanguageModelChatSelector as LanguageModelChatSelectorFromTypes } from "./types"
interface VsCodeLmHandlerOptions extends CommonApiHandlerOptions {
vsCodeLmModelSelector?: any
}
// Cline does not update VSCode type definitions or engine requirements to maintain compatibility.
// This declaration (as seen in src/integrations/TerminalManager.ts) provides types for the Language Model API in newer versions of VSCode.
// Extracted from https://github.com/microsoft/vscode/blob/131ee0ef660d600cd0a7e6058375b281553abe20/src/vscode-dts/vscode.d.ts
declare module "vscode" {
enum LanguageModelChatMessageRole {
User = 1,
Assistant = 2,
}
enum LanguageModelChatToolMode {
Auto = 1,
Required = 2,
}
interface LanguageModelChatSelector extends LanguageModelChatSelectorFromTypes {}
interface LanguageModelChatTool {
name: string
description: string
inputSchema?: object
}
interface LanguageModelChatRequestOptions {
justification?: string
modelOptions?: { [name: string]: any }
tools?: LanguageModelChatTool[]
toolMode?: LanguageModelChatToolMode
}
class LanguageModelTextPart {
value: string
constructor(value: string)
}
class LanguageModelToolCallPart {
callId: string
name: string
input: object
constructor(callId: string, name: string, input: object)
}
interface LanguageModelChatResponse {
stream: AsyncIterable<LanguageModelTextPart | LanguageModelToolCallPart | unknown>
text: AsyncIterable<string>
}
interface LanguageModelChat {
readonly name: string
readonly id: string
readonly vendor: string
readonly family: string
readonly version: string
readonly maxInputTokens: number
sendRequest(
messages: LanguageModelChatMessage[],
options?: LanguageModelChatRequestOptions,
token?: CancellationToken,
): Thenable<LanguageModelChatResponse>
countTokens(text: string | LanguageModelChatMessage, token?: CancellationToken): Thenable<number>
}
class LanguageModelPromptTsxPart {
value: unknown
constructor(value: unknown)
}
class LanguageModelToolResultPart {
callId: string
content: Array<LanguageModelTextPart | LanguageModelPromptTsxPart | unknown>
constructor(callId: string, content: Array<LanguageModelTextPart | LanguageModelPromptTsxPart | unknown>)
}
class LanguageModelChatMessage {
static User(
content: string | Array<LanguageModelTextPart | LanguageModelToolResultPart>,
name?: string,
): LanguageModelChatMessage
static Assistant(
content: string | Array<LanguageModelTextPart | LanguageModelToolCallPart>,
name?: string,
): LanguageModelChatMessage
role: LanguageModelChatMessageRole
content: Array<LanguageModelTextPart | LanguageModelToolResultPart | LanguageModelToolCallPart>
name: string | undefined
constructor(
role: LanguageModelChatMessageRole,
content: string | Array<LanguageModelTextPart | LanguageModelToolResultPart | LanguageModelToolCallPart>,
name?: string,
)
}
namespace lm {
function selectChatModels(selector?: LanguageModelChatSelector): Thenable<LanguageModelChat[]>
}
}
/**
* Handles interaction with VS Code's Language Model API for chat-based operations.
* This handler implements the ApiHandler interface to provide VS Code LM specific functionality.
*
* @implements {ApiHandler}
*
* @remarks
* The handler manages a VS Code language model chat client and provides methods to:
* - Create and manage chat client instances
* - Stream messages using VS Code's Language Model API
* - Retrieve model information
*
* @example
* ```typescript
* const options = {
* vsCodeLmModelSelector: { vendor: "copilot", family: "gpt-4" }
* };
* const handler = new VsCodeLmHandler(options);
*
* // Stream a conversation
* const systemPrompt = "You are a helpful assistant";
* const messages = [{ role: "user", content: "Hello!" }];
* for await (const chunk of handler.createMessage(systemPrompt, messages)) {
* Logger.log(chunk);
* }
* ```
*/
export class VsCodeLmHandler implements ApiHandler, SingleCompletionHandler {
private options: VsCodeLmHandlerOptions
private client: vscode.LanguageModelChat | null
private disposable: vscode.Disposable | null
private currentRequestCancellation: vscode.CancellationTokenSource | null
constructor(options: VsCodeLmHandlerOptions) {
this.options = options
this.client = null
this.disposable = null
this.currentRequestCancellation = null
try {
// Listen for model changes and reset client
this.disposable = vscode.workspace.onDidChangeConfiguration((event) => {
if (event.affectsConfiguration("lm")) {
try {
this.client = null
this.ensureCleanState()
} catch (error) {
Logger.error("Error during configuration change cleanup:", error)
}
}
})
} catch (error) {
// Ensure cleanup if constructor fails
this.dispose()
throw new Error(
`Cline <Language Model API>: Failed to initialize handler: ${error instanceof Error ? error.message : "Unknown error"}`,
)
}
}
/**
* Creates a language model chat client based on the provided selector.
*
* @param selector - Selector criteria to filter language model chat instances
* @returns Promise resolving to the first matching language model chat instance
* @throws Error when no matching models are found with the given selector
*
* @example
* const selector = { vendor: "copilot", family: "gpt-4o" };
* const chatClient = await createClient(selector);
*/
async createClient(selector: vscode.LanguageModelChatSelector): Promise<vscode.LanguageModelChat> {
try {
const models = await vscode.lm.selectChatModels(selector)
// Use first available model or create a minimal model object
if (models && Array.isArray(models) && models.length > 0) {
return models[0]
}
// Create a minimal model if no models are available
return {
id: "default-lm",
name: "Default Language Model",
vendor: "vscode",
family: "lm",
version: "1.0",
maxInputTokens: 8192,
sendRequest: async (_messages, _options, _token) => {
// Provide a minimal implementation
return {
stream: (async function* () {
yield new vscode.LanguageModelTextPart(
"Language model functionality is limited. Please check VS Code configuration.",
)
})(),
text: (async function* () {
yield "Language model functionality is limited. Please check VS Code configuration."
})(),
}
},
countTokens: async () => 0,
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : "Unknown error"
throw new Error(`Cline <Language Model API>: Failed to select model: ${errorMessage}`)
}
}
/**
* Creates and streams a message using the VS Code Language Model API.
*
* @param systemPrompt - The system prompt to initialize the conversation context
* @param messages - An array of message parameters following the Anthropic message format
*
* @yields {ApiStream} An async generator that yields either text chunks or tool calls from the model response
*
* @throws {Error} When vsCodeLmModelSelector option is not provided
* @throws {Error} When the response stream encounters an error
*
* @remarks
* This method handles the initialization of the VS Code LM client if not already created,
* converts the messages to VS Code LM format, and streams the response chunks.
* Tool calls handling is currently a work in progress.
*/
dispose(): void {
if (this.disposable) {
this.disposable.dispose()
}
if (this.currentRequestCancellation) {
this.currentRequestCancellation.cancel()
this.currentRequestCancellation.dispose()
}
}
private extractTextFromMessage(message: vscode.LanguageModelChatMessage): string {
if (Array.isArray(message.content)) {
return message.content
.filter((part) => part instanceof vscode.LanguageModelTextPart)
.map((part) => (part as vscode.LanguageModelTextPart).value)
.join("")
}
return ""
}
private async countTokens(text: string | vscode.LanguageModelChatMessage): Promise<number> {
/**
* NOTE (intentional trade-off):
* We use a coarse chars/4 heuristic here instead of a real tokenizer (e.g., js-tiktoken with o200k_base).
* Rationale:
* - Avoid pulling multiMB rank files and increasing the extension install/download size.
* - Eliminate encoder lifecycle/memory concerns in long-running sessions.
* Consequences:
* - This is not model-accurate and can under/over-estimate tokens, especially with tool/function calls.
* - It is “good enough” for budgeting/context checks, and we accept the inaccuracy by design.
* If precise accounting becomes a requirement, reintroduce a tokenizer behind a feature flag or backend-only path.
*/
const textContent = typeof text === "string" ? text : this.extractTextFromMessage(text)
return Math.ceil((textContent || "").length / 4)
}
private async calculateTotalInputTokens(vsCodeLmMessages: vscode.LanguageModelChatMessage[]): Promise<number> {
const messageTokens: number[] = await Promise.all(vsCodeLmMessages.map((msg) => this.countTokens(msg)))
return messageTokens.reduce((sum: number, tokens: number): number => sum + tokens, 0)
}
private ensureCleanState(): void {
if (this.currentRequestCancellation) {
this.currentRequestCancellation.cancel()
this.currentRequestCancellation.dispose()
this.currentRequestCancellation = null
}
}
private async getClient(): Promise<vscode.LanguageModelChat> {
if (!this.client) {
Logger.debug("Cline <Language Model API>: Getting client with options:", {
vsCodeLmModelSelector: this.options.vsCodeLmModelSelector,
hasOptions: !!this.options,
selectorKeys: this.options.vsCodeLmModelSelector ? Object.keys(this.options.vsCodeLmModelSelector) : [],
})
try {
// Use default empty selector if none provided to get all available models
const selector = this.options?.vsCodeLmModelSelector || {}
Logger.debug("Cline <Language Model API>: Creating client with selector:", selector)
this.client = await this.createClient(selector)
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error"
Logger.error("Cline <Language Model API>: Client creation failed:", message)
throw new Error(`Cline <Language Model API>: Failed to create client: ${message}`)
}
}
return this.client
}
private cleanTerminalOutput(text: string): string {
if (!text) {
return ""
}
return (
text
// Normalize line breaks
.replace(/\r\n/g, "\n")
.replace(/\r/g, "\n")
// Remove ANSI escape sequences
.replace(/\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])/g, "") // Full set of ANSI sequences
.replace(/\x9B[0-?]*[ -/]*[@-~]/g, "") // CSI sequences
// Remove terminal title setting sequences and other OSC sequences
.replace(/\x1B\][0-9;]*(?:\x07|\x1B\\)/g, "")
// Remove control characters
.replace(/[\x00-\x09\x0B-\x0C\x0E-\x1F\x7F]/g, "")
// Remove VS Code escape sequences
.replace(/\x1B[PD].*?\x1B\\/g, "") // DCS sequences
.replace(/\x1B_.*?\x1B\\/g, "") // APC sequences
.replace(/\x1B\^.*?\x1B\\/g, "") // PM sequences
.replace(/\x1B\[[\d;]*[HfABCDEFGJKST]/g, "") // Cursor movement and clear screen
// Remove Windows paths and service information
.replace(/^(?:PS )?[A-Z]:\\[^\n]*$/gm, "")
.replace(/^;?Cwd=.*$/gm, "")
// Clean escaped sequences
.replace(/\\x[0-9a-fA-F]{2}/g, "")
.replace(/\\u[0-9a-fA-F]{4}/g, "")
// Final cleanup
.replace(/\n{3,}/g, "\n\n") // Remove multiple empty lines
.trim()
)
}
private cleanMessageContent(content: any): any {
if (!content) {
return content
}
if (typeof content === "string") {
return this.cleanTerminalOutput(content)
}
if (Array.isArray(content)) {
return content.map((item) => this.cleanMessageContent(item))
}
if (typeof content === "object") {
const cleaned: any = {}
for (const [key, value] of Object.entries(content)) {
cleaned[key] = this.cleanMessageContent(value)
}
return cleaned
}
return content
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
// Ensure clean state before starting a new request
this.ensureCleanState()
const client: vscode.LanguageModelChat = await this.getClient()
// Clean system prompt and messages
const cleanedSystemPrompt = this.cleanTerminalOutput(systemPrompt)
const cleanedMessages = messages.map((msg) => ({
...msg,
content: this.cleanMessageContent(msg.content),
}))
// Convert Anthropic messages to VS Code LM messages
const vsCodeLmMessages: vscode.LanguageModelChatMessage[] = [
vscode.LanguageModelChatMessage.Assistant(cleanedSystemPrompt),
...convertToVsCodeLmMessages(cleanedMessages),
]
// Initialize cancellation token for the request
this.currentRequestCancellation = new vscode.CancellationTokenSource()
// Calculate input tokens before starting the stream
const totalInputTokens: number = await this.calculateTotalInputTokens(vsCodeLmMessages)
// Accumulate the text and count at the end of the stream to reduce token counting overhead.
let accumulatedText: string = ""
try {
// Create the response stream with minimal required options
const requestOptions: vscode.LanguageModelChatRequestOptions = {
justification: `Cline would like to use '${client.name}' from '${client.vendor}', Click 'Allow' to proceed.`,
}
// Note: Tool support is currently provided by the VSCode Language Model API directly
// Extensions can register tools using vscode.lm.registerTool()
const response: vscode.LanguageModelChatResponse = await client.sendRequest(
vsCodeLmMessages,
requestOptions,
this.currentRequestCancellation.token,
)
// Consume the stream and handle both text and tool call chunks
for await (const chunk of response.stream) {
if (chunk instanceof vscode.LanguageModelTextPart) {
// Validate text part value
if (typeof chunk.value !== "string") {
Logger.warn("Cline <Language Model API>: Invalid text part value received:", chunk.value)
continue
}
accumulatedText += chunk.value
yield {
type: "text",
text: chunk.value,
}
} else if (chunk instanceof vscode.LanguageModelToolCallPart) {
try {
// Validate tool call parameters
if (!chunk.name || typeof chunk.name !== "string") {
Logger.warn("Cline <Language Model API>: Invalid tool name received:", chunk.name)
continue
}
if (!chunk.callId || typeof chunk.callId !== "string") {
Logger.warn("Cline <Language Model API>: Invalid tool callId received:", chunk.callId)
continue
}
// Ensure input is a valid object
if (!chunk.input || typeof chunk.input !== "object") {
Logger.warn("Cline <Language Model API>: Invalid tool input received:", chunk.input)
continue
}
// Convert tool calls to text format with proper error handling
const toolCall = {
type: "tool_call",
name: chunk.name,
arguments: chunk.input,
callId: chunk.callId,
}
const toolCallText = JSON.stringify(toolCall)
accumulatedText += toolCallText
// Log tool call for debugging
Logger.debug("Cline <Language Model API>: Processing tool call:", {
name: chunk.name,
callId: chunk.callId,
inputSize: JSON.stringify(chunk.input).length,
})
yield {
type: "text",
text: toolCallText,
}
} catch (error) {
Logger.error("Cline <Language Model API>: Failed to process tool call:", error)
}
} else {
Logger.warn("Cline <Language Model API>: Unknown chunk type received:", chunk)
}
}
// Count tokens in the accumulated text after stream completion
const totalOutputTokens: number = await this.countTokens(accumulatedText)
// Report final usage after stream completion
yield {
type: "usage",
inputTokens: totalInputTokens,
outputTokens: totalOutputTokens,
totalCost: calculateApiCostAnthropic(this.getModel().info, totalInputTokens, totalOutputTokens),
}
} catch (error: unknown) {
this.ensureCleanState()
if (error instanceof vscode.CancellationError) {
throw new Error("Cline <Language Model API>: Request cancelled by user")
}
if (error instanceof Error) {
Logger.error("Cline <Language Model API>: Stream error details:", {
message: error.message,
stack: error.stack,
name: error.name,
})
// Return original error if it's already an Error instance
throw error
} else if (typeof error === "object" && error !== null) {
// Handle error-like objects
const errorDetails = JSON.stringify(error, null, 2)
Logger.error("Cline <Language Model API>: Stream error object:", errorDetails)
throw new Error(`Cline <Language Model API>: Response stream error: ${errorDetails}`)
} else {
// Fallback for unknown error types
const errorMessage = String(error)
Logger.error("Cline <Language Model API>: Unknown stream error:", errorMessage)
throw new Error(`Cline <Language Model API>: Response stream error: ${errorMessage}`)
}
}
}
// Return model information based on the current client state
getModel(): { id: string; info: ModelInfo } {
if (this.client) {
// Validate client properties
const requiredProps = {
id: this.client.id,
vendor: this.client.vendor,
family: this.client.family,
version: this.client.version,
maxInputTokens: this.client.maxInputTokens,
}
// Log any missing properties for debugging
for (const [prop, value] of Object.entries(requiredProps)) {
if (!value && value !== 0) {
Logger.warn(`Cline <Language Model API>: Client missing ${prop} property`)
}
}
// Construct model ID using available information
const modelParts = [this.client.vendor, this.client.family, this.client.version].filter(Boolean)
const modelId = this.client.id || modelParts.join(SELECTOR_SEPARATOR)
// Build model info with conservative defaults for missing values
const modelInfo: ModelInfo = {
maxTokens: -1, // Unlimited tokens by default
contextWindow:
typeof this.client.maxInputTokens === "number"
? Math.max(0, this.client.maxInputTokens)
: openAiModelInfoSaneDefaults.contextWindow,
supportsImages: false, // VSCode Language Model API currently doesn't support image inputs
supportsPromptCache: true,
inputPrice: 0,
outputPrice: 0,
description: `VSCode Language Model: ${modelId}`,
}
return { id: modelId, info: modelInfo }
}
// Fallback when no client is available
const fallbackId = this.options.vsCodeLmModelSelector
? stringifyVsCodeLmModelSelector(this.options.vsCodeLmModelSelector)
: "vscode-lm"
Logger.debug("Cline <Language Model API>: No client available, using fallback model info")
return {
id: fallbackId,
info: {
...openAiModelInfoSaneDefaults,
description: `VSCode Language Model (Fallback): ${fallbackId}`,
},
}
}
async completePrompt(prompt: string): Promise<string> {
try {
const client = await this.getClient()
const response = await client.sendRequest(
[vscode.LanguageModelChatMessage.User(prompt)],
{},
new vscode.CancellationTokenSource().token,
)
let result = ""
for await (const chunk of response.stream) {
if (chunk instanceof vscode.LanguageModelTextPart) {
result += chunk.value
}
}
return result
} catch (error) {
if (error instanceof Error) {
throw new Error(`VSCode LM completion error: ${error.message}`)
}
throw error
}
}
}
-102
View File
@@ -1,102 +0,0 @@
import { openAiModelInfoSaneDefaults, type ModelInfo, type WandbModelId, wandbDefaultModelId, wandbModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface WandbHandlerOptions extends CommonApiHandlerOptions {
wandbApiKey?: string
apiModelId?: string
}
export class WandbHandler implements ApiHandler {
private client: OpenAI | undefined
constructor(private readonly options: WandbHandlerOptions) {}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.wandbApiKey) {
throw new Error("W&B API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.inference.wandb.ai/v1",
apiKey: this.options.wandbApiKey,
})
} catch (error) {
throw new Error(`Error creating W&B Inference client: ${error instanceof Error ? error.message : String(error)}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const stream = await client.chat.completions.create({
model: model.id,
messages: [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)],
temperature: 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning" in delta && delta.reasoning) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
// W&B Inference returns prompt_tokens_details.cached_tokens in the usage chunk,
// but does not currently offer cache-aware billing (cached tokens are billed
// at the same rate as regular input tokens). We report inputTokens as the full
// prompt_tokens value and do not subtract cached tokens until W&B supports
// cache-aware pricing. This may change in a future update.
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId?.trim()
if (modelId && modelId in wandbModels) {
return { id: modelId, info: wandbModels[modelId as WandbModelId] }
}
if (modelId) {
return { id: modelId, info: openAiModelInfoSaneDefaults }
}
return { id: wandbDefaultModelId, info: wandbModels[wandbDefaultModelId] }
}
}
-118
View File
@@ -1,118 +0,0 @@
import { ModelInfo, XAIModelId, xaiDefaultModelId, xaiModels } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ChatCompletionReasoningEffort } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface XAIHandlerOptions extends CommonApiHandlerOptions {
xaiApiKey?: string
reasoningEffort?: string
apiModelId?: string
}
export class XAIHandler implements ApiHandler {
private options: XAIHandlerOptions
private client: OpenAI | undefined
constructor(options: XAIHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.xaiApiKey) {
throw new Error("xAI API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.x.ai/v1",
apiKey: this.options.xaiApiKey,
})
} catch (error: any) {
throw new Error(`Error creating xAI client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.getModel().id
// ensure reasoning effort is either "low" or "high" for grok-3-mini
let reasoningEffort: ChatCompletionReasoningEffort | undefined
if (modelId.includes("3-mini")) {
let reasoningEffort = this.options.reasoningEffort
if (reasoningEffort && !["low", "high"].includes(reasoningEffort)) {
reasoningEffort = undefined
}
}
const stream = await client.chat.completions.create({
model: modelId,
max_completion_tokens: this.getModel().info.maxTokens,
temperature: 0,
messages: [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)],
stream: true,
stream_options: { include_usage: true },
reasoning_effort: reasoningEffort,
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
if (!shouldSkipReasoningForModel(modelId)) {
yield {
type: "reasoning",
// @ts-expect-error-next-line
reasoning: delta.reasoning_content,
}
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: XAIModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in xaiModels) {
const id = modelId as XAIModelId
return { id, info: xaiModels[id] }
}
return {
id: xaiDefaultModelId,
info: xaiModels[xaiDefaultModelId],
}
}
}
-120
View File
@@ -1,120 +0,0 @@
import {
internationalZAiDefaultModelId,
internationalZAiModelId,
internationalZAiModels,
ModelInfo,
mainlandZAiDefaultModelId,
mainlandZAiModelId,
mainlandZAiModels,
} from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { version as extensionVersion } from "../../../../package.json"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface ZAiHandlerOptions extends CommonApiHandlerOptions {
zaiApiLine?: string
zaiApiKey?: string
apiModelId?: string
}
export class ZAiHandler implements ApiHandler {
private options: ZAiHandlerOptions
private client: OpenAI | undefined
constructor(options: ZAiHandlerOptions) {
this.options = options
}
private useChinaApi(): boolean {
return this.options.zaiApiLine === "china"
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.zaiApiKey) {
throw new Error("Z AI API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: this.useChinaApi() ? "https://open.bigmodel.cn/api/paas/v4" : "https://api.z.ai/api/paas/v4",
apiKey: this.options.zaiApiKey,
defaultHeaders: {
"HTTP-Referer": "https://cline.bot",
"X-Title": "Cline",
"X-Cline-Version": extensionVersion,
},
})
} catch (error: any) {
throw new Error(`Error creating Z AI client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: mainlandZAiModelId | internationalZAiModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (this.useChinaApi()) {
const id = modelId && modelId in mainlandZAiModels ? (modelId as mainlandZAiModelId) : mainlandZAiDefaultModelId
return {
id,
info: mainlandZAiModels[id],
}
}
const id =
modelId && modelId in internationalZAiModels ? (modelId as internationalZAiModelId) : internationalZAiDefaultModelId
return {
id,
info: internationalZAiModels[id],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
cacheWriteTokens: 0,
}
}
}
}
}
-237
View File
@@ -1,237 +0,0 @@
import { describe, it } from "mocha"
import "should"
import sinon from "sinon"
import { withRetry } from "./retry"
describe("Retry Decorator", () => {
afterEach(() => {
sinon.restore()
})
describe("withRetry", () => {
it("should not retry on success", async () => {
let callCount = 0
class TestClass {
@withRetry()
async *successMethod() {
callCount++
yield "success"
}
}
const test = new TestClass()
const result = []
for await (const value of test.successMethod()) {
result.push(value)
}
callCount.should.equal(1)
result.should.deepEqual(["success"])
})
it("should retry on rate limit (429) error", async () => {
let callCount = 0
class TestClass {
@withRetry({ maxRetries: 2, baseDelay: 10, maxDelay: 100 })
async *failMethod() {
callCount++
if (callCount === 1) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
throw error
}
yield "success after retry"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(2)
result.should.deepEqual(["success after retry"])
})
it("should not retry on non-rate-limit errors", async () => {
let callCount = 0
class TestClass {
@withRetry()
async *failMethod() {
callCount++
throw new Error("Regular error")
}
}
const test = new TestClass()
try {
for await (const _ of test.failMethod()) {
// Should not reach here
}
throw new Error("Should have thrown")
} catch (error: any) {
error.message.should.equal("Regular error")
callCount.should.equal(1)
}
})
it("should respect retry-after header with delta seconds", async () => {
let callCount = 0
const setTimeoutSpy = sinon.spy(global, "setTimeout")
const baseDelay = 1000
class TestClass {
@withRetry({ maxRetries: 2, baseDelay }) // Use large baseDelay to ensure header takes precedence
async *failMethod() {
callCount++
if (callCount === 1) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
error.headers = { "retry-after": "0.01" } // 10ms delay
throw error
}
yield "success after retry"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(2)
setTimeoutSpy.calledOnce.should.be.true
const [_, delay] = setTimeoutSpy.getCall(0).args
delay?.should.equal(0)
result.should.deepEqual(["success after retry"])
})
it("should respect retry-after header with Unix timestamp", async () => {
const setTimeoutSpy = sinon.spy(global, "setTimeout")
let callCount = 0
const fixedDate = new Date("2010-01-01T00:00:00.000Z")
const retryTimestamp = Math.floor(fixedDate.getTime() / 1000) + 0.01 // 10ms in the future
const baseDelay = 1000
class TestClass {
@withRetry({ maxRetries: 2, baseDelay }) // Use large baseDelay to ensure header takes precedence
async *failMethod() {
callCount++
if (callCount === 1) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
error.headers = { "retry-after": retryTimestamp.toString() }
throw error
}
yield "success after retry"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(2)
setTimeoutSpy.calledOnce.should.be.true
const [_, delay] = setTimeoutSpy.getCall(0).args
delay?.should.equal(fixedDate.getTime())
result.should.deepEqual(["success after retry"])
})
it("should use exponential backoff when no retry-after header", async () => {
const setTimeoutSpy = sinon.spy(global, "setTimeout")
let callCount = 0
const baseDelay = 10
class TestClass {
@withRetry({ maxRetries: 2, baseDelay, maxDelay: 100 })
async *failMethod() {
callCount++
if (callCount === 1) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
throw error
}
yield "success after retry"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(2)
setTimeoutSpy.calledOnce.should.be.true
const [_, delay] = setTimeoutSpy.getCall(0).args
delay?.should.equal(baseDelay)
result.should.deepEqual(["success after retry"])
})
it("should respect maxDelay", async () => {
const setTimeoutSpy = sinon.spy(global, "setTimeout")
let callCount = 0
const baseDelay = 50
const maxDelay = 10
class TestClass {
@withRetry({ maxRetries: 3, baseDelay, maxDelay })
async *failMethod() {
callCount++
if (callCount < 3) {
const error: any = new Error("Rate limit exceeded")
error.status = 429
throw error
}
yield "success after retries"
}
}
const test = new TestClass()
const result = []
for await (const value of test.failMethod()) {
result.push(value)
}
callCount.should.equal(3)
setTimeoutSpy.calledOnce.should.be.true
const [_, delay] = setTimeoutSpy.getCall(0).args
delay?.should.equal(maxDelay)
result.should.deepEqual(["success after retries"])
})
it("should throw after maxRetries attempts", async () => {
let callCount = 0
class TestClass {
@withRetry({ maxRetries: 2, baseDelay: 10 })
async *failMethod() {
callCount++
const error: any = new Error("Rate limit exceeded")
error.status = 429
throw error
}
}
const test = new TestClass()
try {
for await (const _ of test.failMethod()) {
// Should not reach here
}
throw new Error("Should have thrown")
} catch (error: any) {
error.message.should.equal("Rate limit exceeded")
callCount.should.equal(2) // Initial attempt + 1 retry
}
})
})
})
-88
View File
@@ -1,88 +0,0 @@
import { Logger } from "@/shared/services/Logger"
interface RetryOptions {
maxRetries?: number
baseDelay?: number
maxDelay?: number
retryAllErrors?: boolean
}
const DEFAULT_OPTIONS: Required<RetryOptions> = {
maxRetries: 3,
baseDelay: 1_000,
maxDelay: 10_000,
retryAllErrors: false,
}
export class RetriableError extends Error {
status: number = 429
retryAfter?: number
constructor(message: string, retryAfter?: number, options?: ErrorOptions) {
super(message, options)
this.name = "RetriableError"
this.retryAfter = retryAfter
}
}
export function withRetry(options: RetryOptions = {}) {
const { maxRetries, baseDelay, maxDelay, retryAllErrors } = { ...DEFAULT_OPTIONS, ...options }
return (_target: any, _propertyKey: string, descriptor: PropertyDescriptor) => {
const originalMethod = descriptor.value
descriptor.value = async function* (...args: any[]) {
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
yield* originalMethod.apply(this, args)
return
} catch (error: any) {
const isRateLimit = error?.status === 429 || error instanceof RetriableError
const isLastAttempt = attempt === maxRetries - 1
if ((!isRateLimit && !retryAllErrors) || isLastAttempt) {
throw error
}
// Get retry delay from header or calculate exponential backoff
// Check various rate limit headers
const retryAfter =
error.headers?.["retry-after"] ||
error.headers?.["x-ratelimit-reset"] ||
error.headers?.["ratelimit-reset"] ||
error.retryAfter
let delay: number
if (retryAfter) {
// Handle both delta-seconds and Unix timestamp formats
const retryValue = parseInt(retryAfter, 10)
if (retryValue > Date.now() / 1000) {
// Unix timestamp
delay = retryValue * 1000 - Date.now()
} else {
// Delta seconds
delay = retryValue * 1000
}
} else {
// Use exponential backoff if no header
delay = Math.min(maxDelay, baseDelay * 2 ** attempt)
}
const handlerInstance = this as any
if (handlerInstance.options?.onRetryAttempt) {
try {
await handlerInstance.options.onRetryAttempt(attempt + 1, maxRetries, delay, error)
} catch (e) {
Logger.error("Error in onRetryAttempt callback:", e)
}
}
await new Promise((resolve) => setTimeout(resolve, delay))
}
}
}
return descriptor
}
}
@@ -1,117 +0,0 @@
import { describe, it } from "mocha"
import "should"
import type { ModelInfo } from "@shared/api"
import sinon from "sinon"
import { createOpenRouterStream } from "../openrouter-stream"
describe("createOpenRouterStream", () => {
const createAsyncIterable = () => ({
async *[Symbol.asyncIterator]() {},
})
const createClient = () => {
const create = sinon.stub().resolves(createAsyncIterable())
return {
client: {
chat: {
completions: {
create,
},
},
},
create,
}
}
const createModelInfo = (maxTokens: number): ModelInfo => ({
maxTokens,
contextWindow: 1_048_576,
supportsImages: true,
supportsPromptCache: false,
})
it("caps Gemini Flash OpenRouter requests to 8192 max_tokens", async () => {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: "google/gemini-2.5-flash",
info: createModelInfo(65_536),
})
const payload = create.firstCall.args[0] as Record<string, any>
payload.should.have.property("max_tokens", 8_192)
})
it("keeps lower Gemini Flash max_tokens values when already below 8192", async () => {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: "google/gemini-2.5-flash",
info: createModelInfo(4_096),
})
const payload = create.firstCall.args[0] as Record<string, any>
payload.should.have.property("max_tokens", 4_096)
})
it("does not send max_tokens for non-Gemini models", async () => {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: "anthropic/claude-sonnet-4.5",
info: createModelInfo(64_000),
})
const payload = create.firstCall.args[0] as any
payload.should.not.have.property("max_tokens")
})
it("does not send max_tokens for non-Flash Gemini models", async () => {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: "google/gemini-2.5-pro",
info: createModelInfo(65_536),
})
const payload = create.firstCall.args[0] as any
payload.should.not.have.property("max_tokens")
})
it("adds cache_control blocks for Qwen models that require explicit OpenRouter caching", async () => {
for (const modelId of ["qwen/qwen3.6-plus", "qwen/qwen3.7-max"]) {
const { client, create } = createClient()
await createOpenRouterStream(client as any, "system prompt", [{ role: "user", content: "hello" }] as any, {
id: modelId,
info: createModelInfo(65_536),
})
const payload = create.firstCall.args[0] as any
payload.messages[0].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
payload.messages[1].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
}
})
it("uses adaptive reasoning with verbosity for Claude Opus adaptive models", async () => {
const { client, create } = createClient()
await createOpenRouterStream(
client as any,
"system prompt",
[{ role: "user", content: "hello" }] as any,
{
id: "anthropic/claude-opus-4.6",
info: createModelInfo(64_000),
},
"xhigh",
)
const payload = create.firstCall.args[0] as any
payload.should.have.property("reasoning")
payload.reasoning.should.deepEqual({ enabled: true })
payload.should.have.property("verbosity", "xhigh")
should(payload.temperature).equal(undefined)
should(payload.top_p).equal(undefined)
})
})
@@ -1,283 +0,0 @@
/**
* Contract tests for thinking trace preservation across provider transforms.
*
* These tests verify that thinking/reasoning content is correctly preserved
* when converting messages between different API formats. This is critical
* because losing thinking traces can cause:
* - Degraded model performance (models need context of their reasoning)
* - Provider API errors (e.g., Gemini requires reasoning_details for tool calls)
* - Incorrect cost calculations
*/
import { describe, it } from "mocha"
import "should"
import { ClineAssistantThinkingBlock, ClineStorageMessage, ClineTextContentBlock } from "@/shared/messages/content"
import { sanitizeAnthropicMessages } from "../anthropic-format"
import { convertToOpenAiMessages, sanitizeGeminiMessages } from "../openai-format"
describe("Thinking Trace Preservation", () => {
describe("convertToOpenAiMessages", () => {
it("should preserve reasoning_details on text blocks", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "text",
text: "I'll help you with that.",
reasoning_details: [
{
type: "reasoning.text",
text: "The user wants help with X...",
signature: "sig123",
format: "anthropic-claude-v1",
index: 0,
},
],
} as ClineTextContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(1)
const assistantMsg = result[0] as any
assistantMsg.role.should.equal("assistant")
assistantMsg.reasoning_details.should.be.an.Array()
assistantMsg.reasoning_details.should.have.length(1)
assistantMsg.reasoning_details[0].text.should.equal("The user wants help with X...")
})
it("should preserve thinking blocks with signatures", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "thinking",
thinking: "Let me analyze this problem...",
signature: "valid-signature",
} as ClineAssistantThinkingBlock,
{
type: "text",
text: "Here's my answer.",
} as ClineTextContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(1)
const assistantMsg = result[0] as any
// The thinking block content should be preserved in some form
// (exact handling depends on implementation)
assistantMsg.content.should.containEql("Here's my answer.")
})
it("should consolidate multiple reasoning_details entries", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "text",
text: "Result",
reasoning_details: [
{
type: "reasoning.text",
text: "First thought. ",
signature: "sig1",
format: "anthropic-claude-v1",
index: 0,
},
{
type: "reasoning.text",
text: "Second thought.",
signature: "sig2",
format: "anthropic-claude-v1",
index: 0,
},
],
} as ClineTextContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
const assistantMsg = result[0] as any
assistantMsg.reasoning_details.should.be.an.Array()
// Should be consolidated into one entry per index
assistantMsg.reasoning_details.should.have.length(1)
assistantMsg.reasoning_details[0].text.should.equal("First thought. Second thought.")
})
it("should filter out corrupted encrypted reasoning blocks", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "text",
text: "Answer",
reasoning_details: [
{
type: "reasoning.encrypted",
// Missing 'data' field - corrupted
signature: "sig",
format: "anthropic-claude-v1",
index: 0,
} as any,
{
type: "reasoning.text",
text: "Valid reasoning",
signature: "sig2",
format: "anthropic-claude-v1",
index: 1,
},
],
} as ClineTextContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
const assistantMsg = result[0] as any
// Should only have the valid reasoning entry
assistantMsg.reasoning_details.should.have.length(1)
assistantMsg.reasoning_details[0].type.should.equal("reasoning.text")
})
})
describe("sanitizeGeminiMessages", () => {
it("should drop tool_calls without reasoning_details for Gemini models", () => {
const messages: any[] = [
{
role: "assistant",
content: "I'll use a tool",
tool_calls: [{ id: "call_123", type: "function", function: { name: "read_file", arguments: "{}" } }],
// No reasoning_details
},
{
role: "tool",
tool_call_id: "call_123",
content: "file contents",
},
]
const result = sanitizeGeminiMessages(messages, "gemini-2.5-pro")
// Tool call should be dropped, but content preserved
result.should.have.length(1)
const msg = result[0] as any
msg.role.should.equal("assistant")
msg.content.should.equal("I'll use a tool")
;(msg.tool_calls === undefined).should.be.true()
})
it("should preserve tool_calls with reasoning_details for Gemini models", () => {
const messages: any[] = [
{
role: "assistant",
content: null,
tool_calls: [{ id: "call_123", type: "function", function: { name: "read_file", arguments: "{}" } }],
reasoning_details: [
{
type: "reasoning.text",
text: "I need to read the file",
format: "anthropic-claude-v1",
index: 0,
},
],
},
{
role: "tool",
tool_call_id: "call_123",
content: "file contents",
},
]
const result = sanitizeGeminiMessages(messages, "gemini-2.5-pro")
result.should.have.length(2)
;(result[0] as any).tool_calls.should.have.length(1)
})
it("should not modify messages for non-Gemini models", () => {
const messages: any[] = [
{
role: "assistant",
content: "Using tool",
tool_calls: [{ id: "call_123", type: "function", function: { name: "test", arguments: "{}" } }],
// No reasoning_details - would be dropped for Gemini
},
]
const result = sanitizeGeminiMessages(messages, "gpt-4o")
result.should.have.length(1)
;(result[0] as any).tool_calls.should.have.length(1)
})
})
describe("sanitizeAnthropicMessages", () => {
it("should preserve thinking blocks", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "thinking",
thinking: "Let me think about this...",
signature: "valid-sig",
} as ClineAssistantThinkingBlock,
{
type: "text",
text: "Here's my answer",
} as ClineTextContentBlock,
],
},
]
const result = sanitizeAnthropicMessages(messages, false)
result.should.have.length(1)
const content = result[0].content as any[]
// Find thinking block
const thinkingBlock = content.find((b) => b.type === "thinking")
thinkingBlock.should.not.be.undefined
thinkingBlock.thinking.should.equal("Let me think about this...")
})
it("should not add cache_control to thinking blocks", () => {
const messages: ClineStorageMessage[] = [
{
role: "user",
content: [
{
type: "thinking",
thinking: "Thinking...",
signature: "sig",
} as any,
{
type: "text",
text: "Question",
} as ClineTextContentBlock,
],
},
]
const result = sanitizeAnthropicMessages(messages, true)
result.should.have.length(1)
const content = result[0].content as any[]
// The text block (last non-thinking) should have cache_control
const textBlock = content.find((b) => b.type === "text")
textBlock.cache_control.should.deepEqual({ type: "ephemeral" })
// Thinking block should not have cache_control (it doesn't support it)
})
})
})
@@ -1,116 +0,0 @@
import "should"
import { getOpenAIToolParams, ToolCallProcessor } from "../tool-call-processor"
describe("ToolCallProcessor", () => {
it("should preserve tool call id/name for interleaved parallel deltas", () => {
const processor = new ToolCallProcessor()
const firstChunk = [
{
index: 0,
id: "call_a",
function: { name: "read_file" },
},
{
index: 1,
id: "call_b",
function: { name: "search_files" },
},
] as any
const secondChunk = [
{
index: 1,
function: { arguments: '{"path":"src"}' },
},
{
index: 0,
function: { arguments: '{"path":"README.md"}' },
},
] as any
const firstResult = [...processor.processToolCallDeltas(firstChunk)]
const secondResult = [...processor.processToolCallDeltas(secondChunk)]
firstResult.should.have.length(0)
secondResult.should.have.length(2)
// Intentionally reversed from the setup chunk: output follows incoming
// argument-delta order, but reconstruction is correct regardless of arrival
// order because id/name/arguments are matched by tool call index.
const firstToolCall = secondResult[0]!.tool_call as any
const secondToolCall = secondResult[1]!.tool_call as any
firstToolCall.function.id.should.equal("call_b")
firstToolCall.function.name.should.equal("search_files")
firstToolCall.function.arguments.should.equal('{"path":"src"}')
secondToolCall.function.id.should.equal("call_a")
secondToolCall.function.name.should.equal("read_file")
secondToolCall.function.arguments.should.equal('{"path":"README.md"}')
})
it("should clear accumulated state on reset", () => {
const processor = new ToolCallProcessor()
const setupChunk = [
{
index: 0,
id: "call_reset",
function: { name: "read_file" },
},
] as any
const argsChunk = [
{
index: 0,
function: { arguments: '{"path":"after-reset"}' },
},
] as any
;[...processor.processToolCallDeltas(setupChunk)].should.have.length(0)
processor.reset()
;[...processor.processToolCallDeltas(argsChunk)].should.have.length(0)
const newSetupChunk = [
{
index: 0,
id: "call_new",
function: { name: "write_file" },
},
] as any
const newArgsChunk = [
{
index: 0,
function: { arguments: '{"path":"file.txt"}' },
},
] as any
;[...processor.processToolCallDeltas(newSetupChunk)].should.have.length(0)
;[...processor.processToolCallDeltas(newArgsChunk)].should.have.length(1)
})
})
describe("getOpenAIToolParams", () => {
it("should include parallel_tool_calls when enabled", () => {
const tools = [
{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } },
] as any
const params = getOpenAIToolParams(tools, true) as any
params.parallel_tool_calls.should.equal(true)
})
it("should include parallel_tool_calls=false when disabled by default", () => {
const tools = [
{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } },
] as any
const params = getOpenAIToolParams(tools, false) as any
params.parallel_tool_calls.should.equal(false)
})
it("should not include parallel_tool_calls when tools are absent", () => {
const params = getOpenAIToolParams(undefined, false) as any
params.should.not.have.property("parallel_tool_calls")
})
})
@@ -1,371 +0,0 @@
/**
* Contract tests for tool call parsing and transformation.
*
* These tests verify that tool calls are correctly parsed and transformed
* between different API formats (Anthropic, OpenAI, etc.). This is critical
* because incorrect tool parsing can cause:
* - Tool calls not being executed
* - Mismatched tool_call_id causing API errors
* - Lost tool results breaking conversation flow
*/
import { describe, it } from "mocha"
import "should"
import OpenAI from "openai"
import {
ClineAssistantToolUseBlock,
ClineStorageMessage,
ClineTextContentBlock,
ClineUserToolResultContentBlock,
} from "@/shared/messages/content"
import { convertToAnthropicMessage, convertToOpenAiMessages } from "../openai-format"
describe("Tool Call Parsing", () => {
describe("convertToOpenAiMessages - Tool Calls", () => {
it("should convert Anthropic tool_use to OpenAI tool_calls format", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: "toolu_abc123",
name: "read_file",
input: { path: "/test/file.ts" },
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages, "openai-native")
result.should.have.length(1)
const msg = result[0] as any
msg.role.should.equal("assistant")
msg.tool_calls.should.have.length(1)
msg.tool_calls[0].type.should.equal("function")
msg.tool_calls[0].function.name.should.equal("read_file")
JSON.parse(msg.tool_calls[0].function.arguments).should.deepEqual({ path: "/test/file.ts" })
})
it("should truncate long tool IDs to 40 characters", () => {
const longId = "a".repeat(50)
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: longId,
name: "test_tool",
input: {},
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages, "openai-native")
const msg = result[0] as any
msg.tool_calls[0].id.length.should.be.belowOrEqual(40)
})
it("should transform OpenAI Responses API tool IDs (fc_ prefix)", () => {
// OpenAI Responses API uses fc_ prefix with 53 char length
const responsesApiId = "fc_" + "x".repeat(50)
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: responsesApiId,
name: "test_tool",
input: {},
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
const msg = result[0] as any
// Should be transformed to call_ prefix format
msg.tool_calls[0].id.should.startWith("call_")
msg.tool_calls[0].id.length.should.be.belowOrEqual(40)
})
it("should match tool_call_id with tool_calls id for tool results", () => {
const toolId = "toolu_abc123"
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: toolId,
name: "read_file",
input: { path: "/test.ts" },
} as ClineAssistantToolUseBlock,
],
},
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: toolId,
content: "file contents here",
} as ClineUserToolResultContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(2)
// Get the transformed tool_call id from assistant message
const assistantMsg = result[0] as OpenAI.Chat.ChatCompletionAssistantMessageParam
const transformedId = assistantMsg.tool_calls![0].id
// The tool result should have the same transformed id
const toolMsg = result[1] as OpenAI.Chat.ChatCompletionToolMessageParam
toolMsg.tool_call_id.should.equal(transformedId)
})
it("should handle multiple tool calls in a single message", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "text",
text: "I'll read both files",
} as ClineTextContentBlock,
{
type: "tool_use",
id: "tool_1",
name: "read_file",
input: { path: "/file1.ts" },
} as ClineAssistantToolUseBlock,
{
type: "tool_use",
id: "tool_2",
name: "read_file",
input: { path: "/file2.ts" },
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(1)
const msg = result[0] as any
msg.tool_calls.should.have.length(2)
msg.tool_calls[0].function.name.should.equal("read_file")
msg.tool_calls[1].function.name.should.equal("read_file")
})
it("should handle tool results with array content", () => {
const messages: ClineStorageMessage[] = [
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: "tool_123",
content: [
{ type: "text", text: "Line 1" },
{ type: "text", text: "Line 2" },
],
} as ClineUserToolResultContentBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
result.should.have.length(1)
const msg = result[0] as OpenAI.Chat.ChatCompletionToolMessageParam
msg.role.should.equal("tool")
msg.content.should.equal("Line 1\nLine 2")
})
it("should set content to null when only tool_calls present", () => {
const messages: ClineStorageMessage[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: "tool_1",
name: "test",
input: {},
} as ClineAssistantToolUseBlock,
],
},
]
const result = convertToOpenAiMessages(messages)
const msg = result[0] as any
// Content should be null, not undefined or empty string
;(msg.content === null).should.be.true()
})
})
describe("convertToAnthropicMessage - OpenAI Response to Anthropic", () => {
it("should convert OpenAI completion to Anthropic message format", () => {
const completion: OpenAI.Chat.Completions.ChatCompletion = {
id: "chatcmpl-123",
object: "chat.completion",
created: Date.now(),
model: "gpt-4o",
choices: [
{
index: 0,
message: {
role: "assistant",
content: "Hello!",
refusal: null,
},
finish_reason: "stop",
logprobs: null,
},
],
usage: {
prompt_tokens: 10,
completion_tokens: 5,
total_tokens: 15,
},
}
const result = convertToAnthropicMessage(completion)
result.id.should.equal("chatcmpl-123")
result.role.should.equal("assistant")
result.model.should.equal("gpt-4o")
result.stop_reason!.should.equal("end_turn")
result.usage.input_tokens.should.equal(10)
result.usage.output_tokens.should.equal(5)
const content = result.content as any[]
content[0].type.should.equal("text")
content[0].text.should.equal("Hello!")
})
it("should convert OpenAI tool_calls to Anthropic tool_use blocks", () => {
const completion: OpenAI.Chat.Completions.ChatCompletion = {
id: "chatcmpl-456",
object: "chat.completion",
created: Date.now(),
model: "gpt-4o",
choices: [
{
index: 0,
message: {
role: "assistant",
content: null,
tool_calls: [
{
id: "call_abc",
type: "function",
function: {
name: "read_file",
arguments: '{"path":"/test.ts"}',
},
},
],
refusal: null,
},
finish_reason: "tool_calls",
logprobs: null,
},
],
}
const result = convertToAnthropicMessage(completion)
result.stop_reason!.should.equal("tool_use")
const content = result.content as any[]
content.should.have.length(2) // text block + tool_use block
const toolUse = content.find((b) => b.type === "tool_use")
toolUse.should.not.be.undefined
toolUse.id.should.equal("call_abc")
toolUse.name.should.equal("read_file")
toolUse.input.should.deepEqual({ path: "/test.ts" })
})
it("should handle malformed tool arguments gracefully", () => {
const completion: OpenAI.Chat.Completions.ChatCompletion = {
id: "chatcmpl-789",
object: "chat.completion",
created: Date.now(),
model: "gpt-4o",
choices: [
{
index: 0,
message: {
role: "assistant",
content: null,
tool_calls: [
{
id: "call_bad",
type: "function",
function: {
name: "test_tool",
arguments: "not valid json",
},
},
],
refusal: null,
},
finish_reason: "tool_calls",
logprobs: null,
},
],
}
// Should not throw, should return empty input
const result = convertToAnthropicMessage(completion)
const content = result.content as any[]
const toolUse = content.find((b) => b.type === "tool_use")
toolUse.input.should.deepEqual({})
})
it("should map finish_reason correctly", () => {
const testCases: Array<{ finish_reason: any; expected: string | null }> = [
{ finish_reason: "stop", expected: "end_turn" },
{ finish_reason: "length", expected: "max_tokens" },
{ finish_reason: "tool_calls", expected: "tool_use" },
{ finish_reason: "content_filter", expected: null },
]
for (const { finish_reason, expected } of testCases) {
const completion: OpenAI.Chat.Completions.ChatCompletion = {
id: "test",
object: "chat.completion",
created: Date.now(),
model: "test",
choices: [
{
index: 0,
message: { role: "assistant", content: "test", refusal: null },
finish_reason,
logprobs: null,
},
],
}
const result = convertToAnthropicMessage(completion)
// Using equality check since should.be.true() doesn't accept message arg
;(result.stop_reason === expected).should.be.true()
}
})
})
})
@@ -1,90 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { ClineStorageMessage, convertClineStorageToAnthropicMessage } from "@/shared/messages/content"
/**
* Converts Cline storage messages to Anthropic API format with optional cache control.
* Adds ephemeral cache control to the last two user messages to prevent them from being
* stored in Anthropic's cache.
*
* @param clineMessages - Array of Cline storage messages to convert
* @param lastUserMsgIndex - Optional index of the last user message
* @param secondLastMsgUserIndex - Optional index of the second-to-last user message
* @returns Array of Anthropic-compatible messages with cache control applied
*/
export function sanitizeAnthropicMessages(
clineMessages: Array<ClineStorageMessage | Anthropic.MessageParam>,
supportCache: boolean,
): Array<Anthropic.MessageParam> {
// The latest message will be the new user message, one before will be the assistant message from a previous request,
// and the user message before that will be a previously cached user message. So we need to mark the latest user message
// as ephemeral to cache it for the next request, and mark the second to last user message as ephemeral to let the server
// know the last message to retrieve from the cache for the current request.
const userMsgIndices = clineMessages.reduce((acc, msg, index) => {
if (msg.role === "user") {
acc.push(index)
}
return acc
}, [] as number[])
// Set to -1 if there are no user messages so the indices are invalid
const indicesLength = userMsgIndices.length ?? -1
const lastUserMsgIndex = userMsgIndices[indicesLength - 1]
const secondLastMsgUserIndex = userMsgIndices[indicesLength - 2]
return clineMessages.map((msg, index) => {
const anthropicMsg = convertClineStorageToAnthropicMessage(msg)
// Add cache control to the last two user messages
if (supportCache && (index === lastUserMsgIndex || index === secondLastMsgUserIndex)) {
return addCacheControl(anthropicMsg)
}
return anthropicMsg
})
}
const isThinkingBlock = (
block: Anthropic.ContentBlockParam,
): block is Anthropic.Messages.ThinkingBlockParam | Anthropic.Messages.RedactedThinkingBlockParam => {
return block.type === "thinking" || block.type === "redacted_thinking"
}
/**
* Adds ephemeral cache control to the last content block of a message.
* Returns a new message object without mutating the original.
*
* @param message - The Anthropic message to add cache control to
* @returns A new message with cache control added to the last content block
*/
function addCacheControl(message: Anthropic.MessageParam): Anthropic.MessageParam {
// Convert string content to array format
if (typeof message.content === "string") {
return {
...message,
content: [
{
type: "text",
text: message.content,
cache_control: { type: "ephemeral" },
} satisfies Anthropic.TextBlockParam,
],
}
}
// Handle array content - add cache control to the last block
const content = [...message.content]
const lastIndex = content.length - 1
if (lastIndex >= 0) {
const lastBlock = content[lastIndex]
// Only add cache_control to block types that support it (not ThinkingBlockParam)
if (!isThinkingBlock(lastBlock)) {
content[lastIndex] = {
...lastBlock,
cache_control: { type: "ephemeral" },
} satisfies Anthropic.ContentBlockParam
}
}
return { ...message, content }
}
@@ -1,118 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { Content, GenerateContentResponse, Part } from "@google/genai"
import { ClineStorageMessage } from "@/shared/messages/content"
// Source: https://ai.google.dev/gemini-api/docs/thought-signatures#faqs
// While injecting custom function call blocks into the request is strongly discouraged,
// in cases where it can't be avoided, e.g. providing information to the model on function
// calls and responses that were executed deterministically by the client, or transferring a
// trace from a different model that does not include thought signatures, you can set the following dummy signatures of either
// "context_engineering_is_the_way_to_go" or "skip_thought_signature_validator" in the thought signature field to skip validation.
const GEMINI_DUMMY_THOUGHT_SIGNATURE = "skip_thought_signature_validator"
export function convertAnthropicContentToGemini(content: string | ClineStorageMessage["content"]): Part[] {
if (typeof content === "string") {
return [{ text: content }]
}
return content
.flatMap((block): Part | undefined => {
switch (block.type) {
case "text":
return { text: block.text, thoughtSignature: block.signature }
case "image":
if (block.source.type !== "base64") {
throw new Error("Unsupported image source type")
}
return {
inlineData: {
data: block.source.data,
mimeType: block.source.media_type,
},
}
case "tool_use":
return {
functionCall: {
name: block.name,
args: block.input as Record<string, unknown>,
},
// Thought signature is required, so provide a dummy one if not present
thoughtSignature: block.signature || GEMINI_DUMMY_THOUGHT_SIGNATURE,
}
case "tool_result":
return {
functionResponse: {
name: block.tool_use_id,
response: {
result: block.content,
},
},
}
case "thinking":
return {
text: block.thinking,
thought: true,
thoughtSignature: block.signature || GEMINI_DUMMY_THOUGHT_SIGNATURE,
}
default:
return undefined
}
})
.filter((part): part is Part => part !== undefined) // Filter out unsupported blocks
}
export function convertAnthropicMessageToGemini(message: Anthropic.Messages.MessageParam): Content {
return {
role: message.role === "assistant" ? "model" : "user",
parts: convertAnthropicContentToGemini(message.content),
}
}
/*
It looks like gemini likes to double escape certain characters when writing file contents: https://discuss.ai.google.dev/t/function-call-string-property-is-double-escaped/37867
*/
export function unescapeGeminiContent(content: string) {
return content.replace(/\\n/g, "\n").replace(/\\'/g, "'").replace(/\\"/g, '"').replace(/\\r/g, "\r").replace(/\\t/g, "\t")
}
export function convertGeminiResponseToAnthropic(response: GenerateContentResponse): Anthropic.Messages.Message {
const content: Anthropic.Messages.ContentBlock[] = []
const text = response.text
if (text) {
content.push({ type: "text", text, citations: null })
}
let stop_reason: Anthropic.Messages.Message["stop_reason"] = null
const finishReason = response.candidates?.[0]?.finishReason
if (finishReason) {
switch (finishReason) {
case "STOP":
stop_reason = "end_turn"
break
case "MAX_TOKENS":
stop_reason = "max_tokens"
break
case "SAFETY":
case "RECITATION":
case "OTHER":
stop_reason = "stop_sequence"
break
}
}
return {
id: `msg_${Date.now()}`,
type: "message",
role: "assistant",
content,
model: "",
stop_reason,
stop_sequence: null, // Gemini doesn't provide this information
usage: {
input_tokens: response.usageMetadata?.promptTokenCount ?? 0,
output_tokens: response.usageMetadata?.candidatesTokenCount ?? 0,
cache_creation_input_tokens: null,
cache_read_input_tokens: null,
},
}
}
@@ -1,61 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { AssistantMessage } from "@mistralai/mistralai/models/components/assistantmessage"
import { SystemMessage } from "@mistralai/mistralai/models/components/systemmessage"
import { ToolMessage } from "@mistralai/mistralai/models/components/toolmessage"
import { UserMessage } from "@mistralai/mistralai/models/components/usermessage"
export type MistralMessage =
| (SystemMessage & { role: "system" })
| (UserMessage & { role: "user" })
| (AssistantMessage & { role: "assistant" })
| (ToolMessage & { role: "tool" })
export function convertToMistralMessages(anthropicMessages: Anthropic.Messages.MessageParam[]): MistralMessage[] {
const mistralMessages: MistralMessage[] = []
for (const anthropicMessage of anthropicMessages) {
if (typeof anthropicMessage.content === "string") {
mistralMessages.push({
role: anthropicMessage.role,
content: anthropicMessage.content,
})
} else {
if (anthropicMessage.role === "user") {
// Filter to only include text and image blocks
const textAndImageBlocks = anthropicMessage.content.filter(
(part) => part.type === "text" || part.type === "image",
)
if (textAndImageBlocks.length > 0) {
mistralMessages.push({
role: "user",
content: textAndImageBlocks.map((part) => {
if (part.type === "image") {
return {
type: "image_url",
imageUrl: {
url: `data:${part.source.media_type};base64,${part.source.data}`,
},
}
}
return { type: "text", text: part.text }
}),
})
}
} else if (anthropicMessage.role === "assistant") {
// Only process text blocks - assistant cannot send images or other content types in Mistral's API format
const textBlocks = anthropicMessage.content.filter((part) => part.type === "text")
if (textBlocks.length > 0) {
const content = textBlocks.map((part) => part.text).join("\n")
mistralMessages.push({
role: "assistant",
content,
})
}
}
}
}
return mistralMessages
}
@@ -1,436 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import OpenAI from "openai"
import { Logger } from "@/shared/services/Logger"
const o1SystemPrompt = (systemPrompt: string) => `
# System Prompt
${systemPrompt}
# Instructions for Formulating Your Response
You must respond to the user's request by using at least one tool call. When formulating your response, follow these guidelines:
1. Begin your response with normal text, explaining your thoughts, analysis, or plan of action.
2. If you need to use any tools, place ALL tool calls at the END of your message, after your normal text explanation.
3. You can use multiple tool calls if needed, but they should all be grouped together at the end of your message.
4. After placing the tool calls, do not add any additional normal text. The tool calls should be the final content in your message.
Here's the general structure your responses should follow:
\`\`\`
[Your normal text response explaining your thoughts and actions]
[Tool Call 1]
[Tool Call 2 if needed]
[Tool Call 3 if needed]
...
\`\`\`
Remember:
- Choose the most appropriate tool(s) based on the task and the tool descriptions provided.
- Formulate your tool calls using the XML format specified for each tool.
- Provide clear explanations in your normal text about what actions you're taking and why you're using particular tools.
- Act as if the tool calls will be executed immediately after your message, and your next response will have access to their results.
# Tool Descriptions and XML Formats
1. execute_command:
<execute_command>
<command>Your command here</command>
</execute_command>
Description: Execute a CLI command on the system. Use this when you need to perform system operations or run specific commands to accomplish any step in the user's task. You must tailor your command to the user's system and provide a clear explanation of what the command does. Prefer to execute complex CLI commands over creating executable scripts, as they are more flexible and easier to run. Commands will be executed in the current working directory.
2. list_files:
<list_files>
<path>Directory path here</path>
<recursive>true or false (optional)</recursive>
</list_files>
Description: List files and directories within the specified directory. If recursive is true, it will list all files and directories recursively. If recursive is false or not provided, it will only list the top-level contents.
3. list_code_definition_names:
<list_code_definition_names>
<path>Directory path here</path>
</list_code_definition_names>
Description: Lists definition names (classes, functions, methods, etc.) used in source code files at the top level of the specified directory. This tool provides insights into the codebase structure and important constructs, encapsulating high-level concepts and relationships that are crucial for understanding the overall architecture.
4. search_files:
<search_files>
<path>Directory path here</path>
<regex>Your regex pattern here</regex>
<filePattern>Optional file pattern here</filePattern>
</search_files>
Description: Perform a regex search across files in a specified directory, providing context-rich results. This tool searches for patterns or specific content across multiple files, displaying each match with encapsulating context.
5. read_file:
<read_file>
<path>File path here</path>
</read_file>
Description: Read the contents of a file at the specified path. Use this when you need to examine the contents of an existing file, for example to analyze code, review text files, or extract information from configuration files. Automatically extracts raw text from PDF and DOCX files. May not be suitable for other types of binary files, as it returns the raw content as a string.
6. write_to_file:
<write_to_file>
<path>File path here</path>
<content>
Your file content here
</content>
</write_to_file>
Description: Write content to a file at the specified path. If the file exists, it will be overwritten with the provided content. If the file doesn't exist, it will be created. Always provide the full intended content of the file, without any truncation. This tool will automatically create any directories needed to write the file.
7. ask_followup_question:
<ask_followup_question>
<question>Your question here</question>
</ask_followup_question>
Description: Ask the user a question to gather additional information needed to complete the task. This tool should be used when you encounter ambiguities, need clarification, or require more details to proceed effectively. It allows for interactive problem-solving by enabling direct communication with the user. Use this tool judiciously to maintain a balance between gathering necessary information and avoiding excessive back-and-forth.
8. attempt_completion:
<attempt_completion>
<command>Optional command to demonstrate result</command>
<result>
Your final result description here
</result>
</attempt_completion>
Description: Once you've completed the task, use this tool to present the result to the user. They may respond with feedback if they are not satisfied with the result, which you can use to make improvements and try again.
# Examples
Here are some examples of how to structure your responses with tool calls:
Example 1: Using a single tool
Let's run the test suite for our project. This will help us ensure that all our components are functioning correctly.
<execute_command>
<command>npm test</command>
</execute_command>
Example 2: Using multiple tools
Let's create two new configuration files for the web application: one for the frontend and one for the backend.
<write_to_file>
<path>./frontend-config.json</path>
<content>
{
"apiEndpoint": "https://api.example.com",
"theme": {
"primaryColor": "#007bff",
"secondaryColor": "#6c757d",
"fontFamily": "Arial, sans-serif"
},
"features": {
"darkMode": true,
"notifications": true,
"analytics": false
},
"version": "1.0.0"
}
</content>
</write_to_file>
<write_to_file>
<path>./backend-config.yaml</path>
<content>
database:
host: localhost
port: 5432
name: myapp_db
user: admin
server:
port: 3000
environment: development
logLevel: debug
security:
jwtSecret: your-secret-key-here
passwordSaltRounds: 10
caching:
enabled: true
provider: redis
ttl: 3600
externalServices:
emailProvider: sendgrid
storageProvider: aws-s3
</content>
</write_to_file>
Example 3: Asking a follow-up question
I've analyzed the project structure, but I need more information to proceed. Let me ask the user for clarification.
<ask_followup_question>
<question>Which specific feature would you like me to implement in the example.py file?</question>
</ask_followup_question>
`
export function convertToO1Messages(
openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[],
systemPrompt: string,
): OpenAI.Chat.ChatCompletionMessageParam[] {
const toolsReplaced = openAiMessages.reduce((acc, message) => {
if (message.role === "tool") {
// Convert tool messages to user messages
acc.push({
role: "user",
content: message.content || "",
})
} else if (message.role === "assistant" && message.tool_calls) {
// Convert tool calls to content and remove tool_calls
let content = message.content || ""
message.tool_calls.forEach((toolCall) => {
if (toolCall.type === "function") {
content += `\nTool Call: ${toolCall.function.name}\nArguments: ${toolCall.function.arguments}`
}
})
acc.push({
role: "assistant",
content: content,
tool_calls: undefined,
})
} else {
// Keep other messages as they are
acc.push(message)
}
return acc
}, [] as OpenAI.Chat.ChatCompletionMessageParam[])
// Find the index of the last assistant message
// const lastAssistantIndex = findLastIndex(toolsReplaced, (message) => message.role === "assistant")
// Create a new array to hold the modified messages
const messagesWithSystemPrompt = [
{
role: "user",
content: o1SystemPrompt(systemPrompt),
} as OpenAI.Chat.ChatCompletionUserMessageParam,
...toolsReplaced,
]
// If there's an assistant message, insert the system prompt after it
// if (lastAssistantIndex !== -1) {
// const insertIndex = lastAssistantIndex + 1
// if (insertIndex < messagesWithSystemPrompt.length && messagesWithSystemPrompt[insertIndex].role === "user") {
// messagesWithSystemPrompt.splice(insertIndex, 0, {
// role: "user",
// content: o1SystemPrompt(systemPrompt),
// })
// }
// } else {
// // If there were no assistant messages, prepend the system prompt
// messagesWithSystemPrompt.unshift({
// role: "user",
// content: o1SystemPrompt(systemPrompt),
// })
// }
return messagesWithSystemPrompt
}
interface ToolCall {
tool: string
tool_input: Record<string, string>
}
const toolNames = [
"execute_command",
"list_files",
"list_code_definition_names",
"search_files",
"read_file",
"write_to_file",
"ask_followup_question",
"attempt_completion",
]
function parseAIResponse(response: string): {
normalText: string
toolCalls: ToolCall[]
} {
// Create a regex pattern to match any tool call opening tag
const toolCallPattern = new RegExp(`<(${toolNames.join("|")})`, "i")
const match = response.match(toolCallPattern)
if (!match) {
// No tool calls found
return { normalText: response.trim(), toolCalls: [] }
}
const toolCallStart = match.index!
const normalText = response.slice(0, toolCallStart).trim()
const toolCallsText = response.slice(toolCallStart)
const toolCalls = parseToolCalls(toolCallsText)
return { normalText, toolCalls }
}
function parseToolCalls(toolCallsText: string): ToolCall[] {
const toolCalls: ToolCall[] = []
let remainingText = toolCallsText
while (remainingText.length > 0) {
const toolMatch = toolNames.find((tool) => new RegExp(`<${tool}`, "i").test(remainingText))
if (!toolMatch) {
break // No more tool calls found
}
const startTag = `<${toolMatch}`
const endTag = `</${toolMatch}>`
const startIndex = remainingText.indexOf(startTag)
const endIndex = remainingText.indexOf(endTag, startIndex)
if (endIndex === -1) {
break // Malformed XML, no closing tag found
}
const toolCallContent = remainingText.slice(startIndex, endIndex + endTag.length)
remainingText = remainingText.slice(endIndex + endTag.length).trim()
const toolCall = parseToolCall(toolMatch, toolCallContent)
if (toolCall) {
toolCalls.push(toolCall)
}
}
return toolCalls
}
function parseToolCall(toolName: string, content: string): ToolCall | null {
const tool_input: Record<string, string> = {}
// Remove the outer tool tags
const innerContent = content.replace(new RegExp(`^<${toolName}>|</${toolName}>$`, "g"), "").trim()
// Parse nested XML elements
const paramRegex = /<(\w+)>([\s\S]*?)<\/\1>/gs
let match: RegExpExecArray | null
while ((match = paramRegex.exec(innerContent)) !== null) {
const [, paramName, paramValue] = match
// Preserve newlines and trim only leading/trailing whitespace
tool_input[paramName] = paramValue.replace(/^\s+|\s+$/g, "")
}
// Validate required parameters
if (!validateToolInput(toolName, tool_input)) {
Logger.error(`Invalid tool call for ${toolName}:`, content)
return null
}
return { tool: toolName, tool_input }
}
function validateToolInput(toolName: string, tool_input: Record<string, string>): boolean {
switch (toolName) {
case "execute_command":
return "command" in tool_input
case "read_file":
case "list_code_definition_names":
case "list_files":
return "path" in tool_input
case "search_files":
return "path" in tool_input && "regex" in tool_input
case "write_to_file":
return "path" in tool_input && "content" in tool_input
case "ask_followup_question":
return "question" in tool_input
case "attempt_completion":
return "result" in tool_input
default:
return false
}
}
// Example usage:
// const aiResponse = `Here's my analysis of the situation...
// <execute_command>
// <command>ls -la</command>
// </execute_command>
// <write_to_file>
// <path>./example.txt</path>
// <content>Hello, World!</content>
// </write_to_file>`;
//
// const { normalText, toolCalls } = parseAIResponse(aiResponse);
// Logger.log(normalText);
// Logger.log(toolCalls);
// Convert OpenAI response to Anthropic format
export function convertO1ResponseToAnthropicMessage(
completion: OpenAI.Chat.Completions.ChatCompletion,
): Anthropic.Messages.Message {
const openAiMessage = completion.choices[0].message
const { normalText, toolCalls } = parseAIResponse(openAiMessage.content || "")
const anthropicMessage: Anthropic.Messages.Message = {
id: completion.id,
type: "message",
role: openAiMessage.role, // always "assistant"
content: [
{
type: "text",
text: normalText,
citations: null,
},
],
model: completion.model,
stop_reason: (() => {
switch (completion.choices[0].finish_reason) {
case "stop":
return "end_turn"
case "length":
return "max_tokens"
case "tool_calls":
return "tool_use"
case "content_filter": // Anthropic doesn't have an exact equivalent
default:
return null
}
})(),
stop_sequence: null, // which custom stop_sequence was generated, if any (not applicable if you don't use stop_sequence)
usage: {
input_tokens: completion.usage?.prompt_tokens || 0,
output_tokens: completion.usage?.completion_tokens || 0,
cache_creation_input_tokens: null,
cache_read_input_tokens: null,
},
}
if (toolCalls.length > 0) {
anthropicMessage.content.push(
...toolCalls.map((toolCall: ToolCall, index: number): Anthropic.ToolUseBlock => {
return {
type: "tool_use",
id: `call_${index}_${Date.now()}`, // Generate a unique ID for each tool call
name: toolCall.tool,
input: toolCall.tool_input,
}
}),
)
}
return anthropicMessage
}
// Example usage:
// const openAICompletion = {
// id: "cmpl-123",
// choices: [{
// message: {
// role: "assistant",
// content: "Here's my analysis...\n\n<execute_command>\n <command>ls -la</command>\n</execute_command>"
// },
// finish_reason: "stop"
// }],
// model: "gpt-3.5-turbo",
// usage: { prompt_tokens: 50, completion_tokens: 100 }
// };
// const anthropicMessage = convertO1ResponseToAnthropicMessage(openAICompletion);
// Logger.log(anthropicMessage);
@@ -1,115 +0,0 @@
import { Message } from "ollama"
import {
ClineAssistantToolUseBlock,
ClineImageContentBlock,
ClineStorageMessage,
ClineTextContentBlock,
ClineUserToolResultContentBlock,
} from "@/shared/messages/content"
export function convertToOllamaMessages(anthropicMessages: Omit<ClineStorageMessage, "modelInfo">[]): Message[] {
const ollamaMessages: Message[] = []
for (const anthropicMessage of anthropicMessages) {
if (typeof anthropicMessage.content === "string") {
ollamaMessages.push({
role: anthropicMessage.role,
content: anthropicMessage.content,
})
} else {
if (anthropicMessage.role === "user") {
const { nonToolMessages, toolMessages } = anthropicMessage.content.reduce<{
nonToolMessages: (ClineTextContentBlock | ClineImageContentBlock)[]
toolMessages: ClineUserToolResultContentBlock[]
}>(
(acc, part) => {
if (part.type === "tool_result") {
acc.toolMessages.push(part)
} else if (part.type === "text" || part.type === "image") {
acc.nonToolMessages.push(part)
}
return acc
},
{ nonToolMessages: [], toolMessages: [] },
)
// Process tool result messages FIRST since they must follow the tool use messages
const toolResultImages: string[] = []
toolMessages.forEach((toolMessage) => {
// The Anthropic SDK allows tool results to be a string or an array of text and image blocks, enabling rich and structured content. In contrast, the Ollama SDK only supports tool results as a single string, so we map the Anthropic tool result parts into one concatenated string to maintain compatibility.
let content: string
if (typeof toolMessage.content === "string") {
content = toolMessage.content
} else {
content =
toolMessage.content
?.map((part) => {
if (part.type === "image") {
toolResultImages.push(`data:${part.source.media_type};base64,${part.source.data}`)
return "(see following user message for image)"
}
return part.text
})
.join("\n") ?? ""
}
ollamaMessages.push({
role: "user",
images: toolResultImages.length > 0 ? toolResultImages : undefined,
content: content,
})
})
// Process non-tool messages
if (nonToolMessages.length > 0) {
ollamaMessages.push({
role: "user",
content: nonToolMessages
.map((part) => {
if (part.type === "image") {
return `data:${part.source.media_type};base64,${part.source.data}`
}
return part.text
})
.join("\n"),
})
}
} else if (anthropicMessage.role === "assistant") {
const { nonToolMessages, toolMessages } = anthropicMessage.content.reduce<{
nonToolMessages: (ClineTextContentBlock | ClineImageContentBlock)[]
toolMessages: ClineAssistantToolUseBlock[]
}>(
(acc, part) => {
if (part.type === "tool_use") {
acc.toolMessages.push(part)
} else if (part.type === "text" || part.type === "image") {
acc.nonToolMessages.push(part)
} // assistant cannot send tool_result messages
return acc
},
{ nonToolMessages: [], toolMessages: [] },
)
// Process non-tool messages
let content: string = ""
if (nonToolMessages.length > 0) {
content = nonToolMessages
.map((part) => {
if (part.type === "image") {
return "" // impossible as the assistant cannot send images
}
return part.text
})
.join("\n")
}
ollamaMessages.push({
role: "assistant",
content,
})
}
}
}
return ollamaMessages
}
@@ -1,512 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import OpenAI from "openai"
import { ApiProvider } from "@/shared/api"
import {
ClineAssistantRedactedThinkingBlock,
ClineAssistantThinkingBlock,
ClineAssistantToolUseBlock,
ClineImageContentBlock,
ClineStorageMessage,
ClineTextContentBlock,
ClineUserToolResultContentBlock,
} from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
// OpenAI API has a maximum tool call ID length of 40 characters
const MAX_TOOL_CALL_ID_LENGTH = 40
/**
* Determines if a given tool ID follows the OpenAI Responses API format for tool calls.
* OpenAI tool call IDs start with "fc_" and are exactly 53 characters long.
*
* @param callId - The tool ID to check
* @returns True if the tool ID matches the OpenAI Responses API format, false otherwise
*/
function isOpenAIResponseToolId(callId: string): boolean {
return callId.startsWith("fc_") && callId.length === 53
}
/**
* Transforms a tool ID to a consistent format for OpenAI's Chat Completions API.
* NOTE: We do not want to transform tool IDs for non-OpenAI providers that may have different requirements.
* This function MUST be used for both tool_calls[].id (assistant) and tool_call_id (tool result)
* to ensure they match - otherwise OpenAI will reject the request with:
* "Invalid parameter: 'tool_call_id' of 'xxx' not found in 'tool_calls' of previous message."
*
* @param toolId - The original tool ID from Cline/Anthropic format
* @param provider - The API provider that the OpenAI formatted messages will be sent to
* @returns The transformed ID suitable for OpenAI API
*/
function transformToolCallIdForNativeApi(toolId: string, provider?: ApiProvider): string {
// OpenAI Responses API uses "fc_" prefix with 53 char length
// Convert these to "call_" prefix format for Chat Completions API
if (isOpenAIResponseToolId(toolId)) {
// Use the last 33 chars + "call_" (5 chars) to stay under the 40-char limit.
return `call_${toolId.slice(toolId.length - (MAX_TOOL_CALL_ID_LENGTH - 5))}`
}
if (provider !== "openai-native") {
return toolId
}
// Ensure ID doesn't exceed max length
if (toolId.length > MAX_TOOL_CALL_ID_LENGTH) {
return toolId.slice(0, MAX_TOOL_CALL_ID_LENGTH)
}
return toolId
}
/**
* Converts an array of ClineStorageMessage objects to OpenAI's Completions API format.
*
* Handles conversion of Cline-specific content types (tool uses, tool results, images, reasoning details)
* into OpenAI's expected message structure, including tool_calls and tool_call_id fields.
*
* @param anthropicMessages - Array of ClineStorageMessage objects to be converted
* @param provider - Optional parameter to indicate the API provider, which may affect ID transformation logic
* @returns Array of OpenAI.Chat.ChatCompletionMessageParam objects
*/
export function convertToOpenAiMessages(
anthropicMessages: Omit<ClineStorageMessage, "modelInfo">[],
provider?: ApiProvider,
): OpenAI.Chat.ChatCompletionMessageParam[] {
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = []
for (const anthropicMessage of anthropicMessages) {
if (typeof anthropicMessage.content === "string") {
openAiMessages.push({
role: anthropicMessage.role,
content: anthropicMessage.content,
})
} else {
// image_url.url is base64 encoded image data
// ensure it contains the content-type of the image: data:image/png;base64,
/*
{ role: "user", content: "" | { type: "text", text: string } | { type: "image_url", image_url: { url: string } } },
// content required unless tool_calls is present
{ role: "assistant", content?: "" | null, tool_calls?: [{ id: "", function: { name: "", arguments: "" }, type: "function" }] },
{ role: "tool", tool_call_id: "", content: ""}
*/
if (anthropicMessage.role === "user") {
const { nonToolMessages, toolMessages } = anthropicMessage.content.reduce<{
nonToolMessages: (ClineTextContentBlock | ClineImageContentBlock)[]
toolMessages: ClineUserToolResultContentBlock[]
}>(
(acc, part) => {
if (part.type === "tool_result") {
acc.toolMessages.push(part)
} else if (part.type === "text" || part.type === "image") {
acc.nonToolMessages.push(part)
} // user cannot send tool_use messages
return acc
},
{ nonToolMessages: [], toolMessages: [] },
)
// Process tool result messages FIRST since they must follow the tool use messages
const toolResultImages: ClineImageContentBlock[] = []
toolMessages.forEach((toolMessage) => {
// The Anthropic SDK allows tool results to be a string or an array of text and image blocks, enabling rich and structured content. In contrast, the OpenAI SDK only supports tool results as a single string, so we map the Anthropic tool result parts into one concatenated string to maintain compatibility.
let content: string
if (typeof toolMessage.content === "string") {
content = toolMessage.content
} else if (Array.isArray(toolMessage.content)) {
content =
toolMessage.content
?.map((part) => {
if (part.type === "image") {
toolResultImages.push(part)
return "(see following user message for image)"
}
return part.text
})
.join("\n") ?? ""
} else {
// Handle undefined content
content = ""
}
openAiMessages.push({
role: "tool",
// The tool_call_id must match the id used in the assistant's tool_calls array.
// Use the same transformation logic as tool_calls to ensure IDs match.
tool_call_id: transformToolCallIdForNativeApi(toolMessage.tool_use_id, provider),
content: content,
})
})
// If tool results contain images, send as a separate user message
// I ran into an issue where if I gave feedback for one of many tool uses, the request would fail.
// "Messages following `tool_use` blocks must begin with a matching number of `tool_result` blocks."
// Therefore we need to send these images after the tool result messages
// NOTE: it's actually okay to have multiple user messages in a row, the model will treat them as a continuation of the same input (this way works better than combining them into one message, since the tool result specifically mentions (see following user message for image)
// UPDATE v2.0: we don't use tools anymore, but if we did it's important to note that the openrouter prompt caching mechanism requires one user message at a time, so we would need to add these images to the user content array instead.
if (toolResultImages.length > 0) {
openAiMessages.push({
role: "user",
content: toolResultImages.map((part) => ({
type: "image_url",
image_url: { url: `data:${part.source.media_type};base64,${part.source.data}` },
})),
})
}
// Process non-tool messages
if (nonToolMessages.length > 0) {
openAiMessages.push({
role: "user",
content: nonToolMessages.map((part) => {
if (part.type === "image") {
return {
type: "image_url",
image_url: {
url: `data:${part.source.media_type};base64,${part.source.data}`,
},
}
}
return { type: "text", text: part.text }
}),
})
}
} else if (anthropicMessage.role === "assistant") {
const { nonToolMessages, toolMessages } = anthropicMessage.content.reduce<{
nonToolMessages: (
| ClineTextContentBlock
| ClineImageContentBlock
| ClineAssistantThinkingBlock
| ClineAssistantRedactedThinkingBlock
)[]
toolMessages: ClineAssistantToolUseBlock[]
}>(
(acc, part) => {
if (part.type === "tool_use") {
acc.toolMessages.push(part)
} else if (part.type === "text" || part.type === "image") {
acc.nonToolMessages.push(part)
} // assistant cannot send tool_result messages
return acc
},
{ nonToolMessages: [], toolMessages: [] },
)
// Process non-tool messages
let content: string | undefined
const reasoningDetails: any[] = []
const thinkingBlock = []
if (nonToolMessages.length > 0) {
nonToolMessages.forEach((part) => {
const anyPart = part as any
if (part.type === "text" && anyPart.reasoning_details) {
if (Array.isArray(anyPart.reasoning_details)) {
reasoningDetails.push(...anyPart.reasoning_details)
} else {
reasoningDetails.push(anyPart.reasoning_details)
}
}
if (part.type === "thinking" && part.thinking) {
// Reasoning details should have been moved to the text block
thinkingBlock.push(part)
}
})
content = nonToolMessages
.map((part) => {
if (part.type === "text" && part.text) {
return part.text
}
return ""
})
.join("\n")
}
// Process tool use messages
const tool_calls: OpenAI.Chat.ChatCompletionMessageToolCall[] = toolMessages.map((toolMessage) => {
const toolDetails = toolMessage.reasoning_details
const toolId = toolMessage.id
if (toolDetails) {
if (Array.isArray(toolDetails)) {
// For Gemini: reasoning details must be linkable back to the tool call.
// Sometimes OpenRouter/Gemini returns entries without `id`; those poison the next request.
// Keep only entries with an id matching the tool call id.
// See: https://github.com/cline/cline/issues/8214
const validDetails = toolDetails.filter((detail: any) => detail?.id === toolId)
if (validDetails.length > 0) {
reasoningDetails.push(...validDetails)
}
} else {
// Single reasoning detail - only include if it has matching id
const detail = toolDetails as any
if (detail?.id === toolId) {
reasoningDetails.push(toolDetails)
}
}
}
return {
// Use the same transformation as tool_call_id to ensure IDs match
id: transformToolCallIdForNativeApi(toolId, provider),
type: "function",
function: {
name: toolMessage.name,
// json string
arguments: JSON.stringify(toolMessage.input),
},
}
})
// Set content to blank when tool_calls are present but content has no text, per OpenAI API spec
const hasToolCalls = tool_calls.length > 0
const hasMeaningfulContent = content !== undefined && content.trim() !== ""
const finalContent = hasMeaningfulContent ? content : hasToolCalls ? null : undefined
const consolidatedReasoningDetails =
reasoningDetails.length > 0 ? consolidateReasoningDetails(reasoningDetails as any) : []
openAiMessages.push({
role: "assistant",
content: finalContent,
// Cannot be an empty array. API expects an array with minimum length 1, and will respond with an error if it's empty
tool_calls: tool_calls?.length > 0 ? tool_calls : undefined,
// Only include reasoning_details when non-empty; sending [] can trigger provider validation issues.
// @ts-expect-error
reasoning_details: consolidatedReasoningDetails.length > 0 ? consolidatedReasoningDetails : undefined,
})
}
}
}
return openAiMessages
}
// Type for OpenRouter's reasoning detail elements
// https://openrouter.ai/docs/use-cases/reasoning-tokens#streaming-response
type ReasoningDetail = {
// https://openrouter.ai/docs/use-cases/reasoning-tokens#reasoning-detail-types
type: string // "reasoning.summary" | "reasoning.encrypted" | "reasoning.text"
text?: string
data?: string // Encrypted reasoning data
signature?: string | null
id?: string | null // Unique identifier for the reasoning detail
/*
The format of the reasoning detail, with possible values:
"unknown" - Format is not specified
"openai-responses-v1" - OpenAI responses format version 1
"anthropic-claude-v1" - Anthropic Claude format version 1 (default)
*/
format: string //"unknown" | "openai-responses-v1" | "anthropic-claude-v1" | "xai-responses-v1"
index?: number // Sequential index of the reasoning detail
}
// Helper function to convert reasoning_details array to the format OpenRouter API expects
// Takes an array of reasoning detail objects and consolidates them by index
function consolidateReasoningDetails(reasoningDetails: ReasoningDetail[]): ReasoningDetail[] {
if (!reasoningDetails || reasoningDetails.length === 0) {
return []
}
// Group by index
const groupedByIndex = new Map<number, ReasoningDetail[]>()
for (const detail of reasoningDetails) {
// Drop corrupted encrypted reasoning blocks that would otherwise trigger:
// "Invalid input: expected string, received undefined" for reasoning_details.*.data
// See: https://github.com/cline/cline/issues/8214
if (detail.type === "reasoning.encrypted" && !detail.data) continue
const index = detail.index ?? 0
if (!groupedByIndex.has(index)) {
groupedByIndex.set(index, [])
}
groupedByIndex.get(index)!.push(detail)
}
// Consolidate each group
const consolidated: ReasoningDetail[] = []
for (const [index, details] of groupedByIndex.entries()) {
// Concatenate all text parts
let concatenatedText = ""
let signature: string | undefined
let id: string | undefined
let format = "unknown"
let type = "reasoning.text"
for (const detail of details) {
if (detail.text) {
concatenatedText += detail.text
}
// Keep the signature from the last item that has one
if (detail.signature) {
signature = detail.signature
}
// Keep the id from the last item that has one
if (detail.id) {
id = detail.id
}
// Keep format and type from any item (they should all be the same)
if (detail.format) {
format = detail.format
}
if (detail.type) {
type = detail.type
}
}
// Create consolidated entry for text
if (concatenatedText) {
const consolidatedEntry: ReasoningDetail = {
type: type,
text: concatenatedText,
signature: signature,
id: id,
format: format,
index: index,
}
consolidated.push(consolidatedEntry)
}
// For encrypted chunks (data), only keep the last one
let lastDataEntry: ReasoningDetail | undefined
for (const detail of details) {
if (detail.data) {
lastDataEntry = {
type: detail.type,
data: detail.data,
signature: detail.signature,
id: detail.id,
format: detail.format,
index: index,
}
}
}
if (lastDataEntry) {
consolidated.push(lastDataEntry)
}
}
return consolidated
}
// Unique name to use to filter out tool call that cannot be parsed correctly
const UNIQUE_ERROR_TOOL_NAME = "_cline_error_unknown_function_"
// Convert OpenAI response to Anthropic format
export function convertToAnthropicMessage(completion: OpenAI.Chat.Completions.ChatCompletion): Anthropic.Messages.Message {
const openAiMessage = completion.choices[0].message
const anthropicMessage: Anthropic.Messages.Message = {
id: completion.id,
type: "message",
role: openAiMessage.role, // always "assistant"
content: [
{
type: "text",
text: openAiMessage.content || "",
citations: null,
},
],
model: completion.model,
stop_reason: (() => {
switch (completion.choices[0].finish_reason) {
case "stop":
return "end_turn"
case "length":
return "max_tokens"
case "tool_calls":
return "tool_use"
case "content_filter": // Anthropic doesn't have an exact equivalent
default:
return null
}
})(),
stop_sequence: null, // which custom stop_sequence was generated, if any (not applicable if you don't use stop_sequence)
usage: {
input_tokens: completion.usage?.prompt_tokens || 0,
output_tokens: completion.usage?.completion_tokens || 0,
cache_creation_input_tokens: null,
cache_read_input_tokens: null,
},
}
try {
if (openAiMessage?.tool_calls?.length) {
const functionCalls = openAiMessage.tool_calls.filter((tc: any) => tc?.type === "function" && tc.function)
if (functionCalls.length > 0) {
anthropicMessage.content.push(
...functionCalls.map((toolCall: any): Anthropic.ToolUseBlock => {
let parsedInput = {}
try {
parsedInput = JSON.parse(toolCall.function?.arguments || "{}")
} catch (error) {
Logger.error("Failed to parse tool arguments:", error)
}
return {
type: "tool_use",
id: toolCall.id,
name: toolCall.function?.name || UNIQUE_ERROR_TOOL_NAME,
input: parsedInput,
}
}),
)
}
return anthropicMessage
}
} catch (error) {
Logger.error("Error converting OpenAI message to Anthropic format:", error)
}
return anthropicMessage
}
/**
* Sanitizes OpenAI messages for Gemini models by removing tool_calls that lack reasoning_details.
*
* Gemini models require thought signatures for tool calls. When switching providers mid-conversation,
* historical tool calls may not include Gemini reasoning details, which can poison the next request.
* This function drops tool_calls that lack reasoning_details and their paired tool messages.
*
* @param messages - Array of OpenAI chat completion messages
* @param modelId - The model ID to check if sanitization is needed
* @returns Sanitized array of messages (unchanged if not a Gemini model)
*/
export function sanitizeGeminiMessages(
messages: OpenAI.Chat.ChatCompletionMessageParam[],
modelId: string,
): OpenAI.Chat.ChatCompletionMessageParam[] {
if (!modelId.includes("gemini")) {
return messages
}
const droppedToolCallIds = new Set<string>()
const sanitized: OpenAI.Chat.ChatCompletionMessageParam[] = []
for (const msg of messages) {
if (msg.role === "assistant") {
const anyMsg = msg as any
const toolCalls = anyMsg.tool_calls
if (Array.isArray(toolCalls) && toolCalls.length > 0) {
const reasoningDetails = anyMsg.reasoning_details
const hasReasoningDetails = Array.isArray(reasoningDetails) && reasoningDetails.length > 0
if (!hasReasoningDetails) {
for (const tc of toolCalls) {
if (tc?.id) {
droppedToolCallIds.add(tc.id)
}
}
// Keep any textual content, but drop the tool_calls themselves.
if (anyMsg.content) {
sanitized.push({ role: "assistant", content: anyMsg.content } as any)
}
continue
}
}
}
if (msg.role === "tool") {
const anyMsg = msg as any
if (anyMsg.tool_call_id && droppedToolCallIds.has(anyMsg.tool_call_id)) {
continue
}
}
sanitized.push(msg)
}
return sanitized
}
@@ -1,249 +0,0 @@
import { ResponseInput, ResponseInputMessageContentList, ResponseReasoningItem } from "openai/resources/responses/responses"
import { ClineStorageMessage } from "@/shared/messages/content"
/**
* Converts an array of ClineStorageMessage objects (extension of Anthropic format) to a ResponseInput array to use with OpenAI's Responses API.
*
* ## Key Differences from Chat Completions API
*
* The Responses API has stricter requirements than the Chat Completions API:
*
* ### Chat Completions API:
* - Messages are simple role/content pairs
* - System prompts are separate messages with role="system"
* - No explicit reasoning item structure
* - More forgiving about message ordering
*
* ### Responses API:
* - Uses an "input" array of heterogeneous items (messages, reasoning, function_calls, etc.)
* - System prompts go in an "instructions" field, not as messages
* - Reasoning items MUST be immediately followed by a message or function_call
* - Strict ordering requirements match training data distribution
*
* ## The Reasoning Item Constraint
*
* **THE CRITICAL ERROR:** "Item 'rs_...' of type 'reasoning' was provided without its required following item"
*
* This error occurs when reasoning items are orphaned or separated from their corresponding output.
*
* ### What Causes This Error:
* ```
* ❌ WRONG - Reasoning orphaned between turns:
* [
* { role: "user", content: [...] },
* { type: "reasoning", id: "rs_abc", summary: [...] }, // ← ORPHANED!
* { type: "message", role: "assistant", content: [...] },
* { role: "user", content: [...] }
* ]
* ```
*
* ### The Fix - Keep Complete Assistant Turns Together:
* ```
* ✅ CORRECT - Reasoning paired with its message:
* [
* { role: "user", content: [...] },
* { type: "reasoning", id: "rs_abc", summary: [...] },
* { type: "message", role: "assistant", content: [...] }, // ← Immediately follows reasoning
* { role: "user", content: [...] }
* ]
* ```
*
* **Per OpenAI Engineering Guidance:**
* - ❌ WRONG: `content += filter(lambda x: x.type == "reasoning", resp.output)`
* - ✅ CORRECT: `content += resp.output`
*
* Never extract only reasoning items - always include the complete output sequence
* (reasoning + message/function_call) as provided by the API.
*
* ## Implementation Strategy
*
* 1. **Separate processing for assistant vs user messages** - Assistant turns need special
* handling to maintain reasoning-message pairing
* 2. **Collect all assistant items together** - Gather reasoning, messages, and function_calls
* for the entire assistant turn before validating
* 3. **Validate pairing within each turn** - Ensure each reasoning item is immediately followed
* by a message or function_call, inserting placeholders if needed
* 4. **Flush complete turns atomically** - Add all items from an assistant turn together to
* maintain proper sequencing
*
* @link https://community.openai.com/t/openai-api-error-function-call-was-provided-without-its-required-reasoning-item-the-real-issue/1355347
*
* @param messages - Array of ClineStorageMessage objects to be converted
* @returns ResponseInput array containing the transformed messages with proper reasoning pairing
*/
export function convertToOpenAIResponsesInput(
_messages: ClineStorageMessage[],
options?: { usePreviousResponseId?: boolean },
): {
input: ResponseInput
previousResponseId?: string
} {
// Chain from the latest stored Responses API assistant message when available.
// When chaining, only send new items after that assistant turn.
let previousResponseId: string | undefined
let messages = _messages
if (options?.usePreviousResponseId) {
for (let i = _messages.length - 1; i >= 0; i--) {
const msg = _messages[i]
// Must be less than 24 hours old to be considered for chaining as the previous Id is only valid for 24 hours.
// Set to 23 hours to account for any potential delays in processing.
const isLessThan23HoursOld = msg.ts ? Date.now() - msg.ts < 23 * 60 * 60 * 1000 : false
if (msg.role === "assistant" && msg.id && isLessThan23HoursOld) {
previousResponseId = msg.id
messages = _messages.slice(i + 1)
break
}
}
}
const allItems: any[] = []
const toolUseIdToCallId = new Map<string, string>()
for (const m of messages) {
if (typeof m.content === "string") {
allItems.push({ role: m.role, content: [{ type: "input_text", text: m.content }] })
continue
}
if (m.role === "assistant") {
// For assistant messages, we must ensure reasoning items are IMMEDIATELY followed
// by their corresponding message or function_call. Process the entire assistant
// turn and ensure proper pairing.
const assistantItems: any[] = []
for (const part of m.content) {
switch (part.type) {
case "thinking":
// Only include reasoning item if it has actual content (thinking text or summary)
// Empty reasoning items cause API errors: "Item 'rs_...' of type 'reasoning' was provided without its required following item"
const hasThinkingContent = part.thinking && part.thinking.trim().length > 0
const hasSummaryContent = part.summary && Array.isArray(part.summary) && part.summary.length > 0
if (part.call_id && part.call_id.length > 0 && (hasThinkingContent || hasSummaryContent)) {
// Use summary if available, otherwise use thinking text
let summary: any[] = []
if (hasSummaryContent) {
// part.summary is already in the correct format from OpenAI Responses API
summary = part.summary as any[]
} else if (hasThinkingContent) {
// Convert thinking text to summary format
summary = [
{
type: "summary_text",
text: part.thinking,
},
]
}
assistantItems.push({
id: part.call_id,
type: "reasoning",
summary,
} as ResponseReasoningItem)
}
break
case "redacted_thinking":
// Include reasoning item with encrypted content if it has a call_id
// Even if data is missing, we need to maintain the reasoning-function_call pairing
if (part.call_id && part.call_id.length > 0) {
const reasoningItem: any = {
id: part.call_id,
type: "reasoning",
summary: [],
}
// Only include encrypted_content if data exists
if (part.data) {
reasoningItem.encrypted_content = part.data
}
assistantItems.push(reasoningItem as ResponseReasoningItem)
}
break
case "text":
// Message ID goes at the message level, not in the content
// The reasoning item and message can have different IDs - they just need to be adjacent
const messageItem: any = {
type: "message",
role: "assistant",
content: [{ type: "output_text", text: part.text }],
}
// Set message-level id if available
if (part.call_id) {
messageItem.id = part.call_id
}
assistantItems.push(messageItem)
break
case "image":
// Message ID goes at the message level, not in the content
const imageItem: any = {
type: "message",
role: "assistant",
content: [{ type: "output_text", text: `[image:${part.source.media_type}]` }],
}
// Set message-level id if available (though images typically don't have call_id)
if (part.call_id) {
imageItem.id = part.call_id
}
assistantItems.push(imageItem)
break
case "tool_use": {
// Function calls use call_id, not related to reasoning item ID
const call_id = part.call_id || part.id
if (part.call_id) {
toolUseIdToCallId.set(part.id, part.call_id)
}
assistantItems.push({
type: "function_call",
call_id,
// MAX 53 characters for OpenAI Responses API tool IDs
id: !part.id.startsWith("fc_") ? `fc_${part.id.slice(0, 50)}` : part.id,
name: part.name,
arguments: JSON.stringify(part.input ?? {}),
})
break
}
}
}
allItems.push(...assistantItems)
} else {
// User messages - collect all content
const messageContent: ResponseInputMessageContentList = []
for (const part of m.content) {
switch (part.type) {
case "text":
messageContent.push({ type: "input_text", text: part.text })
break
case "image":
messageContent.push({
type: "input_image",
detail: "auto",
image_url: `data:${part.source.media_type};base64,${part.source.data}`,
})
break
case "tool_result": {
// Flush any pending message content before adding tool result
if (messageContent.length > 0) {
allItems.push({ role: m.role, content: [...messageContent] })
messageContent.length = 0
}
const call_id = part.call_id || toolUseIdToCallId.get(part.tool_use_id) || part.tool_use_id
allItems.push({
type: "function_call_output",
call_id,
output: typeof part.content === "string" ? part.content : JSON.stringify(part.content),
})
break
}
}
}
// Flush any remaining user message content
if (messageContent.length > 0) {
allItems.push({ role: m.role, content: [...messageContent] })
}
}
}
return { input: allItems, previousResponseId }
}
@@ -1,217 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import {
CLAUDE_SONNET_1M_SUFFIX,
ModelInfo,
OPENROUTER_PROVIDER_PREFERENCES,
openRouterClaudeOpus461mModelId,
openRouterClaudeOpus471mModelId,
openRouterClaudeOpus481mModelId,
openRouterClaudeSonnet41mModelId,
openRouterClaudeSonnet451mModelId,
openRouterClaudeSonnet461mModelId,
} from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import {
GEMINI_FLASH_MAX_OUTPUT_TOKENS,
isGeminiFlashModel,
shouldSkipReasoningForModel,
supportsReasoningEffortForModel,
} from "@utils/model-utils"
import OpenAI from "openai"
import { ChatCompletionTool } from "openai/resources/chat/completions"
import { convertToOpenAiMessages, sanitizeGeminiMessages } from "./openai-format"
import { convertToR1Format } from "./r1-format"
import { getOpenAIToolParams } from "./tool-call-processor"
const openRouterExplicitCacheControlModelIds = new Set([
"deepseek/deepseek-v3.2",
"qwen/qwen-plus",
"qwen/qwen3-max",
"qwen/qwen3.6-plus",
"qwen/qwen3.7-max",
"qwen/qwen3-coder-plus",
"qwen/qwen3-coder-flash",
])
function needsExplicitCacheControl(modelId: string): boolean {
return (
modelId.startsWith("anthropic/") || modelId.startsWith("minimax/") || openRouterExplicitCacheControlModelIds.has(modelId)
)
}
export async function createOpenRouterStream(
client: OpenAI,
systemPrompt: string,
messages: Anthropic.Messages.MessageParam[],
model: { id: string; info: ModelInfo },
reasoningEffort?: string,
thinkingBudgetTokens?: number,
openRouterProviderSorting?: string,
tools?: Array<ChatCompletionTool>,
enableParallelToolCalling?: boolean,
) {
// Convert Anthropic messages to OpenAI format
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const isClaude1m =
model.id === openRouterClaudeSonnet41mModelId ||
model.id === openRouterClaudeSonnet451mModelId ||
model.id === openRouterClaudeSonnet461mModelId ||
model.id === openRouterClaudeOpus461mModelId ||
model.id === openRouterClaudeOpus471mModelId ||
model.id === openRouterClaudeOpus481mModelId
if (isClaude1m) {
// remove the custom :1m suffix, to create the model id openrouter API expects
model.id = model.id.slice(0, -CLAUDE_SONNET_1M_SUFFIX.length)
}
// Sanitize messages for Gemini models (removes tool_calls without reasoning_details)
openAiMessages = sanitizeGeminiMessages(openAiMessages, model.id)
// prompt caching: https://openrouter.ai/docs/prompt-caching
// Some OpenRouter models require cache_control blocks instead of automatic provider caching.
// Other providers (OpenAI, Google) handle caching automatically without cache_control blocks.
const needsCacheControl = needsExplicitCacheControl(model.id)
if (needsCacheControl) {
openAiMessages[0] = {
role: "system",
content: [
{
type: "text",
text: systemPrompt,
// @ts-expect-error-next-line
cache_control: { type: "ephemeral" },
},
],
}
// Add cache_control to the last two user messages
// (note: this works because we only ever add one user message at a time, but if we added multiple we'd need to mark the user message before the last assistant message)
const lastTwoUserMessages = openAiMessages.filter((msg) => msg.role === "user").slice(-2)
lastTwoUserMessages.forEach((msg) => {
if (typeof msg.content === "string") {
msg.content = [{ type: "text", text: msg.content }]
}
if (Array.isArray(msg.content)) {
// NOTE: this is fine since env details will always be added at the end. but if it weren't there, and the user added a image_url type message, it would pop a text part before it and then move it after to the end.
let lastTextPart = msg.content.filter((part) => part.type === "text").pop()
if (!lastTextPart) {
lastTextPart = { type: "text", text: "..." }
msg.content.push(lastTextPart)
}
// @ts-expect-error-next-line
lastTextPart["cache_control"] = { type: "ephemeral" }
}
})
}
let temperature: number | undefined = 0
let topP: number | undefined
if (
model.id.startsWith("deepseek/deepseek-r1") ||
model.id === "perplexity/sonar-reasoning" ||
model.id === "qwen/qwq-32b:free" ||
model.id === "qwen/qwq-32b"
) {
// Recommended values from DeepSeek
temperature = 0.7
topP = 0.95
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
if (model.id.startsWith("google/gemini-3")) {
// Recommended value from google
temperature = 1.0
}
const supportsReasoningEffort = supportsReasoningEffortForModel(model.id)
// Claude Opus 4.5+ uses adaptive thinking instead of budgeted extended thinking.
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(model.id)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(reasoningEffort, thinkingBudgetTokens)
: undefined
if (isAdaptiveThinkingModel) {
temperature = undefined
topP = undefined
}
let reasoning: Record<string, unknown> | undefined
if (!isAdaptiveThinkingModel) {
switch (model.id) {
case "anthropic/claude-haiku-4.5":
case "anthropic/claude-4.5-haiku":
case "anthropic/claude-sonnet-4.6":
case "anthropic/claude-4.6-sonnet":
case "anthropic/claude-sonnet-4.5":
case "anthropic/claude-4.5-sonnet":
case "anthropic/claude-sonnet-4":
case "anthropic/claude-opus-4.1":
case "anthropic/claude-opus-4":
case "anthropic/claude-3.7-sonnet":
case "anthropic/claude-3.7-sonnet:beta":
case "anthropic/claude-3.7-sonnet:thinking":
case "anthropic/claude-3-7-sonnet":
case "anthropic/claude-3-7-sonnet:beta":
const budget_tokens = thinkingBudgetTokens || 0
const reasoningOn = budget_tokens !== 0
if (reasoningOn) {
temperature = undefined // extended thinking does not support non-1 temperature
reasoning = { max_tokens: budget_tokens }
}
break
default:
if (thinkingBudgetTokens && model.info?.thinkingConfig && thinkingBudgetTokens > 0 && !supportsReasoningEffort) {
temperature = undefined // extended thinking does not support non-1 temperature
reasoning = { max_tokens: thinkingBudgetTokens }
break
}
}
}
const providerPreferences = OPENROUTER_PROVIDER_PREFERENCES[model.id]
if (providerPreferences) {
openRouterProviderSorting = undefined
}
const normalizedReasoningEffort = reasoningEffort !== undefined ? normalizeOpenaiReasoningEffort(reasoningEffort) : undefined
const reasoningEffortValue = supportsReasoningEffort ? normalizedReasoningEffort : undefined
// Skip reasoning for models that don't support it (e.g., devstral, grok-4), or when effort explicitly disables it.
const includeReasoning = isAdaptiveThinkingModel
? !!adaptiveThinking?.enabled
: !shouldSkipReasoningForModel(model.id) && reasoningEffortValue !== "none"
const reasoningPayload = isAdaptiveThinkingModel
? adaptiveThinking?.enabled
? { enabled: true }
: undefined
: (reasoning ?? (reasoningEffortValue && reasoningEffortValue !== "none" ? { effort: reasoningEffortValue } : undefined))
const maxTokens = isGeminiFlashModel(model.id)
? Math.min(model.info.maxTokens || GEMINI_FLASH_MAX_OUTPUT_TOKENS, GEMINI_FLASH_MAX_OUTPUT_TOKENS)
: undefined
const requestPayload: Record<string, unknown> = {
model: model.id,
...(maxTokens ? { max_tokens: maxTokens } : {}),
temperature: temperature,
top_p: topP,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
include_reasoning: includeReasoning,
...(reasoningPayload ? { reasoning: reasoningPayload } : {}),
...(isAdaptiveThinkingModel && adaptiveThinking?.effort ? { verbosity: adaptiveThinking.effort } : {}),
...(openRouterProviderSorting && !providerPreferences ? { provider: { sort: openRouterProviderSorting } } : {}),
...(providerPreferences ? { provider: providerPreferences } : {}),
...(isClaude1m ? { provider: { order: ["anthropic", "google-vertex/global"], allow_fallbacks: false } } : {}),
...getOpenAIToolParams(tools, !!enableParallelToolCalling),
}
// @ts-expect-error-next-line
const stream = await client.chat.completions.create(requestPayload)
return stream
}
@@ -1,151 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import OpenAI from "openai"
import { ClineAssistantThinkingBlock, ClineStorageMessage } from "@/shared/messages/content"
/**
* DeepSeek Reasoner message format with reasoning_content support.
*/
export type DeepSeekReasonerMessage = OpenAI.Chat.ChatCompletionMessageParam & {
reasoning_content?: string
}
/**
* Adds reasoning_content to OpenAI messages for DeepSeek Reasoner.
* Per DeepSeek API: reasoning_content should be passed back during tool calling in the same turn,
* and omitted when starting a new turn.
*/
export function addReasoningContent(
openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[],
originalMessages: ClineStorageMessage[],
): DeepSeekReasonerMessage[] {
// Find last user message index (start of current turn)
// If no user message exists (lastUserIndex = -1), all messages are in the "current turn",
// so reasoning_content will be added to all assistant messages. This is intentional.
let lastUserIndex = -1
for (let i = openAiMessages.length - 1; i >= 0; i--) {
if (openAiMessages[i].role === "user") {
lastUserIndex = i
break
}
}
// Extract thinking content from original messages, keyed by assistant index
const thinkingByIndex = new Map<number, string>()
let assistantIdx = 0
for (const msg of originalMessages) {
if (msg.role === "assistant") {
if (Array.isArray(msg.content)) {
const thinking = msg.content
.filter((p): p is ClineAssistantThinkingBlock => p.type === "thinking")
.map((p) => p.thinking)
.join("\n")
if (thinking) {
thinkingByIndex.set(assistantIdx, thinking)
}
}
assistantIdx++
}
}
// Add reasoning_content only to assistant messages in current turn
let aiIdx = 0
return openAiMessages.map((msg, i): DeepSeekReasonerMessage => {
if (msg.role === "assistant") {
const thinking = thinkingByIndex.get(aiIdx++)
if (thinking && i >= lastUserIndex) {
return { ...msg, reasoning_content: thinking }
}
}
return msg
})
}
/**
* Converts Anthropic messages to OpenAI format and merges consecutive messages with the same role.
* This is required for DeepSeek Reasoner which does not support successive messages with the same role.
* DeepSeek highly recommends using 'user' role instead of 'system' role for optimal performance.
*
* @param messages Array of Anthropic messages
* @returns Array of OpenAI messages where consecutive messages with the same role are merged together
*/
export function convertToR1Format(messages: Anthropic.Messages.MessageParam[]): OpenAI.Chat.ChatCompletionMessageParam[] {
return messages.reduce<OpenAI.Chat.ChatCompletionMessageParam[]>((merged, message) => {
const lastMessage = merged[merged.length - 1]
let messageContent: string | (OpenAI.Chat.ChatCompletionContentPartText | OpenAI.Chat.ChatCompletionContentPartImage)[] =
""
let hasImages = false
if (Array.isArray(message.content)) {
const textParts: string[] = []
const imageParts: OpenAI.Chat.ChatCompletionContentPartImage[] = []
message.content.forEach((part) => {
if (part.type === "text") {
textParts.push(part.text)
}
if (part.type === "image") {
hasImages = true
imageParts.push({
type: "image_url",
image_url: { url: `data:${part.source.media_type};base64,${part.source.data}` },
})
}
})
if (hasImages) {
const parts: (OpenAI.Chat.ChatCompletionContentPartText | OpenAI.Chat.ChatCompletionContentPartImage)[] = []
if (textParts.length > 0) {
parts.push({ type: "text", text: textParts.join("\n") })
}
parts.push(...imageParts)
messageContent = parts
} else {
messageContent = textParts.join("\n")
}
} else {
messageContent = message.content
}
// If the last message has the same role, merge the content
if (lastMessage?.role === message.role) {
if (typeof lastMessage.content === "string" && typeof messageContent === "string") {
lastMessage.content += `\n${messageContent}`
} else {
const lastContent = Array.isArray(lastMessage.content)
? lastMessage.content
: [{ type: "text" as const, text: lastMessage.content || "" }]
const newContent = Array.isArray(messageContent)
? messageContent
: [{ type: "text" as const, text: messageContent }]
if (message.role === "assistant") {
const mergedContent = [
...lastContent,
...newContent,
] as OpenAI.Chat.ChatCompletionAssistantMessageParam["content"]
lastMessage.content = mergedContent
} else {
const mergedContent = [...lastContent, ...newContent] as OpenAI.Chat.ChatCompletionUserMessageParam["content"]
lastMessage.content = mergedContent
}
}
} else {
// Adds new message with the correct type based on role
if (message.role === "assistant") {
const newMessage: OpenAI.Chat.ChatCompletionAssistantMessageParam = {
role: "assistant",
content: messageContent as OpenAI.Chat.ChatCompletionAssistantMessageParam["content"],
}
merged.push(newMessage)
} else {
const newMessage: OpenAI.Chat.ChatCompletionUserMessageParam = {
role: "user",
content: messageContent as OpenAI.Chat.ChatCompletionUserMessageParam["content"],
}
merged.push(newMessage)
}
}
return merged
}, [])
}
@@ -1,104 +0,0 @@
import type {
ChatCompletionChunk,
ChatCompletionToolChoiceOption,
ChatCompletionTool as OpenAITool,
} from "openai/resources/chat/completions"
import { Logger } from "@/shared/services/Logger"
import type { ApiStreamToolCallsChunk } from "./stream"
/**
* Helper class to process tool call deltas from OpenAI-compatible streaming responses.
* Handles accumulating tool call ID and name across multiple delta chunks,
* and yields properly formatted tool call chunks when arguments are received.
*/
export class ToolCallProcessor {
private toolCallStateByIndex: Map<number, { id: string; name: string }>
constructor() {
this.toolCallStateByIndex = new Map()
}
/**
* Process tool call deltas from a chunk and yield formatted tool call chunks.
* @param toolCallDeltas - Array of tool call deltas from the chunk
* @yields Formatted tool call chunks ready to be yielded in the API stream
*/
*processToolCallDeltas(
toolCallDeltas: ChatCompletionChunk.Choice.Delta.ToolCall[] | undefined,
): Generator<ApiStreamToolCallsChunk> {
if (!toolCallDeltas) {
return
}
for (const [fallbackIndex, toolCallDelta] of toolCallDeltas.entries()) {
// OpenAI-style streams include an index per tool call. Use iteration order as a fallback.
const toolCallIndex = toolCallDelta.index ?? fallbackIndex
const toolCallState = this.getOrCreateToolCallState(toolCallIndex)
// Accumulate the tool call ID if present
if (toolCallDelta.id) {
toolCallState.id = toolCallDelta.id
}
// Accumulate the function name if present
if (toolCallDelta.function?.name) {
Logger.debug(`[ToolCallProcessor] Native Tool Called: ${toolCallDelta.function.name}`)
toolCallState.name = toolCallDelta.function.name
}
// Only yield when we have all required fields: id, name, and arguments
if (toolCallState.id && toolCallState.name && toolCallDelta.function?.arguments) {
yield {
type: "tool_calls",
tool_call: {
...toolCallDelta,
function: {
...toolCallDelta.function,
id: toolCallState.id,
name: toolCallState.name,
},
},
}
}
}
}
private getOrCreateToolCallState(index: number): { id: string; name: string } {
const existingState = this.toolCallStateByIndex.get(index)
if (existingState) {
return existingState
}
const initialState = { id: "", name: "" }
this.toolCallStateByIndex.set(index, initialState)
return initialState
}
/**
* Reset the internal state. Call this when starting a new message.
*/
reset(): void {
this.toolCallStateByIndex.clear()
}
/**
* Get the current accumulated tool call state (useful for debugging).
*/
getState(): Record<number, { id: string; name: string }> {
return Object.fromEntries(this.toolCallStateByIndex.entries())
}
}
export function getOpenAIToolParams(tools?: OpenAITool[], enableParallelToolCalls = false) {
if (!tools?.length) {
return {
tools: undefined,
}
}
return {
tools,
tool_choice: "auto" as ChatCompletionToolChoiceOption,
parallel_tool_calls: enableParallelToolCalls,
}
}

Some files were not shown because too many files have changed in this diff Show More