Compare commits

..

58 Commits

Author SHA1 Message Date
Max Paulus 🥪 3d3a257b08 bedrock changes 2026-06-01 14:06:30 -07:00
Max Paulus 🥪 293d1cee0c fix xai provider
- xai provider settings now properly updates providers.json
2026-06-01 13:07:32 -07:00
Max Paulus 🥪 5dcfa9cf79 updat gitignore 2026-06-01 13:03:11 -07:00
Dominic Cooney 1e3972bcf0 fix(vscode): widen ZAiProvider getEventValue to accept VSCodeDropdown event union
The nightly 'vscode test' job (npm run ci:build -> build:webview -> tsc -b) failed on both Linux and windows-latest:

  ZAiProvider.tsx: Argument of type 'Event | FormEvent<HTMLElement>' is not assignable to parameter of type 'Event'

VSCodeDropdown's onChange supplies 'Event | React.FormEvent<HTMLElement>', but the getEventValue helper (added when the zai provider was moved to providers.json) was typed to accept only Event. The helper only reads target.value, which exists on both, so widen the parameter to the same union the dropdown provides.

Note: this only surfaces under 'tsc -b' / 'tsc --noEmit -p tsconfig.app.json'; the webview's root tsconfig.json is a solution file with files:[] so a bare 'tsc --noEmit' checks nothing.

The separate 'test / test' job failure is an aggregate gate that fails because vscode-test failed; fixing this resolves it too.
2026-06-01 12:28:08 -07:00
Dominic Cooney 5a5e23b79f fix(vscode): load ambient vscode LM type decls in unit-test ts-node program
The nightly CI 'Unit Tests with coverage - Linux' step (npm run test:unit) failed in ts-node compilation:

  state-keys.ts: Module 'vscode' has no exported member 'LanguageModelChatSelector'

This surfaced as a misleading 'Cannot find package @shared/...' ERR_MODULE_NOT_FOUND: Mocha tried require() first (which threw the ts-node TSError), then fell back to import(), whose ESM resolver cannot resolve the @shared/* path alias.

tsconfig.unit-test.json is driven by ts-node, which defaults to files:false and compiles modules on demand from their imports. The loose ambient augmentation in src/types/vscode-language-model.d.ts was therefore never loaded, so state-keys.ts failed to compile.

Set ts-node.files=true and add src/types/**/*.d.ts to include so the augmentation is part of the unit-test program. Mirrors the earlier tsconfig.test.json fix for the separate build toolchain.
2026-06-01 12:23:00 -07:00
Max Paulus 🥪 70b5f4bc6f fix litellm provider 2026-06-01 12:16:32 -07:00
Max Paulus 🥪 15bf76beb3 change zai provider to user providers.json instead of statemanager 2026-06-01 11:28:28 -07:00
Dominic Cooney 07a8cb28b8 fix(vscode): include ambient vscode LM type decls in test tsconfig
The nightly CI "Build Tests and Extension" step (npm run ci:build) failed
in compile-tests (tsc -p tsconfig.test.json) with:
  getVsCodeLmModels.ts: Property 'lm' does not exist on type 'typeof import("vscode")'
  state-keys.ts: Module '"vscode"' has no exported member 'LanguageModelChatSelector'

@types/vscode is pinned to 1.84.0, which predates the Language Model API.
The repo compensates with an ambient augmentation in
src/types/vscode-language-model.d.ts, which the main tsconfig picks up via
its 'src/**/*' include. tsconfig.test.json overrides include to only
'src/**/*.test.ts'; listing src/types under typeRoots does not load a loose
.d.ts (typeRoots only auto-loads @types-style package folders), so the
augmentation was missing from the test program.

Add 'src/types/**/*.d.ts' to tsconfig.test.json include so the augmentation
is part of the test compilation.
2026-06-01 11:09:59 -07:00
Max Paulus 🥪 08e32a26f6 fix open task conversation file 2026-06-01 11:08:15 -07:00
Max Paulus 🥪 44257db17e fix history view bugs
- deleting entries works
- favoriting works
2026-06-01 10:53:37 -07:00
Max Paulus 🥪 d93481d511 improve OCA provider
- oca provider login now works
2026-06-01 10:40:38 -07:00
Dominic Cooney 3f2976b290 fix(vscode): show Start New Task after multi-iteration completion turns
The completion signal (attemptCompletionSeen) was scoped to the message
translator's per-iteration reset(), so a turn that called the completion tool
and then ran another iteration before 'done' lost the signal — the turn ended
as awaiting_followup instead of completed and the footer showed no
'Start New Task' button despite the green Task Completed box.

Make the completion signal turn-scoped: reset() only clears streaming pointers;
a new clearTurnOutcome() clears it at genuine turn/task boundaries (initTask,
reinitExistingTaskFromId, askResponse). Also recognize the SDK's built-in
submit_and_exit completion tool (summary field) alongside the VSCode
attempt_completion tool (result field).

Removes the webview-message-state design/findings docs and rewrites the
related comments to describe the system as-is.
2026-06-01 10:36:03 -07:00
Max Paulus 🥪 877ba07f0d improve openai compatible provider settings 2026-06-01 10:36:03 -07:00
Dominic Cooney d21433e7ca fix(vscode): fix stuck/missing footer buttons from TurnState regressions
Webview now gates turnState by seq (a stale snapshot can no longer revert
streaming->idle), never empties a live transcript on a lone newer-epoch partial,
and routes a follow-up after completed/awaiting_followup to askResponse instead
of starting a new task.

Backend phase emission fixes surfaced by live testing:
- post state on turn end even when the done event carries no messages
- askResponse sets phase=streaming (resume/continue shows Cancel, unblocks send)
- a post-cancel turn-complete straggler no longer clobbers resumable

Adds reducer, send-routing, and session-event-coordinator unit tests. See
src/sdk/docs/webview-message-state-design.md §11 for the full debugging log.
2026-06-01 10:36:03 -07:00
Dominic Cooney 8e956fa3b9 refactor(vscode): translator hygiene — suppress ask_question row, drop done synthetic ask
S7 (a + c) of the message-state redesign — removes two order-dependent hacks now that
TurnState is authoritative.

(a) ask_question / ask_followup_question are suppressed from the generic say:"tool"
    renderer (both content_start and content_end). The SdkInteractionCoordinator services
    these and emits the proper ask:"followup"; the generic tool row was an orphan partial
    that never finalized and defeated the tail heuristics. The CLI already does this.

(c) The `done` handler no longer synthesizes a trailing ask:"completion_result" — that
    was the "must be last message" hack (ENG-1887) that existed only because the webview
    inferred UI mode from the array tail. `done` now emits no transcript message and only
    signals turnComplete; the webview reads phase from TurnState (completed when
    attempt_completion was used, else awaiting_followup). The green "Task Completed" box
    still comes from the say:"completion_result" emitted at attempt_completion content_end.

Translator unit tests updated to the new contract (done → 0 messages, turnComplete=true).

Deferred to a follow-up (lower risk if left): (b) collapsing the approval ask onto the
streaming tool's id (needs cross-coordinator id threading), and (d) the mistake_limit
forced abort (now harmless since phase is authoritative). The persisted-history renderer
still appends its own trailing ask so reopened tasks show the resume affordance —
intentional.

428 SDK unit tests pass; tsc + biome clean.
2026-06-01 10:36:03 -07:00
Dominic Cooney 3fdf8fc135 fix(vscode): cancel raises the epoch fence before aborting (no post-cancel stragglers)
S6 of the message-state redesign. sdkHost.abort() is cooperative — the SDK can emit a
few more events after it. Previously cancelTask aborted first and only the (post-abort)
!isRunning filter stripped two trailing ask types, letting say:* stragglers land after
the resume_task ask and wedge the UI.

Now cancel raises the fence SYNCHRONOUSLY before the abort:
- SdkController.cancelTask sets turnState.phase = "resumable" (already in S4), then
- SdkTaskControlCoordinator.cancelTask calls raiseCancelFence() (epoch bump) BEFORE
  awaiting sdkHost.abort().

Any event the SDK emits after the abort request therefore carries the OLD epoch and is
dropped by the webview's convergent reducer; the authoritative phase is "resumable"
(Resume Task), independent of the message tail. Order matters and is covered by a unit
test (fence before abort).

Usage accounting is exempt from the fence — it was never gated by the message filter, so
a post-cancel usage event still bills the tokens the provider actually generated.

The legacy !isRunning ask-only filter is now redundant but kept as defense-in-depth.
428 SDK unit tests pass; tsc + biome clean.
2026-06-01 10:36:03 -07:00
Dominic Cooney 991e33f385 fix(vscode): drive webview footer + buttons from TurnState (fixes RC1)
S5 of the message-state redesign. The webview decided "thinking vs approving vs done"
and which buttons to show by inspecting the TAIL of clineMessages. Because the backend
appends bookkeeping (api_req_started usage) after content and even after approval asks,
the tail routinely meant the wrong thing — producing stuck "Thinking", vanishing
Approve/Reject, and the footer disagreeing with the buttons (RC1, reproduced on camera).

Now the webview reads the authoritative backend-owned TurnState (added in S4):

- buttonConfig: add buttonsForPhase(turnState, anchoredMessage) and the dispatcher
  getButtonConfigFromState(messages, turnState, mode). The button SET is chosen by phase;
  approval labels (Approve vs Save, Run Command, MCP, subagents) come from the anchored
  message (turnState.anchorTs). When turnState is absent (classic/older state) it falls
  back to the legacy tail-walking getButtonConfigForMessages.
- ActionButtons reads turnState from useExtensionState and uses getButtonConfigFromState.
- MessagesArea.isWaitingForResponse short-circuits to `phase === "streaming"` when
  turnState is present (and only shows the footer loader until a content row is actually
  streaming); the legacy tail inference is kept as the fallback.

Button actions (approve/reject/proceed/new_task) already send a fixed responseType
independent of clineAsk, and the SDK backend resolves the pending approval/followup
promise — so routing is correct under TurnState. Classic fallback paths are untouched.

New unit tests cover buttonsForPhase (every phase + anchored-label selection +
mistake_limit-vs-api_req_failed) and getButtonConfigFromState (prefers TurnState over a
trailing bookkeeping tail; legacy fallback). 68 webview chat tests pass; tsc + biome clean.
2026-06-01 10:36:03 -07:00
Dominic Cooney 2bd7bbd44e feat(vscode): add authoritative TurnState (backend-owned UI mode)
S4 of the message-state redesign. Introduces the single source of truth for the
webview's UI mode so it no longer has to be inferred from the tail of clineMessages
(the root of RC1: missing/stuck "Thinking", vanishing approval buttons, footer and
buttons disagreeing).

- Add TurnPhase / TurnState to shared types and ExtensionState.turnState (rides inside
  state_json; no proto change).
- Add TurnStateTracker, owned by SdkController, sharing the one id/seq/epoch authority.
  Each transition stamps a fresh seq so the webview keeps only the newest TurnState.
- Set the phase at the exact lifecycle points where the backend knows it:
    streaming         — initTask / reinit / user responded (resolvePending*)
    awaiting_approval — handleRequestToolApproval (anchored on the ask)
    awaiting_followup — handleAskQuestion; and a turn that ends WITHOUT attempt_completion
    completed         — turn ends and attempt_completion was used
    error             — onSendError; mistake_limit
    resumable         — cancelTask (set before abort)
    idle              — clearTask
  The completed-vs-awaiting_followup decision uses
  MessageTranslatorState.wasAttemptCompletionSeen().
- getStateToPostToWebview now ships turnState in every snapshot.

This is ADDITIVE: the webview does not read turnState yet (S5 wires footer/buttons to
it and deletes the tail heuristics). Classic/legacy paths leave turnState undefined and
keep the legacy behavior. 427 SDK unit tests pass (incl. new turn-state-tracker tests);
tsc and biome clean.
2026-06-01 10:36:03 -07:00
Dominic Cooney db823a5b96 fix(vscode): converge webview transcript via a pure reducer (fixes last-message-missing)
S3 of the message-state redesign. The webview received the same conversation over
two unordered, fire-and-forget channels — incremental partial messages and full
state snapshots — and the state handler REPLACED clineMessages wholesale ("// HACK:
Preserve clineMessages if currentTaskItem is the same"). A stale snapshot captured
before the last message landed could clobber the transcript, dropping the final
message and leaving the UI stuck on "Thinking…" (reproduced earlier on camera).

Introduce a pure convergent-replica reducer (messageReducer.ts) keyed on the three
extension-stamped quantities from S2:
  - ts    : identity / merge key
  - seq   : freshness (higher seq wins for the same ts)
  - epoch : conversation/replica fence (newer replaces, older is dropped, equal merges)

applyMessage / applyStateSnapshot are total and side-effect free:
  - older epoch  -> drop (straggler from a previous task/render)
  - newer epoch  -> replace the transcript wholesale (new task / history load)
  - same epoch   -> merge by ts keeping the higher seq; a snapshot may ADD/UPDATE
                    rows but NEVER truncate, so it can't drop a message the partial
                    stream already delivered. Stale (older stateVersion) snapshots are
                    ignored wholesale.

ExtensionStateContext now feeds both subscription callbacks through the reducer via a
replicaRef, replacing the wholesale-replace HACK and the findLastIndex append. Classic/
legacy state is unstamped (epoch 0 / version 0) and merges by ts exactly as before.

Tests (messageReducer.test.ts): deterministic cases mapping 1:1 to the bugs (stale
snapshot must not shrink the transcript; lower-seq ignored; partial->final in place;
older-epoch straggler dropped; older-version snapshot ignored), PLUS an
order-independence proof — all 120 permutations of a causal log, with duplication and
with non-final drops, converge to the same canonical state. This is the high-confidence
guarantee that the webview cannot get stuck regardless of delivery timing.

62 webview chat tests pass; tsc and biome clean.
2026-06-01 10:36:02 -07:00
Dominic Cooney c17dec92ea refactor(vscode): stamp seq/epoch on messages and state; fire-and-forget delivery
S2 of the message-state redesign. Lays the groundwork for the convergent-replica
webview reducer (S3) so the webview can never get stuck on stale/out-of-order
delivery.

Stamping (extension-owned, synchronous, from the single MessageIdMinter):
- Every ClineMessage flowing to the webview is stamped in SdkMessageCoordinator with
  a fresh `seq` (freshness) and the current `epoch` (conversation/replica fence),
  before it is stored or emitted. The same object references go to both the message
  state handler and the partial-message stream, so both channels carry identical
  stamps. An updated message (partial -> final, same ts) passes through again and
  gets a higher seq, so the freshest copy always wins regardless of arrival order.
- Every state snapshot is stamped in SdkController.getStateToPostToWebview with a
  fresh `stateVersion` (sampled from the same counter) and the current `epoch`.
- `epoch` is bumped at every conversation boundary via a new
  resetMessageTranslatorAndFence() wired into the existing resetMessageTranslator
  sites (task start/clear, history open, reinit, mode rebuild, new-session
  follow-up). iteration_start streaming resets do NOT bump it.

Transport:
- ClineMessage proto gains seq/epoch (fields 24/25) and the conversions carry them.
- ExtensionState gains stateVersion/epoch (ride inside state_json, no proto change).

Fire-and-forget delivery:
- sendPartialMessageEvent and sendStateUpdate no longer await postMessage to the
  webview. A hidden/reloaded/closed webview can make postMessage hang or resolve
  false; awaiting it could stall the backend turn loop. Correctness no longer
  depends on any single delivery — the webview will be a convergent replica (S3).

All new fields are optional/default-0 so the classic/legacy path is unaffected.
Cancel's epoch bump + fence-before-abort and the remaining one-off Date.now() mint
sites are handled in later steps (S6/S2-tail). 423 unit tests pass; tsc clean.
2026-06-01 10:36:02 -07:00
Dominic Cooney a37ab9366d refactor(vscode): unify ClineMessage id minting behind one MessageIdMinter
Message ids (ClineMessage.ts) were minted from Date.now() in two independent
generators: the live message translator (pure ++counter seeded once) and the
interaction coordinator (Math.max(Date.now(), last+1)). Because the translator
counter drifts behind wall-clock, it can later catch up to a clock-based id
minted by the interaction coordinator, producing colliding ids for different
messages. That breaks any merge-by-id scheme on the webview side.

Introduce a single process-wide MessageIdMinter (pure monotonic id/seq/epoch
counters, never reads the clock) owned by MessageTranslatorState and shared by:
- live SDK event translation,
- the interaction coordinator (tool approval / ask_question / user_feedback),
- history rendering (sdkMessagesToClineMessages).

This makes every id globally unique within the process, so regenerated history
ids never overlap live-session ids. Behavior is otherwise unchanged.

Also adds the message-state pipeline design + investigation docs under
src/sdk/docs.

S1 of the message-state redesign; seq/epoch stamping and the remaining one-off
Date.now() mint sites follow in S2.
2026-06-01 10:36:02 -07:00
Dominic Cooney c7bbbda086 docs(vscode): refine sdk-consolidation TODOs for live model fetching
After inspecting the SDK's generic models-URL fetcher
(sdk/packages/core/src/services/providers/model-source.ts
`fetchModelIdsFromSource` + `resolveModelsSourceUrl`), update the TODOs on the
bespoke refresh*Models handlers to capture the real constraint discovered:

- The SDK fetcher exists and is provider-agnostic, but returns model *ids only*
  (unknown ids get placeholder ModelInfo with no real pricing/capabilities).
- `mergeKnownModels` treats a registered `modelsSourceUrl` as the authoritative
  "installed" list (Ollama/LM Studio semantics) and DISCARDS the curated catalog
  when the live fetch returns results.

So simply registering `modelsSourceUrl` for Groq/Baseten/Hicap/HuggingFace/
Vercel/OpenRouter would regress rich model metadata. Proper consolidation needs
an SDK enhancement first (merge-mode or richer per-provider parsing), then the
extension handlers + RPCs can be deleted. refreshGroqModels.ts carries the
detailed note; the others reference it.
2026-06-01 10:36:02 -07:00
Dominic Cooney 9e777c0f4b refactor(vscode): delegate Cline recommended-models fetch to the SDK; TODO others
refreshClineRecommendedModels now delegates the HTTP fetch + response
normalization + offline fallback to the SDK's fetchClineRecommendedModels
(@cline/core), removing ~80 lines of duplicated logic. The extension wrapper
keeps its distinct behavior: the CLINE_RECOMMENDED_MODELS_UPSTREAM feature-flag
gate, the in-memory TTL cache, and in-flight dedup. The proxy-aware fetch
(@/shared/net) and the configured apiBaseUrl are passed through to preserve
network/proxy behavior. The SDK's offline fallback list is identical to
CLINE_RECOMMENDED_MODELS_FALLBACK, so offline behavior is unchanged.

Because the module now imports the ESM-only @cline/core, its unit test moves
from mocha to vitest (joining the other SDK-touching models tests): added to
vitest include + mocha ignore, and fetchClineRecommendedModels added to the
vitest @cline/core stub (and the mocha/integration @cline/core mocks for
completeness). Rewrote the test vitest-native, asserting flag-gate, delegation,
and flag re-check.

Also:
- Add TODO(sdk-consolidation) notes to the remaining bespoke live-model-refresh
  handlers (Groq, Baseten, Hicap, HuggingFace, Vercel AI Gateway, OpenRouter)
  documenting the path to share them with the CLI via the SDK (register
  modelsSourceUrl) and then delete the extension-only handlers + RPCs. These
  are NOT migrated yet because the SDK does not currently live-fetch those
  providers (only ollama/lmstudio register modelsSourceUrl), so deleting them
  today would regress to the curated catalog.
- Remove an unused local type (RuleLoadPart) found while reviewing biome
  noUnusedVariables output.

tsc --noEmit clean; vitest 439 passing.
2026-06-01 10:36:02 -07:00
Dominic Cooney 57df42db8e refactor(vscode): remove dead sapAiCoreModelDescription const
Unused leftover from the deleted SAP AI Core provider handler (found via biome noUnusedVariables). tsc + vitest green.
2026-06-01 10:36:02 -07:00
Dominic Cooney 1f86e4bc37 refactor(vscode): delete genuinely-unused telemetry/auth helpers
- services/telemetry/events/EventHandlerBase.ts: abstract base for telemetry
  event handlers whose concrete subclasses were removed with the classic task
  code; no remaining references (it was the only file left in events/).
- services/auth/AuthServiceMock.ts: test mock with no importers after the
  associated test was deleted.

Note: the rest of services/* (TelemetryService, McpHub, FeatureFlagsService,
ErrorService and their IFoo interfaces) IS live — the interfaces only looked
unreachable to the esbuild oracle because they're consumed via type-only
imports (erased at emit). Verified via importer cross-check; tsc + vitest green.
2026-06-01 10:36:02 -07:00
Dominic Cooney af3d81cf99 refactor(vscode): remove dead cost utils and createOpenAIClient
- Delete src/utils/cost.ts (+ test): calculateApiCostAnthropic/OpenAI/Qwen had
  no consumers left after the provider handlers were removed (only the test
  referenced them).
- Remove createOpenAIClient from src/shared/net.ts (no remaining callers) and
  its now-unused openai + EnvUtils imports. The proxy-aware fetch and
  getAxiosSettings exports remain.

tsc --noEmit clean, vitest 436 passing.
2026-06-01 10:36:02 -07:00
Dominic Cooney 068688d162 chore(vscode): drop npm deps only used by deleted provider handlers
Remove dependencies that became unused after the legacy API provider handlers
and tree-sitter service were deleted (no remaining imports in src, webview, or
build config):

  @anthropic-ai/vertex-sdk, @aws-sdk/client-bedrock-runtime,
  @aws-sdk/credential-providers, @azure/identity,
  @cerebras/cerebras_cloud_sdk, @google-cloud/vertexai, @mistralai/mistralai,
  @sap-ai-sdk/ai-api, @sap-ai-sdk/orchestration, @sap-cloud-sdk/connectivity,
  ollama, tree-sitter-wasms, web-tree-sitter

Also remove the now-dead copyWasmFiles esbuild plugin (it only copied
tree-sitter WASM files for the deleted code-definition service). Kept openai,
@anthropic-ai/sdk, @google/genai, and aws4fetch — still imported by live code.

Verified: extension + standalone esbuild builds succeed, tsc --noEmit clean,
vitest 436 passing.
2026-06-01 10:36:02 -07:00
Dominic Cooney 6f8522e9c4 fix(vscode): drop dead barrel re-exports of deleted files
The previous deletion commit (2c9bd62) removed
core/assistant-message/parse-assistant-message.ts and
core/permissions/CommandPermissionController.ts, but the corresponding barrel
edits (removing their re-exports from index.ts) were dropped by lint-staged's
stash and never committed, leaving HEAD referencing deleted modules. Remove the
dead re-exports so the barrels only export live members.
2026-06-01 10:36:01 -07:00
Dominic Cooney e67f31a684 chore(vscode): add scripts/find-dead-src.mjs dead-code oracle
Computes src files unreachable from the shipped entry points (extension host + standalone host used by JetBrains/CLI) plus webview shared refs, using esbuild metafile reachability. Used to drive the post-SDK-migration dead-code deletions; keep for future pruning. Note: results still need a tsc-gated importer cross-check because esbuild drops import-type-only edges.
2026-06-01 10:36:01 -07:00
Dominic Cooney 6777756a8e refactor(vscode): delete more dead classic code (hooks, permissions, claude-code, misc)
Removes additional source files unreachable from any shipped entry point
(extension host, standalone host, webview) after the SDK migration, verified
via esbuild reachability + a per-file importer cross-check that excludes any
file still referenced by a live non-test survivor or generated/test glue, then
gated on tsc --noEmit + vitest (436 passing):

- services/ripgrep, integrations/notifications, utils/string, utils/tabFiltering
- core/assistant-message/parse-assistant-message
- core/hooks: hook-model-context, notification-hook, precompact-executor,
  PreToolUseHookCancellationError
- core/permissions/CommandPermissionController
- core/workspace/detection
- integrations/claude-code: run, message-filter
- integrations/editor: FileEditProvider, detect-omission
- integrations/misc: extract-file-content, extract-images

Dropped the now-dead re-exports from core/assistant-message/index.ts and
core/permissions/index.ts (those barrels stay; they still export live types
used by generated host glue).
2026-06-01 10:36:01 -07:00
Dominic Cooney 0f30864f8a refactor(vscode): delete dead tree-sitter code-definition service
src/services/tree-sitter/** (language parsers + queries for the classic
list_code_definition_names path) is unreachable from any shipped entry point
after the SDK migration. Verified via scripts/find-dead-src.mjs (esbuild
reachability from extension + standalone entries, plus webview shared refs) and
an importer cross-check (no live value or type importers), gated on
tsc --noEmit + vitest (436 passing).
2026-06-01 10:36:01 -07:00
Dominic Cooney 367446c5d1 refactor(vscode): delete dead classic system-prompt + slash-command code
The classic system-prompt builder (core/prompts/system-prompt/**), the
deep-planning prompt variants, core/prompts/commands.ts, and the
core/slash-commands handler are no longer reachable from any shipped entry
point (extension host, standalone host, or webview) after the SDK migration:
the SDK provides prompt construction (buildClineSystemPrompt) and slash-command
handling. Removed them and their orphaned tests.

Kept core/prompts/responses.ts (still live) and its tests.

Dead-code reachability was computed with scripts/find-dead-src.mjs (esbuild
metafile reachability from src/extension.ts + src/standalone/cline-core.ts,
plus webview src/shared references), then gated on tsc --noEmit + vitest.
2026-06-01 10:36:01 -07:00
Dominic Cooney d2c5e739fb refactor(vscode): delete legacy API provider handlers and dead transforms
Now that buildApiHandler routes through the @cline/llms SDK, the legacy
per-provider handler classes and their supporting code are unused. Remove them:

- apps/vscode/src/core/api/providers/** (all 40+ handler classes, types, tests)
- apps/vscode/src/core/api/transform/** except stream.ts (format/stream/
  tool-call helpers only the handlers used)
- apps/vscode/src/core/api/utils/** (messages/responses API support)
- apps/vscode/src/core/api/retry.ts (+ test)
- apps/vscode/src/shared/sdk-handler-models.ts (getProviderModelFromSdk;
  only the deleted handlers' getModel() used it)

core/api now contains just index.ts (types + SDK re-exports), transform/stream.ts
(ApiStream types still referenced by the local ApiHandler interface), and
adapters/.

Supporting changes to keep everything compiling/working:
- context-window-utils: drop the dead `api instanceof OpenAiHandler` DeepSeek
  branch (handlers are SDK GatewayApiHandlers now); the 64k switch case already
  handles DeepSeek context sizing.
- Relocate fetchLiteLlmModelsInfo from the deleted litellm handler into
  core/controller/models/fetchLiteLlmModels.ts (used by refreshLiteLlmModels).
- Move the `declare module "vscode"` Language Model API augmentation (previously
  carried by the vscode-lm handler) into src/types/vscode-language-model.d.ts so
  live consumers (getVsCodeLmModels, vsCodeSelectorUtils) keep their types.
- Move the @google/genai test mock out of the deleted providers dir to
  src/test/fixtures/google-genai-mock.ts and repoint test-setup.js.
2026-06-01 10:36:01 -07:00
Dominic Cooney 87a0048968 refactor(vscode): route buildApiHandler through the SDK; remove OpenRouter handler
Replace the legacy per-provider buildApiHandler factory with an SDK-backed
handler (apps/vscode/src/sdk/sdk-api-handler.ts) built via @cline/llms
createHandler(). The two standalone callers (commit-message generation and
explain-changes) now import buildApiHandler directly from the SDK module;
@core/api stays types-only (re-exporting SDK types) so it can keep being
imported widely without pulling the SDK runtime graph into activation.

Also:
- Delete the now-dead OpenRouterHandler and its test (createOpenRouterStream
  stays; it is still used by ClineHandler).
- buildSdkProviderConfig never sends both reasoning.effort and
  reasoning.max_tokens (some providers reject it), and supports
  disableReasoning for fast one-shot utility calls; commit-message and
  explain-changes opt in.
- commit-message generation surfaces the real SDK stream error instead of a
  generic "empty API response".
- getGitDiff: run git diff --staged even before the first commit; only gate
  the git diff HEAD fallback on having commits (fixes "no changes" for the
  initial commit of a new repo).
2026-06-01 10:36:01 -07:00
Dominic Cooney 7d7708b1c9 refactor: remove legacy Cline model overrides, wire reasoning effort through SDK
- Delete refreshClineModels.ts, refreshClineModelsRpc.ts, and test
- Remove refreshClineModelsRpc proto RPC
- Remove clineModels state from ExtensionStateContext, StateManager cache, disk
- ClineModelPicker and useOnboardingModels use SDK catalog directly
- Replace ThinkingBudgetSlider with ReasoningEffortSelector for Cline provider
- Use supportsReasoning from SDK catalog instead of hardcoded model names
- Add ProviderReasoningPatch to proto and ProviderConfigPatch contract
- Wire reasoning effort changes through writeProviderConfig to SDK ProviderSettingsManager
- Remove hardcoded Claude switch in openrouter-stream.ts
- Add supportsReasoning to supportsReasoningEffort check in openrouter-stream
2026-06-01 10:36:00 -07:00
Dominic Cooney 05d07e2bd7 fix(vscode): load real @cline/llms in unit-test harness; fix/trim provider tests
The mocha unit-test harness stubbed @cline/llms with an empty catalog, so the SDK-migrated provider handlers (which read the real catalog) failed ~51 tests under the nightly suite.

- src/test/requires.ts now loads the real @cline/llms by resolving its package directory and requiring the ESM entry by absolute path (Node 22 require(esm)), bypassing the package's import-only exports map. This restores real catalog data to provider unit tests.
- vertexModelSupportsGlobalEndpoint: also match legacy ':' context-window/speed suffixes (e.g. claude-opus-4-7:1m), not just '@' snapshot variants.
- refreshClineModels: derive prompt-cache support from reported input_cache_read pricing for any provider, not just openai/google prefixes.
- Delete the failing tests for the classic provider handlers (bedrock cross-region/native-tool-calling, gemini metadata, cline/openrouter qwen cache, wandb unknown-model). These handlers are only reached via buildApiHandler (explain-changes + commit-message generation) and the provider/catalog domain is owned and tested by the SDK; they will be removed when buildApiHandler is retired.
2026-06-01 10:36:00 -07:00
Dominic Cooney 69fa94804c fix(vscode): fix webview provider-model tests and a Windows path test
The nightly publish runs the full webview-ui suite and the Windows extension suite, which surfaced failures the extension-only run does not.

- Harden useProviderModels against a missing providerModelsByProvider map so a partially-mocked ExtensionStateContext no longer crashes the hook.
- Update the SapAiCoreModelPicker and APIOptions specs to provide the provider model-list context the components now read from the SDK catalog, and seed the model ids each test asserts. Removes the now-dead @shared/api sapAiCoreModels mock.
- Compare resolveDataDir() against path.join() instead of a hardcoded POSIX path so the CLINE_DIR fallback test passes on Windows.
2026-06-01 10:36:00 -07:00
Dominic Cooney 04623ebe04 fix(vscode): support OpenAI Compatible provider on the SDK adapter
Selecting the OpenAI Compatible provider failed with 'Unknown provider "openai"', and manually entered model ids were displayed as the catalog default (gpt-4o).

- Map the extension's 'openai' provider id to the SDK's 'openai-compatible' built-in at the SDK boundary (toSdkProviderId), and convert before handing the provider id to core when building a session config.
- Treat openai-compatible as a custom-model-id provider so model resolution honors a user-entered model id instead of coercing it to the catalog default. Adds providerAllowsCustomModelIds() as the shared signal.
- Bump @cline/core, @cline/llms, @cline/shared, @cline/agents to ^0.0.42 (which registers the openai-compatible built-in) and dedupe the dependency tree.
- Carry the tool name on reconstructed Anthropic-format tool_result blocks to satisfy the SDK's ToolResultContent contract.
- Update tests for the refreshed SDK model catalog (Gemini default).
2026-06-01 10:36:00 -07:00
Dominic Cooney b8849c49cd chore(vscode): clean up SDK migration branch (comments, dead code, scree, tests)
- Remove porting scree: docs/sdk-model-catalog/* planning docs, TODO-resume-session.md, and stale doc references in code/comments.

- Rewrite before/after narrative comments in the 'eternal now' style; drop transient 'Step N'/'Phase N' labels.

- Remove leftover [HistoryPerf] diagnostic logging and a dead try/catch rethrow; minor readability/naming.

- Restore .clinerules/network.md (still relevant) and update sdk-migration.md to drop dangling references.

- Fix latent circular-init TDZ in openai-codex-models (lazy catalog build).

- toggleRemoteConfigSetting no longer returns a never-resolving promise.

- Move vitest config into apps/vscode and rename script test:sdk -> test:vitest; wire it into CI.

- Repair/reimplement the SDK-adapter vitest suites (auth-service, provider-migration, sdk-task-history) to match current behavior; all 432 tests pass.
2026-06-01 10:35:14 -07:00
Dominic Cooney 87bf1bf727 fix(vscode): preserve provider model selection fields 2026-06-01 10:35:14 -07:00
Dominic Cooney bd55f2d328 refactor(vscode): source provider model catalogs from @cline/llms SDK (ENG-2116)
Migrates the vscode extension off its hand-curated static model catalogs
in apps/vscode/src/shared/api.ts and on to the @cline/llms SDK as the
single source of truth for provider/model metadata, end-to-end across
the extension host and the webview.

Net impact on the static catalog file:
  apps/vscode/src/shared/api.ts: 5092 -> 468 lines (~90% gone).

What changed at each layer
--------------------------

SDK / catalog plumbing (apps/vscode/src/sdk/model-catalog/):
  - New `ProviderCatalog.peekModels(providerId)` synchronous cache read.
  - `resolveModelInfo` rewritten: committed selection -> catalog peek
    -> await catalog.resolveModels on cache miss. No race with a
    background warmer; if the catalog truly has nothing, returns
    source: "unknown" and the webview renders a neutral loading state.
  - `applyHostModelInfoOverrides` is the canonical seam for the few
    fields the SDK does not yet carry. Today it carries only the Vertex
    `supportsGlobalEndpoint` allowlist (vertex-global-endpoint.ts, with
    a TODO to upstream into the SDK).
  - `ProviderListing` extended with SDK metadata (`is_popular`,
    `popular_rank`, `usage_cost_display`, `capabilities[]`) and plumbed
    through proto + conversion.

Extension-host handlers (apps/vscode/src/core/api/providers/):
  - New shared helper `apps/vscode/src/shared/sdk-handler-models.ts`:
    `getProviderModelFromSdk(providerId, requestedModelId, committedInfo?)`
    returns `{ id, info }` from `getProviderCollectionSync` with
    Vertex global-endpoint overrides applied.
  - 27 handlers converted to a one-liner `getModel()` through that
    helper. Per-handler nuances preserved:
      * Anthropic: strips `:fast` and `:1m` host-side suffixes before
        SDK lookup; carries them back on the returned id so the
        per-request betas still flip.
      * Bedrock: keeps the custom Application Inference Profile ARN
        branch; base-model info from the SDK.
      * Cerebras: keeps the `qwen-3-coder-480b-free` -> `qwen-3-coder-480b`
        paid alias.
      * Qwen / ZAi: SDK has a single catalog each; handlers keep the
        regional base-URL switch but no longer fork the catalog.
      * Wandb: keeps the "unknown id falls through to safe defaults"
        escape hatch via `MODEL_COLLECTIONS_BY_PROVIDER_ID`.

Refresh-models background tasks:
  - `refreshBasetenModels`, `refreshGroqModels`, `refreshHuggingFaceModels`
    source their offline-fallback catalog from the SDK via
    `getProviderCollectionSync` + `adaptSdkModelInfo`. Live fetch path
    unchanged; only the seeding/fallback data changed.

Webview (apps/vscode/webview-ui/):
  - `useNormalizedApiConfiguration` always routes through gRPC
    `resolveModelInfo`. Removed the `isMigratedSdkProvider` /
    `MIGRATED_SDK_PROVIDER_IDS` feature flag and the legacy
    `normalizeApiConfiguration` switch entirely.
  - New `useStaticProviderSelection` hook for the 22 settings
    components whose catalog is now SDK-driven, and
    `useDynamicProviderSelection` for the 12 dynamic-list pickers
    (openrouter, cline, openai-compatible, ollama, lmstudio, requesty,
    litellm, hicap, groq, baseten, huggingface, vercel-ai-gateway,
    aihubmix, oca, huawei-cloud-maas, dify, fireworks, together,
    vscode-lm) so all of them stop calling the legacy switch.
  - `ModelInfoView` reads its `isGemini` check via
    `useProviderModels("gemini")` instead of importing `geminiModels`.
  - `App.stories.tsx` ships a small inline fixture instead of
    importing `bedrockModels`.
  - `ExtensionStateContext` no longer seeds `groqModelsState` /
    `basetenModelsState` from the deleted catalog; the slices start
    empty and the SDK-curated catalog is layered in by the pickers at
    render time.

Misc:
  - `src/utils/model-utils.ts`: `isAnthropicModelId` consults
    `MODEL_COLLECTIONS_BY_PROVIDER_ID["anthropic"]` instead of the
    deleted `anthropicModels` map.
  - `src/shared/storage/provider-keys.ts`: `getProviderDefaultModelId`
    no longer hard-codes 24 per-provider defaults. The function now
    consults the SDK catalog and only keeps an override map for
    providers whose default is intentionally not the SDK default
    (openrouter-shared dynamic providers and local-only providers).
  - `src/shared/openai-codex-models.ts`: relative path for
    `shape-adapter` import so both the extension and webview build
    contexts resolve it identically.

Tests retargeted to assert SDK behavior, not static-map shapes
--------------------------------------------------------------
  - claude-code, anthropic, bedrock, vertex, wandb, provider-keys
    test suites had assertions tied to deleted shapes. Rewrote them
    to either assert through the SDK catalog
    (anthropic compares against `adaptSdkModelInfo(sdkCollection.models[id])`,
    wandb uses the SDK-declared default, etc.) or focus on the
    host-side semantics (bedrock's "global endpoint" block now tests
    `vertexModelSupportsGlobalEndpoint` directly).
  - claude-code test trimmed its 8 `[1m]`/version-pin variants down
    to three SDK-shaped cases. The 8 deleted assertions exercised
    extension-only model-id derivations that the SDK does not carry;
    matching the CLI's behavior was the explicit goal.
  - resolveModelInfo test rewritten around the new peek -> await
    -> unknown contract.
  - proto-lint: added missing `go_package` option to
    `proto/cline/remote_config.proto`.

Verification (npm scripts under apps/vscode/):
  - npm run protos          OK
  - npm run check-types     OK  (apps/vscode + apps/vscode/webview-ui)
  - npm run lint            OK  (biome + proto-lint)
  - npm run build:webview   OK  (tsc -b && vite build)
  - node esbuild.mjs        OK  (dist/extension.js produced)
  - Runtime smoke test: 27/27 provider collections resolve from the
    SDK with correct model counts, defaults, and usage-cost-display
    flags. openai-codex returns cost=hide as expected; every other
    provider returns cost=show.
2026-06-01 10:35:14 -07:00
Max Paulus 🥪 bbdd9d34a8 fix soft-lock on auth fail retry 2026-06-01 10:35:14 -07:00
Max Paulus 🥪 3a9c97b322 fix sesion usubscriptions 2026-06-01 10:35:14 -07:00
Max Paulus 🥪 e17ba43260 instead of listHistory, use host.get(sessionId) instead 2026-06-01 10:35:14 -07:00
Max Paulus 🥪 e5f35422a4 step one for removing src/core/api folder 2026-06-01 10:35:13 -07:00
Max Paulus 🥪 ed3401bfcc remove outdated samples 2026-06-01 10:35:13 -07:00
Max Paulus 🥪 5dfc32daf8 remove old md files 2026-06-01 10:35:13 -07:00
Dominic Cooney a039cded0f ci: run publish-nightly job inside apps/vscode workspace
The publish job ran `npm ci --include=optional` and `npm run publish:marketplace:nightly` at the repo root, but the VS Code extension package and lockfile live at apps/vscode/. The job failed with `npm ci ... can only install with an existing package-lock.json` because there is no root package.json/lockfile anymore.

Mirror ext-vscode-test.yml: set defaults.run.working-directory = apps/vscode for the whole job, and point setup-node cache-dependency-path at the apps/vscode lockfiles. The Checkout and Tag-published-commit steps already use their own explicit working-directory (or are GitHub Actions where it does not apply), so the job-level default does not affect them.
2026-06-01 10:35:13 -07:00
Dominic Cooney fee494cd17 fix(test): stub telemetry helpers in unit-test @cline/core mock
Mirror the integration-test fix from bf147b939 in apps/vscode/src/test/requires.ts so the unit-test @cline/core mock also exposes `createClineTelemetryServiceConfig` and `createConfiguredTelemetryHandle`. Currently only marketplace-filtering.test.ts constructs a Controller, but any future unit test that does so would hit the same `TypeError: createClineTelemetryServiceConfig is not a function` and these two shims should stay in sync.
2026-06-01 10:35:13 -07:00
Max Paulus 🥪 97ccdbbcd2 fix integration tests 2026-06-01 10:35:13 -07:00
Dominic Cooney 28c69b7d9f fix: declare missing direct dependencies in apps/vscode
jwt-decode, js-yaml, and tar are imported by the apps/vscode workspace (jwt-decode in ClineAuthProvider/OCA, js-yaml in frontmatter parsing, tar in scripts/download-ripgrep.mjs) but were not declared in apps/vscode/package.json. They were either dropped when firebase was retired (1335fa545) or never declared in the first place because npm hoisting from a transitive `@sap/xssec`/etc. made them resolve locally. On a clean CI install they did not resolve.

Adds:

  - jwt-decode @ ^4.0.0 (dependencies)

  - js-yaml   @ ^4.1.1 (dependencies, version pinned via existing overrides)

  - tar       @ ^7.5.2 (devDependencies, only used by scripts/download-ripgrep.mjs)

Regenerates package-lock.json via `npm install --package-lock-only` so `npm ci --include=optional` passes.
2026-06-01 10:35:13 -07:00
Dominic Cooney f75044c635 chore: fix lint and format on the vscode app
Lint fixes:

- hook-factory.test.ts: merge duplicate top-level beforeEach hooks (noDuplicateTestHooks)

- system-prompt integration tests: extract shared mockProviderInfo into a new prompt-test-fixtures.ts helper so it is no longer exported from a *.test.ts file (noExportsInTest); update importers accordingly

Format fixes:

- biome format --changed --write across the vscode app; purely cosmetic line-collapse/break adjustments, no semantic changes
2026-06-01 10:34:42 -07:00
Max Paulus 🥪 3b31317881 remove timing code 2026-06-01 10:34:42 -07:00
Max Paulus 🥪 f5b3c4fe4a harden perf improvements 2026-06-01 10:34:42 -07:00
Max Paulus 🥪 5f29ab5953 improve task startup perf 2026-06-01 10:34:42 -07:00
Max Paulus 🥪 51f0bf7b9f add telemetry to sdk extension
Wire SDK/core telemetry into the SDK-backed VS Code session path so
`@cline/core` can emit telemetry events through the SDK
`ITelemetryService` interface, while preserving VS Code and Cline user
opt-out behavior.

- Use the SDK's own telemetry service, not the legacy extension
`TelemetryService`, for SDK/core events.
- Configure the SDK telemetry service using SDK-style OpenTelemetry
environment variables:
  - `OTEL_TELEMETRY_ENABLED`
  - `OTEL_METRICS_EXPORTER`
  - `OTEL_LOGS_EXPORTER`
  - `OTEL_TRACES_EXPORTER`
  - `OTEL_EXPORTER_OTLP_PROTOCOL`
  - `OTEL_EXPORTER_OTLP_ENDPOINT`
  - `OTEL_EXPORTER_OTLP_HEADERS`
  - `OTEL_METRIC_EXPORT_INTERVAL`
- Do not bridge the legacy extension `CLINE_OTEL_*` config into this SDK
telemetry path.
- Do not make the legacy extension `TelemetryService` implement SDK
`ITelemetryService`.
- Do not inject telemetry from `src/sdk/cline-session-factory.ts`; keep
that factory focused on session config construction from
state/provider/model settings.
- Own the shared SDK telemetry handle in `SdkController`, because it
owns the SDK session lifecycle for the VS Code extension.
- Pass the shared telemetry service down through:
  - `SdkController`
  - `SdkSessionLifecycle`
  - `VscodeSessionHost`
  - `ClineCore.create({ telemetry })`
  - `CoreSessionConfig.telemetry` via
`VscodeSessionHost.prepare.applyToStartSessionInput(...)`
- Preserve existing per-session telemetry if remote config or another
prepare step already set `config.telemetry`:

  ```ts
  telemetry: inputWithRemoteConfig.config.telemetry ?? options.telemetry
  ```

The SDK telemetry service must be wrapped by a VS Code/Cline policy
gate.

Ordinary telemetry includes:

- `capture(...)`
- `recordCounter(...)`
- `recordHistogram(...)`
- `recordGauge(...)`

These calls are allowed only when both are true:

1. VS Code/host telemetry is enabled.
2. Cline's `telemetrySetting` is not `"disabled"`.

`"unset"` counts as allowed, matching the existing extension behavior.

`captureRequired(...)` bypasses Cline's `telemetrySetting ===
"disabled"`, but still respects VS Code/host telemetry disabled.

This keeps VS Code's global telemetry setting as the hard privacy gate.

The wrapper starts with host telemetry disabled until
`HostProvider.env.getTelemetrySettings({})` resolves.

This is privacy-conservative: early events are dropped rather than
emitted before VS Code's host telemetry setting is known.

The wrapper also subscribes to
`HostProvider.env.subscribeToTelemetrySettings(...)` so runtime VS Code
telemetry changes are reflected.

- `SdkController` creates the shared SDK telemetry handle once.
- `SdkController.dispose()` disposes the shared SDK telemetry handle.
- `VscodeSessionHost` does not own or dispose the telemetry handle.
- `ClineCore.dispose()` does not dispose the telemetry object it
receives, so passing the shared telemetry service into per-session
`ClineCore` instances is safe.

- [x] Add `src/sdk/sdk-telemetry.ts`.
- [x] In `sdk-telemetry.ts`, create a VS Code SDK telemetry handle using
`createConfiguredTelemetryHandle(createClineTelemetryServiceConfig(...))`.
- [x] In `sdk-telemetry.ts`, add a policy wrapper implementing SDK
`ITelemetryService`.
- [x] Gate ordinary telemetry on both VS Code host telemetry and Cline
`telemetrySetting !== "disabled"`.
- [x] Gate `captureRequired(...)` on VS Code host telemetry only.
- [x] Initialize host telemetry state asynchronously from
`HostProvider.env.getTelemetrySettings({})`, defaulting to disabled
until resolved.
- [x] Subscribe to host telemetry changes via
`HostProvider.env.subscribeToTelemetrySettings(...)`.
- [x] Ensure metadata/common-property mutator methods always delegate to
the underlying SDK telemetry service.
- [x] Ensure `flush()` and `dispose()` delegate to the SDK telemetry
handle and clean up any local subscription state.
- [x] Add a shared SDK telemetry field to `SdkController`.
- [x] Create the shared telemetry handle in `SdkController`
construction.
- [x] Dispose the shared telemetry handle in `SdkController.dispose()`.
- [x] Add `telemetry?: ITelemetryService` to
`SdkSessionLifecycleOptions`.
- [x] Pass telemetry from `SdkController` into `SdkSessionLifecycle`.
- [x] Pass telemetry from `SdkSessionLifecycle` into
`VscodeSessionHost.create(...)`.
- [x] Add `telemetry?: ITelemetryService` to `VscodeSessionHostOptions`.
- [x] Pass `options.telemetry` to `ClineCore.create({ telemetry:
options.telemetry, ... })`.
- [x] In `VscodeSessionHost.prepare.applyToStartSessionInput(...)`, set
`config.telemetry` to existing `config.telemetry` or
`options.telemetry`.
- [x] Remove any telemetry injection from
`src/sdk/cline-session-factory.ts`.
- [x] Add unit coverage for the policy wrapper behavior.
- [x] Add/adjust tests for telemetry propagation through
`SdkSessionLifecycle` and `VscodeSessionHost`.
- [x] Run targeted tests for changed SDK files.
- [x] Run TypeScript validation for the touched paths.

Evidence to collect during implementation:

- SDK `session.started` is emitted through the shared SDK telemetry
service when allowed.
- SDK local runtime events that read `config.telemetry` receive the same
service.
- Ordinary events are dropped when Cline `telemetrySetting` is
`"disabled"`.
- Ordinary and required events are dropped when VS Code host telemetry
is disabled.
- Required events still emit when Cline telemetry is disabled but VS
Code host telemetry is enabled.
- Remote-config-provided `config.telemetry` is preserved and not
overwritten by the VS Code default telemetry service.
2026-06-01 10:34:42 -07:00
Dominic Cooney 7fc6dd3479 fix(mcp): accept CLI-authored nested transport format, preserve oauth/metadata, improve schema error messages
The Cline CLI (cline mcp add) writes servers in a nested transport format:
  { transport: { type, url }, disabled, oauth }

The VSCode extension only accepted the flat format it writes:
  { type, url, disabled, autoApprove }

This caused all MCP servers to silently disappear with a generic
'Invalid MCP settings schema.' error that told users nothing useful.

Changes:
- schemas.ts: Add nestedTransportConfigSchema as the first union arm in
  ServerConfigSchema, placed first so the 'transport:' key acts as an
  unambiguous discriminator. The transform flattens nested -> flat format
  with zero downstream impact (connection logic unchanged).
- schemas.ts: Add oauth and metadata passthrough fields to BaseConfigSchema
  so CLI-written OAuth state and metadata survive round-trips when the
  extension modifies the file (e.g. toggling disabled).
- McpHub.ts: Dramatically improve error messages — include file path,
  per-server breakdown of which fields failed (from Zod error paths), and
  an 'Open Settings File' button for one-click navigation.
- schemas.test.ts: 14 new tests covering nested format, flat format,
  mixed files, oauth/metadata preservation, and error rejection.
2026-06-01 10:34:42 -07:00
Dominic Cooney 10d8dbb884 sdk migration: squashed pre-2026-05-22 work
Collapses the early SDK-migration history (through 2026-05-20) into a single commit. Later commits are preserved individually.
2026-06-01 10:34:42 -07:00
1784 changed files with 51082 additions and 121723 deletions
+121
View File
@@ -0,0 +1,121 @@
# Debug Harness
HTTP-controlled debugger for the VSCode extension at `src/dev/debug-harness/server.ts`.
## Quick start
```bash
# Build extension first if needed (protos + esbuild):
npm run protos && IS_DEV=true node esbuild.mjs
# Launch (skip-build if already built):
npx tsx src/dev/debug-harness/server.ts --skip-build --auto-launch
# In another terminal:
curl localhost:19229/api -d '{"method":"status"}'
```
## Data Isolation
The debugee runs with `CLINE_DIR=~/.cline2` by default, separate from your real `~/.cline`.
This prevents the debugee's logout from logging out the debugger, and vice versa.
Override with `--cline-dir /tmp/test-dir`. Check with `status()``clineDir`.
## Browser Capture & OAuth
The debugee runs with `CLINE_CAPTURE_BROWSER=1`, which intercepts `openExternal()` in
`src/utils/env.ts`. URLs are captured instead of opening a real browser:
- Logged to `$CLINE_DIR/data/debug-captured-urls.jsonl`
- POSTed in real-time to `/captured-url` on the harness server
- Queryable via `oauth.captured_urls`
### OAuth API
- **`oauth.captured_urls`** `{clear?}` — URLs the debugee tried to open
- **`oauth.read_stored_token`** — Check auth token presence in secrets.json
- **`oauth.simulate_callback`** `{path, code?, state?, provider?, token?}` — Build vscode:// callback URI
- **`oauth.read_captured_urls_file`** — Read on-disk JSONL of captured URLs
### OAuth testing flow
For **Cline OAuth** (SDK local callback): The SDK starts a local HTTP server, the auth URL
is captured. To complete: open the captured URL in a real browser (it redirects back to the
SDK's callback server), OR extract the callback port and `curl http://127.0.0.1:PORT/callback?code=...`.
For **MCP/Provider OAuth** (vscode:// URI): The redirect goes to a vscode:// URI. Use
`oauth.simulate_callback` to build it, then inject via `ext.evaluate` calling the URI handler.
## Navigating Views — Use Commands, Not Clicks
Don't try to find/click small sidebar icons. Use VSCode commands via command palette.
Registered in `src/registry.ts`:
| Command | View |
|---------|------|
| `cline.accountButtonClicked` | Account / sign-in |
| `cline.historyButtonClicked` | Task history |
| `cline.settingsButtonClicked` | Settings |
| `cline.mcpButtonClicked` | MCP servers |
| `cline.plusButtonClicked` | New task (chat) |
| `cline.worktreesButtonClicked` | Worktrees |
```bash
curl localhost:19229/api -d '{"method":"ui.command_palette","params":{"command":"cline.accountButtonClicked"}}'
```
## Key commands
All via `POST localhost:19229/api` with `{"method":"...", "params":{...}}`:
- **`launch`** / **`shutdown`** — lifecycle
- **`ui.screenshot`** — screenshot to `/tmp/cline-debug/`; returns `{path}`**use `read_file` on the path to examine, do NOT `open` the file** (Preview.app covers the VSCode window)
- **`ui.open_sidebar`** — open the Cline sidebar
- **`ext.set_breakpoint`** `{file, line, condition?}` — breakpoint by source file (sourcemap-resolved)
- **`ext.evaluate`** `{expression, callFrameId?}` — eval in extension host
- **`ext.resume`** / **`ext.step_over`** / **`ext.step_into`** — stepping
- **`ext.call_stack`** — inspect when paused
- **`web.evaluate`** `{expression}` — eval in webview
- **`web.post_message`** `{message}` — send postMessage to extension host via exposed vsCodeApi
- **`wait_for_pause`** `{timeout?}` — block until breakpoint hit
- **`ui.locator`** `{role?, testId?, text?, frame?}` — Playwright locator (auto-retries on stale sidebar frame)
- **`ui.react_input`** `{text, selector?, clear?, submit?}` — set React textarea value via `execCommand('insertText')`; works reliably across multiple tasks
- **`ui.send_message`** `{text, images?, files?, responseType?}` — send chat message bypassing the textarea entirely (via gRPC postMessage)
- **`ui.command_palette`** `{command}` — run VSCode command
## Typical Session
```bash
# 1. Launch
curl localhost:19229/api -d '{"method":"launch","params":{"skipBuild":true}}'
# 2. Open sidebar + dismiss overlays (ALWAYS do this first)
curl localhost:19229/api -d '{"method":"ui.open_sidebar"}'
curl localhost:19229/api -d '{"method":"web.evaluate","params":{"expression":"document.querySelectorAll(\".sr-only\").forEach(el => el.parentElement?.click())"}}'
# 3. Navigate to view
curl localhost:19229/api -d '{"method":"ui.command_palette","params":{"command":"cline.accountButtonClicked"}}'
# 4. Check captured OAuth URLs if testing auth
curl localhost:19229/api -d '{"method":"oauth.captured_urls"}'
# 5. Verify
curl localhost:19229/api -d '{"method":"ui.screenshot"}'
```
## Caveats
- **⚠️ Dismiss promotional overlays FIRST**: On fresh launches, full-screen promo overlays block the sidebar. **Dismiss immediately after `ui.open_sidebar`**, before any other interaction or screenshot. May need to run twice:
```bash
curl localhost:19229/api -d '{"method": "ui.open_sidebar"}'
curl localhost:19229/api -d '{"method": "web.evaluate", "params": {"expression": "document.querySelectorAll(\".sr-only\").forEach(el => el.parentElement?.click())"}}'
```
- **Screenshots — don't open the file**: `ui.screenshot` and `ui.sidebar_screenshot` save PNGs to `/tmp/cline-debug/` and return the `{path}`. Use `read_file` on that path to examine screenshots. Running `open <path>` launches Preview.app on macOS which covers the VSCode window.
- **Scripts count = 0 after launch**: CDP connects after extension host starts, so scripts parsed during startup aren't tracked. Breakpoints still work via sourcemap resolution.
- **Port 9230**: Extension host inspector. If another VSCode instance uses this port, the harness will fail to connect. Kill other debug instances first.
- **macOS only** for now (Playwright Electron launch behavior).
- **Webview CDP**: `connect_webview` may fail depending on Electron version. `web.evaluate` still works via Playwright's `frame.evaluate()` fallback.
- **Sourcemap paths**: esbuild outputs relative paths like `../src/extension.ts` in the sourcemap. The resolver handles this, but if a file isn't found, use `ext.source_files` to see exact paths.
- **OAuth with fake codes**: Browser capture intercepts the URL but doesn't provide a valid auth code. For real OAuth testing, open the captured URL in a browser. For unit testing, mock the token exchange.
See `src/dev/debug-harness/README.md` for full API reference.
+57 -96
View File
@@ -13,11 +13,55 @@ This file is the secret sauce for working effectively in this codebase. It captu
**What NOT to add:** Stuff you can figure out from reading a few files, obvious patterns, or standard practices. This file should be high-signal, not comprehensive.
## Miscellaneous
- Avoid provider-specific string matching / hardcoded provider branches when fixing provider/config plumbing. Prefer provider metadata, shared catalog/defaults, explicit protocol/client capabilities, or centralized normalization utilities that apply by data shape rather than `providerId === "..."`. If a provider exception seems necessary, stop and explain why instead of adding ad-hoc string matching.
- This is a VS Code extension—check `package.json` for available scripts before trying to verify builds (e.g., `npm run compile`, not `npm run build`).
- When creating PRs, contributors should not create changelog-entry files. Maintainers handle release versioning and changelog curation during the release process.
- When adding new feature flags, see this PR as a reference https://github.com/cline/cline/pull/7566
- Additional instructions about making requests: @.clinerules/network.md
## Searching the Codebase — Avoiding Build Output
Several directories contain build output or generated code that produces
noisy or unusable results with `search_files` / `grep`:
| Directory | What it is | Why it's a problem |
|-----------|-----------|-------------------|
| `out/` | esbuild bundle output | Mirrors `src/` structure as minified JS — every search gets duplicate hits on single-line files |
| `dist/` | Packaged extension | Entire extension bundled into one minified `extension.js` (~1 long line) |
| `dist-standalone/` | Standalone build output | Same minification issue |
| `src/generated/` | Generated protobuf code | Auto-generated from `proto/`; not the source of truth |
| `src/shared/proto/` | Generated proto type defs | Auto-generated from `proto/`; not the source of truth |
| `node_modules/` | Dependencies | Huge, not project source |
### How to skip build output
**`search_files`** — Point at `src/` (not the project root) and use `file_pattern`:
```
search_files(path="src/core", regex="myFunction", file_pattern="*.ts")
```
The `file_pattern` parameter is the most effective filter — e.g. `"*.ts"`,
`"*.tsx"`, `"*.proto"`.
**`grep` directly** — Exclude build dirs and restrict to source extensions:
```bash
grep -rn "myFunction" src/ --include="*.ts" --exclude-dir={out,dist,node_modules,generated}
```
### When you must search minified files
Sometimes you need to verify what got bundled (e.g., checking if a change
made it into the build). Minified files are typically one long line, so
normal `grep` shows the entire file as context. Use these approaches:
- **`grep -oP`** to extract just the match with limited surrounding context:
```bash
grep -oP '.{0,40}myFunction.{0,40}' dist/extension.js
```
- **`read_file`** on files in `out/src/` — these have source maps and are
more readable than `dist/extension.js` (which is the fully bundled output).
- **Source maps** — `out/src/*.js.map` and `dist/extension.js.map` can be
used to trace minified output back to original source locations.
## gRPC/Protobuf Communication
The extension and webview communicate via gRPC-like protocol over VS Code message passing.
@@ -48,102 +92,17 @@ The extension and webview communicate via gRPC-like protocol over VS Code messag
- `src/core/controller/task/explainChanges.ts` - Handler implementation
- `webview-ui/src/components/chat/ChatRow.tsx` - UI rendering
## Adding a New API Provider
When adding a new provider (e.g., "openai-codex"), you must update the proto conversion layer in THREE places or the provider will silently reset to Anthropic:
1. `proto/cline/models.proto` - Add to the `ApiProvider` enum (e.g., `OPENAI_CODEX = 40;`)
2. `convertApiProviderToProto()` in `src/shared/proto-conversions/models/api-configuration-conversion.ts` - Add case mapping string to proto enum
3. `convertProtoToApiProvider()` in the same file - Add case mapping proto enum back to string
**Why this matters:** Without these, the provider string hits the `default` case and returns `ANTHROPIC`. The webview, provider list, and handler all work fine, but the state silently resets when it round-trips through proto serialization. No error is thrown.
**Other files to update when adding a provider:**
- `src/shared/api.ts` - Add to `ApiProvider` union type, define models
- `src/shared/providers/providers.json` - Add to provider list for dropdown
- `src/core/api/index.ts` - Register handler in `createHandlerForProvider()`
- `webview-ui/src/components/settings/utils/providerUtils.ts` - Add cases in `getModelsForProvider()` and `normalizeApiConfiguration()`
- `webview-ui/src/utils/validate.ts` - Add validation case
- `webview-ui/src/components/settings/ApiOptions.tsx` - Render provider component
## Responses API Providers (OpenAI Codex, OpenAI Native)
Providers using OpenAI's Responses API require native tool calling. XML tools don't work with the Responses API.
**Symptoms of broken native tool calling:**
- Tools get called multiple times (e.g., `ask_followup_question` asks the same question twice)
- Tool arguments get duplicated or malformed
- The model responds but tools aren't recognized
**Root causes to check:**
1. **Provider missing from `isNextGenModelProvider()`** in `src/utils/model-utils.ts`. The native variant matchers (e.g., `native-gpt-5/config.ts`) call this function. If your provider isn't in the list, the matcher returns false and falls back to XML tools.
2. **Model missing `apiFormat: ApiFormat.OPENAI_RESPONSES`** in its model info (`src/shared/api.ts`). This property signals that the model requires native tool calling. The task runner in `src/core/task/index.ts` checks this and forces `enableNativeToolCalls: true` regardless of user settings.
**When adding a new Responses API provider:**
1. Add provider to `isNextGenModelProvider()` list in `src/utils/model-utils.ts`
2. Set `apiFormat: ApiFormat.OPENAI_RESPONSES` on all models that use the Responses API
3. The variant matcher and task runner will handle the rest automatically
## Adding Tools to System Prompt
This is tricky—multiple prompt variants and configs. **Always search for existing similar tools first and follow their pattern.** Look at the full chain from prompt definition → variant configs → handler → UI before implementing.
1. **Add to `ClineDefaultTool` enum** in `src/shared/tools.ts`
2. **Tool definition** in `src/core/prompts/system-prompt/tools/` (create file like `generate_explanation.ts`)
- Define variants for each `ModelFamily` (generic, next-gen, xs, etc.)
- Export variants array (e.g., `export const my_tool_variants = [GENERIC, NATIVE_NEXT_GEN, XS]`)
- **Fallback behavior**: If a variant isn't defined for a model family, `ClineToolSet.getToolByNameWithFallback()` automatically falls back to GENERIC. So you only need to export `[GENERIC]` unless the tool needs model-specific behavior.
3. **Register in `src/core/prompts/system-prompt/tools/init.ts`** - Import and spread into `allToolVariants`
4. **Add to variant configs** - Each model family has its own config in `src/core/prompts/system-prompt/variants/*/config.ts`. Add your tool's enum to the `.tools()` list:
- `generic/config.ts`, `next-gen/config.ts`, `gpt-5/config.ts`, `native-gpt-5/config.ts`, `native-gpt-5-1/config.ts`, `native-next-gen/config.ts`, `gemini-3/config.ts`, `glm/config.ts`, `hermes/config.ts`, `xs/config.ts`
- **Important**: If you add to a variant's config, make sure the tool spec exports a variant for that ModelFamily (or relies on GENERIC fallback)
5. **Create handler** in `src/core/task/tools/handlers/`
6. **Wire up in `ToolExecutor.ts`** if needed for execution flow
7. **Add to tool parsing** in `src/core/assistant-message/index.ts` if needed
8. **If tool has UI feedback**: add `ClineSay` enum in proto, update `src/shared/ExtensionMessage.ts`, update `src/shared/proto-conversions/cline-message.ts`, update `webview-ui/src/components/chat/ChatRow.tsx`
## Modifying System Prompt
**Read these first:** `src/core/prompts/system-prompt/README.md`, `tools/README.md`, `__tests__/README.md`
System prompt is modular: **components** (reusable sections) + **variants** (model-specific configs) + **templates** (with `{{PLACEHOLDER}}` resolution).
**Key directories:**
- `components/` - Shared sections: `rules.ts`, `capabilities.ts`, `editing_files.ts`, etc.
- `variants/` - Model-specific: `generic/`, `next-gen/`, `xs/`, `gpt-5/`, `gemini-3/`, `hermes/`, `glm/`, etc.
- `templates/` - Template engine and placeholder definitions
**Variant tiers (ask user which to modify):**
- **Next-gen** (Claude 4, GPT-5, Gemini 2.5): `next-gen/`, `native-next-gen/`, `native-gpt-5/`, `native-gpt-5-1/`, `gemini-3/`, `gpt-5/`
- **Standard** (default fallback): `generic/`
- **Local/small models**: `xs/`, `hermes/`, `glm/`
**How overrides work:** Variants can override components via `componentOverrides` in their `config.ts`, or provide a custom template in `template.ts` (e.g., `next-gen/template.ts` exports `rules_template`). If no override, the shared component from `components/` is used.
**Example: Adding a rule to RULES section**
1. Check if variant overrides rules: look for `rules_template` in `variants/*/template.ts` or `componentOverrides.RULES` in `config.ts`
2. If shared: modify `components/rules.ts`
3. If overridden: modify that variant's template
4. XS variant is special—has heavily condensed inline content in `template.ts`
**After any changes, regenerate snapshots:**
```bash
UPDATE_SNAPSHOTS=true npm run test:unit
```
Snapshots live in `__tests__/__snapshots__/`. Tests validate across model families and context variations (browser, MCP, focus chain).
## Modifying Default Slash Commands
Three places need updates:
- `src/core/slash-commands/index.ts` - Command definitions
- `src/core/prompts/commands.ts` - System prompt integration
- `webview-ui/src/utils/slash-commands.ts` - Webview autocomplete
## Adding New Global State Keys
Adding a new key to global state requires updates in multiple places. Missing any step causes silent failures.
Required steps:
1. Type definition in `src/shared/storage/state-keys.ts` - Add to `GlobalState` or `Settings` interface
2. Add any default value or transform in `src/shared/storage/state-keys.ts` if the key needs one
3. Read and write the value through `StateManager` (`setGlobalState()` / `getGlobalStateKey()`) after initialization
2. Read from globalState in `src/core/storage/utils/state-helpers.ts`:
- Add `const myKey = context.globalState.get<GlobalStateAndSettings["myKey"]>("myKey")` in `readGlobalStateFromDisk()`
- Add to the return object: `myKey: myKey ?? defaultValue,`
3. StateManager handles read/write via `setGlobalState()`/`getGlobalStateKey()` after initialization
Persistent state is file-backed through `StateManager`; do not add new runtime reads or writes against VS Code `ExtensionContext` storage. That storage is only a legacy migration source.
Common mistake: Adding only the return value without the `context.globalState.get()` call. This compiles but the value is always `undefined` on load.
Settings plumbing gotcha: if a key is user-toggleable from settings, wire both controller update paths:
- `src/core/controller/state/updateSettings.ts` for webview `updateSetting(...)`
@@ -157,20 +116,22 @@ Webview toggle gotcha: settings changes must also round-trip back in state paylo
If this round-trip wiring is missing, the backend value can update but the toggle in webview appears stuck or reverts.
## StateManager Cache vs Direct globalState Access
StateManager uses an in-memory cache populated during `StateManager.initialize()` from file-backed storage. For most state, use `controller.stateManager.setGlobalState()`/`getGlobalStateKey()`.
StateManager uses an in-memory cache populated during `StateManager.initialize(context)` in `common.ts`. For most state, use `controller.stateManager.setGlobalState()`/`getGlobalStateKey()`.
Exception: host migration code may read legacy VS Code storage before file-backed storage is initialized.
Exception: State needed immediately at extension startup (before cache is ready)
Example pattern:
When Window A sets state and immediately opens Window B, the new window's StateManager cache is populated from `context.globalState` during initialization. If you need to read state in Window B right at startup (e.g., in `common.ts` during `initialize()`), read directly from `context.globalState.get()` instead of StateManager's cache.
Example pattern (see `lastShownAnnouncementId` and `worktreeAutoOpenPath`):
```typescript
// Writing (normal pattern)
controller.stateManager.setGlobalState("myKey", value)
// Reading after initialization
const value = controller.stateManager.getGlobalStateKey("myKey")
// Reading at startup in common.ts (bypass cache)
const value = context.globalState.get<string>("myKey")
```
Use `context.globalState` only in VS Code migration code that copies legacy ExtensionContext values into the shared file-backed stores.
This is only needed for cross-window state read during the brief startup window before StateManager cache is fully usable. Normal state access after initialization should use StateManager.
## ChatRow Cancelled/Interrupted States
When a ChatRow displays a loading/in-progress state (spinner), you must handle what happens when the task is cancelled. This is non-obvious because cancellation doesn't update the message content—you have to infer it from context.
+26
View File
@@ -0,0 +1,26 @@
# SDK Adapter
The VSCode extension runs on the Cline SDK (`@cline/core`, `@cline/llms`,
`@cline/shared`) through an adapter layer in `apps/vscode/src/sdk/`. The
webview still talks gRPC; the adapter translates between gRPC handlers and SDK
calls. See `apps/vscode/src/dev/debug-harness/README.md` for the debug harness.
## Conventions
1. **Look up SDK APIs, don't guess.** Use `kb_search(name="sdk", query="...")`
before implementing against an SDK surface.
2. **Reference the pre-SDK implementation when replacing a module.** Add a
`// Replaces classic src/core/... (see origin/main)` header and use
`kb_search(name="cline", commit="origin/main")` or
`git show origin/main:path` to consult the prior implementation.
3. **Single entry point.** There is one codepath — the SDK adapter. No
`CLINE_SDK` env flag.
4. **Use `{appBaseUrl}`**, never hardcode `app.cline.bot`.
5. **Avoid `as` casts.** Use explicit conversion functions with tests. The
branded types in `apps/vscode/src/sdk/model-catalog/contracts.ts` exist so
casts are unnecessary outside parse/compute boundaries.
## Debug harness
- **Dismiss the Kanban/promo overlay** before any debug harness interaction.
- **Use the command palette** to navigate tabs in the debug harness.
+1 -1
View File
@@ -44,7 +44,7 @@ For Responses API providers: add to `isNextGenModelProvider()` in `src/utils/mod
Modular: `components/` (shared) + `variants/` (model-specific) + `templates/` (`{{PLACEHOLDER}}`). Variants override components via `componentOverrides` in `config.ts` or custom `template.ts`. XS variant is heavily condensed inline. Always regenerate snapshots after changes.
## Global State Keys (silent failure risk)
Adding a key requires updating the typed storage definitions in `src/shared/storage/state-keys.ts`; runtime reads and writes should go through `StateManager`, not VS Code `ExtensionContext` storage. Persistent state is file-backed so it works across VS Code, CLI, and JetBrains hosts.
Adding a key requires: type in `src/shared/storage/state-keys.ts`, read via `context.globalState.get()` in `src/core/storage/utils/state-helpers.ts` `readGlobalStateFromDisk()`, and add to return object. Missing the `.get()` call compiles fine but value is always `undefined`.
## Slash Commands (3 places)
- `src/core/slash-commands/index.ts` — definitions.
+7 -7
View File
@@ -33,7 +33,7 @@ permissions:
defaults:
run:
working-directory: .
working-directory: sdk
jobs:
publish-main:
@@ -105,12 +105,12 @@ jobs:
PACKAGE_VERSION=$(node -p "require('./apps/cli/package.json').version")
if [ "$PACKAGE_VERSION" != "$VERSION" ]; then
echo "apps/cli/package.json version ${PACKAGE_VERSION} does not match ${TAG}"
echo "sdk/apps/cli/package.json version ${PACKAGE_VERSION} does not match ${TAG}"
exit 1
fi
if ! printf "%s\n" "$VERSION" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$'; then
echo "apps/cli/package.json has invalid version: ${VERSION}"
echo "sdk/apps/cli/package.json has invalid version: ${VERSION}"
exit 1
fi
@@ -147,7 +147,7 @@ jobs:
- name: Build platform binaries
run: bun script/build.ts --install-native-variants --skip-sdk-build
working-directory: apps/cli
working-directory: sdk/apps/cli
env:
TELEMETRY_SERVICE_API_KEY: ${{ secrets.TELEMETRY_SERVICE_API_KEY }}
ERROR_SERVICE_API_KEY: ${{ secrets.ERROR_SERVICE_API_KEY }}
@@ -194,7 +194,7 @@ jobs:
env:
NPM_CONFIG_PROVENANCE: "true"
run: bun script/publish-npm.ts --tag latest
working-directory: apps/cli
working-directory: sdk/apps/cli
- name: Get Previous CLI Tag
id: prev_tag
@@ -375,7 +375,7 @@ jobs:
- name: Build platform binaries
if: steps.check_commits.outputs.skip != 'true'
run: bun script/build.ts --install-native-variants --skip-sdk-build
working-directory: apps/cli
working-directory: sdk/apps/cli
env:
TELEMETRY_SERVICE_API_KEY: ${{ secrets.TELEMETRY_SERVICE_API_KEY }}
ERROR_SERVICE_API_KEY: ${{ secrets.ERROR_SERVICE_API_KEY }}
@@ -424,7 +424,7 @@ jobs:
env:
NPM_CONFIG_PROVENANCE: "true"
run: bun script/publish-npm.ts --tag nightly
working-directory: apps/cli
working-directory: sdk/apps/cli
- name: Summary
if: steps.check_commits.outputs.skip != 'true'
@@ -31,6 +31,9 @@ jobs:
if: github.repository == 'cline/cline' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dpc/sdk-migration-simpler-login')
runs-on: ubuntu-latest
environment: PublishNightly
# The VS Code extension's package.json and lockfiles live under apps/vscode/
# (the repo root has no package.json). Mirror ext-vscode-test.yml so install
# and publish steps run in the correct workspace.
defaults:
run:
working-directory: apps/vscode
@@ -56,14 +59,16 @@ jobs:
# Keep publish environment aligned with test workflow/tooling lockfile expectations.
# Newer LTS (Node 24 / npm 11) can make `npm list` fail with ELSPROBLEMS during vsce packaging.
node-version: 22
cache: 'npm'
cache-dependency-path: |
apps/vscode/package-lock.json
apps/vscode/webview-ui/package-lock.json
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode ci --include=optional
- name: Install root dependencies
run: npm ci --include=optional
- name: Install webview-ui dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/webview-ui ci --include=optional
run: cd webview-ui && npm ci --include=optional
- name: Install Publishing Tools
run: npm install -g @vscode/vsce ovsx
@@ -114,13 +114,11 @@ jobs:
with:
node-version: 22
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode install --include=optional
- name: Install root dependencies
run: npm install --include=optional
- name: Install webview-ui dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/webview-ui install --include=optional
run: cd webview-ui && npm install --include=optional
- name: Install Publishing Tools
run: npm install -g @vscode/vsce ovsx
+3 -5
View File
@@ -128,15 +128,13 @@ jobs:
restore-keys: |
playwright-browsers-${{ runner.os }}-
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode ci
- name: Install root dependencies
run: npm ci
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/webview-ui ci
run: cd webview-ui && npm ci
- name: Install vsce
run: npm install -g @vscode/vsce
+15 -18
View File
@@ -91,15 +91,13 @@ jobs:
apps/vscode/package-lock.json
apps/vscode/webview-ui/package-lock.json
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode ci
- name: Install root dependencies
run: npm ci --include=optional
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/webview-ui ci
run: cd webview-ui && npm ci --include=optional
- name: Run Quality Checks (Parallel)
run: npm run ci:check-all
@@ -132,19 +130,16 @@ jobs:
apps/vscode/package-lock.json
apps/vscode/webview-ui/package-lock.json
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode ci
- name: Install root dependencies
run: npm ci --include=optional
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/webview-ui ci
run: cd webview-ui && npm ci --include=optional
- name: Set up NPM on Windows
if: runner.os == 'Windows'
working-directory: ${{ github.workspace }}
run: |
npm config set script-shell "C:\\Program Files\\Git\\bin\\bash.exe"
@@ -160,6 +155,11 @@ jobs:
id: build_step
run: npm run ci:build
- name: Vitest Suites (SDK adapter + model catalog)
id: vitest_tests
if: ${{ !cancelled() && steps.build_step.outcome == 'success' }}
run: npm run test:vitest
- name: Unit Tests with coverage - Linux
id: unit_tests_linux
if: ${{ !cancelled() && steps.build_step.outcome == 'success' && runner.os == 'Linux' }}
@@ -234,15 +234,13 @@ jobs:
apps/vscode/webview-ui/package-lock.json
apps/vscode/testing-platform/package-lock.json
- name: Install extension dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode ci
- name: Install root dependencies
run: npm ci --include=optional
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Install webview-ui dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/webview-ui ci
run: cd webview-ui && npm ci --include=optional
- name: Download ripgrep binaries
run: npm run download-ripgrep
@@ -251,8 +249,7 @@ jobs:
run: npm run compile-standalone
- name: Install testing platform dependencies
working-directory: ${{ github.workspace }}
run: npm --prefix apps/vscode/testing-platform ci
run: cd testing-platform && npm ci --include=optional
- name: Running testing platform integration spec tests
timeout-minutes: 7
+9 -9
View File
@@ -26,7 +26,7 @@ on:
defaults:
run:
working-directory: .
working-directory: sdk
jobs:
test:
@@ -148,7 +148,7 @@ jobs:
env:
CHANNEL: ${{ steps.channel.outputs.channel }}
run: |
BASE_VERSION=$(node -p "require('./sdk/packages/llms/package.json').version")
BASE_VERSION=$(node -p "require('./packages/llms/package.json').version")
if [ "$CHANNEL" = "nightly" ]; then
TIMESTAMP=$(date +%s)
@@ -166,11 +166,11 @@ jobs:
if: steps.check_commits.outputs.skip != 'true'
env:
VERSION: ${{ steps.version.outputs.version }}
run: bun sdk/scripts/version.ts "$VERSION"
run: bun scripts/version.ts "$VERSION"
- name: Verify publishability
if: steps.check_commits.outputs.skip != 'true'
run: bun sdk/scripts/check-publish.ts
run: bun scripts/check-publish.ts
- name: Prepare package tarball directory
if: steps.check_commits.outputs.skip != 'true'
@@ -187,7 +187,7 @@ jobs:
VERSION: ${{ steps.version.outputs.version }}
run: |
echo "Publishing @cline/shared@${VERSION} with tag '${CHANNEL}'..."
cd sdk/packages/shared
cd packages/shared
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
@@ -199,7 +199,7 @@ jobs:
VERSION: ${{ steps.version.outputs.version }}
run: |
echo "Publishing @cline/llms@${VERSION} with tag '${CHANNEL}'..."
cd sdk/packages/llms
cd packages/llms
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
@@ -211,7 +211,7 @@ jobs:
VERSION: ${{ steps.version.outputs.version }}
run: |
echo "Publishing @cline/agents@${VERSION} with tag '${CHANNEL}'..."
cd sdk/packages/agents
cd packages/agents
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
@@ -223,7 +223,7 @@ jobs:
VERSION: ${{ steps.version.outputs.version }}
run: |
echo "Publishing @cline/core@${VERSION} with tag '${CHANNEL}'..."
cd sdk/packages/core
cd packages/core
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
@@ -235,7 +235,7 @@ jobs:
VERSION: ${{ steps.version.outputs.version }}
run: |
echo "Publishing @cline/sdk@${VERSION} with tag '${CHANNEL}'..."
cd sdk/packages/sdk
cd packages/sdk
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
+4 -4
View File
@@ -21,7 +21,7 @@ permissions:
defaults:
run:
working-directory: .
working-directory: sdk
jobs:
quality-checks:
@@ -96,12 +96,12 @@ jobs:
- name: Run SDK Tests (Windows)
if: ${{ !cancelled() && steps.build_sdk_step.outcome == 'success' && steps.build_cli_step.outcome == 'success' && matrix.os == 'windows-latest' }}
run: bun -F './sdk/packages/**' test
run: bun -F './packages/**' test
- name: Smoke test SQLite under Node
if: ${{ !cancelled() && steps.build_sdk_step.outcome == 'success' && matrix.os != 'windows-latest' }}
timeout-minutes: 10
run: bun sdk/scripts/ci-node-smoke.ts
run: bun scripts/ci-node-smoke.ts
- name: Run TUI e2e tests
if: ${{ !cancelled() && steps.build_sdk_step.outcome == 'success' && steps.build_cli_step.outcome == 'success' && matrix.os == 'ubuntu-latest' && matrix.node-version == '24.x' }}
@@ -109,4 +109,4 @@ jobs:
- name: Verify packages are publishable
if: ${{ !cancelled() && steps.build_sdk_step.outcome == 'success' && steps.build_cli_step.outcome == 'success' && matrix.os == 'ubuntu-latest' && matrix.node-version == '24.x' }}
run: bun sdk/scripts/check-publish.ts
run: bun scripts/check-publish.ts
+3 -11
View File
@@ -13,6 +13,9 @@ tmp
pnpm-lock.yaml
.clineignore
.cline/enterprise
.cline/remote-config
**/.cline/remote-config
.venv
.actrc
CLAUDE.local.md
@@ -61,17 +64,6 @@ tests/**/cache
# Should never be committed: only exists if a publish aborts mid-swap.
.README.github.bak
# Tauri generated code
apps/*/src-tauri/gen
apps/*/src-tauri/bin
apps/examples/*/src-tauri/gen
apps/examples/*/src-tauri/bin
# Tauri UI test snapshots
apps/*/src/tests/.tui-test
apps/*/src/tests/tui-traces
apps/vscode/webview-ui/src/**/*.js
apps/vscode/webview-ui/src/**/*.js.map
# SDK Session files / User data
.cline/data
+1 -10
View File
@@ -1,10 +1 @@
if ! command -v gitleaks >/dev/null 2>&1; then
echo "gitleaks is required for the pre-commit secret scan."
echo "Install it with: brew install gitleaks"
echo "Other install options: https://github.com/gitleaks/gitleaks#installing"
exit 1
fi
gitleaks git --pre-commit --redact --staged --verbose || exit 1
lint-staged
cd apps/vscode && lint-staged
+21 -22
View File
@@ -5,8 +5,8 @@
"tasks": [
{
"label": "compile-standalone",
"type": "shell",
"command": "npm run compile-standalone",
"type": "npm",
"script": "compile-standalone",
"group": "build",
"problemMatcher": [],
"presentation": {
@@ -18,8 +18,8 @@
},
{
"label": "npm: protos",
"type": "shell",
"command": "npm run protos",
"type": "npm",
"script": "protos",
"problemMatcher": [],
"isBackground": false,
"presentation": {
@@ -64,8 +64,8 @@
"group": "build"
},
{
"type": "shell",
"command": "npm run build:webview",
"type": "npm",
"script": "build:webview",
"group": "build",
"problemMatcher": [],
"isBackground": true,
@@ -85,8 +85,8 @@
}
},
{
"type": "shell",
"command": "npm run build:webview:test",
"type": "npm",
"script": "build:webview:test",
"group": "build",
"problemMatcher": [],
"isBackground": true,
@@ -107,8 +107,8 @@
}
},
{
"type": "shell",
"command": "npm run dev:webview",
"type": "npm",
"script": "dev:webview",
"group": "build",
"problemMatcher": [
{
@@ -144,8 +144,8 @@
}
},
{
"type": "shell",
"command": "npm run watch:esbuild",
"type": "npm",
"script": "watch:esbuild",
"group": "build",
"problemMatcher": {
"pattern": [
@@ -183,8 +183,8 @@
}
},
{
"type": "shell",
"command": "npm run watch:esbuild:test",
"type": "npm",
"script": "watch:esbuild:test",
"group": "build",
"problemMatcher": {
"pattern": [
@@ -223,8 +223,8 @@
}
},
{
"type": "shell",
"command": "npm run watch:tsc",
"type": "npm",
"script": "watch:tsc",
"group": "build",
"problemMatcher": "$tsc-watch",
"isBackground": true,
@@ -241,9 +241,8 @@
}
},
{
"type": "shell",
"command": "npm run watch-tests",
"label": "npm: watch-tests",
"type": "npm",
"script": "watch-tests",
"problemMatcher": "$tsc-watch",
"isBackground": true,
"dependsOn": [
@@ -281,8 +280,8 @@
"command": "rm -rf ${workspaceFolder}/apps/vscode/dist/tmp/user && mkdir -p ${workspaceFolder}/apps/vscode/dist/tmp/user"
},
{
"type": "shell",
"command": "npm run storybook",
"type": "npm",
"script": "storybook",
"group": "build",
"problemMatcher": [],
"isBackground": false,
@@ -309,7 +308,7 @@
"$tsc"
],
"options": {
"cwd": "${workspaceFolder}"
"cwd": "${workspaceFolder}/sdk"
}
}
],
-25
View File
@@ -1,30 +1,5 @@
# Changelog
## [3.88.0]
### Added
- Add the latest Fireworks AI serverless models and update the default Fireworks model to Kimi K2.6.
### Fixed
- Fix MCP server delete/add flows so settings writes do not cause the MCP server list to be emptied by the file watcher.
- Remove stale Fireworks AI models and correct Fireworks model metadata and cache pricing.
### Changed
- Always use the upstream Cline recommended models endpoint instead of gating it behind a feature flag.
## [3.87.0]
### Added
- Add MiniMax M3 model support.
### Fixed
- Update VS Code extension dependencies to resolve security issues in `@xmldom/xmldom`, `basic-ftp`, `axios`, `undici`, and other direct/transitive packages.
## [3.86.2]
### Fixed
-1
View File
@@ -1,2 +1 @@
@.clinerules/general.md
@.clinerules/network.md
+3 -7
View File
@@ -51,7 +51,7 @@ for CI/CD and scripting.
npm i -g cline
```
<a href="./apps/cli/README.md">Learn more</a>
<a href="./sdk/apps/cli/README.md">Learn more</a>
<br><br>
</td>
@@ -129,7 +129,7 @@ npm install @cline/sdk
| Product | Description | Location | CHANGELOG |
|---------|------------|--------------|--------------|
| **SDK** | Node.js programmatic agent API and extension exports. | [`sdk/`](https://github.com/cline/cline/tree/main/sdk) | [CHANGELOG.md](https://github.com/cline/cline/blob/main/sdk/CHANGELOG.md) |
| **CLI** | Terminal UI, headless mode, shell commands, and CLI-specific flows. | [`apps/cli/`](https://github.com/cline/cline/tree/main/apps/cli) | [CHANGELOG.md](https://github.com/cline/cline/blob/main/apps/cli/CHANGELOG.md) |
| **CLI** | Terminal UI, headless mode, shell commands, and CLI-specific flows. | [`sdk/apps/cli/`](https://github.com/cline/cline/tree/main/sdk/apps/cli) | [CHANGELOG.md](https://github.com/cline/cline/blob/main/sdk/apps/cli/CHANGELOG.md) |
| **VS Code Extension** | The Marketplace extension and extension host integration. | [`/`](https://github.com/cline/cline/tree/main) (WIP migrating) | [CHANGELOG.md](https://github.com/cline/cline/blob/main/CHANGELOG.md) |
| **JetBrains Plugin** | JetBrains-hosted client that talks to the shared agent core. | Currently we are not open-sourcing JetBrains plugins | - |
| **Kanban** | Web-based multi-agent task board. | [`cline/kanban`](https://github.com/cline/kanban) | [CHANGELOG.md](https://github.com/cline/kanban/blob/main/CHANGELOG.md) |
@@ -212,12 +212,8 @@ cline schedule create "PR summary" \
Chat with your agent from any messaging platform: Telegram, Slack, Discord, Google Chat, WhatsApp, and Linear. Each conversation thread maps to an agent session with full context. Set up access control to restrict who can interact with your agent.
```bash
# Connect to Telegram
cline connect telegram -k $BOT_TOKEN
# Connect to Slack through webhook
cline connect slack --bot-token $SLACK_TOKEN --signing-secret $SECRET --base-url $URL
# Connect to Slack using socket mode
cline connect slack --bot-token $SLACK_TOKEN --app-token $SLACK_APP_TOKEN
cline connect slack --token $SLACK_TOKEN --signing-secret $SECRET --base-url $URL
```
## Headless CLI for CI/CD
-12
View File
@@ -1,12 +0,0 @@
{
"$schema": "https://biomejs.dev/schemas/2.4.5/schema.json",
"root": false,
"extends": ["../sdk/biome.json"],
"linter": {
"rules": {
"a11y": {
"noStaticElementInteractions": "warn"
}
}
}
}
-184
View File
@@ -1,184 +0,0 @@
import { mkdirSync, mkdtempSync } from "node:fs";
import { arch, platform, tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { runDashboardCommand, waitForProcessShutdown } from "./dashboard";
const ENV_KEYS = [
"WORKSPACE_ROOT",
"HOST",
"CLINE_HUB_DASHBOARD_PORT",
"PUBLIC_URL",
"ROOM_SECRET",
"CLINE_HUB_WEBVIEW_DIST_DIR",
"CLINE_WRAPPER_PATH",
] as const;
const originalEnv = Object.fromEntries(
ENV_KEYS.map((key) => [key, process.env[key]]),
);
afterEach(() => {
for (const key of ENV_KEYS) {
const value = originalEnv[key];
if (value === undefined) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
});
describe("runDashboardCommand", () => {
it("starts the dashboard server, opens the invite URL, and waits for shutdown", async () => {
const output: string[] = [];
const errors: string[] = [];
const opened: string[] = [];
const stop = vi.fn();
let observedEnv:
| {
workspaceRoot: string | undefined;
host: string | undefined;
port: string | undefined;
publicUrl: string | undefined;
roomSecret: string | undefined;
webviewDistDir: string | undefined;
}
| undefined;
const webviewDistDir = mkdtempSync(join(tmpdir(), "cline-webview-dist-"));
mkdirSync(webviewDistDir, { recursive: true });
process.env.CLINE_HUB_WEBVIEW_DIST_DIR = webviewDistDir;
const exitCode = await runDashboardCommand({
cwd: "sdk",
host: "127.0.0.1",
port: "9090",
publicUrl: "http://127.0.0.1:9090",
roomSecret: "secret",
io: {
writeln: (text) => output.push(text ?? ""),
writeErr: (text) => errors.push(text),
},
startServer: async () => {
observedEnv = {
workspaceRoot: process.env.WORKSPACE_ROOT,
host: process.env.HOST,
port: process.env.CLINE_HUB_DASHBOARD_PORT,
publicUrl: process.env.PUBLIC_URL,
roomSecret: process.env.ROOM_SECRET,
webviewDistDir: process.env.CLINE_HUB_WEBVIEW_DIST_DIR,
};
return {
listenUrl: "http://127.0.0.1:9090/",
publicUrl: "http://127.0.0.1:9090",
inviteUrl: "http://127.0.0.1:9090/?roomSecret=secret",
hubUrl: "ws://127.0.0.1:25463/hub",
stop,
};
},
openUrl: async (url) => {
opened.push(url);
},
waitForShutdown: async (server) => {
await server.stop();
},
});
expect(exitCode).toBe(0);
expect(observedEnv).toEqual({
workspaceRoot: resolve("sdk"),
host: "127.0.0.1",
port: "9090",
publicUrl: "http://127.0.0.1:9090",
roomSecret: "secret",
webviewDistDir,
});
expect(opened).toEqual(["http://127.0.0.1:9090/?roomSecret=secret"]);
expect(stop).toHaveBeenCalledTimes(1);
expect(output.join("\n")).toContain("Cline dashboard listening at");
expect(output.join("\n")).toContain("ws://127.0.0.1:25463/hub");
expect(errors).toEqual([]);
expect(process.env.WORKSPACE_ROOT).toBe(originalEnv.WORKSPACE_ROOT);
expect(process.env.CLINE_HUB_WEBVIEW_DIST_DIR).toBe(webviewDistDir);
});
it("honors --no-open behavior", async () => {
const openUrl = vi.fn();
const exitCode = await runDashboardCommand({
openBrowser: false,
io: {
writeln: () => {},
writeErr: () => {},
},
startServer: async () => ({
listenUrl: "http://127.0.0.1:8787/",
publicUrl: "http://127.0.0.1:8787",
inviteUrl: "http://127.0.0.1:8787",
stop: vi.fn(),
}),
openUrl,
waitForShutdown: async () => {},
});
expect(exitCode).toBe(0);
expect(openUrl).not.toHaveBeenCalled();
});
it("finds webview assets from the published wrapper package layout", async () => {
const root = mkdtempSync(join(tmpdir(), "cline-wrapper-layout-"));
const wrapperPath = join(root, "node_modules", "cline", "bin", "cline");
const platformName = platform() === "win32" ? "windows" : platform();
const webviewDistDir = join(
root,
"node_modules",
"cline",
"node_modules",
"@cline",
`cli-${platformName}-${arch()}`,
"cline-hub",
"webview",
);
mkdirSync(join(wrapperPath, ".."), { recursive: true });
mkdirSync(webviewDistDir, { recursive: true });
process.env.CLINE_WRAPPER_PATH = wrapperPath;
delete process.env.CLINE_HUB_WEBVIEW_DIST_DIR;
let observedWebviewDistDir: string | undefined;
const exitCode = await runDashboardCommand({
openBrowser: false,
io: {
writeln: () => {},
writeErr: () => {},
},
startServer: async () => {
observedWebviewDistDir = process.env.CLINE_HUB_WEBVIEW_DIST_DIR;
return {
listenUrl: "http://127.0.0.1:8787/",
publicUrl: "http://127.0.0.1:8787",
inviteUrl: "http://127.0.0.1:8787",
stop: vi.fn(),
};
},
waitForShutdown: async () => {},
});
expect(exitCode).toBe(0);
expect(observedWebviewDistDir).toBe(webviewDistDir);
});
it("settles shutdown when server stop rejects", async () => {
const shutdown = waitForProcessShutdown({
listenUrl: "http://127.0.0.1:8787/",
publicUrl: "http://127.0.0.1:8787",
inviteUrl: "http://127.0.0.1:8787",
stop: vi.fn(async () => {
throw new Error("stop failed");
}),
});
process.emit("SIGINT", "SIGINT");
await expect(shutdown).rejects.toThrow("stop failed");
});
});
-195
View File
@@ -1,195 +0,0 @@
import { existsSync } from "node:fs";
import { arch, platform } from "node:os";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import open from "open";
import { c } from "../utils/output";
export interface DashboardServerHandle {
listenUrl: string;
publicUrl: string;
inviteUrl: string;
hubUrl?: string;
stop: () => void | Promise<void>;
}
interface DashboardCommandIo {
writeln: (text?: string) => void;
writeErr: (text: string) => void;
}
export interface RunDashboardCommandOptions {
cwd?: string;
host?: string;
port?: string;
publicUrl?: string;
roomSecret?: string;
openBrowser?: boolean;
io: DashboardCommandIo;
startServer?: () => Promise<DashboardServerHandle>;
openUrl?: (url: string) => Promise<void>;
waitForShutdown?: (server: DashboardServerHandle) => Promise<void>;
}
const DASHBOARD_PORT_ENV = "CLINE_HUB_DASHBOARD_PORT";
const WEBVIEW_DIST_ENV = "CLINE_HUB_WEBVIEW_DIST_DIR";
function setEnvValue(name: string, value: string | undefined): () => void {
const previous = process.env[name];
if (value === undefined) {
return () => {};
}
process.env[name] = value;
return () => {
if (previous === undefined) {
delete process.env[name];
} else {
process.env[name] = previous;
}
};
}
async function withDashboardEnvironment<T>(
options: RunDashboardCommandOptions,
fn: () => Promise<T>,
): Promise<T> {
const restore = [
setEnvValue(
"WORKSPACE_ROOT",
options.cwd ? resolve(options.cwd) : undefined,
),
setEnvValue("HOST", options.host),
setEnvValue(DASHBOARD_PORT_ENV, options.port),
setEnvValue("PUBLIC_URL", options.publicUrl),
setEnvValue("ROOM_SECRET", options.roomSecret),
setEnvValue(WEBVIEW_DIST_ENV, resolveDefaultWebviewDistDir()),
];
try {
return await fn();
} finally {
for (let i = restore.length - 1; i >= 0; i--) {
restore[i]?.();
}
}
}
function resolveDefaultWebviewDistDir(): string | undefined {
if (process.env[WEBVIEW_DIST_ENV]?.trim()) {
return undefined;
}
const moduleDir = dirname(fileURLToPath(import.meta.url));
const candidates = [
...resolveInstalledPlatformPackageWebviewCandidates(),
// Source checkout: apps/cli/src/commands/dashboard.ts
join(moduleDir, "../../../cline-hub/dist/webview"),
// Node bundle: apps/cli/dist/index.js
join(moduleDir, "cline-hub/webview"),
// Compiled platform package: apps/cli/dist/<platform>/bin/cline
join(dirname(process.execPath), "../cline-hub/webview"),
];
return candidates.find((candidate) => existsSync(candidate));
}
function resolveInstalledPlatformPackageWebviewCandidates(): string[] {
const packageName = resolvePlatformPackageName();
const starts = [
process.env.CLINE_WRAPPER_PATH
? dirname(process.env.CLINE_WRAPPER_PATH)
: undefined,
dirname(process.execPath),
].filter((value): value is string => !!value?.trim());
const candidates: string[] = [];
for (const start of starts) {
let current = start;
for (;;) {
candidates.push(
join(current, "node_modules", packageName, "cline-hub/webview"),
);
const parent = dirname(current);
if (parent === current) break;
current = parent;
}
}
return candidates;
}
function resolvePlatformPackageName(): string {
const platformName = platform() === "win32" ? "windows" : platform();
return `@cline/cli-${platformName}-${arch()}`;
}
async function startDefaultDashboardServer(): Promise<DashboardServerHandle> {
const { startClineHubDashboardServer } = await import("@cline/cline-hub");
return await startClineHubDashboardServer();
}
async function openDefaultUrl(url: string): Promise<void> {
await open(url, { wait: false });
}
export function waitForProcessShutdown(
server: DashboardServerHandle,
): Promise<void> {
return new Promise<void>((resolveShutdown, rejectShutdown) => {
let settled = false;
const cleanup = () => {
process.off("SIGINT", handleSignal);
process.off("SIGTERM", handleSignal);
};
const stop = async () => {
if (settled) return;
settled = true;
cleanup();
try {
await server.stop();
resolveShutdown();
} catch (error) {
rejectShutdown(error);
}
};
function handleSignal() {
void stop();
}
process.on("SIGINT", handleSignal);
process.on("SIGTERM", handleSignal);
});
}
export async function runDashboardCommand(
options: RunDashboardCommandOptions,
): Promise<number> {
try {
const server = await withDashboardEnvironment(options, () =>
(options.startServer ?? startDefaultDashboardServer)(),
);
const dashboardUrl =
server.inviteUrl || server.publicUrl || server.listenUrl;
options.io.writeln(
`${c.green}Cline dashboard listening at${c.reset} ${dashboardUrl}`,
);
if (server.hubUrl) {
options.io.writeln(`${c.dim}Hub endpoint: ${server.hubUrl}${c.reset}`);
}
if (options.openBrowser !== false) {
try {
await (options.openUrl ?? openDefaultUrl)(dashboardUrl);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
options.io.writeErr(`Failed to open browser: ${message}`);
}
}
await (options.waitForShutdown ?? waitForProcessShutdown)(server);
return 0;
} catch (error) {
options.io.writeErr(error instanceof Error ? error.message : String(error));
return 1;
}
}
-180
View File
@@ -1,180 +0,0 @@
import { Llms } from "@cline/core";
import type { ChoiceContext } from "@opentui-ui/dialog";
import type { DialogActions } from "@opentui-ui/dialog/react";
import { useCallback, useMemo } from "react";
import type {
InteractiveConfigData,
InteractiveConfigItem,
InteractiveConfigTab,
LoadInteractiveConfigDataOptions,
} from "../../tui/interactive-config";
import type { CliCompactionMode, Config } from "../../utils/types";
import {
ConfigErrorContent,
DeleteConfigItemConfirmContent,
ExtDetailContent,
} from "../components/dialogs/config-dialogs";
import { withLoadingDialog } from "../components/dialogs/loading-dialog";
import { ConfigPanelContent } from "../views/config-view";
import type { ConfigAction } from "../views/config-view-helpers";
import type { OpenModelSelectorOptions } from "./use-model-selector";
export interface OpenConfigOptions {
initialTab?: InteractiveConfigTab;
}
export function useConfigPanel(opts: {
dialog: DialogActions;
config: Config;
sessionUiMode: string;
compactionMode: CliCompactionMode;
toggleMode: () => void;
toggleAutoApprove: () => void;
setCompactionMode: (mode: CliCompactionMode) => void;
termHeight: number;
loadConfigData: (
options?: LoadInteractiveConfigDataOptions,
) => Promise<InteractiveConfigData>;
onToggleConfigItem?: (
item: InteractiveConfigItem,
options?: LoadInteractiveConfigDataOptions,
) => Promise<InteractiveConfigData | undefined>;
onDeleteConfigItem?: (
item: InteractiveConfigItem,
options?: LoadInteractiveConfigDataOptions,
) => Promise<InteractiveConfigData | undefined>;
openModelSelector: (options?: OpenModelSelectorOptions) => Promise<void>;
openMcpManager: (options?: { refocus?: boolean }) => Promise<boolean>;
refocusTextarea: () => void;
}) {
const emptyConfigData = useMemo(
() => ({
workflows: [] as InteractiveConfigItem[],
rules: [] as InteractiveConfigItem[],
skills: [] as InteractiveConfigItem[],
hooks: [] as InteractiveConfigItem[],
agents: [] as InteractiveConfigItem[],
plugins: [] as InteractiveConfigItem[],
mcp: [] as InteractiveConfigItem[],
tools: [] as InteractiveConfigItem[],
workflowSlashCommands: [],
}),
[],
);
const openConfig = useCallback(
async (options: OpenConfigOptions = {}) => {
let keepOpen = true;
let activeTab = options.initialTab;
while (keepOpen) {
const [data, providerInfo] = await withLoadingDialog(
opts.dialog,
"Loading settings...",
async () =>
await Promise.all([
opts
.loadConfigData({ includePluginTools: false })
.catch(() => emptyConfigData),
Llms.getProvider(opts.config.providerId).catch(() => undefined),
]),
);
const providerDisplayName =
providerInfo?.name ?? opts.config.providerId;
const action = await opts.dialog.choice<ConfigAction>({
size: "large",
style: { maxHeight: opts.termHeight - 2 },
closeOnEscape: false,
content: (ctx: ChoiceContext<ConfigAction>) => (
<ConfigPanelContent
{...ctx}
config={opts.config}
configData={data}
loadConfigData={opts.loadConfigData}
providerDisplayName={providerDisplayName}
currentMode={opts.sessionUiMode}
currentCompactionMode={opts.compactionMode}
initialTab={activeTab}
onActiveTabChange={(tab) => {
activeTab = tab;
}}
onToggleConfigItem={opts.onToggleConfigItem}
onDeleteConfigItem={opts.onDeleteConfigItem}
onToggleMode={opts.toggleMode}
onToggleAutoApprove={opts.toggleAutoApprove}
onSetCompactionMode={opts.setCompactionMode}
/>
),
});
if (!action) {
keepOpen = false;
continue;
}
if (action.kind === "open-provider") {
await opts.openModelSelector({
startWithProviderChange: true,
onCancel: () => {},
});
} else if (action.kind === "open-model") {
await opts.openModelSelector({ onCancel: () => {} });
} else if (action.kind === "toggle-item") {
await opts.onToggleConfigItem?.(action.item);
} else if (action.kind === "delete-item") {
const confirmed = await opts.dialog.choice<boolean>({
closeOnEscape: true,
content: (ctx: ChoiceContext<boolean>) => (
<DeleteConfigItemConfirmContent {...ctx} item={action.item} />
),
});
if (confirmed && opts.onDeleteConfigItem) {
try {
await withLoadingDialog(
opts.dialog,
`Deleting ${action.item.name}...`,
async () =>
await opts.onDeleteConfigItem?.(action.item, {
includePluginTools: false,
}),
);
} catch (error) {
await opts.dialog.choice<void>({
closeOnEscape: true,
content: (ctx: ChoiceContext<void>) => (
<ConfigErrorContent
{...ctx}
title="Plugin delete failed"
message={
error instanceof Error ? error.message : String(error)
}
/>
),
});
}
}
} else if (action.kind === "ext-detail") {
await opts.dialog.choice<void>({
style: { maxHeight: opts.termHeight - 2 },
closeOnEscape: false,
content: (ctx: ChoiceContext<void>) => (
<ExtDetailContent
{...ctx}
item={action.item}
onToggleConfigItem={opts.onToggleConfigItem}
/>
),
});
} else if (action.kind === "open-mcp") {
const changed = await opts.openMcpManager({ refocus: false });
if (changed) {
keepOpen = false;
}
}
}
opts.refocusTextarea();
},
[opts, emptyConfigData],
);
return openConfig;
}
@@ -1,81 +0,0 @@
import { describe, expect, it } from "vitest";
import { PLATFORMS, shouldIncludeField } from "./platforms";
describe("connect wizard platform security fields", () => {
it("does not ask Telegram users to re-enter the bot username", () => {
const telegram = PLATFORMS.find((platform) => platform.id === "telegram");
expect(telegram?.fields.map((field) => field.label)).toEqual(["Bot token"]);
expect(telegram?.fields.map((field) => field.flag)).toEqual(["-k"]);
});
it("rejects unsafe Telegram and Slack access restriction identifiers", () => {
const telegram = PLATFORMS.find((platform) => platform.id === "telegram");
const slack = PLATFORMS.find((platform) => platform.id === "slack");
const telegramUser = telegram?.security?.fields.find(
(field) => field.key === "userId",
);
const slackTeam = slack?.security?.fields.find(
(field) => field.key === "teamId",
);
const slackUser = slack?.security?.fields.find(
(field) => field.key === "userId",
);
expect(telegramUser?.validate?.("123456")).toBeUndefined();
expect(telegramUser?.validate?.("123; rm -rf /")).toContain("digits");
expect(slackTeam?.validate?.("T01ABC123")).toBeUndefined();
expect(slackTeam?.validate?.("T01;bad")).toContain("Slack workspace");
expect(slackUser?.validate?.("U01ABC123")).toBeUndefined();
expect(slackUser?.validate?.("U01$(bad)")).toContain("Slack member");
});
it("uses the Telegram allowed user ID flag for wizard security", () => {
const telegram = PLATFORMS.find((platform) => platform.id === "telegram");
const args = telegram?.security?.buildArgs({
userId: "123456",
});
expect(args).toEqual(["--allowed-user-id", "123456"]);
});
it("builds an exact-match Slack authorization hook", () => {
const slack = PLATFORMS.find((platform) => platform.id === "slack");
const args = slack?.security?.buildArgs({
teamId: "T01ABC123",
userId: "U01ABC123",
});
expect(args).toEqual([
"--hook-command",
`jq -r ".payload.actor.participantKey" | grep -qx "slack:team:T01ABC123:user:U01ABC123" && echo '{"action":"allow"}' || echo '{"action":"deny"}'`,
]);
});
it("asks Slack users for mode-specific setup fields", () => {
const slack = PLATFORMS.find((platform) => platform.id === "slack");
const fields = slack?.fields ?? [];
const webhookValues = { "--base-url": "https://example.test" };
const socketValues = { "--base-url": "" };
expect(fields.map((field) => field.flag)).toEqual([
"--bot-token",
"--base-url",
"--signing-secret",
"--app-token",
]);
expect(
fields
.filter((field) => shouldIncludeField(field, webhookValues))
.map((field) => field.flag),
).toEqual(["--bot-token", "--base-url", "--signing-secret"]);
expect(
fields
.filter((field) => shouldIncludeField(field, socketValues))
.map((field) => field.flag),
).toEqual(["--bot-token", "--base-url", "--app-token"]);
});
});
-255
View File
@@ -1,255 +0,0 @@
import { CORE_BUILD_VERSION } from "@cline/core";
import { isNonLocalBindHost } from "./options";
import {
handleToolApprovalResponse,
rejectOrphanedApprovals,
} from "./server/approvals";
import {
browserConfig,
host,
inviteUrl,
port,
publicUrl,
roomSecret,
webviewDistDir,
} from "./server/deps";
import { handleDesktopCommand } from "./server/desktop-commands";
import { createJsonResponse, WebviewAssets } from "./server/http";
import {
attachHub,
detachHub,
restartHub,
syncHubClientsAndSessions,
syncHubHealth,
} from "./server/hub";
import {
loadModels,
runProviderOAuthLogin,
saveProviderSettings,
sendProviderCatalog,
} from "./server/providers";
import {
abortPeerTurn,
deleteSession,
forkPeerSession,
initializePeer,
resetPeer,
restorePeerSession,
selectSession,
sendMessage,
} from "./server/sessions";
import { HubContext } from "./server/state";
import { broadcastHubState, hubStatusPayload } from "./server/state-payloads";
import type { BrowserFrame, BrowserPeer } from "./server/types";
export interface ClineHubDashboardServer {
listenUrl: string;
publicUrl: string;
inviteUrl: string;
bindHost: string;
inviteRequired: boolean;
hubUrl: string | undefined;
stop: () => Promise<void>;
}
export async function startClineHubDashboardServer(): Promise<ClineHubDashboardServer> {
const ctx = new HubContext();
const assets = new WebviewAssets(webviewDistDir);
const syncClientsAndSessions = () => syncHubClientsAndSessions(ctx);
let stopped = false;
function isAuthorizedBrowserRequest(url: URL): boolean {
if (!roomSecret) return true;
return url.searchParams.get("roomSecret") === roomSecret;
}
await attachHub(ctx);
const healthInterval = setInterval(() => {
void (async () => {
await syncHubHealth(ctx);
broadcastHubState(ctx);
})();
}, 5_000);
const server = Bun.serve<BrowserPeer>({
port,
hostname: host,
async fetch(req, server) {
const url = new URL(req.url);
if (url.pathname === "/version") {
return createJsonResponse({ coreVersion: CORE_BUILD_VERSION });
}
if (url.pathname === "/health") {
await syncHubHealth(ctx);
return createJsonResponse(hubStatusPayload(ctx));
}
if (url.pathname === "/browser") {
if (!isAuthorizedBrowserRequest(url)) {
return createJsonResponse({ error: "invalid_room_secret" }, 401);
}
const displayName = `Browser ${Math.random().toString(36).slice(2, 6)}`;
const data = {
socket: undefined as never,
displayName,
sending: false,
};
if (server.upgrade(req, { data })) return undefined;
return new Response("upgrade failed", { status: 400 });
}
if (url.pathname === "/config.json") {
return createJsonResponse(browserConfig);
}
return assets.serve(url.pathname);
},
websocket: {
async open(socket) {
const peer = socket.data;
peer.socket = socket;
ctx.peers.add(peer);
},
async message(socket, raw) {
const peer = socket.data;
try {
const frame = JSON.parse(String(raw)) as BrowserFrame;
if (frame.type === "desktopCommand") {
try {
const result = await handleDesktopCommand(
ctx,
frame.command,
frame.args,
);
ctx.send(peer, {
type: "desktopCommandResult",
id: frame.id,
ok: true,
result,
});
} catch (error) {
ctx.send(peer, {
type: "desktopCommandResult",
id: frame.id,
ok: false,
error: error instanceof Error ? error.message : String(error),
});
}
} else if (frame.type === "ready") {
await initializePeer(ctx, peer, syncClientsAndSessions);
} else if (frame.type === "loadModels") {
await loadModels(ctx, peer, frame.providerId);
} else if (frame.type === "loadProviderCatalog") {
await sendProviderCatalog(ctx, peer);
} else if (frame.type === "saveProviderSettings") {
await saveProviderSettings(ctx, peer, frame);
} else if (frame.type === "runProviderOAuthLogin") {
await runProviderOAuthLogin(ctx, peer, frame.providerId);
} else if (frame.type === "attachSession") {
await selectSession(ctx, peer, frame.sessionId);
} else if (frame.type === "deleteSession") {
await deleteSession(ctx, peer, frame.sessionId);
} else if (frame.type === "updateSessionMetadata") {
if (!ctx.cline) throw new Error("Hub is not connected.");
const session = await ctx.cline.get(frame.sessionId);
const metadata =
session?.metadata && typeof session.metadata === "object"
? (session.metadata as Record<string, unknown>)
: {};
await ctx.cline.update(frame.sessionId, {
metadata: { ...metadata, ...frame.metadata },
});
await syncHubClientsAndSessions(ctx);
broadcastHubState(ctx);
} else if (frame.type === "approval_response") {
handleToolApprovalResponse(ctx, frame);
} else if (frame.type === "abort") {
await abortPeerTurn(ctx, peer);
} else if (frame.type === "reset") {
await resetPeer(ctx, peer);
} else if (frame.type === "send") {
if (peer.sending) {
ctx.send(peer, {
type: "status",
text: "A turn is already in progress.",
});
return;
}
peer.sending = true;
try {
await sendMessage(
ctx,
peer,
frame.prompt,
frame.config,
frame.attachments,
);
} finally {
peer.sending = false;
}
} else if (frame.type === "forkSession") {
await forkPeerSession(ctx, peer, syncClientsAndSessions);
} else if (frame.type === "restore") {
await restorePeerSession(
ctx,
peer,
frame.checkpointRunCount,
syncClientsAndSessions,
);
} else if (frame.type === "restart_hub") {
await restartHub(ctx);
}
} catch (error) {
ctx.send(peer, {
type: "error",
text: error instanceof Error ? error.message : String(error),
});
}
},
close(socket) {
const peer = socket.data;
peer.unsubscribeEvents?.();
ctx.peers.delete(peer);
rejectOrphanedApprovals(ctx);
},
},
});
return {
listenUrl: server.url.toString(),
publicUrl,
inviteUrl,
bindHost: host,
inviteRequired: Boolean(roomSecret),
hubUrl: ctx.hubUrl,
stop: async () => {
if (stopped) return;
stopped = true;
clearInterval(healthInterval);
try {
server.stop(true);
} finally {
await detachHub(ctx);
}
},
};
}
export function printClineHubDashboardServerInfo(
server: ClineHubDashboardServer,
): void {
console.log(`Cline Hub dashboard listening: ${server.listenUrl}`);
console.log(`Cline Hub public URL: ${server.publicUrl}`);
console.log(`hub endpoint: ${server.hubUrl}`);
if (server.inviteRequired) {
console.log(`Cline Hub invite URL: ${server.inviteUrl}`);
} else if (isNonLocalBindHost(server.bindHost)) {
console.warn("WARNING: non-local bind without ROOM_SECRET is not allowed.");
} else {
console.log(
"ROOM_SECRET is not set; this local-only instance accepts browser connections without an invite token.",
);
}
}
if (import.meta.main) {
const server = await startClineHubDashboardServer();
printClineHubDashboardServerInfo(server);
}
-21
View File
@@ -1,21 +0,0 @@
{
"extends": "../../sdk/packages/tsconfig.base.json",
"compilerOptions": {
"noEmit": true,
"types": ["bun", "node"],
"paths": {
"@cline/core": ["../../sdk/packages/core/src/index.ts"],
"@cline/core/*": [
"../../sdk/packages/core/src/*",
"../../sdk/packages/core/src/*/index.ts"
],
"@cline/shared": ["../../sdk/packages/shared/src/index.ts"],
"@cline/shared/*": [
"../../sdk/packages/shared/src/*",
"../../sdk/packages/shared/src/*/index.ts"
]
}
},
"include": ["src/**/*.ts"],
"exclude": ["src/webview/**"]
}
@@ -1,23 +0,0 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"paths": {
"@/*": ["./*"],
"@cline/agents": ["../../../sdk/packages/agents/src/index.ts"],
"@cline/core": ["../../../sdk/packages/core/src/index.ts"],
"@cline/core/hub": ["../../../sdk/packages/core/src/hub/index.ts"],
"@cline/llms": ["../../../sdk/packages/llms/src/index.ts"],
"@cline/shared": ["../../../sdk/packages/shared/src/index.ts"],
"@cline/shared/storage": [
"../../../sdk/packages/shared/src/storage/index.ts"
],
"@cline/shared/db": ["../../../sdk/packages/shared/src/db/index.ts"],
"@cline/shared/*": [
"../../../sdk/packages/shared/src/*",
"../../../sdk/packages/shared/src/*/index.ts"
]
}
},
"include": ["sidecar/**/*.ts", "scripts/**/*.ts", "global.d.ts", "bun.mts"],
"exclude": ["node_modules", "webview"]
}
@@ -1,28 +0,0 @@
{
"extends": "../tsconfig.json",
"compilerOptions": {
"paths": {
"@/*": ["./*"],
"@cline/agents": ["../../../../sdk/packages/agents/src/index.ts"],
"@cline/core": ["../../../../sdk/packages/core/src/index.ts"],
"@cline/llms": ["../../../../sdk/packages/llms/src/index.ts"],
"@cline/shared": ["../../../../sdk/packages/shared/src/index.ts"],
"@cline/shared/storage": [
"../../../../sdk/packages/shared/src/storage/index.ts"
],
"@cline/shared/db": ["../../../../sdk/packages/shared/src/db/index.ts"],
"@cline/shared/*": [
"../../../../sdk/packages/shared/src/*",
"../../../../sdk/packages/shared/src/*/index.ts"
]
}
},
"include": [
"next-env.d.ts",
"**/*.ts",
"**/*.tsx",
".next/types/**/*.ts",
".next/dev/types/**/*.ts"
],
"exclude": ["node_modules"]
}
-35
View File
@@ -1,35 +0,0 @@
{
"extends": "../../../sdk/packages/tsconfig.base.json",
"compilerOptions": {
"outDir": "dist",
"module": "ESNext",
"moduleResolution": "Bundler",
"target": "ESNext",
"paths": {
"@cline/agents": ["../../../sdk/packages/agents/src/index.ts"],
"@cline/agents/*": [
"../../../sdk/packages/agents/src/*",
"../../../sdk/packages/agents/src/*/index.ts"
],
"@cline/core": ["../../../sdk/packages/core/src/index.ts"],
"@cline/core/hub/daemon-entry": [
"../../../sdk/packages/core/src/hub/daemon/entry.ts"
],
"@cline/core/*": [
"../../../sdk/packages/core/src/*",
"../../../sdk/packages/core/src/*/index.ts"
],
"@cline/llms": ["../../../sdk/packages/llms/src/index.ts"],
"@cline/shared": ["../../../sdk/packages/shared/src/index.ts"],
"@cline/shared/storage": [
"../../../sdk/packages/shared/src/storage/index.ts"
],
"@cline/shared/db": ["../../../sdk/packages/shared/src/db/index.ts"],
"@cline/shared/*": [
"../../../sdk/packages/shared/src/*",
"../../../sdk/packages/shared/src/*/index.ts"
]
}
},
"include": ["sidecar/**/*.ts"]
}
-31
View File
@@ -1,31 +0,0 @@
{
"compilerOptions": {
"paths": {
"@cline/agents": ["../sdk/packages/agents/src/index.ts"],
"@cline/agents/*": [
"../sdk/packages/agents/src/*",
"../sdk/packages/agents/src/*/index.ts"
],
"@cline/cline-hub": ["./cline-hub/src/server.ts"],
"@cline/core": ["../sdk/packages/core/src/index.ts"],
"@cline/core/hub/daemon-entry": [
"../sdk/packages/core/src/hub/daemon/entry.ts"
],
"@cline/core/*": [
"../sdk/packages/core/src/*",
"../sdk/packages/core/src/*/index.ts"
],
"@cline/core/telemetry": [
"../sdk/packages/core/src/services/telemetry/index.ts"
],
"@cline/llms": ["../sdk/packages/llms/src/index.ts"],
"@cline/shared": ["../sdk/packages/shared/src/index.ts"],
"@cline/shared/storage": ["../sdk/packages/shared/src/storage/index.ts"],
"@cline/shared/db": ["../sdk/packages/shared/src/db/index.ts"],
"@cline/shared/*": [
"../sdk/packages/shared/src/*",
"../sdk/packages/shared/src/*/index.ts"
]
}
}
}
+7
View File
@@ -6,6 +6,13 @@
"src/**/__tests__/*.ts",
"src/test/services/**/*.test.ts"
],
"ignore": [
"src/core/controller/models/__tests__/providerCatalogHandlers.test.ts",
"src/core/controller/models/__tests__/providerCatalogSmoke.test.ts",
"src/core/controller/models/__tests__/providerSwitchNormalization.test.ts",
"src/core/controller/models/__tests__/resolveModelInfo.test.ts",
"src/core/controller/models/__tests__/refreshClineRecommendedModels.test.ts"
],
"require": [
"ts-node/register",
"source-map-support/register",
+5 -1
View File
@@ -1,9 +1,13 @@
import { defineConfig } from "@vscode/test-cli"
import path from "path"
const vscodeTestVersion = process.env.VSCODE_TEST_VERSION ?? "stable"
export default defineConfig({
files: "{out/**/*.test.js,src/**/*.test.js,!src/test/e2e/**/*.test.js,!out/src/test/e2e/**/*.test.js}",
files: [
"out/src/{core,test,utils,shared,integrations,hosts,services}/**/*.test.js",
"src/{core,test,utils,shared,integrations,hosts,services}/**/*.test.js",
],
mocha: {
ui: "bdd",
timeout: 20000, // Maximum time (in ms) that a test can run before failing
-201
View File
@@ -1,201 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 Cline Bot Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+12 -5
View File
@@ -122,7 +122,6 @@
"!!**/out",
"!!**/evals",
"!!**/playwright",
"!!**/.vscode-test",
"!!**/test-results",
"!!**/node_modules",
"!!**/webview-ui/build",
@@ -131,7 +130,9 @@
"!!**/tests/specs"
]
},
"plugins": ["src/dev/grit/process-env.grit"],
"plugins": [
"src/dev/grit/process-env.grit"
],
"overrides": [
{
"includes": [
@@ -146,11 +147,15 @@
"!!src/integrations/terminal/**",
"!!src/core/controller/ui/openWalkthrough.ts"
],
"plugins": ["src/dev/grit/vscode-api.grit"]
"plugins": [
"src/dev/grit/vscode-api.grit"
]
},
{
// Do not use console logging directly, use the Logger service instead.
"plugins": ["src/dev/grit/console-log.grit"],
"plugins": [
"src/dev/grit/console-log.grit"
],
"includes": [
"**",
"!!**/esbuild.*",
@@ -183,7 +188,9 @@
"!!src/core/storage/utils/state-helpers.ts",
"!!src/extension.ts"
],
"plugins": ["src/dev/grit/use-cache-service.grit"]
"plugins": [
"src/dev/grit/use-cache-service.grit"
]
}
]
}
-39
View File
@@ -85,44 +85,6 @@ const esbuildProblemMatcherPlugin = {
},
}
const copyWasmFiles = {
name: "copy-wasm-files",
setup(build) {
build.onEnd(() => {
// tree sitter
const sourceDir = path.join(__dirname, "node_modules", "web-tree-sitter")
const targetDir = path.join(__dirname, destDir)
// Copy tree-sitter.wasm
fs.copyFileSync(path.join(sourceDir, "tree-sitter.wasm"), path.join(targetDir, "tree-sitter.wasm"))
// Copy language-specific WASM files
const languageWasmDir = path.join(__dirname, "node_modules", "tree-sitter-wasms", "out")
const languages = [
"typescript",
"tsx",
"python",
"rust",
"javascript",
"go",
"cpp",
"c",
"c_sharp",
"ruby",
"java",
"php",
"swift",
"kotlin",
]
languages.forEach((lang) => {
const filename = `tree-sitter-${lang}.wasm`
fs.copyFileSync(path.join(languageWasmDir, filename), path.join(targetDir, filename))
})
})
},
}
const buildEnvVars = {
"import.meta.url": "_importMetaUrl",
"process.env.IS_STANDALONE": JSON.stringify(standalone ? "true" : "false"),
@@ -176,7 +138,6 @@ const baseConfig = {
define: buildEnvVars,
tsconfig: path.resolve(__dirname, "tsconfig.json"),
plugins: [
copyWasmFiles,
aliasResolverPlugin,
/* add to the end of plugins array */
esbuildProblemMatcherPlugin,
+14907 -14960
View File
File diff suppressed because it is too large Load Diff
+22 -22
View File
@@ -2,8 +2,11 @@
"name": "claude-dev",
"displayName": "Cline",
"description": "Autonomous coding agent right in your IDE, capable of creating/editing files, running commands, using the browser, and more with your permission every step of the way.",
"version": "3.88.0",
"version": "3.86.2",
"icon": "assets/icons/icon.png",
"workspaces": [
"."
],
"engines": {
"vscode": "^1.84.0"
},
@@ -389,24 +392,26 @@
"protos": "node scripts/build-proto.mjs",
"protos-python": "node scripts/build-python-proto.mjs",
"download-ripgrep": "node scripts/download-ripgrep.mjs",
"postprotos": "biome format --config-path ./biome.jsonc src/shared/proto src/core/controller src/hosts/ webview-ui/src/services src/generated --write --no-errors-on-unmatched",
"postprotos": "biome format src/shared/proto src/core/controller src/hosts/ webview-ui/src/services src/generated --write --no-errors-on-unmatched",
"clean:build": "rimraf dist dist-standalone webview-ui/build src/generated out/",
"clean:deps": "rimraf node_modules webview-ui/node_modules",
"clean:all": "npm run clean:build && npm run clean:deps",
"compile-tests": "node ./scripts/build-tests.js",
"watch-tests": "tsc -p . -w --outDir out",
"check-types": "npm run protos && tsc --noEmit && cd webview-ui && tsc --noEmit",
"lint": "biome lint --config-path ./biome.jsonc --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error && npm run lint:proto",
"check-types": "npm run protos && npx tsc --noEmit && cd webview-ui && npx tsc --noEmit",
"lint": "biome lint --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error && npm run lint:proto",
"lint:proto": "bash ./scripts/proto-lint.sh",
"format": "biome format --config-path ./biome.jsonc --changed --since main --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error",
"format:fix": "biome check --config-path ./biome.jsonc --changed --since main --no-errors-on-unmatched --files-ignore-unknown=true --write",
"fix:all": "biome check --config-path ./biome.jsonc --no-errors-on-unmatched --files-ignore-unknown=true --write --diagnostic-level=error --unsafe",
"format": "biome format --changed --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error",
"format:fix": "biome check --changed --no-errors-on-unmatched --files-ignore-unknown=true --write",
"fix:all": "biome check --no-errors-on-unmatched --files-ignore-unknown=true --write --diagnostic-level=error --unsafe",
"ci:check-all": "npx npm-run-all -p check-types lint format",
"ci:build": "npm run protos && npm run build:webview && node esbuild.mjs && npm run compile-tests",
"pretest": "npm run compile && npm run compile-tests && npm run compile-standalone && npm run lint",
"test": "npx npm-run-all test:unit test:integration",
"test:integration": "vscode-test",
"test:unit": "cross-env TS_NODE_PROJECT=./tsconfig.unit-test.json mocha",
"test:vitest": "vitest run --config vitest.config.ts",
"test:vitest:watch": "vitest --config vitest.config.ts",
"test:coverage": "vscode-test --coverage",
"test:sca-server": "npx tsx watch scripts/test-standalone-core-api-server.ts",
"test:tp-orchestrator": "npx tsx scripts/testing-platform-orchestrator.ts",
@@ -479,25 +484,24 @@
"rimraf": "^6.0.1",
"should": "^13.2.3",
"sinon": "^21.0.3",
"tar": "^7.5.2",
"tree-kill": "^1.2.2",
"ts-node": "^10.9.2",
"ts-proto": "^2.6.1",
"tsconfig-paths": "^4.2.0",
"typescript": "^5.4.5"
"typescript": "^5.4.5",
"vitest": "^4.0.17"
},
"dependencies": {
"@anthropic-ai/sdk": "^0.37.0",
"@anthropic-ai/vertex-sdk": "^0.6.4",
"@aws-sdk/client-bedrock-runtime": "^3.922.0",
"@aws-sdk/credential-providers": "^3.922.0",
"@azure/identity": "^4.13.0",
"@bufbuild/protobuf": "^2.2.5",
"@cerebras/cerebras_cloud_sdk": "^1.35.0",
"@google-cloud/vertexai": "^1.9.3",
"@cline/agents": "^0.0.42",
"@cline/core": "^0.0.42",
"@cline/llms": "^0.0.42",
"@cline/shared": "^0.0.42",
"@google/genai": "^1.30.0",
"@grpc/grpc-js": "^1.9.15",
"@grpc/reflection": "^1.0.4",
"@mistralai/mistralai": "^1.5.0",
"@modelcontextprotocol/sdk": "^1.25.1",
"@opentelemetry/api": "^1.9.0",
"@opentelemetry/core": "^2.1.0",
@@ -519,9 +523,6 @@
"@opentelemetry/sdk-trace-node": "^1.30.1",
"@opentelemetry/semantic-conventions": "^1.37.0",
"@playwright/test": "^1.55.1",
"@sap-ai-sdk/ai-api": "^2.7.0",
"@sap-ai-sdk/orchestration": "^2.7.0",
"@sap-cloud-sdk/connectivity": "^4.6.0",
"@streamparser/json": "^0.0.22",
"@tailwindcss/vite": "^4.1.14",
"@types/uuid": "^10.0.0",
@@ -547,13 +548,14 @@
"ignore": "^7.0.3",
"image-size": "^2.0.2",
"isbinaryfile": "^5.0.2",
"js-yaml": "^4.1.1",
"jschardet": "^3.1.4",
"json5": "^2.2.3",
"jwt-decode": "^4.0.0",
"mammoth": "^1.11.0",
"nanoid": "^5.1.6",
"nice-grpc": "^2.1.12",
"node-machine-id": "^1.1.12",
"ollama": "^0.5.13",
"open": "^10.1.2",
"open-graph-scraper": "^6.9.0",
"openai": "^6.21.0",
@@ -572,15 +574,13 @@
"simple-git": "3.36.0",
"strip-ansi": "^7.1.2",
"tailwindcss": "^4.1.14",
"tree-sitter-wasms": "^0.1.11",
"ts-morph": "^25.0.1",
"turndown": "^7.2.0",
"ulid": "^2.4.0",
"undici": "^7.26.0",
"uuid": "^11.1.0",
"vscode-uri": "^3.1.0",
"web-tree-sitter": "^0.22.6",
"zod": "^3.24.2"
"zod": "^4.3.6"
},
"overrides": {
"tar-fs": ">=3.1.1",
+153 -2
View File
@@ -21,8 +21,6 @@ service ModelsService {
rpc refreshOpenRouterModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns recommended and free Cline models
rpc refreshClineRecommendedModelsRpc(EmptyRequest) returns (ClineRecommendedModelsResponse);
// Refreshes and returns Cline provider models
rpc refreshClineModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns Hugging Face models
rpc refreshHuggingFaceModels(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns OpenAI models
@@ -55,6 +53,18 @@ service ModelsService {
rpc getAihubmixModels(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Refreshes and returns Vercel AI Gateway models
rpc refreshVercelAiGatewayModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
// Lists providers available from the unified SDK-backed catalog
rpc listProviders(Empty) returns (ProviderListingsResponse);
// Resolves model metadata for a provider through the unified SDK-backed catalog
rpc resolveProviderModels(ResolveProviderModelsRequest) returns (ProviderModelsResponse);
// Resolves model metadata for a provider/model without refreshing model lists
rpc resolveModelInfo(ResolveModelInfoRequest) returns (ResolveModelInfoResponse);
// Reads redacted effective provider configuration
rpc readProviderConfig(StringRequest) returns (ProviderConfigResponse);
// Writes provider configuration fields and returns redacted effective configuration
rpc writeProviderConfig(WriteProviderConfigRequest) returns (ProviderConfigResponse);
// Commits a mode-specific model selection atomically with its model metadata
rpc commitModelSelection(CommitModelSelectionRequest) returns (Empty);
}
// List of VS Code LM models
@@ -117,6 +127,147 @@ message OpenRouterCompatibleModelInfo {
map<string, OpenRouterModelInfo> models = 1;
}
// Lightweight provider entry for the top-level model/provider picker.
// Does not include the full model list; use resolveProviderModels for models.
message ProviderListing {
string id = 1;
string name = 2;
optional string default_model_id = 3;
optional string family = 4;
optional string protocol = 5;
optional string auth_description = 6;
optional string base_url_description = 7;
bool allows_custom_model_ids = 8;
// SDK-driven hint for cost display. Values: "show" (default) or "hide".
// Sourced from `resolveProviderUsageCostDisplay(provider.metadata)` in
// `@cline/llms`. When "hide", consumers must suppress per-token pricing
// and total cost displays (matches the CLI's `shouldShowCliUsageCost`).
string usage_cost_display = 11;
}
message ProviderListingsResponse {
repeated ProviderListing providers = 1;
}
message ResolveProviderModelsRequest {
string provider_id = 1;
bool force_refresh = 2;
optional string request_id = 3;
}
message CatalogErrorInfo {
string kind = 1;
string message = 2;
optional string code = 3;
optional bool retryable = 4;
}
message ProviderModelsResponse {
string provider_id = 1;
string request_id = 2;
string config_fingerprint = 3;
int64 fetched_at = 4;
bool ok = 5;
map<string, OpenRouterModelInfo> models = 6;
optional string default_model_id = 7;
optional string source = 8;
optional CatalogErrorInfo error = 9;
}
message ResolveModelInfoRequest {
string provider_id = 1;
optional string model_id = 2;
}
message ResolveModelInfoResponse {
string provider_id = 1;
string model_id = 2;
optional OpenRouterModelInfo model_info = 3;
string source = 4;
}
message ProviderConfigResponse {
string provider_id = 1;
optional string base_url = 2;
optional string api_line = 3;
map<string, string> headers = 4;
optional string region = 5;
int64 api_key_length = 6;
bool has_access_token = 7;
bool has_refresh_token = 8;
optional string account_id = 9;
optional CommittedModelSelection plan_selection = 10;
optional CommittedModelSelection act_selection = 11;
optional AwsProviderConfig aws = 12;
}
message CommittedModelSelection {
string provider_id = 1;
string model_id = 2;
OpenRouterModelInfo model_info = 3;
}
message ProviderReasoningPatch {
optional bool enabled = 1;
optional string effort = 2; // "none" | "low" | "medium" | "high" | "xhigh"
optional int32 budget_tokens = 3;
}
message AwsProviderConfig {
optional string access_key = 1;
optional string secret_key = 2;
optional string session_token = 3;
optional string region = 4;
optional string profile = 5;
optional string authentication = 6; // "iam" | "api-key" | "apikey" | "profile"
optional bool use_prompt_cache = 7;
optional bool use_cross_region_inference = 8;
optional bool use_global_inference = 9;
optional string endpoint = 10;
optional string custom_model_base_id = 11;
}
message AwsProviderConfigPatch {
optional string access_key = 1;
optional string secret_key = 2;
optional string session_token = 3;
optional string region = 4;
optional string profile = 5;
optional string authentication = 6; // "iam" | "api-key" | "apikey" | "profile"
optional bool use_prompt_cache = 7;
optional bool use_cross_region_inference = 8;
optional bool use_global_inference = 9;
optional string endpoint = 10;
optional string custom_model_base_id = 11;
}
message WriteProviderConfigPatch {
optional string api_key = 1;
optional string base_url = 2;
map<string, string> headers = 3;
optional string region = 4;
optional string api_line = 5;
optional string access_token = 6;
optional string refresh_token = 7;
optional string account_id = 8;
optional ProviderReasoningPatch reasoning = 9;
optional bool clear_headers = 10;
optional AwsProviderConfigPatch aws = 11;
}
message WriteProviderConfigRequest {
string provider_id = 1;
WriteProviderConfigPatch patch = 2;
}
message CommitModelSelectionRequest {
string provider_id = 1;
string mode = 2;
string model_id = 3;
OpenRouterModelInfo model_info = 4;
}
message ClineRecommendedModel {
string id = 1;
string name = 2;
@@ -0,0 +1,32 @@
syntax = "proto3";
package cline;
import "cline/common.proto";
option go_package = "github.com/cline/grpc-go/cline";
option java_multiple_files = true;
option java_package = "bot.cline.proto";
enum RemoteConfigType {
RULE = 0;
WORKFLOW = 1;
SKILL = 2;
}
message RemoteConfigSetting {
RemoteConfigType type = 1;
string name = 2;
string content = 3;
bool enabled = 4;
bool locked = 5;
}
message RemoteConfigSettingsResponse {
repeated RemoteConfigSetting settings = 1;
}
service RemoteConfigService {
rpc getRemoteConfigSettings(Empty) returns (RemoteConfigSettingsResponse);
rpc toggleRemoteConfigSetting(StringRequest) returns (RemoteConfigSetting);
}
+2 -4
View File
@@ -250,7 +250,6 @@ message Settings {
optional string default_terminal_profile = 137;
optional int32 terminal_output_line_limit = 138;
optional int32 max_consecutive_mistakes = 139;
optional bool strict_plan_mode_enabled = 141;
optional bool yolo_mode_toggled = 142;
optional bool use_auto_condense = 143;
optional bool cline_web_tools_enabled = 144;
@@ -286,7 +285,6 @@ message Settings {
optional string act_mode_cline_model_id = 180;
optional OpenRouterModelInfo act_mode_cline_model_info = 181;
optional bool show_feature_tips = 182;
optional bool lazy_teammate_mode_enabled = 183;
}
message State {
@@ -391,6 +389,7 @@ message UpdateSettingsRequest {
reserved 15; // was openai_reasoning_effort (moved to mode-scoped reasoning effort)
reserved 23; // was dictation_settings (dictation removed)
reserved 38; // was skills_enabled (removed - now always enabled)
reserved 43; // was lazy_teammate_mode_enabled (removed)
Metadata metadata = 1;
optional ModelsApiConfiguration api_configuration = 2;
@@ -405,7 +404,7 @@ message UpdateSettingsRequest {
optional int32 terminal_output_line_limit = 12;
optional PlanActMode mode = 13;
optional string preferred_language = 14;
optional bool strict_plan_mode_enabled = 16;
reserved 16; // was strict_plan_mode_enabled (removed)
optional FocusChainSettings focus_chain_settings = 17;
optional bool use_auto_condense = 18;
optional string custom_prompt = 19;
@@ -429,7 +428,6 @@ message UpdateSettingsRequest {
optional bool worktrees_enabled = 40;
optional bool double_check_completion_enabled = 41;
optional bool show_feature_tips = 42;
optional bool lazy_teammate_mode_enabled = 43;
}
message UpdateTerminalConnectionTimeoutRequest {
+3 -1
View File
@@ -82,12 +82,14 @@ message GetTaskHistoryRequest {
string search_query = 3;
string sort_by = 4;
bool current_workspace_only = 5;
int32 limit = 6;
int32 offset = 7;
}
// Response for task history
message TaskHistoryArray {
repeated TaskItem tasks = 1;
int32 total_count = 2;
bool has_more = 2;
}
// Task item details for history list
+6
View File
@@ -226,6 +226,12 @@ message ClineMessage {
ClineAskNewTask ask_new_task = 21;
ClineApiReqInfo api_req_info = 22;
ClineModelInfo model_info = 23;
// Convergent-replica fields (see webview-message-state-design.md):
// seq = monotonic freshness (higher seq wins for the same ts/identity)
// epoch = conversation/replica fence (older epoch is dropped by the webview)
int64 seq = 24;
int64 epoch = 25;
}
message ShowWebviewEvent {
+129
View File
@@ -0,0 +1,129 @@
// Dead-source finder: uses esbuild's own bundle reachability (the same analysis
// that drives tree-shaking + minification mangling) to compute which src/ files
// are reachable from BOTH shipped entry points:
// - src/extension.ts (VS Code extension host)
// - src/standalone/cline-core.ts (standalone host used by JetBrains + CLI)
//
// A src/*.ts file that is NOT in the union of metafile inputs for those two
// builds is unreachable from any shipped entry => dead (modulo dynamic import()
// of computed specifiers, which esbuild surfaces separately).
//
// Run: node scripts/find-dead-src.mjs
import fs from "node:fs"
import path from "node:path"
import { fileURLToPath } from "node:url"
import * as esbuild from "esbuild"
import { glob } from "glob"
const __dirname = path.dirname(fileURLToPath(import.meta.url))
const root = path.resolve(__dirname, "..")
const aliases = {
"@": path.join(root, "src"),
"@core": path.join(root, "src/core"),
"@integrations": path.join(root, "src/integrations"),
"@services": path.join(root, "src/services"),
"@shared": path.join(root, "src/shared"),
"@utils": path.join(root, "src/utils"),
"@packages": path.join(root, "src/packages"),
}
const aliasResolverPlugin = {
name: "alias-resolver",
setup(build) {
for (const [alias, aliasPath] of Object.entries(aliases)) {
const aliasRegex = new RegExp(`^${alias}($|/.*)`)
build.onResolve({ filter: aliasRegex }, (args) => {
const importPath = args.path.replace(alias, aliasPath)
const exts = [".ts", ".tsx", ".js", ".jsx"]
if (fs.existsSync(importPath)) {
const stats = fs.statSync(importPath)
if (stats.isDirectory()) {
for (const ext of exts) {
const idx = path.join(importPath, `index${ext}`)
if (fs.existsSync(idx)) return { path: idx }
}
} else {
return { path: importPath }
}
}
for (const ext of exts) {
if (fs.existsSync(`${importPath}${ext}`)) return { path: `${importPath}${ext}` }
}
return undefined
})
}
},
}
const common = {
bundle: true,
minify: false,
sourcemap: false,
logLevel: "silent",
format: "cjs",
platform: "node",
metafile: true,
write: false,
absWorkingDir: root,
tsconfig: path.join(root, "tsconfig.json"),
packages: "external",
plugins: [aliasResolverPlugin],
define: { "process.env.IS_DEV": "false", "process.env.IS_TEST": "false" },
banner: { js: "const _importMetaUrl=require('url').pathToFileURL(__filename)" },
}
async function inputsFor(entry, external) {
const r = await esbuild.build({ ...common, entryPoints: [entry], external })
return new Set(Object.keys(r.metafile.inputs).filter((f) => f.startsWith("src/") && /\.tsx?$/.test(f)))
}
const ext = await inputsFor("src/extension.ts", ["vscode"])
const standalone = await inputsFor("src/standalone/cline-core.ts", [
"vscode",
"@grpc/reflection",
"grpc-health-check",
"better-sqlite3",
])
const live = new Set([...ext, ...standalone])
// Third consumer: the webview (webview-ui/) is a separate Vite/React build that
// imports extension code ONLY from src/shared (via "@shared/*" alias or relative
// "../src/shared/*" paths). Any src/shared file referenced from webview-ui/src is
// therefore live even if the extension-host/standalone bundles don't reach it.
// Conservatively mark every src/shared file mentioned by the webview as live.
const webviewFiles = await glob("webview-ui/src/**/*.{ts,tsx}", { cwd: root })
const sharedMentionedByWebview = new Set()
for (const wf of webviewFiles) {
const text = fs.readFileSync(path.join(root, wf), "utf8")
// Match @shared/X or .../src/shared/X import specifiers and map to src/shared/X
const re = /(?:@shared\/|src\/shared\/)([A-Za-z0-9_./-]+)/g
let m
while ((m = re.exec(text))) {
const rel = m[1].replace(/\.(ts|tsx|js|jsx)$/, "")
for (const cand of [`src/shared/${rel}.ts`, `src/shared/${rel}.tsx`, `src/shared/${rel}/index.ts`]) {
if (fs.existsSync(path.join(root, cand))) sharedMentionedByWebview.add(cand)
}
}
}
for (const f of sharedMentionedByWebview) live.add(f)
console.log(`src/shared files referenced by webview: ${sharedMentionedByWebview.size}`)
// All non-test, non-.d.ts source files on disk.
const allSrc = (await glob("src/**/*.{ts,tsx}", { cwd: root }))
.filter((f) => !/\.test\.tsx?$/.test(f))
.filter((f) => !f.endsWith(".d.ts"))
.filter((f) => !f.includes("/__tests__/"))
.filter((f) => !f.startsWith("src/test/"))
.filter((f) => !f.startsWith("src/generated/")) // generated host glue
.filter((f) => !f.startsWith("src/dev/")) // dev-only tooling
const dead = allSrc.filter((f) => !live.has(f)).sort()
console.log(`extension inputs: ${ext.size}`)
console.log(`standalone inputs: ${standalone.size}`)
console.log(`union live src files: ${live.size}`)
console.log(`candidate dead files: ${dead.length}`)
fs.writeFileSync("/tmp/dead-src.json", JSON.stringify(dead, null, "\t"))
console.log("--- dead candidates written to /tmp/dead-src.json ---")
@@ -87,6 +87,12 @@ function createMockService<T extends grpc.UntypedServiceImplementation>(serviceN
})
return
case "openExternal":
simulateOAuthBrowserCallback(call.request?.value || "")
.then(() => callback(null, {}))
.catch((error) => callback(error))
return
case "getWebviewHtml":
callback(null, {
html: "<html><body>Fake Webview</body></html>",
@@ -143,6 +149,41 @@ function createMockService<T extends grpc.UntypedServiceImplementation>(serviceN
return new Proxy({} as T, handler)
}
async function simulateOAuthBrowserCallback(urlString: string): Promise<void> {
let url: URL
try {
url = new URL(urlString)
} catch {
return
}
if (!isLoopbackHost(url.hostname) || url.pathname !== "/api/v1/auth/authorize") {
return
}
const callbackUrl = url.searchParams.get("callback_url") ?? url.searchParams.get("redirect_uri")
if (!callbackUrl) {
return
}
const callback = new URL(callbackUrl)
if (!isLoopbackHost(callback.hostname) || callback.pathname !== "/auth") {
return
}
callback.searchParams.set("code", "test-personal-token")
callback.searchParams.set("provider", "cline")
const response = await fetch(callback.toString())
if (!response.ok) {
throw new Error(`Mock OAuth callback failed: ${response.status} ${response.statusText}`)
}
}
function isLoopbackHost(hostname: string): boolean {
return hostname === "localhost" || hostname === "127.0.0.1" || hostname === "::1"
}
if (require.main === module) {
startTestHostBridgeServer().catch((err) => {
console.error("Failed to start test host bridge server:", err)
@@ -115,7 +115,8 @@ async function main(): Promise<void> {
const covDir = path.join(projectRoot, `coverage/coverage-core-${PROTOBUS_PORT}`)
const baseArgs = ["--enable-source-maps", path.join(distDir, "cline-core.js")]
const workosFetchMockPath = path.join(projectRoot, "scripts", "testing-platform-workos-fetch-mock.cjs")
const baseArgs = ["--enable-source-maps", "--require", workosFetchMockPath, path.join(distDir, "cline-core.js")]
const spawnArgs = USE_C8 ? ["c8", "--report-dir", covDir, "node", ...baseArgs] : ["node", ...baseArgs]
@@ -0,0 +1,56 @@
// Preload used by the standalone testing platform.
// It makes the SDK WorkOS device-auth flow deterministic and fully local while
// leaving production auth code on the same device-auth path used by users.
const originalFetch = globalThis.fetch?.bind(globalThis)
const WORKOS_ORIGIN = "https://api.workos.com"
const DEVICE_CODE = "test-device-code"
const USER_CODE = "PTBC-TXTP"
const ACCESS_TOKEN = "test-personal-token"
const REFRESH_TOKEN = "test-personal-token_refresh"
function jsonResponse(body, init = {}) {
return new Response(JSON.stringify(body), {
status: init.status ?? 200,
headers: { "Content-Type": "application/json", ...(init.headers ?? {}) },
})
}
function inputUrl(input) {
if (typeof input === "string") return input
if (input instanceof URL) return input.toString()
if (input && typeof input === "object" && "url" in input) return input.url
return String(input)
}
globalThis.fetch = async (input, init) => {
const urlString = inputUrl(input)
let url
try {
url = new URL(urlString)
} catch {
return originalFetch(input, init)
}
if (url.origin === WORKOS_ORIGIN && url.pathname === "/user_management/authorize/device") {
return jsonResponse({
device_code: DEVICE_CODE,
user_code: USER_CODE,
verification_uri: "https://login.workos.test/device",
verification_uri_complete: `https://login.workos.test/device?user_code=${USER_CODE}`,
expires_in: 300,
interval: 1,
})
}
if (url.origin === WORKOS_ORIGIN && url.pathname === "/user_management/authenticate") {
return jsonResponse({
access_token: ACCESS_TOKEN,
refresh_token: REFRESH_TOKEN,
token_type: "Bearer",
})
}
return originalFetch(input, init)
}
+1 -3
View File
@@ -21,9 +21,7 @@ describe("ClineEndpoint configuration", () => {
// Stub os.homedir to return our temp directory
originalHomedir = os.homedir
sandbox
.stub(os, "homedir")
.returns(tempDir)
sandbox.stub(os, "homedir").returns(tempDir)
// Reset the singleton state using internal method
;(ClineEndpoint as any)._instance = null
+1 -4
View File
@@ -4,7 +4,6 @@ import "./utils/path" // necessary to have access to String.prototype.toPosix
import { HostProvider } from "@/hosts/host-provider"
import { Logger } from "@/shared/services/Logger"
import type { StorageContext } from "@/shared/storage/storage-context"
import { FileContextTracker } from "./core/context/context-tracking/FileContextTracker"
import { clearOnboardingModelsCache } from "./core/controller/models/getClineOnboardingModels"
import { HookDiscoveryCache } from "./core/hooks/HookDiscoveryCache"
import { HookProcessRegistry } from "./core/hooks/HookProcessRegistry"
@@ -74,8 +73,6 @@ export async function initialize(storageContext: StorageContext): Promise<Webvie
syncWorker().init({ ...blobStoreSettings, userDistinctId: getDistinctId() })
// Clean up old temp files in background (non-blocking) and start periodic cleanup every 24 hours
ClineTempManager.startPeriodicCleanup()
// Clean up orphaned file context warnings (startup cleanup)
FileContextTracker.cleanupOrphanedWarnings(stateManager)
telemetryService.captureExtensionActivated()
@@ -106,7 +103,7 @@ async function showVersionUpdateAnnouncement(stateManager: StateManager) {
})
}
// Always update the main version tracker for the next launch.
await stateManager.setGlobalState("clineVersion", currentVersion)
stateManager.setGlobalState("clineVersion", currentVersion)
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error)
+16 -477
View File
@@ -1,52 +1,24 @@
import { ApiConfiguration, ModelInfo, QwenApiRegions } from "@shared/api"
import { type ApiHandler as SdkApiHandler, type ApiStreamChunk as SdkApiStreamChunk } from "@cline/llms"
import { ApiConfiguration, ModelInfo } from "@shared/api"
import { Mode } from "@shared/storage/types"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { ClineTool } from "@/shared/tools"
import { AIhubmixHandler } from "./providers/aihubmix"
import { AnthropicHandler } from "./providers/anthropic"
import { AskSageHandler } from "./providers/asksage"
import { BasetenHandler } from "./providers/baseten"
import { AwsBedrockHandler } from "./providers/bedrock"
import { CerebrasHandler } from "./providers/cerebras"
import { ClaudeCodeHandler } from "./providers/claude-code"
import { ClineHandler } from "./providers/cline"
import { DeepSeekHandler } from "./providers/deepseek"
import { DifyHandler } from "./providers/dify"
import { DoubaoHandler } from "./providers/doubao"
import { FireworksHandler } from "./providers/fireworks"
import { GeminiHandler } from "./providers/gemini"
import { GroqHandler } from "./providers/groq"
import { HicapHandler } from "./providers/hicap"
import { HuaweiCloudMaaSHandler } from "./providers/huawei-cloud-maas"
import { HuggingFaceHandler } from "./providers/huggingface"
import { LiteLlmHandler } from "./providers/litellm"
import { LmStudioHandler } from "./providers/lmstudio"
import { MinimaxHandler } from "./providers/minimax"
import { MistralHandler } from "./providers/mistral"
import { MoonshotHandler } from "./providers/moonshot"
import { NebiusHandler } from "./providers/nebius"
import { NousResearchHandler } from "./providers/nousresearch"
import { OcaHandler } from "./providers/oca"
import { OllamaHandler } from "./providers/ollama"
import { OpenAiHandler } from "./providers/openai"
import { OpenAiCodexHandler } from "./providers/openai-codex"
import { OpenAiNativeHandler } from "./providers/openai-native"
import { OpenRouterHandler } from "./providers/openrouter"
import { QwenHandler } from "./providers/qwen"
import { QwenCodeHandler } from "./providers/qwen-code"
import { RequestyHandler } from "./providers/requesty"
import { SambanovaHandler } from "./providers/sambanova"
import { SapAiCoreHandler } from "./providers/sapaicore"
import { TogetherHandler } from "./providers/together"
import { VercelAIGatewayHandler } from "./providers/vercel-ai-gateway"
import { VertexHandler } from "./providers/vertex"
import { VsCodeLmHandler } from "./providers/vscode-lm"
import { WandbHandler } from "./providers/wandb"
import { XAIHandler } from "./providers/xai"
import { ZAiHandler } from "./providers/zai"
import { ApiStream, ApiStreamUsageChunk } from "./transform/stream"
// buildApiHandler now routes inference through the Cline SDK. It lives in
// apps/vscode/src/sdk/sdk-api-handler.ts and callers import it directly from
// there. It is deliberately NOT re-exported here: this barrel is imported
// widely for *types* only, and re-exporting a value from the SDK module would
// pull the entire SDK/session-factory runtime graph into every type importer
// at module-eval time (which can break extension activation). Keep this file
// types-only.
// Re-export the SDK inference contracts so callers can depend on the SDK types
// through the existing @core/api entry point. These are the canonical handler
// and stream types going forward; the local interfaces below remain for the
// classic provider classes until they are removed.
export type { SdkApiHandler, SdkApiStreamChunk }
export type CommonApiHandlerOptions = {
onRetryAttempt?: ApiConfiguration["onRetryAttempt"]
}
@@ -72,436 +44,3 @@ export interface ApiProviderInfo {
export interface SingleCompletionHandler {
completePrompt(prompt: string): Promise<string>
}
function createHandlerForProvider(
apiProvider: string | undefined,
options: Omit<ApiConfiguration, "apiProvider">,
mode: Mode,
): ApiHandler {
switch (apiProvider) {
case "anthropic":
return new AnthropicHandler({
onRetryAttempt: options.onRetryAttempt,
apiKey: options.apiKey,
anthropicBaseUrl: options.anthropicBaseUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "openrouter":
return new OpenRouterHandler({
onRetryAttempt: options.onRetryAttempt,
openRouterApiKey: options.openRouterApiKey,
openRouterModelId: mode === "plan" ? options.planModeOpenRouterModelId : options.actModeOpenRouterModelId,
openRouterModelInfo: mode === "plan" ? options.planModeOpenRouterModelInfo : options.actModeOpenRouterModelInfo,
openRouterProviderSorting: options.openRouterProviderSorting,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
enableParallelToolCalling: options.enableParallelToolCalling,
})
case "bedrock":
return new AwsBedrockHandler({
onRetryAttempt: options.onRetryAttempt,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
awsAccessKey: options.awsAccessKey,
awsSecretKey: options.awsSecretKey,
awsSessionToken: options.awsSessionToken,
awsRegion: options.awsRegion,
awsAuthentication: options.awsAuthentication,
awsBedrockApiKey: options.awsBedrockApiKey,
awsUseCrossRegionInference: options.awsUseCrossRegionInference,
awsUseGlobalInference: options.awsUseGlobalInference,
awsBedrockUsePromptCache: options.awsBedrockUsePromptCache,
awsUseProfile: options.awsUseProfile,
awsProfile: options.awsProfile,
awsBedrockEndpoint: options.awsBedrockEndpoint,
awsBedrockCustomSelected:
mode === "plan" ? options.planModeAwsBedrockCustomSelected : options.actModeAwsBedrockCustomSelected,
awsBedrockCustomModelBaseId:
mode === "plan" ? options.planModeAwsBedrockCustomModelBaseId : options.actModeAwsBedrockCustomModelBaseId,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "vertex":
return new VertexHandler({
onRetryAttempt: options.onRetryAttempt,
vertexProjectId: options.vertexProjectId,
vertexRegion: options.vertexRegion,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
geminiApiKey: options.geminiApiKey,
geminiBaseUrl: options.geminiBaseUrl,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
ulid: options.ulid,
})
case "openai":
return new OpenAiHandler({
onRetryAttempt: options.onRetryAttempt,
openAiApiKey: options.openAiApiKey,
openAiBaseUrl: options.openAiBaseUrl,
azureApiVersion: options.azureApiVersion,
azureIdentity: options.azureIdentity,
openAiHeaders: options.openAiHeaders,
openAiModelId: mode === "plan" ? options.planModeOpenAiModelId : options.actModeOpenAiModelId,
openAiModelInfo: mode === "plan" ? options.planModeOpenAiModelInfo : options.actModeOpenAiModelInfo,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
})
case "ollama":
return new OllamaHandler({
onRetryAttempt: options.onRetryAttempt,
ollamaBaseUrl: options.ollamaBaseUrl,
ollamaApiKey: options.ollamaApiKey,
ollamaModelId: mode === "plan" ? options.planModeOllamaModelId : options.actModeOllamaModelId,
ollamaApiOptionsCtxNum: options.ollamaApiOptionsCtxNum,
requestTimeoutMs: options.requestTimeoutMs,
})
case "lmstudio":
return new LmStudioHandler({
onRetryAttempt: options.onRetryAttempt,
lmStudioBaseUrl: options.lmStudioBaseUrl,
lmStudioModelId: mode === "plan" ? options.planModeLmStudioModelId : options.actModeLmStudioModelId,
lmStudioMaxTokens: options.lmStudioMaxTokens,
})
case "gemini":
return new GeminiHandler({
onRetryAttempt: options.onRetryAttempt,
vertexProjectId: options.vertexProjectId,
vertexRegion: options.vertexRegion,
geminiApiKey: options.geminiApiKey,
geminiBaseUrl: options.geminiBaseUrl,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
ulid: options.ulid,
})
case "openai-native":
return new OpenAiNativeHandler({
onRetryAttempt: options.onRetryAttempt,
openAiNativeApiKey: options.openAiNativeApiKey,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "openai-codex":
return new OpenAiCodexHandler({
onRetryAttempt: options.onRetryAttempt,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "deepseek":
return new DeepSeekHandler({
onRetryAttempt: options.onRetryAttempt,
deepSeekApiKey: options.deepSeekApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "requesty":
return new RequestyHandler({
onRetryAttempt: options.onRetryAttempt,
requestyBaseUrl: options.requestyBaseUrl,
requestyApiKey: options.requestyApiKey,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
requestyModelId: mode === "plan" ? options.planModeRequestyModelId : options.actModeRequestyModelId,
requestyModelInfo: mode === "plan" ? options.planModeRequestyModelInfo : options.actModeRequestyModelInfo,
})
case "fireworks":
return new FireworksHandler({
onRetryAttempt: options.onRetryAttempt,
fireworksApiKey: options.fireworksApiKey,
fireworksModelId: mode === "plan" ? options.planModeFireworksModelId : options.actModeFireworksModelId,
})
case "together":
return new TogetherHandler({
onRetryAttempt: options.onRetryAttempt,
togetherApiKey: options.togetherApiKey,
togetherModelId: mode === "plan" ? options.planModeTogetherModelId : options.actModeTogetherModelId,
})
case "qwen":
return new QwenHandler({
onRetryAttempt: options.onRetryAttempt,
qwenApiKey: options.qwenApiKey,
qwenApiLine:
options.qwenApiLine === QwenApiRegions.INTERNATIONAL ? QwenApiRegions.INTERNATIONAL : QwenApiRegions.CHINA,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "qwen-code":
return new QwenCodeHandler({
onRetryAttempt: options.onRetryAttempt,
qwenCodeOauthPath: options.qwenCodeOauthPath,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "doubao":
return new DoubaoHandler({
onRetryAttempt: options.onRetryAttempt,
doubaoApiKey: options.doubaoApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "mistral":
return new MistralHandler({
onRetryAttempt: options.onRetryAttempt,
mistralApiKey: options.mistralApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "vscode-lm":
return new VsCodeLmHandler({
onRetryAttempt: options.onRetryAttempt,
vsCodeLmModelSelector:
mode === "plan" ? options.planModeVsCodeLmModelSelector : options.actModeVsCodeLmModelSelector,
})
case "cline": {
const clineModelId =
(mode === "plan" ? options.planModeClineModelId : options.actModeClineModelId) ||
(mode === "plan" ? options.planModeOpenRouterModelId : options.actModeOpenRouterModelId)
const clineModelInfo =
(mode === "plan" ? options.planModeClineModelInfo : options.actModeClineModelInfo) ||
(mode === "plan" ? options.planModeOpenRouterModelInfo : options.actModeOpenRouterModelInfo)
return new ClineHandler({
onRetryAttempt: options.onRetryAttempt,
clineAccountId: options.clineAccountId,
clineApiKey: options.clineApiKey,
ulid: options.ulid,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
openRouterProviderSorting: options.openRouterProviderSorting,
openRouterModelId: clineModelId,
openRouterModelInfo: clineModelInfo,
enableParallelToolCalling: options.enableParallelToolCalling,
})
}
case "litellm":
return new LiteLlmHandler({
onRetryAttempt: options.onRetryAttempt,
liteLlmApiKey: options.liteLlmApiKey,
liteLlmBaseUrl: options.liteLlmBaseUrl,
liteLlmModelId: mode === "plan" ? options.planModeLiteLlmModelId : options.actModeLiteLlmModelId,
liteLlmModelInfo: mode === "plan" ? options.planModeLiteLlmModelInfo : options.actModeLiteLlmModelInfo,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
liteLlmUsePromptCache: options.liteLlmUsePromptCache,
ulid: options.ulid,
})
case "moonshot":
return new MoonshotHandler({
onRetryAttempt: options.onRetryAttempt,
moonshotApiKey: options.moonshotApiKey,
moonshotApiLine: options.moonshotApiLine,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "huggingface":
return new HuggingFaceHandler({
onRetryAttempt: options.onRetryAttempt,
huggingFaceApiKey: options.huggingFaceApiKey,
huggingFaceModelId: mode === "plan" ? options.planModeHuggingFaceModelId : options.actModeHuggingFaceModelId,
huggingFaceModelInfo:
mode === "plan" ? options.planModeHuggingFaceModelInfo : options.actModeHuggingFaceModelInfo,
})
case "nebius":
return new NebiusHandler({
onRetryAttempt: options.onRetryAttempt,
nebiusApiKey: options.nebiusApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "asksage":
return new AskSageHandler({
onRetryAttempt: options.onRetryAttempt,
asksageApiKey: options.asksageApiKey,
asksageApiUrl: options.asksageApiUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "xai":
return new XAIHandler({
onRetryAttempt: options.onRetryAttempt,
xaiApiKey: options.xaiApiKey,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "sambanova":
return new SambanovaHandler({
onRetryAttempt: options.onRetryAttempt,
sambanovaApiKey: options.sambanovaApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "cerebras":
return new CerebrasHandler({
onRetryAttempt: options.onRetryAttempt,
cerebrasApiKey: options.cerebrasApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "groq":
return new GroqHandler({
onRetryAttempt: options.onRetryAttempt,
groqApiKey: options.groqApiKey,
groqModelId: mode === "plan" ? options.planModeGroqModelId : options.actModeGroqModelId,
groqModelInfo: mode === "plan" ? options.planModeGroqModelInfo : options.actModeGroqModelInfo,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "baseten":
return new BasetenHandler({
onRetryAttempt: options.onRetryAttempt,
basetenApiKey: options.basetenApiKey,
basetenModelId: mode === "plan" ? options.planModeBasetenModelId : options.actModeBasetenModelId,
basetenModelInfo: mode === "plan" ? options.planModeBasetenModelInfo : options.actModeBasetenModelInfo,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "sapaicore":
return new SapAiCoreHandler({
onRetryAttempt: options.onRetryAttempt,
sapAiCoreClientId: options.sapAiCoreClientId,
sapAiCoreClientSecret: options.sapAiCoreClientSecret,
sapAiCoreTokenUrl: options.sapAiCoreTokenUrl,
sapAiResourceGroup: options.sapAiResourceGroup,
sapAiCoreBaseUrl: options.sapAiCoreBaseUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
deploymentId: mode === "plan" ? options.planModeSapAiCoreDeploymentId : options.actModeSapAiCoreDeploymentId,
sapAiCoreUseOrchestrationMode: options.sapAiCoreUseOrchestrationMode,
})
case "claude-code":
return new ClaudeCodeHandler({
onRetryAttempt: options.onRetryAttempt,
claudeCodePath: options.claudeCodePath,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "huawei-cloud-maas":
return new HuaweiCloudMaaSHandler({
onRetryAttempt: options.onRetryAttempt,
huaweiCloudMaasApiKey: options.huaweiCloudMaasApiKey,
huaweiCloudMaasModelId:
mode === "plan" ? options.planModeHuaweiCloudMaasModelId : options.actModeHuaweiCloudMaasModelId,
huaweiCloudMaasModelInfo:
mode === "plan" ? options.planModeHuaweiCloudMaasModelInfo : options.actModeHuaweiCloudMaasModelInfo,
})
case "dify": // Add Dify.ai handler
return new DifyHandler({
difyApiKey: options.difyApiKey,
difyBaseUrl: options.difyBaseUrl,
})
case "vercel-ai-gateway":
return new VercelAIGatewayHandler({
onRetryAttempt: options.onRetryAttempt,
vercelAiGatewayApiKey: options.vercelAiGatewayApiKey,
openRouterModelId:
mode === "plan" ? options.planModeVercelAiGatewayModelId : options.actModeVercelAiGatewayModelId,
openRouterModelInfo:
mode === "plan" ? options.planModeVercelAiGatewayModelInfo : options.actModeVercelAiGatewayModelInfo,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "zai":
return new ZAiHandler({
onRetryAttempt: options.onRetryAttempt,
zaiApiLine: options.zaiApiLine,
zaiApiKey: options.zaiApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
case "oca":
return new OcaHandler({
ocaMode: options.ocaMode || "internal",
ocaBaseUrl: options.ocaBaseUrl,
ocaModelId: mode === "plan" ? options.planModeOcaModelId : options.actModeOcaModelId,
ocaModelInfo: mode === "plan" ? options.planModeOcaModelInfo : options.actModeOcaModelInfo,
ocaReasoningEffort: mode === "plan" ? options.planModeOcaReasoningEffort : options.actModeOcaReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
ocaUsePromptCache:
mode === "plan"
? options.planModeOcaModelInfo?.supportsPromptCache
: options.actModeOcaModelInfo?.supportsPromptCache,
taskId: options.ulid,
})
case "aihubmix":
return new AIhubmixHandler({
onRetryAttempt: options.onRetryAttempt,
apiKey: options.aihubmixApiKey,
baseURL: options.aihubmixBaseUrl,
appCode: options.aihubmixAppCode,
modelId: mode === "plan" ? (options as any).planModeAihubmixModelId : (options as any).actModeAihubmixModelId,
modelInfo:
mode === "plan" ? (options as any).planModeAihubmixModelInfo : (options as any).actModeAihubmixModelInfo,
})
case "minimax":
return new MinimaxHandler({
onRetryAttempt: options.onRetryAttempt,
minimaxApiKey: options.minimaxApiKey,
minimaxApiLine: options.minimaxApiLine,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
case "hicap":
return new HicapHandler({
onRetryAttempt: options.onRetryAttempt,
hicapApiKey: options.hicapApiKey,
hicapModelId: mode === "plan" ? options.planModeHicapModelId : options.actModeHicapModelId,
})
case "nousResearch":
return new NousResearchHandler({
onRetryAttempt: options.onRetryAttempt,
nousResearchApiKey: options.nousResearchApiKey,
apiModelId: mode === "plan" ? options.planModeNousResearchModelId : options.actModeNousResearchModelId,
})
case "wandb":
return new WandbHandler({
onRetryAttempt: options.onRetryAttempt,
wandbApiKey: options.wandbApiKey,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
})
default:
return new AnthropicHandler({
onRetryAttempt: options.onRetryAttempt,
apiKey: options.apiKey,
anthropicBaseUrl: options.anthropicBaseUrl,
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
thinkingBudgetTokens:
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
})
}
}
export function buildApiHandler(configuration: ApiConfiguration, mode: Mode): ApiHandler {
const { planModeApiProvider, actModeApiProvider, ...options } = configuration
const apiProvider = mode === "plan" ? planModeApiProvider : actModeApiProvider
// Validate thinking budget tokens against model's maxTokens to prevent API errors
// wrapped in a try-catch for safety, but this should never throw
try {
const thinkingBudgetTokens = mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens
if (thinkingBudgetTokens && thinkingBudgetTokens > 0) {
const handler = createHandlerForProvider(apiProvider, options, mode)
const modelInfo = handler.getModel().info
if (modelInfo?.maxTokens && modelInfo.maxTokens > 0 && thinkingBudgetTokens > modelInfo.maxTokens) {
const clippedValue = modelInfo.maxTokens - 1
if (mode === "plan") {
options.planModeThinkingBudgetTokens = clippedValue
} else {
options.actModeThinkingBudgetTokens = clippedValue
}
} else {
return handler // don't rebuild unless its necessary
}
}
} catch (error) {
Logger.error("buildApiHandler error:", error)
}
return createHandlerForProvider(apiProvider, options, mode)
}
@@ -1,236 +0,0 @@
import { afterEach, describe, it } from "mocha"
import sinon from "sinon"
import "should"
import { anthropicModels } from "@shared/api"
import { ANTHROPIC_FAST_MODE_BETA, AnthropicHandler } from "../anthropic"
describe("AnthropicHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: readonly unknown[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
describe("getModel", () => {
it("should return the fast mode model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:fast",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-6:fast")
result.info.should.deepEqual(anthropicModels["claude-opus-4-6:fast"])
})
it("should return the 1m fast mode model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:1m:fast",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-6:1m:fast")
result.info.should.deepEqual(anthropicModels["claude-opus-4-6:1m:fast"])
})
it("should return the 4.7 model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-7")
result.info.should.deepEqual(anthropicModels["claude-opus-4-7"])
})
it("should return the 4.7 1m model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7:1m",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-7:1m")
result.info.should.deepEqual(anthropicModels["claude-opus-4-7:1m"])
})
it("should return the 4.8 model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-8",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-8")
result.info.should.deepEqual(anthropicModels["claude-opus-4-8"])
})
it("should return the 4.8 1m model when configured", () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-8:1m",
})
const result = handler.getModel()
result.id.should.equal("claude-opus-4-8:1m")
result.info.should.deepEqual(anthropicModels["claude-opus-4-8:1m"])
})
})
describe("createMessage", () => {
it("should route fast mode requests through the beta messages API", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:fast",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
const betaCreate = sinon.stub().callsFake(function (this: { _client?: object }, _params: unknown) {
should.exist(this._client)
return Promise.resolve(createAsyncIterable())
})
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: betaCreate,
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.notCalled(standardCreate)
sinon.assert.calledOnce(betaCreate)
sinon.assert.calledWithMatch(betaCreate, {
model: "claude-opus-4-6",
betas: [ANTHROPIC_FAST_MODE_BETA],
speed: "fast",
stream: true,
})
})
it("should include the 1m beta when routing 1m fast mode requests through the beta messages API", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-6:1m:fast",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
const betaCreate = sinon.stub().callsFake(function (this: { _client?: object }, _params: unknown) {
should.exist(this._client)
return Promise.resolve(createAsyncIterable())
})
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: betaCreate,
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.notCalled(standardCreate)
sinon.assert.calledOnce(betaCreate)
sinon.assert.calledWithMatch(betaCreate, {
model: "claude-opus-4-6",
betas: [ANTHROPIC_FAST_MODE_BETA, "context-1m-2025-08-07"],
speed: "fast",
stream: true,
})
})
it("should include the 1m beta header for Claude Opus 4.7 1m requests", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7:1m",
reasoningEffort: "high",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: sinon.stub().resolves(createAsyncIterable()),
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.calledOnce(standardCreate)
const requestBody = standardCreate.firstCall.args[0] as Record<string, any>
const requestOptions = standardCreate.firstCall.args[1] as Record<string, any>
requestBody.model.should.equal("claude-opus-4-7")
requestBody.thinking.should.deepEqual({ type: "adaptive" })
requestOptions.should.deepEqual({
headers: {
"anthropic-beta": "context-1m-2025-08-07",
},
})
})
it("should use adaptive thinking and output_config for Claude Opus adaptive models", async () => {
const handler = new AnthropicHandler({
apiKey: "test-api-key",
apiModelId: "claude-opus-4-7",
reasoningEffort: "xhigh",
})
const standardCreate = sinon.stub().resolves(createAsyncIterable())
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
messages: {
create: standardCreate,
},
beta: {
messages: {
_client: {},
create: sinon.stub().resolves(createAsyncIterable()),
},
},
})
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
}
sinon.assert.calledOnce(standardCreate)
const requestBody = standardCreate.firstCall.args[0] as Record<string, any>
requestBody.should.have.property("thinking")
requestBody.thinking.should.deepEqual({ type: "adaptive" })
requestBody.should.have.property("output_config")
requestBody.output_config.should.deepEqual({ effort: "xhigh" })
should(requestBody.temperature).equal(undefined)
})
})
})
File diff suppressed because it is too large Load Diff
@@ -1,468 +0,0 @@
import { afterEach, beforeEach, describe, it } from "mocha"
import sinon from "sinon"
import "should"
import { ClaudeCodeHandler } from "@core/api/providers/claude-code"
import { ClineStorageMessage } from "@/shared/messages/content"
describe("ClaudeCodeHandler", () => {
let handler: ClaudeCodeHandler
let sandbox: sinon.SinonSandbox
beforeEach(() => {
sandbox = sinon.createSandbox()
handler = new ClaudeCodeHandler({
claudeCodePath: "/mock/path",
apiModelId: "claude-opus-4-1-20250805",
})
})
afterEach(() => {
sandbox.restore()
})
describe("token counting", () => {
it("should correctly handle token usage from assistant messages", async () => {
// The 'input_tokens' field represents the TOTAL number of input tokens used.
// See https://docs.anthropic.com/en/api/messages#usage-object
// Mock the runClaudeCode function
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
// Create a proper async generator mock for the Claude Code response
async function* mockGenerator() {
// First yield the system init
yield {
type: "system",
subtype: "init",
apiKeySource: "api",
}
// Yield assistant message with usage data
// Example: If base input is 70 tokens, cache read is 20, and cache creation is 10,
// then input_tokens from Anthropic API will be 100 (70 + 20 + 10)
yield {
type: "assistant",
message: {
content: [
{
type: "text",
text: "Test response",
},
],
usage: {
input_tokens: 100, // Total including cache (per Anthropic docs)
output_tokens: 50,
cache_read_input_tokens: 20, // Already included in input_tokens
cache_creation_input_tokens: 10, // Already included in input_tokens
},
stop_reason: "end_turn",
},
}
// Yield result with cost
yield {
type: "result",
result: {},
total_cost_usd: 0.005,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const usageData: any[] = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "usage") {
usageData.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
cacheReadTokens: chunk.cacheReadTokens,
cacheWriteTokens: chunk.cacheWriteTokens,
totalCost: chunk.totalCost,
})
}
}
// Verify token counting follows Anthropic API specification
usageData.should.have.length(1)
usageData[0].should.deepEqual({
inputTokens: 100, // Total including cache tokens (per Anthropic API docs)
outputTokens: 50,
cacheReadTokens: 20, // Tracked separately for reporting
cacheWriteTokens: 10, // Tracked separately for reporting
totalCost: 0.005,
})
// CRITICAL ASSERTION: Verify that input_tokens is NOT inflated by re-adding cache tokens
// The bug would have caused inputTokens to be incorrectly calculated as 130 (100 + 20 + 10)
// The fix ensures it remains 100, as per Anthropic's specification
usageData[0].inputTokens.should.equal(100) // Correct: matches API response
usageData[0].inputTokens.should.not.equal(130) // Would be wrong: double-counting cache tokens
})
it("should handle missing usage fields with nullish coalescing", async () => {
// Mock the runClaudeCode function
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
// Create a proper async generator mock with missing/undefined usage fields
async function* mockGenerator() {
yield {
type: "assistant",
message: {
content: [
{
type: "text",
text: "Test response",
},
],
usage: {
input_tokens: 100,
output_tokens: 50,
// cache fields are undefined/missing
},
stop_reason: "end_turn",
},
}
yield {
type: "result",
result: {},
total_cost_usd: 0.005,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const usageData: any[] = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "usage") {
usageData.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
cacheReadTokens: chunk.cacheReadTokens,
cacheWriteTokens: chunk.cacheWriteTokens,
})
}
}
// Verify that undefined cache tokens default to 0
usageData.should.have.length(1)
usageData[0].should.deepEqual({
inputTokens: 100,
outputTokens: 50,
cacheReadTokens: 0, // Should default to 0
cacheWriteTokens: 0, // Should default to 0
})
})
it("should handle completely missing usage object", async () => {
// Mock the runClaudeCode function
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
// Create a proper async generator mock with missing usage object
async function* mockGenerator() {
yield {
type: "assistant",
message: {
content: [
{
type: "text",
text: "Test response",
},
],
// usage is undefined
usage: undefined,
stop_reason: "end_turn",
},
}
// Need to yield a result chunk to trigger usage data emission
yield {
type: "result",
result: {},
total_cost_usd: 0,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const usageData: any[] = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "usage") {
usageData.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
cacheReadTokens: chunk.cacheReadTokens,
cacheWriteTokens: chunk.cacheWriteTokens,
})
}
}
// All token counts should default to 0 when usage is undefined
usageData.should.have.length(1)
usageData[0].should.deepEqual({
inputTokens: 0,
outputTokens: 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
})
})
})
describe("error handling", () => {
it("should not crash when assistant message has empty content array", async () => {
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
async function* mockGenerator() {
yield {
type: "assistant",
message: {
content: [], // empty content — triggered TypeError in older code
usage: {
input_tokens: 10,
output_tokens: 0,
},
stop_reason: "end_turn",
},
}
yield {
type: "result",
result: {},
total_cost_usd: 0,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const chunks: any[] = []
// Should not throw
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
const usageChunk = chunks.find((c) => c.type === "usage")
usageChunk.should.be.ok()
usageChunk.inputTokens.should.equal(10)
})
it("should throw when result has is_error=true (e.g. rate limit with no assistant message)", async () => {
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
async function* mockGenerator() {
yield {
type: "system",
subtype: "init",
apiKeySource: "none",
}
yield {
type: "system",
subtype: "rate_limit_event",
message: "Rate limit hit",
retryAfterSeconds: 30,
}
// No assistant message — CLI hit rate limit and gave up
yield {
type: "result",
subtype: "error",
is_error: true,
result: "Rate limit exceeded",
total_cost_usd: 0,
duration_ms: 1000,
duration_api_ms: 500,
num_turns: 0,
session_id: "test",
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
let thrownError: Error | undefined
try {
for await (const _ of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
// consume
}
} catch (err) {
thrownError = err as Error
}
thrownError!.message.should.containEql("Rate limit exceeded")
})
it("should ignore rate_limit_event system messages without throwing", async () => {
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
async function* mockGenerator() {
yield {
type: "system",
subtype: "init",
apiKeySource: "none",
}
// Newer Claude Code CLI emits this during rate limiting
yield {
type: "system",
subtype: "rate_limit_event",
message: "Rate limit hit, retrying...",
retryAfterSeconds: 30,
}
yield {
type: "assistant",
message: {
content: [{ type: "text", text: "Response after retry" }],
usage: { input_tokens: 20, output_tokens: 10 },
stop_reason: "end_turn",
},
}
yield {
type: "result",
result: {},
total_cost_usd: 0,
}
}
runClaudeCodeStub.returns(mockGenerator() as any)
const textChunks: string[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
if (chunk.type === "text") textChunks.push(chunk.text)
}
textChunks.should.deepEqual(["Response after retry"])
})
})
describe("getModel", () => {
it("should return the correct model when specified", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-sonnet-4-5-20250929",
})
const model = handler.getModel()
model.id.should.equal("claude-sonnet-4-5-20250929")
})
it("should support Opus 4.6 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-6[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-6[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Opus 4.7 model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-7",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-7")
model.info.contextWindow.should.equal(200_000)
})
it("should support Opus 4.7 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-7[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-7[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Opus 4.8 model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-8",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-8")
model.info.contextWindow.should.equal(200_000)
})
it("should support Opus 4.8 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-opus-4-8[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-opus-4-8[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Opus 1m alias model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "opus[1m]",
})
const model = handler.getModel()
model.id.should.equal("opus[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Sonnet 1m alias model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "sonnet[1m]",
})
const model = handler.getModel()
model.id.should.equal("sonnet[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Sonnet 4.5 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-sonnet-4-5-20250929[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-sonnet-4-5-20250929[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should support Sonnet 4.6 1m model id", () => {
const handler = new ClaudeCodeHandler({
apiModelId: "claude-sonnet-4-6[1m]",
})
const model = handler.getModel()
model.id.should.equal("claude-sonnet-4-6[1m]")
model.info.contextWindow.should.equal(1_000_000)
})
it("should return default model when not specified", () => {
const handler = new ClaudeCodeHandler({})
const model = handler.getModel()
// The default model should be set
model.id.should.be.type("string")
model.info.should.be.type("object")
})
})
})
@@ -1,166 +0,0 @@
import "should"
import { openRouterDefaultModelInfo } from "@shared/api"
import sinon from "sinon"
import { ClineAccountService } from "@/services/account/ClineAccountService"
import { AuthService } from "@/services/auth/AuthService"
import { ClineHandler } from "../cline"
describe("ClineHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
const createHandler = (options: ConstructorParameters<typeof ClineHandler>[0]) => {
sinon.stub(ClineAccountService, "getInstance").returns({} as any)
sinon.stub(AuthService, "getInstance").returns({} as any)
return new ClineHandler(options)
}
it("should handle usage-only chunks when delta is missing", async () => {
const handler = createHandler({})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 17,
completion_tokens: 9,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "openai/gpt-4o-mini",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: 0,
inputTokens: 17,
outputTokens: 9,
totalCost: 0,
},
])
})
it("should read Anthropic-style cache creation and read tokens from usage chunks", async () => {
const handler = createHandler({})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 1000,
completion_tokens: 200,
prompt_tokens_details: {
cached_tokens: 500,
},
cache_creation_input_tokens: 300,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "anthropic/claude-sonnet-4.6",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 300,
cacheReadTokens: 500,
inputTokens: 200,
outputTokens: 200,
totalCost: 0,
},
])
})
it("should forward enableParallelToolCalling to OpenRouter payload", async () => {
const handler = createHandler({ enableParallelToolCalling: true })
const createStub = sinon.stub().resolves(createAsyncIterable([]))
const fakeClient = {
chat: {
completions: {
create: createStub,
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "openai/gpt-4o-mini",
info: openRouterDefaultModelInfo,
})
const tools = [
{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } },
] as any
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
// drain stream
}
const payload = createStub.firstCall.args[0]
payload.parallel_tool_calls.should.equal(true)
})
it("should send cache_control for qwen3.7-max without changing the selected Cline model id", async () => {
const handler = createHandler({
openRouterModelId: "qwen/qwen3.7-max",
openRouterModelInfo: openRouterDefaultModelInfo,
})
const createStub = sinon.stub().resolves(createAsyncIterable([]))
const fakeClient = {
chat: {
completions: {
create: createStub,
},
},
}
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
sinon.stub(handler as any, "getFreeModelIdSet").resolves(new Set())
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
// drain stream
}
handler.getModel().id.should.equal("qwen/qwen3.7-max")
const payload = createStub.firstCall.args[0]
payload.model.should.equal("qwen/qwen3.7-max")
payload.messages[0].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
payload.messages[1].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
})
})
@@ -1,97 +0,0 @@
import "should"
import sinon from "sinon"
import { FireworksHandler } from "../fireworks"
describe("FireworksHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
it("should handle usage-only chunks when delta is missing", async () => {
const handler = new FireworksHandler({
fireworksApiKey: "test-api-key",
fireworksModelId: "accounts/fireworks/models/llama-v3p1-8b-instruct",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 19,
completion_tokens: 4,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
inputTokens: 19,
outputTokens: 4,
cacheReadTokens: 0,
cacheWriteTokens: 0,
},
])
})
it("should read cache hits from prompt_tokens_details when hit tokens are not present", async () => {
const handler = new FireworksHandler({
fireworksApiKey: "test-api-key",
fireworksModelId: "accounts/fireworks/models/llama-v3p1-8b-instruct",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 60,
completion_tokens: 12,
prompt_tokens_details: { cached_tokens: 20 },
prompt_cache_miss_tokens: 40,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
inputTokens: 60,
outputTokens: 12,
cacheReadTokens: 20,
cacheWriteTokens: 40,
},
])
})
})
@@ -1,235 +0,0 @@
import "should"
import sinon from "sinon"
import { GeminiHandler } from "../gemini"
describe("GeminiHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
it("caps maxOutputTokens to 8192 for Flash models", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
apiModelId: "gemini-2.5-flash",
})
const generateContentStream = sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp-1",
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 20,
cachedContentTokenCount: 0,
thoughtsTokenCount: 0,
},
},
]),
)
sinon.stub(handler as any, "ensureClient").returns({
models: { generateContentStream },
} as any)
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
// Consume stream to trigger request execution.
}
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
requestArgs.config.should.have.property("maxOutputTokens", 8_192)
})
it("supports Gemini 3.5 Flash model metadata", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
apiModelId: "gemini-3.5-flash",
})
const model = handler.getModel()
model.id.should.equal("gemini-3.5-flash")
model.info.contextWindow!.should.equal(1_048_576)
model.info.inputPrice!.should.equal(1.5)
model.info.outputPrice!.should.equal(9)
model.info.cacheReadsPrice!.should.equal(0.15)
model.info.supportsReasoning!.should.equal(true)
const generateContentStream = sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp-35",
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 20,
cachedContentTokenCount: 0,
thoughtsTokenCount: 0,
},
},
]),
)
sinon.stub(handler as any, "ensureClient").returns({
models: { generateContentStream },
} as any)
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
// Consume stream to trigger request execution.
}
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
requestArgs.model.should.equal("gemini-3.5-flash")
requestArgs.config.should.have.property("maxOutputTokens", 8_192)
requestArgs.config.thinkingConfig.should.deepEqual({
thinkingBudget: undefined,
thinkingLevel: "LOW",
includeThoughts: true,
})
})
it("does not set maxOutputTokens for non-Flash models", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
apiModelId: "gemini-2.5-pro",
})
const generateContentStream = sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp-2",
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 20,
cachedContentTokenCount: 0,
thoughtsTokenCount: 0,
},
},
]),
)
sinon.stub(handler as any, "ensureClient").returns({
models: { generateContentStream },
} as any)
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
// Consume stream to trigger request execution.
}
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
requestArgs.config.should.not.have.property("maxOutputTokens")
})
it("should emit unique tool call IDs when multiple function calls share one responseId", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
})
const fakeClient = {
models: {
generateContentStream: sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp_1",
candidates: [
{
content: {
parts: [
{
functionCall: {
name: "read_file",
args: { path: ".nvmrc" },
},
},
],
},
},
],
},
{
responseId: "resp_1",
candidates: [
{
content: {
parts: [
{
functionCall: {
name: "read_file",
args: { path: ".gitattributes" },
},
},
],
},
},
],
},
]),
),
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const tools = [{ name: "read_file", description: "read file", parameters: { type: "OBJECT" } }] as any
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
if (chunk.type === "tool_calls") {
chunks.push(chunk)
}
}
chunks.should.have.length(2)
chunks[0].tool_call.function.id.should.equal("resp_1-tool-0")
chunks[1].tool_call.function.id.should.equal("resp_1-tool-1")
chunks[0].tool_call.call_id.should.equal(chunks[0].tool_call.function.id)
chunks[1].tool_call.call_id.should.equal(chunks[1].tool_call.function.id)
JSON.parse(chunks[0].tool_call.function.arguments).path.should.equal(".nvmrc")
JSON.parse(chunks[1].tool_call.function.arguments).path.should.equal(".gitattributes")
})
it("should preserve Gemini-provided functionCall.id when present", async () => {
const handler = new GeminiHandler({
geminiApiKey: "test-api-key",
})
const fakeClient = {
models: {
generateContentStream: sinon.stub().resolves(
createAsyncIterable([
{
responseId: "resp_2",
candidates: [
{
content: {
parts: [
{
functionCall: {
id: "call_alpha",
name: "read_file",
args: { path: ".nvmrc" },
},
},
],
},
},
],
},
]),
),
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const tools = [{ name: "read_file", description: "read file", parameters: { type: "OBJECT" } }] as any
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
if (chunk.type === "tool_calls") {
chunks.push(chunk)
}
}
chunks.should.have.length(1)
chunks[0].tool_call.function.id.should.equal("call_alpha")
chunks[0].tool_call.call_id.should.equal("call_alpha")
JSON.parse(chunks[0].tool_call.function.arguments).path.should.equal(".nvmrc")
})
})
@@ -1,326 +0,0 @@
import { LiteLlmHandler, type LiteLlmModelInfoResponse } from "@core/api/providers/litellm"
import { convertToOpenAiMessages } from "@core/api/transform/openai-format"
import { liteLlmModelInfoSaneDefaults } from "@shared/api" // used in getModel tests
import { expect } from "chai"
import sinon from "sinon"
import { StateManager } from "@/core/storage/StateManager" // used in getModel tests
import { ClineStorageMessage } from "@/shared/messages/content"
import { mockFetchForTesting } from "@/shared/net"
const fakeClient = {
chat: {
completions: {
create: sinon.stub(),
},
},
baseURL: "https://fake.example",
}
describe("LiteLlmHandler", () => {
const mockFetch = sinon.stub()
let doneMockingFetch: (value: any) => void = () => {}
const mockModelFetch = (modelInfo: LiteLlmModelInfoResponse["data"][number]) => {
mockFetch.resolves({
ok: true,
json: () =>
Promise.resolve({
data: [modelInfo],
}),
})
}
let handler: LiteLlmHandler
const mockHandlerChat = () => {
sinon.stub(handler, "ensureClient" as any).returns(fakeClient)
}
const initializeHandler = (model: string) => {
handler = new LiteLlmHandler({
liteLlmApiKey: "test-api-key",
liteLlmBaseUrl: "http://localhost:4000",
liteLlmUsePromptCache: true,
liteLlmModelId: model,
})
mockHandlerChat()
}
beforeEach(() => {
fakeClient.chat.completions.create.resetHistory()
mockFetchForTesting(mockFetch, () => {
return new Promise((resolve) => {
doneMockingFetch = resolve
})
})
// Configure the stub to return a stream that closes immediately with usage data
fakeClient.chat.completions.create.resolves(
createAsyncIterable([
{
choices: [{ delta: { content: "test response" } }],
},
{
choices: [{}],
usage: {
prompt_tokens: 100,
completion_tokens: 50,
cache_creation_input_tokens: 20,
cache_read_input_tokens: 10,
},
},
]),
)
})
afterEach(() => {
sinon.reset()
doneMockingFetch(void 0)
})
const createAsyncIterable = (data: any[] = []) => {
return {
[Symbol.asyncIterator]: async function* () {
yield* data
},
}
}
describe("prompt cache", () => {
const setModelData = (model: string, supportsPromptCaching: boolean) => {
mockModelFetch({
model_name: model,
litellm_params: {
model,
},
model_info: {
supports_prompt_caching: supportsPromptCaching,
input_cost_per_token: 0.01,
output_cost_per_token: 0.02,
},
})
}
describe("when the model doesn't support prompt caching", () => {
const model = "openai/gpt-5"
beforeEach(() => {
initializeHandler(model)
setModelData(model, false)
})
it("sends the system prompt and messages with the openai format", async () => {
const systemPrompt = "Test System Prompt"
const messages: ClineStorageMessage[] = [
{
role: "user",
content: "first message",
},
{
role: "assistant",
content: "first response",
},
{
role: "user",
content: [
{
type: "text",
text: "test",
},
{
type: "text",
text: "second message",
},
],
},
]
for await (const _ of handler.createMessage(systemPrompt, messages)) {
}
sinon.assert.calledOnce(fakeClient.chat.completions.create)
const callArgs = fakeClient.chat.completions.create.getCall(0).args[0]
const systemPromptMessage = callArgs.messages.shift()
expect(systemPromptMessage).to.deep.equal({
role: "system",
content: systemPrompt,
})
expect(callArgs.messages).to.deep.equal(convertToOpenAiMessages(messages))
})
})
describe("when the model supports prompt caching", () => {
const model = "anthropic/claude-sonnet-4-20250514"
beforeEach(() => {
initializeHandler(model)
setModelData(model, true)
})
it("inserts the cache control in the system prompt and the last two user messages", async () => {
const systemPrompt = "Test System Prompt"
const messages: ClineStorageMessage[] = [
{
role: "user",
content: "first message",
},
{
role: "assistant",
content: "first response",
},
{
role: "user",
content: [
{
type: "text",
text: "test",
},
{
type: "text",
text: "second message",
},
],
},
]
for await (const _ of handler.createMessage(systemPrompt, messages)) {
}
sinon.assert.calledOnce(fakeClient.chat.completions.create)
const callArgs = fakeClient.chat.completions.create.getCall(0).args[0]
expect(callArgs.messages[0]).to.deep.equal({
role: "system",
content: [
{
text: systemPrompt,
type: "text",
cache_control: {
type: "ephemeral",
},
},
],
})
const sentMessages = callArgs.messages
expect(sentMessages.length).to.equal(4)
const firstUserMessage = sentMessages[1]
expect(firstUserMessage).to.deep.equal({
role: "user",
content: [
{
type: "text",
text: "first message",
cache_control: {
type: "ephemeral",
},
},
],
})
const lastUserMessage = sentMessages[3]
expect(lastUserMessage.content[0]).to.deep.equal({
type: "text",
text: "test",
})
const lastContentBlock = lastUserMessage.content[lastUserMessage.content.length - 1]
expect(lastContentBlock).to.deep.equal({
type: "text",
text: "second message",
cache_control: {
type: "ephemeral",
},
})
expect(callArgs.model).to.be.a("string")
expect(callArgs.stream).to.equal(true)
expect(callArgs.stream_options).to.deep.equal({ include_usage: true })
})
})
})
describe("getModel", () => {
let stateManagerStub: sinon.SinonStub
beforeEach(() => {
stateManagerStub = sinon.stub(StateManager, "get").returns({
getModelInfo: () => null,
} as any)
})
afterEach(() => {
stateManagerStub.restore()
})
it("returns sane defaults when no liteLlmModelInfo option is provided", () => {
const h = new LiteLlmHandler({
liteLlmApiKey: "test",
liteLlmModelId: "some-model",
})
const model = h.getModel()
expect(model.id).to.equal("some-model")
expect(model.info.contextWindow).to.equal(liteLlmModelInfoSaneDefaults.contextWindow)
})
it("returns user-configured model info when liteLlmModelInfo is provided and no cache exists", () => {
const h = new LiteLlmHandler({
liteLlmApiKey: "test",
liteLlmModelId: "claude-sonnet-4-6",
liteLlmModelInfo: {
contextWindow: 1_000_000,
maxTokens: 8192,
supportsImages: true,
supportsPromptCache: true,
inputPrice: 3,
outputPrice: 15,
cacheWritesPrice: 3.75,
cacheReadsPrice: 0.3,
},
})
const model = h.getModel()
expect(model.id).to.equal("claude-sonnet-4-6")
expect(model.info.contextWindow).to.equal(1_000_000)
})
it("prefers StateManager cached model info over user-configured liteLlmModelInfo", () => {
const cachedInfo = {
contextWindow: 200_000,
maxTokens: 4096,
supportsImages: true,
supportsPromptCache: true,
inputPrice: 0,
outputPrice: 0,
}
stateManagerStub.returns({
getModelInfo: () => cachedInfo,
} as any)
const h = new LiteLlmHandler({
liteLlmApiKey: "test",
liteLlmModelId: "claude-sonnet-4-6",
liteLlmModelInfo: {
contextWindow: 1_000_000,
maxTokens: 8192,
supportsImages: true,
supportsPromptCache: true,
inputPrice: 3,
outputPrice: 15,
cacheWritesPrice: 3.75,
cacheReadsPrice: 0.3,
},
})
const model = h.getModel()
expect(model.info.contextWindow).to.equal(200_000)
})
})
})
@@ -1,53 +0,0 @@
import "should"
import { moonshotModels } from "@shared/api"
import type { ClineStorageMessage } from "@shared/messages/content"
import sinon from "sinon"
import { MoonshotHandler } from "../moonshot"
interface MoonshotRequestPayload {
model: string
temperature: number
max_tokens: number
}
describe("MoonshotHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: unknown[] = []): AsyncIterable<unknown> => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
it("supports kimi-k2.6 model metadata", async () => {
const handler = new MoonshotHandler({
moonshotApiKey: "test-api-key",
apiModelId: "kimi-k2.6",
})
const model = handler.getModel()
model.id.should.equal("kimi-k2.6")
model.info.should.deepEqual(moonshotModels["kimi-k2.6"])
const createStub = sinon.stub().resolves(createAsyncIterable([]))
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
chat: {
completions: {
create: createStub,
},
},
})
const messages: ClineStorageMessage[] = [{ role: "user", content: "hi" }]
for await (const _chunk of handler.createMessage("system", messages)) {
// Consume stream to trigger request execution.
}
const payload = createStub.firstCall.args[0] as MoonshotRequestPayload
payload.model.should.equal("kimi-k2.6")
payload.temperature.should.equal(moonshotModels["kimi-k2.6"].temperature)
payload.max_tokens.should.equal(moonshotModels["kimi-k2.6"].maxTokens)
})
})
@@ -1,93 +0,0 @@
import { expect } from "chai"
import { afterEach, describe, it } from "mocha"
import sinon from "sinon"
import { ClineStorageMessage } from "@/shared/messages/content"
import { ApiFormat } from "@/shared/proto/index.cline"
import { OcaHandler } from "../oca"
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
async function collectChunks(stream: AsyncGenerator<any>) {
const chunks: any[] = []
for await (const chunk of stream) {
chunks.push(chunk)
}
return chunks
}
describe("OcaHandler.createMessage", () => {
afterEach(() => {
sinon.restore()
})
it("routes OPENAI_RESPONSES models to createMessageResponsesApi", async () => {
const handler = new OcaHandler({
ocaModelInfo: { apiFormat: ApiFormat.OPENAI_RESPONSES } as any,
})
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
yield { type: "text", text: "chat" }
})
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
yield { type: "text", text: "responses" }
})
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
yield { type: "text", text: "messages" }
})
const chunks = await collectChunks(handler.createMessage("system", messages))
expect(chunks).to.deep.equal([{ type: "text", text: "responses" }])
sinon.assert.notCalled(chatStub)
sinon.assert.calledOnce(responsesStub)
sinon.assert.notCalled(messagesStub)
})
it("routes ANTHROPIC_CHAT models to createMessageMessagesApi", async () => {
const handler = new OcaHandler({
ocaModelInfo: { apiFormat: ApiFormat.ANTHROPIC_CHAT } as any,
})
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
yield { type: "text", text: "chat" }
})
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
yield { type: "text", text: "responses" }
})
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
yield { type: "text", text: "messages" }
})
const chunks = await collectChunks(handler.createMessage("system", messages))
expect(chunks).to.deep.equal([{ type: "text", text: "messages" }])
sinon.assert.notCalled(chatStub)
sinon.assert.notCalled(responsesStub)
sinon.assert.calledOnce(messagesStub)
})
it("defaults to createMessageChatApi for OPENAI_CHAT and undefined apiFormat", async () => {
for (const apiFormat of [ApiFormat.OPENAI_CHAT, undefined]) {
const handler = new OcaHandler({
ocaModelInfo: { apiFormat } as any,
})
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
yield { type: "text", text: "chat" }
})
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
yield { type: "text", text: "responses" }
})
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
yield { type: "text", text: "messages" }
})
const chunks = await collectChunks(handler.createMessage("system", messages))
expect(chunks).to.deep.equal([{ type: "text", text: "chat" }])
sinon.assert.calledOnce(chatStub)
sinon.assert.notCalled(responsesStub)
sinon.assert.notCalled(messagesStub)
}
})
})
@@ -1,231 +0,0 @@
import { afterEach, before, beforeEach, describe, it } from "mocha"
import "should"
import { ApiHandlerOptions } from "@shared/api"
import axios from "axios"
import sinon from "sinon"
import { ClineStorageMessage } from "@/shared/messages/content"
import { OllamaHandler } from "../ollama"
describe("OllamaHandler", () => {
let ollamaAvailable = false
// Check if Ollama is running before running tests
before(async function () {
this.timeout(5000)
try {
await axios.get("http://localhost:11434/api/version", { timeout: 2000 })
ollamaAvailable = true
} catch (_error) {
console.log("Ollama server not available, skipping tests")
ollamaAvailable = false
}
})
let handler: OllamaHandler
let options: ApiHandlerOptions
let clock: sinon.SinonFakeTimers
beforeEach(() => {
options = {
actModeOllamaModelId: "llama2",
ollamaBaseUrl: "http://localhost:11434",
}
handler = new OllamaHandler(options)
// Use fake timers for testing timeouts
clock = sinon.useFakeTimers()
})
afterEach(() => {
clock.restore()
sinon.restore()
})
describe("createMessage", () => {
it("should handle successful responses", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(5000)
// Ensure client is initialized
const client = (handler as any).ensureClient()
// Mock the Ollama client's chat method
const chatStub = sinon.stub(client, "chat").resolves({
[Symbol.asyncIterator]: async function* () {
yield {
message: { content: "Hello, world!" },
eval_count: 10,
prompt_eval_count: 20,
}
},
} as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const result = []
const usageInfo = []
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "text") {
result.push(chunk.text)
} else if (chunk.type === "usage") {
usageInfo.push({
inputTokens: chunk.inputTokens,
outputTokens: chunk.outputTokens,
})
}
}
// Verify the results
result.should.deepEqual(["Hello, world!"])
usageInfo.should.deepEqual([{ inputTokens: 20, outputTokens: 10 }])
chatStub.calledOnce.should.be.true()
})
it("should handle timeout errors", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(10000)
// Restore real timers for this test
clock.restore()
// Create a handler with a very short timeout for testing
const testHandler = new OllamaHandler(options)
// Replace the createMessage method with one that has a shorter timeout
testHandler.createMessage = async function* (_systemPrompt, _messages) {
try {
// Create a promise that rejects after a short timeout
const timeoutPromise = new Promise<never>((_, reject) => {
setTimeout(() => reject(new Error("Ollama request timed out after 120 seconds")), 100)
})
// Create a promise that never resolves
const neverPromise = new Promise(() => {})
// Race them
await Promise.race([timeoutPromise, neverPromise])
} catch (error: any) {
// Enhance error reporting
console.error(`Ollama API error: ${error.message}`)
throw error
}
}
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
// Start the request and catch the error
let errorMessage = ""
try {
for await (const _ of testHandler.createMessage(systemPrompt, messages)) {
// This should not be reached
}
} catch (error: any) {
errorMessage = error.message
}
// Check the result
errorMessage.should.equal("Ollama request timed out after 120 seconds")
// Restore the fake timers for other tests
clock = sinon.useFakeTimers()
})
it("should retry on errors when using the withRetry decorator", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(10000)
// Restore real timers for this test
clock.restore()
// Ensure client is initialized and mock the Ollama client's chat method to fail on first call and succeed on second
const client = (handler as any).ensureClient()
const chatStub = sinon.stub(client, "chat")
// First call throws an error
chatStub.onFirstCall().rejects(new Error("API Error"))
// Second call succeeds
chatStub.onSecondCall().resolves({
[Symbol.asyncIterator]: async function* () {
yield {
message: { content: "Success after retry" },
}
},
} as any)
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const result = []
// Add a small delay to ensure the retry mechanism has time to work
await new Promise((resolve) => setTimeout(resolve, 100))
// Collect the results
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
if (chunk.type === "text") {
result.push(chunk.text)
}
}
// Verify the results
result.should.deepEqual(["Success after retry"])
chatStub.calledTwice.should.be.true()
// Restore the fake timers for other tests
clock = sinon.useFakeTimers()
})
it("should handle stream processing errors", async function () {
if (!ollamaAvailable) {
this.skip()
}
this.timeout(10000)
// Restore real timers for this test
clock.restore()
// Create a handler with a custom implementation for testing
const testHandler = new OllamaHandler(options)
// Replace the createMessage method with one that simulates a stream error
testHandler.createMessage = async function* (_systemPrompt, _messages) {
// First yield a successful chunk
yield {
type: "text",
text: "Partial response",
}
// Then throw an error in the stream
throw new Error("Ollama stream processing error: Stream error")
}
const systemPrompt = "You are a helpful assistant."
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
const result = []
// Collect the results and catch the error
let errorMessage = ""
try {
for await (const chunk of testHandler.createMessage(systemPrompt, messages)) {
if (chunk.type === "text") {
result.push(chunk.text)
}
}
} catch (error: any) {
errorMessage = error.message
}
// Verify the results
errorMessage.should.equal("Ollama stream processing error: Stream error")
result.should.deepEqual(["Partial response"])
// Restore the fake timers for other tests
clock = sinon.useFakeTimers()
})
})
})
@@ -1,164 +0,0 @@
import "should"
import { openRouterDefaultModelInfo } from "@shared/api"
import sinon from "sinon"
import { OpenRouterHandler } from "../openrouter"
describe("OpenRouterHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
const tools = [{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } }] as any
it("should handle usage-only chunks when delta is missing", async () => {
const handler = new OpenRouterHandler({
openRouterApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 13,
completion_tokens: 5,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "openai/gpt-4o-mini",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: 0,
inputTokens: 13,
outputTokens: 5,
totalCost: 0,
},
])
})
it("should read cache_write_tokens from prompt_tokens_details", async () => {
const handler = new OpenRouterHandler({
openRouterApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 1000,
completion_tokens: 200,
prompt_tokens_details: {
cached_tokens: 500,
cache_write_tokens: 300,
},
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: "anthropic/claude-sonnet-4.6",
info: openRouterDefaultModelInfo,
})
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 300,
cacheReadTokens: 500,
inputTokens: 200,
outputTokens: 200,
totalCost: 0,
},
])
})
type ParallelToolCallsTestCase = {
modelId: string
enableParallelToolCalling: boolean
expectedParallelToolCalls: boolean
}
const parallelToolCallsTestCases: ParallelToolCallsTestCase[] = [
{
modelId: "openai/gpt-4o-mini",
enableParallelToolCalling: true,
expectedParallelToolCalls: true,
},
{
modelId: "openai/gpt-4o-mini",
enableParallelToolCalling: false,
expectedParallelToolCalls: false,
},
{
modelId: "google/gemini-3-flash-preview",
enableParallelToolCalling: true,
expectedParallelToolCalls: true,
},
]
for (const testCase of parallelToolCallsTestCases) {
const settingLabel = testCase.enableParallelToolCalling ? "enabled" : "disabled"
it(`should set parallel_tool_calls=${testCase.expectedParallelToolCalls} for ${testCase.modelId} when setting is ${settingLabel}`, async () => {
const handler = new OpenRouterHandler({
openRouterApiKey: "test-api-key",
enableParallelToolCalling: testCase.enableParallelToolCalling,
})
const createStub = sinon.stub().resolves(createAsyncIterable([]))
const fakeClient = {
chat: {
completions: {
create: createStub,
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
sinon.stub(handler, "getModel").returns({
id: testCase.modelId,
info: openRouterDefaultModelInfo,
})
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
// drain stream
}
const payload = createStub.firstCall.args[0]
payload.parallel_tool_calls.should.equal(testCase.expectedParallelToolCalls)
})
}
})
@@ -1,132 +0,0 @@
import "should"
import { Anthropic } from "@anthropic-ai/sdk"
import { SapAiCoreHandler } from "../sapaicore"
describe("SapAiCoreHandler", () => {
let handler: SapAiCoreHandler
beforeEach(() => {
const mockOptions = {
sapAiCoreClientId: "test-client-id",
sapAiCoreClientSecret: "test-client-secret",
sapAiCoreTokenUrl: "https://test.auth.sap.com",
sapAiResourceGroup: "default",
sapAiCoreBaseUrl: "https://test.api.sap.com",
apiModelId: "anthropic--claude-3.5-sonnet",
}
handler = new SapAiCoreHandler(mockOptions)
})
describe("image processing", () => {
// Test image processing through the public interface
// This tests the complete flow including processImageContent internally
it("should handle image processing for Claude 4 models", () => {
// Create handler with Claude 4 model
const claude4Handler = new SapAiCoreHandler({
sapAiCoreClientId: "test-client-id",
sapAiCoreClientSecret: "test-client-secret",
sapAiCoreTokenUrl: "https://test.auth.sap.com",
sapAiResourceGroup: "default",
sapAiCoreBaseUrl: "https://test.api.sap.com",
apiModelId: "anthropic--claude-4-sonnet",
})
const model = claude4Handler.getModel()
model.id.should.equal("anthropic--claude-4-sonnet")
model.info.should.have.property("supportsImages", true)
})
it("should create proper user readable request with images", () => {
const testImageData =
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=="
const userContent: (Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[] = [
{
type: "text",
text: "Here's an image:",
},
{
type: "image",
source: {
type: "base64",
media_type: "image/png",
data: testImageData,
},
},
]
const result = handler.createUserReadableRequest(userContent)
result.should.have.property("model")
result.should.have.property("max_tokens")
result.should.have.property("system")
result.should.have.property("messages")
result.messages.should.be.Array()
result.messages[1].should.have.property("role", "user")
result.messages[1].should.have.property("content", userContent)
})
it("should support different Claude model variants", () => {
const modelVariants = [
"anthropic--claude-4.6-sonnet",
"anthropic--claude-4-sonnet",
"anthropic--claude-4-opus",
"anthropic--claude-3.7-sonnet",
"anthropic--claude-3.5-sonnet",
"anthropic--claude-3-sonnet",
"anthropic--claude-3-haiku",
"anthropic--claude-3-opus",
]
modelVariants.forEach((modelId) => {
const testHandler = new SapAiCoreHandler({
apiModelId: modelId,
})
const model = testHandler.getModel()
model.id.should.equal(modelId)
model.info.should.have.property("maxTokens")
model.info.should.have.property("contextWindow")
})
})
})
describe("getModel", () => {
it("should return default model when no apiModelId is provided", () => {
const result = handler.getModel()
result.should.have.property("id")
result.should.have.property("info")
result.info.should.have.property("maxTokens")
})
it("should return specified model when apiModelId is provided", () => {
const customHandler = new SapAiCoreHandler({
apiModelId: "anthropic--claude-4-sonnet",
})
const result = customHandler.getModel()
result.id.should.equal("anthropic--claude-4-sonnet")
})
})
describe("createUserReadableRequest", () => {
it("should create a readable request format", () => {
const userContent: Anthropic.TextBlockParam[] = [
{
type: "text",
text: "Hello, world!",
},
]
const result = handler.createUserReadableRequest(userContent)
result.should.have.property("model")
result.should.have.property("max_tokens")
result.should.have.property("system")
result.should.have.property("messages")
result.should.have.property("tools")
result.should.have.property("tool_choice")
})
})
})
@@ -1,138 +0,0 @@
import "should"
import { openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import sinon from "sinon"
import { VercelAIGatewayHandler } from "../vercel-ai-gateway"
describe("VercelAIGatewayHandler", () => {
afterEach(() => {
sinon.restore()
})
const createAsyncIterable = (data: any[] = []) => ({
[Symbol.asyncIterator]: async function* () {
yield* data
},
})
describe("getModel", () => {
it("should return configured model and info when both are provided", () => {
const customModelInfo = {
...openRouterDefaultModelInfo,
maxTokens: 123456,
}
const handler = new VercelAIGatewayHandler({
openRouterModelId: "google/gemini-3.1-pro-preview",
openRouterModelInfo: customModelInfo,
})
const result = handler.getModel()
result.id.should.equal("google/gemini-3.1-pro-preview")
result.info.should.deepEqual(customModelInfo)
})
it("should preserve configured model ID when model info is missing", () => {
const handler = new VercelAIGatewayHandler({
openRouterModelId: "google/gemini-3.1-pro-preview",
})
const result = handler.getModel()
result.id.should.equal("google/gemini-3.1-pro-preview")
result.info.should.deepEqual(openRouterDefaultModelInfo)
})
it("should fall back to default model when model ID is missing", () => {
const handler = new VercelAIGatewayHandler({})
const result = handler.getModel()
result.id.should.equal(openRouterDefaultModelId)
result.info.should.deepEqual(openRouterDefaultModelInfo)
})
})
describe("createMessage", () => {
it("should handle usage-only chunks when delta is missing", async () => {
const handler = new VercelAIGatewayHandler({
vercelAiGatewayApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 11,
completion_tokens: 7,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: 0,
inputTokens: 11,
outputTokens: 7,
totalCost: 0,
},
])
})
it("should read Anthropic-style cache creation and read tokens from usage chunks", async () => {
const handler = new VercelAIGatewayHandler({
vercelAiGatewayApiKey: "test-api-key",
})
const fakeClient = {
chat: {
completions: {
create: sinon.stub().resolves(
createAsyncIterable([
{
choices: [{}],
usage: {
prompt_tokens: 1000,
completion_tokens: 200,
prompt_tokens_details: {
cached_tokens: 500,
},
cache_creation_input_tokens: 300,
},
},
]),
),
},
},
}
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
const chunks: any[] = []
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
chunks.push(chunk)
}
chunks.should.deepEqual([
{
type: "usage",
cacheWriteTokens: 300,
cacheReadTokens: 500,
inputTokens: 200,
outputTokens: 200,
totalCost: 0,
},
])
})
})
})
@@ -1,23 +0,0 @@
import "should"
import { vertexGlobalModels } from "@shared/api"
import { VertexHandler } from "../vertex"
describe("VertexHandler", () => {
it("supports Gemini 3.5 Flash model metadata", () => {
const handler = new VertexHandler({
vertexProjectId: "test-project",
vertexRegion: "global",
apiModelId: "gemini-3.5-flash",
})
const model = handler.getModel()
model.id.should.equal("gemini-3.5-flash")
model.info.contextWindow!.should.equal(1_048_576)
model.info.inputPrice!.should.equal(1.5)
model.info.outputPrice!.should.equal(9)
model.info.cacheReadsPrice!.should.equal(0.15)
model.info.supportsGlobalEndpoint!.should.equal(true)
model.info.supportsReasoning!.should.equal(true)
vertexGlobalModels.should.have.property("gemini-3.5-flash")
})
})
@@ -1,42 +0,0 @@
import "should"
import { openAiModelInfoSaneDefaults, wandbDefaultModelId, wandbModels } from "@shared/api"
import { WandbHandler } from "../wandb"
describe("WandbHandler", () => {
it("returns known catalog model metadata when model id is recognized", () => {
const modelId = "meta-llama/Llama-3.3-70B-Instruct"
const handler = new WandbHandler({
wandbApiKey: "test-api-key",
apiModelId: modelId,
})
const model = handler.getModel()
model.id.should.equal(modelId)
model.info.should.deepEqual(wandbModels[modelId])
})
it("passes through an explicit unknown model id instead of silently falling back", () => {
const unknownModelId = "moonshotai/Kimi-K2.5"
const handler = new WandbHandler({
wandbApiKey: "test-api-key",
apiModelId: unknownModelId,
})
const model = handler.getModel()
model.id.should.equal(unknownModelId)
model.info.should.deepEqual(openAiModelInfoSaneDefaults)
})
it("uses the default W&B model when no model id is configured", () => {
const handler = new WandbHandler({
wandbApiKey: "test-api-key",
})
const model = handler.getModel()
model.id.should.equal(wandbDefaultModelId)
model.info.should.deepEqual(wandbModels[wandbDefaultModelId])
})
})
@@ -1,327 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { GenerateContentConfig, GoogleGenAI } from "@google/genai"
import { ModelInfo } from "@shared/api"
import OpenAI from "openai"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { convertAnthropicMessageToGemini } from "../transform/gemini-format"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface AIhubmixHandlerOptions extends CommonApiHandlerOptions {
apiKey?: string
baseURL?: string
appCode?: string
modelId?: string
modelInfo?: ModelInfo
thinkingBudgetTokens?: number
}
export class AIhubmixHandler implements ApiHandler {
private options: AIhubmixHandlerOptions
private anthropicClient: Anthropic | undefined
private openaiClient: OpenAI | undefined
private geminiClient: GoogleGenAI | undefined
constructor(options: AIhubmixHandlerOptions) {
const { baseURL, appCode, ...rest } = options
this.options = {
baseURL: baseURL ?? "https://aihubmix.com",
appCode: appCode ?? "KUWF9311",
...rest,
}
}
private ensureAnthropicClient(): Anthropic {
if (!this.anthropicClient) {
if (!this.options.apiKey) {
throw new Error("AIhubmix API key is required")
}
try {
this.anthropicClient = new Anthropic({
apiKey: this.options.apiKey,
baseURL: this.options.baseURL,
defaultHeaders: {
"APP-Code": this.options.appCode,
...buildExternalBasicHeaders(),
},
})
} catch (error) {
throw new Error(`Error creating Anthropic client: ${error.message}`)
}
}
return this.anthropicClient
}
private ensureOpenaiClient(): OpenAI {
if (!this.openaiClient) {
if (!this.options.apiKey) {
throw new Error("AIhubmix API key is required")
}
try {
this.openaiClient = new OpenAI({
apiKey: this.options.apiKey,
baseURL: `${this.options.baseURL}/v1`,
defaultHeaders: {
"APP-Code": this.options.appCode,
...buildExternalBasicHeaders(),
},
})
} catch (error) {
throw new Error(`Error creating OpenAI client: ${error.message}`)
}
}
return this.openaiClient
}
private ensureGeminiClient(): GoogleGenAI {
if (!this.geminiClient) {
if (!this.options.apiKey) {
throw new Error("AIhubmix API key is required")
}
try {
this.geminiClient = new GoogleGenAI({
apiKey: this.options.apiKey,
httpOptions: {
baseUrl: `${this.options.baseURL}/gemini`,
headers: {
// @ts-expect-error
"APP-Code": this.options.appCode,
Authorization: `Bearer ${this.options.apiKey ?? ""}`,
...buildExternalBasicHeaders(),
},
},
})
} catch (error) {
throw new Error(`Error creating Gemini client: ${error.message}`)
}
}
return this.geminiClient
}
private routeModel(modelName: string): "anthropic" | "openai" | "gemini" | "openai-response" {
const id = modelName || ""
if (id.startsWith("claude")) {
return "anthropic"
}
if (id.startsWith("gemini") && !id.endsWith("-nothink") && !id.endsWith("-search")) {
return "gemini"
}
if (id === "gpt-5-pro" || id === "gpt-5-codex") {
return "openai-response"
}
return "openai"
}
private fixToolChoice(requestBody: any): any {
if (requestBody.tools?.length === 0 && requestBody.tool_choice) {
delete requestBody.tool_choice
}
return requestBody
}
@withRetry()
async *createMessage(systemPrompt: string, messages: any[]): ApiStream {
const modelId = this.options.modelId || ""
const route = this.routeModel(modelId)
switch (route) {
case "anthropic":
yield* this.createAnthropicMessage(systemPrompt, messages)
break
case "gemini":
yield* this.createGeminiMessage(systemPrompt, messages)
break
case "openai-response":
yield* this.createOpenaiResponseMessage(systemPrompt, messages)
break
case "openai":
yield* this.createOpenaiMessage(systemPrompt, messages)
break
default:
throw new Error(`Unsupported model route: ${route}`)
}
}
private async *createAnthropicMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureAnthropicClient()
const modelId = this.options.modelId || "claude-3-5-sonnet-20241022"
// Sanitize messages to remove Cline-specific fields like call_id that are not allowed by Anthropic API
const sanitizedMessages = sanitizeAnthropicMessages(messages, false)
const stream = await client.messages.create({
model: modelId,
temperature: 0,
max_tokens: this.options.modelInfo?.maxTokens || 8192,
system: [{ text: systemPrompt, type: "text" }],
messages: sanitizedMessages,
stream: true,
})
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start":
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
break
case "message_delta":
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage.output_tokens || 0,
}
break
case "content_block_start":
if (chunk.content_block.type === "text") {
yield {
type: "text",
text: chunk.content_block.text,
}
}
break
case "content_block_delta":
if (chunk.delta.type === "text_delta") {
yield {
type: "text",
text: chunk.delta.text,
}
}
break
}
}
}
private async *createOpenaiResponseMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureOpenaiClient()
const modelId = this.options.modelId || "gpt-4o-mini"
const input = (messages || []).map((m: any) => {
const role = m.role || "user"
const contentArray = Array.isArray(m.content) ? m.content : [{ type: "text", text: m.content }]
const content = contentArray
.filter((c: any) => c != null)
.map((c: any) => {
if (c.type === "image" || c.type === "input_image" || c.type === "image_url") {
return { type: "input_image", image_url: c.image_url || c.url || c.source?.url }
}
const text = c.text ?? (typeof c === "string" ? c : "")
return { type: role === "assistant" ? "output_text" : "input_text", text }
})
return { role, content }
})
const stream = await (client as any).responses.stream({
model: modelId,
instructions: systemPrompt,
input,
})
for await (const event of stream as any) {
if (event?.type === "response.output_text.delta") {
yield { type: "text", text: event.delta || "" }
continue
}
if (event?.type === "response.completed") {
const usage = event.response?.usage || {}
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
}
continue
}
if (event?.type === "response.error") {
throw new Error(event.error?.message || "responses error")
}
}
}
private async *createOpenaiMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureOpenaiClient()
const modelId = this.options.modelId || "gpt-4o-mini"
const openaiMessages = [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
const requestBody = {
model: modelId,
messages: openaiMessages,
temperature: 0,
stream: true,
}
const fixedRequestBody = this.fixToolChoice(requestBody)
const stream = await client.chat.completions.create(fixedRequestBody)
for await (const chunk of stream as any) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
private async *createGeminiMessage(systemPrompt: string, messages: any[]): ApiStream {
const client = this.ensureGeminiClient()
const modelId = this.options.modelId || "gemini-2.0-flash-exp"
const contents = messages.map(convertAnthropicMessageToGemini)
const requestConfig: GenerateContentConfig = {
systemInstruction: systemPrompt,
temperature: 0,
}
if (this.options.thinkingBudgetTokens) {
requestConfig.thinkingConfig = {
thinkingBudget: this.options.thinkingBudgetTokens,
includeThoughts: true,
}
}
const stream = await client.models.generateContentStream({
model: modelId,
contents,
config: requestConfig,
})
for await (const chunk of stream as any) {
if (chunk?.text) {
yield { type: "text", text: chunk.text }
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.modelId || "gpt-4o-mini",
info: this.options.modelInfo || {
maxTokens: 8192,
contextWindow: 128000,
supportsImages: true,
supportsPromptCache: false,
description: "AIhubmix unified model provider",
},
}
}
}
@@ -1,316 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import type {
MessageCreateParamsStreaming as BetaMessageCreateParamsStreaming,
BetaRawMessageStreamEvent,
} from "@anthropic-ai/sdk/resources/beta/messages/messages"
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
import type { MessageCreateParamsStreaming as AnthropicMessageCreateParamsStreaming } from "@anthropic-ai/sdk/resources/messages/messages"
import { Stream as AnthropicStream } from "@anthropic-ai/sdk/streaming"
import {
ANTHROPIC_FAST_MODE_SUFFIX,
AnthropicModelId,
anthropicDefaultModelId,
anthropicModels,
CLAUDE_SONNET_1M_SUFFIX,
ModelInfo,
} from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { ApiStream } from "../transform/stream"
export const ANTHROPIC_FAST_MODE_BETA = "fast-mode-2026-02-01"
interface AnthropicHandlerOptions extends CommonApiHandlerOptions {
apiKey?: string
anthropicBaseUrl?: string
apiModelId?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
}
export class AnthropicHandler implements ApiHandler {
private options: AnthropicHandlerOptions
private client: Anthropic | undefined
constructor(options: AnthropicHandlerOptions) {
this.options = options
}
private ensureClient(): Anthropic {
if (!this.client) {
if (!this.options.apiKey) {
throw new Error("Anthropic API key is required")
}
try {
this.client = new Anthropic({
apiKey: this.options.apiKey,
baseURL: this.options.anthropicBaseUrl || undefined,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating Anthropic client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: AnthropicTool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
let stream: AnthropicStream<Anthropic.RawMessageStreamEvent> | AsyncIterable<BetaRawMessageStreamEvent>
const useFastMode = model.id.endsWith(ANTHROPIC_FAST_MODE_SUFFIX)
const baseModelId = useFastMode ? model.id.slice(0, -ANTHROPIC_FAST_MODE_SUFFIX.length) : model.id
const modelId = baseModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
? baseModelId.slice(0, -CLAUDE_SONNET_1M_SUFFIX.length)
: baseModelId
const enable1mContextWindow = baseModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
const fastModeBetas = enable1mContextWindow
? [ANTHROPIC_FAST_MODE_BETA, "context-1m-2025-08-07"]
: [ANTHROPIC_FAST_MODE_BETA]
const createFastModeMessage = (
body: AnthropicMessageCreateParamsStreaming,
): Promise<AsyncIterable<BetaRawMessageStreamEvent>> => {
return (
client.beta.messages.create as unknown as (
params: BetaMessageCreateParamsStreaming & { speed: "fast" },
) => Promise<AsyncIterable<BetaRawMessageStreamEvent>>
)({
...body,
betas: fastModeBetas,
speed: "fast",
})
}
const budget_tokens = this.options.thinkingBudgetTokens || 0
// Tools are available only when native tools are enabled.
const nativeToolsOn = tools?.length && tools?.length > 0
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
// Claude Opus 4.5+ uses adaptive thinking instead of budgeted extended thinking.
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budget_tokens)
: undefined
const adaptiveThinkingEnabled = adaptiveThinking?.enabled === true
const adaptiveThinkingEffort = adaptiveThinking?.effort
const thinkingEnabled = isAdaptiveThinkingModel ? adaptiveThinkingEnabled : reasoningOn
const thinkingConfig = thinkingEnabled
? isAdaptiveThinkingModel
? ({ type: "adaptive" } as any)
: { type: "enabled", budget_tokens: budget_tokens }
: undefined
const outputConfig = isAdaptiveThinkingModel && adaptiveThinkingEffort ? { effort: adaptiveThinkingEffort } : undefined
if (model.info.supportsPromptCache) {
const anthropicMessages = sanitizeAnthropicMessages(messages, true)
const requestBody: AnthropicMessageCreateParamsStreaming & Record<string, unknown> = {
model: modelId,
thinking: thinkingConfig,
max_tokens: model.info.maxTokens || 8192,
// "Thinking isn't compatible with temperature, top_p, or top_k modifications as well as forced tool use."
// (https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#important-considerations-when-using-extended-thinking)
// Adaptive Claude Opus models do not support temperature.
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
system: [
{
text: systemPrompt,
type: "text",
cache_control: { type: "ephemeral" },
},
], // setting cache breakpoint for system prompt so new tasks can reuse it
messages: anthropicMessages,
// tools, // cache breakpoints go from tools > system > messages, and since tools dont change, we can just set the breakpoint at the end of system (this avoids having to set a breakpoint at the end of tools which by itself does not meet min requirements for haiku caching)
stream: true,
tools: nativeToolsOn ? tools : undefined,
// tool_choice options:
// - none: disables tool use, even if tools are provided. Claude will not call any tools.
// - auto: allows Claude to decide whether to call any provided tools or not. This is the default value when tools are provided.
// - any: tells Claude that it must use one of the provided tools, but doesnt force a particular tool.
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
tool_choice: nativeToolsOn && !thinkingEnabled ? { type: "any" } : undefined,
}
if (outputConfig) {
requestBody.output_config = outputConfig
}
stream = useFastMode
? await createFastModeMessage(requestBody)
: await client.messages.create(
requestBody,
(() => {
// 1m context window beta header
if (enable1mContextWindow) {
return {
headers: {
"anthropic-beta": "context-1m-2025-08-07",
},
}
}
return undefined
})(),
)
} else {
const requestBody: AnthropicMessageCreateParamsStreaming & Record<string, unknown> = {
model: modelId,
max_tokens: model.info.maxTokens || 8192,
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
system: [{ text: systemPrompt, type: "text" }],
messages: sanitizeAnthropicMessages(messages, false),
tools: nativeToolsOn ? tools : undefined,
tool_choice: thinkingEnabled ? undefined : { type: "auto" },
stream: true,
thinking: thinkingConfig,
}
if (outputConfig) {
requestBody.output_config = outputConfig
}
stream = useFastMode ? await createFastModeMessage(requestBody) : await client.messages.create(requestBody)
}
const lastStartedToolCall = { id: "", name: "", arguments: "" }
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start":
{
// tells us cache reads/writes/input/output
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
}
break
case "message_delta":
// tells us stop_reason, stop_sequence, and output tokens along the way and at the end of the message
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage.output_tokens || 0,
}
break
case "message_stop":
// no usage data, just an indicator that the message is done
break
case "content_block_start":
switch (chunk.content_block.type) {
case "thinking":
yield {
type: "reasoning",
reasoning: chunk.content_block.thinking || "",
signature: chunk.content_block.signature,
}
break
case "redacted_thinking":
// Content is encrypted, and we don't to pass placeholder text back to the API
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
redacted_data: chunk.content_block.data,
}
break
case "tool_use":
if (chunk.content_block.id && chunk.content_block.name) {
// Convert Anthropic tool_use to OpenAI-compatible format
lastStartedToolCall.id = chunk.content_block.id
lastStartedToolCall.name = chunk.content_block.name
lastStartedToolCall.arguments = ""
}
break
case "text":
// we may receive multiple text blocks, in which case just insert a line break between them
if (chunk.index > 0) {
yield {
type: "text",
text: "\n",
}
}
yield {
type: "text",
text: chunk.content_block.text,
}
break
}
break
case "content_block_delta":
switch (chunk.delta.type) {
case "thinking_delta":
// 'reasoning' type just displays in the UI, but ant_thinking will be used to send the thinking traces back to the API
yield {
type: "reasoning",
reasoning: chunk.delta.thinking,
}
break
case "signature_delta":
// It's used when sending the thinking block back to the API
// API expects this in completed form, not as array of deltas
if (chunk.delta.signature) {
yield {
type: "reasoning",
reasoning: "", // reasoning text is already sent via thinking_delta
signature: chunk.delta.signature,
}
}
break
case "text_delta":
yield {
type: "text",
text: chunk.delta.text,
}
break
case "input_json_delta":
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
// // Convert Anthropic tool_use to OpenAI-compatible format
yield {
type: "tool_calls",
tool_call: {
...lastStartedToolCall,
function: {
...lastStartedToolCall,
id: lastStartedToolCall.id,
name: lastStartedToolCall.name,
arguments: chunk.delta.partial_json,
},
},
}
}
break
}
break
case "content_block_stop":
lastStartedToolCall.id = ""
lastStartedToolCall.name = ""
lastStartedToolCall.arguments = ""
break
}
}
}
getModel(): { id: AnthropicModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in anthropicModels) {
const id = modelId as AnthropicModelId
return { id, info: anthropicModels[id] }
}
return {
id: anthropicDefaultModelId,
info: anthropicModels[anthropicDefaultModelId],
}
}
}
@@ -1,200 +0,0 @@
import { AskSageModelId, askSageDefaultModelId, askSageDefaultURL, askSageModels, ModelInfo } from "@shared/api"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
interface AskSageHandlerOptions extends CommonApiHandlerOptions {
asksageApiKey?: string
asksageApiUrl?: string
apiModelId?: string
}
type AskSageRequest = {
system_prompt: string
message: {
user: "gpt" | "me"
message: string
}[]
model: string
dataset: "none"
usage: boolean
}
type AskSageUsage = {
model_tokens: {
completion_tokens: number
prompt_tokens: number
total_tokens: number
}
asksage_tokens: number
}
type AskSageResponse = {
uuid: string
status: number
// Response status
response: string
// Generated response message
message: string
// whether embedding & vector systems are down
embedding_down: boolean
vectors_down: boolean
// references if dataset is not none
references: string
type: string
added_obj: any
tool_calls: any
// usage metrics
usage: AskSageUsage | null
tool_responses: any[]
tool_calls_unified: any[]
}
export class AskSageHandler implements ApiHandler {
private options: AskSageHandlerOptions
private apiUrl: string
private apiKey: string
constructor(options: AskSageHandlerOptions) {
Logger.log("init api url", options.asksageApiUrl, askSageDefaultURL)
this.options = options
this.apiKey = options.asksageApiKey || ""
this.apiUrl = options.asksageApiUrl || askSageDefaultURL
if (!this.apiKey) {
throw new Error("AskSage API key is required")
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
try {
const model = this.getModel()
// Transform messages into AskSageRequest format
const formattedMessages = messages.map((msg) => {
const content = Array.isArray(msg.content)
? msg.content.map((block) => ("text" in block ? block.text : "")).join("")
: msg.content
return {
user: msg.role === "assistant" ? ("gpt" as const) : ("me" as const),
message: content,
}
})
const request: AskSageRequest = {
system_prompt: systemPrompt,
message: formattedMessages,
model: model.id,
dataset: "none",
usage: true,
}
// Make request to AskSage API
const response = await fetch(`${this.apiUrl}/query`, {
method: "POST",
headers: this.headers(),
body: JSON.stringify(request),
})
if (!response.ok) {
const error = await response.text()
throw new Error(`AskSage API error: ${error}`)
}
const result = (await response.json()) as AskSageResponse
if (!result.message) {
throw new Error("No content in AskSage response")
}
// Yield tool responses if they exist
if (result.tool_responses && result.tool_responses.length > 0) {
for (const toolResponse of result.tool_responses) {
yield {
type: "text",
text: `[Tool Response: ${JSON.stringify(toolResponse)}]\n`,
}
}
}
// Yield the main response text
yield {
type: "text",
text: result.message,
}
// Yield usage information if available
if (result.usage) {
yield {
type: "usage",
inputTokens: result.usage.model_tokens.prompt_tokens,
outputTokens: result.usage.model_tokens.completion_tokens,
cacheReadTokens: 0,
cacheWriteTokens: 0,
totalCost: result.usage.asksage_tokens, // Cost = Consumed AskSage tokens
}
}
} catch (error) {
if (error instanceof Error) {
throw new Error(`AskSage request failed: ${error.message}`)
}
throw error
}
}
async getApiStreamUsage() {
if (!this.apiKey) {
return undefined
}
try {
const response = await fetch(`${this.apiUrl}/count-monthly-tokens`, {
method: "POST",
headers: this.headers(),
body: JSON.stringify({ app_name: "asksage" }),
})
if (!response.ok) {
Logger.error("Failed to fetch AskSage usage", await response.text())
return undefined
}
const data = await response.json()
const usedTokens = data.response as number
return {
type: "usage" as const,
inputTokens: usedTokens,
outputTokens: 0,
}
} catch (error) {
Logger.error("Error fetching AskSage usage:", error)
return undefined
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in askSageModels) {
const id = modelId as AskSageModelId
return { id, info: askSageModels[id] }
}
return {
id: askSageDefaultModelId,
info: askSageModels[askSageDefaultModelId],
}
}
private headers() {
return {
"Content-Type": "application/json",
"x-access-tokens": this.apiKey,
...buildExternalBasicHeaders(),
}
}
}
@@ -1,172 +0,0 @@
import { BasetenModelId, basetenDefaultModelId, basetenModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
interface BasetenHandlerOptions extends CommonApiHandlerOptions {
basetenApiKey?: string
basetenModelId?: string
basetenModelInfo?: ModelInfo
apiModelId?: string // For backward compatibility
}
export class BasetenHandler implements ApiHandler {
private options: BasetenHandlerOptions
private client: OpenAI | undefined
constructor(options: BasetenHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.basetenApiKey) {
throw new Error("Baseten API key is required")
}
try {
this.client = new OpenAI({
baseURL: "https://inference.baseten.co/v1",
apiKey: this.options.basetenApiKey,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating Baseten client: ${error.message}`)
}
}
return this.client
}
/**
* Gets the optimal max_tokens based on model capabilities
*/
private getOptimalMaxTokens(model: { id: BasetenModelId; info: ModelInfo }): number {
// Use model-specific max tokens if available
if (model.info.maxTokens && model.info.maxTokens > 0) {
return model.info.maxTokens
}
// Default fallback
return 8192
}
getModel(): { id: BasetenModelId; info: ModelInfo } {
// First priority: basetenModelId and basetenModelInfo
const basetenModelId = this.options.basetenModelId
const basetenModelInfo = this.options.basetenModelInfo
if (basetenModelId && basetenModelInfo) {
return { id: basetenModelId as BasetenModelId, info: basetenModelInfo }
}
// Second priority: basetenModelId with static model info
if (basetenModelId && basetenModelId in basetenModels) {
const id = basetenModelId as BasetenModelId
return { id, info: basetenModels[id] }
}
// Third priority: apiModelId (for backward compatibility)
const apiModelId = this.options.apiModelId
if (apiModelId && apiModelId in basetenModels) {
const id = apiModelId as BasetenModelId
return { id, info: basetenModels[id] }
}
// Default fallback
return {
id: basetenDefaultModelId,
info: basetenModels[basetenDefaultModelId],
}
}
private async *yieldUsage(modelInfo: ModelInfo, usage: any): ApiStream {
if (usage.prompt_tokens || usage.completion_tokens) {
const cost = calculateApiCostOpenAI(modelInfo, usage.prompt_tokens || 0, usage.completion_tokens || 0)
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
cacheWriteTokens: 0,
cacheReadTokens: 0,
totalCost: cost,
}
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const maxTokens = this.getOptimalMaxTokens(model)
const toolCallProcessor = new ToolCallProcessor()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_tokens: maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
tools,
tool_choice: tools && tools.length > 0 ? "auto" : undefined,
})
let didOutputUsage = false
for await (const chunk of stream) {
const delta = chunk?.choices?.[0]?.delta
// Handle reasoning field if present (for reasoning models with parsed output)
if (delta && "reasoning" in delta && delta?.reasoning) {
const reasoning = typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning)
yield {
type: "reasoning",
reasoning,
}
}
// Handle content field
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle usage information - only output once
if (!didOutputUsage && chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
didOutputUsage = true
}
}
}
/**
* Checks if the current model supports tools
*/
supportsTools(): boolean {
const model = this.getModel()
const modelInfo = model.info as any
// Use dynamic API data when available, fallback to true since all current Baseten models support tools
// (as of 2025-09-16 - could change if Baseten add non-tool models in future, currently no plans to do so)
return modelInfo.supportedFeatures ? modelInfo.supportedFeatures.includes("tools") : true
}
}
File diff suppressed because it is too large Load Diff
@@ -1,275 +0,0 @@
import Cerebras from "@cerebras/cerebras_cloud_sdk"
import { CerebrasModelId, cerebrasDefaultModelId, cerebrasModels, ModelInfo } from "@shared/api"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
interface CerebrasHandlerOptions extends CommonApiHandlerOptions {
cerebrasApiKey?: string
apiModelId?: string
}
// Conservative max_tokens for Cerebras to avoid premature rate limiting.
// Cerebras rate limiter estimates token consumption using max_completion_tokens upfront,
// so requesting the model maximum (e.g., 64K) reserves that quota even if actual usage is low.
// 16K is sufficient for most agentic tool use while preserving rate limit headroom.
const CEREBRAS_DEFAULT_MAX_TOKENS = 16_384
export class CerebrasHandler implements ApiHandler {
private options: CerebrasHandlerOptions
private client: Cerebras | undefined
constructor(options: CerebrasHandlerOptions) {
this.options = options
}
private ensureClient(): Cerebras {
if (!this.client) {
// Clean and validate the API key
const cleanApiKey = this.options.cerebrasApiKey?.trim()
if (!cleanApiKey) {
throw new Error("Cerebras API key is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
this.client = new Cerebras({
apiKey: cleanApiKey,
timeout: 30000, // 30 second timeout
fetch, // Use configured fetch with proxy support
defaultHeaders: {
...externalHeaders,
"X-Cerebras-3rd-Party-Integration": "cline",
},
})
} catch (error) {
throw new Error(`Error creating Cerebras client: ${error.message}`)
}
}
return this.client
}
@withRetry({
maxRetries: 6, // More retries to be patient with rate limits
baseDelay: 5000, // Start with 5 second delay
maxDelay: 60000, // Allow up to 60 second delays to respect rate limits
})
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
// Convert Anthropic messages to Cerebras format
const cerebrasMessages: Array<{
role: "system" | "user" | "assistant"
content: string
}> = [{ role: "system", content: systemPrompt }]
// Helper function to strip thinking tags from content
const stripThinkingTags = (content: string): string => {
return content.replace(/<think>[\s\S]*?<\/think>/g, "").trim()
}
// Check if this is a reasoning model that uses thinking tags
const modelId = this.getModel().id
const isReasoningModel = modelId.includes("qwen")
// Convert Anthropic messages to Cerebras format
for (const message of messages) {
if (message.role === "user") {
const content = Array.isArray(message.content)
? message.content
.map((block) => {
if (block.type === "text") {
return block.text
}
if (block.type === "image") {
return "[Image content not supported in Cerebras]"
}
return ""
})
.join("\n")
: message.content
cerebrasMessages.push({ role: "user", content })
} else if (message.role === "assistant") {
let content = Array.isArray(message.content)
? message.content
.map((block) => {
if (block.type === "text") {
return block.text
}
return ""
})
.join("\n")
: message.content || ""
// Strip thinking tags from assistant messages for reasoning models
// so the model doesn't see its own thinking in the conversation history
if (isReasoningModel) {
content = stripThinkingTags(content)
}
cerebrasMessages.push({ role: "assistant", content })
}
}
try {
const model = this.getModel()
const stream = await client.chat.completions.create({
model: model.id,
messages: cerebrasMessages,
temperature: model.info.temperature ?? 0,
stream: true,
max_tokens: CEREBRAS_DEFAULT_MAX_TOKENS,
})
// Handle streaming response
let reasoning: string | null = null // Track reasoning content for models that support thinking
for await (const chunk of stream as any) {
// Type assertion for the streaming chunk
const streamChunk = chunk as any
if (streamChunk.choices?.[0]?.delta?.content) {
const content = streamChunk.choices[0].delta.content
// Handle reasoning models (Qwen and DeepSeek R1 Distill) that use <think> tags
if (isReasoningModel) {
// Check if we're entering or continuing reasoning mode
if (reasoning || content.includes("<think>")) {
reasoning = (reasoning || "") + content
// Clean the content by removing think tags for display
const cleanContent = content.replace(/<think>/g, "").replace(/<\/think>/g, "")
// Only yield reasoning content if there's actual content after cleaning
if (cleanContent.trim()) {
yield {
type: "reasoning",
reasoning: cleanContent,
}
}
// Check if reasoning is complete
if (reasoning.includes("</think>")) {
reasoning = null
}
} else {
// Regular content outside of thinking tags
yield {
type: "text",
text: content,
}
}
} else {
// Non-reasoning models - just yield text content
yield {
type: "text",
text: content,
}
}
}
// Handle usage information from Cerebras API
// Usage is typically only available in the final chunk
if (streamChunk.usage) {
const totalCost = this.calculateCost({
inputTokens: streamChunk.usage.prompt_tokens || 0,
outputTokens: streamChunk.usage.completion_tokens || 0,
})
yield {
type: "usage",
inputTokens: streamChunk.usage.prompt_tokens || 0,
outputTokens: streamChunk.usage.completion_tokens || 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
totalCost,
}
}
}
} catch (error: any) {
// Enhanced error handling for Cerebras API
if (error?.status === 429 || error?.code === "rate_limit_exceeded") {
// Rate limit error - will be handled by retry decorator with patient backoff
const _limits = this.getRateLimits()
throw new Error(`Cerebras API rate limit exceeded.`)
}
if (error?.status === 401) {
throw new Error("Cerebras API authentication failed. Please check your API key.")
}
if (error?.status === 403) {
throw new Error("Cerebras API access denied. Please check your API key permissions.")
}
if (error?.status >= 500) {
// Server errors - retryable
throw new Error(`Cerebras API server error (${error.status}): ${error.message || "Unknown server error"}`)
}
if (error?.status === 400) {
// Client errors - not retryable
throw new Error(`Cerebras API bad request: ${error.message || "Invalid request parameters"}`)
}
// Re-throw original error for other cases
throw error
}
}
getModel(): { id: string; info: ModelInfo } {
const originalModelId = this.options.apiModelId
let apiModelId = originalModelId
if (originalModelId === "qwen-3-coder-480b-free") {
apiModelId = "qwen-3-coder-480b"
return { id: apiModelId, info: cerebrasModels[originalModelId as CerebrasModelId] }
}
if (originalModelId && originalModelId in cerebrasModels) {
const id = originalModelId as CerebrasModelId
return { id, info: cerebrasModels[id] }
}
return {
id: cerebrasDefaultModelId,
info: cerebrasModels[cerebrasDefaultModelId],
}
}
/**
* Get rate limit information for the current model
*
* These limits are used for informational purposes and to calculate appropriate
* retry delays. Since Cerebras inference is extremely fast, users hit these limits
* quickly, so we need to be patient with retries to maximize usage efficiency.
*
* @returns Rate limit configuration for the model
*/
private getRateLimits(): { requestsPerMinute: number; tokensPerMinute: number } {
const modelId = this.getModel().id
switch (modelId) {
case "qwen-3-coder-480b":
case "qwen-3-coder-480b-free":
return { requestsPerMinute: 10, tokensPerMinute: 150_000 }
case "qwen-3-235b-a22b-instruct-2507":
case "qwen-3-235b-a22b-thinking-2507":
return { requestsPerMinute: 30, tokensPerMinute: 60_000 }
case "gpt-oss-120b":
return { requestsPerMinute: 30, tokensPerMinute: 64_000 }
default:
// Default rate limits for unknown models
return { requestsPerMinute: 30, tokensPerMinute: 60_000 }
}
}
private calculateCost({ inputTokens, outputTokens }: { inputTokens: number; outputTokens: number }): number {
const model = this.getModel()
const inputPrice = model.info.inputPrice || 0
const outputPrice = model.info.outputPrice || 0
const inputCost = (inputPrice / 1_000_000) * inputTokens
const outputCost = (outputPrice / 1_000_000) * outputTokens
return inputCost + outputCost
}
}
@@ -1,229 +0,0 @@
import { filterMessagesForClaudeCode } from "@/integrations/claude-code/message-filter"
import { runClaudeCode } from "@/integrations/claude-code/run"
import { ClaudeCodeModelId, claudeCodeDefaultModelId, claudeCodeModels } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { type ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { type ApiStream, ApiStreamUsageChunk } from "../transform/stream"
interface ClaudeCodeHandlerOptions extends CommonApiHandlerOptions {
claudeCodePath?: string
apiModelId?: string
thinkingBudgetTokens?: number
}
export class ClaudeCodeHandler implements ApiHandler {
private options: ClaudeCodeHandlerOptions
constructor(options: ClaudeCodeHandlerOptions) {
this.options = options
}
@withRetry({
maxRetries: 4,
baseDelay: 2000,
maxDelay: 15000,
})
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
// Filter out image blocks since Claude Code doesn't support them
const filteredMessages = filterMessagesForClaudeCode(messages)
const claudeProcess = runClaudeCode({
systemPrompt,
messages: filteredMessages,
path: this.options.claudeCodePath,
modelId: this.getModel().id,
thinkingBudgetTokens: this.options.thinkingBudgetTokens,
})
// Usage is included with assistant messages,
// but cost is included in the result chunk
const usage: ApiStreamUsageChunk = {
type: "usage",
inputTokens: 0,
outputTokens: 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
}
let isPaidUsage = true
for await (const chunk of claudeProcess) {
if (typeof chunk === "string") {
yield {
type: "text",
text: chunk,
}
continue
}
// Handle system init messages
if (chunk.type === "system" && "subtype" in chunk) {
if (chunk.subtype === "init") {
// Based on my tests, subscription usage sets the `apiKeySource` to "none"
isPaidUsage = (chunk as any).apiKeySource !== "none"
}
// Also handles legacy rate_limit_event format (type: "system", subtype: "rate_limit_event")
// by falling through — no special handling needed.
continue
}
// Handle rate_limit_event (newer CLI format: top-level type)
if (chunk.type === "rate_limit_event") {
// Rate limit events are informational. Log them but don't yield anything.
// If the rate limit blocks the response, the stream will end without
// assistant messages and the task loop will handle the empty response.
Logger.log("Claude Code rate limit event:", JSON.stringify(chunk))
continue
}
// Skip user messages (tool results from Claude Code's own tool execution)
if (chunk.type === "user") {
continue
}
if (chunk.type === "assistant" && "message" in chunk) {
const message = chunk.message
// Check for error field on the message (newer CLI format)
if (message.error) {
const firstContent = message.content?.[0]
const errorText = firstContent && "text" in firstContent ? firstContent.text : undefined
throw new Error(errorText ?? `Claude Code error: ${message.error}`)
}
if (message.stop_reason !== null) {
const firstContent = message.content?.[0]
const content = firstContent && "text" in firstContent ? firstContent : undefined
// Check if content exists before accessing its properties
if (content && content.text.startsWith(`API Error`)) {
// Error messages are formatted as: `API Error: <<status code>> <<json>>`
const errorMessageStart = content.text.indexOf("{")
const errorMessage = content.text.slice(errorMessageStart)
const error = this.attemptParse(errorMessage)
if (!error) {
throw new Error(content.text)
}
if (error.error.message.includes("Invalid model name")) {
throw new Error(
content.text +
`\n\nAPI keys and subscription plans allow different models. Make sure the selected model is included in your plan.`,
)
}
throw new Error(errorMessage)
}
}
for (const content of message.content) {
switch (content.type) {
case "text":
yield {
type: "text",
text: content.text,
}
break
case "thinking":
yield {
type: "reasoning",
reasoning: content.thinking || "",
}
break
case "redacted_thinking":
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
}
break
case "tool_use":
// Yield tool_use blocks to the streaming pipeline for proper tool execution
yield {
type: "tool_calls",
tool_call: {
call_id: content.id,
function: {
id: content.id,
name: content.name,
arguments: JSON.stringify(content.input),
},
},
}
break
default: {
// Handle unknown content block types gracefully.
// Newer Anthropic models or CLI versions may introduce new content types
// (e.g., server_tool_use, mcp_tool_use). Log them instead of silently dropping.
const unknownBlock = content as { type: string; text?: string }
Logger.warn(`Unhandled content type in Claude Code response: ${unknownBlock.type}`)
// If the unknown block has a text-like field, try to yield it as text
if (typeof unknownBlock.text === "string") {
yield {
type: "text",
text: unknownBlock.text,
}
}
break
}
}
}
// According to Anthropic's API documentation:
// https://docs.anthropic.com/en/api/messages#usage-object
// The `input_tokens` field already includes both `cache_read_input_tokens` and `cache_creation_input_tokens`.
// Therefore, we should not add cache tokens to the input_tokens count again, as this would result in double-counting.
usage.inputTokens = message.usage?.input_tokens ?? 0
usage.outputTokens = message.usage?.output_tokens ?? 0
usage.cacheReadTokens = message.usage?.cache_read_input_tokens ?? 0
usage.cacheWriteTokens = message.usage?.cache_creation_input_tokens ?? 0
continue
}
if (chunk.type === "result" && "result" in chunk) {
if (chunk.is_error) {
throw new Error(`Claude Code returned an error: ${chunk.result}`)
}
usage.totalCost = isPaidUsage ? chunk.total_cost_usd : 0
yield usage
continue
}
// ErrorMessage — log it explicitly and skip
if ((chunk as any).type === "error") {
Logger.warn("Claude Code emitted an error-type chunk:", JSON.stringify(chunk))
continue
}
// Any completely unrecognized chunk type — log and skip
Logger.warn(`Unrecognized Claude Code chunk type: ${(chunk as any).type}`)
}
}
private attemptParse(str: string) {
try {
return JSON.parse(str)
} catch (_err) {
return null
}
}
getModel() {
const modelId = this.options.apiModelId
if (modelId && modelId in claudeCodeModels) {
const id = modelId as ClaudeCodeModelId
return { id, info: claudeCodeModels[id] }
}
return {
id: claudeCodeDefaultModelId,
info: claudeCodeModels[claudeCodeDefaultModelId],
}
}
}
-343
View File
@@ -1,343 +0,0 @@
import { type ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import axios from "axios"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineEnv } from "@/config"
import { refreshClineRecommendedModels } from "@/core/controller/models/refreshClineRecommendedModels"
import { ClineAccountService } from "@/services/account/ClineAccountService"
import { AuthService } from "@/services/auth/AuthService"
import { buildClineExtraHeaders } from "@/services/EnvUtils"
import { CLINE_ACCOUNT_AUTH_ERROR_MESSAGE } from "@/shared/ClineAccount"
import { CLINE_RECOMMENDED_MODELS_FALLBACK } from "@/shared/cline/recommended-models"
import type { ClineStorageMessage } from "@/shared/messages/content"
import { fetch, getAxiosSettings } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import type { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { createOpenRouterStream } from "../transform/openrouter-stream"
import type { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
import type { OpenRouterErrorResponse } from "./types"
interface ClineHandlerOptions extends CommonApiHandlerOptions {
ulid?: string
taskId?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
openRouterProviderSorting?: string
openRouterModelId?: string
openRouterModelInfo?: ModelInfo
clineAccountId?: string
clineApiKey?: string
enableParallelToolCalling?: boolean
}
function normalizeModelId(modelId: string): string {
return modelId.trim().toLowerCase()
}
const CLINE_FREE_MODEL_IDS = new Set(CLINE_RECOMMENDED_MODELS_FALLBACK.free.map((model) => normalizeModelId(model.id)))
function getCacheReadTokens(usage: any): number {
return usage?.prompt_tokens_details?.cached_tokens || usage?.cache_read_input_tokens || 0
}
function getCacheWriteTokens(usage: any): number {
return usage?.prompt_tokens_details?.cache_write_tokens || usage?.cache_creation_input_tokens || 0
}
export class ClineHandler implements ApiHandler {
private options: ClineHandlerOptions
private clineAccountService = ClineAccountService.getInstance()
private _authService: AuthService
private client: OpenAI | undefined
lastGenerationId?: string
private lastRequestId?: string
private get _baseUrl(): string {
return ClineEnv.config().apiBaseUrl
}
constructor(options: ClineHandlerOptions) {
this.options = options
this._authService = AuthService.getInstance()
}
private async getFreeModelIdSet(): Promise<Set<string>> {
try {
const models = await refreshClineRecommendedModels()
const freeModelIds = models.free.map((model) => normalizeModelId(model.id)).filter((modelId) => modelId.length > 0)
if (freeModelIds.length > 0) {
return new Set(freeModelIds)
}
} catch (error) {
Logger.error("Error resolving Cline free model IDs from recommended models:", error)
}
return CLINE_FREE_MODEL_IDS
}
private async ensureClient(): Promise<OpenAI> {
const clineAccountAuthToken = this.options.clineApiKey || (await this._authService.getAuthToken())
if (!clineAccountAuthToken) {
throw new Error(CLINE_ACCOUNT_AUTH_ERROR_MESSAGE)
}
if (!this.client) {
try {
const defaultHeaders: Record<string, string> = {
"HTTP-Referer": "https://cline.bot",
"X-Title": "Cline",
"X-Task-ID": this.options.ulid || "",
}
Object.assign(defaultHeaders, await buildClineExtraHeaders())
this.client = new OpenAI({
baseURL: `${this._baseUrl}/api/v1`,
apiKey: clineAccountAuthToken,
defaultHeaders,
// Capture real HTTP request ID from initial streaming response headers
fetch: async (...args: Parameters<typeof fetch>): Promise<Awaited<ReturnType<typeof fetch>>> => {
const [input, init] = args
const resp = await fetch(input, init)
try {
let urlStr = ""
if (typeof input === "string") {
urlStr = input
} else if (input instanceof URL) {
urlStr = input.toString()
} else if (typeof (input as { url?: unknown }).url === "string") {
urlStr = (input as { url: string }).url
}
// Only record for chat completions (the primary streaming request)
if (urlStr.includes("/chat/completions")) {
const rid = resp.headers.get("x-request-id") || resp.headers.get("request-id")
if (rid) {
this.lastRequestId = rid
}
}
} catch {
// ignore header capture errors
}
return resp
},
})
} catch (error: any) {
throw new Error(`Error creating Cline client: ${error.message}`)
}
}
// Ensure the client is always using the latest auth token
this.client.apiKey = clineAccountAuthToken
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
try {
const client = await this.ensureClient()
this.lastGenerationId = undefined
this.lastRequestId = undefined
let didOutputUsage = false
const freeModelIds = await this.getFreeModelIdSet()
const stream = await createOpenRouterStream(
client,
systemPrompt,
messages,
this.getModel(),
this.options.reasoningEffort,
this.options.thinkingBudgetTokens,
this.options.openRouterProviderSorting,
tools,
this.options.enableParallelToolCalling,
)
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
Logger.debug("ClineHandler chunk:" + JSON.stringify(chunk))
// openrouter returns an error object instead of the openai sdk throwing an error
if ("error" in chunk) {
const error = chunk.error as OpenRouterErrorResponse["error"]
Logger.error(`Cline API Error: ${error?.code} - ${error?.message}`)
// Include metadata in the error message if available
const metadataStr = error.metadata ? `\nMetadata: ${JSON.stringify(error.metadata, null, 2)}` : ""
throw new Error(`Cline API Error ${error.code}: ${error.message}${metadataStr}`)
}
if (!this.lastGenerationId && chunk.id) {
this.lastGenerationId = chunk.id
}
// Check for mid-stream error via finish_reason
const choice = chunk.choices?.[0]
// OpenRouter may return finish_reason = "error" with error details
if ((choice?.finish_reason as string) === "error") {
const choiceWithError = choice as any
if (choiceWithError.error) {
const error = choiceWithError.error
Logger.error(`Cline Mid-Stream Error: ${error.code || error.type || "Unknown"} - ${error.message}`)
throw new Error(`Cline Mid-Stream Error: ${error.code || error.type || "Unknown"} - ${error.message}`)
}
throw new Error("Cline Mid-Stream Error: Stream terminated with error status but no error details provided")
}
const delta = choice?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Reasoning tokens are returned separately from the content
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
if (
delta &&
"reasoning" in delta &&
delta.reasoning &&
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
/*
OpenRouter passes reasoning details that we can pass back unmodified in api requests to preserve reasoning traces for model
- The reasoning_details array in each chunk may contain one or more reasoning objects
- For encrypted reasoning, the content may appear as [REDACTED] in streaming responses
- The complete reasoning sequence is built by concatenating all chunks in order
See: https://openrouter.ai/docs/use-cases/reasoning-tokens#preserving-reasoning-blocks
*/
if (
delta &&
"reasoning_details" in delta &&
delta.reasoning_details &&
// @ts-expect-error-next-line
delta?.reasoning_details?.length && // exists and non-0
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: "",
details: delta.reasoning_details,
}
}
if (!didOutputUsage && chunk.usage) {
// @ts-expect-error-next-line
let totalCost = (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0)
const modelId = this.getModel().id
const isFreeModel = freeModelIds.has(normalizeModelId(modelId))
const cacheReadTokens = getCacheReadTokens(chunk.usage)
const cacheWriteTokens = getCacheWriteTokens(chunk.usage)
if (isFreeModel) {
totalCost = 0
}
yield {
type: "usage",
cacheWriteTokens,
cacheReadTokens,
inputTokens: Math.max(0, (chunk.usage.prompt_tokens || 0) - cacheReadTokens - cacheWriteTokens),
outputTokens: chunk.usage.completion_tokens || 0,
totalCost,
}
didOutputUsage = true
}
}
// Fallback to generation endpoint if usage chunk not returned
if (!didOutputUsage) {
Logger.warn("Cline API did not return usage chunk, fetching from generation endpoint")
const apiStreamUsage = await this.getApiStreamUsage(freeModelIds)
if (apiStreamUsage) {
yield apiStreamUsage
}
}
} catch (error) {
Logger.error("Cline API Error:", error)
throw error
}
}
async getApiStreamUsage(freeModelIds?: Set<string>): Promise<ApiStreamUsageChunk | undefined> {
if (this.lastGenerationId) {
try {
const resolvedFreeModelIds = freeModelIds || (await this.getFreeModelIdSet())
const clineAccountAuthToken = await this._authService.getAuthToken()
if (!clineAccountAuthToken) {
throw new Error(CLINE_ACCOUNT_AUTH_ERROR_MESSAGE)
}
const headers: Record<string, string> = {
// Align with backend auth expectations
Authorization: `Bearer ${clineAccountAuthToken}`,
}
Object.assign(headers, await buildClineExtraHeaders())
const response = await axios.get(`${this.clineAccountService.baseUrl}/generation?id=${this.lastGenerationId}`, {
headers,
timeout: 15_000, // this request hangs sometimes
...getAxiosSettings(),
})
const generation = response.data
let totalCost = generation?.total_cost || 0
const modelId = this.getModel().id
const isFreeModel = resolvedFreeModelIds.has(normalizeModelId(modelId))
if (isFreeModel) {
totalCost = 0
}
return {
type: "usage",
cacheWriteTokens: generation?.native_tokens_cache_write || 0,
cacheReadTokens: generation?.native_tokens_cached || 0,
// openrouter generation endpoint fails often
inputTokens: Math.max(
0,
(generation?.native_tokens_prompt || 0) -
(generation?.native_tokens_cached || 0) -
(generation?.native_tokens_cache_write || 0),
),
outputTokens: generation?.native_tokens_completion || 0,
totalCost,
}
} catch (error) {
// ignore if fails
Logger.error("Error fetching cline generation details:", error)
}
}
return undefined
}
// Expose the last HTTP request ID captured from response headers (X-Request-ID)
getLastRequestId(): string | undefined {
return this.lastRequestId
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.openRouterModelId
const modelInfo = this.options.openRouterModelInfo
if (modelId && modelInfo) {
return { id: modelId, info: modelInfo }
}
// If we have a model ID but no model info (e.g., CLI featured models),
// use the ID with default model info rather than falling back to a different model
if (modelId) {
return { id: modelId, info: openRouterDefaultModelInfo }
}
return { id: openRouterDefaultModelId, info: openRouterDefaultModelInfo }
}
}
@@ -1,142 +0,0 @@
import { DeepSeekModelId, deepSeekDefaultModelId, deepSeekModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { addReasoningContent } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface DeepSeekHandlerOptions extends CommonApiHandlerOptions {
deepSeekApiKey?: string
apiModelId?: string
}
export class DeepSeekHandler implements ApiHandler {
private options: DeepSeekHandlerOptions
private client: OpenAI | undefined
constructor(options: DeepSeekHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.deepSeekApiKey) {
throw new Error("DeepSeek API key is required")
}
try {
this.client = new OpenAI({
baseURL: "https://api.deepseek.com/v1",
apiKey: this.options.deepSeekApiKey,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating DeepSeek client: ${error.message}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
// Deepseek reports total input AND cache reads/writes,
// see context caching: https://api-docs.deepseek.com/guides/kv_cache)
// where the input tokens is the sum of the cache hits/misses, just like OpenAI.
// This affects:
// 1) context management truncation algorithm, and
// 2) cost calculation
// Deepseek usage includes extra fields.
// Safely cast the prompt token details section to the appropriate structure.
interface DeepSeekUsage extends OpenAI.CompletionUsage {
prompt_cache_hit_tokens?: number
prompt_cache_miss_tokens?: number
}
const deepUsage = usage as DeepSeekUsage
const inputTokens = deepUsage?.prompt_tokens || 0 // sum of cache hits and misses
const outputTokens = deepUsage?.completion_tokens || 0
const cacheReadTokens = deepUsage?.prompt_cache_hit_tokens || 0
const cacheWriteTokens = deepUsage?.prompt_cache_miss_tokens || 0
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens) // this will always be 0
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
totalCost: totalCost,
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const isDeepSeekThinkingModel =
model.id.includes("deepseek-reasoner") || model.id === "deepseek-v4-flash" || model.id === "deepseek-v4-pro"
const convertedMessages = convertToOpenAiMessages(messages)
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = isDeepSeekThinkingModel
? [{ role: "system", content: systemPrompt }, ...addReasoningContent(convertedMessages, messages)]
: [{ role: "system", content: systemPrompt }, ...convertedMessages]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
// Only set temperature for non-thinking models
...(isDeepSeekThinkingModel ? {} : { temperature: 0 }),
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
}
}
}
getModel(): { id: DeepSeekModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in deepSeekModels) {
const id = modelId as DeepSeekModelId
return { id, info: deepSeekModels[id] }
}
return {
id: deepSeekDefaultModelId,
info: deepSeekModels[deepSeekDefaultModelId],
}
}
}
-657
View File
@@ -1,657 +0,0 @@
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ModelInfo } from "../../../shared/api"
import { ApiHandler } from "../index"
import { ApiStream } from "../transform/stream"
interface DifyHandlerOptions {
difyApiKey?: string
difyBaseUrl?: string
}
// Dify API Response Types
export interface DifyFileResponse {
id: string
name: string
size: number
extension: string
mime_type: string
created_by: string
created_at: number
}
export interface DifyMessage {
id: string
conversation_id: string
inputs: Record<string, any>
query: string
message_files: Array<{
id: string
type: string
url: string
belongs_to: string
}>
answer: string
created_at: number
feedback?: {
rating: string
}
retriever_resources?: any[]
}
interface DifyHistoryResponse {
data: DifyMessage[]
has_more: boolean
limit: number
}
interface DifyConversation {
id: string
name: string
inputs: Record<string, any>
status: string
introduction: string
created_at: number
updated_at: number
}
interface DifyConversationsResponse {
data: DifyConversation[]
has_more: boolean
limit: number
}
interface DifyConversationResponse {
id: string
name: string
inputs: Record<string, any>
status: string
introduction: string
created_at: number
updated_at: number
}
export class DifyHandler implements ApiHandler {
private options: DifyHandlerOptions
private baseUrl: string
private apiKey: string
private conversationId: string | null = null
private currentTaskId: string | null = null
private abortController: AbortController | null = null
constructor(options: DifyHandlerOptions) {
this.options = options
this.apiKey = options.difyApiKey || ""
this.baseUrl = options.difyBaseUrl || ""
Logger.log("[DIFY DEBUG] Constructor called with:", {
hasApiKey: !!this.apiKey,
baseUrl: this.baseUrl,
})
if (!this.apiKey) {
throw new Error("Dify API key is required")
}
if (!this.baseUrl) {
throw new Error("Dify base URL is required")
}
}
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
Logger.log("[DIFY DEBUG] createMessage called with:", {
systemPromptLength: systemPrompt?.length || 0,
messagesCount: messages?.length || 0,
})
// Convert messages to Dify format
const query = this.convertMessagesToQuery(systemPrompt, messages)
const requestBody = {
inputs: {},
query: query,
response_mode: "streaming",
conversation_id: this.conversationId || "",
user: "cline-user", // A unique user identifier
files: [],
}
const fullUrl = `${this.baseUrl}/chat-messages`
Logger.log("[DIFY DEBUG] Making request to:", fullUrl)
Logger.log("[DIFY DEBUG] Request body:", JSON.stringify(requestBody, null, 2))
let response: Response
try {
response = await fetch(fullUrl, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify(requestBody),
})
} catch (error: any) {
Logger.error("[DIFY DEBUG] Network error during fetch:", error)
const cause = error.cause ? ` | Cause: ${error.cause}` : ""
throw new Error(`Dify API network error: ${error.message}${cause}`)
}
Logger.log("[DIFY DEBUG] Response status:", response.status)
const headersObj: Record<string, string> = {}
response.headers.forEach((value, key) => {
headersObj[key] = value
})
Logger.log("[DIFY DEBUG] Response headers:", headersObj)
if (!response.ok) {
const errorText = await response.text()
Logger.error("[DIFY DEBUG] Error response:", errorText)
throw new Error(`Dify API error: ${response.status} ${response.statusText} - ${errorText}`)
}
if (!response.body) {
throw new Error("No response body from Dify API")
}
const reader = response.body.getReader()
const decoder = new TextDecoder()
let buffer = ""
let fullText = ""
let hasYieldedContent = false
const processedEvents: string[] = []
let lastEventTime = Date.now()
Logger.log("[DIFY DEBUG] Starting to read streaming response...")
try {
while (true) {
const { done, value } = await reader.read()
if (done) {
Logger.log("[DIFY DEBUG] Stream ended naturally")
Logger.log(
"[DIFY DEBUG] Final state - hasYieldedContent:",
hasYieldedContent,
"fullText length:",
fullText.length,
"processedEvents:",
processedEvents,
)
break
}
const chunk = decoder.decode(value, { stream: true })
Logger.log("[DIFY DEBUG] Raw chunk received:", JSON.stringify(chunk))
buffer += chunk
const lines = buffer.split("\n")
// Keep the last incomplete line in the buffer
buffer = lines.pop() || ""
for (const line of lines) {
Logger.log("[DIFY DEBUG] Processing line:", JSON.stringify(line))
if (line.startsWith("data: ")) {
const data = line.slice(6).trim()
Logger.log("[DIFY DEBUG] Extracted data:", JSON.stringify(data))
if (data === "[DONE]") {
Logger.log("[DIFY DEBUG] Received [DONE] signal")
break
}
if (data === "") {
Logger.log("[DIFY DEBUG] Empty data line, skipping")
continue
}
try {
const parsed = JSON.parse(data)
Logger.log("[DIFY DEBUG] Parsed JSON:", parsed)
processedEvents.push(parsed.event || "unknown")
lastEventTime = Date.now()
// Capture conversation_id as soon as it's available
if (parsed.conversation_id && !this.conversationId) {
this.conversationId = parsed.conversation_id
Logger.log("[DIFY DEBUG] Captured conversation_id:", this.conversationId)
}
// Handle different Dify event types based on actual Dify API
if (parsed.event === "message") {
Logger.log("[DIFY DEBUG] Message event, answer:", parsed.answer)
// Dify sends the full text in each "answer" chunk, so we replace.
if (typeof parsed.answer === "string") {
fullText = parsed.answer
Logger.log("[DIFY DEBUG] Updated fullText length:", fullText.length)
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
} else if (parsed.event === "message_replace") {
Logger.log("[DIFY DEBUG] Replace message event:", parsed)
if (parsed.answer) {
fullText = parsed.answer // Replace instead of append
Logger.log("[DIFY DEBUG] Replaced fullText length:", fullText.length)
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
} else if (parsed.event === "message_end") {
Logger.log("[DIFY DEBUG] Message end event", parsed)
// Message completed. Yield final text if we have any.
if (fullText) {
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
// Yield usage data if available
if (parsed.usage) {
yield {
type: "usage",
inputTokens: parsed.usage.prompt_tokens || 0,
outputTokens: parsed.usage.completion_tokens || parsed.usage.total_tokens || 0,
totalCost: parsed.usage.total_price || 0,
}
}
return // End of stream
} else if (parsed.event === "error") {
Logger.error("[DIFY DEBUG] Error event:", parsed)
throw new Error(`Dify API error: ${parsed.message || "Unknown error"}`)
} else if (parsed.event === "workflow_started" || parsed.event === "workflow_finished") {
Logger.log("[DIFY DEBUG] Workflow event:", parsed.event)
// These are informational events, continue processing
} else if (parsed.event === "node_started" || parsed.event === "node_finished") {
Logger.log("[DIFY DEBUG] Node event:", parsed.event, parsed.data)
// These are informational events, continue processing
} else if (parsed.event === "ping") {
Logger.log("[DIFY DEBUG] Ping event received, keeping connection alive.")
// Ping event, do nothing
} else {
Logger.log("[DIFY DEBUG] Unknown event type:", parsed.event, "Full object:", parsed)
// Try to extract text from other possible fields
if (parsed.text) {
fullText += parsed.text
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
} else if (parsed.content) {
fullText += parsed.content
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
} else if (parsed.answer) {
// Fallback: some events might have answer field even if not "message" type
fullText += parsed.answer
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
}
} catch (e) {
Logger.warn("[DIFY DEBUG] Failed to parse JSON:", data, "Error:", e)
}
} else if (line.trim() !== "") {
Logger.log(
"[DIFY DEBUG] Non-data line (not starting with 'data:'), trying to parse as direct JSON:",
JSON.stringify(line),
)
// Try to parse as direct JSON (fallback for non-SSE responses, though Dify uses SSE)
try {
const parsed = JSON.parse(line.trim())
Logger.log("[DIFY DEBUG] Parsed direct JSON:", parsed)
processedEvents.push(parsed.event || "direct-json")
// Handle the same event types as above
if (parsed.event === "message" && parsed.answer) {
fullText += parsed.answer
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
} else if (parsed.event === "message_end") {
if (fullText) {
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
return
} else if (parsed.event === "error") {
Logger.error("[DIFY DEBUG] Direct JSON Error event:", parsed)
throw new Error(`Dify API error: ${parsed.message || "Unknown error"}`)
} else if (parsed.answer || parsed.text || parsed.content) {
// Fallback for any content in direct JSON
const content = parsed.answer || parsed.text || parsed.content
fullText += content
yield {
type: "text",
text: fullText,
}
hasYieldedContent = true
}
} catch (e) {
// Not JSON, continue
Logger.log("[DIFY DEBUG] Line is not direct JSON, continuing")
}
}
}
}
// Final check - if we haven't yielded any content, provide diagnostic information
if (!hasYieldedContent) {
const diagnosticInfo = {
processedEvents,
finalFullTextLength: fullText.length,
finalFullText: fullText,
streamDuration: Date.now() - lastEventTime,
conversationId: this.conversationId,
}
Logger.error("[DIFY DEBUG] No content was yielded! Diagnostic info:", diagnosticInfo)
// If we have any accumulated text at all, yield it as a fallback
if (fullText.trim()) {
Logger.log("[DIFY DEBUG] Yielding accumulated text as fallback:", fullText)
yield {
type: "text",
text: fullText,
}
} else {
// Provide a more informative error
throw new Error(
`Dify API did not provide any assistant messages. ` +
`Events processed: [${processedEvents.join(", ")}]. ` +
`Check your Dify application configuration and ensure it's properly set up to return responses. ` +
`API URL: ${fullUrl}. Conversation ID: ${this.conversationId || "none"}.`,
)
}
}
} finally {
reader.releaseLock()
Logger.log("[DIFY DEBUG] Stream reader released")
}
}
private convertMessagesToQuery(systemPrompt: string, messages: ClineStorageMessage[]): string {
// Dify's context is managed by `conversation_id`. The `query` should be the last user message.
// The system prompt is typically configured in the Dify App itself.
const lastUserMessage = messages.filter((m) => m.role === "user").pop()
if (!lastUserMessage) {
return "" // Should not happen in normal flow
}
const userQuery = Array.isArray(lastUserMessage.content)
? lastUserMessage.content.map((c) => ("text" in c ? c.text : "")).join("\n")
: (lastUserMessage.content as string)
// Only prepend the system prompt if it's the very first message of a new conversation.
if (!this.conversationId && systemPrompt) {
Logger.log("[DIFY DEBUG] Prepending system prompt for new conversation.")
return `${systemPrompt}\n\n---\n\n${userQuery}`
}
return userQuery
}
getModel(): { id: string; info: ModelInfo } {
return {
id: "dify-workflow",
info: {
maxTokens: 8192,
contextWindow: 128000,
supportsImages: true,
supportsPromptCache: false,
inputPrice: 0,
outputPrice: 0,
description: "Dify workflow - model selection is configured in your Dify application",
},
}
}
// Additional Dify API Methods
/**
* Upload a file for use in conversations
* @param file File buffer to upload
* @param filename Name of the file
* @param user User identifier (defaults to "cline-user")
* @returns Promise with file upload response
*/
async uploadFile(file: Buffer, filename: string, user: string = "cline-user"): Promise<DifyFileResponse> {
const formData = new FormData()
formData.append("file", new Blob([new Uint8Array(file)]), filename)
formData.append("user", user)
const response = await fetch(`${this.baseUrl}/files/upload`, {
method: "POST",
headers: this.headers(),
body: formData,
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify file upload error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Stop generation for a specific task
* @param taskId Task ID from streaming response
* @param user User identifier (defaults to "cline-user")
* @returns Promise that resolves when generation is stopped
*/
async stopGeneration(taskId: string, user: string = "cline-user"): Promise<void> {
const response = await fetch(`${this.baseUrl}/chat-messages/${taskId}/stop`, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify({ user }),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify stop generation error: ${response.status} ${response.statusText} - ${errorText}`)
}
}
/**
* Get conversation history messages with pagination
* @param conversationId Conversation ID
* @param user User identifier (defaults to "cline-user")
* @param firstId First message ID for pagination (optional)
* @param limit Number of messages to return (default: 20)
* @returns Promise with conversation history
*/
async getConversationHistory(
conversationId: string,
user: string = "cline-user",
firstId?: string,
limit: number = 20,
): Promise<DifyHistoryResponse> {
const params = new URLSearchParams({ user, limit: limit.toString() })
if (firstId) {
params.append("first_id", firstId)
}
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}/messages?${params}`, {
headers: this.headers(),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify get conversation history error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Get list of conversations for a user
* @param user User identifier (defaults to "cline-user")
* @param lastId Last conversation ID for pagination (optional)
* @param limit Number of conversations to return (default: 20)
* @param sortBy Sort field (default: "-updated_at")
* @returns Promise with conversations list
*/
async getConversations(
user: string = "cline-user",
lastId?: string,
limit: number = 20,
sortBy: string = "-updated_at",
): Promise<DifyConversationsResponse> {
const params = new URLSearchParams({
user,
limit: limit.toString(),
sort_by: sortBy,
})
if (lastId) {
params.append("last_id", lastId)
}
const response = await fetch(`${this.baseUrl}/conversations?${params}`, {
headers: this.headers(),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify get conversations error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Delete a conversation
* @param conversationId Conversation ID to delete
* @param user User identifier (defaults to "cline-user")
* @returns Promise that resolves when conversation is deleted
*/
async deleteConversation(conversationId: string, user: string = "cline-user"): Promise<void> {
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}`, {
method: "DELETE",
headers: this.jsonHeaders(),
body: JSON.stringify({ user }),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify delete conversation error: ${response.status} ${response.statusText} - ${errorText}`)
}
}
/**
* Rename a conversation
* @param conversationId Conversation ID to rename
* @param user User identifier (defaults to "cline-user")
* @param name New conversation name (optional if auto_generate is true)
* @param autoGenerate Whether to auto-generate the name (default: false)
* @returns Promise with updated conversation details
*/
async renameConversation(
conversationId: string,
user: string = "cline-user",
name?: string,
autoGenerate: boolean = false,
): Promise<DifyConversationResponse> {
const body: any = { user, auto_generate: autoGenerate }
if (name) {
body.name = name
}
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}/name`, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify(body),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify rename conversation error: ${response.status} ${response.statusText} - ${errorText}`)
}
return response.json()
}
/**
* Submit feedback for a message
* @param messageId Message ID to provide feedback for
* @param rating Rating: "like" or "dislike"
* @param content Optional feedback content
* @param user User identifier (defaults to "cline-user")
* @returns Promise that resolves when feedback is submitted
*/
async submitMessageFeedback(
messageId: string,
rating: "like" | "dislike",
content?: string,
user: string = "cline-user",
): Promise<void> {
const body: any = { rating, user }
if (content) {
body.content = content
}
const response = await fetch(`${this.baseUrl}/messages/${messageId}/feedbacks`, {
method: "POST",
headers: this.jsonHeaders(),
body: JSON.stringify(body),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Dify submit feedback error: ${response.status} ${response.statusText} - ${errorText}`)
}
}
/**
* Get current conversation ID
* @returns Current conversation ID or null
*/
getCurrentConversationId(): string | null {
return this.conversationId
}
/**
* Set conversation ID for continuing existing conversations
* @param conversationId Conversation ID to set
*/
setConversationId(conversationId: string): void {
this.conversationId = conversationId
}
/**
* Reset conversation ID to start a new conversation
*/
resetConversation(): void {
this.conversationId = null
this.currentTaskId = null
}
private jsonHeaders() {
return {
...this.headers(),
"Content-Type": "application/json",
}
}
private headers() {
const externalHeaders = buildExternalBasicHeaders()
return {
...externalHeaders,
Authorization: `Bearer ${this.apiKey}`,
}
}
}
@@ -1,90 +0,0 @@
import { DoubaoModelId, doubaoDefaultModelId, doubaoModels, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface DoubaoHandlerOptions extends CommonApiHandlerOptions {
doubaoApiKey?: string
apiModelId?: string
}
export class DoubaoHandler implements ApiHandler {
private options: DoubaoHandlerOptions
private client: OpenAI | undefined
constructor(options: DoubaoHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.doubaoApiKey) {
throw new Error("Doubao API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://ark.cn-beijing.volces.com/api/v3/",
apiKey: this.options.doubaoApiKey,
})
} catch (error) {
throw new Error(`Error creating Doubao client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: DoubaoModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in doubaoModels) {
const id = modelId as DoubaoModelId
return { id, info: doubaoModels[id] }
}
return {
id: doubaoDefaultModelId,
info: doubaoModels[doubaoDefaultModelId],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
// @ts-expect-error-next-line
cacheReadTokens: chunk.usage.prompt_cache_hit_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
}
@@ -1,116 +0,0 @@
import { FireworksModelId, fireworksDefaultModelId, fireworksModels, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface FireworksHandlerOptions extends CommonApiHandlerOptions {
fireworksApiKey?: string
fireworksModelId?: string
fireworksModelMaxCompletionTokens?: number
fireworksModelMaxTokens?: number
}
export class FireworksHandler implements ApiHandler {
private options: FireworksHandlerOptions
private client: OpenAI | undefined
constructor(options: FireworksHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.fireworksApiKey) {
throw new Error("Fireworks API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.fireworks.ai/inference/v1",
apiKey: this.options.fireworksApiKey,
})
} catch (error) {
throw new Error(`Error creating Fireworks client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.fireworksModelId ?? ""
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
})
let reasoning: string | null = null
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (reasoning || delta?.content?.includes("<think>")) {
reasoning = (reasoning || "") + (delta.content ?? "")
}
if (delta?.content && !reasoning) {
yield {
type: "text",
text: delta.content,
}
}
if (reasoning || (delta && "reasoning_content" in delta && delta.reasoning_content)) {
yield {
type: "reasoning",
reasoning: delta.content || ((delta as any).reasoning_content as string | undefined) || "",
}
if (reasoning?.includes("</think>")) {
// Reset so the next chunk is regular content
reasoning = null
}
}
if (chunk.usage) {
const usage = chunk.usage as OpenAI.CompletionUsage & {
prompt_cache_hit_tokens?: number
prompt_cache_miss_tokens?: number
prompt_tokens_details?: {
cached_tokens?: number
}
}
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
// Fireworks can return cache hits either as prompt_cache_hit_tokens or prompt_tokens_details.cached_tokens.
cacheReadTokens: usage.prompt_cache_hit_tokens ?? usage.prompt_tokens_details?.cached_tokens ?? 0,
cacheWriteTokens: usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: FireworksModelId; info: ModelInfo } {
const modelId = this.options.fireworksModelId
if (modelId && modelId in fireworksModels) {
const id = modelId as FireworksModelId
return { id, info: fireworksModels[id] }
}
return {
id: fireworksDefaultModelId,
info: fireworksModels[fireworksDefaultModelId],
}
}
}
@@ -1,570 +0,0 @@
// Restore GenerateContentConfig import and add GenerateContentResponseUsageMetadata
import {
ApiError,
FunctionCallingConfigMode,
type GenerateContentConfig,
type GenerateContentResponseUsageMetadata,
GoogleGenAI,
FunctionDeclaration as GoogleTool,
ThinkingLevel,
} from "@google/genai"
import { GeminiModelId, geminiDefaultModelId, geminiModels, ModelInfo } from "@shared/api"
import { GEMINI_FLASH_MAX_OUTPUT_TOKENS, isGeminiFlashModel } from "@utils/model-utils"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { telemetryService } from "@/services/telemetry"
import { ClineStorageMessage } from "@/shared/messages/content"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { RetriableError, withRetry } from "../retry"
import { convertAnthropicMessageToGemini } from "../transform/gemini-format"
import { ApiStream } from "../transform/stream"
const rateLimitPatterns = [/got status: 429/i, /429 Too Many Requests/i, /rate limit exceeded/i, /too many requests/i]
interface GeminiHandlerOptions extends CommonApiHandlerOptions {
isVertex?: boolean
vertexProjectId?: string
vertexRegion?: string
geminiApiKey?: string
geminiBaseUrl?: string
thinkingBudgetTokens?: number
reasoningEffort?: string
apiModelId?: string
ulid?: string
}
function mapReasoningEffortToGeminiThinkingLevel(effort: string): ThinkingLevel {
switch (effort) {
case "low":
case "medium":
return ThinkingLevel.LOW
case "high":
case "xhigh":
return ThinkingLevel.HIGH
default:
return ThinkingLevel.LOW
}
}
function getGeminiMaxOutputTokens(modelId: string, modelMaxTokens?: number): number | undefined {
if (!isGeminiFlashModel(modelId)) {
return undefined
}
if (modelMaxTokens && modelMaxTokens > 0) {
return Math.min(modelMaxTokens, GEMINI_FLASH_MAX_OUTPUT_TOKENS)
}
return GEMINI_FLASH_MAX_OUTPUT_TOKENS
}
/**
* Handler for Google's Gemini API with optimized caching strategy and accurate cost accounting.
*
* Key features:
* - One cache per task: Creates a single cache per task and reuses it for subsequent turns
* - Stable cache keys: Uses ulid as a stable identifier for caches
* - Efficient cache updates: Only updates caches when there's new content to add
* - Split cost accounting: Separates immediate costs from ongoing cache storage costs
*
* Cost accounting approach:
* - Immediate costs (per message): Input tokens, output tokens, and cache read costs
* - Ongoing costs (per task): Cache storage costs for the TTL period
*
* Gemini's caching system is unique in that it charges for holding tokens in cache by the hour.
* This implementation optimizes for both performance and cost by:
* 1. Minimizing redundant cache creations
* 2. Properly accounting for cache costs in the billing calculations
* 3. Using a stable cache key to ensure cache reuse across turns
* 4. Separating immediate costs from ongoing costs to avoid double-counting
*/
export class GeminiHandler implements ApiHandler {
private options: GeminiHandlerOptions
private client: GoogleGenAI | undefined
constructor(options: GeminiHandlerOptions) {
// Store the options
this.options = options
}
private ensureClient(): GoogleGenAI {
if (!this.client) {
const options = this.options as GeminiHandlerOptions
const externalHeaders = buildExternalBasicHeaders()
if (options.isVertex) {
// Initialize with Vertex AI configuration
const project = this.options.vertexProjectId ?? "not-provided"
const location = this.options.vertexRegion ?? "not-provided"
try {
this.client = new GoogleGenAI({
vertexai: true,
project,
location,
httpOptions: {
headers: externalHeaders,
},
})
} catch (error) {
throw new Error(`Error creating Gemini Vertex AI client: ${error.message}`)
}
} else {
// Initialize with standard API key
if (!options.geminiApiKey) {
throw new Error("API key is required for Google Gemini when not using Vertex AI")
}
try {
this.client = new GoogleGenAI({
apiKey: options.geminiApiKey,
httpOptions: {
headers: externalHeaders,
},
})
} catch (error) {
throw new Error(`Error creating Gemini client: ${error.message}`)
}
}
}
return this.client
}
/**
* Creates a message using the Gemini API with implicit caching.
*
* Cost accounting:
* - Immediate costs (returned in the usage object): Input tokens, output tokens, cache read costs
*
* @param systemPrompt The system prompt to use for the message
* @param messages The conversation history to include in the message
* @returns An async generator that yields chunks of the response with accurate immediate costs
*/
@withRetry({
maxRetries: 4,
baseDelay: 2000,
maxDelay: 15000,
})
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: GoogleTool[]): ApiStream {
const client = this.ensureClient()
const { id: modelId, info } = this.getModel()
const contents = messages.map(convertAnthropicMessageToGemini)
// Gemini may emit multiple function calls under the same responseId and without functionCall.id.
// Track a local sequence so each emitted tool call has a stable unique ID.
const responseToolCallCount = new Map<string, number>()
// Configure thinking budget/level if supported
const _thinkingBudget = this.options.thinkingBudgetTokens ?? 0
const maxBudget = info.thinkingConfig?.maxBudget ?? 24576
const thinkingBudget = Math.min(_thinkingBudget, maxBudget)
// When ThinkingLevel is defined, thinking budget cannot be zero
// and only level is used to control thinking behavior.
// Only set thinkingLevel for models that support it
let thinkingLevel: ThinkingLevel | undefined
const rawReasoningEffort = (this.options.reasoningEffort || "").toLowerCase()
const normalizedReasoningEffort = !rawReasoningEffort || rawReasoningEffort === "none" ? "low" : rawReasoningEffort
if (info.thinkingConfig?.supportsThinkingLevel) {
thinkingLevel = mapReasoningEffortToGeminiThinkingLevel(normalizedReasoningEffort)
}
// Set up base generation config
const maxOutputTokens = getGeminiMaxOutputTokens(modelId, info.maxTokens)
const requestConfig: GenerateContentConfig = {
// Add base URL if configured
httpOptions: this.options.geminiBaseUrl ? { baseUrl: this.options.geminiBaseUrl } : undefined,
systemInstruction: systemPrompt,
// Set temperature (default to 0)
// Gemini 3 recommends 1.0
temperature: info.temperature ?? 1,
...(maxOutputTokens !== undefined ? { maxOutputTokens } : {}),
}
// Add thinking config only if the model supports it
if (info.thinkingConfig) {
requestConfig.thinkingConfig = {
// Turn off thinking:
// thinkingBudget: 0
// Turn on dynamic thinking:
// thinkingBudget: -1
// Turn on fixed thinking budget:
thinkingBudget: thinkingLevel ? undefined : thinkingBudget, // Use budget only if thinkingLevel is not set
thinkingLevel,
includeThoughts: thinkingBudget > 0 || !!thinkingLevel,
}
}
// Generate content using the configured parameters
const sdkCallStartTime = Date.now()
let responseId: string | undefined
let sdkFirstChunkTime: number | undefined
let ttftSdkMs: number | undefined
let apiSuccess = false
let apiError: string | undefined
let promptTokens = 0
let outputTokens = 0
let cacheReadTokens = 0
let thoughtsTokenCount = 0 // Initialize thought token counts
let lastUsageMetadata: GenerateContentResponseUsageMetadata | undefined
const isNativeToolCallsEnabled = tools?.length
if (isNativeToolCallsEnabled) {
requestConfig.tools = [{ functionDeclarations: tools }]
requestConfig.toolConfig = {
// Force the model to call 'any' function.
functionCallingConfig: {
mode: FunctionCallingConfigMode.ANY,
},
}
}
try {
const result = await client.models.generateContentStream({
model: modelId,
contents: contents,
config: {
...requestConfig,
},
})
let isFirstSdkChunk = true
for await (const chunk of result) {
const responseKey = chunk.responseId || "gemini-response"
if (isFirstSdkChunk) {
sdkFirstChunkTime = Date.now()
ttftSdkMs = sdkFirstChunkTime - sdkCallStartTime
isFirstSdkChunk = false
}
// Handle thinking content from Gemini's response
const parts = chunk?.candidates?.[0]?.content?.parts || []
for (const part of parts) {
if (part.thought && part.text) {
yield {
type: "reasoning",
id: chunk.responseId,
reasoning: part.text || "",
signature: part.thoughtSignature,
}
} else if (part.text) {
yield {
type: "text",
text: part.text,
id: chunk.responseId,
signature: part.thoughtSignature,
}
}
if (part.functionCall) {
const functionCall = part.functionCall
const args = Object.entries(functionCall.args || {}).filter(([_key, val]) => !!val)
if (functionCall.args && args.length > 0) {
const existingId = functionCall.id?.trim()
const toolCallId =
existingId ??
(() => {
const sequenceNumber = responseToolCallCount.get(responseKey) ?? 0
responseToolCallCount.set(responseKey, sequenceNumber + 1)
return `${responseKey}-tool-${sequenceNumber}`
})()
yield {
type: "tool_calls",
id: chunk.responseId,
tool_call: {
call_id: toolCallId,
function: {
id: toolCallId,
name: functionCall.name,
arguments: JSON.stringify(functionCall.args),
},
},
signature: part.thoughtSignature,
}
}
}
}
if (chunk.usageMetadata) {
responseId = chunk.responseId
lastUsageMetadata = chunk.usageMetadata
promptTokens = lastUsageMetadata.promptTokenCount ?? promptTokens
outputTokens = lastUsageMetadata.candidatesTokenCount ?? outputTokens
thoughtsTokenCount = lastUsageMetadata.thoughtsTokenCount ?? thoughtsTokenCount
cacheReadTokens = lastUsageMetadata.cachedContentTokenCount ?? cacheReadTokens
}
}
apiSuccess = true
if (lastUsageMetadata) {
const totalCost = this.calculateCost({
info,
inputTokens: promptTokens,
outputTokens,
thoughtsTokenCount,
cacheReadTokens,
})
yield {
type: "usage",
inputTokens: promptTokens - cacheReadTokens,
outputTokens,
thoughtsTokenCount,
cacheReadTokens,
cacheWriteTokens: 0,
totalCost,
id: responseId,
}
}
} catch (error) {
apiSuccess = false
// Let the error propagate to be handled by withRetry or Task.ts
// Telemetry will be sent in the finally block.
if (error instanceof Error) {
apiError = error.message
if (error instanceof ApiError) {
if (error.status === 429) {
// The API includes more details in the message
// https://github.com/googleapis/js-genai/blob/v1.11.0/src/_api_client.ts#L758
const response = this.attemptParse(error.message)
if (response && response.error) {
const responseBody = this.attemptParse(response.error.message)
if (responseBody.error) {
const detail = responseBody.error.details?.find(
(d: any) => d["@type"] === "type.googleapis.com/google.rpc.RetryInfo",
)
const detailedError = new RetriableError(
apiError,
this.parseRetryDelay(detail?.retryDelay) || undefined,
{
cause: error,
},
)
throw detailedError
}
}
throw new RetriableError(apiError, undefined, { cause: error })
}
// Fallback in case Gemini throws a rate limit error without a 429 status code
// https://github.com/cline/cline/pull/5205#discussion_r2311761559
const isRateLimit = rateLimitPatterns.some((pattern) => pattern.test(error.message))
if (isRateLimit) {
throw new RetriableError(apiError, undefined, { cause: error })
}
}
} else {
apiError = String(error)
}
throw error
} finally {
const sdkCallEndTime = Date.now()
const totalDurationSdkMs = sdkCallEndTime - sdkCallStartTime
const cacheHit = cacheReadTokens > 0
const cacheHitPercentage = promptTokens > 0 ? (cacheReadTokens / promptTokens) * 100 : undefined
const throughputTokensPerSecSdk =
totalDurationSdkMs > 0 && outputTokens > 0 ? outputTokens / (totalDurationSdkMs / 1000) : undefined
if (this.options.ulid) {
telemetryService.captureGeminiApiPerformance(this.options.ulid, modelId, {
ttftSec: ttftSdkMs !== undefined ? ttftSdkMs / 1000 : undefined,
totalDurationSec: totalDurationSdkMs / 1000,
promptTokens,
outputTokens,
cacheReadTokens,
cacheHit,
cacheHitPercentage,
apiSuccess,
apiError,
throughputTokensPerSec: throughputTokensPerSecSdk,
})
} else {
Logger.warn("GeminiHandler: ulid not available for telemetry in createMessage.")
}
}
}
/**
* Calculate the immediate dollar cost of the API call based on token usage and model pricing.
*
* This method accounts for the immediate costs of the API call:
* - Input token costs (for uncached tokens)
* - Output token costs
* - Cache read costs
* - Gemini implicit caching has no write costs
*
*/
public calculateCost({
info,
inputTokens,
outputTokens,
thoughtsTokenCount = 0,
cacheReadTokens = 0,
}: {
info: ModelInfo
inputTokens: number
outputTokens: number
thoughtsTokenCount: number
cacheReadTokens?: number
}) {
// Exit early if any required pricing information is missing
if (!info.inputPrice || !info.outputPrice) {
return undefined
}
let inputPrice = info.inputPrice
let outputPrice = info.outputPrice
// Right now, we only show the immediate costs of caching and not the ongoing costs of storing the cache
let cacheReadsPrice = info.cacheReadsPrice ?? 0
// If there's tiered pricing then adjust prices based on the input tokens used
if (info.tiers) {
const tier = info.tiers.find((tier) => inputTokens <= tier.contextWindow)
if (tier) {
inputPrice = tier.inputPrice ?? inputPrice
outputPrice = tier.outputPrice ?? outputPrice
cacheReadsPrice = tier.cacheReadsPrice ?? cacheReadsPrice
}
}
// Subtract the cached input tokens from the total input tokens
const uncachedInputTokens = inputTokens - (cacheReadTokens ?? 0)
// Calculate immediate costs only
// 1. Input token costs (for uncached tokens)
const inputTokensCost = inputPrice * (uncachedInputTokens / 1_000_000)
// 2. Output token costs
const responseTokensCost = outputPrice * ((outputTokens + thoughtsTokenCount) / 1_000_000)
// 3. Cache read costs (immediate)
const cacheReadCost = (cacheReadTokens ?? 0) > 0 ? cacheReadsPrice * ((cacheReadTokens ?? 0) / 1_000_000) : 0
// Calculate total immediate cost (excluding cache write/storage costs)
const totalCost = inputTokensCost + responseTokensCost + cacheReadCost
// Create the trace object for debugging
const trace: Record<string, { price: number; tokens: number; cost: number }> = {
input: { price: inputPrice, tokens: uncachedInputTokens, cost: inputTokensCost },
output: { price: outputPrice, tokens: outputTokens, cost: responseTokensCost },
}
// Only include cache read costs in the trace (cache write costs are tracked separately)
if ((cacheReadTokens ?? 0) > 0) {
trace.cacheRead = { price: cacheReadsPrice, tokens: cacheReadTokens ?? 0, cost: cacheReadCost }
}
// Logger.log(`[GeminiHandler] calculateCost -> ${totalCost}`, trace)
return totalCost
}
/**
* Get the model ID and info for the current configuration
*/
getModel(): { id: GeminiModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in geminiModels) {
const id = modelId as GeminiModelId
return { id, info: geminiModels[id] }
}
return {
id: geminiDefaultModelId,
info: geminiModels[geminiDefaultModelId],
}
}
/**
* Count tokens in content using the Gemini API
*/
async countTokens(content: Array<any>): Promise<number> {
try {
const client = this.ensureClient()
const { id: model } = this.getModel()
// Convert content to Gemini format
const geminiContent = content.map((block) => {
if (typeof block === "string") {
return { text: block }
}
return { text: JSON.stringify(block) }
})
// Use Gemini's token counting API
const response = await client.models.countTokens({
model,
contents: [{ parts: geminiContent }],
})
if (response.totalTokens === undefined) {
Logger.warn("Gemini token counting returned undefined, using fallback")
return this.estimateTokens(content)
}
return response.totalTokens
} catch (error) {
Logger.warn("Gemini token counting failed, using fallback", error)
return this.estimateTokens(content)
}
}
/**
* Fallback token estimation method
*/
private estimateTokens(content: Array<any>): number {
// Simple estimation: ~4 characters per token
const totalChars = content.reduce((total, block) => {
if (typeof block === "string") {
return total + block.length
}
if (block && typeof block === "object") {
// Safely stringify the object
try {
const jsonStr = JSON.stringify(block)
return total + jsonStr.length
} catch (e) {
Logger.warn("Failed to stringify block for token estimation", e)
return total
}
}
return total
}, 0)
return Math.ceil(totalChars / 4)
}
private parseRetryDelay(retryAfter?: string): number {
if (!retryAfter) {
return 0
}
const unit = retryAfter.at(-1)
const value = Number.parseInt(retryAfter, 10)
if (Number.isNaN(value)) {
return 0
}
if (unit === "s") {
return value
}
if (unit === "m") {
return value * 60 // Convert minutes to seconds
}
if (unit === "h") {
return value * 60 * 60 // Convert hours to seconds
}
return value
}
private attemptParse(str: string) {
try {
return JSON.parse(str)
} catch (_) {
return null
}
}
}
-320
View File
@@ -1,320 +0,0 @@
import { GroqModelId, groqDefaultModelId, groqModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface GroqHandlerOptions extends CommonApiHandlerOptions {
groqApiKey?: string
groqModelId?: string
groqModelInfo?: ModelInfo
apiModelId?: string // For backward compatibility
}
// Enhanced usage interface to support Groq's cached token fields
interface GroqUsage extends OpenAI.CompletionUsage {
prompt_tokens_details?: {
cached_tokens?: number
}
}
// Model family definitions for enhanced behavior
interface GroqModelFamily {
name: string
supportedFeatures: {
streaming: boolean
temperature: boolean
vision: boolean
tools: boolean
}
maxTokensOverride?: number
specialParams?: Record<string, any>
}
const MODEL_FAMILIES: Record<string, GroqModelFamily> = {
// Moonshort 4 Family - Latest generation with vision support
"kimi-k2": {
name: "kimi-k2",
supportedFeatures: { streaming: true, temperature: true, vision: true, tools: true },
maxTokensOverride: 8192,
},
// Llama 4 Family - Latest generation with vision support
llama4: {
name: "Llama 4",
supportedFeatures: { streaming: true, temperature: true, vision: true, tools: true },
maxTokensOverride: 8192,
},
// Llama 3.3 Family - Balanced performance
"llama3.3": {
name: "Llama 3.3",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 32768,
},
// Llama 3.1 Family - Fast inference
"llama3.1": {
name: "Llama 3.1",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 131072,
},
// DeepSeek Family - Reasoning-optimized
deepseek: {
name: "DeepSeek",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 8192,
specialParams: {
top_p: 0.95,
reasoning_format: "parsed",
},
},
// Qwen Family - Enhanced for Q&A
qwen: {
name: "Qwen",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 32768,
},
// Compound Models - Hybrid architectures
compound: {
name: "Compound",
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
maxTokensOverride: 8192,
},
}
export class GroqHandler implements ApiHandler {
private options: GroqHandlerOptions
private client: OpenAI | undefined
constructor(options: GroqHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.groqApiKey) {
throw new Error("Groq API key is required")
}
try {
this.client = new OpenAI({
baseURL: "https://api.groq.com/openai/v1",
apiKey: this.options.groqApiKey,
defaultHeaders: buildExternalBasicHeaders(),
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating Groq client: ${error.message}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: GroqUsage | undefined): ApiStream {
const inputTokens = usage?.prompt_tokens || 0
const outputTokens = usage?.completion_tokens || 0
const cacheReadTokens = usage?.prompt_tokens_details?.cached_tokens || 0
// Groq does not track cache writes
const cacheWriteTokens = 0
// Calculate cost using OpenAI-compatible cost calculation
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
// Calculate non-cached input tokens for proper reporting
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens,
cacheWriteTokens,
cacheReadTokens,
totalCost,
}
}
/**
* Detects the model family based on the model ID
*/
private detectModelFamily(modelId: string): GroqModelFamily {
if (modelId.includes("kimi-k2")) {
return MODEL_FAMILIES["kimi-k2"]
}
// Llama 4 variants
if (modelId.includes("llama-4") || modelId.includes("llama/llama-4")) {
return MODEL_FAMILIES.llama4
}
// Llama 3.3 variants
if (modelId.includes("llama-3.3")) {
return MODEL_FAMILIES["llama3.3"]
}
// Llama 3.1 variants
if (modelId.includes("llama-3.1")) {
return MODEL_FAMILIES["llama3.1"]
}
// DeepSeek variants
if (modelId.includes("deepseek")) {
return MODEL_FAMILIES.deepseek
}
// Qwen variants
if (modelId.includes("qwen")) {
return MODEL_FAMILIES.qwen
}
// Compound variants
if (modelId.includes("compound")) {
return MODEL_FAMILIES.compound
}
// Default fallback to Llama 3.3 behavior
return MODEL_FAMILIES["kimi-k2"]
}
/**
* Gets the optimal max_tokens based on model family and capabilities
*/
private getOptimalMaxTokens(model: { id: string; info: ModelInfo }, modelFamily: GroqModelFamily): number {
// Use model-specific max tokens if available
if (model.info.maxTokens && model.info.maxTokens > 0) {
return model.info.maxTokens
}
// Use family override if available
if (modelFamily.maxTokensOverride) {
return modelFamily.maxTokensOverride
}
// Default fallback
return 8192
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const modelFamily = this.detectModelFamily(model.id)
// Optimize parameters based on model family
const temperature = 0
const maxTokens = this.getOptimalMaxTokens(model, modelFamily)
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
// Build request parameters with model-specific optimizations
const requestParams: OpenAI.Chat.ChatCompletionCreateParamsStreaming & {
reasoning_format?: "parsed" | "raw" | "hidden"
top_p?: number
} = {
model: model.id,
max_tokens: maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature,
...getOpenAIToolParams(tools),
}
// Add any special parameters for specific model families
if (modelFamily.specialParams) {
Object.assign(requestParams, modelFamily.specialParams)
}
const toolCallProcessor = new ToolCallProcessor()
const stream = await client.chat.completions.create(requestParams)
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle reasoning field if present (for reasoning models with parsed output)
if ((delta as any)?.reasoning) {
const reasoningContent = (delta as any).reasoning as string
yield {
type: "reasoning",
reasoning: reasoningContent,
}
continue
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle content field - trust the parsed output from Groq
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
// Handle usage information
if (chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
}
}
}
/**
* Checks if the current model supports vision/images
*/
supportsImages(): boolean {
const model = this.getModel()
return model.info.supportsImages === true
}
/**
* Checks if the current model supports tools
*/
supportsTools(): boolean {
const model = this.getModel()
const modelFamily = this.detectModelFamily(model.id)
return modelFamily.supportedFeatures.tools
}
/**
* Gets model information with enhanced family detection
*/
getModel(): { id: string; info: ModelInfo } {
// First priority: groqModelId and groqModelInfo (like Requesty does)
const groqModelId = this.options.groqModelId
const groqModelInfo = this.options.groqModelInfo
if (groqModelId && groqModelInfo) {
return { id: groqModelId, info: groqModelInfo }
}
// Second priority: groqModelId with static model info
if (groqModelId && groqModelId in groqModels) {
const id = groqModelId as GroqModelId
return { id, info: groqModels[id] }
}
// Third priority: apiModelId (for backward compatibility)
const apiModelId = this.options.apiModelId
if (apiModelId && apiModelId in groqModels) {
const id = apiModelId as GroqModelId
return { id, info: groqModels[id] }
}
// Default fallback
return {
id: groqDefaultModelId,
info: groqModels[groqDefaultModelId],
}
}
/**
* Gets model family information for debugging/introspection
*/
getModelFamily(): GroqModelFamily {
const model = this.getModel()
return this.detectModelFamily(model.id)
}
}
-104
View File
@@ -1,104 +0,0 @@
import { hicapModelInfoSaneDefaults, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionReasoningEffort } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface OpenAiHandlerOptions extends CommonApiHandlerOptions {
hicapApiKey?: string
hicapModelId?: string
}
export class HicapHandler implements ApiHandler {
private options: OpenAiHandlerOptions
private client: OpenAI | undefined
constructor(options: OpenAiHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.hicapApiKey) {
throw new Error("Hicap API key is required")
}
if (!this.options.hicapModelId) {
throw new Error("Model ID is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.hicap.ai/v2/openai",
apiKey: this.options.hicapApiKey,
defaultHeaders: {
"api-key": this.options.hicapApiKey,
},
})
} catch (error: any) {
throw new Error(`Error creating OpenAI client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.hicapModelId ?? ""
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const temperature: number = 1
let reasoningEffort: ChatCompletionReasoningEffort | undefined
let maxTokens: number | undefined
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
temperature,
max_tokens: maxTokens,
reasoning_effort: reasoningEffort,
stream: true,
stream_options: { include_usage: true },
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.hicapModelId ?? "",
info: hicapModelInfoSaneDefaults,
}
}
}
@@ -1,142 +0,0 @@
import { HuaweiCloudMaasModelId, huaweiCloudMaasDefaultModelId, huaweiCloudMaasModels, ModelInfo } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface HuaweiCloudMaaSHandlerOptions extends CommonApiHandlerOptions {
huaweiCloudMaasApiKey?: string
huaweiCloudMaasModelId?: string
huaweiCloudMaasModelInfo?: ModelInfo
}
export class HuaweiCloudMaaSHandler implements ApiHandler {
private options: HuaweiCloudMaaSHandlerOptions
private client: OpenAI | undefined
constructor(options: HuaweiCloudMaaSHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.huaweiCloudMaasApiKey) {
throw new Error("Huawei Cloud MaaS API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.modelarts-maas.com/v1/",
apiKey: this.options.huaweiCloudMaasApiKey,
})
} catch (error) {
throw new Error(`Error creating Huawei Cloud MaaS client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: HuaweiCloudMaasModelId; info: ModelInfo } {
// First priority: huaweiCloudMaasModelId and huaweiCloudMaasModelInfo (like Groq does)
const huaweiCloudMaasModelId = this.options.huaweiCloudMaasModelId
const huaweiCloudMaasModelInfo = this.options.huaweiCloudMaasModelInfo
if (huaweiCloudMaasModelId && huaweiCloudMaasModelInfo) {
return { id: huaweiCloudMaasModelId as HuaweiCloudMaasModelId, info: huaweiCloudMaasModelInfo }
}
// Second priority: huaweiCloudMaasModelId with static model info
if (huaweiCloudMaasModelId && huaweiCloudMaasModelId in huaweiCloudMaasModels) {
const id = huaweiCloudMaasModelId as HuaweiCloudMaasModelId
return { id, info: huaweiCloudMaasModels[id] }
}
// Default fallback
return {
id: huaweiCloudMaasDefaultModelId,
info: huaweiCloudMaasModels[huaweiCloudMaasDefaultModelId],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
...getOpenAIToolParams(tools),
})
let reasoning: string | null = null
let didOutputUsage: boolean = false
let finalUsage: any = null
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle reasoning content detection
if (delta?.content) {
if (reasoning || delta.content.includes("<think>")) {
reasoning = (reasoning || "") + delta.content
} else if (!reasoning) {
yield {
type: "text",
text: delta.content,
}
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle reasoning output
if (reasoning || (delta && "reasoning_content" in delta && delta.reasoning_content)) {
const reasoningContent = delta?.content || ((delta as any)?.reasoning_content as string | undefined) || ""
if (reasoningContent.trim()) {
yield {
type: "reasoning",
reasoning: reasoningContent,
}
}
// Check if reasoning is complete
if (reasoning?.includes("</think>")) {
reasoning = null
}
}
// Store usage information for later output
if (chunk.usage) {
finalUsage = chunk.usage
}
// Output usage when stream is finished
if (!didOutputUsage && chunk.choices?.[0]?.finish_reason) {
if (finalUsage) {
yield {
type: "usage",
inputTokens: finalUsage.prompt_tokens || 0,
outputTokens: finalUsage.completion_tokens || 0,
cacheWriteTokens: 0,
cacheReadTokens: 0,
}
}
didOutputUsage = true
}
}
}
}
@@ -1,148 +0,0 @@
import { HuggingFaceModelId, huggingFaceDefaultModelId, huggingFaceModels, ModelInfo } from "@shared/api"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface HuggingFaceHandlerOptions extends CommonApiHandlerOptions {
huggingFaceApiKey?: string
huggingFaceModelId?: string
huggingFaceModelInfo?: ModelInfo
}
export class HuggingFaceHandler implements ApiHandler {
private options: HuggingFaceHandlerOptions
private client: OpenAI | undefined
private cachedModel: { id: HuggingFaceModelId; info: ModelInfo } | undefined
constructor(options: HuggingFaceHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.huggingFaceApiKey) {
throw new Error("Hugging Face API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://router.huggingface.co/v1",
apiKey: this.options.huggingFaceApiKey,
})
} catch (error: any) {
throw new Error(`Error creating Hugging Face client: ${error.message}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
if (!usage) {
return
}
const inputTokens = usage.prompt_tokens || 0
const outputTokens = usage.completion_tokens || 0
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens)
const usageData = {
type: "usage" as const,
inputTokens: inputTokens,
outputTokens: outputTokens,
cacheWriteTokens: 0,
cacheReadTokens: 0,
totalCost: totalCost,
}
yield usageData
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
try {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const requestParams = {
model: model.id,
max_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature: 0,
...getOpenAIToolParams(tools),
}
const toolCallProcessor = new ToolCallProcessor()
const stream = (await client.chat.completions.create(requestParams)) as any
let _chunkCount = 0
let _totalContent = ""
for await (const chunk of stream) {
_chunkCount++
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
_totalContent += delta.content
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield* this.yieldUsage(model.info, chunk.usage)
}
}
} catch (error: any) {
throw error
}
}
getModel(): { id: HuggingFaceModelId; info: ModelInfo } {
// Return cached model if available
if (this.cachedModel) {
return this.cachedModel
}
const modelId = this.options.huggingFaceModelId
// List all available models for debugging
const _availableModels = Object.keys(huggingFaceModels)
let result: { id: HuggingFaceModelId; info: ModelInfo }
if (modelId && modelId in huggingFaceModels) {
const id = modelId as HuggingFaceModelId
const modelInfo = huggingFaceModels[id]
result = { id, info: modelInfo }
} else {
const defaultInfo = huggingFaceModels[huggingFaceDefaultModelId]
result = {
id: huggingFaceDefaultModelId,
info: defaultInfo,
}
}
// Cache the result for future calls
this.cachedModel = result
return result
}
}
@@ -1,403 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { LiteLLMModelInfo, liteLlmDefaultModelId, liteLlmModelInfoSaneDefaults } from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import OpenAI from "openai"
import { StateManager } from "@/core/storage/StateManager"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient, fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { isAnthropicModelId } from "@/utils/model-utils"
import { ApiHandler, CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface LiteLlmHandlerOptions extends CommonApiHandlerOptions {
liteLlmApiKey?: string
liteLlmBaseUrl?: string
liteLlmModelId?: string
liteLlmModelInfo?: LiteLLMModelInfo
reasoningEffort?: string
thinkingBudgetTokens?: number
liteLlmUsePromptCache?: boolean
ulid?: string
}
/**
* Extended chat completion parameters that include LiteLLM-specific options
* not present in the standard OpenAI SDK types
*/
interface LiteLlmChatCompletionCreateParams extends OpenAI.Chat.ChatCompletionCreateParamsStreaming {
drop_params?: boolean
}
export interface LiteLlmModelInfoResponse {
data: Array<{
model_name: string
litellm_params: {
model: string
[key: string]: any
}
model_info: {
input_cost_per_token: number
output_cost_per_token: number
cache_creation_input_token_cost?: number
cache_read_input_token_cost?: number
supports_prompt_caching?: boolean
[key: string]: any
}
}>
}
/**
* Exported utility function to fetch LiteLLM model info
* @param baseUrl The base URL for the LiteLLM API
* @param apiKey The API key for authentication
* @returns The model info response or undefined if fetch fails
*/
export async function fetchLiteLlmModelsInfo(baseUrl: string, apiKey: string): Promise<LiteLlmModelInfoResponse | undefined> {
// Handle base URLs that already include /v1 to avoid double /v1/v1/
const normalizedBaseUrl = baseUrl.endsWith("/v1") ? baseUrl : `${baseUrl}/v1`
const url = `${normalizedBaseUrl}/model/info`
try {
const response = await fetch(url, {
method: "GET",
headers: {
accept: "application/json",
"x-litellm-api-key": apiKey,
...buildExternalBasicHeaders(),
},
})
if (response.ok) {
const data: LiteLlmModelInfoResponse = await response.json()
return data
}
Logger.error("Failed to fetch LiteLLM model info:", response.statusText)
// Try with Authorization header instead
const retryResponse = await fetch(url, {
method: "GET",
headers: {
accept: "application/json",
Authorization: `Bearer ${apiKey}`,
...buildExternalBasicHeaders(),
},
})
if (retryResponse.ok) {
const data: LiteLlmModelInfoResponse = await retryResponse.json()
return data
}
Logger.error("Failed to fetch LiteLLM model info with Authorization header:", retryResponse.statusText)
throw new Error(`Failed to fetch LiteLLM model info: ${retryResponse.statusText}`)
} catch (error) {
Logger.error("Error fetching LiteLLM model info:", error)
throw error
}
}
export class LiteLlmHandler implements ApiHandler {
private options: LiteLlmHandlerOptions
private client: OpenAI | undefined
private modelInfoCache: LiteLlmModelInfoResponse | undefined
private modelInfoCacheTimestamp = 0
private readonly modelInfoCacheTTL = 5 * 60 * 1000 // 5 minutes
constructor(options: LiteLlmHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.liteLlmApiKey) {
throw new Error("LiteLLM API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: this.options.liteLlmBaseUrl || "http://localhost:4000",
apiKey: this.options.liteLlmApiKey || "noop",
})
} catch (error) {
throw new Error(`Error creating LiteLLM client: ${error.message}`)
}
}
return this.client
}
private async modelInfo(publicModelName: string): Promise<LiteLlmModelInfoResponse["data"][number] | undefined> {
const modelInfo = await this.fetchModelsInfo()
if (!modelInfo?.data) {
return undefined
}
return modelInfo.data.find((model) => model.model_name === publicModelName)
}
private async fetchModelsInfo(): Promise<LiteLlmModelInfoResponse | undefined> {
// Check if cache is still valid
const now = Date.now()
if (this.modelInfoCache && now - this.modelInfoCacheTimestamp < this.modelInfoCacheTTL) {
return this.modelInfoCache
}
const client = this.ensureClient()
const data = await fetchLiteLlmModelsInfo(client.baseURL, this.options.liteLlmApiKey || "")
if (data) {
this.modelInfoCache = data
this.modelInfoCacheTimestamp = now
}
return data
}
private async getModelCostInfo(publicModelName: string): Promise<{
inputCostPerToken: number
outputCostPerToken: number
cacheCreationCostPerToken?: number
cacheReadCostPerToken?: number
}> {
try {
const matchingModel = await this.modelInfo(publicModelName)
if (matchingModel) {
return {
inputCostPerToken: matchingModel.model_info.input_cost_per_token || 0,
outputCostPerToken: matchingModel.model_info.output_cost_per_token || 0,
cacheCreationCostPerToken: matchingModel.model_info.cache_creation_input_token_cost,
cacheReadCostPerToken: matchingModel.model_info.cache_read_input_token_cost,
}
}
} catch (error) {
Logger.warn("Error getting LiteLLM model cost info:", error)
}
// Fallback to zero costs if we can't get the information
return {
inputCostPerToken: 0,
outputCostPerToken: 0,
}
}
async calculateCost(
prompt_tokens: number,
completion_tokens: number,
cache_creation_tokens?: number,
cache_read_tokens?: number,
): Promise<number | undefined> {
const publicModelId = this.options.liteLlmModelId || liteLlmDefaultModelId
try {
const costInfo = await this.getModelCostInfo(publicModelId)
// Calculate costs for different token types
const inputCost = Math.max(0, prompt_tokens - (cache_read_tokens || 0)) * costInfo.inputCostPerToken
const outputCost = completion_tokens * costInfo.outputCostPerToken
const cacheCreationCost = (cache_creation_tokens || 0) * (costInfo.cacheCreationCostPerToken || 0)
const cacheReadCost = (cache_read_tokens || 0) * (costInfo.cacheReadCostPerToken || 0)
const totalCost = inputCost + outputCost + cacheCreationCost + cacheReadCost
return totalCost
} catch (error) {
Logger.error("Error calculating spend:", error)
return undefined
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const formattedMessages = convertToOpenAiMessages(messages)
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam | Anthropic.Messages.TextBlockParam = {
role: "system",
content: systemPrompt,
}
const modelId = this.options.liteLlmModelId || liteLlmDefaultModelId
const isOminiModel = modelId.includes("o1-mini") || modelId.includes("o3-mini") || modelId.includes("o4-mini")
const isCodexModel = modelId.toLowerCase().includes("codex")
// Configuration for extended thinking
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = budgetTokens !== 0
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budgetTokens)
: undefined
const thinkingConfig = isAdaptiveThinkingModel
? adaptiveThinking?.enabled
? ({ type: "adaptive" } as any)
: undefined
: reasoningOn
? { type: "enabled", budget_tokens: budgetTokens }
: undefined
let temperature: number | undefined = this.options.liteLlmModelInfo?.temperature ?? 1
if (isAdaptiveThinkingModel) {
temperature = undefined
} else if ((isOminiModel || isAnthropicModelId(modelId)) && reasoningOn) {
temperature = undefined // OAI omni and Anthropic extended thinking mode doesn't support temperature
}
const modelInfo = await this.modelInfo(modelId)
// Automatically enable caching if the model supports it
const cacheControl =
(modelInfo?.model_info.supports_prompt_caching ?? false) ? { cache_control: { type: "ephemeral" } } : undefined
if (cacheControl) {
// Add cache_control to system message if enabled
// https://docs.litellm.ai/docs/providers/anthropic#caching---large-context-caching
systemMessage.content = [
{
text: systemPrompt,
type: "text",
...cacheControl,
},
] as Anthropic.Messages.TextBlockParam[]
}
// Find the last two user messages to apply caching
const userMsgIndices = formattedMessages.reduce(
(acc, msg, index) => (msg.role === "user" ? [...acc, index] : acc),
[] as number[],
)
const lastUserMsgIndex = userMsgIndices[userMsgIndices.length - 1] ?? -1
const secondLastUserMsgIndex = userMsgIndices[userMsgIndices.length - 2] ?? -1
// Apply cache_control to the last two user messages if enabled
// https://docs.litellm.ai/docs/providers/anthropic#caching---large-context-caching
const enhancedMessages: OpenAI.Chat.ChatCompletionMessageParam[] = formattedMessages.map(
(message, index): OpenAI.Chat.ChatCompletionMessageParam => {
if ((index === lastUserMsgIndex || index === secondLastUserMsgIndex) && cacheControl) {
// Handle both string and array content types
if (typeof message.content === "string") {
return {
...message,
content: [
{
type: "text",
text: message.content,
...cacheControl,
},
] as any,
}
}
if (Array.isArray(message.content)) {
// Apply cache control to the last content item in the array
return {
...message,
content: message.content.map((item, contentIndex) =>
contentIndex === (message.content?.length || 0) - 1
? {
...item,
...cacheControl,
}
: item,
) as any,
}
}
return {
...message,
...cacheControl,
}
}
return message
},
)
const stream = await client.chat.completions.create({
model: this.options.liteLlmModelId || liteLlmDefaultModelId,
messages: [systemMessage, ...enhancedMessages],
temperature,
stream: true,
drop_params: true,
...(!isCodexModel && { stream_options: { include_usage: true } }), // Codex models are only on the responses api, which doesn't take the stream_options parameter. we will need to migrate to the responses api for this to work
...(thinkingConfig && { thinking: thinkingConfig }), // Add thinking configuration when applicable
...(isAdaptiveThinkingModel && adaptiveThinking?.effort
? { output_config: { effort: adaptiveThinking.effort } }
: {}),
...(this.options.ulid && { litellm_session_id: `cline-${this.options.ulid}` }), // Add session ID for LiteLLM tracking
} as LiteLlmChatCompletionCreateParams)
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle normal text content
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
// Handle reasoning events
// This is not in the standard types but may be in the response
interface ThinkingDelta {
reasoning_content?: string
}
if ((delta as ThinkingDelta)?.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta as ThinkingDelta).reasoning_content || "",
}
}
// Handle token usage information
if (chunk.usage) {
// Extract cache-related information if available
// Need to use type assertion since these properties are not in the standard OpenAI types
const usage = chunk.usage as {
prompt_tokens: number
completion_tokens: number
cache_creation_input_tokens?: number
prompt_cache_miss_tokens?: number
cache_read_input_tokens?: number
prompt_cache_hit_tokens?: number
}
const cacheWriteTokens = usage.cache_creation_input_tokens || usage.prompt_cache_miss_tokens || 0
const cacheReadTokens = usage.cache_read_input_tokens || usage.prompt_cache_hit_tokens || 0
// Calculate cost using the actual token usage including cache tokens
const totalCost =
(await this.calculateCost(
usage.prompt_tokens || 0,
usage.completion_tokens || 0,
cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
cacheReadTokens > 0 ? cacheReadTokens : undefined,
)) || 0
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
cacheWriteTokens: cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
cacheReadTokens: cacheReadTokens > 0 ? cacheReadTokens : undefined,
totalCost,
}
}
}
}
getModel() {
const modelId = this.options.liteLlmModelId || liteLlmDefaultModelId
// Try to get model info from StateManager cache first
const cachedModelInfo = StateManager.get().getModelInfo("liteLlm", modelId)
// Fall back to provided model info or defaults if not in cache
const modelInfo = cachedModelInfo || this.options.liteLlmModelInfo || liteLlmModelInfoSaneDefaults
return {
id: modelId,
info: modelInfo,
}
}
}
@@ -1,109 +0,0 @@
import { type ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import type { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import type { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface LmStudioHandlerOptions extends CommonApiHandlerOptions {
lmStudioBaseUrl?: string
lmStudioModelId?: string
lmStudioMaxTokens?: string
}
export class LmStudioHandler implements ApiHandler {
private options: LmStudioHandlerOptions
private client: OpenAI | undefined
constructor(options: LmStudioHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
try {
this.client = createOpenAIClient({
// Docs on the new v0 api endpoint: https://lmstudio.ai/docs/app/api/endpoints/rest
baseURL: new URL("api/v0", this.options.lmStudioBaseUrl || "http://localhost:1234").toString(),
apiKey: "noop",
})
} catch (error) {
throw new Error(`Error creating LM Studio client: ${error.message}`)
}
}
return this.client
}
@withRetry({ retryAllErrors: true })
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
try {
const stream = await client.chat.completions.create({
model: this.getModel().id,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
max_completion_tokens: this.options.lmStudioMaxTokens ? Number(this.options.lmStudioMaxTokens) : undefined,
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const choice = chunk.choices?.[0]
const delta = choice?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
}
}
}
} catch {
// LM Studio doesn't return an error code/body for now
throw new Error(
"Please check the LM Studio developer logs to debug what went wrong. You may need to load the model with a larger context length to work with Cline's prompts. Alternatively, try enabling Compact Prompt in your settings when working with a limited context window.",
)
}
}
getModel(): { id: string; info: ModelInfo } {
const info = { ...openAiModelInfoSaneDefaults }
const maxTokens = Number(this.options.lmStudioMaxTokens)
if (!Number.isNaN(maxTokens)) {
info.contextWindow = maxTokens
}
return {
id: this.options.lmStudioModelId || "",
info,
}
}
}
@@ -1,208 +0,0 @@
import { Anthropic } from "@anthropic-ai/sdk"
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
import { Stream as AnthropicStream } from "@anthropic-ai/sdk/streaming"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { MinimaxModelId, ModelInfo, minimaxDefaultModelId, minimaxModels } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ClineTool } from "@/shared/tools"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
interface MinimaxHandlerOptions extends CommonApiHandlerOptions {
minimaxApiKey?: string
minimaxApiLine?: string
apiModelId?: string
thinkingBudgetTokens?: number
}
export class MinimaxHandler implements ApiHandler {
private options: MinimaxHandlerOptions
private client: Anthropic | undefined
constructor(options: MinimaxHandlerOptions) {
this.options = options
}
private ensureClient(): Anthropic {
if (!this.client) {
if (!this.options.minimaxApiKey) {
throw new Error("MiniMax API key is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
this.client = new Anthropic({
apiKey: this.options.minimaxApiKey,
baseURL:
this.options.minimaxApiLine === "china"
? "https://api.minimaxi.com/anthropic"
: "https://api.minimax.io/anthropic",
defaultHeaders: externalHeaders,
fetch, // Use configured fetch with proxy support
})
} catch (error) {
throw new Error(`Error creating MiniMax client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ClineTool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
// Tools are available only when native tools are enabled
const nativeToolsOn = tools?.length && tools?.length > 0
const budget_tokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
// MiniMax M2 uses Anthropic API format
const stream: AnthropicStream<Anthropic.RawMessageStreamEvent> = await client.messages.create({
model: model.id,
max_tokens: model.info.maxTokens || 8192,
system: [{ text: systemPrompt, type: "text" }],
messages,
stream: true,
tools: nativeToolsOn ? (tools as AnthropicTool[]) : undefined,
thinking: reasoningOn ? { type: "enabled", budget_tokens: budget_tokens } : undefined,
// "Thinking isn't compatible with temperature, top_p, or top_k modifications"
temperature: reasoningOn ? undefined : 1.0, // MiniMax recommends 1.0, range is (0.0, 1.0]
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
tool_choice: nativeToolsOn && !reasoningOn ? { type: "any" } : undefined,
})
const lastStartedToolCall = { id: "", name: "", arguments: "" }
for await (const chunk of stream) {
switch (chunk?.type) {
case "message_start": {
// tells us cache reads/writes/input/output
const usage = chunk.message.usage
yield {
type: "usage",
inputTokens: usage.input_tokens || 0,
outputTokens: usage.output_tokens || 0,
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
cacheReadTokens: usage.cache_read_input_tokens || undefined,
}
break
}
case "message_delta":
// tells us stop_reason, stop_sequence, and output tokens along the way and at the end of the message
yield {
type: "usage",
inputTokens: 0,
outputTokens: chunk.usage.output_tokens || 0,
}
break
case "message_stop":
// no usage data, just an indicator that the message is done
break
case "content_block_start":
switch (chunk.content_block.type) {
case "thinking":
yield {
type: "reasoning",
reasoning: chunk.content_block.thinking || "",
signature: chunk.content_block.signature,
}
break
case "redacted_thinking":
// Content is encrypted, and we don't want to pass placeholder text back to the API
yield {
type: "reasoning",
reasoning: "[Redacted thinking block]",
redacted_data: chunk.content_block.data,
}
break
case "tool_use":
if (chunk.content_block.id && chunk.content_block.name) {
// Store tool call information for streaming
lastStartedToolCall.id = chunk.content_block.id
lastStartedToolCall.name = chunk.content_block.name
lastStartedToolCall.arguments = ""
}
break
case "text":
// we may receive multiple text blocks, in which case just insert a line break between them
if (chunk.index > 0) {
yield {
type: "text",
text: "\n",
}
}
yield {
type: "text",
text: chunk.content_block.text,
}
break
}
break
case "content_block_delta":
switch (chunk.delta.type) {
case "thinking_delta":
// 'reasoning' type just displays in the UI, but reasoning with signature will be used to send the thinking traces back to the API
yield {
type: "reasoning",
reasoning: chunk.delta.thinking,
}
break
case "signature_delta":
// It's used when sending the thinking block back to the API
// API expects this in completed form, not as array of deltas
if (chunk.delta.signature) {
yield {
type: "reasoning",
reasoning: "",
signature: chunk.delta.signature,
}
}
break
case "text_delta":
yield {
type: "text",
text: chunk.delta.text,
}
break
case "input_json_delta":
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
// Convert Anthropic tool_use to OpenAI-compatible format for internal processing
yield {
type: "tool_calls",
tool_call: {
...lastStartedToolCall,
function: {
...lastStartedToolCall,
id: lastStartedToolCall.id,
name: lastStartedToolCall.name,
arguments: chunk.delta.partial_json,
},
},
}
}
break
}
break
case "content_block_stop":
lastStartedToolCall.id = ""
lastStartedToolCall.name = ""
lastStartedToolCall.arguments = ""
break
}
}
}
getModel(): { id: MinimaxModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in minimaxModels) {
const id = modelId as MinimaxModelId
return { id, info: minimaxModels[id] }
}
return { id: minimaxDefaultModelId, info: minimaxModels[minimaxDefaultModelId] }
}
}
@@ -1,154 +0,0 @@
import { Mistral } from "@mistralai/mistralai"
import { HTTPClient } from "@mistralai/mistralai/lib/http"
import { Tool as MistralTool } from "@mistralai/mistralai/models/components/tool"
import { MistralModelId, ModelInfo, mistralDefaultModelId, mistralModels } from "@shared/api"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToMistralMessages } from "../transform/mistral-format"
import { ApiStream } from "../transform/stream"
interface MistralHandlerOptions extends CommonApiHandlerOptions {
mistralApiKey?: string
apiModelId?: string
}
export class MistralHandler implements ApiHandler {
private options: MistralHandlerOptions
private client: Mistral | undefined
constructor(options: MistralHandlerOptions) {
this.options = options
}
private ensureClient(): Mistral {
if (!this.client) {
if (!this.options.mistralApiKey) {
throw new Error("Mistral API key is required")
}
try {
const externalHeaders = buildExternalBasicHeaders()
// Create HTTP client with custom fetch for proxy support
// The Mistral SDK's HTTPClient passes a Request object to the fetcher,
// but we need to extract the URL and init options to pass to our fetch wrapper
// which properly handles proxy configuration in standalone mode (JetBrains/CLI)
const httpClient = new HTTPClient({
fetcher: async (input: RequestInfo | URL, init?: RequestInit) => {
// Handle both string/URL and Request object inputs
if (input instanceof Request) {
Object.keys(externalHeaders).forEach((key) => {
if (!input.headers.has(key)) {
input.headers.set(key, externalHeaders[key])
}
})
return fetch(input.url, {
method: input.method,
headers: input.headers,
body: input.body,
redirect: input.redirect,
signal: input.signal,
// duplex is required when sending a body stream in Node.js/undici
duplex: input.body ? "half" : undefined,
...init,
} as RequestInit)
}
// Merge external headers with existing headers
const mergedInit = {
...init,
headers: {
...externalHeaders,
...(init?.headers || {}),
},
}
return fetch(input, mergedInit)
},
})
this.client = new Mistral({
apiKey: this.options.mistralApiKey,
httpClient,
})
} catch (error) {
throw new Error(`Error creating Mistral client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const stream = await client.chat
.stream({
model: this.getModel().id,
// max_completion_tokens: this.getModel().info.maxTokens,
temperature: 0,
messages: [{ role: "system", content: systemPrompt }, ...convertToMistralMessages(messages)],
stream: true,
tools: tools?.length ? (tools as MistralTool[]) : undefined,
toolChoice: tools?.length ? "any" : undefined,
})
.catch((err) => {
// The Mistal SDK uses statusCode instead of status
// However, if they introduce status for something, I don't want to override it
if ("statusCode" in err && !("status" in err)) {
err.status = err.statusCode
}
throw err
})
for await (const chunk of stream) {
const delta = chunk.data.choices[0]?.delta
if (delta.toolCalls) {
for (const toolCall of delta.toolCalls) {
yield {
type: "tool_calls",
tool_call: {
function: {
id: toolCall.id,
name: toolCall.function.name,
arguments: JSON.stringify(toolCall.function.arguments),
},
},
}
}
} else if (delta?.content) {
let content: string = ""
if (typeof delta.content === "string") {
content = delta.content
} else if (Array.isArray(delta.content)) {
content = delta.content.map((c) => (c.type === "text" ? c.text : "")).join("")
}
yield {
type: "text",
text: content,
}
}
if (chunk.data.usage) {
yield {
type: "usage",
inputTokens: chunk.data.usage.promptTokens || 0,
outputTokens: chunk.data.usage.completionTokens || 0,
}
}
}
}
getModel(): { id: MistralModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in mistralModels) {
const id = modelId as MistralModelId
return { id, info: mistralModels[id] }
}
return {
id: mistralDefaultModelId,
info: mistralModels[mistralDefaultModelId],
}
}
}
@@ -1,110 +0,0 @@
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ModelInfo, MoonshotModelId, moonshotDefaultModelId, moonshotModels } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface MoonshotHandlerOptions extends CommonApiHandlerOptions {
moonshotApiKey?: string
moonshotApiLine?: string
apiModelId?: string
}
// Enhanced usage interface to support Moonshot's cached token field
interface MoonshotUsage extends OpenAI.CompletionUsage {
cached_tokens?: number
}
export class MoonshotHandler implements ApiHandler {
private client: OpenAI | undefined
constructor(private readonly options: MoonshotHandlerOptions) {}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.moonshotApiKey) {
throw new Error("Moonshot API key is required")
}
try {
this.client = createOpenAIClient({
baseURL:
this.options.moonshotApiLine === "china" ? "https://api.moonshot.cn/v1" : "https://api.moonshot.ai/v1",
apiKey: this.options.moonshotApiKey,
})
} catch (error) {
throw new Error(`Error creating Moonshot client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
messages: openAiMessages,
temperature: model.info.temperature,
max_tokens: model.info.maxTokens,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
const usage = chunk.usage as MoonshotUsage
yield {
type: "usage",
cacheWriteTokens: 0,
cacheReadTokens: usage.cached_tokens ?? 0,
inputTokens: (usage.prompt_tokens || 0) - (usage.cached_tokens ?? 0),
outputTokens: usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: MoonshotModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in moonshotModels) {
const id = modelId as MoonshotModelId
return { id, info: moonshotModels[id] }
}
return { id: moonshotDefaultModelId, info: moonshotModels[moonshotDefaultModelId] }
}
}
@@ -1,96 +0,0 @@
import { type ModelInfo, type NebiusModelId, nebiusDefaultModelId, nebiusModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface NebiusHandlerOptions extends CommonApiHandlerOptions {
nebiusApiKey?: string
apiModelId?: string
}
export class NebiusHandler implements ApiHandler {
private client: OpenAI | undefined
constructor(private readonly options: NebiusHandlerOptions) {}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.nebiusApiKey) {
throw new Error("Nebius API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.studio.nebius.ai/v1",
apiKey: this.options.nebiusApiKey,
})
} catch (error) {
throw new Error(`Error creating Nebius client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = model.id.includes("DeepSeek-R1")
? convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
: [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
const stream = await client.chat.completions.create({
model: model.id,
messages: openAiMessages,
temperature: 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId !== undefined && modelId in nebiusModels) {
return { id: modelId, info: nebiusModels[modelId as NebiusModelId] }
}
return { id: nebiusDefaultModelId, info: nebiusModels[nebiusDefaultModelId] }
}
}
@@ -1,93 +0,0 @@
import { ModelInfo, NousResearchModelId, nousResearchDefaultModelId, nousResearchModels } from "@shared/api"
import OpenAI from "openai"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface NousResearchHandlerOptions extends CommonApiHandlerOptions {
nousResearchApiKey?: string
apiModelId?: string
}
export class NousResearchHandler implements ApiHandler {
private options: NousResearchHandlerOptions
private client: OpenAI | undefined
constructor(options: NousResearchHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.nousResearchApiKey) {
throw new Error("NousResearch API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://inference-api.nousResearch.com/v1",
apiKey: this.options.nousResearchApiKey,
})
} catch (error: any) {
throw new Error(`Error creating NousResearch client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const stream = await client.chat.completions.create({
model: model.id,
messages: openAiMessages,
temperature: 0,
stream: true,
stream_options: { include_usage: true },
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: NousResearchModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in nousResearchModels) {
const id = modelId as NousResearchModelId
return { id, info: nousResearchModels[id] }
}
return { id: nousResearchDefaultModelId, info: nousResearchModels[nousResearchDefaultModelId] }
}
}
-473
View File
@@ -1,473 +0,0 @@
import { Anthropic, APIError as AnthropicAPIError } from "@anthropic-ai/sdk"
import { liteLlmDefaultModelId, liteLlmModelInfoSaneDefaults, ModelInfo } from "@shared/api"
import OpenAI, { APIError as OpenAIAPIError, OpenAIError } from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { OcaAuthService } from "@/services/auth/oca/OcaAuthService"
import {
DEFAULT_EXTERNAL_OCA_BASE_URL,
DEFAULT_INTERNAL_OCA_BASE_URL,
OCI_HEADER_OPC_REQUEST_ID,
} from "@/services/auth/oca/utils/constants"
import { createOcaHeaders } from "@/services/auth/oca/utils/utils"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { OcaModelInfo } from "@/shared/api"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiFormat } from "@/shared/proto/index.cline"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, type CommonApiHandlerOptions } from ".."
import { withRetry } from "../retry"
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
import { convertOpenAIToolsToAnthropicTools, handleAnthropicMessagesApiStreamResponse } from "../utils/messages_api_support"
import { handleResponsesApiStreamResponse } from "../utils/responses_api_support"
export interface OcaHandlerOptions extends CommonApiHandlerOptions {
ocaBaseUrl?: string
ocaModelId?: string
ocaModelInfo?: OcaModelInfo
ocaReasoningEffort?: string
thinkingBudgetTokens?: number
ocaUsePromptCache?: boolean
taskId?: string
ocaMode?: string // "internal" or "external"
}
export class OcaHandler implements ApiHandler {
protected options: OcaHandlerOptions
protected openAIClient: OpenAI | undefined
protected anthropicClient: Anthropic | undefined
protected externalHeaders: Record<string, string> = {}
constructor(options: OcaHandlerOptions) {
this.options = options
}
protected initializeOpenAIClient(options: OcaHandlerOptions): OpenAI {
const externalHeaders = buildExternalBasicHeaders()
return new (class OCIOpenAI extends OpenAI {
protected override async prepareOptions(opts: any): Promise<void> {
const token = await OcaAuthService.getInstance().getAuthToken()
if (!token) {
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
}
opts.headers ??= {}
// OCA Headers
const ociHeaders = await createOcaHeaders(token, options.taskId!)
opts.headers = { ...opts.headers, ...externalHeaders, ...ociHeaders }
Logger.log(`Making request with customer opc-request-id: ${opts.headers?.["opc-request-id"]}`)
return super.prepareOptions(opts)
}
protected override makeStatusError(
status: number | undefined,
error: Object | undefined,
message: string | undefined,
headers: any | undefined,
): OpenAIAPIError {
interface OciError {
code?: string
message?: string
}
let ociErrorMessage = message
if (typeof error === "object" && error !== null) {
try {
ociErrorMessage = JSON.stringify(error)
const ociErr = error as OciError
if (ociErr.code !== undefined && ociErr.message !== undefined) {
ociErrorMessage = `${ociErr.code}: ${ociErr.message}`
}
} catch {}
}
const opcRequestId = headers?.[OCI_HEADER_OPC_REQUEST_ID]
if (opcRequestId) {
ociErrorMessage += `\n(${OCI_HEADER_OPC_REQUEST_ID}: ${opcRequestId})`
}
const statusCode = typeof status === "number" ? status : 500
return super.makeStatusError(statusCode, error ?? {}, ociErrorMessage, headers)
}
})({
baseURL:
options.ocaBaseUrl ||
(options.ocaMode === "internal" ? DEFAULT_INTERNAL_OCA_BASE_URL : DEFAULT_EXTERNAL_OCA_BASE_URL),
apiKey: "noop",
fetch, // Use configured fetch with proxy support
})
}
protected initializeAnthropicClient(options: OcaHandlerOptions): Anthropic {
const externalHeaders = buildExternalBasicHeaders()
return new (class OCIAnthropic extends Anthropic {
protected override async prepareOptions(opts: any): Promise<void> {
const token = await OcaAuthService.getInstance().getAuthToken()
if (!token) {
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
}
opts.headers ??= {}
// OCA Headers
const ociHeaders = await createOcaHeaders(token, options.taskId!)
opts.headers = { ...opts.headers, ...externalHeaders, ...ociHeaders }
Logger.log(`Making request with customer opc-request-id: ${opts.headers?.["opc-request-id"]}`)
return super.prepareOptions(opts)
}
protected override makeStatusError(
status: number | undefined,
error: Object | undefined,
message: string | undefined,
headers: any | undefined,
): AnthropicAPIError {
interface OciError {
code?: string
message?: string
}
let ociErrorMessage = message
if (typeof error === "object" && error !== null) {
try {
ociErrorMessage = JSON.stringify(error)
const ociErr = error as OciError
if (ociErr.code !== undefined && ociErr.message !== undefined) {
ociErrorMessage = `${ociErr.code}: ${ociErr.message}`
}
} catch {}
}
const opcRequestId = headers?.[OCI_HEADER_OPC_REQUEST_ID]
if (opcRequestId) {
ociErrorMessage += `\n(${OCI_HEADER_OPC_REQUEST_ID}: ${opcRequestId})`
}
const statusCode = typeof status === "number" ? status : 500
return super.makeStatusError(statusCode, error ?? {}, ociErrorMessage, headers)
}
})({
baseURL:
options.ocaBaseUrl ||
(options.ocaMode === "internal" ? DEFAULT_INTERNAL_OCA_BASE_URL : DEFAULT_EXTERNAL_OCA_BASE_URL),
apiKey: "noop",
fetch, // Use configured fetch with proxy support
})
}
protected ensureOpenAIClient(): OpenAI {
if (!this.openAIClient) {
if (!this.options.ocaModelId) {
throw new Error("Oracle Code Assist (OCA) model is not selected")
}
try {
this.openAIClient = this.initializeOpenAIClient(this.options)
} catch (error) {
throw new Error(`Error creating Oracle Code Assist (OCA) client: ${error.message}`)
}
}
return this.openAIClient
}
protected ensureAnthropicClient(): Anthropic {
if (!this.anthropicClient) {
if (!this.options.ocaModelId) {
throw new Error("Oracle Code Assist (OCA) model is not selected")
}
try {
this.anthropicClient = this.initializeAnthropicClient(this.options)
} catch (error) {
throw new Error(`Error creating Oracle Code Assist (OCA) client: ${error.message}`)
}
}
return this.anthropicClient
}
async getApiCosts(prompt_tokens: number, completion_tokens: number): Promise<number | undefined> {
// Reference: https://github.com/BerriAI/litellm/blob/122ee634f434014267af104814022af1d9a0882f/litellm/proxy/spend_tracking/spend_management_endpoints.py#L1473
const client = this.ensureOpenAIClient()
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
const token = await OcaAuthService.getInstance().getAuthToken()
if (!token) {
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
}
const externalHeaders = buildExternalBasicHeaders()
const ociHeaders = await createOcaHeaders(token, this.options.taskId!)
Logger.log(`Making calculate cost request with customer opc-request-id: ${ociHeaders["opc-request-id"]}`)
try {
const response = await fetch(`${client.baseURL}/spend/calculate`, {
method: "POST",
headers: { ...externalHeaders, ...ociHeaders },
body: JSON.stringify({
completion_response: {
model: modelId,
usage: {
prompt_tokens,
completion_tokens,
},
},
}),
})
if (response.ok) {
const data: { cost: number } = await response.json()
return data.cost
}
Logger.error("Error calculating spend:", response.statusText)
return undefined
} catch (error) {
Logger.error("Error calculating spend:", error)
return undefined
}
}
async calculateCost(
modelInfo: ModelInfo,
inputTokens: number,
outputTokens: number,
_cacheWriteTokens?: number,
_cacheReadTokens?: number,
) {
const inputCost = (await this.getApiCosts(1e6, 0)) || 0
const outputCost = (await this.getApiCosts(0, 1e6)) || 0
const totalCost = (inputCost * inputTokens) / 1e6 + (outputCost * outputTokens) / 1e6
return totalCost
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
if (this.options.ocaModelInfo?.apiFormat == ApiFormat.OPENAI_RESPONSES) {
yield* this.createMessageResponsesApi(systemPrompt, messages, tools)
} else if (this.options.ocaModelInfo?.apiFormat == ApiFormat.ANTHROPIC_CHAT) {
yield* this.createMessageMessagesApi(systemPrompt, messages, tools)
} else {
yield* this.createMessageChatApi(systemPrompt, messages, tools)
}
}
async *createMessageChatApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureOpenAIClient()
const formattedMessages = convertToOpenAiMessages(messages)
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam = {
role: "system",
content: systemPrompt,
}
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
const isOminiModel = modelId.includes("o1-mini") || modelId.includes("o3-mini") || modelId.includes("o4-mini")
// Configuration for extended thinking
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = budgetTokens !== 0
const thinkingConfig = reasoningOn ? { type: "enabled", budget_tokens: budgetTokens } : undefined
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
const maxTokens: number | undefined = this.options.ocaModelInfo?.maxTokens
if (isOminiModel && reasoningOn) {
temperature = undefined // Thinking mode doesn't support temperature
}
// Define cache control object if prompt caching is enabled
const cacheControl = this.options.ocaUsePromptCache ? { cache_control: { type: "ephemeral" } } : undefined
// Add cache_control to system message if enabled
const enhancedSystemMessage = {
...systemMessage,
...(cacheControl && cacheControl),
}
// Find the last two user messages to apply caching
const userMsgIndices = formattedMessages.reduce((acc, msg, index) => {
if (msg.role === "user") {
acc.push(index)
}
return acc
}, [] as number[])
const lastUserMsgIndex = userMsgIndices[userMsgIndices.length - 1] ?? -1
const secondLastUserMsgIndex = userMsgIndices[userMsgIndices.length - 2] ?? -1
// Apply cache_control to the last two user messages if enabled
const enhancedMessages = formattedMessages.map((message, index) => {
if ((index === lastUserMsgIndex || index === secondLastUserMsgIndex) && cacheControl) {
return {
...message,
...cacheControl,
}
}
return message
})
const toolCallProcessor = new ToolCallProcessor()
const chatCompletionsParams: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming = {
model: this.options.ocaModelId || liteLlmDefaultModelId,
messages: [enhancedSystemMessage, ...enhancedMessages],
temperature,
stream: true,
max_completion_tokens: maxTokens,
max_tokens: maxTokens,
stream_options: { include_usage: true },
...(thinkingConfig && { thinking: thinkingConfig }), // Add thinking configuration when applicable
...(this.options.taskId && {
litellm_session_id: `cline-${this.options.taskId}`,
...getOpenAIToolParams(tools),
}), // Add session ID for LiteLLM tracking
}
if (this.options.ocaModelInfo?.supportsReasoningEffort) {
chatCompletionsParams["reasoning_effort"] = this.options.ocaReasoningEffort || ("medium" as any)
}
const stream = await client.chat.completions.create(chatCompletionsParams)
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
// Handle normal text content
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
// Handle reasoning events (thinking)
// Thinking is not in the standard types but may be in the response
interface ThinkingDelta {
thinking?: string
}
if ((delta as ThinkingDelta)?.thinking) {
yield {
type: "reasoning",
reasoning: (delta as ThinkingDelta).thinking || "",
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle token usage information
if (chunk.usage) {
const totalCost = await this.calculateCost(
this.options.ocaModelInfo!,
chunk.usage.prompt_tokens,
chunk.usage.completion_tokens,
)
// Extract cache-related information if available
// Need to use type assertion since these properties are not in the standard OpenAI types
const usage = chunk.usage as {
prompt_tokens: number
completion_tokens: number
cache_creation_input_tokens?: number
prompt_cache_miss_tokens?: number
cache_read_input_tokens?: number
prompt_cache_hit_tokens?: number
}
const cacheWriteTokens = usage.cache_creation_input_tokens || usage.prompt_cache_miss_tokens || 0
const cacheReadTokens = usage.cache_read_input_tokens || usage.prompt_cache_hit_tokens || 0
yield {
type: "usage",
inputTokens: usage.prompt_tokens || 0,
outputTokens: usage.completion_tokens || 0,
cacheWriteTokens: cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
cacheReadTokens: cacheReadTokens > 0 ? cacheReadTokens : undefined,
totalCost,
}
}
}
}
async *createMessageResponsesApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureOpenAIClient()
const inputMessages = convertToOpenAIResponsesInput(messages, { usePreviousResponseId: false }).input
// Convert messages to Responses API input format
const input: OpenAI.Responses.ResponseInputItem[] = [{ role: "system", content: systemPrompt }, ...inputMessages]
// Convert ChatCompletion tools to Responses API format if provided
const responseTools = tools
?.filter((tool) => tool?.type === "function")
.map((tool: any) => ({
type: "function" as const,
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters,
strict: tool.function.strict ?? true, // Responses API defaults to strict mode
}))
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
const maxOutputTokens: number | undefined = this.options.ocaModelInfo?.maxTokens
const ocaModelInfo = this.options.ocaModelInfo
if (!ocaModelInfo) {
throw new Error("Oracle Code Assist (OCA) model info is required for Responses API")
}
const reasoningOn = !!ocaModelInfo.supportsReasoning
if (reasoningOn) {
temperature = undefined
}
const responsesParams: OpenAI.Responses.ResponseCreateParamsStreaming = {
model: this.options.ocaModelId || liteLlmDefaultModelId,
input,
stream: true,
tools: responseTools,
...(typeof temperature === "number" ? { temperature } : {}),
...(typeof maxOutputTokens === "number" && maxOutputTokens > 0 ? { max_output_tokens: maxOutputTokens } : {}),
}
if (reasoningOn) {
responsesParams.reasoning = { effort: this.options.ocaReasoningEffort as any, summary: "auto" }
}
// Create the response using Responses API
const stream = await client.responses.create(responsesParams)
yield* handleResponsesApiStreamResponse(stream, ocaModelInfo, this.calculateCost.bind(this))
}
async *createMessageMessagesApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureAnthropicClient()
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = this.options.ocaModelInfo?.supportsReasoning && budgetTokens !== 0
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
const maxTokens: number | undefined = this.options.ocaModelInfo?.maxTokens || 8192
if (reasoningOn) {
temperature = 0
}
const anthropicTools = convertOpenAIToolsToAnthropicTools(tools)
const anthropicMessages = sanitizeAnthropicMessages(messages, this.options.ocaUsePromptCache ?? false)
const stream = await client.messages.create({
model: modelId,
max_tokens: maxTokens,
temperature: reasoningOn ? undefined : temperature,
system: [
{
text: systemPrompt,
type: "text",
cache_control: this.options.ocaUsePromptCache ? { type: "ephemeral" } : undefined,
},
],
messages: anthropicMessages,
stream: true,
tools: anthropicTools,
thinking: reasoningOn ? { type: "enabled", budget_tokens: budgetTokens } : undefined,
})
yield* handleAnthropicMessagesApiStreamResponse(stream)
}
getModel() {
return {
id: this.options.ocaModelId || liteLlmDefaultModelId,
info: this.options.ocaModelInfo || liteLlmModelInfoSaneDefaults,
}
}
}
@@ -1,159 +0,0 @@
import { type ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import { type Config, type Message, Ollama } from "ollama"
import type { ChatCompletionTool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import type { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOllamaMessages } from "../transform/ollama-format"
import type { ApiStream } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
interface OllamaHandlerOptions extends CommonApiHandlerOptions {
ollamaBaseUrl?: string
ollamaApiKey?: string
ollamaModelId?: string
ollamaApiOptionsCtxNum?: string
requestTimeoutMs?: number
}
const DEFAULT_CONTEXT_WINDOW = 32768
export class OllamaHandler implements ApiHandler {
private options: OllamaHandlerOptions
private client: Ollama | undefined
constructor(options: OllamaHandlerOptions) {
const ollamaApiOptionsCtxNum = (options.ollamaApiOptionsCtxNum ?? DEFAULT_CONTEXT_WINDOW).toString()
this.options = { ...options, ollamaApiOptionsCtxNum }
}
private ensureClient(): Ollama {
if (!this.client) {
try {
const externalHeaders = buildExternalBasicHeaders()
const clientOptions: Partial<Config> = {
host: this.options.ollamaBaseUrl,
fetch,
headers: externalHeaders,
}
// Add API key if provided (for Ollama cloud or authenticated instances)
if (this.options.ollamaApiKey) {
clientOptions.headers = {
...clientOptions.headers,
Authorization: `Bearer ${this.options.ollamaApiKey}`,
}
}
this.client = new Ollama(clientOptions)
} catch (error) {
throw new Error(`Error creating Ollama client: ${error.message}`)
}
}
return this.client
}
@withRetry({ retryAllErrors: true })
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
const client = this.ensureClient()
const ollamaMessages: Message[] = [{ role: "system", content: systemPrompt }, ...convertToOllamaMessages(messages)]
try {
// Create a promise that rejects after timeout
const timeoutMs = this.options.requestTimeoutMs || 30000
const timeoutPromise = new Promise<never>((_, reject) => {
setTimeout(() => reject(new Error(`Ollama request timed out after ${timeoutMs / 1000} seconds`)), timeoutMs)
})
// Create the actual API request promise
const apiPromise = client.chat({
model: this.getModel().id,
messages: ollamaMessages,
stream: true,
options: {
num_ctx: Number(this.options.ollamaApiOptionsCtxNum),
},
tools: tools as any,
})
const toolCallProcessor = new ToolCallProcessor()
// Race the API request against the timeout
const stream = (await Promise.race([apiPromise, timeoutPromise])) as Awaited<typeof apiPromise>
try {
for await (const chunk of stream) {
Logger.debug("[OllamaHandler] Message Chunk" + JSON.stringify(chunk))
const delta = chunk.message
if (delta?.tool_calls) {
Logger.debug(`[OllamaHandler] Tool Calls Detected: ${JSON.stringify(delta.tool_calls)}`)
yield* toolCallProcessor.processToolCallDeltas(
delta.tool_calls?.map((tc, inx) => ({
index: inx,
id: `ollama-tool-${inx}`,
function: {
name: tc.function.name,
arguments:
typeof tc.function.arguments === "string"
? tc.function.arguments
: JSON.stringify(tc.function.arguments),
},
type: "function",
})),
)
}
if (typeof delta.content === "string") {
yield {
type: "text",
text: delta.content,
}
}
// Handle token usage if available
if (chunk.eval_count !== undefined || chunk.prompt_eval_count !== undefined) {
yield {
type: "usage",
inputTokens: chunk.prompt_eval_count || 0,
outputTokens: chunk.eval_count || 0,
}
}
}
} catch (streamError: any) {
Logger.error("Error processing Ollama stream:", streamError)
throw new Error(`Ollama stream processing error: ${streamError.message || "Unknown error"}`)
}
} catch (error) {
// Check if it's a timeout error
if (error?.message?.includes("timed out")) {
const timeoutMs = this.options.requestTimeoutMs || 30000
throw new Error(`Ollama request timed out after ${timeoutMs / 1000} seconds`)
}
// Enhance error reporting
const statusCode = error.status || error.statusCode
const errorMessage = error.message || "Unknown error"
Logger.error(`Ollama API error (${statusCode || "unknown"}): ${errorMessage}`)
throw error
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.ollamaModelId || "",
info: {
...openAiModelInfoSaneDefaults,
contextWindow: Number(this.options.ollamaApiOptionsCtxNum),
},
}
}
abort(): void {
this.client?.abort()
}
}
@@ -1,732 +0,0 @@
import { ModelInfo, OpenAiCodexModelId, openAiCodexDefaultModelId, openAiCodexModels } from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import OpenAI from "openai"
import type { ChatCompletionTool } from "openai/resources/chat/completions"
import * as os from "os"
import { MessageEvent as UndiciMessageEvent, WebSocket as UndiciWebSocket } from "undici"
import { v7 as uuidv7 } from "uuid"
import { openAiCodexOAuthManager } from "@/integrations/openai-codex/oauth"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { featureFlagsService } from "@/services/feature-flags"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { ApiFormat } from "@/shared/proto/cline/models"
import { FeatureFlag } from "@/shared/services/feature-flags/feature-flags"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
import { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
/**
* OpenAI Codex base URL for API requests
* Routes to chatgpt.com/backend-api/codex
*/
const CODEX_API_BASE_URL = "https://chatgpt.com/backend-api/codex"
const CODEX_RESPONSES_WEBSOCKET_URL = "wss://chatgpt.com/backend-api/codex/responses"
interface OpenAiCodexHandlerOptions extends CommonApiHandlerOptions {
reasoningEffort?: string
apiModelId?: string
}
/**
* OpenAiCodexHandler - Uses OpenAI Responses API with OAuth authentication
*
* Key differences from OpenAiNativeHandler:
* - Uses OAuth Bearer tokens instead of API keys
* - Routes requests to Codex backend (chatgpt.com/backend-api/codex)
* - Subscription-based pricing (no per-token costs)
* - Limited model subset
* - Custom headers for Codex backend
*/
export class OpenAiCodexHandler implements ApiHandler {
private options: OpenAiCodexHandlerOptions
private client?: OpenAI
private responsesWs: UndiciWebSocket | undefined
private websocketRequestInFlight = false
// Session ID for the Codex API (persists for the lifetime of the handler)
private readonly sessionId: string
// Abort controller for cancelling ongoing requests
private abortController?: AbortController
// Track tool call identity for streaming
private pendingToolCallId: string | undefined
private pendingToolCallName: string | undefined
constructor(options: OpenAiCodexHandlerOptions) {
this.options = options
this.sessionId = uuidv7()
}
private normalizeUsage(usage: any, _model: { id: string; info: ModelInfo }): ApiStreamUsageChunk | undefined {
if (!usage) {
return undefined
}
const inputDetails = usage.input_tokens_details ?? usage.prompt_tokens_details
const hasCachedTokens = typeof inputDetails?.cached_tokens === "number"
const hasCacheMissTokens = typeof inputDetails?.cache_miss_tokens === "number"
const cachedFromDetails = hasCachedTokens ? inputDetails.cached_tokens : 0
const missFromDetails = hasCacheMissTokens ? inputDetails.cache_miss_tokens : 0
let totalInputTokens = usage.input_tokens ?? usage.prompt_tokens ?? 0
if (totalInputTokens === 0 && inputDetails && (cachedFromDetails > 0 || missFromDetails > 0)) {
totalInputTokens = cachedFromDetails + missFromDetails
}
const totalOutputTokens = usage.output_tokens ?? usage.completion_tokens ?? 0
const cacheWriteTokens = usage.cache_creation_input_tokens ?? usage.cache_write_tokens ?? 0
const cacheReadTokens =
usage.cache_read_input_tokens ?? usage.cache_read_tokens ?? usage.cached_tokens ?? cachedFromDetails ?? 0
const reasoningTokens =
typeof usage.output_tokens_details?.reasoning_tokens === "number"
? usage.output_tokens_details.reasoning_tokens
: undefined
// Subscription-based: no per-token costs
const out: ApiStreamUsageChunk = {
type: "usage",
inputTokens: totalInputTokens,
outputTokens: totalOutputTokens,
cacheWriteTokens,
cacheReadTokens,
...(typeof reasoningTokens === "number" ? { reasoningTokens } : {}),
totalCost: 0, // Subscription-based pricing
}
return out
}
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
const model = this.getModel()
// Reset state for this request
this.pendingToolCallId = undefined
this.pendingToolCallName = undefined
// Get access token from OAuth manager
let accessToken = await openAiCodexOAuthManager.getAccessToken()
if (!accessToken) {
throw new Error("Not authenticated with OpenAI Codex. Please sign in using the OpenAI Codex OAuth flow in settings.")
}
const useWebsocketMode = this.useWebsocketMode(model.info.apiFormat)
const { input, previousResponseId } = convertToOpenAIResponsesInput(messages, { usePreviousResponseId: useWebsocketMode })
const usePreviousResponseId = useWebsocketMode && !!previousResponseId
// Build request body
const requestBody = this.buildRequestBody(model, input, systemPrompt, tools, previousResponseId)
const fallbackRequestBody = this.buildRequestBody(model, input, systemPrompt, tools)
// Make the request with retry on auth failure
for (let attempt = 0; attempt < 2; attempt++) {
try {
yield* this.executeRequest(requestBody, fallbackRequestBody, model, accessToken, usePreviousResponseId)
return
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
const isAuthFailure = /unauthorized|invalid token|not authenticated|authentication|401/i.test(message)
if (attempt === 0 && isAuthFailure) {
// Force refresh the token for retry
const refreshed = await openAiCodexOAuthManager.forceRefreshAccessToken()
if (!refreshed) {
throw new Error(
"Not authenticated with OpenAI Codex. Please sign in using the OpenAI Codex OAuth flow in settings.",
)
}
accessToken = refreshed
continue
}
throw error
}
}
}
private useWebsocketMode(apiFormat?: ApiFormat): boolean {
if (featureFlagsService.getBooleanFlagEnabled(FeatureFlag.OPENAI_RESPONSES_WEBSOCKET_MODE)) {
return apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE
}
return false
}
private buildRequestBody(
model: { id: string; info: ModelInfo },
formattedInput: any,
systemPrompt: string,
tools?: ChatCompletionTool[],
previousResponseId?: string,
): any {
// Determine reasoning effort
const reasoningEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
const includeReasoning = reasoningEffort !== "none"
const body: any = {
model: model.id,
input: formattedInput,
stream: true,
store: false,
instructions: systemPrompt,
...(previousResponseId ? { previous_response_id: previousResponseId } : {}),
...(includeReasoning ? { include: ["reasoning.encrypted_content"] } : {}),
...(includeReasoning
? {
reasoning: {
effort: reasoningEffort,
summary: "auto",
},
}
: {}),
}
// Add tools if provided
// Pass through strict value from tool (MCP/custom tools have strict: false, built-in tools default to true)
if (tools && tools.length > 0) {
body.tools = tools
.filter((tool: any) => tool?.type === "function")
.map((tool: any) => ({
type: "function",
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters,
strict: tool.function.strict ?? true,
}))
}
return body
}
private async *executeRequest(
requestBody: any,
fallbackRequestBody: any,
model: { id: string; info: ModelInfo },
accessToken: string,
useWebsocketMode: boolean,
): ApiStream {
// Create AbortController for cancellation
this.abortController = new AbortController()
try {
// Get ChatGPT account ID for organization subscriptions
const accountId = await openAiCodexOAuthManager.getAccountId()
// Build Codex-specific headers
const codexHeaders: Record<string, string> = {
originator: "cline",
session_id: this.sessionId,
"User-Agent": `cline/${process.env.npm_package_version || "1.0.0"} (${os.platform()} ${os.release()}; ${os.arch()}) node/${process.version.slice(1)}`,
...(accountId ? { "ChatGPT-Account-Id": accountId } : {}),
...buildExternalBasicHeaders(),
}
if (useWebsocketMode) {
try {
yield* this.createResponseStreamWebsocket(requestBody, fallbackRequestBody, accessToken, codexHeaders, model)
return
} catch (error) {
Logger.error("OpenAI Codex websocket mode failed, falling back to HTTP Responses API:", error)
this.closeResponsesWebsocket()
}
}
// Try using OpenAI SDK first
try {
const client =
this.client ??
new OpenAI({
apiKey: accessToken,
baseURL: CODEX_API_BASE_URL,
defaultHeaders: codexHeaders,
fetch, // Use shared fetch for proxy support
})
const stream = (await (client as any).responses.create(requestBody, {
signal: this.abortController.signal,
headers: codexHeaders,
})) as AsyncIterable<any>
if (typeof (stream as any)?.[Symbol.asyncIterator] !== "function") {
throw new Error("OpenAI SDK did not return an AsyncIterable")
}
for await (const event of stream) {
if (this.abortController.signal.aborted) {
break
}
for await (const outChunk of this.processEvent(event, model)) {
yield outChunk
}
}
} catch (_sdkErr) {
// Fallback to manual SSE via fetch
yield* this.makeCodexRequest(requestBody, model, accessToken)
}
} finally {
this.abortController = undefined
}
}
private async *createResponseStreamWebsocket(
primaryParams: OpenAI.Responses.ResponseCreateParamsStreaming,
fallbackParams: OpenAI.Responses.ResponseCreateParamsStreaming,
accessToken: string,
codexHeaders: Record<string, string>,
model: { id: string; info: ModelInfo },
): ApiStream {
try {
for await (const event of this.createResponseEventsViaWebsocket(primaryParams, accessToken, codexHeaders)) {
if (this.abortController?.signal.aborted) {
return
}
yield* this.processEvent(event, model)
}
} catch (error) {
if (this.shouldRetryWebsocketWithFullContext(error, !!primaryParams.previous_response_id)) {
Logger.log(
"Retrying Codex websocket response with full context after previous_response_not_found or socket reset",
)
this.closeResponsesWebsocket()
for await (const event of this.createResponseEventsViaWebsocket(fallbackParams, accessToken, codexHeaders)) {
if (this.abortController?.signal.aborted) {
return
}
yield* this.processEvent(event, model)
}
return
}
throw error
}
}
private shouldRetryWebsocketWithFullContext(error: unknown, hadPreviousResponseId: boolean): boolean {
const errorCode =
typeof error === "object" && error && "code" in error && typeof (error as { code: unknown }).code === "string"
? (error as { code: string }).code
: undefined
if (hadPreviousResponseId && errorCode === "previous_response_not_found") {
return true
}
if (errorCode === "websocket_closed" || errorCode === "websocket_error") {
return true
}
return false
}
private async ensureResponsesWebsocket(accessToken: string, codexHeaders: Record<string, string>): Promise<UndiciWebSocket> {
if (this.responsesWs && this.responsesWs.readyState === UndiciWebSocket.OPEN) {
return this.responsesWs
}
this.closeResponsesWebsocket()
const ws = new UndiciWebSocket(CODEX_RESPONSES_WEBSOCKET_URL, {
headers: {
Authorization: `Bearer ${accessToken}`,
"OpenAI-Beta": "responses_websockets=2026-02-06",
...codexHeaders,
},
})
await new Promise<void>((resolve, reject) => {
const cleanup = () => {
ws.removeEventListener("open", handleOpen)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
}
const handleOpen = () => {
cleanup()
resolve()
}
const handleError = () => {
cleanup()
reject(new Error("Failed to open Codex Responses websocket"))
}
const handleClose = () => {
cleanup()
reject(new Error("Codex Responses websocket closed before opening"))
}
ws.addEventListener("open", handleOpen)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
})
this.responsesWs = ws
return ws
}
private closeResponsesWebsocket() {
if (this.responsesWs) {
try {
this.responsesWs.close()
} catch {}
this.responsesWs = undefined
}
}
private async *createResponseEventsViaWebsocket(
params: OpenAI.Responses.ResponseCreateParamsStreaming,
accessToken: string,
codexHeaders: Record<string, string>,
): AsyncGenerator<OpenAI.Responses.ResponseStreamEvent> {
if (this.websocketRequestInFlight) {
const error: Error & { code?: string } = new Error("Websocket response.create is already in progress")
error.code = "websocket_concurrency_limit"
throw error
}
const ws = await this.ensureResponsesWebsocket(accessToken, codexHeaders)
this.websocketRequestInFlight = true
const eventQueue: OpenAI.Responses.ResponseStreamEvent[] = []
let resolver: (() => void) | undefined
let completed = false
let failure: (Error & { code?: string }) | undefined
const wake = () => {
const next = resolver
resolver = undefined
next?.()
}
const handleMessage = (evt: UndiciMessageEvent) => {
try {
let raw = ""
if (typeof evt.data === "string") {
raw = evt.data
} else if (evt.data instanceof ArrayBuffer) {
raw = new TextDecoder().decode(new Uint8Array(evt.data))
} else if (ArrayBuffer.isView(evt.data)) {
raw = new TextDecoder().decode(new Uint8Array(evt.data.buffer, evt.data.byteOffset, evt.data.byteLength))
} else {
raw = String(evt.data)
}
const parsed = JSON.parse(raw)
if (parsed?.type === "error" && parsed?.error) {
const error: Error & { code?: string } = new Error(parsed.error.message || "Codex Responses websocket error")
error.code = parsed.error.code
failure = error
completed = true
wake()
return
}
eventQueue.push(parsed as OpenAI.Responses.ResponseStreamEvent)
if (parsed?.type === "response.completed" || parsed?.type === "response.failed") {
completed = true
}
wake()
} catch (error) {
const parseError: Error & { code?: string } = new Error(
`Failed to parse websocket event: ${error instanceof Error ? error.message : String(error)}`,
)
parseError.code = "websocket_parse_error"
failure = parseError
completed = true
wake()
}
}
const handleError = () => {
const error: Error & { code?: string } = new Error("Codex Responses websocket emitted an error event")
error.code = "websocket_error"
failure = error
completed = true
wake()
}
const handleClose = () => {
if (!completed) {
const error: Error & { code?: string } = new Error("Codex Responses websocket closed during response stream")
error.code = "websocket_closed"
failure = error
completed = true
wake()
}
}
ws.addEventListener("message", handleMessage)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
try {
ws.send(
JSON.stringify({
type: "response.create",
...params,
}),
)
while (!completed || eventQueue.length > 0) {
if (eventQueue.length === 0) {
await new Promise<void>((resolve) => {
resolver = resolve
})
continue
}
const event = eventQueue.shift()
if (event) {
yield event
}
}
if (failure) {
throw failure
}
} finally {
ws.removeEventListener("message", handleMessage)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
this.websocketRequestInFlight = false
}
}
private async *makeCodexRequest(requestBody: any, model: { id: string; info: ModelInfo }, accessToken: string): ApiStream {
const url = `${CODEX_API_BASE_URL}/responses`
// Get ChatGPT account ID for organization subscriptions
const accountId = await openAiCodexOAuthManager.getAccountId()
// Build headers with required Codex-specific fields
const headers: Record<string, string> = {
"Content-Type": "application/json",
Authorization: `Bearer ${accessToken}`,
originator: "cline",
session_id: this.sessionId,
"User-Agent": `cline/${process.env.npm_package_version || "1.0.0"} (${os.platform()} ${os.release()}; ${os.arch()}) node/${process.version.slice(1)}`,
}
// Add ChatGPT-Account-Id if available
if (accountId) {
headers["ChatGPT-Account-Id"] = accountId
}
try {
const response = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify(requestBody),
signal: this.abortController?.signal,
})
if (!response.ok) {
const errorText = await response.text()
let errorMessage = `Codex API request failed: ${response.status}`
try {
const errorJson = JSON.parse(errorText)
if (errorJson.error?.message) {
errorMessage = errorJson.error.message
} else if (errorJson.message) {
errorMessage = errorJson.message
}
} catch {
if (errorText) {
errorMessage += ` - ${errorText}`
}
}
throw new Error(errorMessage)
}
if (!response.body) {
throw new Error("No response body from Codex API")
}
yield* this.handleStreamResponse(response.body, model)
} catch (error) {
if (error instanceof Error) {
throw new Error(`Codex API error: ${error.message}`)
}
throw new Error("Unexpected error connecting to Codex API")
}
}
private async *handleStreamResponse(body: ReadableStream<Uint8Array>, model: { id: string; info: ModelInfo }): ApiStream {
const reader = body.getReader()
const decoder = new TextDecoder()
let buffer = ""
try {
while (true) {
if (this.abortController?.signal.aborted) {
break
}
const { done, value } = await reader.read()
if (done) {
break
}
buffer += decoder.decode(value, { stream: true })
const lines = buffer.split("\n")
buffer = lines.pop() || ""
for (const line of lines) {
if (line.startsWith("data: ")) {
const data = line.slice(6).trim()
if (data === "[DONE]") {
continue
}
try {
const parsed = JSON.parse(data)
for await (const outChunk of this.processEvent(parsed, model)) {
yield outChunk
}
} catch (e) {
if (!(e instanceof SyntaxError)) {
throw e
}
}
}
}
}
} finally {
reader.releaseLock()
}
}
private async *processEvent(event: any, model: { id: string; info: ModelInfo }): ApiStream {
// Handle text deltas
if (event?.type === "response.text.delta" || event?.type === "response.output_text.delta") {
if (event?.delta) {
yield { type: "text", text: event.delta }
}
return
}
// Handle reasoning deltas
if (
event?.type === "response.reasoning.delta" ||
event?.type === "response.reasoning_text.delta" ||
event?.type === "response.reasoning_summary.delta" ||
event?.type === "response.reasoning_summary_text.delta"
) {
if (event?.delta) {
yield { type: "reasoning", reasoning: event.delta }
}
return
}
// Handle refusal deltas
if (event?.type === "response.refusal.delta") {
if (event?.delta) {
yield { type: "text", text: `[Refusal] ${event.delta}` }
}
return
}
// Handle tool/function call deltas
if (event?.type === "response.tool_call_arguments.delta" || event?.type === "response.function_call_arguments.delta") {
const callId = event.call_id || event.tool_call_id || event.id || this.pendingToolCallId
const name = event.name || event.function_name || this.pendingToolCallName
const args = event.delta || event.arguments
if (typeof callId === "string" && callId.length > 0 && typeof name === "string" && name.length > 0) {
yield {
type: "tool_calls",
tool_call: {
call_id: callId,
function: {
id: callId,
name,
arguments: typeof args === "string" ? args : "",
},
},
}
}
return
}
// Handle output item events
if (event?.type === "response.output_item.added" || event?.type === "response.output_item.done") {
const item = event?.item
if (item) {
// Capture tool identity for subsequent argument deltas
if (item.type === "function_call" || item.type === "tool_call") {
const callId = item.call_id || item.tool_call_id || item.id
const name = item.name || item.function?.name || item.function_name
if (typeof callId === "string" && callId.length > 0) {
this.pendingToolCallId = callId
this.pendingToolCallName = typeof name === "string" ? name : undefined
}
}
if (item.type === "text" && item.text) {
yield { type: "text", text: item.text }
} else if (item.type === "reasoning" && item.text) {
yield { type: "reasoning", reasoning: item.text }
} else if (item.type === "message" && Array.isArray(item.content)) {
for (const content of item.content) {
if ((content?.type === "text" || content?.type === "output_text") && content?.text) {
yield { type: "text", text: content.text }
}
}
} else if (
(item.type === "function_call" || item.type === "tool_call") &&
event.type === "response.output_item.done"
) {
const callId = item.call_id || item.tool_call_id || item.id
if (callId) {
const args = item.arguments || item.function?.arguments || item.function_arguments
yield {
type: "tool_calls",
id: callId,
tool_call: {
call_id: callId,
function: {
id: callId,
name: item.name || item.function?.name || item.function_name || "",
arguments: typeof args === "string" ? args : "{}",
},
},
}
}
}
}
return
}
// Handle completion events
if (event?.type === "response.done" || event?.type === "response.completed") {
const usage = event?.response?.usage || event?.usage || undefined
const usageData = this.normalizeUsage(usage, model)
if (usageData) {
yield usageData
}
return
}
// Fallbacks for legacy formats
if (event?.choices?.[0]?.delta?.content) {
yield { type: "text", text: event.choices[0].delta.content }
return
}
if (event?.usage) {
const usageData = this.normalizeUsage(event.usage, model)
if (usageData) {
yield usageData
}
}
}
abort(): void {
this.closeResponsesWebsocket()
this.abortController?.abort()
}
getModel(): { id: OpenAiCodexModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
const id = modelId && modelId in openAiCodexModels ? (modelId as OpenAiCodexModelId) : openAiCodexDefaultModelId
const info: ModelInfo = openAiCodexModels[id]
return { id, info }
}
}
@@ -1,702 +0,0 @@
import {
ModelInfo,
OpenAiCompatibleModelInfo,
OpenAiNativeModelId,
openAiNativeDefaultModelId,
openAiNativeModels,
} from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import type {
ChatCompletionFunctionTool,
ChatCompletionReasoningEffort,
ChatCompletionTool,
} from "openai/resources/chat/completions"
import { MessageEvent as UndiciMessageEvent, WebSocket as UndiciWebSocket } from "undici"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { featureFlagsService } from "@/services/feature-flags"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiFormat } from "@/shared/proto/cline/models"
import { FeatureFlag } from "@/shared/services/feature-flags/feature-flags"
import { Logger } from "@/shared/services/Logger"
import { isGPT5ModelFamily } from "@/utils/model-utils"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface OpenAiNativeHandlerOptions extends CommonApiHandlerOptions {
openAiNativeApiKey?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
apiModelId?: string
openAiNativeUseResponsesWebsocket?: boolean
}
export class OpenAiNativeHandler implements ApiHandler {
private options: OpenAiNativeHandlerOptions
private client: OpenAI | undefined
private responsesWs: UndiciWebSocket | undefined
private responsesWsReadyPromise: Promise<UndiciWebSocket> | undefined
private websocketRequestInFlight = false
private abortController?: AbortController
constructor(options: OpenAiNativeHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.openAiNativeApiKey) {
throw new Error("OpenAI API key is required")
}
try {
this.client = createOpenAIClient({
apiKey: this.options.openAiNativeApiKey,
})
} catch (error) {
throw new Error(`Error creating OpenAI client: ${error instanceof Error ? error.message : String(error)}`)
}
}
return this.client
}
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
const inputTokens = usage?.prompt_tokens || 0 // sum of cache hits and misses
const outputTokens = usage?.completion_tokens || 0
const cacheReadTokens = usage?.prompt_tokens_details?.cached_tokens || 0
const cacheWriteTokens = 0
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
totalCost: totalCost,
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
// Responses API requires tool format to be set to OPENAI_RESPONSES with native tools calling enabled
const apiFormat = this.getModel()?.info?.apiFormat
if (apiFormat === ApiFormat.OPENAI_RESPONSES || apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE) {
if (!tools?.length) {
throw new Error("Native Tool Call must be enabled in your setting for OpenAI Responses API")
}
yield* this.createResponseStream(systemPrompt, messages, tools)
} else {
yield* this.createCompletionStream(systemPrompt, messages, tools)
}
}
private async *createCompletionStream(
systemPrompt: string,
messages: ClineStorageMessage[],
tools?: ChatCompletionTool[],
): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const toolCallProcessor = new ToolCallProcessor()
this.abortController = new AbortController()
// Handle o1 models separately as they don't support streaming
if (model.info.supportsStreaming === false) {
const response = await client.chat.completions.create(
{
model: model.id,
messages: [{ role: "user", content: systemPrompt }, ...convertToOpenAiMessages(messages, "openai-native")],
},
{ signal: this.abortController?.signal },
)
yield {
type: "text",
text: response.choices[0]?.message.content || "",
}
yield* this.yieldUsage(model.info, response.usage)
return
}
const systemRole = model.info.systemRole ?? "system"
const includeReasoning = model.info.supportsReasoningEffort
const includeTools = model.info.supportsTools ?? true
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
const reasoningEffort =
includeReasoning && requestedEffort !== "none" ? (requestedEffort as ChatCompletionReasoningEffort) : undefined
const stream = await client.chat.completions.create({
model: model.id,
messages: [{ role: systemRole, content: systemPrompt }, ...convertToOpenAiMessages(messages, "openai-native")],
stream: true,
stream_options: { include_usage: true },
reasoning_effort: reasoningEffort,
...(model.info.temperature !== undefined ? { temperature: model.info.temperature } : {}),
...(includeTools ? getOpenAIToolParams(tools, isGPT5ModelFamily(model.id)) : {}),
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
try {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
} catch (error) {
Logger.error("Error processing tool call delta:", error, delta.tool_calls)
}
}
if (chunk.usage) {
// Only last chunk contains usage
yield* this.yieldUsage(model.info, chunk.usage)
}
}
}
private async *createResponseStream(
systemPrompt: string,
messages: ClineStorageMessage[],
tools: ChatCompletionTool[],
): ApiStream {
const model = this.getModel()
const usePreviousResponseId = this.useWebsocketMode(model.info.apiFormat)
// Warm websocket connection early in websocket mode so the first response.create avoids handshake latency.
if (usePreviousResponseId) {
this.preconnectResponsesWebsocket()
}
const { input, previousResponseId } = convertToOpenAIResponsesInput(messages, { usePreviousResponseId })
const responseTools = this.mapResponseTools(tools)
this.abortController = new AbortController()
const params = this.buildResponseCreateParams({
modelId: model.id,
systemPrompt,
input,
previousResponseId,
tools: responseTools,
})
const fallbackParams = this.buildResponseCreateParams({
modelId: model.id,
systemPrompt,
input,
tools: responseTools,
})
if (usePreviousResponseId && previousResponseId) {
try {
yield* this.createResponseStreamWebsocket(model.info, params, fallbackParams)
return
} catch (error) {
Logger.error("OpenAI websocket mode failed, falling back to HTTP Responses API:", error)
this.closeResponsesWebsocket()
}
}
yield* this.createResponseStreamHttp(model.info, params)
}
private preconnectResponsesWebsocket(): void {
void this.ensureResponsesWebsocket().catch((error) => {
Logger.debug("OpenAI websocket preconnect failed:", error)
this.closeResponsesWebsocket()
})
}
private useWebsocketMode(apiFormat?: ApiFormat): boolean {
if (featureFlagsService.getBooleanFlagEnabled(FeatureFlag.OPENAI_RESPONSES_WEBSOCKET_MODE)) {
return apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE
}
return false
}
private mapResponseTools(tools: ChatCompletionTool[]): OpenAI.Responses.Tool[] {
return tools
?.filter((tool): tool is ChatCompletionFunctionTool => tool?.type === "function")
.map((tool) => ({
type: "function" as const,
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters ?? null,
strict: tool.function.strict ?? true,
}))
}
private buildResponseCreateParams(args: {
modelId: string
systemPrompt: string
input: OpenAI.Responses.ResponseInput
tools: OpenAI.Responses.Tool[]
previousResponseId?: string
}): OpenAI.Responses.ResponseCreateParamsStreaming {
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
const reasoning: { effort: ChatCompletionReasoningEffort; summary: "auto" } | undefined =
requestedEffort === "none"
? undefined
: {
effort: requestedEffort,
summary: "auto",
}
return {
model: args.modelId,
instructions: args.systemPrompt,
input: args.input,
stream: true,
tools: args.tools,
store: !args.previousResponseId, // Do not use store when websocket mode is enabled.
...(args.previousResponseId ? { previous_response_id: args.previousResponseId } : {}),
...(reasoning ? { reasoning } : {}),
}
}
private async *createResponseStreamHttp(
modelInfo: ModelInfo,
params: OpenAI.Responses.ResponseCreateParamsStreaming,
): ApiStream {
const client = this.ensureClient()
Logger.debug(`OpenAI Responses Input (HTTP): ${JSON.stringify(params.input)}`)
const stream = await client.responses.create(params, { signal: this.abortController?.signal })
yield* this.processResponsesEvents(stream, modelInfo)
}
private async *createResponseStreamWebsocket(
modelInfo: ModelInfo,
primaryParams: OpenAI.Responses.ResponseCreateParamsStreaming,
fallbackParams: OpenAI.Responses.ResponseCreateParamsStreaming,
): ApiStream {
Logger.debug(`OpenAI Responses Input (WebSocket): ${JSON.stringify(primaryParams.input)}`)
try {
yield* this.processResponsesEvents(this.createResponseEventsViaWebsocket(primaryParams), modelInfo)
} catch (error) {
if (this.shouldRetryWebsocketWithFullContext(error, !!primaryParams.previous_response_id)) {
Logger.log("Retrying websocket response with full context after previous_response_not_found or socket reset")
this.closeResponsesWebsocket()
yield* this.processResponsesEvents(this.createResponseEventsViaWebsocket(fallbackParams), modelInfo)
return
}
throw error
}
}
private shouldRetryWebsocketWithFullContext(error: unknown, hadPreviousResponseId: boolean): boolean {
const errorCode =
typeof error === "object" && error && "code" in error && typeof (error as { code: unknown }).code === "string"
? (error as { code: string }).code
: undefined
if (hadPreviousResponseId && errorCode === "previous_response_not_found") {
return true
}
if (errorCode === "websocket_closed" || errorCode === "websocket_error") {
return true
}
return false
}
private async ensureResponsesWebsocket(): Promise<UndiciWebSocket> {
if (this.responsesWs && this.responsesWs.readyState === UndiciWebSocket.OPEN) {
return this.responsesWs
}
if (this.responsesWsReadyPromise) {
return this.responsesWsReadyPromise
}
this.closeResponsesWebsocket()
if (!this.options.openAiNativeApiKey) {
throw new Error("OpenAI API key is required")
}
const ws = new UndiciWebSocket("wss://api.openai.com/v1/responses", {
headers: {
Authorization: `Bearer ${this.options.openAiNativeApiKey}`,
"OpenAI-Beta": "responses_websockets=2026-02-06",
...buildExternalBasicHeaders(),
},
})
this.responsesWs = ws
const readyPromise = new Promise<UndiciWebSocket>((resolve, reject) => {
const cleanup = () => {
ws.removeEventListener("open", handleOpen)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
}
const handleOpen = () => {
cleanup()
resolve(ws)
}
const handleError = () => {
cleanup()
reject(new Error("Failed to open Responses websocket"))
}
const handleClose = () => {
cleanup()
reject(new Error("Responses websocket closed before opening"))
}
ws.addEventListener("open", handleOpen)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
})
this.responsesWsReadyPromise = readyPromise
try {
return await readyPromise
} catch (error) {
if (this.responsesWs === ws) {
this.responsesWs = undefined
}
throw error
} finally {
if (this.responsesWsReadyPromise === readyPromise) {
this.responsesWsReadyPromise = undefined
}
}
}
private closeResponsesWebsocket() {
this.responsesWsReadyPromise = undefined
if (this.responsesWs) {
try {
this.responsesWs.close()
} catch {}
this.responsesWs = undefined
}
}
private async *createResponseEventsViaWebsocket(
params: OpenAI.Responses.ResponseCreateParamsStreaming,
): AsyncGenerator<OpenAI.Responses.ResponseStreamEvent> {
if (this.websocketRequestInFlight) {
const error: Error & { code?: string } = new Error("Websocket response.create is already in progress")
error.code = "websocket_concurrency_limit"
throw error
}
const ws = await this.ensureResponsesWebsocket()
this.websocketRequestInFlight = true
const eventQueue: OpenAI.Responses.ResponseStreamEvent[] = []
let resolver: (() => void) | undefined
let completed = false
let failure: (Error & { code?: string }) | undefined
const wake = () => {
const next = resolver
resolver = undefined
next?.()
}
const handleMessage = (evt: UndiciMessageEvent) => {
try {
let raw = ""
if (typeof evt.data === "string") {
raw = evt.data
} else if (evt.data instanceof ArrayBuffer) {
raw = new TextDecoder().decode(new Uint8Array(evt.data))
} else if (ArrayBuffer.isView(evt.data)) {
raw = new TextDecoder().decode(new Uint8Array(evt.data.buffer, evt.data.byteOffset, evt.data.byteLength))
} else {
raw = String(evt.data)
}
const parsed = JSON.parse(raw)
if (parsed?.type === "error" && parsed?.error) {
const error: Error & { code?: string } = new Error(parsed.error.message || "Responses websocket error")
error.code = parsed.error.code
failure = error
completed = true
wake()
return
}
eventQueue.push(parsed as OpenAI.Responses.ResponseStreamEvent)
if (parsed?.type === "response.completed" || parsed?.type === "response.failed") {
completed = true
}
wake()
} catch (error) {
const parseError: Error & { code?: string } = new Error(
`Failed to parse websocket event: ${error instanceof Error ? error.message : String(error)}`,
)
parseError.code = "websocket_parse_error"
failure = parseError
completed = true
wake()
}
}
const handleError = () => {
const error: Error & { code?: string } = new Error("Responses websocket emitted an error event")
error.code = "websocket_error"
failure = error
completed = true
wake()
}
const handleClose = () => {
if (!completed) {
const error: Error & { code?: string } = new Error("Responses websocket closed during response stream")
error.code = "websocket_closed"
failure = error
completed = true
wake()
}
}
ws.addEventListener("message", handleMessage)
ws.addEventListener("error", handleError)
ws.addEventListener("close", handleClose)
try {
ws.send(
JSON.stringify({
type: "response.create",
...params,
}),
)
while (!completed || eventQueue.length > 0) {
if (eventQueue.length === 0) {
await new Promise<void>((resolve) => {
resolver = resolve
})
continue
}
const event = eventQueue.shift()
if (event) {
yield event
}
}
if (failure) {
throw failure
}
} finally {
ws.removeEventListener("message", handleMessage)
ws.removeEventListener("error", handleError)
ws.removeEventListener("close", handleClose)
this.websocketRequestInFlight = false
}
}
private async *processResponsesEvents(
stream: AsyncIterable<OpenAI.Responses.ResponseStreamEvent>,
modelInfo: ModelInfo,
): ApiStream {
const functionCallByItemId = new Map<string, { call_id?: string; name?: string; id?: string }>()
for await (const chunk of stream) {
Logger.debug(`OpenAI Responses Chunk: ${JSON.stringify(chunk)}`)
if (chunk.type === "response.output_item.added") {
const item = chunk.item
if (item.type === "function_call" && item.id) {
functionCallByItemId.set(item.id, { call_id: item.call_id, name: item.name, id: item.id })
yield {
type: "tool_calls",
id: item.id,
tool_call: {
call_id: item.call_id,
function: {
id: item.id,
name: item.name,
arguments: item.arguments,
},
},
}
}
if (item.type === "reasoning" && item.encrypted_content && item.id) {
yield {
type: "reasoning",
id: item.id,
reasoning: "",
redacted_data: item.encrypted_content,
}
}
}
if (chunk.type === "response.output_item.done") {
const item = chunk.item
if (item.type === "function_call") {
if (item.id) {
functionCallByItemId.set(item.id, { call_id: item.call_id, name: item.name, id: item.id })
}
yield {
type: "tool_calls",
id: item.id || item.call_id,
tool_call: {
call_id: item.call_id,
function: {
id: item.id,
name: item.name,
arguments: item.arguments,
},
},
}
}
if (item.type === "reasoning") {
yield {
type: "reasoning",
id: item.id,
details: item.summary,
reasoning: "",
}
}
}
if (chunk.type === "response.reasoning_summary_part.added") {
yield {
type: "reasoning",
id: chunk.item_id,
reasoning: chunk.part.text,
}
}
if (chunk.type === "response.reasoning_summary_text.delta") {
yield {
type: "reasoning",
id: chunk.item_id,
reasoning: chunk.delta,
}
}
if (chunk.type === "response.reasoning_summary_part.done") {
yield {
type: "reasoning",
id: chunk.item_id,
details: chunk.part,
reasoning: "",
}
}
if (chunk.type === "response.output_text.delta") {
if (chunk.delta) {
yield {
id: chunk.item_id,
type: "text",
text: chunk.delta,
}
}
}
if (chunk.type === "response.reasoning_text.delta") {
if (chunk.delta) {
yield {
id: chunk.item_id,
type: "reasoning",
reasoning: chunk.delta,
}
}
}
if (chunk.type === "response.function_call_arguments.delta") {
const pendingCall = functionCallByItemId.get(chunk.item_id)
const callId = pendingCall?.call_id
const functionName = pendingCall?.name
const functionId = pendingCall?.id || chunk.item_id
yield {
type: "tool_calls",
tool_call: {
call_id: callId,
function: {
id: functionId,
name: functionName,
arguments: chunk.delta,
},
},
}
}
if (chunk.type === "response.function_call_arguments.done") {
if (chunk.item_id && chunk.name && chunk.arguments) {
const pendingCall = functionCallByItemId.get(chunk.item_id)
const callId = pendingCall?.call_id
const functionId = pendingCall?.id || chunk.item_id
yield {
type: "tool_calls",
tool_call: {
call_id: callId,
function: {
id: functionId,
name: chunk.name,
arguments: chunk.arguments,
},
},
}
}
}
if (
chunk.type === "response.incomplete" &&
chunk.response?.status === "incomplete" &&
chunk.response?.incomplete_details?.reason === "max_output_tokens"
) {
if (chunk.response?.output_text?.length > 0) {
Logger.log("Partial output:", chunk.response.output_text)
} else {
Logger.log("Ran out of tokens during reasoning")
}
}
if (chunk.type === "response.completed" && chunk.response?.usage) {
const usage = chunk.response.usage
const inputTokens = usage.input_tokens || 0
const outputTokens = usage.output_tokens || 0
const cacheReadTokens = usage.input_tokens_details?.cached_tokens || 0
const cacheWriteTokens = 0
const reasoningTokens = usage.output_tokens_details?.reasoning_tokens || 0
const totalTokens = usage.total_tokens || 0
Logger.log(`Total tokens from Responses API usage: ${totalTokens}`)
const totalCost = calculateApiCostOpenAI(
modelInfo,
inputTokens,
outputTokens + reasoningTokens,
cacheWriteTokens,
cacheReadTokens,
)
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
yield {
type: "usage",
inputTokens: nonCachedInputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
thoughtsTokenCount: reasoningTokens,
totalCost: totalCost,
id: chunk.response.id,
}
}
}
}
abort(): void {
this.closeResponsesWebsocket()
this.abortController?.abort()
this.abortController = undefined
}
getModel(): { id: OpenAiNativeModelId; info: OpenAiCompatibleModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in openAiNativeModels) {
const id = modelId as OpenAiNativeModelId
const info = openAiNativeModels[id]
return { id, info: { ...info } }
}
return {
id: openAiNativeDefaultModelId,
info: { ...openAiNativeModels[openAiNativeDefaultModelId] },
}
}
}
@@ -1,188 +0,0 @@
import { DefaultAzureCredential, getBearerTokenProvider } from "@azure/identity"
import { azureOpenAiDefaultApiVersion, ModelInfo, OpenAiCompatibleModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
import OpenAI, { AzureOpenAI } from "openai"
import type { ChatCompletionReasoningEffort, ChatCompletionTool } from "openai/resources/chat/completions"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient, fetch } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface OpenAiHandlerOptions extends CommonApiHandlerOptions {
openAiApiKey?: string
openAiBaseUrl?: string
azureApiVersion?: string
azureIdentity?: boolean
openAiHeaders?: Record<string, string>
openAiModelId?: string
openAiModelInfo?: OpenAiCompatibleModelInfo
reasoningEffort?: string
}
export class OpenAiHandler implements ApiHandler {
private options: OpenAiHandlerOptions
private client: OpenAI | undefined
constructor(options: OpenAiHandlerOptions) {
this.options = options
}
private getAzureAudienceScope(baseUrl?: string): string {
const url = baseUrl?.toLowerCase() ?? ""
if (url.includes("azure.us")) return "https://cognitiveservices.azure.us/.default"
if (url.includes("azure.com")) return "https://cognitiveservices.azure.com/.default"
return "https://cognitiveservices.azure.com/.default"
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.openAiApiKey && !this.options.azureIdentity) {
throw new Error("OpenAI API key or Azure Identity Authentication is required")
}
try {
const baseUrl = this.options.openAiBaseUrl?.toLowerCase() ?? ""
const isAzureDomain = baseUrl.includes("azure.com") || baseUrl.includes("azure.us")
const externalHeaders = buildExternalBasicHeaders()
// Azure API shape slightly differs from the core API shape...
if (
this.options.azureApiVersion ||
(isAzureDomain && !this.options.openAiModelId?.toLowerCase().includes("deepseek"))
) {
if (this.options.azureIdentity) {
this.client = new AzureOpenAI({
baseURL: this.options.openAiBaseUrl,
azureADTokenProvider: getBearerTokenProvider(
new DefaultAzureCredential(),
this.getAzureAudienceScope(this.options.openAiBaseUrl),
),
apiVersion: this.options.azureApiVersion || azureOpenAiDefaultApiVersion,
defaultHeaders: {
...externalHeaders,
...this.options.openAiHeaders,
},
fetch,
})
} else {
this.client = new AzureOpenAI({
baseURL: this.options.openAiBaseUrl,
apiKey: this.options.openAiApiKey,
apiVersion: this.options.azureApiVersion || azureOpenAiDefaultApiVersion,
defaultHeaders: {
...externalHeaders,
...this.options.openAiHeaders,
},
fetch,
})
}
} else {
this.client = createOpenAIClient({
baseURL: this.options.openAiBaseUrl,
apiKey: this.options.openAiApiKey,
defaultHeaders: this.options.openAiHeaders,
})
}
} catch (error: any) {
throw new Error(`Error creating OpenAI client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.openAiModelId ?? ""
const isDeepseekReasoner = modelId.includes("deepseek-reasoner")
const isR1FormatRequired = this.options.openAiModelInfo?.isR1FormatRequired ?? false
const isReasoningModelFamily =
["o1", "o3", "o4", "gpt-5"].some((prefix) => modelId.includes(prefix)) && !modelId.includes("chat")
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
let temperature: number | undefined
if (this.options.openAiModelInfo?.temperature !== undefined) {
const tempValue = Number(this.options.openAiModelInfo.temperature)
temperature = tempValue === 0 ? undefined : tempValue
} else {
temperature = openAiModelInfoSaneDefaults.temperature
}
let reasoningEffort: ChatCompletionReasoningEffort | undefined
let maxTokens: number | undefined
if (this.options.openAiModelInfo?.maxTokens && this.options.openAiModelInfo.maxTokens > 0) {
maxTokens = Number(this.options.openAiModelInfo.maxTokens)
} else {
maxTokens = undefined
}
if (isDeepseekReasoner || isR1FormatRequired) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
if (isReasoningModelFamily) {
openAiMessages = [{ role: "developer", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
temperature = undefined // does not support temperature
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
reasoningEffort = requestedEffort === "none" ? undefined : (requestedEffort as ChatCompletionReasoningEffort)
}
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
temperature,
max_tokens: maxTokens,
reasoning_effort: reasoningEffort,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.openAiModelId ?? "",
info: this.options.openAiModelInfo ?? openAiModelInfoSaneDefaults,
}
}
}
@@ -1,235 +0,0 @@
import { setTimeout as setTimeoutPromise } from "node:timers/promises"
import { StateManager } from "@core/storage/StateManager"
import { ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import axios from "axios"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient, getAxiosSettings } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { createOpenRouterStream } from "../transform/openrouter-stream"
import { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
import { OpenRouterErrorResponse } from "./types"
interface OpenRouterHandlerOptions extends CommonApiHandlerOptions {
openRouterApiKey?: string
openRouterModelId?: string
openRouterModelInfo?: ModelInfo
openRouterProviderSorting?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
enableParallelToolCalling?: boolean
}
export class OpenRouterHandler implements ApiHandler {
private options: OpenRouterHandlerOptions
private client: OpenAI | undefined
lastGenerationId?: string
constructor(options: OpenRouterHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.openRouterApiKey) {
throw new Error("OpenRouter API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://openrouter.ai/api/v1",
apiKey: this.options.openRouterApiKey,
defaultHeaders: {
"HTTP-Referer": "https://cline.bot", // Optional, for including your app on openrouter.ai rankings.
"X-Title": "Cline", // Optional. Shows in rankings on openrouter.ai.
},
})
} catch (error: any) {
throw new Error(`Error creating OpenRouter client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
this.lastGenerationId = undefined
const stream = await createOpenRouterStream(
client,
systemPrompt,
messages,
this.getModel(),
this.options.reasoningEffort,
this.options.thinkingBudgetTokens,
this.options.openRouterProviderSorting,
tools,
this.options.enableParallelToolCalling,
)
let didOutputUsage = false
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
// openrouter returns an error object instead of the openai sdk throwing an error
// Check for error field directly on chunk
if ("error" in chunk) {
const error = chunk.error as OpenRouterErrorResponse["error"]
Logger.error(`OpenRouter API Error: ${error?.code} - ${error?.message}`)
// Include metadata in the error message if available
const metadataStr = error.metadata ? `\nMetadata: ${JSON.stringify(error.metadata, null, 2)}` : ""
throw new Error(`OpenRouter API Error ${error.code}: ${error.message}${metadataStr}`)
}
// Check for error in choices[0].finish_reason
// OpenRouter may return errors in a non-standard way within choices
const choice = chunk.choices?.[0]
// Use type assertion since OpenRouter uses non-standard "error" finish_reason
if ((choice?.finish_reason as string) === "error") {
// Use type assertion since OpenRouter adds non-standard error property
const choiceWithError = choice as any
if (choiceWithError.error) {
const error = choiceWithError.error
Logger.error(
`OpenRouter Mid-Stream Error: ${error?.code || "Unknown"} - ${error?.message || "Unknown error"}`,
)
// Format error details
const errorDetails = typeof error === "object" ? JSON.stringify(error, null, 2) : String(error)
throw new Error(`OpenRouter Mid-Stream Error: ${errorDetails}`)
}
// Fallback if error details are not available
throw new Error(`OpenRouter Mid-Stream Error: Stream terminated with error status but no error details provided`)
}
if (!this.lastGenerationId && chunk.id) {
this.lastGenerationId = chunk.id
}
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Reasoning tokens are returned separately from the content
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
if (
delta &&
"reasoning" in delta &&
delta.reasoning &&
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
// OpenRouter passes reasoning details that we can pass back unmodified in api requests to preserve reasoning traces for model
// See: https://openrouter.ai/docs/use-cases/reasoning-tokens#preserving-reasoning-blocks
if (
delta &&
"reasoning_details" in delta &&
delta.reasoning_details &&
// @ts-expect-error-next-line
delta.reasoning_details.length && // exists and non-0
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: "",
details: delta.reasoning_details,
}
}
if (!didOutputUsage && chunk.usage) {
// @ts-expect-error-next-line -- OpenRouter returns cache_write_tokens for Anthropic models
const cacheWriteTokens = chunk.usage.prompt_tokens_details?.cache_write_tokens || 0
yield {
type: "usage",
cacheWriteTokens,
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
inputTokens:
(chunk.usage.prompt_tokens || 0) -
(chunk.usage.prompt_tokens_details?.cached_tokens || 0) -
(cacheWriteTokens || 0),
outputTokens: chunk.usage.completion_tokens || 0,
// @ts-expect-error-next-line
totalCost: (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0),
}
didOutputUsage = true
}
}
// Fallback to generation endpoint if usage chunk not returned
if (!didOutputUsage) {
const apiStreamUsage = await this.getApiStreamUsage()
if (apiStreamUsage) {
yield apiStreamUsage
}
}
}
async getApiStreamUsage(): Promise<ApiStreamUsageChunk | undefined> {
if (this.lastGenerationId) {
await setTimeoutPromise(500) // FIXME: necessary delay to ensure generation endpoint is ready
try {
const generationIterator = this.fetchGenerationDetails(this.lastGenerationId)
const generation = (await generationIterator.next()).value
// Logger.log("OpenRouter generation details:", generation)
return {
type: "usage",
cacheWriteTokens: generation?.native_tokens_cache_write || 0,
cacheReadTokens: generation?.native_tokens_cached || 0,
// openrouter generation endpoint fails often
inputTokens: (generation?.native_tokens_prompt || 0) - (generation?.native_tokens_cached || 0),
outputTokens: generation?.native_tokens_completion || 0,
totalCost: generation?.total_cost || 0,
}
} catch (error) {
// ignore if fails
Logger.error("Error fetching OpenRouter generation details:", error)
}
}
return undefined
}
@withRetry({ maxRetries: 4, baseDelay: 250, maxDelay: 1000, retryAllErrors: true })
async *fetchGenerationDetails(genId: string) {
// Logger.log("Fetching generation details for:", genId)
try {
const response = await axios.get(`https://openrouter.ai/api/v1/generation?id=${genId}`, {
headers: {
Authorization: `Bearer ${this.options.openRouterApiKey}`,
},
timeout: 15_000, // this request hangs sometimes
...getAxiosSettings(),
})
yield response.data?.data
} catch (error) {
// ignore if fails
Logger.error("Error fetching OpenRouter generation details:", error)
throw error
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.openRouterModelId || openRouterDefaultModelId
const cachedModelInfo = StateManager.get().getModelInfo("openRouter", modelId)
return {
id: modelId,
info: cachedModelInfo || openRouterDefaultModelInfo,
}
}
}
@@ -1,286 +0,0 @@
import { promises as fs } from "node:fs"
import { ModelInfo, QwenCodeModelId, qwenCodeDefaultModelId, qwenCodeModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import * as os from "os"
import * as path from "path"
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
import { ClineStorageMessage } from "@/shared/messages/content"
import { fetch } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
// --- Constants for Qwen OAuth2 ---
const QWEN_OAUTH_BASE_URL = "https://chat.qwen.ai"
const QWEN_OAUTH_TOKEN_ENDPOINT = `${QWEN_OAUTH_BASE_URL}/api/v1/oauth2/token`
const QWEN_OAUTH_CLIENT_ID = "f0304373b74a44d2b584a3fb70ca9e56"
const QWEN_DIR = ".qwen"
const QWEN_CREDENTIAL_FILENAME = "oauth_creds.json"
interface QwenOAuthCredentials {
access_token: string
refresh_token: string
token_type: string
expiry_date: number
resource_url?: string
}
interface QwenCodeHandlerOptions extends CommonApiHandlerOptions {
qwenCodeOauthPath?: string
apiModelId?: string
}
function getQwenCachedCredentialPath(customPath?: string): string {
if (customPath) {
// Support custom path that starts with ~/ or is absolute
if (customPath.startsWith("~/")) {
return path.join(os.homedir(), customPath.slice(2))
}
return path.resolve(customPath)
}
return path.join(os.homedir(), QWEN_DIR, QWEN_CREDENTIAL_FILENAME)
}
function objectToUrlEncoded(data: Record<string, string>): string {
return Object.keys(data)
.map((key) => `${encodeURIComponent(key)}=${encodeURIComponent(data[key])}`)
.join("&")
}
export class QwenCodeHandler implements ApiHandler {
private options: QwenCodeHandlerOptions
private credentials: QwenOAuthCredentials | null = null
private client: OpenAI | undefined
constructor(options: QwenCodeHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
// Create the client instance with dummy key initially
// The API key will be updated dynamically via ensureAuthenticated
const externalHeaders = buildExternalBasicHeaders()
this.client = new OpenAI({
apiKey: "dummy-key-will-be-replaced",
baseURL: "https://dashscope.aliyuncs.com/compatible-mode/v1",
defaultHeaders: externalHeaders,
fetch,
})
}
return this.client
}
private async loadCachedQwenCredentials(): Promise<QwenOAuthCredentials> {
try {
const keyFile = getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)
const credsStr = await fs.readFile(keyFile, "utf-8")
return JSON.parse(credsStr)
} catch (error) {
Logger.error(
`Error reading or parsing credentials file at ${getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)}`,
)
throw new Error(`Failed to load Qwen OAuth credentials: ${error}`)
}
}
private async refreshAccessToken(credentials: QwenOAuthCredentials): Promise<QwenOAuthCredentials> {
if (!credentials.refresh_token) {
throw new Error("No refresh token available in credentials.")
}
const bodyData = {
grant_type: "refresh_token",
refresh_token: credentials.refresh_token,
client_id: QWEN_OAUTH_CLIENT_ID,
}
const response = await fetch(QWEN_OAUTH_TOKEN_ENDPOINT, {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded",
Accept: "application/json",
},
body: objectToUrlEncoded(bodyData),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Token refresh failed: ${response.status} ${response.statusText}. Response: ${errorText}`)
}
const tokenData = await response.json()
if (tokenData.error) {
throw new Error(`Token refresh failed: ${tokenData.error} - ${tokenData.error_description}`)
}
const newCredentials = {
...credentials,
access_token: tokenData.access_token,
token_type: tokenData.token_type,
refresh_token: tokenData.refresh_token || credentials.refresh_token,
expiry_date: Date.now() + tokenData.expires_in * 1000,
}
const filePath = getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)
await fs.writeFile(filePath, JSON.stringify(newCredentials, null, 2))
return newCredentials
}
private isTokenValid(credentials: QwenOAuthCredentials): boolean {
const TOKEN_REFRESH_BUFFER_MS = 30 * 1000 // 30s buffer
if (!credentials.expiry_date) {
return false
}
return Date.now() < credentials.expiry_date - TOKEN_REFRESH_BUFFER_MS
}
private async ensureAuthenticated(): Promise<void> {
if (!this.credentials) {
this.credentials = await this.loadCachedQwenCredentials()
}
if (!this.isTokenValid(this.credentials)) {
this.credentials = await this.refreshAccessToken(this.credentials)
}
// After authentication, update the apiKey and baseURL on the existing client
const client = this.ensureClient()
client.apiKey = this.credentials.access_token
client.baseURL = this.getBaseUrl(this.credentials)
}
private getBaseUrl(creds: QwenOAuthCredentials): string {
let baseUrl = creds.resource_url || "https://dashscope.aliyuncs.com/compatible-mode/v1"
if (!baseUrl.startsWith("http://") && !baseUrl.startsWith("https://")) {
baseUrl = `https://${baseUrl}`
}
return baseUrl.endsWith("/v1") ? baseUrl : `${baseUrl}/v1`
}
private async callApiWithRetry<T>(apiCall: () => Promise<T>): Promise<T> {
try {
return await apiCall()
} catch (error: any) {
if (error.status === 401) {
// Token expired, refresh and retry
this.credentials = await this.refreshAccessToken(this.credentials!)
const client = this.ensureClient()
client.apiKey = this.credentials.access_token
client.baseURL = this.getBaseUrl(this.credentials)
return await apiCall()
} else {
throw error
}
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
await this.ensureAuthenticated()
const client = this.ensureClient()
const model = this.getModel()
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam = {
role: "system",
content: systemPrompt,
}
const convertedMessages = [systemMessage, ...convertToOpenAiMessages(messages)]
const requestOptions: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming = {
model: model.id,
temperature: 0,
messages: convertedMessages,
stream: true,
stream_options: { include_usage: true },
max_completion_tokens: model.info.maxTokens,
...getOpenAIToolParams(tools),
}
const stream = await this.callApiWithRetry(() => client.chat.completions.create(requestOptions))
const toolCallProcessor = new ToolCallProcessor()
let fullContent = ""
for await (const apiChunk of stream) {
const delta = apiChunk.choices[0]?.delta ?? {}
if (delta.content) {
let newText = delta.content
if (newText.startsWith(fullContent)) {
newText = newText.substring(fullContent.length)
}
fullContent = delta.content
if (newText) {
// Check for thinking blocks
if (newText.includes("<think>") || newText.includes("</think>")) {
// Simple parsing for thinking blocks
const parts = newText.split(/<\/?think>/g)
for (let i = 0; i < parts.length; i++) {
if (parts[i]) {
if (i % 2 === 0) {
// Outside thinking block
yield {
type: "text",
text: parts[i],
}
} else {
// Inside thinking block
yield {
type: "reasoning",
reasoning: parts[i],
}
}
}
}
} else {
yield {
type: "text",
text: newText,
}
}
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Handle reasoning content (o1-style)
if ("reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (apiChunk.usage) {
yield {
type: "usage",
inputTokens: apiChunk.usage.prompt_tokens || 0,
outputTokens: apiChunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: QwenCodeModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in qwenCodeModels) {
const id = modelId as QwenCodeModelId
return { id, info: qwenCodeModels[id] }
}
return {
id: qwenCodeDefaultModelId,
info: qwenCodeModels[qwenCodeDefaultModelId],
}
}
}
-163
View File
@@ -1,163 +0,0 @@
import {
InternationalQwenModelId,
internationalQwenDefaultModelId,
internationalQwenModels,
MainlandQwenModelId,
ModelInfo,
mainlandQwenDefaultModelId,
mainlandQwenModels,
QwenApiRegions,
} from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface QwenHandlerOptions extends CommonApiHandlerOptions {
qwenApiKey?: string
qwenApiLine?: QwenApiRegions
apiModelId?: string
thinkingBudgetTokens?: number
}
export class QwenHandler implements ApiHandler {
private options: QwenHandlerOptions
private client: OpenAI | undefined
constructor(options: QwenHandlerOptions) {
// Ensure options start with defaults but allow overrides
this.options = {
qwenApiLine: QwenApiRegions.CHINA,
...options,
}
}
private useChinaApi(): boolean {
return this.options.qwenApiLine === QwenApiRegions.CHINA
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.qwenApiKey) {
throw new Error("Alibaba API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: this.useChinaApi()
? "https://dashscope.aliyuncs.com/compatible-mode/v1"
: "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
apiKey: this.options.qwenApiKey,
})
} catch (error: any) {
throw new Error(`Error creating Alibaba client: ${error.message}`)
}
}
return this.client
}
getModel(): { id: MainlandQwenModelId | InternationalQwenModelId; info: ModelInfo } {
const modelId = this.options.apiModelId
// Branch based on API line to let poor typescript know what to do
if (this.useChinaApi()) {
const id = modelId && modelId in mainlandQwenModels ? (modelId as MainlandQwenModelId) : mainlandQwenDefaultModelId
return {
id,
info: mainlandQwenModels[id],
}
}
const id =
modelId && modelId in internationalQwenModels
? (modelId as InternationalQwenModelId)
: internationalQwenDefaultModelId
return {
id,
info: internationalQwenModels[id],
}
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const isDeepseekReasoner = model.id.includes("deepseek-r1")
const isReasoningModelFamily = model.id.includes("qwen3") || ["qwen-plus-latest", "qwen-turbo-latest"].includes(model.id)
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
let temperature: number | undefined = 0
// Configuration for extended thinking
const budgetTokens = this.options.thinkingBudgetTokens || 0
const reasoningOn = budgetTokens !== 0
const thinkingArgs = isReasoningModelFamily
? {
enable_thinking: reasoningOn,
thinking_budget: reasoningOn ? budgetTokens : undefined,
}
: undefined
if (isDeepseekReasoner || (reasoningOn && isReasoningModelFamily)) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
temperature = undefined
}
const stream = await client.chat.completions.create({
model: model.id,
max_completion_tokens: model.info.maxTokens,
messages: openAiMessages,
stream: true,
stream_options: { include_usage: true },
temperature,
...thinkingArgs,
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
try {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
} catch (error) {
Logger.error("Error processing tool call delta:", error, delta.tool_calls)
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
// @ts-expect-error-next-line
cacheReadTokens: chunk.usage.prompt_cache_hit_tokens || 0,
// @ts-expect-error-next-line
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
}
}
}
}
}
@@ -1,162 +0,0 @@
import { ModelInfo, requestyDefaultModelId, requestyDefaultModelInfo } from "@shared/api"
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
import { calculateApiCostOpenAI } from "@utils/cost"
import OpenAI from "openai"
import { toRequestyServiceStringUrl } from "@/shared/clients/requesty"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { ApiStream } from "../transform/stream"
interface RequestyHandlerOptions extends CommonApiHandlerOptions {
requestyBaseUrl?: string
requestyApiKey?: string
reasoningEffort?: string
thinkingBudgetTokens?: number
requestyModelId?: string
requestyModelInfo?: ModelInfo
}
// Requesty usage includes an extra field for Anthropic use cases.
// Safely cast the prompt token details section to the appropriate structure.
interface RequestyUsage extends OpenAI.CompletionUsage {
prompt_tokens_details?: {
caching_tokens?: number
cached_tokens?: number
}
total_cost?: number
}
export class RequestyHandler implements ApiHandler {
private options: RequestyHandlerOptions
private client: OpenAI | undefined
constructor(options: RequestyHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.requestyApiKey) {
throw new Error("Requesty API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: toRequestyServiceStringUrl(this.options.requestyBaseUrl),
apiKey: this.options.requestyApiKey,
defaultHeaders: {
"HTTP-Referer": "https://cline.bot",
"X-Title": "Cline",
},
})
} catch (error: any) {
throw new Error(`Error creating Requesty client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const reasoningEffort = this.options.reasoningEffort || "medium"
const reasoning = { reasoning_effort: reasoningEffort }
const reasoningArgs = model.id.startsWith("openai/o") ? reasoning : {}
const thinkingBudget = this.options.thinkingBudgetTokens || 0
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(model.id)
const adaptiveThinking = isAdaptiveThinkingModel
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, thinkingBudget)
: undefined
const thinking =
thinkingBudget > 0
? { thinking: { type: "enabled", budget_tokens: thinkingBudget } }
: { thinking: { type: "disabled" } }
const supportsLegacyClaudeThinking =
!isAdaptiveThinkingModel &&
(model.id.includes("claude-3-7-sonnet") ||
model.id.includes("claude-4.6-sonnet") ||
model.id.includes("claude-sonnet-4") ||
model.id.includes("claude-opus-4"))
const thinkingArgs = isAdaptiveThinkingModel
? adaptiveThinking?.enabled
? {
thinking: { type: "adaptive" },
...(adaptiveThinking.effort ? { output_config: { effort: adaptiveThinking.effort } } : {}),
}
: {}
: supportsLegacyClaudeThinking
? thinking
: {}
const stream = await client.chat.completions.create({
model: model.id,
max_tokens: model.info.maxTokens || undefined,
messages: openAiMessages,
...(isAdaptiveThinkingModel ? {} : { temperature: 0 }),
stream: true,
stream_options: { include_usage: true },
...reasoningArgs,
...thinkingArgs,
})
let lastUsage: any
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
lastUsage = chunk.usage
}
}
if (lastUsage) {
const usage = lastUsage as RequestyUsage
const inputTokens = usage.prompt_tokens || 0
const outputTokens = usage.completion_tokens || 0
const cacheWriteTokens = usage.prompt_tokens_details?.caching_tokens || undefined
const cacheReadTokens = usage.prompt_tokens_details?.cached_tokens || undefined
const totalCost = calculateApiCostOpenAI(model.info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
yield {
type: "usage",
inputTokens: inputTokens,
outputTokens: outputTokens,
cacheWriteTokens: cacheWriteTokens,
cacheReadTokens: cacheReadTokens,
totalCost: totalCost,
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.requestyModelId
const modelInfo = this.options.requestyModelInfo
if (modelId && modelInfo) {
return { id: modelId, info: modelInfo }
}
return { id: requestyDefaultModelId, info: requestyDefaultModelInfo }
}
}
@@ -1,103 +0,0 @@
import { ModelInfo, SambanovaModelId, sambanovaDefaultModelId, sambanovaModels } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface SambanovaHandlerOptions extends CommonApiHandlerOptions {
sambanovaApiKey?: string
apiModelId?: string
}
export class SambanovaHandler implements ApiHandler {
private options: SambanovaHandlerOptions
private client: OpenAI | undefined
constructor(options: SambanovaHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.sambanovaApiKey) {
throw new Error("SambaNova API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.sambanova.ai/v1",
apiKey: this.options.sambanovaApiKey,
})
} catch (error: any) {
throw new Error(`Error creating SambaNova client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const model = this.getModel()
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
const modelId = model.id.toLowerCase()
if (modelId.includes("deepseek") || modelId.includes("qwen3")) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
const toolCallProcessor = new ToolCallProcessor()
const stream = await client.chat.completions.create({
model: this.getModel().id,
messages: openAiMessages,
temperature: model.info.temperature ?? 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.apiModelId
if (modelId && modelId in sambanovaModels) {
const id = modelId as SambanovaModelId
return { id, info: sambanovaModels[id] }
}
return {
id: sambanovaDefaultModelId,
info: sambanovaModels[sambanovaDefaultModelId],
}
}
}
File diff suppressed because it is too large Load Diff
@@ -1,103 +0,0 @@
import { ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { convertToOpenAiMessages } from "../transform/openai-format"
import { convertToR1Format } from "../transform/r1-format"
import { ApiStream } from "../transform/stream"
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
interface TogetherHandlerOptions extends CommonApiHandlerOptions {
togetherApiKey?: string
togetherModelId?: string
}
export class TogetherHandler implements ApiHandler {
private options: TogetherHandlerOptions
private client: OpenAI | undefined
constructor(options: TogetherHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.togetherApiKey) {
throw new Error("Together API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://api.together.xyz/v1",
apiKey: this.options.togetherApiKey,
})
} catch (error: any) {
throw new Error(`Error creating Together client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.options.togetherModelId ?? ""
const isDeepseekReasoner = modelId.includes("deepseek-reasoner")
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: systemPrompt },
...convertToOpenAiMessages(messages),
]
if (isDeepseekReasoner) {
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
}
const stream = await client.chat.completions.create({
model: modelId,
messages: openAiMessages,
temperature: 0,
stream: true,
stream_options: { include_usage: true },
...getOpenAIToolParams(tools),
})
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
yield {
type: "reasoning",
reasoning: (delta.reasoning_content as string | undefined) || "",
}
}
if (chunk.usage) {
yield {
type: "usage",
inputTokens: chunk.usage.prompt_tokens || 0,
outputTokens: chunk.usage.completion_tokens || 0,
}
}
}
}
getModel(): { id: string; info: ModelInfo } {
return {
id: this.options.togetherModelId ?? "",
info: openAiModelInfoSaneDefaults,
}
}
}
@@ -1,29 +0,0 @@
// For the following openrouter error type sources, see the docs here:
// https://openrouter.ai/docs/api-reference/errors
export interface LanguageModelChatSelector {
vendor?: string
family?: string
version?: string
id?: string
}
export type OpenRouterErrorResponse = {
error: {
message: string
code: number
metadata?: OpenRouterProviderErrorMetadata | OpenRouterModerationErrorMetadata | Record<string, unknown>
}
}
export type OpenRouterProviderErrorMetadata = {
provider_name: string // The name of the provider that encountered the error
raw: unknown // The raw error from the provider
}
export type OpenRouterModerationErrorMetadata = {
reasons: string[] // Why your input was flagged
flagged_input: string // The text segment that was flagged, limited to 100 characters. If the flagged input is longer than 100 characters, it will be truncated in the middle and replaced with ...
provider_name: string // The name of the provider that requested moderation
model_slug: string
}
@@ -1,164 +0,0 @@
import { ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
import { shouldSkipReasoningForModel } from "@utils/model-utils"
import OpenAI from "openai"
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
import { ClineStorageMessage } from "@/shared/messages/content"
import { createOpenAIClient } from "@/shared/net"
import { Logger } from "@/shared/services/Logger"
import { ApiHandler, CommonApiHandlerOptions } from "../index"
import { withRetry } from "../retry"
import { ApiStream } from "../transform/stream"
import { ToolCallProcessor } from "../transform/tool-call-processor"
import { createVercelAIGatewayStream } from "../transform/vercel-ai-gateway-stream"
interface VercelAIGatewayHandlerOptions extends CommonApiHandlerOptions {
vercelAiGatewayApiKey?: string
openRouterModelId?: string
openRouterModelInfo?: ModelInfo
reasoningEffort?: string
thinkingBudgetTokens?: number
}
function getCacheReadTokens(usage: any): number {
return usage?.prompt_tokens_details?.cached_tokens || usage?.cache_read_input_tokens || 0
}
function getCacheWriteTokens(usage: any): number {
return usage?.prompt_tokens_details?.cache_write_tokens || usage?.cache_creation_input_tokens || 0
}
export class VercelAIGatewayHandler implements ApiHandler {
private options: VercelAIGatewayHandlerOptions
private client: OpenAI | undefined
constructor(options: VercelAIGatewayHandlerOptions) {
this.options = options
}
private ensureClient(): OpenAI {
if (!this.client) {
if (!this.options.vercelAiGatewayApiKey) {
throw new Error("Vercel AI Gateway API key is required")
}
try {
this.client = createOpenAIClient({
baseURL: "https://ai-gateway.vercel.sh/v1",
apiKey: this.options.vercelAiGatewayApiKey,
defaultHeaders: {
"http-referer": "https://cline.bot",
"x-title": "Cline",
},
})
} catch (error: any) {
throw new Error(`Error creating Vercel AI Gateway client: ${error.message}`)
}
}
return this.client
}
@withRetry()
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
const client = this.ensureClient()
const modelId = this.getModel().id
const modelInfo = this.getModel().info
try {
const stream = await createVercelAIGatewayStream(
client,
systemPrompt,
messages,
{ id: modelId, info: modelInfo },
this.options.reasoningEffort,
this.options.thinkingBudgetTokens,
tools,
)
let didOutputUsage = false
const toolCallProcessor = new ToolCallProcessor()
for await (const chunk of stream) {
const delta = chunk.choices?.[0]?.delta
if (delta?.content) {
yield {
type: "text",
text: delta.content,
}
}
if (delta?.tool_calls) {
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
}
// Reasoning tokens are returned separately from the content
// Skip reasoning content for models that don't support it (e.g., devstral, grok-4)
if (
delta &&
"reasoning" in delta &&
delta.reasoning &&
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
}
}
// Reasoning details that can be passed back in API requests to preserve reasoning traces
if (
delta &&
"reasoning_details" in delta &&
delta.reasoning_details &&
// @ts-expect-error-next-line
delta.reasoning_details.length && // exists and non-0
!shouldSkipReasoningForModel(this.options.openRouterModelId)
) {
yield {
type: "reasoning",
reasoning: "",
details: delta.reasoning_details,
}
}
if (!didOutputUsage && chunk.usage) {
// @ts-expect-error - Vercel AI Gateway extends OpenAI types
const totalCost = (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0)
const cacheReadTokens = getCacheReadTokens(chunk.usage)
const cacheWriteTokens = getCacheWriteTokens(chunk.usage)
yield {
type: "usage",
cacheWriteTokens,
cacheReadTokens,
inputTokens: Math.max(0, (chunk.usage.prompt_tokens || 0) - cacheReadTokens - cacheWriteTokens),
outputTokens: chunk.usage.completion_tokens || 0,
totalCost,
}
didOutputUsage = true
}
}
if (!didOutputUsage) {
Logger.warn("Vercel AI Gateway did not provide usage information in stream")
}
} catch (error: any) {
Logger.error("Vercel AI Gateway error details:", error)
Logger.error("Error stack:", error.stack)
throw new Error(`Vercel AI Gateway error: ${error.message}`)
}
}
getModel(): { id: string; info: ModelInfo } {
const modelId = this.options.openRouterModelId
const modelInfo = this.options.openRouterModelInfo
if (modelId && modelInfo) {
return { id: modelId, info: modelInfo }
}
// If we have a model ID but no model info, preserve the selected model ID
// and fall back only the metadata to defaults.
if (modelId) {
return { id: modelId, info: openRouterDefaultModelInfo }
}
return { id: openRouterDefaultModelId, info: openRouterDefaultModelInfo }
}
}

Some files were not shown because too many files have changed in this diff Show More