mirror of
https://github.com/cline/cline.git
synced 2026-09-05 05:02:27 +08:00
Compare commits
58 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 3d3a257b08 | |||
| 293d1cee0c | |||
| 5dcfa9cf79 | |||
| 1e3972bcf0 | |||
| 5a5e23b79f | |||
| 70b5f4bc6f | |||
| 15bf76beb3 | |||
| 07a8cb28b8 | |||
| 08e32a26f6 | |||
| 44257db17e | |||
| d93481d511 | |||
| 3f2976b290 | |||
| 877ba07f0d | |||
| d21433e7ca | |||
| 8e956fa3b9 | |||
| 3fdf8fc135 | |||
| 991e33f385 | |||
| 2bd7bbd44e | |||
| db823a5b96 | |||
| c17dec92ea | |||
| a37ab9366d | |||
| c7bbbda086 | |||
| 9e777c0f4b | |||
| 57df42db8e | |||
| 1f86e4bc37 | |||
| af3d81cf99 | |||
| 068688d162 | |||
| 6f8522e9c4 | |||
| e67f31a684 | |||
| 6777756a8e | |||
| 0f30864f8a | |||
| 367446c5d1 | |||
| d2c5e739fb | |||
| 87a0048968 | |||
| 7d7708b1c9 | |||
| 05d07e2bd7 | |||
| 69fa94804c | |||
| 04623ebe04 | |||
| b8849c49cd | |||
| 87bf1bf727 | |||
| bd55f2d328 | |||
| bbdd9d34a8 | |||
| 3a9c97b322 | |||
| e17ba43260 | |||
| e5f35422a4 | |||
| ed3401bfcc | |||
| 5dfc32daf8 | |||
| a039cded0f | |||
| fee494cd17 | |||
| 97ccdbbcd2 | |||
| 28c69b7d9f | |||
| f75044c635 | |||
| 3b31317881 | |||
| f5b3c4fe4a | |||
| 5f29ab5953 | |||
| 51f0bf7b9f | |||
| 7fc6dd3479 | |||
| 10d8dbb884 |
@@ -0,0 +1,121 @@
|
||||
# Debug Harness
|
||||
|
||||
HTTP-controlled debugger for the VSCode extension at `src/dev/debug-harness/server.ts`.
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# Build extension first if needed (protos + esbuild):
|
||||
npm run protos && IS_DEV=true node esbuild.mjs
|
||||
|
||||
# Launch (skip-build if already built):
|
||||
npx tsx src/dev/debug-harness/server.ts --skip-build --auto-launch
|
||||
|
||||
# In another terminal:
|
||||
curl localhost:19229/api -d '{"method":"status"}'
|
||||
```
|
||||
|
||||
## Data Isolation
|
||||
|
||||
The debugee runs with `CLINE_DIR=~/.cline2` by default, separate from your real `~/.cline`.
|
||||
This prevents the debugee's logout from logging out the debugger, and vice versa.
|
||||
Override with `--cline-dir /tmp/test-dir`. Check with `status()` → `clineDir`.
|
||||
|
||||
## Browser Capture & OAuth
|
||||
|
||||
The debugee runs with `CLINE_CAPTURE_BROWSER=1`, which intercepts `openExternal()` in
|
||||
`src/utils/env.ts`. URLs are captured instead of opening a real browser:
|
||||
|
||||
- Logged to `$CLINE_DIR/data/debug-captured-urls.jsonl`
|
||||
- POSTed in real-time to `/captured-url` on the harness server
|
||||
- Queryable via `oauth.captured_urls`
|
||||
|
||||
### OAuth API
|
||||
|
||||
- **`oauth.captured_urls`** `{clear?}` — URLs the debugee tried to open
|
||||
- **`oauth.read_stored_token`** — Check auth token presence in secrets.json
|
||||
- **`oauth.simulate_callback`** `{path, code?, state?, provider?, token?}` — Build vscode:// callback URI
|
||||
- **`oauth.read_captured_urls_file`** — Read on-disk JSONL of captured URLs
|
||||
|
||||
### OAuth testing flow
|
||||
|
||||
For **Cline OAuth** (SDK local callback): The SDK starts a local HTTP server, the auth URL
|
||||
is captured. To complete: open the captured URL in a real browser (it redirects back to the
|
||||
SDK's callback server), OR extract the callback port and `curl http://127.0.0.1:PORT/callback?code=...`.
|
||||
|
||||
For **MCP/Provider OAuth** (vscode:// URI): The redirect goes to a vscode:// URI. Use
|
||||
`oauth.simulate_callback` to build it, then inject via `ext.evaluate` calling the URI handler.
|
||||
|
||||
## Navigating Views — Use Commands, Not Clicks
|
||||
|
||||
Don't try to find/click small sidebar icons. Use VSCode commands via command palette.
|
||||
Registered in `src/registry.ts`:
|
||||
|
||||
| Command | View |
|
||||
|---------|------|
|
||||
| `cline.accountButtonClicked` | Account / sign-in |
|
||||
| `cline.historyButtonClicked` | Task history |
|
||||
| `cline.settingsButtonClicked` | Settings |
|
||||
| `cline.mcpButtonClicked` | MCP servers |
|
||||
| `cline.plusButtonClicked` | New task (chat) |
|
||||
| `cline.worktreesButtonClicked` | Worktrees |
|
||||
|
||||
```bash
|
||||
curl localhost:19229/api -d '{"method":"ui.command_palette","params":{"command":"cline.accountButtonClicked"}}'
|
||||
```
|
||||
|
||||
## Key commands
|
||||
|
||||
All via `POST localhost:19229/api` with `{"method":"...", "params":{...}}`:
|
||||
|
||||
- **`launch`** / **`shutdown`** — lifecycle
|
||||
- **`ui.screenshot`** — screenshot to `/tmp/cline-debug/`; returns `{path}` — **use `read_file` on the path to examine, do NOT `open` the file** (Preview.app covers the VSCode window)
|
||||
- **`ui.open_sidebar`** — open the Cline sidebar
|
||||
- **`ext.set_breakpoint`** `{file, line, condition?}` — breakpoint by source file (sourcemap-resolved)
|
||||
- **`ext.evaluate`** `{expression, callFrameId?}` — eval in extension host
|
||||
- **`ext.resume`** / **`ext.step_over`** / **`ext.step_into`** — stepping
|
||||
- **`ext.call_stack`** — inspect when paused
|
||||
- **`web.evaluate`** `{expression}` — eval in webview
|
||||
- **`web.post_message`** `{message}` — send postMessage to extension host via exposed vsCodeApi
|
||||
- **`wait_for_pause`** `{timeout?}` — block until breakpoint hit
|
||||
- **`ui.locator`** `{role?, testId?, text?, frame?}` — Playwright locator (auto-retries on stale sidebar frame)
|
||||
- **`ui.react_input`** `{text, selector?, clear?, submit?}` — set React textarea value via `execCommand('insertText')`; works reliably across multiple tasks
|
||||
- **`ui.send_message`** `{text, images?, files?, responseType?}` — send chat message bypassing the textarea entirely (via gRPC postMessage)
|
||||
- **`ui.command_palette`** `{command}` — run VSCode command
|
||||
|
||||
## Typical Session
|
||||
|
||||
```bash
|
||||
# 1. Launch
|
||||
curl localhost:19229/api -d '{"method":"launch","params":{"skipBuild":true}}'
|
||||
|
||||
# 2. Open sidebar + dismiss overlays (ALWAYS do this first)
|
||||
curl localhost:19229/api -d '{"method":"ui.open_sidebar"}'
|
||||
curl localhost:19229/api -d '{"method":"web.evaluate","params":{"expression":"document.querySelectorAll(\".sr-only\").forEach(el => el.parentElement?.click())"}}'
|
||||
|
||||
# 3. Navigate to view
|
||||
curl localhost:19229/api -d '{"method":"ui.command_palette","params":{"command":"cline.accountButtonClicked"}}'
|
||||
|
||||
# 4. Check captured OAuth URLs if testing auth
|
||||
curl localhost:19229/api -d '{"method":"oauth.captured_urls"}'
|
||||
|
||||
# 5. Verify
|
||||
curl localhost:19229/api -d '{"method":"ui.screenshot"}'
|
||||
```
|
||||
|
||||
## Caveats
|
||||
|
||||
- **⚠️ Dismiss promotional overlays FIRST**: On fresh launches, full-screen promo overlays block the sidebar. **Dismiss immediately after `ui.open_sidebar`**, before any other interaction or screenshot. May need to run twice:
|
||||
```bash
|
||||
curl localhost:19229/api -d '{"method": "ui.open_sidebar"}'
|
||||
curl localhost:19229/api -d '{"method": "web.evaluate", "params": {"expression": "document.querySelectorAll(\".sr-only\").forEach(el => el.parentElement?.click())"}}'
|
||||
```
|
||||
- **Screenshots — don't open the file**: `ui.screenshot` and `ui.sidebar_screenshot` save PNGs to `/tmp/cline-debug/` and return the `{path}`. Use `read_file` on that path to examine screenshots. Running `open <path>` launches Preview.app on macOS which covers the VSCode window.
|
||||
- **Scripts count = 0 after launch**: CDP connects after extension host starts, so scripts parsed during startup aren't tracked. Breakpoints still work via sourcemap resolution.
|
||||
- **Port 9230**: Extension host inspector. If another VSCode instance uses this port, the harness will fail to connect. Kill other debug instances first.
|
||||
- **macOS only** for now (Playwright Electron launch behavior).
|
||||
- **Webview CDP**: `connect_webview` may fail depending on Electron version. `web.evaluate` still works via Playwright's `frame.evaluate()` fallback.
|
||||
- **Sourcemap paths**: esbuild outputs relative paths like `../src/extension.ts` in the sourcemap. The resolver handles this, but if a file isn't found, use `ext.source_files` to see exact paths.
|
||||
- **OAuth with fake codes**: Browser capture intercepts the URL but doesn't provide a valid auth code. For real OAuth testing, open the captured URL in a browser. For unit testing, mock the token exchange.
|
||||
|
||||
See `src/dev/debug-harness/README.md` for full API reference.
|
||||
+57
-96
@@ -13,11 +13,55 @@ This file is the secret sauce for working effectively in this codebase. It captu
|
||||
**What NOT to add:** Stuff you can figure out from reading a few files, obvious patterns, or standard practices. This file should be high-signal, not comprehensive.
|
||||
|
||||
## Miscellaneous
|
||||
- Avoid provider-specific string matching / hardcoded provider branches when fixing provider/config plumbing. Prefer provider metadata, shared catalog/defaults, explicit protocol/client capabilities, or centralized normalization utilities that apply by data shape rather than `providerId === "..."`. If a provider exception seems necessary, stop and explain why instead of adding ad-hoc string matching.
|
||||
- This is a VS Code extension—check `package.json` for available scripts before trying to verify builds (e.g., `npm run compile`, not `npm run build`).
|
||||
- When creating PRs, contributors should not create changelog-entry files. Maintainers handle release versioning and changelog curation during the release process.
|
||||
- When adding new feature flags, see this PR as a reference https://github.com/cline/cline/pull/7566
|
||||
- Additional instructions about making requests: @.clinerules/network.md
|
||||
|
||||
## Searching the Codebase — Avoiding Build Output
|
||||
|
||||
Several directories contain build output or generated code that produces
|
||||
noisy or unusable results with `search_files` / `grep`:
|
||||
|
||||
| Directory | What it is | Why it's a problem |
|
||||
|-----------|-----------|-------------------|
|
||||
| `out/` | esbuild bundle output | Mirrors `src/` structure as minified JS — every search gets duplicate hits on single-line files |
|
||||
| `dist/` | Packaged extension | Entire extension bundled into one minified `extension.js` (~1 long line) |
|
||||
| `dist-standalone/` | Standalone build output | Same minification issue |
|
||||
| `src/generated/` | Generated protobuf code | Auto-generated from `proto/`; not the source of truth |
|
||||
| `src/shared/proto/` | Generated proto type defs | Auto-generated from `proto/`; not the source of truth |
|
||||
| `node_modules/` | Dependencies | Huge, not project source |
|
||||
|
||||
### How to skip build output
|
||||
|
||||
**`search_files`** — Point at `src/` (not the project root) and use `file_pattern`:
|
||||
```
|
||||
search_files(path="src/core", regex="myFunction", file_pattern="*.ts")
|
||||
```
|
||||
The `file_pattern` parameter is the most effective filter — e.g. `"*.ts"`,
|
||||
`"*.tsx"`, `"*.proto"`.
|
||||
|
||||
**`grep` directly** — Exclude build dirs and restrict to source extensions:
|
||||
```bash
|
||||
grep -rn "myFunction" src/ --include="*.ts" --exclude-dir={out,dist,node_modules,generated}
|
||||
```
|
||||
|
||||
### When you must search minified files
|
||||
|
||||
Sometimes you need to verify what got bundled (e.g., checking if a change
|
||||
made it into the build). Minified files are typically one long line, so
|
||||
normal `grep` shows the entire file as context. Use these approaches:
|
||||
|
||||
- **`grep -oP`** to extract just the match with limited surrounding context:
|
||||
```bash
|
||||
grep -oP '.{0,40}myFunction.{0,40}' dist/extension.js
|
||||
```
|
||||
- **`read_file`** on files in `out/src/` — these have source maps and are
|
||||
more readable than `dist/extension.js` (which is the fully bundled output).
|
||||
- **Source maps** — `out/src/*.js.map` and `dist/extension.js.map` can be
|
||||
used to trace minified output back to original source locations.
|
||||
|
||||
## gRPC/Protobuf Communication
|
||||
The extension and webview communicate via gRPC-like protocol over VS Code message passing.
|
||||
|
||||
@@ -48,102 +92,17 @@ The extension and webview communicate via gRPC-like protocol over VS Code messag
|
||||
- `src/core/controller/task/explainChanges.ts` - Handler implementation
|
||||
- `webview-ui/src/components/chat/ChatRow.tsx` - UI rendering
|
||||
|
||||
## Adding a New API Provider
|
||||
When adding a new provider (e.g., "openai-codex"), you must update the proto conversion layer in THREE places or the provider will silently reset to Anthropic:
|
||||
|
||||
1. `proto/cline/models.proto` - Add to the `ApiProvider` enum (e.g., `OPENAI_CODEX = 40;`)
|
||||
2. `convertApiProviderToProto()` in `src/shared/proto-conversions/models/api-configuration-conversion.ts` - Add case mapping string to proto enum
|
||||
3. `convertProtoToApiProvider()` in the same file - Add case mapping proto enum back to string
|
||||
|
||||
**Why this matters:** Without these, the provider string hits the `default` case and returns `ANTHROPIC`. The webview, provider list, and handler all work fine, but the state silently resets when it round-trips through proto serialization. No error is thrown.
|
||||
|
||||
**Other files to update when adding a provider:**
|
||||
- `src/shared/api.ts` - Add to `ApiProvider` union type, define models
|
||||
- `src/shared/providers/providers.json` - Add to provider list for dropdown
|
||||
- `src/core/api/index.ts` - Register handler in `createHandlerForProvider()`
|
||||
- `webview-ui/src/components/settings/utils/providerUtils.ts` - Add cases in `getModelsForProvider()` and `normalizeApiConfiguration()`
|
||||
- `webview-ui/src/utils/validate.ts` - Add validation case
|
||||
- `webview-ui/src/components/settings/ApiOptions.tsx` - Render provider component
|
||||
|
||||
## Responses API Providers (OpenAI Codex, OpenAI Native)
|
||||
Providers using OpenAI's Responses API require native tool calling. XML tools don't work with the Responses API.
|
||||
|
||||
**Symptoms of broken native tool calling:**
|
||||
- Tools get called multiple times (e.g., `ask_followup_question` asks the same question twice)
|
||||
- Tool arguments get duplicated or malformed
|
||||
- The model responds but tools aren't recognized
|
||||
|
||||
**Root causes to check:**
|
||||
1. **Provider missing from `isNextGenModelProvider()`** in `src/utils/model-utils.ts`. The native variant matchers (e.g., `native-gpt-5/config.ts`) call this function. If your provider isn't in the list, the matcher returns false and falls back to XML tools.
|
||||
|
||||
2. **Model missing `apiFormat: ApiFormat.OPENAI_RESPONSES`** in its model info (`src/shared/api.ts`). This property signals that the model requires native tool calling. The task runner in `src/core/task/index.ts` checks this and forces `enableNativeToolCalls: true` regardless of user settings.
|
||||
|
||||
**When adding a new Responses API provider:**
|
||||
1. Add provider to `isNextGenModelProvider()` list in `src/utils/model-utils.ts`
|
||||
2. Set `apiFormat: ApiFormat.OPENAI_RESPONSES` on all models that use the Responses API
|
||||
3. The variant matcher and task runner will handle the rest automatically
|
||||
|
||||
## Adding Tools to System Prompt
|
||||
This is tricky—multiple prompt variants and configs. **Always search for existing similar tools first and follow their pattern.** Look at the full chain from prompt definition → variant configs → handler → UI before implementing.
|
||||
|
||||
1. **Add to `ClineDefaultTool` enum** in `src/shared/tools.ts`
|
||||
2. **Tool definition** in `src/core/prompts/system-prompt/tools/` (create file like `generate_explanation.ts`)
|
||||
- Define variants for each `ModelFamily` (generic, next-gen, xs, etc.)
|
||||
- Export variants array (e.g., `export const my_tool_variants = [GENERIC, NATIVE_NEXT_GEN, XS]`)
|
||||
- **Fallback behavior**: If a variant isn't defined for a model family, `ClineToolSet.getToolByNameWithFallback()` automatically falls back to GENERIC. So you only need to export `[GENERIC]` unless the tool needs model-specific behavior.
|
||||
3. **Register in `src/core/prompts/system-prompt/tools/init.ts`** - Import and spread into `allToolVariants`
|
||||
4. **Add to variant configs** - Each model family has its own config in `src/core/prompts/system-prompt/variants/*/config.ts`. Add your tool's enum to the `.tools()` list:
|
||||
- `generic/config.ts`, `next-gen/config.ts`, `gpt-5/config.ts`, `native-gpt-5/config.ts`, `native-gpt-5-1/config.ts`, `native-next-gen/config.ts`, `gemini-3/config.ts`, `glm/config.ts`, `hermes/config.ts`, `xs/config.ts`
|
||||
- **Important**: If you add to a variant's config, make sure the tool spec exports a variant for that ModelFamily (or relies on GENERIC fallback)
|
||||
5. **Create handler** in `src/core/task/tools/handlers/`
|
||||
6. **Wire up in `ToolExecutor.ts`** if needed for execution flow
|
||||
7. **Add to tool parsing** in `src/core/assistant-message/index.ts` if needed
|
||||
8. **If tool has UI feedback**: add `ClineSay` enum in proto, update `src/shared/ExtensionMessage.ts`, update `src/shared/proto-conversions/cline-message.ts`, update `webview-ui/src/components/chat/ChatRow.tsx`
|
||||
|
||||
## Modifying System Prompt
|
||||
**Read these first:** `src/core/prompts/system-prompt/README.md`, `tools/README.md`, `__tests__/README.md`
|
||||
|
||||
System prompt is modular: **components** (reusable sections) + **variants** (model-specific configs) + **templates** (with `{{PLACEHOLDER}}` resolution).
|
||||
|
||||
**Key directories:**
|
||||
- `components/` - Shared sections: `rules.ts`, `capabilities.ts`, `editing_files.ts`, etc.
|
||||
- `variants/` - Model-specific: `generic/`, `next-gen/`, `xs/`, `gpt-5/`, `gemini-3/`, `hermes/`, `glm/`, etc.
|
||||
- `templates/` - Template engine and placeholder definitions
|
||||
|
||||
**Variant tiers (ask user which to modify):**
|
||||
- **Next-gen** (Claude 4, GPT-5, Gemini 2.5): `next-gen/`, `native-next-gen/`, `native-gpt-5/`, `native-gpt-5-1/`, `gemini-3/`, `gpt-5/`
|
||||
- **Standard** (default fallback): `generic/`
|
||||
- **Local/small models**: `xs/`, `hermes/`, `glm/`
|
||||
|
||||
**How overrides work:** Variants can override components via `componentOverrides` in their `config.ts`, or provide a custom template in `template.ts` (e.g., `next-gen/template.ts` exports `rules_template`). If no override, the shared component from `components/` is used.
|
||||
|
||||
**Example: Adding a rule to RULES section**
|
||||
1. Check if variant overrides rules: look for `rules_template` in `variants/*/template.ts` or `componentOverrides.RULES` in `config.ts`
|
||||
2. If shared: modify `components/rules.ts`
|
||||
3. If overridden: modify that variant's template
|
||||
4. XS variant is special—has heavily condensed inline content in `template.ts`
|
||||
|
||||
**After any changes, regenerate snapshots:**
|
||||
```bash
|
||||
UPDATE_SNAPSHOTS=true npm run test:unit
|
||||
```
|
||||
Snapshots live in `__tests__/__snapshots__/`. Tests validate across model families and context variations (browser, MCP, focus chain).
|
||||
|
||||
## Modifying Default Slash Commands
|
||||
Three places need updates:
|
||||
- `src/core/slash-commands/index.ts` - Command definitions
|
||||
- `src/core/prompts/commands.ts` - System prompt integration
|
||||
- `webview-ui/src/utils/slash-commands.ts` - Webview autocomplete
|
||||
|
||||
## Adding New Global State Keys
|
||||
Adding a new key to global state requires updates in multiple places. Missing any step causes silent failures.
|
||||
|
||||
Required steps:
|
||||
1. Type definition in `src/shared/storage/state-keys.ts` - Add to `GlobalState` or `Settings` interface
|
||||
2. Add any default value or transform in `src/shared/storage/state-keys.ts` if the key needs one
|
||||
3. Read and write the value through `StateManager` (`setGlobalState()` / `getGlobalStateKey()`) after initialization
|
||||
2. Read from globalState in `src/core/storage/utils/state-helpers.ts`:
|
||||
- Add `const myKey = context.globalState.get<GlobalStateAndSettings["myKey"]>("myKey")` in `readGlobalStateFromDisk()`
|
||||
- Add to the return object: `myKey: myKey ?? defaultValue,`
|
||||
3. StateManager handles read/write via `setGlobalState()`/`getGlobalStateKey()` after initialization
|
||||
|
||||
Persistent state is file-backed through `StateManager`; do not add new runtime reads or writes against VS Code `ExtensionContext` storage. That storage is only a legacy migration source.
|
||||
Common mistake: Adding only the return value without the `context.globalState.get()` call. This compiles but the value is always `undefined` on load.
|
||||
|
||||
Settings plumbing gotcha: if a key is user-toggleable from settings, wire both controller update paths:
|
||||
- `src/core/controller/state/updateSettings.ts` for webview `updateSetting(...)`
|
||||
@@ -157,20 +116,22 @@ Webview toggle gotcha: settings changes must also round-trip back in state paylo
|
||||
If this round-trip wiring is missing, the backend value can update but the toggle in webview appears stuck or reverts.
|
||||
|
||||
## StateManager Cache vs Direct globalState Access
|
||||
StateManager uses an in-memory cache populated during `StateManager.initialize()` from file-backed storage. For most state, use `controller.stateManager.setGlobalState()`/`getGlobalStateKey()`.
|
||||
StateManager uses an in-memory cache populated during `StateManager.initialize(context)` in `common.ts`. For most state, use `controller.stateManager.setGlobalState()`/`getGlobalStateKey()`.
|
||||
|
||||
Exception: host migration code may read legacy VS Code storage before file-backed storage is initialized.
|
||||
Exception: State needed immediately at extension startup (before cache is ready)
|
||||
|
||||
Example pattern:
|
||||
When Window A sets state and immediately opens Window B, the new window's StateManager cache is populated from `context.globalState` during initialization. If you need to read state in Window B right at startup (e.g., in `common.ts` during `initialize()`), read directly from `context.globalState.get()` instead of StateManager's cache.
|
||||
|
||||
Example pattern (see `lastShownAnnouncementId` and `worktreeAutoOpenPath`):
|
||||
```typescript
|
||||
// Writing (normal pattern)
|
||||
controller.stateManager.setGlobalState("myKey", value)
|
||||
|
||||
// Reading after initialization
|
||||
const value = controller.stateManager.getGlobalStateKey("myKey")
|
||||
// Reading at startup in common.ts (bypass cache)
|
||||
const value = context.globalState.get<string>("myKey")
|
||||
```
|
||||
|
||||
Use `context.globalState` only in VS Code migration code that copies legacy ExtensionContext values into the shared file-backed stores.
|
||||
This is only needed for cross-window state read during the brief startup window before StateManager cache is fully usable. Normal state access after initialization should use StateManager.
|
||||
|
||||
## ChatRow Cancelled/Interrupted States
|
||||
When a ChatRow displays a loading/in-progress state (spinner), you must handle what happens when the task is cancelled. This is non-obvious because cancellation doesn't update the message content—you have to infer it from context.
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# SDK Adapter
|
||||
|
||||
The VSCode extension runs on the Cline SDK (`@cline/core`, `@cline/llms`,
|
||||
`@cline/shared`) through an adapter layer in `apps/vscode/src/sdk/`. The
|
||||
webview still talks gRPC; the adapter translates between gRPC handlers and SDK
|
||||
calls. See `apps/vscode/src/dev/debug-harness/README.md` for the debug harness.
|
||||
|
||||
## Conventions
|
||||
|
||||
1. **Look up SDK APIs, don't guess.** Use `kb_search(name="sdk", query="...")`
|
||||
before implementing against an SDK surface.
|
||||
2. **Reference the pre-SDK implementation when replacing a module.** Add a
|
||||
`// Replaces classic src/core/... (see origin/main)` header and use
|
||||
`kb_search(name="cline", commit="origin/main")` or
|
||||
`git show origin/main:path` to consult the prior implementation.
|
||||
3. **Single entry point.** There is one codepath — the SDK adapter. No
|
||||
`CLINE_SDK` env flag.
|
||||
4. **Use `{appBaseUrl}`**, never hardcode `app.cline.bot`.
|
||||
5. **Avoid `as` casts.** Use explicit conversion functions with tests. The
|
||||
branded types in `apps/vscode/src/sdk/model-catalog/contracts.ts` exist so
|
||||
casts are unnecessary outside parse/compute boundaries.
|
||||
|
||||
## Debug harness
|
||||
|
||||
- **Dismiss the Kanban/promo overlay** before any debug harness interaction.
|
||||
- **Use the command palette** to navigate tabs in the debug harness.
|
||||
@@ -44,7 +44,7 @@ For Responses API providers: add to `isNextGenModelProvider()` in `src/utils/mod
|
||||
Modular: `components/` (shared) + `variants/` (model-specific) + `templates/` (`{{PLACEHOLDER}}`). Variants override components via `componentOverrides` in `config.ts` or custom `template.ts`. XS variant is heavily condensed inline. Always regenerate snapshots after changes.
|
||||
|
||||
## Global State Keys (silent failure risk)
|
||||
Adding a key requires updating the typed storage definitions in `src/shared/storage/state-keys.ts`; runtime reads and writes should go through `StateManager`, not VS Code `ExtensionContext` storage. Persistent state is file-backed so it works across VS Code, CLI, and JetBrains hosts.
|
||||
Adding a key requires: type in `src/shared/storage/state-keys.ts`, read via `context.globalState.get()` in `src/core/storage/utils/state-helpers.ts` `readGlobalStateFromDisk()`, and add to return object. Missing the `.get()` call compiles fine but value is always `undefined`.
|
||||
|
||||
## Slash Commands (3 places)
|
||||
- `src/core/slash-commands/index.ts` — definitions.
|
||||
|
||||
@@ -33,7 +33,7 @@ permissions:
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: .
|
||||
working-directory: sdk
|
||||
|
||||
jobs:
|
||||
publish-main:
|
||||
@@ -105,12 +105,12 @@ jobs:
|
||||
PACKAGE_VERSION=$(node -p "require('./apps/cli/package.json').version")
|
||||
|
||||
if [ "$PACKAGE_VERSION" != "$VERSION" ]; then
|
||||
echo "apps/cli/package.json version ${PACKAGE_VERSION} does not match ${TAG}"
|
||||
echo "sdk/apps/cli/package.json version ${PACKAGE_VERSION} does not match ${TAG}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! printf "%s\n" "$VERSION" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$'; then
|
||||
echo "apps/cli/package.json has invalid version: ${VERSION}"
|
||||
echo "sdk/apps/cli/package.json has invalid version: ${VERSION}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -147,7 +147,7 @@ jobs:
|
||||
|
||||
- name: Build platform binaries
|
||||
run: bun script/build.ts --install-native-variants --skip-sdk-build
|
||||
working-directory: apps/cli
|
||||
working-directory: sdk/apps/cli
|
||||
env:
|
||||
TELEMETRY_SERVICE_API_KEY: ${{ secrets.TELEMETRY_SERVICE_API_KEY }}
|
||||
ERROR_SERVICE_API_KEY: ${{ secrets.ERROR_SERVICE_API_KEY }}
|
||||
@@ -194,7 +194,7 @@ jobs:
|
||||
env:
|
||||
NPM_CONFIG_PROVENANCE: "true"
|
||||
run: bun script/publish-npm.ts --tag latest
|
||||
working-directory: apps/cli
|
||||
working-directory: sdk/apps/cli
|
||||
|
||||
- name: Get Previous CLI Tag
|
||||
id: prev_tag
|
||||
@@ -375,7 +375,7 @@ jobs:
|
||||
- name: Build platform binaries
|
||||
if: steps.check_commits.outputs.skip != 'true'
|
||||
run: bun script/build.ts --install-native-variants --skip-sdk-build
|
||||
working-directory: apps/cli
|
||||
working-directory: sdk/apps/cli
|
||||
env:
|
||||
TELEMETRY_SERVICE_API_KEY: ${{ secrets.TELEMETRY_SERVICE_API_KEY }}
|
||||
ERROR_SERVICE_API_KEY: ${{ secrets.ERROR_SERVICE_API_KEY }}
|
||||
@@ -424,7 +424,7 @@ jobs:
|
||||
env:
|
||||
NPM_CONFIG_PROVENANCE: "true"
|
||||
run: bun script/publish-npm.ts --tag nightly
|
||||
working-directory: apps/cli
|
||||
working-directory: sdk/apps/cli
|
||||
|
||||
- name: Summary
|
||||
if: steps.check_commits.outputs.skip != 'true'
|
||||
|
||||
@@ -31,6 +31,9 @@ jobs:
|
||||
if: github.repository == 'cline/cline' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dpc/sdk-migration-simpler-login')
|
||||
runs-on: ubuntu-latest
|
||||
environment: PublishNightly
|
||||
# The VS Code extension's package.json and lockfiles live under apps/vscode/
|
||||
# (the repo root has no package.json). Mirror ext-vscode-test.yml so install
|
||||
# and publish steps run in the correct workspace.
|
||||
defaults:
|
||||
run:
|
||||
working-directory: apps/vscode
|
||||
@@ -56,14 +59,16 @@ jobs:
|
||||
# Keep publish environment aligned with test workflow/tooling lockfile expectations.
|
||||
# Newer LTS (Node 24 / npm 11) can make `npm list` fail with ELSPROBLEMS during vsce packaging.
|
||||
node-version: 22
|
||||
cache: 'npm'
|
||||
cache-dependency-path: |
|
||||
apps/vscode/package-lock.json
|
||||
apps/vscode/webview-ui/package-lock.json
|
||||
|
||||
- name: Install extension dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode ci --include=optional
|
||||
- name: Install root dependencies
|
||||
run: npm ci --include=optional
|
||||
|
||||
- name: Install webview-ui dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode/webview-ui ci --include=optional
|
||||
run: cd webview-ui && npm ci --include=optional
|
||||
|
||||
- name: Install Publishing Tools
|
||||
run: npm install -g @vscode/vsce ovsx
|
||||
|
||||
@@ -114,13 +114,11 @@ jobs:
|
||||
with:
|
||||
node-version: 22
|
||||
|
||||
- name: Install extension dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode install --include=optional
|
||||
- name: Install root dependencies
|
||||
run: npm install --include=optional
|
||||
|
||||
- name: Install webview-ui dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode/webview-ui install --include=optional
|
||||
run: cd webview-ui && npm install --include=optional
|
||||
|
||||
- name: Install Publishing Tools
|
||||
run: npm install -g @vscode/vsce ovsx
|
||||
|
||||
@@ -128,15 +128,13 @@ jobs:
|
||||
restore-keys: |
|
||||
playwright-browsers-${{ runner.os }}-
|
||||
|
||||
- name: Install extension dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode ci
|
||||
- name: Install root dependencies
|
||||
run: npm ci
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Install webview-ui dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode/webview-ui ci
|
||||
run: cd webview-ui && npm ci
|
||||
|
||||
- name: Install vsce
|
||||
run: npm install -g @vscode/vsce
|
||||
|
||||
@@ -91,15 +91,13 @@ jobs:
|
||||
apps/vscode/package-lock.json
|
||||
apps/vscode/webview-ui/package-lock.json
|
||||
|
||||
- name: Install extension dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode ci
|
||||
- name: Install root dependencies
|
||||
run: npm ci --include=optional
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Install webview-ui dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode/webview-ui ci
|
||||
run: cd webview-ui && npm ci --include=optional
|
||||
|
||||
- name: Run Quality Checks (Parallel)
|
||||
run: npm run ci:check-all
|
||||
@@ -132,19 +130,16 @@ jobs:
|
||||
apps/vscode/package-lock.json
|
||||
apps/vscode/webview-ui/package-lock.json
|
||||
|
||||
- name: Install extension dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode ci
|
||||
- name: Install root dependencies
|
||||
run: npm ci --include=optional
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Install webview-ui dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode/webview-ui ci
|
||||
run: cd webview-ui && npm ci --include=optional
|
||||
|
||||
- name: Set up NPM on Windows
|
||||
if: runner.os == 'Windows'
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: |
|
||||
npm config set script-shell "C:\\Program Files\\Git\\bin\\bash.exe"
|
||||
|
||||
@@ -160,6 +155,11 @@ jobs:
|
||||
id: build_step
|
||||
run: npm run ci:build
|
||||
|
||||
- name: Vitest Suites (SDK adapter + model catalog)
|
||||
id: vitest_tests
|
||||
if: ${{ !cancelled() && steps.build_step.outcome == 'success' }}
|
||||
run: npm run test:vitest
|
||||
|
||||
- name: Unit Tests with coverage - Linux
|
||||
id: unit_tests_linux
|
||||
if: ${{ !cancelled() && steps.build_step.outcome == 'success' && runner.os == 'Linux' }}
|
||||
@@ -234,15 +234,13 @@ jobs:
|
||||
apps/vscode/webview-ui/package-lock.json
|
||||
apps/vscode/testing-platform/package-lock.json
|
||||
|
||||
- name: Install extension dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode ci
|
||||
- name: Install root dependencies
|
||||
run: npm ci --include=optional
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Install webview-ui dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode/webview-ui ci
|
||||
run: cd webview-ui && npm ci --include=optional
|
||||
|
||||
- name: Download ripgrep binaries
|
||||
run: npm run download-ripgrep
|
||||
@@ -251,8 +249,7 @@ jobs:
|
||||
run: npm run compile-standalone
|
||||
|
||||
- name: Install testing platform dependencies
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: npm --prefix apps/vscode/testing-platform ci
|
||||
run: cd testing-platform && npm ci --include=optional
|
||||
|
||||
- name: Running testing platform integration spec tests
|
||||
timeout-minutes: 7
|
||||
|
||||
@@ -26,7 +26,7 @@ on:
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: .
|
||||
working-directory: sdk
|
||||
|
||||
jobs:
|
||||
test:
|
||||
@@ -148,7 +148,7 @@ jobs:
|
||||
env:
|
||||
CHANNEL: ${{ steps.channel.outputs.channel }}
|
||||
run: |
|
||||
BASE_VERSION=$(node -p "require('./sdk/packages/llms/package.json').version")
|
||||
BASE_VERSION=$(node -p "require('./packages/llms/package.json').version")
|
||||
|
||||
if [ "$CHANNEL" = "nightly" ]; then
|
||||
TIMESTAMP=$(date +%s)
|
||||
@@ -166,11 +166,11 @@ jobs:
|
||||
if: steps.check_commits.outputs.skip != 'true'
|
||||
env:
|
||||
VERSION: ${{ steps.version.outputs.version }}
|
||||
run: bun sdk/scripts/version.ts "$VERSION"
|
||||
run: bun scripts/version.ts "$VERSION"
|
||||
|
||||
- name: Verify publishability
|
||||
if: steps.check_commits.outputs.skip != 'true'
|
||||
run: bun sdk/scripts/check-publish.ts
|
||||
run: bun scripts/check-publish.ts
|
||||
|
||||
- name: Prepare package tarball directory
|
||||
if: steps.check_commits.outputs.skip != 'true'
|
||||
@@ -187,7 +187,7 @@ jobs:
|
||||
VERSION: ${{ steps.version.outputs.version }}
|
||||
run: |
|
||||
echo "Publishing @cline/shared@${VERSION} with tag '${CHANNEL}'..."
|
||||
cd sdk/packages/shared
|
||||
cd packages/shared
|
||||
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
|
||||
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
|
||||
|
||||
@@ -199,7 +199,7 @@ jobs:
|
||||
VERSION: ${{ steps.version.outputs.version }}
|
||||
run: |
|
||||
echo "Publishing @cline/llms@${VERSION} with tag '${CHANNEL}'..."
|
||||
cd sdk/packages/llms
|
||||
cd packages/llms
|
||||
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
|
||||
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
|
||||
|
||||
@@ -211,7 +211,7 @@ jobs:
|
||||
VERSION: ${{ steps.version.outputs.version }}
|
||||
run: |
|
||||
echo "Publishing @cline/agents@${VERSION} with tag '${CHANNEL}'..."
|
||||
cd sdk/packages/agents
|
||||
cd packages/agents
|
||||
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
|
||||
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
|
||||
|
||||
@@ -223,7 +223,7 @@ jobs:
|
||||
VERSION: ${{ steps.version.outputs.version }}
|
||||
run: |
|
||||
echo "Publishing @cline/core@${VERSION} with tag '${CHANNEL}'..."
|
||||
cd sdk/packages/core
|
||||
cd packages/core
|
||||
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
|
||||
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
|
||||
|
||||
@@ -235,7 +235,7 @@ jobs:
|
||||
VERSION: ${{ steps.version.outputs.version }}
|
||||
run: |
|
||||
echo "Publishing @cline/sdk@${VERSION} with tag '${CHANNEL}'..."
|
||||
cd sdk/packages/sdk
|
||||
cd packages/sdk
|
||||
TARBALL=$(bun pm pack --destination "$RUNNER_TEMP/sdk-npm-packs" --quiet)
|
||||
npm publish "$RUNNER_TEMP/sdk-npm-packs/$(basename "$TARBALL")" --tag "$CHANNEL" --access public
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ permissions:
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: .
|
||||
working-directory: sdk
|
||||
|
||||
jobs:
|
||||
quality-checks:
|
||||
@@ -96,12 +96,12 @@ jobs:
|
||||
|
||||
- name: Run SDK Tests (Windows)
|
||||
if: ${{ !cancelled() && steps.build_sdk_step.outcome == 'success' && steps.build_cli_step.outcome == 'success' && matrix.os == 'windows-latest' }}
|
||||
run: bun -F './sdk/packages/**' test
|
||||
run: bun -F './packages/**' test
|
||||
|
||||
- name: Smoke test SQLite under Node
|
||||
if: ${{ !cancelled() && steps.build_sdk_step.outcome == 'success' && matrix.os != 'windows-latest' }}
|
||||
timeout-minutes: 10
|
||||
run: bun sdk/scripts/ci-node-smoke.ts
|
||||
run: bun scripts/ci-node-smoke.ts
|
||||
|
||||
- name: Run TUI e2e tests
|
||||
if: ${{ !cancelled() && steps.build_sdk_step.outcome == 'success' && steps.build_cli_step.outcome == 'success' && matrix.os == 'ubuntu-latest' && matrix.node-version == '24.x' }}
|
||||
@@ -109,4 +109,4 @@ jobs:
|
||||
|
||||
- name: Verify packages are publishable
|
||||
if: ${{ !cancelled() && steps.build_sdk_step.outcome == 'success' && steps.build_cli_step.outcome == 'success' && matrix.os == 'ubuntu-latest' && matrix.node-version == '24.x' }}
|
||||
run: bun sdk/scripts/check-publish.ts
|
||||
run: bun scripts/check-publish.ts
|
||||
|
||||
+3
-11
@@ -13,6 +13,9 @@ tmp
|
||||
pnpm-lock.yaml
|
||||
|
||||
.clineignore
|
||||
.cline/enterprise
|
||||
.cline/remote-config
|
||||
**/.cline/remote-config
|
||||
.venv
|
||||
.actrc
|
||||
CLAUDE.local.md
|
||||
@@ -61,17 +64,6 @@ tests/**/cache
|
||||
# Should never be committed: only exists if a publish aborts mid-swap.
|
||||
.README.github.bak
|
||||
|
||||
# Tauri generated code
|
||||
apps/*/src-tauri/gen
|
||||
apps/*/src-tauri/bin
|
||||
apps/examples/*/src-tauri/gen
|
||||
apps/examples/*/src-tauri/bin
|
||||
# Tauri UI test snapshots
|
||||
apps/*/src/tests/.tui-test
|
||||
apps/*/src/tests/tui-traces
|
||||
apps/vscode/webview-ui/src/**/*.js
|
||||
apps/vscode/webview-ui/src/**/*.js.map
|
||||
|
||||
|
||||
# SDK Session files / User data
|
||||
.cline/data
|
||||
|
||||
+1
-10
@@ -1,10 +1 @@
|
||||
if ! command -v gitleaks >/dev/null 2>&1; then
|
||||
echo "gitleaks is required for the pre-commit secret scan."
|
||||
echo "Install it with: brew install gitleaks"
|
||||
echo "Other install options: https://github.com/gitleaks/gitleaks#installing"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
gitleaks git --pre-commit --redact --staged --verbose || exit 1
|
||||
|
||||
lint-staged
|
||||
cd apps/vscode && lint-staged
|
||||
Vendored
+21
-22
@@ -5,8 +5,8 @@
|
||||
"tasks": [
|
||||
{
|
||||
"label": "compile-standalone",
|
||||
"type": "shell",
|
||||
"command": "npm run compile-standalone",
|
||||
"type": "npm",
|
||||
"script": "compile-standalone",
|
||||
"group": "build",
|
||||
"problemMatcher": [],
|
||||
"presentation": {
|
||||
@@ -18,8 +18,8 @@
|
||||
},
|
||||
{
|
||||
"label": "npm: protos",
|
||||
"type": "shell",
|
||||
"command": "npm run protos",
|
||||
"type": "npm",
|
||||
"script": "protos",
|
||||
"problemMatcher": [],
|
||||
"isBackground": false,
|
||||
"presentation": {
|
||||
@@ -64,8 +64,8 @@
|
||||
"group": "build"
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"command": "npm run build:webview",
|
||||
"type": "npm",
|
||||
"script": "build:webview",
|
||||
"group": "build",
|
||||
"problemMatcher": [],
|
||||
"isBackground": true,
|
||||
@@ -85,8 +85,8 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"command": "npm run build:webview:test",
|
||||
"type": "npm",
|
||||
"script": "build:webview:test",
|
||||
"group": "build",
|
||||
"problemMatcher": [],
|
||||
"isBackground": true,
|
||||
@@ -107,8 +107,8 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"command": "npm run dev:webview",
|
||||
"type": "npm",
|
||||
"script": "dev:webview",
|
||||
"group": "build",
|
||||
"problemMatcher": [
|
||||
{
|
||||
@@ -144,8 +144,8 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"command": "npm run watch:esbuild",
|
||||
"type": "npm",
|
||||
"script": "watch:esbuild",
|
||||
"group": "build",
|
||||
"problemMatcher": {
|
||||
"pattern": [
|
||||
@@ -183,8 +183,8 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"command": "npm run watch:esbuild:test",
|
||||
"type": "npm",
|
||||
"script": "watch:esbuild:test",
|
||||
"group": "build",
|
||||
"problemMatcher": {
|
||||
"pattern": [
|
||||
@@ -223,8 +223,8 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"command": "npm run watch:tsc",
|
||||
"type": "npm",
|
||||
"script": "watch:tsc",
|
||||
"group": "build",
|
||||
"problemMatcher": "$tsc-watch",
|
||||
"isBackground": true,
|
||||
@@ -241,9 +241,8 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"command": "npm run watch-tests",
|
||||
"label": "npm: watch-tests",
|
||||
"type": "npm",
|
||||
"script": "watch-tests",
|
||||
"problemMatcher": "$tsc-watch",
|
||||
"isBackground": true,
|
||||
"dependsOn": [
|
||||
@@ -281,8 +280,8 @@
|
||||
"command": "rm -rf ${workspaceFolder}/apps/vscode/dist/tmp/user && mkdir -p ${workspaceFolder}/apps/vscode/dist/tmp/user"
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"command": "npm run storybook",
|
||||
"type": "npm",
|
||||
"script": "storybook",
|
||||
"group": "build",
|
||||
"problemMatcher": [],
|
||||
"isBackground": false,
|
||||
@@ -309,7 +308,7 @@
|
||||
"$tsc"
|
||||
],
|
||||
"options": {
|
||||
"cwd": "${workspaceFolder}"
|
||||
"cwd": "${workspaceFolder}/sdk"
|
||||
}
|
||||
}
|
||||
],
|
||||
|
||||
@@ -1,30 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## [3.88.0]
|
||||
|
||||
### Added
|
||||
|
||||
- Add the latest Fireworks AI serverless models and update the default Fireworks model to Kimi K2.6.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Fix MCP server delete/add flows so settings writes do not cause the MCP server list to be emptied by the file watcher.
|
||||
- Remove stale Fireworks AI models and correct Fireworks model metadata and cache pricing.
|
||||
|
||||
### Changed
|
||||
|
||||
- Always use the upstream Cline recommended models endpoint instead of gating it behind a feature flag.
|
||||
|
||||
## [3.87.0]
|
||||
|
||||
### Added
|
||||
|
||||
- Add MiniMax M3 model support.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Update VS Code extension dependencies to resolve security issues in `@xmldom/xmldom`, `basic-ftp`, `axios`, `undici`, and other direct/transitive packages.
|
||||
|
||||
## [3.86.2]
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -51,7 +51,7 @@ for CI/CD and scripting.
|
||||
npm i -g cline
|
||||
```
|
||||
|
||||
<a href="./apps/cli/README.md">Learn more</a>
|
||||
<a href="./sdk/apps/cli/README.md">Learn more</a>
|
||||
<br><br>
|
||||
|
||||
</td>
|
||||
@@ -129,7 +129,7 @@ npm install @cline/sdk
|
||||
| Product | Description | Location | CHANGELOG |
|
||||
|---------|------------|--------------|--------------|
|
||||
| **SDK** | Node.js programmatic agent API and extension exports. | [`sdk/`](https://github.com/cline/cline/tree/main/sdk) | [CHANGELOG.md](https://github.com/cline/cline/blob/main/sdk/CHANGELOG.md) |
|
||||
| **CLI** | Terminal UI, headless mode, shell commands, and CLI-specific flows. | [`apps/cli/`](https://github.com/cline/cline/tree/main/apps/cli) | [CHANGELOG.md](https://github.com/cline/cline/blob/main/apps/cli/CHANGELOG.md) |
|
||||
| **CLI** | Terminal UI, headless mode, shell commands, and CLI-specific flows. | [`sdk/apps/cli/`](https://github.com/cline/cline/tree/main/sdk/apps/cli) | [CHANGELOG.md](https://github.com/cline/cline/blob/main/sdk/apps/cli/CHANGELOG.md) |
|
||||
| **VS Code Extension** | The Marketplace extension and extension host integration. | [`/`](https://github.com/cline/cline/tree/main) (WIP migrating) | [CHANGELOG.md](https://github.com/cline/cline/blob/main/CHANGELOG.md) |
|
||||
| **JetBrains Plugin** | JetBrains-hosted client that talks to the shared agent core. | Currently we are not open-sourcing JetBrains plugins | - |
|
||||
| **Kanban** | Web-based multi-agent task board. | [`cline/kanban`](https://github.com/cline/kanban) | [CHANGELOG.md](https://github.com/cline/kanban/blob/main/CHANGELOG.md) |
|
||||
@@ -212,12 +212,8 @@ cline schedule create "PR summary" \
|
||||
Chat with your agent from any messaging platform: Telegram, Slack, Discord, Google Chat, WhatsApp, and Linear. Each conversation thread maps to an agent session with full context. Set up access control to restrict who can interact with your agent.
|
||||
|
||||
```bash
|
||||
# Connect to Telegram
|
||||
cline connect telegram -k $BOT_TOKEN
|
||||
# Connect to Slack through webhook
|
||||
cline connect slack --bot-token $SLACK_TOKEN --signing-secret $SECRET --base-url $URL
|
||||
# Connect to Slack using socket mode
|
||||
cline connect slack --bot-token $SLACK_TOKEN --app-token $SLACK_APP_TOKEN
|
||||
cline connect slack --token $SLACK_TOKEN --signing-secret $SECRET --base-url $URL
|
||||
```
|
||||
|
||||
## Headless CLI for CI/CD
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
{
|
||||
"$schema": "https://biomejs.dev/schemas/2.4.5/schema.json",
|
||||
"root": false,
|
||||
"extends": ["../sdk/biome.json"],
|
||||
"linter": {
|
||||
"rules": {
|
||||
"a11y": {
|
||||
"noStaticElementInteractions": "warn"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,184 +0,0 @@
|
||||
import { mkdirSync, mkdtempSync } from "node:fs";
|
||||
import { arch, platform, tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { runDashboardCommand, waitForProcessShutdown } from "./dashboard";
|
||||
|
||||
const ENV_KEYS = [
|
||||
"WORKSPACE_ROOT",
|
||||
"HOST",
|
||||
"CLINE_HUB_DASHBOARD_PORT",
|
||||
"PUBLIC_URL",
|
||||
"ROOM_SECRET",
|
||||
"CLINE_HUB_WEBVIEW_DIST_DIR",
|
||||
"CLINE_WRAPPER_PATH",
|
||||
] as const;
|
||||
|
||||
const originalEnv = Object.fromEntries(
|
||||
ENV_KEYS.map((key) => [key, process.env[key]]),
|
||||
);
|
||||
|
||||
afterEach(() => {
|
||||
for (const key of ENV_KEYS) {
|
||||
const value = originalEnv[key];
|
||||
if (value === undefined) {
|
||||
delete process.env[key];
|
||||
} else {
|
||||
process.env[key] = value;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
describe("runDashboardCommand", () => {
|
||||
it("starts the dashboard server, opens the invite URL, and waits for shutdown", async () => {
|
||||
const output: string[] = [];
|
||||
const errors: string[] = [];
|
||||
const opened: string[] = [];
|
||||
const stop = vi.fn();
|
||||
let observedEnv:
|
||||
| {
|
||||
workspaceRoot: string | undefined;
|
||||
host: string | undefined;
|
||||
port: string | undefined;
|
||||
publicUrl: string | undefined;
|
||||
roomSecret: string | undefined;
|
||||
webviewDistDir: string | undefined;
|
||||
}
|
||||
| undefined;
|
||||
const webviewDistDir = mkdtempSync(join(tmpdir(), "cline-webview-dist-"));
|
||||
mkdirSync(webviewDistDir, { recursive: true });
|
||||
process.env.CLINE_HUB_WEBVIEW_DIST_DIR = webviewDistDir;
|
||||
|
||||
const exitCode = await runDashboardCommand({
|
||||
cwd: "sdk",
|
||||
host: "127.0.0.1",
|
||||
port: "9090",
|
||||
publicUrl: "http://127.0.0.1:9090",
|
||||
roomSecret: "secret",
|
||||
io: {
|
||||
writeln: (text) => output.push(text ?? ""),
|
||||
writeErr: (text) => errors.push(text),
|
||||
},
|
||||
startServer: async () => {
|
||||
observedEnv = {
|
||||
workspaceRoot: process.env.WORKSPACE_ROOT,
|
||||
host: process.env.HOST,
|
||||
port: process.env.CLINE_HUB_DASHBOARD_PORT,
|
||||
publicUrl: process.env.PUBLIC_URL,
|
||||
roomSecret: process.env.ROOM_SECRET,
|
||||
webviewDistDir: process.env.CLINE_HUB_WEBVIEW_DIST_DIR,
|
||||
};
|
||||
return {
|
||||
listenUrl: "http://127.0.0.1:9090/",
|
||||
publicUrl: "http://127.0.0.1:9090",
|
||||
inviteUrl: "http://127.0.0.1:9090/?roomSecret=secret",
|
||||
hubUrl: "ws://127.0.0.1:25463/hub",
|
||||
stop,
|
||||
};
|
||||
},
|
||||
openUrl: async (url) => {
|
||||
opened.push(url);
|
||||
},
|
||||
waitForShutdown: async (server) => {
|
||||
await server.stop();
|
||||
},
|
||||
});
|
||||
|
||||
expect(exitCode).toBe(0);
|
||||
expect(observedEnv).toEqual({
|
||||
workspaceRoot: resolve("sdk"),
|
||||
host: "127.0.0.1",
|
||||
port: "9090",
|
||||
publicUrl: "http://127.0.0.1:9090",
|
||||
roomSecret: "secret",
|
||||
webviewDistDir,
|
||||
});
|
||||
expect(opened).toEqual(["http://127.0.0.1:9090/?roomSecret=secret"]);
|
||||
expect(stop).toHaveBeenCalledTimes(1);
|
||||
expect(output.join("\n")).toContain("Cline dashboard listening at");
|
||||
expect(output.join("\n")).toContain("ws://127.0.0.1:25463/hub");
|
||||
expect(errors).toEqual([]);
|
||||
expect(process.env.WORKSPACE_ROOT).toBe(originalEnv.WORKSPACE_ROOT);
|
||||
expect(process.env.CLINE_HUB_WEBVIEW_DIST_DIR).toBe(webviewDistDir);
|
||||
});
|
||||
|
||||
it("honors --no-open behavior", async () => {
|
||||
const openUrl = vi.fn();
|
||||
|
||||
const exitCode = await runDashboardCommand({
|
||||
openBrowser: false,
|
||||
io: {
|
||||
writeln: () => {},
|
||||
writeErr: () => {},
|
||||
},
|
||||
startServer: async () => ({
|
||||
listenUrl: "http://127.0.0.1:8787/",
|
||||
publicUrl: "http://127.0.0.1:8787",
|
||||
inviteUrl: "http://127.0.0.1:8787",
|
||||
stop: vi.fn(),
|
||||
}),
|
||||
openUrl,
|
||||
waitForShutdown: async () => {},
|
||||
});
|
||||
|
||||
expect(exitCode).toBe(0);
|
||||
expect(openUrl).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("finds webview assets from the published wrapper package layout", async () => {
|
||||
const root = mkdtempSync(join(tmpdir(), "cline-wrapper-layout-"));
|
||||
const wrapperPath = join(root, "node_modules", "cline", "bin", "cline");
|
||||
const platformName = platform() === "win32" ? "windows" : platform();
|
||||
const webviewDistDir = join(
|
||||
root,
|
||||
"node_modules",
|
||||
"cline",
|
||||
"node_modules",
|
||||
"@cline",
|
||||
`cli-${platformName}-${arch()}`,
|
||||
"cline-hub",
|
||||
"webview",
|
||||
);
|
||||
mkdirSync(join(wrapperPath, ".."), { recursive: true });
|
||||
mkdirSync(webviewDistDir, { recursive: true });
|
||||
process.env.CLINE_WRAPPER_PATH = wrapperPath;
|
||||
delete process.env.CLINE_HUB_WEBVIEW_DIST_DIR;
|
||||
let observedWebviewDistDir: string | undefined;
|
||||
|
||||
const exitCode = await runDashboardCommand({
|
||||
openBrowser: false,
|
||||
io: {
|
||||
writeln: () => {},
|
||||
writeErr: () => {},
|
||||
},
|
||||
startServer: async () => {
|
||||
observedWebviewDistDir = process.env.CLINE_HUB_WEBVIEW_DIST_DIR;
|
||||
return {
|
||||
listenUrl: "http://127.0.0.1:8787/",
|
||||
publicUrl: "http://127.0.0.1:8787",
|
||||
inviteUrl: "http://127.0.0.1:8787",
|
||||
stop: vi.fn(),
|
||||
};
|
||||
},
|
||||
waitForShutdown: async () => {},
|
||||
});
|
||||
|
||||
expect(exitCode).toBe(0);
|
||||
expect(observedWebviewDistDir).toBe(webviewDistDir);
|
||||
});
|
||||
|
||||
it("settles shutdown when server stop rejects", async () => {
|
||||
const shutdown = waitForProcessShutdown({
|
||||
listenUrl: "http://127.0.0.1:8787/",
|
||||
publicUrl: "http://127.0.0.1:8787",
|
||||
inviteUrl: "http://127.0.0.1:8787",
|
||||
stop: vi.fn(async () => {
|
||||
throw new Error("stop failed");
|
||||
}),
|
||||
});
|
||||
|
||||
process.emit("SIGINT", "SIGINT");
|
||||
|
||||
await expect(shutdown).rejects.toThrow("stop failed");
|
||||
});
|
||||
});
|
||||
@@ -1,195 +0,0 @@
|
||||
import { existsSync } from "node:fs";
|
||||
import { arch, platform } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import open from "open";
|
||||
import { c } from "../utils/output";
|
||||
|
||||
export interface DashboardServerHandle {
|
||||
listenUrl: string;
|
||||
publicUrl: string;
|
||||
inviteUrl: string;
|
||||
hubUrl?: string;
|
||||
stop: () => void | Promise<void>;
|
||||
}
|
||||
|
||||
interface DashboardCommandIo {
|
||||
writeln: (text?: string) => void;
|
||||
writeErr: (text: string) => void;
|
||||
}
|
||||
|
||||
export interface RunDashboardCommandOptions {
|
||||
cwd?: string;
|
||||
host?: string;
|
||||
port?: string;
|
||||
publicUrl?: string;
|
||||
roomSecret?: string;
|
||||
openBrowser?: boolean;
|
||||
io: DashboardCommandIo;
|
||||
startServer?: () => Promise<DashboardServerHandle>;
|
||||
openUrl?: (url: string) => Promise<void>;
|
||||
waitForShutdown?: (server: DashboardServerHandle) => Promise<void>;
|
||||
}
|
||||
|
||||
const DASHBOARD_PORT_ENV = "CLINE_HUB_DASHBOARD_PORT";
|
||||
const WEBVIEW_DIST_ENV = "CLINE_HUB_WEBVIEW_DIST_DIR";
|
||||
|
||||
function setEnvValue(name: string, value: string | undefined): () => void {
|
||||
const previous = process.env[name];
|
||||
if (value === undefined) {
|
||||
return () => {};
|
||||
}
|
||||
process.env[name] = value;
|
||||
return () => {
|
||||
if (previous === undefined) {
|
||||
delete process.env[name];
|
||||
} else {
|
||||
process.env[name] = previous;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
async function withDashboardEnvironment<T>(
|
||||
options: RunDashboardCommandOptions,
|
||||
fn: () => Promise<T>,
|
||||
): Promise<T> {
|
||||
const restore = [
|
||||
setEnvValue(
|
||||
"WORKSPACE_ROOT",
|
||||
options.cwd ? resolve(options.cwd) : undefined,
|
||||
),
|
||||
setEnvValue("HOST", options.host),
|
||||
setEnvValue(DASHBOARD_PORT_ENV, options.port),
|
||||
setEnvValue("PUBLIC_URL", options.publicUrl),
|
||||
setEnvValue("ROOM_SECRET", options.roomSecret),
|
||||
setEnvValue(WEBVIEW_DIST_ENV, resolveDefaultWebviewDistDir()),
|
||||
];
|
||||
try {
|
||||
return await fn();
|
||||
} finally {
|
||||
for (let i = restore.length - 1; i >= 0; i--) {
|
||||
restore[i]?.();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function resolveDefaultWebviewDistDir(): string | undefined {
|
||||
if (process.env[WEBVIEW_DIST_ENV]?.trim()) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const moduleDir = dirname(fileURLToPath(import.meta.url));
|
||||
const candidates = [
|
||||
...resolveInstalledPlatformPackageWebviewCandidates(),
|
||||
// Source checkout: apps/cli/src/commands/dashboard.ts
|
||||
join(moduleDir, "../../../cline-hub/dist/webview"),
|
||||
// Node bundle: apps/cli/dist/index.js
|
||||
join(moduleDir, "cline-hub/webview"),
|
||||
// Compiled platform package: apps/cli/dist/<platform>/bin/cline
|
||||
join(dirname(process.execPath), "../cline-hub/webview"),
|
||||
];
|
||||
|
||||
return candidates.find((candidate) => existsSync(candidate));
|
||||
}
|
||||
|
||||
function resolveInstalledPlatformPackageWebviewCandidates(): string[] {
|
||||
const packageName = resolvePlatformPackageName();
|
||||
const starts = [
|
||||
process.env.CLINE_WRAPPER_PATH
|
||||
? dirname(process.env.CLINE_WRAPPER_PATH)
|
||||
: undefined,
|
||||
dirname(process.execPath),
|
||||
].filter((value): value is string => !!value?.trim());
|
||||
const candidates: string[] = [];
|
||||
for (const start of starts) {
|
||||
let current = start;
|
||||
for (;;) {
|
||||
candidates.push(
|
||||
join(current, "node_modules", packageName, "cline-hub/webview"),
|
||||
);
|
||||
const parent = dirname(current);
|
||||
if (parent === current) break;
|
||||
current = parent;
|
||||
}
|
||||
}
|
||||
return candidates;
|
||||
}
|
||||
|
||||
function resolvePlatformPackageName(): string {
|
||||
const platformName = platform() === "win32" ? "windows" : platform();
|
||||
return `@cline/cli-${platformName}-${arch()}`;
|
||||
}
|
||||
|
||||
async function startDefaultDashboardServer(): Promise<DashboardServerHandle> {
|
||||
const { startClineHubDashboardServer } = await import("@cline/cline-hub");
|
||||
return await startClineHubDashboardServer();
|
||||
}
|
||||
|
||||
async function openDefaultUrl(url: string): Promise<void> {
|
||||
await open(url, { wait: false });
|
||||
}
|
||||
|
||||
export function waitForProcessShutdown(
|
||||
server: DashboardServerHandle,
|
||||
): Promise<void> {
|
||||
return new Promise<void>((resolveShutdown, rejectShutdown) => {
|
||||
let settled = false;
|
||||
|
||||
const cleanup = () => {
|
||||
process.off("SIGINT", handleSignal);
|
||||
process.off("SIGTERM", handleSignal);
|
||||
};
|
||||
|
||||
const stop = async () => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
cleanup();
|
||||
try {
|
||||
await server.stop();
|
||||
resolveShutdown();
|
||||
} catch (error) {
|
||||
rejectShutdown(error);
|
||||
}
|
||||
};
|
||||
|
||||
function handleSignal() {
|
||||
void stop();
|
||||
}
|
||||
|
||||
process.on("SIGINT", handleSignal);
|
||||
process.on("SIGTERM", handleSignal);
|
||||
});
|
||||
}
|
||||
|
||||
export async function runDashboardCommand(
|
||||
options: RunDashboardCommandOptions,
|
||||
): Promise<number> {
|
||||
try {
|
||||
const server = await withDashboardEnvironment(options, () =>
|
||||
(options.startServer ?? startDefaultDashboardServer)(),
|
||||
);
|
||||
const dashboardUrl =
|
||||
server.inviteUrl || server.publicUrl || server.listenUrl;
|
||||
options.io.writeln(
|
||||
`${c.green}Cline dashboard listening at${c.reset} ${dashboardUrl}`,
|
||||
);
|
||||
if (server.hubUrl) {
|
||||
options.io.writeln(`${c.dim}Hub endpoint: ${server.hubUrl}${c.reset}`);
|
||||
}
|
||||
|
||||
if (options.openBrowser !== false) {
|
||||
try {
|
||||
await (options.openUrl ?? openDefaultUrl)(dashboardUrl);
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
options.io.writeErr(`Failed to open browser: ${message}`);
|
||||
}
|
||||
}
|
||||
|
||||
await (options.waitForShutdown ?? waitForProcessShutdown)(server);
|
||||
return 0;
|
||||
} catch (error) {
|
||||
options.io.writeErr(error instanceof Error ? error.message : String(error));
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
@@ -1,180 +0,0 @@
|
||||
import { Llms } from "@cline/core";
|
||||
import type { ChoiceContext } from "@opentui-ui/dialog";
|
||||
import type { DialogActions } from "@opentui-ui/dialog/react";
|
||||
import { useCallback, useMemo } from "react";
|
||||
import type {
|
||||
InteractiveConfigData,
|
||||
InteractiveConfigItem,
|
||||
InteractiveConfigTab,
|
||||
LoadInteractiveConfigDataOptions,
|
||||
} from "../../tui/interactive-config";
|
||||
import type { CliCompactionMode, Config } from "../../utils/types";
|
||||
import {
|
||||
ConfigErrorContent,
|
||||
DeleteConfigItemConfirmContent,
|
||||
ExtDetailContent,
|
||||
} from "../components/dialogs/config-dialogs";
|
||||
import { withLoadingDialog } from "../components/dialogs/loading-dialog";
|
||||
import { ConfigPanelContent } from "../views/config-view";
|
||||
import type { ConfigAction } from "../views/config-view-helpers";
|
||||
import type { OpenModelSelectorOptions } from "./use-model-selector";
|
||||
|
||||
export interface OpenConfigOptions {
|
||||
initialTab?: InteractiveConfigTab;
|
||||
}
|
||||
|
||||
export function useConfigPanel(opts: {
|
||||
dialog: DialogActions;
|
||||
config: Config;
|
||||
sessionUiMode: string;
|
||||
compactionMode: CliCompactionMode;
|
||||
toggleMode: () => void;
|
||||
toggleAutoApprove: () => void;
|
||||
setCompactionMode: (mode: CliCompactionMode) => void;
|
||||
termHeight: number;
|
||||
loadConfigData: (
|
||||
options?: LoadInteractiveConfigDataOptions,
|
||||
) => Promise<InteractiveConfigData>;
|
||||
onToggleConfigItem?: (
|
||||
item: InteractiveConfigItem,
|
||||
options?: LoadInteractiveConfigDataOptions,
|
||||
) => Promise<InteractiveConfigData | undefined>;
|
||||
onDeleteConfigItem?: (
|
||||
item: InteractiveConfigItem,
|
||||
options?: LoadInteractiveConfigDataOptions,
|
||||
) => Promise<InteractiveConfigData | undefined>;
|
||||
openModelSelector: (options?: OpenModelSelectorOptions) => Promise<void>;
|
||||
openMcpManager: (options?: { refocus?: boolean }) => Promise<boolean>;
|
||||
refocusTextarea: () => void;
|
||||
}) {
|
||||
const emptyConfigData = useMemo(
|
||||
() => ({
|
||||
workflows: [] as InteractiveConfigItem[],
|
||||
rules: [] as InteractiveConfigItem[],
|
||||
skills: [] as InteractiveConfigItem[],
|
||||
hooks: [] as InteractiveConfigItem[],
|
||||
agents: [] as InteractiveConfigItem[],
|
||||
plugins: [] as InteractiveConfigItem[],
|
||||
mcp: [] as InteractiveConfigItem[],
|
||||
tools: [] as InteractiveConfigItem[],
|
||||
workflowSlashCommands: [],
|
||||
}),
|
||||
[],
|
||||
);
|
||||
|
||||
const openConfig = useCallback(
|
||||
async (options: OpenConfigOptions = {}) => {
|
||||
let keepOpen = true;
|
||||
let activeTab = options.initialTab;
|
||||
while (keepOpen) {
|
||||
const [data, providerInfo] = await withLoadingDialog(
|
||||
opts.dialog,
|
||||
"Loading settings...",
|
||||
async () =>
|
||||
await Promise.all([
|
||||
opts
|
||||
.loadConfigData({ includePluginTools: false })
|
||||
.catch(() => emptyConfigData),
|
||||
Llms.getProvider(opts.config.providerId).catch(() => undefined),
|
||||
]),
|
||||
);
|
||||
const providerDisplayName =
|
||||
providerInfo?.name ?? opts.config.providerId;
|
||||
const action = await opts.dialog.choice<ConfigAction>({
|
||||
size: "large",
|
||||
style: { maxHeight: opts.termHeight - 2 },
|
||||
closeOnEscape: false,
|
||||
content: (ctx: ChoiceContext<ConfigAction>) => (
|
||||
<ConfigPanelContent
|
||||
{...ctx}
|
||||
config={opts.config}
|
||||
configData={data}
|
||||
loadConfigData={opts.loadConfigData}
|
||||
providerDisplayName={providerDisplayName}
|
||||
currentMode={opts.sessionUiMode}
|
||||
currentCompactionMode={opts.compactionMode}
|
||||
initialTab={activeTab}
|
||||
onActiveTabChange={(tab) => {
|
||||
activeTab = tab;
|
||||
}}
|
||||
onToggleConfigItem={opts.onToggleConfigItem}
|
||||
onDeleteConfigItem={opts.onDeleteConfigItem}
|
||||
onToggleMode={opts.toggleMode}
|
||||
onToggleAutoApprove={opts.toggleAutoApprove}
|
||||
onSetCompactionMode={opts.setCompactionMode}
|
||||
/>
|
||||
),
|
||||
});
|
||||
|
||||
if (!action) {
|
||||
keepOpen = false;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (action.kind === "open-provider") {
|
||||
await opts.openModelSelector({
|
||||
startWithProviderChange: true,
|
||||
onCancel: () => {},
|
||||
});
|
||||
} else if (action.kind === "open-model") {
|
||||
await opts.openModelSelector({ onCancel: () => {} });
|
||||
} else if (action.kind === "toggle-item") {
|
||||
await opts.onToggleConfigItem?.(action.item);
|
||||
} else if (action.kind === "delete-item") {
|
||||
const confirmed = await opts.dialog.choice<boolean>({
|
||||
closeOnEscape: true,
|
||||
content: (ctx: ChoiceContext<boolean>) => (
|
||||
<DeleteConfigItemConfirmContent {...ctx} item={action.item} />
|
||||
),
|
||||
});
|
||||
if (confirmed && opts.onDeleteConfigItem) {
|
||||
try {
|
||||
await withLoadingDialog(
|
||||
opts.dialog,
|
||||
`Deleting ${action.item.name}...`,
|
||||
async () =>
|
||||
await opts.onDeleteConfigItem?.(action.item, {
|
||||
includePluginTools: false,
|
||||
}),
|
||||
);
|
||||
} catch (error) {
|
||||
await opts.dialog.choice<void>({
|
||||
closeOnEscape: true,
|
||||
content: (ctx: ChoiceContext<void>) => (
|
||||
<ConfigErrorContent
|
||||
{...ctx}
|
||||
title="Plugin delete failed"
|
||||
message={
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}
|
||||
/>
|
||||
),
|
||||
});
|
||||
}
|
||||
}
|
||||
} else if (action.kind === "ext-detail") {
|
||||
await opts.dialog.choice<void>({
|
||||
style: { maxHeight: opts.termHeight - 2 },
|
||||
closeOnEscape: false,
|
||||
content: (ctx: ChoiceContext<void>) => (
|
||||
<ExtDetailContent
|
||||
{...ctx}
|
||||
item={action.item}
|
||||
onToggleConfigItem={opts.onToggleConfigItem}
|
||||
/>
|
||||
),
|
||||
});
|
||||
} else if (action.kind === "open-mcp") {
|
||||
const changed = await opts.openMcpManager({ refocus: false });
|
||||
if (changed) {
|
||||
keepOpen = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
opts.refocusTextarea();
|
||||
},
|
||||
[opts, emptyConfigData],
|
||||
);
|
||||
|
||||
return openConfig;
|
||||
}
|
||||
@@ -1,81 +0,0 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { PLATFORMS, shouldIncludeField } from "./platforms";
|
||||
|
||||
describe("connect wizard platform security fields", () => {
|
||||
it("does not ask Telegram users to re-enter the bot username", () => {
|
||||
const telegram = PLATFORMS.find((platform) => platform.id === "telegram");
|
||||
|
||||
expect(telegram?.fields.map((field) => field.label)).toEqual(["Bot token"]);
|
||||
expect(telegram?.fields.map((field) => field.flag)).toEqual(["-k"]);
|
||||
});
|
||||
|
||||
it("rejects unsafe Telegram and Slack access restriction identifiers", () => {
|
||||
const telegram = PLATFORMS.find((platform) => platform.id === "telegram");
|
||||
const slack = PLATFORMS.find((platform) => platform.id === "slack");
|
||||
|
||||
const telegramUser = telegram?.security?.fields.find(
|
||||
(field) => field.key === "userId",
|
||||
);
|
||||
const slackTeam = slack?.security?.fields.find(
|
||||
(field) => field.key === "teamId",
|
||||
);
|
||||
const slackUser = slack?.security?.fields.find(
|
||||
(field) => field.key === "userId",
|
||||
);
|
||||
|
||||
expect(telegramUser?.validate?.("123456")).toBeUndefined();
|
||||
expect(telegramUser?.validate?.("123; rm -rf /")).toContain("digits");
|
||||
expect(slackTeam?.validate?.("T01ABC123")).toBeUndefined();
|
||||
expect(slackTeam?.validate?.("T01;bad")).toContain("Slack workspace");
|
||||
expect(slackUser?.validate?.("U01ABC123")).toBeUndefined();
|
||||
expect(slackUser?.validate?.("U01$(bad)")).toContain("Slack member");
|
||||
});
|
||||
|
||||
it("uses the Telegram allowed user ID flag for wizard security", () => {
|
||||
const telegram = PLATFORMS.find((platform) => platform.id === "telegram");
|
||||
|
||||
const args = telegram?.security?.buildArgs({
|
||||
userId: "123456",
|
||||
});
|
||||
|
||||
expect(args).toEqual(["--allowed-user-id", "123456"]);
|
||||
});
|
||||
|
||||
it("builds an exact-match Slack authorization hook", () => {
|
||||
const slack = PLATFORMS.find((platform) => platform.id === "slack");
|
||||
|
||||
const args = slack?.security?.buildArgs({
|
||||
teamId: "T01ABC123",
|
||||
userId: "U01ABC123",
|
||||
});
|
||||
|
||||
expect(args).toEqual([
|
||||
"--hook-command",
|
||||
`jq -r ".payload.actor.participantKey" | grep -qx "slack:team:T01ABC123:user:U01ABC123" && echo '{"action":"allow"}' || echo '{"action":"deny"}'`,
|
||||
]);
|
||||
});
|
||||
|
||||
it("asks Slack users for mode-specific setup fields", () => {
|
||||
const slack = PLATFORMS.find((platform) => platform.id === "slack");
|
||||
const fields = slack?.fields ?? [];
|
||||
const webhookValues = { "--base-url": "https://example.test" };
|
||||
const socketValues = { "--base-url": "" };
|
||||
|
||||
expect(fields.map((field) => field.flag)).toEqual([
|
||||
"--bot-token",
|
||||
"--base-url",
|
||||
"--signing-secret",
|
||||
"--app-token",
|
||||
]);
|
||||
expect(
|
||||
fields
|
||||
.filter((field) => shouldIncludeField(field, webhookValues))
|
||||
.map((field) => field.flag),
|
||||
).toEqual(["--bot-token", "--base-url", "--signing-secret"]);
|
||||
expect(
|
||||
fields
|
||||
.filter((field) => shouldIncludeField(field, socketValues))
|
||||
.map((field) => field.flag),
|
||||
).toEqual(["--bot-token", "--base-url", "--app-token"]);
|
||||
});
|
||||
});
|
||||
@@ -1,255 +0,0 @@
|
||||
import { CORE_BUILD_VERSION } from "@cline/core";
|
||||
import { isNonLocalBindHost } from "./options";
|
||||
import {
|
||||
handleToolApprovalResponse,
|
||||
rejectOrphanedApprovals,
|
||||
} from "./server/approvals";
|
||||
import {
|
||||
browserConfig,
|
||||
host,
|
||||
inviteUrl,
|
||||
port,
|
||||
publicUrl,
|
||||
roomSecret,
|
||||
webviewDistDir,
|
||||
} from "./server/deps";
|
||||
import { handleDesktopCommand } from "./server/desktop-commands";
|
||||
import { createJsonResponse, WebviewAssets } from "./server/http";
|
||||
import {
|
||||
attachHub,
|
||||
detachHub,
|
||||
restartHub,
|
||||
syncHubClientsAndSessions,
|
||||
syncHubHealth,
|
||||
} from "./server/hub";
|
||||
import {
|
||||
loadModels,
|
||||
runProviderOAuthLogin,
|
||||
saveProviderSettings,
|
||||
sendProviderCatalog,
|
||||
} from "./server/providers";
|
||||
import {
|
||||
abortPeerTurn,
|
||||
deleteSession,
|
||||
forkPeerSession,
|
||||
initializePeer,
|
||||
resetPeer,
|
||||
restorePeerSession,
|
||||
selectSession,
|
||||
sendMessage,
|
||||
} from "./server/sessions";
|
||||
import { HubContext } from "./server/state";
|
||||
import { broadcastHubState, hubStatusPayload } from "./server/state-payloads";
|
||||
import type { BrowserFrame, BrowserPeer } from "./server/types";
|
||||
|
||||
export interface ClineHubDashboardServer {
|
||||
listenUrl: string;
|
||||
publicUrl: string;
|
||||
inviteUrl: string;
|
||||
bindHost: string;
|
||||
inviteRequired: boolean;
|
||||
hubUrl: string | undefined;
|
||||
stop: () => Promise<void>;
|
||||
}
|
||||
|
||||
export async function startClineHubDashboardServer(): Promise<ClineHubDashboardServer> {
|
||||
const ctx = new HubContext();
|
||||
const assets = new WebviewAssets(webviewDistDir);
|
||||
const syncClientsAndSessions = () => syncHubClientsAndSessions(ctx);
|
||||
let stopped = false;
|
||||
|
||||
function isAuthorizedBrowserRequest(url: URL): boolean {
|
||||
if (!roomSecret) return true;
|
||||
return url.searchParams.get("roomSecret") === roomSecret;
|
||||
}
|
||||
|
||||
await attachHub(ctx);
|
||||
const healthInterval = setInterval(() => {
|
||||
void (async () => {
|
||||
await syncHubHealth(ctx);
|
||||
broadcastHubState(ctx);
|
||||
})();
|
||||
}, 5_000);
|
||||
|
||||
const server = Bun.serve<BrowserPeer>({
|
||||
port,
|
||||
hostname: host,
|
||||
async fetch(req, server) {
|
||||
const url = new URL(req.url);
|
||||
if (url.pathname === "/version") {
|
||||
return createJsonResponse({ coreVersion: CORE_BUILD_VERSION });
|
||||
}
|
||||
if (url.pathname === "/health") {
|
||||
await syncHubHealth(ctx);
|
||||
return createJsonResponse(hubStatusPayload(ctx));
|
||||
}
|
||||
if (url.pathname === "/browser") {
|
||||
if (!isAuthorizedBrowserRequest(url)) {
|
||||
return createJsonResponse({ error: "invalid_room_secret" }, 401);
|
||||
}
|
||||
const displayName = `Browser ${Math.random().toString(36).slice(2, 6)}`;
|
||||
const data = {
|
||||
socket: undefined as never,
|
||||
displayName,
|
||||
sending: false,
|
||||
};
|
||||
if (server.upgrade(req, { data })) return undefined;
|
||||
return new Response("upgrade failed", { status: 400 });
|
||||
}
|
||||
if (url.pathname === "/config.json") {
|
||||
return createJsonResponse(browserConfig);
|
||||
}
|
||||
return assets.serve(url.pathname);
|
||||
},
|
||||
websocket: {
|
||||
async open(socket) {
|
||||
const peer = socket.data;
|
||||
peer.socket = socket;
|
||||
ctx.peers.add(peer);
|
||||
},
|
||||
async message(socket, raw) {
|
||||
const peer = socket.data;
|
||||
try {
|
||||
const frame = JSON.parse(String(raw)) as BrowserFrame;
|
||||
if (frame.type === "desktopCommand") {
|
||||
try {
|
||||
const result = await handleDesktopCommand(
|
||||
ctx,
|
||||
frame.command,
|
||||
frame.args,
|
||||
);
|
||||
ctx.send(peer, {
|
||||
type: "desktopCommandResult",
|
||||
id: frame.id,
|
||||
ok: true,
|
||||
result,
|
||||
});
|
||||
} catch (error) {
|
||||
ctx.send(peer, {
|
||||
type: "desktopCommandResult",
|
||||
id: frame.id,
|
||||
ok: false,
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
});
|
||||
}
|
||||
} else if (frame.type === "ready") {
|
||||
await initializePeer(ctx, peer, syncClientsAndSessions);
|
||||
} else if (frame.type === "loadModels") {
|
||||
await loadModels(ctx, peer, frame.providerId);
|
||||
} else if (frame.type === "loadProviderCatalog") {
|
||||
await sendProviderCatalog(ctx, peer);
|
||||
} else if (frame.type === "saveProviderSettings") {
|
||||
await saveProviderSettings(ctx, peer, frame);
|
||||
} else if (frame.type === "runProviderOAuthLogin") {
|
||||
await runProviderOAuthLogin(ctx, peer, frame.providerId);
|
||||
} else if (frame.type === "attachSession") {
|
||||
await selectSession(ctx, peer, frame.sessionId);
|
||||
} else if (frame.type === "deleteSession") {
|
||||
await deleteSession(ctx, peer, frame.sessionId);
|
||||
} else if (frame.type === "updateSessionMetadata") {
|
||||
if (!ctx.cline) throw new Error("Hub is not connected.");
|
||||
const session = await ctx.cline.get(frame.sessionId);
|
||||
const metadata =
|
||||
session?.metadata && typeof session.metadata === "object"
|
||||
? (session.metadata as Record<string, unknown>)
|
||||
: {};
|
||||
await ctx.cline.update(frame.sessionId, {
|
||||
metadata: { ...metadata, ...frame.metadata },
|
||||
});
|
||||
await syncHubClientsAndSessions(ctx);
|
||||
broadcastHubState(ctx);
|
||||
} else if (frame.type === "approval_response") {
|
||||
handleToolApprovalResponse(ctx, frame);
|
||||
} else if (frame.type === "abort") {
|
||||
await abortPeerTurn(ctx, peer);
|
||||
} else if (frame.type === "reset") {
|
||||
await resetPeer(ctx, peer);
|
||||
} else if (frame.type === "send") {
|
||||
if (peer.sending) {
|
||||
ctx.send(peer, {
|
||||
type: "status",
|
||||
text: "A turn is already in progress.",
|
||||
});
|
||||
return;
|
||||
}
|
||||
peer.sending = true;
|
||||
try {
|
||||
await sendMessage(
|
||||
ctx,
|
||||
peer,
|
||||
frame.prompt,
|
||||
frame.config,
|
||||
frame.attachments,
|
||||
);
|
||||
} finally {
|
||||
peer.sending = false;
|
||||
}
|
||||
} else if (frame.type === "forkSession") {
|
||||
await forkPeerSession(ctx, peer, syncClientsAndSessions);
|
||||
} else if (frame.type === "restore") {
|
||||
await restorePeerSession(
|
||||
ctx,
|
||||
peer,
|
||||
frame.checkpointRunCount,
|
||||
syncClientsAndSessions,
|
||||
);
|
||||
} else if (frame.type === "restart_hub") {
|
||||
await restartHub(ctx);
|
||||
}
|
||||
} catch (error) {
|
||||
ctx.send(peer, {
|
||||
type: "error",
|
||||
text: error instanceof Error ? error.message : String(error),
|
||||
});
|
||||
}
|
||||
},
|
||||
close(socket) {
|
||||
const peer = socket.data;
|
||||
peer.unsubscribeEvents?.();
|
||||
ctx.peers.delete(peer);
|
||||
rejectOrphanedApprovals(ctx);
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
listenUrl: server.url.toString(),
|
||||
publicUrl,
|
||||
inviteUrl,
|
||||
bindHost: host,
|
||||
inviteRequired: Boolean(roomSecret),
|
||||
hubUrl: ctx.hubUrl,
|
||||
stop: async () => {
|
||||
if (stopped) return;
|
||||
stopped = true;
|
||||
clearInterval(healthInterval);
|
||||
try {
|
||||
server.stop(true);
|
||||
} finally {
|
||||
await detachHub(ctx);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function printClineHubDashboardServerInfo(
|
||||
server: ClineHubDashboardServer,
|
||||
): void {
|
||||
console.log(`Cline Hub dashboard listening: ${server.listenUrl}`);
|
||||
console.log(`Cline Hub public URL: ${server.publicUrl}`);
|
||||
console.log(`hub endpoint: ${server.hubUrl}`);
|
||||
if (server.inviteRequired) {
|
||||
console.log(`Cline Hub invite URL: ${server.inviteUrl}`);
|
||||
} else if (isNonLocalBindHost(server.bindHost)) {
|
||||
console.warn("WARNING: non-local bind without ROOM_SECRET is not allowed.");
|
||||
} else {
|
||||
console.log(
|
||||
"ROOM_SECRET is not set; this local-only instance accepts browser connections without an invite token.",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (import.meta.main) {
|
||||
const server = await startClineHubDashboardServer();
|
||||
printClineHubDashboardServerInfo(server);
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"extends": "../../sdk/packages/tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"noEmit": true,
|
||||
"types": ["bun", "node"],
|
||||
"paths": {
|
||||
"@cline/core": ["../../sdk/packages/core/src/index.ts"],
|
||||
"@cline/core/*": [
|
||||
"../../sdk/packages/core/src/*",
|
||||
"../../sdk/packages/core/src/*/index.ts"
|
||||
],
|
||||
"@cline/shared": ["../../sdk/packages/shared/src/index.ts"],
|
||||
"@cline/shared/*": [
|
||||
"../../sdk/packages/shared/src/*",
|
||||
"../../sdk/packages/shared/src/*/index.ts"
|
||||
]
|
||||
}
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["src/webview/**"]
|
||||
}
|
||||
@@ -1,23 +0,0 @@
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"paths": {
|
||||
"@/*": ["./*"],
|
||||
"@cline/agents": ["../../../sdk/packages/agents/src/index.ts"],
|
||||
"@cline/core": ["../../../sdk/packages/core/src/index.ts"],
|
||||
"@cline/core/hub": ["../../../sdk/packages/core/src/hub/index.ts"],
|
||||
"@cline/llms": ["../../../sdk/packages/llms/src/index.ts"],
|
||||
"@cline/shared": ["../../../sdk/packages/shared/src/index.ts"],
|
||||
"@cline/shared/storage": [
|
||||
"../../../sdk/packages/shared/src/storage/index.ts"
|
||||
],
|
||||
"@cline/shared/db": ["../../../sdk/packages/shared/src/db/index.ts"],
|
||||
"@cline/shared/*": [
|
||||
"../../../sdk/packages/shared/src/*",
|
||||
"../../../sdk/packages/shared/src/*/index.ts"
|
||||
]
|
||||
}
|
||||
},
|
||||
"include": ["sidecar/**/*.ts", "scripts/**/*.ts", "global.d.ts", "bun.mts"],
|
||||
"exclude": ["node_modules", "webview"]
|
||||
}
|
||||
@@ -1,28 +0,0 @@
|
||||
{
|
||||
"extends": "../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"paths": {
|
||||
"@/*": ["./*"],
|
||||
"@cline/agents": ["../../../../sdk/packages/agents/src/index.ts"],
|
||||
"@cline/core": ["../../../../sdk/packages/core/src/index.ts"],
|
||||
"@cline/llms": ["../../../../sdk/packages/llms/src/index.ts"],
|
||||
"@cline/shared": ["../../../../sdk/packages/shared/src/index.ts"],
|
||||
"@cline/shared/storage": [
|
||||
"../../../../sdk/packages/shared/src/storage/index.ts"
|
||||
],
|
||||
"@cline/shared/db": ["../../../../sdk/packages/shared/src/db/index.ts"],
|
||||
"@cline/shared/*": [
|
||||
"../../../../sdk/packages/shared/src/*",
|
||||
"../../../../sdk/packages/shared/src/*/index.ts"
|
||||
]
|
||||
}
|
||||
},
|
||||
"include": [
|
||||
"next-env.d.ts",
|
||||
"**/*.ts",
|
||||
"**/*.tsx",
|
||||
".next/types/**/*.ts",
|
||||
".next/dev/types/**/*.ts"
|
||||
],
|
||||
"exclude": ["node_modules"]
|
||||
}
|
||||
@@ -1,35 +0,0 @@
|
||||
{
|
||||
"extends": "../../../sdk/packages/tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"outDir": "dist",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"target": "ESNext",
|
||||
"paths": {
|
||||
"@cline/agents": ["../../../sdk/packages/agents/src/index.ts"],
|
||||
"@cline/agents/*": [
|
||||
"../../../sdk/packages/agents/src/*",
|
||||
"../../../sdk/packages/agents/src/*/index.ts"
|
||||
],
|
||||
"@cline/core": ["../../../sdk/packages/core/src/index.ts"],
|
||||
"@cline/core/hub/daemon-entry": [
|
||||
"../../../sdk/packages/core/src/hub/daemon/entry.ts"
|
||||
],
|
||||
"@cline/core/*": [
|
||||
"../../../sdk/packages/core/src/*",
|
||||
"../../../sdk/packages/core/src/*/index.ts"
|
||||
],
|
||||
"@cline/llms": ["../../../sdk/packages/llms/src/index.ts"],
|
||||
"@cline/shared": ["../../../sdk/packages/shared/src/index.ts"],
|
||||
"@cline/shared/storage": [
|
||||
"../../../sdk/packages/shared/src/storage/index.ts"
|
||||
],
|
||||
"@cline/shared/db": ["../../../sdk/packages/shared/src/db/index.ts"],
|
||||
"@cline/shared/*": [
|
||||
"../../../sdk/packages/shared/src/*",
|
||||
"../../../sdk/packages/shared/src/*/index.ts"
|
||||
]
|
||||
}
|
||||
},
|
||||
"include": ["sidecar/**/*.ts"]
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"paths": {
|
||||
"@cline/agents": ["../sdk/packages/agents/src/index.ts"],
|
||||
"@cline/agents/*": [
|
||||
"../sdk/packages/agents/src/*",
|
||||
"../sdk/packages/agents/src/*/index.ts"
|
||||
],
|
||||
"@cline/cline-hub": ["./cline-hub/src/server.ts"],
|
||||
"@cline/core": ["../sdk/packages/core/src/index.ts"],
|
||||
"@cline/core/hub/daemon-entry": [
|
||||
"../sdk/packages/core/src/hub/daemon/entry.ts"
|
||||
],
|
||||
"@cline/core/*": [
|
||||
"../sdk/packages/core/src/*",
|
||||
"../sdk/packages/core/src/*/index.ts"
|
||||
],
|
||||
"@cline/core/telemetry": [
|
||||
"../sdk/packages/core/src/services/telemetry/index.ts"
|
||||
],
|
||||
"@cline/llms": ["../sdk/packages/llms/src/index.ts"],
|
||||
"@cline/shared": ["../sdk/packages/shared/src/index.ts"],
|
||||
"@cline/shared/storage": ["../sdk/packages/shared/src/storage/index.ts"],
|
||||
"@cline/shared/db": ["../sdk/packages/shared/src/db/index.ts"],
|
||||
"@cline/shared/*": [
|
||||
"../sdk/packages/shared/src/*",
|
||||
"../sdk/packages/shared/src/*/index.ts"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,13 @@
|
||||
"src/**/__tests__/*.ts",
|
||||
"src/test/services/**/*.test.ts"
|
||||
],
|
||||
"ignore": [
|
||||
"src/core/controller/models/__tests__/providerCatalogHandlers.test.ts",
|
||||
"src/core/controller/models/__tests__/providerCatalogSmoke.test.ts",
|
||||
"src/core/controller/models/__tests__/providerSwitchNormalization.test.ts",
|
||||
"src/core/controller/models/__tests__/resolveModelInfo.test.ts",
|
||||
"src/core/controller/models/__tests__/refreshClineRecommendedModels.test.ts"
|
||||
],
|
||||
"require": [
|
||||
"ts-node/register",
|
||||
"source-map-support/register",
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
import { defineConfig } from "@vscode/test-cli"
|
||||
import path from "path"
|
||||
|
||||
const vscodeTestVersion = process.env.VSCODE_TEST_VERSION ?? "stable"
|
||||
|
||||
export default defineConfig({
|
||||
files: "{out/**/*.test.js,src/**/*.test.js,!src/test/e2e/**/*.test.js,!out/src/test/e2e/**/*.test.js}",
|
||||
files: [
|
||||
"out/src/{core,test,utils,shared,integrations,hosts,services}/**/*.test.js",
|
||||
"src/{core,test,utils,shared,integrations,hosts,services}/**/*.test.js",
|
||||
],
|
||||
mocha: {
|
||||
ui: "bdd",
|
||||
timeout: 20000, // Maximum time (in ms) that a test can run before failing
|
||||
|
||||
@@ -1,201 +0,0 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright 2026 Cline Bot Inc.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
+12
-5
@@ -122,7 +122,6 @@
|
||||
"!!**/out",
|
||||
"!!**/evals",
|
||||
"!!**/playwright",
|
||||
"!!**/.vscode-test",
|
||||
"!!**/test-results",
|
||||
"!!**/node_modules",
|
||||
"!!**/webview-ui/build",
|
||||
@@ -131,7 +130,9 @@
|
||||
"!!**/tests/specs"
|
||||
]
|
||||
},
|
||||
"plugins": ["src/dev/grit/process-env.grit"],
|
||||
"plugins": [
|
||||
"src/dev/grit/process-env.grit"
|
||||
],
|
||||
"overrides": [
|
||||
{
|
||||
"includes": [
|
||||
@@ -146,11 +147,15 @@
|
||||
"!!src/integrations/terminal/**",
|
||||
"!!src/core/controller/ui/openWalkthrough.ts"
|
||||
],
|
||||
"plugins": ["src/dev/grit/vscode-api.grit"]
|
||||
"plugins": [
|
||||
"src/dev/grit/vscode-api.grit"
|
||||
]
|
||||
},
|
||||
{
|
||||
// Do not use console logging directly, use the Logger service instead.
|
||||
"plugins": ["src/dev/grit/console-log.grit"],
|
||||
"plugins": [
|
||||
"src/dev/grit/console-log.grit"
|
||||
],
|
||||
"includes": [
|
||||
"**",
|
||||
"!!**/esbuild.*",
|
||||
@@ -183,7 +188,9 @@
|
||||
"!!src/core/storage/utils/state-helpers.ts",
|
||||
"!!src/extension.ts"
|
||||
],
|
||||
"plugins": ["src/dev/grit/use-cache-service.grit"]
|
||||
"plugins": [
|
||||
"src/dev/grit/use-cache-service.grit"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -85,44 +85,6 @@ const esbuildProblemMatcherPlugin = {
|
||||
},
|
||||
}
|
||||
|
||||
const copyWasmFiles = {
|
||||
name: "copy-wasm-files",
|
||||
setup(build) {
|
||||
build.onEnd(() => {
|
||||
// tree sitter
|
||||
const sourceDir = path.join(__dirname, "node_modules", "web-tree-sitter")
|
||||
const targetDir = path.join(__dirname, destDir)
|
||||
|
||||
// Copy tree-sitter.wasm
|
||||
fs.copyFileSync(path.join(sourceDir, "tree-sitter.wasm"), path.join(targetDir, "tree-sitter.wasm"))
|
||||
|
||||
// Copy language-specific WASM files
|
||||
const languageWasmDir = path.join(__dirname, "node_modules", "tree-sitter-wasms", "out")
|
||||
const languages = [
|
||||
"typescript",
|
||||
"tsx",
|
||||
"python",
|
||||
"rust",
|
||||
"javascript",
|
||||
"go",
|
||||
"cpp",
|
||||
"c",
|
||||
"c_sharp",
|
||||
"ruby",
|
||||
"java",
|
||||
"php",
|
||||
"swift",
|
||||
"kotlin",
|
||||
]
|
||||
|
||||
languages.forEach((lang) => {
|
||||
const filename = `tree-sitter-${lang}.wasm`
|
||||
fs.copyFileSync(path.join(languageWasmDir, filename), path.join(targetDir, filename))
|
||||
})
|
||||
})
|
||||
},
|
||||
}
|
||||
|
||||
const buildEnvVars = {
|
||||
"import.meta.url": "_importMetaUrl",
|
||||
"process.env.IS_STANDALONE": JSON.stringify(standalone ? "true" : "false"),
|
||||
@@ -176,7 +138,6 @@ const baseConfig = {
|
||||
define: buildEnvVars,
|
||||
tsconfig: path.resolve(__dirname, "tsconfig.json"),
|
||||
plugins: [
|
||||
copyWasmFiles,
|
||||
aliasResolverPlugin,
|
||||
/* add to the end of plugins array */
|
||||
esbuildProblemMatcherPlugin,
|
||||
|
||||
Generated
+14907
-14960
File diff suppressed because it is too large
Load Diff
+22
-22
@@ -2,8 +2,11 @@
|
||||
"name": "claude-dev",
|
||||
"displayName": "Cline",
|
||||
"description": "Autonomous coding agent right in your IDE, capable of creating/editing files, running commands, using the browser, and more with your permission every step of the way.",
|
||||
"version": "3.88.0",
|
||||
"version": "3.86.2",
|
||||
"icon": "assets/icons/icon.png",
|
||||
"workspaces": [
|
||||
"."
|
||||
],
|
||||
"engines": {
|
||||
"vscode": "^1.84.0"
|
||||
},
|
||||
@@ -389,24 +392,26 @@
|
||||
"protos": "node scripts/build-proto.mjs",
|
||||
"protos-python": "node scripts/build-python-proto.mjs",
|
||||
"download-ripgrep": "node scripts/download-ripgrep.mjs",
|
||||
"postprotos": "biome format --config-path ./biome.jsonc src/shared/proto src/core/controller src/hosts/ webview-ui/src/services src/generated --write --no-errors-on-unmatched",
|
||||
"postprotos": "biome format src/shared/proto src/core/controller src/hosts/ webview-ui/src/services src/generated --write --no-errors-on-unmatched",
|
||||
"clean:build": "rimraf dist dist-standalone webview-ui/build src/generated out/",
|
||||
"clean:deps": "rimraf node_modules webview-ui/node_modules",
|
||||
"clean:all": "npm run clean:build && npm run clean:deps",
|
||||
"compile-tests": "node ./scripts/build-tests.js",
|
||||
"watch-tests": "tsc -p . -w --outDir out",
|
||||
"check-types": "npm run protos && tsc --noEmit && cd webview-ui && tsc --noEmit",
|
||||
"lint": "biome lint --config-path ./biome.jsonc --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error && npm run lint:proto",
|
||||
"check-types": "npm run protos && npx tsc --noEmit && cd webview-ui && npx tsc --noEmit",
|
||||
"lint": "biome lint --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error && npm run lint:proto",
|
||||
"lint:proto": "bash ./scripts/proto-lint.sh",
|
||||
"format": "biome format --config-path ./biome.jsonc --changed --since main --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error",
|
||||
"format:fix": "biome check --config-path ./biome.jsonc --changed --since main --no-errors-on-unmatched --files-ignore-unknown=true --write",
|
||||
"fix:all": "biome check --config-path ./biome.jsonc --no-errors-on-unmatched --files-ignore-unknown=true --write --diagnostic-level=error --unsafe",
|
||||
"format": "biome format --changed --no-errors-on-unmatched --files-ignore-unknown=true --diagnostic-level=error",
|
||||
"format:fix": "biome check --changed --no-errors-on-unmatched --files-ignore-unknown=true --write",
|
||||
"fix:all": "biome check --no-errors-on-unmatched --files-ignore-unknown=true --write --diagnostic-level=error --unsafe",
|
||||
"ci:check-all": "npx npm-run-all -p check-types lint format",
|
||||
"ci:build": "npm run protos && npm run build:webview && node esbuild.mjs && npm run compile-tests",
|
||||
"pretest": "npm run compile && npm run compile-tests && npm run compile-standalone && npm run lint",
|
||||
"test": "npx npm-run-all test:unit test:integration",
|
||||
"test:integration": "vscode-test",
|
||||
"test:unit": "cross-env TS_NODE_PROJECT=./tsconfig.unit-test.json mocha",
|
||||
"test:vitest": "vitest run --config vitest.config.ts",
|
||||
"test:vitest:watch": "vitest --config vitest.config.ts",
|
||||
"test:coverage": "vscode-test --coverage",
|
||||
"test:sca-server": "npx tsx watch scripts/test-standalone-core-api-server.ts",
|
||||
"test:tp-orchestrator": "npx tsx scripts/testing-platform-orchestrator.ts",
|
||||
@@ -479,25 +484,24 @@
|
||||
"rimraf": "^6.0.1",
|
||||
"should": "^13.2.3",
|
||||
"sinon": "^21.0.3",
|
||||
"tar": "^7.5.2",
|
||||
"tree-kill": "^1.2.2",
|
||||
"ts-node": "^10.9.2",
|
||||
"ts-proto": "^2.6.1",
|
||||
"tsconfig-paths": "^4.2.0",
|
||||
"typescript": "^5.4.5"
|
||||
"typescript": "^5.4.5",
|
||||
"vitest": "^4.0.17"
|
||||
},
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sdk": "^0.37.0",
|
||||
"@anthropic-ai/vertex-sdk": "^0.6.4",
|
||||
"@aws-sdk/client-bedrock-runtime": "^3.922.0",
|
||||
"@aws-sdk/credential-providers": "^3.922.0",
|
||||
"@azure/identity": "^4.13.0",
|
||||
"@bufbuild/protobuf": "^2.2.5",
|
||||
"@cerebras/cerebras_cloud_sdk": "^1.35.0",
|
||||
"@google-cloud/vertexai": "^1.9.3",
|
||||
"@cline/agents": "^0.0.42",
|
||||
"@cline/core": "^0.0.42",
|
||||
"@cline/llms": "^0.0.42",
|
||||
"@cline/shared": "^0.0.42",
|
||||
"@google/genai": "^1.30.0",
|
||||
"@grpc/grpc-js": "^1.9.15",
|
||||
"@grpc/reflection": "^1.0.4",
|
||||
"@mistralai/mistralai": "^1.5.0",
|
||||
"@modelcontextprotocol/sdk": "^1.25.1",
|
||||
"@opentelemetry/api": "^1.9.0",
|
||||
"@opentelemetry/core": "^2.1.0",
|
||||
@@ -519,9 +523,6 @@
|
||||
"@opentelemetry/sdk-trace-node": "^1.30.1",
|
||||
"@opentelemetry/semantic-conventions": "^1.37.0",
|
||||
"@playwright/test": "^1.55.1",
|
||||
"@sap-ai-sdk/ai-api": "^2.7.0",
|
||||
"@sap-ai-sdk/orchestration": "^2.7.0",
|
||||
"@sap-cloud-sdk/connectivity": "^4.6.0",
|
||||
"@streamparser/json": "^0.0.22",
|
||||
"@tailwindcss/vite": "^4.1.14",
|
||||
"@types/uuid": "^10.0.0",
|
||||
@@ -547,13 +548,14 @@
|
||||
"ignore": "^7.0.3",
|
||||
"image-size": "^2.0.2",
|
||||
"isbinaryfile": "^5.0.2",
|
||||
"js-yaml": "^4.1.1",
|
||||
"jschardet": "^3.1.4",
|
||||
"json5": "^2.2.3",
|
||||
"jwt-decode": "^4.0.0",
|
||||
"mammoth": "^1.11.0",
|
||||
"nanoid": "^5.1.6",
|
||||
"nice-grpc": "^2.1.12",
|
||||
"node-machine-id": "^1.1.12",
|
||||
"ollama": "^0.5.13",
|
||||
"open": "^10.1.2",
|
||||
"open-graph-scraper": "^6.9.0",
|
||||
"openai": "^6.21.0",
|
||||
@@ -572,15 +574,13 @@
|
||||
"simple-git": "3.36.0",
|
||||
"strip-ansi": "^7.1.2",
|
||||
"tailwindcss": "^4.1.14",
|
||||
"tree-sitter-wasms": "^0.1.11",
|
||||
"ts-morph": "^25.0.1",
|
||||
"turndown": "^7.2.0",
|
||||
"ulid": "^2.4.0",
|
||||
"undici": "^7.26.0",
|
||||
"uuid": "^11.1.0",
|
||||
"vscode-uri": "^3.1.0",
|
||||
"web-tree-sitter": "^0.22.6",
|
||||
"zod": "^3.24.2"
|
||||
"zod": "^4.3.6"
|
||||
},
|
||||
"overrides": {
|
||||
"tar-fs": ">=3.1.1",
|
||||
|
||||
@@ -21,8 +21,6 @@ service ModelsService {
|
||||
rpc refreshOpenRouterModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
|
||||
// Refreshes and returns recommended and free Cline models
|
||||
rpc refreshClineRecommendedModelsRpc(EmptyRequest) returns (ClineRecommendedModelsResponse);
|
||||
// Refreshes and returns Cline provider models
|
||||
rpc refreshClineModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
|
||||
// Refreshes and returns Hugging Face models
|
||||
rpc refreshHuggingFaceModels(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
|
||||
// Refreshes and returns OpenAI models
|
||||
@@ -55,6 +53,18 @@ service ModelsService {
|
||||
rpc getAihubmixModels(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
|
||||
// Refreshes and returns Vercel AI Gateway models
|
||||
rpc refreshVercelAiGatewayModelsRpc(EmptyRequest) returns (OpenRouterCompatibleModelInfo);
|
||||
// Lists providers available from the unified SDK-backed catalog
|
||||
rpc listProviders(Empty) returns (ProviderListingsResponse);
|
||||
// Resolves model metadata for a provider through the unified SDK-backed catalog
|
||||
rpc resolveProviderModels(ResolveProviderModelsRequest) returns (ProviderModelsResponse);
|
||||
// Resolves model metadata for a provider/model without refreshing model lists
|
||||
rpc resolveModelInfo(ResolveModelInfoRequest) returns (ResolveModelInfoResponse);
|
||||
// Reads redacted effective provider configuration
|
||||
rpc readProviderConfig(StringRequest) returns (ProviderConfigResponse);
|
||||
// Writes provider configuration fields and returns redacted effective configuration
|
||||
rpc writeProviderConfig(WriteProviderConfigRequest) returns (ProviderConfigResponse);
|
||||
// Commits a mode-specific model selection atomically with its model metadata
|
||||
rpc commitModelSelection(CommitModelSelectionRequest) returns (Empty);
|
||||
}
|
||||
|
||||
// List of VS Code LM models
|
||||
@@ -117,6 +127,147 @@ message OpenRouterCompatibleModelInfo {
|
||||
map<string, OpenRouterModelInfo> models = 1;
|
||||
}
|
||||
|
||||
// Lightweight provider entry for the top-level model/provider picker.
|
||||
// Does not include the full model list; use resolveProviderModels for models.
|
||||
message ProviderListing {
|
||||
string id = 1;
|
||||
string name = 2;
|
||||
optional string default_model_id = 3;
|
||||
optional string family = 4;
|
||||
optional string protocol = 5;
|
||||
optional string auth_description = 6;
|
||||
optional string base_url_description = 7;
|
||||
bool allows_custom_model_ids = 8;
|
||||
|
||||
// SDK-driven hint for cost display. Values: "show" (default) or "hide".
|
||||
// Sourced from `resolveProviderUsageCostDisplay(provider.metadata)` in
|
||||
// `@cline/llms`. When "hide", consumers must suppress per-token pricing
|
||||
// and total cost displays (matches the CLI's `shouldShowCliUsageCost`).
|
||||
string usage_cost_display = 11;
|
||||
}
|
||||
|
||||
message ProviderListingsResponse {
|
||||
repeated ProviderListing providers = 1;
|
||||
}
|
||||
|
||||
message ResolveProviderModelsRequest {
|
||||
string provider_id = 1;
|
||||
bool force_refresh = 2;
|
||||
optional string request_id = 3;
|
||||
}
|
||||
|
||||
message CatalogErrorInfo {
|
||||
string kind = 1;
|
||||
string message = 2;
|
||||
optional string code = 3;
|
||||
optional bool retryable = 4;
|
||||
}
|
||||
|
||||
message ProviderModelsResponse {
|
||||
string provider_id = 1;
|
||||
string request_id = 2;
|
||||
string config_fingerprint = 3;
|
||||
int64 fetched_at = 4;
|
||||
bool ok = 5;
|
||||
map<string, OpenRouterModelInfo> models = 6;
|
||||
optional string default_model_id = 7;
|
||||
optional string source = 8;
|
||||
optional CatalogErrorInfo error = 9;
|
||||
}
|
||||
|
||||
message ResolveModelInfoRequest {
|
||||
string provider_id = 1;
|
||||
optional string model_id = 2;
|
||||
}
|
||||
|
||||
message ResolveModelInfoResponse {
|
||||
string provider_id = 1;
|
||||
string model_id = 2;
|
||||
optional OpenRouterModelInfo model_info = 3;
|
||||
string source = 4;
|
||||
}
|
||||
|
||||
message ProviderConfigResponse {
|
||||
string provider_id = 1;
|
||||
optional string base_url = 2;
|
||||
optional string api_line = 3;
|
||||
map<string, string> headers = 4;
|
||||
optional string region = 5;
|
||||
int64 api_key_length = 6;
|
||||
bool has_access_token = 7;
|
||||
bool has_refresh_token = 8;
|
||||
optional string account_id = 9;
|
||||
optional CommittedModelSelection plan_selection = 10;
|
||||
optional CommittedModelSelection act_selection = 11;
|
||||
optional AwsProviderConfig aws = 12;
|
||||
}
|
||||
|
||||
message CommittedModelSelection {
|
||||
string provider_id = 1;
|
||||
string model_id = 2;
|
||||
OpenRouterModelInfo model_info = 3;
|
||||
}
|
||||
|
||||
message ProviderReasoningPatch {
|
||||
optional bool enabled = 1;
|
||||
optional string effort = 2; // "none" | "low" | "medium" | "high" | "xhigh"
|
||||
optional int32 budget_tokens = 3;
|
||||
}
|
||||
|
||||
message AwsProviderConfig {
|
||||
optional string access_key = 1;
|
||||
optional string secret_key = 2;
|
||||
optional string session_token = 3;
|
||||
optional string region = 4;
|
||||
optional string profile = 5;
|
||||
optional string authentication = 6; // "iam" | "api-key" | "apikey" | "profile"
|
||||
optional bool use_prompt_cache = 7;
|
||||
optional bool use_cross_region_inference = 8;
|
||||
optional bool use_global_inference = 9;
|
||||
optional string endpoint = 10;
|
||||
optional string custom_model_base_id = 11;
|
||||
}
|
||||
|
||||
message AwsProviderConfigPatch {
|
||||
optional string access_key = 1;
|
||||
optional string secret_key = 2;
|
||||
optional string session_token = 3;
|
||||
optional string region = 4;
|
||||
optional string profile = 5;
|
||||
optional string authentication = 6; // "iam" | "api-key" | "apikey" | "profile"
|
||||
optional bool use_prompt_cache = 7;
|
||||
optional bool use_cross_region_inference = 8;
|
||||
optional bool use_global_inference = 9;
|
||||
optional string endpoint = 10;
|
||||
optional string custom_model_base_id = 11;
|
||||
}
|
||||
|
||||
message WriteProviderConfigPatch {
|
||||
optional string api_key = 1;
|
||||
optional string base_url = 2;
|
||||
map<string, string> headers = 3;
|
||||
optional string region = 4;
|
||||
optional string api_line = 5;
|
||||
optional string access_token = 6;
|
||||
optional string refresh_token = 7;
|
||||
optional string account_id = 8;
|
||||
optional ProviderReasoningPatch reasoning = 9;
|
||||
optional bool clear_headers = 10;
|
||||
optional AwsProviderConfigPatch aws = 11;
|
||||
}
|
||||
|
||||
message WriteProviderConfigRequest {
|
||||
string provider_id = 1;
|
||||
WriteProviderConfigPatch patch = 2;
|
||||
}
|
||||
|
||||
message CommitModelSelectionRequest {
|
||||
string provider_id = 1;
|
||||
string mode = 2;
|
||||
string model_id = 3;
|
||||
OpenRouterModelInfo model_info = 4;
|
||||
}
|
||||
|
||||
message ClineRecommendedModel {
|
||||
string id = 1;
|
||||
string name = 2;
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
syntax = "proto3";
|
||||
|
||||
package cline;
|
||||
|
||||
import "cline/common.proto";
|
||||
|
||||
option go_package = "github.com/cline/grpc-go/cline";
|
||||
option java_multiple_files = true;
|
||||
option java_package = "bot.cline.proto";
|
||||
|
||||
enum RemoteConfigType {
|
||||
RULE = 0;
|
||||
WORKFLOW = 1;
|
||||
SKILL = 2;
|
||||
}
|
||||
|
||||
message RemoteConfigSetting {
|
||||
RemoteConfigType type = 1;
|
||||
string name = 2;
|
||||
string content = 3;
|
||||
bool enabled = 4;
|
||||
bool locked = 5;
|
||||
}
|
||||
|
||||
message RemoteConfigSettingsResponse {
|
||||
repeated RemoteConfigSetting settings = 1;
|
||||
}
|
||||
|
||||
service RemoteConfigService {
|
||||
rpc getRemoteConfigSettings(Empty) returns (RemoteConfigSettingsResponse);
|
||||
rpc toggleRemoteConfigSetting(StringRequest) returns (RemoteConfigSetting);
|
||||
}
|
||||
@@ -250,7 +250,6 @@ message Settings {
|
||||
optional string default_terminal_profile = 137;
|
||||
optional int32 terminal_output_line_limit = 138;
|
||||
optional int32 max_consecutive_mistakes = 139;
|
||||
optional bool strict_plan_mode_enabled = 141;
|
||||
optional bool yolo_mode_toggled = 142;
|
||||
optional bool use_auto_condense = 143;
|
||||
optional bool cline_web_tools_enabled = 144;
|
||||
@@ -286,7 +285,6 @@ message Settings {
|
||||
optional string act_mode_cline_model_id = 180;
|
||||
optional OpenRouterModelInfo act_mode_cline_model_info = 181;
|
||||
optional bool show_feature_tips = 182;
|
||||
optional bool lazy_teammate_mode_enabled = 183;
|
||||
}
|
||||
|
||||
message State {
|
||||
@@ -391,6 +389,7 @@ message UpdateSettingsRequest {
|
||||
reserved 15; // was openai_reasoning_effort (moved to mode-scoped reasoning effort)
|
||||
reserved 23; // was dictation_settings (dictation removed)
|
||||
reserved 38; // was skills_enabled (removed - now always enabled)
|
||||
reserved 43; // was lazy_teammate_mode_enabled (removed)
|
||||
|
||||
Metadata metadata = 1;
|
||||
optional ModelsApiConfiguration api_configuration = 2;
|
||||
@@ -405,7 +404,7 @@ message UpdateSettingsRequest {
|
||||
optional int32 terminal_output_line_limit = 12;
|
||||
optional PlanActMode mode = 13;
|
||||
optional string preferred_language = 14;
|
||||
optional bool strict_plan_mode_enabled = 16;
|
||||
reserved 16; // was strict_plan_mode_enabled (removed)
|
||||
optional FocusChainSettings focus_chain_settings = 17;
|
||||
optional bool use_auto_condense = 18;
|
||||
optional string custom_prompt = 19;
|
||||
@@ -429,7 +428,6 @@ message UpdateSettingsRequest {
|
||||
optional bool worktrees_enabled = 40;
|
||||
optional bool double_check_completion_enabled = 41;
|
||||
optional bool show_feature_tips = 42;
|
||||
optional bool lazy_teammate_mode_enabled = 43;
|
||||
}
|
||||
|
||||
message UpdateTerminalConnectionTimeoutRequest {
|
||||
|
||||
@@ -82,12 +82,14 @@ message GetTaskHistoryRequest {
|
||||
string search_query = 3;
|
||||
string sort_by = 4;
|
||||
bool current_workspace_only = 5;
|
||||
int32 limit = 6;
|
||||
int32 offset = 7;
|
||||
}
|
||||
|
||||
// Response for task history
|
||||
message TaskHistoryArray {
|
||||
repeated TaskItem tasks = 1;
|
||||
int32 total_count = 2;
|
||||
bool has_more = 2;
|
||||
}
|
||||
|
||||
// Task item details for history list
|
||||
|
||||
@@ -226,6 +226,12 @@ message ClineMessage {
|
||||
ClineAskNewTask ask_new_task = 21;
|
||||
ClineApiReqInfo api_req_info = 22;
|
||||
ClineModelInfo model_info = 23;
|
||||
|
||||
// Convergent-replica fields (see webview-message-state-design.md):
|
||||
// seq = monotonic freshness (higher seq wins for the same ts/identity)
|
||||
// epoch = conversation/replica fence (older epoch is dropped by the webview)
|
||||
int64 seq = 24;
|
||||
int64 epoch = 25;
|
||||
}
|
||||
|
||||
message ShowWebviewEvent {
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
// Dead-source finder: uses esbuild's own bundle reachability (the same analysis
|
||||
// that drives tree-shaking + minification mangling) to compute which src/ files
|
||||
// are reachable from BOTH shipped entry points:
|
||||
// - src/extension.ts (VS Code extension host)
|
||||
// - src/standalone/cline-core.ts (standalone host used by JetBrains + CLI)
|
||||
//
|
||||
// A src/*.ts file that is NOT in the union of metafile inputs for those two
|
||||
// builds is unreachable from any shipped entry => dead (modulo dynamic import()
|
||||
// of computed specifiers, which esbuild surfaces separately).
|
||||
//
|
||||
// Run: node scripts/find-dead-src.mjs
|
||||
|
||||
import fs from "node:fs"
|
||||
import path from "node:path"
|
||||
import { fileURLToPath } from "node:url"
|
||||
import * as esbuild from "esbuild"
|
||||
import { glob } from "glob"
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url))
|
||||
const root = path.resolve(__dirname, "..")
|
||||
|
||||
const aliases = {
|
||||
"@": path.join(root, "src"),
|
||||
"@core": path.join(root, "src/core"),
|
||||
"@integrations": path.join(root, "src/integrations"),
|
||||
"@services": path.join(root, "src/services"),
|
||||
"@shared": path.join(root, "src/shared"),
|
||||
"@utils": path.join(root, "src/utils"),
|
||||
"@packages": path.join(root, "src/packages"),
|
||||
}
|
||||
|
||||
const aliasResolverPlugin = {
|
||||
name: "alias-resolver",
|
||||
setup(build) {
|
||||
for (const [alias, aliasPath] of Object.entries(aliases)) {
|
||||
const aliasRegex = new RegExp(`^${alias}($|/.*)`)
|
||||
build.onResolve({ filter: aliasRegex }, (args) => {
|
||||
const importPath = args.path.replace(alias, aliasPath)
|
||||
const exts = [".ts", ".tsx", ".js", ".jsx"]
|
||||
if (fs.existsSync(importPath)) {
|
||||
const stats = fs.statSync(importPath)
|
||||
if (stats.isDirectory()) {
|
||||
for (const ext of exts) {
|
||||
const idx = path.join(importPath, `index${ext}`)
|
||||
if (fs.existsSync(idx)) return { path: idx }
|
||||
}
|
||||
} else {
|
||||
return { path: importPath }
|
||||
}
|
||||
}
|
||||
for (const ext of exts) {
|
||||
if (fs.existsSync(`${importPath}${ext}`)) return { path: `${importPath}${ext}` }
|
||||
}
|
||||
return undefined
|
||||
})
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
const common = {
|
||||
bundle: true,
|
||||
minify: false,
|
||||
sourcemap: false,
|
||||
logLevel: "silent",
|
||||
format: "cjs",
|
||||
platform: "node",
|
||||
metafile: true,
|
||||
write: false,
|
||||
absWorkingDir: root,
|
||||
tsconfig: path.join(root, "tsconfig.json"),
|
||||
packages: "external",
|
||||
plugins: [aliasResolverPlugin],
|
||||
define: { "process.env.IS_DEV": "false", "process.env.IS_TEST": "false" },
|
||||
banner: { js: "const _importMetaUrl=require('url').pathToFileURL(__filename)" },
|
||||
}
|
||||
|
||||
async function inputsFor(entry, external) {
|
||||
const r = await esbuild.build({ ...common, entryPoints: [entry], external })
|
||||
return new Set(Object.keys(r.metafile.inputs).filter((f) => f.startsWith("src/") && /\.tsx?$/.test(f)))
|
||||
}
|
||||
|
||||
const ext = await inputsFor("src/extension.ts", ["vscode"])
|
||||
const standalone = await inputsFor("src/standalone/cline-core.ts", [
|
||||
"vscode",
|
||||
"@grpc/reflection",
|
||||
"grpc-health-check",
|
||||
"better-sqlite3",
|
||||
])
|
||||
const live = new Set([...ext, ...standalone])
|
||||
|
||||
// Third consumer: the webview (webview-ui/) is a separate Vite/React build that
|
||||
// imports extension code ONLY from src/shared (via "@shared/*" alias or relative
|
||||
// "../src/shared/*" paths). Any src/shared file referenced from webview-ui/src is
|
||||
// therefore live even if the extension-host/standalone bundles don't reach it.
|
||||
// Conservatively mark every src/shared file mentioned by the webview as live.
|
||||
const webviewFiles = await glob("webview-ui/src/**/*.{ts,tsx}", { cwd: root })
|
||||
const sharedMentionedByWebview = new Set()
|
||||
for (const wf of webviewFiles) {
|
||||
const text = fs.readFileSync(path.join(root, wf), "utf8")
|
||||
// Match @shared/X or .../src/shared/X import specifiers and map to src/shared/X
|
||||
const re = /(?:@shared\/|src\/shared\/)([A-Za-z0-9_./-]+)/g
|
||||
let m
|
||||
while ((m = re.exec(text))) {
|
||||
const rel = m[1].replace(/\.(ts|tsx|js|jsx)$/, "")
|
||||
for (const cand of [`src/shared/${rel}.ts`, `src/shared/${rel}.tsx`, `src/shared/${rel}/index.ts`]) {
|
||||
if (fs.existsSync(path.join(root, cand))) sharedMentionedByWebview.add(cand)
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const f of sharedMentionedByWebview) live.add(f)
|
||||
console.log(`src/shared files referenced by webview: ${sharedMentionedByWebview.size}`)
|
||||
|
||||
// All non-test, non-.d.ts source files on disk.
|
||||
const allSrc = (await glob("src/**/*.{ts,tsx}", { cwd: root }))
|
||||
.filter((f) => !/\.test\.tsx?$/.test(f))
|
||||
.filter((f) => !f.endsWith(".d.ts"))
|
||||
.filter((f) => !f.includes("/__tests__/"))
|
||||
.filter((f) => !f.startsWith("src/test/"))
|
||||
.filter((f) => !f.startsWith("src/generated/")) // generated host glue
|
||||
.filter((f) => !f.startsWith("src/dev/")) // dev-only tooling
|
||||
|
||||
const dead = allSrc.filter((f) => !live.has(f)).sort()
|
||||
|
||||
console.log(`extension inputs: ${ext.size}`)
|
||||
console.log(`standalone inputs: ${standalone.size}`)
|
||||
console.log(`union live src files: ${live.size}`)
|
||||
console.log(`candidate dead files: ${dead.length}`)
|
||||
fs.writeFileSync("/tmp/dead-src.json", JSON.stringify(dead, null, "\t"))
|
||||
console.log("--- dead candidates written to /tmp/dead-src.json ---")
|
||||
@@ -87,6 +87,12 @@ function createMockService<T extends grpc.UntypedServiceImplementation>(serviceN
|
||||
})
|
||||
return
|
||||
|
||||
case "openExternal":
|
||||
simulateOAuthBrowserCallback(call.request?.value || "")
|
||||
.then(() => callback(null, {}))
|
||||
.catch((error) => callback(error))
|
||||
return
|
||||
|
||||
case "getWebviewHtml":
|
||||
callback(null, {
|
||||
html: "<html><body>Fake Webview</body></html>",
|
||||
@@ -143,6 +149,41 @@ function createMockService<T extends grpc.UntypedServiceImplementation>(serviceN
|
||||
return new Proxy({} as T, handler)
|
||||
}
|
||||
|
||||
async function simulateOAuthBrowserCallback(urlString: string): Promise<void> {
|
||||
let url: URL
|
||||
try {
|
||||
url = new URL(urlString)
|
||||
} catch {
|
||||
return
|
||||
}
|
||||
|
||||
if (!isLoopbackHost(url.hostname) || url.pathname !== "/api/v1/auth/authorize") {
|
||||
return
|
||||
}
|
||||
|
||||
const callbackUrl = url.searchParams.get("callback_url") ?? url.searchParams.get("redirect_uri")
|
||||
if (!callbackUrl) {
|
||||
return
|
||||
}
|
||||
|
||||
const callback = new URL(callbackUrl)
|
||||
if (!isLoopbackHost(callback.hostname) || callback.pathname !== "/auth") {
|
||||
return
|
||||
}
|
||||
|
||||
callback.searchParams.set("code", "test-personal-token")
|
||||
callback.searchParams.set("provider", "cline")
|
||||
|
||||
const response = await fetch(callback.toString())
|
||||
if (!response.ok) {
|
||||
throw new Error(`Mock OAuth callback failed: ${response.status} ${response.statusText}`)
|
||||
}
|
||||
}
|
||||
|
||||
function isLoopbackHost(hostname: string): boolean {
|
||||
return hostname === "localhost" || hostname === "127.0.0.1" || hostname === "::1"
|
||||
}
|
||||
|
||||
if (require.main === module) {
|
||||
startTestHostBridgeServer().catch((err) => {
|
||||
console.error("Failed to start test host bridge server:", err)
|
||||
|
||||
@@ -115,7 +115,8 @@ async function main(): Promise<void> {
|
||||
|
||||
const covDir = path.join(projectRoot, `coverage/coverage-core-${PROTOBUS_PORT}`)
|
||||
|
||||
const baseArgs = ["--enable-source-maps", path.join(distDir, "cline-core.js")]
|
||||
const workosFetchMockPath = path.join(projectRoot, "scripts", "testing-platform-workos-fetch-mock.cjs")
|
||||
const baseArgs = ["--enable-source-maps", "--require", workosFetchMockPath, path.join(distDir, "cline-core.js")]
|
||||
|
||||
const spawnArgs = USE_C8 ? ["c8", "--report-dir", covDir, "node", ...baseArgs] : ["node", ...baseArgs]
|
||||
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
// Preload used by the standalone testing platform.
|
||||
// It makes the SDK WorkOS device-auth flow deterministic and fully local while
|
||||
// leaving production auth code on the same device-auth path used by users.
|
||||
|
||||
const originalFetch = globalThis.fetch?.bind(globalThis)
|
||||
|
||||
const WORKOS_ORIGIN = "https://api.workos.com"
|
||||
const DEVICE_CODE = "test-device-code"
|
||||
const USER_CODE = "PTBC-TXTP"
|
||||
const ACCESS_TOKEN = "test-personal-token"
|
||||
const REFRESH_TOKEN = "test-personal-token_refresh"
|
||||
|
||||
function jsonResponse(body, init = {}) {
|
||||
return new Response(JSON.stringify(body), {
|
||||
status: init.status ?? 200,
|
||||
headers: { "Content-Type": "application/json", ...(init.headers ?? {}) },
|
||||
})
|
||||
}
|
||||
|
||||
function inputUrl(input) {
|
||||
if (typeof input === "string") return input
|
||||
if (input instanceof URL) return input.toString()
|
||||
if (input && typeof input === "object" && "url" in input) return input.url
|
||||
return String(input)
|
||||
}
|
||||
|
||||
globalThis.fetch = async (input, init) => {
|
||||
const urlString = inputUrl(input)
|
||||
let url
|
||||
try {
|
||||
url = new URL(urlString)
|
||||
} catch {
|
||||
return originalFetch(input, init)
|
||||
}
|
||||
|
||||
if (url.origin === WORKOS_ORIGIN && url.pathname === "/user_management/authorize/device") {
|
||||
return jsonResponse({
|
||||
device_code: DEVICE_CODE,
|
||||
user_code: USER_CODE,
|
||||
verification_uri: "https://login.workos.test/device",
|
||||
verification_uri_complete: `https://login.workos.test/device?user_code=${USER_CODE}`,
|
||||
expires_in: 300,
|
||||
interval: 1,
|
||||
})
|
||||
}
|
||||
|
||||
if (url.origin === WORKOS_ORIGIN && url.pathname === "/user_management/authenticate") {
|
||||
return jsonResponse({
|
||||
access_token: ACCESS_TOKEN,
|
||||
refresh_token: REFRESH_TOKEN,
|
||||
token_type: "Bearer",
|
||||
})
|
||||
}
|
||||
|
||||
return originalFetch(input, init)
|
||||
}
|
||||
@@ -21,9 +21,7 @@ describe("ClineEndpoint configuration", () => {
|
||||
|
||||
// Stub os.homedir to return our temp directory
|
||||
originalHomedir = os.homedir
|
||||
sandbox
|
||||
.stub(os, "homedir")
|
||||
.returns(tempDir)
|
||||
sandbox.stub(os, "homedir").returns(tempDir)
|
||||
|
||||
// Reset the singleton state using internal method
|
||||
;(ClineEndpoint as any)._instance = null
|
||||
|
||||
@@ -4,7 +4,6 @@ import "./utils/path" // necessary to have access to String.prototype.toPosix
|
||||
import { HostProvider } from "@/hosts/host-provider"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import type { StorageContext } from "@/shared/storage/storage-context"
|
||||
import { FileContextTracker } from "./core/context/context-tracking/FileContextTracker"
|
||||
import { clearOnboardingModelsCache } from "./core/controller/models/getClineOnboardingModels"
|
||||
import { HookDiscoveryCache } from "./core/hooks/HookDiscoveryCache"
|
||||
import { HookProcessRegistry } from "./core/hooks/HookProcessRegistry"
|
||||
@@ -74,8 +73,6 @@ export async function initialize(storageContext: StorageContext): Promise<Webvie
|
||||
syncWorker().init({ ...blobStoreSettings, userDistinctId: getDistinctId() })
|
||||
// Clean up old temp files in background (non-blocking) and start periodic cleanup every 24 hours
|
||||
ClineTempManager.startPeriodicCleanup()
|
||||
// Clean up orphaned file context warnings (startup cleanup)
|
||||
FileContextTracker.cleanupOrphanedWarnings(stateManager)
|
||||
|
||||
telemetryService.captureExtensionActivated()
|
||||
|
||||
@@ -106,7 +103,7 @@ async function showVersionUpdateAnnouncement(stateManager: StateManager) {
|
||||
})
|
||||
}
|
||||
// Always update the main version tracker for the next launch.
|
||||
await stateManager.setGlobalState("clineVersion", currentVersion)
|
||||
stateManager.setGlobalState("clineVersion", currentVersion)
|
||||
}
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : String(error)
|
||||
|
||||
@@ -1,52 +1,24 @@
|
||||
import { ApiConfiguration, ModelInfo, QwenApiRegions } from "@shared/api"
|
||||
import { type ApiHandler as SdkApiHandler, type ApiStreamChunk as SdkApiStreamChunk } from "@cline/llms"
|
||||
import { ApiConfiguration, ModelInfo } from "@shared/api"
|
||||
import { Mode } from "@shared/storage/types"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ClineTool } from "@/shared/tools"
|
||||
import { AIhubmixHandler } from "./providers/aihubmix"
|
||||
import { AnthropicHandler } from "./providers/anthropic"
|
||||
import { AskSageHandler } from "./providers/asksage"
|
||||
import { BasetenHandler } from "./providers/baseten"
|
||||
import { AwsBedrockHandler } from "./providers/bedrock"
|
||||
import { CerebrasHandler } from "./providers/cerebras"
|
||||
import { ClaudeCodeHandler } from "./providers/claude-code"
|
||||
import { ClineHandler } from "./providers/cline"
|
||||
import { DeepSeekHandler } from "./providers/deepseek"
|
||||
import { DifyHandler } from "./providers/dify"
|
||||
import { DoubaoHandler } from "./providers/doubao"
|
||||
import { FireworksHandler } from "./providers/fireworks"
|
||||
import { GeminiHandler } from "./providers/gemini"
|
||||
import { GroqHandler } from "./providers/groq"
|
||||
import { HicapHandler } from "./providers/hicap"
|
||||
import { HuaweiCloudMaaSHandler } from "./providers/huawei-cloud-maas"
|
||||
import { HuggingFaceHandler } from "./providers/huggingface"
|
||||
import { LiteLlmHandler } from "./providers/litellm"
|
||||
import { LmStudioHandler } from "./providers/lmstudio"
|
||||
import { MinimaxHandler } from "./providers/minimax"
|
||||
import { MistralHandler } from "./providers/mistral"
|
||||
import { MoonshotHandler } from "./providers/moonshot"
|
||||
import { NebiusHandler } from "./providers/nebius"
|
||||
import { NousResearchHandler } from "./providers/nousresearch"
|
||||
import { OcaHandler } from "./providers/oca"
|
||||
import { OllamaHandler } from "./providers/ollama"
|
||||
import { OpenAiHandler } from "./providers/openai"
|
||||
import { OpenAiCodexHandler } from "./providers/openai-codex"
|
||||
import { OpenAiNativeHandler } from "./providers/openai-native"
|
||||
import { OpenRouterHandler } from "./providers/openrouter"
|
||||
import { QwenHandler } from "./providers/qwen"
|
||||
import { QwenCodeHandler } from "./providers/qwen-code"
|
||||
import { RequestyHandler } from "./providers/requesty"
|
||||
import { SambanovaHandler } from "./providers/sambanova"
|
||||
import { SapAiCoreHandler } from "./providers/sapaicore"
|
||||
import { TogetherHandler } from "./providers/together"
|
||||
import { VercelAIGatewayHandler } from "./providers/vercel-ai-gateway"
|
||||
import { VertexHandler } from "./providers/vertex"
|
||||
import { VsCodeLmHandler } from "./providers/vscode-lm"
|
||||
import { WandbHandler } from "./providers/wandb"
|
||||
import { XAIHandler } from "./providers/xai"
|
||||
import { ZAiHandler } from "./providers/zai"
|
||||
import { ApiStream, ApiStreamUsageChunk } from "./transform/stream"
|
||||
|
||||
// buildApiHandler now routes inference through the Cline SDK. It lives in
|
||||
// apps/vscode/src/sdk/sdk-api-handler.ts and callers import it directly from
|
||||
// there. It is deliberately NOT re-exported here: this barrel is imported
|
||||
// widely for *types* only, and re-exporting a value from the SDK module would
|
||||
// pull the entire SDK/session-factory runtime graph into every type importer
|
||||
// at module-eval time (which can break extension activation). Keep this file
|
||||
// types-only.
|
||||
|
||||
// Re-export the SDK inference contracts so callers can depend on the SDK types
|
||||
// through the existing @core/api entry point. These are the canonical handler
|
||||
// and stream types going forward; the local interfaces below remain for the
|
||||
// classic provider classes until they are removed.
|
||||
export type { SdkApiHandler, SdkApiStreamChunk }
|
||||
|
||||
export type CommonApiHandlerOptions = {
|
||||
onRetryAttempt?: ApiConfiguration["onRetryAttempt"]
|
||||
}
|
||||
@@ -72,436 +44,3 @@ export interface ApiProviderInfo {
|
||||
export interface SingleCompletionHandler {
|
||||
completePrompt(prompt: string): Promise<string>
|
||||
}
|
||||
|
||||
function createHandlerForProvider(
|
||||
apiProvider: string | undefined,
|
||||
options: Omit<ApiConfiguration, "apiProvider">,
|
||||
mode: Mode,
|
||||
): ApiHandler {
|
||||
switch (apiProvider) {
|
||||
case "anthropic":
|
||||
return new AnthropicHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
apiKey: options.apiKey,
|
||||
anthropicBaseUrl: options.anthropicBaseUrl,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
})
|
||||
case "openrouter":
|
||||
return new OpenRouterHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
openRouterApiKey: options.openRouterApiKey,
|
||||
openRouterModelId: mode === "plan" ? options.planModeOpenRouterModelId : options.actModeOpenRouterModelId,
|
||||
openRouterModelInfo: mode === "plan" ? options.planModeOpenRouterModelInfo : options.actModeOpenRouterModelInfo,
|
||||
openRouterProviderSorting: options.openRouterProviderSorting,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
enableParallelToolCalling: options.enableParallelToolCalling,
|
||||
})
|
||||
case "bedrock":
|
||||
return new AwsBedrockHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
awsAccessKey: options.awsAccessKey,
|
||||
awsSecretKey: options.awsSecretKey,
|
||||
awsSessionToken: options.awsSessionToken,
|
||||
awsRegion: options.awsRegion,
|
||||
awsAuthentication: options.awsAuthentication,
|
||||
awsBedrockApiKey: options.awsBedrockApiKey,
|
||||
awsUseCrossRegionInference: options.awsUseCrossRegionInference,
|
||||
awsUseGlobalInference: options.awsUseGlobalInference,
|
||||
awsBedrockUsePromptCache: options.awsBedrockUsePromptCache,
|
||||
awsUseProfile: options.awsUseProfile,
|
||||
awsProfile: options.awsProfile,
|
||||
awsBedrockEndpoint: options.awsBedrockEndpoint,
|
||||
awsBedrockCustomSelected:
|
||||
mode === "plan" ? options.planModeAwsBedrockCustomSelected : options.actModeAwsBedrockCustomSelected,
|
||||
awsBedrockCustomModelBaseId:
|
||||
mode === "plan" ? options.planModeAwsBedrockCustomModelBaseId : options.actModeAwsBedrockCustomModelBaseId,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
})
|
||||
case "vertex":
|
||||
return new VertexHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
vertexProjectId: options.vertexProjectId,
|
||||
vertexRegion: options.vertexRegion,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
geminiApiKey: options.geminiApiKey,
|
||||
geminiBaseUrl: options.geminiBaseUrl,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
ulid: options.ulid,
|
||||
})
|
||||
case "openai":
|
||||
return new OpenAiHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
openAiApiKey: options.openAiApiKey,
|
||||
openAiBaseUrl: options.openAiBaseUrl,
|
||||
azureApiVersion: options.azureApiVersion,
|
||||
azureIdentity: options.azureIdentity,
|
||||
openAiHeaders: options.openAiHeaders,
|
||||
openAiModelId: mode === "plan" ? options.planModeOpenAiModelId : options.actModeOpenAiModelId,
|
||||
openAiModelInfo: mode === "plan" ? options.planModeOpenAiModelInfo : options.actModeOpenAiModelInfo,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
})
|
||||
case "ollama":
|
||||
return new OllamaHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
ollamaBaseUrl: options.ollamaBaseUrl,
|
||||
ollamaApiKey: options.ollamaApiKey,
|
||||
ollamaModelId: mode === "plan" ? options.planModeOllamaModelId : options.actModeOllamaModelId,
|
||||
ollamaApiOptionsCtxNum: options.ollamaApiOptionsCtxNum,
|
||||
requestTimeoutMs: options.requestTimeoutMs,
|
||||
})
|
||||
case "lmstudio":
|
||||
return new LmStudioHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
lmStudioBaseUrl: options.lmStudioBaseUrl,
|
||||
lmStudioModelId: mode === "plan" ? options.planModeLmStudioModelId : options.actModeLmStudioModelId,
|
||||
lmStudioMaxTokens: options.lmStudioMaxTokens,
|
||||
})
|
||||
case "gemini":
|
||||
return new GeminiHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
vertexProjectId: options.vertexProjectId,
|
||||
vertexRegion: options.vertexRegion,
|
||||
geminiApiKey: options.geminiApiKey,
|
||||
geminiBaseUrl: options.geminiBaseUrl,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
ulid: options.ulid,
|
||||
})
|
||||
case "openai-native":
|
||||
return new OpenAiNativeHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
openAiNativeApiKey: options.openAiNativeApiKey,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
})
|
||||
case "openai-codex":
|
||||
return new OpenAiCodexHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "deepseek":
|
||||
return new DeepSeekHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
deepSeekApiKey: options.deepSeekApiKey,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "requesty":
|
||||
return new RequestyHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
requestyBaseUrl: options.requestyBaseUrl,
|
||||
requestyApiKey: options.requestyApiKey,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
requestyModelId: mode === "plan" ? options.planModeRequestyModelId : options.actModeRequestyModelId,
|
||||
requestyModelInfo: mode === "plan" ? options.planModeRequestyModelInfo : options.actModeRequestyModelInfo,
|
||||
})
|
||||
case "fireworks":
|
||||
return new FireworksHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
fireworksApiKey: options.fireworksApiKey,
|
||||
fireworksModelId: mode === "plan" ? options.planModeFireworksModelId : options.actModeFireworksModelId,
|
||||
})
|
||||
case "together":
|
||||
return new TogetherHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
togetherApiKey: options.togetherApiKey,
|
||||
togetherModelId: mode === "plan" ? options.planModeTogetherModelId : options.actModeTogetherModelId,
|
||||
})
|
||||
case "qwen":
|
||||
return new QwenHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
qwenApiKey: options.qwenApiKey,
|
||||
qwenApiLine:
|
||||
options.qwenApiLine === QwenApiRegions.INTERNATIONAL ? QwenApiRegions.INTERNATIONAL : QwenApiRegions.CHINA,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
})
|
||||
case "qwen-code":
|
||||
return new QwenCodeHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
qwenCodeOauthPath: options.qwenCodeOauthPath,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "doubao":
|
||||
return new DoubaoHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
doubaoApiKey: options.doubaoApiKey,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "mistral":
|
||||
return new MistralHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
mistralApiKey: options.mistralApiKey,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "vscode-lm":
|
||||
return new VsCodeLmHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
vsCodeLmModelSelector:
|
||||
mode === "plan" ? options.planModeVsCodeLmModelSelector : options.actModeVsCodeLmModelSelector,
|
||||
})
|
||||
case "cline": {
|
||||
const clineModelId =
|
||||
(mode === "plan" ? options.planModeClineModelId : options.actModeClineModelId) ||
|
||||
(mode === "plan" ? options.planModeOpenRouterModelId : options.actModeOpenRouterModelId)
|
||||
const clineModelInfo =
|
||||
(mode === "plan" ? options.planModeClineModelInfo : options.actModeClineModelInfo) ||
|
||||
(mode === "plan" ? options.planModeOpenRouterModelInfo : options.actModeOpenRouterModelInfo)
|
||||
return new ClineHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
clineAccountId: options.clineAccountId,
|
||||
clineApiKey: options.clineApiKey,
|
||||
ulid: options.ulid,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
openRouterProviderSorting: options.openRouterProviderSorting,
|
||||
openRouterModelId: clineModelId,
|
||||
openRouterModelInfo: clineModelInfo,
|
||||
enableParallelToolCalling: options.enableParallelToolCalling,
|
||||
})
|
||||
}
|
||||
case "litellm":
|
||||
return new LiteLlmHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
liteLlmApiKey: options.liteLlmApiKey,
|
||||
liteLlmBaseUrl: options.liteLlmBaseUrl,
|
||||
liteLlmModelId: mode === "plan" ? options.planModeLiteLlmModelId : options.actModeLiteLlmModelId,
|
||||
liteLlmModelInfo: mode === "plan" ? options.planModeLiteLlmModelInfo : options.actModeLiteLlmModelInfo,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
liteLlmUsePromptCache: options.liteLlmUsePromptCache,
|
||||
ulid: options.ulid,
|
||||
})
|
||||
case "moonshot":
|
||||
return new MoonshotHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
moonshotApiKey: options.moonshotApiKey,
|
||||
moonshotApiLine: options.moonshotApiLine,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "huggingface":
|
||||
return new HuggingFaceHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
huggingFaceApiKey: options.huggingFaceApiKey,
|
||||
huggingFaceModelId: mode === "plan" ? options.planModeHuggingFaceModelId : options.actModeHuggingFaceModelId,
|
||||
huggingFaceModelInfo:
|
||||
mode === "plan" ? options.planModeHuggingFaceModelInfo : options.actModeHuggingFaceModelInfo,
|
||||
})
|
||||
case "nebius":
|
||||
return new NebiusHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
nebiusApiKey: options.nebiusApiKey,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "asksage":
|
||||
return new AskSageHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
asksageApiKey: options.asksageApiKey,
|
||||
asksageApiUrl: options.asksageApiUrl,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "xai":
|
||||
return new XAIHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
xaiApiKey: options.xaiApiKey,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "sambanova":
|
||||
return new SambanovaHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
sambanovaApiKey: options.sambanovaApiKey,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "cerebras":
|
||||
return new CerebrasHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
cerebrasApiKey: options.cerebrasApiKey,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "groq":
|
||||
return new GroqHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
groqApiKey: options.groqApiKey,
|
||||
groqModelId: mode === "plan" ? options.planModeGroqModelId : options.actModeGroqModelId,
|
||||
groqModelInfo: mode === "plan" ? options.planModeGroqModelInfo : options.actModeGroqModelInfo,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "baseten":
|
||||
return new BasetenHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
basetenApiKey: options.basetenApiKey,
|
||||
basetenModelId: mode === "plan" ? options.planModeBasetenModelId : options.actModeBasetenModelId,
|
||||
basetenModelInfo: mode === "plan" ? options.planModeBasetenModelInfo : options.actModeBasetenModelInfo,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "sapaicore":
|
||||
return new SapAiCoreHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
sapAiCoreClientId: options.sapAiCoreClientId,
|
||||
sapAiCoreClientSecret: options.sapAiCoreClientSecret,
|
||||
sapAiCoreTokenUrl: options.sapAiCoreTokenUrl,
|
||||
sapAiResourceGroup: options.sapAiResourceGroup,
|
||||
sapAiCoreBaseUrl: options.sapAiCoreBaseUrl,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
deploymentId: mode === "plan" ? options.planModeSapAiCoreDeploymentId : options.actModeSapAiCoreDeploymentId,
|
||||
sapAiCoreUseOrchestrationMode: options.sapAiCoreUseOrchestrationMode,
|
||||
})
|
||||
case "claude-code":
|
||||
return new ClaudeCodeHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
claudeCodePath: options.claudeCodePath,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
})
|
||||
case "huawei-cloud-maas":
|
||||
return new HuaweiCloudMaaSHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
huaweiCloudMaasApiKey: options.huaweiCloudMaasApiKey,
|
||||
huaweiCloudMaasModelId:
|
||||
mode === "plan" ? options.planModeHuaweiCloudMaasModelId : options.actModeHuaweiCloudMaasModelId,
|
||||
huaweiCloudMaasModelInfo:
|
||||
mode === "plan" ? options.planModeHuaweiCloudMaasModelInfo : options.actModeHuaweiCloudMaasModelInfo,
|
||||
})
|
||||
case "dify": // Add Dify.ai handler
|
||||
return new DifyHandler({
|
||||
difyApiKey: options.difyApiKey,
|
||||
difyBaseUrl: options.difyBaseUrl,
|
||||
})
|
||||
case "vercel-ai-gateway":
|
||||
return new VercelAIGatewayHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
vercelAiGatewayApiKey: options.vercelAiGatewayApiKey,
|
||||
openRouterModelId:
|
||||
mode === "plan" ? options.planModeVercelAiGatewayModelId : options.actModeVercelAiGatewayModelId,
|
||||
openRouterModelInfo:
|
||||
mode === "plan" ? options.planModeVercelAiGatewayModelInfo : options.actModeVercelAiGatewayModelInfo,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
})
|
||||
case "zai":
|
||||
return new ZAiHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
zaiApiLine: options.zaiApiLine,
|
||||
zaiApiKey: options.zaiApiKey,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
case "oca":
|
||||
return new OcaHandler({
|
||||
ocaMode: options.ocaMode || "internal",
|
||||
ocaBaseUrl: options.ocaBaseUrl,
|
||||
ocaModelId: mode === "plan" ? options.planModeOcaModelId : options.actModeOcaModelId,
|
||||
ocaModelInfo: mode === "plan" ? options.planModeOcaModelInfo : options.actModeOcaModelInfo,
|
||||
ocaReasoningEffort: mode === "plan" ? options.planModeOcaReasoningEffort : options.actModeOcaReasoningEffort,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
ocaUsePromptCache:
|
||||
mode === "plan"
|
||||
? options.planModeOcaModelInfo?.supportsPromptCache
|
||||
: options.actModeOcaModelInfo?.supportsPromptCache,
|
||||
taskId: options.ulid,
|
||||
})
|
||||
case "aihubmix":
|
||||
return new AIhubmixHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
apiKey: options.aihubmixApiKey,
|
||||
baseURL: options.aihubmixBaseUrl,
|
||||
appCode: options.aihubmixAppCode,
|
||||
modelId: mode === "plan" ? (options as any).planModeAihubmixModelId : (options as any).actModeAihubmixModelId,
|
||||
modelInfo:
|
||||
mode === "plan" ? (options as any).planModeAihubmixModelInfo : (options as any).actModeAihubmixModelInfo,
|
||||
})
|
||||
case "minimax":
|
||||
return new MinimaxHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
minimaxApiKey: options.minimaxApiKey,
|
||||
minimaxApiLine: options.minimaxApiLine,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
})
|
||||
case "hicap":
|
||||
return new HicapHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
hicapApiKey: options.hicapApiKey,
|
||||
hicapModelId: mode === "plan" ? options.planModeHicapModelId : options.actModeHicapModelId,
|
||||
})
|
||||
case "nousResearch":
|
||||
return new NousResearchHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
nousResearchApiKey: options.nousResearchApiKey,
|
||||
apiModelId: mode === "plan" ? options.planModeNousResearchModelId : options.actModeNousResearchModelId,
|
||||
})
|
||||
case "wandb":
|
||||
return new WandbHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
wandbApiKey: options.wandbApiKey,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
})
|
||||
default:
|
||||
return new AnthropicHandler({
|
||||
onRetryAttempt: options.onRetryAttempt,
|
||||
apiKey: options.apiKey,
|
||||
anthropicBaseUrl: options.anthropicBaseUrl,
|
||||
apiModelId: mode === "plan" ? options.planModeApiModelId : options.actModeApiModelId,
|
||||
reasoningEffort: mode === "plan" ? options.planModeReasoningEffort : options.actModeReasoningEffort,
|
||||
thinkingBudgetTokens:
|
||||
mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
export function buildApiHandler(configuration: ApiConfiguration, mode: Mode): ApiHandler {
|
||||
const { planModeApiProvider, actModeApiProvider, ...options } = configuration
|
||||
|
||||
const apiProvider = mode === "plan" ? planModeApiProvider : actModeApiProvider
|
||||
|
||||
// Validate thinking budget tokens against model's maxTokens to prevent API errors
|
||||
// wrapped in a try-catch for safety, but this should never throw
|
||||
try {
|
||||
const thinkingBudgetTokens = mode === "plan" ? options.planModeThinkingBudgetTokens : options.actModeThinkingBudgetTokens
|
||||
if (thinkingBudgetTokens && thinkingBudgetTokens > 0) {
|
||||
const handler = createHandlerForProvider(apiProvider, options, mode)
|
||||
|
||||
const modelInfo = handler.getModel().info
|
||||
if (modelInfo?.maxTokens && modelInfo.maxTokens > 0 && thinkingBudgetTokens > modelInfo.maxTokens) {
|
||||
const clippedValue = modelInfo.maxTokens - 1
|
||||
if (mode === "plan") {
|
||||
options.planModeThinkingBudgetTokens = clippedValue
|
||||
} else {
|
||||
options.actModeThinkingBudgetTokens = clippedValue
|
||||
}
|
||||
} else {
|
||||
return handler // don't rebuild unless its necessary
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
Logger.error("buildApiHandler error:", error)
|
||||
}
|
||||
|
||||
return createHandlerForProvider(apiProvider, options, mode)
|
||||
}
|
||||
|
||||
@@ -1,236 +0,0 @@
|
||||
import { afterEach, describe, it } from "mocha"
|
||||
import sinon from "sinon"
|
||||
import "should"
|
||||
import { anthropicModels } from "@shared/api"
|
||||
import { ANTHROPIC_FAST_MODE_BETA, AnthropicHandler } from "../anthropic"
|
||||
|
||||
describe("AnthropicHandler", () => {
|
||||
afterEach(() => {
|
||||
sinon.restore()
|
||||
})
|
||||
|
||||
const createAsyncIterable = (data: readonly unknown[] = []) => ({
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield* data
|
||||
},
|
||||
})
|
||||
|
||||
describe("getModel", () => {
|
||||
it("should return the fast mode model when configured", () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-6:fast",
|
||||
})
|
||||
|
||||
const result = handler.getModel()
|
||||
|
||||
result.id.should.equal("claude-opus-4-6:fast")
|
||||
result.info.should.deepEqual(anthropicModels["claude-opus-4-6:fast"])
|
||||
})
|
||||
|
||||
it("should return the 1m fast mode model when configured", () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-6:1m:fast",
|
||||
})
|
||||
|
||||
const result = handler.getModel()
|
||||
|
||||
result.id.should.equal("claude-opus-4-6:1m:fast")
|
||||
result.info.should.deepEqual(anthropicModels["claude-opus-4-6:1m:fast"])
|
||||
})
|
||||
|
||||
it("should return the 4.7 model when configured", () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-7",
|
||||
})
|
||||
|
||||
const result = handler.getModel()
|
||||
|
||||
result.id.should.equal("claude-opus-4-7")
|
||||
result.info.should.deepEqual(anthropicModels["claude-opus-4-7"])
|
||||
})
|
||||
|
||||
it("should return the 4.7 1m model when configured", () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-7:1m",
|
||||
})
|
||||
|
||||
const result = handler.getModel()
|
||||
|
||||
result.id.should.equal("claude-opus-4-7:1m")
|
||||
result.info.should.deepEqual(anthropicModels["claude-opus-4-7:1m"])
|
||||
})
|
||||
|
||||
it("should return the 4.8 model when configured", () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-8",
|
||||
})
|
||||
|
||||
const result = handler.getModel()
|
||||
|
||||
result.id.should.equal("claude-opus-4-8")
|
||||
result.info.should.deepEqual(anthropicModels["claude-opus-4-8"])
|
||||
})
|
||||
|
||||
it("should return the 4.8 1m model when configured", () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-8:1m",
|
||||
})
|
||||
|
||||
const result = handler.getModel()
|
||||
|
||||
result.id.should.equal("claude-opus-4-8:1m")
|
||||
result.info.should.deepEqual(anthropicModels["claude-opus-4-8:1m"])
|
||||
})
|
||||
})
|
||||
|
||||
describe("createMessage", () => {
|
||||
it("should route fast mode requests through the beta messages API", async () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-6:fast",
|
||||
})
|
||||
|
||||
const standardCreate = sinon.stub().resolves(createAsyncIterable())
|
||||
const betaCreate = sinon.stub().callsFake(function (this: { _client?: object }, _params: unknown) {
|
||||
should.exist(this._client)
|
||||
return Promise.resolve(createAsyncIterable())
|
||||
})
|
||||
|
||||
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
|
||||
messages: {
|
||||
create: standardCreate,
|
||||
},
|
||||
beta: {
|
||||
messages: {
|
||||
_client: {},
|
||||
create: betaCreate,
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
|
||||
}
|
||||
|
||||
sinon.assert.notCalled(standardCreate)
|
||||
sinon.assert.calledOnce(betaCreate)
|
||||
sinon.assert.calledWithMatch(betaCreate, {
|
||||
model: "claude-opus-4-6",
|
||||
betas: [ANTHROPIC_FAST_MODE_BETA],
|
||||
speed: "fast",
|
||||
stream: true,
|
||||
})
|
||||
})
|
||||
|
||||
it("should include the 1m beta when routing 1m fast mode requests through the beta messages API", async () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-6:1m:fast",
|
||||
})
|
||||
|
||||
const standardCreate = sinon.stub().resolves(createAsyncIterable())
|
||||
const betaCreate = sinon.stub().callsFake(function (this: { _client?: object }, _params: unknown) {
|
||||
should.exist(this._client)
|
||||
return Promise.resolve(createAsyncIterable())
|
||||
})
|
||||
|
||||
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
|
||||
messages: {
|
||||
create: standardCreate,
|
||||
},
|
||||
beta: {
|
||||
messages: {
|
||||
_client: {},
|
||||
create: betaCreate,
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
|
||||
}
|
||||
|
||||
sinon.assert.notCalled(standardCreate)
|
||||
sinon.assert.calledOnce(betaCreate)
|
||||
sinon.assert.calledWithMatch(betaCreate, {
|
||||
model: "claude-opus-4-6",
|
||||
betas: [ANTHROPIC_FAST_MODE_BETA, "context-1m-2025-08-07"],
|
||||
speed: "fast",
|
||||
stream: true,
|
||||
})
|
||||
})
|
||||
|
||||
it("should include the 1m beta header for Claude Opus 4.7 1m requests", async () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-7:1m",
|
||||
reasoningEffort: "high",
|
||||
})
|
||||
|
||||
const standardCreate = sinon.stub().resolves(createAsyncIterable())
|
||||
|
||||
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
|
||||
messages: {
|
||||
create: standardCreate,
|
||||
},
|
||||
beta: {
|
||||
messages: {
|
||||
_client: {},
|
||||
create: sinon.stub().resolves(createAsyncIterable()),
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
|
||||
}
|
||||
|
||||
sinon.assert.calledOnce(standardCreate)
|
||||
const requestBody = standardCreate.firstCall.args[0] as Record<string, any>
|
||||
const requestOptions = standardCreate.firstCall.args[1] as Record<string, any>
|
||||
requestBody.model.should.equal("claude-opus-4-7")
|
||||
requestBody.thinking.should.deepEqual({ type: "adaptive" })
|
||||
requestOptions.should.deepEqual({
|
||||
headers: {
|
||||
"anthropic-beta": "context-1m-2025-08-07",
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it("should use adaptive thinking and output_config for Claude Opus adaptive models", async () => {
|
||||
const handler = new AnthropicHandler({
|
||||
apiKey: "test-api-key",
|
||||
apiModelId: "claude-opus-4-7",
|
||||
reasoningEffort: "xhigh",
|
||||
})
|
||||
|
||||
const standardCreate = sinon.stub().resolves(createAsyncIterable())
|
||||
|
||||
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
|
||||
messages: {
|
||||
create: standardCreate,
|
||||
},
|
||||
beta: {
|
||||
messages: {
|
||||
_client: {},
|
||||
create: sinon.stub().resolves(createAsyncIterable()),
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
for await (const _chunk of handler.createMessage("system prompt", [{ role: "user", content: "Hello" }])) {
|
||||
}
|
||||
|
||||
sinon.assert.calledOnce(standardCreate)
|
||||
const requestBody = standardCreate.firstCall.args[0] as Record<string, any>
|
||||
requestBody.should.have.property("thinking")
|
||||
requestBody.thinking.should.deepEqual({ type: "adaptive" })
|
||||
requestBody.should.have.property("output_config")
|
||||
requestBody.output_config.should.deepEqual({ effort: "xhigh" })
|
||||
should(requestBody.temperature).equal(undefined)
|
||||
})
|
||||
})
|
||||
})
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,468 +0,0 @@
|
||||
import { afterEach, beforeEach, describe, it } from "mocha"
|
||||
import sinon from "sinon"
|
||||
import "should"
|
||||
import { ClaudeCodeHandler } from "@core/api/providers/claude-code"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
|
||||
describe("ClaudeCodeHandler", () => {
|
||||
let handler: ClaudeCodeHandler
|
||||
let sandbox: sinon.SinonSandbox
|
||||
|
||||
beforeEach(() => {
|
||||
sandbox = sinon.createSandbox()
|
||||
handler = new ClaudeCodeHandler({
|
||||
claudeCodePath: "/mock/path",
|
||||
apiModelId: "claude-opus-4-1-20250805",
|
||||
})
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
sandbox.restore()
|
||||
})
|
||||
|
||||
describe("token counting", () => {
|
||||
it("should correctly handle token usage from assistant messages", async () => {
|
||||
// The 'input_tokens' field represents the TOTAL number of input tokens used.
|
||||
// See https://docs.anthropic.com/en/api/messages#usage-object
|
||||
|
||||
// Mock the runClaudeCode function
|
||||
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
|
||||
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
|
||||
|
||||
// Create a proper async generator mock for the Claude Code response
|
||||
async function* mockGenerator() {
|
||||
// First yield the system init
|
||||
yield {
|
||||
type: "system",
|
||||
subtype: "init",
|
||||
apiKeySource: "api",
|
||||
}
|
||||
|
||||
// Yield assistant message with usage data
|
||||
// Example: If base input is 70 tokens, cache read is 20, and cache creation is 10,
|
||||
// then input_tokens from Anthropic API will be 100 (70 + 20 + 10)
|
||||
yield {
|
||||
type: "assistant",
|
||||
message: {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: "Test response",
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
input_tokens: 100, // Total including cache (per Anthropic docs)
|
||||
output_tokens: 50,
|
||||
cache_read_input_tokens: 20, // Already included in input_tokens
|
||||
cache_creation_input_tokens: 10, // Already included in input_tokens
|
||||
},
|
||||
stop_reason: "end_turn",
|
||||
},
|
||||
}
|
||||
|
||||
// Yield result with cost
|
||||
yield {
|
||||
type: "result",
|
||||
result: {},
|
||||
total_cost_usd: 0.005,
|
||||
}
|
||||
}
|
||||
|
||||
runClaudeCodeStub.returns(mockGenerator() as any)
|
||||
|
||||
const systemPrompt = "You are a helpful assistant."
|
||||
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
|
||||
|
||||
const usageData: any[] = []
|
||||
|
||||
// Collect the results
|
||||
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
|
||||
if (chunk.type === "usage") {
|
||||
usageData.push({
|
||||
inputTokens: chunk.inputTokens,
|
||||
outputTokens: chunk.outputTokens,
|
||||
cacheReadTokens: chunk.cacheReadTokens,
|
||||
cacheWriteTokens: chunk.cacheWriteTokens,
|
||||
totalCost: chunk.totalCost,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Verify token counting follows Anthropic API specification
|
||||
usageData.should.have.length(1)
|
||||
usageData[0].should.deepEqual({
|
||||
inputTokens: 100, // Total including cache tokens (per Anthropic API docs)
|
||||
outputTokens: 50,
|
||||
cacheReadTokens: 20, // Tracked separately for reporting
|
||||
cacheWriteTokens: 10, // Tracked separately for reporting
|
||||
totalCost: 0.005,
|
||||
})
|
||||
|
||||
// CRITICAL ASSERTION: Verify that input_tokens is NOT inflated by re-adding cache tokens
|
||||
// The bug would have caused inputTokens to be incorrectly calculated as 130 (100 + 20 + 10)
|
||||
// The fix ensures it remains 100, as per Anthropic's specification
|
||||
usageData[0].inputTokens.should.equal(100) // Correct: matches API response
|
||||
usageData[0].inputTokens.should.not.equal(130) // Would be wrong: double-counting cache tokens
|
||||
})
|
||||
|
||||
it("should handle missing usage fields with nullish coalescing", async () => {
|
||||
// Mock the runClaudeCode function
|
||||
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
|
||||
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
|
||||
|
||||
// Create a proper async generator mock with missing/undefined usage fields
|
||||
async function* mockGenerator() {
|
||||
yield {
|
||||
type: "assistant",
|
||||
message: {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: "Test response",
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
input_tokens: 100,
|
||||
output_tokens: 50,
|
||||
// cache fields are undefined/missing
|
||||
},
|
||||
stop_reason: "end_turn",
|
||||
},
|
||||
}
|
||||
|
||||
yield {
|
||||
type: "result",
|
||||
result: {},
|
||||
total_cost_usd: 0.005,
|
||||
}
|
||||
}
|
||||
|
||||
runClaudeCodeStub.returns(mockGenerator() as any)
|
||||
|
||||
const systemPrompt = "You are a helpful assistant."
|
||||
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
|
||||
|
||||
const usageData: any[] = []
|
||||
|
||||
// Collect the results
|
||||
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
|
||||
if (chunk.type === "usage") {
|
||||
usageData.push({
|
||||
inputTokens: chunk.inputTokens,
|
||||
outputTokens: chunk.outputTokens,
|
||||
cacheReadTokens: chunk.cacheReadTokens,
|
||||
cacheWriteTokens: chunk.cacheWriteTokens,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Verify that undefined cache tokens default to 0
|
||||
usageData.should.have.length(1)
|
||||
usageData[0].should.deepEqual({
|
||||
inputTokens: 100,
|
||||
outputTokens: 50,
|
||||
cacheReadTokens: 0, // Should default to 0
|
||||
cacheWriteTokens: 0, // Should default to 0
|
||||
})
|
||||
})
|
||||
|
||||
it("should handle completely missing usage object", async () => {
|
||||
// Mock the runClaudeCode function
|
||||
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
|
||||
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
|
||||
|
||||
// Create a proper async generator mock with missing usage object
|
||||
async function* mockGenerator() {
|
||||
yield {
|
||||
type: "assistant",
|
||||
message: {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: "Test response",
|
||||
},
|
||||
],
|
||||
// usage is undefined
|
||||
usage: undefined,
|
||||
stop_reason: "end_turn",
|
||||
},
|
||||
}
|
||||
|
||||
// Need to yield a result chunk to trigger usage data emission
|
||||
yield {
|
||||
type: "result",
|
||||
result: {},
|
||||
total_cost_usd: 0,
|
||||
}
|
||||
}
|
||||
|
||||
runClaudeCodeStub.returns(mockGenerator() as any)
|
||||
|
||||
const systemPrompt = "You are a helpful assistant."
|
||||
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
|
||||
|
||||
const usageData: any[] = []
|
||||
|
||||
// Collect the results
|
||||
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
|
||||
if (chunk.type === "usage") {
|
||||
usageData.push({
|
||||
inputTokens: chunk.inputTokens,
|
||||
outputTokens: chunk.outputTokens,
|
||||
cacheReadTokens: chunk.cacheReadTokens,
|
||||
cacheWriteTokens: chunk.cacheWriteTokens,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// All token counts should default to 0 when usage is undefined
|
||||
usageData.should.have.length(1)
|
||||
usageData[0].should.deepEqual({
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
cacheWriteTokens: 0,
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe("error handling", () => {
|
||||
it("should not crash when assistant message has empty content array", async () => {
|
||||
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
|
||||
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
|
||||
|
||||
async function* mockGenerator() {
|
||||
yield {
|
||||
type: "assistant",
|
||||
message: {
|
||||
content: [], // empty content — triggered TypeError in older code
|
||||
usage: {
|
||||
input_tokens: 10,
|
||||
output_tokens: 0,
|
||||
},
|
||||
stop_reason: "end_turn",
|
||||
},
|
||||
}
|
||||
|
||||
yield {
|
||||
type: "result",
|
||||
result: {},
|
||||
total_cost_usd: 0,
|
||||
}
|
||||
}
|
||||
|
||||
runClaudeCodeStub.returns(mockGenerator() as any)
|
||||
|
||||
const chunks: any[] = []
|
||||
// Should not throw
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
const usageChunk = chunks.find((c) => c.type === "usage")
|
||||
usageChunk.should.be.ok()
|
||||
usageChunk.inputTokens.should.equal(10)
|
||||
})
|
||||
|
||||
it("should throw when result has is_error=true (e.g. rate limit with no assistant message)", async () => {
|
||||
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
|
||||
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
|
||||
|
||||
async function* mockGenerator() {
|
||||
yield {
|
||||
type: "system",
|
||||
subtype: "init",
|
||||
apiKeySource: "none",
|
||||
}
|
||||
|
||||
yield {
|
||||
type: "system",
|
||||
subtype: "rate_limit_event",
|
||||
message: "Rate limit hit",
|
||||
retryAfterSeconds: 30,
|
||||
}
|
||||
|
||||
// No assistant message — CLI hit rate limit and gave up
|
||||
yield {
|
||||
type: "result",
|
||||
subtype: "error",
|
||||
is_error: true,
|
||||
result: "Rate limit exceeded",
|
||||
total_cost_usd: 0,
|
||||
duration_ms: 1000,
|
||||
duration_api_ms: 500,
|
||||
num_turns: 0,
|
||||
session_id: "test",
|
||||
}
|
||||
}
|
||||
|
||||
runClaudeCodeStub.returns(mockGenerator() as any)
|
||||
|
||||
let thrownError: Error | undefined
|
||||
try {
|
||||
for await (const _ of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
// consume
|
||||
}
|
||||
} catch (err) {
|
||||
thrownError = err as Error
|
||||
}
|
||||
|
||||
thrownError!.message.should.containEql("Rate limit exceeded")
|
||||
})
|
||||
|
||||
it("should ignore rate_limit_event system messages without throwing", async () => {
|
||||
const runClaudeCodeModule = await import("@/integrations/claude-code/run")
|
||||
const runClaudeCodeStub = sandbox.stub(runClaudeCodeModule, "runClaudeCode")
|
||||
|
||||
async function* mockGenerator() {
|
||||
yield {
|
||||
type: "system",
|
||||
subtype: "init",
|
||||
apiKeySource: "none",
|
||||
}
|
||||
|
||||
// Newer Claude Code CLI emits this during rate limiting
|
||||
yield {
|
||||
type: "system",
|
||||
subtype: "rate_limit_event",
|
||||
message: "Rate limit hit, retrying...",
|
||||
retryAfterSeconds: 30,
|
||||
}
|
||||
|
||||
yield {
|
||||
type: "assistant",
|
||||
message: {
|
||||
content: [{ type: "text", text: "Response after retry" }],
|
||||
usage: { input_tokens: 20, output_tokens: 10 },
|
||||
stop_reason: "end_turn",
|
||||
},
|
||||
}
|
||||
|
||||
yield {
|
||||
type: "result",
|
||||
result: {},
|
||||
total_cost_usd: 0,
|
||||
}
|
||||
}
|
||||
|
||||
runClaudeCodeStub.returns(mockGenerator() as any)
|
||||
|
||||
const textChunks: string[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
if (chunk.type === "text") textChunks.push(chunk.text)
|
||||
}
|
||||
|
||||
textChunks.should.deepEqual(["Response after retry"])
|
||||
})
|
||||
})
|
||||
|
||||
describe("getModel", () => {
|
||||
it("should return the correct model when specified", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "claude-sonnet-4-5-20250929",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("claude-sonnet-4-5-20250929")
|
||||
})
|
||||
|
||||
it("should support Opus 4.6 1m model id", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "claude-opus-4-6[1m]",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("claude-opus-4-6[1m]")
|
||||
model.info.contextWindow.should.equal(1_000_000)
|
||||
})
|
||||
|
||||
it("should support Opus 4.7 model id", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "claude-opus-4-7",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("claude-opus-4-7")
|
||||
model.info.contextWindow.should.equal(200_000)
|
||||
})
|
||||
|
||||
it("should support Opus 4.7 1m model id", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "claude-opus-4-7[1m]",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("claude-opus-4-7[1m]")
|
||||
model.info.contextWindow.should.equal(1_000_000)
|
||||
})
|
||||
|
||||
it("should support Opus 4.8 model id", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "claude-opus-4-8",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("claude-opus-4-8")
|
||||
model.info.contextWindow.should.equal(200_000)
|
||||
})
|
||||
|
||||
it("should support Opus 4.8 1m model id", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "claude-opus-4-8[1m]",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("claude-opus-4-8[1m]")
|
||||
model.info.contextWindow.should.equal(1_000_000)
|
||||
})
|
||||
|
||||
it("should support Opus 1m alias model id", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "opus[1m]",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("opus[1m]")
|
||||
model.info.contextWindow.should.equal(1_000_000)
|
||||
})
|
||||
|
||||
it("should support Sonnet 1m alias model id", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "sonnet[1m]",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("sonnet[1m]")
|
||||
model.info.contextWindow.should.equal(1_000_000)
|
||||
})
|
||||
|
||||
it("should support Sonnet 4.5 1m model id", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "claude-sonnet-4-5-20250929[1m]",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("claude-sonnet-4-5-20250929[1m]")
|
||||
model.info.contextWindow.should.equal(1_000_000)
|
||||
})
|
||||
|
||||
it("should support Sonnet 4.6 1m model id", () => {
|
||||
const handler = new ClaudeCodeHandler({
|
||||
apiModelId: "claude-sonnet-4-6[1m]",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("claude-sonnet-4-6[1m]")
|
||||
model.info.contextWindow.should.equal(1_000_000)
|
||||
})
|
||||
|
||||
it("should return default model when not specified", () => {
|
||||
const handler = new ClaudeCodeHandler({})
|
||||
|
||||
const model = handler.getModel()
|
||||
// The default model should be set
|
||||
model.id.should.be.type("string")
|
||||
model.info.should.be.type("object")
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,166 +0,0 @@
|
||||
import "should"
|
||||
import { openRouterDefaultModelInfo } from "@shared/api"
|
||||
import sinon from "sinon"
|
||||
import { ClineAccountService } from "@/services/account/ClineAccountService"
|
||||
import { AuthService } from "@/services/auth/AuthService"
|
||||
import { ClineHandler } from "../cline"
|
||||
|
||||
describe("ClineHandler", () => {
|
||||
afterEach(() => {
|
||||
sinon.restore()
|
||||
})
|
||||
|
||||
const createAsyncIterable = (data: any[] = []) => ({
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield* data
|
||||
},
|
||||
})
|
||||
|
||||
const createHandler = (options: ConstructorParameters<typeof ClineHandler>[0]) => {
|
||||
sinon.stub(ClineAccountService, "getInstance").returns({} as any)
|
||||
sinon.stub(AuthService, "getInstance").returns({} as any)
|
||||
return new ClineHandler(options)
|
||||
}
|
||||
|
||||
it("should handle usage-only chunks when delta is missing", async () => {
|
||||
const handler = createHandler({})
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
choices: [{}],
|
||||
usage: {
|
||||
prompt_tokens: 17,
|
||||
completion_tokens: 9,
|
||||
},
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
|
||||
sinon.stub(handler, "getModel").returns({
|
||||
id: "openai/gpt-4o-mini",
|
||||
info: openRouterDefaultModelInfo,
|
||||
})
|
||||
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
chunks.should.deepEqual([
|
||||
{
|
||||
type: "usage",
|
||||
cacheWriteTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
inputTokens: 17,
|
||||
outputTokens: 9,
|
||||
totalCost: 0,
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it("should read Anthropic-style cache creation and read tokens from usage chunks", async () => {
|
||||
const handler = createHandler({})
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
choices: [{}],
|
||||
usage: {
|
||||
prompt_tokens: 1000,
|
||||
completion_tokens: 200,
|
||||
prompt_tokens_details: {
|
||||
cached_tokens: 500,
|
||||
},
|
||||
cache_creation_input_tokens: 300,
|
||||
},
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
|
||||
sinon.stub(handler, "getModel").returns({
|
||||
id: "anthropic/claude-sonnet-4.6",
|
||||
info: openRouterDefaultModelInfo,
|
||||
})
|
||||
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
chunks.should.deepEqual([
|
||||
{
|
||||
type: "usage",
|
||||
cacheWriteTokens: 300,
|
||||
cacheReadTokens: 500,
|
||||
inputTokens: 200,
|
||||
outputTokens: 200,
|
||||
totalCost: 0,
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it("should forward enableParallelToolCalling to OpenRouter payload", async () => {
|
||||
const handler = createHandler({ enableParallelToolCalling: true })
|
||||
const createStub = sinon.stub().resolves(createAsyncIterable([]))
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: createStub,
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
|
||||
sinon.stub(handler, "getModel").returns({
|
||||
id: "openai/gpt-4o-mini",
|
||||
info: openRouterDefaultModelInfo,
|
||||
})
|
||||
|
||||
const tools = [
|
||||
{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } },
|
||||
] as any
|
||||
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
|
||||
// drain stream
|
||||
}
|
||||
|
||||
const payload = createStub.firstCall.args[0]
|
||||
payload.parallel_tool_calls.should.equal(true)
|
||||
})
|
||||
|
||||
it("should send cache_control for qwen3.7-max without changing the selected Cline model id", async () => {
|
||||
const handler = createHandler({
|
||||
openRouterModelId: "qwen/qwen3.7-max",
|
||||
openRouterModelInfo: openRouterDefaultModelInfo,
|
||||
})
|
||||
const createStub = sinon.stub().resolves(createAsyncIterable([]))
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: createStub,
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").resolves(fakeClient as any)
|
||||
sinon.stub(handler as any, "getFreeModelIdSet").resolves(new Set())
|
||||
|
||||
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
// drain stream
|
||||
}
|
||||
|
||||
handler.getModel().id.should.equal("qwen/qwen3.7-max")
|
||||
const payload = createStub.firstCall.args[0]
|
||||
payload.model.should.equal("qwen/qwen3.7-max")
|
||||
payload.messages[0].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
|
||||
payload.messages[1].content[0].cache_control.should.deepEqual({ type: "ephemeral" })
|
||||
})
|
||||
})
|
||||
@@ -1,97 +0,0 @@
|
||||
import "should"
|
||||
import sinon from "sinon"
|
||||
import { FireworksHandler } from "../fireworks"
|
||||
|
||||
describe("FireworksHandler", () => {
|
||||
afterEach(() => {
|
||||
sinon.restore()
|
||||
})
|
||||
|
||||
const createAsyncIterable = (data: any[] = []) => ({
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield* data
|
||||
},
|
||||
})
|
||||
|
||||
it("should handle usage-only chunks when delta is missing", async () => {
|
||||
const handler = new FireworksHandler({
|
||||
fireworksApiKey: "test-api-key",
|
||||
fireworksModelId: "accounts/fireworks/models/llama-v3p1-8b-instruct",
|
||||
})
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
choices: [{}],
|
||||
usage: {
|
||||
prompt_tokens: 19,
|
||||
completion_tokens: 4,
|
||||
},
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
|
||||
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
chunks.should.deepEqual([
|
||||
{
|
||||
type: "usage",
|
||||
inputTokens: 19,
|
||||
outputTokens: 4,
|
||||
cacheReadTokens: 0,
|
||||
cacheWriteTokens: 0,
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it("should read cache hits from prompt_tokens_details when hit tokens are not present", async () => {
|
||||
const handler = new FireworksHandler({
|
||||
fireworksApiKey: "test-api-key",
|
||||
fireworksModelId: "accounts/fireworks/models/llama-v3p1-8b-instruct",
|
||||
})
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
choices: [{}],
|
||||
usage: {
|
||||
prompt_tokens: 60,
|
||||
completion_tokens: 12,
|
||||
prompt_tokens_details: { cached_tokens: 20 },
|
||||
prompt_cache_miss_tokens: 40,
|
||||
},
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
|
||||
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
chunks.should.deepEqual([
|
||||
{
|
||||
type: "usage",
|
||||
inputTokens: 60,
|
||||
outputTokens: 12,
|
||||
cacheReadTokens: 20,
|
||||
cacheWriteTokens: 40,
|
||||
},
|
||||
])
|
||||
})
|
||||
})
|
||||
@@ -1,235 +0,0 @@
|
||||
import "should"
|
||||
import sinon from "sinon"
|
||||
import { GeminiHandler } from "../gemini"
|
||||
|
||||
describe("GeminiHandler", () => {
|
||||
afterEach(() => {
|
||||
sinon.restore()
|
||||
})
|
||||
|
||||
const createAsyncIterable = (data: any[] = []) => ({
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield* data
|
||||
},
|
||||
})
|
||||
|
||||
it("caps maxOutputTokens to 8192 for Flash models", async () => {
|
||||
const handler = new GeminiHandler({
|
||||
geminiApiKey: "test-api-key",
|
||||
apiModelId: "gemini-2.5-flash",
|
||||
})
|
||||
|
||||
const generateContentStream = sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
responseId: "resp-1",
|
||||
usageMetadata: {
|
||||
promptTokenCount: 10,
|
||||
candidatesTokenCount: 20,
|
||||
cachedContentTokenCount: 0,
|
||||
thoughtsTokenCount: 0,
|
||||
},
|
||||
},
|
||||
]),
|
||||
)
|
||||
sinon.stub(handler as any, "ensureClient").returns({
|
||||
models: { generateContentStream },
|
||||
} as any)
|
||||
|
||||
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
|
||||
// Consume stream to trigger request execution.
|
||||
}
|
||||
|
||||
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
|
||||
requestArgs.config.should.have.property("maxOutputTokens", 8_192)
|
||||
})
|
||||
|
||||
it("supports Gemini 3.5 Flash model metadata", async () => {
|
||||
const handler = new GeminiHandler({
|
||||
geminiApiKey: "test-api-key",
|
||||
apiModelId: "gemini-3.5-flash",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("gemini-3.5-flash")
|
||||
model.info.contextWindow!.should.equal(1_048_576)
|
||||
model.info.inputPrice!.should.equal(1.5)
|
||||
model.info.outputPrice!.should.equal(9)
|
||||
model.info.cacheReadsPrice!.should.equal(0.15)
|
||||
model.info.supportsReasoning!.should.equal(true)
|
||||
|
||||
const generateContentStream = sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
responseId: "resp-35",
|
||||
usageMetadata: {
|
||||
promptTokenCount: 10,
|
||||
candidatesTokenCount: 20,
|
||||
cachedContentTokenCount: 0,
|
||||
thoughtsTokenCount: 0,
|
||||
},
|
||||
},
|
||||
]),
|
||||
)
|
||||
sinon.stub(handler as any, "ensureClient").returns({
|
||||
models: { generateContentStream },
|
||||
} as any)
|
||||
|
||||
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
|
||||
// Consume stream to trigger request execution.
|
||||
}
|
||||
|
||||
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
|
||||
requestArgs.model.should.equal("gemini-3.5-flash")
|
||||
requestArgs.config.should.have.property("maxOutputTokens", 8_192)
|
||||
requestArgs.config.thinkingConfig.should.deepEqual({
|
||||
thinkingBudget: undefined,
|
||||
thinkingLevel: "LOW",
|
||||
includeThoughts: true,
|
||||
})
|
||||
})
|
||||
|
||||
it("does not set maxOutputTokens for non-Flash models", async () => {
|
||||
const handler = new GeminiHandler({
|
||||
geminiApiKey: "test-api-key",
|
||||
apiModelId: "gemini-2.5-pro",
|
||||
})
|
||||
|
||||
const generateContentStream = sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
responseId: "resp-2",
|
||||
usageMetadata: {
|
||||
promptTokenCount: 10,
|
||||
candidatesTokenCount: 20,
|
||||
cachedContentTokenCount: 0,
|
||||
thoughtsTokenCount: 0,
|
||||
},
|
||||
},
|
||||
]),
|
||||
)
|
||||
sinon.stub(handler as any, "ensureClient").returns({
|
||||
models: { generateContentStream },
|
||||
} as any)
|
||||
|
||||
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }] as any)) {
|
||||
// Consume stream to trigger request execution.
|
||||
}
|
||||
|
||||
const requestArgs = generateContentStream.firstCall.args[0] as Record<string, any>
|
||||
requestArgs.config.should.not.have.property("maxOutputTokens")
|
||||
})
|
||||
|
||||
it("should emit unique tool call IDs when multiple function calls share one responseId", async () => {
|
||||
const handler = new GeminiHandler({
|
||||
geminiApiKey: "test-api-key",
|
||||
})
|
||||
|
||||
const fakeClient = {
|
||||
models: {
|
||||
generateContentStream: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
responseId: "resp_1",
|
||||
candidates: [
|
||||
{
|
||||
content: {
|
||||
parts: [
|
||||
{
|
||||
functionCall: {
|
||||
name: "read_file",
|
||||
args: { path: ".nvmrc" },
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
responseId: "resp_1",
|
||||
candidates: [
|
||||
{
|
||||
content: {
|
||||
parts: [
|
||||
{
|
||||
functionCall: {
|
||||
name: "read_file",
|
||||
args: { path: ".gitattributes" },
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
|
||||
|
||||
const tools = [{ name: "read_file", description: "read file", parameters: { type: "OBJECT" } }] as any
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
|
||||
if (chunk.type === "tool_calls") {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
}
|
||||
|
||||
chunks.should.have.length(2)
|
||||
chunks[0].tool_call.function.id.should.equal("resp_1-tool-0")
|
||||
chunks[1].tool_call.function.id.should.equal("resp_1-tool-1")
|
||||
chunks[0].tool_call.call_id.should.equal(chunks[0].tool_call.function.id)
|
||||
chunks[1].tool_call.call_id.should.equal(chunks[1].tool_call.function.id)
|
||||
JSON.parse(chunks[0].tool_call.function.arguments).path.should.equal(".nvmrc")
|
||||
JSON.parse(chunks[1].tool_call.function.arguments).path.should.equal(".gitattributes")
|
||||
})
|
||||
|
||||
it("should preserve Gemini-provided functionCall.id when present", async () => {
|
||||
const handler = new GeminiHandler({
|
||||
geminiApiKey: "test-api-key",
|
||||
})
|
||||
|
||||
const fakeClient = {
|
||||
models: {
|
||||
generateContentStream: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
responseId: "resp_2",
|
||||
candidates: [
|
||||
{
|
||||
content: {
|
||||
parts: [
|
||||
{
|
||||
functionCall: {
|
||||
id: "call_alpha",
|
||||
name: "read_file",
|
||||
args: { path: ".nvmrc" },
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
|
||||
|
||||
const tools = [{ name: "read_file", description: "read file", parameters: { type: "OBJECT" } }] as any
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
|
||||
if (chunk.type === "tool_calls") {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
}
|
||||
|
||||
chunks.should.have.length(1)
|
||||
chunks[0].tool_call.function.id.should.equal("call_alpha")
|
||||
chunks[0].tool_call.call_id.should.equal("call_alpha")
|
||||
JSON.parse(chunks[0].tool_call.function.arguments).path.should.equal(".nvmrc")
|
||||
})
|
||||
})
|
||||
@@ -1,326 +0,0 @@
|
||||
import { LiteLlmHandler, type LiteLlmModelInfoResponse } from "@core/api/providers/litellm"
|
||||
import { convertToOpenAiMessages } from "@core/api/transform/openai-format"
|
||||
import { liteLlmModelInfoSaneDefaults } from "@shared/api" // used in getModel tests
|
||||
import { expect } from "chai"
|
||||
import sinon from "sinon"
|
||||
import { StateManager } from "@/core/storage/StateManager" // used in getModel tests
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { mockFetchForTesting } from "@/shared/net"
|
||||
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: sinon.stub(),
|
||||
},
|
||||
},
|
||||
baseURL: "https://fake.example",
|
||||
}
|
||||
|
||||
describe("LiteLlmHandler", () => {
|
||||
const mockFetch = sinon.stub()
|
||||
let doneMockingFetch: (value: any) => void = () => {}
|
||||
|
||||
const mockModelFetch = (modelInfo: LiteLlmModelInfoResponse["data"][number]) => {
|
||||
mockFetch.resolves({
|
||||
ok: true,
|
||||
json: () =>
|
||||
Promise.resolve({
|
||||
data: [modelInfo],
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
let handler: LiteLlmHandler
|
||||
|
||||
const mockHandlerChat = () => {
|
||||
sinon.stub(handler, "ensureClient" as any).returns(fakeClient)
|
||||
}
|
||||
|
||||
const initializeHandler = (model: string) => {
|
||||
handler = new LiteLlmHandler({
|
||||
liteLlmApiKey: "test-api-key",
|
||||
liteLlmBaseUrl: "http://localhost:4000",
|
||||
liteLlmUsePromptCache: true,
|
||||
liteLlmModelId: model,
|
||||
})
|
||||
|
||||
mockHandlerChat()
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
fakeClient.chat.completions.create.resetHistory()
|
||||
|
||||
mockFetchForTesting(mockFetch, () => {
|
||||
return new Promise((resolve) => {
|
||||
doneMockingFetch = resolve
|
||||
})
|
||||
})
|
||||
|
||||
// Configure the stub to return a stream that closes immediately with usage data
|
||||
fakeClient.chat.completions.create.resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
choices: [{ delta: { content: "test response" } }],
|
||||
},
|
||||
{
|
||||
choices: [{}],
|
||||
usage: {
|
||||
prompt_tokens: 100,
|
||||
completion_tokens: 50,
|
||||
cache_creation_input_tokens: 20,
|
||||
cache_read_input_tokens: 10,
|
||||
},
|
||||
},
|
||||
]),
|
||||
)
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
sinon.reset()
|
||||
doneMockingFetch(void 0)
|
||||
})
|
||||
|
||||
const createAsyncIterable = (data: any[] = []) => {
|
||||
return {
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield* data
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
describe("prompt cache", () => {
|
||||
const setModelData = (model: string, supportsPromptCaching: boolean) => {
|
||||
mockModelFetch({
|
||||
model_name: model,
|
||||
litellm_params: {
|
||||
model,
|
||||
},
|
||||
model_info: {
|
||||
supports_prompt_caching: supportsPromptCaching,
|
||||
input_cost_per_token: 0.01,
|
||||
output_cost_per_token: 0.02,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
describe("when the model doesn't support prompt caching", () => {
|
||||
const model = "openai/gpt-5"
|
||||
|
||||
beforeEach(() => {
|
||||
initializeHandler(model)
|
||||
setModelData(model, false)
|
||||
})
|
||||
|
||||
it("sends the system prompt and messages with the openai format", async () => {
|
||||
const systemPrompt = "Test System Prompt"
|
||||
const messages: ClineStorageMessage[] = [
|
||||
{
|
||||
role: "user",
|
||||
content: "first message",
|
||||
},
|
||||
{
|
||||
role: "assistant",
|
||||
content: "first response",
|
||||
},
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: "test",
|
||||
},
|
||||
{
|
||||
type: "text",
|
||||
text: "second message",
|
||||
},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
for await (const _ of handler.createMessage(systemPrompt, messages)) {
|
||||
}
|
||||
|
||||
sinon.assert.calledOnce(fakeClient.chat.completions.create)
|
||||
|
||||
const callArgs = fakeClient.chat.completions.create.getCall(0).args[0]
|
||||
|
||||
const systemPromptMessage = callArgs.messages.shift()
|
||||
expect(systemPromptMessage).to.deep.equal({
|
||||
role: "system",
|
||||
content: systemPrompt,
|
||||
})
|
||||
|
||||
expect(callArgs.messages).to.deep.equal(convertToOpenAiMessages(messages))
|
||||
})
|
||||
})
|
||||
|
||||
describe("when the model supports prompt caching", () => {
|
||||
const model = "anthropic/claude-sonnet-4-20250514"
|
||||
|
||||
beforeEach(() => {
|
||||
initializeHandler(model)
|
||||
|
||||
setModelData(model, true)
|
||||
})
|
||||
|
||||
it("inserts the cache control in the system prompt and the last two user messages", async () => {
|
||||
const systemPrompt = "Test System Prompt"
|
||||
const messages: ClineStorageMessage[] = [
|
||||
{
|
||||
role: "user",
|
||||
content: "first message",
|
||||
},
|
||||
{
|
||||
role: "assistant",
|
||||
content: "first response",
|
||||
},
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: "test",
|
||||
},
|
||||
{
|
||||
type: "text",
|
||||
text: "second message",
|
||||
},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
for await (const _ of handler.createMessage(systemPrompt, messages)) {
|
||||
}
|
||||
|
||||
sinon.assert.calledOnce(fakeClient.chat.completions.create)
|
||||
|
||||
const callArgs = fakeClient.chat.completions.create.getCall(0).args[0]
|
||||
|
||||
expect(callArgs.messages[0]).to.deep.equal({
|
||||
role: "system",
|
||||
content: [
|
||||
{
|
||||
text: systemPrompt,
|
||||
type: "text",
|
||||
cache_control: {
|
||||
type: "ephemeral",
|
||||
},
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const sentMessages = callArgs.messages
|
||||
expect(sentMessages.length).to.equal(4)
|
||||
|
||||
const firstUserMessage = sentMessages[1]
|
||||
|
||||
expect(firstUserMessage).to.deep.equal({
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: "first message",
|
||||
cache_control: {
|
||||
type: "ephemeral",
|
||||
},
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const lastUserMessage = sentMessages[3]
|
||||
expect(lastUserMessage.content[0]).to.deep.equal({
|
||||
type: "text",
|
||||
text: "test",
|
||||
})
|
||||
|
||||
const lastContentBlock = lastUserMessage.content[lastUserMessage.content.length - 1]
|
||||
expect(lastContentBlock).to.deep.equal({
|
||||
type: "text",
|
||||
text: "second message",
|
||||
cache_control: {
|
||||
type: "ephemeral",
|
||||
},
|
||||
})
|
||||
|
||||
expect(callArgs.model).to.be.a("string")
|
||||
expect(callArgs.stream).to.equal(true)
|
||||
expect(callArgs.stream_options).to.deep.equal({ include_usage: true })
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe("getModel", () => {
|
||||
let stateManagerStub: sinon.SinonStub
|
||||
|
||||
beforeEach(() => {
|
||||
stateManagerStub = sinon.stub(StateManager, "get").returns({
|
||||
getModelInfo: () => null,
|
||||
} as any)
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
stateManagerStub.restore()
|
||||
})
|
||||
|
||||
it("returns sane defaults when no liteLlmModelInfo option is provided", () => {
|
||||
const h = new LiteLlmHandler({
|
||||
liteLlmApiKey: "test",
|
||||
liteLlmModelId: "some-model",
|
||||
})
|
||||
const model = h.getModel()
|
||||
expect(model.id).to.equal("some-model")
|
||||
expect(model.info.contextWindow).to.equal(liteLlmModelInfoSaneDefaults.contextWindow)
|
||||
})
|
||||
|
||||
it("returns user-configured model info when liteLlmModelInfo is provided and no cache exists", () => {
|
||||
const h = new LiteLlmHandler({
|
||||
liteLlmApiKey: "test",
|
||||
liteLlmModelId: "claude-sonnet-4-6",
|
||||
liteLlmModelInfo: {
|
||||
contextWindow: 1_000_000,
|
||||
maxTokens: 8192,
|
||||
supportsImages: true,
|
||||
supportsPromptCache: true,
|
||||
inputPrice: 3,
|
||||
outputPrice: 15,
|
||||
cacheWritesPrice: 3.75,
|
||||
cacheReadsPrice: 0.3,
|
||||
},
|
||||
})
|
||||
const model = h.getModel()
|
||||
expect(model.id).to.equal("claude-sonnet-4-6")
|
||||
expect(model.info.contextWindow).to.equal(1_000_000)
|
||||
})
|
||||
|
||||
it("prefers StateManager cached model info over user-configured liteLlmModelInfo", () => {
|
||||
const cachedInfo = {
|
||||
contextWindow: 200_000,
|
||||
maxTokens: 4096,
|
||||
supportsImages: true,
|
||||
supportsPromptCache: true,
|
||||
inputPrice: 0,
|
||||
outputPrice: 0,
|
||||
}
|
||||
stateManagerStub.returns({
|
||||
getModelInfo: () => cachedInfo,
|
||||
} as any)
|
||||
|
||||
const h = new LiteLlmHandler({
|
||||
liteLlmApiKey: "test",
|
||||
liteLlmModelId: "claude-sonnet-4-6",
|
||||
liteLlmModelInfo: {
|
||||
contextWindow: 1_000_000,
|
||||
maxTokens: 8192,
|
||||
supportsImages: true,
|
||||
supportsPromptCache: true,
|
||||
inputPrice: 3,
|
||||
outputPrice: 15,
|
||||
cacheWritesPrice: 3.75,
|
||||
cacheReadsPrice: 0.3,
|
||||
},
|
||||
})
|
||||
const model = h.getModel()
|
||||
expect(model.info.contextWindow).to.equal(200_000)
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,53 +0,0 @@
|
||||
import "should"
|
||||
import { moonshotModels } from "@shared/api"
|
||||
import type { ClineStorageMessage } from "@shared/messages/content"
|
||||
import sinon from "sinon"
|
||||
import { MoonshotHandler } from "../moonshot"
|
||||
|
||||
interface MoonshotRequestPayload {
|
||||
model: string
|
||||
temperature: number
|
||||
max_tokens: number
|
||||
}
|
||||
|
||||
describe("MoonshotHandler", () => {
|
||||
afterEach(() => {
|
||||
sinon.restore()
|
||||
})
|
||||
|
||||
const createAsyncIterable = (data: unknown[] = []): AsyncIterable<unknown> => ({
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield* data
|
||||
},
|
||||
})
|
||||
|
||||
it("supports kimi-k2.6 model metadata", async () => {
|
||||
const handler = new MoonshotHandler({
|
||||
moonshotApiKey: "test-api-key",
|
||||
apiModelId: "kimi-k2.6",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("kimi-k2.6")
|
||||
model.info.should.deepEqual(moonshotModels["kimi-k2.6"])
|
||||
|
||||
const createStub = sinon.stub().resolves(createAsyncIterable([]))
|
||||
sinon.stub(handler as unknown as { ensureClient: () => unknown }, "ensureClient").returns({
|
||||
chat: {
|
||||
completions: {
|
||||
create: createStub,
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
const messages: ClineStorageMessage[] = [{ role: "user", content: "hi" }]
|
||||
for await (const _chunk of handler.createMessage("system", messages)) {
|
||||
// Consume stream to trigger request execution.
|
||||
}
|
||||
|
||||
const payload = createStub.firstCall.args[0] as MoonshotRequestPayload
|
||||
payload.model.should.equal("kimi-k2.6")
|
||||
payload.temperature.should.equal(moonshotModels["kimi-k2.6"].temperature)
|
||||
payload.max_tokens.should.equal(moonshotModels["kimi-k2.6"].maxTokens)
|
||||
})
|
||||
})
|
||||
@@ -1,93 +0,0 @@
|
||||
import { expect } from "chai"
|
||||
import { afterEach, describe, it } from "mocha"
|
||||
import sinon from "sinon"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { ApiFormat } from "@/shared/proto/index.cline"
|
||||
import { OcaHandler } from "../oca"
|
||||
|
||||
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
|
||||
|
||||
async function collectChunks(stream: AsyncGenerator<any>) {
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of stream) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
return chunks
|
||||
}
|
||||
|
||||
describe("OcaHandler.createMessage", () => {
|
||||
afterEach(() => {
|
||||
sinon.restore()
|
||||
})
|
||||
|
||||
it("routes OPENAI_RESPONSES models to createMessageResponsesApi", async () => {
|
||||
const handler = new OcaHandler({
|
||||
ocaModelInfo: { apiFormat: ApiFormat.OPENAI_RESPONSES } as any,
|
||||
})
|
||||
|
||||
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
|
||||
yield { type: "text", text: "chat" }
|
||||
})
|
||||
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
|
||||
yield { type: "text", text: "responses" }
|
||||
})
|
||||
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
|
||||
yield { type: "text", text: "messages" }
|
||||
})
|
||||
|
||||
const chunks = await collectChunks(handler.createMessage("system", messages))
|
||||
|
||||
expect(chunks).to.deep.equal([{ type: "text", text: "responses" }])
|
||||
sinon.assert.notCalled(chatStub)
|
||||
sinon.assert.calledOnce(responsesStub)
|
||||
sinon.assert.notCalled(messagesStub)
|
||||
})
|
||||
|
||||
it("routes ANTHROPIC_CHAT models to createMessageMessagesApi", async () => {
|
||||
const handler = new OcaHandler({
|
||||
ocaModelInfo: { apiFormat: ApiFormat.ANTHROPIC_CHAT } as any,
|
||||
})
|
||||
|
||||
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
|
||||
yield { type: "text", text: "chat" }
|
||||
})
|
||||
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
|
||||
yield { type: "text", text: "responses" }
|
||||
})
|
||||
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
|
||||
yield { type: "text", text: "messages" }
|
||||
})
|
||||
|
||||
const chunks = await collectChunks(handler.createMessage("system", messages))
|
||||
|
||||
expect(chunks).to.deep.equal([{ type: "text", text: "messages" }])
|
||||
sinon.assert.notCalled(chatStub)
|
||||
sinon.assert.notCalled(responsesStub)
|
||||
sinon.assert.calledOnce(messagesStub)
|
||||
})
|
||||
|
||||
it("defaults to createMessageChatApi for OPENAI_CHAT and undefined apiFormat", async () => {
|
||||
for (const apiFormat of [ApiFormat.OPENAI_CHAT, undefined]) {
|
||||
const handler = new OcaHandler({
|
||||
ocaModelInfo: { apiFormat } as any,
|
||||
})
|
||||
|
||||
const chatStub = sinon.stub(handler as any, "createMessageChatApi").callsFake(async function* () {
|
||||
yield { type: "text", text: "chat" }
|
||||
})
|
||||
const responsesStub = sinon.stub(handler as any, "createMessageResponsesApi").callsFake(async function* () {
|
||||
yield { type: "text", text: "responses" }
|
||||
})
|
||||
const messagesStub = sinon.stub(handler as any, "createMessageMessagesApi").callsFake(async function* () {
|
||||
yield { type: "text", text: "messages" }
|
||||
})
|
||||
|
||||
const chunks = await collectChunks(handler.createMessage("system", messages))
|
||||
|
||||
expect(chunks).to.deep.equal([{ type: "text", text: "chat" }])
|
||||
sinon.assert.calledOnce(chatStub)
|
||||
sinon.assert.notCalled(responsesStub)
|
||||
sinon.assert.notCalled(messagesStub)
|
||||
}
|
||||
})
|
||||
})
|
||||
@@ -1,231 +0,0 @@
|
||||
import { afterEach, before, beforeEach, describe, it } from "mocha"
|
||||
import "should"
|
||||
import { ApiHandlerOptions } from "@shared/api"
|
||||
import axios from "axios"
|
||||
import sinon from "sinon"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { OllamaHandler } from "../ollama"
|
||||
|
||||
describe("OllamaHandler", () => {
|
||||
let ollamaAvailable = false
|
||||
|
||||
// Check if Ollama is running before running tests
|
||||
before(async function () {
|
||||
this.timeout(5000)
|
||||
try {
|
||||
await axios.get("http://localhost:11434/api/version", { timeout: 2000 })
|
||||
ollamaAvailable = true
|
||||
} catch (_error) {
|
||||
console.log("Ollama server not available, skipping tests")
|
||||
ollamaAvailable = false
|
||||
}
|
||||
})
|
||||
let handler: OllamaHandler
|
||||
let options: ApiHandlerOptions
|
||||
let clock: sinon.SinonFakeTimers
|
||||
|
||||
beforeEach(() => {
|
||||
options = {
|
||||
actModeOllamaModelId: "llama2",
|
||||
ollamaBaseUrl: "http://localhost:11434",
|
||||
}
|
||||
handler = new OllamaHandler(options)
|
||||
// Use fake timers for testing timeouts
|
||||
clock = sinon.useFakeTimers()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
clock.restore()
|
||||
sinon.restore()
|
||||
})
|
||||
|
||||
describe("createMessage", () => {
|
||||
it("should handle successful responses", async function () {
|
||||
if (!ollamaAvailable) {
|
||||
this.skip()
|
||||
}
|
||||
this.timeout(5000)
|
||||
// Ensure client is initialized
|
||||
const client = (handler as any).ensureClient()
|
||||
// Mock the Ollama client's chat method
|
||||
const chatStub = sinon.stub(client, "chat").resolves({
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield {
|
||||
message: { content: "Hello, world!" },
|
||||
eval_count: 10,
|
||||
prompt_eval_count: 20,
|
||||
}
|
||||
},
|
||||
} as any)
|
||||
|
||||
const systemPrompt = "You are a helpful assistant."
|
||||
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
|
||||
|
||||
const result = []
|
||||
const usageInfo = []
|
||||
|
||||
// Collect the results
|
||||
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
|
||||
if (chunk.type === "text") {
|
||||
result.push(chunk.text)
|
||||
} else if (chunk.type === "usage") {
|
||||
usageInfo.push({
|
||||
inputTokens: chunk.inputTokens,
|
||||
outputTokens: chunk.outputTokens,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Verify the results
|
||||
result.should.deepEqual(["Hello, world!"])
|
||||
usageInfo.should.deepEqual([{ inputTokens: 20, outputTokens: 10 }])
|
||||
chatStub.calledOnce.should.be.true()
|
||||
})
|
||||
|
||||
it("should handle timeout errors", async function () {
|
||||
if (!ollamaAvailable) {
|
||||
this.skip()
|
||||
}
|
||||
this.timeout(10000)
|
||||
// Restore real timers for this test
|
||||
clock.restore()
|
||||
|
||||
// Create a handler with a very short timeout for testing
|
||||
const testHandler = new OllamaHandler(options)
|
||||
|
||||
// Replace the createMessage method with one that has a shorter timeout
|
||||
testHandler.createMessage = async function* (_systemPrompt, _messages) {
|
||||
try {
|
||||
// Create a promise that rejects after a short timeout
|
||||
const timeoutPromise = new Promise<never>((_, reject) => {
|
||||
setTimeout(() => reject(new Error("Ollama request timed out after 120 seconds")), 100)
|
||||
})
|
||||
|
||||
// Create a promise that never resolves
|
||||
const neverPromise = new Promise(() => {})
|
||||
|
||||
// Race them
|
||||
await Promise.race([timeoutPromise, neverPromise])
|
||||
} catch (error: any) {
|
||||
// Enhance error reporting
|
||||
console.error(`Ollama API error: ${error.message}`)
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
const systemPrompt = "You are a helpful assistant."
|
||||
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
|
||||
|
||||
// Start the request and catch the error
|
||||
let errorMessage = ""
|
||||
try {
|
||||
for await (const _ of testHandler.createMessage(systemPrompt, messages)) {
|
||||
// This should not be reached
|
||||
}
|
||||
} catch (error: any) {
|
||||
errorMessage = error.message
|
||||
}
|
||||
|
||||
// Check the result
|
||||
errorMessage.should.equal("Ollama request timed out after 120 seconds")
|
||||
|
||||
// Restore the fake timers for other tests
|
||||
clock = sinon.useFakeTimers()
|
||||
})
|
||||
|
||||
it("should retry on errors when using the withRetry decorator", async function () {
|
||||
if (!ollamaAvailable) {
|
||||
this.skip()
|
||||
}
|
||||
this.timeout(10000)
|
||||
// Restore real timers for this test
|
||||
clock.restore()
|
||||
|
||||
// Ensure client is initialized and mock the Ollama client's chat method to fail on first call and succeed on second
|
||||
const client = (handler as any).ensureClient()
|
||||
const chatStub = sinon.stub(client, "chat")
|
||||
|
||||
// First call throws an error
|
||||
chatStub.onFirstCall().rejects(new Error("API Error"))
|
||||
|
||||
// Second call succeeds
|
||||
chatStub.onSecondCall().resolves({
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield {
|
||||
message: { content: "Success after retry" },
|
||||
}
|
||||
},
|
||||
} as any)
|
||||
|
||||
const systemPrompt = "You are a helpful assistant."
|
||||
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
|
||||
|
||||
const result = []
|
||||
|
||||
// Add a small delay to ensure the retry mechanism has time to work
|
||||
await new Promise((resolve) => setTimeout(resolve, 100))
|
||||
|
||||
// Collect the results
|
||||
for await (const chunk of handler.createMessage(systemPrompt, messages)) {
|
||||
if (chunk.type === "text") {
|
||||
result.push(chunk.text)
|
||||
}
|
||||
}
|
||||
|
||||
// Verify the results
|
||||
result.should.deepEqual(["Success after retry"])
|
||||
chatStub.calledTwice.should.be.true()
|
||||
|
||||
// Restore the fake timers for other tests
|
||||
clock = sinon.useFakeTimers()
|
||||
})
|
||||
|
||||
it("should handle stream processing errors", async function () {
|
||||
if (!ollamaAvailable) {
|
||||
this.skip()
|
||||
}
|
||||
this.timeout(10000)
|
||||
// Restore real timers for this test
|
||||
clock.restore()
|
||||
|
||||
// Create a handler with a custom implementation for testing
|
||||
const testHandler = new OllamaHandler(options)
|
||||
|
||||
// Replace the createMessage method with one that simulates a stream error
|
||||
testHandler.createMessage = async function* (_systemPrompt, _messages) {
|
||||
// First yield a successful chunk
|
||||
yield {
|
||||
type: "text",
|
||||
text: "Partial response",
|
||||
}
|
||||
|
||||
// Then throw an error in the stream
|
||||
throw new Error("Ollama stream processing error: Stream error")
|
||||
}
|
||||
|
||||
const systemPrompt = "You are a helpful assistant."
|
||||
const messages: ClineStorageMessage[] = [{ role: "user", content: "Hello" }]
|
||||
|
||||
const result = []
|
||||
|
||||
// Collect the results and catch the error
|
||||
let errorMessage = ""
|
||||
try {
|
||||
for await (const chunk of testHandler.createMessage(systemPrompt, messages)) {
|
||||
if (chunk.type === "text") {
|
||||
result.push(chunk.text)
|
||||
}
|
||||
}
|
||||
} catch (error: any) {
|
||||
errorMessage = error.message
|
||||
}
|
||||
|
||||
// Verify the results
|
||||
errorMessage.should.equal("Ollama stream processing error: Stream error")
|
||||
result.should.deepEqual(["Partial response"])
|
||||
|
||||
// Restore the fake timers for other tests
|
||||
clock = sinon.useFakeTimers()
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,164 +0,0 @@
|
||||
import "should"
|
||||
import { openRouterDefaultModelInfo } from "@shared/api"
|
||||
import sinon from "sinon"
|
||||
import { OpenRouterHandler } from "../openrouter"
|
||||
|
||||
describe("OpenRouterHandler", () => {
|
||||
afterEach(() => {
|
||||
sinon.restore()
|
||||
})
|
||||
|
||||
const createAsyncIterable = (data: any[] = []) => ({
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield* data
|
||||
},
|
||||
})
|
||||
|
||||
const tools = [{ type: "function", function: { name: "read_file", description: "", parameters: { type: "object" } } }] as any
|
||||
|
||||
it("should handle usage-only chunks when delta is missing", async () => {
|
||||
const handler = new OpenRouterHandler({
|
||||
openRouterApiKey: "test-api-key",
|
||||
})
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
choices: [{}],
|
||||
usage: {
|
||||
prompt_tokens: 13,
|
||||
completion_tokens: 5,
|
||||
},
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
|
||||
sinon.stub(handler, "getModel").returns({
|
||||
id: "openai/gpt-4o-mini",
|
||||
info: openRouterDefaultModelInfo,
|
||||
})
|
||||
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
chunks.should.deepEqual([
|
||||
{
|
||||
type: "usage",
|
||||
cacheWriteTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
inputTokens: 13,
|
||||
outputTokens: 5,
|
||||
totalCost: 0,
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it("should read cache_write_tokens from prompt_tokens_details", async () => {
|
||||
const handler = new OpenRouterHandler({
|
||||
openRouterApiKey: "test-api-key",
|
||||
})
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
choices: [{}],
|
||||
usage: {
|
||||
prompt_tokens: 1000,
|
||||
completion_tokens: 200,
|
||||
prompt_tokens_details: {
|
||||
cached_tokens: 500,
|
||||
cache_write_tokens: 300,
|
||||
},
|
||||
},
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
|
||||
sinon.stub(handler, "getModel").returns({
|
||||
id: "anthropic/claude-sonnet-4.6",
|
||||
info: openRouterDefaultModelInfo,
|
||||
})
|
||||
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
chunks.should.deepEqual([
|
||||
{
|
||||
type: "usage",
|
||||
cacheWriteTokens: 300,
|
||||
cacheReadTokens: 500,
|
||||
inputTokens: 200,
|
||||
outputTokens: 200,
|
||||
totalCost: 0,
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
type ParallelToolCallsTestCase = {
|
||||
modelId: string
|
||||
enableParallelToolCalling: boolean
|
||||
expectedParallelToolCalls: boolean
|
||||
}
|
||||
|
||||
const parallelToolCallsTestCases: ParallelToolCallsTestCase[] = [
|
||||
{
|
||||
modelId: "openai/gpt-4o-mini",
|
||||
enableParallelToolCalling: true,
|
||||
expectedParallelToolCalls: true,
|
||||
},
|
||||
{
|
||||
modelId: "openai/gpt-4o-mini",
|
||||
enableParallelToolCalling: false,
|
||||
expectedParallelToolCalls: false,
|
||||
},
|
||||
{
|
||||
modelId: "google/gemini-3-flash-preview",
|
||||
enableParallelToolCalling: true,
|
||||
expectedParallelToolCalls: true,
|
||||
},
|
||||
]
|
||||
|
||||
for (const testCase of parallelToolCallsTestCases) {
|
||||
const settingLabel = testCase.enableParallelToolCalling ? "enabled" : "disabled"
|
||||
it(`should set parallel_tool_calls=${testCase.expectedParallelToolCalls} for ${testCase.modelId} when setting is ${settingLabel}`, async () => {
|
||||
const handler = new OpenRouterHandler({
|
||||
openRouterApiKey: "test-api-key",
|
||||
enableParallelToolCalling: testCase.enableParallelToolCalling,
|
||||
})
|
||||
const createStub = sinon.stub().resolves(createAsyncIterable([]))
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: createStub,
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
|
||||
sinon.stub(handler, "getModel").returns({
|
||||
id: testCase.modelId,
|
||||
info: openRouterDefaultModelInfo,
|
||||
})
|
||||
|
||||
for await (const _chunk of handler.createMessage("system", [{ role: "user", content: "hi" }], tools)) {
|
||||
// drain stream
|
||||
}
|
||||
|
||||
const payload = createStub.firstCall.args[0]
|
||||
payload.parallel_tool_calls.should.equal(testCase.expectedParallelToolCalls)
|
||||
})
|
||||
}
|
||||
})
|
||||
@@ -1,132 +0,0 @@
|
||||
import "should"
|
||||
import { Anthropic } from "@anthropic-ai/sdk"
|
||||
import { SapAiCoreHandler } from "../sapaicore"
|
||||
|
||||
describe("SapAiCoreHandler", () => {
|
||||
let handler: SapAiCoreHandler
|
||||
|
||||
beforeEach(() => {
|
||||
const mockOptions = {
|
||||
sapAiCoreClientId: "test-client-id",
|
||||
sapAiCoreClientSecret: "test-client-secret",
|
||||
sapAiCoreTokenUrl: "https://test.auth.sap.com",
|
||||
sapAiResourceGroup: "default",
|
||||
sapAiCoreBaseUrl: "https://test.api.sap.com",
|
||||
apiModelId: "anthropic--claude-3.5-sonnet",
|
||||
}
|
||||
handler = new SapAiCoreHandler(mockOptions)
|
||||
})
|
||||
|
||||
describe("image processing", () => {
|
||||
// Test image processing through the public interface
|
||||
// This tests the complete flow including processImageContent internally
|
||||
|
||||
it("should handle image processing for Claude 4 models", () => {
|
||||
// Create handler with Claude 4 model
|
||||
const claude4Handler = new SapAiCoreHandler({
|
||||
sapAiCoreClientId: "test-client-id",
|
||||
sapAiCoreClientSecret: "test-client-secret",
|
||||
sapAiCoreTokenUrl: "https://test.auth.sap.com",
|
||||
sapAiResourceGroup: "default",
|
||||
sapAiCoreBaseUrl: "https://test.api.sap.com",
|
||||
apiModelId: "anthropic--claude-4-sonnet",
|
||||
})
|
||||
|
||||
const model = claude4Handler.getModel()
|
||||
model.id.should.equal("anthropic--claude-4-sonnet")
|
||||
model.info.should.have.property("supportsImages", true)
|
||||
})
|
||||
|
||||
it("should create proper user readable request with images", () => {
|
||||
const testImageData =
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=="
|
||||
|
||||
const userContent: (Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[] = [
|
||||
{
|
||||
type: "text",
|
||||
text: "Here's an image:",
|
||||
},
|
||||
{
|
||||
type: "image",
|
||||
source: {
|
||||
type: "base64",
|
||||
media_type: "image/png",
|
||||
data: testImageData,
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
const result = handler.createUserReadableRequest(userContent)
|
||||
|
||||
result.should.have.property("model")
|
||||
result.should.have.property("max_tokens")
|
||||
result.should.have.property("system")
|
||||
result.should.have.property("messages")
|
||||
result.messages.should.be.Array()
|
||||
result.messages[1].should.have.property("role", "user")
|
||||
result.messages[1].should.have.property("content", userContent)
|
||||
})
|
||||
|
||||
it("should support different Claude model variants", () => {
|
||||
const modelVariants = [
|
||||
"anthropic--claude-4.6-sonnet",
|
||||
"anthropic--claude-4-sonnet",
|
||||
"anthropic--claude-4-opus",
|
||||
"anthropic--claude-3.7-sonnet",
|
||||
"anthropic--claude-3.5-sonnet",
|
||||
"anthropic--claude-3-sonnet",
|
||||
"anthropic--claude-3-haiku",
|
||||
"anthropic--claude-3-opus",
|
||||
]
|
||||
|
||||
modelVariants.forEach((modelId) => {
|
||||
const testHandler = new SapAiCoreHandler({
|
||||
apiModelId: modelId,
|
||||
})
|
||||
|
||||
const model = testHandler.getModel()
|
||||
model.id.should.equal(modelId)
|
||||
model.info.should.have.property("maxTokens")
|
||||
model.info.should.have.property("contextWindow")
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe("getModel", () => {
|
||||
it("should return default model when no apiModelId is provided", () => {
|
||||
const result = handler.getModel()
|
||||
result.should.have.property("id")
|
||||
result.should.have.property("info")
|
||||
result.info.should.have.property("maxTokens")
|
||||
})
|
||||
|
||||
it("should return specified model when apiModelId is provided", () => {
|
||||
const customHandler = new SapAiCoreHandler({
|
||||
apiModelId: "anthropic--claude-4-sonnet",
|
||||
})
|
||||
|
||||
const result = customHandler.getModel()
|
||||
result.id.should.equal("anthropic--claude-4-sonnet")
|
||||
})
|
||||
})
|
||||
|
||||
describe("createUserReadableRequest", () => {
|
||||
it("should create a readable request format", () => {
|
||||
const userContent: Anthropic.TextBlockParam[] = [
|
||||
{
|
||||
type: "text",
|
||||
text: "Hello, world!",
|
||||
},
|
||||
]
|
||||
|
||||
const result = handler.createUserReadableRequest(userContent)
|
||||
|
||||
result.should.have.property("model")
|
||||
result.should.have.property("max_tokens")
|
||||
result.should.have.property("system")
|
||||
result.should.have.property("messages")
|
||||
result.should.have.property("tools")
|
||||
result.should.have.property("tool_choice")
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,138 +0,0 @@
|
||||
import "should"
|
||||
import { openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
|
||||
import sinon from "sinon"
|
||||
import { VercelAIGatewayHandler } from "../vercel-ai-gateway"
|
||||
|
||||
describe("VercelAIGatewayHandler", () => {
|
||||
afterEach(() => {
|
||||
sinon.restore()
|
||||
})
|
||||
|
||||
const createAsyncIterable = (data: any[] = []) => ({
|
||||
[Symbol.asyncIterator]: async function* () {
|
||||
yield* data
|
||||
},
|
||||
})
|
||||
|
||||
describe("getModel", () => {
|
||||
it("should return configured model and info when both are provided", () => {
|
||||
const customModelInfo = {
|
||||
...openRouterDefaultModelInfo,
|
||||
maxTokens: 123456,
|
||||
}
|
||||
|
||||
const handler = new VercelAIGatewayHandler({
|
||||
openRouterModelId: "google/gemini-3.1-pro-preview",
|
||||
openRouterModelInfo: customModelInfo,
|
||||
})
|
||||
|
||||
const result = handler.getModel()
|
||||
result.id.should.equal("google/gemini-3.1-pro-preview")
|
||||
result.info.should.deepEqual(customModelInfo)
|
||||
})
|
||||
|
||||
it("should preserve configured model ID when model info is missing", () => {
|
||||
const handler = new VercelAIGatewayHandler({
|
||||
openRouterModelId: "google/gemini-3.1-pro-preview",
|
||||
})
|
||||
|
||||
const result = handler.getModel()
|
||||
result.id.should.equal("google/gemini-3.1-pro-preview")
|
||||
result.info.should.deepEqual(openRouterDefaultModelInfo)
|
||||
})
|
||||
|
||||
it("should fall back to default model when model ID is missing", () => {
|
||||
const handler = new VercelAIGatewayHandler({})
|
||||
const result = handler.getModel()
|
||||
|
||||
result.id.should.equal(openRouterDefaultModelId)
|
||||
result.info.should.deepEqual(openRouterDefaultModelInfo)
|
||||
})
|
||||
})
|
||||
|
||||
describe("createMessage", () => {
|
||||
it("should handle usage-only chunks when delta is missing", async () => {
|
||||
const handler = new VercelAIGatewayHandler({
|
||||
vercelAiGatewayApiKey: "test-api-key",
|
||||
})
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
choices: [{}],
|
||||
usage: {
|
||||
prompt_tokens: 11,
|
||||
completion_tokens: 7,
|
||||
},
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
|
||||
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
chunks.should.deepEqual([
|
||||
{
|
||||
type: "usage",
|
||||
cacheWriteTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
inputTokens: 11,
|
||||
outputTokens: 7,
|
||||
totalCost: 0,
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it("should read Anthropic-style cache creation and read tokens from usage chunks", async () => {
|
||||
const handler = new VercelAIGatewayHandler({
|
||||
vercelAiGatewayApiKey: "test-api-key",
|
||||
})
|
||||
const fakeClient = {
|
||||
chat: {
|
||||
completions: {
|
||||
create: sinon.stub().resolves(
|
||||
createAsyncIterable([
|
||||
{
|
||||
choices: [{}],
|
||||
usage: {
|
||||
prompt_tokens: 1000,
|
||||
completion_tokens: 200,
|
||||
prompt_tokens_details: {
|
||||
cached_tokens: 500,
|
||||
},
|
||||
cache_creation_input_tokens: 300,
|
||||
},
|
||||
},
|
||||
]),
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
sinon.stub(handler as any, "ensureClient").returns(fakeClient as any)
|
||||
|
||||
const chunks: any[] = []
|
||||
for await (const chunk of handler.createMessage("system", [{ role: "user", content: "hi" }])) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
chunks.should.deepEqual([
|
||||
{
|
||||
type: "usage",
|
||||
cacheWriteTokens: 300,
|
||||
cacheReadTokens: 500,
|
||||
inputTokens: 200,
|
||||
outputTokens: 200,
|
||||
totalCost: 0,
|
||||
},
|
||||
])
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,23 +0,0 @@
|
||||
import "should"
|
||||
import { vertexGlobalModels } from "@shared/api"
|
||||
import { VertexHandler } from "../vertex"
|
||||
|
||||
describe("VertexHandler", () => {
|
||||
it("supports Gemini 3.5 Flash model metadata", () => {
|
||||
const handler = new VertexHandler({
|
||||
vertexProjectId: "test-project",
|
||||
vertexRegion: "global",
|
||||
apiModelId: "gemini-3.5-flash",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
model.id.should.equal("gemini-3.5-flash")
|
||||
model.info.contextWindow!.should.equal(1_048_576)
|
||||
model.info.inputPrice!.should.equal(1.5)
|
||||
model.info.outputPrice!.should.equal(9)
|
||||
model.info.cacheReadsPrice!.should.equal(0.15)
|
||||
model.info.supportsGlobalEndpoint!.should.equal(true)
|
||||
model.info.supportsReasoning!.should.equal(true)
|
||||
vertexGlobalModels.should.have.property("gemini-3.5-flash")
|
||||
})
|
||||
})
|
||||
@@ -1,42 +0,0 @@
|
||||
import "should"
|
||||
import { openAiModelInfoSaneDefaults, wandbDefaultModelId, wandbModels } from "@shared/api"
|
||||
import { WandbHandler } from "../wandb"
|
||||
|
||||
describe("WandbHandler", () => {
|
||||
it("returns known catalog model metadata when model id is recognized", () => {
|
||||
const modelId = "meta-llama/Llama-3.3-70B-Instruct"
|
||||
const handler = new WandbHandler({
|
||||
wandbApiKey: "test-api-key",
|
||||
apiModelId: modelId,
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
|
||||
model.id.should.equal(modelId)
|
||||
model.info.should.deepEqual(wandbModels[modelId])
|
||||
})
|
||||
|
||||
it("passes through an explicit unknown model id instead of silently falling back", () => {
|
||||
const unknownModelId = "moonshotai/Kimi-K2.5"
|
||||
const handler = new WandbHandler({
|
||||
wandbApiKey: "test-api-key",
|
||||
apiModelId: unknownModelId,
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
|
||||
model.id.should.equal(unknownModelId)
|
||||
model.info.should.deepEqual(openAiModelInfoSaneDefaults)
|
||||
})
|
||||
|
||||
it("uses the default W&B model when no model id is configured", () => {
|
||||
const handler = new WandbHandler({
|
||||
wandbApiKey: "test-api-key",
|
||||
})
|
||||
|
||||
const model = handler.getModel()
|
||||
|
||||
model.id.should.equal(wandbDefaultModelId)
|
||||
model.info.should.deepEqual(wandbModels[wandbDefaultModelId])
|
||||
})
|
||||
})
|
||||
@@ -1,327 +0,0 @@
|
||||
import { Anthropic } from "@anthropic-ai/sdk"
|
||||
import { GenerateContentConfig, GoogleGenAI } from "@google/genai"
|
||||
import { ModelInfo } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
|
||||
import { convertAnthropicMessageToGemini } from "../transform/gemini-format"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface AIhubmixHandlerOptions extends CommonApiHandlerOptions {
|
||||
apiKey?: string
|
||||
baseURL?: string
|
||||
appCode?: string
|
||||
modelId?: string
|
||||
modelInfo?: ModelInfo
|
||||
thinkingBudgetTokens?: number
|
||||
}
|
||||
|
||||
export class AIhubmixHandler implements ApiHandler {
|
||||
private options: AIhubmixHandlerOptions
|
||||
private anthropicClient: Anthropic | undefined
|
||||
private openaiClient: OpenAI | undefined
|
||||
private geminiClient: GoogleGenAI | undefined
|
||||
|
||||
constructor(options: AIhubmixHandlerOptions) {
|
||||
const { baseURL, appCode, ...rest } = options
|
||||
this.options = {
|
||||
baseURL: baseURL ?? "https://aihubmix.com",
|
||||
appCode: appCode ?? "KUWF9311",
|
||||
...rest,
|
||||
}
|
||||
}
|
||||
|
||||
private ensureAnthropicClient(): Anthropic {
|
||||
if (!this.anthropicClient) {
|
||||
if (!this.options.apiKey) {
|
||||
throw new Error("AIhubmix API key is required")
|
||||
}
|
||||
try {
|
||||
this.anthropicClient = new Anthropic({
|
||||
apiKey: this.options.apiKey,
|
||||
baseURL: this.options.baseURL,
|
||||
defaultHeaders: {
|
||||
"APP-Code": this.options.appCode,
|
||||
...buildExternalBasicHeaders(),
|
||||
},
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Anthropic client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.anthropicClient
|
||||
}
|
||||
|
||||
private ensureOpenaiClient(): OpenAI {
|
||||
if (!this.openaiClient) {
|
||||
if (!this.options.apiKey) {
|
||||
throw new Error("AIhubmix API key is required")
|
||||
}
|
||||
try {
|
||||
this.openaiClient = new OpenAI({
|
||||
apiKey: this.options.apiKey,
|
||||
baseURL: `${this.options.baseURL}/v1`,
|
||||
defaultHeaders: {
|
||||
"APP-Code": this.options.appCode,
|
||||
...buildExternalBasicHeaders(),
|
||||
},
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating OpenAI client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.openaiClient
|
||||
}
|
||||
|
||||
private ensureGeminiClient(): GoogleGenAI {
|
||||
if (!this.geminiClient) {
|
||||
if (!this.options.apiKey) {
|
||||
throw new Error("AIhubmix API key is required")
|
||||
}
|
||||
try {
|
||||
this.geminiClient = new GoogleGenAI({
|
||||
apiKey: this.options.apiKey,
|
||||
httpOptions: {
|
||||
baseUrl: `${this.options.baseURL}/gemini`,
|
||||
headers: {
|
||||
// @ts-expect-error
|
||||
"APP-Code": this.options.appCode,
|
||||
Authorization: `Bearer ${this.options.apiKey ?? ""}`,
|
||||
...buildExternalBasicHeaders(),
|
||||
},
|
||||
},
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Gemini client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.geminiClient
|
||||
}
|
||||
|
||||
private routeModel(modelName: string): "anthropic" | "openai" | "gemini" | "openai-response" {
|
||||
const id = modelName || ""
|
||||
if (id.startsWith("claude")) {
|
||||
return "anthropic"
|
||||
}
|
||||
if (id.startsWith("gemini") && !id.endsWith("-nothink") && !id.endsWith("-search")) {
|
||||
return "gemini"
|
||||
}
|
||||
if (id === "gpt-5-pro" || id === "gpt-5-codex") {
|
||||
return "openai-response"
|
||||
}
|
||||
return "openai"
|
||||
}
|
||||
|
||||
private fixToolChoice(requestBody: any): any {
|
||||
if (requestBody.tools?.length === 0 && requestBody.tool_choice) {
|
||||
delete requestBody.tool_choice
|
||||
}
|
||||
return requestBody
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: any[]): ApiStream {
|
||||
const modelId = this.options.modelId || ""
|
||||
const route = this.routeModel(modelId)
|
||||
|
||||
switch (route) {
|
||||
case "anthropic":
|
||||
yield* this.createAnthropicMessage(systemPrompt, messages)
|
||||
break
|
||||
case "gemini":
|
||||
yield* this.createGeminiMessage(systemPrompt, messages)
|
||||
break
|
||||
case "openai-response":
|
||||
yield* this.createOpenaiResponseMessage(systemPrompt, messages)
|
||||
break
|
||||
case "openai":
|
||||
yield* this.createOpenaiMessage(systemPrompt, messages)
|
||||
break
|
||||
default:
|
||||
throw new Error(`Unsupported model route: ${route}`)
|
||||
}
|
||||
}
|
||||
|
||||
private async *createAnthropicMessage(systemPrompt: string, messages: any[]): ApiStream {
|
||||
const client = this.ensureAnthropicClient()
|
||||
const modelId = this.options.modelId || "claude-3-5-sonnet-20241022"
|
||||
|
||||
// Sanitize messages to remove Cline-specific fields like call_id that are not allowed by Anthropic API
|
||||
const sanitizedMessages = sanitizeAnthropicMessages(messages, false)
|
||||
|
||||
const stream = await client.messages.create({
|
||||
model: modelId,
|
||||
temperature: 0,
|
||||
max_tokens: this.options.modelInfo?.maxTokens || 8192,
|
||||
system: [{ text: systemPrompt, type: "text" }],
|
||||
messages: sanitizedMessages,
|
||||
stream: true,
|
||||
})
|
||||
|
||||
for await (const chunk of stream) {
|
||||
switch (chunk?.type) {
|
||||
case "message_start":
|
||||
const usage = chunk.message.usage
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: usage.input_tokens || 0,
|
||||
outputTokens: usage.output_tokens || 0,
|
||||
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
|
||||
cacheReadTokens: usage.cache_read_input_tokens || undefined,
|
||||
}
|
||||
break
|
||||
case "message_delta":
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: 0,
|
||||
outputTokens: chunk.usage.output_tokens || 0,
|
||||
}
|
||||
break
|
||||
case "content_block_start":
|
||||
if (chunk.content_block.type === "text") {
|
||||
yield {
|
||||
type: "text",
|
||||
text: chunk.content_block.text,
|
||||
}
|
||||
}
|
||||
break
|
||||
case "content_block_delta":
|
||||
if (chunk.delta.type === "text_delta") {
|
||||
yield {
|
||||
type: "text",
|
||||
text: chunk.delta.text,
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async *createOpenaiResponseMessage(systemPrompt: string, messages: any[]): ApiStream {
|
||||
const client = this.ensureOpenaiClient()
|
||||
const modelId = this.options.modelId || "gpt-4o-mini"
|
||||
|
||||
const input = (messages || []).map((m: any) => {
|
||||
const role = m.role || "user"
|
||||
const contentArray = Array.isArray(m.content) ? m.content : [{ type: "text", text: m.content }]
|
||||
const content = contentArray
|
||||
.filter((c: any) => c != null)
|
||||
.map((c: any) => {
|
||||
if (c.type === "image" || c.type === "input_image" || c.type === "image_url") {
|
||||
return { type: "input_image", image_url: c.image_url || c.url || c.source?.url }
|
||||
}
|
||||
const text = c.text ?? (typeof c === "string" ? c : "")
|
||||
return { type: role === "assistant" ? "output_text" : "input_text", text }
|
||||
})
|
||||
return { role, content }
|
||||
})
|
||||
|
||||
const stream = await (client as any).responses.stream({
|
||||
model: modelId,
|
||||
instructions: systemPrompt,
|
||||
input,
|
||||
})
|
||||
|
||||
for await (const event of stream as any) {
|
||||
if (event?.type === "response.output_text.delta") {
|
||||
yield { type: "text", text: event.delta || "" }
|
||||
continue
|
||||
}
|
||||
if (event?.type === "response.completed") {
|
||||
const usage = event.response?.usage || {}
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: usage.input_tokens || 0,
|
||||
outputTokens: usage.output_tokens || 0,
|
||||
}
|
||||
continue
|
||||
}
|
||||
if (event?.type === "response.error") {
|
||||
throw new Error(event.error?.message || "responses error")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async *createOpenaiMessage(systemPrompt: string, messages: any[]): ApiStream {
|
||||
const client = this.ensureOpenaiClient()
|
||||
const modelId = this.options.modelId || "gpt-4o-mini"
|
||||
|
||||
const openaiMessages = [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
|
||||
|
||||
const requestBody = {
|
||||
model: modelId,
|
||||
messages: openaiMessages,
|
||||
temperature: 0,
|
||||
stream: true,
|
||||
}
|
||||
|
||||
const fixedRequestBody = this.fixToolChoice(requestBody)
|
||||
|
||||
const stream = await client.chat.completions.create(fixedRequestBody)
|
||||
|
||||
for await (const chunk of stream as any) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async *createGeminiMessage(systemPrompt: string, messages: any[]): ApiStream {
|
||||
const client = this.ensureGeminiClient()
|
||||
const modelId = this.options.modelId || "gemini-2.0-flash-exp"
|
||||
|
||||
const contents = messages.map(convertAnthropicMessageToGemini)
|
||||
|
||||
const requestConfig: GenerateContentConfig = {
|
||||
systemInstruction: systemPrompt,
|
||||
temperature: 0,
|
||||
}
|
||||
|
||||
if (this.options.thinkingBudgetTokens) {
|
||||
requestConfig.thinkingConfig = {
|
||||
thinkingBudget: this.options.thinkingBudgetTokens,
|
||||
includeThoughts: true,
|
||||
}
|
||||
}
|
||||
|
||||
const stream = await client.models.generateContentStream({
|
||||
model: modelId,
|
||||
contents,
|
||||
config: requestConfig,
|
||||
})
|
||||
|
||||
for await (const chunk of stream as any) {
|
||||
if (chunk?.text) {
|
||||
yield { type: "text", text: chunk.text }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
return {
|
||||
id: this.options.modelId || "gpt-4o-mini",
|
||||
info: this.options.modelInfo || {
|
||||
maxTokens: 8192,
|
||||
contextWindow: 128000,
|
||||
supportsImages: true,
|
||||
supportsPromptCache: false,
|
||||
description: "AIhubmix unified model provider",
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,316 +0,0 @@
|
||||
import { Anthropic } from "@anthropic-ai/sdk"
|
||||
import type {
|
||||
MessageCreateParamsStreaming as BetaMessageCreateParamsStreaming,
|
||||
BetaRawMessageStreamEvent,
|
||||
} from "@anthropic-ai/sdk/resources/beta/messages/messages"
|
||||
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
|
||||
import type { MessageCreateParamsStreaming as AnthropicMessageCreateParamsStreaming } from "@anthropic-ai/sdk/resources/messages/messages"
|
||||
import { Stream as AnthropicStream } from "@anthropic-ai/sdk/streaming"
|
||||
import {
|
||||
ANTHROPIC_FAST_MODE_SUFFIX,
|
||||
AnthropicModelId,
|
||||
anthropicDefaultModelId,
|
||||
anthropicModels,
|
||||
CLAUDE_SONNET_1M_SUFFIX,
|
||||
ModelInfo,
|
||||
} from "@shared/api"
|
||||
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
export const ANTHROPIC_FAST_MODE_BETA = "fast-mode-2026-02-01"
|
||||
|
||||
interface AnthropicHandlerOptions extends CommonApiHandlerOptions {
|
||||
apiKey?: string
|
||||
anthropicBaseUrl?: string
|
||||
apiModelId?: string
|
||||
reasoningEffort?: string
|
||||
thinkingBudgetTokens?: number
|
||||
}
|
||||
|
||||
export class AnthropicHandler implements ApiHandler {
|
||||
private options: AnthropicHandlerOptions
|
||||
private client: Anthropic | undefined
|
||||
|
||||
constructor(options: AnthropicHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): Anthropic {
|
||||
if (!this.client) {
|
||||
if (!this.options.apiKey) {
|
||||
throw new Error("Anthropic API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = new Anthropic({
|
||||
apiKey: this.options.apiKey,
|
||||
baseURL: this.options.anthropicBaseUrl || undefined,
|
||||
defaultHeaders: buildExternalBasicHeaders(),
|
||||
fetch, // Use configured fetch with proxy support
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Anthropic client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: AnthropicTool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
|
||||
const model = this.getModel()
|
||||
let stream: AnthropicStream<Anthropic.RawMessageStreamEvent> | AsyncIterable<BetaRawMessageStreamEvent>
|
||||
|
||||
const useFastMode = model.id.endsWith(ANTHROPIC_FAST_MODE_SUFFIX)
|
||||
const baseModelId = useFastMode ? model.id.slice(0, -ANTHROPIC_FAST_MODE_SUFFIX.length) : model.id
|
||||
const modelId = baseModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
|
||||
? baseModelId.slice(0, -CLAUDE_SONNET_1M_SUFFIX.length)
|
||||
: baseModelId
|
||||
const enable1mContextWindow = baseModelId.endsWith(CLAUDE_SONNET_1M_SUFFIX)
|
||||
const fastModeBetas = enable1mContextWindow
|
||||
? [ANTHROPIC_FAST_MODE_BETA, "context-1m-2025-08-07"]
|
||||
: [ANTHROPIC_FAST_MODE_BETA]
|
||||
const createFastModeMessage = (
|
||||
body: AnthropicMessageCreateParamsStreaming,
|
||||
): Promise<AsyncIterable<BetaRawMessageStreamEvent>> => {
|
||||
return (
|
||||
client.beta.messages.create as unknown as (
|
||||
params: BetaMessageCreateParamsStreaming & { speed: "fast" },
|
||||
) => Promise<AsyncIterable<BetaRawMessageStreamEvent>>
|
||||
)({
|
||||
...body,
|
||||
betas: fastModeBetas,
|
||||
speed: "fast",
|
||||
})
|
||||
}
|
||||
|
||||
const budget_tokens = this.options.thinkingBudgetTokens || 0
|
||||
|
||||
// Tools are available only when native tools are enabled.
|
||||
const nativeToolsOn = tools?.length && tools?.length > 0
|
||||
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
|
||||
|
||||
// Claude Opus 4.5+ uses adaptive thinking instead of budgeted extended thinking.
|
||||
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
|
||||
const adaptiveThinking = isAdaptiveThinkingModel
|
||||
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budget_tokens)
|
||||
: undefined
|
||||
const adaptiveThinkingEnabled = adaptiveThinking?.enabled === true
|
||||
const adaptiveThinkingEffort = adaptiveThinking?.effort
|
||||
const thinkingEnabled = isAdaptiveThinkingModel ? adaptiveThinkingEnabled : reasoningOn
|
||||
const thinkingConfig = thinkingEnabled
|
||||
? isAdaptiveThinkingModel
|
||||
? ({ type: "adaptive" } as any)
|
||||
: { type: "enabled", budget_tokens: budget_tokens }
|
||||
: undefined
|
||||
const outputConfig = isAdaptiveThinkingModel && adaptiveThinkingEffort ? { effort: adaptiveThinkingEffort } : undefined
|
||||
|
||||
if (model.info.supportsPromptCache) {
|
||||
const anthropicMessages = sanitizeAnthropicMessages(messages, true)
|
||||
const requestBody: AnthropicMessageCreateParamsStreaming & Record<string, unknown> = {
|
||||
model: modelId,
|
||||
thinking: thinkingConfig,
|
||||
max_tokens: model.info.maxTokens || 8192,
|
||||
// "Thinking isn't compatible with temperature, top_p, or top_k modifications as well as forced tool use."
|
||||
// (https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#important-considerations-when-using-extended-thinking)
|
||||
// Adaptive Claude Opus models do not support temperature.
|
||||
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
|
||||
system: [
|
||||
{
|
||||
text: systemPrompt,
|
||||
type: "text",
|
||||
cache_control: { type: "ephemeral" },
|
||||
},
|
||||
], // setting cache breakpoint for system prompt so new tasks can reuse it
|
||||
messages: anthropicMessages,
|
||||
// tools, // cache breakpoints go from tools > system > messages, and since tools dont change, we can just set the breakpoint at the end of system (this avoids having to set a breakpoint at the end of tools which by itself does not meet min requirements for haiku caching)
|
||||
stream: true,
|
||||
tools: nativeToolsOn ? tools : undefined,
|
||||
// tool_choice options:
|
||||
// - none: disables tool use, even if tools are provided. Claude will not call any tools.
|
||||
// - auto: allows Claude to decide whether to call any provided tools or not. This is the default value when tools are provided.
|
||||
// - any: tells Claude that it must use one of the provided tools, but doesn’t force a particular tool.
|
||||
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
|
||||
tool_choice: nativeToolsOn && !thinkingEnabled ? { type: "any" } : undefined,
|
||||
}
|
||||
if (outputConfig) {
|
||||
requestBody.output_config = outputConfig
|
||||
}
|
||||
|
||||
stream = useFastMode
|
||||
? await createFastModeMessage(requestBody)
|
||||
: await client.messages.create(
|
||||
requestBody,
|
||||
(() => {
|
||||
// 1m context window beta header
|
||||
if (enable1mContextWindow) {
|
||||
return {
|
||||
headers: {
|
||||
"anthropic-beta": "context-1m-2025-08-07",
|
||||
},
|
||||
}
|
||||
}
|
||||
return undefined
|
||||
})(),
|
||||
)
|
||||
} else {
|
||||
const requestBody: AnthropicMessageCreateParamsStreaming & Record<string, unknown> = {
|
||||
model: modelId,
|
||||
max_tokens: model.info.maxTokens || 8192,
|
||||
temperature: isAdaptiveThinkingModel ? undefined : reasoningOn ? undefined : 0,
|
||||
system: [{ text: systemPrompt, type: "text" }],
|
||||
messages: sanitizeAnthropicMessages(messages, false),
|
||||
tools: nativeToolsOn ? tools : undefined,
|
||||
tool_choice: thinkingEnabled ? undefined : { type: "auto" },
|
||||
stream: true,
|
||||
thinking: thinkingConfig,
|
||||
}
|
||||
if (outputConfig) {
|
||||
requestBody.output_config = outputConfig
|
||||
}
|
||||
|
||||
stream = useFastMode ? await createFastModeMessage(requestBody) : await client.messages.create(requestBody)
|
||||
}
|
||||
|
||||
const lastStartedToolCall = { id: "", name: "", arguments: "" }
|
||||
|
||||
for await (const chunk of stream) {
|
||||
switch (chunk?.type) {
|
||||
case "message_start":
|
||||
{
|
||||
// tells us cache reads/writes/input/output
|
||||
const usage = chunk.message.usage
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: usage.input_tokens || 0,
|
||||
outputTokens: usage.output_tokens || 0,
|
||||
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
|
||||
cacheReadTokens: usage.cache_read_input_tokens || undefined,
|
||||
}
|
||||
}
|
||||
break
|
||||
case "message_delta":
|
||||
// tells us stop_reason, stop_sequence, and output tokens along the way and at the end of the message
|
||||
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: 0,
|
||||
outputTokens: chunk.usage.output_tokens || 0,
|
||||
}
|
||||
break
|
||||
case "message_stop":
|
||||
// no usage data, just an indicator that the message is done
|
||||
break
|
||||
case "content_block_start":
|
||||
switch (chunk.content_block.type) {
|
||||
case "thinking":
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: chunk.content_block.thinking || "",
|
||||
signature: chunk.content_block.signature,
|
||||
}
|
||||
break
|
||||
case "redacted_thinking":
|
||||
// Content is encrypted, and we don't to pass placeholder text back to the API
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: "[Redacted thinking block]",
|
||||
redacted_data: chunk.content_block.data,
|
||||
}
|
||||
break
|
||||
case "tool_use":
|
||||
if (chunk.content_block.id && chunk.content_block.name) {
|
||||
// Convert Anthropic tool_use to OpenAI-compatible format
|
||||
lastStartedToolCall.id = chunk.content_block.id
|
||||
lastStartedToolCall.name = chunk.content_block.name
|
||||
lastStartedToolCall.arguments = ""
|
||||
}
|
||||
break
|
||||
case "text":
|
||||
// we may receive multiple text blocks, in which case just insert a line break between them
|
||||
if (chunk.index > 0) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: "\n",
|
||||
}
|
||||
}
|
||||
yield {
|
||||
type: "text",
|
||||
text: chunk.content_block.text,
|
||||
}
|
||||
break
|
||||
}
|
||||
break
|
||||
case "content_block_delta":
|
||||
switch (chunk.delta.type) {
|
||||
case "thinking_delta":
|
||||
// 'reasoning' type just displays in the UI, but ant_thinking will be used to send the thinking traces back to the API
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: chunk.delta.thinking,
|
||||
}
|
||||
break
|
||||
case "signature_delta":
|
||||
// It's used when sending the thinking block back to the API
|
||||
// API expects this in completed form, not as array of deltas
|
||||
if (chunk.delta.signature) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: "", // reasoning text is already sent via thinking_delta
|
||||
signature: chunk.delta.signature,
|
||||
}
|
||||
}
|
||||
break
|
||||
case "text_delta":
|
||||
yield {
|
||||
type: "text",
|
||||
text: chunk.delta.text,
|
||||
}
|
||||
break
|
||||
case "input_json_delta":
|
||||
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
|
||||
// // Convert Anthropic tool_use to OpenAI-compatible format
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
tool_call: {
|
||||
...lastStartedToolCall,
|
||||
function: {
|
||||
...lastStartedToolCall,
|
||||
id: lastStartedToolCall.id,
|
||||
name: lastStartedToolCall.name,
|
||||
arguments: chunk.delta.partial_json,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
break
|
||||
|
||||
case "content_block_stop":
|
||||
lastStartedToolCall.id = ""
|
||||
lastStartedToolCall.name = ""
|
||||
lastStartedToolCall.arguments = ""
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: AnthropicModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in anthropicModels) {
|
||||
const id = modelId as AnthropicModelId
|
||||
return { id, info: anthropicModels[id] }
|
||||
}
|
||||
return {
|
||||
id: anthropicDefaultModelId,
|
||||
info: anthropicModels[anthropicDefaultModelId],
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,200 +0,0 @@
|
||||
import { AskSageModelId, askSageDefaultModelId, askSageDefaultURL, askSageModels, ModelInfo } from "@shared/api"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from ".."
|
||||
import { withRetry } from "../retry"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface AskSageHandlerOptions extends CommonApiHandlerOptions {
|
||||
asksageApiKey?: string
|
||||
asksageApiUrl?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
type AskSageRequest = {
|
||||
system_prompt: string
|
||||
message: {
|
||||
user: "gpt" | "me"
|
||||
message: string
|
||||
}[]
|
||||
model: string
|
||||
dataset: "none"
|
||||
usage: boolean
|
||||
}
|
||||
|
||||
type AskSageUsage = {
|
||||
model_tokens: {
|
||||
completion_tokens: number
|
||||
prompt_tokens: number
|
||||
total_tokens: number
|
||||
}
|
||||
asksage_tokens: number
|
||||
}
|
||||
|
||||
type AskSageResponse = {
|
||||
uuid: string
|
||||
status: number
|
||||
// Response status
|
||||
response: string
|
||||
// Generated response message
|
||||
message: string
|
||||
// whether embedding & vector systems are down
|
||||
embedding_down: boolean
|
||||
vectors_down: boolean
|
||||
// references if dataset is not none
|
||||
references: string
|
||||
type: string
|
||||
added_obj: any
|
||||
tool_calls: any
|
||||
// usage metrics
|
||||
usage: AskSageUsage | null
|
||||
tool_responses: any[]
|
||||
tool_calls_unified: any[]
|
||||
}
|
||||
|
||||
export class AskSageHandler implements ApiHandler {
|
||||
private options: AskSageHandlerOptions
|
||||
private apiUrl: string
|
||||
private apiKey: string
|
||||
|
||||
constructor(options: AskSageHandlerOptions) {
|
||||
Logger.log("init api url", options.asksageApiUrl, askSageDefaultURL)
|
||||
this.options = options
|
||||
this.apiKey = options.asksageApiKey || ""
|
||||
this.apiUrl = options.asksageApiUrl || askSageDefaultURL
|
||||
|
||||
if (!this.apiKey) {
|
||||
throw new Error("AskSage API key is required")
|
||||
}
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
try {
|
||||
const model = this.getModel()
|
||||
// Transform messages into AskSageRequest format
|
||||
const formattedMessages = messages.map((msg) => {
|
||||
const content = Array.isArray(msg.content)
|
||||
? msg.content.map((block) => ("text" in block ? block.text : "")).join("")
|
||||
: msg.content
|
||||
|
||||
return {
|
||||
user: msg.role === "assistant" ? ("gpt" as const) : ("me" as const),
|
||||
message: content,
|
||||
}
|
||||
})
|
||||
|
||||
const request: AskSageRequest = {
|
||||
system_prompt: systemPrompt,
|
||||
message: formattedMessages,
|
||||
model: model.id,
|
||||
dataset: "none",
|
||||
usage: true,
|
||||
}
|
||||
|
||||
// Make request to AskSage API
|
||||
const response = await fetch(`${this.apiUrl}/query`, {
|
||||
method: "POST",
|
||||
headers: this.headers(),
|
||||
body: JSON.stringify(request),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const error = await response.text()
|
||||
throw new Error(`AskSage API error: ${error}`)
|
||||
}
|
||||
|
||||
const result = (await response.json()) as AskSageResponse
|
||||
|
||||
if (!result.message) {
|
||||
throw new Error("No content in AskSage response")
|
||||
}
|
||||
|
||||
// Yield tool responses if they exist
|
||||
if (result.tool_responses && result.tool_responses.length > 0) {
|
||||
for (const toolResponse of result.tool_responses) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: `[Tool Response: ${JSON.stringify(toolResponse)}]\n`,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Yield the main response text
|
||||
yield {
|
||||
type: "text",
|
||||
text: result.message,
|
||||
}
|
||||
|
||||
// Yield usage information if available
|
||||
if (result.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: result.usage.model_tokens.prompt_tokens,
|
||||
outputTokens: result.usage.model_tokens.completion_tokens,
|
||||
cacheReadTokens: 0,
|
||||
cacheWriteTokens: 0,
|
||||
totalCost: result.usage.asksage_tokens, // Cost = Consumed AskSage tokens
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
if (error instanceof Error) {
|
||||
throw new Error(`AskSage request failed: ${error.message}`)
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async getApiStreamUsage() {
|
||||
if (!this.apiKey) {
|
||||
return undefined
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(`${this.apiUrl}/count-monthly-tokens`, {
|
||||
method: "POST",
|
||||
headers: this.headers(),
|
||||
body: JSON.stringify({ app_name: "asksage" }),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
Logger.error("Failed to fetch AskSage usage", await response.text())
|
||||
return undefined
|
||||
}
|
||||
|
||||
const data = await response.json()
|
||||
const usedTokens = data.response as number
|
||||
|
||||
return {
|
||||
type: "usage" as const,
|
||||
inputTokens: usedTokens,
|
||||
outputTokens: 0,
|
||||
}
|
||||
} catch (error) {
|
||||
Logger.error("Error fetching AskSage usage:", error)
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in askSageModels) {
|
||||
const id = modelId as AskSageModelId
|
||||
return { id, info: askSageModels[id] }
|
||||
}
|
||||
return {
|
||||
id: askSageDefaultModelId,
|
||||
info: askSageModels[askSageDefaultModelId],
|
||||
}
|
||||
}
|
||||
|
||||
private headers() {
|
||||
return {
|
||||
"Content-Type": "application/json",
|
||||
"x-access-tokens": this.apiKey,
|
||||
...buildExternalBasicHeaders(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,172 +0,0 @@
|
||||
import { BasetenModelId, basetenDefaultModelId, basetenModels, ModelInfo } from "@shared/api"
|
||||
import { calculateApiCostOpenAI } from "@utils/cost"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface BasetenHandlerOptions extends CommonApiHandlerOptions {
|
||||
basetenApiKey?: string
|
||||
basetenModelId?: string
|
||||
basetenModelInfo?: ModelInfo
|
||||
apiModelId?: string // For backward compatibility
|
||||
}
|
||||
|
||||
export class BasetenHandler implements ApiHandler {
|
||||
private options: BasetenHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: BasetenHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.basetenApiKey) {
|
||||
throw new Error("Baseten API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = new OpenAI({
|
||||
baseURL: "https://inference.baseten.co/v1",
|
||||
apiKey: this.options.basetenApiKey,
|
||||
defaultHeaders: buildExternalBasicHeaders(),
|
||||
fetch, // Use configured fetch with proxy support
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Baseten client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the optimal max_tokens based on model capabilities
|
||||
*/
|
||||
private getOptimalMaxTokens(model: { id: BasetenModelId; info: ModelInfo }): number {
|
||||
// Use model-specific max tokens if available
|
||||
if (model.info.maxTokens && model.info.maxTokens > 0) {
|
||||
return model.info.maxTokens
|
||||
}
|
||||
|
||||
// Default fallback
|
||||
return 8192
|
||||
}
|
||||
|
||||
getModel(): { id: BasetenModelId; info: ModelInfo } {
|
||||
// First priority: basetenModelId and basetenModelInfo
|
||||
const basetenModelId = this.options.basetenModelId
|
||||
const basetenModelInfo = this.options.basetenModelInfo
|
||||
if (basetenModelId && basetenModelInfo) {
|
||||
return { id: basetenModelId as BasetenModelId, info: basetenModelInfo }
|
||||
}
|
||||
|
||||
// Second priority: basetenModelId with static model info
|
||||
if (basetenModelId && basetenModelId in basetenModels) {
|
||||
const id = basetenModelId as BasetenModelId
|
||||
return { id, info: basetenModels[id] }
|
||||
}
|
||||
|
||||
// Third priority: apiModelId (for backward compatibility)
|
||||
const apiModelId = this.options.apiModelId
|
||||
if (apiModelId && apiModelId in basetenModels) {
|
||||
const id = apiModelId as BasetenModelId
|
||||
return { id, info: basetenModels[id] }
|
||||
}
|
||||
|
||||
// Default fallback
|
||||
return {
|
||||
id: basetenDefaultModelId,
|
||||
info: basetenModels[basetenDefaultModelId],
|
||||
}
|
||||
}
|
||||
|
||||
private async *yieldUsage(modelInfo: ModelInfo, usage: any): ApiStream {
|
||||
if (usage.prompt_tokens || usage.completion_tokens) {
|
||||
const cost = calculateApiCostOpenAI(modelInfo, usage.prompt_tokens || 0, usage.completion_tokens || 0)
|
||||
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: usage.prompt_tokens || 0,
|
||||
outputTokens: usage.completion_tokens || 0,
|
||||
cacheWriteTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
totalCost: cost,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
const maxTokens = this.getOptimalMaxTokens(model)
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
max_tokens: maxTokens,
|
||||
messages: openAiMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
temperature: 0,
|
||||
tools,
|
||||
tool_choice: tools && tools.length > 0 ? "auto" : undefined,
|
||||
})
|
||||
|
||||
let didOutputUsage = false
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk?.choices?.[0]?.delta
|
||||
|
||||
// Handle reasoning field if present (for reasoning models with parsed output)
|
||||
if (delta && "reasoning" in delta && delta?.reasoning) {
|
||||
const reasoning = typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning)
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning,
|
||||
}
|
||||
}
|
||||
|
||||
// Handle content field
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
// Handle usage information - only output once
|
||||
if (!didOutputUsage && chunk.usage) {
|
||||
yield* this.yieldUsage(model.info, chunk.usage)
|
||||
didOutputUsage = true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks if the current model supports tools
|
||||
*/
|
||||
supportsTools(): boolean {
|
||||
const model = this.getModel()
|
||||
const modelInfo = model.info as any
|
||||
|
||||
// Use dynamic API data when available, fallback to true since all current Baseten models support tools
|
||||
// (as of 2025-09-16 - could change if Baseten add non-tool models in future, currently no plans to do so)
|
||||
return modelInfo.supportedFeatures ? modelInfo.supportedFeatures.includes("tools") : true
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,275 +0,0 @@
|
||||
import Cerebras from "@cerebras/cerebras_cloud_sdk"
|
||||
import { CerebrasModelId, cerebrasDefaultModelId, cerebrasModels, ModelInfo } from "@shared/api"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface CerebrasHandlerOptions extends CommonApiHandlerOptions {
|
||||
cerebrasApiKey?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
// Conservative max_tokens for Cerebras to avoid premature rate limiting.
|
||||
// Cerebras rate limiter estimates token consumption using max_completion_tokens upfront,
|
||||
// so requesting the model maximum (e.g., 64K) reserves that quota even if actual usage is low.
|
||||
// 16K is sufficient for most agentic tool use while preserving rate limit headroom.
|
||||
const CEREBRAS_DEFAULT_MAX_TOKENS = 16_384
|
||||
|
||||
export class CerebrasHandler implements ApiHandler {
|
||||
private options: CerebrasHandlerOptions
|
||||
private client: Cerebras | undefined
|
||||
|
||||
constructor(options: CerebrasHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): Cerebras {
|
||||
if (!this.client) {
|
||||
// Clean and validate the API key
|
||||
const cleanApiKey = this.options.cerebrasApiKey?.trim()
|
||||
|
||||
if (!cleanApiKey) {
|
||||
throw new Error("Cerebras API key is required")
|
||||
}
|
||||
|
||||
try {
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
this.client = new Cerebras({
|
||||
apiKey: cleanApiKey,
|
||||
timeout: 30000, // 30 second timeout
|
||||
fetch, // Use configured fetch with proxy support
|
||||
defaultHeaders: {
|
||||
...externalHeaders,
|
||||
"X-Cerebras-3rd-Party-Integration": "cline",
|
||||
},
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Cerebras client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry({
|
||||
maxRetries: 6, // More retries to be patient with rate limits
|
||||
baseDelay: 5000, // Start with 5 second delay
|
||||
maxDelay: 60000, // Allow up to 60 second delays to respect rate limits
|
||||
})
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
|
||||
// Convert Anthropic messages to Cerebras format
|
||||
const cerebrasMessages: Array<{
|
||||
role: "system" | "user" | "assistant"
|
||||
content: string
|
||||
}> = [{ role: "system", content: systemPrompt }]
|
||||
|
||||
// Helper function to strip thinking tags from content
|
||||
const stripThinkingTags = (content: string): string => {
|
||||
return content.replace(/<think>[\s\S]*?<\/think>/g, "").trim()
|
||||
}
|
||||
|
||||
// Check if this is a reasoning model that uses thinking tags
|
||||
const modelId = this.getModel().id
|
||||
const isReasoningModel = modelId.includes("qwen")
|
||||
|
||||
// Convert Anthropic messages to Cerebras format
|
||||
for (const message of messages) {
|
||||
if (message.role === "user") {
|
||||
const content = Array.isArray(message.content)
|
||||
? message.content
|
||||
.map((block) => {
|
||||
if (block.type === "text") {
|
||||
return block.text
|
||||
}
|
||||
if (block.type === "image") {
|
||||
return "[Image content not supported in Cerebras]"
|
||||
}
|
||||
return ""
|
||||
})
|
||||
.join("\n")
|
||||
: message.content
|
||||
cerebrasMessages.push({ role: "user", content })
|
||||
} else if (message.role === "assistant") {
|
||||
let content = Array.isArray(message.content)
|
||||
? message.content
|
||||
.map((block) => {
|
||||
if (block.type === "text") {
|
||||
return block.text
|
||||
}
|
||||
return ""
|
||||
})
|
||||
.join("\n")
|
||||
: message.content || ""
|
||||
|
||||
// Strip thinking tags from assistant messages for reasoning models
|
||||
// so the model doesn't see its own thinking in the conversation history
|
||||
if (isReasoningModel) {
|
||||
content = stripThinkingTags(content)
|
||||
}
|
||||
|
||||
cerebrasMessages.push({ role: "assistant", content })
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
const model = this.getModel()
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
messages: cerebrasMessages,
|
||||
temperature: model.info.temperature ?? 0,
|
||||
stream: true,
|
||||
max_tokens: CEREBRAS_DEFAULT_MAX_TOKENS,
|
||||
})
|
||||
|
||||
// Handle streaming response
|
||||
let reasoning: string | null = null // Track reasoning content for models that support thinking
|
||||
|
||||
for await (const chunk of stream as any) {
|
||||
// Type assertion for the streaming chunk
|
||||
const streamChunk = chunk as any
|
||||
|
||||
if (streamChunk.choices?.[0]?.delta?.content) {
|
||||
const content = streamChunk.choices[0].delta.content
|
||||
|
||||
// Handle reasoning models (Qwen and DeepSeek R1 Distill) that use <think> tags
|
||||
if (isReasoningModel) {
|
||||
// Check if we're entering or continuing reasoning mode
|
||||
if (reasoning || content.includes("<think>")) {
|
||||
reasoning = (reasoning || "") + content
|
||||
|
||||
// Clean the content by removing think tags for display
|
||||
const cleanContent = content.replace(/<think>/g, "").replace(/<\/think>/g, "")
|
||||
|
||||
// Only yield reasoning content if there's actual content after cleaning
|
||||
if (cleanContent.trim()) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: cleanContent,
|
||||
}
|
||||
}
|
||||
|
||||
// Check if reasoning is complete
|
||||
if (reasoning.includes("</think>")) {
|
||||
reasoning = null
|
||||
}
|
||||
} else {
|
||||
// Regular content outside of thinking tags
|
||||
yield {
|
||||
type: "text",
|
||||
text: content,
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Non-reasoning models - just yield text content
|
||||
yield {
|
||||
type: "text",
|
||||
text: content,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Handle usage information from Cerebras API
|
||||
// Usage is typically only available in the final chunk
|
||||
if (streamChunk.usage) {
|
||||
const totalCost = this.calculateCost({
|
||||
inputTokens: streamChunk.usage.prompt_tokens || 0,
|
||||
outputTokens: streamChunk.usage.completion_tokens || 0,
|
||||
})
|
||||
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: streamChunk.usage.prompt_tokens || 0,
|
||||
outputTokens: streamChunk.usage.completion_tokens || 0,
|
||||
cacheReadTokens: 0,
|
||||
cacheWriteTokens: 0,
|
||||
totalCost,
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error: any) {
|
||||
// Enhanced error handling for Cerebras API
|
||||
if (error?.status === 429 || error?.code === "rate_limit_exceeded") {
|
||||
// Rate limit error - will be handled by retry decorator with patient backoff
|
||||
const _limits = this.getRateLimits()
|
||||
throw new Error(`Cerebras API rate limit exceeded.`)
|
||||
}
|
||||
if (error?.status === 401) {
|
||||
throw new Error("Cerebras API authentication failed. Please check your API key.")
|
||||
}
|
||||
if (error?.status === 403) {
|
||||
throw new Error("Cerebras API access denied. Please check your API key permissions.")
|
||||
}
|
||||
if (error?.status >= 500) {
|
||||
// Server errors - retryable
|
||||
throw new Error(`Cerebras API server error (${error.status}): ${error.message || "Unknown server error"}`)
|
||||
}
|
||||
if (error?.status === 400) {
|
||||
// Client errors - not retryable
|
||||
throw new Error(`Cerebras API bad request: ${error.message || "Invalid request parameters"}`)
|
||||
}
|
||||
|
||||
// Re-throw original error for other cases
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
const originalModelId = this.options.apiModelId
|
||||
let apiModelId = originalModelId
|
||||
if (originalModelId === "qwen-3-coder-480b-free") {
|
||||
apiModelId = "qwen-3-coder-480b"
|
||||
return { id: apiModelId, info: cerebrasModels[originalModelId as CerebrasModelId] }
|
||||
}
|
||||
|
||||
if (originalModelId && originalModelId in cerebrasModels) {
|
||||
const id = originalModelId as CerebrasModelId
|
||||
return { id, info: cerebrasModels[id] }
|
||||
}
|
||||
return {
|
||||
id: cerebrasDefaultModelId,
|
||||
info: cerebrasModels[cerebrasDefaultModelId],
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get rate limit information for the current model
|
||||
*
|
||||
* These limits are used for informational purposes and to calculate appropriate
|
||||
* retry delays. Since Cerebras inference is extremely fast, users hit these limits
|
||||
* quickly, so we need to be patient with retries to maximize usage efficiency.
|
||||
*
|
||||
* @returns Rate limit configuration for the model
|
||||
*/
|
||||
private getRateLimits(): { requestsPerMinute: number; tokensPerMinute: number } {
|
||||
const modelId = this.getModel().id
|
||||
|
||||
switch (modelId) {
|
||||
case "qwen-3-coder-480b":
|
||||
case "qwen-3-coder-480b-free":
|
||||
return { requestsPerMinute: 10, tokensPerMinute: 150_000 }
|
||||
case "qwen-3-235b-a22b-instruct-2507":
|
||||
case "qwen-3-235b-a22b-thinking-2507":
|
||||
return { requestsPerMinute: 30, tokensPerMinute: 60_000 }
|
||||
case "gpt-oss-120b":
|
||||
return { requestsPerMinute: 30, tokensPerMinute: 64_000 }
|
||||
default:
|
||||
// Default rate limits for unknown models
|
||||
return { requestsPerMinute: 30, tokensPerMinute: 60_000 }
|
||||
}
|
||||
}
|
||||
|
||||
private calculateCost({ inputTokens, outputTokens }: { inputTokens: number; outputTokens: number }): number {
|
||||
const model = this.getModel()
|
||||
const inputPrice = model.info.inputPrice || 0
|
||||
const outputPrice = model.info.outputPrice || 0
|
||||
|
||||
const inputCost = (inputPrice / 1_000_000) * inputTokens
|
||||
const outputCost = (outputPrice / 1_000_000) * outputTokens
|
||||
|
||||
return inputCost + outputCost
|
||||
}
|
||||
}
|
||||
@@ -1,229 +0,0 @@
|
||||
import { filterMessagesForClaudeCode } from "@/integrations/claude-code/message-filter"
|
||||
import { runClaudeCode } from "@/integrations/claude-code/run"
|
||||
import { ClaudeCodeModelId, claudeCodeDefaultModelId, claudeCodeModels } from "@/shared/api"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { type ApiHandler, CommonApiHandlerOptions } from ".."
|
||||
import { withRetry } from "../retry"
|
||||
import { type ApiStream, ApiStreamUsageChunk } from "../transform/stream"
|
||||
|
||||
interface ClaudeCodeHandlerOptions extends CommonApiHandlerOptions {
|
||||
claudeCodePath?: string
|
||||
apiModelId?: string
|
||||
thinkingBudgetTokens?: number
|
||||
}
|
||||
|
||||
export class ClaudeCodeHandler implements ApiHandler {
|
||||
private options: ClaudeCodeHandlerOptions
|
||||
|
||||
constructor(options: ClaudeCodeHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
@withRetry({
|
||||
maxRetries: 4,
|
||||
baseDelay: 2000,
|
||||
maxDelay: 15000,
|
||||
})
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
// Filter out image blocks since Claude Code doesn't support them
|
||||
const filteredMessages = filterMessagesForClaudeCode(messages)
|
||||
|
||||
const claudeProcess = runClaudeCode({
|
||||
systemPrompt,
|
||||
messages: filteredMessages,
|
||||
path: this.options.claudeCodePath,
|
||||
modelId: this.getModel().id,
|
||||
thinkingBudgetTokens: this.options.thinkingBudgetTokens,
|
||||
})
|
||||
|
||||
// Usage is included with assistant messages,
|
||||
// but cost is included in the result chunk
|
||||
const usage: ApiStreamUsageChunk = {
|
||||
type: "usage",
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
cacheWriteTokens: 0,
|
||||
}
|
||||
|
||||
let isPaidUsage = true
|
||||
|
||||
for await (const chunk of claudeProcess) {
|
||||
if (typeof chunk === "string") {
|
||||
yield {
|
||||
type: "text",
|
||||
text: chunk,
|
||||
}
|
||||
|
||||
continue
|
||||
}
|
||||
|
||||
// Handle system init messages
|
||||
if (chunk.type === "system" && "subtype" in chunk) {
|
||||
if (chunk.subtype === "init") {
|
||||
// Based on my tests, subscription usage sets the `apiKeySource` to "none"
|
||||
isPaidUsage = (chunk as any).apiKeySource !== "none"
|
||||
}
|
||||
// Also handles legacy rate_limit_event format (type: "system", subtype: "rate_limit_event")
|
||||
// by falling through — no special handling needed.
|
||||
continue
|
||||
}
|
||||
|
||||
// Handle rate_limit_event (newer CLI format: top-level type)
|
||||
if (chunk.type === "rate_limit_event") {
|
||||
// Rate limit events are informational. Log them but don't yield anything.
|
||||
// If the rate limit blocks the response, the stream will end without
|
||||
// assistant messages and the task loop will handle the empty response.
|
||||
Logger.log("Claude Code rate limit event:", JSON.stringify(chunk))
|
||||
continue
|
||||
}
|
||||
|
||||
// Skip user messages (tool results from Claude Code's own tool execution)
|
||||
if (chunk.type === "user") {
|
||||
continue
|
||||
}
|
||||
|
||||
if (chunk.type === "assistant" && "message" in chunk) {
|
||||
const message = chunk.message
|
||||
|
||||
// Check for error field on the message (newer CLI format)
|
||||
if (message.error) {
|
||||
const firstContent = message.content?.[0]
|
||||
const errorText = firstContent && "text" in firstContent ? firstContent.text : undefined
|
||||
throw new Error(errorText ?? `Claude Code error: ${message.error}`)
|
||||
}
|
||||
|
||||
if (message.stop_reason !== null) {
|
||||
const firstContent = message.content?.[0]
|
||||
const content = firstContent && "text" in firstContent ? firstContent : undefined
|
||||
|
||||
// Check if content exists before accessing its properties
|
||||
if (content && content.text.startsWith(`API Error`)) {
|
||||
// Error messages are formatted as: `API Error: <<status code>> <<json>>`
|
||||
const errorMessageStart = content.text.indexOf("{")
|
||||
const errorMessage = content.text.slice(errorMessageStart)
|
||||
|
||||
const error = this.attemptParse(errorMessage)
|
||||
if (!error) {
|
||||
throw new Error(content.text)
|
||||
}
|
||||
|
||||
if (error.error.message.includes("Invalid model name")) {
|
||||
throw new Error(
|
||||
content.text +
|
||||
`\n\nAPI keys and subscription plans allow different models. Make sure the selected model is included in your plan.`,
|
||||
)
|
||||
}
|
||||
|
||||
throw new Error(errorMessage)
|
||||
}
|
||||
}
|
||||
|
||||
for (const content of message.content) {
|
||||
switch (content.type) {
|
||||
case "text":
|
||||
yield {
|
||||
type: "text",
|
||||
text: content.text,
|
||||
}
|
||||
break
|
||||
case "thinking":
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: content.thinking || "",
|
||||
}
|
||||
break
|
||||
case "redacted_thinking":
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: "[Redacted thinking block]",
|
||||
}
|
||||
break
|
||||
case "tool_use":
|
||||
// Yield tool_use blocks to the streaming pipeline for proper tool execution
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
tool_call: {
|
||||
call_id: content.id,
|
||||
function: {
|
||||
id: content.id,
|
||||
name: content.name,
|
||||
arguments: JSON.stringify(content.input),
|
||||
},
|
||||
},
|
||||
}
|
||||
break
|
||||
default: {
|
||||
// Handle unknown content block types gracefully.
|
||||
// Newer Anthropic models or CLI versions may introduce new content types
|
||||
// (e.g., server_tool_use, mcp_tool_use). Log them instead of silently dropping.
|
||||
const unknownBlock = content as { type: string; text?: string }
|
||||
Logger.warn(`Unhandled content type in Claude Code response: ${unknownBlock.type}`)
|
||||
// If the unknown block has a text-like field, try to yield it as text
|
||||
if (typeof unknownBlock.text === "string") {
|
||||
yield {
|
||||
type: "text",
|
||||
text: unknownBlock.text,
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// According to Anthropic's API documentation:
|
||||
// https://docs.anthropic.com/en/api/messages#usage-object
|
||||
// The `input_tokens` field already includes both `cache_read_input_tokens` and `cache_creation_input_tokens`.
|
||||
// Therefore, we should not add cache tokens to the input_tokens count again, as this would result in double-counting.
|
||||
usage.inputTokens = message.usage?.input_tokens ?? 0
|
||||
usage.outputTokens = message.usage?.output_tokens ?? 0
|
||||
usage.cacheReadTokens = message.usage?.cache_read_input_tokens ?? 0
|
||||
usage.cacheWriteTokens = message.usage?.cache_creation_input_tokens ?? 0
|
||||
|
||||
continue
|
||||
}
|
||||
|
||||
if (chunk.type === "result" && "result" in chunk) {
|
||||
if (chunk.is_error) {
|
||||
throw new Error(`Claude Code returned an error: ${chunk.result}`)
|
||||
}
|
||||
|
||||
usage.totalCost = isPaidUsage ? chunk.total_cost_usd : 0
|
||||
|
||||
yield usage
|
||||
continue
|
||||
}
|
||||
|
||||
// ErrorMessage — log it explicitly and skip
|
||||
if ((chunk as any).type === "error") {
|
||||
Logger.warn("Claude Code emitted an error-type chunk:", JSON.stringify(chunk))
|
||||
continue
|
||||
}
|
||||
|
||||
// Any completely unrecognized chunk type — log and skip
|
||||
Logger.warn(`Unrecognized Claude Code chunk type: ${(chunk as any).type}`)
|
||||
}
|
||||
}
|
||||
|
||||
private attemptParse(str: string) {
|
||||
try {
|
||||
return JSON.parse(str)
|
||||
} catch (_err) {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
getModel() {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in claudeCodeModels) {
|
||||
const id = modelId as ClaudeCodeModelId
|
||||
return { id, info: claudeCodeModels[id] }
|
||||
}
|
||||
|
||||
return {
|
||||
id: claudeCodeDefaultModelId,
|
||||
info: claudeCodeModels[claudeCodeDefaultModelId],
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,343 +0,0 @@
|
||||
import { type ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
|
||||
import { shouldSkipReasoningForModel } from "@utils/model-utils"
|
||||
import axios from "axios"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineEnv } from "@/config"
|
||||
import { refreshClineRecommendedModels } from "@/core/controller/models/refreshClineRecommendedModels"
|
||||
import { ClineAccountService } from "@/services/account/ClineAccountService"
|
||||
import { AuthService } from "@/services/auth/AuthService"
|
||||
import { buildClineExtraHeaders } from "@/services/EnvUtils"
|
||||
import { CLINE_ACCOUNT_AUTH_ERROR_MESSAGE } from "@/shared/ClineAccount"
|
||||
import { CLINE_RECOMMENDED_MODELS_FALLBACK } from "@/shared/cline/recommended-models"
|
||||
import type { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch, getAxiosSettings } from "@/shared/net"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import type { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { createOpenRouterStream } from "../transform/openrouter-stream"
|
||||
import type { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
|
||||
import { ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
import type { OpenRouterErrorResponse } from "./types"
|
||||
|
||||
interface ClineHandlerOptions extends CommonApiHandlerOptions {
|
||||
ulid?: string
|
||||
taskId?: string
|
||||
reasoningEffort?: string
|
||||
thinkingBudgetTokens?: number
|
||||
openRouterProviderSorting?: string
|
||||
openRouterModelId?: string
|
||||
openRouterModelInfo?: ModelInfo
|
||||
clineAccountId?: string
|
||||
clineApiKey?: string
|
||||
enableParallelToolCalling?: boolean
|
||||
}
|
||||
|
||||
function normalizeModelId(modelId: string): string {
|
||||
return modelId.trim().toLowerCase()
|
||||
}
|
||||
|
||||
const CLINE_FREE_MODEL_IDS = new Set(CLINE_RECOMMENDED_MODELS_FALLBACK.free.map((model) => normalizeModelId(model.id)))
|
||||
|
||||
function getCacheReadTokens(usage: any): number {
|
||||
return usage?.prompt_tokens_details?.cached_tokens || usage?.cache_read_input_tokens || 0
|
||||
}
|
||||
|
||||
function getCacheWriteTokens(usage: any): number {
|
||||
return usage?.prompt_tokens_details?.cache_write_tokens || usage?.cache_creation_input_tokens || 0
|
||||
}
|
||||
|
||||
export class ClineHandler implements ApiHandler {
|
||||
private options: ClineHandlerOptions
|
||||
private clineAccountService = ClineAccountService.getInstance()
|
||||
private _authService: AuthService
|
||||
private client: OpenAI | undefined
|
||||
lastGenerationId?: string
|
||||
private lastRequestId?: string
|
||||
|
||||
private get _baseUrl(): string {
|
||||
return ClineEnv.config().apiBaseUrl
|
||||
}
|
||||
|
||||
constructor(options: ClineHandlerOptions) {
|
||||
this.options = options
|
||||
this._authService = AuthService.getInstance()
|
||||
}
|
||||
|
||||
private async getFreeModelIdSet(): Promise<Set<string>> {
|
||||
try {
|
||||
const models = await refreshClineRecommendedModels()
|
||||
const freeModelIds = models.free.map((model) => normalizeModelId(model.id)).filter((modelId) => modelId.length > 0)
|
||||
if (freeModelIds.length > 0) {
|
||||
return new Set(freeModelIds)
|
||||
}
|
||||
} catch (error) {
|
||||
Logger.error("Error resolving Cline free model IDs from recommended models:", error)
|
||||
}
|
||||
|
||||
return CLINE_FREE_MODEL_IDS
|
||||
}
|
||||
|
||||
private async ensureClient(): Promise<OpenAI> {
|
||||
const clineAccountAuthToken = this.options.clineApiKey || (await this._authService.getAuthToken())
|
||||
if (!clineAccountAuthToken) {
|
||||
throw new Error(CLINE_ACCOUNT_AUTH_ERROR_MESSAGE)
|
||||
}
|
||||
if (!this.client) {
|
||||
try {
|
||||
const defaultHeaders: Record<string, string> = {
|
||||
"HTTP-Referer": "https://cline.bot",
|
||||
"X-Title": "Cline",
|
||||
"X-Task-ID": this.options.ulid || "",
|
||||
}
|
||||
Object.assign(defaultHeaders, await buildClineExtraHeaders())
|
||||
|
||||
this.client = new OpenAI({
|
||||
baseURL: `${this._baseUrl}/api/v1`,
|
||||
apiKey: clineAccountAuthToken,
|
||||
defaultHeaders,
|
||||
// Capture real HTTP request ID from initial streaming response headers
|
||||
fetch: async (...args: Parameters<typeof fetch>): Promise<Awaited<ReturnType<typeof fetch>>> => {
|
||||
const [input, init] = args
|
||||
const resp = await fetch(input, init)
|
||||
try {
|
||||
let urlStr = ""
|
||||
if (typeof input === "string") {
|
||||
urlStr = input
|
||||
} else if (input instanceof URL) {
|
||||
urlStr = input.toString()
|
||||
} else if (typeof (input as { url?: unknown }).url === "string") {
|
||||
urlStr = (input as { url: string }).url
|
||||
}
|
||||
// Only record for chat completions (the primary streaming request)
|
||||
if (urlStr.includes("/chat/completions")) {
|
||||
const rid = resp.headers.get("x-request-id") || resp.headers.get("request-id")
|
||||
if (rid) {
|
||||
this.lastRequestId = rid
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// ignore header capture errors
|
||||
}
|
||||
return resp
|
||||
},
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating Cline client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
// Ensure the client is always using the latest auth token
|
||||
this.client.apiKey = clineAccountAuthToken
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
try {
|
||||
const client = await this.ensureClient()
|
||||
|
||||
this.lastGenerationId = undefined
|
||||
this.lastRequestId = undefined
|
||||
|
||||
let didOutputUsage = false
|
||||
const freeModelIds = await this.getFreeModelIdSet()
|
||||
|
||||
const stream = await createOpenRouterStream(
|
||||
client,
|
||||
systemPrompt,
|
||||
messages,
|
||||
this.getModel(),
|
||||
this.options.reasoningEffort,
|
||||
this.options.thinkingBudgetTokens,
|
||||
this.options.openRouterProviderSorting,
|
||||
tools,
|
||||
this.options.enableParallelToolCalling,
|
||||
)
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
Logger.debug("ClineHandler chunk:" + JSON.stringify(chunk))
|
||||
// openrouter returns an error object instead of the openai sdk throwing an error
|
||||
if ("error" in chunk) {
|
||||
const error = chunk.error as OpenRouterErrorResponse["error"]
|
||||
Logger.error(`Cline API Error: ${error?.code} - ${error?.message}`)
|
||||
// Include metadata in the error message if available
|
||||
const metadataStr = error.metadata ? `\nMetadata: ${JSON.stringify(error.metadata, null, 2)}` : ""
|
||||
throw new Error(`Cline API Error ${error.code}: ${error.message}${metadataStr}`)
|
||||
}
|
||||
|
||||
if (!this.lastGenerationId && chunk.id) {
|
||||
this.lastGenerationId = chunk.id
|
||||
}
|
||||
|
||||
// Check for mid-stream error via finish_reason
|
||||
const choice = chunk.choices?.[0]
|
||||
// OpenRouter may return finish_reason = "error" with error details
|
||||
if ((choice?.finish_reason as string) === "error") {
|
||||
const choiceWithError = choice as any
|
||||
if (choiceWithError.error) {
|
||||
const error = choiceWithError.error
|
||||
Logger.error(`Cline Mid-Stream Error: ${error.code || error.type || "Unknown"} - ${error.message}`)
|
||||
throw new Error(`Cline Mid-Stream Error: ${error.code || error.type || "Unknown"} - ${error.message}`)
|
||||
}
|
||||
throw new Error("Cline Mid-Stream Error: Stream terminated with error status but no error details provided")
|
||||
}
|
||||
|
||||
const delta = choice?.delta
|
||||
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
// Reasoning tokens are returned separately from the content
|
||||
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
|
||||
if (
|
||||
delta &&
|
||||
"reasoning" in delta &&
|
||||
delta.reasoning &&
|
||||
!shouldSkipReasoningForModel(this.options.openRouterModelId)
|
||||
) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
OpenRouter passes reasoning details that we can pass back unmodified in api requests to preserve reasoning traces for model
|
||||
- The reasoning_details array in each chunk may contain one or more reasoning objects
|
||||
- For encrypted reasoning, the content may appear as [REDACTED] in streaming responses
|
||||
- The complete reasoning sequence is built by concatenating all chunks in order
|
||||
See: https://openrouter.ai/docs/use-cases/reasoning-tokens#preserving-reasoning-blocks
|
||||
*/
|
||||
if (
|
||||
delta &&
|
||||
"reasoning_details" in delta &&
|
||||
delta.reasoning_details &&
|
||||
// @ts-expect-error-next-line
|
||||
delta?.reasoning_details?.length && // exists and non-0
|
||||
!shouldSkipReasoningForModel(this.options.openRouterModelId)
|
||||
) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: "",
|
||||
details: delta.reasoning_details,
|
||||
}
|
||||
}
|
||||
|
||||
if (!didOutputUsage && chunk.usage) {
|
||||
// @ts-expect-error-next-line
|
||||
let totalCost = (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0)
|
||||
const modelId = this.getModel().id
|
||||
const isFreeModel = freeModelIds.has(normalizeModelId(modelId))
|
||||
const cacheReadTokens = getCacheReadTokens(chunk.usage)
|
||||
const cacheWriteTokens = getCacheWriteTokens(chunk.usage)
|
||||
|
||||
if (isFreeModel) {
|
||||
totalCost = 0
|
||||
}
|
||||
|
||||
yield {
|
||||
type: "usage",
|
||||
cacheWriteTokens,
|
||||
cacheReadTokens,
|
||||
inputTokens: Math.max(0, (chunk.usage.prompt_tokens || 0) - cacheReadTokens - cacheWriteTokens),
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
totalCost,
|
||||
}
|
||||
didOutputUsage = true
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback to generation endpoint if usage chunk not returned
|
||||
if (!didOutputUsage) {
|
||||
Logger.warn("Cline API did not return usage chunk, fetching from generation endpoint")
|
||||
const apiStreamUsage = await this.getApiStreamUsage(freeModelIds)
|
||||
if (apiStreamUsage) {
|
||||
yield apiStreamUsage
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
Logger.error("Cline API Error:", error)
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async getApiStreamUsage(freeModelIds?: Set<string>): Promise<ApiStreamUsageChunk | undefined> {
|
||||
if (this.lastGenerationId) {
|
||||
try {
|
||||
const resolvedFreeModelIds = freeModelIds || (await this.getFreeModelIdSet())
|
||||
const clineAccountAuthToken = await this._authService.getAuthToken()
|
||||
if (!clineAccountAuthToken) {
|
||||
throw new Error(CLINE_ACCOUNT_AUTH_ERROR_MESSAGE)
|
||||
}
|
||||
const headers: Record<string, string> = {
|
||||
// Align with backend auth expectations
|
||||
Authorization: `Bearer ${clineAccountAuthToken}`,
|
||||
}
|
||||
Object.assign(headers, await buildClineExtraHeaders())
|
||||
|
||||
const response = await axios.get(`${this.clineAccountService.baseUrl}/generation?id=${this.lastGenerationId}`, {
|
||||
headers,
|
||||
timeout: 15_000, // this request hangs sometimes
|
||||
...getAxiosSettings(),
|
||||
})
|
||||
|
||||
const generation = response.data
|
||||
let totalCost = generation?.total_cost || 0
|
||||
const modelId = this.getModel().id
|
||||
const isFreeModel = resolvedFreeModelIds.has(normalizeModelId(modelId))
|
||||
|
||||
if (isFreeModel) {
|
||||
totalCost = 0
|
||||
}
|
||||
|
||||
return {
|
||||
type: "usage",
|
||||
cacheWriteTokens: generation?.native_tokens_cache_write || 0,
|
||||
cacheReadTokens: generation?.native_tokens_cached || 0,
|
||||
// openrouter generation endpoint fails often
|
||||
inputTokens: Math.max(
|
||||
0,
|
||||
(generation?.native_tokens_prompt || 0) -
|
||||
(generation?.native_tokens_cached || 0) -
|
||||
(generation?.native_tokens_cache_write || 0),
|
||||
),
|
||||
outputTokens: generation?.native_tokens_completion || 0,
|
||||
totalCost,
|
||||
}
|
||||
} catch (error) {
|
||||
// ignore if fails
|
||||
Logger.error("Error fetching cline generation details:", error)
|
||||
}
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
// Expose the last HTTP request ID captured from response headers (X-Request-ID)
|
||||
getLastRequestId(): string | undefined {
|
||||
return this.lastRequestId
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
const modelId = this.options.openRouterModelId
|
||||
const modelInfo = this.options.openRouterModelInfo
|
||||
if (modelId && modelInfo) {
|
||||
return { id: modelId, info: modelInfo }
|
||||
}
|
||||
// If we have a model ID but no model info (e.g., CLI featured models),
|
||||
// use the ID with default model info rather than falling back to a different model
|
||||
if (modelId) {
|
||||
return { id: modelId, info: openRouterDefaultModelInfo }
|
||||
}
|
||||
return { id: openRouterDefaultModelId, info: openRouterDefaultModelInfo }
|
||||
}
|
||||
}
|
||||
@@ -1,142 +0,0 @@
|
||||
import { DeepSeekModelId, deepSeekDefaultModelId, deepSeekModels, ModelInfo } from "@shared/api"
|
||||
import { calculateApiCostOpenAI } from "@utils/cost"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { addReasoningContent } from "../transform/r1-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface DeepSeekHandlerOptions extends CommonApiHandlerOptions {
|
||||
deepSeekApiKey?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
export class DeepSeekHandler implements ApiHandler {
|
||||
private options: DeepSeekHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: DeepSeekHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.deepSeekApiKey) {
|
||||
throw new Error("DeepSeek API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = new OpenAI({
|
||||
baseURL: "https://api.deepseek.com/v1",
|
||||
apiKey: this.options.deepSeekApiKey,
|
||||
defaultHeaders: buildExternalBasicHeaders(),
|
||||
fetch, // Use configured fetch with proxy support
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating DeepSeek client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
|
||||
// Deepseek reports total input AND cache reads/writes,
|
||||
// see context caching: https://api-docs.deepseek.com/guides/kv_cache)
|
||||
// where the input tokens is the sum of the cache hits/misses, just like OpenAI.
|
||||
// This affects:
|
||||
// 1) context management truncation algorithm, and
|
||||
// 2) cost calculation
|
||||
|
||||
// Deepseek usage includes extra fields.
|
||||
// Safely cast the prompt token details section to the appropriate structure.
|
||||
interface DeepSeekUsage extends OpenAI.CompletionUsage {
|
||||
prompt_cache_hit_tokens?: number
|
||||
prompt_cache_miss_tokens?: number
|
||||
}
|
||||
const deepUsage = usage as DeepSeekUsage
|
||||
|
||||
const inputTokens = deepUsage?.prompt_tokens || 0 // sum of cache hits and misses
|
||||
const outputTokens = deepUsage?.completion_tokens || 0
|
||||
const cacheReadTokens = deepUsage?.prompt_cache_hit_tokens || 0
|
||||
const cacheWriteTokens = deepUsage?.prompt_cache_miss_tokens || 0
|
||||
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
|
||||
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens) // this will always be 0
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: nonCachedInputTokens,
|
||||
outputTokens: outputTokens,
|
||||
cacheWriteTokens: cacheWriteTokens,
|
||||
cacheReadTokens: cacheReadTokens,
|
||||
totalCost: totalCost,
|
||||
}
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
|
||||
const isDeepSeekThinkingModel =
|
||||
model.id.includes("deepseek-reasoner") || model.id === "deepseek-v4-flash" || model.id === "deepseek-v4-pro"
|
||||
|
||||
const convertedMessages = convertToOpenAiMessages(messages)
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = isDeepSeekThinkingModel
|
||||
? [{ role: "system", content: systemPrompt }, ...addReasoningContent(convertedMessages, messages)]
|
||||
: [{ role: "system", content: systemPrompt }, ...convertedMessages]
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
max_completion_tokens: model.info.maxTokens,
|
||||
messages: openAiMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
// Only set temperature for non-thinking models
|
||||
...(isDeepSeekThinkingModel ? {} : { temperature: 0 }),
|
||||
...getOpenAIToolParams(tools),
|
||||
})
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield* this.yieldUsage(model.info, chunk.usage)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: DeepSeekModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in deepSeekModels) {
|
||||
const id = modelId as DeepSeekModelId
|
||||
return { id, info: deepSeekModels[id] }
|
||||
}
|
||||
return {
|
||||
id: deepSeekDefaultModelId,
|
||||
info: deepSeekModels[deepSeekDefaultModelId],
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,657 +0,0 @@
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ModelInfo } from "../../../shared/api"
|
||||
import { ApiHandler } from "../index"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface DifyHandlerOptions {
|
||||
difyApiKey?: string
|
||||
difyBaseUrl?: string
|
||||
}
|
||||
|
||||
// Dify API Response Types
|
||||
export interface DifyFileResponse {
|
||||
id: string
|
||||
name: string
|
||||
size: number
|
||||
extension: string
|
||||
mime_type: string
|
||||
created_by: string
|
||||
created_at: number
|
||||
}
|
||||
|
||||
export interface DifyMessage {
|
||||
id: string
|
||||
conversation_id: string
|
||||
inputs: Record<string, any>
|
||||
query: string
|
||||
message_files: Array<{
|
||||
id: string
|
||||
type: string
|
||||
url: string
|
||||
belongs_to: string
|
||||
}>
|
||||
answer: string
|
||||
created_at: number
|
||||
feedback?: {
|
||||
rating: string
|
||||
}
|
||||
retriever_resources?: any[]
|
||||
}
|
||||
|
||||
interface DifyHistoryResponse {
|
||||
data: DifyMessage[]
|
||||
has_more: boolean
|
||||
limit: number
|
||||
}
|
||||
|
||||
interface DifyConversation {
|
||||
id: string
|
||||
name: string
|
||||
inputs: Record<string, any>
|
||||
status: string
|
||||
introduction: string
|
||||
created_at: number
|
||||
updated_at: number
|
||||
}
|
||||
|
||||
interface DifyConversationsResponse {
|
||||
data: DifyConversation[]
|
||||
has_more: boolean
|
||||
limit: number
|
||||
}
|
||||
|
||||
interface DifyConversationResponse {
|
||||
id: string
|
||||
name: string
|
||||
inputs: Record<string, any>
|
||||
status: string
|
||||
introduction: string
|
||||
created_at: number
|
||||
updated_at: number
|
||||
}
|
||||
|
||||
export class DifyHandler implements ApiHandler {
|
||||
private options: DifyHandlerOptions
|
||||
private baseUrl: string
|
||||
private apiKey: string
|
||||
private conversationId: string | null = null
|
||||
private currentTaskId: string | null = null
|
||||
private abortController: AbortController | null = null
|
||||
|
||||
constructor(options: DifyHandlerOptions) {
|
||||
this.options = options
|
||||
this.apiKey = options.difyApiKey || ""
|
||||
this.baseUrl = options.difyBaseUrl || ""
|
||||
|
||||
Logger.log("[DIFY DEBUG] Constructor called with:", {
|
||||
hasApiKey: !!this.apiKey,
|
||||
baseUrl: this.baseUrl,
|
||||
})
|
||||
|
||||
if (!this.apiKey) {
|
||||
throw new Error("Dify API key is required")
|
||||
}
|
||||
if (!this.baseUrl) {
|
||||
throw new Error("Dify base URL is required")
|
||||
}
|
||||
}
|
||||
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
Logger.log("[DIFY DEBUG] createMessage called with:", {
|
||||
systemPromptLength: systemPrompt?.length || 0,
|
||||
messagesCount: messages?.length || 0,
|
||||
})
|
||||
|
||||
// Convert messages to Dify format
|
||||
const query = this.convertMessagesToQuery(systemPrompt, messages)
|
||||
const requestBody = {
|
||||
inputs: {},
|
||||
query: query,
|
||||
response_mode: "streaming",
|
||||
conversation_id: this.conversationId || "",
|
||||
user: "cline-user", // A unique user identifier
|
||||
files: [],
|
||||
}
|
||||
|
||||
const fullUrl = `${this.baseUrl}/chat-messages`
|
||||
Logger.log("[DIFY DEBUG] Making request to:", fullUrl)
|
||||
Logger.log("[DIFY DEBUG] Request body:", JSON.stringify(requestBody, null, 2))
|
||||
|
||||
let response: Response
|
||||
try {
|
||||
response = await fetch(fullUrl, {
|
||||
method: "POST",
|
||||
headers: this.jsonHeaders(),
|
||||
body: JSON.stringify(requestBody),
|
||||
})
|
||||
} catch (error: any) {
|
||||
Logger.error("[DIFY DEBUG] Network error during fetch:", error)
|
||||
const cause = error.cause ? ` | Cause: ${error.cause}` : ""
|
||||
throw new Error(`Dify API network error: ${error.message}${cause}`)
|
||||
}
|
||||
|
||||
Logger.log("[DIFY DEBUG] Response status:", response.status)
|
||||
const headersObj: Record<string, string> = {}
|
||||
response.headers.forEach((value, key) => {
|
||||
headersObj[key] = value
|
||||
})
|
||||
Logger.log("[DIFY DEBUG] Response headers:", headersObj)
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
Logger.error("[DIFY DEBUG] Error response:", errorText)
|
||||
throw new Error(`Dify API error: ${response.status} ${response.statusText} - ${errorText}`)
|
||||
}
|
||||
|
||||
if (!response.body) {
|
||||
throw new Error("No response body from Dify API")
|
||||
}
|
||||
|
||||
const reader = response.body.getReader()
|
||||
const decoder = new TextDecoder()
|
||||
let buffer = ""
|
||||
let fullText = ""
|
||||
let hasYieldedContent = false
|
||||
const processedEvents: string[] = []
|
||||
let lastEventTime = Date.now()
|
||||
|
||||
Logger.log("[DIFY DEBUG] Starting to read streaming response...")
|
||||
|
||||
try {
|
||||
while (true) {
|
||||
const { done, value } = await reader.read()
|
||||
if (done) {
|
||||
Logger.log("[DIFY DEBUG] Stream ended naturally")
|
||||
Logger.log(
|
||||
"[DIFY DEBUG] Final state - hasYieldedContent:",
|
||||
hasYieldedContent,
|
||||
"fullText length:",
|
||||
fullText.length,
|
||||
"processedEvents:",
|
||||
processedEvents,
|
||||
)
|
||||
break
|
||||
}
|
||||
|
||||
const chunk = decoder.decode(value, { stream: true })
|
||||
Logger.log("[DIFY DEBUG] Raw chunk received:", JSON.stringify(chunk))
|
||||
|
||||
buffer += chunk
|
||||
const lines = buffer.split("\n")
|
||||
|
||||
// Keep the last incomplete line in the buffer
|
||||
buffer = lines.pop() || ""
|
||||
|
||||
for (const line of lines) {
|
||||
Logger.log("[DIFY DEBUG] Processing line:", JSON.stringify(line))
|
||||
|
||||
if (line.startsWith("data: ")) {
|
||||
const data = line.slice(6).trim()
|
||||
Logger.log("[DIFY DEBUG] Extracted data:", JSON.stringify(data))
|
||||
|
||||
if (data === "[DONE]") {
|
||||
Logger.log("[DIFY DEBUG] Received [DONE] signal")
|
||||
break
|
||||
}
|
||||
|
||||
if (data === "") {
|
||||
Logger.log("[DIFY DEBUG] Empty data line, skipping")
|
||||
continue
|
||||
}
|
||||
|
||||
try {
|
||||
const parsed = JSON.parse(data)
|
||||
Logger.log("[DIFY DEBUG] Parsed JSON:", parsed)
|
||||
processedEvents.push(parsed.event || "unknown")
|
||||
lastEventTime = Date.now()
|
||||
|
||||
// Capture conversation_id as soon as it's available
|
||||
if (parsed.conversation_id && !this.conversationId) {
|
||||
this.conversationId = parsed.conversation_id
|
||||
Logger.log("[DIFY DEBUG] Captured conversation_id:", this.conversationId)
|
||||
}
|
||||
|
||||
// Handle different Dify event types based on actual Dify API
|
||||
if (parsed.event === "message") {
|
||||
Logger.log("[DIFY DEBUG] Message event, answer:", parsed.answer)
|
||||
// Dify sends the full text in each "answer" chunk, so we replace.
|
||||
if (typeof parsed.answer === "string") {
|
||||
fullText = parsed.answer
|
||||
Logger.log("[DIFY DEBUG] Updated fullText length:", fullText.length)
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
hasYieldedContent = true
|
||||
}
|
||||
} else if (parsed.event === "message_replace") {
|
||||
Logger.log("[DIFY DEBUG] Replace message event:", parsed)
|
||||
if (parsed.answer) {
|
||||
fullText = parsed.answer // Replace instead of append
|
||||
Logger.log("[DIFY DEBUG] Replaced fullText length:", fullText.length)
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
hasYieldedContent = true
|
||||
}
|
||||
} else if (parsed.event === "message_end") {
|
||||
Logger.log("[DIFY DEBUG] Message end event", parsed)
|
||||
// Message completed. Yield final text if we have any.
|
||||
if (fullText) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
hasYieldedContent = true
|
||||
}
|
||||
// Yield usage data if available
|
||||
if (parsed.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: parsed.usage.prompt_tokens || 0,
|
||||
outputTokens: parsed.usage.completion_tokens || parsed.usage.total_tokens || 0,
|
||||
totalCost: parsed.usage.total_price || 0,
|
||||
}
|
||||
}
|
||||
return // End of stream
|
||||
} else if (parsed.event === "error") {
|
||||
Logger.error("[DIFY DEBUG] Error event:", parsed)
|
||||
throw new Error(`Dify API error: ${parsed.message || "Unknown error"}`)
|
||||
} else if (parsed.event === "workflow_started" || parsed.event === "workflow_finished") {
|
||||
Logger.log("[DIFY DEBUG] Workflow event:", parsed.event)
|
||||
// These are informational events, continue processing
|
||||
} else if (parsed.event === "node_started" || parsed.event === "node_finished") {
|
||||
Logger.log("[DIFY DEBUG] Node event:", parsed.event, parsed.data)
|
||||
// These are informational events, continue processing
|
||||
} else if (parsed.event === "ping") {
|
||||
Logger.log("[DIFY DEBUG] Ping event received, keeping connection alive.")
|
||||
// Ping event, do nothing
|
||||
} else {
|
||||
Logger.log("[DIFY DEBUG] Unknown event type:", parsed.event, "Full object:", parsed)
|
||||
// Try to extract text from other possible fields
|
||||
if (parsed.text) {
|
||||
fullText += parsed.text
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
hasYieldedContent = true
|
||||
} else if (parsed.content) {
|
||||
fullText += parsed.content
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
hasYieldedContent = true
|
||||
} else if (parsed.answer) {
|
||||
// Fallback: some events might have answer field even if not "message" type
|
||||
fullText += parsed.answer
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
hasYieldedContent = true
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
Logger.warn("[DIFY DEBUG] Failed to parse JSON:", data, "Error:", e)
|
||||
}
|
||||
} else if (line.trim() !== "") {
|
||||
Logger.log(
|
||||
"[DIFY DEBUG] Non-data line (not starting with 'data:'), trying to parse as direct JSON:",
|
||||
JSON.stringify(line),
|
||||
)
|
||||
// Try to parse as direct JSON (fallback for non-SSE responses, though Dify uses SSE)
|
||||
try {
|
||||
const parsed = JSON.parse(line.trim())
|
||||
Logger.log("[DIFY DEBUG] Parsed direct JSON:", parsed)
|
||||
processedEvents.push(parsed.event || "direct-json")
|
||||
|
||||
// Handle the same event types as above
|
||||
if (parsed.event === "message" && parsed.answer) {
|
||||
fullText += parsed.answer
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
hasYieldedContent = true
|
||||
} else if (parsed.event === "message_end") {
|
||||
if (fullText) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
hasYieldedContent = true
|
||||
}
|
||||
return
|
||||
} else if (parsed.event === "error") {
|
||||
Logger.error("[DIFY DEBUG] Direct JSON Error event:", parsed)
|
||||
throw new Error(`Dify API error: ${parsed.message || "Unknown error"}`)
|
||||
} else if (parsed.answer || parsed.text || parsed.content) {
|
||||
// Fallback for any content in direct JSON
|
||||
const content = parsed.answer || parsed.text || parsed.content
|
||||
fullText += content
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
hasYieldedContent = true
|
||||
}
|
||||
} catch (e) {
|
||||
// Not JSON, continue
|
||||
Logger.log("[DIFY DEBUG] Line is not direct JSON, continuing")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Final check - if we haven't yielded any content, provide diagnostic information
|
||||
if (!hasYieldedContent) {
|
||||
const diagnosticInfo = {
|
||||
processedEvents,
|
||||
finalFullTextLength: fullText.length,
|
||||
finalFullText: fullText,
|
||||
streamDuration: Date.now() - lastEventTime,
|
||||
conversationId: this.conversationId,
|
||||
}
|
||||
Logger.error("[DIFY DEBUG] No content was yielded! Diagnostic info:", diagnosticInfo)
|
||||
|
||||
// If we have any accumulated text at all, yield it as a fallback
|
||||
if (fullText.trim()) {
|
||||
Logger.log("[DIFY DEBUG] Yielding accumulated text as fallback:", fullText)
|
||||
yield {
|
||||
type: "text",
|
||||
text: fullText,
|
||||
}
|
||||
} else {
|
||||
// Provide a more informative error
|
||||
throw new Error(
|
||||
`Dify API did not provide any assistant messages. ` +
|
||||
`Events processed: [${processedEvents.join(", ")}]. ` +
|
||||
`Check your Dify application configuration and ensure it's properly set up to return responses. ` +
|
||||
`API URL: ${fullUrl}. Conversation ID: ${this.conversationId || "none"}.`,
|
||||
)
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock()
|
||||
Logger.log("[DIFY DEBUG] Stream reader released")
|
||||
}
|
||||
}
|
||||
|
||||
private convertMessagesToQuery(systemPrompt: string, messages: ClineStorageMessage[]): string {
|
||||
// Dify's context is managed by `conversation_id`. The `query` should be the last user message.
|
||||
// The system prompt is typically configured in the Dify App itself.
|
||||
const lastUserMessage = messages.filter((m) => m.role === "user").pop()
|
||||
|
||||
if (!lastUserMessage) {
|
||||
return "" // Should not happen in normal flow
|
||||
}
|
||||
|
||||
const userQuery = Array.isArray(lastUserMessage.content)
|
||||
? lastUserMessage.content.map((c) => ("text" in c ? c.text : "")).join("\n")
|
||||
: (lastUserMessage.content as string)
|
||||
|
||||
// Only prepend the system prompt if it's the very first message of a new conversation.
|
||||
if (!this.conversationId && systemPrompt) {
|
||||
Logger.log("[DIFY DEBUG] Prepending system prompt for new conversation.")
|
||||
return `${systemPrompt}\n\n---\n\n${userQuery}`
|
||||
}
|
||||
|
||||
return userQuery
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
return {
|
||||
id: "dify-workflow",
|
||||
info: {
|
||||
maxTokens: 8192,
|
||||
contextWindow: 128000,
|
||||
supportsImages: true,
|
||||
supportsPromptCache: false,
|
||||
inputPrice: 0,
|
||||
outputPrice: 0,
|
||||
description: "Dify workflow - model selection is configured in your Dify application",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// Additional Dify API Methods
|
||||
|
||||
/**
|
||||
* Upload a file for use in conversations
|
||||
* @param file File buffer to upload
|
||||
* @param filename Name of the file
|
||||
* @param user User identifier (defaults to "cline-user")
|
||||
* @returns Promise with file upload response
|
||||
*/
|
||||
async uploadFile(file: Buffer, filename: string, user: string = "cline-user"): Promise<DifyFileResponse> {
|
||||
const formData = new FormData()
|
||||
formData.append("file", new Blob([new Uint8Array(file)]), filename)
|
||||
formData.append("user", user)
|
||||
|
||||
const response = await fetch(`${this.baseUrl}/files/upload`, {
|
||||
method: "POST",
|
||||
headers: this.headers(),
|
||||
body: formData,
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(`Dify file upload error: ${response.status} ${response.statusText} - ${errorText}`)
|
||||
}
|
||||
|
||||
return response.json()
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop generation for a specific task
|
||||
* @param taskId Task ID from streaming response
|
||||
* @param user User identifier (defaults to "cline-user")
|
||||
* @returns Promise that resolves when generation is stopped
|
||||
*/
|
||||
async stopGeneration(taskId: string, user: string = "cline-user"): Promise<void> {
|
||||
const response = await fetch(`${this.baseUrl}/chat-messages/${taskId}/stop`, {
|
||||
method: "POST",
|
||||
headers: this.jsonHeaders(),
|
||||
body: JSON.stringify({ user }),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(`Dify stop generation error: ${response.status} ${response.statusText} - ${errorText}`)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get conversation history messages with pagination
|
||||
* @param conversationId Conversation ID
|
||||
* @param user User identifier (defaults to "cline-user")
|
||||
* @param firstId First message ID for pagination (optional)
|
||||
* @param limit Number of messages to return (default: 20)
|
||||
* @returns Promise with conversation history
|
||||
*/
|
||||
async getConversationHistory(
|
||||
conversationId: string,
|
||||
user: string = "cline-user",
|
||||
firstId?: string,
|
||||
limit: number = 20,
|
||||
): Promise<DifyHistoryResponse> {
|
||||
const params = new URLSearchParams({ user, limit: limit.toString() })
|
||||
if (firstId) {
|
||||
params.append("first_id", firstId)
|
||||
}
|
||||
|
||||
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}/messages?${params}`, {
|
||||
headers: this.headers(),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(`Dify get conversation history error: ${response.status} ${response.statusText} - ${errorText}`)
|
||||
}
|
||||
|
||||
return response.json()
|
||||
}
|
||||
|
||||
/**
|
||||
* Get list of conversations for a user
|
||||
* @param user User identifier (defaults to "cline-user")
|
||||
* @param lastId Last conversation ID for pagination (optional)
|
||||
* @param limit Number of conversations to return (default: 20)
|
||||
* @param sortBy Sort field (default: "-updated_at")
|
||||
* @returns Promise with conversations list
|
||||
*/
|
||||
async getConversations(
|
||||
user: string = "cline-user",
|
||||
lastId?: string,
|
||||
limit: number = 20,
|
||||
sortBy: string = "-updated_at",
|
||||
): Promise<DifyConversationsResponse> {
|
||||
const params = new URLSearchParams({
|
||||
user,
|
||||
limit: limit.toString(),
|
||||
sort_by: sortBy,
|
||||
})
|
||||
if (lastId) {
|
||||
params.append("last_id", lastId)
|
||||
}
|
||||
|
||||
const response = await fetch(`${this.baseUrl}/conversations?${params}`, {
|
||||
headers: this.headers(),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(`Dify get conversations error: ${response.status} ${response.statusText} - ${errorText}`)
|
||||
}
|
||||
|
||||
return response.json()
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete a conversation
|
||||
* @param conversationId Conversation ID to delete
|
||||
* @param user User identifier (defaults to "cline-user")
|
||||
* @returns Promise that resolves when conversation is deleted
|
||||
*/
|
||||
async deleteConversation(conversationId: string, user: string = "cline-user"): Promise<void> {
|
||||
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}`, {
|
||||
method: "DELETE",
|
||||
headers: this.jsonHeaders(),
|
||||
body: JSON.stringify({ user }),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(`Dify delete conversation error: ${response.status} ${response.statusText} - ${errorText}`)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Rename a conversation
|
||||
* @param conversationId Conversation ID to rename
|
||||
* @param user User identifier (defaults to "cline-user")
|
||||
* @param name New conversation name (optional if auto_generate is true)
|
||||
* @param autoGenerate Whether to auto-generate the name (default: false)
|
||||
* @returns Promise with updated conversation details
|
||||
*/
|
||||
async renameConversation(
|
||||
conversationId: string,
|
||||
user: string = "cline-user",
|
||||
name?: string,
|
||||
autoGenerate: boolean = false,
|
||||
): Promise<DifyConversationResponse> {
|
||||
const body: any = { user, auto_generate: autoGenerate }
|
||||
if (name) {
|
||||
body.name = name
|
||||
}
|
||||
|
||||
const response = await fetch(`${this.baseUrl}/conversations/${conversationId}/name`, {
|
||||
method: "POST",
|
||||
headers: this.jsonHeaders(),
|
||||
body: JSON.stringify(body),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(`Dify rename conversation error: ${response.status} ${response.statusText} - ${errorText}`)
|
||||
}
|
||||
|
||||
return response.json()
|
||||
}
|
||||
|
||||
/**
|
||||
* Submit feedback for a message
|
||||
* @param messageId Message ID to provide feedback for
|
||||
* @param rating Rating: "like" or "dislike"
|
||||
* @param content Optional feedback content
|
||||
* @param user User identifier (defaults to "cline-user")
|
||||
* @returns Promise that resolves when feedback is submitted
|
||||
*/
|
||||
async submitMessageFeedback(
|
||||
messageId: string,
|
||||
rating: "like" | "dislike",
|
||||
content?: string,
|
||||
user: string = "cline-user",
|
||||
): Promise<void> {
|
||||
const body: any = { rating, user }
|
||||
if (content) {
|
||||
body.content = content
|
||||
}
|
||||
|
||||
const response = await fetch(`${this.baseUrl}/messages/${messageId}/feedbacks`, {
|
||||
method: "POST",
|
||||
headers: this.jsonHeaders(),
|
||||
body: JSON.stringify(body),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(`Dify submit feedback error: ${response.status} ${response.statusText} - ${errorText}`)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current conversation ID
|
||||
* @returns Current conversation ID or null
|
||||
*/
|
||||
getCurrentConversationId(): string | null {
|
||||
return this.conversationId
|
||||
}
|
||||
|
||||
/**
|
||||
* Set conversation ID for continuing existing conversations
|
||||
* @param conversationId Conversation ID to set
|
||||
*/
|
||||
setConversationId(conversationId: string): void {
|
||||
this.conversationId = conversationId
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset conversation ID to start a new conversation
|
||||
*/
|
||||
resetConversation(): void {
|
||||
this.conversationId = null
|
||||
this.currentTaskId = null
|
||||
}
|
||||
|
||||
private jsonHeaders() {
|
||||
return {
|
||||
...this.headers(),
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
}
|
||||
|
||||
private headers() {
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
return {
|
||||
...externalHeaders,
|
||||
Authorization: `Bearer ${this.apiKey}`,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,90 +0,0 @@
|
||||
import { DoubaoModelId, doubaoDefaultModelId, doubaoModels, ModelInfo } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from ".."
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface DoubaoHandlerOptions extends CommonApiHandlerOptions {
|
||||
doubaoApiKey?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
export class DoubaoHandler implements ApiHandler {
|
||||
private options: DoubaoHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
constructor(options: DoubaoHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.doubaoApiKey) {
|
||||
throw new Error("Doubao API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://ark.cn-beijing.volces.com/api/v3/",
|
||||
apiKey: this.options.doubaoApiKey,
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Doubao client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
getModel(): { id: DoubaoModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in doubaoModels) {
|
||||
const id = modelId as DoubaoModelId
|
||||
return { id, info: doubaoModels[id] }
|
||||
}
|
||||
return {
|
||||
id: doubaoDefaultModelId,
|
||||
info: doubaoModels[doubaoDefaultModelId],
|
||||
}
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
max_completion_tokens: model.info.maxTokens,
|
||||
messages: openAiMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
temperature: 0,
|
||||
})
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
// @ts-expect-error-next-line
|
||||
cacheReadTokens: chunk.usage.prompt_cache_hit_tokens || 0,
|
||||
// @ts-expect-error-next-line
|
||||
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,116 +0,0 @@
|
||||
import { FireworksModelId, fireworksDefaultModelId, fireworksModels, ModelInfo } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from ".."
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface FireworksHandlerOptions extends CommonApiHandlerOptions {
|
||||
fireworksApiKey?: string
|
||||
fireworksModelId?: string
|
||||
fireworksModelMaxCompletionTokens?: number
|
||||
fireworksModelMaxTokens?: number
|
||||
}
|
||||
|
||||
export class FireworksHandler implements ApiHandler {
|
||||
private options: FireworksHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: FireworksHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.fireworksApiKey) {
|
||||
throw new Error("Fireworks API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://api.fireworks.ai/inference/v1",
|
||||
apiKey: this.options.fireworksApiKey,
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Fireworks client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const modelId = this.options.fireworksModelId ?? ""
|
||||
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: modelId,
|
||||
messages: openAiMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
temperature: 0,
|
||||
})
|
||||
|
||||
let reasoning: string | null = null
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (reasoning || delta?.content?.includes("<think>")) {
|
||||
reasoning = (reasoning || "") + (delta.content ?? "")
|
||||
}
|
||||
|
||||
if (delta?.content && !reasoning) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (reasoning || (delta && "reasoning_content" in delta && delta.reasoning_content)) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: delta.content || ((delta as any).reasoning_content as string | undefined) || "",
|
||||
}
|
||||
if (reasoning?.includes("</think>")) {
|
||||
// Reset so the next chunk is regular content
|
||||
reasoning = null
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
const usage = chunk.usage as OpenAI.CompletionUsage & {
|
||||
prompt_cache_hit_tokens?: number
|
||||
prompt_cache_miss_tokens?: number
|
||||
prompt_tokens_details?: {
|
||||
cached_tokens?: number
|
||||
}
|
||||
}
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: usage.prompt_tokens || 0,
|
||||
outputTokens: usage.completion_tokens || 0,
|
||||
// Fireworks can return cache hits either as prompt_cache_hit_tokens or prompt_tokens_details.cached_tokens.
|
||||
cacheReadTokens: usage.prompt_cache_hit_tokens ?? usage.prompt_tokens_details?.cached_tokens ?? 0,
|
||||
cacheWriteTokens: usage.prompt_cache_miss_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: FireworksModelId; info: ModelInfo } {
|
||||
const modelId = this.options.fireworksModelId
|
||||
if (modelId && modelId in fireworksModels) {
|
||||
const id = modelId as FireworksModelId
|
||||
return { id, info: fireworksModels[id] }
|
||||
}
|
||||
return {
|
||||
id: fireworksDefaultModelId,
|
||||
info: fireworksModels[fireworksDefaultModelId],
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,570 +0,0 @@
|
||||
// Restore GenerateContentConfig import and add GenerateContentResponseUsageMetadata
|
||||
import {
|
||||
ApiError,
|
||||
FunctionCallingConfigMode,
|
||||
type GenerateContentConfig,
|
||||
type GenerateContentResponseUsageMetadata,
|
||||
GoogleGenAI,
|
||||
FunctionDeclaration as GoogleTool,
|
||||
ThinkingLevel,
|
||||
} from "@google/genai"
|
||||
import { GeminiModelId, geminiDefaultModelId, geminiModels, ModelInfo } from "@shared/api"
|
||||
import { GEMINI_FLASH_MAX_OUTPUT_TOKENS, isGeminiFlashModel } from "@utils/model-utils"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { telemetryService } from "@/services/telemetry"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { RetriableError, withRetry } from "../retry"
|
||||
import { convertAnthropicMessageToGemini } from "../transform/gemini-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
const rateLimitPatterns = [/got status: 429/i, /429 Too Many Requests/i, /rate limit exceeded/i, /too many requests/i]
|
||||
|
||||
interface GeminiHandlerOptions extends CommonApiHandlerOptions {
|
||||
isVertex?: boolean
|
||||
vertexProjectId?: string
|
||||
vertexRegion?: string
|
||||
geminiApiKey?: string
|
||||
geminiBaseUrl?: string
|
||||
thinkingBudgetTokens?: number
|
||||
reasoningEffort?: string
|
||||
apiModelId?: string
|
||||
ulid?: string
|
||||
}
|
||||
|
||||
function mapReasoningEffortToGeminiThinkingLevel(effort: string): ThinkingLevel {
|
||||
switch (effort) {
|
||||
case "low":
|
||||
case "medium":
|
||||
return ThinkingLevel.LOW
|
||||
case "high":
|
||||
case "xhigh":
|
||||
return ThinkingLevel.HIGH
|
||||
default:
|
||||
return ThinkingLevel.LOW
|
||||
}
|
||||
}
|
||||
|
||||
function getGeminiMaxOutputTokens(modelId: string, modelMaxTokens?: number): number | undefined {
|
||||
if (!isGeminiFlashModel(modelId)) {
|
||||
return undefined
|
||||
}
|
||||
|
||||
if (modelMaxTokens && modelMaxTokens > 0) {
|
||||
return Math.min(modelMaxTokens, GEMINI_FLASH_MAX_OUTPUT_TOKENS)
|
||||
}
|
||||
|
||||
return GEMINI_FLASH_MAX_OUTPUT_TOKENS
|
||||
}
|
||||
|
||||
/**
|
||||
* Handler for Google's Gemini API with optimized caching strategy and accurate cost accounting.
|
||||
*
|
||||
* Key features:
|
||||
* - One cache per task: Creates a single cache per task and reuses it for subsequent turns
|
||||
* - Stable cache keys: Uses ulid as a stable identifier for caches
|
||||
* - Efficient cache updates: Only updates caches when there's new content to add
|
||||
* - Split cost accounting: Separates immediate costs from ongoing cache storage costs
|
||||
*
|
||||
* Cost accounting approach:
|
||||
* - Immediate costs (per message): Input tokens, output tokens, and cache read costs
|
||||
* - Ongoing costs (per task): Cache storage costs for the TTL period
|
||||
*
|
||||
* Gemini's caching system is unique in that it charges for holding tokens in cache by the hour.
|
||||
* This implementation optimizes for both performance and cost by:
|
||||
* 1. Minimizing redundant cache creations
|
||||
* 2. Properly accounting for cache costs in the billing calculations
|
||||
* 3. Using a stable cache key to ensure cache reuse across turns
|
||||
* 4. Separating immediate costs from ongoing costs to avoid double-counting
|
||||
*/
|
||||
export class GeminiHandler implements ApiHandler {
|
||||
private options: GeminiHandlerOptions
|
||||
private client: GoogleGenAI | undefined
|
||||
|
||||
constructor(options: GeminiHandlerOptions) {
|
||||
// Store the options
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): GoogleGenAI {
|
||||
if (!this.client) {
|
||||
const options = this.options as GeminiHandlerOptions
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
|
||||
if (options.isVertex) {
|
||||
// Initialize with Vertex AI configuration
|
||||
const project = this.options.vertexProjectId ?? "not-provided"
|
||||
const location = this.options.vertexRegion ?? "not-provided"
|
||||
|
||||
try {
|
||||
this.client = new GoogleGenAI({
|
||||
vertexai: true,
|
||||
project,
|
||||
location,
|
||||
httpOptions: {
|
||||
headers: externalHeaders,
|
||||
},
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Gemini Vertex AI client: ${error.message}`)
|
||||
}
|
||||
} else {
|
||||
// Initialize with standard API key
|
||||
if (!options.geminiApiKey) {
|
||||
throw new Error("API key is required for Google Gemini when not using Vertex AI")
|
||||
}
|
||||
|
||||
try {
|
||||
this.client = new GoogleGenAI({
|
||||
apiKey: options.geminiApiKey,
|
||||
httpOptions: {
|
||||
headers: externalHeaders,
|
||||
},
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Gemini client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a message using the Gemini API with implicit caching.
|
||||
*
|
||||
* Cost accounting:
|
||||
* - Immediate costs (returned in the usage object): Input tokens, output tokens, cache read costs
|
||||
*
|
||||
* @param systemPrompt The system prompt to use for the message
|
||||
* @param messages The conversation history to include in the message
|
||||
* @returns An async generator that yields chunks of the response with accurate immediate costs
|
||||
*/
|
||||
@withRetry({
|
||||
maxRetries: 4,
|
||||
baseDelay: 2000,
|
||||
maxDelay: 15000,
|
||||
})
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: GoogleTool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const { id: modelId, info } = this.getModel()
|
||||
const contents = messages.map(convertAnthropicMessageToGemini)
|
||||
// Gemini may emit multiple function calls under the same responseId and without functionCall.id.
|
||||
// Track a local sequence so each emitted tool call has a stable unique ID.
|
||||
const responseToolCallCount = new Map<string, number>()
|
||||
|
||||
// Configure thinking budget/level if supported
|
||||
const _thinkingBudget = this.options.thinkingBudgetTokens ?? 0
|
||||
const maxBudget = info.thinkingConfig?.maxBudget ?? 24576
|
||||
const thinkingBudget = Math.min(_thinkingBudget, maxBudget)
|
||||
// When ThinkingLevel is defined, thinking budget cannot be zero
|
||||
// and only level is used to control thinking behavior.
|
||||
// Only set thinkingLevel for models that support it
|
||||
let thinkingLevel: ThinkingLevel | undefined
|
||||
const rawReasoningEffort = (this.options.reasoningEffort || "").toLowerCase()
|
||||
const normalizedReasoningEffort = !rawReasoningEffort || rawReasoningEffort === "none" ? "low" : rawReasoningEffort
|
||||
if (info.thinkingConfig?.supportsThinkingLevel) {
|
||||
thinkingLevel = mapReasoningEffortToGeminiThinkingLevel(normalizedReasoningEffort)
|
||||
}
|
||||
|
||||
// Set up base generation config
|
||||
const maxOutputTokens = getGeminiMaxOutputTokens(modelId, info.maxTokens)
|
||||
const requestConfig: GenerateContentConfig = {
|
||||
// Add base URL if configured
|
||||
httpOptions: this.options.geminiBaseUrl ? { baseUrl: this.options.geminiBaseUrl } : undefined,
|
||||
systemInstruction: systemPrompt,
|
||||
// Set temperature (default to 0)
|
||||
// Gemini 3 recommends 1.0
|
||||
temperature: info.temperature ?? 1,
|
||||
...(maxOutputTokens !== undefined ? { maxOutputTokens } : {}),
|
||||
}
|
||||
|
||||
// Add thinking config only if the model supports it
|
||||
if (info.thinkingConfig) {
|
||||
requestConfig.thinkingConfig = {
|
||||
// Turn off thinking:
|
||||
// thinkingBudget: 0
|
||||
// Turn on dynamic thinking:
|
||||
// thinkingBudget: -1
|
||||
// Turn on fixed thinking budget:
|
||||
thinkingBudget: thinkingLevel ? undefined : thinkingBudget, // Use budget only if thinkingLevel is not set
|
||||
thinkingLevel,
|
||||
includeThoughts: thinkingBudget > 0 || !!thinkingLevel,
|
||||
}
|
||||
}
|
||||
|
||||
// Generate content using the configured parameters
|
||||
const sdkCallStartTime = Date.now()
|
||||
let responseId: string | undefined
|
||||
let sdkFirstChunkTime: number | undefined
|
||||
let ttftSdkMs: number | undefined
|
||||
let apiSuccess = false
|
||||
let apiError: string | undefined
|
||||
let promptTokens = 0
|
||||
let outputTokens = 0
|
||||
let cacheReadTokens = 0
|
||||
let thoughtsTokenCount = 0 // Initialize thought token counts
|
||||
let lastUsageMetadata: GenerateContentResponseUsageMetadata | undefined
|
||||
|
||||
const isNativeToolCallsEnabled = tools?.length
|
||||
if (isNativeToolCallsEnabled) {
|
||||
requestConfig.tools = [{ functionDeclarations: tools }]
|
||||
requestConfig.toolConfig = {
|
||||
// Force the model to call 'any' function.
|
||||
functionCallingConfig: {
|
||||
mode: FunctionCallingConfigMode.ANY,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await client.models.generateContentStream({
|
||||
model: modelId,
|
||||
contents: contents,
|
||||
config: {
|
||||
...requestConfig,
|
||||
},
|
||||
})
|
||||
|
||||
let isFirstSdkChunk = true
|
||||
for await (const chunk of result) {
|
||||
const responseKey = chunk.responseId || "gemini-response"
|
||||
if (isFirstSdkChunk) {
|
||||
sdkFirstChunkTime = Date.now()
|
||||
ttftSdkMs = sdkFirstChunkTime - sdkCallStartTime
|
||||
isFirstSdkChunk = false
|
||||
}
|
||||
|
||||
// Handle thinking content from Gemini's response
|
||||
const parts = chunk?.candidates?.[0]?.content?.parts || []
|
||||
for (const part of parts) {
|
||||
if (part.thought && part.text) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
id: chunk.responseId,
|
||||
reasoning: part.text || "",
|
||||
signature: part.thoughtSignature,
|
||||
}
|
||||
} else if (part.text) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: part.text,
|
||||
id: chunk.responseId,
|
||||
signature: part.thoughtSignature,
|
||||
}
|
||||
}
|
||||
if (part.functionCall) {
|
||||
const functionCall = part.functionCall
|
||||
const args = Object.entries(functionCall.args || {}).filter(([_key, val]) => !!val)
|
||||
if (functionCall.args && args.length > 0) {
|
||||
const existingId = functionCall.id?.trim()
|
||||
const toolCallId =
|
||||
existingId ??
|
||||
(() => {
|
||||
const sequenceNumber = responseToolCallCount.get(responseKey) ?? 0
|
||||
responseToolCallCount.set(responseKey, sequenceNumber + 1)
|
||||
return `${responseKey}-tool-${sequenceNumber}`
|
||||
})()
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
id: chunk.responseId,
|
||||
tool_call: {
|
||||
call_id: toolCallId,
|
||||
function: {
|
||||
id: toolCallId,
|
||||
name: functionCall.name,
|
||||
arguments: JSON.stringify(functionCall.args),
|
||||
},
|
||||
},
|
||||
signature: part.thoughtSignature,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usageMetadata) {
|
||||
responseId = chunk.responseId
|
||||
lastUsageMetadata = chunk.usageMetadata
|
||||
promptTokens = lastUsageMetadata.promptTokenCount ?? promptTokens
|
||||
outputTokens = lastUsageMetadata.candidatesTokenCount ?? outputTokens
|
||||
thoughtsTokenCount = lastUsageMetadata.thoughtsTokenCount ?? thoughtsTokenCount
|
||||
cacheReadTokens = lastUsageMetadata.cachedContentTokenCount ?? cacheReadTokens
|
||||
}
|
||||
}
|
||||
apiSuccess = true
|
||||
|
||||
if (lastUsageMetadata) {
|
||||
const totalCost = this.calculateCost({
|
||||
info,
|
||||
inputTokens: promptTokens,
|
||||
outputTokens,
|
||||
thoughtsTokenCount,
|
||||
cacheReadTokens,
|
||||
})
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: promptTokens - cacheReadTokens,
|
||||
outputTokens,
|
||||
thoughtsTokenCount,
|
||||
cacheReadTokens,
|
||||
cacheWriteTokens: 0,
|
||||
totalCost,
|
||||
id: responseId,
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
apiSuccess = false
|
||||
// Let the error propagate to be handled by withRetry or Task.ts
|
||||
// Telemetry will be sent in the finally block.
|
||||
if (error instanceof Error) {
|
||||
apiError = error.message
|
||||
|
||||
if (error instanceof ApiError) {
|
||||
if (error.status === 429) {
|
||||
// The API includes more details in the message
|
||||
// https://github.com/googleapis/js-genai/blob/v1.11.0/src/_api_client.ts#L758
|
||||
const response = this.attemptParse(error.message)
|
||||
|
||||
if (response && response.error) {
|
||||
const responseBody = this.attemptParse(response.error.message)
|
||||
|
||||
if (responseBody.error) {
|
||||
const detail = responseBody.error.details?.find(
|
||||
(d: any) => d["@type"] === "type.googleapis.com/google.rpc.RetryInfo",
|
||||
)
|
||||
|
||||
const detailedError = new RetriableError(
|
||||
apiError,
|
||||
this.parseRetryDelay(detail?.retryDelay) || undefined,
|
||||
{
|
||||
cause: error,
|
||||
},
|
||||
)
|
||||
throw detailedError
|
||||
}
|
||||
}
|
||||
|
||||
throw new RetriableError(apiError, undefined, { cause: error })
|
||||
}
|
||||
|
||||
// Fallback in case Gemini throws a rate limit error without a 429 status code
|
||||
// https://github.com/cline/cline/pull/5205#discussion_r2311761559
|
||||
const isRateLimit = rateLimitPatterns.some((pattern) => pattern.test(error.message))
|
||||
if (isRateLimit) {
|
||||
throw new RetriableError(apiError, undefined, { cause: error })
|
||||
}
|
||||
}
|
||||
} else {
|
||||
apiError = String(error)
|
||||
}
|
||||
|
||||
throw error
|
||||
} finally {
|
||||
const sdkCallEndTime = Date.now()
|
||||
const totalDurationSdkMs = sdkCallEndTime - sdkCallStartTime
|
||||
const cacheHit = cacheReadTokens > 0
|
||||
const cacheHitPercentage = promptTokens > 0 ? (cacheReadTokens / promptTokens) * 100 : undefined
|
||||
const throughputTokensPerSecSdk =
|
||||
totalDurationSdkMs > 0 && outputTokens > 0 ? outputTokens / (totalDurationSdkMs / 1000) : undefined
|
||||
|
||||
if (this.options.ulid) {
|
||||
telemetryService.captureGeminiApiPerformance(this.options.ulid, modelId, {
|
||||
ttftSec: ttftSdkMs !== undefined ? ttftSdkMs / 1000 : undefined,
|
||||
totalDurationSec: totalDurationSdkMs / 1000,
|
||||
promptTokens,
|
||||
outputTokens,
|
||||
cacheReadTokens,
|
||||
cacheHit,
|
||||
cacheHitPercentage,
|
||||
apiSuccess,
|
||||
apiError,
|
||||
throughputTokensPerSec: throughputTokensPerSecSdk,
|
||||
})
|
||||
} else {
|
||||
Logger.warn("GeminiHandler: ulid not available for telemetry in createMessage.")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate the immediate dollar cost of the API call based on token usage and model pricing.
|
||||
*
|
||||
* This method accounts for the immediate costs of the API call:
|
||||
* - Input token costs (for uncached tokens)
|
||||
* - Output token costs
|
||||
* - Cache read costs
|
||||
* - Gemini implicit caching has no write costs
|
||||
*
|
||||
*/
|
||||
public calculateCost({
|
||||
info,
|
||||
inputTokens,
|
||||
outputTokens,
|
||||
thoughtsTokenCount = 0,
|
||||
cacheReadTokens = 0,
|
||||
}: {
|
||||
info: ModelInfo
|
||||
inputTokens: number
|
||||
outputTokens: number
|
||||
thoughtsTokenCount: number
|
||||
cacheReadTokens?: number
|
||||
}) {
|
||||
// Exit early if any required pricing information is missing
|
||||
if (!info.inputPrice || !info.outputPrice) {
|
||||
return undefined
|
||||
}
|
||||
|
||||
let inputPrice = info.inputPrice
|
||||
let outputPrice = info.outputPrice
|
||||
// Right now, we only show the immediate costs of caching and not the ongoing costs of storing the cache
|
||||
let cacheReadsPrice = info.cacheReadsPrice ?? 0
|
||||
|
||||
// If there's tiered pricing then adjust prices based on the input tokens used
|
||||
if (info.tiers) {
|
||||
const tier = info.tiers.find((tier) => inputTokens <= tier.contextWindow)
|
||||
if (tier) {
|
||||
inputPrice = tier.inputPrice ?? inputPrice
|
||||
outputPrice = tier.outputPrice ?? outputPrice
|
||||
cacheReadsPrice = tier.cacheReadsPrice ?? cacheReadsPrice
|
||||
}
|
||||
}
|
||||
|
||||
// Subtract the cached input tokens from the total input tokens
|
||||
const uncachedInputTokens = inputTokens - (cacheReadTokens ?? 0)
|
||||
|
||||
// Calculate immediate costs only
|
||||
|
||||
// 1. Input token costs (for uncached tokens)
|
||||
const inputTokensCost = inputPrice * (uncachedInputTokens / 1_000_000)
|
||||
|
||||
// 2. Output token costs
|
||||
const responseTokensCost = outputPrice * ((outputTokens + thoughtsTokenCount) / 1_000_000)
|
||||
|
||||
// 3. Cache read costs (immediate)
|
||||
const cacheReadCost = (cacheReadTokens ?? 0) > 0 ? cacheReadsPrice * ((cacheReadTokens ?? 0) / 1_000_000) : 0
|
||||
|
||||
// Calculate total immediate cost (excluding cache write/storage costs)
|
||||
const totalCost = inputTokensCost + responseTokensCost + cacheReadCost
|
||||
|
||||
// Create the trace object for debugging
|
||||
const trace: Record<string, { price: number; tokens: number; cost: number }> = {
|
||||
input: { price: inputPrice, tokens: uncachedInputTokens, cost: inputTokensCost },
|
||||
output: { price: outputPrice, tokens: outputTokens, cost: responseTokensCost },
|
||||
}
|
||||
|
||||
// Only include cache read costs in the trace (cache write costs are tracked separately)
|
||||
if ((cacheReadTokens ?? 0) > 0) {
|
||||
trace.cacheRead = { price: cacheReadsPrice, tokens: cacheReadTokens ?? 0, cost: cacheReadCost }
|
||||
}
|
||||
|
||||
// Logger.log(`[GeminiHandler] calculateCost -> ${totalCost}`, trace)
|
||||
return totalCost
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the model ID and info for the current configuration
|
||||
*/
|
||||
getModel(): { id: GeminiModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in geminiModels) {
|
||||
const id = modelId as GeminiModelId
|
||||
return { id, info: geminiModels[id] }
|
||||
}
|
||||
return {
|
||||
id: geminiDefaultModelId,
|
||||
info: geminiModels[geminiDefaultModelId],
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Count tokens in content using the Gemini API
|
||||
*/
|
||||
async countTokens(content: Array<any>): Promise<number> {
|
||||
try {
|
||||
const client = this.ensureClient()
|
||||
const { id: model } = this.getModel()
|
||||
|
||||
// Convert content to Gemini format
|
||||
const geminiContent = content.map((block) => {
|
||||
if (typeof block === "string") {
|
||||
return { text: block }
|
||||
}
|
||||
return { text: JSON.stringify(block) }
|
||||
})
|
||||
|
||||
// Use Gemini's token counting API
|
||||
const response = await client.models.countTokens({
|
||||
model,
|
||||
contents: [{ parts: geminiContent }],
|
||||
})
|
||||
|
||||
if (response.totalTokens === undefined) {
|
||||
Logger.warn("Gemini token counting returned undefined, using fallback")
|
||||
return this.estimateTokens(content)
|
||||
}
|
||||
|
||||
return response.totalTokens
|
||||
} catch (error) {
|
||||
Logger.warn("Gemini token counting failed, using fallback", error)
|
||||
return this.estimateTokens(content)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fallback token estimation method
|
||||
*/
|
||||
private estimateTokens(content: Array<any>): number {
|
||||
// Simple estimation: ~4 characters per token
|
||||
const totalChars = content.reduce((total, block) => {
|
||||
if (typeof block === "string") {
|
||||
return total + block.length
|
||||
}
|
||||
if (block && typeof block === "object") {
|
||||
// Safely stringify the object
|
||||
try {
|
||||
const jsonStr = JSON.stringify(block)
|
||||
return total + jsonStr.length
|
||||
} catch (e) {
|
||||
Logger.warn("Failed to stringify block for token estimation", e)
|
||||
return total
|
||||
}
|
||||
}
|
||||
return total
|
||||
}, 0)
|
||||
|
||||
return Math.ceil(totalChars / 4)
|
||||
}
|
||||
|
||||
private parseRetryDelay(retryAfter?: string): number {
|
||||
if (!retryAfter) {
|
||||
return 0
|
||||
}
|
||||
|
||||
const unit = retryAfter.at(-1)
|
||||
const value = Number.parseInt(retryAfter, 10)
|
||||
|
||||
if (Number.isNaN(value)) {
|
||||
return 0
|
||||
}
|
||||
|
||||
if (unit === "s") {
|
||||
return value
|
||||
}
|
||||
if (unit === "m") {
|
||||
return value * 60 // Convert minutes to seconds
|
||||
}
|
||||
if (unit === "h") {
|
||||
return value * 60 * 60 // Convert hours to seconds
|
||||
}
|
||||
|
||||
return value
|
||||
}
|
||||
|
||||
private attemptParse(str: string) {
|
||||
try {
|
||||
return JSON.parse(str)
|
||||
} catch (_) {
|
||||
return null
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,320 +0,0 @@
|
||||
import { GroqModelId, groqDefaultModelId, groqModels, ModelInfo } from "@shared/api"
|
||||
import { calculateApiCostOpenAI } from "@utils/cost"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface GroqHandlerOptions extends CommonApiHandlerOptions {
|
||||
groqApiKey?: string
|
||||
groqModelId?: string
|
||||
groqModelInfo?: ModelInfo
|
||||
apiModelId?: string // For backward compatibility
|
||||
}
|
||||
|
||||
// Enhanced usage interface to support Groq's cached token fields
|
||||
interface GroqUsage extends OpenAI.CompletionUsage {
|
||||
prompt_tokens_details?: {
|
||||
cached_tokens?: number
|
||||
}
|
||||
}
|
||||
|
||||
// Model family definitions for enhanced behavior
|
||||
interface GroqModelFamily {
|
||||
name: string
|
||||
supportedFeatures: {
|
||||
streaming: boolean
|
||||
temperature: boolean
|
||||
vision: boolean
|
||||
tools: boolean
|
||||
}
|
||||
maxTokensOverride?: number
|
||||
specialParams?: Record<string, any>
|
||||
}
|
||||
|
||||
const MODEL_FAMILIES: Record<string, GroqModelFamily> = {
|
||||
// Moonshort 4 Family - Latest generation with vision support
|
||||
"kimi-k2": {
|
||||
name: "kimi-k2",
|
||||
supportedFeatures: { streaming: true, temperature: true, vision: true, tools: true },
|
||||
maxTokensOverride: 8192,
|
||||
},
|
||||
// Llama 4 Family - Latest generation with vision support
|
||||
llama4: {
|
||||
name: "Llama 4",
|
||||
supportedFeatures: { streaming: true, temperature: true, vision: true, tools: true },
|
||||
maxTokensOverride: 8192,
|
||||
},
|
||||
// Llama 3.3 Family - Balanced performance
|
||||
"llama3.3": {
|
||||
name: "Llama 3.3",
|
||||
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
|
||||
maxTokensOverride: 32768,
|
||||
},
|
||||
// Llama 3.1 Family - Fast inference
|
||||
"llama3.1": {
|
||||
name: "Llama 3.1",
|
||||
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
|
||||
maxTokensOverride: 131072,
|
||||
},
|
||||
// DeepSeek Family - Reasoning-optimized
|
||||
deepseek: {
|
||||
name: "DeepSeek",
|
||||
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
|
||||
maxTokensOverride: 8192,
|
||||
specialParams: {
|
||||
top_p: 0.95,
|
||||
reasoning_format: "parsed",
|
||||
},
|
||||
},
|
||||
// Qwen Family - Enhanced for Q&A
|
||||
qwen: {
|
||||
name: "Qwen",
|
||||
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
|
||||
maxTokensOverride: 32768,
|
||||
},
|
||||
// Compound Models - Hybrid architectures
|
||||
compound: {
|
||||
name: "Compound",
|
||||
supportedFeatures: { streaming: true, temperature: true, vision: false, tools: true },
|
||||
maxTokensOverride: 8192,
|
||||
},
|
||||
}
|
||||
|
||||
export class GroqHandler implements ApiHandler {
|
||||
private options: GroqHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: GroqHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.groqApiKey) {
|
||||
throw new Error("Groq API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = new OpenAI({
|
||||
baseURL: "https://api.groq.com/openai/v1",
|
||||
apiKey: this.options.groqApiKey,
|
||||
defaultHeaders: buildExternalBasicHeaders(),
|
||||
fetch, // Use configured fetch with proxy support
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Groq client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
private async *yieldUsage(info: ModelInfo, usage: GroqUsage | undefined): ApiStream {
|
||||
const inputTokens = usage?.prompt_tokens || 0
|
||||
const outputTokens = usage?.completion_tokens || 0
|
||||
|
||||
const cacheReadTokens = usage?.prompt_tokens_details?.cached_tokens || 0
|
||||
|
||||
// Groq does not track cache writes
|
||||
const cacheWriteTokens = 0
|
||||
|
||||
// Calculate cost using OpenAI-compatible cost calculation
|
||||
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
|
||||
|
||||
// Calculate non-cached input tokens for proper reporting
|
||||
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
|
||||
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: nonCachedInputTokens,
|
||||
outputTokens,
|
||||
cacheWriteTokens,
|
||||
cacheReadTokens,
|
||||
totalCost,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Detects the model family based on the model ID
|
||||
*/
|
||||
private detectModelFamily(modelId: string): GroqModelFamily {
|
||||
if (modelId.includes("kimi-k2")) {
|
||||
return MODEL_FAMILIES["kimi-k2"]
|
||||
}
|
||||
// Llama 4 variants
|
||||
if (modelId.includes("llama-4") || modelId.includes("llama/llama-4")) {
|
||||
return MODEL_FAMILIES.llama4
|
||||
}
|
||||
// Llama 3.3 variants
|
||||
if (modelId.includes("llama-3.3")) {
|
||||
return MODEL_FAMILIES["llama3.3"]
|
||||
}
|
||||
// Llama 3.1 variants
|
||||
if (modelId.includes("llama-3.1")) {
|
||||
return MODEL_FAMILIES["llama3.1"]
|
||||
}
|
||||
// DeepSeek variants
|
||||
if (modelId.includes("deepseek")) {
|
||||
return MODEL_FAMILIES.deepseek
|
||||
}
|
||||
// Qwen variants
|
||||
if (modelId.includes("qwen")) {
|
||||
return MODEL_FAMILIES.qwen
|
||||
}
|
||||
// Compound variants
|
||||
if (modelId.includes("compound")) {
|
||||
return MODEL_FAMILIES.compound
|
||||
}
|
||||
|
||||
// Default fallback to Llama 3.3 behavior
|
||||
return MODEL_FAMILIES["kimi-k2"]
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the optimal max_tokens based on model family and capabilities
|
||||
*/
|
||||
private getOptimalMaxTokens(model: { id: string; info: ModelInfo }, modelFamily: GroqModelFamily): number {
|
||||
// Use model-specific max tokens if available
|
||||
if (model.info.maxTokens && model.info.maxTokens > 0) {
|
||||
return model.info.maxTokens
|
||||
}
|
||||
|
||||
// Use family override if available
|
||||
if (modelFamily.maxTokensOverride) {
|
||||
return modelFamily.maxTokensOverride
|
||||
}
|
||||
|
||||
// Default fallback
|
||||
return 8192
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
const modelFamily = this.detectModelFamily(model.id)
|
||||
|
||||
// Optimize parameters based on model family
|
||||
const temperature = 0
|
||||
const maxTokens = this.getOptimalMaxTokens(model, modelFamily)
|
||||
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
// Build request parameters with model-specific optimizations
|
||||
const requestParams: OpenAI.Chat.ChatCompletionCreateParamsStreaming & {
|
||||
reasoning_format?: "parsed" | "raw" | "hidden"
|
||||
top_p?: number
|
||||
} = {
|
||||
model: model.id,
|
||||
max_tokens: maxTokens,
|
||||
messages: openAiMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
temperature,
|
||||
...getOpenAIToolParams(tools),
|
||||
}
|
||||
|
||||
// Add any special parameters for specific model families
|
||||
if (modelFamily.specialParams) {
|
||||
Object.assign(requestParams, modelFamily.specialParams)
|
||||
}
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
const stream = await client.chat.completions.create(requestParams)
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
|
||||
// Handle reasoning field if present (for reasoning models with parsed output)
|
||||
if ((delta as any)?.reasoning) {
|
||||
const reasoningContent = (delta as any).reasoning as string
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: reasoningContent,
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
// Handle content field - trust the parsed output from Groq
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
// Handle usage information
|
||||
if (chunk.usage) {
|
||||
yield* this.yieldUsage(model.info, chunk.usage)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks if the current model supports vision/images
|
||||
*/
|
||||
supportsImages(): boolean {
|
||||
const model = this.getModel()
|
||||
return model.info.supportsImages === true
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks if the current model supports tools
|
||||
*/
|
||||
supportsTools(): boolean {
|
||||
const model = this.getModel()
|
||||
const modelFamily = this.detectModelFamily(model.id)
|
||||
return modelFamily.supportedFeatures.tools
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets model information with enhanced family detection
|
||||
*/
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
// First priority: groqModelId and groqModelInfo (like Requesty does)
|
||||
const groqModelId = this.options.groqModelId
|
||||
const groqModelInfo = this.options.groqModelInfo
|
||||
if (groqModelId && groqModelInfo) {
|
||||
return { id: groqModelId, info: groqModelInfo }
|
||||
}
|
||||
|
||||
// Second priority: groqModelId with static model info
|
||||
if (groqModelId && groqModelId in groqModels) {
|
||||
const id = groqModelId as GroqModelId
|
||||
return { id, info: groqModels[id] }
|
||||
}
|
||||
|
||||
// Third priority: apiModelId (for backward compatibility)
|
||||
const apiModelId = this.options.apiModelId
|
||||
if (apiModelId && apiModelId in groqModels) {
|
||||
const id = apiModelId as GroqModelId
|
||||
return { id, info: groqModels[id] }
|
||||
}
|
||||
|
||||
// Default fallback
|
||||
return {
|
||||
id: groqDefaultModelId,
|
||||
info: groqModels[groqDefaultModelId],
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets model family information for debugging/introspection
|
||||
*/
|
||||
getModelFamily(): GroqModelFamily {
|
||||
const model = this.getModel()
|
||||
return this.detectModelFamily(model.id)
|
||||
}
|
||||
}
|
||||
@@ -1,104 +0,0 @@
|
||||
import { hicapModelInfoSaneDefaults, ModelInfo } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionReasoningEffort } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface OpenAiHandlerOptions extends CommonApiHandlerOptions {
|
||||
hicapApiKey?: string
|
||||
hicapModelId?: string
|
||||
}
|
||||
|
||||
export class HicapHandler implements ApiHandler {
|
||||
private options: OpenAiHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: OpenAiHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.hicapApiKey) {
|
||||
throw new Error("Hicap API key is required")
|
||||
}
|
||||
if (!this.options.hicapModelId) {
|
||||
throw new Error("Model ID is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://api.hicap.ai/v2/openai",
|
||||
apiKey: this.options.hicapApiKey,
|
||||
defaultHeaders: {
|
||||
"api-key": this.options.hicapApiKey,
|
||||
},
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating OpenAI client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const modelId = this.options.hicapModelId ?? ""
|
||||
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
const temperature: number = 1
|
||||
let reasoningEffort: ChatCompletionReasoningEffort | undefined
|
||||
let maxTokens: number | undefined
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: modelId,
|
||||
messages: openAiMessages,
|
||||
temperature,
|
||||
max_tokens: maxTokens,
|
||||
reasoning_effort: reasoningEffort,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
})
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
|
||||
// @ts-expect-error-next-line
|
||||
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
return {
|
||||
id: this.options.hicapModelId ?? "",
|
||||
info: hicapModelInfoSaneDefaults,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,142 +0,0 @@
|
||||
import { HuaweiCloudMaasModelId, huaweiCloudMaasDefaultModelId, huaweiCloudMaasModels, ModelInfo } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from ".."
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface HuaweiCloudMaaSHandlerOptions extends CommonApiHandlerOptions {
|
||||
huaweiCloudMaasApiKey?: string
|
||||
huaweiCloudMaasModelId?: string
|
||||
huaweiCloudMaasModelInfo?: ModelInfo
|
||||
}
|
||||
|
||||
export class HuaweiCloudMaaSHandler implements ApiHandler {
|
||||
private options: HuaweiCloudMaaSHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
constructor(options: HuaweiCloudMaaSHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.huaweiCloudMaasApiKey) {
|
||||
throw new Error("Huawei Cloud MaaS API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://api.modelarts-maas.com/v1/",
|
||||
apiKey: this.options.huaweiCloudMaasApiKey,
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Huawei Cloud MaaS client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
getModel(): { id: HuaweiCloudMaasModelId; info: ModelInfo } {
|
||||
// First priority: huaweiCloudMaasModelId and huaweiCloudMaasModelInfo (like Groq does)
|
||||
const huaweiCloudMaasModelId = this.options.huaweiCloudMaasModelId
|
||||
const huaweiCloudMaasModelInfo = this.options.huaweiCloudMaasModelInfo
|
||||
if (huaweiCloudMaasModelId && huaweiCloudMaasModelInfo) {
|
||||
return { id: huaweiCloudMaasModelId as HuaweiCloudMaasModelId, info: huaweiCloudMaasModelInfo }
|
||||
}
|
||||
|
||||
// Second priority: huaweiCloudMaasModelId with static model info
|
||||
if (huaweiCloudMaasModelId && huaweiCloudMaasModelId in huaweiCloudMaasModels) {
|
||||
const id = huaweiCloudMaasModelId as HuaweiCloudMaasModelId
|
||||
return { id, info: huaweiCloudMaasModels[id] }
|
||||
}
|
||||
|
||||
// Default fallback
|
||||
return {
|
||||
id: huaweiCloudMaasDefaultModelId,
|
||||
info: huaweiCloudMaasModels[huaweiCloudMaasDefaultModelId],
|
||||
}
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
max_completion_tokens: model.info.maxTokens,
|
||||
messages: openAiMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
temperature: 0,
|
||||
...getOpenAIToolParams(tools),
|
||||
})
|
||||
|
||||
let reasoning: string | null = null
|
||||
let didOutputUsage: boolean = false
|
||||
let finalUsage: any = null
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
|
||||
// Handle reasoning content detection
|
||||
if (delta?.content) {
|
||||
if (reasoning || delta.content.includes("<think>")) {
|
||||
reasoning = (reasoning || "") + delta.content
|
||||
} else if (!reasoning) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
// Handle reasoning output
|
||||
if (reasoning || (delta && "reasoning_content" in delta && delta.reasoning_content)) {
|
||||
const reasoningContent = delta?.content || ((delta as any)?.reasoning_content as string | undefined) || ""
|
||||
if (reasoningContent.trim()) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: reasoningContent,
|
||||
}
|
||||
}
|
||||
|
||||
// Check if reasoning is complete
|
||||
if (reasoning?.includes("</think>")) {
|
||||
reasoning = null
|
||||
}
|
||||
}
|
||||
|
||||
// Store usage information for later output
|
||||
if (chunk.usage) {
|
||||
finalUsage = chunk.usage
|
||||
}
|
||||
|
||||
// Output usage when stream is finished
|
||||
if (!didOutputUsage && chunk.choices?.[0]?.finish_reason) {
|
||||
if (finalUsage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: finalUsage.prompt_tokens || 0,
|
||||
outputTokens: finalUsage.completion_tokens || 0,
|
||||
cacheWriteTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
}
|
||||
}
|
||||
didOutputUsage = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,148 +0,0 @@
|
||||
import { HuggingFaceModelId, huggingFaceDefaultModelId, huggingFaceModels, ModelInfo } from "@shared/api"
|
||||
import { calculateApiCostOpenAI } from "@utils/cost"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface HuggingFaceHandlerOptions extends CommonApiHandlerOptions {
|
||||
huggingFaceApiKey?: string
|
||||
huggingFaceModelId?: string
|
||||
huggingFaceModelInfo?: ModelInfo
|
||||
}
|
||||
|
||||
export class HuggingFaceHandler implements ApiHandler {
|
||||
private options: HuggingFaceHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
private cachedModel: { id: HuggingFaceModelId; info: ModelInfo } | undefined
|
||||
|
||||
constructor(options: HuggingFaceHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.huggingFaceApiKey) {
|
||||
throw new Error("Hugging Face API key is required")
|
||||
}
|
||||
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://router.huggingface.co/v1",
|
||||
apiKey: this.options.huggingFaceApiKey,
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating Hugging Face client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
|
||||
if (!usage) {
|
||||
return
|
||||
}
|
||||
|
||||
const inputTokens = usage.prompt_tokens || 0
|
||||
const outputTokens = usage.completion_tokens || 0
|
||||
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens)
|
||||
|
||||
const usageData = {
|
||||
type: "usage" as const,
|
||||
inputTokens: inputTokens,
|
||||
outputTokens: outputTokens,
|
||||
cacheWriteTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
totalCost: totalCost,
|
||||
}
|
||||
|
||||
yield usageData
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
try {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
const requestParams = {
|
||||
model: model.id,
|
||||
max_tokens: model.info.maxTokens,
|
||||
messages: openAiMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
temperature: 0,
|
||||
...getOpenAIToolParams(tools),
|
||||
}
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
const stream = (await client.chat.completions.create(requestParams)) as any
|
||||
|
||||
let _chunkCount = 0
|
||||
let _totalContent = ""
|
||||
|
||||
for await (const chunk of stream) {
|
||||
_chunkCount++
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
_totalContent += delta.content
|
||||
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield* this.yieldUsage(model.info, chunk.usage)
|
||||
}
|
||||
}
|
||||
} catch (error: any) {
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: HuggingFaceModelId; info: ModelInfo } {
|
||||
// Return cached model if available
|
||||
if (this.cachedModel) {
|
||||
return this.cachedModel
|
||||
}
|
||||
|
||||
const modelId = this.options.huggingFaceModelId
|
||||
|
||||
// List all available models for debugging
|
||||
const _availableModels = Object.keys(huggingFaceModels)
|
||||
let result: { id: HuggingFaceModelId; info: ModelInfo }
|
||||
|
||||
if (modelId && modelId in huggingFaceModels) {
|
||||
const id = modelId as HuggingFaceModelId
|
||||
const modelInfo = huggingFaceModels[id]
|
||||
result = { id, info: modelInfo }
|
||||
} else {
|
||||
const defaultInfo = huggingFaceModels[huggingFaceDefaultModelId]
|
||||
result = {
|
||||
id: huggingFaceDefaultModelId,
|
||||
info: defaultInfo,
|
||||
}
|
||||
}
|
||||
|
||||
// Cache the result for future calls
|
||||
this.cachedModel = result
|
||||
|
||||
return result
|
||||
}
|
||||
}
|
||||
@@ -1,403 +0,0 @@
|
||||
import { Anthropic } from "@anthropic-ai/sdk"
|
||||
import { LiteLLMModelInfo, liteLlmDefaultModelId, liteLlmModelInfoSaneDefaults } from "@shared/api"
|
||||
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
|
||||
import OpenAI from "openai"
|
||||
import { StateManager } from "@/core/storage/StateManager"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient, fetch } from "@/shared/net"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { isAnthropicModelId } from "@/utils/model-utils"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from ".."
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface LiteLlmHandlerOptions extends CommonApiHandlerOptions {
|
||||
liteLlmApiKey?: string
|
||||
liteLlmBaseUrl?: string
|
||||
liteLlmModelId?: string
|
||||
liteLlmModelInfo?: LiteLLMModelInfo
|
||||
reasoningEffort?: string
|
||||
thinkingBudgetTokens?: number
|
||||
liteLlmUsePromptCache?: boolean
|
||||
ulid?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Extended chat completion parameters that include LiteLLM-specific options
|
||||
* not present in the standard OpenAI SDK types
|
||||
*/
|
||||
interface LiteLlmChatCompletionCreateParams extends OpenAI.Chat.ChatCompletionCreateParamsStreaming {
|
||||
drop_params?: boolean
|
||||
}
|
||||
|
||||
export interface LiteLlmModelInfoResponse {
|
||||
data: Array<{
|
||||
model_name: string
|
||||
litellm_params: {
|
||||
model: string
|
||||
[key: string]: any
|
||||
}
|
||||
model_info: {
|
||||
input_cost_per_token: number
|
||||
output_cost_per_token: number
|
||||
cache_creation_input_token_cost?: number
|
||||
cache_read_input_token_cost?: number
|
||||
supports_prompt_caching?: boolean
|
||||
[key: string]: any
|
||||
}
|
||||
}>
|
||||
}
|
||||
|
||||
/**
|
||||
* Exported utility function to fetch LiteLLM model info
|
||||
* @param baseUrl The base URL for the LiteLLM API
|
||||
* @param apiKey The API key for authentication
|
||||
* @returns The model info response or undefined if fetch fails
|
||||
*/
|
||||
export async function fetchLiteLlmModelsInfo(baseUrl: string, apiKey: string): Promise<LiteLlmModelInfoResponse | undefined> {
|
||||
// Handle base URLs that already include /v1 to avoid double /v1/v1/
|
||||
const normalizedBaseUrl = baseUrl.endsWith("/v1") ? baseUrl : `${baseUrl}/v1`
|
||||
const url = `${normalizedBaseUrl}/model/info`
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
method: "GET",
|
||||
headers: {
|
||||
accept: "application/json",
|
||||
"x-litellm-api-key": apiKey,
|
||||
...buildExternalBasicHeaders(),
|
||||
},
|
||||
})
|
||||
|
||||
if (response.ok) {
|
||||
const data: LiteLlmModelInfoResponse = await response.json()
|
||||
return data
|
||||
}
|
||||
Logger.error("Failed to fetch LiteLLM model info:", response.statusText)
|
||||
// Try with Authorization header instead
|
||||
const retryResponse = await fetch(url, {
|
||||
method: "GET",
|
||||
headers: {
|
||||
accept: "application/json",
|
||||
Authorization: `Bearer ${apiKey}`,
|
||||
...buildExternalBasicHeaders(),
|
||||
},
|
||||
})
|
||||
|
||||
if (retryResponse.ok) {
|
||||
const data: LiteLlmModelInfoResponse = await retryResponse.json()
|
||||
return data
|
||||
}
|
||||
Logger.error("Failed to fetch LiteLLM model info with Authorization header:", retryResponse.statusText)
|
||||
throw new Error(`Failed to fetch LiteLLM model info: ${retryResponse.statusText}`)
|
||||
} catch (error) {
|
||||
Logger.error("Error fetching LiteLLM model info:", error)
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
export class LiteLlmHandler implements ApiHandler {
|
||||
private options: LiteLlmHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
private modelInfoCache: LiteLlmModelInfoResponse | undefined
|
||||
private modelInfoCacheTimestamp = 0
|
||||
private readonly modelInfoCacheTTL = 5 * 60 * 1000 // 5 minutes
|
||||
|
||||
constructor(options: LiteLlmHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.liteLlmApiKey) {
|
||||
throw new Error("LiteLLM API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: this.options.liteLlmBaseUrl || "http://localhost:4000",
|
||||
apiKey: this.options.liteLlmApiKey || "noop",
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating LiteLLM client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
private async modelInfo(publicModelName: string): Promise<LiteLlmModelInfoResponse["data"][number] | undefined> {
|
||||
const modelInfo = await this.fetchModelsInfo()
|
||||
|
||||
if (!modelInfo?.data) {
|
||||
return undefined
|
||||
}
|
||||
|
||||
return modelInfo.data.find((model) => model.model_name === publicModelName)
|
||||
}
|
||||
|
||||
private async fetchModelsInfo(): Promise<LiteLlmModelInfoResponse | undefined> {
|
||||
// Check if cache is still valid
|
||||
const now = Date.now()
|
||||
if (this.modelInfoCache && now - this.modelInfoCacheTimestamp < this.modelInfoCacheTTL) {
|
||||
return this.modelInfoCache
|
||||
}
|
||||
|
||||
const client = this.ensureClient()
|
||||
const data = await fetchLiteLlmModelsInfo(client.baseURL, this.options.liteLlmApiKey || "")
|
||||
|
||||
if (data) {
|
||||
this.modelInfoCache = data
|
||||
this.modelInfoCacheTimestamp = now
|
||||
}
|
||||
|
||||
return data
|
||||
}
|
||||
|
||||
private async getModelCostInfo(publicModelName: string): Promise<{
|
||||
inputCostPerToken: number
|
||||
outputCostPerToken: number
|
||||
cacheCreationCostPerToken?: number
|
||||
cacheReadCostPerToken?: number
|
||||
}> {
|
||||
try {
|
||||
const matchingModel = await this.modelInfo(publicModelName)
|
||||
|
||||
if (matchingModel) {
|
||||
return {
|
||||
inputCostPerToken: matchingModel.model_info.input_cost_per_token || 0,
|
||||
outputCostPerToken: matchingModel.model_info.output_cost_per_token || 0,
|
||||
cacheCreationCostPerToken: matchingModel.model_info.cache_creation_input_token_cost,
|
||||
cacheReadCostPerToken: matchingModel.model_info.cache_read_input_token_cost,
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
Logger.warn("Error getting LiteLLM model cost info:", error)
|
||||
}
|
||||
|
||||
// Fallback to zero costs if we can't get the information
|
||||
return {
|
||||
inputCostPerToken: 0,
|
||||
outputCostPerToken: 0,
|
||||
}
|
||||
}
|
||||
|
||||
async calculateCost(
|
||||
prompt_tokens: number,
|
||||
completion_tokens: number,
|
||||
cache_creation_tokens?: number,
|
||||
cache_read_tokens?: number,
|
||||
): Promise<number | undefined> {
|
||||
const publicModelId = this.options.liteLlmModelId || liteLlmDefaultModelId
|
||||
|
||||
try {
|
||||
const costInfo = await this.getModelCostInfo(publicModelId)
|
||||
|
||||
// Calculate costs for different token types
|
||||
const inputCost = Math.max(0, prompt_tokens - (cache_read_tokens || 0)) * costInfo.inputCostPerToken
|
||||
const outputCost = completion_tokens * costInfo.outputCostPerToken
|
||||
const cacheCreationCost = (cache_creation_tokens || 0) * (costInfo.cacheCreationCostPerToken || 0)
|
||||
const cacheReadCost = (cache_read_tokens || 0) * (costInfo.cacheReadCostPerToken || 0)
|
||||
|
||||
const totalCost = inputCost + outputCost + cacheCreationCost + cacheReadCost
|
||||
|
||||
return totalCost
|
||||
} catch (error) {
|
||||
Logger.error("Error calculating spend:", error)
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
|
||||
const formattedMessages = convertToOpenAiMessages(messages)
|
||||
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam | Anthropic.Messages.TextBlockParam = {
|
||||
role: "system",
|
||||
content: systemPrompt,
|
||||
}
|
||||
const modelId = this.options.liteLlmModelId || liteLlmDefaultModelId
|
||||
const isOminiModel = modelId.includes("o1-mini") || modelId.includes("o3-mini") || modelId.includes("o4-mini")
|
||||
const isCodexModel = modelId.toLowerCase().includes("codex")
|
||||
|
||||
// Configuration for extended thinking
|
||||
const budgetTokens = this.options.thinkingBudgetTokens || 0
|
||||
const reasoningOn = budgetTokens !== 0
|
||||
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(modelId)
|
||||
const adaptiveThinking = isAdaptiveThinkingModel
|
||||
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, budgetTokens)
|
||||
: undefined
|
||||
const thinkingConfig = isAdaptiveThinkingModel
|
||||
? adaptiveThinking?.enabled
|
||||
? ({ type: "adaptive" } as any)
|
||||
: undefined
|
||||
: reasoningOn
|
||||
? { type: "enabled", budget_tokens: budgetTokens }
|
||||
: undefined
|
||||
|
||||
let temperature: number | undefined = this.options.liteLlmModelInfo?.temperature ?? 1
|
||||
|
||||
if (isAdaptiveThinkingModel) {
|
||||
temperature = undefined
|
||||
} else if ((isOminiModel || isAnthropicModelId(modelId)) && reasoningOn) {
|
||||
temperature = undefined // OAI omni and Anthropic extended thinking mode doesn't support temperature
|
||||
}
|
||||
|
||||
const modelInfo = await this.modelInfo(modelId)
|
||||
// Automatically enable caching if the model supports it
|
||||
const cacheControl =
|
||||
(modelInfo?.model_info.supports_prompt_caching ?? false) ? { cache_control: { type: "ephemeral" } } : undefined
|
||||
|
||||
if (cacheControl) {
|
||||
// Add cache_control to system message if enabled
|
||||
// https://docs.litellm.ai/docs/providers/anthropic#caching---large-context-caching
|
||||
systemMessage.content = [
|
||||
{
|
||||
text: systemPrompt,
|
||||
type: "text",
|
||||
...cacheControl,
|
||||
},
|
||||
] as Anthropic.Messages.TextBlockParam[]
|
||||
}
|
||||
|
||||
// Find the last two user messages to apply caching
|
||||
const userMsgIndices = formattedMessages.reduce(
|
||||
(acc, msg, index) => (msg.role === "user" ? [...acc, index] : acc),
|
||||
[] as number[],
|
||||
)
|
||||
const lastUserMsgIndex = userMsgIndices[userMsgIndices.length - 1] ?? -1
|
||||
const secondLastUserMsgIndex = userMsgIndices[userMsgIndices.length - 2] ?? -1
|
||||
|
||||
// Apply cache_control to the last two user messages if enabled
|
||||
// https://docs.litellm.ai/docs/providers/anthropic#caching---large-context-caching
|
||||
const enhancedMessages: OpenAI.Chat.ChatCompletionMessageParam[] = formattedMessages.map(
|
||||
(message, index): OpenAI.Chat.ChatCompletionMessageParam => {
|
||||
if ((index === lastUserMsgIndex || index === secondLastUserMsgIndex) && cacheControl) {
|
||||
// Handle both string and array content types
|
||||
if (typeof message.content === "string") {
|
||||
return {
|
||||
...message,
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: message.content,
|
||||
...cacheControl,
|
||||
},
|
||||
] as any,
|
||||
}
|
||||
}
|
||||
if (Array.isArray(message.content)) {
|
||||
// Apply cache control to the last content item in the array
|
||||
return {
|
||||
...message,
|
||||
content: message.content.map((item, contentIndex) =>
|
||||
contentIndex === (message.content?.length || 0) - 1
|
||||
? {
|
||||
...item,
|
||||
...cacheControl,
|
||||
}
|
||||
: item,
|
||||
) as any,
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
...message,
|
||||
...cacheControl,
|
||||
}
|
||||
}
|
||||
return message
|
||||
},
|
||||
)
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: this.options.liteLlmModelId || liteLlmDefaultModelId,
|
||||
messages: [systemMessage, ...enhancedMessages],
|
||||
temperature,
|
||||
stream: true,
|
||||
drop_params: true,
|
||||
...(!isCodexModel && { stream_options: { include_usage: true } }), // Codex models are only on the responses api, which doesn't take the stream_options parameter. we will need to migrate to the responses api for this to work
|
||||
...(thinkingConfig && { thinking: thinkingConfig }), // Add thinking configuration when applicable
|
||||
...(isAdaptiveThinkingModel && adaptiveThinking?.effort
|
||||
? { output_config: { effort: adaptiveThinking.effort } }
|
||||
: {}),
|
||||
...(this.options.ulid && { litellm_session_id: `cline-${this.options.ulid}` }), // Add session ID for LiteLLM tracking
|
||||
} as LiteLlmChatCompletionCreateParams)
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
|
||||
// Handle normal text content
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
// Handle reasoning events
|
||||
// This is not in the standard types but may be in the response
|
||||
interface ThinkingDelta {
|
||||
reasoning_content?: string
|
||||
}
|
||||
|
||||
if ((delta as ThinkingDelta)?.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta as ThinkingDelta).reasoning_content || "",
|
||||
}
|
||||
}
|
||||
|
||||
// Handle token usage information
|
||||
if (chunk.usage) {
|
||||
// Extract cache-related information if available
|
||||
// Need to use type assertion since these properties are not in the standard OpenAI types
|
||||
const usage = chunk.usage as {
|
||||
prompt_tokens: number
|
||||
completion_tokens: number
|
||||
cache_creation_input_tokens?: number
|
||||
prompt_cache_miss_tokens?: number
|
||||
cache_read_input_tokens?: number
|
||||
prompt_cache_hit_tokens?: number
|
||||
}
|
||||
|
||||
const cacheWriteTokens = usage.cache_creation_input_tokens || usage.prompt_cache_miss_tokens || 0
|
||||
const cacheReadTokens = usage.cache_read_input_tokens || usage.prompt_cache_hit_tokens || 0
|
||||
|
||||
// Calculate cost using the actual token usage including cache tokens
|
||||
const totalCost =
|
||||
(await this.calculateCost(
|
||||
usage.prompt_tokens || 0,
|
||||
usage.completion_tokens || 0,
|
||||
cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
|
||||
cacheReadTokens > 0 ? cacheReadTokens : undefined,
|
||||
)) || 0
|
||||
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: usage.prompt_tokens || 0,
|
||||
outputTokens: usage.completion_tokens || 0,
|
||||
cacheWriteTokens: cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
|
||||
cacheReadTokens: cacheReadTokens > 0 ? cacheReadTokens : undefined,
|
||||
totalCost,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel() {
|
||||
const modelId = this.options.liteLlmModelId || liteLlmDefaultModelId
|
||||
|
||||
// Try to get model info from StateManager cache first
|
||||
const cachedModelInfo = StateManager.get().getModelInfo("liteLlm", modelId)
|
||||
|
||||
// Fall back to provided model info or defaults if not in cache
|
||||
const modelInfo = cachedModelInfo || this.options.liteLlmModelInfo || liteLlmModelInfoSaneDefaults
|
||||
|
||||
return {
|
||||
id: modelId,
|
||||
info: modelInfo,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,109 +0,0 @@
|
||||
import { type ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import type { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import type { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface LmStudioHandlerOptions extends CommonApiHandlerOptions {
|
||||
lmStudioBaseUrl?: string
|
||||
lmStudioModelId?: string
|
||||
lmStudioMaxTokens?: string
|
||||
}
|
||||
|
||||
export class LmStudioHandler implements ApiHandler {
|
||||
private options: LmStudioHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: LmStudioHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
// Docs on the new v0 api endpoint: https://lmstudio.ai/docs/app/api/endpoints/rest
|
||||
baseURL: new URL("api/v0", this.options.lmStudioBaseUrl || "http://localhost:1234").toString(),
|
||||
apiKey: "noop",
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating LM Studio client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry({ retryAllErrors: true })
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
try {
|
||||
const stream = await client.chat.completions.create({
|
||||
model: this.getModel().id,
|
||||
messages: openAiMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
max_completion_tokens: this.options.lmStudioMaxTokens ? Number(this.options.lmStudioMaxTokens) : undefined,
|
||||
...getOpenAIToolParams(tools),
|
||||
})
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const choice = chunk.choices?.[0]
|
||||
const delta = choice?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// LM Studio doesn't return an error code/body for now
|
||||
throw new Error(
|
||||
"Please check the LM Studio developer logs to debug what went wrong. You may need to load the model with a larger context length to work with Cline's prompts. Alternatively, try enabling Compact Prompt in your settings when working with a limited context window.",
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
const info = { ...openAiModelInfoSaneDefaults }
|
||||
const maxTokens = Number(this.options.lmStudioMaxTokens)
|
||||
if (!Number.isNaN(maxTokens)) {
|
||||
info.contextWindow = maxTokens
|
||||
}
|
||||
return {
|
||||
id: this.options.lmStudioModelId || "",
|
||||
info,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,208 +0,0 @@
|
||||
import { Anthropic } from "@anthropic-ai/sdk"
|
||||
import { Tool as AnthropicTool } from "@anthropic-ai/sdk/resources/index"
|
||||
import { Stream as AnthropicStream } from "@anthropic-ai/sdk/streaming"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { MinimaxModelId, ModelInfo, minimaxDefaultModelId, minimaxModels } from "@/shared/api"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { ClineTool } from "@/shared/tools"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface MinimaxHandlerOptions extends CommonApiHandlerOptions {
|
||||
minimaxApiKey?: string
|
||||
minimaxApiLine?: string
|
||||
apiModelId?: string
|
||||
thinkingBudgetTokens?: number
|
||||
}
|
||||
|
||||
export class MinimaxHandler implements ApiHandler {
|
||||
private options: MinimaxHandlerOptions
|
||||
private client: Anthropic | undefined
|
||||
|
||||
constructor(options: MinimaxHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): Anthropic {
|
||||
if (!this.client) {
|
||||
if (!this.options.minimaxApiKey) {
|
||||
throw new Error("MiniMax API key is required")
|
||||
}
|
||||
try {
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
this.client = new Anthropic({
|
||||
apiKey: this.options.minimaxApiKey,
|
||||
baseURL:
|
||||
this.options.minimaxApiLine === "china"
|
||||
? "https://api.minimaxi.com/anthropic"
|
||||
: "https://api.minimax.io/anthropic",
|
||||
defaultHeaders: externalHeaders,
|
||||
fetch, // Use configured fetch with proxy support
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating MiniMax client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ClineTool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
|
||||
// Tools are available only when native tools are enabled
|
||||
const nativeToolsOn = tools?.length && tools?.length > 0
|
||||
|
||||
const budget_tokens = this.options.thinkingBudgetTokens || 0
|
||||
const reasoningOn = (model.info.supportsReasoning ?? false) && budget_tokens !== 0
|
||||
|
||||
// MiniMax M2 uses Anthropic API format
|
||||
const stream: AnthropicStream<Anthropic.RawMessageStreamEvent> = await client.messages.create({
|
||||
model: model.id,
|
||||
max_tokens: model.info.maxTokens || 8192,
|
||||
system: [{ text: systemPrompt, type: "text" }],
|
||||
messages,
|
||||
stream: true,
|
||||
tools: nativeToolsOn ? (tools as AnthropicTool[]) : undefined,
|
||||
thinking: reasoningOn ? { type: "enabled", budget_tokens: budget_tokens } : undefined,
|
||||
// "Thinking isn't compatible with temperature, top_p, or top_k modifications"
|
||||
temperature: reasoningOn ? undefined : 1.0, // MiniMax recommends 1.0, range is (0.0, 1.0]
|
||||
// NOTE: Forcing tool use when tools are provided will result in error when thinking is also enabled.
|
||||
tool_choice: nativeToolsOn && !reasoningOn ? { type: "any" } : undefined,
|
||||
})
|
||||
|
||||
const lastStartedToolCall = { id: "", name: "", arguments: "" }
|
||||
|
||||
for await (const chunk of stream) {
|
||||
switch (chunk?.type) {
|
||||
case "message_start": {
|
||||
// tells us cache reads/writes/input/output
|
||||
const usage = chunk.message.usage
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: usage.input_tokens || 0,
|
||||
outputTokens: usage.output_tokens || 0,
|
||||
cacheWriteTokens: usage.cache_creation_input_tokens || undefined,
|
||||
cacheReadTokens: usage.cache_read_input_tokens || undefined,
|
||||
}
|
||||
break
|
||||
}
|
||||
case "message_delta":
|
||||
// tells us stop_reason, stop_sequence, and output tokens along the way and at the end of the message
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: 0,
|
||||
outputTokens: chunk.usage.output_tokens || 0,
|
||||
}
|
||||
break
|
||||
case "message_stop":
|
||||
// no usage data, just an indicator that the message is done
|
||||
break
|
||||
case "content_block_start":
|
||||
switch (chunk.content_block.type) {
|
||||
case "thinking":
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: chunk.content_block.thinking || "",
|
||||
signature: chunk.content_block.signature,
|
||||
}
|
||||
break
|
||||
case "redacted_thinking":
|
||||
// Content is encrypted, and we don't want to pass placeholder text back to the API
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: "[Redacted thinking block]",
|
||||
redacted_data: chunk.content_block.data,
|
||||
}
|
||||
break
|
||||
case "tool_use":
|
||||
if (chunk.content_block.id && chunk.content_block.name) {
|
||||
// Store tool call information for streaming
|
||||
lastStartedToolCall.id = chunk.content_block.id
|
||||
lastStartedToolCall.name = chunk.content_block.name
|
||||
lastStartedToolCall.arguments = ""
|
||||
}
|
||||
break
|
||||
case "text":
|
||||
// we may receive multiple text blocks, in which case just insert a line break between them
|
||||
if (chunk.index > 0) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: "\n",
|
||||
}
|
||||
}
|
||||
yield {
|
||||
type: "text",
|
||||
text: chunk.content_block.text,
|
||||
}
|
||||
break
|
||||
}
|
||||
break
|
||||
case "content_block_delta":
|
||||
switch (chunk.delta.type) {
|
||||
case "thinking_delta":
|
||||
// 'reasoning' type just displays in the UI, but reasoning with signature will be used to send the thinking traces back to the API
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: chunk.delta.thinking,
|
||||
}
|
||||
break
|
||||
case "signature_delta":
|
||||
// It's used when sending the thinking block back to the API
|
||||
// API expects this in completed form, not as array of deltas
|
||||
if (chunk.delta.signature) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: "",
|
||||
signature: chunk.delta.signature,
|
||||
}
|
||||
}
|
||||
break
|
||||
case "text_delta":
|
||||
yield {
|
||||
type: "text",
|
||||
text: chunk.delta.text,
|
||||
}
|
||||
break
|
||||
case "input_json_delta":
|
||||
if (lastStartedToolCall.id && lastStartedToolCall.name && chunk.delta.partial_json) {
|
||||
// Convert Anthropic tool_use to OpenAI-compatible format for internal processing
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
tool_call: {
|
||||
...lastStartedToolCall,
|
||||
function: {
|
||||
...lastStartedToolCall,
|
||||
id: lastStartedToolCall.id,
|
||||
name: lastStartedToolCall.name,
|
||||
arguments: chunk.delta.partial_json,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
break
|
||||
|
||||
case "content_block_stop":
|
||||
lastStartedToolCall.id = ""
|
||||
lastStartedToolCall.name = ""
|
||||
lastStartedToolCall.arguments = ""
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: MinimaxModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
|
||||
if (modelId && modelId in minimaxModels) {
|
||||
const id = modelId as MinimaxModelId
|
||||
return { id, info: minimaxModels[id] }
|
||||
}
|
||||
return { id: minimaxDefaultModelId, info: minimaxModels[minimaxDefaultModelId] }
|
||||
}
|
||||
}
|
||||
@@ -1,154 +0,0 @@
|
||||
import { Mistral } from "@mistralai/mistralai"
|
||||
import { HTTPClient } from "@mistralai/mistralai/lib/http"
|
||||
import { Tool as MistralTool } from "@mistralai/mistralai/models/components/tool"
|
||||
import { MistralModelId, ModelInfo, mistralDefaultModelId, mistralModels } from "@shared/api"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToMistralMessages } from "../transform/mistral-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface MistralHandlerOptions extends CommonApiHandlerOptions {
|
||||
mistralApiKey?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
export class MistralHandler implements ApiHandler {
|
||||
private options: MistralHandlerOptions
|
||||
private client: Mistral | undefined
|
||||
|
||||
constructor(options: MistralHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): Mistral {
|
||||
if (!this.client) {
|
||||
if (!this.options.mistralApiKey) {
|
||||
throw new Error("Mistral API key is required")
|
||||
}
|
||||
try {
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
// Create HTTP client with custom fetch for proxy support
|
||||
// The Mistral SDK's HTTPClient passes a Request object to the fetcher,
|
||||
// but we need to extract the URL and init options to pass to our fetch wrapper
|
||||
// which properly handles proxy configuration in standalone mode (JetBrains/CLI)
|
||||
const httpClient = new HTTPClient({
|
||||
fetcher: async (input: RequestInfo | URL, init?: RequestInit) => {
|
||||
// Handle both string/URL and Request object inputs
|
||||
if (input instanceof Request) {
|
||||
Object.keys(externalHeaders).forEach((key) => {
|
||||
if (!input.headers.has(key)) {
|
||||
input.headers.set(key, externalHeaders[key])
|
||||
}
|
||||
})
|
||||
return fetch(input.url, {
|
||||
method: input.method,
|
||||
headers: input.headers,
|
||||
body: input.body,
|
||||
redirect: input.redirect,
|
||||
signal: input.signal,
|
||||
// duplex is required when sending a body stream in Node.js/undici
|
||||
duplex: input.body ? "half" : undefined,
|
||||
...init,
|
||||
} as RequestInit)
|
||||
}
|
||||
|
||||
// Merge external headers with existing headers
|
||||
const mergedInit = {
|
||||
...init,
|
||||
headers: {
|
||||
...externalHeaders,
|
||||
...(init?.headers || {}),
|
||||
},
|
||||
}
|
||||
return fetch(input, mergedInit)
|
||||
},
|
||||
})
|
||||
|
||||
this.client = new Mistral({
|
||||
apiKey: this.options.mistralApiKey,
|
||||
httpClient,
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Mistral client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const stream = await client.chat
|
||||
.stream({
|
||||
model: this.getModel().id,
|
||||
// max_completion_tokens: this.getModel().info.maxTokens,
|
||||
temperature: 0,
|
||||
messages: [{ role: "system", content: systemPrompt }, ...convertToMistralMessages(messages)],
|
||||
stream: true,
|
||||
tools: tools?.length ? (tools as MistralTool[]) : undefined,
|
||||
toolChoice: tools?.length ? "any" : undefined,
|
||||
})
|
||||
.catch((err) => {
|
||||
// The Mistal SDK uses statusCode instead of status
|
||||
// However, if they introduce status for something, I don't want to override it
|
||||
if ("statusCode" in err && !("status" in err)) {
|
||||
err.status = err.statusCode
|
||||
}
|
||||
|
||||
throw err
|
||||
})
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.data.choices[0]?.delta
|
||||
if (delta.toolCalls) {
|
||||
for (const toolCall of delta.toolCalls) {
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
tool_call: {
|
||||
function: {
|
||||
id: toolCall.id,
|
||||
name: toolCall.function.name,
|
||||
arguments: JSON.stringify(toolCall.function.arguments),
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
} else if (delta?.content) {
|
||||
let content: string = ""
|
||||
if (typeof delta.content === "string") {
|
||||
content = delta.content
|
||||
} else if (Array.isArray(delta.content)) {
|
||||
content = delta.content.map((c) => (c.type === "text" ? c.text : "")).join("")
|
||||
}
|
||||
yield {
|
||||
type: "text",
|
||||
text: content,
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.data.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.data.usage.promptTokens || 0,
|
||||
outputTokens: chunk.data.usage.completionTokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: MistralModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in mistralModels) {
|
||||
const id = modelId as MistralModelId
|
||||
return { id, info: mistralModels[id] }
|
||||
}
|
||||
return {
|
||||
id: mistralDefaultModelId,
|
||||
info: mistralModels[mistralDefaultModelId],
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,110 +0,0 @@
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ModelInfo, MoonshotModelId, moonshotDefaultModelId, moonshotModels } from "@/shared/api"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface MoonshotHandlerOptions extends CommonApiHandlerOptions {
|
||||
moonshotApiKey?: string
|
||||
moonshotApiLine?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
// Enhanced usage interface to support Moonshot's cached token field
|
||||
interface MoonshotUsage extends OpenAI.CompletionUsage {
|
||||
cached_tokens?: number
|
||||
}
|
||||
|
||||
export class MoonshotHandler implements ApiHandler {
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(private readonly options: MoonshotHandlerOptions) {}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.moonshotApiKey) {
|
||||
throw new Error("Moonshot API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL:
|
||||
this.options.moonshotApiLine === "china" ? "https://api.moonshot.cn/v1" : "https://api.moonshot.ai/v1",
|
||||
apiKey: this.options.moonshotApiKey,
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Moonshot client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
messages: openAiMessages,
|
||||
temperature: model.info.temperature,
|
||||
max_tokens: model.info.maxTokens,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
...getOpenAIToolParams(tools),
|
||||
})
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
const usage = chunk.usage as MoonshotUsage
|
||||
yield {
|
||||
type: "usage",
|
||||
cacheWriteTokens: 0,
|
||||
cacheReadTokens: usage.cached_tokens ?? 0,
|
||||
inputTokens: (usage.prompt_tokens || 0) - (usage.cached_tokens ?? 0),
|
||||
outputTokens: usage.completion_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: MoonshotModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
|
||||
if (modelId && modelId in moonshotModels) {
|
||||
const id = modelId as MoonshotModelId
|
||||
return { id, info: moonshotModels[id] }
|
||||
}
|
||||
return { id: moonshotDefaultModelId, info: moonshotModels[moonshotDefaultModelId] }
|
||||
}
|
||||
}
|
||||
@@ -1,96 +0,0 @@
|
||||
import { type ModelInfo, type NebiusModelId, nebiusDefaultModelId, nebiusModels } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { convertToR1Format } from "../transform/r1-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface NebiusHandlerOptions extends CommonApiHandlerOptions {
|
||||
nebiusApiKey?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
export class NebiusHandler implements ApiHandler {
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(private readonly options: NebiusHandlerOptions) {}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.nebiusApiKey) {
|
||||
throw new Error("Nebius API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://api.studio.nebius.ai/v1",
|
||||
apiKey: this.options.nebiusApiKey,
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Nebius client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = model.id.includes("DeepSeek-R1")
|
||||
? convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
|
||||
: [{ role: "system", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
messages: openAiMessages,
|
||||
temperature: 0,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
...getOpenAIToolParams(tools),
|
||||
})
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
|
||||
if (modelId !== undefined && modelId in nebiusModels) {
|
||||
return { id: modelId, info: nebiusModels[modelId as NebiusModelId] }
|
||||
}
|
||||
return { id: nebiusDefaultModelId, info: nebiusModels[nebiusDefaultModelId] }
|
||||
}
|
||||
}
|
||||
@@ -1,93 +0,0 @@
|
||||
import { ModelInfo, NousResearchModelId, nousResearchDefaultModelId, nousResearchModels } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface NousResearchHandlerOptions extends CommonApiHandlerOptions {
|
||||
nousResearchApiKey?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
export class NousResearchHandler implements ApiHandler {
|
||||
private options: NousResearchHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: NousResearchHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.nousResearchApiKey) {
|
||||
throw new Error("NousResearch API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://inference-api.nousResearch.com/v1",
|
||||
apiKey: this.options.nousResearchApiKey,
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating NousResearch client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
messages: openAiMessages,
|
||||
temperature: 0,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
})
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: NousResearchModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
|
||||
if (modelId && modelId in nousResearchModels) {
|
||||
const id = modelId as NousResearchModelId
|
||||
return { id, info: nousResearchModels[id] }
|
||||
}
|
||||
return { id: nousResearchDefaultModelId, info: nousResearchModels[nousResearchDefaultModelId] }
|
||||
}
|
||||
}
|
||||
@@ -1,473 +0,0 @@
|
||||
import { Anthropic, APIError as AnthropicAPIError } from "@anthropic-ai/sdk"
|
||||
import { liteLlmDefaultModelId, liteLlmModelInfoSaneDefaults, ModelInfo } from "@shared/api"
|
||||
import OpenAI, { APIError as OpenAIAPIError, OpenAIError } from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { OcaAuthService } from "@/services/auth/oca/OcaAuthService"
|
||||
import {
|
||||
DEFAULT_EXTERNAL_OCA_BASE_URL,
|
||||
DEFAULT_INTERNAL_OCA_BASE_URL,
|
||||
OCI_HEADER_OPC_REQUEST_ID,
|
||||
} from "@/services/auth/oca/utils/constants"
|
||||
import { createOcaHeaders } from "@/services/auth/oca/utils/utils"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { OcaModelInfo } from "@/shared/api"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { ApiFormat } from "@/shared/proto/index.cline"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ApiHandler, type CommonApiHandlerOptions } from ".."
|
||||
import { withRetry } from "../retry"
|
||||
import { sanitizeAnthropicMessages } from "../transform/anthropic-format"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
import { convertOpenAIToolsToAnthropicTools, handleAnthropicMessagesApiStreamResponse } from "../utils/messages_api_support"
|
||||
import { handleResponsesApiStreamResponse } from "../utils/responses_api_support"
|
||||
|
||||
export interface OcaHandlerOptions extends CommonApiHandlerOptions {
|
||||
ocaBaseUrl?: string
|
||||
ocaModelId?: string
|
||||
ocaModelInfo?: OcaModelInfo
|
||||
ocaReasoningEffort?: string
|
||||
thinkingBudgetTokens?: number
|
||||
ocaUsePromptCache?: boolean
|
||||
taskId?: string
|
||||
ocaMode?: string // "internal" or "external"
|
||||
}
|
||||
|
||||
export class OcaHandler implements ApiHandler {
|
||||
protected options: OcaHandlerOptions
|
||||
protected openAIClient: OpenAI | undefined
|
||||
protected anthropicClient: Anthropic | undefined
|
||||
protected externalHeaders: Record<string, string> = {}
|
||||
|
||||
constructor(options: OcaHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
protected initializeOpenAIClient(options: OcaHandlerOptions): OpenAI {
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
return new (class OCIOpenAI extends OpenAI {
|
||||
protected override async prepareOptions(opts: any): Promise<void> {
|
||||
const token = await OcaAuthService.getInstance().getAuthToken()
|
||||
if (!token) {
|
||||
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
|
||||
}
|
||||
opts.headers ??= {}
|
||||
// OCA Headers
|
||||
const ociHeaders = await createOcaHeaders(token, options.taskId!)
|
||||
opts.headers = { ...opts.headers, ...externalHeaders, ...ociHeaders }
|
||||
Logger.log(`Making request with customer opc-request-id: ${opts.headers?.["opc-request-id"]}`)
|
||||
return super.prepareOptions(opts)
|
||||
}
|
||||
|
||||
protected override makeStatusError(
|
||||
status: number | undefined,
|
||||
error: Object | undefined,
|
||||
message: string | undefined,
|
||||
headers: any | undefined,
|
||||
): OpenAIAPIError {
|
||||
interface OciError {
|
||||
code?: string
|
||||
message?: string
|
||||
}
|
||||
let ociErrorMessage = message
|
||||
if (typeof error === "object" && error !== null) {
|
||||
try {
|
||||
ociErrorMessage = JSON.stringify(error)
|
||||
const ociErr = error as OciError
|
||||
if (ociErr.code !== undefined && ociErr.message !== undefined) {
|
||||
ociErrorMessage = `${ociErr.code}: ${ociErr.message}`
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
const opcRequestId = headers?.[OCI_HEADER_OPC_REQUEST_ID]
|
||||
if (opcRequestId) {
|
||||
ociErrorMessage += `\n(${OCI_HEADER_OPC_REQUEST_ID}: ${opcRequestId})`
|
||||
}
|
||||
const statusCode = typeof status === "number" ? status : 500
|
||||
return super.makeStatusError(statusCode, error ?? {}, ociErrorMessage, headers)
|
||||
}
|
||||
})({
|
||||
baseURL:
|
||||
options.ocaBaseUrl ||
|
||||
(options.ocaMode === "internal" ? DEFAULT_INTERNAL_OCA_BASE_URL : DEFAULT_EXTERNAL_OCA_BASE_URL),
|
||||
apiKey: "noop",
|
||||
fetch, // Use configured fetch with proxy support
|
||||
})
|
||||
}
|
||||
|
||||
protected initializeAnthropicClient(options: OcaHandlerOptions): Anthropic {
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
return new (class OCIAnthropic extends Anthropic {
|
||||
protected override async prepareOptions(opts: any): Promise<void> {
|
||||
const token = await OcaAuthService.getInstance().getAuthToken()
|
||||
if (!token) {
|
||||
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
|
||||
}
|
||||
opts.headers ??= {}
|
||||
// OCA Headers
|
||||
const ociHeaders = await createOcaHeaders(token, options.taskId!)
|
||||
opts.headers = { ...opts.headers, ...externalHeaders, ...ociHeaders }
|
||||
Logger.log(`Making request with customer opc-request-id: ${opts.headers?.["opc-request-id"]}`)
|
||||
return super.prepareOptions(opts)
|
||||
}
|
||||
|
||||
protected override makeStatusError(
|
||||
status: number | undefined,
|
||||
error: Object | undefined,
|
||||
message: string | undefined,
|
||||
headers: any | undefined,
|
||||
): AnthropicAPIError {
|
||||
interface OciError {
|
||||
code?: string
|
||||
message?: string
|
||||
}
|
||||
let ociErrorMessage = message
|
||||
if (typeof error === "object" && error !== null) {
|
||||
try {
|
||||
ociErrorMessage = JSON.stringify(error)
|
||||
const ociErr = error as OciError
|
||||
if (ociErr.code !== undefined && ociErr.message !== undefined) {
|
||||
ociErrorMessage = `${ociErr.code}: ${ociErr.message}`
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
const opcRequestId = headers?.[OCI_HEADER_OPC_REQUEST_ID]
|
||||
if (opcRequestId) {
|
||||
ociErrorMessage += `\n(${OCI_HEADER_OPC_REQUEST_ID}: ${opcRequestId})`
|
||||
}
|
||||
const statusCode = typeof status === "number" ? status : 500
|
||||
return super.makeStatusError(statusCode, error ?? {}, ociErrorMessage, headers)
|
||||
}
|
||||
})({
|
||||
baseURL:
|
||||
options.ocaBaseUrl ||
|
||||
(options.ocaMode === "internal" ? DEFAULT_INTERNAL_OCA_BASE_URL : DEFAULT_EXTERNAL_OCA_BASE_URL),
|
||||
apiKey: "noop",
|
||||
fetch, // Use configured fetch with proxy support
|
||||
})
|
||||
}
|
||||
|
||||
protected ensureOpenAIClient(): OpenAI {
|
||||
if (!this.openAIClient) {
|
||||
if (!this.options.ocaModelId) {
|
||||
throw new Error("Oracle Code Assist (OCA) model is not selected")
|
||||
}
|
||||
try {
|
||||
this.openAIClient = this.initializeOpenAIClient(this.options)
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Oracle Code Assist (OCA) client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.openAIClient
|
||||
}
|
||||
|
||||
protected ensureAnthropicClient(): Anthropic {
|
||||
if (!this.anthropicClient) {
|
||||
if (!this.options.ocaModelId) {
|
||||
throw new Error("Oracle Code Assist (OCA) model is not selected")
|
||||
}
|
||||
try {
|
||||
this.anthropicClient = this.initializeAnthropicClient(this.options)
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Oracle Code Assist (OCA) client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.anthropicClient
|
||||
}
|
||||
|
||||
async getApiCosts(prompt_tokens: number, completion_tokens: number): Promise<number | undefined> {
|
||||
// Reference: https://github.com/BerriAI/litellm/blob/122ee634f434014267af104814022af1d9a0882f/litellm/proxy/spend_tracking/spend_management_endpoints.py#L1473
|
||||
const client = this.ensureOpenAIClient()
|
||||
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
|
||||
const token = await OcaAuthService.getInstance().getAuthToken()
|
||||
if (!token) {
|
||||
throw new OpenAIError("Unable to handle auth, Oracle Code Assist (OCA) access token is not available")
|
||||
}
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
const ociHeaders = await createOcaHeaders(token, this.options.taskId!)
|
||||
Logger.log(`Making calculate cost request with customer opc-request-id: ${ociHeaders["opc-request-id"]}`)
|
||||
try {
|
||||
const response = await fetch(`${client.baseURL}/spend/calculate`, {
|
||||
method: "POST",
|
||||
headers: { ...externalHeaders, ...ociHeaders },
|
||||
body: JSON.stringify({
|
||||
completion_response: {
|
||||
model: modelId,
|
||||
usage: {
|
||||
prompt_tokens,
|
||||
completion_tokens,
|
||||
},
|
||||
},
|
||||
}),
|
||||
})
|
||||
|
||||
if (response.ok) {
|
||||
const data: { cost: number } = await response.json()
|
||||
return data.cost
|
||||
}
|
||||
Logger.error("Error calculating spend:", response.statusText)
|
||||
return undefined
|
||||
} catch (error) {
|
||||
Logger.error("Error calculating spend:", error)
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
async calculateCost(
|
||||
modelInfo: ModelInfo,
|
||||
inputTokens: number,
|
||||
outputTokens: number,
|
||||
_cacheWriteTokens?: number,
|
||||
_cacheReadTokens?: number,
|
||||
) {
|
||||
const inputCost = (await this.getApiCosts(1e6, 0)) || 0
|
||||
const outputCost = (await this.getApiCosts(0, 1e6)) || 0
|
||||
const totalCost = (inputCost * inputTokens) / 1e6 + (outputCost * outputTokens) / 1e6
|
||||
return totalCost
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
if (this.options.ocaModelInfo?.apiFormat == ApiFormat.OPENAI_RESPONSES) {
|
||||
yield* this.createMessageResponsesApi(systemPrompt, messages, tools)
|
||||
} else if (this.options.ocaModelInfo?.apiFormat == ApiFormat.ANTHROPIC_CHAT) {
|
||||
yield* this.createMessageMessagesApi(systemPrompt, messages, tools)
|
||||
} else {
|
||||
yield* this.createMessageChatApi(systemPrompt, messages, tools)
|
||||
}
|
||||
}
|
||||
|
||||
async *createMessageChatApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureOpenAIClient()
|
||||
const formattedMessages = convertToOpenAiMessages(messages)
|
||||
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam = {
|
||||
role: "system",
|
||||
content: systemPrompt,
|
||||
}
|
||||
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
|
||||
const isOminiModel = modelId.includes("o1-mini") || modelId.includes("o3-mini") || modelId.includes("o4-mini")
|
||||
|
||||
// Configuration for extended thinking
|
||||
const budgetTokens = this.options.thinkingBudgetTokens || 0
|
||||
const reasoningOn = budgetTokens !== 0
|
||||
const thinkingConfig = reasoningOn ? { type: "enabled", budget_tokens: budgetTokens } : undefined
|
||||
|
||||
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
|
||||
const maxTokens: number | undefined = this.options.ocaModelInfo?.maxTokens
|
||||
|
||||
if (isOminiModel && reasoningOn) {
|
||||
temperature = undefined // Thinking mode doesn't support temperature
|
||||
}
|
||||
|
||||
// Define cache control object if prompt caching is enabled
|
||||
const cacheControl = this.options.ocaUsePromptCache ? { cache_control: { type: "ephemeral" } } : undefined
|
||||
|
||||
// Add cache_control to system message if enabled
|
||||
const enhancedSystemMessage = {
|
||||
...systemMessage,
|
||||
...(cacheControl && cacheControl),
|
||||
}
|
||||
|
||||
// Find the last two user messages to apply caching
|
||||
const userMsgIndices = formattedMessages.reduce((acc, msg, index) => {
|
||||
if (msg.role === "user") {
|
||||
acc.push(index)
|
||||
}
|
||||
return acc
|
||||
}, [] as number[])
|
||||
const lastUserMsgIndex = userMsgIndices[userMsgIndices.length - 1] ?? -1
|
||||
const secondLastUserMsgIndex = userMsgIndices[userMsgIndices.length - 2] ?? -1
|
||||
|
||||
// Apply cache_control to the last two user messages if enabled
|
||||
const enhancedMessages = formattedMessages.map((message, index) => {
|
||||
if ((index === lastUserMsgIndex || index === secondLastUserMsgIndex) && cacheControl) {
|
||||
return {
|
||||
...message,
|
||||
...cacheControl,
|
||||
}
|
||||
}
|
||||
return message
|
||||
})
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
const chatCompletionsParams: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming = {
|
||||
model: this.options.ocaModelId || liteLlmDefaultModelId,
|
||||
messages: [enhancedSystemMessage, ...enhancedMessages],
|
||||
temperature,
|
||||
stream: true,
|
||||
max_completion_tokens: maxTokens,
|
||||
max_tokens: maxTokens,
|
||||
stream_options: { include_usage: true },
|
||||
...(thinkingConfig && { thinking: thinkingConfig }), // Add thinking configuration when applicable
|
||||
...(this.options.taskId && {
|
||||
litellm_session_id: `cline-${this.options.taskId}`,
|
||||
...getOpenAIToolParams(tools),
|
||||
}), // Add session ID for LiteLLM tracking
|
||||
}
|
||||
|
||||
if (this.options.ocaModelInfo?.supportsReasoningEffort) {
|
||||
chatCompletionsParams["reasoning_effort"] = this.options.ocaReasoningEffort || ("medium" as any)
|
||||
}
|
||||
|
||||
const stream = await client.chat.completions.create(chatCompletionsParams)
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
|
||||
// Handle normal text content
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
// Handle reasoning events (thinking)
|
||||
// Thinking is not in the standard types but may be in the response
|
||||
interface ThinkingDelta {
|
||||
thinking?: string
|
||||
}
|
||||
|
||||
if ((delta as ThinkingDelta)?.thinking) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta as ThinkingDelta).thinking || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
// Handle token usage information
|
||||
if (chunk.usage) {
|
||||
const totalCost = await this.calculateCost(
|
||||
this.options.ocaModelInfo!,
|
||||
chunk.usage.prompt_tokens,
|
||||
chunk.usage.completion_tokens,
|
||||
)
|
||||
|
||||
// Extract cache-related information if available
|
||||
// Need to use type assertion since these properties are not in the standard OpenAI types
|
||||
const usage = chunk.usage as {
|
||||
prompt_tokens: number
|
||||
completion_tokens: number
|
||||
cache_creation_input_tokens?: number
|
||||
prompt_cache_miss_tokens?: number
|
||||
cache_read_input_tokens?: number
|
||||
prompt_cache_hit_tokens?: number
|
||||
}
|
||||
|
||||
const cacheWriteTokens = usage.cache_creation_input_tokens || usage.prompt_cache_miss_tokens || 0
|
||||
const cacheReadTokens = usage.cache_read_input_tokens || usage.prompt_cache_hit_tokens || 0
|
||||
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: usage.prompt_tokens || 0,
|
||||
outputTokens: usage.completion_tokens || 0,
|
||||
cacheWriteTokens: cacheWriteTokens > 0 ? cacheWriteTokens : undefined,
|
||||
cacheReadTokens: cacheReadTokens > 0 ? cacheReadTokens : undefined,
|
||||
totalCost,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async *createMessageResponsesApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureOpenAIClient()
|
||||
const inputMessages = convertToOpenAIResponsesInput(messages, { usePreviousResponseId: false }).input
|
||||
// Convert messages to Responses API input format
|
||||
const input: OpenAI.Responses.ResponseInputItem[] = [{ role: "system", content: systemPrompt }, ...inputMessages]
|
||||
|
||||
// Convert ChatCompletion tools to Responses API format if provided
|
||||
const responseTools = tools
|
||||
?.filter((tool) => tool?.type === "function")
|
||||
.map((tool: any) => ({
|
||||
type: "function" as const,
|
||||
name: tool.function.name,
|
||||
description: tool.function.description,
|
||||
parameters: tool.function.parameters,
|
||||
strict: tool.function.strict ?? true, // Responses API defaults to strict mode
|
||||
}))
|
||||
|
||||
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
|
||||
const maxOutputTokens: number | undefined = this.options.ocaModelInfo?.maxTokens
|
||||
|
||||
const ocaModelInfo = this.options.ocaModelInfo
|
||||
if (!ocaModelInfo) {
|
||||
throw new Error("Oracle Code Assist (OCA) model info is required for Responses API")
|
||||
}
|
||||
|
||||
const reasoningOn = !!ocaModelInfo.supportsReasoning
|
||||
if (reasoningOn) {
|
||||
temperature = undefined
|
||||
}
|
||||
|
||||
const responsesParams: OpenAI.Responses.ResponseCreateParamsStreaming = {
|
||||
model: this.options.ocaModelId || liteLlmDefaultModelId,
|
||||
input,
|
||||
stream: true,
|
||||
tools: responseTools,
|
||||
...(typeof temperature === "number" ? { temperature } : {}),
|
||||
...(typeof maxOutputTokens === "number" && maxOutputTokens > 0 ? { max_output_tokens: maxOutputTokens } : {}),
|
||||
}
|
||||
|
||||
if (reasoningOn) {
|
||||
responsesParams.reasoning = { effort: this.options.ocaReasoningEffort as any, summary: "auto" }
|
||||
}
|
||||
|
||||
// Create the response using Responses API
|
||||
const stream = await client.responses.create(responsesParams)
|
||||
|
||||
yield* handleResponsesApiStreamResponse(stream, ocaModelInfo, this.calculateCost.bind(this))
|
||||
}
|
||||
|
||||
async *createMessageMessagesApi(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureAnthropicClient()
|
||||
|
||||
const modelId = this.options.ocaModelId || liteLlmDefaultModelId
|
||||
|
||||
const budgetTokens = this.options.thinkingBudgetTokens || 0
|
||||
const reasoningOn = this.options.ocaModelInfo?.supportsReasoning && budgetTokens !== 0
|
||||
|
||||
let temperature: number | undefined = this.options.ocaModelInfo?.temperature ?? 0
|
||||
const maxTokens: number | undefined = this.options.ocaModelInfo?.maxTokens || 8192
|
||||
|
||||
if (reasoningOn) {
|
||||
temperature = 0
|
||||
}
|
||||
|
||||
const anthropicTools = convertOpenAIToolsToAnthropicTools(tools)
|
||||
const anthropicMessages = sanitizeAnthropicMessages(messages, this.options.ocaUsePromptCache ?? false)
|
||||
|
||||
const stream = await client.messages.create({
|
||||
model: modelId,
|
||||
max_tokens: maxTokens,
|
||||
temperature: reasoningOn ? undefined : temperature,
|
||||
system: [
|
||||
{
|
||||
text: systemPrompt,
|
||||
type: "text",
|
||||
cache_control: this.options.ocaUsePromptCache ? { type: "ephemeral" } : undefined,
|
||||
},
|
||||
],
|
||||
messages: anthropicMessages,
|
||||
stream: true,
|
||||
tools: anthropicTools,
|
||||
thinking: reasoningOn ? { type: "enabled", budget_tokens: budgetTokens } : undefined,
|
||||
})
|
||||
|
||||
yield* handleAnthropicMessagesApiStreamResponse(stream)
|
||||
}
|
||||
|
||||
getModel() {
|
||||
return {
|
||||
id: this.options.ocaModelId || liteLlmDefaultModelId,
|
||||
info: this.options.ocaModelInfo || liteLlmModelInfoSaneDefaults,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,159 +0,0 @@
|
||||
import { type ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
|
||||
import { type Config, type Message, Ollama } from "ollama"
|
||||
import type { ChatCompletionTool } from "openai/resources/chat/completions"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import type { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOllamaMessages } from "../transform/ollama-format"
|
||||
import type { ApiStream } from "../transform/stream"
|
||||
import { ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface OllamaHandlerOptions extends CommonApiHandlerOptions {
|
||||
ollamaBaseUrl?: string
|
||||
ollamaApiKey?: string
|
||||
ollamaModelId?: string
|
||||
ollamaApiOptionsCtxNum?: string
|
||||
requestTimeoutMs?: number
|
||||
}
|
||||
|
||||
const DEFAULT_CONTEXT_WINDOW = 32768
|
||||
|
||||
export class OllamaHandler implements ApiHandler {
|
||||
private options: OllamaHandlerOptions
|
||||
private client: Ollama | undefined
|
||||
|
||||
constructor(options: OllamaHandlerOptions) {
|
||||
const ollamaApiOptionsCtxNum = (options.ollamaApiOptionsCtxNum ?? DEFAULT_CONTEXT_WINDOW).toString()
|
||||
this.options = { ...options, ollamaApiOptionsCtxNum }
|
||||
}
|
||||
|
||||
private ensureClient(): Ollama {
|
||||
if (!this.client) {
|
||||
try {
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
const clientOptions: Partial<Config> = {
|
||||
host: this.options.ollamaBaseUrl,
|
||||
fetch,
|
||||
headers: externalHeaders,
|
||||
}
|
||||
|
||||
// Add API key if provided (for Ollama cloud or authenticated instances)
|
||||
if (this.options.ollamaApiKey) {
|
||||
clientOptions.headers = {
|
||||
...clientOptions.headers,
|
||||
Authorization: `Bearer ${this.options.ollamaApiKey}`,
|
||||
}
|
||||
}
|
||||
|
||||
this.client = new Ollama(clientOptions)
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating Ollama client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry({ retryAllErrors: true })
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const ollamaMessages: Message[] = [{ role: "system", content: systemPrompt }, ...convertToOllamaMessages(messages)]
|
||||
|
||||
try {
|
||||
// Create a promise that rejects after timeout
|
||||
const timeoutMs = this.options.requestTimeoutMs || 30000
|
||||
const timeoutPromise = new Promise<never>((_, reject) => {
|
||||
setTimeout(() => reject(new Error(`Ollama request timed out after ${timeoutMs / 1000} seconds`)), timeoutMs)
|
||||
})
|
||||
|
||||
// Create the actual API request promise
|
||||
const apiPromise = client.chat({
|
||||
model: this.getModel().id,
|
||||
messages: ollamaMessages,
|
||||
stream: true,
|
||||
options: {
|
||||
num_ctx: Number(this.options.ollamaApiOptionsCtxNum),
|
||||
},
|
||||
tools: tools as any,
|
||||
})
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
// Race the API request against the timeout
|
||||
const stream = (await Promise.race([apiPromise, timeoutPromise])) as Awaited<typeof apiPromise>
|
||||
|
||||
try {
|
||||
for await (const chunk of stream) {
|
||||
Logger.debug("[OllamaHandler] Message Chunk" + JSON.stringify(chunk))
|
||||
|
||||
const delta = chunk.message
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
Logger.debug(`[OllamaHandler] Tool Calls Detected: ${JSON.stringify(delta.tool_calls)}`)
|
||||
yield* toolCallProcessor.processToolCallDeltas(
|
||||
delta.tool_calls?.map((tc, inx) => ({
|
||||
index: inx,
|
||||
id: `ollama-tool-${inx}`,
|
||||
function: {
|
||||
name: tc.function.name,
|
||||
arguments:
|
||||
typeof tc.function.arguments === "string"
|
||||
? tc.function.arguments
|
||||
: JSON.stringify(tc.function.arguments),
|
||||
},
|
||||
type: "function",
|
||||
})),
|
||||
)
|
||||
}
|
||||
|
||||
if (typeof delta.content === "string") {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
// Handle token usage if available
|
||||
if (chunk.eval_count !== undefined || chunk.prompt_eval_count !== undefined) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.prompt_eval_count || 0,
|
||||
outputTokens: chunk.eval_count || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (streamError: any) {
|
||||
Logger.error("Error processing Ollama stream:", streamError)
|
||||
throw new Error(`Ollama stream processing error: ${streamError.message || "Unknown error"}`)
|
||||
}
|
||||
} catch (error) {
|
||||
// Check if it's a timeout error
|
||||
if (error?.message?.includes("timed out")) {
|
||||
const timeoutMs = this.options.requestTimeoutMs || 30000
|
||||
throw new Error(`Ollama request timed out after ${timeoutMs / 1000} seconds`)
|
||||
}
|
||||
|
||||
// Enhance error reporting
|
||||
const statusCode = error.status || error.statusCode
|
||||
const errorMessage = error.message || "Unknown error"
|
||||
|
||||
Logger.error(`Ollama API error (${statusCode || "unknown"}): ${errorMessage}`)
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
return {
|
||||
id: this.options.ollamaModelId || "",
|
||||
info: {
|
||||
...openAiModelInfoSaneDefaults,
|
||||
contextWindow: Number(this.options.ollamaApiOptionsCtxNum),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
abort(): void {
|
||||
this.client?.abort()
|
||||
}
|
||||
}
|
||||
@@ -1,732 +0,0 @@
|
||||
import { ModelInfo, OpenAiCodexModelId, openAiCodexDefaultModelId, openAiCodexModels } from "@shared/api"
|
||||
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool } from "openai/resources/chat/completions"
|
||||
import * as os from "os"
|
||||
import { MessageEvent as UndiciMessageEvent, WebSocket as UndiciWebSocket } from "undici"
|
||||
import { v7 as uuidv7 } from "uuid"
|
||||
import { openAiCodexOAuthManager } from "@/integrations/openai-codex/oauth"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { featureFlagsService } from "@/services/feature-flags"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { ApiFormat } from "@/shared/proto/cline/models"
|
||||
import { FeatureFlag } from "@/shared/services/feature-flags/feature-flags"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
|
||||
import { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
|
||||
|
||||
/**
|
||||
* OpenAI Codex base URL for API requests
|
||||
* Routes to chatgpt.com/backend-api/codex
|
||||
*/
|
||||
const CODEX_API_BASE_URL = "https://chatgpt.com/backend-api/codex"
|
||||
const CODEX_RESPONSES_WEBSOCKET_URL = "wss://chatgpt.com/backend-api/codex/responses"
|
||||
|
||||
interface OpenAiCodexHandlerOptions extends CommonApiHandlerOptions {
|
||||
reasoningEffort?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* OpenAiCodexHandler - Uses OpenAI Responses API with OAuth authentication
|
||||
*
|
||||
* Key differences from OpenAiNativeHandler:
|
||||
* - Uses OAuth Bearer tokens instead of API keys
|
||||
* - Routes requests to Codex backend (chatgpt.com/backend-api/codex)
|
||||
* - Subscription-based pricing (no per-token costs)
|
||||
* - Limited model subset
|
||||
* - Custom headers for Codex backend
|
||||
*/
|
||||
export class OpenAiCodexHandler implements ApiHandler {
|
||||
private options: OpenAiCodexHandlerOptions
|
||||
private client?: OpenAI
|
||||
private responsesWs: UndiciWebSocket | undefined
|
||||
private websocketRequestInFlight = false
|
||||
// Session ID for the Codex API (persists for the lifetime of the handler)
|
||||
private readonly sessionId: string
|
||||
// Abort controller for cancelling ongoing requests
|
||||
private abortController?: AbortController
|
||||
// Track tool call identity for streaming
|
||||
private pendingToolCallId: string | undefined
|
||||
private pendingToolCallName: string | undefined
|
||||
|
||||
constructor(options: OpenAiCodexHandlerOptions) {
|
||||
this.options = options
|
||||
this.sessionId = uuidv7()
|
||||
}
|
||||
|
||||
private normalizeUsage(usage: any, _model: { id: string; info: ModelInfo }): ApiStreamUsageChunk | undefined {
|
||||
if (!usage) {
|
||||
return undefined
|
||||
}
|
||||
|
||||
const inputDetails = usage.input_tokens_details ?? usage.prompt_tokens_details
|
||||
|
||||
const hasCachedTokens = typeof inputDetails?.cached_tokens === "number"
|
||||
const hasCacheMissTokens = typeof inputDetails?.cache_miss_tokens === "number"
|
||||
const cachedFromDetails = hasCachedTokens ? inputDetails.cached_tokens : 0
|
||||
const missFromDetails = hasCacheMissTokens ? inputDetails.cache_miss_tokens : 0
|
||||
|
||||
let totalInputTokens = usage.input_tokens ?? usage.prompt_tokens ?? 0
|
||||
if (totalInputTokens === 0 && inputDetails && (cachedFromDetails > 0 || missFromDetails > 0)) {
|
||||
totalInputTokens = cachedFromDetails + missFromDetails
|
||||
}
|
||||
|
||||
const totalOutputTokens = usage.output_tokens ?? usage.completion_tokens ?? 0
|
||||
const cacheWriteTokens = usage.cache_creation_input_tokens ?? usage.cache_write_tokens ?? 0
|
||||
const cacheReadTokens =
|
||||
usage.cache_read_input_tokens ?? usage.cache_read_tokens ?? usage.cached_tokens ?? cachedFromDetails ?? 0
|
||||
|
||||
const reasoningTokens =
|
||||
typeof usage.output_tokens_details?.reasoning_tokens === "number"
|
||||
? usage.output_tokens_details.reasoning_tokens
|
||||
: undefined
|
||||
|
||||
// Subscription-based: no per-token costs
|
||||
const out: ApiStreamUsageChunk = {
|
||||
type: "usage",
|
||||
inputTokens: totalInputTokens,
|
||||
outputTokens: totalOutputTokens,
|
||||
cacheWriteTokens,
|
||||
cacheReadTokens,
|
||||
...(typeof reasoningTokens === "number" ? { reasoningTokens } : {}),
|
||||
totalCost: 0, // Subscription-based pricing
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
|
||||
const model = this.getModel()
|
||||
|
||||
// Reset state for this request
|
||||
this.pendingToolCallId = undefined
|
||||
this.pendingToolCallName = undefined
|
||||
|
||||
// Get access token from OAuth manager
|
||||
let accessToken = await openAiCodexOAuthManager.getAccessToken()
|
||||
if (!accessToken) {
|
||||
throw new Error("Not authenticated with OpenAI Codex. Please sign in using the OpenAI Codex OAuth flow in settings.")
|
||||
}
|
||||
const useWebsocketMode = this.useWebsocketMode(model.info.apiFormat)
|
||||
const { input, previousResponseId } = convertToOpenAIResponsesInput(messages, { usePreviousResponseId: useWebsocketMode })
|
||||
const usePreviousResponseId = useWebsocketMode && !!previousResponseId
|
||||
|
||||
// Build request body
|
||||
const requestBody = this.buildRequestBody(model, input, systemPrompt, tools, previousResponseId)
|
||||
const fallbackRequestBody = this.buildRequestBody(model, input, systemPrompt, tools)
|
||||
|
||||
// Make the request with retry on auth failure
|
||||
for (let attempt = 0; attempt < 2; attempt++) {
|
||||
try {
|
||||
yield* this.executeRequest(requestBody, fallbackRequestBody, model, accessToken, usePreviousResponseId)
|
||||
return
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error)
|
||||
const isAuthFailure = /unauthorized|invalid token|not authenticated|authentication|401/i.test(message)
|
||||
|
||||
if (attempt === 0 && isAuthFailure) {
|
||||
// Force refresh the token for retry
|
||||
const refreshed = await openAiCodexOAuthManager.forceRefreshAccessToken()
|
||||
if (!refreshed) {
|
||||
throw new Error(
|
||||
"Not authenticated with OpenAI Codex. Please sign in using the OpenAI Codex OAuth flow in settings.",
|
||||
)
|
||||
}
|
||||
accessToken = refreshed
|
||||
continue
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private useWebsocketMode(apiFormat?: ApiFormat): boolean {
|
||||
if (featureFlagsService.getBooleanFlagEnabled(FeatureFlag.OPENAI_RESPONSES_WEBSOCKET_MODE)) {
|
||||
return apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
private buildRequestBody(
|
||||
model: { id: string; info: ModelInfo },
|
||||
formattedInput: any,
|
||||
systemPrompt: string,
|
||||
tools?: ChatCompletionTool[],
|
||||
previousResponseId?: string,
|
||||
): any {
|
||||
// Determine reasoning effort
|
||||
const reasoningEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
|
||||
const includeReasoning = reasoningEffort !== "none"
|
||||
|
||||
const body: any = {
|
||||
model: model.id,
|
||||
input: formattedInput,
|
||||
stream: true,
|
||||
store: false,
|
||||
instructions: systemPrompt,
|
||||
...(previousResponseId ? { previous_response_id: previousResponseId } : {}),
|
||||
...(includeReasoning ? { include: ["reasoning.encrypted_content"] } : {}),
|
||||
...(includeReasoning
|
||||
? {
|
||||
reasoning: {
|
||||
effort: reasoningEffort,
|
||||
summary: "auto",
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
}
|
||||
|
||||
// Add tools if provided
|
||||
// Pass through strict value from tool (MCP/custom tools have strict: false, built-in tools default to true)
|
||||
if (tools && tools.length > 0) {
|
||||
body.tools = tools
|
||||
.filter((tool: any) => tool?.type === "function")
|
||||
.map((tool: any) => ({
|
||||
type: "function",
|
||||
name: tool.function.name,
|
||||
description: tool.function.description,
|
||||
parameters: tool.function.parameters,
|
||||
strict: tool.function.strict ?? true,
|
||||
}))
|
||||
}
|
||||
|
||||
return body
|
||||
}
|
||||
|
||||
private async *executeRequest(
|
||||
requestBody: any,
|
||||
fallbackRequestBody: any,
|
||||
model: { id: string; info: ModelInfo },
|
||||
accessToken: string,
|
||||
useWebsocketMode: boolean,
|
||||
): ApiStream {
|
||||
// Create AbortController for cancellation
|
||||
this.abortController = new AbortController()
|
||||
|
||||
try {
|
||||
// Get ChatGPT account ID for organization subscriptions
|
||||
const accountId = await openAiCodexOAuthManager.getAccountId()
|
||||
|
||||
// Build Codex-specific headers
|
||||
const codexHeaders: Record<string, string> = {
|
||||
originator: "cline",
|
||||
session_id: this.sessionId,
|
||||
"User-Agent": `cline/${process.env.npm_package_version || "1.0.0"} (${os.platform()} ${os.release()}; ${os.arch()}) node/${process.version.slice(1)}`,
|
||||
...(accountId ? { "ChatGPT-Account-Id": accountId } : {}),
|
||||
...buildExternalBasicHeaders(),
|
||||
}
|
||||
|
||||
if (useWebsocketMode) {
|
||||
try {
|
||||
yield* this.createResponseStreamWebsocket(requestBody, fallbackRequestBody, accessToken, codexHeaders, model)
|
||||
return
|
||||
} catch (error) {
|
||||
Logger.error("OpenAI Codex websocket mode failed, falling back to HTTP Responses API:", error)
|
||||
this.closeResponsesWebsocket()
|
||||
}
|
||||
}
|
||||
|
||||
// Try using OpenAI SDK first
|
||||
try {
|
||||
const client =
|
||||
this.client ??
|
||||
new OpenAI({
|
||||
apiKey: accessToken,
|
||||
baseURL: CODEX_API_BASE_URL,
|
||||
defaultHeaders: codexHeaders,
|
||||
fetch, // Use shared fetch for proxy support
|
||||
})
|
||||
|
||||
const stream = (await (client as any).responses.create(requestBody, {
|
||||
signal: this.abortController.signal,
|
||||
headers: codexHeaders,
|
||||
})) as AsyncIterable<any>
|
||||
|
||||
if (typeof (stream as any)?.[Symbol.asyncIterator] !== "function") {
|
||||
throw new Error("OpenAI SDK did not return an AsyncIterable")
|
||||
}
|
||||
|
||||
for await (const event of stream) {
|
||||
if (this.abortController.signal.aborted) {
|
||||
break
|
||||
}
|
||||
|
||||
for await (const outChunk of this.processEvent(event, model)) {
|
||||
yield outChunk
|
||||
}
|
||||
}
|
||||
} catch (_sdkErr) {
|
||||
// Fallback to manual SSE via fetch
|
||||
yield* this.makeCodexRequest(requestBody, model, accessToken)
|
||||
}
|
||||
} finally {
|
||||
this.abortController = undefined
|
||||
}
|
||||
}
|
||||
|
||||
private async *createResponseStreamWebsocket(
|
||||
primaryParams: OpenAI.Responses.ResponseCreateParamsStreaming,
|
||||
fallbackParams: OpenAI.Responses.ResponseCreateParamsStreaming,
|
||||
accessToken: string,
|
||||
codexHeaders: Record<string, string>,
|
||||
model: { id: string; info: ModelInfo },
|
||||
): ApiStream {
|
||||
try {
|
||||
for await (const event of this.createResponseEventsViaWebsocket(primaryParams, accessToken, codexHeaders)) {
|
||||
if (this.abortController?.signal.aborted) {
|
||||
return
|
||||
}
|
||||
yield* this.processEvent(event, model)
|
||||
}
|
||||
} catch (error) {
|
||||
if (this.shouldRetryWebsocketWithFullContext(error, !!primaryParams.previous_response_id)) {
|
||||
Logger.log(
|
||||
"Retrying Codex websocket response with full context after previous_response_not_found or socket reset",
|
||||
)
|
||||
this.closeResponsesWebsocket()
|
||||
for await (const event of this.createResponseEventsViaWebsocket(fallbackParams, accessToken, codexHeaders)) {
|
||||
if (this.abortController?.signal.aborted) {
|
||||
return
|
||||
}
|
||||
yield* this.processEvent(event, model)
|
||||
}
|
||||
return
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
private shouldRetryWebsocketWithFullContext(error: unknown, hadPreviousResponseId: boolean): boolean {
|
||||
const errorCode =
|
||||
typeof error === "object" && error && "code" in error && typeof (error as { code: unknown }).code === "string"
|
||||
? (error as { code: string }).code
|
||||
: undefined
|
||||
|
||||
if (hadPreviousResponseId && errorCode === "previous_response_not_found") {
|
||||
return true
|
||||
}
|
||||
if (errorCode === "websocket_closed" || errorCode === "websocket_error") {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
private async ensureResponsesWebsocket(accessToken: string, codexHeaders: Record<string, string>): Promise<UndiciWebSocket> {
|
||||
if (this.responsesWs && this.responsesWs.readyState === UndiciWebSocket.OPEN) {
|
||||
return this.responsesWs
|
||||
}
|
||||
|
||||
this.closeResponsesWebsocket()
|
||||
|
||||
const ws = new UndiciWebSocket(CODEX_RESPONSES_WEBSOCKET_URL, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${accessToken}`,
|
||||
"OpenAI-Beta": "responses_websockets=2026-02-06",
|
||||
...codexHeaders,
|
||||
},
|
||||
})
|
||||
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const cleanup = () => {
|
||||
ws.removeEventListener("open", handleOpen)
|
||||
ws.removeEventListener("error", handleError)
|
||||
ws.removeEventListener("close", handleClose)
|
||||
}
|
||||
const handleOpen = () => {
|
||||
cleanup()
|
||||
resolve()
|
||||
}
|
||||
const handleError = () => {
|
||||
cleanup()
|
||||
reject(new Error("Failed to open Codex Responses websocket"))
|
||||
}
|
||||
const handleClose = () => {
|
||||
cleanup()
|
||||
reject(new Error("Codex Responses websocket closed before opening"))
|
||||
}
|
||||
ws.addEventListener("open", handleOpen)
|
||||
ws.addEventListener("error", handleError)
|
||||
ws.addEventListener("close", handleClose)
|
||||
})
|
||||
|
||||
this.responsesWs = ws
|
||||
return ws
|
||||
}
|
||||
|
||||
private closeResponsesWebsocket() {
|
||||
if (this.responsesWs) {
|
||||
try {
|
||||
this.responsesWs.close()
|
||||
} catch {}
|
||||
this.responsesWs = undefined
|
||||
}
|
||||
}
|
||||
|
||||
private async *createResponseEventsViaWebsocket(
|
||||
params: OpenAI.Responses.ResponseCreateParamsStreaming,
|
||||
accessToken: string,
|
||||
codexHeaders: Record<string, string>,
|
||||
): AsyncGenerator<OpenAI.Responses.ResponseStreamEvent> {
|
||||
if (this.websocketRequestInFlight) {
|
||||
const error: Error & { code?: string } = new Error("Websocket response.create is already in progress")
|
||||
error.code = "websocket_concurrency_limit"
|
||||
throw error
|
||||
}
|
||||
|
||||
const ws = await this.ensureResponsesWebsocket(accessToken, codexHeaders)
|
||||
this.websocketRequestInFlight = true
|
||||
|
||||
const eventQueue: OpenAI.Responses.ResponseStreamEvent[] = []
|
||||
let resolver: (() => void) | undefined
|
||||
let completed = false
|
||||
let failure: (Error & { code?: string }) | undefined
|
||||
|
||||
const wake = () => {
|
||||
const next = resolver
|
||||
resolver = undefined
|
||||
next?.()
|
||||
}
|
||||
|
||||
const handleMessage = (evt: UndiciMessageEvent) => {
|
||||
try {
|
||||
let raw = ""
|
||||
if (typeof evt.data === "string") {
|
||||
raw = evt.data
|
||||
} else if (evt.data instanceof ArrayBuffer) {
|
||||
raw = new TextDecoder().decode(new Uint8Array(evt.data))
|
||||
} else if (ArrayBuffer.isView(evt.data)) {
|
||||
raw = new TextDecoder().decode(new Uint8Array(evt.data.buffer, evt.data.byteOffset, evt.data.byteLength))
|
||||
} else {
|
||||
raw = String(evt.data)
|
||||
}
|
||||
|
||||
const parsed = JSON.parse(raw)
|
||||
if (parsed?.type === "error" && parsed?.error) {
|
||||
const error: Error & { code?: string } = new Error(parsed.error.message || "Codex Responses websocket error")
|
||||
error.code = parsed.error.code
|
||||
failure = error
|
||||
completed = true
|
||||
wake()
|
||||
return
|
||||
}
|
||||
|
||||
eventQueue.push(parsed as OpenAI.Responses.ResponseStreamEvent)
|
||||
if (parsed?.type === "response.completed" || parsed?.type === "response.failed") {
|
||||
completed = true
|
||||
}
|
||||
wake()
|
||||
} catch (error) {
|
||||
const parseError: Error & { code?: string } = new Error(
|
||||
`Failed to parse websocket event: ${error instanceof Error ? error.message : String(error)}`,
|
||||
)
|
||||
parseError.code = "websocket_parse_error"
|
||||
failure = parseError
|
||||
completed = true
|
||||
wake()
|
||||
}
|
||||
}
|
||||
|
||||
const handleError = () => {
|
||||
const error: Error & { code?: string } = new Error("Codex Responses websocket emitted an error event")
|
||||
error.code = "websocket_error"
|
||||
failure = error
|
||||
completed = true
|
||||
wake()
|
||||
}
|
||||
|
||||
const handleClose = () => {
|
||||
if (!completed) {
|
||||
const error: Error & { code?: string } = new Error("Codex Responses websocket closed during response stream")
|
||||
error.code = "websocket_closed"
|
||||
failure = error
|
||||
completed = true
|
||||
wake()
|
||||
}
|
||||
}
|
||||
|
||||
ws.addEventListener("message", handleMessage)
|
||||
ws.addEventListener("error", handleError)
|
||||
ws.addEventListener("close", handleClose)
|
||||
|
||||
try {
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
type: "response.create",
|
||||
...params,
|
||||
}),
|
||||
)
|
||||
|
||||
while (!completed || eventQueue.length > 0) {
|
||||
if (eventQueue.length === 0) {
|
||||
await new Promise<void>((resolve) => {
|
||||
resolver = resolve
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
const event = eventQueue.shift()
|
||||
if (event) {
|
||||
yield event
|
||||
}
|
||||
}
|
||||
|
||||
if (failure) {
|
||||
throw failure
|
||||
}
|
||||
} finally {
|
||||
ws.removeEventListener("message", handleMessage)
|
||||
ws.removeEventListener("error", handleError)
|
||||
ws.removeEventListener("close", handleClose)
|
||||
this.websocketRequestInFlight = false
|
||||
}
|
||||
}
|
||||
|
||||
private async *makeCodexRequest(requestBody: any, model: { id: string; info: ModelInfo }, accessToken: string): ApiStream {
|
||||
const url = `${CODEX_API_BASE_URL}/responses`
|
||||
|
||||
// Get ChatGPT account ID for organization subscriptions
|
||||
const accountId = await openAiCodexOAuthManager.getAccountId()
|
||||
|
||||
// Build headers with required Codex-specific fields
|
||||
const headers: Record<string, string> = {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${accessToken}`,
|
||||
originator: "cline",
|
||||
session_id: this.sessionId,
|
||||
"User-Agent": `cline/${process.env.npm_package_version || "1.0.0"} (${os.platform()} ${os.release()}; ${os.arch()}) node/${process.version.slice(1)}`,
|
||||
}
|
||||
|
||||
// Add ChatGPT-Account-Id if available
|
||||
if (accountId) {
|
||||
headers["ChatGPT-Account-Id"] = accountId
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
method: "POST",
|
||||
headers,
|
||||
body: JSON.stringify(requestBody),
|
||||
signal: this.abortController?.signal,
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
let errorMessage = `Codex API request failed: ${response.status}`
|
||||
|
||||
try {
|
||||
const errorJson = JSON.parse(errorText)
|
||||
if (errorJson.error?.message) {
|
||||
errorMessage = errorJson.error.message
|
||||
} else if (errorJson.message) {
|
||||
errorMessage = errorJson.message
|
||||
}
|
||||
} catch {
|
||||
if (errorText) {
|
||||
errorMessage += ` - ${errorText}`
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error(errorMessage)
|
||||
}
|
||||
|
||||
if (!response.body) {
|
||||
throw new Error("No response body from Codex API")
|
||||
}
|
||||
|
||||
yield* this.handleStreamResponse(response.body, model)
|
||||
} catch (error) {
|
||||
if (error instanceof Error) {
|
||||
throw new Error(`Codex API error: ${error.message}`)
|
||||
}
|
||||
throw new Error("Unexpected error connecting to Codex API")
|
||||
}
|
||||
}
|
||||
|
||||
private async *handleStreamResponse(body: ReadableStream<Uint8Array>, model: { id: string; info: ModelInfo }): ApiStream {
|
||||
const reader = body.getReader()
|
||||
const decoder = new TextDecoder()
|
||||
let buffer = ""
|
||||
|
||||
try {
|
||||
while (true) {
|
||||
if (this.abortController?.signal.aborted) {
|
||||
break
|
||||
}
|
||||
|
||||
const { done, value } = await reader.read()
|
||||
if (done) {
|
||||
break
|
||||
}
|
||||
|
||||
buffer += decoder.decode(value, { stream: true })
|
||||
const lines = buffer.split("\n")
|
||||
buffer = lines.pop() || ""
|
||||
|
||||
for (const line of lines) {
|
||||
if (line.startsWith("data: ")) {
|
||||
const data = line.slice(6).trim()
|
||||
if (data === "[DONE]") {
|
||||
continue
|
||||
}
|
||||
|
||||
try {
|
||||
const parsed = JSON.parse(data)
|
||||
|
||||
for await (const outChunk of this.processEvent(parsed, model)) {
|
||||
yield outChunk
|
||||
}
|
||||
} catch (e) {
|
||||
if (!(e instanceof SyntaxError)) {
|
||||
throw e
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock()
|
||||
}
|
||||
}
|
||||
|
||||
private async *processEvent(event: any, model: { id: string; info: ModelInfo }): ApiStream {
|
||||
// Handle text deltas
|
||||
if (event?.type === "response.text.delta" || event?.type === "response.output_text.delta") {
|
||||
if (event?.delta) {
|
||||
yield { type: "text", text: event.delta }
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Handle reasoning deltas
|
||||
if (
|
||||
event?.type === "response.reasoning.delta" ||
|
||||
event?.type === "response.reasoning_text.delta" ||
|
||||
event?.type === "response.reasoning_summary.delta" ||
|
||||
event?.type === "response.reasoning_summary_text.delta"
|
||||
) {
|
||||
if (event?.delta) {
|
||||
yield { type: "reasoning", reasoning: event.delta }
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Handle refusal deltas
|
||||
if (event?.type === "response.refusal.delta") {
|
||||
if (event?.delta) {
|
||||
yield { type: "text", text: `[Refusal] ${event.delta}` }
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Handle tool/function call deltas
|
||||
if (event?.type === "response.tool_call_arguments.delta" || event?.type === "response.function_call_arguments.delta") {
|
||||
const callId = event.call_id || event.tool_call_id || event.id || this.pendingToolCallId
|
||||
const name = event.name || event.function_name || this.pendingToolCallName
|
||||
const args = event.delta || event.arguments
|
||||
|
||||
if (typeof callId === "string" && callId.length > 0 && typeof name === "string" && name.length > 0) {
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
tool_call: {
|
||||
call_id: callId,
|
||||
function: {
|
||||
id: callId,
|
||||
name,
|
||||
arguments: typeof args === "string" ? args : "",
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Handle output item events
|
||||
if (event?.type === "response.output_item.added" || event?.type === "response.output_item.done") {
|
||||
const item = event?.item
|
||||
if (item) {
|
||||
// Capture tool identity for subsequent argument deltas
|
||||
if (item.type === "function_call" || item.type === "tool_call") {
|
||||
const callId = item.call_id || item.tool_call_id || item.id
|
||||
const name = item.name || item.function?.name || item.function_name
|
||||
if (typeof callId === "string" && callId.length > 0) {
|
||||
this.pendingToolCallId = callId
|
||||
this.pendingToolCallName = typeof name === "string" ? name : undefined
|
||||
}
|
||||
}
|
||||
|
||||
if (item.type === "text" && item.text) {
|
||||
yield { type: "text", text: item.text }
|
||||
} else if (item.type === "reasoning" && item.text) {
|
||||
yield { type: "reasoning", reasoning: item.text }
|
||||
} else if (item.type === "message" && Array.isArray(item.content)) {
|
||||
for (const content of item.content) {
|
||||
if ((content?.type === "text" || content?.type === "output_text") && content?.text) {
|
||||
yield { type: "text", text: content.text }
|
||||
}
|
||||
}
|
||||
} else if (
|
||||
(item.type === "function_call" || item.type === "tool_call") &&
|
||||
event.type === "response.output_item.done"
|
||||
) {
|
||||
const callId = item.call_id || item.tool_call_id || item.id
|
||||
if (callId) {
|
||||
const args = item.arguments || item.function?.arguments || item.function_arguments
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
id: callId,
|
||||
tool_call: {
|
||||
call_id: callId,
|
||||
function: {
|
||||
id: callId,
|
||||
name: item.name || item.function?.name || item.function_name || "",
|
||||
arguments: typeof args === "string" ? args : "{}",
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Handle completion events
|
||||
if (event?.type === "response.done" || event?.type === "response.completed") {
|
||||
const usage = event?.response?.usage || event?.usage || undefined
|
||||
const usageData = this.normalizeUsage(usage, model)
|
||||
if (usageData) {
|
||||
yield usageData
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Fallbacks for legacy formats
|
||||
if (event?.choices?.[0]?.delta?.content) {
|
||||
yield { type: "text", text: event.choices[0].delta.content }
|
||||
return
|
||||
}
|
||||
|
||||
if (event?.usage) {
|
||||
const usageData = this.normalizeUsage(event.usage, model)
|
||||
if (usageData) {
|
||||
yield usageData
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
abort(): void {
|
||||
this.closeResponsesWebsocket()
|
||||
this.abortController?.abort()
|
||||
}
|
||||
|
||||
getModel(): { id: OpenAiCodexModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
|
||||
const id = modelId && modelId in openAiCodexModels ? (modelId as OpenAiCodexModelId) : openAiCodexDefaultModelId
|
||||
|
||||
const info: ModelInfo = openAiCodexModels[id]
|
||||
|
||||
return { id, info }
|
||||
}
|
||||
}
|
||||
@@ -1,702 +0,0 @@
|
||||
import {
|
||||
ModelInfo,
|
||||
OpenAiCompatibleModelInfo,
|
||||
OpenAiNativeModelId,
|
||||
openAiNativeDefaultModelId,
|
||||
openAiNativeModels,
|
||||
} from "@shared/api"
|
||||
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
|
||||
import { calculateApiCostOpenAI } from "@utils/cost"
|
||||
import OpenAI from "openai"
|
||||
import type {
|
||||
ChatCompletionFunctionTool,
|
||||
ChatCompletionReasoningEffort,
|
||||
ChatCompletionTool,
|
||||
} from "openai/resources/chat/completions"
|
||||
import { MessageEvent as UndiciMessageEvent, WebSocket as UndiciWebSocket } from "undici"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { featureFlagsService } from "@/services/feature-flags"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiFormat } from "@/shared/proto/cline/models"
|
||||
import { FeatureFlag } from "@/shared/services/feature-flags/feature-flags"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { isGPT5ModelFamily } from "@/utils/model-utils"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { convertToOpenAIResponsesInput } from "../transform/openai-response-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface OpenAiNativeHandlerOptions extends CommonApiHandlerOptions {
|
||||
openAiNativeApiKey?: string
|
||||
reasoningEffort?: string
|
||||
thinkingBudgetTokens?: number
|
||||
apiModelId?: string
|
||||
openAiNativeUseResponsesWebsocket?: boolean
|
||||
}
|
||||
|
||||
export class OpenAiNativeHandler implements ApiHandler {
|
||||
private options: OpenAiNativeHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
private responsesWs: UndiciWebSocket | undefined
|
||||
private responsesWsReadyPromise: Promise<UndiciWebSocket> | undefined
|
||||
private websocketRequestInFlight = false
|
||||
private abortController?: AbortController
|
||||
|
||||
constructor(options: OpenAiNativeHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.openAiNativeApiKey) {
|
||||
throw new Error("OpenAI API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
apiKey: this.options.openAiNativeApiKey,
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(`Error creating OpenAI client: ${error instanceof Error ? error.message : String(error)}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
private async *yieldUsage(info: ModelInfo, usage: OpenAI.Completions.CompletionUsage | undefined): ApiStream {
|
||||
const inputTokens = usage?.prompt_tokens || 0 // sum of cache hits and misses
|
||||
const outputTokens = usage?.completion_tokens || 0
|
||||
const cacheReadTokens = usage?.prompt_tokens_details?.cached_tokens || 0
|
||||
const cacheWriteTokens = 0
|
||||
const totalCost = calculateApiCostOpenAI(info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
|
||||
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: nonCachedInputTokens,
|
||||
outputTokens: outputTokens,
|
||||
cacheWriteTokens: cacheWriteTokens,
|
||||
cacheReadTokens: cacheReadTokens,
|
||||
totalCost: totalCost,
|
||||
}
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
|
||||
// Responses API requires tool format to be set to OPENAI_RESPONSES with native tools calling enabled
|
||||
const apiFormat = this.getModel()?.info?.apiFormat
|
||||
if (apiFormat === ApiFormat.OPENAI_RESPONSES || apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE) {
|
||||
if (!tools?.length) {
|
||||
throw new Error("Native Tool Call must be enabled in your setting for OpenAI Responses API")
|
||||
}
|
||||
yield* this.createResponseStream(systemPrompt, messages, tools)
|
||||
} else {
|
||||
yield* this.createCompletionStream(systemPrompt, messages, tools)
|
||||
}
|
||||
}
|
||||
|
||||
private async *createCompletionStream(
|
||||
systemPrompt: string,
|
||||
messages: ClineStorageMessage[],
|
||||
tools?: ChatCompletionTool[],
|
||||
): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
this.abortController = new AbortController()
|
||||
|
||||
// Handle o1 models separately as they don't support streaming
|
||||
if (model.info.supportsStreaming === false) {
|
||||
const response = await client.chat.completions.create(
|
||||
{
|
||||
model: model.id,
|
||||
messages: [{ role: "user", content: systemPrompt }, ...convertToOpenAiMessages(messages, "openai-native")],
|
||||
},
|
||||
{ signal: this.abortController?.signal },
|
||||
)
|
||||
yield {
|
||||
type: "text",
|
||||
text: response.choices[0]?.message.content || "",
|
||||
}
|
||||
yield* this.yieldUsage(model.info, response.usage)
|
||||
return
|
||||
}
|
||||
|
||||
const systemRole = model.info.systemRole ?? "system"
|
||||
const includeReasoning = model.info.supportsReasoningEffort
|
||||
const includeTools = model.info.supportsTools ?? true
|
||||
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
|
||||
const reasoningEffort =
|
||||
includeReasoning && requestedEffort !== "none" ? (requestedEffort as ChatCompletionReasoningEffort) : undefined
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
messages: [{ role: systemRole, content: systemPrompt }, ...convertToOpenAiMessages(messages, "openai-native")],
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
reasoning_effort: reasoningEffort,
|
||||
...(model.info.temperature !== undefined ? { temperature: model.info.temperature } : {}),
|
||||
...(includeTools ? getOpenAIToolParams(tools, isGPT5ModelFamily(model.id)) : {}),
|
||||
})
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
try {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
} catch (error) {
|
||||
Logger.error("Error processing tool call delta:", error, delta.tool_calls)
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
// Only last chunk contains usage
|
||||
yield* this.yieldUsage(model.info, chunk.usage)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async *createResponseStream(
|
||||
systemPrompt: string,
|
||||
messages: ClineStorageMessage[],
|
||||
tools: ChatCompletionTool[],
|
||||
): ApiStream {
|
||||
const model = this.getModel()
|
||||
const usePreviousResponseId = this.useWebsocketMode(model.info.apiFormat)
|
||||
|
||||
// Warm websocket connection early in websocket mode so the first response.create avoids handshake latency.
|
||||
if (usePreviousResponseId) {
|
||||
this.preconnectResponsesWebsocket()
|
||||
}
|
||||
|
||||
const { input, previousResponseId } = convertToOpenAIResponsesInput(messages, { usePreviousResponseId })
|
||||
const responseTools = this.mapResponseTools(tools)
|
||||
this.abortController = new AbortController()
|
||||
|
||||
const params = this.buildResponseCreateParams({
|
||||
modelId: model.id,
|
||||
systemPrompt,
|
||||
input,
|
||||
previousResponseId,
|
||||
tools: responseTools,
|
||||
})
|
||||
|
||||
const fallbackParams = this.buildResponseCreateParams({
|
||||
modelId: model.id,
|
||||
systemPrompt,
|
||||
input,
|
||||
tools: responseTools,
|
||||
})
|
||||
|
||||
if (usePreviousResponseId && previousResponseId) {
|
||||
try {
|
||||
yield* this.createResponseStreamWebsocket(model.info, params, fallbackParams)
|
||||
return
|
||||
} catch (error) {
|
||||
Logger.error("OpenAI websocket mode failed, falling back to HTTP Responses API:", error)
|
||||
this.closeResponsesWebsocket()
|
||||
}
|
||||
}
|
||||
|
||||
yield* this.createResponseStreamHttp(model.info, params)
|
||||
}
|
||||
|
||||
private preconnectResponsesWebsocket(): void {
|
||||
void this.ensureResponsesWebsocket().catch((error) => {
|
||||
Logger.debug("OpenAI websocket preconnect failed:", error)
|
||||
this.closeResponsesWebsocket()
|
||||
})
|
||||
}
|
||||
|
||||
private useWebsocketMode(apiFormat?: ApiFormat): boolean {
|
||||
if (featureFlagsService.getBooleanFlagEnabled(FeatureFlag.OPENAI_RESPONSES_WEBSOCKET_MODE)) {
|
||||
return apiFormat === ApiFormat.OPENAI_RESPONSES_WEBSOCKET_MODE
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
private mapResponseTools(tools: ChatCompletionTool[]): OpenAI.Responses.Tool[] {
|
||||
return tools
|
||||
?.filter((tool): tool is ChatCompletionFunctionTool => tool?.type === "function")
|
||||
.map((tool) => ({
|
||||
type: "function" as const,
|
||||
name: tool.function.name,
|
||||
description: tool.function.description,
|
||||
parameters: tool.function.parameters ?? null,
|
||||
strict: tool.function.strict ?? true,
|
||||
}))
|
||||
}
|
||||
|
||||
private buildResponseCreateParams(args: {
|
||||
modelId: string
|
||||
systemPrompt: string
|
||||
input: OpenAI.Responses.ResponseInput
|
||||
tools: OpenAI.Responses.Tool[]
|
||||
previousResponseId?: string
|
||||
}): OpenAI.Responses.ResponseCreateParamsStreaming {
|
||||
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
|
||||
const reasoning: { effort: ChatCompletionReasoningEffort; summary: "auto" } | undefined =
|
||||
requestedEffort === "none"
|
||||
? undefined
|
||||
: {
|
||||
effort: requestedEffort,
|
||||
summary: "auto",
|
||||
}
|
||||
|
||||
return {
|
||||
model: args.modelId,
|
||||
instructions: args.systemPrompt,
|
||||
input: args.input,
|
||||
stream: true,
|
||||
tools: args.tools,
|
||||
store: !args.previousResponseId, // Do not use store when websocket mode is enabled.
|
||||
...(args.previousResponseId ? { previous_response_id: args.previousResponseId } : {}),
|
||||
...(reasoning ? { reasoning } : {}),
|
||||
}
|
||||
}
|
||||
|
||||
private async *createResponseStreamHttp(
|
||||
modelInfo: ModelInfo,
|
||||
params: OpenAI.Responses.ResponseCreateParamsStreaming,
|
||||
): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
Logger.debug(`OpenAI Responses Input (HTTP): ${JSON.stringify(params.input)}`)
|
||||
const stream = await client.responses.create(params, { signal: this.abortController?.signal })
|
||||
yield* this.processResponsesEvents(stream, modelInfo)
|
||||
}
|
||||
|
||||
private async *createResponseStreamWebsocket(
|
||||
modelInfo: ModelInfo,
|
||||
primaryParams: OpenAI.Responses.ResponseCreateParamsStreaming,
|
||||
fallbackParams: OpenAI.Responses.ResponseCreateParamsStreaming,
|
||||
): ApiStream {
|
||||
Logger.debug(`OpenAI Responses Input (WebSocket): ${JSON.stringify(primaryParams.input)}`)
|
||||
try {
|
||||
yield* this.processResponsesEvents(this.createResponseEventsViaWebsocket(primaryParams), modelInfo)
|
||||
} catch (error) {
|
||||
if (this.shouldRetryWebsocketWithFullContext(error, !!primaryParams.previous_response_id)) {
|
||||
Logger.log("Retrying websocket response with full context after previous_response_not_found or socket reset")
|
||||
this.closeResponsesWebsocket()
|
||||
yield* this.processResponsesEvents(this.createResponseEventsViaWebsocket(fallbackParams), modelInfo)
|
||||
return
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
private shouldRetryWebsocketWithFullContext(error: unknown, hadPreviousResponseId: boolean): boolean {
|
||||
const errorCode =
|
||||
typeof error === "object" && error && "code" in error && typeof (error as { code: unknown }).code === "string"
|
||||
? (error as { code: string }).code
|
||||
: undefined
|
||||
|
||||
if (hadPreviousResponseId && errorCode === "previous_response_not_found") {
|
||||
return true
|
||||
}
|
||||
if (errorCode === "websocket_closed" || errorCode === "websocket_error") {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
private async ensureResponsesWebsocket(): Promise<UndiciWebSocket> {
|
||||
if (this.responsesWs && this.responsesWs.readyState === UndiciWebSocket.OPEN) {
|
||||
return this.responsesWs
|
||||
}
|
||||
|
||||
if (this.responsesWsReadyPromise) {
|
||||
return this.responsesWsReadyPromise
|
||||
}
|
||||
|
||||
this.closeResponsesWebsocket()
|
||||
|
||||
if (!this.options.openAiNativeApiKey) {
|
||||
throw new Error("OpenAI API key is required")
|
||||
}
|
||||
|
||||
const ws = new UndiciWebSocket("wss://api.openai.com/v1/responses", {
|
||||
headers: {
|
||||
Authorization: `Bearer ${this.options.openAiNativeApiKey}`,
|
||||
"OpenAI-Beta": "responses_websockets=2026-02-06",
|
||||
...buildExternalBasicHeaders(),
|
||||
},
|
||||
})
|
||||
|
||||
this.responsesWs = ws
|
||||
const readyPromise = new Promise<UndiciWebSocket>((resolve, reject) => {
|
||||
const cleanup = () => {
|
||||
ws.removeEventListener("open", handleOpen)
|
||||
ws.removeEventListener("error", handleError)
|
||||
ws.removeEventListener("close", handleClose)
|
||||
}
|
||||
const handleOpen = () => {
|
||||
cleanup()
|
||||
resolve(ws)
|
||||
}
|
||||
const handleError = () => {
|
||||
cleanup()
|
||||
reject(new Error("Failed to open Responses websocket"))
|
||||
}
|
||||
const handleClose = () => {
|
||||
cleanup()
|
||||
reject(new Error("Responses websocket closed before opening"))
|
||||
}
|
||||
ws.addEventListener("open", handleOpen)
|
||||
ws.addEventListener("error", handleError)
|
||||
ws.addEventListener("close", handleClose)
|
||||
})
|
||||
|
||||
this.responsesWsReadyPromise = readyPromise
|
||||
|
||||
try {
|
||||
return await readyPromise
|
||||
} catch (error) {
|
||||
if (this.responsesWs === ws) {
|
||||
this.responsesWs = undefined
|
||||
}
|
||||
throw error
|
||||
} finally {
|
||||
if (this.responsesWsReadyPromise === readyPromise) {
|
||||
this.responsesWsReadyPromise = undefined
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private closeResponsesWebsocket() {
|
||||
this.responsesWsReadyPromise = undefined
|
||||
if (this.responsesWs) {
|
||||
try {
|
||||
this.responsesWs.close()
|
||||
} catch {}
|
||||
this.responsesWs = undefined
|
||||
}
|
||||
}
|
||||
|
||||
private async *createResponseEventsViaWebsocket(
|
||||
params: OpenAI.Responses.ResponseCreateParamsStreaming,
|
||||
): AsyncGenerator<OpenAI.Responses.ResponseStreamEvent> {
|
||||
if (this.websocketRequestInFlight) {
|
||||
const error: Error & { code?: string } = new Error("Websocket response.create is already in progress")
|
||||
error.code = "websocket_concurrency_limit"
|
||||
throw error
|
||||
}
|
||||
|
||||
const ws = await this.ensureResponsesWebsocket()
|
||||
this.websocketRequestInFlight = true
|
||||
|
||||
const eventQueue: OpenAI.Responses.ResponseStreamEvent[] = []
|
||||
let resolver: (() => void) | undefined
|
||||
let completed = false
|
||||
let failure: (Error & { code?: string }) | undefined
|
||||
|
||||
const wake = () => {
|
||||
const next = resolver
|
||||
resolver = undefined
|
||||
next?.()
|
||||
}
|
||||
|
||||
const handleMessage = (evt: UndiciMessageEvent) => {
|
||||
try {
|
||||
let raw = ""
|
||||
if (typeof evt.data === "string") {
|
||||
raw = evt.data
|
||||
} else if (evt.data instanceof ArrayBuffer) {
|
||||
raw = new TextDecoder().decode(new Uint8Array(evt.data))
|
||||
} else if (ArrayBuffer.isView(evt.data)) {
|
||||
raw = new TextDecoder().decode(new Uint8Array(evt.data.buffer, evt.data.byteOffset, evt.data.byteLength))
|
||||
} else {
|
||||
raw = String(evt.data)
|
||||
}
|
||||
const parsed = JSON.parse(raw)
|
||||
|
||||
if (parsed?.type === "error" && parsed?.error) {
|
||||
const error: Error & { code?: string } = new Error(parsed.error.message || "Responses websocket error")
|
||||
error.code = parsed.error.code
|
||||
failure = error
|
||||
completed = true
|
||||
wake()
|
||||
return
|
||||
}
|
||||
|
||||
eventQueue.push(parsed as OpenAI.Responses.ResponseStreamEvent)
|
||||
if (parsed?.type === "response.completed" || parsed?.type === "response.failed") {
|
||||
completed = true
|
||||
}
|
||||
wake()
|
||||
} catch (error) {
|
||||
const parseError: Error & { code?: string } = new Error(
|
||||
`Failed to parse websocket event: ${error instanceof Error ? error.message : String(error)}`,
|
||||
)
|
||||
parseError.code = "websocket_parse_error"
|
||||
failure = parseError
|
||||
completed = true
|
||||
wake()
|
||||
}
|
||||
}
|
||||
|
||||
const handleError = () => {
|
||||
const error: Error & { code?: string } = new Error("Responses websocket emitted an error event")
|
||||
error.code = "websocket_error"
|
||||
failure = error
|
||||
completed = true
|
||||
wake()
|
||||
}
|
||||
|
||||
const handleClose = () => {
|
||||
if (!completed) {
|
||||
const error: Error & { code?: string } = new Error("Responses websocket closed during response stream")
|
||||
error.code = "websocket_closed"
|
||||
failure = error
|
||||
completed = true
|
||||
wake()
|
||||
}
|
||||
}
|
||||
|
||||
ws.addEventListener("message", handleMessage)
|
||||
ws.addEventListener("error", handleError)
|
||||
ws.addEventListener("close", handleClose)
|
||||
|
||||
try {
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
type: "response.create",
|
||||
...params,
|
||||
}),
|
||||
)
|
||||
|
||||
while (!completed || eventQueue.length > 0) {
|
||||
if (eventQueue.length === 0) {
|
||||
await new Promise<void>((resolve) => {
|
||||
resolver = resolve
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
const event = eventQueue.shift()
|
||||
if (event) {
|
||||
yield event
|
||||
}
|
||||
}
|
||||
|
||||
if (failure) {
|
||||
throw failure
|
||||
}
|
||||
} finally {
|
||||
ws.removeEventListener("message", handleMessage)
|
||||
ws.removeEventListener("error", handleError)
|
||||
ws.removeEventListener("close", handleClose)
|
||||
this.websocketRequestInFlight = false
|
||||
}
|
||||
}
|
||||
|
||||
private async *processResponsesEvents(
|
||||
stream: AsyncIterable<OpenAI.Responses.ResponseStreamEvent>,
|
||||
modelInfo: ModelInfo,
|
||||
): ApiStream {
|
||||
const functionCallByItemId = new Map<string, { call_id?: string; name?: string; id?: string }>()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
Logger.debug(`OpenAI Responses Chunk: ${JSON.stringify(chunk)}`)
|
||||
|
||||
if (chunk.type === "response.output_item.added") {
|
||||
const item = chunk.item
|
||||
if (item.type === "function_call" && item.id) {
|
||||
functionCallByItemId.set(item.id, { call_id: item.call_id, name: item.name, id: item.id })
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
id: item.id,
|
||||
tool_call: {
|
||||
call_id: item.call_id,
|
||||
function: {
|
||||
id: item.id,
|
||||
name: item.name,
|
||||
arguments: item.arguments,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
if (item.type === "reasoning" && item.encrypted_content && item.id) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
id: item.id,
|
||||
reasoning: "",
|
||||
redacted_data: item.encrypted_content,
|
||||
}
|
||||
}
|
||||
}
|
||||
if (chunk.type === "response.output_item.done") {
|
||||
const item = chunk.item
|
||||
if (item.type === "function_call") {
|
||||
if (item.id) {
|
||||
functionCallByItemId.set(item.id, { call_id: item.call_id, name: item.name, id: item.id })
|
||||
}
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
id: item.id || item.call_id,
|
||||
tool_call: {
|
||||
call_id: item.call_id,
|
||||
function: {
|
||||
id: item.id,
|
||||
name: item.name,
|
||||
arguments: item.arguments,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
if (item.type === "reasoning") {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
id: item.id,
|
||||
details: item.summary,
|
||||
reasoning: "",
|
||||
}
|
||||
}
|
||||
}
|
||||
if (chunk.type === "response.reasoning_summary_part.added") {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
id: chunk.item_id,
|
||||
reasoning: chunk.part.text,
|
||||
}
|
||||
}
|
||||
if (chunk.type === "response.reasoning_summary_text.delta") {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
id: chunk.item_id,
|
||||
reasoning: chunk.delta,
|
||||
}
|
||||
}
|
||||
if (chunk.type === "response.reasoning_summary_part.done") {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
id: chunk.item_id,
|
||||
details: chunk.part,
|
||||
reasoning: "",
|
||||
}
|
||||
}
|
||||
if (chunk.type === "response.output_text.delta") {
|
||||
if (chunk.delta) {
|
||||
yield {
|
||||
id: chunk.item_id,
|
||||
type: "text",
|
||||
text: chunk.delta,
|
||||
}
|
||||
}
|
||||
}
|
||||
if (chunk.type === "response.reasoning_text.delta") {
|
||||
if (chunk.delta) {
|
||||
yield {
|
||||
id: chunk.item_id,
|
||||
type: "reasoning",
|
||||
reasoning: chunk.delta,
|
||||
}
|
||||
}
|
||||
}
|
||||
if (chunk.type === "response.function_call_arguments.delta") {
|
||||
const pendingCall = functionCallByItemId.get(chunk.item_id)
|
||||
const callId = pendingCall?.call_id
|
||||
const functionName = pendingCall?.name
|
||||
const functionId = pendingCall?.id || chunk.item_id
|
||||
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
tool_call: {
|
||||
call_id: callId,
|
||||
function: {
|
||||
id: functionId,
|
||||
name: functionName,
|
||||
arguments: chunk.delta,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
if (chunk.type === "response.function_call_arguments.done") {
|
||||
if (chunk.item_id && chunk.name && chunk.arguments) {
|
||||
const pendingCall = functionCallByItemId.get(chunk.item_id)
|
||||
const callId = pendingCall?.call_id
|
||||
const functionId = pendingCall?.id || chunk.item_id
|
||||
|
||||
yield {
|
||||
type: "tool_calls",
|
||||
tool_call: {
|
||||
call_id: callId,
|
||||
function: {
|
||||
id: functionId,
|
||||
name: chunk.name,
|
||||
arguments: chunk.arguments,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (
|
||||
chunk.type === "response.incomplete" &&
|
||||
chunk.response?.status === "incomplete" &&
|
||||
chunk.response?.incomplete_details?.reason === "max_output_tokens"
|
||||
) {
|
||||
if (chunk.response?.output_text?.length > 0) {
|
||||
Logger.log("Partial output:", chunk.response.output_text)
|
||||
} else {
|
||||
Logger.log("Ran out of tokens during reasoning")
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.type === "response.completed" && chunk.response?.usage) {
|
||||
const usage = chunk.response.usage
|
||||
const inputTokens = usage.input_tokens || 0
|
||||
const outputTokens = usage.output_tokens || 0
|
||||
const cacheReadTokens = usage.input_tokens_details?.cached_tokens || 0
|
||||
const cacheWriteTokens = 0
|
||||
const reasoningTokens = usage.output_tokens_details?.reasoning_tokens || 0
|
||||
const totalTokens = usage.total_tokens || 0
|
||||
Logger.log(`Total tokens from Responses API usage: ${totalTokens}`)
|
||||
const totalCost = calculateApiCostOpenAI(
|
||||
modelInfo,
|
||||
inputTokens,
|
||||
outputTokens + reasoningTokens,
|
||||
cacheWriteTokens,
|
||||
cacheReadTokens,
|
||||
)
|
||||
const nonCachedInputTokens = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens)
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: nonCachedInputTokens,
|
||||
outputTokens: outputTokens,
|
||||
cacheWriteTokens: cacheWriteTokens,
|
||||
cacheReadTokens: cacheReadTokens,
|
||||
thoughtsTokenCount: reasoningTokens,
|
||||
totalCost: totalCost,
|
||||
id: chunk.response.id,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
abort(): void {
|
||||
this.closeResponsesWebsocket()
|
||||
this.abortController?.abort()
|
||||
this.abortController = undefined
|
||||
}
|
||||
|
||||
getModel(): { id: OpenAiNativeModelId; info: OpenAiCompatibleModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in openAiNativeModels) {
|
||||
const id = modelId as OpenAiNativeModelId
|
||||
const info = openAiNativeModels[id]
|
||||
return { id, info: { ...info } }
|
||||
}
|
||||
return {
|
||||
id: openAiNativeDefaultModelId,
|
||||
info: { ...openAiNativeModels[openAiNativeDefaultModelId] },
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,188 +0,0 @@
|
||||
import { DefaultAzureCredential, getBearerTokenProvider } from "@azure/identity"
|
||||
import { azureOpenAiDefaultApiVersion, ModelInfo, OpenAiCompatibleModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
|
||||
import { normalizeOpenaiReasoningEffort } from "@shared/storage/types"
|
||||
import OpenAI, { AzureOpenAI } from "openai"
|
||||
import type { ChatCompletionReasoningEffort, ChatCompletionTool } from "openai/resources/chat/completions"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient, fetch } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { convertToR1Format } from "../transform/r1-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface OpenAiHandlerOptions extends CommonApiHandlerOptions {
|
||||
openAiApiKey?: string
|
||||
openAiBaseUrl?: string
|
||||
azureApiVersion?: string
|
||||
azureIdentity?: boolean
|
||||
openAiHeaders?: Record<string, string>
|
||||
openAiModelId?: string
|
||||
openAiModelInfo?: OpenAiCompatibleModelInfo
|
||||
reasoningEffort?: string
|
||||
}
|
||||
|
||||
export class OpenAiHandler implements ApiHandler {
|
||||
private options: OpenAiHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: OpenAiHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private getAzureAudienceScope(baseUrl?: string): string {
|
||||
const url = baseUrl?.toLowerCase() ?? ""
|
||||
if (url.includes("azure.us")) return "https://cognitiveservices.azure.us/.default"
|
||||
if (url.includes("azure.com")) return "https://cognitiveservices.azure.com/.default"
|
||||
return "https://cognitiveservices.azure.com/.default"
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.openAiApiKey && !this.options.azureIdentity) {
|
||||
throw new Error("OpenAI API key or Azure Identity Authentication is required")
|
||||
}
|
||||
try {
|
||||
const baseUrl = this.options.openAiBaseUrl?.toLowerCase() ?? ""
|
||||
const isAzureDomain = baseUrl.includes("azure.com") || baseUrl.includes("azure.us")
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
// Azure API shape slightly differs from the core API shape...
|
||||
if (
|
||||
this.options.azureApiVersion ||
|
||||
(isAzureDomain && !this.options.openAiModelId?.toLowerCase().includes("deepseek"))
|
||||
) {
|
||||
if (this.options.azureIdentity) {
|
||||
this.client = new AzureOpenAI({
|
||||
baseURL: this.options.openAiBaseUrl,
|
||||
azureADTokenProvider: getBearerTokenProvider(
|
||||
new DefaultAzureCredential(),
|
||||
this.getAzureAudienceScope(this.options.openAiBaseUrl),
|
||||
),
|
||||
apiVersion: this.options.azureApiVersion || azureOpenAiDefaultApiVersion,
|
||||
defaultHeaders: {
|
||||
...externalHeaders,
|
||||
...this.options.openAiHeaders,
|
||||
},
|
||||
fetch,
|
||||
})
|
||||
} else {
|
||||
this.client = new AzureOpenAI({
|
||||
baseURL: this.options.openAiBaseUrl,
|
||||
apiKey: this.options.openAiApiKey,
|
||||
apiVersion: this.options.azureApiVersion || azureOpenAiDefaultApiVersion,
|
||||
defaultHeaders: {
|
||||
...externalHeaders,
|
||||
...this.options.openAiHeaders,
|
||||
},
|
||||
fetch,
|
||||
})
|
||||
}
|
||||
} else {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: this.options.openAiBaseUrl,
|
||||
apiKey: this.options.openAiApiKey,
|
||||
defaultHeaders: this.options.openAiHeaders,
|
||||
})
|
||||
}
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating OpenAI client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const modelId = this.options.openAiModelId ?? ""
|
||||
const isDeepseekReasoner = modelId.includes("deepseek-reasoner")
|
||||
const isR1FormatRequired = this.options.openAiModelInfo?.isR1FormatRequired ?? false
|
||||
const isReasoningModelFamily =
|
||||
["o1", "o3", "o4", "gpt-5"].some((prefix) => modelId.includes(prefix)) && !modelId.includes("chat")
|
||||
|
||||
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
let temperature: number | undefined
|
||||
if (this.options.openAiModelInfo?.temperature !== undefined) {
|
||||
const tempValue = Number(this.options.openAiModelInfo.temperature)
|
||||
temperature = tempValue === 0 ? undefined : tempValue
|
||||
} else {
|
||||
temperature = openAiModelInfoSaneDefaults.temperature
|
||||
}
|
||||
let reasoningEffort: ChatCompletionReasoningEffort | undefined
|
||||
let maxTokens: number | undefined
|
||||
|
||||
if (this.options.openAiModelInfo?.maxTokens && this.options.openAiModelInfo.maxTokens > 0) {
|
||||
maxTokens = Number(this.options.openAiModelInfo.maxTokens)
|
||||
} else {
|
||||
maxTokens = undefined
|
||||
}
|
||||
|
||||
if (isDeepseekReasoner || isR1FormatRequired) {
|
||||
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
|
||||
}
|
||||
|
||||
if (isReasoningModelFamily) {
|
||||
openAiMessages = [{ role: "developer", content: systemPrompt }, ...convertToOpenAiMessages(messages)]
|
||||
temperature = undefined // does not support temperature
|
||||
const requestedEffort = normalizeOpenaiReasoningEffort(this.options.reasoningEffort)
|
||||
reasoningEffort = requestedEffort === "none" ? undefined : (requestedEffort as ChatCompletionReasoningEffort)
|
||||
}
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: modelId,
|
||||
messages: openAiMessages,
|
||||
temperature,
|
||||
max_tokens: maxTokens,
|
||||
reasoning_effort: reasoningEffort,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
...getOpenAIToolParams(tools),
|
||||
})
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
|
||||
// @ts-expect-error-next-line
|
||||
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
return {
|
||||
id: this.options.openAiModelId ?? "",
|
||||
info: this.options.openAiModelInfo ?? openAiModelInfoSaneDefaults,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,235 +0,0 @@
|
||||
import { setTimeout as setTimeoutPromise } from "node:timers/promises"
|
||||
import { StateManager } from "@core/storage/StateManager"
|
||||
import { ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
|
||||
import { shouldSkipReasoningForModel } from "@utils/model-utils"
|
||||
import axios from "axios"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient, getAxiosSettings } from "@/shared/net"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { createOpenRouterStream } from "../transform/openrouter-stream"
|
||||
import { ApiStream, ApiStreamUsageChunk } from "../transform/stream"
|
||||
import { ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
import { OpenRouterErrorResponse } from "./types"
|
||||
|
||||
interface OpenRouterHandlerOptions extends CommonApiHandlerOptions {
|
||||
openRouterApiKey?: string
|
||||
openRouterModelId?: string
|
||||
openRouterModelInfo?: ModelInfo
|
||||
openRouterProviderSorting?: string
|
||||
reasoningEffort?: string
|
||||
thinkingBudgetTokens?: number
|
||||
enableParallelToolCalling?: boolean
|
||||
}
|
||||
|
||||
export class OpenRouterHandler implements ApiHandler {
|
||||
private options: OpenRouterHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
lastGenerationId?: string
|
||||
|
||||
constructor(options: OpenRouterHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.openRouterApiKey) {
|
||||
throw new Error("OpenRouter API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://openrouter.ai/api/v1",
|
||||
apiKey: this.options.openRouterApiKey,
|
||||
defaultHeaders: {
|
||||
"HTTP-Referer": "https://cline.bot", // Optional, for including your app on openrouter.ai rankings.
|
||||
"X-Title": "Cline", // Optional. Shows in rankings on openrouter.ai.
|
||||
},
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating OpenRouter client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
this.lastGenerationId = undefined
|
||||
|
||||
const stream = await createOpenRouterStream(
|
||||
client,
|
||||
systemPrompt,
|
||||
messages,
|
||||
this.getModel(),
|
||||
this.options.reasoningEffort,
|
||||
this.options.thinkingBudgetTokens,
|
||||
this.options.openRouterProviderSorting,
|
||||
tools,
|
||||
this.options.enableParallelToolCalling,
|
||||
)
|
||||
|
||||
let didOutputUsage = false
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
// openrouter returns an error object instead of the openai sdk throwing an error
|
||||
// Check for error field directly on chunk
|
||||
if ("error" in chunk) {
|
||||
const error = chunk.error as OpenRouterErrorResponse["error"]
|
||||
Logger.error(`OpenRouter API Error: ${error?.code} - ${error?.message}`)
|
||||
// Include metadata in the error message if available
|
||||
const metadataStr = error.metadata ? `\nMetadata: ${JSON.stringify(error.metadata, null, 2)}` : ""
|
||||
throw new Error(`OpenRouter API Error ${error.code}: ${error.message}${metadataStr}`)
|
||||
}
|
||||
|
||||
// Check for error in choices[0].finish_reason
|
||||
// OpenRouter may return errors in a non-standard way within choices
|
||||
const choice = chunk.choices?.[0]
|
||||
// Use type assertion since OpenRouter uses non-standard "error" finish_reason
|
||||
if ((choice?.finish_reason as string) === "error") {
|
||||
// Use type assertion since OpenRouter adds non-standard error property
|
||||
const choiceWithError = choice as any
|
||||
if (choiceWithError.error) {
|
||||
const error = choiceWithError.error
|
||||
Logger.error(
|
||||
`OpenRouter Mid-Stream Error: ${error?.code || "Unknown"} - ${error?.message || "Unknown error"}`,
|
||||
)
|
||||
// Format error details
|
||||
const errorDetails = typeof error === "object" ? JSON.stringify(error, null, 2) : String(error)
|
||||
throw new Error(`OpenRouter Mid-Stream Error: ${errorDetails}`)
|
||||
}
|
||||
// Fallback if error details are not available
|
||||
throw new Error(`OpenRouter Mid-Stream Error: Stream terminated with error status but no error details provided`)
|
||||
}
|
||||
|
||||
if (!this.lastGenerationId && chunk.id) {
|
||||
this.lastGenerationId = chunk.id
|
||||
}
|
||||
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
// Reasoning tokens are returned separately from the content
|
||||
// Skip reasoning content for Grok 4 models since it only displays "thinking" without providing useful information
|
||||
if (
|
||||
delta &&
|
||||
"reasoning" in delta &&
|
||||
delta.reasoning &&
|
||||
!shouldSkipReasoningForModel(this.options.openRouterModelId)
|
||||
) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
|
||||
}
|
||||
}
|
||||
|
||||
// OpenRouter passes reasoning details that we can pass back unmodified in api requests to preserve reasoning traces for model
|
||||
// See: https://openrouter.ai/docs/use-cases/reasoning-tokens#preserving-reasoning-blocks
|
||||
if (
|
||||
delta &&
|
||||
"reasoning_details" in delta &&
|
||||
delta.reasoning_details &&
|
||||
// @ts-expect-error-next-line
|
||||
delta.reasoning_details.length && // exists and non-0
|
||||
!shouldSkipReasoningForModel(this.options.openRouterModelId)
|
||||
) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: "",
|
||||
details: delta.reasoning_details,
|
||||
}
|
||||
}
|
||||
|
||||
if (!didOutputUsage && chunk.usage) {
|
||||
// @ts-expect-error-next-line -- OpenRouter returns cache_write_tokens for Anthropic models
|
||||
const cacheWriteTokens = chunk.usage.prompt_tokens_details?.cache_write_tokens || 0
|
||||
yield {
|
||||
type: "usage",
|
||||
cacheWriteTokens,
|
||||
cacheReadTokens: chunk.usage.prompt_tokens_details?.cached_tokens || 0,
|
||||
inputTokens:
|
||||
(chunk.usage.prompt_tokens || 0) -
|
||||
(chunk.usage.prompt_tokens_details?.cached_tokens || 0) -
|
||||
(cacheWriteTokens || 0),
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
// @ts-expect-error-next-line
|
||||
totalCost: (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0),
|
||||
}
|
||||
didOutputUsage = true
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback to generation endpoint if usage chunk not returned
|
||||
if (!didOutputUsage) {
|
||||
const apiStreamUsage = await this.getApiStreamUsage()
|
||||
if (apiStreamUsage) {
|
||||
yield apiStreamUsage
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async getApiStreamUsage(): Promise<ApiStreamUsageChunk | undefined> {
|
||||
if (this.lastGenerationId) {
|
||||
await setTimeoutPromise(500) // FIXME: necessary delay to ensure generation endpoint is ready
|
||||
try {
|
||||
const generationIterator = this.fetchGenerationDetails(this.lastGenerationId)
|
||||
const generation = (await generationIterator.next()).value
|
||||
// Logger.log("OpenRouter generation details:", generation)
|
||||
return {
|
||||
type: "usage",
|
||||
cacheWriteTokens: generation?.native_tokens_cache_write || 0,
|
||||
cacheReadTokens: generation?.native_tokens_cached || 0,
|
||||
// openrouter generation endpoint fails often
|
||||
inputTokens: (generation?.native_tokens_prompt || 0) - (generation?.native_tokens_cached || 0),
|
||||
outputTokens: generation?.native_tokens_completion || 0,
|
||||
totalCost: generation?.total_cost || 0,
|
||||
}
|
||||
} catch (error) {
|
||||
// ignore if fails
|
||||
Logger.error("Error fetching OpenRouter generation details:", error)
|
||||
}
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
@withRetry({ maxRetries: 4, baseDelay: 250, maxDelay: 1000, retryAllErrors: true })
|
||||
async *fetchGenerationDetails(genId: string) {
|
||||
// Logger.log("Fetching generation details for:", genId)
|
||||
try {
|
||||
const response = await axios.get(`https://openrouter.ai/api/v1/generation?id=${genId}`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${this.options.openRouterApiKey}`,
|
||||
},
|
||||
timeout: 15_000, // this request hangs sometimes
|
||||
...getAxiosSettings(),
|
||||
})
|
||||
yield response.data?.data
|
||||
} catch (error) {
|
||||
// ignore if fails
|
||||
Logger.error("Error fetching OpenRouter generation details:", error)
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
const modelId = this.options.openRouterModelId || openRouterDefaultModelId
|
||||
const cachedModelInfo = StateManager.get().getModelInfo("openRouter", modelId)
|
||||
return {
|
||||
id: modelId,
|
||||
info: cachedModelInfo || openRouterDefaultModelInfo,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,286 +0,0 @@
|
||||
import { promises as fs } from "node:fs"
|
||||
import { ModelInfo, QwenCodeModelId, qwenCodeDefaultModelId, qwenCodeModels } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import * as os from "os"
|
||||
import * as path from "path"
|
||||
import { buildExternalBasicHeaders } from "@/services/EnvUtils"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { fetch } from "@/shared/net"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
// --- Constants for Qwen OAuth2 ---
|
||||
const QWEN_OAUTH_BASE_URL = "https://chat.qwen.ai"
|
||||
const QWEN_OAUTH_TOKEN_ENDPOINT = `${QWEN_OAUTH_BASE_URL}/api/v1/oauth2/token`
|
||||
const QWEN_OAUTH_CLIENT_ID = "f0304373b74a44d2b584a3fb70ca9e56"
|
||||
const QWEN_DIR = ".qwen"
|
||||
const QWEN_CREDENTIAL_FILENAME = "oauth_creds.json"
|
||||
|
||||
interface QwenOAuthCredentials {
|
||||
access_token: string
|
||||
refresh_token: string
|
||||
token_type: string
|
||||
expiry_date: number
|
||||
resource_url?: string
|
||||
}
|
||||
|
||||
interface QwenCodeHandlerOptions extends CommonApiHandlerOptions {
|
||||
qwenCodeOauthPath?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
function getQwenCachedCredentialPath(customPath?: string): string {
|
||||
if (customPath) {
|
||||
// Support custom path that starts with ~/ or is absolute
|
||||
if (customPath.startsWith("~/")) {
|
||||
return path.join(os.homedir(), customPath.slice(2))
|
||||
}
|
||||
return path.resolve(customPath)
|
||||
}
|
||||
return path.join(os.homedir(), QWEN_DIR, QWEN_CREDENTIAL_FILENAME)
|
||||
}
|
||||
|
||||
function objectToUrlEncoded(data: Record<string, string>): string {
|
||||
return Object.keys(data)
|
||||
.map((key) => `${encodeURIComponent(key)}=${encodeURIComponent(data[key])}`)
|
||||
.join("&")
|
||||
}
|
||||
|
||||
export class QwenCodeHandler implements ApiHandler {
|
||||
private options: QwenCodeHandlerOptions
|
||||
private credentials: QwenOAuthCredentials | null = null
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: QwenCodeHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
// Create the client instance with dummy key initially
|
||||
// The API key will be updated dynamically via ensureAuthenticated
|
||||
const externalHeaders = buildExternalBasicHeaders()
|
||||
this.client = new OpenAI({
|
||||
apiKey: "dummy-key-will-be-replaced",
|
||||
baseURL: "https://dashscope.aliyuncs.com/compatible-mode/v1",
|
||||
defaultHeaders: externalHeaders,
|
||||
fetch,
|
||||
})
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
private async loadCachedQwenCredentials(): Promise<QwenOAuthCredentials> {
|
||||
try {
|
||||
const keyFile = getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)
|
||||
const credsStr = await fs.readFile(keyFile, "utf-8")
|
||||
return JSON.parse(credsStr)
|
||||
} catch (error) {
|
||||
Logger.error(
|
||||
`Error reading or parsing credentials file at ${getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)}`,
|
||||
)
|
||||
throw new Error(`Failed to load Qwen OAuth credentials: ${error}`)
|
||||
}
|
||||
}
|
||||
|
||||
private async refreshAccessToken(credentials: QwenOAuthCredentials): Promise<QwenOAuthCredentials> {
|
||||
if (!credentials.refresh_token) {
|
||||
throw new Error("No refresh token available in credentials.")
|
||||
}
|
||||
|
||||
const bodyData = {
|
||||
grant_type: "refresh_token",
|
||||
refresh_token: credentials.refresh_token,
|
||||
client_id: QWEN_OAUTH_CLIENT_ID,
|
||||
}
|
||||
|
||||
const response = await fetch(QWEN_OAUTH_TOKEN_ENDPOINT, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/x-www-form-urlencoded",
|
||||
Accept: "application/json",
|
||||
},
|
||||
body: objectToUrlEncoded(bodyData),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(`Token refresh failed: ${response.status} ${response.statusText}. Response: ${errorText}`)
|
||||
}
|
||||
|
||||
const tokenData = await response.json()
|
||||
|
||||
if (tokenData.error) {
|
||||
throw new Error(`Token refresh failed: ${tokenData.error} - ${tokenData.error_description}`)
|
||||
}
|
||||
|
||||
const newCredentials = {
|
||||
...credentials,
|
||||
access_token: tokenData.access_token,
|
||||
token_type: tokenData.token_type,
|
||||
refresh_token: tokenData.refresh_token || credentials.refresh_token,
|
||||
expiry_date: Date.now() + tokenData.expires_in * 1000,
|
||||
}
|
||||
|
||||
const filePath = getQwenCachedCredentialPath(this.options.qwenCodeOauthPath)
|
||||
await fs.writeFile(filePath, JSON.stringify(newCredentials, null, 2))
|
||||
|
||||
return newCredentials
|
||||
}
|
||||
|
||||
private isTokenValid(credentials: QwenOAuthCredentials): boolean {
|
||||
const TOKEN_REFRESH_BUFFER_MS = 30 * 1000 // 30s buffer
|
||||
if (!credentials.expiry_date) {
|
||||
return false
|
||||
}
|
||||
return Date.now() < credentials.expiry_date - TOKEN_REFRESH_BUFFER_MS
|
||||
}
|
||||
|
||||
private async ensureAuthenticated(): Promise<void> {
|
||||
if (!this.credentials) {
|
||||
this.credentials = await this.loadCachedQwenCredentials()
|
||||
}
|
||||
|
||||
if (!this.isTokenValid(this.credentials)) {
|
||||
this.credentials = await this.refreshAccessToken(this.credentials)
|
||||
}
|
||||
|
||||
// After authentication, update the apiKey and baseURL on the existing client
|
||||
const client = this.ensureClient()
|
||||
client.apiKey = this.credentials.access_token
|
||||
client.baseURL = this.getBaseUrl(this.credentials)
|
||||
}
|
||||
|
||||
private getBaseUrl(creds: QwenOAuthCredentials): string {
|
||||
let baseUrl = creds.resource_url || "https://dashscope.aliyuncs.com/compatible-mode/v1"
|
||||
if (!baseUrl.startsWith("http://") && !baseUrl.startsWith("https://")) {
|
||||
baseUrl = `https://${baseUrl}`
|
||||
}
|
||||
return baseUrl.endsWith("/v1") ? baseUrl : `${baseUrl}/v1`
|
||||
}
|
||||
|
||||
private async callApiWithRetry<T>(apiCall: () => Promise<T>): Promise<T> {
|
||||
try {
|
||||
return await apiCall()
|
||||
} catch (error: any) {
|
||||
if (error.status === 401) {
|
||||
// Token expired, refresh and retry
|
||||
this.credentials = await this.refreshAccessToken(this.credentials!)
|
||||
const client = this.ensureClient()
|
||||
client.apiKey = this.credentials.access_token
|
||||
client.baseURL = this.getBaseUrl(this.credentials)
|
||||
return await apiCall()
|
||||
} else {
|
||||
throw error
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
await this.ensureAuthenticated()
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
|
||||
const systemMessage: OpenAI.Chat.ChatCompletionSystemMessageParam = {
|
||||
role: "system",
|
||||
content: systemPrompt,
|
||||
}
|
||||
|
||||
const convertedMessages = [systemMessage, ...convertToOpenAiMessages(messages)]
|
||||
|
||||
const requestOptions: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming = {
|
||||
model: model.id,
|
||||
temperature: 0,
|
||||
messages: convertedMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
max_completion_tokens: model.info.maxTokens,
|
||||
...getOpenAIToolParams(tools),
|
||||
}
|
||||
|
||||
const stream = await this.callApiWithRetry(() => client.chat.completions.create(requestOptions))
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
let fullContent = ""
|
||||
|
||||
for await (const apiChunk of stream) {
|
||||
const delta = apiChunk.choices[0]?.delta ?? {}
|
||||
|
||||
if (delta.content) {
|
||||
let newText = delta.content
|
||||
if (newText.startsWith(fullContent)) {
|
||||
newText = newText.substring(fullContent.length)
|
||||
}
|
||||
fullContent = delta.content
|
||||
|
||||
if (newText) {
|
||||
// Check for thinking blocks
|
||||
if (newText.includes("<think>") || newText.includes("</think>")) {
|
||||
// Simple parsing for thinking blocks
|
||||
const parts = newText.split(/<\/?think>/g)
|
||||
for (let i = 0; i < parts.length; i++) {
|
||||
if (parts[i]) {
|
||||
if (i % 2 === 0) {
|
||||
// Outside thinking block
|
||||
yield {
|
||||
type: "text",
|
||||
text: parts[i],
|
||||
}
|
||||
} else {
|
||||
// Inside thinking block
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: parts[i],
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
yield {
|
||||
type: "text",
|
||||
text: newText,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
// Handle reasoning content (o1-style)
|
||||
if ("reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (apiChunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: apiChunk.usage.prompt_tokens || 0,
|
||||
outputTokens: apiChunk.usage.completion_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: QwenCodeModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in qwenCodeModels) {
|
||||
const id = modelId as QwenCodeModelId
|
||||
return { id, info: qwenCodeModels[id] }
|
||||
}
|
||||
return {
|
||||
id: qwenCodeDefaultModelId,
|
||||
info: qwenCodeModels[qwenCodeDefaultModelId],
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,163 +0,0 @@
|
||||
import {
|
||||
InternationalQwenModelId,
|
||||
internationalQwenDefaultModelId,
|
||||
internationalQwenModels,
|
||||
MainlandQwenModelId,
|
||||
ModelInfo,
|
||||
mainlandQwenDefaultModelId,
|
||||
mainlandQwenModels,
|
||||
QwenApiRegions,
|
||||
} from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { convertToR1Format } from "../transform/r1-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface QwenHandlerOptions extends CommonApiHandlerOptions {
|
||||
qwenApiKey?: string
|
||||
qwenApiLine?: QwenApiRegions
|
||||
apiModelId?: string
|
||||
thinkingBudgetTokens?: number
|
||||
}
|
||||
|
||||
export class QwenHandler implements ApiHandler {
|
||||
private options: QwenHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: QwenHandlerOptions) {
|
||||
// Ensure options start with defaults but allow overrides
|
||||
this.options = {
|
||||
qwenApiLine: QwenApiRegions.CHINA,
|
||||
...options,
|
||||
}
|
||||
}
|
||||
|
||||
private useChinaApi(): boolean {
|
||||
return this.options.qwenApiLine === QwenApiRegions.CHINA
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.qwenApiKey) {
|
||||
throw new Error("Alibaba API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: this.useChinaApi()
|
||||
? "https://dashscope.aliyuncs.com/compatible-mode/v1"
|
||||
: "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
|
||||
apiKey: this.options.qwenApiKey,
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating Alibaba client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
getModel(): { id: MainlandQwenModelId | InternationalQwenModelId; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
// Branch based on API line to let poor typescript know what to do
|
||||
if (this.useChinaApi()) {
|
||||
const id = modelId && modelId in mainlandQwenModels ? (modelId as MainlandQwenModelId) : mainlandQwenDefaultModelId
|
||||
return {
|
||||
id,
|
||||
info: mainlandQwenModels[id],
|
||||
}
|
||||
}
|
||||
const id =
|
||||
modelId && modelId in internationalQwenModels
|
||||
? (modelId as InternationalQwenModelId)
|
||||
: internationalQwenDefaultModelId
|
||||
return {
|
||||
id,
|
||||
info: internationalQwenModels[id],
|
||||
}
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
const isDeepseekReasoner = model.id.includes("deepseek-r1")
|
||||
const isReasoningModelFamily = model.id.includes("qwen3") || ["qwen-plus-latest", "qwen-turbo-latest"].includes(model.id)
|
||||
|
||||
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
let temperature: number | undefined = 0
|
||||
// Configuration for extended thinking
|
||||
const budgetTokens = this.options.thinkingBudgetTokens || 0
|
||||
const reasoningOn = budgetTokens !== 0
|
||||
const thinkingArgs = isReasoningModelFamily
|
||||
? {
|
||||
enable_thinking: reasoningOn,
|
||||
thinking_budget: reasoningOn ? budgetTokens : undefined,
|
||||
}
|
||||
: undefined
|
||||
|
||||
if (isDeepseekReasoner || (reasoningOn && isReasoningModelFamily)) {
|
||||
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
|
||||
temperature = undefined
|
||||
}
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
max_completion_tokens: model.info.maxTokens,
|
||||
messages: openAiMessages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
temperature,
|
||||
...thinkingArgs,
|
||||
...getOpenAIToolParams(tools),
|
||||
})
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
try {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
} catch (error) {
|
||||
Logger.error("Error processing tool call delta:", error, delta.tool_calls)
|
||||
}
|
||||
}
|
||||
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
// @ts-expect-error-next-line
|
||||
cacheReadTokens: chunk.usage.prompt_cache_hit_tokens || 0,
|
||||
// @ts-expect-error-next-line
|
||||
cacheWriteTokens: chunk.usage.prompt_cache_miss_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,162 +0,0 @@
|
||||
import { ModelInfo, requestyDefaultModelId, requestyDefaultModelInfo } from "@shared/api"
|
||||
import { isClaudeOpusAdaptiveThinkingModel, resolveClaudeOpusAdaptiveThinking } from "@shared/utils/reasoning-support"
|
||||
import { calculateApiCostOpenAI } from "@utils/cost"
|
||||
import OpenAI from "openai"
|
||||
import { toRequestyServiceStringUrl } from "@/shared/clients/requesty"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
|
||||
interface RequestyHandlerOptions extends CommonApiHandlerOptions {
|
||||
requestyBaseUrl?: string
|
||||
requestyApiKey?: string
|
||||
reasoningEffort?: string
|
||||
thinkingBudgetTokens?: number
|
||||
requestyModelId?: string
|
||||
requestyModelInfo?: ModelInfo
|
||||
}
|
||||
|
||||
// Requesty usage includes an extra field for Anthropic use cases.
|
||||
// Safely cast the prompt token details section to the appropriate structure.
|
||||
interface RequestyUsage extends OpenAI.CompletionUsage {
|
||||
prompt_tokens_details?: {
|
||||
caching_tokens?: number
|
||||
cached_tokens?: number
|
||||
}
|
||||
total_cost?: number
|
||||
}
|
||||
|
||||
export class RequestyHandler implements ApiHandler {
|
||||
private options: RequestyHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: RequestyHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.requestyApiKey) {
|
||||
throw new Error("Requesty API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: toRequestyServiceStringUrl(this.options.requestyBaseUrl),
|
||||
apiKey: this.options.requestyApiKey,
|
||||
defaultHeaders: {
|
||||
"HTTP-Referer": "https://cline.bot",
|
||||
"X-Title": "Cline",
|
||||
},
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating Requesty client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
|
||||
const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
const reasoningEffort = this.options.reasoningEffort || "medium"
|
||||
const reasoning = { reasoning_effort: reasoningEffort }
|
||||
const reasoningArgs = model.id.startsWith("openai/o") ? reasoning : {}
|
||||
|
||||
const thinkingBudget = this.options.thinkingBudgetTokens || 0
|
||||
const isAdaptiveThinkingModel = isClaudeOpusAdaptiveThinkingModel(model.id)
|
||||
const adaptiveThinking = isAdaptiveThinkingModel
|
||||
? resolveClaudeOpusAdaptiveThinking(this.options.reasoningEffort, thinkingBudget)
|
||||
: undefined
|
||||
const thinking =
|
||||
thinkingBudget > 0
|
||||
? { thinking: { type: "enabled", budget_tokens: thinkingBudget } }
|
||||
: { thinking: { type: "disabled" } }
|
||||
const supportsLegacyClaudeThinking =
|
||||
!isAdaptiveThinkingModel &&
|
||||
(model.id.includes("claude-3-7-sonnet") ||
|
||||
model.id.includes("claude-4.6-sonnet") ||
|
||||
model.id.includes("claude-sonnet-4") ||
|
||||
model.id.includes("claude-opus-4"))
|
||||
const thinkingArgs = isAdaptiveThinkingModel
|
||||
? adaptiveThinking?.enabled
|
||||
? {
|
||||
thinking: { type: "adaptive" },
|
||||
...(adaptiveThinking.effort ? { output_config: { effort: adaptiveThinking.effort } } : {}),
|
||||
}
|
||||
: {}
|
||||
: supportsLegacyClaudeThinking
|
||||
? thinking
|
||||
: {}
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: model.id,
|
||||
max_tokens: model.info.maxTokens || undefined,
|
||||
messages: openAiMessages,
|
||||
...(isAdaptiveThinkingModel ? {} : { temperature: 0 }),
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
...reasoningArgs,
|
||||
...thinkingArgs,
|
||||
})
|
||||
|
||||
let lastUsage: any
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
lastUsage = chunk.usage
|
||||
}
|
||||
}
|
||||
|
||||
if (lastUsage) {
|
||||
const usage = lastUsage as RequestyUsage
|
||||
const inputTokens = usage.prompt_tokens || 0
|
||||
const outputTokens = usage.completion_tokens || 0
|
||||
const cacheWriteTokens = usage.prompt_tokens_details?.caching_tokens || undefined
|
||||
const cacheReadTokens = usage.prompt_tokens_details?.cached_tokens || undefined
|
||||
const totalCost = calculateApiCostOpenAI(model.info, inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens)
|
||||
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: inputTokens,
|
||||
outputTokens: outputTokens,
|
||||
cacheWriteTokens: cacheWriteTokens,
|
||||
cacheReadTokens: cacheReadTokens,
|
||||
totalCost: totalCost,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
const modelId = this.options.requestyModelId
|
||||
const modelInfo = this.options.requestyModelInfo
|
||||
if (modelId && modelInfo) {
|
||||
return { id: modelId, info: modelInfo }
|
||||
}
|
||||
return { id: requestyDefaultModelId, info: requestyDefaultModelInfo }
|
||||
}
|
||||
}
|
||||
@@ -1,103 +0,0 @@
|
||||
import { ModelInfo, SambanovaModelId, sambanovaDefaultModelId, sambanovaModels } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { convertToR1Format } from "../transform/r1-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface SambanovaHandlerOptions extends CommonApiHandlerOptions {
|
||||
sambanovaApiKey?: string
|
||||
apiModelId?: string
|
||||
}
|
||||
|
||||
export class SambanovaHandler implements ApiHandler {
|
||||
private options: SambanovaHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: SambanovaHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.sambanovaApiKey) {
|
||||
throw new Error("SambaNova API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://api.sambanova.ai/v1",
|
||||
apiKey: this.options.sambanovaApiKey,
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating SambaNova client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const model = this.getModel()
|
||||
|
||||
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
const modelId = model.id.toLowerCase()
|
||||
|
||||
if (modelId.includes("deepseek") || modelId.includes("qwen3")) {
|
||||
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
|
||||
}
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
const stream = await client.chat.completions.create({
|
||||
model: this.getModel().id,
|
||||
messages: openAiMessages,
|
||||
temperature: model.info.temperature ?? 0,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
...getOpenAIToolParams(tools),
|
||||
})
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
const modelId = this.options.apiModelId
|
||||
if (modelId && modelId in sambanovaModels) {
|
||||
const id = modelId as SambanovaModelId
|
||||
return { id, info: sambanovaModels[id] }
|
||||
}
|
||||
return {
|
||||
id: sambanovaDefaultModelId,
|
||||
info: sambanovaModels[sambanovaDefaultModelId],
|
||||
}
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,103 +0,0 @@
|
||||
import { ModelInfo, openAiModelInfoSaneDefaults } from "@shared/api"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { convertToOpenAiMessages } from "../transform/openai-format"
|
||||
import { convertToR1Format } from "../transform/r1-format"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { getOpenAIToolParams, ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
|
||||
interface TogetherHandlerOptions extends CommonApiHandlerOptions {
|
||||
togetherApiKey?: string
|
||||
togetherModelId?: string
|
||||
}
|
||||
|
||||
export class TogetherHandler implements ApiHandler {
|
||||
private options: TogetherHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: TogetherHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.togetherApiKey) {
|
||||
throw new Error("Together API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://api.together.xyz/v1",
|
||||
apiKey: this.options.togetherApiKey,
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating Together client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const modelId = this.options.togetherModelId ?? ""
|
||||
const isDeepseekReasoner = modelId.includes("deepseek-reasoner")
|
||||
|
||||
let openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
|
||||
{ role: "system", content: systemPrompt },
|
||||
...convertToOpenAiMessages(messages),
|
||||
]
|
||||
|
||||
if (isDeepseekReasoner) {
|
||||
openAiMessages = convertToR1Format([{ role: "user", content: systemPrompt }, ...messages])
|
||||
}
|
||||
|
||||
const stream = await client.chat.completions.create({
|
||||
model: modelId,
|
||||
messages: openAiMessages,
|
||||
temperature: 0,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
...getOpenAIToolParams(tools),
|
||||
})
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
if (delta && "reasoning_content" in delta && delta.reasoning_content) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: (delta.reasoning_content as string | undefined) || "",
|
||||
}
|
||||
}
|
||||
|
||||
if (chunk.usage) {
|
||||
yield {
|
||||
type: "usage",
|
||||
inputTokens: chunk.usage.prompt_tokens || 0,
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
return {
|
||||
id: this.options.togetherModelId ?? "",
|
||||
info: openAiModelInfoSaneDefaults,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
// For the following openrouter error type sources, see the docs here:
|
||||
// https://openrouter.ai/docs/api-reference/errors
|
||||
|
||||
export interface LanguageModelChatSelector {
|
||||
vendor?: string
|
||||
family?: string
|
||||
version?: string
|
||||
id?: string
|
||||
}
|
||||
|
||||
export type OpenRouterErrorResponse = {
|
||||
error: {
|
||||
message: string
|
||||
code: number
|
||||
metadata?: OpenRouterProviderErrorMetadata | OpenRouterModerationErrorMetadata | Record<string, unknown>
|
||||
}
|
||||
}
|
||||
|
||||
export type OpenRouterProviderErrorMetadata = {
|
||||
provider_name: string // The name of the provider that encountered the error
|
||||
raw: unknown // The raw error from the provider
|
||||
}
|
||||
|
||||
export type OpenRouterModerationErrorMetadata = {
|
||||
reasons: string[] // Why your input was flagged
|
||||
flagged_input: string // The text segment that was flagged, limited to 100 characters. If the flagged input is longer than 100 characters, it will be truncated in the middle and replaced with ...
|
||||
provider_name: string // The name of the provider that requested moderation
|
||||
model_slug: string
|
||||
}
|
||||
@@ -1,164 +0,0 @@
|
||||
import { ModelInfo, openRouterDefaultModelId, openRouterDefaultModelInfo } from "@shared/api"
|
||||
import { shouldSkipReasoningForModel } from "@utils/model-utils"
|
||||
import OpenAI from "openai"
|
||||
import type { ChatCompletionTool as OpenAITool } from "openai/resources/chat/completions"
|
||||
import { ClineStorageMessage } from "@/shared/messages/content"
|
||||
import { createOpenAIClient } from "@/shared/net"
|
||||
import { Logger } from "@/shared/services/Logger"
|
||||
import { ApiHandler, CommonApiHandlerOptions } from "../index"
|
||||
import { withRetry } from "../retry"
|
||||
import { ApiStream } from "../transform/stream"
|
||||
import { ToolCallProcessor } from "../transform/tool-call-processor"
|
||||
import { createVercelAIGatewayStream } from "../transform/vercel-ai-gateway-stream"
|
||||
|
||||
interface VercelAIGatewayHandlerOptions extends CommonApiHandlerOptions {
|
||||
vercelAiGatewayApiKey?: string
|
||||
openRouterModelId?: string
|
||||
openRouterModelInfo?: ModelInfo
|
||||
reasoningEffort?: string
|
||||
thinkingBudgetTokens?: number
|
||||
}
|
||||
|
||||
function getCacheReadTokens(usage: any): number {
|
||||
return usage?.prompt_tokens_details?.cached_tokens || usage?.cache_read_input_tokens || 0
|
||||
}
|
||||
|
||||
function getCacheWriteTokens(usage: any): number {
|
||||
return usage?.prompt_tokens_details?.cache_write_tokens || usage?.cache_creation_input_tokens || 0
|
||||
}
|
||||
|
||||
export class VercelAIGatewayHandler implements ApiHandler {
|
||||
private options: VercelAIGatewayHandlerOptions
|
||||
private client: OpenAI | undefined
|
||||
|
||||
constructor(options: VercelAIGatewayHandlerOptions) {
|
||||
this.options = options
|
||||
}
|
||||
|
||||
private ensureClient(): OpenAI {
|
||||
if (!this.client) {
|
||||
if (!this.options.vercelAiGatewayApiKey) {
|
||||
throw new Error("Vercel AI Gateway API key is required")
|
||||
}
|
||||
try {
|
||||
this.client = createOpenAIClient({
|
||||
baseURL: "https://ai-gateway.vercel.sh/v1",
|
||||
apiKey: this.options.vercelAiGatewayApiKey,
|
||||
defaultHeaders: {
|
||||
"http-referer": "https://cline.bot",
|
||||
"x-title": "Cline",
|
||||
},
|
||||
})
|
||||
} catch (error: any) {
|
||||
throw new Error(`Error creating Vercel AI Gateway client: ${error.message}`)
|
||||
}
|
||||
}
|
||||
return this.client
|
||||
}
|
||||
|
||||
@withRetry()
|
||||
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: OpenAITool[]): ApiStream {
|
||||
const client = this.ensureClient()
|
||||
const modelId = this.getModel().id
|
||||
const modelInfo = this.getModel().info
|
||||
|
||||
try {
|
||||
const stream = await createVercelAIGatewayStream(
|
||||
client,
|
||||
systemPrompt,
|
||||
messages,
|
||||
{ id: modelId, info: modelInfo },
|
||||
this.options.reasoningEffort,
|
||||
this.options.thinkingBudgetTokens,
|
||||
tools,
|
||||
)
|
||||
let didOutputUsage = false
|
||||
|
||||
const toolCallProcessor = new ToolCallProcessor()
|
||||
|
||||
for await (const chunk of stream) {
|
||||
const delta = chunk.choices?.[0]?.delta
|
||||
|
||||
if (delta?.content) {
|
||||
yield {
|
||||
type: "text",
|
||||
text: delta.content,
|
||||
}
|
||||
}
|
||||
|
||||
if (delta?.tool_calls) {
|
||||
yield* toolCallProcessor.processToolCallDeltas(delta.tool_calls)
|
||||
}
|
||||
|
||||
// Reasoning tokens are returned separately from the content
|
||||
// Skip reasoning content for models that don't support it (e.g., devstral, grok-4)
|
||||
if (
|
||||
delta &&
|
||||
"reasoning" in delta &&
|
||||
delta.reasoning &&
|
||||
!shouldSkipReasoningForModel(this.options.openRouterModelId)
|
||||
) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: typeof delta.reasoning === "string" ? delta.reasoning : JSON.stringify(delta.reasoning),
|
||||
}
|
||||
}
|
||||
|
||||
// Reasoning details that can be passed back in API requests to preserve reasoning traces
|
||||
if (
|
||||
delta &&
|
||||
"reasoning_details" in delta &&
|
||||
delta.reasoning_details &&
|
||||
// @ts-expect-error-next-line
|
||||
delta.reasoning_details.length && // exists and non-0
|
||||
!shouldSkipReasoningForModel(this.options.openRouterModelId)
|
||||
) {
|
||||
yield {
|
||||
type: "reasoning",
|
||||
reasoning: "",
|
||||
details: delta.reasoning_details,
|
||||
}
|
||||
}
|
||||
|
||||
if (!didOutputUsage && chunk.usage) {
|
||||
// @ts-expect-error - Vercel AI Gateway extends OpenAI types
|
||||
const totalCost = (chunk.usage.cost || 0) + (chunk.usage.cost_details?.upstream_inference_cost || 0)
|
||||
const cacheReadTokens = getCacheReadTokens(chunk.usage)
|
||||
const cacheWriteTokens = getCacheWriteTokens(chunk.usage)
|
||||
|
||||
yield {
|
||||
type: "usage",
|
||||
cacheWriteTokens,
|
||||
cacheReadTokens,
|
||||
inputTokens: Math.max(0, (chunk.usage.prompt_tokens || 0) - cacheReadTokens - cacheWriteTokens),
|
||||
outputTokens: chunk.usage.completion_tokens || 0,
|
||||
totalCost,
|
||||
}
|
||||
didOutputUsage = true
|
||||
}
|
||||
}
|
||||
|
||||
if (!didOutputUsage) {
|
||||
Logger.warn("Vercel AI Gateway did not provide usage information in stream")
|
||||
}
|
||||
} catch (error: any) {
|
||||
Logger.error("Vercel AI Gateway error details:", error)
|
||||
Logger.error("Error stack:", error.stack)
|
||||
throw new Error(`Vercel AI Gateway error: ${error.message}`)
|
||||
}
|
||||
}
|
||||
|
||||
getModel(): { id: string; info: ModelInfo } {
|
||||
const modelId = this.options.openRouterModelId
|
||||
const modelInfo = this.options.openRouterModelInfo
|
||||
if (modelId && modelInfo) {
|
||||
return { id: modelId, info: modelInfo }
|
||||
}
|
||||
// If we have a model ID but no model info, preserve the selected model ID
|
||||
// and fall back only the metadata to defaults.
|
||||
if (modelId) {
|
||||
return { id: modelId, info: openRouterDefaultModelInfo }
|
||||
}
|
||||
return { id: openRouterDefaultModelId, info: openRouterDefaultModelInfo }
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user