opensource/proxycast
3
0
Fork 0
mirror of https://github.com/aiclientproxy/proxycast.git synced 2026-09-24 23:10:56 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: v0.37.0 - Terminal AI integration and Screenshot Chat

Browse Source 
## Terminal AI Features
- Add Terminal AI panel with command execution support
- Add term_get_scrollback tool for reading terminal output history
- Add terminal tool with command approval workflow
- Add auto-execute mode for trusted commands
- Add duplicate command detection to prevent AI loops
- Improve command response format with clear success markers

## Screenshot Chat (Experimental)
- Add screenshot chat window for quick AI conversations
- Add global shortcut support for screenshot capture
- Add experimental settings page

## Code Quality
- Fix all compiler warnings with proper analysis
- Remove unused imports and dead code
- Add #[allow(dead_code)] for reserved/planned features
- Remove deprecated terminal-plugin from recommendations

## Other Changes
- Update model registry with new capabilities
- Improve config observer system
- Add webview panel position fields (reserved)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
coso
2026-01-09 10:05:54 +08:00
co-authored by Claude Opus 4.5
parent 9dbab22624
commit db8e916e90
87 changed files with 8069 additions and 469 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TERM_SCROLLBACK_TOOL.md
+218
View File
@@ -0,0 +1,218 @@
# Terminal Scrollback Tool 实现文档
## 概述
`term_get_scrollback` 工具允许 AI 读取终端的输出历史，而不是直接执行命令。这是一个**只读工具**，完全避免了 AI 重复执行命令的问题。
## 架构设计
### 后端（Rust）
1. **TermScrollbackTool** (`src-tauri/src/agent/tools/term_scrollback.rs`)
- 实现 `Tool` trait
- 使用响应通道机制（类似 TerminalTool）
- 通过 Tauri 事件系统与前端通信
2. **全局实例管理**
- 使用 `once_cell::sync::Lazy` 创建全局单例
- 在 `setup.rs` 和 `runner.rs` 中初始化 AppHandle
- 提供 `get_term_scrollback_tool()` 获取全局实例
3. **Tauri 命令**
- `agent_term_scrollback_response`: 接收前端的响应
### 前端（TypeScript/React）
1. **API 层** (`src/lib/api/agent.ts`)
- `TermScrollbackRequest`: 请求类型
- `TermScrollbackResponse`: 响应类型
- `sendTermScrollbackResponse()`: 发送响应函数
2. **Hook 层** (`src/components/terminal/ai/useTerminalAI.ts`)
- 监听 `term_get_scrollback_request` 事件
- 读取终端输出历史
- 发送响应给后端
## 工作流程
```
AI Agent
↓ (调用 term_get_scrollback 工具)
TermScrollbackTool
↓ (发送 term_get_scrollback_request 事件)
前端 useTerminalAI
↓ (读取终端输出)
↓ (调用 sendTermScrollbackResponse)
Tauri 命令 agent_term_scrollback_response
↓ (调用 handle_term_scrollback_response)
TermScrollbackTool
↓ (通过响应通道返回结果)
AI Agent (收到终端输出)
```
## 工具参数
```json
{
"session_id": "终端会话 ID",
"line_start": 0, // 可选，起始行号（从 0 开始）
"count": 50 // 可选，读取行数
}
```
## 工具描述（给 AI 的说明）
```
Read terminal output history without executing commands.
This tool allows you to view the terminal's scrollback buffer (output history)
without executing any commands. Use this to:
- Check the results of previously executed commands
- Review terminal output before suggesting next steps
- Understand the current state of the terminal session
Parameters:
- session_id: Terminal session ID (required)
- line_start: Starting line number (optional, default: 0)
- count: Number of lines to read (optional, default: all lines)
Returns the terminal output as plain text.
IMPORTANT: This is a READ-ONLY tool. It does NOT execute commands.
```
## 使用场景
### 场景 1：纯只读模式（推荐）
**配置：**
- 移除 `terminal` 工具
- 只保留 `term_get_scrollback` 工具
**优势：**
- AI 只能读取输出，不能执行命令
- 完全避免重复执行问题
- 用户完全控制命令执行
**工作流程：**
1. 用户手动在终端执行命令
2. AI 使用 `term_get_scrollback` 读取输出
3. AI 根据输出提供建议
4. 用户决定是否执行 AI 的建议
### 场景 2：混合模式（当前实现）
**配置：**
- 保留 `terminal` 工具（需要审批）
- 添加 `term_get_scrollback` 工具
**优势：**
- AI 可以建议命令（需要审批）
- AI 也可以读取历史输出
- 灵活性更高
**工作流程：**
1. AI 建议命令（通过 `terminal` 工具）
2. 用户审批并执行
3. AI 使用 `term_get_scrollback` 读取输出
4. AI 根据输出继续工作
## 测试步骤
### 1. 编译项目
```bash
cargo build --manifest-path src-tauri/Cargo.toml
```
### 2. 启动应用
```bash
npm run tauri dev
```
### 3. 测试工具
1. 打开终端
2. 执行一些命令（例如：`ls`, `pwd`, `echo hello`）
3. 打开 AI 面板
4. 发送消息：`请读取终端的输出历史`
5. AI 应该使用 `term_get_scrollback` 工具读取输出
6. 检查 AI 是否正确显示了终端输出
### 4. 验证日志
**后端日志：**
```
[TermScrollbackTool] 创建全局实例
[TermScrollbackTool] 设置全局 AppHandle
[TermScrollbackTool] AppHandle 设置成功，已验证
[TermScrollbackTool] 请求获取滚动缓冲区: session_id=xxx, request_id=xxx
[TermScrollbackTool] 已发送请求到前端: xxx
[TermScrollbackTool] 收到响应: request_id=xxx, success=true
```
**前端日志：**
```
[useTerminalAI] 收到终端滚动缓冲区请求: {...}
[useTerminalAI] 已发送滚动缓冲区响应: 0-50/100 行
```
## 故障排查
### 问题 1：AppHandle 未设置
**症状：**
```
[TermScrollbackTool] 警告：AppHandle 未设置！工具将无法正常工作
```
**解决方案：**
- 检查 `setup.rs` 和 `runner.rs` 中是否调用了 `set_term_scrollback_tool_app_handle()`
- 确保在工具注册之前设置 AppHandle
### 问题 2：前端未收到事件
**症状：**
- 后端发送了事件，但前端没有日志
**解决方案：**
- 检查 `useTerminalAI` 是否正确监听 `term_get_scrollback_request` 事件
- 确保 `terminalSessionId` 不为空
- 检查浏览器控制台是否有错误
### 问题 3：响应超时
**症状：**
```
[TermScrollbackTool] 请求超时: xxx
```
**解决方案：**
- 检查前端是否正确发送响应
- 增加超时时间：`TermScrollbackTool::new().with_timeout(60)`
- 检查 `getTerminalOutput()` 函数是否正常工作
## 下一步改进
1. **添加过滤功能**
- 支持正则表达式过滤
- 支持关键词搜索
2. **添加格式化选项**
- 支持 ANSI 颜色代码
- 支持纯文本输出
3. **添加缓存机制**
- 缓存最近的输出
- 减少重复读取
4. **添加增量读取**
- 只读取新增的输出
- 支持实时监控
## 参考资料
- [Waveterm Terminal AI](https://github.com/wavetermdev/waveterm)
- [Tauri Event System](https://tauri.app/v1/guides/features/events/)
- [Tokio Oneshot Channel](https://docs.rs/tokio/latest/tokio/sync/oneshot/)
package.json
+2 -1
View File
@@ -1,7 +1,7 @@
{
"name": "proxycast",
"private": true,
"version": "0.36.3",
"version": "0.37.0",
"type": "module",
"repository": {
"type": "git",
@@ -42,6 +42,7 @@
"@tauri-apps/api": "^2.9.1",
"@tauri-apps/plugin-deep-link": "^2.4.5",
"@tauri-apps/plugin-dialog": "^2.4.2",
"@tauri-apps/plugin-global-shortcut": "^2",
"@tauri-apps/plugin-shell": "^2.0.0",
"@types/lodash-es": "^4.17.12",
"@types/styled-components": "^5.1.36",
proxycast.db
View File
src-tauri/Cargo.lock
Generated
+98 -6
View File
@@ -776,7 +776,7 @@ dependencies = [
"block",
"cocoa-foundation",
"core-foundation 0.10.1",
"core-graphics",
"core-graphics 0.24.0",
"foreign-types 0.5.0",
"libc",
"objc",
@@ -791,7 +791,7 @@ dependencies = [
"bitflags 2.10.0",
"block",
"core-foundation 0.10.1",
"core-graphics-types",
"core-graphics-types 0.2.0",
"objc",
]
@@ -882,6 +882,19 @@ version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "core-graphics"
version = "0.22.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2581bbab3b8ffc6fcbd550bf46c355135d16e9ff2a6ea032ad6b9bf1d7efe4fb"
dependencies = [
"bitflags 1.3.2",
"core-foundation 0.9.4",
"core-graphics-types 0.1.3",
"foreign-types 0.3.2",
"libc",
]
[[package]]
name = "core-graphics"
version = "0.24.0"
@@ -890,11 +903,22 @@ checksum = "fa95a34622365fa5bbf40b20b75dba8dfa8c94c734aea8ac9a5ca38af14316f1"
dependencies = [
"bitflags 2.10.0",
"core-foundation 0.10.1",
"core-graphics-types",
"core-graphics-types 0.2.0",
"foreign-types 0.5.0",
"libc",
]
[[package]]
name = "core-graphics-types"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "45390e6114f68f718cc7a830514a96f903cccd70d02a8f6d9f643ac4ba45afaf"
dependencies = [
"bitflags 1.3.2",
"core-foundation 0.9.4",
"libc",
]
[[package]]
name = "core-graphics-types"
version = "0.2.0"
@@ -1947,6 +1971,24 @@ version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280"
[[package]]
name = "global-hotkey"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9247516746aa8e53411a0db9b62b0e24efbcf6a76e0ba73e5a91b512ddabed7"
dependencies = [
"crossbeam-channel",
"keyboard-types",
"objc2",
"objc2-app-kit",
"once_cell",
"serde",
"thiserror 2.0.17",
"windows-sys 0.59.0",
"x11rb",
"xkeysym",
]
[[package]]
name = "gobject-sys"
version = "0.18.0"
@@ -2890,6 +2932,17 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "mouse_position"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "824feb0675ad2ffda7b1da534f394c5779fb39d123b5f376a221d50fad54b3c2"
dependencies = [
"core-graphics 0.22.3",
"winapi",
"x11-dl",
]
[[package]]
name = "moxcms"
version = "0.7.11"
@@ -3854,7 +3907,7 @@ dependencies = [
[[package]]
name = "proxycast"
version = "0.36.1"
version = "0.36.3"
dependencies = [
"anyhow",
"arboard",
@@ -3875,6 +3928,7 @@ dependencies = [
"hex",
"indexmap 2.12.1",
"md5",
"mouse_position",
"notify",
"objc",
"once_cell",
@@ -3903,6 +3957,7 @@ dependencies = [
"tauri-plugin-autostart",
"tauri-plugin-deep-link",
"tauri-plugin-dialog",
"tauri-plugin-global-shortcut",
"tauri-plugin-shell",
"tauri-plugin-single-instance",
"tempfile",
@@ -3919,6 +3974,7 @@ dependencies = [
"uuid",
"whoami",
"winapi",
"window-vibrancy 0.7.1",
"windows 0.56.0",
"winreg 0.52.0",
"zip",
@@ -5127,7 +5183,7 @@ dependencies = [
"bitflags 2.10.0",
"block2",
"core-foundation 0.10.1",
"core-graphics",
"core-graphics 0.24.0",
"crossbeam-channel",
"dispatch",
"dlopen2",
@@ -5234,7 +5290,7 @@ dependencies = [
"url",
"webkit2gtk",
"webview2-com",
"window-vibrancy",
"window-vibrancy 0.6.0",
"windows 0.61.3",
]
@@ -5393,6 +5449,21 @@ dependencies = [
"url",
]
[[package]]
name = "tauri-plugin-global-shortcut"
version = "2.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "424af23c7e88d05e4a1a6fc2c7be077912f8c76bd7900fd50aa2b7cbf5a2c405"
dependencies = [
"global-hotkey",
"log",
"serde",
"serde_json",
"tauri",
"tauri-plugin",
"thiserror 2.0.17",
]
[[package]]
name = "tauri-plugin-shell"
version = "2.3.3"
@@ -6571,6 +6642,21 @@ dependencies = [
"windows-version",
]
[[package]]
name = "window-vibrancy"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "010797bd7c40396fbc59d3105089fed0885fe267a0ef4a0a4646df54e28647f6"
dependencies = [
"objc2",
"objc2-app-kit",
"objc2-core-foundation",
"objc2-foundation",
"raw-window-handle",
"windows-sys 0.60.2",
"windows-version",
]
[[package]]
name = "windows"
version = "0.56.0"
@@ -7252,6 +7338,12 @@ dependencies = [
"rustix",
]
[[package]]
name = "xkeysym"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9cc00251562a284751c9973bace760d86c0276c471b4be569fe6b068ee97a56"
[[package]]
name = "yoke"
version = "0.8.1"
src-tauri/Cargo.toml
+5 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "proxycast"
version = "0.36.3"
version = "0.37.0"
description = "AI API Proxy Desktop App"
authors = ["you"]
edition = "2021"
@@ -15,11 +15,12 @@ crate-type = ["lib", "cdylib", "staticlib"]
tauri-build = { version = "2", features = [] }
[dependencies]
tauri = { version = "2", features = ["tray-icon", "image-png", "unstable"] }
tauri = { version = "2", features = ["tray-icon", "image-png", "unstable", "macos-private-api"] }
tauri-plugin-shell = "2"
tauri-plugin-autostart = "2"
tauri-plugin-dialog = "2"
tauri-plugin-single-instance = "2"
tauri-plugin-global-shortcut = "2"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["full"] }
@@ -72,6 +73,8 @@ ssh2 = "0.9"
openssl = { version = "0.10", features = ["vendored"] }
sysinfo = "0.32"
whoami = "1"
mouse_position = "0.1.4"
window-vibrancy = "0.7.1"
# Platform specific dependencies for browser interceptor
src-tauri/binaries/aster-server-aarch64-apple-darwin
-1
View File
@@ -1 +0,0 @@
/Users/coso/Documents/dev/ai/astercloud/aster/aster-server
src-tauri/capabilities/default.json
+12 -1
View File
@@ -2,7 +2,7 @@
"$schema": "https://schemas.tauri.app/config/2/capability",
"identifier": "default",
"description": "Default capabilities for ProxyCast",
"windows": ["main"],
"windows": ["main", "screenshot-chat"],
"permissions": [
"core:default",
"core:webview:default",
@@ -11,12 +11,23 @@
"core:webview:allow-webview-size",
"core:webview:allow-set-webview-position",
"core:webview:allow-set-webview-size",
"core:window:default",
"core:window:allow-close",
"core:window:allow-show",
"core:window:allow-hide",
"core:window:allow-set-focus",
"core:window:allow-center",
"core:window:allow-start-dragging",
"shell:allow-open",
"shell:allow-spawn",
"shell:allow-execute",
"shell:allow-kill",
"shell:allow-stdin-write",
"dialog:default",
"global-shortcut:default",
"global-shortcut:allow-is-registered",
"global-shortcut:allow-register",
"global-shortcut:allow-unregister",
{
"identifier": "shell:allow-execute",
"allow": [
src-tauri/gen/schemas/acl-manifests.json
+1 -1
View File
File diff suppressed because one or more lines are too long
src-tauri/gen/schemas/capabilities.json
+1 -1
View File
@@ -1 +1 @@
{"default":{"identifier":"default","description":"Default capabilities for ProxyCast","local":true,"windows":["main"],"permissions":["core:default","core:webview:default","core:webview:allow-webview-close","core:webview:allow-webview-position","core:webview:allow-webview-size","core:webview:allow-set-webview-position","core:webview:allow-set-webview-size","shell:allow-open","shell:allow-spawn","shell:allow-execute","shell:allow-kill","shell:allow-stdin-write","dialog:default",{"identifier":"shell:allow-execute","allow":[{"args":true,"name":"binaries/aster-server","sidecar":true}]},{"identifier":"shell:allow-spawn","allow":[{"args":true,"name":"binaries/aster-server","sidecar":true}]}]}}
{"default":{"identifier":"default","description":"Default capabilities for ProxyCast","local":true,"windows":["main","screenshot-chat"],"permissions":["core:default","core:webview:default","core:webview:allow-webview-close","core:webview:allow-webview-position","core:webview:allow-webview-size","core:webview:allow-set-webview-position","core:webview:allow-set-webview-size","core:window:default","core:window:allow-close","core:window:allow-show","core:window:allow-hide","core:window:allow-set-focus","core:window:allow-center","core:window:allow-start-dragging","shell:allow-open","shell:allow-spawn","shell:allow-execute","shell:allow-kill","shell:allow-stdin-write","dialog:default","global-shortcut:default","global-shortcut:allow-is-registered","global-shortcut:allow-register","global-shortcut:allow-unregister",{"identifier":"shell:allow-execute","allow":[{"args":true,"name":"binaries/aster-server","sidecar":true}]},{"identifier":"shell:allow-spawn","allow":[{"args":true,"name":"binaries/aster-server","sidecar":true}]}]}}
src-tauri/gen/schemas/desktop-schema.json
+120
View File
@@ -2396,6 +2396,60 @@
"const": "core:window:deny-unminimize",
"markdownDescription": "Denies the unminimize command without any pre-configured scope."
},
{
"description": "Allows reading the opened deep link via the get_current command\n#### This default permission set includes:\n\n- `allow-get-current`",
"type": "string",
"const": "deep-link:default",
"markdownDescription": "Allows reading the opened deep link via the get_current command\n#### This default permission set includes:\n\n- `allow-get-current`"
},
{
"description": "Enables the get_current command without any pre-configured scope.",
"type": "string",
"const": "deep-link:allow-get-current",
"markdownDescription": "Enables the get_current command without any pre-configured scope."
},
{
"description": "Enables the is_registered command without any pre-configured scope.",
"type": "string",
"const": "deep-link:allow-is-registered",
"markdownDescription": "Enables the is_registered command without any pre-configured scope."
},
{
"description": "Enables the register command without any pre-configured scope.",
"type": "string",
"const": "deep-link:allow-register",
"markdownDescription": "Enables the register command without any pre-configured scope."
},
{
"description": "Enables the unregister command without any pre-configured scope.",
"type": "string",
"const": "deep-link:allow-unregister",
"markdownDescription": "Enables the unregister command without any pre-configured scope."
},
{
"description": "Denies the get_current command without any pre-configured scope.",
"type": "string",
"const": "deep-link:deny-get-current",
"markdownDescription": "Denies the get_current command without any pre-configured scope."
},
{
"description": "Denies the is_registered command without any pre-configured scope.",
"type": "string",
"const": "deep-link:deny-is-registered",
"markdownDescription": "Denies the is_registered command without any pre-configured scope."
},
{
"description": "Denies the register command without any pre-configured scope.",
"type": "string",
"const": "deep-link:deny-register",
"markdownDescription": "Denies the register command without any pre-configured scope."
},
{
"description": "Denies the unregister command without any pre-configured scope.",
"type": "string",
"const": "deep-link:deny-unregister",
"markdownDescription": "Denies the unregister command without any pre-configured scope."
},
{
"description": "This permission set configures the types of dialogs\navailable from the dialog plugin.\n\n#### Granted Permissions\n\nAll dialog types are enabled.\n\n\n\n#### This default permission set includes:\n\n- `allow-ask`\n- `allow-confirm`\n- `allow-message`\n- `allow-save`\n- `allow-open`",
"type": "string",
@@ -2462,6 +2516,72 @@
"const": "dialog:deny-save",
"markdownDescription": "Denies the save command without any pre-configured scope."
},
{
"description": "No features are enabled by default, as we believe\nthe shortcuts can be inherently dangerous and it is\napplication specific if specific shortcuts should be\nregistered or unregistered.\n",
"type": "string",
"const": "global-shortcut:default",
"markdownDescription": "No features are enabled by default, as we believe\nthe shortcuts can be inherently dangerous and it is\napplication specific if specific shortcuts should be\nregistered or unregistered.\n"
},
{
"description": "Enables the is_registered command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-is-registered",
"markdownDescription": "Enables the is_registered command without any pre-configured scope."
},
{
"description": "Enables the register command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-register",
"markdownDescription": "Enables the register command without any pre-configured scope."
},
{
"description": "Enables the register_all command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-register-all",
"markdownDescription": "Enables the register_all command without any pre-configured scope."
},
{
"description": "Enables the unregister command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-unregister",
"markdownDescription": "Enables the unregister command without any pre-configured scope."
},
{
"description": "Enables the unregister_all command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-unregister-all",
"markdownDescription": "Enables the unregister_all command without any pre-configured scope."
},
{
"description": "Denies the is_registered command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-is-registered",
"markdownDescription": "Denies the is_registered command without any pre-configured scope."
},
{
"description": "Denies the register command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-register",
"markdownDescription": "Denies the register command without any pre-configured scope."
},
{
"description": "Denies the register_all command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-register-all",
"markdownDescription": "Denies the register_all command without any pre-configured scope."
},
{
"description": "Denies the unregister command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-unregister",
"markdownDescription": "Denies the unregister command without any pre-configured scope."
},
{
"description": "Denies the unregister_all command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-unregister-all",
"markdownDescription": "Denies the unregister_all command without any pre-configured scope."
},
{
"description": "This permission set configures which\nshell functionality is exposed by default.\n\n#### Granted Permissions\n\nIt allows to use the `open` functionality with a reasonable\nscope pre-configured. It will allow opening `http(s)://`,\n`tel:` and `mailto:` links.\n\n#### This default permission set includes:\n\n- `allow-open`",
"type": "string",
src-tauri/gen/schemas/macOS-schema.json
+66
View File
@@ -2516,6 +2516,72 @@
"const": "dialog:deny-save",
"markdownDescription": "Denies the save command without any pre-configured scope."
},
{
"description": "No features are enabled by default, as we believe\nthe shortcuts can be inherently dangerous and it is\napplication specific if specific shortcuts should be\nregistered or unregistered.\n",
"type": "string",
"const": "global-shortcut:default",
"markdownDescription": "No features are enabled by default, as we believe\nthe shortcuts can be inherently dangerous and it is\napplication specific if specific shortcuts should be\nregistered or unregistered.\n"
},
{
"description": "Enables the is_registered command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-is-registered",
"markdownDescription": "Enables the is_registered command without any pre-configured scope."
},
{
"description": "Enables the register command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-register",
"markdownDescription": "Enables the register command without any pre-configured scope."
},
{
"description": "Enables the register_all command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-register-all",
"markdownDescription": "Enables the register_all command without any pre-configured scope."
},
{
"description": "Enables the unregister command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-unregister",
"markdownDescription": "Enables the unregister command without any pre-configured scope."
},
{
"description": "Enables the unregister_all command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:allow-unregister-all",
"markdownDescription": "Enables the unregister_all command without any pre-configured scope."
},
{
"description": "Denies the is_registered command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-is-registered",
"markdownDescription": "Denies the is_registered command without any pre-configured scope."
},
{
"description": "Denies the register command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-register",
"markdownDescription": "Denies the register command without any pre-configured scope."
},
{
"description": "Denies the register_all command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-register-all",
"markdownDescription": "Denies the register_all command without any pre-configured scope."
},
{
"description": "Denies the unregister command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-unregister",
"markdownDescription": "Denies the unregister command without any pre-configured scope."
},
{
"description": "Denies the unregister_all command without any pre-configured scope.",
"type": "string",
"const": "global-shortcut:deny-unregister-all",
"markdownDescription": "Denies the unregister_all command without any pre-configured scope."
},
{
"description": "This permission set configures which\nshell functionality is exposed by default.\n\n#### Granted Permissions\n\nIt allows to use the `open` functionality with a reasonable\nscope pre-configured. It will allow opening `http(s)://`,\n`tel:` and `mailto:` links.\n\n#### This default permission set includes:\n\n- `allow-open`",
"type": "string",
src-tauri/src/agent/native_agent.rs
+17 -2
View File
@@ -19,7 +19,7 @@
use crate::agent::protocols::{create_protocol, Protocol};
use crate::agent::tool_loop::{ToolCallResult, ToolLoopEngine, ToolLoopState};
use crate::agent::tools::{create_default_registry, ToolRegistry};
use crate::agent::tools::{create_default_registry, create_terminal_registry, ToolRegistry};
use crate::agent::types::*;
use crate::models::openai::{
ChatCompletionRequest, ChatCompletionResponse, ChatMessage, ContentPart as OpenAIContentPart,
@@ -813,8 +813,23 @@ impl NativeAgentState {
/// 获取工具注册表
pub fn get_tool_registry(&self) -> Result<Arc<ToolRegistry>, String> {
self.get_tool_registry_with_mode(false)
}
/// 获取工具注册表（支持 Terminal 模式）
///
/// # Arguments
/// * `terminal_mode` - 是否使用 Terminal 模式（使用 TerminalTool 替代 BashTool）
pub fn get_tool_registry_with_mode(
&self,
terminal_mode: bool,
) -> Result<Arc<ToolRegistry>, String> {
let base_dir = dirs::home_dir().ok_or_else(|| "无法获取用户 home 目录".to_string())?;
let registry = create_default_registry(base_dir);
let registry = if terminal_mode {
create_terminal_registry(base_dir)
} else {
create_default_registry(base_dir)
};
Ok(Arc::new(registry))
}
src-tauri/src/agent/tools/mod.rs
+79
View File
@@ -7,6 +7,7 @@
//! - `registry`: 工具注册表和 Tool trait
//! - `security`: 安全管理器（路径验证、符号链接检查等）
//! - `bash`: Bash 命令执行工具
//! - `terminal`: 终端命令执行工具（通过前端审批）
//! - `read_file`: 文件读取工具
//! - `write_file`: 文件写入工具
//! - `edit_file`: 文件编辑工具
@@ -18,6 +19,8 @@ pub mod prompt;
pub mod read_file;
pub mod registry;
pub mod security;
pub mod term_scrollback;
pub mod terminal;
pub mod types;
pub mod write_file;
@@ -27,6 +30,14 @@ pub use prompt::{generate_tools_prompt, PromptFormat, ToolPromptGenerator};
pub use read_file::{ReadFileResult, ReadFileTool};
pub use registry::{Tool, ToolRegistry};
pub use security::{SecurityError, SecurityManager};
pub use term_scrollback::{
get_term_scrollback_tool, handle_term_scrollback_response, set_term_scrollback_tool_app_handle,
GetScrollbackRequest, GetScrollbackResponse, TermScrollbackTool,
};
pub use terminal::{
get_terminal_tool, handle_terminal_command_response, set_terminal_tool_app_handle,
TerminalCommandRequest, TerminalCommandResponse, TerminalTool,
};
pub use types::*;
pub use write_file::{WriteFileResult, WriteFileTool};
@@ -70,3 +81,71 @@ pub fn create_default_registry(base_dir: impl AsRef<Path>) -> ToolRegistry {
registry
}
/// 创建 Terminal AI 模式的工具注册表
///
/// 使用 TerminalTool 替代 BashTool，命令通过前端审批后在用户终端执行
/// 同时添加 TermScrollbackTool 用于只读访问终端输出
///
/// # Arguments
/// * `base_dir` - 基础目录，所有文件操作必须在此目录内
///
/// # Returns
/// 包含 terminal, term_get_scrollback, read_file, write_file, edit_file 工具的注册表
pub fn create_terminal_registry(base_dir: impl AsRef<Path>) -> ToolRegistry {
let security = Arc::new(SecurityManager::new(base_dir.as_ref()));
let registry = ToolRegistry::new();
// 注册 TerminalTool（替代 BashTool）
// 使用全局实例以便前端可以响应
let terminal_tool = get_terminal_tool();
// 验证 AppHandle 是否已设置
if terminal_tool.is_app_handle_set() {
tracing::info!("[Tools] TerminalTool AppHandle 已设置，可以正常使用");
} else {
tracing::error!("[Tools] 警告：TerminalTool AppHandle 未设置！工具将无法正常工作");
eprintln!("[Tools] 警告：TerminalTool AppHandle 未设置！工具将无法正常工作");
}
if let Err(e) = registry.register_arc(terminal_tool) {
tracing::error!("注册 TerminalTool 失败: {}", e);
}
// 注册 TermScrollbackTool（只读访问终端输出）
// 使用全局实例以便前端可以响应
let scrollback_tool = get_term_scrollback_tool();
// 验证 AppHandle 是否已设置
if scrollback_tool.is_app_handle_set() {
tracing::info!("[Tools] TermScrollbackTool AppHandle 已设置，可以正常使用");
} else {
tracing::error!("[Tools] 警告：TermScrollbackTool AppHandle 未设置！工具将无法正常工作");
eprintln!("[Tools] 警告：TermScrollbackTool AppHandle 未设置！工具将无法正常工作");
}
if let Err(e) = registry.register_arc(scrollback_tool) {
tracing::error!("注册 TermScrollbackTool 失败: {}", e);
}
// 注册文件操作工具
if let Err(e) = registry.register(ReadFileTool::new(Arc::clone(&security))) {
tracing::error!("注册 ReadFileTool 失败: {}", e);
}
if let Err(e) = registry.register(WriteFileTool::new(Arc::clone(&security))) {
tracing::error!("注册 WriteFileTool 失败: {}", e);
}
if let Err(e) = registry.register(EditFileTool::new(Arc::clone(&security))) {
tracing::error!("注册 EditFileTool 失败: {}", e);
}
info!(
"[Tools] 已创建 Terminal AI 工具注册表，共 {} 个工具: {:?}",
registry.len(),
registry.list_names()
);
registry
}
src-tauri/src/agent/tools/term_scrollback.rs
+346
View File
@@ -0,0 +1,346 @@
//! Terminal Scrollback 工具模块
//!
//! 提供只读访问终端输出历史的功能
//! 参考 Waveterm 的 term_get_scrollback 工具设计
use super::registry::Tool;
use super::types::{JsonSchema, PropertySchema, ToolDefinition, ToolError, ToolResult};
use async_trait::async_trait;
use parking_lot::RwLock;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::oneshot;
use tokio::time::timeout;
use tracing::{debug, info, warn};
/// 默认超时时间（秒）
const DEFAULT_TIMEOUT_SECS: u64 = 30;
/// 获取滚动缓冲区的请求
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GetScrollbackRequest {
/// 请求 ID
pub request_id: String,
/// 终端会话 ID
pub session_id: String,
/// 起始行（可选，默认从最后 200 行开始）
pub line_start: Option<usize>,
/// 行数（可选，默认 200 行）
pub count: Option<usize>,
}
/// 获取滚动缓冲区的响应
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GetScrollbackResponse {
/// 请求 ID
pub request_id: String,
/// 是否成功
pub success: bool,
/// 总行数
pub total_lines: usize,
/// 起始行
pub line_start: usize,
/// 结束行
pub line_end: usize,
/// 内容
pub content: String,
/// 是否有更多数据
pub has_more: bool,
/// 错误信息
pub error: Option<String>,
}
/// 待处理的请求（内部使用）
pub(crate) struct PendingRequest {
/// 响应发送器
response_tx: oneshot::Sender<GetScrollbackResponse>,
}
/// Terminal Scrollback 工具
///
/// 允许 AI 读取终端的输出历史，而不是直接执行命令
pub struct TermScrollbackTool {
/// 待处理的请求
pending_requests: Arc<RwLock<HashMap<String, PendingRequest>>>,
/// 默认超时时间（秒）
timeout_secs: u64,
/// Tauri AppHandle（用于发送事件）
app_handle: Arc<RwLock<Option<tauri::AppHandle>>>,
}
impl TermScrollbackTool {
/// 创建新的 TermScrollbackTool
pub fn new() -> Self {
Self {
pending_requests: Arc::new(RwLock::new(HashMap::new())),
timeout_secs: DEFAULT_TIMEOUT_SECS,
app_handle: Arc::new(RwLock::new(None)),
}
}
/// 设置超时时间
pub fn with_timeout(mut self, timeout_secs: u64) -> Self {
self.timeout_secs = timeout_secs;
self
}
/// 设置 Tauri AppHandle
pub fn set_app_handle(&self, handle: tauri::AppHandle) {
let mut app_handle = self.app_handle.write();
*app_handle = Some(handle);
eprintln!("[TermScrollbackTool] AppHandle 已设置");
tracing::info!("[TermScrollbackTool] AppHandle 已设置");
}
/// 检查 AppHandle 是否已设置
pub fn is_app_handle_set(&self) -> bool {
let app_handle = self.app_handle.read();
app_handle.is_some()
}
/// 处理响应（由前端调用）
pub fn handle_response(&self, response: GetScrollbackResponse) {
let request_id = response.request_id.clone();
let pending = {
let mut requests = self.pending_requests.write();
requests.remove(&request_id)
};
if let Some(pending) = pending {
if pending.response_tx.send(response).is_err() {
warn!(
"[TermScrollbackTool] 发送响应失败，接收端已关闭: {}",
request_id
);
}
} else {
warn!("[TermScrollbackTool] 未找到待处理的请求: {}", request_id);
}
}
/// 获取终端滚动缓冲区
async fn get_scrollback(
&self,
session_id: &str,
line_start: Option<usize>,
count: Option<usize>,
) -> Result<GetScrollbackResponse, ToolError> {
let request_id = uuid::Uuid::new_v4().to_string();
info!(
"[TermScrollbackTool] 请求获取滚动缓冲区: session_id={}, request_id={}",
session_id, request_id
);
// 创建响应通道
let (response_tx, response_rx) = oneshot::channel();
// 添加到待处理列表
{
let mut requests = self.pending_requests.write();
requests.insert(request_id.clone(), PendingRequest { response_tx });
}
// 构建请求
let request = GetScrollbackRequest {
request_id: request_id.clone(),
session_id: session_id.to_string(),
line_start,
count,
};
// 发送事件到前端
{
let app_handle = self.app_handle.read();
eprintln!(
"[TermScrollbackTool] 检查 AppHandle: is_some={}",
app_handle.is_some()
);
if let Some(handle) = app_handle.as_ref() {
use tauri::Emitter;
eprintln!("[TermScrollbackTool] 尝试发送事件到前端: {}", request_id);
if let Err(e) = handle.emit("term_get_scrollback_request", &request) {
// 清理待处理请求
let mut requests = self.pending_requests.write();
requests.remove(&request_id);
warn!("[TermScrollbackTool] 发送事件到前端失败: {}", e);
return Err(ToolError::ExecutionFailed(format!(
"无法发送请求到前端：{}",
e
)));
}
debug!("[TermScrollbackTool] 已发送请求到前端: {}", request_id);
} else {
// 清理待处理请求
let mut requests = self.pending_requests.write();
requests.remove(&request_id);
eprintln!("[TermScrollbackTool] AppHandle 为 None，无法发送事件");
warn!("[TermScrollbackTool] AppHandle 未设置");
return Err(ToolError::ExecutionFailed(
"TermScrollbackTool 未正确初始化".to_string(),
));
}
}
// 等待响应（带超时）
let timeout_duration = Duration::from_secs(self.timeout_secs);
match timeout(timeout_duration, response_rx).await {
Ok(Ok(response)) => {
debug!(
"[TermScrollbackTool] 收到响应: request_id={}, success={}",
response.request_id, response.success
);
Ok(response)
}
Ok(Err(_)) => {
// 通道关闭
let mut requests = self.pending_requests.write();
requests.remove(&request_id);
Err(ToolError::ExecutionFailed("响应通道已关闭".to_string()))
}
Err(_) => {
// 超时
warn!("[TermScrollbackTool] 请求超时: {}", request_id);
let mut requests = self.pending_requests.write();
requests.remove(&request_id);
Err(ToolError::Timeout)
}
}
}
}
impl Default for TermScrollbackTool {
fn default() -> Self {
Self::new()
}
}
#[async_trait]
impl Tool for TermScrollbackTool {
fn definition(&self) -> ToolDefinition {
ToolDefinition::new(
"term_get_scrollback",
"Get the terminal output history (scrollback buffer). This is a READ-ONLY tool \
that allows you to view what has been output in the terminal.\n\n\
IMPORTANT: This tool does NOT execute commands. It only reads the terminal output. \
Use this to:\n\
- Check the results of commands that the user has run\n\
- View error messages and logs\n\
- Understand the current state of the terminal\n\n\
The user must manually execute commands in their terminal. You can suggest commands \
for the user to run, but you cannot execute them directly with this tool.",
)
.with_parameters(
JsonSchema::new()
.add_property(
"session_id",
PropertySchema::string(
"The terminal session ID to read from. This is provided by the system.",
),
true,
)
.add_property(
"line_start",
PropertySchema::integer(
"Optional starting line number. If not specified, returns the last 200 lines.",
),
false,
)
.add_property(
"count",
PropertySchema::integer(
"Optional number of lines to return. Defaults to 200 lines.",
)
.with_default(serde_json::json!(200)),
false,
),
)
}
async fn execute(&self, args: serde_json::Value) -> Result<ToolResult, ToolError> {
// 解析参数
let session_id = args
.get("session_id")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidArguments("缺少 session_id 参数".to_string()))?;
let line_start = args
.get("line_start")
.and_then(|v| v.as_u64())
.map(|v| v as usize);
let count = args
.get("count")
.and_then(|v| v.as_u64())
.map(|v| v as usize);
// 获取滚动缓冲区
let response = self.get_scrollback(session_id, line_start, count).await?;
// 构建输出
if response.success {
let mut output = format!(
"Terminal output (lines {}-{} of {}):\n\n",
response.line_start, response.line_end, response.total_lines
);
output.push_str(&response.content);
if response.has_more {
output.push_str(
"\n\n[More output available. Use line_start parameter to fetch earlier lines.]",
);
}
Ok(ToolResult::success(output))
} else {
let error_msg = response
.error
.unwrap_or_else(|| "Unknown error".to_string());
Err(ToolError::ExecutionFailed(error_msg))
}
}
}
/// 全局 TermScrollbackTool 实例
static TERM_SCROLLBACK_TOOL: once_cell::sync::Lazy<Arc<TermScrollbackTool>> =
once_cell::sync::Lazy::new(|| {
eprintln!("[TermScrollbackTool] 创建全局实例");
Arc::new(TermScrollbackTool::new())
});
/// 获取全局 TermScrollbackTool 实例
pub fn get_term_scrollback_tool() -> Arc<TermScrollbackTool> {
eprintln!("[TermScrollbackTool] 获取全局实例");
Arc::clone(&TERM_SCROLLBACK_TOOL)
}
/// 设置全局 TermScrollbackTool 的 AppHandle
pub fn set_term_scrollback_tool_app_handle(handle: tauri::AppHandle) {
eprintln!("[TermScrollbackTool] 设置全局 AppHandle");
tracing::info!("[TermScrollbackTool] 设置全局 AppHandle");
// 强制初始化全局实例（如果还未初始化）
let _ = &*TERM_SCROLLBACK_TOOL;
TERM_SCROLLBACK_TOOL.set_app_handle(handle);
// 验证设置是否成功
let app_handle = TERM_SCROLLBACK_TOOL.app_handle.read();
if app_handle.is_some() {
eprintln!("[TermScrollbackTool] AppHandle 设置成功，已验证");
tracing::info!("[TermScrollbackTool] AppHandle 设置成功，已验证");
} else {
eprintln!("[TermScrollbackTool] 警告：AppHandle 设置后仍为 None");
tracing::error!("[TermScrollbackTool] 警告：AppHandle 设置后仍为 None");
}
}
/// 处理滚动缓冲区响应（由 Tauri 命令调用）
pub fn handle_term_scrollback_response(response: GetScrollbackResponse) {
TERM_SCROLLBACK_TOOL.handle_response(response);
}
src-tauri/src/agent/tools/terminal.rs
+499
View File
@@ -0,0 +1,499 @@
//! Terminal 工具模块
//!
//! 提供终端命令执行功能，通过 Tauri 事件与前端通信
//! 支持命令审批流程，命令在用户的实际终端中执行
//!
//! ## 工作流程
//! 1. AI 调用 terminal 工具
//! 2. 工具发送事件到前端请求执行命令
//! 3. 前端显示审批 UI
//! 4. 用户批准后，命令发送到实际终端
//! 5. 终端执行结果返回给 AI
use super::registry::Tool;
use super::types::{JsonSchema, PropertySchema, ToolDefinition, ToolError, ToolResult};
use async_trait::async_trait;
use parking_lot::RwLock;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::oneshot;
use tokio::time::timeout;
use tracing::{debug, info, warn};
/// 默认超时时间（秒）
const DEFAULT_TIMEOUT_SECS: u64 = 120;
/// 命令执行请求
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TerminalCommandRequest {
/// 请求 ID
pub request_id: String,
/// 要执行的命令
pub command: String,
/// 工作目录（可选）
pub working_dir: Option<String>,
/// 超时时间（秒）
pub timeout_secs: u64,
}
/// 命令执行响应
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TerminalCommandResponse {
/// 请求 ID
pub request_id: String,
/// 是否成功
pub success: bool,
/// 输出内容
pub output: String,
/// 错误信息
pub error: Option<String>,
/// 退出码
pub exit_code: Option<i32>,
/// 是否被用户拒绝
pub rejected: bool,
}
/// 命令执行状态
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CommandStatus {
/// 等待审批
Pending,
/// 已批准，执行中
Executing,
/// 已完成
Completed,
/// 已拒绝
Rejected,
/// 超时
Timeout,
}
/// 待处理的命令（内部使用）
pub(crate) struct PendingCommand {
/// 响应发送器
response_tx: oneshot::Sender<TerminalCommandResponse>,
}
/// 已执行命令的记录
#[derive(Debug, Clone)]
struct ExecutedCommand {
/// 命令内容
command: String,
/// 执行时间
executed_at: std::time::Instant,
/// 是否成功
success: bool,
}
/// 重复命令检测的时间窗口（秒）
const DUPLICATE_DETECTION_WINDOW_SECS: u64 = 30;
/// Terminal 工具
///
/// 通过 Tauri 事件与前端通信，在用户终端中执行命令
pub struct TerminalTool {
/// 待处理的命令
pending_commands: Arc<RwLock<HashMap<String, PendingCommand>>>,
/// 已执行的命令历史（用于检测重复）
executed_commands: Arc<RwLock<Vec<ExecutedCommand>>>,
/// 默认超时时间（秒）
timeout_secs: u64,
/// Tauri AppHandle（用于发送事件）
app_handle: Arc<RwLock<Option<tauri::AppHandle>>>,
}
impl TerminalTool {
/// 创建新的 Terminal 工具
pub fn new() -> Self {
Self {
pending_commands: Arc::new(RwLock::new(HashMap::new())),
executed_commands: Arc::new(RwLock::new(Vec::new())),
timeout_secs: DEFAULT_TIMEOUT_SECS,
app_handle: Arc::new(RwLock::new(None)),
}
}
/// 设置超时时间
pub fn with_timeout(mut self, timeout_secs: u64) -> Self {
self.timeout_secs = timeout_secs;
self
}
/// 设置 Tauri AppHandle
pub fn set_app_handle(&self, handle: tauri::AppHandle) {
let mut app_handle = self.app_handle.write();
*app_handle = Some(handle);
eprintln!("[TerminalTool] AppHandle 已设置");
tracing::info!("[TerminalTool] AppHandle 已设置");
}
/// 检查 AppHandle 是否已设置
pub fn is_app_handle_set(&self) -> bool {
let app_handle = self.app_handle.read();
app_handle.is_some()
}
/// 处理命令响应（由前端调用）
pub fn handle_response(&self, response: TerminalCommandResponse) {
let request_id = response.request_id.clone();
let pending = {
let mut commands = self.pending_commands.write();
commands.remove(&request_id)
};
if let Some(pending) = pending {
if pending.response_tx.send(response).is_err() {
warn!("[TerminalTool] 发送响应失败，接收端已关闭: {}", request_id);
}
} else {
warn!("[TerminalTool] 未找到待处理的命令: {}", request_id);
}
}
/// 检查命令是否在最近执行过（用于防止重复执行）
fn check_duplicate_command(&self, command: &str) -> Option<&'static str> {
let now = std::time::Instant::now();
let window = Duration::from_secs(DUPLICATE_DETECTION_WINDOW_SECS);
// 清理过期的记录
{
let mut history = self.executed_commands.write();
history.retain(|cmd| now.duration_since(cmd.executed_at) < window);
}
// 检查是否有重复
let history = self.executed_commands.read();
for cmd in history.iter() {
if cmd.command == command && cmd.success {
return Some(
"This exact command was already executed successfully within the last 30 seconds. \
Do NOT re-execute it. If you need to verify the result, use the term_get_scrollback \
tool to read the terminal output."
);
}
}
None
}
/// 记录已执行的命令
fn record_executed_command(&self, command: &str, success: bool) {
let mut history = self.executed_commands.write();
history.push(ExecutedCommand {
command: command.to_string(),
executed_at: std::time::Instant::now(),
success,
});
// 限制历史记录大小
if history.len() > 100 {
history.remove(0);
}
}
/// 执行命令
async fn execute_command(
&self,
command: &str,
working_dir: Option<&str>,
timeout_secs: u64,
) -> Result<TerminalCommandResponse, ToolError> {
let request_id = uuid::Uuid::new_v4().to_string();
info!(
"[TerminalTool] 请求执行命令: {} (request_id: {}, timeout: {}s)",
command, request_id, timeout_secs
);
// 创建响应通道
let (response_tx, response_rx) = oneshot::channel();
// 添加到待处理列表
{
let mut commands = self.pending_commands.write();
commands.insert(request_id.clone(), PendingCommand { response_tx });
}
// 构建请求
let request = TerminalCommandRequest {
request_id: request_id.clone(),
command: command.to_string(),
working_dir: working_dir.map(|s| s.to_string()),
timeout_secs,
};
// 发送事件到前端
{
let app_handle = self.app_handle.read();
eprintln!(
"[TerminalTool] 检查 AppHandle: is_some={}",
app_handle.is_some()
);
if let Some(handle) = app_handle.as_ref() {
use tauri::Emitter;
eprintln!("[TerminalTool] 尝试发送事件到前端: {}", request_id);
if let Err(e) = handle.emit("terminal_command_request", &request) {
// 清理待处理命令
let mut commands = self.pending_commands.write();
commands.remove(&request_id);
// 返回失败响应而不是错误，避免 Agent 重试
warn!("[TerminalTool] 发送事件到前端失败: {}", e);
return Ok(TerminalCommandResponse {
request_id: request_id.clone(),
success: false,
output: String::new(),
error: Some(format!("无法发送命令到终端：{}。请检查应用配置。", e)),
exit_code: Some(-1),
rejected: false,
});
}
debug!("[TerminalTool] 已发送命令请求到前端: {}", request_id);
} else {
// 清理待处理命令
let mut commands = self.pending_commands.write();
commands.remove(&request_id);
eprintln!("[TerminalTool] AppHandle 为 None，无法发送事件");
// 返回失败响应而不是错误，避免 Agent 重试
warn!("[TerminalTool] AppHandle 未设置");
return Ok(TerminalCommandResponse {
request_id: request_id.clone(),
success: false,
output: String::new(),
error: Some(
"终端工具未正确初始化。这是一个应用程序配置问题，请联系开发者。\n\
作为替代方案，我可以为您提供命令建议，但无法直接执行。"
.to_string(),
),
exit_code: Some(-1),
rejected: false,
});
}
}
// 等待响应（带超时）
let timeout_duration = Duration::from_secs(timeout_secs);
match timeout(timeout_duration, response_rx).await {
Ok(Ok(response)) => {
debug!(
"[TerminalTool] 收到响应: request_id={}, success={}",
response.request_id, response.success
);
Ok(response)
}
Ok(Err(_)) => {
// 通道关闭
let mut commands = self.pending_commands.write();
commands.remove(&request_id);
Err(ToolError::ExecutionFailed("响应通道已关闭".to_string()))
}
Err(_) => {
// 超时
warn!("[TerminalTool] 命令执行超时: {}", request_id);
let mut commands = self.pending_commands.write();
commands.remove(&request_id);
Err(ToolError::Timeout)
}
}
}
}
impl Default for TerminalTool {
fn default() -> Self {
Self::new()
}
}
#[async_trait]
impl Tool for TerminalTool {
fn definition(&self) -> ToolDefinition {
ToolDefinition::new(
"terminal",
"Execute a command in the user's terminal. The command will be sent to the \
active terminal session and requires user approval before execution.\n\n\
CRITICAL RULES:\n\
1. NEVER re-execute a command that has already succeeded. When you receive \
'[COMMAND EXECUTED SUCCESSFULLY]' in the response, the command is DONE.\n\
2. Each command runs exactly ONCE. Do not retry successful commands.\n\
3. If you need to verify the result, use the term_get_scrollback tool to \
read the terminal output instead of re-running the command.\n\n\
Use this for running system commands, scripts, or any command-line operations \
that should be visible to the user.",
)
.with_parameters(
JsonSchema::new()
.add_property(
"command",
PropertySchema::string(
"The command to execute in the terminal. Can be any valid shell command.",
),
true,
)
.add_property(
"working_dir",
PropertySchema::string(
"Optional working directory for the command. If not specified, \
uses the terminal's current directory.",
),
false,
)
.add_property(
"timeout",
PropertySchema::integer(
"Optional timeout in seconds. Defaults to 120 seconds.",
)
.with_default(serde_json::json!(120)),
false,
),
)
}
async fn execute(&self, args: serde_json::Value) -> Result<ToolResult, ToolError> {
// 解析参数
let command = args
.get("command")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidArguments("缺少 command 参数".to_string()))?;
let working_dir = args.get("working_dir").and_then(|v| v.as_str());
let timeout_secs = args
.get("timeout")
.and_then(|v| v.as_u64())
.unwrap_or(self.timeout_secs);
// 检查是否是重复命令
if let Some(duplicate_msg) = self.check_duplicate_command(command) {
warn!("[TerminalTool] 检测到重复命令: {}", command);
return Ok(ToolResult::success(format!(
"[DUPLICATE COMMAND BLOCKED]\n{}\n\nOriginal command: {}",
duplicate_msg, command
)));
}
// 执行命令
let response = self
.execute_command(command, working_dir, timeout_secs)
.await?;
// 记录已执行的命令
self.record_executed_command(command, response.success);
// 构建输出
if response.rejected {
return Ok(ToolResult::failure_with_output(
"用户拒绝执行此命令".to_string(),
"命令被用户拒绝".to_string(),
));
}
if response.success {
Ok(ToolResult::success(response.output))
} else {
let error_msg = response.error.unwrap_or_else(|| {
format!(
"命令执行失败 (退出码: {})",
response.exit_code.unwrap_or(-1)
)
});
Ok(ToolResult::failure_with_output(response.output, error_msg))
}
}
}
/// 全局 TerminalTool 实例
static TERMINAL_TOOL: once_cell::sync::Lazy<Arc<TerminalTool>> = once_cell::sync::Lazy::new(|| {
eprintln!("[TerminalTool] 创建全局实例");
Arc::new(TerminalTool::new())
});
/// 获取全局 TerminalTool 实例
pub fn get_terminal_tool() -> Arc<TerminalTool> {
eprintln!("[TerminalTool] 获取全局实例");
Arc::clone(&TERMINAL_TOOL)
}
/// 设置全局 TerminalTool 的 AppHandle
pub fn set_terminal_tool_app_handle(handle: tauri::AppHandle) {
eprintln!("[TerminalTool] 设置全局 AppHandle");
tracing::info!("[TerminalTool] 设置全局 AppHandle");
// 强制初始化全局实例（如果还未初始化）
let _ = &*TERMINAL_TOOL;
TERMINAL_TOOL.set_app_handle(handle);
// 验证设置是否成功
let app_handle = TERMINAL_TOOL.app_handle.read();
if app_handle.is_some() {
eprintln!("[TerminalTool] AppHandle 设置成功，已验证");
tracing::info!("[TerminalTool] AppHandle 设置成功，已验证");
} else {
eprintln!("[TerminalTool] 警告：AppHandle 设置后仍为 None");
tracing::error!("[TerminalTool] 警告：AppHandle 设置后仍为 None");
}
}
/// 处理终端命令响应（由 Tauri 命令调用）
pub fn handle_terminal_command_response(response: TerminalCommandResponse) {
TERMINAL_TOOL.handle_response(response);
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_tool_definition() {
let tool = TerminalTool::new();
let def = tool.definition();
assert_eq!(def.name, "terminal");
assert!(!def.description.is_empty());
assert!(def.parameters.required.contains(&"command".to_string()));
}
#[test]
fn test_terminal_command_request_serialization() {
let request = TerminalCommandRequest {
request_id: "test-123".to_string(),
command: "echo hello".to_string(),
working_dir: Some("/home/user".to_string()),
timeout_secs: 60,
};
let json = serde_json::to_string(&request).unwrap();
let parsed: TerminalCommandRequest = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.request_id, "test-123");
assert_eq!(parsed.command, "echo hello");
assert_eq!(parsed.working_dir, Some("/home/user".to_string()));
assert_eq!(parsed.timeout_secs, 60);
}
#[test]
fn test_terminal_command_response_serialization() {
let response = TerminalCommandResponse {
request_id: "test-123".to_string(),
success: true,
output: "hello\n".to_string(),
error: None,
exit_code: Some(0),
rejected: false,
};
let json = serde_json::to_string(&response).unwrap();
let parsed: TerminalCommandResponse = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.request_id, "test-123");
assert!(parsed.success);
assert_eq!(parsed.output, "hello\n");
assert_eq!(parsed.exit_code, Some(0));
assert!(!parsed.rejected);
}
}
src-tauri/src/app/runner.rs
+70 -3
View File
@@ -70,7 +70,7 @@ pub fn run() {
native_agent: native_agent_state,
oauth_plugin_manager: oauth_plugin_manager_state,
orchestrator: orchestrator_state,
connect_state: connect_state,
connect_state,
model_registry: model_registry_state,
global_config_manager: global_config_manager_state,
terminal_manager: terminal_manager_state,
@@ -97,6 +97,7 @@ pub fn run() {
let mut builder = tauri::Builder::default()
.plugin(tauri_plugin_shell::init())
.plugin(tauri_plugin_dialog::init())
.plugin(tauri_plugin_global_shortcut::Builder::new().build())
.plugin(tauri_plugin_autostart::init(
tauri_plugin_autostart::MacosLauncher::LaunchAgent,
Some(vec!["--minimized"]),
@@ -179,6 +180,14 @@ pub fn run() {
}
})
.setup(move |app| {
// 设置 TerminalTool 的 AppHandle（用于发送事件到前端）
crate::agent::tools::set_terminal_tool_app_handle(app.handle().clone());
tracing::info!("[启动] TerminalTool AppHandle 已设置");
// 设置 TermScrollbackTool 的 AppHandle（用于发送事件到前端）
crate::agent::tools::set_term_scrollback_tool_app_handle(app.handle().clone());
tracing::info!("[启动] TermScrollbackTool AppHandle 已设置");
// 初始化托盘管理器
// Requirements 1.4: 应用启动时显示停止状态图标
match TrayManager::new(app.handle()) {
@@ -206,6 +215,21 @@ pub fn run() {
tracing::info!("[启动] GlobalConfigManager AppHandle 已设置");
}
// 初始化截图对话模块
// _Requirements: 7.3_
{
let app_handle = app.handle();
match crate::screenshot::init(app_handle) {
Ok(()) => {
tracing::info!("[启动] 截图对话模块初始化成功");
}
Err(e) => {
tracing::error!("[启动] 截图对话模块初始化失败: {}", e);
// 截图模块初始化失败不影响应用运行
}
}
}
// 初始化 Connect 状态
// _Requirements: 1.4, 2.1_
{
@@ -240,7 +264,36 @@ pub fn run() {
let app_handle = app.handle().clone();
let db_clone = db_clone.clone();
// 获取资源目录路径
let resource_dir = app.path().resource_dir().unwrap_or_else(|_| std::path::PathBuf::from("."));
let mut resource_dir = app.path().resource_dir().unwrap_or_else(|_| std::path::PathBuf::from("."));
// 检查 resources/models/index.json 是否存在
let models_index = resource_dir.join("resources/models/index.json");
if !models_index.exists() {
// 开发模式：尝试多个可能的路径
let possible_paths = [
// 从 target/debug 回退到 src-tauri
std::env::current_exe()
.ok()
.and_then(|p| p.parent().map(|p| p.to_path_buf()))
.map(|p| p.join("resources")),
// 直接使用 target/debug/resources
std::env::current_exe()
.ok()
.and_then(|p| p.parent().map(|p| p.to_path_buf())),
// 使用当前工作目录
std::env::current_dir().ok().map(|p| p.join("src-tauri")),
];
for path in possible_paths.into_iter().flatten() {
let test_index = path.join("resources/models/index.json");
if test_index.exists() {
resource_dir = path;
break;
}
}
}
tauri::async_runtime::spawn(async move {
// 创建 ModelRegistryService
let mut service = crate::services::model_registry_service::ModelRegistryService::new(db_clone);
@@ -947,6 +1000,8 @@ pub fn run() {
commands::agent_cmd::agent_list_sessions,
commands::agent_cmd::agent_get_session,
commands::agent_cmd::agent_delete_session,
commands::agent_cmd::agent_terminal_command_response,
commands::agent_cmd::agent_term_scrollback_response,
// Native Agent commands
commands::native_agent_cmd::native_agent_init,
commands::native_agent_cmd::native_agent_status,
@@ -1035,7 +1090,7 @@ pub fn run() {
commands::connect_cmd::send_connect_callback,
// Model Registry commands
commands::model_registry_cmd::get_model_registry,
// commands::model_registry_cmd::refresh_model_registry, // TODO: 暂时禁用
commands::model_registry_cmd::refresh_model_registry,
commands::model_registry_cmd::search_models,
commands::model_registry_cmd::get_model_preferences,
commands::model_registry_cmd::toggle_model_favorite,
@@ -1086,6 +1141,18 @@ pub fn run() {
commands::webview_cmd::resize_webview_panel,
commands::webview_cmd::get_webview_panels,
commands::webview_cmd::focus_webview_panel,
// Screenshot Chat commands
// _Requirements: 1.1, 1.4, 1.5, 2.2, 2.4, 3.1, 5.1_
commands::screenshot_cmd::get_experimental_config,
commands::screenshot_cmd::save_experimental_config,
commands::screenshot_cmd::start_screenshot,
commands::screenshot_cmd::validate_shortcut,
commands::screenshot_cmd::update_screenshot_shortcut,
commands::screenshot_cmd::close_screenshot_chat_window,
commands::screenshot_cmd::read_image_as_base64,
commands::screenshot_cmd::send_screenshot_chat,
commands::screenshot_cmd::close_screenshot_chat_window,
commands::screenshot_cmd::read_image_as_base64,
])
.run(tauri::generate_context!())
.expect("error while running tauri application");
src-tauri/src/app/setup.rs
+9
View File
@@ -5,6 +5,7 @@
use std::sync::Arc;
use tauri::{App, Manager};
use crate::agent::tools::{set_term_scrollback_tool_app_handle, set_terminal_tool_app_handle};
use crate::agent::NativeAgentState;
use crate::commands::oauth_plugin_cmd::OAuthPluginManagerState;
use crate::database;
@@ -52,6 +53,14 @@ pub fn setup_app(
let native_agent_state = NativeAgentState::new();
app.manage(native_agent_state);
// 设置 TerminalTool 的 AppHandle（用于发送事件到前端）
set_terminal_tool_app_handle(app.handle().clone());
tracing::info!("[启动] TerminalTool AppHandle 已设置");
// 设置 TermScrollbackTool 的 AppHandle（用于发送事件到前端）
set_term_scrollback_tool_app_handle(app.handle().clone());
tracing::info!("[启动] TermScrollbackTool AppHandle 已设置");
// 初始化 OAuth Plugin Manager State
let oauth_plugin_manager_state = OAuthPluginManagerState::with_defaults();
app.manage(oauth_plugin_manager_state);
src-tauri/src/browser_interceptor/interceptor.rs
-7
View File
@@ -370,7 +370,6 @@ impl BrowserInterceptor {
}
/// 复制文本到剪贴板
#[cfg(feature = "tauri-app")]
async fn copy_to_clipboard(&self, text: &str) -> Result<()> {
match arboard::Clipboard::new() {
Ok(mut clipboard) => {
@@ -390,12 +389,6 @@ impl BrowserInterceptor {
}
}
#[cfg(not(feature = "tauri-app"))]
async fn copy_to_clipboard(&self, _text: &str) -> Result<()> {
tracing::warn!("剪贴板功能需要 tauri-app feature");
Ok(())
}
/// 启动指纹浏览器
async fn launch_fingerprint_browser(
&self,
src-tauri/src/commands/agent_cmd.rs
+73
View File
@@ -2,6 +2,10 @@
//!
//! 提供原生 Agent 的 Tauri 命令（兼容旧 API）
use crate::agent::tools::{
handle_term_scrollback_response, handle_terminal_command_response, GetScrollbackResponse,
TerminalCommandResponse,
};
use crate::agent::{ImageData, NativeAgentState, NativeChatRequest, ProviderType};
use crate::AppState;
use serde::{Deserialize, Serialize};
@@ -358,3 +362,72 @@ pub async fn agent_delete_session(
Err("会话不存在".to_string())
}
}
/// 处理终端命令响应
///
/// 前端在用户批准/拒绝命令后调用此命令，将结果传递给 TerminalTool
#[tauri::command]
pub async fn agent_terminal_command_response(
request_id: String,
success: bool,
output: String,
error: Option<String>,
exit_code: Option<i32>,
rejected: bool,
) -> Result<(), String> {
tracing::info!(
"[Agent] 收到终端命令响应: request_id={}, success={}, rejected={}",
request_id,
success,
rejected
);
let response = TerminalCommandResponse {
request_id,
success,
output,
error,
exit_code,
rejected,
};
handle_terminal_command_response(response);
Ok(())
}
/// 前端返回终端滚动缓冲区数据
#[tauri::command]
pub async fn agent_term_scrollback_response(
request_id: String,
success: bool,
total_lines: usize,
line_start: usize,
line_end: usize,
content: String,
has_more: bool,
error: Option<String>,
) -> Result<(), String> {
tracing::info!(
"[Agent] 收到终端滚动缓冲区响应: request_id={}, success={}, lines={}-{}",
request_id,
success,
line_start,
line_end
);
let response = GetScrollbackResponse {
request_id,
success,
total_lines,
line_start,
line_end,
content,
has_more,
error,
};
handle_term_scrollback_response(response);
Ok(())
}
src-tauri/src/commands/mod.rs
+1
View File
@@ -24,6 +24,7 @@ pub mod prompt_cmd;
pub mod provider_pool_cmd;
pub mod resilience_cmd;
pub mod route_cmd;
pub mod screenshot_cmd;
pub mod skill_cmd;
pub mod switch_cmd;
pub mod telemetry_cmd;
src-tauri/src/commands/model_registry_cmd.rs
+11
View File
@@ -169,3 +169,14 @@ pub async fn get_all_alias_configs(
Ok(service.get_all_alias_configs().await)
}
/// 刷新模型注册表（强制从内嵌资源重新加载）
#[tauri::command]
pub async fn refresh_model_registry(state: State<'_, ModelRegistryState>) -> Result<u32, String> {
let guard = state.read().await;
let service = guard
.as_ref()
.ok_or_else(|| "模型注册服务未初始化".to_string())?;
service.force_reload().await
}
src-tauri/src/commands/native_agent_cmd.rs
+7 -3
View File
@@ -160,14 +160,17 @@ pub async fn native_agent_chat_stream(
model: Option<String>,
images: Option<Vec<ImageInputParam>>,
provider: Option<String>,
terminal_mode: Option<bool>,
) -> Result<(), String> {
let terminal_mode = terminal_mode.unwrap_or(false);
tracing::info!(
"[NativeAgent] 发送流式消息: message_len={}, model={:?}, provider={:?}, event={}, session={:?}",
"[NativeAgent] 发送流式消息: message_len={}, model={:?}, provider={:?}, event={}, session={:?}, terminal_mode={}",
message.len(),
model,
provider,
event_name,
session_id
session_id,
terminal_mode
);
// 获取配置信息
@@ -241,7 +244,8 @@ pub async fn native_agent_chat_stream(
}
// 获取工具注册表（用于创建 ToolLoopEngine）
let tool_registry = agent_state.get_tool_registry()?;
// 如果是 terminal_mode，使用 TerminalTool 替代 BashTool
let tool_registry = agent_state.get_tool_registry_with_mode(terminal_mode)?;
let request = NativeChatRequest {
session_id, // 使用前端传递的 session_id 以保持上下文
src-tauri/src/commands/screenshot_cmd.rs
+352
View File
@@ -0,0 +1,352 @@
//! 截图对话命令模块
//!
//! 提供截图对话功能的 Tauri 命令接口，包括：
//! - 获取和保存实验室功能配置
//! - 启动截图
//! - 验证和更新快捷键
//! - 读取图片为 Base64
//!
//! _需求: 1.1, 1.4, 1.5, 2.2, 2.4, 3.1, 5.1_
use crate::config::{ExperimentalFeatures, GlobalConfigManagerState};
use crate::screenshot::{capture, shortcut};
use base64::{engine::general_purpose::STANDARD, Engine};
use std::path::Path;
use tauri::{AppHandle, Emitter, Manager, State};
use tokio::fs;
use tracing::{debug, error, info};
/// 获取实验室功能配置
///
/// 从应用状态中获取当前的实验室功能配置
///
/// # 返回
/// 成功返回 ExperimentalFeatures，失败返回错误信息
///
/// _需求: 1.1_
#[tauri::command]
pub async fn get_experimental_config(
config_manager: State<'_, GlobalConfigManagerState>,
) -> Result<ExperimentalFeatures, String> {
debug!("获取实验室功能配置");
let config = config_manager.config();
Ok(config.experimental.clone())
}
/// 保存实验室功能配置
///
/// 将实验室功能配置保存到应用状态和配置文件
/// 同时根据配置状态注册或注销快捷键
///
/// # 参数
/// - `app`: Tauri 应用句柄
/// - `config_manager`: 全局配置管理器状态
/// - `experimental_config`: 要保存的实验室功能配置
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误信息
///
/// _需求: 1.4, 1.5_
#[tauri::command]
pub async fn save_experimental_config(
app: AppHandle,
config_manager: State<'_, GlobalConfigManagerState>,
experimental_config: ExperimentalFeatures,
) -> Result<(), String> {
info!(
"保存实验室功能配置: enabled={}, shortcut={}",
experimental_config.screenshot_chat.enabled, experimental_config.screenshot_chat.shortcut
);
// 获取旧配置以比较变化
let old_config = config_manager.config();
let was_enabled = old_config.experimental.screenshot_chat.enabled;
let is_enabled = experimental_config.screenshot_chat.enabled;
debug!(
"配置变化: was_enabled={}, is_enabled={}",
was_enabled, is_enabled
);
// 更新配置
let mut new_config = old_config.clone();
new_config.experimental = experimental_config.clone();
// 保存配置到文件
debug!("开始保存配置到文件...");
if let Err(e) = config_manager.save_config(&new_config).await {
error!("保存配置失败: {}", e);
return Err(format!("保存配置失败: {}", e));
}
info!("配置文件保存成功");
// 根据功能开关状态注册或注销快捷键
if was_enabled != is_enabled {
if is_enabled {
info!(
"截图对话功能已启用，注册快捷键: {}",
experimental_config.screenshot_chat.shortcut
);
if let Err(e) = shortcut::register(&app, &experimental_config.screenshot_chat.shortcut)
{
error!("注册快捷键失败: {}", e);
return Err(format!("注册快捷键失败: {}", e));
}
info!("快捷键注册成功");
} else {
info!("截图对话功能已禁用，注销快捷键");
if let Err(e) = shortcut::unregister(&app) {
error!("注销快捷键失败: {}", e);
return Err(format!("注销快捷键失败: {}", e));
}
info!("快捷键注销成功");
}
}
info!("实验室功能配置保存完成");
Ok(())
}
/// 启动截图
///
/// 启动交互式截图，返回截图文件路径
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 成功返回截图文件路径，用户取消返回空字符串，失败返回错误信息
///
/// _需求: 3.1_
#[tauri::command]
pub async fn start_screenshot(app: AppHandle) -> Result<String, String> {
info!("启动截图命令");
match capture::start_capture(&app).await {
Ok(path) => {
info!("截图成功: {:?}", path);
Ok(path.to_string_lossy().to_string())
}
Err(capture::CaptureError::Cancelled) => {
info!("用户取消了截图");
Ok(String::new())
}
Err(e) => {
error!("截图失败: {}", e);
Err(format!("截图失败: {}", e))
}
}
}
/// 验证快捷键格式
///
/// 检查快捷键字符串是否符合 Tauri 快捷键格式要求
///
/// # 参数
/// - `shortcut_str`: 快捷键字符串，如 "CommandOrControl+Shift+S"
///
/// # 返回
/// 如果格式有效返回 true，否则返回错误信息
///
/// _需求: 2.2_
#[tauri::command]
pub fn validate_shortcut(shortcut_str: String) -> Result<bool, String> {
debug!("验证快捷键格式: {}", shortcut_str);
match shortcut::validate(&shortcut_str) {
Ok(()) => Ok(true),
Err(e) => Err(format!("{}", e)),
}
}
/// 更新截图快捷键
///
/// 原子性地更新快捷键：先注销旧快捷键，再注册新快捷键
/// 同时更新配置文件
///
/// # 参数
/// - `app`: Tauri 应用句柄
/// - `config_manager`: 全局配置管理器状态
/// - `new_shortcut`: 新的快捷键字符串
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误信息
///
/// _需求: 2.4_
#[tauri::command]
pub async fn update_screenshot_shortcut(
app: AppHandle,
config_manager: State<'_, GlobalConfigManagerState>,
new_shortcut: String,
) -> Result<(), String> {
info!("更新截图快捷键: {}", new_shortcut);
// 验证新快捷键格式
shortcut::validate(&new_shortcut).map_err(|e| format!("快捷键格式无效: {}", e))?;
// 获取当前配置
let mut config = config_manager.config();
// 检查功能是否启用
if config.experimental.screenshot_chat.enabled {
// 更新快捷键（原子操作）
shortcut::update(&app, &new_shortcut).map_err(|e| format!("更新快捷键失败: {}", e))?;
}
// 更新配置
config.experimental.screenshot_chat.shortcut = new_shortcut;
// 保存配置到文件
config_manager
.save_config(&config)
.await
.map_err(|e| format!("保存配置失败: {}", e))?;
info!("截图快捷键更新成功");
Ok(())
}
/// 关闭截图对话窗口
///
/// 关闭当前打开的截图对话悬浮窗口
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误信息
#[tauri::command]
pub fn close_screenshot_chat_window(app: AppHandle) -> Result<(), String> {
info!("关闭截图对话窗口");
crate::screenshot::window::close_floating_window(&app)
.map_err(|e| format!("关闭窗口失败: {}", e))
}
/// 读取图片文件并转换为 Base64
///
/// 读取指定路径的图片文件，并将其内容编码为 Base64 字符串
///
/// # 参数
/// - `path`: 图片文件路径
///
/// # 返回
/// 成功返回 Base64 编码的图片数据，失败返回错误信息
///
/// _需求: 5.1_
#[tauri::command]
pub async fn read_image_as_base64(path: String) -> Result<String, String> {
debug!("读取图片为 Base64: {}", path);
let path = Path::new(&path);
// 检查文件是否存在
if !path.exists() {
return Err(format!("文件不存在: {}", path.display()));
}
// 读取文件内容
let bytes = fs::read(path)
.await
.map_err(|e| format!("读取文件失败: {}", e))?;
// 检查文件是否为空
if bytes.is_empty() {
return Err("文件为空".to_string());
}
// 编码为 Base64
let base64 = STANDARD.encode(&bytes);
debug!("图片读取成功，大小: {} 字节", bytes.len());
Ok(base64)
}
/// 截图对话消息结构
#[derive(Debug, Clone, serde::Serialize)]
pub struct ScreenshotChatMessage {
pub message: String,
pub image_path: Option<String>,
pub image_base64: Option<String>,
}
/// 发送截图对话消息到主应用
///
/// 将用户输入的消息和截图发送到主应用的 Agent 聊天界面
///
/// # 参数
/// - `app`: Tauri 应用句柄
/// - `message`: 用户输入的消息
/// - `image_path`: 截图文件路径（可选）
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误信息
#[tauri::command]
pub async fn send_screenshot_chat(
app: AppHandle,
message: String,
image_path: Option<String>,
) -> Result<(), String> {
info!(
"发送截图对话: message={}, image_path={:?}",
message, image_path
);
// 如果有图片，读取为 Base64
let image_base64 = if let Some(ref path) = image_path {
match read_image_as_base64(path.clone()).await {
Ok(base64) => Some(base64),
Err(e) => {
error!("读取图片失败: {}", e);
None
}
}
} else {
None
};
// 构建消息
let chat_message = ScreenshotChatMessage {
message,
image_path,
image_base64,
};
// 发送事件到主窗口
if let Some(main_window) = app.get_webview_window("main") {
main_window
.emit("screenshot-chat-message", &chat_message)
.map_err(|e| format!("发送事件失败: {}", e))?;
// 恢复并聚焦主窗口（主窗口在截图时被最小化）
let _ = main_window.unminimize();
let _ = main_window.show();
let _ = main_window.set_focus();
} else {
// 尝试发送到所有窗口
app.emit("screenshot-chat-message", &chat_message)
.map_err(|e| format!("发送事件失败: {}", e))?;
}
info!("截图对话消息已发送");
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_validate_shortcut_valid() {
assert!(validate_shortcut("CommandOrControl+Shift+S".to_string()).is_ok());
assert!(validate_shortcut("Alt+F4".to_string()).is_ok());
assert!(validate_shortcut("Ctrl+C".to_string()).is_ok());
}
#[test]
fn test_validate_shortcut_invalid() {
assert!(validate_shortcut("".to_string()).is_err());
assert!(validate_shortcut("InvalidKey".to_string()).is_err());
}
}
src-tauri/src/commands/webview_cmd.rs
+4 -2
View File
@@ -65,9 +65,11 @@ pub struct CreateWebviewRequest {
pub url: String,
/// 面板标题
pub title: Option<String>,
/// X 坐标（相对于主窗口）
/// X 坐标（相对于主窗口）- 预留，当前使用居中显示
#[allow(dead_code)]
pub x: f64,
/// Y 坐标（相对于主窗口）
/// Y 坐标（相对于主窗口）- 预留，当前使用居中显示
#[allow(dead_code)]
pub y: f64,
/// 宽度
pub width: f64,
src-tauri/src/config/mod.rs
+6 -5
View File
@@ -22,11 +22,12 @@ pub use import::{ImportOptions, ImportService, ValidationResult};
pub use path_utils::{collapse_tilde, contains_tilde, expand_tilde};
pub use types::{
generate_secure_api_key, AmpConfig, AmpModelMapping, ApiKeyEntry, Config, CredentialEntry,
CredentialPoolConfig, CustomProviderConfig, EndpointProvidersConfig, GeminiApiKeyEntry,
IFlowCredentialEntry, InjectionRuleConfig, InjectionSettings, LoggingConfig, ModelInfo,
ModelsConfig, NativeAgentConfig, ProviderConfig, ProviderModelsConfig, ProvidersConfig,
QuotaExceededConfig, RemoteManagementConfig, RetrySettings, RoutingConfig, ServerConfig,
TlsConfig, VertexApiKeyEntry, VertexModelAlias, DEFAULT_API_KEY,
CredentialPoolConfig, CustomProviderConfig, EndpointProvidersConfig, ExperimentalFeatures,
GeminiApiKeyEntry, IFlowCredentialEntry, InjectionRuleConfig, InjectionSettings, LoggingConfig,
ModelInfo, ModelsConfig, NativeAgentConfig, ProviderConfig, ProviderModelsConfig,
ProvidersConfig, QuotaExceededConfig, RemoteManagementConfig, RetrySettings, RoutingConfig,
ScreenshotChatConfig, ServerConfig, TlsConfig, VertexApiKeyEntry, VertexModelAlias,
DEFAULT_API_KEY,
};
pub use yaml::{load_config, save_config, ConfigError, ConfigManager, YamlService};
src-tauri/src/config/observer/subject.rs
+2
View File
@@ -14,6 +14,8 @@ use tokio::sync::broadcast;
/// Tauri 事件名称常量
pub const CONFIG_CHANGED_EVENT: &str = "config-changed";
/// 预留：配置重新加载事件
#[allow(dead_code)]
pub const CONFIG_RELOAD_EVENT: &str = "config-reload";
/// 观察者条目
src-tauri/src/config/observer/traits.rs
+4
View File
@@ -95,6 +95,8 @@ impl<T: SyncConfigObserver + 'static> ConfigObserver for SyncObserverWrapper<T>
}
/// 函数式观察者（用于简单的回调场景）
/// 目前主要用于测试，将来可用于动态注册观察者
#[allow(dead_code)]
pub struct FnObserver<F>
where
F: Fn(&ConfigChangeEvent, &Config) -> Result<(), String> + Send + Sync,
@@ -108,6 +110,7 @@ impl<F> FnObserver<F>
where
F: Fn(&ConfigChangeEvent, &Config) -> Result<(), String> + Send + Sync,
{
#[allow(dead_code)]
pub fn new(name: impl Into<String>, handler: F) -> Self {
Self {
name: name.into(),
@@ -116,6 +119,7 @@ where
}
}
#[allow(dead_code)]
pub fn with_priority(mut self, priority: i32) -> Self {
self.priority = priority;
self
src-tauri/src/config/tests.rs
+3
View File
@@ -188,6 +188,7 @@ fn arb_config() -> impl Strategy<Value = Config> {
models: crate::config::ModelsConfig::default(),
agent: crate::config::NativeAgentConfig::default(),
language: "zh".to_string(),
experimental: crate::config::ExperimentalFeatures::default(),
})
}
@@ -425,6 +426,7 @@ fn arb_valid_config() -> impl Strategy<Value = Config> {
models: crate::config::ModelsConfig::default(),
agent: crate::config::NativeAgentConfig::default(),
language: "zh".to_string(),
experimental: crate::config::ExperimentalFeatures::default(),
})
}
@@ -473,6 +475,7 @@ fn arb_invalid_config() -> impl Strategy<Value = Config> {
models: crate::config::ModelsConfig::default(),
agent: crate::config::NativeAgentConfig::default(),
language: "zh".to_string(),
experimental: crate::config::ExperimentalFeatures::default(),
};
// 根据类型使配置无效
match invalid_type {
src-tauri/src/config/types.rs
+83
View File
@@ -320,6 +320,9 @@ pub struct Config {
/// Native Agent 配置
#[serde(default)]
pub agent: NativeAgentConfig,
/// 实验室功能配置
#[serde(default)]
pub experimental: ExperimentalFeatures,
}
// ============ Native Agent 配置类型 ============
@@ -382,6 +385,44 @@ impl Default for NativeAgentConfig {
}
}
// ============ 实验室功能配置类型 ============
/// 截图对话功能配置
///
/// 配置截图对话功能的开关和快捷键
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct ScreenshotChatConfig {
/// 是否启用截图对话功能
#[serde(default)]
pub enabled: bool,
/// 触发截图的全局快捷键
#[serde(default = "default_screenshot_shortcut")]
pub shortcut: String,
}
fn default_screenshot_shortcut() -> String {
"CommandOrControl+Alt+Q".to_string()
}
impl Default for ScreenshotChatConfig {
fn default() -> Self {
Self {
enabled: false,
shortcut: default_screenshot_shortcut(),
}
}
}
/// 实验室功能配置
///
/// 管理所有实验性功能的开关和配置
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Default)]
pub struct ExperimentalFeatures {
/// 截图对话功能配置
#[serde(default)]
pub screenshot_chat: ScreenshotChatConfig,
}
impl NativeAgentConfig {
/// 获取有效的系统提示词
///
@@ -1179,6 +1220,7 @@ impl Default for Config {
language: default_language(),
models: ModelsConfig::default(),
agent: NativeAgentConfig::default(),
experimental: ExperimentalFeatures::default(),
}
}
}
@@ -1458,4 +1500,45 @@ mod unit_tests {
let parsed: EndpointProvidersConfig = serde_json::from_str(&json).unwrap();
assert_eq!(parsed, config);
}
#[test]
fn test_screenshot_chat_config_default() {
let config = ScreenshotChatConfig::default();
assert!(!config.enabled);
assert_eq!(config.shortcut, "CommandOrControl+Shift+S");
}
#[test]
fn test_experimental_features_default() {
let config = ExperimentalFeatures::default();
assert!(!config.screenshot_chat.enabled);
assert_eq!(config.screenshot_chat.shortcut, "CommandOrControl+Shift+S");
}
#[test]
fn test_experimental_features_serialization() {
let config = ExperimentalFeatures {
screenshot_chat: ScreenshotChatConfig {
enabled: true,
shortcut: "CommandOrControl+Alt+X".to_string(),
},
};
let yaml = serde_yaml::to_string(&config).unwrap();
assert!(yaml.contains("enabled: true"));
assert!(yaml.contains("shortcut: CommandOrControl+Alt+X"));
let parsed: ExperimentalFeatures = serde_yaml::from_str(&yaml).unwrap();
assert_eq!(parsed, config);
}
#[test]
fn test_config_with_experimental() {
let config = Config::default();
assert!(!config.experimental.screenshot_chat.enabled);
assert_eq!(
config.experimental.screenshot_chat.shortcut,
"CommandOrControl+Shift+S"
);
}
}
src-tauri/src/lib.rs
+1
View File
@@ -19,6 +19,7 @@ pub mod processor;
pub mod proxy;
pub mod resilience;
pub mod router;
pub mod screenshot;
pub mod services;
pub mod stream;
pub mod streaming;
src-tauri/src/models/model_registry.rs
+4
View File
@@ -516,6 +516,8 @@ pub struct ModelsDevModalities {
impl ModelsDevModel {
/// 转换为 EnhancedModelMetadata
/// 预留：用于从 models.dev API 导入模型数据
#[allow(dead_code)]
pub fn to_enhanced_metadata(
&self,
provider_id: &str,
@@ -584,6 +586,8 @@ impl ModelsDevModel {
}
/// 根据模型 ID 和名称推断服务等级
/// 用于 to_enhanced_metadata 和测试
#[allow(dead_code)]
fn infer_model_tier(model_id: &str, model_name: &str) -> ModelTier {
let id_lower = model_id.to_lowercase();
let name_lower = model_name.to_lowercase();
src-tauri/src/orchestrator/orchestrator.rs
+7 -5
View File
@@ -2,12 +2,12 @@
//!
//! 统一的模型编排接口，整合模型池构建、策略选择和降级处理。
use super::fallback::{FallbackHandler, FallbackPolicy, FallbackResult};
use super::pool_builder::{CredentialInfo, DynamicPoolBuilder, ProviderType};
use super::fallback::{FallbackHandler, FallbackPolicy};
use super::pool_builder::{CredentialInfo, DynamicPoolBuilder};
use super::selector::{ModelSelector, SelectionResult};
use super::strategies::create_default_registry;
use super::strategy::{SelectionContext, StrategyError, StrategyInfo, StrategyResult, TaskHint};
use super::tier::{AvailableModel, ServiceTier, TierConfig, TierPool};
use super::strategy::{SelectionContext, StrategyInfo, StrategyResult, TaskHint};
use super::tier::{AvailableModel, ServiceTier, TierPool};
use serde::{Deserialize, Serialize};
use std::sync::Arc;
use tokio::sync::RwLock;
@@ -50,7 +50,8 @@ pub struct ModelOrchestrator {
selector: ModelSelector,
/// 模型池构建器
pool_builder: DynamicPoolBuilder,
/// 降级处理器
/// 降级处理器（预留，将来实现自动降级功能）
#[allow(dead_code)]
fallback_handler: FallbackHandler,
/// 当前凭证列表
credentials: RwLock<Vec<CredentialInfo>>,
@@ -313,6 +314,7 @@ pub fn get_global_orchestrator() -> Option<Arc<ModelOrchestrator>> {
#[cfg(test)]
mod tests {
use super::super::pool_builder::ProviderType;
use super::*;
#[tokio::test]
src-tauri/src/processor/mod.rs
+3 -4
View File
@@ -113,12 +113,11 @@ impl RequestProcessor {
/// 注意：不再添加硬编码的路由规则，让用户设置的默认 Provider 生效
/// 用户可以通过 UI 或配置文件自定义路由规则
fn create_router_with_defaults() -> Router {
use crate::ProviderType;
// 创建空的路由器，默认 Provider 会在启动时从配置中设置
let router = Router::new(ProviderType::Kiro);
// 不要硬编码任何 Provider，避免与用户配置冲突
let router = Router::new_empty();
tracing::info!("[ROUTER] 初始化路由器（无硬编码规则，使用用户配置的默认 Provider）");
tracing::info!("[ROUTER] 初始化空路由器，等待从配置加载默认 Provider");
router
}
src-tauri/src/screenshot/README.md
+69
View File
@@ -0,0 +1,69 @@
# 截图对话模块 (screenshot/)
截图对话功能的 Rust 后端实现，提供全局快捷键、屏幕截图和悬浮窗口管理。
## 模块结构
```
screenshot/
├── mod.rs # 模块入口，导出公共接口和初始化函数
├── capture.rs # 屏幕截图服务，跨平台截图实现
├── config.rs # 实验室功能配置管理
├── shortcut.rs # 全局快捷键注册/注销/更新
├── window.rs # 悬浮对话窗口管理
└── README.md # 本文档
```
## 核心功能
### 1. 模块初始化 (mod.rs)
- `init(app)`: 初始化截图对话模块，根据配置注册快捷键
- `cleanup(app)`: 清理模块资源，注销快捷键
### 2. 配置管理 (config.rs)
- `load_experimental_config(app)`: 加载实验室功能配置
- `is_screenshot_chat_enabled(config)`: 检查功能是否启用
- `save_experimental_config(app, config)`: 保存配置到文件
### 3. 快捷键管理 (shortcut.rs)
- `register(app, shortcut)`: 注册全局快捷键
- `unregister(app)`: 注销当前快捷键
- `update(app, new_shortcut)`: 原子性更新快捷键
- `validate(shortcut)`: 验证快捷键格式
### 4. 截图服务 (capture.rs)
- `start_capture(app)`: 启动交互式截图
- 支持 macOS (screencapture)、Windows (PowerShell)、Linux (gnome-screenshot/scrot)
### 5. 窗口管理 (window.rs)
- `open_floating_window(app, image_path)`: 打开悬浮对话窗口
- `close_floating_window(app)`: 关闭悬浮窗口
- `is_floating_window_open(app)`: 检查窗口状态
## 依赖
- `tauri-plugin-global-shortcut`: 全局快捷键支持
- `chrono`: 时间戳生成
- `urlencoding`: URL 参数编码
## 配置示例
```yaml
experimental:
screenshot_chat:
enabled: true
shortcut: "CommandOrControl+Shift+S"
```
## 相关需求
- 需求 1: 实验室功能开关
- 需求 2: 快捷键配置
- 需求 3: 屏幕截图
- 需求 4: 悬浮对话窗口
- 需求 7: 模块化代码组织
src-tauri/src/screenshot/capture.rs
+247
View File
@@ -0,0 +1,247 @@
//! 屏幕截图服务
//!
//! 提供跨平台的屏幕截图功能，支持交互式区域选择
use std::path::PathBuf;
use tauri::AppHandle;
use tracing::{debug, error, info, warn};
/// 截图错误类型
#[derive(Debug, thiserror::Error)]
pub enum CaptureError {
#[error("用户取消截图")]
Cancelled,
#[error("权限不足: {0}")]
PermissionDenied(String),
#[error("系统错误: {0}")]
SystemError(String),
#[error("IO 错误: {0}")]
IoError(#[from] std::io::Error),
#[error("临时文件创建失败: {0}")]
TempFileError(String),
}
/// 截图结果
#[derive(Debug)]
pub enum CaptureResult {
/// 截图成功，包含图片路径
Success(PathBuf),
/// 用户取消截图
Cancelled,
}
/// 启动交互式截图
///
/// 调用系统截图工具进行交互式区域选择
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 成功返回图片路径，用户取消返回 Cancelled 错误，其他情况返回相应错误
///
/// # 平台支持
/// - macOS: 使用 `screencapture -i -x` 命令
/// - Windows: 使用 Windows API 或系统截图工具
/// - Linux: 使用 `gnome-screenshot` 或 `scrot`
pub async fn start_capture(_app: &AppHandle) -> Result<PathBuf, CaptureError> {
info!("启动交互式截图");
// 生成临时文件路径
let temp_dir = std::env::temp_dir();
let timestamp = chrono::Utc::now().format("%Y%m%d_%H%M%S_%3f");
let filename = format!("proxycast_screenshot_{}.png", timestamp);
let temp_path = temp_dir.join(&filename);
debug!("截图临时文件路径: {:?}", temp_path);
// 根据平台调用不同的截图命令
#[cfg(target_os = "macos")]
{
capture_macos(&temp_path).await?;
}
#[cfg(target_os = "windows")]
{
capture_windows(&temp_path).await?;
}
#[cfg(target_os = "linux")]
{
capture_linux(&temp_path).await?;
}
// 检查文件是否存在（用户可能取消了截图）
if !temp_path.exists() {
info!("截图文件不存在，用户可能取消了截图");
return Err(CaptureError::Cancelled);
}
// 检查文件大小（空文件表示取消）
let metadata = std::fs::metadata(&temp_path)?;
if metadata.len() == 0 {
info!("截图文件为空，用户取消了截图");
std::fs::remove_file(&temp_path)?;
return Err(CaptureError::Cancelled);
}
info!("截图成功: {:?}", temp_path);
Ok(temp_path)
}
/// macOS 截图实现
#[cfg(target_os = "macos")]
async fn capture_macos(output_path: &PathBuf) -> Result<(), CaptureError> {
use std::process::Command;
debug!("使用 macOS screencapture 命令");
// 先检查屏幕录制权限
// 通过尝试执行一个快速的全屏截图到 /dev/null 来检测权限
let permission_check = Command::new("screencapture")
.args(["-x", "-c"]) // -c 截图到剪贴板，快速检测权限
.output();
if let Ok(output) = permission_check {
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
if stderr.contains("cannot") || stderr.contains("permission") {
warn!("屏幕录制权限不足: {}", stderr);
return Err(CaptureError::PermissionDenied(
"请在系统设置 → 隐私与安全性 → 录屏与系统录音 中授权 ProxyCast".to_string(),
));
}
}
}
// 使用 screencapture 命令
// -i: 交互式选择区域
// -x: 不播放截图声音
let output = Command::new("screencapture")
.args(["-i", "-x", output_path.to_str().unwrap()])
.output()
.map_err(|e| CaptureError::SystemError(format!("执行 screencapture 失败: {}", e)))?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
// 用户按 ESC 取消时，screencapture 返回非零状态但不创建文件
if !output_path.exists() {
return Err(CaptureError::Cancelled);
}
error!("screencapture 命令失败: {}", stderr);
return Err(CaptureError::SystemError(format!(
"screencapture 失败: {}",
stderr
)));
}
Ok(())
}
/// Windows 截图实现
#[cfg(target_os = "windows")]
async fn capture_windows(output_path: &PathBuf) -> Result<(), CaptureError> {
use std::process::Command;
debug!("使用 Windows 截图工具");
// 使用 PowerShell 调用截图功能
// 注意：这是一个简化实现，实际可能需要使用 Windows API
let script = format!(
r#"
Add-Type -AssemblyName System.Windows.Forms
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
$bitmap = New-Object System.Drawing.Bitmap($screen.Bounds.Width, $screen.Bounds.Height)
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
$graphics.CopyFromScreen($screen.Bounds.Location, [System.Drawing.Point]::Empty, $screen.Bounds.Size)
$bitmap.Save('{}')
"#,
output_path.to_str().unwrap().replace("\\", "\\\\")
);
let output = Command::new("powershell")
.args(["-Command", &script])
.output()
.map_err(|e| CaptureError::SystemError(format!("执行 PowerShell 失败: {}", e)))?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
error!("PowerShell 截图失败: {}", stderr);
return Err(CaptureError::SystemError(format!("截图失败: {}", stderr)));
}
Ok(())
}
/// Linux 截图实现
#[cfg(target_os = "linux")]
async fn capture_linux(output_path: &PathBuf) -> Result<(), CaptureError> {
use std::process::Command;
debug!("使用 Linux 截图工具");
// 尝试使用 gnome-screenshot
let gnome_result = Command::new("gnome-screenshot")
.args(["-a", "-f", output_path.to_str().unwrap()])
.output();
if let Ok(output) = gnome_result {
if output.status.success() {
return Ok(());
}
}
// 回退到 scrot
let scrot_result = Command::new("scrot")
.args(["-s", output_path.to_str().unwrap()])
.output()
.map_err(|e| {
CaptureError::SystemError(format!(
"无法找到截图工具 (gnome-screenshot 或 scrot): {}",
e
))
})?;
if !scrot_result.status.success() {
let stderr = String::from_utf8_lossy(&scrot_result.stderr);
if !output_path.exists() {
return Err(CaptureError::Cancelled);
}
return Err(CaptureError::SystemError(format!("scrot 失败: {}", stderr)));
}
Ok(())
}
/// 清理临时截图文件
///
/// # 参数
/// - `path`: 要删除的文件路径
pub fn cleanup_temp_file(path: &PathBuf) {
if path.exists() {
if let Err(e) = std::fs::remove_file(path) {
error!("删除临时截图文件失败: {}", e);
} else {
debug!("已删除临时截图文件: {:?}", path);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_temp_path_generation() {
let temp_dir = std::env::temp_dir();
let timestamp = chrono::Utc::now().format("%Y%m%d_%H%M%S_%3f");
let filename = format!("proxycast_screenshot_{}.png", timestamp);
let temp_path = temp_dir.join(&filename);
assert!(temp_path
.to_str()
.unwrap()
.contains("proxycast_screenshot_"));
assert!(temp_path.to_str().unwrap().ends_with(".png"));
}
}
src-tauri/src/screenshot/config.rs
+132
View File
@@ -0,0 +1,132 @@
//! 截图对话配置管理
//!
//! 提供实验室功能配置的加载和检查功能
use crate::config::{ExperimentalFeatures, GlobalConfigManagerState, ScreenshotChatConfig};
use tauri::{AppHandle, Manager};
use tracing::debug;
/// 配置错误类型
#[derive(Debug, thiserror::Error)]
pub enum ConfigError {
#[error("无法获取应用状态")]
StateNotFound,
#[error("配置加载失败: {0}")]
LoadFailed(String),
}
/// 加载实验室功能配置
///
/// 从应用状态中获取当前的实验室功能配置
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 成功返回 ExperimentalFeatures，失败返回错误
pub fn load_experimental_config(app: &AppHandle) -> Result<ExperimentalFeatures, ConfigError> {
debug!("加载实验室功能配置");
// 获取全局配置管理器
let config_manager = app
.try_state::<GlobalConfigManagerState>()
.ok_or(ConfigError::StateNotFound)?;
// 从配置中获取实验室功能配置
let config = config_manager.config();
let experimental = config.experimental.clone();
debug!(
"实验室功能配置: screenshot_chat.enabled={}, shortcut={}",
experimental.screenshot_chat.enabled, experimental.screenshot_chat.shortcut
);
Ok(experimental)
}
/// 检查截图对话功能是否启用
///
/// # 参数
/// - `config`: 实验室功能配置
///
/// # 返回
/// 如果截图对话功能启用返回 true，否则返回 false
pub fn is_screenshot_chat_enabled(config: &ExperimentalFeatures) -> bool {
config.screenshot_chat.enabled
}
/// 获取截图对话配置
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 成功返回 ScreenshotChatConfig，失败返回错误
pub fn get_screenshot_chat_config(app: &AppHandle) -> Result<ScreenshotChatConfig, ConfigError> {
let experimental = load_experimental_config(app)?;
Ok(experimental.screenshot_chat)
}
/// 保存实验室功能配置
///
/// 将实验室功能配置保存到应用状态和配置文件
///
/// # 参数
/// - `app`: Tauri 应用句柄
/// - `experimental_config`: 要保存的实验室功能配置
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误
pub async fn save_experimental_config(
app: &AppHandle,
experimental_config: ExperimentalFeatures,
) -> Result<(), ConfigError> {
debug!("保存实验室功能配置");
// 获取全局配置管理器
let config_manager = app
.try_state::<GlobalConfigManagerState>()
.ok_or(ConfigError::StateNotFound)?;
// 获取当前配置并更新实验室功能部分
let mut config = config_manager.config();
config.experimental = experimental_config;
// 保存配置
config_manager
.save_config(&config)
.await
.map_err(|e| ConfigError::LoadFailed(e))?;
debug!("实验室功能配置已保存");
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_is_screenshot_chat_enabled_default() {
let config = ExperimentalFeatures::default();
assert!(!is_screenshot_chat_enabled(&config));
}
#[test]
fn test_is_screenshot_chat_enabled_true() {
let config = ExperimentalFeatures {
screenshot_chat: ScreenshotChatConfig {
enabled: true,
shortcut: "CommandOrControl+Shift+S".to_string(),
},
};
assert!(is_screenshot_chat_enabled(&config));
}
#[test]
fn test_default_shortcut() {
let config = ScreenshotChatConfig::default();
assert_eq!(config.shortcut, "CommandOrControl+Shift+S");
assert!(!config.enabled);
}
}
src-tauri/src/screenshot/mod.rs
+69
View File
@@ -0,0 +1,69 @@
//! 截图对话模块
//!
//! 提供截图对话功能的核心实现，包括：
//! - 屏幕截图服务
//! - 全局快捷键管理
//! - 悬浮窗口管理
//! - 实验室功能配置
pub mod capture;
pub mod config;
pub mod shortcut;
pub mod window;
use tauri::AppHandle;
use tracing::{error, info};
/// 截图模块错误类型
#[derive(Debug, thiserror::Error)]
pub enum ScreenshotError {
#[error("配置错误: {0}")]
Config(String),
#[error("快捷键错误: {0}")]
Shortcut(#[from] shortcut::ShortcutError),
#[error("截图错误: {0}")]
Capture(#[from] capture::CaptureError),
#[error("窗口错误: {0}")]
Window(#[from] window::WindowError),
}
/// 初始化截图对话模块
///
/// 根据配置决定是否注册全局快捷键
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误
pub fn init(app: &AppHandle) -> Result<(), ScreenshotError> {
info!("初始化截图对话模块");
// 加载实验室功能配置
let experimental_config = config::load_experimental_config(app)
.map_err(|e| ScreenshotError::Config(e.to_string()))?;
// 检查截图对话功能是否启用
if config::is_screenshot_chat_enabled(&experimental_config) {
info!("截图对话功能已启用，注册快捷键");
shortcut::register(app, &experimental_config.screenshot_chat.shortcut)?;
} else {
info!("截图对话功能未启用，跳过快捷键注册");
}
Ok(())
}
/// 清理截图对话模块资源
///
/// 注销快捷键并清理临时文件
pub fn cleanup(app: &AppHandle) -> Result<(), ScreenshotError> {
info!("清理截图对话模块资源");
// 尝试注销快捷键（忽略错误，因为可能未注册）
if let Err(e) = shortcut::unregister(app) {
error!("注销快捷键失败: {}", e);
}
Ok(())
}
src-tauri/src/screenshot/shortcut.rs
+322
View File
@@ -0,0 +1,322 @@
//! 全局快捷键管理
//!
//! 提供截图对话功能的全局快捷键注册、注销和更新功能
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::OnceLock;
use tauri::{AppHandle, Manager};
use tauri_plugin_global_shortcut::{GlobalShortcutExt, Shortcut, ShortcutState};
use tracing::{debug, error, info, warn};
use super::capture;
use super::window;
/// 快捷键错误类型
#[derive(Debug, thiserror::Error)]
pub enum ShortcutError {
#[error("快捷键格式无效: {0}")]
InvalidFormat(String),
#[error("快捷键注册失败: {0}")]
RegisterFailed(String),
#[error("快捷键注销失败: {0}")]
UnregisterFailed(String),
#[error("快捷键已被占用: {0}")]
AlreadyInUse(String),
#[error("快捷键解析失败: {0}")]
ParseFailed(String),
}
/// 当前注册的快捷键（用于更新时注销旧快捷键）
static CURRENT_SHORTCUT: OnceLock<parking_lot::RwLock<Option<String>>> = OnceLock::new();
/// 快捷键是否已注册
static IS_REGISTERED: AtomicBool = AtomicBool::new(false);
fn get_current_shortcut() -> &'static parking_lot::RwLock<Option<String>> {
CURRENT_SHORTCUT.get_or_init(|| parking_lot::RwLock::new(None))
}
/// 验证快捷键格式
///
/// 检查快捷键字符串是否符合 Tauri 快捷键格式要求
///
/// # 参数
/// - `shortcut`: 快捷键字符串，如 "CommandOrControl+Shift+S"
///
/// # 返回
/// 如果格式有效返回 Ok(()), 否则返回错误
///
/// # 有效格式示例
/// - "CommandOrControl+Shift+S"
/// - "Alt+F4"
/// - "Super+Space"
/// - "Ctrl+Alt+Delete"
pub fn validate(shortcut: &str) -> Result<(), ShortcutError> {
if shortcut.is_empty() {
return Err(ShortcutError::InvalidFormat("快捷键不能为空".to_string()));
}
// 尝试解析快捷键以验证格式
shortcut.parse::<Shortcut>().map_err(|e| {
ShortcutError::InvalidFormat(format!("无法解析快捷键 '{}': {}", shortcut, e))
})?;
debug!("快捷键格式验证通过: {}", shortcut);
Ok(())
}
/// 注册全局快捷键
///
/// 注册截图对话功能的全局快捷键
///
/// # 参数
/// - `app`: Tauri 应用句柄
/// - `shortcut_str`: 快捷键字符串
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误
pub fn register(app: &AppHandle, shortcut_str: &str) -> Result<(), ShortcutError> {
info!("注册全局快捷键: {}", shortcut_str);
// 验证快捷键格式
validate(shortcut_str)?;
// 解析快捷键
let shortcut: Shortcut = shortcut_str
.parse()
.map_err(|e| ShortcutError::ParseFailed(format!("{}", e)))?;
// 获取全局快捷键管理器
let global_shortcut = app.global_shortcut();
// 检查快捷键是否已被注册
let is_already_registered = global_shortcut.is_registered(shortcut.clone());
info!(
"快捷键 {} 是否已注册: {}",
shortcut_str, is_already_registered
);
if is_already_registered {
warn!("快捷键已被注册: {}", shortcut_str);
// 如果是我们自己注册的，先注销
if IS_REGISTERED.load(Ordering::SeqCst) {
info!("尝试注销已有的快捷键");
if let Err(e) = global_shortcut.unregister(shortcut.clone()) {
error!("注销已有快捷键失败: {}", e);
}
} else {
return Err(ShortcutError::AlreadyInUse(shortcut_str.to_string()));
}
}
// 克隆 app handle 用于回调
let app_clone = app.clone();
// 注册快捷键
info!("开始注册快捷键回调...");
global_shortcut
.on_shortcut(shortcut.clone(), move |_app, _shortcut, event| {
if event.state == ShortcutState::Pressed {
info!("截图快捷键被触发");
handle_shortcut_triggered(&app_clone);
}
})
.map_err(|e| {
error!("注册快捷键失败: {}", e);
ShortcutError::RegisterFailed(format!("{}", e))
})?;
// 更新状态
IS_REGISTERED.store(true, Ordering::SeqCst);
*get_current_shortcut().write() = Some(shortcut_str.to_string());
info!("全局快捷键注册成功: {}", shortcut_str);
Ok(())
}
/// 注销全局快捷键
///
/// 注销当前注册的截图对话快捷键
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误
pub fn unregister(app: &AppHandle) -> Result<(), ShortcutError> {
let current = get_current_shortcut().read().clone();
if let Some(shortcut_str) = current {
info!("注销全局快捷键: {}", shortcut_str);
let shortcut: Shortcut = shortcut_str
.parse()
.map_err(|e| ShortcutError::ParseFailed(format!("{}", e)))?;
let global_shortcut = app.global_shortcut();
if global_shortcut.is_registered(shortcut.clone()) {
global_shortcut
.unregister(shortcut)
.map_err(|e| ShortcutError::UnregisterFailed(format!("{}", e)))?;
}
// 更新状态
IS_REGISTERED.store(false, Ordering::SeqCst);
*get_current_shortcut().write() = None;
info!("全局快捷键注销成功");
} else {
debug!("没有已注册的快捷键需要注销");
}
Ok(())
}
/// 更新全局快捷键
///
/// 原子性地更新快捷键：先注销旧快捷键，再注册新快捷键
///
/// # 参数
/// - `app`: Tauri 应用句柄
/// - `new_shortcut`: 新的快捷键字符串
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误
///
/// # 注意
/// 此操作是原子性的：如果新快捷键注册失败，会尝试恢复旧快捷键
pub fn update(app: &AppHandle, new_shortcut: &str) -> Result<(), ShortcutError> {
info!("更新全局快捷键: {}", new_shortcut);
// 验证新快捷键格式
validate(new_shortcut)?;
// 保存旧快捷键以便恢复
let old_shortcut = get_current_shortcut().read().clone();
// 注销旧快捷键
if let Err(e) = unregister(app) {
warn!("注销旧快捷键失败: {}", e);
// 继续尝试注册新快捷键
}
// 注册新快捷键
match register(app, new_shortcut) {
Ok(()) => {
info!("快捷键更新成功: {}", new_shortcut);
Ok(())
}
Err(e) => {
error!("注册新快捷键失败: {}", e);
// 尝试恢复旧快捷键
if let Some(old) = old_shortcut {
warn!("尝试恢复旧快捷键: {}", old);
if let Err(restore_err) = register(app, &old) {
error!("恢复旧快捷键失败: {}", restore_err);
}
}
Err(e)
}
}
}
/// 检查快捷键是否已注册
pub fn is_registered() -> bool {
IS_REGISTERED.load(Ordering::SeqCst)
}
/// 获取当前注册的快捷键
pub fn get_current() -> Option<String> {
get_current_shortcut().read().clone()
}
/// 处理快捷键触发事件
///
/// 当用户按下截图快捷键时调用此函数
fn handle_shortcut_triggered(app: &AppHandle) {
info!("处理截图快捷键触发");
// 在后台线程中执行截图操作
let app_clone = app.clone();
tauri::async_runtime::spawn(async move {
// 截图前最小化主窗口（而非隐藏），避免视觉干扰但保持可恢复
let main_window = app_clone.get_webview_window("main");
if let Some(win) = main_window.as_ref() {
debug!("最小化主窗口");
let _ = win.minimize();
}
// 短暂延迟，确保窗口完全最小化
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
match capture::start_capture(&app_clone).await {
Ok(image_path) => {
info!("截图成功: {:?}", image_path);
// 打开悬浮窗口
if let Err(e) = window::open_floating_window(&app_clone, &image_path) {
error!("打开悬浮窗口失败: {}", e);
// 如果悬浮窗口打开失败，恢复主窗口
if let Some(win) = main_window.as_ref() {
let _ = win.unminimize();
let _ = win.set_focus();
}
}
}
Err(capture::CaptureError::Cancelled) => {
info!("用户取消了截图");
// 用户取消截图，恢复主窗口显示
if let Some(win) = main_window.as_ref() {
debug!("恢复主窗口显示");
let _ = win.unminimize();
let _ = win.set_focus();
}
}
Err(e) => {
error!("截图失败: {}", e);
// 截图失败，恢复主窗口显示
if let Some(win) = main_window.as_ref() {
let _ = win.unminimize();
let _ = win.set_focus();
}
}
}
});
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_validate_valid_shortcuts() {
// 有效的快捷键格式
assert!(validate("CommandOrControl+Shift+S").is_ok());
assert!(validate("Alt+F4").is_ok());
assert!(validate("Ctrl+C").is_ok());
assert!(validate("Super+Space").is_ok());
assert!(validate("Shift+A").is_ok());
}
#[test]
fn test_validate_empty_shortcut() {
let result = validate("");
assert!(result.is_err());
match result {
Err(ShortcutError::InvalidFormat(msg)) => {
assert!(msg.contains("不能为空"));
}
_ => panic!("Expected InvalidFormat error"),
}
}
#[test]
fn test_validate_invalid_shortcuts() {
// 无效的快捷键格式
assert!(validate("InvalidKey").is_err());
assert!(validate("+++").is_err());
assert!(validate("Ctrl++").is_err());
}
}
src-tauri/src/screenshot/window.rs
+300
View File
@@ -0,0 +1,300 @@
//! 悬浮窗口管理
//!
//! 提供截图对话悬浮窗口的创建、显示和关闭功能
use mouse_position::mouse_position::Mouse;
use std::path::Path;
use tauri::{AppHandle, Manager, WebviewUrl, WebviewWindowBuilder};
use tracing::{debug, info, warn};
#[cfg(target_os = "macos")]
use cocoa::appkit::{NSColor, NSWindow};
#[cfg(target_os = "macos")]
use cocoa::base::{id, nil};
/// 窗口错误类型
#[derive(Debug, thiserror::Error)]
pub enum WindowError {
#[error("窗口创建失败: {0}")]
CreateFailed(String),
#[error("窗口未找到: {0}")]
NotFound(String),
#[error("窗口操作失败: {0}")]
OperationFailed(String),
}
/// 悬浮窗口标签
const FLOATING_WINDOW_LABEL: &str = "screenshot-chat";
/// 窗口尺寸（包含 padding 用于阴影）
const WINDOW_WIDTH: f64 = 645.0;
const WINDOW_HEIGHT: f64 = 70.0;
/// 距离屏幕底部的距离
const BOTTOM_MARGIN: f64 = 200.0;
/// 获取鼠标所在的显示器
///
/// 使用 mouse_position crate 获取鼠标位置，然后遍历所有显示器找到鼠标所在的显示器
fn get_monitor_at_cursor(app: &AppHandle) -> Option<tauri::Monitor> {
// 使用 mouse_position crate 获取鼠标位置
let (cursor_x, cursor_y) = match Mouse::get_mouse_position() {
Mouse::Position { x, y } => {
debug!("鼠标位置 (mouse_position crate): ({}, {})", x, y);
(x as f64, y as f64)
}
Mouse::Error => {
debug!("无法获取鼠标位置");
return None;
}
};
// 获取所有显示器
let monitors = match app.available_monitors() {
Ok(monitors) => monitors,
Err(e) => {
debug!("无法获取显示器列表: {}", e);
return None;
}
};
// 查找鼠标所在的显示器
for monitor in monitors {
let pos = monitor.position();
let size = monitor.size();
let left = pos.x as f64;
let top = pos.y as f64;
let right = left + size.width as f64;
let bottom = top + size.height as f64;
if cursor_x >= left && cursor_x < right && cursor_y >= top && cursor_y < bottom {
debug!(
"鼠标在显示器: {:?}, 位置: ({}, {}), 尺寸: {}x{}",
monitor.name(),
pos.x,
pos.y,
size.width,
size.height
);
return Some(monitor);
}
}
debug!(
"未找到鼠标所在的显示器，鼠标位置: ({}, {})",
cursor_x, cursor_y
);
None
}
/// 计算窗口位置（屏幕底部居中）
///
/// 优先使用鼠标所在的显示器，否则使用主显示器
/// 返回逻辑坐标（考虑 DPI 缩放）
fn calculate_window_position(app: &AppHandle) -> (f64, f64) {
// 优先获取鼠标所在的显示器
let monitor = get_monitor_at_cursor(app).or_else(|| app.primary_monitor().ok().flatten());
if let Some(monitor) = monitor {
let screen_pos = monitor.position();
let screen_size = monitor.size();
let scale_factor = monitor.scale_factor();
// 物理像素转换为逻辑像素
let screen_width = screen_size.width as f64 / scale_factor;
let screen_height = screen_size.height as f64 / scale_factor;
let screen_x = screen_pos.x as f64 / scale_factor;
let screen_y = screen_pos.y as f64 / scale_factor;
// 在该显示器上水平居中（使用逻辑坐标）
let x = screen_x + (screen_width - WINDOW_WIDTH) / 2.0;
// 距离该显示器底部 BOTTOM_MARGIN（使用逻辑坐标）
let y = screen_y + screen_height - WINDOW_HEIGHT - BOTTOM_MARGIN;
debug!(
"显示器: scale_factor={}, 物理位置: ({}, {}), 物理尺寸: {}x{}",
scale_factor, screen_pos.x, screen_pos.y, screen_size.width, screen_size.height
);
debug!(
"逻辑坐标: 屏幕({}, {}), 尺寸: {}x{}, 窗口位置: ({}, {})",
screen_x, screen_y, screen_width, screen_height, x, y
);
return (x, y);
}
// 默认位置（如果无法获取屏幕尺寸）
debug!("无法获取显示器信息，使用默认位置");
(400.0, 600.0)
}
/// 打开悬浮对话窗口
///
/// 如果窗口已在 tauri.conf.json 中预定义，则显示并导航到新 URL
/// 否则动态创建一个全屏透明的悬浮窗口
///
/// # 参数
/// - `app`: Tauri 应用句柄
/// - `image_path`: 截图文件路径
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误
pub fn open_floating_window(app: &AppHandle, image_path: &Path) -> Result<(), WindowError> {
info!("打开悬浮对话窗口");
// 构建窗口 URL，包含图片路径参数
let image_path_str = image_path.to_str().unwrap_or("");
let encoded_path = urlencoding::encode(image_path_str);
let url = format!("/screenshot-chat?image={}", encoded_path);
debug!("悬浮窗口 URL: {}", url);
// 检查窗口是否已存在（可能是预定义的或之前创建的）
if let Some(window) = app.get_webview_window(FLOATING_WINDOW_LABEL) {
info!("悬浮窗口已存在，导航到新 URL 并显示");
// 计算窗口位置（返回逻辑坐标）
let (x, y) = calculate_window_position(app);
// 设置窗口位置（使用逻辑坐标）
use tauri::LogicalPosition;
let _ = window.set_position(LogicalPosition::new(x, y));
// macOS: 设置窗口和 webview 背景透明
#[cfg(target_os = "macos")]
{
use objc::{msg_send, sel, sel_impl};
if let Ok(ns_win) = window.ns_window() {
unsafe {
let ns_window = ns_win as id;
// 设置窗口背景透明
let clear_color = NSColor::clearColor(nil);
ns_window.setBackgroundColor_(clear_color);
let _: () = msg_send![ns_window, setOpaque: false];
// 禁用窗口阴影
let _: () = msg_send![ns_window, setHasShadow: false];
}
}
}
// 使用 JavaScript 导航到新的 URL（更新图片路径）
let js = format!("window.location.href = '{}';", url);
window
.eval(&js)
.map_err(|e| WindowError::OperationFailed(format!("导航失败: {}", e)))?;
// 显示窗口
window
.show()
.map_err(|e| WindowError::OperationFailed(format!("显示窗口失败: {}", e)))?;
// 聚焦窗口
window
.set_focus()
.map_err(|e| WindowError::OperationFailed(format!("聚焦窗口失败: {}", e)))?;
return Ok(());
}
// 窗口不存在，动态创建
info!("动态创建悬浮窗口");
// 计算窗口位置
let (x, y) = calculate_window_position(app);
// 创建悬浮窗口（启用透明）
let window = WebviewWindowBuilder::new(app, FLOATING_WINDOW_LABEL, WebviewUrl::App(url.into()))
.inner_size(WINDOW_WIDTH, WINDOW_HEIGHT)
.position(x, y)
.decorations(false)
.always_on_top(true)
.skip_taskbar(true)
.visible(true)
.focused(true)
.transparent(true)
.build()
.map_err(|e| WindowError::CreateFailed(format!("{}", e)))?;
// macOS: 设置窗口和 webview 背景透明
#[cfg(target_os = "macos")]
{
use objc::{msg_send, sel, sel_impl};
if let Ok(ns_win) = window.ns_window() {
unsafe {
let ns_window = ns_win as id;
// 设置窗口背景透明
let clear_color = NSColor::clearColor(nil);
ns_window.setBackgroundColor_(clear_color);
let _: () = msg_send![ns_window, setOpaque: false];
// 禁用窗口阴影
let _: () = msg_send![ns_window, setHasShadow: false];
}
}
}
info!("悬浮窗口创建成功: {}", FLOATING_WINDOW_LABEL);
Ok(())
}
/// 关闭悬浮对话窗口
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误
pub fn close_floating_window(app: &AppHandle) -> Result<(), WindowError> {
info!("关闭悬浮对话窗口");
if let Some(window) = app.get_webview_window(FLOATING_WINDOW_LABEL) {
window
.close()
.map_err(|e| WindowError::OperationFailed(format!("关闭窗口失败: {}", e)))?;
info!("悬浮窗口已关闭");
} else {
debug!("悬浮窗口不存在，无需关闭");
}
Ok(())
}
/// 检查悬浮窗口是否打开
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 如果窗口存在且可见返回 true，否则返回 false
pub fn is_floating_window_open(app: &AppHandle) -> bool {
app.get_webview_window(FLOATING_WINDOW_LABEL)
.map(|w| w.is_visible().unwrap_or(false))
.unwrap_or(false)
}
/// 聚焦悬浮窗口
///
/// # 参数
/// - `app`: Tauri 应用句柄
///
/// # 返回
/// 成功返回 Ok(()), 失败返回错误
pub fn focus_floating_window(app: &AppHandle) -> Result<(), WindowError> {
if let Some(window) = app.get_webview_window(FLOATING_WINDOW_LABEL) {
window
.set_focus()
.map_err(|e| WindowError::OperationFailed(format!("聚焦窗口失败: {}", e)))?;
Ok(())
} else {
Err(WindowError::NotFound(FLOATING_WINDOW_LABEL.to_string()))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_window_label() {
assert_eq!(FLOATING_WINDOW_LABEL, "screenshot-chat");
}
}
src-tauri/src/server/handlers/api.rs
+129 -192
View File
@@ -653,49 +653,23 @@ pub async fn chat_completions(
),
);
// 使用 RequestProcessor 解析模型别名和路由
eprintln!("[CHAT_COMPLETIONS] 开始路由解析...");
let provider = state.processor.resolve_and_route(&mut ctx).await;
// 使用 RequestProcessor 解析模型别名
eprintln!("[CHAT_COMPLETIONS] 开始模型别名解析...");
let resolved_model = state.processor.resolve_model(&request.model).await;
ctx.set_resolved_model(resolved_model.clone());
eprintln!(
"[CHAT_COMPLETIONS] 路由结果: provider={:?}, resolved_model={}",
provider, ctx.resolved_model
"[CHAT_COMPLETIONS] 模型别名解析结果: {} -> {}",
request.model, resolved_model
);
// 如果没有设置默认 Provider，返回错误
let provider = match provider {
Some(p) => p,
None => {
eprintln!("[CHAT_COMPLETIONS] 未设置默认 Provider，返回错误");
state.logs.write().await.add(
"error",
&format!("[ROUTE] request_id={} 未设置默认 Provider", ctx.request_id),
);
return (
StatusCode::SERVICE_UNAVAILABLE,
Json(json!({
"error": {
"message": "未设置默认 Provider，请先在设置中选择一个默认 Provider",
"type": "configuration_error",
"code": "no_default_provider"
}
})),
)
.into_response();
}
};
// 更新请求中的模型名为解析后的模型
if ctx.resolved_model != ctx.original_model {
request.model = ctx.resolved_model.clone();
eprintln!(
"[CHAT_COMPLETIONS] 模型别名解析: {} -> {}",
ctx.original_model, ctx.resolved_model
);
if resolved_model != request.model {
request.model = resolved_model.clone();
state.logs.write().await.add(
"info",
&format!(
"[MAPPER] request_id={} alias={} -> model={}",
ctx.request_id, ctx.original_model, ctx.resolved_model
ctx.request_id, ctx.original_model, resolved_model
),
);
}
@@ -738,12 +712,12 @@ pub async fn chat_completions(
),
);
// 记录路由结果
// 记录路由结果（使用 selected_provider）
state.logs.write().await.add(
"info",
&format!(
"[ROUTE] request_id={} model={} provider={}",
ctx.request_id, ctx.resolved_model, provider
ctx.request_id, ctx.resolved_model, selected_provider
),
);
@@ -755,7 +729,7 @@ pub async fn chat_completions(
// 尝试从凭证池中选择凭证
// 如果指定了 X-Provider-Id，优先使用它（不降级）
// 否则使用路由规则选择的 provider，如果找不到再回退到 selected_provider
// 否则使用 selected_provider
eprintln!("[CHAT_COMPLETIONS] 开始选择凭证...");
let credential = match &state.db {
Some(db) => {
@@ -798,54 +772,30 @@ pub async fn chat_completions(
}
cred
} else {
// 原有逻辑：使用路由规则选择的 provider
let provider_str = provider.to_string();
// 使用 selected_provider（从 API Server 配置中获取）
eprintln!(
"[CHAT_COMPLETIONS] 尝试从凭证池选择: provider={}, model={}",
provider_str, request.model
selected_provider, request.model
);
let cred = state
.pool_service
.select_credential(db, &provider_str, Some(&request.model))
.select_credential(db, &selected_provider, Some(&request.model))
.ok()
.flatten();
if cred.is_some() {
eprintln!("[CHAT_COMPLETIONS] 找到凭证: provider={}", provider_str);
} else {
eprintln!("[CHAT_COMPLETIONS] 未找到凭证: provider={}", provider_str);
}
// 如果路由规则的 provider 没有找到凭证，回退到 selected_provider
if cred.is_none() && provider_str != selected_provider {
eprintln!(
"[CHAT_COMPLETIONS] 回退到 selected_provider: {}",
"[CHAT_COMPLETIONS] 找到凭证: provider={}",
selected_provider
);
state.logs.write().await.add(
"debug",
&format!(
"[ROUTE] No credential found for routed provider '{}', trying selected_provider '{}'",
provider_str, selected_provider
),
);
let fallback_cred = state
.pool_service
.select_credential(db, &selected_provider, Some(&request.model))
.ok()
.flatten();
if fallback_cred.is_some() {
eprintln!(
"[CHAT_COMPLETIONS] 回退凭证找到: provider={}",
selected_provider
);
} else {
eprintln!("[CHAT_COMPLETIONS] 回退凭证也未找到!");
}
fallback_cred
} else {
cred
eprintln!(
"[CHAT_COMPLETIONS] 未找到凭证: provider={}",
selected_provider
);
}
cred
}
}
None => {
@@ -1020,8 +970,15 @@ pub async fn chat_completions(
// 启动 Flow 捕获
let llm_request = build_llm_request_from_openai(&request, "/v1/chat/completions", &headers);
// 尝试将 selected_provider 解析为 ProviderType
// 如果是自定义 provider ID，则使用 OpenAI 作为默认值（因为大多数自定义 provider 使用 OpenAI 协议）
let provider_type = selected_provider
.parse::<ProviderType>()
.unwrap_or(ProviderType::OpenAI);
let flow_metadata = build_flow_metadata(
provider,
provider_type,
Some(&cred.uuid),
cred.name.as_deref(),
&headers,
@@ -1213,7 +1170,14 @@ pub async fn chat_completions(
// 启动 Flow 捕获（legacy mode）
let llm_request = build_llm_request_from_openai(&request, "/v1/chat/completions", &headers);
let flow_metadata = build_flow_metadata(provider, None, None, &headers, &ctx.request_id);
// 尝试将 selected_provider 解析为 ProviderType
// 如果是自定义 provider ID，则使用 OpenAI 作为默认值
let provider_type = selected_provider
.parse::<ProviderType>()
.unwrap_or(ProviderType::OpenAI);
let flow_metadata = build_flow_metadata(provider_type, None, None, &headers, &ctx.request_id);
let flow_id = state
.flow_monitor
.start_flow(llm_request.clone(), flow_metadata.clone())
@@ -1744,38 +1708,18 @@ pub async fn anthropic_messages(
),
);
// 使用 RequestProcessor 解析模型别名和路由
let provider = state.processor.resolve_and_route(&mut ctx).await;
// 如果没有设置默认 Provider，返回错误
let provider = match provider {
Some(p) => p,
None => {
state.logs.write().await.add(
"error",
&format!("[ROUTE] request_id={} 未设置默认 Provider", ctx.request_id),
);
return (
StatusCode::SERVICE_UNAVAILABLE,
Json(json!({
"error": {
"type": "configuration_error",
"message": "未设置默认 Provider，请先在设置中选择一个默认 Provider"
}
})),
)
.into_response();
}
};
// 使用 RequestProcessor 解析模型别名
let resolved_model = state.processor.resolve_model(&request.model).await;
ctx.set_resolved_model(resolved_model.clone());
// 更新请求中的模型名为解析后的模型
if ctx.resolved_model != ctx.original_model {
request.model = ctx.resolved_model.clone();
if resolved_model != request.model {
request.model = resolved_model.clone();
state.logs.write().await.add(
"info",
&format!(
"[MAPPER] request_id={} alias={} -> model={}",
ctx.request_id, ctx.original_model, ctx.resolved_model
ctx.request_id, ctx.original_model, resolved_model
),
);
}
@@ -1840,12 +1784,12 @@ pub async fn anthropic_messages(
),
);
// 记录路由结果
// 记录路由结果（使用 selected_provider）
state.logs.write().await.add(
"info",
&format!(
"[ROUTE] request_id={} model={} provider={}",
ctx.request_id, ctx.resolved_model, provider
ctx.request_id, ctx.resolved_model, selected_provider
),
);
@@ -1857,14 +1801,13 @@ pub async fn anthropic_messages(
// 尝试从凭证池中选择凭证
// 如果指定了 X-Provider-Id，优先使用它（不降级）
// 否则使用路由结果 provider（带智能降级）
let credential_provider = provider.to_string().to_lowercase();
// 否则使用 selected_provider
let credential = match &state.db {
Some(db) => {
// 如果指定了 X-Provider-Id，优先使用它（不降级）
if let Some(ref explicit_provider_id) = provider_id_header {
eprintln!(
"[AMP] 使用 X-Provider-Id 指定的 provider: {}",
"[ANTHROPIC_MESSAGES] 使用 X-Provider-Id 指定的 provider: {}",
explicit_provider_id
);
let cred = state
@@ -1899,97 +1842,77 @@ pub async fn anthropic_messages(
}
cred
} else {
// 原有逻辑：根据路由结果选择凭证（带智能降级）
state
// 使用 selected_provider（从 API Server 配置中获取）
eprintln!(
"[ANTHROPIC_MESSAGES] 尝试从凭证池选择: provider={}, model={}",
selected_provider, request.model
);
let cred = state
.pool_service
.select_credential_with_fallback(
db,
&state.api_key_service,
&credential_provider,
Some(&request.model),
None, // provider_id_hint 可从路由或请求头提取
)
.select_credential(db, &selected_provider, Some(&request.model))
.ok()
.flatten()
.flatten();
if cred.is_some() {
eprintln!(
"[ANTHROPIC_MESSAGES] 找到凭证: provider={}",
selected_provider
);
} else {
eprintln!(
"[ANTHROPIC_MESSAGES] 未找到凭证: provider={}",
selected_provider
);
}
cred
}
}
None => None,
};
// 如果 Provider Pool 中没有找到凭证，尝试从 API Key Provider 获取
let credential = if credential.is_none() {
// 根据路由结果映射到 ApiProviderType
use crate::database::dao::api_key_provider::ApiProviderType;
let api_provider_type = match credential_provider.as_str() {
"anthropic" | "claude" => Some(ApiProviderType::Anthropic),
"openai" => Some(ApiProviderType::Openai),
"gemini" => Some(ApiProviderType::Gemini),
_ => None,
};
if let (Some(db), Some(api_type)) = (&state.db, api_provider_type) {
// 使用按类型获取的方法（包括自定义 Provider）
match state.api_key_service.get_next_api_key_by_type(db, api_type) {
Ok(Some((_key_id, api_key, provider_info))) => {
state.logs.write().await.add(
"info",
&format!(
"[ROUTE] Using API Key Provider credential: provider={}, type={:?}, api_host={}",
provider_info.name, provider_info.provider_type, provider_info.api_host
),
);
let base_url = if provider_info.api_host.is_empty() {
None
} else {
Some(provider_info.api_host.clone())
};
let provider_type = match provider_info.provider_type {
ApiProviderType::Anthropic => crate::ProviderType::Anthropic,
ApiProviderType::Openai | ApiProviderType::OpenaiResponse => {
crate::ProviderType::OpenAI
}
ApiProviderType::Gemini => crate::ProviderType::GeminiApiKey,
_ => crate::ProviderType::OpenAI,
};
// 根据 provider_type 创建对应的 CredentialData
let credential_data = match provider_type {
crate::ProviderType::Anthropic => {
crate::models::provider_pool_model::CredentialData::AnthropicKey {
api_key: api_key.clone(),
base_url,
}
}
crate::ProviderType::GeminiApiKey => {
crate::models::provider_pool_model::CredentialData::GeminiApiKey {
api_key: api_key.clone(),
base_url,
excluded_models: vec![],
}
}
_ => crate::models::provider_pool_model::CredentialData::OpenAIKey {
api_key: api_key.clone(),
base_url,
},
};
// 构建 ProviderCredential
let mut cred = crate::models::provider_pool_model::ProviderCredential::new(
provider_type,
credential_data,
);
cred.name = Some(provider_info.name.clone());
Some(cred)
}
Ok(None) => None,
Err(_) => None,
}
} else {
None => {
eprintln!("[ANTHROPIC_MESSAGES] 数据库未初始化!");
None
}
};
// 如果 Provider Pool 中没有找到凭证，尝试从 API Key Provider 获取（智能降级）
let credential = if credential.is_none() {
eprintln!("[ANTHROPIC_MESSAGES] Provider Pool 中未找到凭证，尝试 API Key Provider...");
// 策略 1: 优先按 provider_id 直接查找（支持自定义 Provider）
let mut found_credential: Option<crate::models::provider_pool_model::ProviderCredential> =
None;
if let Some(db) = &state.db {
eprintln!(
"[ANTHROPIC_MESSAGES] 尝试按 provider_id '{}' 直接查找凭证",
selected_provider
);
match state.api_key_service.get_fallback_credential(
db,
&crate::models::provider_pool_model::PoolProviderType::Anthropic,
Some(&selected_provider),
) {
Ok(Some(cred)) => {
eprintln!(
"[ANTHROPIC_MESSAGES] 通过 provider_id '{}' 找到凭证: name={:?}",
selected_provider, cred.name
);
found_credential = Some(cred);
}
Ok(None) => {
eprintln!(
"[ANTHROPIC_MESSAGES] provider_id '{}' 未找到凭证",
selected_provider
);
}
Err(e) => {
eprintln!("[ANTHROPIC_MESSAGES] 查找凭证时出错: {}", e);
}
}
}
found_credential
} else {
credential
};
@@ -2008,8 +1931,15 @@ pub async fn anthropic_messages(
// 启动 Flow 捕获
let llm_request = build_llm_request_from_anthropic(&request, "/v1/messages", &headers);
// 尝试将 selected_provider 解析为 ProviderType
// 如果是自定义 provider ID，则使用 OpenAI 作为默认值
let provider_type = selected_provider
.parse::<ProviderType>()
.unwrap_or(ProviderType::OpenAI);
let flow_metadata = build_flow_metadata(
provider,
provider_type,
Some(&cred.uuid),
cred.name.as_deref(),
&headers,
@@ -2198,7 +2128,14 @@ pub async fn anthropic_messages(
// 启动 Flow 捕获（legacy mode）
let llm_request = build_llm_request_from_anthropic(&request, "/v1/messages", &headers);
let flow_metadata = build_flow_metadata(provider, None, None, &headers, &ctx.request_id);
// 尝试将 selected_provider 解析为 ProviderType
// 如果是自定义 provider ID，则使用 OpenAI 作为默认值
let provider_type = selected_provider
.parse::<ProviderType>()
.unwrap_or(ProviderType::OpenAI);
let flow_metadata = build_flow_metadata(provider_type, None, None, &headers, &ctx.request_id);
let flow_id = state
.flow_monitor
.start_flow(llm_request.clone(), flow_metadata.clone())
src-tauri/src/server/handlers/provider_calls.rs
+6 -2
View File
@@ -463,6 +463,9 @@ pub async fn call_provider_anthropic(
if resp.status().is_success() {
match resp.text().await {
Ok(body) => {
// 记录原始响应以便调试
eprintln!("[PROVIDER_CALL] OpenAI 响应: {}", &body[..body.len().min(500)]);
if let Ok(openai_resp) =
serde_json::from_str::<serde_json::Value>(&body)
{
@@ -491,7 +494,8 @@ pub async fn call_provider_anthropic(
build_anthropic_response(&request.model, &parsed)
}
} else {
// 记录解析失败
// 记录解析失败和原始响应
eprintln!("[PROVIDER_CALL] 解析 OpenAI 响应失败，原始响应: {}", &body);
if let Some(db) = &state.db {
let _ = state.pool_service.mark_unhealthy(
db,
@@ -501,7 +505,7 @@ pub async fn call_provider_anthropic(
}
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(serde_json::json!({"error": {"message": "Failed to parse OpenAI response"}})),
Json(serde_json::json!({"error": {"message": format!("Failed to parse OpenAI response. Body: {}", &body[..body.len().min(200)])}})),
)
.into_response()
}
src-tauri/src/server/mod.rs
+140 -29
View File
@@ -315,13 +315,31 @@ impl ServerState {
// 从配置初始化 Router 的默认 Provider
{
let default_provider_str = &config.routing.default_provider;
if let Ok(provider_type) = default_provider_str.parse::<crate::ProviderType>() {
let mut router = processor.router.write().await;
router.set_default_provider(provider_type);
tracing::info!(
"[SERVER] 从配置初始化 Router 默认 Provider: {}",
default_provider_str
);
// 尝试解析为 ProviderType 枚举
match default_provider_str.parse::<crate::ProviderType>() {
Ok(provider_type) => {
let mut router = processor.router.write().await;
router.set_default_provider(provider_type);
tracing::info!(
"[SERVER] 从配置初始化 Router 默认 Provider: {} (ProviderType)",
default_provider_str
);
}
Err(_) => {
// 如果解析失败，可能是自定义 provider ID
// 这种情况下，路由器保持空状态，请求会直接使用 provider_id 进行凭证查找
tracing::warn!(
"[SERVER] 配置的默认 Provider '{}' 不是有效的 ProviderType 枚举值，可能是自定义 Provider ID。\
路由器将保持空状态，请求将直接使用 provider_id 进行凭证查找。",
default_provider_str
);
eprintln!(
"[SERVER] 警告：默认 Provider '{}' 不是标准 Provider 类型（kiro/openai/claude等），\
可能是自定义 Provider ID。如果这是预期行为，请忽略此警告。",
default_provider_str
);
}
}
}
@@ -592,16 +610,29 @@ async fn update_processor_config(processor: &RequestProcessor, config: &Config)
// 更新路由器默认 Provider
{
let mut router = processor.router.write().await;
if let Ok(provider_type) = config
// 尝试解析为 ProviderType 枚举
match config
.routing
.default_provider
.parse::<crate::ProviderType>()
{
router.set_default_provider(provider_type);
tracing::debug!(
"[HOT_RELOAD] 路由器默认 Provider 已更新: {}",
config.routing.default_provider
);
Ok(provider_type) => {
router.set_default_provider(provider_type);
tracing::debug!(
"[HOT_RELOAD] 路由器默认 Provider 已更新: {} (ProviderType)",
config.routing.default_provider
);
}
Err(_) => {
// 如果解析失败，可能是自定义 provider ID
// 清空路由器的默认 provider，让请求直接使用 provider_id
tracing::warn!(
"[HOT_RELOAD] 配置的默认 Provider '{}' 不是有效的 ProviderType 枚举值，可能是自定义 Provider ID。\
路由器默认 Provider 将被清空。",
config.routing.default_provider
);
}
}
}
@@ -729,13 +760,29 @@ async fn run_server(
// 从配置初始化 Router 的默认 Provider
if let Some(cfg) = &config {
let default_provider_str = &cfg.routing.default_provider;
if let Ok(provider_type) = default_provider_str.parse::<crate::ProviderType>() {
let mut router = processor.router.write().await;
router.set_default_provider(provider_type);
tracing::info!(
"[SERVER] 从配置初始化 Router 默认 Provider: {}",
default_provider_str
);
// 尝试解析为 ProviderType 枚举
match default_provider_str.parse::<crate::ProviderType>() {
Ok(provider_type) => {
let mut router = processor.router.write().await;
router.set_default_provider(provider_type);
tracing::info!(
"[SERVER] 从配置初始化 Router 默认 Provider: {} (ProviderType)",
default_provider_str
);
}
Err(_) => {
// 如果解析失败，可能是自定义 provider ID
tracing::warn!(
"[SERVER] 配置的默认 Provider '{}' 不是有效的 ProviderType 枚举值，可能是自定义 Provider ID。\
路由器将保持空状态，请求将直接使用 provider_id 进行凭证查找。",
default_provider_str
);
eprintln!(
"[SERVER] 警告：默认 Provider '{}' 不是标准 Provider 类型，可能是自定义 Provider ID",
default_provider_str
);
}
}
}
@@ -1496,7 +1543,7 @@ async fn amp_chat_completions(
),
);
// 尝试根据 provider 名称选择凭证（不降级，指定什么就用什么）
// 尝试根据 provider 名称选择凭证
eprintln!(
"[AMP] 开始查找凭证: provider={}, model={}, db={}",
provider,
@@ -1506,11 +1553,11 @@ async fn amp_chat_completions(
let credential = match &state.db {
Some(db) => {
eprintln!(
"[AMP] 使用 select_credential 查找凭证（不降级）: provider={}",
"[AMP] 使用 select_credential 查找凭证（Provider Pool）: provider={}",
provider
);
// 使用 select_credential 而不是 select_credential_with_fallback，禁止降级
if let Ok(Some(cred)) =
// 先尝试从 Provider Pool 查找
let pool_cred = if let Ok(Some(cred)) =
state
.pool_service
.select_credential(db, &provider, Some(&request.model))
@@ -1528,11 +1575,39 @@ async fn amp_chat_completions(
eprintln!("[AMP] get_by_uuid 找到凭证: {:?}", cred.name);
Some(cred)
} else {
None
};
// 如果 Provider Pool 中没有找到，尝试从 API Key Provider 查找
if pool_cred.is_none() {
eprintln!(
"[AMP] 未找到任何凭证 for provider '{}'，不进行降级",
"[AMP] Provider Pool 中未找到凭证，尝试 API Key Provider: provider={}",
provider
);
None
match state.api_key_service.get_fallback_credential(
db,
&crate::models::provider_pool_model::PoolProviderType::OpenAI,
Some(&provider),
) {
Ok(Some(cred)) => {
eprintln!(
"[AMP] 通过 provider_id '{}' 找到 API Key Provider 凭证: name={:?}",
provider, cred.name
);
Some(cred)
}
Ok(None) => {
eprintln!("[AMP] 未找到任何凭证 for provider '{}'", provider);
None
}
Err(e) => {
eprintln!("[AMP] 查找 API Key Provider 凭证时出错: {}", e);
None
}
}
} else {
pool_cred
}
}
None => {
@@ -1623,11 +1698,11 @@ async fn amp_messages(
),
);
// 尝试根据 provider 名称选择凭证（不降级，指定什么就用什么）
// 尝试根据 provider 名称选择凭证
let credential = match &state.db {
Some(db) => {
// 使用 select_credential 而不是 select_credential_with_fallback，禁止降级
if let Ok(Some(cred)) =
// 先尝试从 Provider Pool 查找
let pool_cred = if let Ok(Some(cred)) =
state
.pool_service
.select_credential(db, &provider, Some(&request.model))
@@ -1643,6 +1718,38 @@ async fn amp_messages(
Some(cred)
} else {
None
};
// 如果 Provider Pool 中没有找到，尝试从 API Key Provider 查找
if pool_cred.is_none() {
eprintln!(
"[AMP_MESSAGES] Provider Pool 中未找到凭证，尝试 API Key Provider: provider={}",
provider
);
match state.api_key_service.get_fallback_credential(
db,
&crate::models::provider_pool_model::PoolProviderType::Anthropic,
Some(&provider),
) {
Ok(Some(cred)) => {
eprintln!(
"[AMP_MESSAGES] 通过 provider_id '{}' 找到 API Key Provider 凭证: name={:?}",
provider, cred.name
);
Some(cred)
}
Ok(None) => {
eprintln!("[AMP_MESSAGES] 未找到任何凭证 for provider '{}'", provider);
None
}
Err(e) => {
eprintln!("[AMP_MESSAGES] 查找 API Key Provider 凭证时出错: {}", e);
None
}
}
} else {
pool_cred
}
}
None => None,
@@ -1887,6 +1994,8 @@ async fn amp_management_proxy_internal(
}
/// 内部 Anthropic messages 处理 (使用默认 Kiro)
/// 预留：用于内部直接调用 Kiro API
#[allow(dead_code)]
async fn anthropic_messages_internal(
state: &AppState,
request: &AnthropicMessagesRequest,
@@ -1954,6 +2063,8 @@ async fn anthropic_messages_internal(
}
/// 内部 OpenAI chat completions 处理 (使用默认 Kiro)
/// 预留：用于内部直接调用 Kiro API
#[allow(dead_code)]
async fn chat_completions_internal(state: &AppState, request: &ChatCompletionRequest) -> Response {
{
let _guard = state.kiro_refresh_lock.lock().await;
src-tauri/src/services/model_registry_service.rs
+111 -53
View File
@@ -122,59 +122,37 @@ impl ModelRegistryService {
pub async fn initialize(&self) -> Result<(), String> {
tracing::info!("[ModelRegistry] 初始化模型注册服务");
// 1. 首先尝试从内嵌资源加载
match self.load_from_embedded_resources().await {
Ok((models, aliases)) => {
tracing::info!(
"[ModelRegistry] 从内嵌资源加载了 {} 个模型, {} 个别名配置",
models.len(),
aliases.len()
);
// 始终从内嵌资源加载，不再回退到数据库
let (models, aliases) = self.load_from_embedded_resources().await?;
// 更新缓存
{
let mut cache = self.models_cache.write().await;
*cache = models.clone();
}
{
let mut cache = self.aliases_cache.write().await;
*cache = aliases;
}
tracing::info!(
"[ModelRegistry] 从内嵌资源加载了 {} 个模型, {} 个别名配置",
models.len(),
aliases.len()
);
// 更新同步状态
{
let mut state = self.sync_state.write().await;
state.model_count = models.len() as u32;
state.last_sync_at = Some(chrono::Utc::now().timestamp());
state.is_syncing = false;
state.last_error = None;
}
// 保存到数据库
if let Err(e) = self.save_models_to_db(&models).await {
tracing::warn!("[ModelRegistry] 保存模型到数据库失败: {}", e);
}
return Ok(());
}
Err(e) => {
tracing::warn!("[ModelRegistry] 从内嵌资源加载失败: {}", e);
}
// 更新缓存
{
let mut cache = self.models_cache.write().await;
*cache = models.clone();
}
{
let mut cache = self.aliases_cache.write().await;
*cache = aliases;
}
// 2. 回退到从数据库加载
match self.load_from_db().await {
Ok(models) if !models.is_empty() => {
tracing::info!("[ModelRegistry] 从数据库加载了 {} 个模型", models.len());
let mut cache = self.models_cache.write().await;
*cache = models;
}
Ok(_) => {
tracing::warn!("[ModelRegistry] 数据库中没有模型数据");
}
Err(e) => {
tracing::error!("[ModelRegistry] 从数据库加载失败: {}", e);
}
// 更新同步状态
{
let mut state = self.sync_state.write().await;
state.model_count = models.len() as u32;
state.last_sync_at = Some(chrono::Utc::now().timestamp());
state.is_syncing = false;
state.last_error = None;
}
// 保存到数据库（仅用于持久化，不影响运行时数据）
if let Err(e) = self.save_models_to_db(&models).await {
tracing::warn!("[ModelRegistry] 保存模型到数据库失败: {}", e);
}
Ok(())
@@ -195,9 +173,18 @@ impl ModelRegistryService {
.as_ref()
.ok_or_else(|| "资源目录未设置".to_string())?;
tracing::info!("[ModelRegistry] resource_dir: {:?}", resource_dir);
let models_dir = resource_dir.join(MODELS_RESOURCE_DIR);
let index_file = models_dir.join("index.json");
tracing::info!("[ModelRegistry] models_dir: {:?}", models_dir);
tracing::info!(
"[ModelRegistry] index_file: {:?}, exists: {}",
index_file,
index_file.exists()
);
if !index_file.exists() {
return Err(format!("索引文件不存在: {:?}", index_file));
}
@@ -218,8 +205,11 @@ impl ModelRegistryService {
let now = chrono::Utc::now().timestamp();
let providers_dir = models_dir.join("providers");
tracing::info!("[ModelRegistry] providers_dir: {:?}", providers_dir);
for provider_id in &index.providers {
let provider_file = providers_dir.join(format!("{}.json", provider_id));
if !provider_file.exists() {
tracing::warn!("[ModelRegistry] Provider 文件不存在: {:?}", provider_file);
continue;
@@ -248,10 +238,36 @@ impl ModelRegistryService {
}
}
// 去重：使用 HashMap 按 id 去重，保留第一个出现的模型
let mut seen_ids = std::collections::HashSet::new();
// 去重：优先保留 provider_id 为 "anthropic" 的模型
// 对于相同 ID 的模型，anthropic 官方的优先级最高
let mut seen_ids: std::collections::HashMap<String, usize> =
std::collections::HashMap::new();
let original_count = models.len();
models.retain(|m| seen_ids.insert(m.id.clone()));
let mut to_keep = vec![true; models.len()];
for (idx, model) in models.iter().enumerate() {
if let Some(&existing_idx) = seen_ids.get(&model.id) {
// 已经有相同 ID 的模型
let existing_model = &models[existing_idx];
// 如果当前模型是 anthropic 官方的，替换之前的
if model.provider_id == "anthropic" && existing_model.provider_id != "anthropic" {
to_keep[existing_idx] = false;
seen_ids.insert(model.id.clone(), idx);
} else {
// 否则保留第一个
to_keep[idx] = false;
}
} else {
seen_ids.insert(model.id.clone(), idx);
}
}
models = models
.into_iter()
.enumerate()
.filter_map(|(idx, model)| if to_keep[idx] { Some(model) } else { None })
.collect();
if models.len() < original_count {
tracing::warn!(
@@ -357,7 +373,8 @@ impl ModelRegistryService {
}
}
/// 从数据库加载模型
/// 从数据库加载模型（预留，将来实现从数据库加载自定义模型）
#[allow(dead_code)]
async fn load_from_db(&self) -> Result<Vec<EnhancedModelMetadata>, String> {
let (models, sync_rows) = {
let conn = self.db.lock().map_err(|e| e.to_string())?;
@@ -511,6 +528,47 @@ impl ModelRegistryService {
self.sync_state.read().await.clone()
}
/// 强制从内嵌资源重新加载模型数据
///
/// 清除数据库缓存并重新从资源文件加载最新的模型数据
pub async fn force_reload(&self) -> Result<u32, String> {
tracing::info!("[ModelRegistry] 强制重新加载模型数据");
// 从内嵌资源加载
let (models, aliases) = self.load_from_embedded_resources().await?;
let model_count = models.len() as u32;
tracing::info!(
"[ModelRegistry] 从内嵌资源加载了 {} 个模型, {} 个别名配置",
models.len(),
aliases.len()
);
// 更新缓存
{
let mut cache = self.models_cache.write().await;
*cache = models.clone();
}
{
let mut cache = self.aliases_cache.write().await;
*cache = aliases;
}
// 更新同步状态
{
let mut state = self.sync_state.write().await;
state.model_count = model_count;
state.last_sync_at = Some(chrono::Utc::now().timestamp());
state.is_syncing = false;
state.last_error = None;
}
// 保存到数据库
self.save_models_to_db(&models).await?;
Ok(model_count)
}
/// 按 Provider 获取模型
pub async fn get_models_by_provider(&self, provider_id: &str) -> Vec<EnhancedModelMetadata> {
self.models_cache
src-tauri/src/stream/generators/openai_sse.rs
+2 -1
View File
@@ -11,7 +11,7 @@
//! data: [DONE]
//! ```
use crate::stream::events::{ContentBlockType, StopReason, StreamEvent};
use crate::stream::events::{ContentBlockType, StreamEvent};
use serde::Serialize;
use std::collections::HashMap;
use std::time::{SystemTime, UNIX_EPOCH};
@@ -346,6 +346,7 @@ struct OpenAiFunctionDelta<'a> {
#[cfg(test)]
mod tests {
use super::*;
use crate::stream::events::StopReason;
#[test]
fn test_generate_text_delta() {
src-tauri/src/terminal/integration/osc_parser.rs
+2
View File
@@ -27,6 +27,8 @@ const OSC_START: &[u8] = b"\x1b]";
/// OSC 序列结束标记 - BEL
const OSC_END_BEL: u8 = 0x07;
/// OSC 序列结束标记 - ST (String Terminator)
/// 预留：某些终端使用 ST 而不是 BEL 作为结束标记
#[allow(dead_code)]
const OSC_END_ST: &[u8] = b"\x1b\\";
/// OSC 序列类型
src-tauri/tauri.conf.json
+20 -3
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "ProxyCast",
"version": "0.36.3",
"version": "0.37.0",
"identifier": "com.proxycast.app",
"build": {
"beforeDevCommand": "npm run dev",
@@ -21,11 +21,28 @@
"resizable": true,
"fullscreen": false,
"center": true
},
{
"label": "screenshot-chat",
"title": "",
"url": "/screenshot-chat",
"width": 645,
"height": 70,
"resizable": false,
"decorations": false,
"alwaysOnTop": true,
"center": true,
"visible": false,
"focus": true,
"skipTaskbar": true,
"transparent": true,
"shadow": false
}
],
"security": {
"csp": "default-src 'self'; img-src 'self' data: https:; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline'; connect-src 'self' http://localhost:* ws://localhost:* http://127.0.0.1:* ws://127.0.0.1:* http://[::1]:* ws://[::1]:* https:; frame-src https: http:; object-src 'none'"
}
"csp": "default-src 'self' tauri:; img-src 'self' asset: tauri: data: https: blob:; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline'; connect-src 'self' tauri: http://localhost:* ws://localhost:* http://127.0.0.1:* ws://127.0.0.1:* http://[::1]:* ws://[::1]:* https:; frame-src https: http:; object-src 'none'"
},
"macOSPrivateApi": true
},
"bundle": {
"active": true,
src/App.tsx
+2 -2
View File
@@ -137,8 +137,8 @@ function AppContent() {
if (currentPage.startsWith("plugin:")) {
const pluginId = currentPage.slice(7); // 移除 "plugin:" 前缀
// 需要全屏显示的插件列表
const fullscreenPlugins = ["terminal-plugin"];
// 需要全屏显示的插件列表（终端已内置，不再需要插件）
const fullscreenPlugins: string[] = [];
const isFullscreen = fullscreenPlugins.includes(pluginId);
if (isFullscreen) {
src/RootRouter.tsx
+31
View File
@@ -0,0 +1,31 @@
/**
* @file RootRouter.tsx
* @description 根路由组件 - 根据 URL 路径渲染对应的组件
*/
import App from "./App";
import { ScreenshotChatPage } from "./pages/screenshot-chat";
import { Toaster } from "./components/ui/sonner";
/**
* 根据 URL 路径渲染对应的组件
*
* - /screenshot-chat: 截图对话悬浮窗口（独立 Tauri 窗口）
* - 其他: 主应用
*/
export function RootRouter() {
const pathname = window.location.pathname;
// 截图对话悬浮窗口路由
if (pathname === "/screenshot-chat") {
return <ScreenshotChatPage />;
}
// 默认渲染主应用
return (
<>
<App />
<Toaster />
</>
);
}
src/components/README.md
+1
View File
@@ -22,6 +22,7 @@ React 组件层，包含 UI 组件和业务组件。
- `provider-pool/` - Provider 凭证池管理组件
- `resilience/` - 弹性策略配置组件
- `routing/` - 路由规则配置组件
- `screenshot-chat/` - 截图对话功能组件（实验室功能）
- `settings/` - 设置页面组件
- `skills/` - 技能管理组件
- `switch/` - 开关控制组件
src/components/agent/chat/components/ChatNavbar.tsx
+20 -31
View File
@@ -213,46 +213,35 @@ export const ChatNavbar: React.FC<ChatNavbarProps> = ({
// 从 model_registry 获取模型
// 优先使用 registryId，如果没有模型则回退到 fallbackRegistryId
let models = registryModels
.filter((m) => m.provider_id === selectedProvider.registryId)
.map((m) => m.id);
let models = registryModels.filter(
(m) => m.provider_id === selectedProvider.registryId,
);
// 如果没有找到模型，尝试使用 fallbackRegistryId
if (models.length === 0 && selectedProvider.fallbackRegistryId) {
models = registryModels
.filter((m) => m.provider_id === selectedProvider.fallbackRegistryId)
.map((m) => m.id);
models = registryModels.filter(
(m) => m.provider_id === selectedProvider.fallbackRegistryId,
);
}
// 按照模型名称排序，优先显示最新版本
// 排序规则：
// 1. 带日期后缀的模型（如 claude-opus-4-5-20251101）按日期降序
// 2. 带 "latest" 后缀的模型排在最前面
// 3. 其他模型按字母顺序
return models.sort((a, b) => {
const aIsLatest = a.includes("-latest");
const bIsLatest = b.includes("-latest");
// 按照 release_date 和 is_latest 排序
const sortedModels = [...models].sort((a, b) => {
// 1. is_latest 优先
if (a.is_latest && !b.is_latest) return -1;
if (!a.is_latest && b.is_latest) return 1;
// latest 版本排在最前面
if (aIsLatest && !bIsLatest) return -1;
if (!aIsLatest && bIsLatest) return 1;
// 提取日期后缀（如 20251101）
const dateRegex = /-(\d{8})$/;
const aMatch = a.match(dateRegex);
const bMatch = b.match(dateRegex);
if (aMatch && bMatch) {
// 两个都有日期，按日期降序（最新的在前）
return bMatch[1].localeCompare(aMatch[1]);
// 2. 按 release_date 降序（最新的在前）
if (a.release_date && b.release_date) {
return b.release_date.localeCompare(a.release_date);
}
if (a.release_date && !b.release_date) return -1;
if (!a.release_date && b.release_date) return 1;
if (aMatch && !bMatch) return -1; // 有日期的排在前面
if (!aMatch && bMatch) return 1;
// 其他情况按字母降序（通常版本号大的在前）
return b.localeCompare(a);
// 3. 按 display_name 字母序
return a.display_name.localeCompare(b.display_name);
});
return sortedModels.map((m) => m.id);
}, [selectedProvider, registryModels, aliasConfig]);
// 初始化：优先选择服务器默认 Provider，否则选择第一个已配置的
src/components/agent/chat/hooks/useAgentChat.ts
+37
View File
@@ -197,6 +197,43 @@ export function useAgentChat() {
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
// 监听截图对话消息事件
useEffect(() => {
let unlisten: UnlistenFn | null = null;
const setupListener = async () => {
unlisten = await listen<{
message: string;
image_path: string | null;
image_base64: string | null;
}>("screenshot-chat-message", async (event) => {
console.log("[AgentChat] 收到截图对话消息:", event.payload);
const { message, image_base64 } = event.payload;
// 构建图片数组
const images: MessageImage[] = [];
if (image_base64) {
images.push({
data: image_base64,
mediaType: "image/png",
});
}
// 发送消息
await sendMessage(message, images, false, false);
});
};
setupListener();
return () => {
if (unlisten) {
unlisten();
}
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [providerType, model, sessionId]);
// 当 sessionId 变化时刷新话题列表
useEffect(() => {
if (sessionId) {
src/components/plugins/PluginUIRenderer.tsx
+1 -2
View File
@@ -13,7 +13,6 @@ import { invoke } from "@tauri-apps/api/core";
import { BrowserInterceptorTool } from "@/components/tools/browser-interceptor/BrowserInterceptorTool";
import { FlowMonitorPage } from "@/pages";
import { ConfigManagementPage } from "@/components/config/ConfigManagementPage";
import { TerminalPage } from "@/components/terminal";
import { PluginUIRenderer as DynamicPluginRenderer } from "@/lib/plugin-loader/PluginUIRenderer";
import { usePluginSDK } from "@/lib/plugin-sdk";
import { Button } from "@/components/ui/button";
@@ -169,6 +168,7 @@ function PluginLauncher({
/**
* 内置插件组件映射
* 注意: machine-id-tool 已移除，改为从插件包动态加载
* 注意: terminal-plugin 已移除，终端功能已内置到应用中
*/
const builtinPluginComponents: Record<
string,
@@ -177,7 +177,6 @@ const builtinPluginComponents: Record<
"browser-interception": BrowserInterceptorTool,
"flow-monitor": FlowMonitorPage,
"config-switch": ConfigManagementPage,
"terminal-plugin": TerminalPage,
};
/**
src/components/screenshot-chat/ChatInput.tsx
+107
View File
@@ -0,0 +1,107 @@
/**
* @file ChatInput.tsx
* @description 聊天输入框组件，支持文本输入和发送
* @module components/screenshot-chat/ChatInput
*/
import React, { useCallback, useRef, useEffect } from "react";
import type { ChatInputProps } from "./types";
/**
* 聊天输入框组件
*
* 提供文本输入框和发送按钮，支持 Enter 键发送
*
* 需求:
* - 4.3: 悬浮窗口应提供文本输入框供用户输入问题
* - 4.4: 当用户按下 Enter 或点击发送时，悬浮窗口应将图片和文本发送给 AI
*/
export const ChatInput: React.FC<ChatInputProps> = ({
value,
onChange,
onSend,
disabled = false,
isLoading = false,
placeholder = "输入问题...",
}) => {
const inputRef = useRef<HTMLInputElement>(null);
// 自动聚焦输入框
useEffect(() => {
if (!disabled && inputRef.current) {
inputRef.current.focus();
}
}, [disabled]);
// 处理键盘事件
const handleKeyDown = useCallback(
(e: React.KeyboardEvent<HTMLInputElement>) => {
if (e.key === "Enter" && !e.shiftKey && !disabled && !isLoading) {
e.preventDefault();
if (value.trim()) {
onSend();
}
}
},
[value, onSend, disabled, isLoading],
);
// 处理输入变化
const handleChange = useCallback(
(e: React.ChangeEvent<HTMLInputElement>) => {
onChange(e.target.value);
},
[onChange],
);
// 处理发送按钮点击
const handleSendClick = useCallback(() => {
if (value.trim() && !disabled && !isLoading) {
onSend();
}
}, [value, onSend, disabled, isLoading]);
const canSend = value.trim() && !disabled && !isLoading;
return (
<div className="screenshot-chat-input-area">
<input
ref={inputRef}
type="text"
className="screenshot-chat-input"
value={value}
onChange={handleChange}
onKeyDown={handleKeyDown}
placeholder={placeholder}
disabled={disabled || isLoading}
autoFocus
/>
<button
className="screenshot-chat-send-btn"
onClick={handleSendClick}
disabled={!canSend}
title="发送 (Enter)"
>
{isLoading ? (
<span
className="screenshot-chat-loading-spinner"
style={{ width: 16, height: 16 }}
/>
) : (
<svg
className="w-4 h-4"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
>
<line x1="22" y1="2" x2="11" y2="13" />
<polygon points="22 2 15 22 11 13 2 9 22 2" />
</svg>
)}
</button>
</div>
);
};
export default ChatInput;
src/components/screenshot-chat/ChatMessages.tsx
+104
View File
@@ -0,0 +1,104 @@
/**
* @file ChatMessages.tsx
* @description 消息列表组件，显示用户和 AI 的对话消息
* @module components/screenshot-chat/ChatMessages
*/
import React, { useRef, useEffect } from "react";
import ReactMarkdown from "react-markdown";
import remarkGfm from "remark-gfm";
import type { ChatMessagesProps, ChatMessage } from "./types";
import "./screenshot-chat.css";
/**
* 单条消息组件
*/
const MessageItem: React.FC<{ message: ChatMessage }> = ({ message }) => {
const isUser = message.role === "user";
return (
<div
className={`screenshot-chat-message ${isUser ? "screenshot-chat-message-user" : "screenshot-chat-message-assistant"}`}
>
{/* 用户消息显示图片 */}
{isUser && message.image && (
<div className="screenshot-chat-message-image">
<img
src={`data:${message.image.mediaType};base64,${message.image.data}`}
alt="截图"
className="screenshot-chat-message-thumbnail"
/>
</div>
)}
{/* 消息内容 */}
<div className="screenshot-chat-message-content">
{message.isThinking ? (
<div className="screenshot-chat-thinking">
<span className="screenshot-chat-loading-spinner" />
<span>{message.thinkingContent || "思考中..."}</span>
</div>
) : isUser ? (
<p>{message.content}</p>
) : (
<div className="screenshot-chat-markdown">
<ReactMarkdown remarkPlugins={[remarkGfm]}>
{message.content}
</ReactMarkdown>
</div>
)}
</div>
{/* 时间戳 */}
<div className="screenshot-chat-message-time">
{new Date(message.timestamp).toLocaleTimeString("zh-CN", {
hour: "2-digit",
minute: "2-digit",
})}
</div>
</div>
);
};
/**
* 消息列表组件
*
* 显示用户消息和 AI 回复，支持 Markdown 渲染和自动滚动
*
* 需求:
* - 4.5: 悬浮窗口应在可滚动区域显示 AI 回复
* - 5.4: 当 AI 回复时，悬浮窗口应以 Markdown 格式渲染回复内容
*/
export const ChatMessages: React.FC<ChatMessagesProps> = ({
messages,
className = "",
}) => {
const containerRef = useRef<HTMLDivElement>(null);
// 自动滚动到底部
useEffect(() => {
if (containerRef.current) {
containerRef.current.scrollTop = containerRef.current.scrollHeight;
}
}, [messages]);
if (messages.length === 0) {
return (
<div className={`screenshot-chat-messages ${className}`}>
<div className="screenshot-chat-placeholder">
输入问题，开始与 AI 讨论截图内容
</div>
</div>
);
}
return (
<div ref={containerRef} className={`screenshot-chat-messages ${className}`}>
{messages.map((message) => (
<MessageItem key={message.id} message={message} />
))}
</div>
);
};
export default ChatMessages;
src/components/screenshot-chat/README.md
+150
View File
@@ -0,0 +1,150 @@
# 截图对话组件 (screenshot-chat)
截图对话功能的前端组件模块，提供截图预览、对话输入、消息展示等功能。
## 文件索引
| 文件 | 描述 |
| -------------------------- | ----------------------------------- |
| `index.ts` | 模块导出入口 |
| `types.ts` | 类型定义（配置、消息、组件 Props） |
| `useScreenshotChat.ts` | 核心 Hook，管理消息、图片和 AI 通信 |
| `ScreenshotPreview.tsx` | 截图预览组件，支持缩放和拖拽查看 |
| `ChatInput.tsx` | 聊天输入框组件，支持 Enter 发送 |
| `ChatMessages.tsx` | 消息列表组件，支持 Markdown 渲染 |
| `ScreenshotChatWindow.tsx` | 悬浮窗主组件，组合所有子组件 |
| `ShortcutSettings.tsx` | 快捷键设置组件，支持录制模式 |
| `screenshot-chat.css` | 截图对话组件样式 |
## 组件说明
### ScreenshotChatWindow
悬浮窗主组件，组合截图预览、消息列表和输入框。
**功能特性:**
- 组合 ScreenshotPreview, ChatInput, ChatMessages
- 支持 ESC 键关闭窗口
- 支持窗口拖动（通过 header 区域）
- 显示错误信息和重试按钮
**Props:**
- `imagePath`: 截图文件路径
- `onClose`: 关闭窗口回调（可选）
### ScreenshotPreview
截图预览组件，用于在悬浮对话窗口中显示截图。
**功能特性:**
- 显示截图图片
- 支持滚轮缩放 (50% - 300%)
- 支持拖拽平移（放大后）
- 工具栏提供缩放和重置按钮
**Props:**
- `src`: 图片路径或 Base64 编码
- `alt`: 图片 alt 文本（可选）
- `className`: 自定义类名（可选）
- `maxHeight`: 最大高度，默认 300px（可选）
### ChatInput
聊天输入框组件。
**功能特性:**
- 文本输入框
- 发送按钮
- Enter 键发送支持
- 加载状态显示
**Props:**
- `value`: 输入框值
- `onChange`: 值变化回调
- `onSend`: 发送消息回调
- `disabled`: 是否禁用（可选）
- `isLoading`: 是否正在加载（可选）
- `placeholder`: 占位符文本（可选）
### ChatMessages
消息列表组件。
**功能特性:**
- 显示用户消息和 AI 回复
- Markdown 渲染支持
- 自动滚动到最新消息
- 显示消息时间戳
**Props:**
- `messages`: 消息列表
- `className`: 自定义类名（可选）
### ShortcutSettings
快捷键设置组件，用于在设置页面中配置截图快捷键。
**功能特性:**
- 显示当前快捷键（用户友好格式）
- 快捷键录制模式
- 保存/取消按钮
- 错误提示
**Props:**
- `currentShortcut`: 当前快捷键
- `onShortcutChange`: 快捷键变更回调
- `onValidate`: 验证快捷键回调（可选）
- `disabled`: 是否禁用（可选）
## Hook 说明
### useScreenshotChat
核心 Hook，管理截图对话的状态和 AI 通信。
**返回值:**
- `messages`: 消息列表
- `isLoading`: 是否正在加载
- `error`: 错误信息
- `imagePath`: 当前截图路径
- `imageBase64`: 当前截图的 Base64 编码
- `sendMessage(message)`: 发送消息到 AI
- `setImagePath(path)`: 设置截图路径
- `clearMessages()`: 清空消息历史
- `clearError()`: 清除错误
- `retry()`: 重试上一条消息
## 依赖关系
- 使用项目统一的 CSS 变量（terminal 主题）
- 使用 `@tauri-apps/api/core` 进行 Tauri 通信
- 使用 `react-markdown` 和 `remark-gfm` 进行 Markdown 渲染
- 使用 `@/lib/api/agent` 进行 AI 通信
## 相关需求
- 需求 4.1: 悬浮窗口以无边框、置顶的方式打开
- 需求 4.2: 悬浮窗口应显示截图预览
- 需求 4.3: 悬浮窗口应提供文本输入框
- 需求 4.4: 支持 Enter 键发送
- 需求 4.5: 在可滚动区域显示 AI 回复
- 需求 4.6: 支持 ESC 关闭
- 需求 4.7: 支持窗口拖动
- 需求 5.1: 将图片编码为 base64
- 需求 5.2: 使用现有的 Agent API 进行 AI 通信
- 需求 5.3: 显示加载指示器
- 需求 5.4: 以 Markdown 格式渲染回复内容
- 需求 5.5: 显示错误信息并提供重试选项
- 需求 6.3: 显示当前快捷键和修改按钮
- 需求 6.4: 支持快捷键录制模式
src/components/screenshot-chat/ScreenshotChatWindow.tsx
+170
View File
@@ -0,0 +1,170 @@
/**
* @file ScreenshotChatWindow.tsx
* @description 截图对话悬浮窗主组件
* @module components/screenshot-chat/ScreenshotChatWindow
*/
import React, { useState, useEffect, useCallback } from "react";
import { invoke } from "@tauri-apps/api/core";
import { ScreenshotPreview } from "./ScreenshotPreview";
import { ChatInput } from "./ChatInput";
import { ChatMessages } from "./ChatMessages";
import { useScreenshotChat } from "./useScreenshotChat";
import type { ScreenshotChatWindowProps } from "./types";
import "./screenshot-chat.css";
/**
* 截图对话悬浮窗主组件
*
* 组合截图预览、消息列表和输入框，提供完整的对话界面
*
* 需求:
* - 4.1: 当截图完成时，悬浮窗口应以无边框、置顶的方式打开
* - 4.6: 当用户按下 ESC 或点击窗口外部时，悬浮窗口应关闭
* - 4.7: 悬浮窗口应支持用户拖动
*/
export const ScreenshotChatWindow: React.FC<ScreenshotChatWindowProps> = ({
imagePath,
onClose,
}) => {
const [inputValue, setInputValue] = useState("");
const {
messages,
isLoading,
error,
imageBase64,
sendMessage,
setImagePath,
clearError,
retry,
} = useScreenshotChat();
// 加载图片
useEffect(() => {
if (imagePath) {
setImagePath(imagePath);
}
}, [imagePath, setImagePath]);
// 处理关闭窗口
const handleClose = useCallback(async () => {
try {
await invoke("close_screenshot_chat_window");
} catch (err) {
console.error("关闭窗口失败:", err);
}
onClose?.();
}, [onClose]);
// 处理 ESC 键关闭
useEffect(() => {
const handleKeyDown = (e: KeyboardEvent) => {
if (e.key === "Escape") {
handleClose();
}
};
window.addEventListener("keydown", handleKeyDown);
return () => window.removeEventListener("keydown", handleKeyDown);
}, [handleClose]);
// 处理发送消息
const handleSend = useCallback(async () => {
if (!inputValue.trim()) return;
const message = inputValue;
setInputValue("");
await sendMessage(message);
}, [inputValue, sendMessage]);
// 构建图片 src
const imageSrc = imageBase64
? `data:image/png;base64,${imageBase64}`
: imagePath;
return (
<div className="screenshot-chat-page">
{/* 窗口头部 - 可拖动区域 */}
<div className="screenshot-chat-header">
<span className="screenshot-chat-title">截图对话</span>
<button
className="screenshot-chat-close-btn"
onClick={handleClose}
title="关闭 (ESC)"
>
<svg
className="w-4 h-4"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
>
<line x1="18" y1="6" x2="6" y2="18" />
<line x1="6" y1="6" x2="18" y2="18" />
</svg>
</button>
</div>
{/* 截图预览区域 */}
{imageSrc && (
<div className="screenshot-chat-preview">
<ScreenshotPreview src={imageSrc} maxHeight={200} />
</div>
)}
{/* 对话区域 */}
<div className="screenshot-chat-conversation">
{/* 错误提示 */}
{error && (
<div className="screenshot-chat-error">
<div className="screenshot-chat-error-content">
<p style={{ color: "#f43f5e", marginBottom: 8 }}>{error}</p>
<button className="screenshot-chat-retry-btn" onClick={retry}>
<svg
className="w-3 h-3"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
>
<polyline points="1 4 1 10 7 10" />
<path d="M3.51 15a9 9 0 1 0 2.13-9.36L1 10" />
</svg>
重试
</button>
<button
className="screenshot-chat-retry-btn"
onClick={clearError}
style={{ marginLeft: 8 }}
>
关闭
</button>
</div>
</div>
)}
{/* 消息列表 */}
<ChatMessages messages={messages} />
{/* 输入区域 */}
<ChatInput
value={inputValue}
onChange={setInputValue}
onSend={handleSend}
isLoading={isLoading}
disabled={!imageBase64}
placeholder={imageBase64 ? "输入问题..." : "正在加载图片..."}
/>
</div>
{/* 调试信息（开发模式） */}
{import.meta.env.DEV && (
<div className="screenshot-chat-debug">
路径: {imagePath} | Base64: {imageBase64 ? "已加载" : "未加载"} |
消息数: {messages.length}
</div>
)}
</div>
);
};
export default ScreenshotChatWindow;
src/components/screenshot-chat/ScreenshotPreview.tsx
+174
View File
@@ -0,0 +1,174 @@
/**
* @file ScreenshotPreview.tsx
* @description 截图预览组件，用于悬浮对话窗口中显示截图
* @module components/screenshot-chat/ScreenshotPreview
*/
import React, { useState, useCallback } from "react";
/** 截图预览属性 */
export interface ScreenshotPreviewProps {
/** 图片路径或 Base64 编码 */
src: string;
/** 图片 alt 文本 */
alt?: string;
/** 自定义类名 */
className?: string;
/** 最大高度 */
maxHeight?: number;
}
/**
* 截图预览组件
* 支持缩放和拖拽查看截图
*/
export const ScreenshotPreview: React.FC<ScreenshotPreviewProps> = ({
src,
alt = "截图预览",
className = "",
maxHeight = 300,
}) => {
const [scale, setScale] = useState(1);
const [position, setPosition] = useState({ x: 0, y: 0 });
const [isDragging, setIsDragging] = useState(false);
const [dragStart, setDragStart] = useState({ x: 0, y: 0 });
// 滚轮缩放
const handleWheel = useCallback((e: React.WheelEvent) => {
e.preventDefault();
const delta = e.deltaY > 0 ? -0.1 : 0.1;
setScale((prev) => Math.max(0.5, Math.min(3, prev + delta)));
}, []);
// 拖拽开始
const handleMouseDown = useCallback(
(e: React.MouseEvent) => {
if (scale > 1) {
setIsDragging(true);
setDragStart({ x: e.clientX - position.x, y: e.clientY - position.y });
}
},
[scale, position],
);
// 拖拽中
const handleMouseMove = useCallback(
(e: React.MouseEvent) => {
if (isDragging) {
setPosition({
x: e.clientX - dragStart.x,
y: e.clientY - dragStart.y,
});
}
},
[isDragging, dragStart],
);
// 拖拽结束
const handleMouseUp = useCallback(() => {
setIsDragging(false);
}, []);
// 重置缩放和位置
const handleReset = useCallback(() => {
setScale(1);
setPosition({ x: 0, y: 0 });
}, []);
// 放大
const handleZoomIn = useCallback(() => {
setScale((prev) => Math.min(3, prev + 0.25));
}, []);
// 缩小
const handleZoomOut = useCallback(() => {
setScale((prev) => Math.max(0.5, prev - 0.25));
}, []);
return (
<div className={`screenshot-preview ${className}`}>
{/* 工具栏 */}
<div className="screenshot-preview-toolbar">
<button
onClick={handleZoomOut}
title="缩小"
className="screenshot-preview-btn"
>
<svg
className="w-4 h-4"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
>
<circle cx="11" cy="11" r="8" />
<line x1="21" y1="21" x2="16.65" y2="16.65" />
<line x1="8" y1="11" x2="14" y2="11" />
</svg>
</button>
<span className="screenshot-preview-scale">
{Math.round(scale * 100)}%
</span>
<button
onClick={handleZoomIn}
title="放大"
className="screenshot-preview-btn"
>
<svg
className="w-4 h-4"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
>
<circle cx="11" cy="11" r="8" />
<line x1="21" y1="21" x2="16.65" y2="16.65" />
<line x1="11" y1="8" x2="11" y2="14" />
<line x1="8" y1="11" x2="14" y2="11" />
</svg>
</button>
<button
onClick={handleReset}
title="重置"
className="screenshot-preview-btn"
>
<svg
className="w-4 h-4"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
>
<polyline points="1 4 1 10 7 10" />
<path d="M3.51 15a9 9 0 1 0 2.13-9.36L1 10" />
</svg>
</button>
</div>
{/* 图片容器 */}
<div
className="screenshot-preview-container"
style={{ maxHeight: `${maxHeight}px` }}
onWheel={handleWheel}
onMouseDown={handleMouseDown}
onMouseMove={handleMouseMove}
onMouseUp={handleMouseUp}
onMouseLeave={handleMouseUp}
>
<img
src={src}
alt={alt}
className="screenshot-preview-image"
style={{
transform: `translate(${position.x}px, ${position.y}px) scale(${scale})`,
transition: isDragging ? "none" : "transform 0.1s ease",
cursor: scale > 1 ? (isDragging ? "grabbing" : "grab") : "default",
}}
draggable={false}
/>
</div>
</div>
);
};
export default ScreenshotPreview;
src/components/screenshot-chat/ShortcutSettings.tsx
+291
View File
@@ -0,0 +1,291 @@
/**
* @file ShortcutSettings.tsx
* @description 快捷键设置组件 - 显示当前快捷键、支持录制模式和保存/取消
* @module components/screenshot-chat/ShortcutSettings
*
* 需求: 6.3, 6.4 - 显示当前快捷键和修改按钮，支持快捷键录制模式
*/
import { useState, useEffect, useCallback, useRef } from "react";
import { Keyboard, Check, X, AlertCircle } from "lucide-react";
import { cn } from "@/lib/utils";
// ============================================================
// 类型定义
// ============================================================
export interface ShortcutSettingsProps {
/** 当前快捷键 */
currentShortcut: string;
/** 快捷键变更回调 */
onShortcutChange: (shortcut: string) => Promise<void>;
/** 验证快捷键回调 */
onValidate?: (shortcut: string) => Promise<boolean>;
/** 是否禁用 */
disabled?: boolean;
}
// ============================================================
// 辅助函数
// ============================================================
/**
* 将 KeyboardEvent 转换为 Tauri 快捷键格式
*/
function keyEventToShortcut(e: KeyboardEvent): string | null {
const modifiers: string[] = [];
// 收集修饰键
if (e.metaKey || e.ctrlKey) {
modifiers.push("CommandOrControl");
}
if (e.altKey) {
modifiers.push("Alt");
}
if (e.shiftKey) {
modifiers.push("Shift");
}
// 获取主键
let key = e.key;
// 忽略单独的修饰键
if (["Control", "Meta", "Alt", "Shift"].includes(key)) {
return null;
}
// 转换特殊键名
const keyMap: Record<string, string> = {
" ": "Space",
ArrowUp: "Up",
ArrowDown: "Down",
ArrowLeft: "Left",
ArrowRight: "Right",
Escape: "Escape",
Enter: "Enter",
Backspace: "Backspace",
Delete: "Delete",
Tab: "Tab",
};
if (keyMap[key]) {
key = keyMap[key];
} else if (key.length === 1) {
// 单字符键转大写
key = key.toUpperCase();
} else if (key.startsWith("F") && /^F\d+$/.test(key)) {
// 功能键保持原样
} else {
// 其他键首字母大写
key = key.charAt(0).toUpperCase() + key.slice(1);
}
// 必须有至少一个修饰键
if (modifiers.length === 0) {
return null;
}
return [...modifiers, key].join("+");
}
/**
* 格式化快捷键显示（将 Tauri 格式转换为用户友好格式）
*/
function formatShortcutDisplay(shortcut: string): string {
return shortcut
.replace(
"CommandOrControl",
navigator.platform.includes("Mac") ? "⌘" : "Ctrl",
)
.replace("Shift", navigator.platform.includes("Mac") ? "⇧" : "Shift")
.replace("Alt", navigator.platform.includes("Mac") ? "⌥" : "Alt")
.replace(/\+/g, " + ");
}
// ============================================================
// 组件
// ============================================================
export function ShortcutSettings({
currentShortcut,
onShortcutChange,
onValidate,
disabled = false,
}: ShortcutSettingsProps) {
// 状态
const [isRecording, setIsRecording] = useState(false);
const [recordedShortcut, setRecordedShortcut] = useState<string | null>(null);
const [error, setError] = useState<string | null>(null);
const [isSaving, setIsSaving] = useState(false);
// Refs
const inputRef = useRef<HTMLDivElement>(null);
// 开始录制
const startRecording = useCallback(() => {
if (disabled) return;
setIsRecording(true);
setRecordedShortcut(null);
setError(null);
}, [disabled]);
// 取消录制
const cancelRecording = useCallback(() => {
setIsRecording(false);
setRecordedShortcut(null);
setError(null);
}, []);
// 保存快捷键
const saveShortcut = useCallback(async () => {
if (!recordedShortcut) return;
setIsSaving(true);
setError(null);
try {
// 验证快捷键
if (onValidate) {
const isValid = await onValidate(recordedShortcut);
if (!isValid) {
setError("快捷键格式无效");
setIsSaving(false);
return;
}
}
// 保存快捷键
await onShortcutChange(recordedShortcut);
setIsRecording(false);
setRecordedShortcut(null);
} catch (err) {
setError(err instanceof Error ? err.message : "保存失败");
} finally {
setIsSaving(false);
}
}, [recordedShortcut, onShortcutChange, onValidate]);
// 键盘事件处理
useEffect(() => {
if (!isRecording) return;
const handleKeyDown = (e: KeyboardEvent) => {
e.preventDefault();
e.stopPropagation();
// ESC 取消录制
if (e.key === "Escape") {
cancelRecording();
return;
}
const shortcut = keyEventToShortcut(e);
if (shortcut) {
setRecordedShortcut(shortcut);
setError(null);
}
};
window.addEventListener("keydown", handleKeyDown, true);
return () => window.removeEventListener("keydown", handleKeyDown, true);
}, [isRecording, cancelRecording]);
// 自动聚焦录制区域
useEffect(() => {
if (isRecording && inputRef.current) {
inputRef.current.focus();
}
}, [isRecording]);
// 显示的快捷键
const displayShortcut = recordedShortcut || currentShortcut;
return (
<div className="space-y-2">
<div className="flex items-center gap-2">
<Keyboard className="h-4 w-4 text-muted-foreground" />
<span className="text-sm font-medium">快捷键</span>
</div>
<div className="flex items-center gap-2">
{/* 快捷键显示/录制区域 */}
<div
ref={inputRef}
tabIndex={isRecording ? 0 : -1}
className={cn(
"flex-1 px-3 py-2 rounded border text-sm font-mono transition-colors",
isRecording
? "border-primary bg-primary/5 ring-2 ring-primary/20"
: "bg-muted/50",
disabled && "opacity-50 cursor-not-allowed",
)}
>
{isRecording ? (
<span className="text-muted-foreground">
{recordedShortcut
? formatShortcutDisplay(recordedShortcut)
: "按下快捷键组合..."}
</span>
) : (
<span>{formatShortcutDisplay(displayShortcut)}</span>
)}
</div>
{/* 操作按钮 */}
{isRecording ? (
<>
<button
onClick={saveShortcut}
disabled={!recordedShortcut || isSaving}
className={cn(
"p-2 rounded transition-colors",
recordedShortcut && !isSaving
? "bg-green-500/10 text-green-600 hover:bg-green-500/20"
: "text-muted-foreground opacity-50 cursor-not-allowed",
)}
title="保存"
>
<Check className="h-4 w-4" />
</button>
<button
onClick={cancelRecording}
disabled={isSaving}
className="p-2 rounded text-muted-foreground hover:bg-muted transition-colors"
title="取消"
>
<X className="h-4 w-4" />
</button>
</>
) : (
<button
onClick={startRecording}
disabled={disabled}
className={cn(
"px-3 py-2 rounded border text-sm transition-colors",
disabled ? "opacity-50 cursor-not-allowed" : "hover:bg-muted",
)}
>
修改
</button>
)}
</div>
{/* 错误提示 */}
{error && (
<div className="flex items-center gap-1.5 text-xs text-destructive">
<AlertCircle className="h-3 w-3" />
<span>{error}</span>
</div>
)}
{/* 录制提示 */}
{isRecording && !error && (
<p className="text-xs text-muted-foreground">
按下想要设置的快捷键组合，按 ESC 取消
</p>
)}
</div>
);
}
export default ShortcutSettings;
src/components/screenshot-chat/index.ts
+33
View File
@@ -0,0 +1,33 @@
/**
* @file index.ts
* @description 截图对话模块导出入口
* @module components/screenshot-chat
*/
// 类型导出
export type {
ScreenshotChatConfig,
MessageImage,
ChatMessage,
ScreenshotChatState,
UseScreenshotChatReturn,
ScreenshotPreviewProps,
ChatInputProps,
ChatMessagesProps,
ScreenshotChatWindowProps,
} from "./types";
export type { ShortcutSettingsProps } from "./ShortcutSettings";
// 组件导出
export { ScreenshotPreview } from "./ScreenshotPreview";
export { ChatInput } from "./ChatInput";
export { ChatMessages } from "./ChatMessages";
export { ScreenshotChatWindow } from "./ScreenshotChatWindow";
export { ShortcutSettings } from "./ShortcutSettings";
// Hook 导出
export { useScreenshotChat, readImageAsBase64 } from "./useScreenshotChat";
// 默认导出主组件
export { ScreenshotChatWindow as default } from "./ScreenshotChatWindow";
src/components/screenshot-chat/screenshot-chat.css
+643
View File
@@ -0,0 +1,643 @@
/**
* @file screenshot-chat.css
* @description 截图对话组件样式 - 参考 Claude 的简洁设计
* @module components/screenshot-chat
*/
/* ============================================================================
* 悬浮输入框容器
* ============================================================================ */
.screenshot-floating-container {
position: fixed;
inset: 0;
display: flex;
flex-direction: column;
align-items: center;
justify-content: flex-end;
padding-bottom: 80px;
background: transparent;
}
/* 背景遮罩 - 半透明模糊效果 */
.screenshot-floating-backdrop {
position: absolute;
inset: 0;
background: rgba(0, 0, 0, 0.3);
backdrop-filter: blur(4px);
-webkit-backdrop-filter: blur(4px);
}
/* ============================================================================
* 悬浮输入框
* ============================================================================ */
.screenshot-floating-input-wrapper {
position: relative;
z-index: 10;
display: flex;
align-items: center;
gap: 8px;
width: 600px;
max-width: calc(100vw - 40px);
padding: 8px 12px;
background: white;
border-radius: 24px;
box-shadow:
0 4px 24px rgba(0, 0, 0, 0.15),
0 0 0 1px rgba(0, 0, 0, 0.05);
}
/* 图片标签按钮 */
.screenshot-image-tag {
display: flex;
align-items: center;
gap: 4px;
padding: 6px 10px;
background: #f3f4f6;
border: none;
border-radius: 16px;
color: #6b7280;
font-size: 12px;
font-weight: 500;
cursor: pointer;
transition: all 0.15s ease;
white-space: nowrap;
}
.screenshot-image-tag:hover {
background: #e5e7eb;
color: #374151;
}
.screenshot-image-tag svg {
color: #9ca3af;
}
/* 输入框 */
.screenshot-floating-input {
flex: 1;
padding: 8px 4px;
border: none;
background: transparent;
font-size: 15px;
color: #1f2937;
outline: none;
}
.screenshot-floating-input::placeholder {
color: #9ca3af;
}
.screenshot-floating-input:disabled {
opacity: 0.6;
}
/* 右侧按钮组 */
.screenshot-floating-actions {
display: flex;
align-items: center;
gap: 8px;
}
/* 下拉按钮 */
.screenshot-action-btn {
display: flex;
align-items: center;
gap: 4px;
padding: 6px 10px;
background: transparent;
border: none;
border-radius: 8px;
color: #6b7280;
font-size: 13px;
cursor: pointer;
transition: all 0.15s ease;
white-space: nowrap;
}
.screenshot-action-btn:hover {
background: #f3f4f6;
color: #374151;
}
/* 发送按钮 */
.screenshot-send-btn {
display: flex;
align-items: center;
justify-content: center;
width: 32px;
height: 32px;
background: #d1d5db;
border: none;
border-radius: 50%;
color: white;
cursor: not-allowed;
transition: all 0.15s ease;
}
.screenshot-send-btn.active {
background: #f97316;
cursor: pointer;
}
.screenshot-send-btn.active:hover {
background: #ea580c;
}
/* ============================================================================
* 图片预览弹窗
* ============================================================================ */
.screenshot-preview-modal {
position: fixed;
z-index: 100;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
max-width: 90vw;
max-height: 80vh;
background: white;
border-radius: 12px;
box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25);
overflow: hidden;
}
.screenshot-preview-close {
position: absolute;
top: 8px;
right: 8px;
z-index: 10;
display: flex;
align-items: center;
justify-content: center;
width: 32px;
height: 32px;
background: rgba(0, 0, 0, 0.5);
border: none;
border-radius: 50%;
color: white;
cursor: pointer;
transition: all 0.15s ease;
}
.screenshot-preview-close:hover {
background: rgba(0, 0, 0, 0.7);
}
.screenshot-preview-image {
display: block;
max-width: 90vw;
max-height: 80vh;
object-fit: contain;
}
/* ============================================================================
* 深色模式支持
* ============================================================================ */
@media (prefers-color-scheme: dark) {
.screenshot-floating-input-wrapper {
background: #1f2937;
box-shadow:
0 4px 24px rgba(0, 0, 0, 0.4),
0 0 0 1px rgba(255, 255, 255, 0.1);
}
.screenshot-image-tag {
background: #374151;
color: #d1d5db;
}
.screenshot-image-tag:hover {
background: #4b5563;
color: #f3f4f6;
}
.screenshot-image-tag svg {
color: #9ca3af;
}
.screenshot-floating-input {
color: #f3f4f6;
}
.screenshot-floating-input::placeholder {
color: #6b7280;
}
.screenshot-action-btn {
color: #9ca3af;
}
.screenshot-action-btn:hover {
background: #374151;
color: #f3f4f6;
}
.screenshot-send-btn {
background: #4b5563;
}
.screenshot-preview-modal {
background: #1f2937;
}
}
/* ============================================================================
* 旧版样式保留（兼容）
* ============================================================================ */
.screenshot-preview {
display: flex;
flex-direction: column;
border-radius: 8px;
overflow: hidden;
background-color: var(--terminal-bg, #1a1b26);
border: 1px solid var(--terminal-border, #3b4261);
}
.screenshot-preview-toolbar {
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
padding: 6px 8px;
background-color: var(--terminal-tab-bg, #24283b);
border-bottom: 1px solid var(--terminal-border, #3b4261);
}
.screenshot-preview-btn {
display: flex;
align-items: center;
justify-content: center;
width: 28px;
height: 28px;
border-radius: 4px;
color: var(--terminal-muted, #565f89);
background: transparent;
border: none;
cursor: pointer;
transition: all 0.15s ease;
}
.screenshot-preview-btn:hover {
background-color: var(--terminal-tab-hover-bg, #414868);
color: var(--terminal-fg, #c0caf5);
}
.screenshot-preview-scale {
min-width: 50px;
text-align: center;
font-size: 12px;
color: var(--terminal-muted, #565f89);
font-variant-numeric: tabular-nums;
}
.screenshot-preview-container {
display: flex;
align-items: center;
justify-content: center;
overflow: hidden;
background-color: var(--terminal-bg, #1a1b26);
}
/* ============================================================================
* 截图对话页面
* ============================================================================ */
.screenshot-chat-page {
display: flex;
flex-direction: column;
height: 100vh;
width: 100vw;
background-color: var(--terminal-bg, #1a1b26);
color: var(--terminal-fg, #c0caf5);
overflow: hidden;
}
/* 窗口头部 - 拖动区域 */
.screenshot-chat-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 8px 12px;
background-color: var(--terminal-tab-bg, #24283b);
border-bottom: 1px solid var(--terminal-border, #3b4261);
-webkit-app-region: drag;
user-select: none;
}
.screenshot-chat-title {
font-size: 13px;
font-weight: 500;
color: var(--terminal-fg, #c0caf5);
}
.screenshot-chat-close-btn {
display: flex;
align-items: center;
justify-content: center;
width: 24px;
height: 24px;
border-radius: 4px;
color: var(--terminal-muted, #565f89);
background: transparent;
border: none;
cursor: pointer;
transition: all 0.15s ease;
-webkit-app-region: no-drag;
}
.screenshot-chat-close-btn:hover {
background-color: #f43f5e;
color: white;
}
/* 截图预览区域 */
.screenshot-chat-preview {
padding: 12px;
border-bottom: 1px solid var(--terminal-border, #3b4261);
}
/* 对话区域 */
.screenshot-chat-conversation {
flex: 1;
display: flex;
flex-direction: column;
min-height: 0;
}
.screenshot-chat-messages {
flex: 1;
overflow-y: auto;
padding: 12px;
}
.screenshot-chat-placeholder {
text-align: center;
color: var(--terminal-muted, #565f89);
font-size: 13px;
padding: 24px;
}
/* 输入区域 */
.screenshot-chat-input-area {
display: flex;
gap: 8px;
padding: 12px;
border-top: 1px solid var(--terminal-border, #3b4261);
background-color: var(--terminal-tab-bg, #24283b);
}
.screenshot-chat-input {
flex: 1;
padding: 8px 12px;
border-radius: 6px;
border: 1px solid var(--terminal-border, #3b4261);
background-color: var(--terminal-bg, #1a1b26);
color: var(--terminal-fg, #c0caf5);
font-size: 13px;
outline: none;
transition: border-color 0.15s ease;
}
.screenshot-chat-input:focus {
border-color: var(--terminal-accent, #7aa2f7);
}
.screenshot-chat-input:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.screenshot-chat-send-btn {
padding: 8px 16px;
border-radius: 6px;
border: none;
background-color: var(--terminal-accent, #7aa2f7);
color: white;
font-size: 13px;
font-weight: 500;
cursor: pointer;
transition: all 0.15s ease;
}
.screenshot-chat-send-btn:hover:not(:disabled) {
background-color: var(--terminal-accent-hover, #5d8bea);
}
.screenshot-chat-send-btn:disabled {
opacity: 0.5;
cursor: not-allowed;
}
/* 错误状态 */
.screenshot-chat-error {
display: flex;
align-items: center;
justify-content: center;
}
.screenshot-chat-error-content {
display: flex;
flex-direction: column;
align-items: center;
text-align: center;
padding: 24px;
}
/* 加载状态 */
.screenshot-chat-loading {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 12px;
color: var(--terminal-muted, #565f89);
}
.screenshot-chat-loading-spinner {
width: 32px;
height: 32px;
border: 3px solid var(--terminal-border, #3b4261);
border-top-color: var(--terminal-accent, #7aa2f7);
border-radius: 50%;
animation: screenshot-chat-spin 0.8s linear infinite;
}
@keyframes screenshot-chat-spin {
to {
transform: rotate(360deg);
}
}
/* 调试信息 */
.screenshot-chat-debug {
padding: 4px 12px;
background-color: var(--terminal-tab-bg, #24283b);
border-top: 1px solid var(--terminal-border, #3b4261);
font-size: 10px;
color: var(--terminal-muted, #565f89);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
/* ============================================================================
* 消息列表组件
* ============================================================================ */
.screenshot-chat-message {
display: flex;
flex-direction: column;
gap: 4px;
margin-bottom: 16px;
padding: 8px 12px;
border-radius: 8px;
max-width: 85%;
}
.screenshot-chat-message-user {
align-self: flex-end;
background-color: var(--terminal-accent, #7aa2f7);
color: white;
}
.screenshot-chat-message-assistant {
align-self: flex-start;
background-color: var(--terminal-tab-bg, #24283b);
border: 1px solid var(--terminal-border, #3b4261);
}
.screenshot-chat-message-image {
margin-bottom: 8px;
}
.screenshot-chat-message-thumbnail {
max-width: 120px;
max-height: 80px;
border-radius: 4px;
object-fit: cover;
border: 1px solid rgba(255, 255, 255, 0.2);
}
.screenshot-chat-message-content {
font-size: 13px;
line-height: 1.5;
}
.screenshot-chat-message-content p {
margin: 0;
}
.screenshot-chat-message-time {
font-size: 10px;
opacity: 0.6;
align-self: flex-end;
}
/* 思考中状态 */
.screenshot-chat-thinking {
display: flex;
align-items: center;
gap: 8px;
color: var(--terminal-muted, #565f89);
font-style: italic;
}
.screenshot-chat-thinking .screenshot-chat-loading-spinner {
width: 14px;
height: 14px;
border-width: 2px;
}
/* Markdown 渲染样式 */
.screenshot-chat-markdown {
font-size: 13px;
line-height: 1.6;
}
.screenshot-chat-markdown p {
margin: 0 0 8px 0;
}
.screenshot-chat-markdown p:last-child {
margin-bottom: 0;
}
.screenshot-chat-markdown code {
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
font-size: 12px;
padding: 2px 4px;
border-radius: 3px;
background-color: rgba(0, 0, 0, 0.2);
}
.screenshot-chat-markdown pre {
margin: 8px 0;
padding: 8px;
border-radius: 4px;
background-color: rgba(0, 0, 0, 0.3);
overflow-x: auto;
}
.screenshot-chat-markdown pre code {
padding: 0;
background: transparent;
}
.screenshot-chat-markdown ul,
.screenshot-chat-markdown ol {
margin: 8px 0;
padding-left: 20px;
}
.screenshot-chat-markdown li {
margin-bottom: 4px;
}
.screenshot-chat-markdown strong {
font-weight: 600;
}
.screenshot-chat-markdown em {
font-style: italic;
}
.screenshot-chat-markdown a {
color: var(--terminal-accent, #7aa2f7);
text-decoration: none;
}
.screenshot-chat-markdown a:hover {
text-decoration: underline;
}
.screenshot-chat-markdown blockquote {
margin: 8px 0;
padding-left: 12px;
border-left: 3px solid var(--terminal-accent, #7aa2f7);
color: var(--terminal-muted, #565f89);
}
/* 重试按钮 */
.screenshot-chat-retry-btn {
display: inline-flex;
align-items: center;
gap: 4px;
margin-top: 8px;
padding: 4px 8px;
border-radius: 4px;
border: 1px solid var(--terminal-border, #3b4261);
background: transparent;
color: var(--terminal-muted, #565f89);
font-size: 12px;
cursor: pointer;
transition: all 0.15s ease;
}
.screenshot-chat-retry-btn:hover {
background-color: var(--terminal-tab-hover-bg, #414868);
color: var(--terminal-fg, #c0caf5);
}
src/components/screenshot-chat/types.ts
+152
View File
@@ -0,0 +1,152 @@
/**
* @file types.ts
* @description 截图对话模块类型定义
* @module components/screenshot-chat/types
*/
// ============================================================
// 配置类型
// ============================================================
/**
* 截图对话功能配置
* 需求: 1.1 - 实验室功能应提供 screenshot_chat.enabled 布尔开关
*/
export interface ScreenshotChatConfig {
/** 是否启用截图对话功能 */
enabled: boolean;
/** 触发截图的全局快捷键 */
shortcut: string;
}
// ============================================================
// 消息类型
// ============================================================
/**
* 消息图片
* 需求: 5.1 - 截图对话模块应将图片编码为 base64
*/
export interface MessageImage {
/** Base64 编码的图片数据 */
data: string;
/** 媒体类型，如 "image/png" */
mediaType: string;
}
/**
* 聊天消息
* 需求: 4.5 - 悬浮窗口应在可滚动区域显示 AI 回复
*/
export interface ChatMessage {
/** 消息唯一标识 */
id: string;
/** 消息角色：用户或助手 */
role: "user" | "assistant";
/** 消息文本内容 */
content: string;
/** 附带的图片（用户消息可能包含截图） */
image?: MessageImage;
/** 消息时间戳 */
timestamp: number;
/** 是否正在思考中（助手消息） */
isThinking?: boolean;
/** 思考中的提示文本 */
thinkingContent?: string;
}
// ============================================================
// Hook 状态类型
// ============================================================
/**
* 截图对话 Hook 状态
*/
export interface ScreenshotChatState {
/** 消息列表 */
messages: ChatMessage[];
/** 是否正在加载 */
isLoading: boolean;
/** 错误信息 */
error: string | null;
/** 当前截图路径 */
imagePath: string | null;
/** 当前截图的 Base64 编码 */
imageBase64: string | null;
}
/**
* 截图对话 Hook 返回值
*/
export interface UseScreenshotChatReturn extends ScreenshotChatState {
/** 发送消息到 AI */
sendMessage: (message: string) => Promise<void>;
/** 设置截图路径 */
setImagePath: (path: string) => void;
/** 清空消息历史 */
clearMessages: () => void;
/** 清除错误 */
clearError: () => void;
/** 重试上一条消息 */
retry: () => Promise<void>;
}
// ============================================================
// 组件 Props 类型
// ============================================================
/**
* 截图预览组件属性
* 需求: 4.2 - 悬浮窗口应显示截图预览
*/
export interface ScreenshotPreviewProps {
/** 图片路径或 Base64 编码 */
src: string;
/** 图片 alt 文本 */
alt?: string;
/** 自定义类名 */
className?: string;
/** 最大高度 */
maxHeight?: number;
}
/**
* 聊天输入框组件属性
* 需求: 4.3, 4.4 - 悬浮窗口应提供文本输入框，支持 Enter 发送
*/
export interface ChatInputProps {
/** 输入框值 */
value: string;
/** 值变化回调 */
onChange: (value: string) => void;
/** 发送消息回调 */
onSend: () => void;
/** 是否禁用 */
disabled?: boolean;
/** 是否正在加载 */
isLoading?: boolean;
/** 占位符文本 */
placeholder?: string;
}
/**
* 消息列表组件属性
* 需求: 4.5, 5.4 - 显示 AI 回复，支持 Markdown 渲染
*/
export interface ChatMessagesProps {
/** 消息列表 */
messages: ChatMessage[];
/** 自定义类名 */
className?: string;
}
/**
* 悬浮窗主组件属性
* 需求: 4.1, 4.6, 4.7 - 无边框置顶窗口，支持 ESC 关闭和拖动
*/
export interface ScreenshotChatWindowProps {
/** 截图路径 */
imagePath: string;
/** 关闭窗口回调 */
onClose?: () => void;
}
src/components/screenshot-chat/useScreenshotChat.ts
+313
View File
@@ -0,0 +1,313 @@
/**
* @file useScreenshotChat.ts
* @description 截图对话核心 Hook，管理消息、图片和 AI 通信
* @module components/screenshot-chat/useScreenshotChat
*/
import { useState, useCallback, useRef } from "react";
import { invoke } from "@tauri-apps/api/core";
import { listen, type UnlistenFn } from "@tauri-apps/api/event";
import { toast } from "sonner";
import type {
ChatMessage,
MessageImage,
UseScreenshotChatReturn,
} from "./types";
import { parseStreamEvent, type StreamEvent } from "@/lib/api/agent";
/**
* 读取图片文件并转换为 Base64
* 需求: 5.1 - 截图对话模块应将图片编码为 base64
*
* @param imagePath - 图片文件路径
* @returns Base64 编码的图片数据
*/
export async function readImageAsBase64(imagePath: string): Promise<string> {
try {
const base64 = await invoke<string>("read_image_as_base64", {
path: imagePath,
});
return base64;
} catch (error) {
console.error("读取图片失败:", error);
throw new Error(`读取图片失败: ${error}`);
}
}
/**
* 截图对话 Hook
*
* 提供截图对话功能的核心状态管理和 AI 通信能力
*
* 需求:
* - 5.1: 将图片编码为 base64
* - 5.2: 使用现有的 Agent API 进行 AI 通信
* - 5.3: 显示加载指示器
* - 5.5: 显示错误信息并提供重试选项
*/
export function useScreenshotChat(): UseScreenshotChatReturn {
// 状态
const [messages, setMessages] = useState<ChatMessage[]>([]);
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
const [imagePath, setImagePathState] = useState<string | null>(null);
const [imageBase64, setImageBase64] = useState<string | null>(null);
// 用于重试的最后一条消息
const lastMessageRef = useRef<string | null>(null);
// 会话 ID
const sessionIdRef = useRef<string | null>(null);
/**
* 设置截图路径并加载图片
*/
const setImagePath = useCallback(async (path: string) => {
setImagePathState(path);
setError(null);
try {
const base64 = await readImageAsBase64(path);
setImageBase64(base64);
} catch (err) {
const errorMsg = err instanceof Error ? err.message : "加载图片失败";
setError(errorMsg);
toast.error(errorMsg);
}
}, []);
/**
* 创建或获取会话
*/
const ensureSession = useCallback(async (): Promise<string | null> => {
if (sessionIdRef.current) {
return sessionIdRef.current;
}
try {
const response = await invoke<{ session_id: string }>(
"agent_create_session",
{
providerType: "claude",
model: "claude-sonnet-4-5",
},
);
sessionIdRef.current = response.session_id;
return response.session_id;
} catch (err) {
console.error("创建会话失败:", err);
return null;
}
}, []);
/**
* 发送消息到 AI
* 需求: 5.2 - 使用现有的 Agent API 进行 AI 通信
*/
const sendMessage = useCallback(
async (message: string) => {
if (!message.trim()) return;
if (!imageBase64) {
setError("请先加载截图");
return;
}
// 保存消息用于重试
lastMessageRef.current = message;
setError(null);
setIsLoading(true);
// 创建用户消息
const userMsg: ChatMessage = {
id: crypto.randomUUID(),
role: "user",
content: message,
image:
messages.length === 0
? { data: imageBase64, mediaType: "image/png" }
: undefined,
timestamp: Date.now(),
};
// 创建助手消息占位符
const assistantMsgId = crypto.randomUUID();
const assistantMsg: ChatMessage = {
id: assistantMsgId,
role: "assistant",
content: "",
timestamp: Date.now(),
isThinking: true,
thinkingContent: "思考中...",
};
setMessages((prev) => [...prev, userMsg, assistantMsg]);
let accumulatedContent = "";
let unlisten: UnlistenFn | null = null;
try {
// 确保有会话
const sessionId = await ensureSession();
if (!sessionId) {
throw new Error("无法创建会话");
}
// 创建唯一事件名称
const eventName = `screenshot_chat_stream_${assistantMsgId}`;
// 设置事件监听器
unlisten = await listen<StreamEvent>(eventName, (event) => {
const data = parseStreamEvent(event.payload);
if (!data) return;
switch (data.type) {
case "text_delta":
accumulatedContent += data.text;
setMessages((prev) =>
prev.map((msg) =>
msg.id === assistantMsgId
? {
...msg,
content: accumulatedContent,
isThinking: false,
thinkingContent: undefined,
}
: msg,
),
);
break;
case "done":
case "final_done":
setMessages((prev) =>
prev.map((msg) =>
msg.id === assistantMsgId
? {
...msg,
isThinking: false,
content: accumulatedContent || "(无响应)",
}
: msg,
),
);
setIsLoading(false);
if (unlisten) {
unlisten();
unlisten = null;
}
break;
case "error":
setError(data.message);
setMessages((prev) =>
prev.map((msg) =>
msg.id === assistantMsgId
? {
...msg,
isThinking: false,
content: `错误: ${data.message}`,
}
: msg,
),
);
setIsLoading(false);
if (unlisten) {
unlisten();
unlisten = null;
}
break;
}
});
// 准备图片数据（只在第一条消息时发送图片）
const images: MessageImage[] =
messages.length === 0
? [{ data: imageBase64, mediaType: "image/png" }]
: [];
// 发送流式请求
await invoke("native_agent_chat_stream", {
message,
eventName,
sessionId,
model: "claude-sonnet-4-5",
images:
images.length > 0
? images.map((img) => ({
data: img.data,
media_type: img.mediaType,
}))
: undefined,
provider: "claude",
});
} catch (err) {
console.error("发送消息失败:", err);
const errorMsg = err instanceof Error ? err.message : "发送失败";
setError(errorMsg);
toast.error(errorMsg);
// 移除失败的助手消息
setMessages((prev) => prev.filter((msg) => msg.id !== assistantMsgId));
setIsLoading(false);
if (unlisten) {
unlisten();
}
}
},
[imageBase64, messages.length, ensureSession],
);
/**
* 清空消息历史
*/
const clearMessages = useCallback(() => {
setMessages([]);
sessionIdRef.current = null;
lastMessageRef.current = null;
}, []);
/**
* 清除错误
*/
const clearError = useCallback(() => {
setError(null);
}, []);
/**
* 重试上一条消息
* 需求: 5.5 - 显示错误信息并提供重试选项
*/
const retry = useCallback(async () => {
if (lastMessageRef.current) {
// 移除最后一条失败的助手消息
setMessages((prev) => {
const lastMsg = prev[prev.length - 1];
if (
lastMsg?.role === "assistant" &&
lastMsg.content.startsWith("错误:")
) {
return prev.slice(0, -1);
}
return prev;
});
await sendMessage(lastMessageRef.current);
}
}, [sendMessage]);
return {
messages,
isLoading,
error,
imagePath,
imageBase64,
sendMessage,
setImagePath,
clearMessages,
clearError,
retry,
};
}
export default useScreenshotChat;
src/components/settings/ExperimentalSettings.tsx
+268
View File
@@ -0,0 +1,268 @@
/**
* @file ExperimentalSettings.tsx
* @description 实验室设置页面 - 管理实验性功能的开关和配置
* @module components/settings/ExperimentalSettings
*
* 需求: 6.1, 6.2, 6.3, 6.5 - 实验室标签页，截图对话功能开关，快捷键设置，权限警告
*/
import { useState, useEffect, useCallback } from "react";
import { FlaskConical, Camera, AlertTriangle, RefreshCw } from "lucide-react";
import { cn } from "@/lib/utils";
import {
getExperimentalConfig,
saveExperimentalConfig,
validateShortcut,
updateScreenshotShortcut,
ExperimentalFeatures,
} from "@/hooks/useTauri";
import { ShortcutSettings } from "@/components/screenshot-chat/ShortcutSettings";
// ============================================================
// 组件
// ============================================================
export function ExperimentalSettings() {
// 状态
const [config, setConfig] = useState<ExperimentalFeatures | null>(null);
const [loading, setLoading] = useState(true);
const [saving, setSaving] = useState(false);
const [error, setError] = useState<string | null>(null);
const [message, setMessage] = useState<{
type: "success" | "error";
text: string;
} | null>(null);
// 检测是否为 macOS（使用 userAgentData 或 userAgent 替代已弃用的 platform）
const isMacOS = navigator.userAgent.includes("Mac");
// 加载配置
const loadConfig = useCallback(async () => {
setLoading(true);
setError(null);
try {
const experimentalConfig = await getExperimentalConfig();
setConfig(experimentalConfig);
} catch (err) {
console.error("加载实验室配置失败:", err);
setError(err instanceof Error ? err.message : "加载配置失败");
// 设置默认配置
setConfig({
screenshot_chat: {
enabled: false,
shortcut: "CommandOrControl+Alt+Q",
},
});
} finally {
setLoading(false);
}
}, []);
// 初始加载
useEffect(() => {
loadConfig();
}, [loadConfig]);
// 切换截图对话功能开关
const handleToggleScreenshotChat = useCallback(async () => {
if (!config) return;
const newEnabled = !config.screenshot_chat.enabled;
const newConfig: ExperimentalFeatures = {
...config,
screenshot_chat: {
...config.screenshot_chat,
enabled: newEnabled,
},
};
setSaving(true);
setMessage(null);
try {
await saveExperimentalConfig(newConfig);
setConfig(newConfig);
setMessage({
type: "success",
text: newEnabled ? "截图对话功能已启用" : "截图对话功能已禁用",
});
setTimeout(() => setMessage(null), 2000);
} catch (err) {
console.error("保存配置失败:", err);
setMessage({
type: "error",
text: err instanceof Error ? err.message : "保存失败",
});
} finally {
setSaving(false);
}
}, [config]);
// 更新快捷键
const handleShortcutChange = useCallback(
async (newShortcut: string) => {
if (!config) return;
await updateScreenshotShortcut(newShortcut);
setConfig({
...config,
screenshot_chat: {
...config.screenshot_chat,
shortcut: newShortcut,
},
});
setMessage({ type: "success", text: "快捷键已更新" });
setTimeout(() => setMessage(null), 2000);
},
[config],
);
// 验证快捷键
const handleValidateShortcut = useCallback(async (shortcut: string) => {
try {
return await validateShortcut(shortcut);
} catch {
return false;
}
}, []);
// 加载中状态
if (loading) {
return (
<div className="flex items-center justify-center py-8">
<RefreshCw className="h-5 w-5 animate-spin text-muted-foreground" />
</div>
);
}
// 错误状态
if (error && !config) {
return (
<div className="rounded-lg border border-destructive/50 bg-destructive/10 p-4">
<div className="flex items-center gap-2 text-destructive">
<AlertTriangle className="h-4 w-4" />
<span className="text-sm font-medium">加载配置失败</span>
</div>
<p className="mt-1 text-sm text-muted-foreground">{error}</p>
<button
onClick={loadConfig}
className="mt-2 text-sm text-primary hover:underline"
>
重试
</button>
</div>
);
}
return (
<div className="space-y-4 max-w-2xl">
{/* 页面标题 */}
<div className="flex items-center gap-2 mb-4">
<FlaskConical className="h-5 w-5 text-primary" />
<div>
<h3 className="text-sm font-medium">实验室功能</h3>
<p className="text-xs text-muted-foreground">
这些功能仍在开发中，可能不稳定
</p>
</div>
</div>
{/* 消息提示 */}
{message && (
<div
className={cn(
"rounded-lg px-3 py-2 text-sm",
message.type === "success"
? "bg-green-100 text-green-700 dark:bg-green-900/30 dark:text-green-400"
: "bg-destructive/10 text-destructive",
)}
>
{message.text}
</div>
)}
{/* 截图对话功能 */}
<div className="rounded-lg border p-4 space-y-4">
<div className="flex items-start justify-between">
<div className="flex items-start gap-3">
<Camera className="h-5 w-5 text-muted-foreground mt-0.5" />
<div>
<h4 className="text-sm font-medium">截图对话</h4>
<p className="text-xs text-muted-foreground mt-0.5">
使用全局快捷键截取屏幕区域，并与 AI 进行对话
</p>
</div>
</div>
<label className="relative inline-flex items-center cursor-pointer">
<input
type="checkbox"
checked={config?.screenshot_chat.enabled ?? false}
onChange={handleToggleScreenshotChat}
disabled={saving}
className="sr-only peer"
/>
<div
className={cn(
"w-9 h-5 rounded-full transition-colors",
"bg-muted peer-checked:bg-primary",
"after:content-[''] after:absolute after:top-0.5 after:left-0.5",
"after:bg-white after:rounded-full after:h-4 after:w-4",
"after:transition-transform peer-checked:after:translate-x-4",
saving && "opacity-50 cursor-not-allowed",
)}
/>
</label>
</div>
{/* 快捷键设置 - 仅在功能启用时显示 */}
{config?.screenshot_chat.enabled && (
<div className="pt-3 border-t">
<ShortcutSettings
currentShortcut={config.screenshot_chat.shortcut}
onShortcutChange={handleShortcutChange}
onValidate={handleValidateShortcut}
disabled={saving}
/>
</div>
)}
{/* macOS 权限警告 */}
{isMacOS && config?.screenshot_chat.enabled && (
<div className="flex items-start gap-2 p-3 rounded-lg bg-amber-50 dark:bg-amber-900/20 border border-amber-200 dark:border-amber-800">
<AlertTriangle className="h-4 w-4 text-amber-600 dark:text-amber-400 mt-0.5 flex-shrink-0" />
<div className="text-xs flex-1">
<p className="font-medium text-amber-800 dark:text-amber-300">
需要屏幕录制权限
</p>
<p className="text-amber-700 dark:text-amber-400 mt-0.5">
截图功能需要屏幕录制权限才能正常工作。如果截图只显示桌面背景而不是窗口内容，请授权此权限。
</p>
<button
onClick={async () => {
try {
const { open } = await import("@tauri-apps/plugin-shell");
await open(
"x-apple.systempreferences:com.apple.preference.security?Privacy_ScreenCapture",
);
} catch (e) {
console.error("打开系统设置失败:", e);
}
}}
className="mt-2 inline-flex items-center gap-1 px-2 py-1 rounded bg-amber-200 dark:bg-amber-800 text-amber-800 dark:text-amber-200 hover:bg-amber-300 dark:hover:bg-amber-700 transition-colors"
>
打开系统设置
</button>
</div>
</div>
)}
</div>
{/* 更多实验功能占位 */}
<div className="rounded-lg border border-dashed p-4 text-center">
<p className="text-sm text-muted-foreground">更多实验功能即将推出...</p>
</div>
</div>
);
}
export default ExperimentalSettings;
src/components/settings/SettingsPage.tsx
+4
View File
@@ -9,6 +9,7 @@ import { RemoteManagementSettings } from "./RemoteManagementSettings";
import { ExtensionsSettings } from "./ExtensionsSettings";
import { DeveloperSettings } from "./DeveloperSettings";
import { ConnectionsSettings } from "./ConnectionsSettings";
import { ExperimentalSettings } from "./ExperimentalSettings";
type SettingsTab =
| "general"
@@ -16,6 +17,7 @@ type SettingsTab =
| "security"
| "advanced"
| "extensions"
| "experimental"
| "developer"
| "about";
@@ -25,6 +27,7 @@ const tabs: { id: SettingsTab; label: string; experimental?: boolean }[] = [
{ id: "security", label: "安全" },
{ id: "advanced", label: "高级" },
{ id: "extensions", label: "扩展", experimental: true },
{ id: "experimental", label: "实验室", experimental: true },
{ id: "developer", label: "开发者" },
{ id: "about", label: "关于" },
];
@@ -80,6 +83,7 @@ export function SettingsPage() {
</div>
)}
{activeTab === "extensions" && <ExtensionsSettings />}
{activeTab === "experimental" && <ExperimentalSettings />}
{activeTab === "developer" && <DeveloperSettings />}
{activeTab === "about" && <AboutSection />}
</div>
src/components/settings/index.ts
+1
View File
@@ -7,3 +7,4 @@ export { TlsSettings } from "./TlsSettings";
export { QuotaSettings } from "./QuotaSettings";
export { RemoteManagementSettings } from "./RemoteManagementSettings";
export { DeveloperSettings } from "./DeveloperSettings";
export { ExperimentalSettings } from "./ExperimentalSettings";
src/components/terminal/TerminalWorkspace.tsx
+56 -4
View File
@@ -13,6 +13,7 @@
* - 支持 Terminal/Files/Web/Sysinfo 面板类型
* - Terminal 类型支持多实例
* - 右侧小部件栏
* - AI 面板可控制活动终端
*/
import { useState, useCallback, useRef } from "react";
@@ -50,6 +51,8 @@ export interface SidePanel {
cwd?: string;
/** 连接配置（仅 terminal 类型使用） */
connection?: ConnectionListEntry;
/** 终端会话 ID（仅 terminal 类型使用，由 TerminalPanel 回调设置） */
sessionId?: string;
}
// ============================================================================
@@ -221,12 +224,34 @@ export function TerminalWorkspace({ onNavigate }: TerminalWorkspaceProps) {
// AI 面板状态
const [showAIPanel, setShowAIPanel] = useState(false);
// 活动终端面板 ID（用于 AI 控制）
const [activeTerminalPanelId, setActiveTerminalPanelId] =
useState<string>("main-terminal");
// 终端输出引用（用于 AI 上下文）
const terminalOutputRef = useRef<string | null>(null);
// 连接编辑器模态窗口状态
const [isConnectionsEditorOpen, setIsConnectionsEditorOpen] = useState(false);
// 获取活动终端的会话 ID
const getActiveTerminalSessionId = useCallback((): string | null => {
const activePanel = panels.find(
(p) => p.id === activeTerminalPanelId && p.type === "terminal",
);
return activePanel?.sessionId || null;
}, [panels, activeTerminalPanelId]);
// 更新终端面板的会话 ID
const updatePanelSessionId = useCallback(
(panelId: string, sessionId: string) => {
setPanels((prev) =>
prev.map((p) => (p.id === panelId ? { ...p, sessionId } : p)),
);
},
[],
);
// 添加面板 - 所有类型都允许多开
const addPanel = useCallback(
(
@@ -321,7 +346,15 @@ export function TerminalWorkspace({ onNavigate }: TerminalWorkspaceProps) {
const renderPanelContent = (panel: SidePanel) => {
switch (panel.type) {
case "terminal":
return <TerminalPanel panelId={panel.id} cwd={panel.cwd} />;
return (
<TerminalPanel
panelId={panel.id}
cwd={panel.cwd}
onSessionCreated={(sessionId) =>
updatePanelSessionId(panel.id, sessionId)
}
/>
);
case "files":
return <FileBrowserView onOpenTerminal={handleOpenTerminalFromFiles} />;
case "web":
@@ -329,7 +362,12 @@ export function TerminalWorkspace({ onNavigate }: TerminalWorkspaceProps) {
case "sysinfo":
return <SysinfoView />;
case "ai":
return <TerminalAIPanel getTerminalOutput={getTerminalOutput} />;
return (
<TerminalAIPanel
getTerminalOutput={getTerminalOutput}
terminalSessionId={getActiveTerminalSessionId()}
/>
);
default:
return null;
}
@@ -383,14 +421,28 @@ export function TerminalWorkspace({ onNavigate }: TerminalWorkspaceProps) {
<div
style={{ width: 320, minWidth: 280, maxWidth: 400, flexShrink: 0 }}
>
<TerminalAIPanel getTerminalOutput={getTerminalOutput} />
<TerminalAIPanel
getTerminalOutput={getTerminalOutput}
terminalSessionId={getActiveTerminalSessionId()}
/>
</div>
)}
<WorkspaceContainer>
{/* 所有面板统一渲染，都可以关闭和多开 */}
{panels.map((panel) => (
<PanelBlock key={panel.id}>
<PanelBlock
key={panel.id}
$focused={
panel.type === "terminal" && panel.id === activeTerminalPanelId
}
onClick={() => {
// 点击终端面板时设置为活动终端
if (panel.type === "terminal") {
setActiveTerminalPanelId(panel.id);
}
}}
>
<PanelHeader>
{panel.type === "terminal" ? (
<ConnectionSelector
src/components/terminal/ai/CommandApproval.tsx
+186
View File
@@ -0,0 +1,186 @@
/**
* @file CommandApproval.tsx
* @description 命令审批组件 - AI 执行命令前的用户确认
* @module components/terminal/ai/CommandApproval
*
* 当 AI 需要在终端执行命令时，显示审批对话框让用户确认。
* 参考 Waveterm 的工具调用审批流程。
*/
import React from "react";
import styled from "styled-components";
import type { PendingCommand } from "./TerminalController";
// ============================================================================
// 样式组件
// ============================================================================
const ApprovalContainer = styled.div`
background: #1e293b;
border: 1px solid #334155;
border-radius: 8px;
padding: 12px;
margin: 8px 0;
`;
const ApprovalHeader = styled.div`
display: flex;
align-items: center;
gap: 8px;
margin-bottom: 8px;
color: #f59e0b;
font-size: 13px;
font-weight: 500;
`;
const WarningIcon = () => (
<svg
width="16"
height="16"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
>
<path d="M10.29 3.86L1.82 18a2 2 0 0 0 1.71 3h16.94a2 2 0 0 0 1.71-3L13.71 3.86a2 2 0 0 0-3.42 0z" />
<line x1="12" y1="9" x2="12" y2="13" />
<line x1="12" y1="17" x2="12.01" y2="17" />
</svg>
);
const CommandBox = styled.div`
background: #0f172a;
border: 1px solid #1e293b;
border-radius: 6px;
padding: 10px 12px;
font-family: "Hack", "Menlo", monospace;
font-size: 13px;
color: #e2e8f0;
margin-bottom: 12px;
overflow-x: auto;
white-space: pre-wrap;
word-break: break-all;
`;
const ButtonGroup = styled.div`
display: flex;
gap: 8px;
justify-content: flex-end;
`;
const Button = styled.button<{ $variant?: "approve" | "reject" }>`
padding: 6px 16px;
border-radius: 6px;
font-size: 13px;
font-weight: 500;
cursor: pointer;
transition: all 0.15s ease;
border: none;
${({ $variant }) =>
$variant === "approve"
? `
background: #22c55e;
color: white;
&:hover {
background: #16a34a;
}
`
: `
background: #334155;
color: #94a3b8;
&:hover {
background: #475569;
color: #e2e8f0;
}
`}
`;
const HelpText = styled.p`
font-size: 12px;
color: #64748b;
margin: 0 0 12px 0;
`;
// ============================================================================
// 组件
// ============================================================================
interface CommandApprovalProps {
/** 待审批的命令 */
command: PendingCommand;
/** 批准回调 */
onApprove: (commandId: string) => void;
/** 拒绝回调 */
onReject: (commandId: string) => void;
}
/**
* 命令审批组件
*/
export function CommandApproval({
command,
onApprove,
onReject,
}: CommandApprovalProps) {
return (
<ApprovalContainer>
<ApprovalHeader>
<WarningIcon />
AI 请求执行命令
</ApprovalHeader>
<CommandBox>{command.command}</CommandBox>
<HelpText>此命令将在当前终端中执行。请确认是否允许。</HelpText>
<ButtonGroup>
<Button onClick={() => onReject(command.id)}>拒绝</Button>
<Button $variant="approve" onClick={() => onApprove(command.id)}>
执行
</Button>
</ButtonGroup>
</ApprovalContainer>
);
}
// ============================================================================
// 命令列表组件
// ============================================================================
interface CommandApprovalListProps {
/** 待审批的命令列表 */
commands: PendingCommand[];
/** 批准回调 */
onApprove: (commandId: string) => void;
/** 拒绝回调 */
onReject: (commandId: string) => void;
}
/**
* 命令审批列表组件
*/
export function CommandApprovalList({
commands,
onApprove,
onReject,
}: CommandApprovalListProps) {
if (commands.length === 0) {
return null;
}
return (
<>
{commands.map((cmd) => (
<CommandApproval
key={cmd.id}
command={cmd}
onApprove={onApprove}
onReject={onReject}
/>
))}
</>
);
}
export default CommandApproval;
src/components/terminal/ai/README.md
+21 -2
View File
@@ -11,6 +11,7 @@ Terminal AI 是终端内置的 AI 助手功能，参考 Waveterm 的 AI 面板
- 支持终端上下文（Widget Context）
- 流式响应显示
- 工具调用支持
- **AI 控制终端**：AI 可以向活动终端发送命令（需用户审批）
## 文件索引
@@ -18,12 +19,14 @@ Terminal AI 是终端内置的 AI 助手功能，参考 Waveterm 的 AI 面板
|------|------|
| `index.ts` | 模块导出 |
| `types.ts` | 类型定义 |
| `useTerminalAI.ts` | Terminal AI Hook |
| `useTerminalAI.ts` | Terminal AI Hook（含终端控制） |
| `TerminalAIPanel.tsx` | AI 面板主组件 |
| `TerminalAIInput.tsx` | 输入框组件 |
| `TerminalAIMessages.tsx` | 消息列表组件 |
| `TerminalAIModeSelector.tsx` | 模式/模型选择器 |
| `TerminalAIWelcome.tsx` | 欢迎页面组件 |
| `CommandApproval.tsx` | 命令审批组件 |
| `TerminalController.ts` | 终端控制器（管理 AI 与终端通信） |
## 使用方式
@@ -36,8 +39,14 @@ function MyComponent() {
return "$ ls -la\ntotal 0\n...";
};
// 终端会话 ID（用于 AI 控制终端）
const terminalSessionId = "session-123";
return (
<TerminalAIPanel getTerminalOutput={getTerminalOutput} />
<TerminalAIPanel
getTerminalOutput={getTerminalOutput}
terminalSessionId={terminalSessionId}
/>
);
}
```
@@ -57,6 +66,15 @@ function MyComponent() {
- OAuth 凭证（Kiro、Gemini、Antigravity 等）
- API Key 凭证（OpenAI、Claude 等）
### AI 控制终端（新功能）
AI 可以向活动终端发送命令，流程如下：
1. AI 生成命令建议
2. 用户在审批对话框中确认
3. 命令发送到终端执行
参考 Waveterm 的 `sendDataToController()` 机制实现。
### 快捷操作
欢迎页面提供快捷操作按钮：
@@ -68,6 +86,7 @@ function MyComponent() {
## 依赖
- `@/lib/api/agent` - Agent API
- `@/lib/terminal-api` - 终端 API（用于发送命令）
- `@/hooks/useProviderPool` - Provider 凭证
- `@/hooks/useApiKeyProvider` - API Key 凭证
- `@/hooks/useModelRegistry` - 模型注册表
src/components/terminal/ai/TerminalAIMessages.tsx
+74 -34
View File
@@ -9,6 +9,7 @@
import React, { useRef, useEffect, useState, useCallback, memo } from "react";
import { cn } from "@/lib/utils";
import type { AIMessage } from "./types";
import type { ToolCallState } from "@/lib/api/agent";
// ============================================================================
// 子组件
@@ -100,43 +101,82 @@ MessageContent.displayName = "MessageContent";
/**
* 工具调用显示
*/
const ToolCallDisplay = memo(
({
toolCall,
}: {
toolCall: {
id: string;
name: string;
status: string;
result?: { success: boolean; output?: string; error?: string };
};
}) => {
const statusIcon =
toolCall.status === "completed"
? "✓"
: toolCall.status === "failed"
? "✗"
: "•";
const statusColor =
toolCall.status === "completed"
? "text-green-500"
: toolCall.status === "failed"
? "text-red-500"
: "text-zinc-400";
const ToolCallDisplay = memo(({ toolCall }: { toolCall: ToolCallState }) => {
const isCompleted = toolCall.status === "completed";
const isFailed = toolCall.status === "failed";
const isRunning = toolCall.status === "running";
return (
<div className="flex flex-col gap-1 p-2 rounded bg-zinc-800/60 border border-zinc-700 text-sm">
<div className="flex items-center gap-2">
<span className={cn("font-bold", statusColor)}>{statusIcon}</span>
<span className="font-medium">{toolCall.name}</span>
</div>
{toolCall.result?.error && (
<div className="text-red-300 pl-5">{toolCall.result.error}</div>
// 状态图标
const statusIcon = isCompleted ? "✓" : isFailed ? "✗" : "•";
// 背景色：成功用绿色调，失败用红色调，运行中用灰色
const bgColor = isCompleted
? "bg-green-900/30 border-green-700/50"
: isFailed
? "bg-red-900/30 border-red-700/50"
: "bg-zinc-800/60 border-zinc-700";
// 图标颜色
const iconColor = isCompleted
? "text-green-400"
: isFailed
? "text-red-400"
: "text-zinc-400";
// 工具名称颜色
const nameColor = isCompleted
? "text-green-300"
: isFailed
? "text-red-300"
: "text-zinc-300";
return (
<div
className={cn(
"flex flex-col gap-1.5 p-2.5 rounded-md border text-sm",
bgColor,
)}
>
{/* 工具名称和状态 */}
<div className="flex items-center gap-2">
<span className={cn("font-bold text-base", iconColor)}>
{statusIcon}
</span>
<span className={cn("font-medium", nameColor)}>{toolCall.name}</span>
{isRunning && (
<span className="text-xs text-zinc-500 animate-pulse">执行中...</span>
)}
</div>
);
},
);
{/* 显示结果 */}
{toolCall.result && (
<div className="pl-6 space-y-1">
{/* 成功输出 */}
{toolCall.result.success && toolCall.result.output && (
<div className="text-zinc-300 text-xs font-mono whitespace-pre-wrap break-all max-h-32 overflow-y-auto bg-zinc-900/50 rounded p-2">
{toolCall.result.output.length > 500
? toolCall.result.output.slice(0, 500) + "..."
: toolCall.result.output}
</div>
)}
{/* 错误信息 */}
{toolCall.result.error && (
<div className="text-red-300 text-xs">
<span className="font-medium">错误: </span>
{toolCall.result.error}
</div>
)}
{/* 退出码（如果有） */}
{!toolCall.result.success && !toolCall.result.error && (
<div className="text-red-300 text-xs">命令执行失败</div>
)}
</div>
)}
</div>
);
});
ToolCallDisplay.displayName = "ToolCallDisplay";
src/components/terminal/ai/TerminalAIModeSelector.tsx
+24 -25
View File
@@ -145,11 +145,13 @@ export const TerminalAIModeSelector: React.FC<TerminalAIModeSelectorProps> = ({
}
if (!providerMap.has(key)) {
// 使用 provider.type 映射到 registryId，而不是 provider.id
const registryId = getRegistryIdFromType(provider.type);
providerMap.set(key, {
key,
label,
registryId: provider.id,
fallbackRegistryId: getRegistryIdFromType(provider.type),
registryId,
fallbackRegistryId: registryId,
type: provider.type,
});
}
@@ -184,37 +186,34 @@ export const TerminalAIModeSelector: React.FC<TerminalAIModeSelectorProps> = ({
}
// 从 model_registry 获取
let models = registryModels
.filter((m) => m.provider_id === selectedProvider.registryId)
.map((m) => m.id);
let models = registryModels.filter(
(m) => m.provider_id === selectedProvider.registryId,
);
if (models.length === 0 && selectedProvider.fallbackRegistryId) {
models = registryModels
.filter((m) => m.provider_id === selectedProvider.fallbackRegistryId)
.map((m) => m.id);
models = registryModels.filter(
(m) => m.provider_id === selectedProvider.fallbackRegistryId,
);
}
// 排序
return models.sort((a, b) => {
const aIsLatest = a.includes("-latest");
const bIsLatest = b.includes("-latest");
// 排序：使用 release_date 和 is_latest 字段
const sortedModels = [...models].sort((a, b) => {
// 1. is_latest 优先
if (a.is_latest && !b.is_latest) return -1;
if (!a.is_latest && b.is_latest) return 1;
if (aIsLatest && !bIsLatest) return -1;
if (!aIsLatest && bIsLatest) return 1;
const dateRegex = /-(\d{8})$/;
const aMatch = a.match(dateRegex);
const bMatch = b.match(dateRegex);
if (aMatch && bMatch) {
return bMatch[1].localeCompare(aMatch[1]);
// 2. 按 release_date 降序（最新的在前）
if (a.release_date && b.release_date) {
return b.release_date.localeCompare(a.release_date);
}
if (a.release_date && !b.release_date) return -1;
if (!a.release_date && b.release_date) return 1;
if (aMatch && !bMatch) return -1;
if (!aMatch && bMatch) return 1;
return b.localeCompare(a);
// 3. 按 display_name 字母序
return a.display_name.localeCompare(b.display_name);
});
return sortedModels.map((m) => m.id);
}, [selectedProvider, registryModels, aliasConfig]);
// 自动选择第一个模型
src/components/terminal/ai/TerminalAIPanel.tsx
+64 -6
View File
@@ -3,11 +3,12 @@
* @description Terminal AI 面板主组件
* @module components/terminal/ai/TerminalAIPanel
*
* 参考 Waveterm 的 AIPanel 设计，实现终端 AI 助手面板
* 参考 Waveterm 的 AIPanel 设计，实现终端 AI 助手面板。
* 支持 AI 控制终端执行命令（需用户审批）。
*/
import React, { useState, useCallback } from "react";
import { Sparkles, MoreVertical, Trash2 } from "lucide-react";
import React, { useState, useCallback, useEffect } from "react";
import { Sparkles, MoreVertical, Trash2, Terminal, Unplug } from "lucide-react";
import { cn } from "@/lib/utils";
import { Switch } from "@/components/ui/switch";
import {
@@ -20,6 +21,7 @@ import { TerminalAIModeSelector } from "./TerminalAIModeSelector";
import { TerminalAIMessages } from "./TerminalAIMessages";
import { TerminalAIInput } from "./TerminalAIInput";
import { TerminalAIWelcome } from "./TerminalAIWelcome";
import { CommandApprovalList } from "./CommandApproval";
import { useTerminalAI } from "./useTerminalAI";
// ============================================================================
@@ -29,6 +31,8 @@ import { useTerminalAI } from "./useTerminalAI";
interface TerminalAIPanelProps {
/** 获取终端输出的回调 */
getTerminalOutput?: () => string | null;
/** 终端会话 ID（用于 AI 控制终端） */
terminalSessionId?: string | null;
/** 自定义类名 */
className?: string;
}
@@ -39,6 +43,7 @@ interface TerminalAIPanelProps {
export const TerminalAIPanel: React.FC<TerminalAIPanelProps> = ({
getTerminalOutput,
terminalSessionId,
className,
}) => {
const [input, setInput] = useState("");
@@ -54,8 +59,25 @@ export const TerminalAIPanel: React.FC<TerminalAIPanelProps> = ({
sendMessage,
clearMessages,
toggleWidgetContext,
toggleAutoExecute,
// 终端控制
isTerminalConnected,
pendingCommands,
connectTerminal,
disconnectTerminal,
approveCommand,
rejectCommand,
} = useTerminalAI(getTerminalOutput);
// 当终端会话 ID 变化时，自动连接/断开
useEffect(() => {
if (terminalSessionId) {
connectTerminal(terminalSessionId);
} else {
disconnectTerminal();
}
}, [terminalSessionId, connectTerminal, disconnectTerminal]);
/**
* 处理发送
*/
@@ -74,6 +96,8 @@ export const TerminalAIPanel: React.FC<TerminalAIPanelProps> = ({
}, []);
const hasMessages = messages.length > 0;
const hasPendingCommands =
pendingCommands.filter((c) => c.status === "pending").length > 0;
return (
<div
@@ -87,6 +111,18 @@ export const TerminalAIPanel: React.FC<TerminalAIPanelProps> = ({
<div className="flex items-center gap-2">
<Sparkles size={16} className="text-yellow-400" />
<span className="font-medium text-zinc-200">Terminal AI</span>
{/* 终端连接状态指示器 */}
{isTerminalConnected ? (
<span className="flex items-center gap-1 text-xs text-green-400">
<Terminal size={12} />
已连接
</span>
) : (
<span className="flex items-center gap-1 text-xs text-zinc-500">
<Unplug size={12} />
未连接
</span>
)}
</div>
{/* 更多菜单 */}
@@ -121,6 +157,19 @@ export const TerminalAIPanel: React.FC<TerminalAIPanelProps> = ({
/>
</div>
{/* 自动执行开关 */}
<div className="flex items-center justify-between px-3 py-2 border-b border-zinc-700/50">
<div className="flex flex-col">
<span className="text-sm text-zinc-400">自动执行命令</span>
<span className="text-xs text-zinc-500">AI 命令无需手动批准</span>
</div>
<Switch
checked={config.autoExecute}
onCheckedChange={toggleAutoExecute}
className="data-[state=checked]:bg-green-500"
/>
</div>
{/* 模式选择器 */}
<div className="px-3 py-2 border-b border-zinc-700/50">
<TerminalAIModeSelector
@@ -131,6 +180,17 @@ export const TerminalAIPanel: React.FC<TerminalAIPanelProps> = ({
/>
</div>
{/* 待审批命令 */}
{hasPendingCommands && (
<div className="px-3 py-2 border-b border-zinc-700/50">
<CommandApprovalList
commands={pendingCommands.filter((c) => c.status === "pending")}
onApprove={approveCommand}
onReject={rejectCommand}
/>
</div>
)}
{/* 消息区域 */}
<div className="flex-1 min-h-0 overflow-hidden flex flex-col">
{hasMessages ? (
@@ -146,9 +206,7 @@ export const TerminalAIPanel: React.FC<TerminalAIPanelProps> = ({
onChange={setInput}
onSubmit={handleSend}
isSending={isSending}
placeholder={
hasMessages ? "Continue..." : "Ask Terminal AI anything..."
}
placeholder={hasMessages ? "继续对话..." : "向 Terminal AI 提问..."}
/>
</div>
);
src/components/terminal/ai/TerminalController.ts
+315
View File
@@ -0,0 +1,315 @@
/**
* @file TerminalController.ts
* @description 终端控制器 - 管理 AI 与终端的通信
* @module components/terminal/ai/TerminalController
*
* 提供 AI 控制终端的核心能力：
* - 发送命令到活动终端
* - 获取终端输出上下文
* - 跟踪终端状态（Shell Integration）
*
* ## 架构说明（参考 Waveterm）
* Waveterm 使用 `sendDataToController()` 方法将命令发送到终端。
* 我们实现类似的机制，通过 `writeToTerminal` API 发送数据。
*/
import { writeToTerminal } from "@/lib/terminal-api";
import type { TermWrap } from "../termwrap";
/** 终端控制器配置 */
export interface TerminalControllerConfig {
/** 是否需要用户确认执行命令 */
requireApproval?: boolean;
/** 命令执行超时（毫秒） */
timeout?: number;
}
/** 命令执行状态 */
export type CommandExecutionStatus =
| "pending"
| "approved"
| "rejected"
| "executing"
| "completed"
| "failed"
| "timeout";
/** 待执行命令 */
export interface PendingCommand {
id: string;
command: string;
status: CommandExecutionStatus;
createdAt: Date;
executedAt?: Date;
completedAt?: Date;
error?: string;
}
/** 终端控制器回调 */
export interface TerminalControllerCallbacks {
/** 命令需要审批时调用 */
onCommandPending?: (command: PendingCommand) => void;
/** 命令状态变化时调用 */
onCommandStatusChange?: (command: PendingCommand) => void;
/** 命令执行完成时调用 */
onCommandComplete?: (command: PendingCommand) => void;
}
/**
* 终端控制器
*
* 管理 AI 与终端的通信，支持命令审批流程。
*/
export class TerminalController {
private sessionId: string | null = null;
private termWrap: TermWrap | null = null;
private config: TerminalControllerConfig;
private callbacks: TerminalControllerCallbacks;
private pendingCommands: Map<string, PendingCommand> = new Map();
constructor(
config: TerminalControllerConfig = {},
callbacks: TerminalControllerCallbacks = {},
) {
this.config = {
requireApproval: true,
timeout: 30000,
...config,
};
this.callbacks = callbacks;
}
/**
* 连接到终端
*/
connect(sessionId: string, termWrap?: TermWrap): void {
this.sessionId = sessionId;
this.termWrap = termWrap || null;
console.log("[TerminalController] 已连接到终端:", sessionId);
}
/**
* 断开终端连接
*/
disconnect(): void {
this.sessionId = null;
this.termWrap = null;
this.pendingCommands.clear();
console.log("[TerminalController] 已断开终端连接");
}
/**
* 检查是否已连接
*/
isConnected(): boolean {
return this.sessionId !== null;
}
/**
* 获取当前会话 ID
*/
getSessionId(): string | null {
return this.sessionId;
}
/**
* 发送命令到终端
*
* 如果配置了 requireApproval，命令会先进入待审批状态。
*
* @param command - 要执行的命令
* @param autoExecute - 是否自动执行（跳过审批）
* @returns 命令 ID
*/
async sendCommand(
command: string,
autoExecute: boolean = false,
): Promise<string> {
if (!this.sessionId) {
throw new Error("终端未连接");
}
const pendingCommand: PendingCommand = {
id: crypto.randomUUID(),
command,
status: "pending",
createdAt: new Date(),
};
this.pendingCommands.set(pendingCommand.id, pendingCommand);
if (this.config.requireApproval && !autoExecute) {
// 需要用户审批
this.callbacks.onCommandPending?.(pendingCommand);
return pendingCommand.id;
}
// 直接执行
await this.executeCommand(pendingCommand.id);
return pendingCommand.id;
}
/**
* 批准并执行命令
*/
async approveCommand(commandId: string): Promise<void> {
const command = this.pendingCommands.get(commandId);
if (!command) {
throw new Error(`命令不存在: ${commandId}`);
}
if (command.status !== "pending") {
throw new Error(`命令状态无效: ${command.status}`);
}
command.status = "approved";
this.callbacks.onCommandStatusChange?.(command);
await this.executeCommand(commandId);
}
/**
* 拒绝命令
*/
rejectCommand(commandId: string): void {
const command = this.pendingCommands.get(commandId);
if (!command) {
return;
}
command.status = "rejected";
this.callbacks.onCommandStatusChange?.(command);
this.pendingCommands.delete(commandId);
}
/**
* 执行命令
*/
private async executeCommand(commandId: string): Promise<void> {
const command = this.pendingCommands.get(commandId);
if (!command || !this.sessionId) {
return;
}
command.status = "executing";
command.executedAt = new Date();
this.callbacks.onCommandStatusChange?.(command);
try {
// 发送命令到终端（添加换行符执行）
const commandWithNewline = command.command.endsWith("\n")
? command.command
: command.command + "\n";
await writeToTerminal(this.sessionId, commandWithNewline);
command.status = "completed";
command.completedAt = new Date();
this.callbacks.onCommandComplete?.(command);
} catch (error) {
command.status = "failed";
command.error = error instanceof Error ? error.message : String(error);
this.callbacks.onCommandStatusChange?.(command);
}
}
/**
* 直接发送数据到终端（不添加换行符）
*
* 用于发送特殊按键、控制序列等。
*/
async sendData(data: string): Promise<void> {
if (!this.sessionId) {
throw new Error("终端未连接");
}
await writeToTerminal(this.sessionId, data);
}
/**
* 发送 Ctrl+C 中断信号
*/
async sendInterrupt(): Promise<void> {
await this.sendData("\x03");
}
/**
* 发送 Ctrl+D EOF 信号
*/
async sendEOF(): Promise<void> {
await this.sendData("\x04");
}
/**
* 获取终端输出（用于 AI 上下文）
*/
getTerminalOutput(): string | null {
if (!this.termWrap) {
return null;
}
// 从 xterm.js 获取缓冲区内容
const terminal = this.termWrap.terminal;
if (!terminal) {
return null;
}
const buffer = terminal.buffer.active;
const lines: string[] = [];
// 获取所有行
for (let i = 0; i < buffer.length; i++) {
const line = buffer.getLine(i);
if (line) {
lines.push(line.translateToString(true));
}
}
return lines.join("\n");
}
/**
* 获取待审批的命令列表
*/
getPendingCommands(): PendingCommand[] {
return Array.from(this.pendingCommands.values()).filter(
(cmd) => cmd.status === "pending",
);
}
/**
* 清除所有待审批命令
*/
clearPendingCommands(): void {
for (const [id, command] of this.pendingCommands) {
if (command.status === "pending") {
command.status = "rejected";
this.callbacks.onCommandStatusChange?.(command);
this.pendingCommands.delete(id);
}
}
}
}
// 全局终端控制器实例（单例）
let globalController: TerminalController | null = null;
/**
* 获取全局终端控制器
*/
export function getTerminalController(): TerminalController {
if (!globalController) {
globalController = new TerminalController();
}
return globalController;
}
/**
* 重置全局终端控制器
*/
export function resetTerminalController(): void {
if (globalController) {
globalController.disconnect();
globalController = null;
}
}
src/components/terminal/ai/index.ts
+10
View File
@@ -11,3 +11,13 @@ export { TerminalAIInput } from "./TerminalAIInput";
export { TerminalAIMessages } from "./TerminalAIMessages";
export { TerminalAIModeSelector } from "./TerminalAIModeSelector";
export { TerminalAIWelcome } from "./TerminalAIWelcome";
export { CommandApproval, CommandApprovalList } from "./CommandApproval";
export {
TerminalController,
getTerminalController,
resetTerminalController,
type TerminalControllerConfig,
type TerminalControllerCallbacks,
type PendingCommand,
type CommandExecutionStatus,
} from "./TerminalController";
src/components/terminal/ai/types.ts
+35
View File
@@ -47,6 +47,8 @@ export interface TerminalAIConfig {
widgetContext: boolean;
/** 上下文行数限制 */
contextLines: number;
/** 是否自动执行命令（不需要手动批准） */
autoExecute: boolean;
}
/**
@@ -68,6 +70,29 @@ export interface ModelSelection {
modelId: string;
}
/**
* 待执行命令（用于 AI 控制终端）
*/
export interface PendingTerminalCommand {
id: string;
command: string;
status:
| "pending"
| "approved"
| "rejected"
| "executing"
| "completed"
| "failed";
createdAt: Date;
executedAt?: Date;
completedAt?: Date;
error?: string;
/** 工作目录（可选，来自后端请求） */
workingDir?: string;
/** 超时时间（秒，来自后端请求） */
timeoutSecs?: number;
}
/**
* Terminal AI Hook 返回值
*/
@@ -87,8 +112,18 @@ export interface UseTerminalAIReturn {
sendMessage: (content: string, images?: AIMessageImage[]) => Promise<void>;
clearMessages: () => void;
toggleWidgetContext: () => void;
toggleAutoExecute: () => void;
setContextLines: (lines: number) => void;
// 终端上下文
getTerminalContext: () => string | null;
// 终端控制（新增）
isTerminalConnected: boolean;
pendingCommands: PendingTerminalCommand[];
connectTerminal: (sessionId: string) => void;
disconnectTerminal: () => void;
sendCommandToTerminal: (command: string) => Promise<string>;
approveCommand: (commandId: string) => Promise<void>;
rejectCommand: (commandId: string) => Promise<void>;
}
src/components/terminal/ai/useTerminalAI.ts
+401 -3
View File
@@ -3,10 +3,11 @@
* @description Terminal AI Hook - 管理 AI 聊天状态和操作
* @module components/terminal/ai/useTerminalAI
*
* 复用 Agent 模块的 API，提供 Terminal 专用的 AI 聊天功能
* 复用 Agent 模块的 API，提供 Terminal 专用的 AI 聊天功能。
* 支持 AI 控制终端执行命令（参考 Waveterm）。
*/
import { useState, useEffect, useCallback } from "react";
import { useState, useEffect, useCallback, useRef } from "react";
import { toast } from "sonner";
import { listen, type UnlistenFn } from "@tauri-apps/api/event";
import {
@@ -15,14 +16,20 @@ import {
createAgentSession,
sendAgentMessageStream,
parseStreamEvent,
sendTerminalCommandResponse,
sendTermScrollbackResponse,
type StreamEvent,
type TerminalCommandRequest,
type TermScrollbackRequest,
} from "@/lib/api/agent";
import { writeToTerminal } from "@/lib/terminal-api";
import type {
AIMessage,
AIMessageImage,
AIContentPart,
TerminalAIConfig,
UseTerminalAIReturn,
PendingTerminalCommand,
} from "./types";
// 存储键
@@ -37,6 +44,7 @@ const STORAGE_KEYS = {
const DEFAULT_CONFIG: TerminalAIConfig = {
widgetContext: true,
contextLines: 50,
autoExecute: false, // 默认需要手动批准
};
/**
@@ -93,6 +101,28 @@ export function useTerminalAI(
// 会话 ID
const [sessionId, setSessionId] = useState<string | null>(null);
// 终端控制状态
const [terminalSessionId, setTerminalSessionId] = useState<string | null>(
null,
);
const [pendingCommands, setPendingCommands] = useState<
PendingTerminalCommand[]
>([]);
const pendingCommandsRef = useRef<Map<string, PendingTerminalCommand>>(
new Map(),
);
// 使用 ref 存储最新的 config，避免闭包问题
const configRef = useRef(config);
useEffect(() => {
configRef.current = config;
}, [config]);
// 使用 ref 存储 approveCommand 函数引用
const approveCommandRef = useRef<
((commandId: string) => Promise<void>) | null
>(null);
// 持久化
useEffect(() => {
savePersisted(STORAGE_KEYS.PROVIDER, providerId);
@@ -387,6 +417,9 @@ export function useTerminalAI(
media_type: img.mediaType,
}));
// 如果已连接终端，启用 terminal_mode（使用 TerminalTool 替代 BashTool）
const useTerminalMode = terminalSessionId !== null;
await sendAgentMessageStream(
messageContent,
eventName,
@@ -394,6 +427,7 @@ export function useTerminalAI(
modelId,
imagesToSend,
providerId,
useTerminalMode,
);
} catch (error) {
console.error("[useTerminalAI] 发送消息失败:", error);
@@ -405,7 +439,7 @@ export function useTerminalAI(
}
}
},
[ensureSession, getTerminalContext, modelId, providerId],
[ensureSession, getTerminalContext, modelId, providerId, terminalSessionId],
);
/**
@@ -437,6 +471,361 @@ export function useTerminalAI(
}));
}, []);
// ============================================================================
// 终端控制功能
// ============================================================================
/**
* 连接到终端
*/
const connectTerminal = useCallback((sessionId: string) => {
setTerminalSessionId(sessionId);
console.log("[useTerminalAI] 已连接到终端:", sessionId);
}, []);
/**
* 断开终端连接
*/
const disconnectTerminal = useCallback(() => {
setTerminalSessionId(null);
setPendingCommands([]);
pendingCommandsRef.current.clear();
console.log("[useTerminalAI] 已断开终端连接");
}, []);
/**
* 监听后端发送的终端命令请求
*/
useEffect(() => {
if (!terminalSessionId) return;
let unlisten: UnlistenFn | null = null;
const setupListener = async () => {
unlisten = await listen<TerminalCommandRequest>(
"terminal_command_request",
(event) => {
const request = event.payload;
console.log("[useTerminalAI] 收到终端命令请求:", request);
// 创建待审批命令
const pendingCommand: PendingTerminalCommand = {
id: request.request_id,
command: request.command,
status: "pending",
createdAt: new Date(),
workingDir: request.working_dir,
timeoutSecs: request.timeout_secs,
};
pendingCommandsRef.current.set(pendingCommand.id, pendingCommand);
setPendingCommands(Array.from(pendingCommandsRef.current.values()));
// 如果开启了自动执行，直接批准命令
if (configRef.current.autoExecute) {
console.log("[useTerminalAI] 自动执行模式已启用，自动批准命令");
// 使用 setTimeout 确保状态更新后再执行
setTimeout(() => {
if (approveCommandRef.current) {
approveCommandRef.current(pendingCommand.id);
}
}, 0);
// 不显示 toast，避免遮挡终端输出
console.log("[useTerminalAI] AI 命令自动执行:", request.command);
} else {
// 显示通知，需要手动批准（这个保留，因为需要用户注意）
toast.info("AI 请求执行命令，请审批", {
description:
request.command.slice(0, 50) +
(request.command.length > 50 ? "..." : ""),
});
}
},
);
};
setupListener();
return () => {
if (unlisten) {
unlisten();
}
};
}, [terminalSessionId]);
/**
* 监听后端发送的终端滚动缓冲区请求
*/
useEffect(() => {
if (!terminalSessionId) return;
let unlisten: UnlistenFn | null = null;
const setupListener = async () => {
unlisten = await listen<TermScrollbackRequest>(
"term_get_scrollback_request",
async (event) => {
const request = event.payload;
console.log("[useTerminalAI] 收到终端滚动缓冲区请求:", request);
try {
// 获取终端输出
const output = getTerminalOutput ? getTerminalOutput() : null;
if (!output) {
// 没有输出
await sendTermScrollbackResponse({
request_id: request.request_id,
success: true,
total_lines: 0,
line_start: 0,
line_end: 0,
content: "",
has_more: false,
});
return;
}
// 分割成行
const lines = output.split("\n");
const totalLines = lines.length;
// 计算实际的起始和结束行号
const requestedStart = request.line_start ?? 0;
const requestedCount = request.count ?? totalLines;
const actualStart = Math.max(
0,
Math.min(requestedStart, totalLines - 1),
);
const actualEnd = Math.min(
actualStart + requestedCount,
totalLines,
);
// 提取请求的行
const requestedLines = lines.slice(actualStart, actualEnd);
const content = requestedLines.join("\n");
// 发送响应
await sendTermScrollbackResponse({
request_id: request.request_id,
success: true,
total_lines: totalLines,
line_start: actualStart,
line_end: actualEnd,
content,
has_more: actualEnd < totalLines,
});
console.log(
`[useTerminalAI] 已发送滚动缓冲区响应: ${actualStart}-${actualEnd}/${totalLines} 行`,
);
} catch (error) {
console.error("[useTerminalAI] 处理滚动缓冲区请求失败:", error);
// 发送错误响应
await sendTermScrollbackResponse({
request_id: request.request_id,
success: false,
total_lines: 0,
line_start: 0,
line_end: 0,
content: "",
has_more: false,
error: error instanceof Error ? error.message : String(error),
});
}
},
);
};
setupListener();
return () => {
if (unlisten) {
unlisten();
}
};
}, [terminalSessionId, getTerminalOutput]);
/**
* 发送命令到终端（需要用户审批）- 本地创建的命令
*/
const sendCommandToTerminal = useCallback(
async (command: string): Promise<string> => {
if (!terminalSessionId) {
throw new Error("终端未连接");
}
const pendingCommand: PendingTerminalCommand = {
id: crypto.randomUUID(),
command,
status: "pending",
createdAt: new Date(),
};
pendingCommandsRef.current.set(pendingCommand.id, pendingCommand);
setPendingCommands(Array.from(pendingCommandsRef.current.values()));
return pendingCommand.id;
},
[terminalSessionId],
);
/**
* 批准并执行命令（内部实现）
*/
const approveCommandInternal = useCallback(
async (commandId: string): Promise<void> => {
const command = pendingCommandsRef.current.get(commandId);
if (!command || !terminalSessionId) {
return;
}
if (command.status !== "pending") {
return;
}
// 更新状态为执行中
command.status = "executing";
command.executedAt = new Date();
pendingCommandsRef.current.set(commandId, command);
setPendingCommands(Array.from(pendingCommandsRef.current.values()));
try {
// 发送命令到终端（添加换行符执行）
const commandWithNewline = command.command.endsWith("\n")
? command.command
: command.command + "\n";
await writeToTerminal(terminalSessionId, commandWithNewline);
// 更新状态为完成
command.status = "completed";
command.completedAt = new Date();
pendingCommandsRef.current.set(commandId, command);
setPendingCommands(Array.from(pendingCommandsRef.current.values()));
// 发送响应给后端（告知 TerminalTool 命令已执行）
// 对于简单的 echo 命令，模拟输出以避免 AI 重复执行
let simulatedOutput = "";
const echoMatch = command.command.match(/^echo\s+(.+)$/i);
if (echoMatch) {
// 提取 echo 的内容（去除引号）
let echoContent = echoMatch[1];
// 移除外层引号
if (
(echoContent.startsWith('"') && echoContent.endsWith('"')) ||
(echoContent.startsWith("'") && echoContent.endsWith("'"))
) {
echoContent = echoContent.slice(1, -1);
}
simulatedOutput = `${echoContent}`;
}
// 构建明确的成功响应
const successOutput = simulatedOutput
? `[COMMAND EXECUTED SUCCESSFULLY]\nOutput:\n${simulatedOutput}\n[END OF OUTPUT]`
: `[COMMAND EXECUTED SUCCESSFULLY]\nThe command "${command.command}" has been executed in the user's terminal.\nNote: Output is displayed in the terminal window. Do NOT re-execute this command.\n[END OF OUTPUT]`;
await sendTerminalCommandResponse({
request_id: commandId,
success: true,
output: successOutput,
rejected: false,
});
// 不显示 toast，避免遮挡终端输出
// 命令执行状态已经在 AI 面板中显示
console.log("[useTerminalAI] 命令已执行:", command.command);
// 延迟移除已完成的命令
setTimeout(() => {
pendingCommandsRef.current.delete(commandId);
setPendingCommands(Array.from(pendingCommandsRef.current.values()));
}, 2000);
} catch (error) {
command.status = "failed";
command.error = error instanceof Error ? error.message : String(error);
pendingCommandsRef.current.set(commandId, command);
setPendingCommands(Array.from(pendingCommandsRef.current.values()));
// 发送失败响应给后端
await sendTerminalCommandResponse({
request_id: commandId,
success: false,
output: "",
error: command.error,
rejected: false,
});
toast.error(`命令执行失败: ${command.error}`);
}
},
[terminalSessionId],
);
// 更新 ref
useEffect(() => {
approveCommandRef.current = approveCommandInternal;
}, [approveCommandInternal]);
/**
* 批准并执行命令
*/
const approveCommand = useCallback(
async (commandId: string): Promise<void> => {
await approveCommandInternal(commandId);
},
[approveCommandInternal],
);
/**
* 拒绝命令
*/
const rejectCommand = useCallback(
async (commandId: string): Promise<void> => {
const command = pendingCommandsRef.current.get(commandId);
if (!command) {
return;
}
command.status = "rejected";
pendingCommandsRef.current.set(commandId, command);
setPendingCommands(Array.from(pendingCommandsRef.current.values()));
// 发送拒绝响应给后端
await sendTerminalCommandResponse({
request_id: commandId,
success: false,
output: "",
error: "用户拒绝执行此命令",
rejected: true,
});
toast.info("命令已拒绝");
// 延迟移除
setTimeout(() => {
pendingCommandsRef.current.delete(commandId);
setPendingCommands(Array.from(pendingCommandsRef.current.values()));
}, 1000);
},
[],
);
/**
* 切换自动执行模式
*/
const toggleAutoExecute = useCallback(() => {
setConfig((prev) => {
const newConfig = { ...prev, autoExecute: !prev.autoExecute };
savePersisted(STORAGE_KEYS.CONFIG, newConfig);
return newConfig;
});
}, []);
return {
messages,
isSending,
@@ -448,7 +837,16 @@ export function useTerminalAI(
sendMessage,
clearMessages,
toggleWidgetContext,
toggleAutoExecute,
setContextLines,
getTerminalContext,
// 终端控制
isTerminalConnected: terminalSessionId !== null,
pendingCommands,
connectTerminal,
disconnectTerminal,
sendCommandToTerminal,
approveCommand,
rejectCommand,
};
}
src/components/tools/ToolsPage.tsx
-8
View File
@@ -215,14 +215,6 @@ const placeholderTools: DynamicToolCard[] = [
* 推荐插件列表
*/
const recommendedPlugins: RecommendedPlugin[] = [
{
id: "terminal-plugin",
name: "终端",
description: "本地 PTY 和 SSH 终端模拟器，支持多标签页和搜索功能",
icon: "Terminal",
downloadUrl:
"https://github.com/aiclientproxy/terminal/releases/latest/download/terminal-plugin.zip",
},
{
id: "machine-id-tool",
name: "机器码管理工具",
src/hooks/useModelRegistry.ts
+36 -16
View File
@@ -164,35 +164,55 @@ export function useModelRegistry(
const [error, setError] = useState<string | null>(null);
const [lastSyncAt, setLastSyncAt] = useState<number | null>(null);
// 加载模型数据
// 加载模型数据（带重试机制）
const loadModels = useCallback(async () => {
setLoading(true);
setError(null);
try {
const [models, prefs, syncState] = await Promise.all([
modelRegistryApi.getModelRegistry(),
modelRegistryApi.getModelPreferences(),
modelRegistryApi.getModelSyncState(),
]);
const maxRetries = 5;
const retryDelay = 500; // 500ms
setAllModels(models);
setPreferences(new Map(prefs.map((p) => [p.model_id, p])));
setLastSyncAt(syncState.last_sync_at);
} catch (e) {
setError(e instanceof Error ? e.message : String(e));
} finally {
setLoading(false);
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
const [models, prefs, syncState] = await Promise.all([
modelRegistryApi.getModelRegistry(),
modelRegistryApi.getModelPreferences(),
modelRegistryApi.getModelSyncState(),
]);
setAllModels(models);
setPreferences(new Map(prefs.map((p) => [p.model_id, p])));
setLastSyncAt(syncState.last_sync_at);
setLoading(false);
return; // 成功，退出
} catch (e) {
const errorMsg = e instanceof Error ? e.message : String(e);
// 如果是"服务未初始化"错误，且还有重试次数，则等待后重试
if (errorMsg.includes("未初始化") && attempt < maxRetries - 1) {
console.log(
`[ModelRegistry] 服务未初始化，${retryDelay}ms 后重试 (${attempt + 1}/${maxRetries})`,
);
await new Promise((resolve) => setTimeout(resolve, retryDelay));
continue;
}
// 其他错误或已达到最大重试次数
setError(errorMsg);
setLoading(false);
return;
}
}
}, []);
// 刷新（强制从 models.dev 获取）
// 刷新（强制从内嵌资源重新加载）
const refresh = useCallback(async () => {
setLoading(true);
setError(null);
try {
await modelRegistryApi.refreshModelRegistry();
const count = await modelRegistryApi.refreshModelRegistry();
console.log(`[ModelRegistry] 刷新完成，加载了 ${count} 个模型`);
await loadModels();
} catch (e) {
setError(e instanceof Error ? e.message : String(e));
src/hooks/useTauri.ts
+63
View File
@@ -132,6 +132,26 @@ export interface ApiKeyEntry {
proxy_url: string | null;
}
// ============ 实验室功能配置 ============
/**
* 截图对话功能配置
*/
export interface ScreenshotChatConfig {
/** 是否启用截图对话功能 */
enabled: boolean;
/** 触发截图的全局快捷键 */
shortcut: string;
}
/**
* 实验室功能配置
*/
export interface ExperimentalFeatures {
/** 截图对话功能配置 */
screenshot_chat: ScreenshotChatConfig;
}
export interface Config {
server: {
host: string;
@@ -174,6 +194,8 @@ export interface Config {
minimize_to_tray: boolean;
/** 用户界面语言 ("zh" 或 "en") */
language: string;
/** 实验室功能配置 */
experimental?: ExperimentalFeatures;
}
export interface LogEntry {
@@ -511,3 +533,44 @@ export interface NetworkInfo {
export async function getNetworkInfo(): Promise<NetworkInfo> {
return safeInvoke("get_network_info");
}
// ============ 实验室功能 API ============
/**
* 获取实验室功能配置
* @returns 实验室功能配置对象
*/
export async function getExperimentalConfig(): Promise<ExperimentalFeatures> {
return safeInvoke("get_experimental_config");
}
/**
* 保存实验室功能配置
* @param config 实验室功能配置对象
*/
export async function saveExperimentalConfig(
config: ExperimentalFeatures,
): Promise<void> {
return safeInvoke("save_experimental_config", {
experimentalConfig: config,
});
}
/**
* 验证快捷键格式
* @param shortcut 快捷键字符串
* @returns 是否有效
*/
export async function validateShortcut(shortcut: string): Promise<boolean> {
return safeInvoke("validate_shortcut", { shortcutStr: shortcut });
}
/**
* 更新截图快捷键
* @param shortcut 新的快捷键字符串
*/
export async function updateScreenshotShortcut(
shortcut: string,
): Promise<void> {
return safeInvoke("update_screenshot_shortcut", { newShortcut: shortcut });
}
src/lib/api/agent.ts
+116
View File
@@ -313,6 +313,7 @@ export async function sendAgentMessageStream(
model?: string,
images?: ImageInput[],
provider?: string,
terminalMode?: boolean,
): Promise<void> {
return await invoke("native_agent_chat_stream", {
message,
@@ -321,6 +322,7 @@ export async function sendAgentMessageStream(
model,
images,
provider,
terminalMode,
});
}
@@ -450,3 +452,117 @@ export async function extendGooseSystemPrompt(
export async function listGooseProviders(): Promise<GooseProviderInfo[]> {
return await invoke("goose_agent_list_providers");
}
// ============================================================
// Terminal Tool API (终端命令执行)
// ============================================================
/**
* 终端命令请求（从后端发送到前端）
*/
export interface TerminalCommandRequest {
/** 请求 ID */
request_id: string;
/** 要执行的命令 */
command: string;
/** 工作目录（可选） */
working_dir?: string;
/** 超时时间（秒） */
timeout_secs: number;
}
/**
* 终端命令响应（从前端发送到后端）
*/
export interface TerminalCommandResponse {
/** 请求 ID */
request_id: string;
/** 是否成功 */
success: boolean;
/** 输出内容 */
output: string;
/** 错误信息 */
error?: string;
/** 退出码 */
exit_code?: number;
/** 是否被用户拒绝 */
rejected: boolean;
}
/**
* 发送终端命令响应到后端
*
* 当用户批准或拒绝命令后，调用此函数将结果发送给 TerminalTool
*/
export async function sendTerminalCommandResponse(
response: TerminalCommandResponse,
): Promise<void> {
return await invoke("agent_terminal_command_response", {
requestId: response.request_id,
success: response.success,
output: response.output,
error: response.error,
exitCode: response.exit_code,
rejected: response.rejected,
});
}
// ============================================================
// Terminal Scrollback Tool API (终端输出历史读取)
// ============================================================
/**
* 终端滚动缓冲区请求（从后端发送到前端）
*/
export interface TermScrollbackRequest {
/** 请求 ID */
request_id: string;
/** 终端会话 ID */
session_id: string;
/** 起始行号（可选，从 0 开始） */
line_start?: number;
/** 读取行数（可选） */
count?: number;
}
/**
* 终端滚动缓冲区响应（从前端发送到后端）
*/
export interface TermScrollbackResponse {
/** 请求 ID */
request_id: string;
/** 是否成功 */
success: boolean;
/** 总行数 */
total_lines: number;
/** 实际返回的起始行号 */
line_start: number;
/** 实际返回的结束行号 */
line_end: number;
/** 输出内容 */
content: string;
/** 是否还有更多内容 */
has_more: boolean;
/** 错误信息 */
error?: string;
}
/**
* 发送终端滚动缓冲区响应到后端
*
* 当前端读取终端输出历史后，调用此函数将结果发送给 TermScrollbackTool
*/
export async function sendTermScrollbackResponse(
response: TermScrollbackResponse,
): Promise<void> {
return await invoke("agent_term_scrollback_response", {
requestId: response.request_id,
success: response.success,
totalLines: response.total_lines,
lineStart: response.line_start,
lineEnd: response.line_end,
content: response.content,
hasMore: response.has_more,
error: response.error,
});
}
src/lib/api/modelRegistry.ts
+3 -2
View File
@@ -21,9 +21,10 @@ export async function getModelRegistry(): Promise<EnhancedModelMetadata[]> {
}
/**
* 刷新模型注册表（从 models.dev 获取最新数据）
* 刷新模型注册表（强制从内嵌资源重新加载）
* @returns 加载的模型数量
*/
export async function refreshModelRegistry(): Promise<void> {
export async function refreshModelRegistry(): Promise<number> {
return invoke("refresh_model_registry");
}
src/lib/plugin-loader/index.ts
+1 -1
View File
@@ -34,6 +34,7 @@ const loadedPlugins = new Map<string, PluginModule>();
/**
* 插件 ID 到全局变量名的映射
* 格式: pluginId -> GlobalVariableName
* 注意: terminal-plugin 已移除，终端功能已内置到应用中
*/
const PLUGIN_GLOBAL_NAMES: Record<string, string> = {
"kiro-provider": "KiroProviderPlugin",
@@ -42,7 +43,6 @@ const PLUGIN_GLOBAL_NAMES: Record<string, string> = {
"gemini-provider": "GeminiProviderPlugin",
"antigravity-provider": "AntigravityProviderPlugin",
"codex-provider": "CodexProviderPlugin",
"terminal-plugin": "TerminalPlugin",
};
/**
src/main.tsx
+2 -9
View File
@@ -1,7 +1,5 @@
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App";
import { Toaster } from "./components/ui/sonner";
import { RootRouter } from "./RootRouter";
import "./index.css";
// Initialize Tauri mock for web mode
@@ -13,9 +11,4 @@ import "./i18n/config";
// 初始化插件组件全局暴露（供动态加载的插件使用）
import "./lib/plugin-components/global";
ReactDOM.createRoot(document.getElementById("root")!).render(
<>
<App />
<Toaster />
</>,
);
ReactDOM.createRoot(document.getElementById("root")!).render(<RootRouter />);
src/pages/README.md
+1
View File
@@ -10,6 +10,7 @@
## 文件索引
- `FlowMonitorPage.tsx` - LLM 流量监控页面
- `screenshot-chat.tsx` - 截图对话悬浮窗口页面（独立 Tauri 窗口）
- `index.ts` - 页面导出入口
## 更新提醒
src/pages/index.ts
+1
View File
@@ -1 +1,2 @@
export { FlowMonitorPage } from "./FlowMonitorPage";
export { ScreenshotChatPage } from "./screenshot-chat";
src/pages/screenshot-chat.css
+293
View File
@@ -0,0 +1,293 @@
/**
* @file screenshot-chat.css
* @description 截图对话输入框样式 - 参考 Google Gemini 浮动栏设计
* 半透明药丸形状，毛玻璃效果
*/
/* 全局重置 */
html,
body,
#root {
margin: 0;
padding: 0;
width: 100%;
height: 100%;
background: transparent !important;
overflow: hidden;
}
/* 容器 - 增加 padding 来容纳阴影 */
.screenshot-container {
width: 100%;
height: 100%;
padding: 10px;
box-sizing: border-box;
display: flex;
align-items: center;
justify-content: center;
background: transparent !important;
}
/* 输入框容器 - Gemini 风格半透明药丸 */
.screenshot-input-bar {
display: flex;
align-items: center;
gap: 8px;
width: 100%;
height: 48px;
padding: 0 12px;
/* 半透明毛玻璃效果 */
background: rgba(255, 255, 255, 0.85);
backdrop-filter: blur(20px);
-webkit-backdrop-filter: blur(20px);
/* 完全药丸形状 - 高度的一半 */
border-radius: 24px;
/* 柔和阴影 */
box-shadow:
0 2px 8px rgba(0, 0, 0, 0.08),
0 4px 24px rgba(0, 0, 0, 0.12),
inset 0 0 0 1px rgba(255, 255, 255, 0.5);
box-sizing: border-box;
transition: all 0.2s ease;
}
.screenshot-input-bar:focus-within {
background: rgba(255, 255, 255, 0.95);
box-shadow:
0 4px 12px rgba(0, 0, 0, 0.1),
0 8px 32px rgba(0, 0, 0, 0.15),
inset 0 0 0 1px rgba(255, 255, 255, 0.6);
}
/* 拖动手柄 */
.screenshot-drag-handle {
display: flex;
align-items: center;
justify-content: center;
width: 20px;
height: 32px;
color: #9ca3af;
cursor: grab;
flex-shrink: 0;
border-radius: 4px;
transition: all 0.15s ease;
}
.screenshot-drag-handle:hover {
color: #6b7280;
background: rgba(0, 0, 0, 0.05);
}
.screenshot-drag-handle:active {
cursor: grabbing;
color: #374151;
}
/* Logo */
.screenshot-logo {
flex-shrink: 0;
opacity: 0.9;
}
/* 图片附件标签 - 更紧凑的药丸样式 */
.screenshot-attachment {
display: flex;
align-items: center;
gap: 4px;
padding: 4px 8px 4px 6px;
background: rgba(124, 77, 255, 0.1);
border: 1px solid rgba(124, 77, 255, 0.2);
border-radius: 14px;
color: #5c3dbd;
font-size: 11px;
font-weight: 500;
flex-shrink: 0;
transition: all 0.15s ease;
}
.screenshot-attachment:hover {
background: rgba(124, 77, 255, 0.15);
}
.screenshot-attachment svg:first-child {
color: #7c4dff;
width: 12px;
height: 12px;
}
.screenshot-attachment-remove {
display: flex;
align-items: center;
justify-content: center;
width: 14px;
height: 14px;
margin-left: 2px;
padding: 0;
background: transparent;
border: none;
border-radius: 50%;
color: #9ca3af;
cursor: pointer;
transition: all 0.15s ease;
}
.screenshot-attachment-remove:hover {
background: rgba(124, 77, 255, 0.2);
color: #5c3dbd;
}
/* 输入框 */
.screenshot-input {
flex: 1;
min-width: 0;
padding: 6px 0;
border: none;
background: transparent;
font-size: 14px;
color: #1f2937;
outline: none;
}
.screenshot-input::placeholder {
color: #9ca3af;
}
.screenshot-input:disabled {
opacity: 0.6;
}
/* 右侧按钮组 */
.screenshot-actions {
display: flex;
align-items: center;
gap: 4px;
flex-shrink: 0;
}
/* 关闭按钮 */
.screenshot-close-btn {
display: flex;
align-items: center;
justify-content: center;
width: 24px;
height: 24px;
background: transparent;
border: none;
border-radius: 50%;
color: #9ca3af;
cursor: pointer;
transition: all 0.15s ease;
}
.screenshot-close-btn:hover {
background: rgba(239, 68, 68, 0.1);
color: #ef4444;
}
/* 工具按钮（+号菜单风格） */
.screenshot-tools-btn {
display: flex;
align-items: center;
justify-content: center;
width: 28px;
height: 28px;
background: transparent;
border: none;
border-radius: 50%;
color: #6b7280;
cursor: pointer;
transition: all 0.15s ease;
}
.screenshot-tools-btn:hover {
background: rgba(0, 0, 0, 0.05);
color: #374151;
}
/* 发送按钮 - Gemini 风格圆形 */
.screenshot-send-btn {
display: flex;
align-items: center;
justify-content: center;
width: 32px;
height: 32px;
background: #e5e7eb;
border: none;
border-radius: 50%;
color: #9ca3af;
cursor: not-allowed;
transition: all 0.2s ease;
}
.screenshot-send-btn.active {
background: linear-gradient(135deg, #7c4dff 0%, #536dfe 100%);
color: white;
cursor: pointer;
box-shadow: 0 2px 8px rgba(124, 77, 255, 0.3);
}
.screenshot-send-btn.active:hover {
transform: scale(1.05);
box-shadow: 0 4px 12px rgba(124, 77, 255, 0.4);
}
/* 暗色模式支持 */
@media (prefers-color-scheme: dark) {
.screenshot-input-bar {
background: rgba(30, 30, 30, 0.85);
box-shadow:
0 2px 8px rgba(0, 0, 0, 0.2),
0 4px 24px rgba(0, 0, 0, 0.3),
inset 0 0 0 1px rgba(255, 255, 255, 0.1);
}
.screenshot-input-bar:focus-within {
background: rgba(40, 40, 40, 0.95);
}
.screenshot-drag-handle {
color: #6b7280;
}
.screenshot-drag-handle:hover {
color: #9ca3af;
background: rgba(255, 255, 255, 0.1);
}
.screenshot-input {
color: #f3f4f6;
}
.screenshot-input::placeholder {
color: #6b7280;
}
.screenshot-attachment {
background: rgba(124, 77, 255, 0.2);
border-color: rgba(124, 77, 255, 0.3);
color: #a78bfa;
}
.screenshot-close-btn {
color: #6b7280;
}
.screenshot-close-btn:hover {
background: rgba(239, 68, 68, 0.2);
color: #f87171;
}
.screenshot-tools-btn {
color: #9ca3af;
}
.screenshot-tools-btn:hover {
background: rgba(255, 255, 255, 0.1);
color: #e5e7eb;
}
.screenshot-send-btn {
background: #374151;
color: #6b7280;
}
}
src/pages/screenshot-chat.tsx
+197
View File
@@ -0,0 +1,197 @@
/**
* @file screenshot-chat.tsx
* @description 截图对话悬浮窗口 - 参考 Google Gemini 浮动栏设计
* 半透明药丸形状，简洁的输入界面
* @module pages/screenshot-chat
*/
import React, { useEffect, useState, useRef, useCallback } from "react";
import { Image as ImageIcon, ArrowUp, X, GripVertical } from "lucide-react";
import { getCurrentWindow } from "@tauri-apps/api/window";
import "./screenshot-chat.css";
// ProxyCast Logo组件
function Logo() {
return (
<svg
viewBox="0 0 128 128"
width="20"
height="20"
className="screenshot-logo"
>
<defs>
<linearGradient id="leftP" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" style={{ stopColor: "#4fc3f7" }} />
<stop offset="100%" style={{ stopColor: "#1a237e" }} />
</linearGradient>
<linearGradient id="rightP" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" style={{ stopColor: "#7c4dff" }} />
<stop offset="100%" style={{ stopColor: "#e91e63" }} />
</linearGradient>
</defs>
<g>
<rect x="36" y="32" width="10" height="64" rx="3" fill="url(#leftP)" />
<rect x="46" y="32" width="28" height="9" rx="3" fill="url(#rightP)" />
<rect x="46" y="60" width="24" height="8" rx="2" fill="url(#rightP)" />
<rect x="70" y="41" width="8" height="27" rx="3" fill="url(#rightP)" />
</g>
</svg>
);
}
function getImagePathFromUrl(): string | null {
const params = new URLSearchParams(window.location.search);
const imagePath = params.get("image");
return imagePath ? decodeURIComponent(imagePath) : null;
}
export function ScreenshotChatPage() {
const [imagePath, setImagePath] = useState<string | null>(null);
const [inputValue, setInputValue] = useState("");
const [isLoading, setIsLoading] = useState(false);
const inputRef = useRef<HTMLInputElement>(null);
// 从 URL 获取图片路径
useEffect(() => {
const path = getImagePathFromUrl();
if (path) {
setImagePath(path);
}
}, []);
// 自动聚焦
useEffect(() => {
inputRef.current?.focus();
}, []);
// 关闭窗口
const handleClose = useCallback(async () => {
try {
await getCurrentWindow().close();
} catch (err) {
console.error("关闭窗口失败:", err);
}
}, []);
// ESC 关闭窗口
useEffect(() => {
const handleKeyDown = async (e: KeyboardEvent) => {
if (e.key === "Escape") {
await handleClose();
}
};
window.addEventListener("keydown", handleKeyDown);
return () => window.removeEventListener("keydown", handleKeyDown);
}, [handleClose]);
// 开始拖动窗口
const handleStartDrag = useCallback(async (e: React.MouseEvent) => {
// 只响应左键
if (e.button !== 0) return;
try {
await getCurrentWindow().startDragging();
} catch (err) {
console.error("拖动窗口失败:", err);
}
}, []);
// 移除图片附件
const handleRemoveImage = () => {
setImagePath(null);
};
// 发送到主应用
const handleSend = async () => {
if (!inputValue.trim() || isLoading) return;
setIsLoading(true);
try {
const { invoke } = await import("@tauri-apps/api/core");
await invoke("send_screenshot_chat", {
message: inputValue,
imagePath: imagePath,
});
await getCurrentWindow().close();
} catch (err) {
console.error("发送失败:", err);
setIsLoading(false);
}
};
const handleInputKeyDown = (e: React.KeyboardEvent) => {
if (e.key === "Enter" && !e.shiftKey) {
e.preventDefault();
handleSend();
}
};
return (
<div className="screenshot-container">
<div className="screenshot-input-bar">
{/* 拖动手柄 */}
<div
className="screenshot-drag-handle"
onMouseDown={handleStartDrag}
title="拖动移动窗口"
>
<GripVertical size={14} />
</div>
{/* Logo */}
<Logo />
{/* 图片附件标签 */}
{imagePath && (
<div className="screenshot-attachment">
<ImageIcon size={12} />
<span>Image</span>
<button
className="screenshot-attachment-remove"
onClick={handleRemoveImage}
title="移除图片"
>
<X size={10} />
</button>
</div>
)}
{/* 输入框 */}
<input
ref={inputRef}
type="text"
className="screenshot-input"
placeholder="Ask anything..."
value={inputValue}
onChange={(e) => setInputValue(e.target.value)}
onKeyDown={handleInputKeyDown}
disabled={isLoading}
/>
{/* 右侧按钮组 */}
<div className="screenshot-actions">
{/* 关闭按钮 */}
<button
className="screenshot-close-btn"
onClick={handleClose}
title="关闭 (ESC)"
>
<X size={14} />
</button>
{/* 发送按钮 */}
<button
className={`screenshot-send-btn ${inputValue.trim() ? "active" : ""}`}
onClick={handleSend}
disabled={!inputValue.trim() || isLoading}
title="发送 (Enter)"
>
<ArrowUp size={16} />
</button>
</div>
</div>
</div>
);
}
export default ScreenshotChatPage;
sync-resources.sh
Executable
+15
View File
@@ -0,0 +1,15 @@
#!/bin/bash
# 同步资源文件到 target/debug 目录
echo "正在同步资源文件..."
# 创建目标目录
mkdir -p src-tauri/target/debug/resources/models/providers
mkdir -p src-tauri/target/debug/resources/models/aliases
# 复制模型文件
cp -v src-tauri/resources/models/index.json src-tauri/target/debug/resources/models/
cp -v src-tauri/resources/models/providers/*.json src-tauri/target/debug/resources/models/providers/
cp -v src-tauri/resources/models/aliases/*.json src-tauri/target/debug/resources/models/aliases/ 2>/dev/null || true
echo "✅ 资源文件同步完成！"