opensource/proxycast
3
0
Fork 0
mirror of https://github.com/aiclientproxy/proxycast.git synced 2026-09-21 12:44:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

release: v1.25.0

Browse Source
This commit is contained in:
coso
2026-04-30 23:27:29 +08:00
parent ad64ac7cce
commit 4652dd633d
207 changed files with 34211 additions and 1412 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+4
View File
@@ -51,6 +51,10 @@ docs/roadmap/*
!docs/roadmap/harness-engine/diagrams.md
!docs/roadmap/warp/
!docs/roadmap/warp/*.md
!docs/roadmap/agentui/
!docs/roadmap/agentui/*.md
!docs/roadmap/voice/
!docs/roadmap/voice/*.md
docs/gongzonghao/
docs/bussniss/
docs/oem/
AGENTS.md
+5 -4
View File
@@ -31,10 +31,11 @@
3. **协议改动必须同步四侧** - `safeInvoke(...)` / `invoke(...)`、`tauri::generate_handler!`、`agentCommandCatalog`、`mockPriorityCommands` / `defaultMocks` 必须保持一致，并执行 `npm run test:contracts`
4. **Lime 是 GUI 桌面产品** - 不能只以 `lint`、`typecheck`、单测通过作为“可交付”判断
5. **高风险 GUI 改动必须做最小冒烟** - 涉及 GUI 壳、DevBridge、Workspace、主路径时执行 `npm run verify:gui-smoke`
6. **用户可见 UI 改动必须补稳定回归** - 优先补现有 `*.test.tsx` 或 snapshot 断言
7. **配置与依赖改动要成组更新** - schema、校验器、消费者、文档、锁文件保持同步
8. **Rust 变更先小测后全量** - 先跑受影响 crate / 模块 / 定向测试；新增模块尽量控制在 `500 LoC` 内，文件接近 `800 LoC` 时优先拆新模块
9. **Harness Engine 只认单一事实源** - handoff / evidence / replay / analysis / review / GUI 统一消费 `agent_runtime_export_evidence_pack`；`requestTelemetry` 需要按 `session/thread/turn` 真实关联导出，无匹配请求时输出空摘要，不再保留伪 `unlinked`
6. **Playwright 续测优先稳定桌面 Chrome 会话** - 真实交互验证优先复用已有 Lime 页签；需要新启浏览器时走持久化 Chrome 上下文，避免本地桌面出现自动化横幅或 `--no-sandbox` 安全横幅，细则见 `docs/aiprompts/playwright-e2e.md`
7. **用户可见 UI 改动必须补稳定回归** - 优先补现有 `*.test.tsx` 或 snapshot 断言
8. **配置与依赖改动要成组更新** - schema、校验器、消费者、文档、锁文件保持同步
9. **Rust 变更先小测后全量** - 先跑受影响 crate / 模块 / 定向测试；新增模块尽量控制在 `500 LoC` 内，文件接近 `800 LoC` 时优先拆新模块
10. **Harness Engine 只认单一事实源** - handoff / evidence / replay / analysis / review / GUI 统一消费 `agent_runtime_export_evidence_pack`；`requestTelemetry` 需要按 `session/thread/turn` 真实关联导出，无匹配请求时输出空摘要，不再保留伪 `unlinked`
## 执行与路线图
RELEASE_NOTES.md
+81 -58
View File
@@ -1,88 +1,111 @@
## Lime v1.24.0
## Lime v1.25.0
发布日期：`2026-04-30`
### 发布概览
- 本次发布目标 tag 为 `v1.24.0`。
- 本次发布聚焦多模态运行合同底座、Browser / PDF / Voice / Research / Text Transform 合同闭环、任务中心与旧会话打开体验、工作台首页技能入口，以及版本发布配置同步。
- 本轮待递交内容覆盖 Rust 后端、Tauri 配置、前端 Agent Workspace / Home / Settings / Provider / DevBridge、治理合同 registry、测试覆盖、版本锁文件、图片素材与执行计划文档。
- 本次发布目标 tag 为 `v1.25.0`，重点把 Lime 的语音输入、音频转写、AgentUI 旧会话体验和多模态运行合同推进到同一条 current 主链。
- 版本文件、Tauri 配置、Cargo / npm lockfile、CLI wrapper、浏览器 mock 与 release updater 测试样例已同步到 `1.25.0`。
- 该版本继续坚持“一个事实源”：语音、音频、转写、任务轻卡、Evidence Pack、Replay 与 GUI 恢复层都消费统一的 runtime contract / task artifact / media task index，而不是新增平行协议。
### 重点更新
### 用户可见更新
#### 1. 版本号同步到 v1.24.0
#### 1. 离线语音模型与 Fn 听写
- 应用版本已同步为 `1.24.0`：
- `package.json`
- `package-lock.json`
- `src-tauri/Cargo.toml`
- `src-tauri/Cargo.lock`
- `src-tauri/tauri.conf.json`
- `src-tauri/tauri.conf.headless.json`
- `packages/lime-cli-npm/package.json` 与 `packages/lime-cli-npm/README.md` 已同步到 `1.24.0`，保持 CLI wrapper 与桌面 release 版本一致。
- 浏览器模式默认 mock 的 update current version 已同步为 `1.24.0`。
- GitHub release asset staging 测试中的当前发布样例已同步到 `v1.24.0`。
- 设置页新增 `SenseVoice Small` 本地语音模型主链：可查看安装状态、下载模型、删除本地模型、设为默认 ASR、执行 WAV 文件测试转写。
- 本地模型按需下载到 Lime 数据目录，不进入桌面安装包；缺失模型文件时会给出明确引导，不会静默拉取大模型。
- `voice-core` 接入 `sherpa-onnx` offline recognizer，支持通过 `model.int8.onnx` 与 `tokens.txt` 运行 SenseVoice 本地转写。
- `voice_asr_service` 新增 `SenseVoiceLocal` 分支，录音采样率非 `16kHz` 时会转换为 `16kHz` mono float samples 后再识别。
- macOS 新增 Fn 按住录音第一刀：按住 Fn 打开语音输入，松开 Fn 停止并转写；Fn 不可用或权限不足时保留普通全局快捷键 fallback。
- 语音设置、快捷键设置、onboarding 语音测试与 ASR Provider UI 已补齐 SenseVoice / Fn 状态展示和回归覆盖。
#### 2. 多模态运行合同底座
#### 2. Audio Transcription 转写任务
- 新增并扩展 `ModalityRuntimeContract` 前端 registry 与 Rust 侧合同常量，统一 `image_generation`、`browser_control`、`pdf_extract`、`voice_generation`、`web_research`、`text_transform` 六类底层运行合同。
- `scripts/check-modality-runtime-contracts.mjs` 与 `npm run governance:modality-contracts` 继续作为合同 registry 的机器校验入口，保护 contract key、entry binding、executor binding 与 metadata path 不漂移。
- 图片生成任务、媒体任务 API、DevBridge mock 与治理目录册同步合同字段，避免入口层直接拥有底层 task / model / artifact / evidence 事实源。
- `docs/roadmap/warp/contract-schema.md` 与 `docs/exec-plans/multimodal-runtime-contract-plan.md` 更新合同 schema、执行阶段与 current / compat / deprecated / dead 分类。
- `@转写 / @transcribe / @Audio Extractor` 归并到 `audio_transcription` 运行合同，入口只作为 binding，不再直接伪造 transcript 事实源。
- 新增 `transcription_generate` task artifact 主链：任务会写入 contract snapshot、transcript 状态、provider 结果、错误码与统一媒体任务索引。
- 最小 `lime-transcription-worker` 接入 OpenAI-compatible `/audio/transcriptions` provider：支持本地 `source_path` 与远程 `source_url`，成功后写入 `.lime/runtime/transcripts/*`。
- 转写失败会保留结构化错误，例如 provider 未配置、source 不可读、provider 请求失败、空 transcript、输出写入失败等；不会退回前端直连 ASR 或普通文件读取。
- 前端转写轻卡和 workspace viewer 可从媒体任务索引恢复完成态或失败态，完成态支持 transcript 文本、语言、输出格式、路径和时间轴信息展示。
- Transcript parser 支持从 verbose JSON、VTT / SRT 内容中恢复分段、时间戳和说话人信息。
- 转写校对产物新增 `transcriptCorrection*` metadata：保存修订版本时保留原始 transcript 不可变，并记录修订文本、分段数量、说话人数量和来源路径。
#### 3. Browser / PDF / Voice / Research / Text Transform 合同闭环
#### 3. 音频生成与多模态任务恢复
- `@浏览器` / Browser Assist 发送链路注入 `browser_control` 合同快照，Rust 侧保留 Browser Assist runtime hint，并把 browser tool timeline metadata 纳入 evidence / replay。
- `@读PDF` 入口收敛到 `pdf_extract` 合同，前端 metadata、Rust prompt 准备、Skill(pdf_read) timeline、evidence pack 与 replay 检查保持同一份合同事实源。
- `@配音` 服务型入口收敛到 `voice_generation` 合同，ServiceSkill / voice runtime 的 request metadata、timeline args 与 replay grader 统一识别音频运行证据。
- `@搜索`、`@深搜`、`@站点搜索`、`@研报`、`@竞品` 统一归入 `web_research` 合同，并补齐 evidence `snapshotIndex`，让联网研究可以按 contract/source/routing/tool trace 检索。
- `@读文件`、`@总结`、`@翻译`、`@分析`、`@发布合规`、`@Logo拆解` 统一归入 `text_transform` 合同，禁止回退为前端直出、ToolSearch / WebSearch 或普通聊天摘要。
- `audio_generate` 与 `transcription_generate` 继续统一到媒体任务索引，任务轻卡、运行时文档、DevBridge mock 和恢复层消费同一份 artifact。
- 音频预览 runtime 会优先读取统一索引中的 audio output 状态，不再依赖隐藏 task JSON；provider 失败时不会保留旧音频路径或伪造可播放结果。
- `TaskMessagePreview`、`taskPreviewFromToolResult`、媒体任务 API 与 mock 输出补齐 audio / transcript 状态字段，减少完成态、失败态、缺 trace 之间的误判。
#### 4. Evidence Pack / Replay / Task Runtime
#### 4. AgentUI 首页、任务中心与旧会话性能
- `runtime_evidence_pack_service` 与 `runtime_replay_case_service` 扩展多模态合同快照、索引、expected checks 与 grader 规则，回放样本能判断是否真实命中对应 executor trace。
- 媒体 runtime 与 `media_task_cmd` 补齐合同字段归一化、能力缺口提示、artifact payload 与任务预览字段。
- 前端 `agentRuntime` media/session 类型、`mediaTasks` API、ImageTask viewer 与工作台预览 runtime 同步消费合同字段，减少 viewer 自行猜测 artifact 类型。
- `modalityRuntimeContracts` 的前端/Rust 双侧测试覆盖合同解析、fallback 与 mock 分发，保持浏览器模式不是假绿。
- Agent Chat 首页补齐 Home Start Surface、引导卡、技能卡片、更多技能抽屉、Starter Chips 与稳定回归。
- 新建任务、打开已有会话、任务中心 tab、侧栏会话列表之间的事件链收敛到 `taskCenterDraftTaskEvents`，减少旧导航和本地 tab 状态双写。
- 旧会话打开继续降载：最近会话首屏请求从大窗口降到 `11`，任务中心 topics 初始请求降到 `21`，归档列表也改为可见数量 `+1` 的哨兵分页。
- 侧栏 hover / focus 旧会话预取延迟到约 `900ms`，点击时取消未触发预取，避免鼠标扫过列表时抢占正式切换链路。
- 侧栏打开旧会话后，路由追平会在短窗口内去重，避免同一用户意图触发两次 `switchTopic` / `agent_runtime_get_session`。
- 旧会话恢复首帧优先正文和输入区，运行轨迹、thread items、queued turns、pending actions 等投影延迟到 idle；正在发送、聚焦 timeline 或存在 A2UI 待处理时不会延迟实时反馈。
- `MessageList` 继续强化旧会话轻量渲染：历史窗口只先渲染最近消息，timeline 延迟构建，长历史 Markdown 使用轻量预览，Provider / Prompt Cache 配置扫描推迟到首帧后。
- 新增 `window.__LIME_AGENTUI_PERF__` 性能采集器，可按 session 汇总 `clickToMessageListPaintMs`、`runtimeGetSessionDurationMs`、隐藏历史数量、最终渲染消息数量和可用堆内存指标。
#### 5. 工作台首页、任务中心与旧会话体验
#### 5. Browser Assist 与工作区展示
- Agent Chat 新增 Home Start Surface、技能卡片画廊、更多技能抽屉与场景技能管理弹窗，并提交对应 home cover 素材。
- EmptyState / Workspace / AppSidebar 接入新的首页入口、补充快捷入口与任务中心事件桥，一级“新建任务”与会话侧栏切换保持可取消草稿语义。
- 旧会话打开性能继续收口：首屏优先最近消息、timeline 延后物化、历史消息与流式 delta overlap 合并，降低旧会话切换卡顿与布局跳动。
- MessageList / Team Workspace / Workspace Send Actions 增补回归，覆盖新建草稿、旧会话切换、图片任务预览和合同 metadata 发送。
- Browser Assist renderer 增强运行结果展示，支持更清晰地呈现浏览器会话、页面、操作结果与可恢复状态。
- Workspace 的音频任务预览、转写任务预览、图片任务预览和 general workbench 恢复逻辑进一步对齐 task artifact / media index 事实源。
- Inputbar 与消息渲染继续收紧多模态、运行态、队列态和任务文件展示，减少过程信息污染最终正文。
#### 6. Provider / 设置页 / 云端能力边界
#### 6. Provider、模型能力与设置体验
- Provider 与 API Key 设置页进一步收口模型能力、Prompt Cache、云端套餐 / 权益 / API Key 展示口径，避免本地设置页重新承载用户中心商业工作台。
- OEM cloud access、LimeHub provider sync、model registry service 与 provider list 工具函数补齐测试，保护登录态、权益摘要、模型支持和 provider capability 展示。
- `withI18nPatch`、Settings Provider 页面与相关 hooks 做格式化和回归更新，保持当前设置入口与 mock / Rust 模型事实源一致。
- API Key Provider 设置页补齐 provider 配置、模型能力和 Prompt Cache 相关展示与回归。
- OpenAI-compatible provider 和 credential bridge 增强，为音频转写、模型能力推断和 provider 解析提供更稳定的后端事实源。
- LimeHub local dev models 与本地 provider 模型列表同步更新，减少设置页和运行时模型认知漂移。
- 外观配色扩展：在原有配色基础上新增 `霓虹`、`青柠`、`暮色`、`极简`、`鲜活`、`文学`、`奢华` 等主题，并调整现有配色名称与描述。
### 待递交范围确认
### 开发者与治理更新
- 版本与发布：版本文件、lockfile、Tauri 配置、CLI wrapper、release updater 测试样例与 release notes。
- Rust 主链：多模态合同、媒体任务、Browser Assist、PDF / Research / Summary / Translation / Analysis / ServiceSkill launch、evidence pack、replay case、model registry。
- 前端主链：Agent Chat Workspace、Home Start Surface、MessageList、ImageTask Viewer、任务中心事件桥、Settings Provider、API Key Provider、DevBridge、tauri mock。
- 治理与文档：`modalityRuntimeContracts` registry / 校验脚本、Warp contract schema、执行计划与旧会话性能计划。
- 素材与测试：home cover 素材、新增/更新的 Vitest、Rust 回归与治理合同测试。
#### 1. 运行合同与治理校验
- `modalityRuntimeContracts` 新增 / 更新 `audio_transcription`，并同步 Rust 合同常量、前端 registry、mock 和校验脚本。
- 新增 `modalityArtifactGraph.json`，把 entry binding、executor binding、artifact、viewer 和 evidence / replay 关系显式化。
- `scripts/check-modality-runtime-contracts.mjs` 扩展校验范围，覆盖 capability、model role、artifact kind、artifact graph 与 current contract 同步关系。
- `npm run test:contracts` 现在覆盖 agent runtime client 生成检查、命令契约、harness 契约、modality contracts 与 cleanup report contract。
#### 2. Evidence Pack 与 Replay
- Evidence Pack 可导出 audio transcription contract snapshot、transcript 索引、audio output 索引、provider 失败状态与 runtime verification 信息。
- Replay case 可携带 `audio_transcription`、`voice_generation`、`pdf_extract`、`browser_control`、`web_research`、`text_transform` 等合同进入 grader checks。
- Replay 对音频和转写失败做更细分类：可区分 provider failure、缺少输出、缺少 trace、合同不匹配等情况。
#### 3. 文档与路线图
- 新增 AgentUI 路线图文档，覆盖目标架构、代码地图、事件流、时序图、后端协作和实施路线。
- 新增离线语音模型路线图，明确 SenseVoice Small、Fn 听写、模型下载、测试转写和后续 VAD / 历史规划。
- Warp roadmap 补齐 artifact graph、contract schema、acceptance 与 evolution guide，用于多模态 / runtime contract 后续演进。
- Playwright E2E 指南补充真实 GUI 续测约束，继续优先复用稳定桌面 Chrome / Lime 页签。
- `AGENTS.md` 与执行计划文档同步更新，保留本轮关键决策和验证记录。
### 已知说明
- SenseVoice Small 当前按需从上游 release 下载；内置清单会记录下载摘要，但上游未在本地清单内提供可信 sha256 时不会声明强校验完成。
- Fn 按住录音当前只作为 macOS 第一刀能力；第三方键盘或缺少系统权限时，用户应继续使用普通语音快捷键。
- `audio_transcription` 执行器当前只接入 OpenAI-compatible transcription adapter；其他 provider runtime type 仍需要后续补 client。
- 真实联网 web search 测试默认 gated，需要设置 `LIME_REAL_API_TEST=1` 才会执行。
### 校验状态
- 已执行：
- 已通过：
- `cargo fmt --manifest-path "src-tauri/Cargo.toml" --all`
- `npm run format`
- `npm run verify:app-version`
- `cargo test --manifest-path "src-tauri/Cargo.toml" --target-dir "src-tauri/target/codex-release-v124"`
- `cargo clippy --manifest-path "src-tauri/Cargo.toml" --target-dir "src-tauri/target/codex-release-v124" --all-targets --all-features`
- `npm run verify:app-version`（版本一致性检查通过：`1.25.0`）
- `npm run lint`
- `npm test`
- `npm run typecheck`
- `npm test`（`44` 个 Vitest 批次通过）
- `npm run test:contracts`
- `npm run verify:gui-smoke`
- `git diff --check`
- 结果：上述校验均已通过；`cargo test` 结果为 `1103 passed; 0 failed; 0 ignored`，另有 2 个真实联网测试按预期 ignored。
- 备注：`cargo clippy` 仍保留既有 warning（`manual_repeat_n`、`too_many_arguments`、`needless_lifetimes`），本轮未扩大 warning 面。
- GUI 主路径：`npm run verify:gui-smoke` 已通过，覆盖 DevBridge、默认 workspace、browser runtime、site adapter catalog、服务技能入口与 runtime tool surface 页面烟测。
- `npm run lint:rust`（命令通过；仓库仍保留若干既有 Clippy warning，未配置为阻断）
- `cargo test --manifest-path "src-tauri/Cargo.toml" -p voice-core`
- `cargo test --manifest-path "src-tauri/Cargo.toml"`（主库 `1120` 项通过，DeepSeek runtime `2` 项通过，真实联网 web search `2` 项按环境变量门禁 ignored）
- `npm run verify:gui-smoke -- --reuse-running --timeout-ms 600000`
- 说明：完整 `cargo test` 首次因本机磁盘空间不足中断；清理已确认无进程占用的临时 GUI smoke target 后重跑通过。
- GUI 主路径：已复用运行中的 headless Tauri 完成最小冒烟，覆盖 DevBridge、默认 workspace、browser runtime、site adapter、Agent service skill entry 与 runtime tool surface。
---
**完整变更**: `v1.23.0` -> `v1.24.0`
**完整变更**: `v1.24.0` -> `v1.25.0`
docs/aiprompts/playwright-e2e.md
+2
View File
@@ -24,6 +24,8 @@
- 优先使用 **Playwright MCP** 做交互验证，不优先编写新的本地 Playwright 测试文件
- 浏览器模式默认首页从 `http://127.0.0.1:1420/` 进入
- 需要稳定的桌面 Chrome 观感时，优先复用已有 Lime 页签；必须新启 headed 浏览器时，使用固定 profile 的 Chrome 持久化上下文，例如 `channel: 'chrome'` + `launchPersistentContext(userDataDir, { headless: false, viewport: null })`
- 本地桌面续测不要传 `--no-sandbox` / `--disable-setuid-sandbox`；如需去掉 Chrome 顶部“自动测试软件控制”提示，只能在受控 launcher 中忽略 `--enable-automation`。CI、容器或 Linux sandbox 受限环境例外，但要在结论中说明原因
- 如果 Playwright 工具当前还在 deferred surface，优先用 `ToolSearch` 的精确选择名，例如 `select:mcp__playwright__browser_click`；不要把 `playwright_browser_click`、`browser click` 之类同义词反复丢给 `ToolSearch`
- 能走真实后端就走真实后端；浏览器模式暂不支持或尚未桥接的能力，允许走 mock
- `verify:gui-smoke` 内部的 browser runtime 校验默认走无界面浏览器会话；它只证明主链可启动，不替代后续真实页面交互验证
docs/exec-plans/agentui-implementation-progress.md
+730
View File
@@ -0,0 +1,730 @@
# AgentUI 实施进度
> 状态：进行中
> 更新时间：2026-04-30
> 路线图：`docs/roadmap/agentui/lime-agentui-implementation-roadmap.md`
## 主目标
逐步实现 Lime AgentUI 下一阶段主线，优先解决旧会话恢复慢、首字慢、tab 卡顿、流式重复吐字和过程信息噪声。
## 当前阶段
P1：Tab 与 Task Capsule，先落旧会话切换期间的轻量投影。
## 本轮执行准则更新：旧 UI 顺路清理
主线仍以旧会话性能数值分析和首字/流式体验优化为优先级；遇到现有 UI 老旧、不适合继续承载 AgentUI 主链时，可以顺路清理，但清理必须直接服务当前交付，不能偏航成纯治理或视觉翻新。
### 准入条件
仅当旧 UI 满足以下任一条件时，本轮允许顺路清理：
- 直接造成旧会话打开、切换、新建对话或流式输出卡顿，例如重复渲染非活跃会话、旧面板抢占 hydrate、旧入口触发额外查询。
- 直接造成用户路径错误，例如新建对话被旧界面跳走、无法同时打开多个历史对话、输入区/对话区位置与 AgentUI 规划冲突。
- 直接造成重复事实源，例如同一状态同时由旧 tab、旧任务面板和新 capsule 独立维护，导致状态不同步或额外 render。
- 直接违反当前 Lime UI 规范并影响可用性，例如半透明主表面、多层套卡、重复标题、伪交互、中文排版被压缩。
### 分类与动作
- `current`：继续沿 AgentUI 当前主链演进；性能采样、消息窗口、tab/capsule、新建对话入口都应向这里收敛。
- `compat`：只允许委托和适配，不新增状态、查询、渲染分支；如果旧 UI 仍被调用，必须写清退出条件。
- `deprecated`：只允许迁移和下线；遇到影响主线的旧 UI，优先从主入口摘除，再登记后续删除。
- `dead`：确认无入口或与 current 规划冲突时直接删除或补治理守卫，防止后续重新接回主链。
### 清理边界
- 每一刀最多顺路清理一个直接阻塞主线的旧 UI surface，其余旧面先登记，不连续深挖。
- 不新增平行的新旧两套组件；能收敛到现有 AgentUI current 组件时，不再补新的 compat 包装层。
- 清理后仍需保持桌面 GUI 气质：实体主表面、清晰边界、中文优先、按钮层级明确，避免半透明主体和过度嵌套套卡。
- 如果清理会触及 Tauri command、Bridge、mock 或 session 数据事实源，必须另起命令边界检查，不把 UI 清理伪装成纯样式改动。
### 验收口径
- 性能侧：用 `window.__LIME_AGENTUI_PERF__.summary()` 对比清理前后，至少看 `clickToMessageListPaintMs`、`runtimeGetSessionDurationMs`、`finalRenderedMessagesCount`、`hiddenHistoryCount`。
- 交互侧：覆盖新建对话、打开旧会话 A、打开旧会话 B、旧会话间切换、发送短句看首字和流式输出。
- UI 侧：补稳定回归或 snapshot，必要时复用现有 Lime 页签做 Playwright E2E；不使用会新开 isolated profile 或 `--no-sandbox` 临时浏览器的验证方式。
- 治理侧：如果删除或下线旧入口，补 `current / compat / deprecated / dead` 分类说明；必要时运行 `npm run governance:legacy-report` 或 `npm run test:contracts`。
## 进度日志
### 2026-04-30：P0 第一刀，流式主链观测与渲染降频
已完成：
- 在 stream request state 中补齐首字慢分段字段：
- listener bound
- submit dispatched / accepted / failed
- first event
- first runtime status
- first text delta
- first text paint
- text render flush / backlog
- 在 `agentStreamTurnEventBinding.ts` 记录 listener bound 和 first event。
- 在 `agentStreamSubmitExecution.ts` 记录 submit dispatched / accepted / failed。
- 在 `agentStreamRuntimeHandler.ts` 记录 first runtime status、first text delta、first text paint、text render flush。
- 保留并补齐已有 text_delta 低频刷新策略，避免每个字符都刷新消息树。
- 保留并补齐已有 thinking 关闭策略，`thinking_delta` 不再污染最终正文。
- 保留并补齐已有 `final_done` reconcile 防线，避免最终文本整段重复追加。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/hooks/agentStreamRuntimeHandler.test.ts" "src/components/agent/chat/hooks/agentStreamTurnEventBinding.test.ts"
npm exec -- vitest run "src/components/agent/chat/hooks/agentStreamRuntimeHandler.test.ts" "src/components/agent/chat/hooks/agentStreamTurnEventBinding.test.ts" "src/components/agent/chat/components/MessageList.test.tsx" "src/components/agent/chat/components/StreamingRenderer.test.tsx"
npm run typecheck
npm run verify:gui-smoke
```
结果：
- 流式定向测试：通过，`17` 个测试通过。
- AgentUI P0 定向测试：通过，`131` 个测试通过。
- TypeScript：通过。
- GUI smoke：通过。
下一步：
1. 跑 touched files 的 lint/type 校验。
2. 继续 P0 旧会话渐进恢复：确认 active tab 与非 active tab 的 hydrate 边界，减少旧会话打开时 MessageList/timeline 同步负担。
3. 再进入 P1 tab/capsule：避免多历史会话同时全量渲染。
### 2026-04-30：P0 第二刀复核，旧会话渐进恢复现状
已确认当前工作区已有以下旧会话恢复优化：
- `useAgentSession.ts`：
- `getSession` 恢复路径统一带 `historyLimit: 40`。
- recent session prefetch 使用 `SESSION_DETAIL_PREFETCH_HISTORY_LIMIT = 40`。
- cached snapshot fresh 时延迟 detail hydrate。
- detail hydrate 进入 `startTransition`，降低切换时主线程抢占。
- “加载更多历史”改为分页：`SESSION_HISTORY_LOAD_PAGE_SIZE = 50`，使用 `historyOffset` / `historyBeforeMessageId`。
- `MessageList.tsx`：
- 旧会话只先渲染最近消息窗口。
- 旧会话隐藏历史不自动逐批补齐，必须用户点击展开。
- 历史 timeline 延迟到 idle，完成历史 timeline 可先折叠为轻量摘要。
- 超长历史助手消息和较长历史助手消息先展示纯文本预览。
- 历史 Markdown 使用 light render mode，避免首帧挂载重 Markdown。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/components/MessageList.test.tsx" "src/components/agent/chat/components/StreamingRenderer.test.tsx"
npm run typecheck
npm run verify:gui-smoke
```
结果：通过，`MessageList` / `StreamingRenderer` 定向测试共 `114` 个通过，TypeScript 检查通过，GUI smoke 通过。
下一步：
1. 进入 P1 tab/capsule 前，先用 Playwright 复测真实多历史会话打开路径，采集 `runtimeGetSession.*`、`switchTopic.*`、`AgentStream.*` 日志。
2. 若 E2E 仍出现切换卡顿，下一刀优先做非活跃 tab snapshot/freeze，而不是继续压 MessageList 单点。
### 2026-04-30：P1 第一刀，旧会话切换期间延迟运行轨迹投影
已完成：
- 在 `useWorkspaceConversationSceneRuntime.tsx` 增加 session runtime projection defer：
- 旧会话恢复首帧继续立即透传 `messages`，保证正文和输入区先可交互。
- `turns`、`threadItems`、`threadRead`、`pendingActions`、`queuedTurns`、`childSubagentSessions` 延迟到 idle 后再挂载，减少点击历史会话时的同步投影和 timeline 构建压力。
- 延迟状态按 `sessionId + 首尾 message + 尾部 turn/item` 绑定，避免从一个历史会话切到另一个同长度历史会话时复用上一会话的“已投影”状态。
- 正在发送、聚焦 timeline、存在 pending A2UI 表单时不延迟，避免影响运行中反馈和用户待处理动作。
- 在 `useWorkspaceConversationSceneRuntime.test.ts` 补回归：
- 恢复旧会话首帧应先透传消息，并延迟运行轨迹投影。
- 切换到另一条同长度旧会话时应重新延迟运行轨迹投影。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/workspace/useWorkspaceConversationSceneRuntime.test.ts" "src/components/agent/chat/workspace/WorkspaceConversationScene.test.tsx" "src/components/agent/chat/components/MessageList.test.tsx"
npm run typecheck
```
结果：
- Workspace / MessageList 定向回归：通过，`96` 个测试通过。
- TypeScript：通过。
未完成验证：
- `npm run verify:gui-smoke` 本轮未完成。原因是本地 `DevBridge` 3030 未就绪，smoke 复用/拉起 headless 链路时触发新的 Rust 编译链，持续数分钟占用 CPU；继续等待会污染“旧会话打开是否卡顿”的真实判断。已清理本轮 `verify-gui-smoke` 派生的临时编译链，未清理用户已有的长期 `tauri:dev:headless` / 其他 Rust 校验进程。
- 真实 Chrome 页签 E2E 续测暂未完成。当前前端 `http://127.0.0.1:1420/` 可访问，但 `http://127.0.0.1:3030/health` 未监听；需先恢复 DevBridge 后再复测多历史会话切换。
下一步：
1. 恢复 DevBridge 3030 后，用现有 Chrome 页签复测：新建对话、打开两个历史会话、来回切换、观察控制台 error 和 `switchTopic.*` 日志。
2. 若旧会话仍有体感卡顿，继续 P1：把 tab/shell 层的 running/queued/needs_input 胶囊化，并进一步降低非当前过程面板的更新频率。
### 2026-04-30：P1 第二刀，覆盖 history window hydrate 后的重投影卡顿
已完成：
- 扩展 `useWorkspaceConversationSceneRuntime.tsx` 的旧会话运行轨迹延迟投影触发条件：
- 不再只依赖 `isAutoRestoringSession`。
- 当 `sessionHistoryWindow.totalMessages > sessionHistoryWindow.loadedMessages` 时，也按旧会话窗口处理，覆盖 cached snapshot 已显示、detail hydrate 后恢复大量 `turns/threadItems` 的场景。
- 同一个 session 尾部追加新消息时不重新延迟投影，避免首字和流式输出被 700ms 延迟误伤。
- 继续保持发送中、timeline 聚焦、存在 pending A2UI 表单时不延迟，保护实时反馈和用户待处理动作。
- 延迟投影状态写入改为“值未变化则返回当前 state”，避免 ready/pending 状态重复写入造成额外 render。
- 补齐 `useWorkspaceConversationSceneRuntime.test.ts` 回归：
- `isAutoRestoringSession=false` 但存在截断历史窗口时，首帧仍先显示消息、延迟运行轨迹。
- 发送中会话不延迟运行轨迹。
- 聚焦 timeline 或存在 A2UI 表单时不延迟运行轨迹。
- 顺手补齐语音快捷键相关测试 fixture 的 `fn_*` 字段，修复当前工作区 `typecheck` 阻塞；该修复只闭合测试 mock 类型，不扩展语音功能面。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/workspace/useWorkspaceConversationSceneRuntime.test.ts"
npm exec -- vitest run "src/components/agent/chat/workspace/useWorkspaceConversationSceneRuntime.test.ts" "src/components/agent/chat/workspace/WorkspaceConversationScene.test.tsx" "src/components/agent/chat/components/MessageList.test.tsx"
npm exec -- vitest run "src/components/onboarding/steps/VoiceShortcutTestStep.test.tsx" "src/components/settings-v2/general/hotkeys/hotkeyCatalog.test.ts"
npm run typecheck
curl -fsS "http://127.0.0.1:3030/health"
```
结果：
- Workspace 单测：通过，`13` 个测试通过。
- Workspace / MessageList 组合回归：通过，`99` 个测试通过。
- 语音 fixture 定向回归：通过，`4` 个测试通过。
- TypeScript：通过。
- DevBridge 健康检查：失败，`127.0.0.1:3030` 未监听；当前只确认 `127.0.0.1:1420` 前端 dev server 在监听。第二轮 render 收紧后复查结果一致。
未完成验证：
- 真实 Chrome 页签 E2E 暂未执行。原因是 DevBridge 3030 未就绪，强行启动 `verify:gui-smoke` 会触发新的 Rust 编译链并污染旧会话卡顿判断。
下一步：
1. 恢复 DevBridge 3030 后，复用现有 Chrome/Lime 页签做多历史会话切换 E2E。
2. 若体感卡顿仍明显，下一刀进入 tab/shell 层：冻结非活跃 tab 的重运行时投影，只保留标题、状态、未读和最后预览。
### 2026-04-30：P1 第三刀，收紧 tab/shell 同步小开销与验证阻塞
已完成：
- `AgentChatWorkspace.tsx` 为 `topics` 建立单次 `topicById` Map：
- 初始会话切换策略、任务中心打开旧会话、detached 判断和顶部 tab item 生成复用 Map 查询。
- 减少旧会话切换路径里多处 `topics.find/some` 扫描；当前非活跃 tab 本身只渲染标题/状态/未读，不渲染正文或 timeline。
- `useWorkspaceConversationSceneRuntime.tsx` 收紧延迟投影状态更新：
- 空投影数组改为模块级稳定常量，避免下游 `useMemo` 因每次新建 `[]` 失效。
- 延迟投影 ready/pending 写入保持“值未变则返回当前 state”，避免无意义二次 render。
- 修复当前工作区两个验证阻塞：
- 语音快捷键测试 fixture 补齐 `fn_*` 字段。
- 媒体任务测试 fixture 与浏览器 mock 输出补齐 transcript 汇总字段，保持 `MediaTaskModalityRuntimeContractIndex` 契约一致。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/index.test.tsx" -t "任务中心初始会话标签"
npm exec -- vitest run "src/components/agent/chat/workspace/useWorkspaceConversationSceneRuntime.test.ts" "src/components/agent/chat/workspace/WorkspaceConversationScene.test.tsx" "src/components/agent/chat/components/MessageList.test.tsx"
npm exec -- vitest run "src/components/agent/chat/workspace/useWorkspaceAudioTaskPreviewRuntime.test.tsx" "src/components/agent/chat/workspace/useWorkspaceImageTaskPreviewRuntime.test.tsx" "src/lib/tauri-mock/core.test.ts"
npx eslint "src/components/agent/chat/AgentChatWorkspace.tsx" "src/components/agent/chat/workspace/useWorkspaceConversationSceneRuntime.tsx" "src/components/agent/chat/workspace/useWorkspaceConversationSceneRuntime.test.ts" "src/components/agent/chat/workspace/useWorkspaceAudioTaskPreviewRuntime.test.tsx" "src/components/agent/chat/workspace/useWorkspaceImageTaskPreviewRuntime.test.tsx" "src/lib/tauri-mock/core.ts" "src/components/onboarding/steps/VoiceShortcutTestStep.test.tsx" "src/components/settings-v2/general/hotkeys/hotkeyCatalog.test.ts" --max-warnings 0
npm run typecheck
npm run test:contracts
curl -fsS "http://127.0.0.1:3030/health"
```
结果：
- 任务中心定向回归：通过，`11` 个测试通过。
- Workspace / MessageList 组合回归：通过，`99` 个测试通过。
- 媒体任务 / mock 回归：通过，`54` 个测试通过。
- ESLint touched files：通过。
- TypeScript：通过。
- Contract：通过。
- DevBridge 健康检查：失败，`127.0.0.1:3030` 未监听；`127.0.0.1:1420` 前端 dev server 仍在监听。
未完成验证：
- 真实 Chrome 页签 E2E 仍未执行。原因同上一刀：DevBridge 3030 未就绪，当前不应启动新的 Playwright profile 或强行触发重编译链。
下一步：
1. 待 DevBridge 恢复后，复用现有 Chrome/Lime 页签采样：打开旧会话、打开第二个旧会话、来回切换、发送 DeepSeek 短句，看首字前占位、真实 first text delta 和 long task。
2. 若旧会话仍慢，下一刀不再继续做 tab 微调，优先做 MessageList 动态高度虚拟化或 `agent_runtime_get_session` 分块返回方案评估。
### 2026-04-30：P1 第四刀，旧会话首帧推迟 Prompt Cache 配置扫描
已完成：
- `MessageList.tsx` 推迟旧会话恢复首帧的 Provider 配置自动加载：
- 旧会话恢复或历史窗口 hydrate 阶段，`useConfiguredProviders` 先以 `autoLoad: false` 运行。
- 首帧消息可见后再在 idle 阶段允许 Provider 配置加载，避免 Prompt Cache 提示为了历史消息扫描抢占旧会话打开主链。
- 保留实时发送、当前会话和普通新会话的 Provider 能力提示，不影响新 token 到达后的说明能力。
- `MessageList.tsx` 同步收紧最新助手消息定位：
- `lastAssistantMessageId` 与当前 timeline 映射不再复制数组反向查找。
- 减少旧会话窗口里消息列表重算时的短期内存峰值。
- `MessageList.test.tsx` 补回归：
- 旧会话恢复首帧应关闭 Provider 配置自动加载。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/components/MessageList.test.tsx"
npx eslint "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx" --max-warnings 0
npm exec -- vitest run "src/components/agent/chat/workspace/useWorkspaceConversationSceneRuntime.test.ts" "src/components/agent/chat/workspace/WorkspaceConversationScene.test.tsx" "src/components/agent/chat/components/MessageList.test.tsx"
npm run typecheck
npm run test:contracts
curl -fsS "http://127.0.0.1:3030/health"
```
结果：
- MessageList 定向回归：通过，`85` 个测试通过。
- MessageList ESLint：通过。
- Workspace / MessageList 组合回归：通过，`100` 个测试通过。
- TypeScript：通过。
- Contract：通过。
- DevBridge 健康检查：通过，返回 `status=ok`。
未完成验证：
- 真实 Chrome 页签 E2E 仍需复用现有 Lime 页签继续；当前 Chrome DevTools MCP 被已有 `chrome-profile` 占用，不能按工具提示启动 isolated 新实例，也不应启用 `--no-sandbox` 的临时 Playwright profile。
下一步：
1. 复用现有 Lime 页签或恢复 Chrome DevTools MCP 会话后，执行旧会话 A / 旧会话 B / 新建对话切换采样。
2. 如果旧会话仍有明显鼠标 loading、CPU 或内存峰值，优先评估 MessageList 动态高度虚拟化，避免继续在 tab shell 做边际优化。
### 2026-04-30：P1 第五刀，点击旧会话不再即时预取抢占切换链路
已完成：
- `AppSidebarConversationShelf.tsx` 取消点击路径上的即时预取：
- 移除 `onPointerDown` 的立即 prefetch。
- `onFocus` 改为延迟预取，并在 `onClick` / `onBlur` / `onPointerLeave` 时取消尚未触发的预取。
- 保留真正悬停或键盘聚焦停留时的旧会话预热，但避免“鼠标按下 -> 预取 hydrate -> 正式切换复用同一 promise”造成点击后主链多做一次旧会话 hydrate。
- `MessageList.tsx` 继续减少热路径小分配：
- `renderGroups` 从 `assistantMessages.map(...).find(...)` 改为单次循环，避免每个消息组创建临时数组。
主线收益：
- 用户点击旧会话时，正式切换链路直接进入 `switchTopic.fetchDetail`，不再先被侧栏 prefetch 的 snapshot hydrate 抢占。
- 只有用户悬停停留足够久时才做预热，符合“后台预取不能拖慢明确点击”的优先级。
已验证：
```bash
npm exec -- vitest run "src/components/AppSidebar.test.tsx" -t "旧会话预取|已有会话"
npm exec -- vitest run "src/components/agent/chat/components/MessageList.test.tsx" -t "旧会话恢复首帧|复杂任务完成后"
npx eslint "src/components/app-sidebar/AppSidebarConversationShelf.tsx" "src/components/AppSidebar.test.tsx" "src/components/agent/chat/components/MessageList.tsx" --max-warnings 0
npm run typecheck
git diff --check -- "docs/exec-plans/agentui-implementation-progress.md" "src/components/app-sidebar/AppSidebarConversationShelf.tsx" "src/components/AppSidebar.test.tsx" "src/components/agent/chat/components/MessageList.tsx"
npm exec -- vitest run "src/components/AppSidebar.test.tsx"
npm exec -- vitest run "src/components/agent/chat/components/MessageList.test.tsx"
```
结果：
- AppSidebar 定向回归：通过，`4` 个测试通过。
- MessageList 定向回归：通过，`2` 个测试通过。
- ESLint touched files：通过。
- TypeScript：通过。
- diff whitespace：通过。
- AppSidebar 全量回归：通过，`38` 个测试通过。
- MessageList 全量回归：通过，`85` 个测试通过。
未完成验证：
- `npm run verify:gui-smoke` 本轮执行到 `smoke:agent-runtime-tool-surface-page` 时失败：`launch_browser_session` 对 DevBridge 的请求连续返回 `fetch failed`，随后 `3030` DevBridge 端口不可用。当前 `tauri dev` 正在重新编译 Rust 侧，暂不能继续真实 E2E。
- Chrome DevTools MCP 仍被既有 `chrome-profile` 占用；不能按工具提示启动 isolated 新实例，也不应使用带 `--no-sandbox` 的临时 Playwright profile。
下一步：
1. 等 DevBridge 3030 恢复后，先只做真实页签 E2E，不再重复触发会重启/重编 Rust 的 GUI smoke。
2. E2E 重点采样：点击旧会话是否还出现鼠标 loading、打开第二个旧会话是否并发卡住、新建对话 tab 是否立即可打开。
3. 若仍慢，下一刀进入真正的 MessageList 虚拟化或后端 `getSession` 分块返回，不继续做 prefetch / tab shell 边际优化。
### 2026-04-30：P1 第六刀，补 AgentUI 性能数值采集
已完成：
- 新增 `src/lib/agentUiPerformanceMetrics.ts`：
- 浏览器内维护 `AgentUI` 性能 ring buffer，最多保留 `500` 条。
- 自动暴露 `window.__LIME_AGENTUI_PERF__`，E2E 可直接读取：
- `window.__LIME_AGENTUI_PERF__.entries()`
- `window.__LIME_AGENTUI_PERF__.summary()`
- `window.__LIME_AGENTUI_PERF__.clear()`
- `summary()` 按 `sessionId` 汇总旧会话打开关键耗时：
- `clickToSwitchStartMs`
- `clickToCachedSnapshotMs`
- `clickToPendingShellMs`
- `clickToFetchStartMs`
- `fetchDetailDurationMs`
- `runtimeGetSessionDurationMs`
- `clickToSwitchSuccessMs`
- `clickToMessageListPaintMs`
- `finalRenderedMessagesCount / hiddenHistoryCount / persistedHiddenHistoryCount`
- 可用时同步采集 `usedJSHeapSize / totalJSHeapSize`
- 采样点已接入：
- `AppSidebarConversationShelf.tsx`：旧会话 hover/focus 预取 schedule / cancel / fire、点击旧会话。
- `sessionClient.ts`：`agentRuntime.listSessions`、`agentRuntime.getSession` start / success / error 及 duration。
- `useAgentSession.ts`：session prefetch、switch start、cached snapshot、pending shell、fetch detail、switch success。
- `MessageList.tsx`：旧会话恢复 / 分页历史窗口 commit 与 post-paint 消息数、turn 数、timeline defer 状态。
- E2E 读取建议：
```ts
await page.evaluate(() => window.__LIME_AGENTUI_PERF__?.clear());
// 执行：点击旧会话 A -> 点击旧会话 B -> 新建对话 -> 切回旧会话
const snapshot = await page.evaluate(() => window.__LIME_AGENTUI_PERF__?.summary());
console.table(snapshot?.sessions ?? []);
```
已验证：
```bash
npm exec -- vitest run "src/lib/agentUiPerformanceMetrics.test.ts" "src/components/agent/chat/components/MessageList.test.tsx" -t "agentUiPerformanceMetrics|旧会话首帧应记录|旧会话恢复首帧"
npm exec -- vitest run "src/components/AppSidebar.test.tsx" -t "旧会话预取|已有会话"
npm exec -- vitest run "src/components/agent/chat/hooks/useAsterAgentChat.test.tsx" -t "thread timeline|hydrate|switchTopic"
npm exec -- vitest run "src/lib/api/agent.test.ts" -t "getSession|listSessions|runtime"
npx eslint "src/lib/agentUiPerformanceMetrics.ts" "src/lib/agentUiPerformanceMetrics.test.ts" "src/lib/api/agentRuntime/sessionClient.ts" "src/components/agent/chat/hooks/useAgentSession.ts" "src/components/app-sidebar/AppSidebarConversationShelf.tsx" "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx" --max-warnings 0
npm run typecheck
```
结果：
- 性能采集器 / MessageList 定向回归：通过，`4` 个测试通过。
- AppSidebar 旧会话点击 / 预取回归：通过，`4` 个测试通过。
- useAsterAgentChat 定向回归：通过，`9` 个测试通过。
- Agent Runtime API 定向回归：通过，`15` 个测试通过。
- ESLint touched files：通过。
- TypeScript：通过。
下一步：
1. 复用现有 Lime 页签时先执行 `window.__LIME_AGENTUI_PERF__.clear()`。
2. 完成多旧会话切换后导出 `summary().sessions`，用 `clickToMessageListPaintMs`、`runtimeGetSessionDurationMs`、`finalRenderedMessagesCount` 判断瓶颈是在桥接、后端查询还是前端渲染。
3. 如果 `runtimeGetSessionDurationMs` 低但 `clickToMessageListPaintMs` 高，下一刀优先 MessageList 虚拟化；如果两者都高，优先后端 `getSession` 分块/缓存。
## 风险记录
- 当前工作区已有多处未提交改动，本计划只记录本轮 AgentUI 主线增量；合并前需继续保护用户已有改动，不做回滚。
- 本轮没有新增 Tauri command，不需要同步 command catalog / mock；后续若新增 session snapshot 或 timeline page command，必须按命令边界四侧同步。
### 2026-04-30：P1 第七刀，旧会话列表分页降载与路由追平去重
采集事实：
- 首轮 Playwright 性能探针显示，点击新建 / 旧会话附近仍有侧边栏 `agent_runtime_list_sessions(limit=37)` 在 DevBridge 通道内运行：浏览器侧记录约 `10.2s`，同时 `workspace_get` 出现 `timeout after 5000ms`，导致旧会话列表和新建任务入口体感卡顿。
- 后端日志对比显示，降载前异常环境下 `limit=37` / `limit=60` 最高可到数秒级；降载后同一工作区常规请求收敛到 `limit=11` / `limit=21`，最近采样约 `47-218ms`。
- 二次 Playwright 探针发现“侧栏打开旧会话 -> 路由追平”会让同一历史会话出现 `switchStartCount=2` / `runtimeGetSessionStartCount=2`，即同一次用户意图触发两次 `getSession`。
已完成：
- `AppSidebar.tsx`：最近 / 归档会话列表改成可见数量 `+1` 的哨兵分页：
- 最近会话首屏请求从强制 `37` 降到 `11`。
- 归档首屏请求从 `17` 降到 `9`。
- 点击“查看更多”时再按当前可见数量继续增加请求窗口，避免首页一次性预取两页以上历史。
- `useAgentSession.ts`：任务中心内部 topics 初始列表请求从 `60` 降到 `21`，保留 `topicsListMayBeTruncatedRef` 作为旧会话不在首屏列表时的 detached 恢复兜底。
- `AppSidebarConversationShelf.tsx`：hover/focus 预取延迟从 `140ms` 提高到 `900ms`，点击路径仍会取消未触发预取，减少鼠标扫过列表时的隐性 `getSession`。
- `useWorkspaceInitialSessionNavigation.ts` / `AgentChatWorkspace.tsx`：新增“外部任务打开已启动”去重标记；侧栏/任务中心事件已经触发 `switchTopic` 后，2s 内路由 `initialSessionId` 追平不再重复打开同一会话。
- `.tmp/agentui-perf-probe.mjs` 临时探针增强：只从侧栏会话 shelf 取目标、记录 invoke command、采集 CDP CPU/heap delta、long task 与 detail 阶段 summary。
Playwright 复测摘要：
- 页面：`http://127.0.0.1:1420/`，Chrome 持久化 profile `.tmp/lime-agentui-e2e-chrome-profile`，DevBridge `3030` 健康检查通过。
- 旧会话 A `AI网关MVP规划`：`switchStartCount=1`，`runtimeGetSessionStartCount=1`，`runtimeGetSessionDurationMs≈82ms`，`clickToSwitchSuccessMs≈685ms`，`clickToMessageListPaintMs≈626ms`，`longTask=0`。
- 旧会话 B `PPT大纲规划`：去重后目标会话 `switchStartCount=1`，`runtimeGetSessionStartCount=1`；`agent_runtime_get_session` 最慢约 `157ms`。
- 切回旧会话 A：目标会话 `switchStartCount=1`，`runtimeGetSessionStartCount=1`，`clickToMessageListPaintMs≈18ms`，`longTask=0`。
- 控制台：`0 error / 1 warning`；warning 为 `useAgentTopicSnapshot.skipWithoutActiveTopic`，不阻塞主链，但提示后续 topics 截断后仍可继续优化 topic snapshot 噪音。
- 网络：最慢 `/invoke` 从上一轮 `agent_runtime_get_session≈6.1s` / `agent_runtime_list_sessions≈10.2s`，恢复到本轮 `agent_runtime_get_session≤157ms`、`agent_runtime_list_sessions≤218ms`；上一轮 6s 主要由探针未等待 detail、连续发起多个旧会话请求叠加当时 Rust 编译负载污染，已用直接 curl 与等待 detail 的探针复核。
已验证：
```bash
npx eslint "src/components/AppSidebar.tsx" "src/components/AppSidebar.test.tsx" "src/components/app-sidebar/AppSidebarConversationShelf.tsx" "src/components/agent/chat/hooks/useAgentSession.ts" "src/components/agent/chat/hooks/useAsterAgentChat.test.tsx" --max-warnings 0
npm exec -- vitest run "src/components/AppSidebar.test.tsx" "src/components/agent/chat/hooks/useAsterAgentChat.test.tsx" -t "最近对话|窗口重新聚焦|打开已有会话|归档动作|加载话题时应后台预热"
npx eslint "src/components/agent/chat/AgentChatWorkspace.tsx" "src/components/agent/chat/workspace/useWorkspaceInitialSessionNavigation.ts" "src/components/agent/chat/index.test.tsx" --max-warnings 0
npm exec -- vitest run "src/components/agent/chat/index.test.tsx" "src/components/agent/chat/workspace/useWorkspaceInitialSessionNavigation.test.tsx" -t "外层侧边栏通知打开历史会话|初始会话|dedupe|连续打开历史会话|任务中心初始会话标签"
npm run typecheck
node ".tmp/agentui-perf-probe.mjs" > ".tmp/agentui-perf-probe-latest.json"
```
结果：
- AppSidebar / useAsterAgentChat 定向回归：通过，`8` 个测试通过。
- AgentChatWorkspace / initial session navigation 定向回归：通过，`16` 个测试通过。
- ESLint touched files：通过。
- TypeScript：通过。
- Playwright 性能探针：完成，旧会话切换不再出现同一目标重复 `switchTopic` / 重复 `getSession`；列表请求窗口已降到 `11/21`。
下一步：
1. 修复或降噪 `useAgentTopicSnapshot.skipWithoutActiveTopic`：在 topics 初始窗口被截断时，active session 已由 detail 恢复，不应持续警告。
2. 新建任务探针当前用性能事件等待会超时，应改成 DOM 空态断言（`home-start-surface` / `青柠一下，灵感即来`）后再纳入数值汇总，避免把探针等待超时误判为产品卡顿。
3. 如果用户继续反馈“打开后消息正文仍慢”，下一刀不要再扩 AppSidebar；优先进入 `agent_runtime_get_session` 首包/分块返回或 MessageList 真虚拟化。
### 2026-04-30：P1 第八刀，新建页旧会话本地打开与探针降噪
采集事实：
- 上一轮 Playwright 探针只剩 `0 error / 1 warning`，warning 来自旧会话 pending shell 阶段：`sessionId` 已切到目标会话，但 topics 还没等 `getSession` detail upsert，`useAgentTopicSnapshot` 误判为 active topic 缺失。
- 新建任务按钮探针此前仍等待 `session.switch.*` / `messageList.paint` 性能事件；新建首页本身不会触发旧会话切换事件，导致 14s 超时被误算成产品卡顿。
- 从新建页点击旧会话时，侧栏原先只在 `agentEntry=claw` 时走任务中心本地事件；在 `new-task` 首页会回到路由跳转路径，容易触发页面切换、路由追平和首刀延迟。
已完成：
- `useAsterAgentChat.ts`：`useAgentTopicSnapshot` 在 `isSessionHydrating=true` 时抑制 active topic 暂缺 warning；真实缺失仍会在非 hydrating 阶段继续暴露。
- `AppSidebar.tsx`：新建任务首页点击已有会话改走 `TASK_CENTER_OPEN_TASK_EVENT`，交给当前 Agent workspace 本地新增/切换标签，不再跳出当前页面做 claw 路由导航。
- `AppSidebar.test.tsx`：更新新建任务首页点击历史会话的回归，断言发出本地 open event 且不调用 `onNavigate`。
- `.tmp/agentui-perf-probe.mjs`：新建任务改为等待 `home-start-surface` / `青柠一下，灵感即来` DOM 就绪；会话选择避开已标记 active 的侧栏项，避免把 no-op 点击计入旧会话恢复耗时。
Playwright 复测摘要：
- 页面：`http://127.0.0.1:1420/`，Chrome 持久化 profile `.tmp/lime-agentui-e2e-chrome-profile`，DevBridge `3030` 健康检查通过。
- 旧会话 B `PPT大纲规划`：`clickToSwitchStartMs≈3ms`，`runtimeGetSessionDurationMs≈71ms`，`clickToSwitchSuccessMs≈81ms`，`switchStartCount=1`，`runtimeGetSessionStartCount=1`，`longTask=0`。
- 切回旧会话 A `AI网关MVP规划`：`clickToSwitchStartMs≈1ms`，`runtimeGetSessionDurationMs≈108ms`，`clickToSwitchSuccessMs≈113ms`，`switchStartCount=1`，`runtimeGetSessionStartCount=1`，`longTaskMax≈51ms`。
- 新建任务：`clickToHomeMs≈83ms`，`home-start-surface=true`，`0` 次 `getSession`，`longTask=0`。
- 控制台：`0 error / 0 warning`；上一轮 `useAgentTopicSnapshot.skipWithoutActiveTopic` 已消失。
- 网络：最慢有效 `/invoke` 为 `agent_runtime_list_sessions≈413ms`，`agent_runtime_get_session≤113ms`；长连接类 `/events` 在浏览器关闭时出现 `ERR_ABORTED`，为探针关闭上下文产生的非阻塞噪音。
已验证：
```bash
node --check ".tmp/agentui-perf-probe.mjs"
npx eslint "src/components/agent/chat/hooks/useAsterAgentChat.ts" --max-warnings 0
npm exec -- vitest run "src/components/agent/chat/hooks/useAgentTopicSnapshot.test.tsx" "src/components/agent/chat/hooks/useAsterAgentChat.test.tsx" -t "detached 会话缺少活动话题|切换到无本地快照的话题"
npx eslint "src/components/AppSidebar.tsx" "src/components/AppSidebar.test.tsx" "src/components/agent/chat/hooks/useAsterAgentChat.ts" --max-warnings 0
npm exec -- vitest run "src/components/AppSidebar.test.tsx" "src/components/agent/chat/index.test.tsx" -t "新建任务首页点击已有会话|new-task 首页收到外层侧栏打开历史会话|任务中心内点击已有会话|打开已有会话时若导航已有缓存任务"
npm run typecheck
npm run bridge:health -- --timeout-ms 120000
node ".tmp/agentui-perf-probe.mjs" > ".tmp/agentui-perf-probe-latest.json"
```
结果：
- Hook 定向回归：通过，`2` 个测试通过。
- AppSidebar / AgentChatPage 定向回归：通过，`4` 个测试通过。
- ESLint touched files：通过。
- TypeScript：通过。
- Playwright 性能探针：完成，旧会话点击到 switch start 已收敛到 `1-3ms`；新建任务首页首帧收敛到 `~83ms`；控制台 warning 清零。
下一步：
1. 如果用户继续感知“旧会话正文慢”，优先看真实重历史会话的 `messageList.paint` 是否稳定产出；当前小会话 detail 很快，但首个 no-op 样本说明探针还需要进一步区分“已在内存中的会话”与“真实冷打开”。
2. 对大历史会话继续采 `renderedMessagesCount / threadItemsCount / timelineGroupsCount / longTaskMaxMs`，若 `getSession≤150ms` 但 paint 或 long task 高，下一刀进入 MessageList 虚拟化 / timeline worker。
3. 当前磁盘空间不足时不跑 `npm run tauri:dev:headless` / `verify:gui-smoke` 这类会触发 Rust 编译的重验证；本轮以已就绪 DevBridge + 浏览器实测覆盖主路径。
### 2026-04-30：P1 第九刀，旧会话 pending shell 直接进入会话布局
采集事实：
- 上一轮探针中，旧会话 `switch.success` 与 `agent_runtime_get_session` 已较快返回，但部分样本 `messageList.paintCount=0`，探针等待到超时，体感上等同“打开旧会话后仍停在新建首页 / 空态”。
- 代码复核发现 `chatLayoutVisibility` 已支持 `isSessionHydrating`，但 `AgentChatWorkspace` 还没有把该状态传入 `shouldShowChatLayout`；同时 `effectiveShowChatPanel` 的 `new-task` 分支也没有把 hydrating 算作会话活动。
- 这会导致旧会话 pending shell 阶段虽然已进入 session switch 主链，但布局层仍按空白新建页处理，MessageList 无法及时挂载并产生 paint 指标。
已完成：
- `AgentChatWorkspace.tsx`：
- `shouldShowChatLayout(...)` 调用补入 `isSessionHydrating`。
- `effectiveShowChatPanel` 在 `agentEntry="new-task"` 时把 `isSessionHydrating` 计入会话活动，旧会话恢复的 pending shell 不再被空白首页分支吞掉。
- `chatLayoutVisibility.ts` / `chatLayoutVisibility.test.ts`：保留第八刀新增的 hydrating 可见性参数与回归，确保旧会话恢复 pending shell 阶段直接进入会话布局。
已验证：
```bash
npx eslint "src/components/agent/chat/AgentChatWorkspace.tsx" "src/components/agent/chat/utils/chatLayoutVisibility.ts" "src/components/agent/chat/utils/chatLayoutVisibility.test.ts" "src/components/agent/chat/index.test.tsx" "src/components/AppSidebar.tsx" "src/components/AppSidebar.test.tsx" "src/components/agent/chat/taskCenterDraftTaskEvents.ts" --max-warnings 0
npm exec -- vitest run "src/components/agent/chat/utils/chatLayoutVisibility.test.ts" "src/components/agent/chat/index.test.tsx" "src/components/AppSidebar.test.tsx" -t "旧会话恢复 pending shell|new-task 首页收到外层侧栏打开历史会话|新建任务首页点击已有会话|任务中心内点击已有会话|空白新建任务首页应保留浏览器式工作区顶栏"
npm run typecheck
```
结果：
- ESLint touched files：通过。
- AgentUI 布局 / 新建页 / 侧栏打开历史会话定向回归：通过，`5` 个测试通过。
- TypeScript：通过。
Playwright / DevBridge 状态：
- 下一次真实复测必须继续按仓库规则使用 Chrome 持久化 profile：`.tmp/lime-agentui-e2e-chrome-profile`，不使用 isolated 临时 profile，不传 `--no-sandbox`。
- 本轮真实 E2E 暂未完成：`npm run bridge:health -- --timeout-ms 5000` 多次失败，`http://127.0.0.1:3030/health` 未监听。
- 尝试拉起 `npm run tauri:dev:headless` 时，当前机器已有其它 Rust / GUI smoke / cargo test 编译链占用 Cargo lock，输出持续停在 `Blocking waiting for file lock on package cache / artifact directory`；同时磁盘可用空间约 `8.8-9.3GiB`，继续叠加编译会污染旧会话卡顿采样。
- 已停止本轮自己拉起的 headless 链路，未清理用户已有的长期 Tauri / cargo 校验进程；待 DevBridge 恢复后再复跑真实旧会话多标签性能探针。
下一步：
1. DevBridge 3030 恢复后，复测：新建任务首页 -> 打开旧会话 A -> 打开旧会话 B -> 切回旧会话 A -> 新建空白任务标签。
2. 采集并对比 `clickToSwitchStartMs`、`runtimeGetSessionDurationMs`、`clickToSwitchSuccessMs`、`messageListPaintCount`、`clickToMessageListPaintMs`、`longTaskMaxMs`。
3. 如果 `messageList.paintCount` 仍为 `0`，下一刀继续查 `WorkspaceShellScene` / `MessageList` 条件渲染；如果 paint 正常但 `longTaskMaxMs` 高，转入 MessageList 虚拟化或 timeline worker。
### 2026-04-30：P1 第十刀，旧会话首帧延后底部运行状态行重计算
采集事实：
- 第九刀修复布局后，旧会话 pending shell 能进入会话布局；下一处首帧风险集中在 `MessageList` 首帧仍会为了最后一条 assistant 的底部运行状态行同步扫描 `threadItems`。
- 已分页旧会话或历史窗口里，timeline 本体虽然已经延后，但 `buildInputbarRuntimeStatusLineModel` / `buildAgentTaskRuntimeCardModel` 仍可能在首帧扫描并过滤大量历史 `threadItems`，造成 CPU 峰值和短期数组分配。
- 这些底部完成态状态行不是旧会话首帧正文可见的必要条件；等待 historical timeline idle 后再补上，不影响发送中、等待输入、排队或 active turn 的实时反馈。
已完成：
- `MessageList.tsx`：
- 对旧会话恢复窗口增加 `shouldDeferTailRuntimeStatusLine`。
- 当旧会话历史 timeline 正在 idle 延后、且没有发送中 / active turn / pending action / queued turn / pending request 时，首帧不再构建底部运行状态行。
- `messageList.commit/paint` 指标增加 `tailRuntimeStatusDeferred`，后续 E2E 可直接判断首帧是否成功避开这段同步计算。
- `MessageList.test.tsx`：旧会话消息少但执行过程多的场景，新增断言：首帧不渲染 `inputbar-runtime-status-line`，idle 后再恢复。
- `inputbarRuntimeStatusLine.ts` / `agentTaskRuntime.ts`：把多处 `[...array].reverse()` 与 `filter(...).find(...)` 改为从尾部循环或单次扫描，减少旧会话打开时的短期数组复制。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/components/MessageList.test.tsx" -t "旧会话消息较少但执行过程很多时也应延后构建 timeline|旧会话首帧应记录可汇总的渲染采样数值|复杂任务完成后应把运行状态"
npm exec -- vitest run "src/components/agent/chat/components/Inputbar/components/InputbarRuntimeStatusLine.test.tsx" "src/components/agent/chat/components/MessageList.test.tsx" -t "旧会话消息较少但执行过程很多时也应延后构建 timeline|复杂任务完成后应把运行状态|InputbarRuntimeStatusLine"
npm exec -- vitest run "src/components/agent/chat/utils/agentTaskRuntime.test.ts"
npx eslint "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx" "src/components/agent/chat/utils/agentTaskRuntime.ts" "src/components/agent/chat/utils/inputbarRuntimeStatusLine.ts" --max-warnings 0
npm run typecheck
npm run bridge:health -- --timeout-ms 5000
```
结果：
- MessageList 定向回归：通过，`3` 个测试通过。
- Inputbar / MessageList 组合回归：通过，`4` 个测试通过。
- Agent task runtime 单测：通过，`4` 个测试通过。
- ESLint touched files：通过。
- TypeScript：通过。
- DevBridge 健康检查：失败，`http://127.0.0.1:3030/health` 仍未监听；因此本轮仍未做真实 Playwright 多旧会话采样。
下一步：
1. DevBridge 恢复后，优先复测旧会话 A/B 切换，确认 `messageList.paint` 中 `tailRuntimeStatusDeferred=true` 的首帧是否降低 `longTaskMaxMs`。
2. 如果 E2E 仍显示 `longTaskMaxMs` 高，下一刀继续把 historical `threadItems` 的摘要计算改成按 `latestTurnId` 建索引或移入 idle/worker。
3. 如果 E2E 显示 `runtimeGetSessionDurationMs` 高于渲染耗时，回到后端 `getSession` 分块/缓存，不再继续前端微调。
### 2026-04-30：P1 第十一刀，旧会话按可见 turns 精确裁剪 threadItems
采集事实：
- 第十刀已把旧会话首帧的底部运行状态行延后，但 `MessageList` 仍会在历史窗口首帧按消息数量粗略截取尾部 `threadItems`。
- 对“消息少、工具过程多”的旧会话，粗略按 `messageCount * factor` 截取容易把与当前可见消息无关的 turn item 也带入 `buildMessageTurnTimeline` / runtime status 计算，继续造成短期 CPU 与数组分配峰值。
- 旧会话分页窗口的首帧只需要渲染尾部可见 assistant 关联的 turns；其它历史 turns 可以等待“加载完整历史”或后续展开再参与计算。
已完成：
- `MessageList.tsx`：
- 移除按消息数倍数裁剪 `threadItems` 的粗略常量。
- 根据当前实际渲染的 assistant 消息数推导旧会话恢复窗口的 `renderedTurns` tail window，并额外保留 `currentTurnId`，避免进行中 turn 被裁掉。
- 在分页旧会话 / hidden history 场景下构建 `renderedTurnIdSet`，`renderedThreadItems` 只保留这些 turns 对应的 `threadItems`。
- `MessageList.test.tsx`：新增“已分页旧会话首帧应只把尾部相关 turns 的 threadItems 纳入计算”回归，覆盖 8 个 turns、40 个 threadItems、首帧只渲染 2 条消息的场景；断言首帧只纳入尾部 2 个 turns / 10 个 threadItems。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/components/MessageList.test.tsx" -t "已分页旧会话首帧应只把尾部相关 turns 的 threadItems 纳入计算|旧会话消息较少但执行过程很多时也应延后构建 timeline|已分页旧会话的完成执行过程应先折叠为轻量摘要"
npx eslint "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx" --max-warnings 0
python3 - <<'PY'
# 120s wrapper for npm run typecheck
PY
```
结果：
- MessageList 定向回归：通过，`3` 个测试通过。
- ESLint touched files：通过。
- TypeScript：本轮 wrapper 在 `120s` 超时后终止；未看到类型错误输出。当前机器仍有其它 Rust / cargo 编译链与编辑器 tsserver 占用 CPU，先记录为环境性未完成，不继续叠加新的全量 typecheck。
- DevBridge / Playwright：尚未复测；需先确认 `3030` 恢复，避免在 bridge 未就绪时把前端优化误判为 GUI 卡顿。
下一步：
1. DevBridge 恢复后，复测多旧会话切换并采集 `messageList.commit/paint`，重点看 `threadItemsCount` 是否随可见 turn 数下降，而不是随全量历史增长。
2. 如果 `getSession≤150ms` 但 `clickToMessageListPaintMs` 或 `longTaskMaxMs` 仍高，下一刀继续把 `buildMessageTurnTimeline` 的历史 mapping / sort 移到 idle 或建立 `turn_id -> items` 索引。
3. 如果 `runtimeGetSessionDurationMs` 高于渲染耗时，转回后端 `getSession` 分块 / 缓存，不再继续前端微调。
### 2026-04-30：P1 第十二刀，旧会话首帧完全跳过 threadItems 扫描
采集事实：
- 第十一刀已经把旧会话恢复窗口的 `threadItems` 从“按消息数粗略截尾”改成“按可见 turns 精确裁剪”，但首帧仍需要遍历全量 `threadItems` 才能筛出尾部 turns。
- 对工具轨迹特别多的旧会话，哪怕最终只渲染 10 条相关 items，首帧的全量数组扫描仍会造成鼠标 loading、CPU 峰值和短期内存分配。
- 旧会话首帧的核心目标是先让消息文本和吸顶布局可见；完成态历史 timeline / 底部运行状态行可以继续等 idle 后补齐。
已完成：
- `MessageList.tsx`：
- `shouldDeferHistoricalTimeline` 改为基于原始 `threadItems.length` 与恢复窗口判断，避免为了判断是否延后而先扫描 / 裁剪 `threadItems`。
- 新增 `shouldDeferThreadItemsScan`：旧会话历史 timeline 尚未 idle-ready、且没有 active turn 时，`renderedThreadItems` 首帧直接返回空数组，不再遍历全量历史 items。
- `timelineHydrationKey` 改用原始 `threadItems.length + lastItemId`，确保后台 item 变化仍能触发 idle hydrate。
- `messageList.commit/paint` 指标增加 `threadItemsScanDeferred`，后续 Playwright 能直接看到首帧是否跳过了全量扫描。
- `MessageList.test.tsx`：更新旧会话窗口测试，首帧断言 `threadItemsScanDeferred=true` 且 `threadItemsCount=0`；idle 后再断言精确裁剪到尾部 2 个 turns / 10 个 `threadItems`。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/components/MessageList.test.tsx" -t "已分页旧会话首帧应只把尾部相关 turns 的 threadItems 纳入计算|旧会话消息较少但执行过程很多时也应延后构建 timeline|已分页旧会话的完成执行过程应先折叠为轻量摘要"
npx eslint "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx" --max-warnings 0
npm run bridge:health -- --timeout-ms 5000
python3 - <<'PY'
# 180s wrapper for npm run typecheck
PY
```
结果：
- MessageList 定向回归：通过，`3` 个测试通过。
- ESLint touched files：通过。
- DevBridge 健康检查：失败，`http://127.0.0.1:3030/health` 未监听；本轮没有进入 Playwright 真实旧会话采样。
- TypeScript：本轮 wrapper 未超时，但 `tsc --noEmit` 约 `40s` 后以 code `241` 退出且无错误诊断；检查后未发现残留 `tsc` 进程。当前仍有其它 `run-vitest-smart` / Vite 进程占用 CPU，先记录为环境性未闭环，避免继续叠加重校验影响用户本机性能。
下一步：
1. DevBridge 3030 恢复后复测旧会话 A/B 切换，重点读取 `messageList.commit/paint` 的 `threadItemsScanDeferred`、`threadItemsCount`、`clickToMessageListPaintMs` 和 `longTaskMaxMs`。
2. 若首帧 `threadItemsScanDeferred=true` 后仍卡顿，下一刀优先检查 `StreamingRenderer` / Markdown light render 的长文本成本，而不是继续优化 timeline。
3. 若 idle 后 timeline hydrate 才出现长任务，则把 `buildMessageTurnTimeline` 的 grouping / sort 继续放到 worker 或分片 idle 队列。
### 2026-04-30：P1 第十三刀，旧会话首帧延后 contentParts 细节扫描 + 流式首字立即可见
采集事实：
- DevBridge `3030` 仍未监听，当前无法用 Playwright 复测真实旧会话 A/B 切换；同时本机已有多条 Rust 编译链占用 CPU，继续启动 `tauri:dev:headless` 会污染“卡顿”采样。
- 第十二刀已经让旧会话首帧跳过全量 `threadItems` 扫描，但 `MessageList` 在渲染每条历史 assistant 时仍会同步 `sanitizeContentPartsForDisplay`、过滤工具/思考片段、计算 inline process coverage。
- 对已完成的旧会话，这些 `contentParts` 细节不是首帧文本可见的必要条件；可以等 historical timeline idle 后再恢复。
- 流式纯文本首次挂载时，`StreamingText` 之前以空字符串作为首帧，必须等下一次 `requestAnimationFrame` 才开始吐字；在模型已有首个分片时，这会放大“首字慢”的体感。
已完成：
- `MessageList.tsx`：
- 增加 `shouldDeferHistoricalAssistantMessageDetails`，旧会话恢复窗口、historical timeline 尚未 ready、没有 active turn / sending / action request / toolCalls / thinking 时，首帧不再处理历史 assistant 的 `contentParts` 细节。
- 增加 `historicalContentPartsDeferredCount` 指标，后续 E2E 可判断是否命中旧会话首帧轻量路径。
- 保留文本正文优先渲染；idle 后再恢复 contentParts / timeline 细节，避免首帧被旧工具过程拖慢。
- `MessageList.test.tsx`：新增“旧会话首帧应延后历史助手 contentParts 细节扫描”回归，断言首帧 `data-content-parts=0`、`historicalContentPartsDeferredCount=1`，idle 后指标恢复为 `0`。
- `StreamingRenderer.tsx`：
- 增加 `STREAMING_TEXT_INITIAL_VISIBLE_CHARS` 和 `resolveInitialStreamingDisplayText`。
- 流式纯文本首次挂载时立即显示前 `12` 个字符；结构化内容仍保持空首帧，避免把未完整的 A2UI / write_file 协议直接露出。
- 当流式文本发生非 append 替换时，也立即种下同样的纯文本前缀，减少重置后的空白等待。
- `StreamingRenderer.test.tsx`：新增“流式纯文本首帧应立即显示前缀”回归，并保留大 backlog 追赶测试。
已验证：
```bash
npm exec -- vitest run "src/components/agent/chat/components/MessageList.test.tsx" -t "旧会话首帧应延后历史助手 contentParts 细节扫描|已分页旧会话首帧应只把尾部相关 turns 的 threadItems 纳入计算|旧会话消息较少但执行过程很多时也应延后构建 timeline"
npm exec -- vitest run "src/components/agent/chat/components/StreamingRenderer.test.tsx" -t "流式纯文本首帧应立即显示前缀|流式正文积压较多时应快速追上最新目标文本|纯文本内容应短路跳过结构化解析"
npx eslint "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx" "src/components/agent/chat/components/StreamingRenderer.tsx" "src/components/agent/chat/components/StreamingRenderer.test.tsx" --max-warnings 0
npm run bridge:health -- --timeout-ms 5000
```
结果：
- MessageList 定向回归：通过，`3` 个测试通过。
- StreamingRenderer 定向回归：通过，`3` 个测试通过。
- ESLint touched files：通过。
- DevBridge 健康检查：失败，`http://127.0.0.1:3030/health` 未监听。
- TypeScript：本轮暂不追加全量 typecheck；当前机器已有多条 Rust 编译链高负载运行，继续叠加全量 TS 校验会影响用户正在反馈的 CPU/鼠标 loading 问题。待编译链空闲后补跑。
下一步：
1. DevBridge 恢复后复测真实旧会话：读取 `threadItemsScanDeferred`、`historicalContentPartsDeferredCount`、`clickToMessageListPaintMs`、`longTaskMaxMs`。
2. 如果旧会话首帧仍慢，下一刀只看 `MarkdownRenderer` 本体：把 restored history 的短正文也先走纯文本预览 / idle 后 Markdown hydrate，避免 ReactMarkdown 在首帧解析多条历史短回复。
3. 如果流式仍首字慢，下一步看后端事件到达时间：区分“前端首帧空白”与“runtime 第一段 delta 到达慢”。
第十三刀补充：
- `agentUiPerformanceMetrics.ts`：summary 增加 `historicalContentPartsDeferredMax` 与 `threadItemsScanDeferredCount`，让 Playwright 不只读 raw entries，也能在 session summary 里直接看到旧会话首帧轻量路径命中次数。
- `agentUiPerformanceMetrics.test.ts`：补充 summary 断言，覆盖 contentParts 延后最大值与 threadItems 扫描延后次数。
- 追加验证：
```bash
npm exec -- vitest run "src/lib/agentUiPerformanceMetrics.test.ts"
npx eslint "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx" "src/components/agent/chat/components/StreamingRenderer.tsx" "src/components/agent/chat/components/StreamingRenderer.test.tsx" "src/lib/agentUiPerformanceMetrics.ts" "src/lib/agentUiPerformanceMetrics.test.ts" --max-warnings 0
```
结果：性能指标汇总单测通过，ESLint touched files 继续通过。
docs/exec-plans/multimodal-runtime-contract-plan.md
+98 -7
View File
@@ -62,6 +62,32 @@ runtime identity
2. `src/lib/governance/modalityCapabilityMatrix.json`
3. contract 守卫检查 `required_capabilities` 与 `routing_slot` 是否引用已登记能力和模型角色
### Phase 3：ModalityExecutionProfile
状态：进行中。
本阶段输出：
1. `docs/roadmap/warp/execution-profile.md`
2. `src/lib/governance/modalityExecutionProfiles.json`
3. `image_generation`、`browser_control`、`pdf_extract`、`voice_generation`、`audio_transcription`、`web_research`、`text_transform` 的最小 execution profile。
4. contract 守卫检查 current contract 必须被 profile 覆盖，且 profile 的模型角色、权限、LimeCore policy、artifact policy 必须覆盖 contract。
5. `src/lib/governance/modalityExecutionProfiles.ts` 把 profile / adapter registry 解析成 launch metadata 快照。
6. 后续仍需把 profile registry 接入 Rust runtime policy merge、thread read、evidence 和 GUI 可视化。
### Phase 5：Executor Adapter registry
状态：进行中。
本阶段输出：
1. `src/lib/governance/modalityExecutionProfiles.json` 的 `executor_adapters`
2. `skill:image_generate`、`browser:browser_assist`、`skill:pdf_read`、`service_skill:voice_runtime`、`skill:transcription_generate`、`skill:research`、`skill:text_transform` 的最小 adapter 声明。
3. contract 守卫检查 current contract 的 `executor_binding` 必须能解析到 `executor_kind:binding_key` adapter。
4. adapter 的 progress / cancel / resume / artifact 支持位、artifact output、permission requirements 与 failure mapping 必须覆盖 contract。
5. 前端 `runtime_contract` snapshot 已携带 `executor_adapter` 摘要，避免上层入口继续只传 executor binding。
6. 后续仍需把 adapter registry 接入真实执行前检查、统一 task index 与 LimeCore policy snapshot。
### Phase 7：上层入口绑定
状态：进行中。
@@ -80,10 +106,11 @@ runtime identity
1. `browser_control` 先作为底层 Browser Assist 运行合同进入 `harness.browser_assist`，不新增独立 browser task 协议。
2. `@浏览器 / @Browser Agent / @Mini Tester` 仍保留原始用户消息，只把 `browser_requirement`、launch URL 与 contract snapshot 交给后端 runtime 决策。
3. `browser_control.bound_entries` 暂不扩展；`@` 命令只是入口层，不能成为底层 contract 事实源。
3. `browser_control.bound_entries` 只登记 `@浏览器`、`@Browser Agent`、`@Mini Tester`、`@Web Scheduler`、`@Web Manage` 的 entry metadata；`@` 命令仍只是入口层，不能写 task / artifact / viewer 事实源。
4. 后端 `BrowserAssistRuntimeHint` 必须解析并保留 `browser_control` 合同快照，browser tool result metadata 需要携带同一组 contract 字段。
5. Evidence Pack 从 browser tool timeline metadata 导出 `browser_control` 的 `modalityRuntimeContracts`，使浏览器动作也进入 replay 可用的事实源。
6. Replay / grader 必须把 `browser_control` 识别成 Browser Assist 回归样本，检查 browser action trace、禁止 WebSearch 替代真实浏览器动作。
6. Evidence Pack 的 `modalityRuntimeContracts.snapshotIndex.browserActionIndex` 必须汇总 action、session、URL、observation 与 screenshot，使 Browser Assist 不再只能靠 raw snapshots 人工查找。
7. Replay / grader 必须把 `browser_control` 识别成 Browser Assist 回归样本，检查 browser action trace / browserActionIndex、禁止 WebSearch 替代真实浏览器动作。
### Phase 9：PDF Extract 底层合同注入
@@ -112,7 +139,7 @@ runtime identity
4. 服务型场景 prompt 必须显式说明当前底层合同，禁止退回 `legacy_tts_test_command` 或伪造“云端已提交”。
5. Evidence Pack 必须能从 voice runtime / service scene timeline args 或 metadata 导出 `voice_generation` 的 `modalityRuntimeContracts`，使配音合同进入 replay 可评分事实源。
6. Replay / grader 必须把 `voice_generation` 识别成本地 ServiceSkill/voice runtime 回归样本，检查 `service_scene_launch(scene_key=voice_runtime)` 或后续 audio_task/audio_output 证据，并禁止 `legacy_tts_test_command`、伪造云端提交、普通聊天文本或通用文件卡替代。
7. 本阶段新增最小 `audio_generate` task artifact 协议，把 `voice_generation` 写入标准 `audio_task/audio_output` 产物；`audio viewer` 与真实音频 worker 继续后置，避免提前拆出第二套音频执行事实源。
7. 本阶段新增最小 `audio_generate` task artifact 协议，把 `voice_generation` 写入标准 `audio_task/audio_output` 产物；聊天区提供最小音频任务卡与运行时文档 viewer，并允许执行器通过同一 task artifact 回写 `audio_output.completed`。当前 `lime-audio-worker` 已接入 OpenAI-compatible speech provider seam：有 current API Key Provider 凭证时写入 `.lime/runtime/audio/*.mp3` 并回写 `payload.audio_output.completed`；未配置 provider / model / client 时必须显式写回 `audio_provider_*` 错误，聊天任务卡与运行时文档 viewer 都从同一 task artifact 展示完成/失败事实，不回退 legacy TTS，也不伪造音频路径。
### Phase 11：Web Research 合同注入
@@ -143,6 +170,20 @@ runtime identity
6. Replay / grader 必须把 `text_transform` 识别成文本转换回归样本，检查文本转换 Skill trace 与必要文件读取证据，并禁止 `frontend_direct_text_transform`、ToolSearch、WebSearch 或普通聊天替代。
7. 本阶段不拆 `summary_generation`、`translation`、`analysis`、`publish_compliance` 或 `logo_decomposition` 平行 contract；只有当独立 artifact、执行器或 viewer 形成稳定事实源时才拆子合同。
### Phase 13：Audio Transcription 合同注入
状态：进行中。
本阶段输出：
1. `audio_transcription` 作为音频转写底层运行合同进入 `harness.transcription_skill_launch.transcription_task`，`@转写 / @transcribe / @Audio Extractor` 仅保留为 entry source。
2. 前端 `@转写` 发送时保留原始消息，并注入 `modality_contract_key=audio_transcription`、`required_capabilities`、`routing_slot=audio_transcription_model` 与 `runtime_contract` 快照。
3. 后端 `prepare_transcription_skill_launch_request_metadata` 需要为旧客户端或缺省 metadata 补齐同一组合同字段，避免回退成前端直连 ASR、普通文件读取或 generic file transcript。
4. `Skill(transcription_generate)` prompt 必须显式说明当前底层合同，继续压制 ToolSearch / WebSearch / Read / Glob / Grep 首刀偏航。
5. 标准 `transcription_generate` task file 必须写入同一 `audio_transcription` 合同与 `transcript.pending` 子产物，CLI / Agent tool 回退入口不能另写平行协议。
6. 媒体任务索引、Evidence Pack 与 Replay / grader 必须能定位 transcript 状态、来源、语言、输出格式与失败码，使 `audio_transcription` 不再只停留在入口 metadata。
7. 本阶段接入最小 `lime-transcription-worker` 与用户可见恢复层：OpenAI-compatible provider 成功时写入 `.lime/runtime/transcripts/*` 并回写 `transcript.completed`，provider/source/contract 失败时回写 `transcript.failed` 与明确错误码；聊天任务卡和 `.lime/runtime/transcription-generate/*.md` 运行时文档优先消费统一媒体任务索引的 `transcript_*` snapshot，并在完成态读取 `transcript_path` 文本内容供复制校对；当前 viewer 已能从 JSON / SRT / VTT transcript 中解析时间轴与说话人，生成可逐段编辑校对的段落表；保存校对稿时复用 ArtifactDocument 版本链，写入 `transcriptCorrection*` / `transcriptSegmentsCorrected` / `transcriptCorrectionDiffSummary` metadata，不改写原始 ASR 输出，并在 viewer 中展示“校对稿已保存”状态；更多 ASR adapter 与本地离线 ASR 后续继续消费同一 task artifact 与 `transcriptIndex`。
## 本轮最小闭环
第一轮已完成治理底座；当前纠偏后的最小闭环必须至少贯通一个真实运行时 vertical slice。
@@ -177,16 +218,40 @@ runtime identity
18. `web_research` 第二十刀把 `@研报 / @竞品` 收为报告型子入口：入口仍走 `report_skill_launch -> Skill(report_generate)`，底层合同、evidence、replay 统一归入 `web_research`。
19. `text_transform` 第二十一刀把 `@读文件 / @总结 / @翻译 / @分析 / @发布合规 / @Logo拆解` 收为文本/文档转换子入口：入口仍分别走 `summary_skill_launch`、`translation_skill_launch`、`analysis_skill_launch`，底层合同、evidence、replay 统一归入 `text_transform`。
20. `voice_generation` 第二十三刀新增最小 `audio_generate` artifact protocol：`create_audio_generation_task_artifact` / `lime_create_audio_generation_task` 写入标准 `audio_task/audio_output` payload，前端 API、DevBridge mock、task index、evidence pack 与 replay grader 都消费同一份 `voice_generation` 合同快照。
21. `voice_generation` 第二十四刀把 `audio_generate` tool result 恢复为聊天区任务卡与 `.lime/runtime/audio-generate/*.md` 运行时文档；打开任务卡不再尝试展示隐藏 `.lime/tasks/audio_generate/*.json`，而是进入可读的音频任务 viewer，同时继续把 task JSON 作为 primary source。
22. `voice_generation` 第二十五刀新增最小 `audio_output` 完成态回写：`complete_audio_generation_task_artifact` 只更新标准 `audio_generate` task file，把 `audio_path / mime_type / duration_ms` 同步写入 payload、`payload.audio_output` 与 `record.result`，并通过同一媒体任务事件刷新前端，不新增云端执行协议。
23. `voice_generation` 第二十六刀把完成态回流接回聊天任务卡：`useWorkspaceAudioTaskPreviewRuntime` 监听同一媒体任务事件并轮询已追踪的 `audio_generate` task file，从 `payload.audio_output` / `record.result.audio_output` 恢复 `audioUrl / mimeType / durationMs`，让执行器回写后能直接在现有轻卡中出现可播放结果。
24. `voice_generation` 第二十七刀新增最小音频执行器骨架：`create_audio_generation_task_artifact` 创建标准 `audio_generate` 后会进入同一媒体任务执行链；在真实音频 provider 尚未接通时，worker 明确把 task 写成 `failed`、`last_error.code=audio_worker_unavailable`、`payload.audio_output.status=failed`，不伪造 `audio_path`，也不回流 `legacy_tts_test_command`。
25. `voice_generation` 第二十八刀把音频执行器从纯骨架推进到 OpenAI-compatible provider seam：worker 通过 current API Key Provider 解析 `provider_id/model/api_key/base_url`，调用 `/audio/speech` 获取真实音频 bytes，保存到 `.lime/runtime/audio/<task_id>.mp3`，并用同一 task artifact 回写 `payload.audio_output.completed`、`record.result.audio_output` 与 `worker_id=lime-audio-worker`；provider 未配置、resolver 缺失或非 OpenAI-compatible client 时分别写回 `audio_provider_unconfigured` / `audio_provider_resolver_unavailable` / `audio_provider_client_missing`，不回 legacy。
26. `voice_generation` 第二十九刀把 worker 失败态回流到聊天音频任务卡：`useWorkspaceAudioTaskPreviewRuntime` 从 `payload.audio_output` / `record.last_error` 解析 `audio_provider_*` 错误码、错误原因与 retryable，不保留失败 artifact 里不存在的 `audio_path`；`TaskMessagePreview` 显示 Provider 错误码和原因，让“未配置 provider / resolver 不可用 / adapter 缺失”成为用户可见事实，而不是泛化成普通失败。
27. `voice_generation` 第三十刀把完成/失败态同步到同一个运行时文档 viewer：音频任务轮询更新 `taskPreview` 的同时，会重建 `.lime/runtime/audio-generate/*.md` 对应的 ArtifactDocument metadata；打开任务卡时看到的是 `audio_output.completed` 的音频路径或 `audio_provider_*` 失败原因，而不是创建任务时的旧“等待执行器”文档。
28. `voice_generation` 第三十一刀把 `audio_output` 细节接入 evidence / replay：`audio_task.modality_runtime_contract` 快照现在携带 `audioOutput.status/audioPath/providerId/model/workerId/errorCode`，`snapshotIndex.audioOutputIndex` 可检索完成/失败态；replay tags、failure modes、expected 与 grader 会区分 `audio_output.completed` 和 Provider 失败，禁止把已完成音频降级成 Markdown 文本、普通聊天或通用文件卡。
29. `voice_generation` 第三十二刀把 `audio_output` 细节接入统一媒体任务索引：`list_media_task_artifacts` 的 `modality_runtime_contracts` 现在输出 `audio_output_count`、`audio_output_statuses`、`audio_output_error_codes` 与每个 snapshot 的 `audio_output_*` 字段；前端类型与浏览器 mock 同步，任务列表/恢复逻辑不必重新打开 task JSON 才能知道音频已完成、失败或缺少 Provider。
30. `voice_generation` 第三十三刀让聊天音频任务恢复优先消费统一媒体任务索引：`useWorkspaceAudioTaskPreviewRuntime` 会先读取 `list_media_task_artifacts(modalityContractKey=voice_generation, taskType=audio_generate)` 的 `audio_output_*` snapshot，把完成态音频路径或 Provider 失败直接同步到任务卡与同一条运行时文档；只有索引缺失该 task 或没有 `audio_output_status` 时才回退读取单个 task artifact。
31. `browser_control` 第三十八刀把同一份 `browserActionIndex` 接入最小 `browser_replay_viewer`：Harness evidence 面板的“打开复盘”会构造 `browser_assist` Artifact，并由 `BrowserAssistRenderer` 消费 `browserActionIndex` 展示 action/session/URL/observation/screenshot 与最近 `browser_session` / `browser_snapshot` 项；本刀不新增 browser task 协议，viewer 仍消费 Evidence Pack 的同一事实源。
32. `audio_transcription` 第三十九刀把 `@转写 / @transcribe / @Audio Extractor` 收为 registry entry binding：前端 `transcription_skill_launch.transcription_task` 与 Rust prompt prepare 都会保留 `audio_transcription` 合同快照，`transcript` artifact graph 也反向登记该 current contract；本刀只建立底层合同，不新增真实转写 worker、完整 transcript viewer 或 replay 索引。
33. `audio_transcription` 第四十刀把 `transcription_generate` task artifact 接到底层合同事实源：Agent tool 与 CLI 回退入口现在写入标准 `.lime/tasks/transcription_generate/*.json`，payload 固化 `audio_transcription` contract、`requested_target=transcript` 与 `transcript.pending`；`list_media_task_artifacts` 输出 transcript 状态/来源/语言/格式，Evidence Pack 输出 `snapshotIndex.transcriptIndex`，Replay / grader 能识别缺失 task trace、缺失 transcript index 与 `transcript.failed`。
34. `audio_transcription` 第四十一刀接入最小 `lime-transcription-worker`：Agent tool 创建 task 后会启动转写执行链，OpenAI-compatible provider 成功时把响应写入 `.lime/runtime/transcripts/<task>.json|txt|srt|vtt` 并回写 `transcript.completed` / `record.result.transcript`；缺少 AppHandle、provider/model、API Key、source 文件、非 OpenAI-compatible adapter 或 provider 请求失败时回写 `transcript.failed` 与 `transcription_*` 错误码，不回退 frontend ASR、普通文件读取或 generic_file transcript。
35. `audio_transcription` 第四十二刀把 transcript 恢复层接回聊天区：`lime_create_transcription_task` 工具结果现在生成 `.lime/runtime/transcription-generate/*.md` 运行时文档；`useWorkspaceTranscriptionTaskPreviewRuntime` 监听同一媒体任务事件并优先按 `audio_transcription + transcription_generate` 查询 `list_media_task_artifacts`，从 `transcript_status/path/source/language/output_format/error_code` snapshot 恢复任务卡和 viewer，索引缺失时才回退读取单个 task artifact。
36. `audio_transcription` 第四十三刀把 transcript 文本接入最小校对 viewer：完成态恢复时读取 `transcript_path` 指向的 `.lime/runtime/transcripts/*` 文本，并把内容嵌入 `.lime/runtime/transcription-generate/*.md` ArtifactDocument 的 `code_block`，打开任务卡即可复制和校对转写文本；失败态仍只展示 `transcription_*` 错误码，不伪造 transcript 内容。
37. `audio_transcription` 第四十四刀把 transcript viewer 从纯文本推进到时间轴/说话人段落层：恢复层会解析 `.json` verbose transcript、SRT 与 VTT 的 segment / speaker / start-end 时间码，写回 `taskPreview.transcriptSegments`、聊天轻卡时间轴预览和 `.lime/runtime/transcription-generate/*.md` 的“转写时间轴（可逐段编辑校对）”表格；仍不读取隐藏 task JSON 作为主路径，也不新增前端 ASR 或 `generic_file` 旁路。
38. `audio_transcription` 第四十五刀把 transcript viewer 从只读校对推进到可保存校对稿：`.lime/runtime/transcription-generate/*.md` ArtifactDocument 会标记 `transcriptCorrectionEnabled`、提示保存不改写原始 ASR 输出；用户编辑“转写文本”或“转写时间轴”并保存时，Workbench 复用同一 ArtifactDocument 版本链写入 `transcriptCorrectionStatus=saved`、`transcriptCorrectionEditedBlockId`、`transcriptCorrectionSourceTranscriptPath`、`transcriptCorrectionTextLength`、`transcriptCorrectionSegmentCount`、`transcriptCorrectionSpeakerCount` 与 `transcriptSegmentsCorrected`，不新增 Tauri command、frontend ASR 或并行 patch 协议。
39. `audio_transcription` 第四十六刀把已保存校对稿显性化：保存后会插入/更新 `transcript-correction-status` callout，显示原始 ASR 输出保持不可变、原文/校对稿文本长度差异、段落差异和说话人数差异；同时写入 `transcriptCorrectionDiffSummary`，让后续专用 transcript viewer、diff 面板或导出器不必重新扫描 block 才能识别校对变更。
40. `Phase 3 / Phase 5` 第四十七刀新增最小 `ModalityExecutionProfile` 与 `ExecutorAdapter` 事实源：`modalityExecutionProfiles.json` 覆盖 7 个 current contracts 与 7 个 executor adapters，`governance:modality-contracts` 会校验 profile 覆盖、adapter 绑定、支持位、产物、权限、LimeCore policy 与 failure mapping；本刀只建立 current 治理事实源，不新增命令、bridge、mock 或运行时执行分支。
41. `Phase 3 / Phase 5` 第四十八刀把 profile / adapter 事实源接入前端 runtime contract resolver：`resolveModalityRuntimeContractBinding()` 现在会从 `modalityExecutionProfiles.json` 解析 `execution_profile` 与 `executor_adapter` 快照，并随 `runtime_contract` 进入所有 current launch metadata；本刀仍不新增 Tauri command、bridge、mock 或 Rust executor 分支，为下一步 Rust runtime preflight / evidence 可视化提供统一输入。
42. `Phase 3 / Phase 5` 第四十九刀把 profile / adapter 快照接入 Evidence Pack：`runtime_evidence_pack_service` 会从 `runtime_contract.execution_profile.profile_key` 与 `runtime_contract.executor_adapter.adapter_key` 提取 `executionProfileKey` / `executorAdapterKey`，并写入 `modalityRuntimeContracts.snapshots[]`、`snapshotIndex.executionProfileKeys`、`snapshotIndex.executorAdapterKeys` 与 `toolTraceIndex.items[]`；本刀仍不改变真实 executor 行为，只让 evidence/replay 主链能看见 Phase 3/5 决策输入。
暂不做：
1. 暂不把所有 `@`、按钮、Scene 批量登记到 `bound_entries`；本轮只绑定 `image_generation` 第一条 vertical slice。
1. 暂不把所有 `@`、按钮、Scene 批量登记到 `bound_entries`；已进入主链的 `image_generation`、`browser_control`、`pdf_extract`、`voice_generation`、`web_research`、`text_transform`、`audio_transcription` 继续按合同逐条收口，未进入 current 事实源的入口不批量扩张。
2. 暂不把 LimeCore 写成默认执行器；LimeCore 仍只作为 catalog / policy / offer / audit 控制面。
3. 暂不批量改造视频、语音、PDF、搜索等其他 viewer；先让 `image_generation` vertical slice 收口。
3. 暂不批量改造视频、PDF、搜索、研报等其他 viewer；已补到最小 viewer / index 的 browser 与 voice 继续沿同一事实源演进，未形成稳定 artifact graph 的能力先不另开 viewer 协议。
4. 暂不新增 PDF task artifact / reader viewer；PDF 本轮只收 executor trace、evidence 和 replay，避免在底层事实源未完全稳定前再开一套产物协议。
5. 暂不新增 audio viewer / 真实音频 worker；voice 本轮只落最小 `audio_generate` artifact protocol，不伪造云端提交，也不把普通聊天文本或通用文件卡当作音频产物。
5. 暂不新增完整音频工作台，也暂不把 Gemini / Azure 专有语音 API、LimeCore 云 run/poll 或语音后处理工作流并入主链；voice 目前只提供最小 `audio_generate` 任务卡、运行时文档 viewer、OpenAI-compatible speech provider seam 与完成/失败态回流。未配置可用 provider 时仍必须显式失败，不伪造云端提交，也不把普通聊天文本或通用文件卡当作音频产物。
6. 暂不新增独立 `report_generation` 合同；`@研报 / @竞品` 继续走 `report_skill_launch -> Skill(report_generate)` 主链，但其底层能力归属先收敛到 `web_research`，避免把 report artifact 协议提前扩张成第二套事实源。
7. 暂不新增独立 `summary_generation`、`translation`、`analysis`、`publish_compliance` 或 `logo_decomposition` 合同；这组轻量文本/文档转换入口先统一收敛到 `text_transform`，避免把上层 `@` 命令提前扩张成平行底层事实源。
8. 暂不新增非 OpenAI-compatible ASR adapter 或本地离线 ASR 执行器；`audio_transcription` 当前交付标准 `transcription_generate` task writer、`lime-transcription-worker`、`transcript.completed/failed` 回写、统一媒体任务索引、聊天任务卡、可编辑校对运行时文档 viewer、JSON/SRT/VTT 时间轴与说话人段落展示、ArtifactDocument 版本化校对稿保存、校对稿状态/差异摘要、Evidence `transcriptIndex` 与 Replay 检查。
9. 暂不在本刀实现 Rust 侧真实 profile merge、tenant policy snapshot、adapter runtime preflight 或 GUI/evidence 可视化；当前先把 Phase 3 / Phase 5 的底层合同关系落成可检查 registry，并接入前端 launch metadata 与 Evidence Pack 快照，后续继续让 Rust runtime 消费同一事实源。
## 分类
@@ -259,9 +324,35 @@ runtime identity
- 2026-04-29：继续第十五刀 `pdf_extract` entry binding：`modalityRuntimeContracts.json` 现在登记 `at_pdf_read_command`，覆盖 `@读PDF / @pdf / @read_pdf`，并把 launch path 固定为 `harness.pdf_read_skill_launch.pdf_read_request`；`useWorkspaceSendActions` 从 `resolvePdfExtractRuntimeContractBinding().boundEntrySources` 读取 `entry_source`，只保留 `PDF_EXTRACT_DEFAULT_ENTRY_SOURCE` 作为 registry 缺省 fallback。
- 2026-04-29：继续第十六刀 `voice_generation`：新增前端 registry resolver `resolveVoiceGenerationRuntimeContractBinding()`，`voice_generation.bound_entries` 登记 `at_voice_command`；`@配音` 发送时在 `harness.service_scene_launch.service_scene_run` 注入 `voice_generation` contract 快照，后端 `prepare_service_scene_launch_request_metadata` 也会为 `voice_runtime` 旧 metadata 补齐合同字段，服务型场景 prompt 会输出底层合同、所需能力与 `runtime_contract(JSON)`，并禁止退回 `legacy_tts_test_command`。
- 2026-04-29：继续第十七刀 `web_research`：新增前端 registry resolver `resolveWebResearchRuntimeContractBinding()`，`web_research.bound_entries` 登记 `at_search_command`、`at_deep_search_command` 与 `at_site_search_command`；`@搜索 / @深搜 / @站点搜索` 发送时分别在 research / deep search / site search request 中注入 `web_research` contract 快照，Rust 侧 prepare 函数也会把缺省合同字段补齐到 launch 与 request，prompt 显式输出底层合同、所需能力与 `runtime_contract(JSON)`，并禁止退回模型记忆、本地文件搜索、通用 WebSearch 或 ToolSearch 偏航。
- 2026-04-30：继续第十八刀 `web_research` evidence/replay：`runtime_evidence_pack_service` 现在会从 `Skill(research)` / `Skill(site_search)` tool timeline 的 args 或 metadata 导出 `web_research` 的 `modalityRuntimeContracts`，`source=web_research_skill_trace.modality_runtime_contract`、`routingEvent=executor_invoked`、`routingOutcome=accepted/failed`、`expectedRoutingSlot=report_generation_model`；`runtime_replay_case_service` 同步增加 `web-research`、`research-skill`、`web-research-trace` tags，并在 expected / grader 中要求继续走 `Skill(research)` / `Skill(site_search)` 与真实 search_query / `lime_site_*` 证据，禁止 `model_memory_only_answer`、`local_file_search_before_research_skill`、ToolSearch 或普通聊天替代。
- 2026-04-30：继续第十八刀 `web_research` evidence/replay：`runtime_evidence_pack_service` 现在会从 `Skill(research)` / `Skill(site_search)` tool timeline 的 args 或 metadata 导出 `web_research` 的 `modalityRuntimeContracts`，`source=web_research_skill_trace.modality_runtime_contract`、`routingEvent=executor_invoked`、`routingOutcome=accepted/failed`、`expectedRoutingSlot=report_generation_model`；`runtime_replay_case_service` 同步增加 `web-research`、`research-skill`、`web-research-trace` tags，并在 expected / grader 中要求继续走 `Skill(research)` / `Skill(site_search)` 与真实 search*query / `lime_site*\*`证据，禁止`model_memory_only_answer`、`local_file_search_before_research_skill`、ToolSearch 或普通聊天替代。
- 2026-04-30：继续第十九刀 `web_research` 索引化：`runtime_evidence_pack_service` 的 `modalityRuntimeContracts` 现在输出 `snapshotIndex`，汇总 `contractKeys`、`sourceCounts`、`routingOutcomeCounts`、`expectedRoutingSlots` 与 `toolTraceIndex.items`；`web_research` replay input 会携带同一份索引，使回归样本能直接定位 `executorBindingKey=research`、`entrySource=at_search_command` 与 `web_research_skill_trace`，不再只能扫描 raw snapshots。
- 2026-04-30：继续第二十刀 `web_research` 报告型子入口：`web_research.bound_entries` 纳入 `at_report_command` 与 `at_competitor_command`，前端 `@研报 / @竞品` 发送会在 `harness.report_skill_launch.report_request` 注入同一份 `web_research` contract；Rust `report_skill_launch` prepare / prompt 也补齐合同字段，同时保留首刀 `Skill(report_generate)`，并把 evidence / replay 扩展到 `report_generate` trace，避免另开 `report_generation` 平行事实源。
- 2026-04-30：继续第二十一刀 `text_transform`：新增 `text_transform` contract 并登记 `at_file_read_command`、`at_summary_command`、`at_translation_command`、`at_analysis_command`、`at_publish_compliance_command`、`at_logo_decomposition_command`；前端 `@读文件 / @总结 / @翻译 / @分析 / @发布合规 / @Logo拆解`、Rust summary/translation/analysis prepare 与 prompt、evidence `text_transform_skill_trace`、replay tags / expected / grader 已统一收敛到同一份底层合同，入口层不再新增 summary / translation / analysis 平行事实源。
- 2026-04-30：继续第二十二刀 `voice_generation` evidence/replay：`runtime_evidence_pack_service` 现在会从 voice runtime / service scene timeline args 或 metadata 导出 `voice_generation` 的 `modalityRuntimeContracts`，`source=voice_generation_service_scene_trace.modality_runtime_contract`、`routingEvent=executor_invoked`、`routingOutcome=accepted/failed`、`expectedRoutingSlot=voice_generation_model`；`runtime_replay_case_service` 同步增加 `voice-generation`、`voice-runtime`、`voice-generation-trace` tags，并在 expected / grader 中要求继续走 `service_scene_launch(scene_key=voice_runtime)` / 本地 ServiceSkill runtime 或后续 audio_task/audio_output 证据，禁止 `legacy_tts_test_command`、伪造云端提交、普通聊天文本或通用文件卡替代。
- 2026-04-30：继续第二十三刀 `voice_generation` audio artifact protocol：新增 `audio_generate` task type、`create_audio_generation_task_artifact` Tauri 命令与 `lime_create_audio_generation_task` Agent tool，只写入标准 `audio_task/audio_output` 产物，不生成真实音频、不提交云端；前端 API、DevBridge mock、`list_media_task_artifacts` contract index、evidence `audio_task.modality_runtime_contract` 与 replay grader 已同步消费同一份 `voice_generation` 合同快照，下一刀再补 audio viewer / 真实 worker。
- 2026-04-30：继续第二十四刀 `voice_generation` audio preview/viewer：`lime_create_audio_generation_task` 工具结果现在会带回 `source_text / voice / audio_path / duration_ms` 等任务元数据，前端 `taskPreviewFromToolResult` 可生成 `audio_generate` 任务卡，并同步生成 `.lime/runtime/audio-generate/*.md` 运行时文档 viewer；聊天区任务卡采用 Lime 现有轻卡视觉，打开后进入可读文档而不是暴露隐藏 task JSON，真实音频 worker 与完整音频工作台继续后置。
- 2026-04-30：继续第二十五刀 `voice_generation` audio_output 回写闭环：新增 `complete_audio_generation_task_artifact` 命令与前端 API / DevBridge mock / governance catalog / mockPriority 同步，执行器可把标准 `audio_generate` task 从 pending 更新为 `succeeded`，并把完成态 `audio_output` 同时写入 payload 与 result；本刀仍不引入真实音频 worker、LimeCore 云执行 run/poll 或 legacy TTS。
- 2026-04-30：继续第二十六刀 `voice_generation` audio preview runtime：新增 `useWorkspaceAudioTaskPreviewRuntime`，聊天工作区会监听 `lime://creation_task_submitted` 并轮询仍在 running 或缺少音频 URL 的 `audio_generate` 任务，从标准 task artifact 恢复完成态 `audio_output` 到任务卡；这一步把第二十五刀写回口真正接回用户可见轻卡，但仍不新增完整音频工作台。
- 2026-04-30：继续第二十七刀 `voice_generation` audio worker skeleton：`create_audio_generation_task_artifact` 现在会在 emit 标准媒体任务事件后启动 `lime-audio-worker` 骨架；骨架先把任务推进到 `running`，随后在真实 provider 未接通时写回 `audio_worker_unavailable` 可重试失败与 `payload.audio_output.status=failed`，不生成假 `audio_path`、不调用 legacy TTS，也不新增 LimeCore 云 run/poll 协议。
- 2026-04-30：继续第二十八刀 `voice_generation` OpenAI-compatible audio provider seam：`lime-audio-worker` 不再停在纯骨架失败，而是从标准 `audio_generate` task payload / 全局 `media_defaults.voice` 解析 provider 与 model，通过 current API Key Provider 主链拿到 OpenAI-compatible `api_key/base_url`，调用 `/audio/speech` 返回真实音频 bytes 后保存到 `.lime/runtime/audio/<task_id>.mp3`，再回写 `payload.audio_output.completed` 与 `record.result.audio_output`；无 AppHandle、DbConnection、ApiKeyProviderService、provider/model/API Key 或非 OpenAI-compatible client 时写回明确 `audio_provider_*` 失败码，不回退 `legacy_tts_test_command`、`test_tts`、伪云端提交或通用文件卡。
- 2026-04-30：继续第二十九刀 `voice_generation` audio failure preview：聊天区 `useWorkspaceAudioTaskPreviewRuntime` 现在会从标准 `audio_generate` task artifact 的 `payload.audio_output` / `record.last_error` 读取 `audio_provider_*` 错误码、错误原因与 retryable，并在失败 artifact 没有 `audio_path` 时清空任务卡音频路径；`TaskMessagePreview` 同步展示 Provider 错误码和原因，让未配置 provider、resolver 不可用或 adapter 缺失直接回到用户可见任务卡，不再泛化成普通失败，也不伪造音频结果。
- 2026-04-30：继续第三十刀 `voice_generation` audio viewer sync：`useWorkspaceAudioTaskPreviewRuntime` 在从标准 `audio_generate` task artifact 恢复完成/失败态时，会同步重建同一条 `.lime/runtime/audio-generate/*.md` 运行时文档的 ArtifactDocument metadata；打开聊天任务卡时，viewer 会展示 `audio_output.completed` 的真实音频路径，或展示 `audio_provider_*` 错误码与原因，不再停留在创建任务时的“等待音频执行器”文档。
- 2026-04-30：继续第三十一刀 `voice_generation` audio evidence/replay details：`runtime_evidence_pack_service` 现在会从 `audio_generate` task artifact 导出 `audioOutput` 摘要，并在 `snapshotIndex.audioOutputIndex` 汇总 audio output 状态、Provider 错误码与 worker；`runtime_replay_case_service` 同步把 `audio-output-completed` / `audio-output-failed` 纳入 suite tags / failure modes，并在 expected / grader 中要求 replay 保留 `audio_output.completed` 与真实 `audio_path`，或显式处理 Provider 失败，不允许回退 legacy TTS、伪造音频路径、普通聊天文本或通用文件卡。
- 2026-04-30：继续第三十二刀 `voice_generation` audio task index details：`media_task_cmd` 的 `list_media_task_artifacts` 索引现在把 `audio_output` 完成/失败事实直接纳入 `modality_runtime_contracts`，包含按状态汇总、Provider 错误码汇总，以及每个 task snapshot 的 `audio_output_status/path/mime_type/duration_ms/error_code/retryable`；`src/lib/api/agentRuntime/types.ts` 与浏览器 fallback mock 同步更新，避免浏览器链路或前端恢复逻辑只能靠重新读取隐藏 task JSON 才能判断音频产物状态。
- 2026-04-30：继续第三十三刀 `voice_generation` audio index restore：聊天工作区的 `useWorkspaceAudioTaskPreviewRuntime` 现在会先按 `voice_generation + audio_generate` 查询统一媒体任务索引，从 snapshot 的 `audio_output_status/path/mime_type/duration_ms/error_code/retryable` 直接恢复任务卡和 `.lime/runtime/audio-generate/*.md` viewer；索引缺失或旧 task 没有 `audio_output_status` 时才保留原来的单 task artifact 读取兜底，避免主路径继续依赖隐藏 task JSON。
- 2026-04-30：继续第三十四刀 `Phase 4 artifact graph`：新增 `docs/roadmap/warp/artifact-graph.md` 与 `src/lib/governance/modalityArtifactGraph.json`，把 `image/audio/browser/pdf/report/presentation/webpage/generic_file` 的 artifact kind、truth source、viewer、evidence 与 task index 字段落成可检查事实源；`governance:modality-contracts` 现在会校验 contract 的 `artifact_kinds` 必须能在 artifact graph 中找到 truth source / viewer / evidence 交集，防止新增多模态能力继续退回未知 viewer 或通用文件卡旁路。
- 2026-04-30：继续第三十五刀 `browser_control entry binding`：`browser_control.bound_entries` 现在登记 `@浏览器 / @browser / @browse`、`@Browser Agent`、`@Mini Tester`、`@Web Scheduler`、`@Web Manage`，所有入口只声明 `harness.browser_assist` launch metadata、默认输入映射与可见性策略；前端 Browser Assist launch 会按触发词从 registry 派生 `entry_source`，不再把所有浏览器别名压成同一个入口来源，也不让入口直接写 task / artifact / viewer。
- 2026-04-30：继续第三十六刀 `browser_control browserActionIndex`：`runtime_evidence_pack_service` 现在会从 Browser Assist tool metadata 提取 `browserAction` 摘要，并在 `modalityRuntimeContracts.snapshotIndex.browserActionIndex` 汇总 actionCount、sessionCount、lastUrl、observationCount、screenshotCount、backend/status/action 计数与逐项 `browser_session` / `browser_snapshot` artifactKind；`runtime_replay_case_service` 同步把 `browser-action-index` 纳入 suite tags / expected / grader，要求 replay 继续导出可查询 browser session/snapshot 索引，而不是只靠 raw snapshots 或 WebSearch 替代。
- 2026-04-30：继续第三十七刀 `browser_control Harness visible index`：Evidence Pack 的 `observabilitySummary.modalityRuntimeContracts.snapshotIndex.browserActionIndex` 现在会携带裁剪后的 Browser Assist 摘要，前端 `AgentRuntimeEvidencePack` 类型与 normalizer 同步解析该索引；`HarnessStatusPanel` 在导出问题证据包后展示 Browser Assist 索引卡，直接暴露 action/session/URL/observation/screenshot 与最近 `browser_session` / `browser_snapshot` 项，证明浏览器复盘已从 runtime.json 人工查找前进到用户可见 evidence 面板。
- 2026-04-30：继续第三十八刀 `browser_control browser_replay_viewer`：`HarnessStatusPanel` 的 Browser Assist 索引卡新增“打开复盘”，以 evidence pack + `browserActionIndex` 构造临时 `browser_assist` Artifact；`BrowserAssistRenderer` 现在可从 artifact meta 或 JSON content 读取 `browserActionIndex`，渲染最小 `browser_replay_viewer`，展示动作统计、最近 URL、会话/target/entry、截图可用性与最近 `browser_session` / `browser_snapshot`。这一步把 evidence 可见摘要推进到 Lime 内部复盘面，但仍不新增 browser task 文件协议；完整交互回放、权限 profile 可视化与截图/DOM/network 深层展开继续后置。
- 2026-04-30：继续第三十九刀 `audio_transcription contract`：新增 current `audio_transcription` contract，绑定 `@转写 / @transcribe / @Audio Extractor` 到 `harness.transcription_skill_launch.transcription_task`；前端 `@转写` metadata 现在携带 `modality_contract_key=audio_transcription`、`required_capabilities=[text_generation,audio_transcription]`、`routing_slot=audio_transcription_model` 与 `runtime_contract`，Rust `transcription_skill_launch` 也会为旧 metadata 补齐同一合同并在 prompt 中禁止 `frontend_direct_asr`、`generic_file_transcript`、转写前 ToolSearch/WebSearch 偏航。`transcript` artifact graph 已反向登记该合同；真实转写 worker、transcript viewer 与 evidence/replay 索引继续后置。
- 2026-04-30：继续第四十刀 `audio_transcription transcript task/index/evidence/replay`：`lime_create_transcription_task`、`create_transcription_task_artifact_inner` 与 CLI `lime task create transcription` / `lime media transcription generate` 现在写入同一标准 `transcription_generate` task file，payload 固化 `audio_transcription` 合同、`requested_target=transcript` 与 `transcript.pending`；`list_media_task_artifacts` 增加 `transcript_count/statuses/error_codes` 与每个 task 的 `transcript_*` snapshot，Evidence Pack 增加 `snapshotIndex.transcriptIndex`，Replay / grader 增加 `audio-transcription`、`transcription-task-trace`、`transcript-index` 标签与缺失/失败阻断检查。真实 ASR worker、完成态 transcript 回写和专属 transcript viewer 仍作为下一刀。
- 2026-04-30：继续第四十一刀 `audio_transcription transcription worker`：`lime_create_transcription_task` 现在在创建标准 `transcription_generate` task 后启动 `lime-transcription-worker`；worker 校验 `audio_transcription` contract、读取本地 source file 或下载 `source_url`，通过 current API Key Provider 解析 OpenAI-compatible credential，调用 `/audio/transcriptions`，成功时写入 `.lime/runtime/transcripts/*` 并回写 `transcript.completed`、`transcript_path`、`provider_id/model` 与 `record.result.transcript`；失败时回写 `transcript.failed` 和 `transcription_provider_*` / `transcription_source_*` / contract 错误码，不回退 frontend ASR、普通文件读取或 generic_file transcript。最小 GUI 恢复层已在第四十二刀接回，文本校对已在第四十三刀接回，时间轴/说话人段落展示已在第四十四刀接回；编辑保存型 transcript viewer、更多 ASR adapter 与本地离线 ASR 继续后置。
- 2026-04-30：继续第四十二刀 `audio_transcription transcript visible restore`：聊天工作区新增 `useWorkspaceTranscriptionTaskPreviewRuntime`，会优先从 `list_media_task_artifacts(modalityContractKey=audio_transcription, taskType=transcription_generate)` 的 `transcript_*` snapshot 恢复完成/失败态，把 `transcript_path`、source、language、outputFormat、Provider 错误码同步到任务卡与 `.lime/runtime/transcription-generate/*.md` 运行时文档；只有索引缺失时才回退读取单个 task artifact，避免用户可见路径继续依赖隐藏 task JSON 或 generic_file transcript。
- 2026-04-30：继续第四十三刀 `audio_transcription transcript proofreading viewer`：`useWorkspaceTranscriptionTaskPreviewRuntime` 在完成态恢复时读取 `transcript_path` 对应文本，并把 `transcriptText` 写回任务预览和 `.lime/runtime/transcription-generate/*.md` 运行时文档；`buildTranscriptionTaskArtifactDocument` 现在输出“转写文本（可编辑校对）”代码块，让打开任务卡即可在 Lime 内部读取、复制与校对 transcript，而不是只看到路径或退回 generic_file。
- 2026-04-30：继续第四十四刀 `audio_transcription transcript timeline viewer`：新增 transcript segment 解析器，支持从 OpenAI verbose JSON、SRT 与 VTT 读取 `start/end/speaker/text`，并把结果回写到聊天任务卡时间轴预览与 `.lime/runtime/transcription-generate/*.md` ArtifactDocument 表格；本刀只消费同一 `transcription_generate` task file 与 `transcript_path` 输出，不新增协议、Tauri command、frontend ASR 或 generic_file 旁路。编辑保存型 transcript viewer、更多 ASR adapter 与本地离线 ASR 继续后置。
- 2026-04-30：继续第四十五刀 `audio_transcription transcript correction save`：复用现有 ArtifactDocument Workbench 编辑保存链路，让转写运行时文档的文本与时间轴从“可复制校对”推进到“可保存校对稿”；生成文档会标记 `transcriptCorrectionEnabled` / `transcriptCorrectionSource=artifact_document_version` 并提示保存不改写原始 ASR 输出，保存时补 `transcriptCorrectionStatus=saved`、编辑 block、原始 transcript 路径、文本长度、段落/说话人数与 `transcriptSegmentsCorrected`。本刀没有新增 Tauri command、Rust worker 分支、frontend ASR 或 generic_file patch 旁路。
- 2026-04-30：继续第四十六刀 `audio_transcription transcript correction status`：保存校对稿时会在同一 ArtifactDocument 中插入/更新 `transcript-correction-status` 成功提示块，明确“校对稿已保存为当前运行时文档的新版本；原始 ASR 输出文件保持不变”，并写入 `transcriptCorrectionDiffSummary`，汇总原文/校对稿文本长度、段落数量、变更段落和说话人数差异。该状态仍随同一 `.lime/runtime/transcription-generate/*.md` 运行时文档版本保存，不新增命令、worker 分支或并行 patch artifact。
- 2026-04-30：继续第四十七刀 `Phase 3 / Phase 5 execution profile registry`：新增 `docs/roadmap/warp/execution-profile.md` 与 `src/lib/governance/modalityExecutionProfiles.json`，把 7 个 current contracts 的 profile、artifact policy、LimeCore policy refs 与 executor adapters 落成机器事实源；`check-modality-runtime-contracts.mjs` 现在会读取 profile registry，校验每个 current contract 都被 profile 覆盖、每个 `executor_binding` 都有 adapter、adapter 支持位/产物/权限/failure mapping 与 contract 对齐。该刀不改 Tauri command、bridge、mock 或真实 executor 行为，只把 Phase 3/5 的主线底座从文档要求推进成可阻断错误配置的 current 守卫。
- 2026-04-30：继续第四十八刀 `Phase 3 / Phase 5 profile resolver`：新增 `src/lib/governance/modalityExecutionProfiles.ts` 与定向测试，`resolveModalityRuntimeContractBinding()` 会把 current contract 对应的 `execution_profile`、`executor_adapter`、`executionProfileKey`、`executorAdapterKey` 注入同一 runtime contract binding；所有现有上层入口继续只调用 runtime contract resolver，即可随 launch metadata 携带 profile / adapter 快照。本刀没有新增命令、bridge、mock、Rust executor 或 GUI surface，只把上一刀的机器事实源推进到前端主路径输入。
- 2026-04-30：继续第四十九刀 `Phase 3 / Phase 5 evidence snapshot`：`runtime_evidence_pack_service` 现在会从 runtime contract 中提取 `executionProfileKey` 与 `executorAdapterKey`，并写入 `modalityRuntimeContracts.snapshots[]`、`snapshotIndex.executionProfileKeys`、`snapshotIndex.executorAdapterKeys` 与 `toolTraceIndex.items[]`；图片 contract preflight 失败样本与 web_research Skill trace 样本都增加断言，证明 profile / adapter 已进入 evidence 主链，而不是只停留在前端 metadata 或治理 JSON。本刀未改命令、bridge、mock、GUI 或真实 executor 行为。
docs/exec-plans/old-session-open-performance-plan.md
+46
View File
@@ -112,6 +112,8 @@
- 同轮收口：任务中心内部 `handleOpenSidebarTaskTopic` / `handleResumeSidebarTask` / `handleResumeRecentSession` / fallback restore 默认走追加/激活语义，只有外部直达会话仍保留 replace 语义，避免旧缓存污染首开深链。
- `2026-04-30` 追加旧会话卡顿收口：MessageList 对恢复中或已分页旧会话使用更小的首屏批次（20 条）和 idle 小批量补齐；已完成的旧会话最后一个 turn 不再按 current turn 处理，避免历史 timeline 在首帧被当作活跃执行过程同步展开。
- `2026-04-30` 追加 invoke 降噪：`workspace_get` 增加 1s 短 TTL 与同 id in-flight 合并，避免旧会话切换期间工作区详情重复抢占 bridge；打开旧会话时自动回填的 `recent_preferences` / `recent_team_selection` 改为同 session 后台合并队列，12s 后 idle 再写，若期间已切走则直接丢弃，手动偏好修改仍保持即时低延迟同步。
- `2026-04-30` 追加旧会话预取：侧边栏会话按钮在 focus / pointer enter / pointer down 时发送 task-center prefetch 事件，当前 Agent 工作区复用 `runtime.getSession(historyLimit: 40)` 预取结果；若用户随后点击同一会话，`switchTopic` 直接等待同一个 in-flight 详情请求，不再重复打 `agent_runtime_get_session`，并把预取结果写入本地 tail snapshot。话题列表就绪后还会在测试环境外 idle 预取最近 3 条非当前会话，降低无本地 snapshot 时的首次点击等待。
- `2026-04-30` 追加路由恢复收口：从首页/侧栏通过 `initialSessionId` 进入旧会话时不再默认 `forceRefresh`，仅 `workspace_error` 会话继续强制刷新；命中 fresh tail snapshot 时先渲染缓存，远端详情刷新延后到 1.2s + idle，避免点击主链马上再打 `agent_runtime_get_session`。
- `npx vitest run "src/components/agent/chat/index.test.tsx" -t "任务中心初始会话标签"` 通过（11 tests），新增覆盖任务中心内部连续打开旧会话、外层侧栏事件 + 路由追平后保留两个历史 tab。
- `npx vitest run "src/components/AppSidebar.test.tsx"` 通过（35 tests），新增覆盖任务中心内点击已有会话会先通知本地标签栏，再继续导航。
- `npx eslint "src/components/AppSidebar.tsx" "src/components/AppSidebar.test.tsx" "src/components/agent/chat/AgentChatWorkspace.tsx" "src/components/agent/chat/index.test.tsx" "src/components/agent/chat/taskCenterDraftTaskEvents.ts" --max-warnings 0` 通过。
@@ -131,3 +133,47 @@
- `npx vitest run "src/components/agent/chat/hooks/useSelectedTeamPreference.test.tsx"` 通过（13 tests），覆盖自动 fallback Team 回填走后台同步、手动切换仍即时回写。
- `npx eslint "src/lib/api/project.ts" "src/lib/api/project.test.ts" "src/components/agent/chat/AgentChatWorkspace.tsx" "src/components/agent/chat/hooks/useSelectedTeamPreference.ts" "src/components/agent/chat/hooks/useSelectedTeamPreference.test.tsx" --max-warnings 0` 通过。
- GUI smoke / Playwright 后置复测暂被本地并发 Cargo/Tauri 构建阻塞：`tauri:dev:headless` 首次暴露非本轮 `creation_tools.rs` 大型 `serde_json::json!` 递归限制，已在 `src-tauri/src/lib.rs` 补 `recursion_limit = "256"` 解除；随后仍有多个非本轮 `cargo test` / `cargo check` 并发进程占用 target/package lock，需环境空闲后重跑 `npm run bridge:health -- --timeout-ms 120000`、`npm run verify:gui-smoke` 与 Playwright 旧会话性能采样。
- `2026-04-30` Playwright 预取前采样（`http://127.0.0.1:1420/`）：刷新后点击 `Slow typing E2E`，早期 invoke 仅 `workspace_get_default` / `workspace_list` / `agent_runtime_get_session(historyLimit:40)`，其中 `get_session` 约 `1261ms`；`localStorage/sessionStorage` 未命中 `aster_session_snapshots*`，说明当前慢点主要是无本地 tail snapshot 时必须等后端详情返回。
- `2026-04-30` Playwright 预取后采样（`http://127.0.0.1:1420/`）：focus `AI Trends Task` 后预取 `agent_runtime_get_session(historyLimit:40)` 约 `116ms` 并写入 `aster_session_snapshots_*`；随后点击同会话，前 `500ms` 内不再出现 `agent_runtime_get_session`，仅剩 route/workspace 初始化类 `workspace_get/default/list`，消息组已可见，`body/html cursor=auto`。
- `npx vitest run "src/components/agent/chat/hooks/useAsterAgentChat.test.tsx" "src/components/AppSidebar.test.tsx"` 通过（2 files / 191 tests），覆盖旧会话预取 in-flight 复用与侧边栏 focus 预取事件。
- `npx eslint "src/components/agent/chat/hooks/useAgentSession.ts" "src/components/agent/chat/hooks/useAsterAgentChat.ts" "src/components/agent/chat/hooks/useAsterAgentChat.test.tsx" "src/components/agent/chat/AgentChatWorkspace.tsx" "src/components/agent/chat/taskCenterDraftTaskEvents.ts" "src/components/AppSidebar.tsx" "src/components/AppSidebar.test.tsx" "src/components/app-sidebar/AppSidebarConversationShelf.tsx" --max-warnings 0` 通过。
- `npx tsc --noEmit --pretty false` 通过。
- `2026-04-30` 追加旧会话首帧渲染降载：历史恢复路径的助手正文新增 `MarkdownRenderer renderMode="light"`，旧会话首屏跳过高成本 `rehypeRaw` / `rehypeKatex` / Prism 代码高亮；实时流式与普通新消息继续走标准模式。已分页旧会话首屏批次从 `20` 缩小到 `10`，idle 补齐批次从 `12` 缩小到 `6`，且恢复路径 idle 延迟从 `120ms` 提高到 `600ms`，避免点击后立即连续吃主线程。
- `2026-04-30` 追加后端历史 payload 裁剪：`agent_thread_items` 的 tail / cursor 历史查询在 `file_artifact.content` 之外，继续裁剪超大的 `tool_call.output`、`command_execution.aggregated_output` 与 `web_search.output` 到约 `16KB` 并附截断提示；完整 `list_items_by_thread` 不裁剪，保留全量诊断能力。这针对本地 runtime.db 中数 MB 级 `social_generate_cover_image` tool output 导致旧会话恢复 CPU/内存尖峰的问题。
- Context7 / WebSearch 校准结论：React 官方文档说明 `useDeferredValue` / `startTransition` 只能降低优先级，不能让慢列表本身变快；TanStack Virtual 对动态高度长列表建议 `useVirtualizer + measureElement + overscan`；Tauri 文档建议大 payload 走 async command 与 Channel/chunk。当前先落地低风险的“减少首帧同步工作 + 裁剪历史 payload”，后续若 40 条历史仍卡，下一刀应评估 MessageList 动态高度虚拟化或 `agent_runtime_get_session` 分块返回。
- Playwright E2E（`http://127.0.0.1:1420/`）：刷新后点击 `Slow typing E2E`，`agent_runtime_get_session(historyLimit:40)` 约 `322ms`，点击到首屏可见约 `605ms`，DOM 约 `491`，`body/html cursor=auto`，存在一次约 `113ms` long task；进一步缩小历史首屏批次后，等待页面稳定再切 `AI Trends Task -> Slow typing E2E`，可见旧会话 DOM 保持约 `480-626`，`cursor=auto`。当前 dev 环境 heap 读数约 `1.4GB`，更像 Vite/WebView 长会话累计值，不能单独当作本轮回归证据。
- `npm test -- "src/components/agent/chat/components/StreamingRenderer.test.tsx"` 通过（29 tests），覆盖 `markdownRenderMode="light"` 透传到 `MarkdownRenderer`。
- `npm test -- "src/components/agent/chat/components/MessageList.test.tsx"` 通过（82 tests），覆盖旧会话轻量 Markdown、长历史正文预览、历史 timeline 折叠与更小首屏批次。
- `npm run lint -- "src/components/agent/chat/components/MarkdownRenderer.tsx" "src/components/agent/chat/components/StreamingRenderer.tsx" "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/StreamingRenderer.test.tsx" "src/components/agent/chat/components/MessageList.test.tsx"` 通过。
- `cd src-tauri && CARGO_TARGET_DIR="/tmp/lime-core-test-target" cargo test -p lime-core item_tail_query_should` 通过（2 tests），覆盖历史 tail 查询裁剪大 file artifact 与大 tool output，完整查询不裁剪。
- `npm run test:contracts` 通过；本轮未新增命令，仅收窄 `agent_runtime_get_session` 历史窗口返回中的超大 inline payload。
- `npm run verify:gui-smoke` 通过；复用现有 headless Tauri 与 DevBridge，workspace-ready、browser-runtime、site-adapters、service-skill-entry、runtime tool-surface 与 runtime tool-surface page smoke 均通过。
- `npm run typecheck` 通过。
- `2026-04-30` 追加旧会话首帧二次降载：已分页/恢复旧会话中的长助手回复不再首帧挂载 `StreamingRenderer` / Markdown 解析，而是先展示纯文本预览，用户点击“展开完整内容”后才按轻量 Markdown 物化；超长历史回复的预览长度从约 `6000` 字收窄到约 `2000` 字，降低点击旧会话后的主线程解析与 DOM 压力。
- 同轮将历史 timeline 的 idle 物化从旧会话首帧后约 `80ms` 延后到约 `900ms`，避免消息刚可见后立刻构建完整 `buildMessageTurnTimeline` 抢占主线程；非旧会话仍保留原 `80ms` 行为。
- `npm test -- "src/components/agent/chat/components/MessageList.test.tsx"` 通过（83 tests），新增覆盖长助手回复纯文本预览与展开后再物化轻量 Markdown。
- Playwright E2E 复测（`http://127.0.0.1:1420/`）：刷新后切 `AI Trends Task -> Slow typing E2E`，旧会话首屏约 `77ms` 可见，DOM 节点约 `403`，历史助手正文纯文本预览 `1` 个，历史 timeline 预览 `1` 个、真实 timeline 挂载 `0` 个，`longTasks=[]`，`body/html cursor=auto`，控制台 `0 error / 0 warning`。invoke trace 中两次 `agent_runtime_get_session` 均携带 `historyLimit: 40`，无 `/invoke` 错误。
- `npm run verify:gui-smoke` 通过；复用现有 headless Tauri 与 DevBridge，workspace-ready、browser-runtime、site-adapters、service-skill-entry、runtime tool-surface 与 runtime tool-surface page smoke 均通过。
- `2026-04-30` 追加旧会话后台渲染节流：恢复/已分页旧会话不再自动 idle 补齐当前窗口里更早的消息，首屏只保留尾部批次，用户点击“立即展开更早消息”时才继续物化，避免打开后数秒内后台持续挂载 Markdown / timeline 造成 CPU 与内存尖峰；普通非旧会话仍保留空闲自动补齐。
- 同轮进一步扩大历史 timeline 延后范围：即使旧会话只有少量消息，只要 execution/thread items 很多，也先显示正文，约 `900ms` 后再生成 timeline 摘要预览；这覆盖 `Slow typing E2E` 这类“消息少但工具轨迹多”的慢会话。
- `npm test -- "src/components/agent/chat/components/MessageList.test.tsx"` 通过（84 tests），新增覆盖旧会话不自动补齐历史窗口、少消息多执行过程延后构建 timeline。
- `npx eslint "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx" --max-warnings 0` 通过。
- `npm run typecheck` 通过。
- 后置 `npm run verify:gui-smoke` 已尝试两次但未完成：当前本机已有非本轮 `cargo test --manifest-path src-tauri/Cargo.toml audio_generation_task` 与旧 `tauri:dev:headless` 长时间占用 Cargo artifact lock，DevBridge 无法就绪；为避免继续拉高 CPU，已停止本轮新拉起的 smoke 进程，未终止非本轮 cargo/tauri 进程。待该锁释放后需重跑 GUI smoke / Playwright。
- `2026-04-30` 追加首字前反馈优化：Playwright MCP 当前 transport closed，改用本地 Playwright + Chrome 复测；真实 DevBridge 恢复后发送 `首字延迟 E2E ...`，点击到用户消息可见约 `295ms`，新的 `assistant-first-token-placeholder` 同步约 `295ms` 可见。后端随后因本机缺少 `claude` 凭证在约 `5.3s` 返回失败，因此本轮无法测真实模型首 token，但已消除“首个模型事件前 assistant 气泡为空/像没响应”的 UI 体感慢点。
- 同轮代码收口：`MessageList` 在 assistant `isThinking` 且正文/parts/timeline 均为空但存在启动态 `runtimeStatus` 时，渲染轻量首字前占位；不把 runtimeStatus 塞回 `StreamingRenderer` 正文，也不渲染底部重复运行态，首个真实文本到来后自动切回正常流式渲染。
- `npm test -- "src/components/agent/chat/components/MessageList.test.tsx"` 通过（84 tests），新增覆盖启动态 runtimeStatus 只渲染轻量首字前占位、已有正文时不重复占位。
- `npx eslint "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx" --max-warnings 0` 通过。
- `npm run typecheck` 通过。
- `git diff --check -- "src/components/agent/chat/components/MessageList.tsx" "src/components/agent/chat/components/MessageList.test.tsx"` 通过。
- `npm run verify:gui-smoke` 通过；复用现有 headless Tauri 与 DevBridge，workspace-ready、browser-runtime、site-adapters、service-skill-entry、runtime tool-surface 与 runtime tool-surface page smoke 均通过。
- `2026-04-30` 追加 DeepSeek 首字续测：Playwright MCP 仍为 `Transport closed`，继续用本地 Chrome + Playwright；模型选择入口需先展开“高级设置”，再点击 `button[role="combobox"]`，可切到 `DeepSeek / deepseek-v4-flash`。
- 同轮复现输出排版问题：DeepSeek 实际流里会返回 `thinking` 分片再返回最终 `text` 分片，例如 `llm_request.2.jsonl` 中先有 `thinking: "我们…好。"`，再有 `text: "好"`；旧前端在 thinking 关闭时仍渲染 thinking block，导致 UI 显示类似 `思考中…好。好`。
- 同轮代码收口：`agentStreamTurnEventBinding` 将本轮 `thinking` 偏好透传给 `agentStreamRuntimeHandler`；当 thinking 关闭时，`thinking_delta` 只激活流状态、不再写入 assistant `thinkingContent/contentParts`，后续 `text_delta` 会清理已有 thinking part，避免 DeepSeek 隐式 reasoning 混入最终正文。
- DeepSeek E2E 成功样本（`/tmp/lime-deepseek-first-token-e2e-retry.json`）：`agent_runtime_create_session` 约 `73ms`，`agent_runtime_submit_turn` 约 `136ms`，首字前占位约 `117ms` 可见，最终 assistant 文本为 `好已完成·00:06`，不再出现 `思考中` / reasoning 重复吐字；过程中仍有一次非阻塞 `workspace_get` 5s timeout 噪音。
- DeepSeek E2E 失败样本（`/tmp/lime-deepseek-first-token-e2e-final.json`）：同样选择 `DeepSeek / deepseek-v4-flash` 后，`agent_runtime_submit_turn` 偶发在 `60s` 前端超时并返回 DevBridge timeout，且本地 `aster/state/logs` 没有对应 LLM request 新日志，说明失败发生在提交链路/bridge 阻塞或运行时排队阶段，未进入 provider 首 token；这是后续首字慢优化的下一刀证据。
- `npm test -- "src/components/agent/chat/hooks/agentStreamRuntimeHandler.test.ts"` 通过（10 tests），新增覆盖 thinking 关闭时不渲染 reasoning_delta。
- `npm test -- "src/components/agent/chat/hooks/agentStreamTurnEventBinding.test.ts" "src/components/agent/chat/hooks/agentStreamSubmitExecution.test.ts"` 通过（7 tests）。
- `npx eslint "src/components/agent/chat/hooks/agentStreamRuntimeHandler.ts" "src/components/agent/chat/hooks/agentStreamTurnEventBinding.ts" "src/components/agent/chat/hooks/agentStreamSubmitExecution.ts" "src/components/agent/chat/hooks/agentStreamRuntimeHandler.test.ts" --max-warnings 0` 通过。
- `npm run typecheck` 通过。
- `npm run verify:gui-smoke` 通过；本轮临时 target 首次编译耗时较长，随后 DevBridge / workspace-ready / browser-runtime / site-adapters / service-skill-entry / runtime tool-surface page smoke 均通过。
docs/roadmap/agentui/README.md
+55
View File
@@ -0,0 +1,55 @@
# Lime AgentUI 路线图文档
> 状态：路线图与架构设计
> 更新时间：2026-04-30
> 范围：Lime 对话工作区下一阶段 AgentUI，包括 UI 架构、代码层级、事件流程、时序图、后端协作与落地顺序。
## 目标
AgentUI 不是再做一个聊天页面，而是把 Lime 已有的 runtime、timeline、artifact、task、team、harness、evidence 能力收束成一个可观察、可控制、可交付的工作台。
本目录回答四类问题：
1. **产品结构**：Lime 的 AgentUI 应该由哪些层组成，哪些信息应该出现在首屏，哪些应该进入展开详情。
2. **代码结构**：现有前端、协议、Tauri command、Rust runtime、service、持久化层分别负责什么。
3. **运行流程**：发送消息、打开旧会话、排队输入、权限确认、产物生成、证据导出如何流动。
4. **落地顺序**：哪些改动先解决体感慢、卡顿、重复吐字和多任务管理，哪些进入中长期演进。
## 阅读顺序
| 文档 | 作用 |
| --- | --- |
| [agent-ui-research-and-lime-direction.md](agent-ui-research-and-lime-direction.md) | 竞品与本地参考调研，说明为什么 Lime 要走“对话 + 过程 + 任务 + 产物 + 证据”路线。 |
| [lime-agentui-target-architecture.md](lime-agentui-target-architecture.md) | 目标 UI 架构图与五层模型，是后续 UI 改造的总图。 |
| [lime-agentui-code-map.md](lime-agentui-code-map.md) | Lime 当前代码层级地图，标出前端、协议、后端、服务和测试入口。 |
| [lime-agentui-event-flow.md](lime-agentui-event-flow.md) | 关键流程图，包括发送消息、旧会话恢复、queue/steer、权限、artifact、evidence。 |
| [lime-agentui-sequence-diagrams.md](lime-agentui-sequence-diagrams.md) | 端到端时序图，适合实现和排查首字慢、恢复慢、流式错乱。 |
| [lime-agentui-backend-coordination.md](lime-agentui-backend-coordination.md) | 后端配合代码架构，定义 UI 需要后端继续补齐的投影、分页、指标与批量接口。 |
| [lime-agentui-implementation-roadmap.md](lime-agentui-implementation-roadmap.md) | P0/P1/P2/P3 落地顺序、验收标准和验证命令。 |
## 当前结论
Lime AgentUI 的主线应保持一个事实源，不新增第二套事件系统：
```text
Agent runtime event
-> session / timeline / thread_read / artifact / evidence projection
-> frontend state
-> Conversation / Process / Task / Artifact / Evidence UI
```
下一阶段 UI 的关键词不是“更像某个竞品”，而是：
- **首屏轻**：旧会话打开先展示 shell、缓存快照、最近消息，再渐进补 timeline / tool / artifact。
- **流式稳**：text、thinking、tool、artifact、runtime status 分型渲染，防止重复吐字和正文污染。
- **任务可压缩**：运行中、排队、needs input、plan ready、failed 统一进入 capsule / task strip。
- **产物离开正文**：最终交付进入 Artifact / Canvas / Workbench，聊天正文负责解释和协作。
- **证据可追溯**：harness、evidence、review、replay 消费同一条 runtime/timeline 事实链。
## 设计约束
1. 不复制 Claude Code、Warp、CodexMonitor 或 Codex TUI 的表面视觉，只借鉴结构模式。
2. 不把过程日志塞回最终回答正文。
3. 不让旧会话恢复阻塞 UI 挂载。
4. 不让 sidebar list、session detail、timeline build、artifact preview 在同一时刻抢主线程和 invoke 通道。
5. 不新增 parallel runtime/event 协议；新增 UI 只消费现有 AgentEvent、timeline、thread_read、artifact、evidence 投影。
docs/roadmap/agentui/agent-ui-research-and-lime-direction.md
+645
View File
@@ -0,0 +1,645 @@
# AI Agent UI 调研与 Lime 对话体验路线图
> 状态：调研与路线图
> 更新时间：2026-04-30
> 目标：把“AI Agent 最佳 UI 是什么”从竞品印象、局部控件和聊天气泡，收敛成 Lime 可执行的对话体验路线图。
> 结论：Lime 不应做纯聊天界面，也不应复制 Claude Code 或 Warp；Lime 应走“对话 + 过程 + 任务 + 产物 + 证据”的可执行工作区 Agent UI。
## 1. 核心判断
当前 AI Agent UI 已经从“输入框 + 消息列表”演进成复合工作台。最好的产品不再只回答“模型说了什么”，而是同时回答：
1. 用户现在在做什么目标？
2. Agent 正在什么阶段？
3. 哪些工具、权限、成本、文件、网页和子代理被使用了？
4. 哪些内容是最终产物，哪些只是推理过程？
5. 结果是否可追溯、可重放、可继续编辑？
这和 Lime 的方向高度一致。Lime 已经有 workspace、artifact、timeline、task、runtime、browser、service skill、team/subagent、evidence pack 等底座。真正要补的不是一个更像 ChatGPT 的聊天页，而是把这些底座组织成清晰、低负担、可长时间使用的 Agent 工作台。
本文档的产品结论是：
- **Chat 负责意图、推理与协作。**
- **Capsule 负责执行状态、后台任务和多 Agent 队列。**
- **Timeline 负责过程证据。**
- **Artifact 负责最终交付。**
- **Evidence 负责复盘、验证和治理。**
一句话：Lime 应当让 Agent 像一个可观察、可控、可交付的工作系统，而不是一个不断吐字的文本框。
## 2. 调研范围
### 2.1 商业产品与官方资料
| 产品 / 资料 | 观察重点 | 对 Lime 的启发 |
| --- | --- | --- |
| OpenAI Agents / AgentKit / ChatKit | Agent 构建、工具、会话组件、任务链路 | Chat UI 应该消费结构化 agent event，而不是只消费文本流 |
| ChatGPT Canvas | 对话之外的编辑画布 | 产物应离开聊天正文，进入可编辑的 Artifact / Canvas |
| ChatGPT Projects / Tasks | 项目上下文、长期任务、跨会话组织 | Lime 的 workspace/session/tab 需要成为一等导航对象 |
| Claude Artifacts | 回复旁路展示代码、文档、图表等产物 | “正文解释 + 右侧产物”比把全部内容塞进消息更稳定 |
| Claude Extended Thinking | 思考内容可控展示 | thinking 应折叠、摘要化、可关闭，不能污染最终答案 |
| Claude Code | 胶囊式后台任务、工具结果压缩、计划审批 | Lime 可借鉴 pill / plan / tool rich output，但不复制终端 UI |
| Gemini Deep Research | 多步研究进度、来源、最终报告 | 研究型任务需要进度、来源和报告产物分离 |
| Gemini Canvas | 对话与创作空间组合 | 编辑型任务应进入持续画布，而不是每次生成一段 Markdown |
| Cursor / Windsurf | IDE 内 agent、工具调用、diff、上下文 | 代码任务 UI 的核心是可审查变更和可恢复控制 |
| Warp Agent Mode | 终端 block、AI block、上下文命令 | 命令和 AI 过程适合 block 化，而不是混成纯聊天流 |
### 2.2 Context7 开源与 SDK 资料
| 资料 | 关键模式 | 对 Lime 的取舍 |
| --- | --- | --- |
| `assistant-ui` | Thread、ThreadList、Composer、Message parts、ActionBar、branch、copy、reload、feedback、attachment、lazy list | 可参考 primitives 分层和 lazy list 思路，不直接引入整套视觉 |
| `Vercel AI SDK` | UIMessage parts、reasoning part、tool part、data part、custom stream、tool progress | Lime 应继续把 text / thinking / tool / status / artifact 映射为结构化 part |
| `CopilotKit` | CopilotChat、Generative UI、Human-in-the-loop、frontend action、shared agent state | Lime 的 action_required、权限确认、工作区状态更新应作为 HITL UI，而不是普通文本 |
### 2.3 本地参考项目
主要参考 `/Users/coso/Documents/dev/js/claudecode`，辅助参考 `/Users/coso/Documents/dev/rust/warp`、`/Users/coso/Documents/dev/rust/CodexMonitor` 与 `/Users/coso/Documents/dev/rust/codex`。
Claude Code 本地源码显示三类尤其值得借鉴：
1. `src/tasks/pillLabel.ts` 把后台任务压缩为短标签，例如 shell、monitor、team、local agent、cloud session、ultraplan、dreaming。只有 `needs_input` 和 `plan_ready` 这种注意力状态才提示 CTA。
2. `src/tools/MCPTool/UI.tsx` 对工具输入输出做分层：非 verbose 截断输入、progress bar、大输出 token warning、图片占位、空输出提示、小 JSON 扁平展示、dominant text payload 解包。
3. `src/tools/ExitPlanModeTool/UI.tsx` 把计划流做成明确状态：退出计划、提交审批、用户批准、保存路径、拒绝计划，而不是只渲染一段 Markdown。
Warp 本地源码与文档显示三类可借鉴方向：
1. Terminal block / AI block 把命令、输出、AI 解释组织为可选择、可复制、可恢复的块。
2. Markdown visuals、Mermaid、本地图片渲染说明 AI 输出不应只是纯文本。
3. Agent Mode 的价值在于贴近工作上下文，而不是开一个独立聊天网页。
CodexMonitor 更接近 Lime 当前桌面 GUI 形态，尤其值得补充：
1. 多 workspace / thread 管理：支持 recent agent activity、unread/running 状态、pin/rename/archive/copy、resume thread 和 remote backend。对 Lime 的启发是 session/tab 不只是列表，而是多工作区执行态索引。
2. Composer 控制层：支持 Queue vs Steer、模型选择、collaboration mode、reasoning effort、access mode、context usage ring、图片附件、dictation、skills / prompts / review / file path autocomplete。对 Lime 的启发是输入区应显式承载“执行意图 + 模型/权限/上下文预算”，不是只有 textarea。
3. 消息渲染层：reasoning、tool、diff、review、userInput、plan follow-up 分成不同 row，tool output 做窗口化末尾展示，图片用 lazy thumbnail + lightbox。对 Lime 的启发是 process item 必须分型渲染。
4. App-server event reference 把支持/缺失事件列成清单，并记录 `item/reasoning/*`、`item/commandExecution/*`、`turn/diff/updated`、`thread/status/changed` 等事件。对 Lime 的启发是 Agent UI 需要协议覆盖表，避免 UI 靠猜事件。
Codex TUI 则提供了更底层的交互与性能经验：
1. `docs/tui-chat-composer.md` 把 composer 当成状态机：popup、slash、file、skill、paste burst、history search、local/persistent history、image rows、draft recovery 分开处理。
2. `docs/tui-request-user-input.md` 把用户输入请求做成 overlay：一题一页、option + freeform notes、焦点路由、紧凑布局降级。对 Lime 的 `action_required` / elicitation UI 很直接。
3. `docs/tui-stream-chunking-*.md` 把流式展示拆成 Smooth / CatchUp 模式，并用 queue depth、oldest age、mode transition、rapid re-entry 等指标验证。对 Lime 的首字慢、流式卡顿、CPU 飙高有直接借鉴价值。
4. `status_indicator_widget.rs`、`unified_exec_footer.rs`、`pending_input_preview.rs` 展示了状态行、后台 terminal 摘要、pending steer / queued messages 预览的压缩方式。对 Lime 的 capsule / queue UI 有参考意义。
### 2.4 Lime 当前事实源
Lime 不是从零开始。当前仓库已有基础：
- `src/components/agent/chat/types.ts` 已有 `ContentPart`：`text`、`thinking`、`tool_use`、`action_required`。
- `src/components/agent/chat/components/StreamingRenderer.tsx` 已有 thinking block、content parts、runtime status、流式渲染与历史轻量渲染入口。
- `src/components/agent/chat/components/MessageList.tsx` 已有首 token placeholder、runtime status pill、timeline 分离、历史 timeline 延迟构建、长历史窗口。
- `docs/roadmap/task/event-chain.md` 已定义事件链方向：不要新造第二条事件系统，应在现有 `AgentEvent -> timeline -> thread_read -> evidence / RequestLog` 主链上补齐任务、路由、经济事件。
- `docs/roadmap/artifacts/roadmap.md` 已明确 Artifact 方向，并包含核心原则：`Chat for reasoning, Artifact for delivery`。
- `docs/roadmap/harness-engine/README.md` 已把 evidence pack、replay、analysis、review、UI 展示纳入同源治理。
- `docs/roadmap/warp/README.md` 已明确 Agent loop、tool protocol、permission、slash、SkillTool、subagent 参考 Claude Code。
这意味着 Agent UI 的下一步不是再加一个 parallel UI，而是把现有结构化事实源的展示层收束起来。
## 3. Agent UI 模式全景
### 3.1 输入区 Composer
| 模式 | 代表形态 | 适用场景 | Lime 建议 |
| --- | --- | --- | --- |
| 底部吸附输入框 | ChatGPT、Claude、assistant-ui | 长对话、持续问答 | 保留，作为默认主入口 |
| 浮动胶囊输入框 | 新建任务、命令模式、轻量 prompt | 首页、空态、新任务 | 适合 Lime 首页和新建对话，但不应覆盖工作区信息 |
| Slash command | Claude Code、Cursor、Warp | 精确触发模式、工具、模板 | 必须支持，且映射到 skill / command / task contract |
| Mention / context chip | Cursor、Windsurf、Copilot 类产品 | 引用文件、网页、任务、会话 | 适合 Lime workspace、artifact、browser、session 引用 |
| 附件托盘 | ChatGPT、Claude、Gemini | 多模态输入、文档、图片 | 应继续做成输入前结构化 context，不只显示文件名 |
| 模型 / Provider 选择 | 多模型产品 | 高级用户、成本控制 | 可以是紧凑 selector，不要干扰普通输入 |
| 权限 / 模式 chip | plan、act、safe、browser、write | 高风险执行前约束 | 应与 execution profile / permission policy 绑定 |
| Queue / Steer 切换 | CodexMonitor、Codex TUI | 当前 turn 运行中继续补充输入 | Lime 应明确区分“排队下一条”和“转向当前 turn” |
| Context usage ring | CodexMonitor | 长会话、上下文预算、压缩前提示 | Lime 应把上下文预算做成低调但常驻的预算信号 |
| History search / draft recovery | Codex TUI | 长期高频输入、误清空恢复 | Lime 可做 P1，避免新建/切换/取消时丢 draft |
| 语音 / 截图 / 拖拽 | 通用 AI 助手 | 快速输入、视觉任务 | 可作为 P2，不应阻塞主链 |
输入区最重要的不是“做成胶囊”本身，而是让用户在发出任务前就能看见：当前上下文、执行模式、目标 workspace、模型/成本策略和权限边界。
Codex TUI 的 composer 状态机对 Lime 很有价值：多行粘贴、IME、slash command、附件 placeholder、历史召回、外部编辑器回填都不应靠 textarea 事件临时判断。Lime 如果继续强化 composer，应把这些行为沉到可测试的状态机，而不是在组件里继续堆条件分支。
### 3.2 消息展示 Message
| 模式 | 优点 | 风险 | Lime 建议 |
| --- | --- | --- | --- |
| 用户气泡 + Assistant 正文 | 清晰区分双方 | 长输出易拥挤 | 默认采用 |
| 双侧气泡 | 聊天感强 | 对桌面工作台效率低 | 不作为主体验 |
| Assistant 无气泡正文 | 文档阅读舒服 | 状态和工具容易混入正文 | 适合最终回答 |
| 分段消息 parts | 可结构化渲染 text/tool/thinking/source | 需要事件协议稳定 | Lime 已有基础，应强化 |
| Branch / 版本切换 | 支持重试和多答案 | 增加复杂度 | 对关键回合 P1 引入 |
| Message action bar | copy/edit/retry/feedback | 操作过多会压视觉 | 保留低调操作栏 |
| Inline citation | 研究、网页、证据 | 无来源时容易假装严谨 | 只对真实 source/evidence 显示 |
Lime 的消息层应做到：最终答案干净、过程信息可见但不抢位、操作入口稳定、历史恢复轻量。
### 3.3 Thinking / Reasoning UI
| 模式 | 表现 | 优点 | 风险 | Lime 建议 |
| --- | --- | --- | --- | --- |
| 完全隐藏 | 只显示最终答复 | 简洁 | 用户不知道是否卡住 | 不作为唯一模式 |
| “正在思考”状态 | 文本或 spinner | 首字前有反馈 | 解释力弱 | 必须有，用于首 token 前 |
| 折叠 thinking block | 默认显示摘要/耗时，可展开 | 平衡透明与清爽 | 摘要质量要稳定 | P0 推荐 |
| 实时 reasoning step | 一步步显示 | 强透明 | 易泄漏、噪声大、卡顿 | 只对支持模型和调试模式 |
| 完成后摘要 | “思考了 N 秒 / 检查了 X 项” | 历史阅读友好 | 可能过度拟人化 | 适合默认态 |
| 原文可展开 | 展示模型 reasoning 内容 | 方便调试 | 不同模型政策不同 | 仅在 provider 明确支持且用户开启 |
| Sources-first reasoning | 先展示来源/证据，再展示结论 | 研究型任务可信 | 对普通任务太重 | 用于 research / browser / evidence 场景 |
Lime 的原则：thinking 是过程层，不是正文层。它可以解释“正在发生什么”，但不能与最终答案重复吐字，也不能把 `<think>`、`thinking_delta` 或工具日志混进 Markdown 正文。
### 3.4 Runtime Status UI
| 状态 | UI 表现 | 说明 |
| --- | --- | --- |
| submitted | 首 token placeholder | 明确“请求已提交，等待首个响应” |
| routing | 阶段 pill | 显示模型/路由/队列决策，不刷屏 |
| streaming | 轻量流式光标 | 不用大 spinner 打断阅读 |
| tool_running | 工具 pill + 可展开详情 | 正文不直接塞工具输出 |
| action_required | 高亮确认卡 | 必须有清晰 CTA |
| retrying | 小型重试状态 | 显示次数和原因 |
| failed | 可恢复错误卡 | retry / copy diagnostic / switch provider |
| cancelled | 安静的取消状态 | 不应显示成失败 |
| completed | 状态收起 | 保留摘要，避免历史噪声 |
首字慢的体感往往来自“用户完全看不到系统是否活着”。Runtime Status UI 的目标不是假装更快，而是在真实首 token 前给出低成本、可信的阶段反馈。
Codex TUI 的 `StatusIndicatorWidget` 说明状态行应把动画、耗时、interrupt hint 和短详情放在同一条稳定表面中，避免布局跳动。Lime 的 runtime status 可以吸收这个结构：一行显示阶段、耗时和可取消动作，详情最多 2 到 3 行，超出进入 timeline 或诊断抽屉。
### 3.5 Tool Call UI
| 模式 | 适用 | Lime 建议 |
| --- | --- | --- |
| 行内工具卡 | 单个小工具、读文件、短命令 | 用于当前 turn 的关键步骤 |
| 批量摘要 | 多个工具、重复搜索、长列表 | 默认压缩，避免刷屏 |
| 详情抽屉 | 大输出、调试、审计 | 与 timeline/evidence 连接 |
| 输入截断 | 参数很长 | 默认显示关键字段，提供展开 |
| 输出预览 | 小 JSON、短文本、图片 | rich output 解析后展示 |
| 大输出警告 | token 很大 | 借鉴 Claude Code，提示上下文影响 |
| 空输出提示 | 工具无内容 | 显示“无输出”，不要留白 |
| 可重跑 | 幂等工具、失败工具 | 高风险工具需要确认 |
| 来源链接 | browser/search/doc | 与 citation/evidence 同源 |
Claude Code 的 MCP UI 给 Lime 一个清晰信号：tool UI 的好坏不在于“展示全部原始数据”，而在于分层、截断、摘要、异常提示和可追溯。
### 3.6 Capsule / Pill UI
胶囊式 UI 是本次调研里最适合 Lime 形成特色的模式。它不应只是装饰，而应成为任务状态入口。
| 胶囊类型 | 示例 | Lime 映射 |
| --- | --- | --- |
| 本地命令 | `1 shell`、`1 monitor` | terminal / command runtime |
| 后台 terminal | `1 background terminal running` | command / terminal dock |
| 后台 workflow | `1 background workflow` | task runtime |
| 本地 Agent | `1 local agent` | local runtime / subagent |
| 云端 Agent | `◇ 1 cloud session` | remote runtime / cloud scene |
| Team | `1 team` | team/subagent orchestration |
| Plan ready | `◆ plan ready` | plan approval / ExitPlanMode |
| Needs input | `◇ needs your input` | action_required / elicitation |
| Dreaming / background idea | `dreaming` | 可选 P2，不作为主链 |
胶囊应该遵循三条规则：
1. **普通 running 只显示事实，不打扰。**
2. **needs input / plan ready / failed 才抢注意力。**
3. **点击胶囊展开任务中心或 timeline，不跳出当前上下文。**
这也能解决多 tab、多会话和后台 agent 带来的卡顿：用户看到的是压缩后的任务索引，而不是所有会话全量渲染。
CodexMonitor 与 Codex TUI 都支持“当前 turn 运行中继续输入”：一个偏 GUI 队列，一个偏 TUI pending input preview。Lime 可以把这类输入合并进胶囊/队列层：pending steer 显示为“将注入当前 turn”，queued follow-up 显示为“本轮后发送”，并允许编辑/删除。
### 3.7 Plan / Human-in-the-loop UI
AI Agent 越能执行，计划和审批 UI 越重要。最佳实践不是让模型说“我准备这么做”，而是把计划变成可批准、可拒绝、可编辑、可追踪的对象。
| 场景 | UI 形态 | Lime 建议 |
| --- | --- | --- |
| 普通计划 | Markdown plan + steps | 可读即可 |
| 高风险计划 | approval card | 必须有批准 / 拒绝 |
| 团队审批 | waiting for lead | 显示审批人/状态 |
| 计划被拒绝 | rejected plan message | 保留理由并可继续编辑 |
| 计划保存 | saved path / artifact | 不只留在聊天正文 |
| 权限升级 | danger confirmation | 与 command policy 统一 |
| 成本确认 | cost / model estimate | 与 task/model routing 统一 |
Lime 已有 `action_required` 和 timeline，可以把 HITL 从“文本解释”升级为“状态明确的交互卡”。
### 3.8 Artifact / Canvas UI
Agent UI 的关键分水岭是：最终产物是否离开聊天正文，进入可继续工作的表面。
| 模式 | 代表 | Lime 建议 |
| --- | --- | --- |
| Inline preview | ChatGPT 小预览、tool result preview | 适合短结果 |
| 右侧 Canvas | Claude Artifacts、ChatGPT Canvas、Gemini Canvas | Lime 应强化为主交付面 |
| 文件树 / 产物列表 | IDE agent、workspace 产品 | 适合多文件、多版本 |
| Diff view | Cursor、Windsurf | 代码/文档修改必须支持 |
| Version history | Artifacts / docs | Artifact Workbench P1/P2 |
| Source drawer | 研究报告、网页任务 | 与 timeline/evidence 同源 |
| Export / share | 文档、图片、报告 | 使用桌面保存与 artifact export 主链 |
Lime 的既有路线图已经说清楚：`Chat for reasoning, Artifact for delivery`。Agent UI 文档应把这条原则提升为所有交互设计的第一约束。
### 3.9 Timeline / Evidence UI
Timeline 和 Evidence 是 Lime 可以走出自己特色的地方。多数竞品只显示“看起来可信”的过程，Lime 可以显示真正可复查的运行事实。
| 层级 | 目的 | 展示方式 |
| --- | --- | --- |
| Inline process | 当前 turn 发生了什么 | 少量关键状态 |
| Turn timeline | 工具、thinking、artifact、action | 折叠时间线 |
| Session timeline | 多回合运行历史 | 默认懒加载 |
| Evidence pack | 可导出事实包 | Harness / review 面板 |
| Replay | 复现问题 | 调试/验证入口 |
| Request log | 模型、provider、路由、成本 | 诊断面 |
重要边界：不是所有事件都要直接展示给用户。事件可以进入事实源，UI 只投影必要摘要。否则 timeline 会变成另一种刷屏。
### 3.10 Multi-agent / Team UI
多 Agent UI 不应一开始做成复杂组织图。先从任务状态和责任边界做起。
| 模式 | 使用时机 | Lime 建议 |
| --- | --- | --- |
| 成员胶囊 | 多个子代理并行 | 顶部/底部状态压缩 |
| 子任务列表 | 可拆解任务 | 任务中心展示 |
| Handoff 卡 | 子代理交接结果 | 与 evidence / artifact 绑定 |
| Leader approval | 需要审批 | 计划 UI 的扩展 |
| Queue view | 多任务排队 | 避免全量 tab 同时渲染 |
| Per-agent trace | 调试失败 | 默认折叠 |
Lime 的 team/subagent 特色应该是“可观察、可交接、可验证”，而不是多几个头像。
### 3.11 Browser / Research UI
研究类 Agent 需要的 UI 与普通问答不同。
| 模式 | 目的 | Lime 建议 |
| --- | --- | --- |
| 查询计划 | 解释将查什么 | 可折叠 |
| 来源列表 | 可信度和复查 | 与 citation/evidence 同源 |
| 浏览器快照 | 证明页面状态 | browser runtime artifact |
| 进度步骤 | 多轮搜索 | capsule + timeline |
| 最终报告 | 交付物 | artifact document |
| 引用回跳 | 从报告到来源 | artifact source drawer |
Gemini Deep Research 的核心不是“搜得多”，而是把研究过程、来源和最终报告分层。Lime 如果结合 browser runtime 和 evidence pack，可以做得更扎实。
### 3.12 Tab / Session UI
用户已经多次反馈：多会话、多 tab 和旧会话恢复会导致卡顿。这里的 UI 不能只学聊天产品，要学习浏览器。
| 浏览器式概念 | Agent 会话映射 | 目的 |
| --- | --- | --- |
| Active tab | 当前可交互会话 | 全量渲染 |
| Recent tab | 最近使用会话 | 轻量快照 |
| Pinned tab | 固定重要任务 | 不被自动丢弃 |
| Suspended tab | 暂停渲染但保留状态 | 降低 CPU/内存 |
| Discarded tab | 释放重资源，保留恢复入口 | 控制长时间使用成本 |
| Restore | 重新加载窗口历史 | 渐进恢复 |
| Tab group | 项目/任务分组 | workspace 维度组织 |
| Remote backend thread | 远端机器上的 Codex session | 显示连接、运行、未读、恢复状态 |
| Worktree / clone agent | 隔离工作区任务 | 作为 session 派生关系，而不是普通平铺 tab |
Lime 的多会话体验应避免“打开一个历史对话就把所有东西恢复出来”。旧会话默认显示窗口化消息、轻量 timeline 摘要和 artifact 索引；只有用户展开时才加载完整历史和细节。
CodexMonitor 的 thread 管理提醒我们：session UI 还要表达 pin、archive、copy、resume、running、unread、remote backend 和 worktree 派生关系。Lime 不宜只做浏览器标签外观，而应把 tab 当成“可恢复执行单元”的索引。
### 3.13 Error / Recovery UI
Agent 产品一定会失败，好的 UI 不是隐藏失败，而是让失败可恢复。
| 错误类型 | UI 方式 |
| --- | --- |
| Provider credential | 指向设置页，保留当前输入 |
| Rate limit | 显示等待/换模型/重试 |
| Tool permission denied | 展示被拒绝动作和替代路径 |
| Tool timeout | 可重跑、可取消、可导出日志 |
| Empty output | 明确无输出，不留白 |
| Bad formatting | 提供重新渲染/原文查看 |
| Stream interrupted | 保留已生成内容，允许继续 |
| Old session corrupt | 轻量恢复 + 诊断导出 |
错误态必须和 runtime/evidence 绑定，不能只在 toast 里一闪而过。
### 3.14 Performance UI
性能不是纯工程问题，也是一种 UI 设计。
| 模式 | 解决问题 | Lime 建议 |
| --- | --- | --- |
| Skeleton | 首屏等待 | 打开旧会话先挂 UI |
| First token placeholder | 首字前无反馈 | 已有基础，继续强化 |
| Progressive hydration | 旧会话重 | 先文本，后 timeline/tool |
| Virtual list | 消息多 | 长历史必须使用窗口化 |
| Lazy timeline | threadItems 重 | 默认摘要，展开再构建 |
| Suspended tab | 多会话卡顿 | 浏览器式 session 管理 |
| Output cap | 工具输出巨大 | 预览 + warning + 详情 |
| Worker parsing | Markdown/timeline 重 | P2 可引入 |
| Adaptive stream chunking | 流式 backlog 和 UI 卡顿 | Smooth / CatchUp 策略，按队列压力切换 |
| Queue pressure metrics | 流式调优无证据 | 记录 queue depth、oldest age、mode transition、rapid re-entry |
| Stable status row | 运行态布局抖动 | 状态行固定高度，详情有最大行数 |
“看起来慢”经常来自三个源头：首 token 前无反馈、历史恢复同步计算、后台会话全量渲染。Agent UI 路线图必须把性能策略写进交互，而不是只靠后端优化。
Codex TUI 的流式分块文档给 Lime 一个可执行验证方向：不要只观察“首字时间”，还要记录 UI 消费流的 backlog。至少应能看到每次渲染提交的队列深度、最老未渲染 delta 年龄、是否进入 catch-up、是否频繁抖动切换。这样才能解释“模型已经吐了但 UI 还慢”。
## 4. Lime 应该形成的 UI 架构
### 4.1 五层模型
Lime Agent UI 建议按五层组织：
| 层 | 负责 | 不负责 |
| --- | --- | --- |
| 对话层 | 用户意图、最终回答、少量协作文本 | 展示全部工具日志 |
| 过程层 | thinking、runtime status、tool summary、timeline 摘要 | 承载最终产物 |
| 任务层 | 胶囊、队列、后台任务、子代理、plan approval | 替代消息列表 |
| 产物层 | Artifact、Canvas、文件、diff、版本、导出 | 解释所有推理 |
| 证据层 | evidence、replay、request log、review、diagnostic | 日常默认展开 |
这五层对应 Lime 的现有能力，不需要新造第二套 runtime。关键是让 UI 只消费同一条事实链的不同投影。
### 4.2 信息优先级
默认屏幕优先级应为：
1. 当前用户目标和输入区。
2. 当前 active turn 的可读状态。
3. 最终回答或 artifact 交付物。
4. 需要用户介入的 action。
5. 后台任务胶囊。
6. 可展开过程和证据。
不应默认抢占注意力的内容：
- 全量 thinking。
- 全量 tool JSON。
- 完整 request log。
- 历史 turn 的所有 timeline 细节。
- 已完成后台任务的长日志。
### 4.3 Lime 视觉语言约束
根据 `docs/aiprompts/design-language.md`，Agent UI 应保持：
- 轻盈、清晰、专业，信息优先。
- 桌面应用感强，避免网页营销风格。
- 主表面使用实体底色，不依赖半透明和磨砂。
- 胶囊可以使用小圆角状态标签，但不要把所有内容都做成卡片。
- 状态色保持语义稳定：成功 emerald，提醒 amber，信息 sky/slate，错误 rose/red。
- 中文排版优先，不使用过大的英文 tracking 破坏阅读。
这意味着 Lime 可以使用胶囊式 UI，但胶囊必须是状态语义，不是视觉噱头。
## 5. 适合 Lime 的优先级
### P0：对话主链 UI 收敛
目标：解决旧会话卡顿、首字慢、重复吐字、排版污染这些用户已经感知到的问题。
建议动作：
1. `thinking`、`tool_use`、`action_required` 与最终正文严格分层。
2. 首 token 前始终显示可信 runtime status，不让用户面对空白等待。
3. tool 输出默认摘要化，正文只保留必要解释。
4. 历史消息默认轻量渲染：文本优先，timeline/tool 详情延后。
5. 长输出和大 JSON 必须有 preview、warning 和详情入口。
验收：
- DeepSeek / reasoning 模型不会把思考内容重复吐到正文。
- 打开旧会话不会触发长时间鼠标 loading 和明显 CPU 飙高。
- 用户能在 1 秒内看到“请求已提交/正在路由/正在等待模型”等阶段反馈。
- 工具执行不会把正文排版挤乱。
### P1：执行胶囊层
目标：让后台任务、多 Agent、计划审批和 needs input 有统一入口。
建议动作：
1. 引入任务胶囊状态模型：running、queued、needs_input、plan_ready、failed、completed。
2. 普通 running 低调显示；needs_input / plan_ready / failed 明确提示。
3. 胶囊点击展开任务中心或当前 session timeline。
4. 胶囊与 tab/session 管理联动，避免所有会话全量恢复。
验收：
- 多个后台任务不会刷屏。
- 用户能一眼知道哪个任务需要介入。
- 打开多个历史对话时，非 active 会话不持续吃 CPU。
### P1：Artifact-first 交付路径
目标：让最终产物离开聊天正文，进入可编辑、可版本化、可导出的工作台。
建议动作：
1. 对文档、代码、报告、图片、网页研究等结果默认创建 artifact。
2. 聊天正文只解释意图、摘要和下一步。
3. Artifact 与 timeline item 双向跳转。
4. 工具来源、引用和 evidence 能回挂到 artifact source drawer。
验收：
- 长报告不再只是一大段聊天 Markdown。
- 用户可从 artifact 找到生成它的 turn、tool、source。
- 修改、重写、导出走 Artifact Workbench，而不是重发一段聊天。
### P2：浏览器式 Session / Tab 管理
目标：解决多会话长期运行的性能和导航问题。
建议动作：
1. Session tab 引入 active、recent、pinned、suspended、discarded 状态。
2. 非 active 历史会话只保留标题、摘要、任务胶囊和 artifact 索引。
3. 恢复旧会话采用 progressive hydration。
4. pinned session 不被自动丢弃，普通 session 可按内存压力降级。
验收：
- 同时打开多个历史对话不会线性增加主线程压力。
- 切回旧会话先显示可读快照，再渐进恢复细节。
- 新建任务不会被旧会话恢复阻塞。
### P2：Team / Evidence 工作台
目标：让 Lime 的团队代理和证据链成为差异化能力。
建议动作：
1. Team/subagent 先以胶囊和任务列表呈现，不急于做复杂图谱。
2. Handoff 结果进入 timeline 和 artifact。
3. Evidence pack 和 replay 入口只在需要诊断、审核、验证时展开。
4. Harness 面板与对话工作台共享同一份 verification facts。
验收：
- 多 Agent 执行能看出责任边界。
- 失败时能导出证据，而不是让用户截图聊天。
- Review / replay / UI 不出现事实漂移。
## 6. 推荐路线图
### Phase 1：对话主链稳定化
交付内容：
- thinking / final text / tool / action_required 的展示边界收紧。
- 首 token placeholder 与 runtime status 统一。
- 历史恢复默认轻量渲染。
- tool rich output 支持截断、预览、大输出警告、空输出提示。
- 流式输出引入 UI backlog 指标，区分模型慢、invoke 慢和前端渲染慢。
- E2E 覆盖新建对话、旧会话、流式、thinking、tool 输出、错误恢复。
这是最先做的一刀，因为它直接对应当前用户痛点：旧会话慢、首字慢、重复吐字、排版不稳。
### Phase 2：胶囊任务层
交付内容：
- 任务胶囊状态模型。
- 胶囊栏或状态区。
- needs input / plan ready / failed 的 attention CTA。
- 胶囊点击进入任务中心或 timeline。
- 后台任务与 session/tab 状态关联。
这一刀会把 Claude Code 的 pill 思路转化为 Lime 自己的桌面工作台语言。
### Phase 3：Artifact 与 Timeline 双向增强
交付内容：
- artifact 默认承载长产物。
- timeline item 可打开 artifact 定位。
- artifact source drawer 可回跳 source / tool / evidence。
- report / code / image / browser snapshot 等 domain artifact 的 viewer 收敛。
这一刀让 Lime 从“聊天里有产物”变成“围绕产物协作”。
### Phase 4：浏览器式会话管理
交付内容：
- active / recent / pinned / suspended / discarded session 状态。
- 非 active session 的资源降级。
- 旧会话 progressive hydration。
- 多 tab 下的 CPU、内存、主线程指标回归。
这一刀专门解决“对话多了以后卡”的长期问题。
### Phase 5：Team / 多 Agent 工作台
交付内容：
- team/subagent 胶囊。
- 子任务列表和 handoff 卡。
- leader approval / plan approval。
- evidence / replay / verification 与 UI 联动。
这一刀把 Lime 的长期特色显性化：不是单个聊天助手，而是可治理的 Agent 工作系统。
## 7. 验收标准
### 7.1 用户体验验收
1. 新建对话可以快速打开，不被旧会话恢复阻塞。
2. 打开旧会话先出现可读内容，再渐进恢复细节。
3. 首 token 前有可信状态，不出现长时间空白。
4. thinking 不与最终答案重复。
5. tool 输出不会破坏正文排版。
6. action_required 有明确 CTA。
7. 后台任务有胶囊入口，且注意力状态清晰。
8. 长报告、代码、图片、研究结果进入 artifact。
9. artifact 能回跳来源、timeline 和 evidence。
10. 多个 tab/session 长时间存在时 CPU 和内存可控。
### 7.2 工程验收
1. 不新增第二套事件系统。
2. UI 消费 `AgentEvent -> timeline -> thread_read -> evidence` 主链投影。
3. `ContentPart` 继续作为消息结构化 part 的前端边界。
4. tool / thinking / action / artifact 的渲染有稳定测试。
5. GUI 主路径补 Playwright / smoke 验证。
6. 旧会话性能改动有 E2E 或性能日志证明。
7. 文档、代码和测试都不扩展 deprecated / compat 路径。
## 8. 明确不做
1. 不直接复制 Claude、ChatGPT、Gemini、Cursor、Warp 的视觉。
2. 不把所有 tool 输出塞进聊天正文。
3. 不把 thinking 当作最终答案的一部分。
4. 不新增与 timeline/evidence 平行的第二套事实系统。
5. 不为了“胶囊好看”把所有状态都做成 pill。
6. 不默认全量恢复每个历史会话。
7. 不为了展示透明度牺牲旧会话性能。
8. 不用营销式 hero 或大面积装饰背景包装 Agent 工作台。
## 9. 参考资料
### 9.1 官方与公开资料
- OpenAI Agents 指南：`https://platform.openai.com/docs/guides/agents`
- OpenAI Agent Builder：`https://platform.openai.com/docs/guides/agent-builder`
- OpenAI ChatKit：`https://platform.openai.com/docs/guides/chatkit`
- OpenAI Canvas 发布文：`https://openai.com/index/introducing-canvas/`
- ChatGPT Projects 帮助文档：`https://help.openai.com/en/articles/10169521-projects-in-chatgpt`
- Claude Artifacts 帮助文档：`https://support.claude.com/en/articles/9487310-what-are-artifacts-and-how-do-i-use-them`
- Claude Extended Thinking 文档：`https://platform.claude.com/docs/en/docs/build-with-claude/extended-thinking`
- Anthropic Computer Use 发布文：`https://www.anthropic.com/news/3-5-models-and-computer-use`
- Gemini Canvas：`https://gemini.google/overview/canvas/`
- Gemini Deep Research 帮助文档：`https://support.google.com/gemini/answer/15719111`
- Warp Agent Mode 文档：`https://docs.warp.dev/agents/warp-ai/agent-mode`
- Cursor Docs：`https://docs.cursor.com/`
- Windsurf Docs：`https://docs.windsurf.com/`
- Microsoft Human-AI Interaction Guidelines：`https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/`
- Google People + AI Guidebook：`https://pair.withgoogle.com/guidebook/`
### 9.2 Context7 查询源
- `assistant-ui`：`/assistant-ui/assistant-ui`
- `Vercel AI SDK`：`/vercel/ai`
- `CopilotKit`：`/copilotkit/copilotkit`
### 9.3 本地参考源
- Claude Code pill：`/Users/coso/Documents/dev/js/claudecode/src/tasks/pillLabel.ts`
- Claude Code MCP tool UI：`/Users/coso/Documents/dev/js/claudecode/src/tools/MCPTool/UI.tsx`
- Claude Code plan UI：`/Users/coso/Documents/dev/js/claudecode/src/tools/ExitPlanModeTool/UI.tsx`
- Warp 项目说明：`/Users/coso/Documents/dev/rust/warp/WARP.md`
- Warp Agent Mode 测试：`/Users/coso/Documents/dev/rust/warp/crates/integration/src/test/agent_mode.rs`
- CodexMonitor README：`/Users/coso/Documents/dev/rust/CodexMonitor/README.md`
- CodexMonitor app-server event reference：`/Users/coso/Documents/dev/rust/CodexMonitor/docs/app-server-events.md`
- CodexMonitor composer：`/Users/coso/Documents/dev/rust/CodexMonitor/src/features/composer/components/Composer.tsx`
- CodexMonitor queue：`/Users/coso/Documents/dev/rust/CodexMonitor/src/features/composer/components/ComposerQueue.tsx`
- CodexMonitor messages：`/Users/coso/Documents/dev/rust/CodexMonitor/src/features/messages/components/Messages.tsx`
- Codex TUI composer state machine：`/Users/coso/Documents/dev/rust/codex/docs/tui-chat-composer.md`
- Codex TUI request-user-input overlay：`/Users/coso/Documents/dev/rust/codex/docs/tui-request-user-input.md`
- Codex TUI stream chunking：`/Users/coso/Documents/dev/rust/codex/docs/tui-stream-chunking-tuning.md`
- Codex TUI stream validation：`/Users/coso/Documents/dev/rust/codex/docs/tui-stream-chunking-validation.md`
- Codex TUI status / queue UI：`/Users/coso/Documents/dev/rust/codex/codex-rs/tui/src/status_indicator_widget.rs`、`/Users/coso/Documents/dev/rust/codex/codex-rs/tui/src/bottom_pane/pending_input_preview.rs`
### 9.4 Lime 事实源
- `src/components/agent/chat/types.ts`
- `src/components/agent/chat/components/StreamingRenderer.tsx`
- `src/components/agent/chat/components/MessageList.tsx`
- `docs/roadmap/task/event-chain.md`
- `docs/roadmap/artifacts/roadmap.md`
- `docs/roadmap/harness-engine/README.md`
- `docs/roadmap/warp/README.md`
## 10. 最终建议
Lime 的 Agent UI 不应把“胶囊式 UI”“思考 UI”“工具 UI”“Artifact UI”当作互相独立的组件库任务。它们应该围绕同一个问题收束：
> 用户如何在一个桌面工作区里，快速知道 Agent 正在做什么、什么时候需要自己介入、产物在哪里、过程是否可信、失败后如何恢复？
因此下一阶段最务实的顺序是：
1. 先修对话主链：thinking、tool、首 token、旧会话性能。
2. 再建胶囊任务层：后台任务、needs input、plan ready、多 session 压缩。
3. 然后强化 Artifact：最终产物离开正文，进入 Workbench。
4. 最后把 session/tab、team/subagent、evidence/replay 组织成 Lime 独有的 Agent 工作系统。
这条路既吸收了 Claude Code 的执行状态、Warp 的 block 化工作上下文、ChatGPT/Claude/Gemini 的 Canvas/Artifact 经验，也保留 Lime 自己最重要的差异化：可执行、可观察、可交付、可验证。
docs/roadmap/agentui/lime-agentui-backend-coordination.md
+361
View File
@@ -0,0 +1,361 @@
# Lime AgentUI 后端协作架构
> 状态：后端协作设计
> 更新时间：2026-04-30
> 目标：明确 AgentUI 下一阶段需要后端稳定提供哪些投影、分页、指标和批量接口，避免前端靠全量 detail 和字符串推断撑 UI。
## 1. 后端职责总览
AgentUI 的后端不只是“把模型输出传给前端”。它需要提供五类能力：
| 能力 | 后端职责 | UI 收益 |
| --- | --- | --- |
| 事件流 | 尽早发 `runtime_status`，分型发 text/thinking/tool/artifact/action/queue | 首字前有反馈，流式不串音 |
| 会话分页 | tail window、cursor、older page、summary snapshot | 旧会话秒开，长历史不卡 |
| 过程持久化 | runtime event -> timeline turn/item | tool/thinking/process 可追溯 |
| 产物持久化 | artifact document、版本、metadata、diff | 交付物离开正文进入 workbench |
| 证据导出 | evidence、review、replay、handoff | 结果可审计、可复盘、可继续 |
后端协作的核心原则：
- 前端首屏只拿必要 summary，不拿全量历史。
- 运行态由事件流驱动，历史态由 session detail 和 timeline 驱动。
- process、artifact、evidence 必须有稳定 id，不能只靠显示文本。
- 慢点日志要前后端同名可拼接。
## 2. 后端模块分层
```mermaid
flowchart TB
FE[Frontend agentRuntime clients] --> Cmd[command_api/runtime_api.rs]
Cmd --> Turn[runtime_turn.rs]
Cmd --> Session[session_runtime.rs]
Cmd --> Action[action_runtime.rs]
Cmd --> Subagent[subagent_runtime.rs]
Turn --> Queue[runtime_queue.rs]
Queue --> Aster[Aster Runtime]
Aster --> Converter[event_converter.rs]
Converter --> Events[AgentEvent stream]
Converter --> Timeline[AgentTimelineRecorder]
Cmd --> Store[session_store.rs]
Store --> Db[(SQLite)]
Timeline --> Db
Turn --> Artifact[artifact services]
Artifact --> ArtifactFiles[.lime/artifacts]
Cmd --> Evidence[evidence/review/replay services]
Evidence --> HarnessFiles[.lime/harness]
Store --> ThreadRead[thread_read projection]
ThreadRead --> FE
Events --> FE
```
## 3. Command API 合同
### 3.1 Session 读取
当前主入口：
- 前端：`getAgentRuntimeSession(sessionId, options)`
- 后端：`agent_runtime_get_session(...)`
当前已支持：
- `historyLimit`
- `historyOffset`
- `historyBeforeMessageId`
- `resumeSessionStartHooks`
- `history_cursor`
- `history_truncated`
- `messages_count`
- `thread_read`
- `queued_turns`
下一阶段建议：
| 需求 | 接口建议 | 说明 |
| --- | --- | --- |
| tab 快速恢复 | `agent_runtime_get_session_snapshot` 或 list summary 扩展 | 只返回 title、last message preview、status、unread/running、capsule summary |
| 旧会话分段 hydrate | 保持 `getSession(historyLimit=40)`，补 `timelineDetailLimit` | 首屏 messages 先到，timeline detail 可更少 |
| 更早历史 | `historyBeforeMessageId` 优先于 offset | offset 对持续插入不稳定，cursor 更适合分页 |
| 非活跃 tab 刷新 | summary refresh | 避免后台 tab 拉 detail |
| list 与 detail 抢通道 | list low priority / debounce | 点击历史会话时 detail 优先 |
### 3.2 Turn 提交
当前主入口：
- 前端：`submitAgentRuntimeTurn(request)`
- 后端：`agent_runtime_submit_turn`
必须保持：
- event listener 先注册。
- command 尽早接受请求。
- runtime 尽早发 `runtime_status`。
- busy 时明确 queue 或 steer。
下一阶段建议：
| 需求 | 后端配合 |
| --- | --- |
| 首字慢观测 | 在 submit accepted、runtime queued、runtime starting、provider request started、first model event 处发 status 或 tracing |
| Queue / Steer 区分 | request 和 event 均带 `submission_mode` / `queue_policy` / `target_turn_id` |
| 取消与中断 | interrupt marker 进入 thread_read，事件流发 cancelled/aborted 状态 |
| provider 慢 | runtime status 包含 provider/model/routing stage，但不暴露敏感配置 |
### 3.3 Action Required
当前主入口：
- 前端：`respondAgentRuntimeAction(request)`
- 后端：`agent_runtime_respond_action`
- 事件：`action_required`
- 投影：`thread_read.pending_requests`
下一阶段建议：
| 需求 | 后端配合 |
| --- | --- |
| needs input capsule | thread_read 输出 pending request count、highest severity、oldest age |
| 权限确认可审计 | approval request item 写入 timeline，并记录 response summary |
| replay pending request | pending request 带 replay metadata 和稳定 request id |
| 表单型请求 | action data 使用稳定 schema，而不是 Markdown 文本 |
### 3.4 Queue
当前主入口：
- 前端：`removeAgentRuntimeQueuedTurn`、`promoteAgentRuntimeQueuedTurn`
- 后端：runtime queue service
- 事件：`queue_added`、`queue_removed`、`queue_started`、`queue_cleared`
下一阶段建议：
| 需求 | 后端配合 |
| --- | --- |
| task capsule | queue snapshot 返回 `queued_turn_id`、preview、created_at、mode、target turn |
| 多 tab 快 | list session summary 带 queue/running counts，不需要 detail |
| 队列操作稳定 | queue mutation 以 id 为键，返回最新 queue snapshot |
| 后台恢复 | `resume_runtime_queue_if_needed` 不由普通 getSession 顺手触发 |
## 4. Session Store 与历史分页
当前关键入口：
- `get_runtime_session_detail_with_history_page`
- `get_session_sync_with_history_page`
- `AgentDao::get_message_window_info`
- `count_session_messages_sync`
当前已做到：
- tail page 读取 messages。
- 对 turns/items 使用同窗口限制。
- limited history 下跳过部分重型 runtime overlay。
- full history 时启用 tool IO eviction。
下一阶段建议后端继续提供三种粒度：
| 粒度 | 用途 | 内容 |
| --- | --- | --- |
| summary | sidebar/tab/capsule | session id、title、preview、status、running/queued/pending counts、last activity |
| window detail | active conversation 首屏 | 最近 N 条 messages、少量 turns/items、thread_read、history_cursor |
| full detail page | 用户主动加载更早历史或展开 timeline | older messages、对应 turns/items、artifact refs |
不要把“打开会话”和“加载所有证据”绑成同一个接口。
## 5. Timeline 与 Process 投影
当前关键入口：
- `AgentTimelineRecorder`
- `AgentTimelineDao::list_turns_by_thread_tail_page`
- `AgentTimelineDao::list_items_by_thread_tail_page`
- `thread_reliability_projection_service`
Timeline 后端应对 UI 提供两层数据：
| 层 | 内容 | UI 使用 |
| --- | --- | --- |
| compact process summary | turn status、tool count、artifact count、last tool、errors、duration | 消息旁 process summary、capsule、tab |
| detailed timeline items | reasoning、tool call、command、web search、approval、artifact | Timeline drawer / evidence |
下一阶段建议：
1. 给 `getSession` 增加可选 `timelineDetailLimit` 或 `includeTimelineDetails=false`。
2. 提供 `getThreadTimelinePage(sessionId, cursor, limit)`，用于展开过程详情时再加载。
3. 对 tool 大输出默认返回 summary + offload ref，详情按需读取。
4. timeline item 需要稳定 type 和 status，前端不要解析中文过程文本。
## 6. Artifact 协作
当前关键入口：
- `artifact_document_service.rs`
- `artifact_ops_service.rs`
- `artifact_protocol.rs`
- `write_artifact_events.rs`
- `AgentThreadTimelineArtifactCard`
- `useWorkspaceArtifactPreviewActions`
后端需要保证：
- artifact id、request id、path key 归一化一致。
- `artifact_snapshot` 能映射到 timeline item 和 ArtifactDocument。
- 大 artifact 内容可以按 preview 读取，不把完整内容塞进 message。
- artifact ops envelope 可增量展示：
- `artifact.begin`
- `artifact.meta.patch`
- `artifact.source.upsert`
- `artifact.block.upsert`
- `artifact.block.remove`
- `artifact.complete`
- `artifact.fail`
下一阶段建议：
| 需求 | 后端配合 |
| --- | --- |
| workbench 自动打开 | artifact snapshot 返回 display title、mime/kind、preview mode |
| artifact 列表快 | session summary 带 latest artifacts summary |
| artifact diff | file checkpoint / artifact diff service 打通 |
| artifact 失败恢复 | artifact fail event 进入 thread_read incident |
## 7. Evidence / Review / Replay 协作
当前关键入口：
- `runtime_evidence_pack_service.rs`
- `runtime_review_decision_service.rs`
- `runtime_replay_case_service.rs`
- `runtime_analysis_handoff_service.rs`
- `runtime_handoff_artifact_service.rs`
证据层应保持：
```text
session detail + thread_read + timeline + artifacts + verification
-> evidence pack
-> review decision / replay / handoff
```
下一阶段建议：
| 需求 | 后端配合 |
| --- | --- |
| evidence 后台导出 | command 返回 job/snapshot 状态，UI 用 capsule 展示 |
| review 状态 | session summary 带 latest review status / pending review |
| replay 可见 | replay case 生成后进入 evidence panel |
| 验证摘要 | thread_read 或 evidence summary 输出 pass/fail/needs_review，不由前端推断 |
## 8. 日志与性能指标合同
### 8.1 旧会话恢复
后端已经输出：
- `total_ms`
- `resume_queue_ms`
- `detail_ms`
- `hooks_ms`
- `queue_snapshots_ms`
- `projection_ms`
- `interrupt_marker_ms`
- `dto_ms`
- `history_limit`
- `history_offset`
- `messages`
- `turns`
- `items`
- `queued_turns`
前端应对齐输出：
- `switchTopic.start`
- `snapshot.apply.ms`
- `runtimeGetSession.start/success/slow`
- `hydrate.messages.ms`
- `messageList.timelineBuild.ms`
- `messageList.firstStablePaint.ms`
- `historicalTimeline.idleComplete.ms`
### 8.2 首字慢
建议后端补齐：
- `submit.accepted.ms`
- `queue.wait.ms`
- `turn.start.ms`
- `provider.request.start.ms`
- `provider.first_event.ms`
- `provider.first_text_delta.ms`
- `tool.first_start.ms`
建议前端补齐：
- `listener.bound.ms`
- `submit.invoke.ms`
- `first_event.ms`
- `first_runtime_status.ms`
- `first_text_delta.ms`
- `first_text_paint.ms`
- `text_delta.queue_depth`
- `text_delta.oldest_unrendered_age_ms`
- `stream.catch_up_mode`
### 8.3 CPU / 内存
建议补：
- active tab count。
- hydrated detail tab count。
- mounted MessageList count。
- mounted timeline item count。
- streaming buffer chars。
- deferred timeline pending count。
- artifact preview loaded bytes。
这些指标可以先通过 debug log 实现，不必一开始接入完整遥测系统。
## 9. UI 需要的后端新增能力清单
| 优先级 | 能力 | 建议位置 | 验收 |
| --- | --- | --- | --- |
| P0 | session summary snapshot | `session_runtime.rs` / DTO | 打开 tab 不拉全量 detail |
| P0 | first event / first text tracing | `runtime_turn.rs` / event converter | 首字慢可分段定位 |
| P0 | getSession 可关闭 timeline detail | `runtime_api.rs` / `session_store.rs` | 旧会话 messages 先渲染 |
| P1 | timeline page API | runtime command + DAO | 展开过程时分页加载 |
| P1 | batch session patch | session update command | access mode/provider/strategy 回填合并 |
| P1 | queue/capsule summary in list | session list DTO | sidebar/tab 不抢 detail |
| P1 | tool detail on-demand | tool IO offload + command | 大工具输出不进正文 |
| P2 | background evidence job | evidence service | 证据导出不阻塞 UI |
| P2 | artifact preview manifest | artifact service | workbench 列表快速加载 |
| P3 | worker-friendly projection DTO | protocol projection | 前端 worker 可直接消费 |
## 10. 命令边界注意事项
任何新增或修改 Tauri command 必须同步四侧：
1. 前端 API client。
2. Rust command 注册。
3. command catalog。
4. mock / browser fallback。
最低校验：
```bash
npm run test:contracts
```
如果改动影响 GUI 主路径，再补：
```bash
npm run verify:gui-smoke
```
本轮文档只定义目标，不新增 command。后续实现时必须按命令边界治理执行。
docs/roadmap/agentui/lime-agentui-code-map.md
+165
View File
@@ -0,0 +1,165 @@
# Lime AgentUI 代码层级地图
> 状态：代码事实地图
> 更新时间：2026-04-30
> 目标：把 AgentUI 相关前端、协议、Tauri command、Rust runtime、service、持久化入口分层，作为后续拆分和性能优化的定位图。
## 1. 总览
```mermaid
flowchart TB
UI[src/components/agent/chat UI]
Hooks[src/components/agent/chat/hooks state + stream]
Api[src/lib/api/agentRuntime + agentProtocol]
Tauri[src-tauri commands/aster_agent_cmd]
AgentCrate[src-tauri/crates/agent runtime crate]
Services[src-tauri/src/services]
Store[(SQLite + .lime files)]
UI --> Hooks
Hooks --> Api
Api --> Tauri
Tauri --> AgentCrate
Tauri --> Services
AgentCrate --> Services
Services --> Store
AgentCrate --> Store
```
当前 AgentUI 不是缺少底座，而是底座集中在少数大入口里。下一步代码架构的重点是把聚合入口拆成稳定子层，而不是复制一条新链。
## 2. 前端 UI 层
| 区域 | 关键文件 | 当前职责 | 后续架构方向 |
| --- | --- | --- | --- |
| 工作区总入口 | `src/components/agent/chat/AgentChatWorkspace.tsx` | 组合 chat、inputbar、workspace、artifact、timeline、team、harness、task center | 保留 shell 入口，逐步拆出 `SessionChrome`、`ConversationPane`、`WorkbenchPane`、`ProcessDrawer` |
| 消息列表 | `src/components/agent/chat/components/MessageList.tsx` | 渲染 messages、timeline、queued、pending action、历史窗口、progressive render | 继续做首屏轻量投影；timeline/tool/artifact detail 延迟或虚拟化 |
| 流式渲染 | `src/components/agent/chat/components/StreamingRenderer.tsx` | 渲染 text/thinking/tool/action/runtime status、A2UI、计划块 | 保持 content part 分型；补 backlog/catch-up 指标和重复吐字防线 |
| 输入区 | `src/components/agent/chat/components/Inputbar/index.tsx` | 输入容器与状态接线 | 收敛为 composer shell |
| 输入核心 | `src/components/agent/chat/components/Inputbar/components/InputbarCore.tsx` | 文本、图片、provider、model、execution strategy、access mode、team、workflow、hint popup | 抽出 composer state machine，明确 send / queue / steer |
| 运行状态行 | `src/components/agent/chat/components/Inputbar/components/InputbarRuntimeStatusLine.tsx` | 输入区状态提示 | 升级为首字前可信 runtime strip，可和 task capsule 共享投影 |
| 队列面板 | `src/components/agent/chat/components/Inputbar/components/QueuedTurnsPanel.tsx` | 展示 queued turns | 作为 task capsule 的详情面板 |
| 时间线 | `src/components/agent/chat/components/AgentThreadTimeline.tsx` | 展示 turn/item/tool/artifact 过程 | 从 MessageList 正文中继续抽离，默认延迟渲染历史项 |
| 工具步骤 | `src/components/agent/chat/components/InlineToolProcessStep.tsx`、`ToolCallDisplay.tsx` | 工具调用、工具结果展示 | 大输出进入详情抽屉，正文只保留摘要 |
| 计划与决策 | `AgentPlanBlock.tsx`、`DecisionPanel.tsx`、`ActionRequestA2UIPreviewCard.tsx` | 计划、审批、action_required | 统一为 HITL pattern：批准、拒绝、编辑、重放 |
| 任务中心 | `TaskCenterTabStrip.tsx`、`A2UITaskCard.tsx` | task center / A2UI 任务卡 | 进入 task layer，和 tab/capsule 统一 |
| Team 工作台 | `components/team-workspace-board/*` | 子代理、团队、画布式协作 | 成为 task layer 的 team 视图，不和普通 chat 首屏抢渲染 |
| Harness | `HarnessStatusPanel.tsx`、`RuntimeReviewDecisionDialog.tsx` | evidence、review、verification 展示 | 成为 evidence layer，默认由 task/evidence 胶囊进入 |
| Workbench | `CanvasWorkbenchLayout.tsx`、`GeneralWorkbench*`、`useWorkspaceArtifactPreviewActions.ts` | artifact/canvas/general workbench | 成为 artifact layer 的首要承载面 |
## 3. 前端状态与 Hook 层
| 文件 | 当前职责 | 关键事实 | 后续方向 |
| --- | --- | --- | --- |
| `hooks/useAgentSession.ts` | 会话主状态、topic、messages、threadTurns、threadItems、queuedTurns、threadRead、executionRuntime | 已有 `SESSION_HISTORY_LOAD_PAGE_SIZE = 50`、`SESSION_DETAIL_PREFETCH_HISTORY_LIMIT = 40`、deferred hydration、prefetch、cached snapshot | 逐步拆为 session controller、history controller、topic/tab controller、runtime projection selector |
| `hooks/agentStreamSubmitExecution.ts` | 提交流式 turn | 先注册 event binding，再 `submitOp` | 保持顺序，避免首事件丢失 |
| `hooks/agentStreamTurnEventBinding.ts` | 注册事件 listener、watchdog、silent recovery | `STREAM_FIRST_EVENT_TIMEOUT_MS = 12_000`、`STREAM_INACTIVITY_TIMEOUT_MS = 120_000` | 首字慢排查应采集 first event / first text / inactivity 指标 |
| `hooks/agentStreamRuntimeHandler.ts` | `handleTurnStreamEvent` 分发事件到前端状态 | `TEXT_DELTA_RENDER_FLUSH_MS = 32`，含 `reconcileFinalContentParts` | 抽成 reducer，补 backlog depth / oldest age / catch-up mode |
| `utils/threadTimelineView.ts` | messages + turns + items -> timeline 投影 | MessageList 同步计算热点之一 | 大会话改为 idle/worker/懒加载 |
| `utils/messageTurnGrouping.ts` | timeline grouping | 与渲染批次绑定 | 与 virtualization 结合 |
| `utils/processDisplayText.ts` | 过程文本规范化 | 防 `<think>` / 工具日志污染正文 | 继续作为 text/thinking/tool 分型防线 |
## 4. 前端协议 / API 层
| 文件 | 当前职责 | 关键能力 |
| --- | --- | --- |
| `src/lib/api/agentRuntime/index.ts` | agent runtime API 统一导出 | session、thread、export、subagent、site、media client |
| `src/lib/api/agentRuntime/sessionClient.ts` | create/list/get/update session | `getAgentRuntimeSession(sessionId, { historyLimit, historyOffset, historyBeforeMessageId })`；有 `runtimeGetSession.slow` 日志 |
| `src/lib/api/agentRuntime/threadClient.ts` | submit、interrupt、compact、resume、respond action、queue 操作、thread read | `submitAgentRuntimeTurn`、`respondAgentRuntimeAction`、`getAgentRuntimeThreadRead` |
| `src/lib/api/agentRuntime/types.ts` | DTO 类型 | `AsterSessionDetail`、`AgentRuntimeThreadReadModel`、`AgentRuntimeSubmitTurnRequest`、history cursor |
| `src/lib/api/agentProtocol.ts` | AgentEvent 类型和兼容 normalizer | `turn_started`、`text_delta`、`thinking_delta`、`tool_start/end`、`artifact_snapshot`、`runtime_status`、queue、subagent、`done/final_done` |
| `src/lib/api/agentTextNormalization.ts` | legacy 文本与 item normalizer | 兼容旧事件/旧 item | 继续收口 compat，不新增旧形态 |
协议层的架构判断：
- UI 只能消费结构化事件和 DTO，不应通过字符串猜测 Agent 状态。
- `parseAgentEvent` 是兼容边界，不是鼓励继续新增 legacy 事件。
- `historyLimit/historyOffset/historyBeforeMessageId` 是旧会话性能主接口，后续 pagination/virtualization 要继续沿用。
## 5. Tauri Command 层
| 文件 | 当前职责 | AgentUI 相关能力 |
| --- | --- | --- |
| `src-tauri/src/commands/aster_agent_cmd/command_api/runtime_api.rs` | runtime Tauri command 主入口 | submit、interrupt、compact、resume、get session、thread read、file checkpoint、evidence/review export |
| `src-tauri/src/commands/aster_agent_cmd/runtime_turn.rs` | turn 执行主链 | 提前发 `runtime_status`、执行 provider/runtime/tool/hook/memory/artifact/auto compact |
| `src-tauri/src/commands/aster_agent_cmd/session_runtime.rs` | session create/list/update/recent runtime context | sidebar/session summary 和 tab 管理基础 |
| `src-tauri/src/commands/aster_agent_cmd/subagent_runtime.rs` | subagent runtime | team/capsule 子代理状态基础 |
| `src-tauri/src/commands/aster_agent_cmd/action_runtime.rs` | action required / response | HITL、权限确认、用户输入请求 |
| `src-tauri/src/commands/aster_agent_cmd/tool_runtime/*` | tool bridge | browser、workspace、service skill、media、mcp、subagent、site |
| `src-tauri/src/commands/aster_agent_cmd/dto.rs` | 后端 DTO | `AgentRuntimeSessionDetail`、history cursor、thread_read 投影 |
`agent_runtime_get_session` 当前关键行为：
- 默认 `RUNTIME_SESSION_OPEN_HISTORY_LIMIT = 40`。
- 最大 `RUNTIME_SESSION_MAX_HISTORY_LIMIT = 2000`。
- `historyLimit = 0` 表示不裁剪。
- 返回 detail、queue snapshots、interrupt marker、thread_read、history cursor。
- tracing 已拆出 `detail_ms`、`hooks_ms`、`queue_snapshots_ms`、`projection_ms`、`dto_ms`。
这意味着旧会话恢复慢的排查不应只看前端感受，必须把前端 `runtimeGetSession.slow` 与后端 `agent_runtime_get_session` 分段日志对齐。
## 6. Rust Agent Crate 层
| 文件 | 当前职责 | AgentUI 意义 |
| --- | --- | --- |
| `src-tauri/crates/agent/src/runtime_queue.rs` | turn queue、resume、queue event | queue/capsule/task layer 的事实源 |
| `src-tauri/crates/agent/src/session_store.rs` | session detail、messages、turns、items、todo、runtime overlay、subagent context | old session restore、history pagination、thread_read 的底层事实源 |
| `src-tauri/crates/agent/src/session_query.rs` | parent/child/cascade 查询 | team/subagent 视图 |
| `src-tauri/crates/agent/src/event_converter.rs` | Aster event -> Lime runtime event | text/thinking/tool/artifact/status 的协议生成点 |
| `src-tauri/crates/agent/src/protocol_projection.rs` | current projection 入口 | 新 UI 应优先跟 current projection 对齐 |
| `src-tauri/crates/agent/src/runtime_projection_snapshot.rs` | runtime summary snapshot | tab/capsule/sidebar summary 可复用 |
| `src-tauri/crates/agent/src/session_execution_runtime.rs` | execution runtime/cost/limit/routing | provider/model/cost/context usage UI |
| `src-tauri/crates/agent/src/queued_turn.rs` | queue snapshot 数据结构 | queue panel、task capsule |
| `src-tauri/crates/agent/src/tool_io_offload.rs` | tool 大输出 offload | tool UI 大输出不卡顿的后端基础 |
| `src-tauri/crates/agent/src/write_artifact_events.rs` | artifact write event | artifact snapshot / workbench 联动 |
## 7. Service 层
| 文件 | 当前职责 | AgentUI 使用方式 |
| --- | --- | --- |
| `src-tauri/src/services/agent_timeline_service.rs` | `AgentTimelineRecorder` 持久化 runtime event -> turn/item | Process layer 与 Evidence layer 的过程事实 |
| `src-tauri/src/services/artifact_document_service.rs` | ArtifactDocument 持久化、版本、`.lime/artifacts` | Artifact layer 主事实源 |
| `src-tauri/src/services/artifact_ops_service.rs` | incremental artifact envelope | artifact.begin/meta/source/block/complete/fail |
| `src-tauri/src/services/runtime_evidence_pack_service.rs` | evidence pack 导出 | Evidence layer 输出 `summary.md/runtime.json/timeline.json/artifacts.json` |
| `src-tauri/src/services/runtime_review_decision_service.rs` | review decision 模板与保存 | Human review 闭环 |
| `src-tauri/src/services/runtime_replay_case_service.rs` | replay case | 失败复现与验证 |
| `src-tauri/src/services/runtime_handoff_artifact_service.rs` | handoff bundle | 跨 agent/人工交接 |
| `src-tauri/src/services/runtime_file_checkpoint_service.rs` | file checkpoint / diff | 代码任务可审查变更 |
| `src-tauri/src/services/thread_reliability_projection_service.rs` | reliability projection | thread_read、incident、pending request |
## 8. 存储与文件事实源
| 事实源 | 内容 | UI 映射 |
| --- | --- | --- |
| SQLite session tables | sessions、messages、metadata | sidebar、tabs、Conversation layer |
| SQLite timeline tables | turns、items、tool、artifact item | Process layer、Evidence layer |
| `.lime/artifacts` | artifact documents、版本、snapshot | Artifact layer |
| `.lime/harness/sessions/<session>/evidence` | evidence pack | Evidence panel |
| `.lime/harness/sessions/<session>/review` | review decision | Review dialog/panel |
| runtime in-memory state | running turns、interrupt marker、queue | Runtime strip、capsule、queue |
## 9. 测试入口
| 范围 | 现有测试入口 |
| --- | --- |
| MessageList | `src/components/agent/chat/components/MessageList.test.tsx` |
| StreamingRenderer | `src/components/agent/chat/components/StreamingRenderer.test.tsx` |
| Stream event binding | `src/components/agent/chat/hooks/agentStreamTurnEventBinding.test.ts` |
| Stream runtime reducer | `src/components/agent/chat/hooks/agentStreamRuntimeHandler.test.ts` |
| Inputbar | `InputbarCore.test.tsx`、`InputbarRuntimeStatusLine.test.tsx`、`QueuedTurnsPanel.test.tsx` |
| Timeline | `AgentThreadTimeline.test.tsx`、`AgentThreadTimelineArtifactCard.test.tsx` |
| Task center / team | `TaskCenterTabStrip.test.tsx`、`TeamWorkspaceBoard*.test.tsx` |
| Harness | `HarnessStatusPanel.test.tsx` |
| Backend history/session | `src-tauri/crates/agent/src/session_store.rs` unit tests |
| Command contract | `npm run test:contracts` |
| GUI smoke | `npm run verify:gui-smoke` |
## 10. 后续拆分边界
建议优先拆这些边界，避免继续扩大主入口：
1. `useAgentSession` 拆出 `useSessionHistoryWindow`、`useTopicTabs`、`useRuntimeProjection`。
2. `agentStreamRuntimeHandler` 抽出纯 reducer，测试 first event、first text、thinking/final 去重、tool/action/artifact 合并。
3. `MessageList` 把 timeline projection 和 rendering 分离，历史 timeline 默认 idle/worker。
4. `AgentChatWorkspace` 把 workbench/harness/team 的状态接线移到各自 controller hook。
5. `InputbarCore` 把 textarea、attachments、slash/hint、queue/steer 拆为 composer state machine。
docs/roadmap/agentui/lime-agentui-event-flow.md
+238
View File
@@ -0,0 +1,238 @@
# Lime AgentUI 事件流程
> 状态：流程设计
> 更新时间：2026-04-30
> 目标：固定 AgentUI 的关键运行流，避免 UI 层用局部状态猜测 runtime 行为。
## 1. 事件事实源
AgentUI 的运行事实来自五类投影：
| 投影 | 入口 | UI 消费 |
| --- | --- | --- |
| AgentEvent stream | `src/lib/api/agentProtocol.ts` | 流式文本、thinking、tool、artifact、runtime status、queue、done |
| Session detail | `getAgentRuntimeSession` / `agent_runtime_get_session` | 打开旧会话、历史窗口、thread items、queued turns、thread read |
| Thread read | `getAgentRuntimeThreadRead` / session detail 内联 | pending requests、last outcome、incident、interrupt、queue |
| Timeline | `AgentTimelineRecorder` 持久化的 turn/item | Process layer、Evidence layer |
| Artifact/Evidence files | `.lime/artifacts`、`.lime/harness` | Workbench、Harness、Review、Replay |
事件流原则：
1. 新 UI 只新增投影或 selector，不新增第二条 runtime event。
2. 对话正文只消费 `text` 和最终答复相关 preview。
3. thinking/tool/status/action/artifact 必须分型，不拼接进同一个 Markdown 字符串。
4. 历史恢复消费 session detail，流式执行消费 AgentEvent，两者最终进入同一前端 state shape。
## 2. 发送消息流程
```mermaid
flowchart TB
Start([用户点击发送]) --> Validate[Inputbar 校验 prompt / attachments / model / access mode]
Validate --> Optimistic[前端创建乐观 user message 和 assistant placeholder]
Optimistic --> Bind[注册 AgentEvent listener]
Bind --> Submit[submitAgentRuntimeTurn]
Submit --> Command[agent_runtime_submit_turn]
Command --> Queue{runtime queue 是否忙}
Queue -->|空闲| StartTurn[启动 turn]
Queue -->|忙碌且允许排队| Enqueue[写入 queued turn 并发 queue_added]
Queue -->|忙碌且 steer| Steer[注入当前 turn 或 pending input]
StartTurn --> EarlyStatus[尽早发 runtime_status preparing/routing]
EarlyStatus --> Runtime[Aster Agent Runtime]
Runtime --> Events[turn_started / thinking_delta / text_delta / tool / artifact / action_required]
Events --> FrontendReducer[handleTurnStreamEvent 更新前端状态]
FrontendReducer --> UI[Conversation / Process / Task / Artifact UI]
Events --> Recorder[AgentTimelineRecorder]
Recorder --> TimelineDb[(timeline DB)]
Runtime --> Done[done / final_done]
Done --> Reconcile[reconcileFinalContentParts 防重复吐字]
Reconcile --> Complete[停止 sending，收起 transient status]
```
关键约束：
- listener 必须先于 submit 注册，避免首事件丢失。
- `runtime_status` 应早于首个 `text_delta`，用于首字前反馈。
- `text_delta` 只进入 text part；`thinking_delta` 只进入 thinking part。
- `final_done` 只能 reconcile，不应把完整 final text 再追加一遍。
- tool/artifact/action 事件进入 process/task/artifact 投影，不污染最终正文。
## 3. 打开旧会话流程
```mermaid
flowchart TB
Click([用户打开历史会话]) --> Tab[创建或激活 topic/tab]
Tab --> Snapshot{有 cached snapshot?}
Snapshot -->|有| ApplySnapshot[立即应用标题/最近消息/运行态快照]
Snapshot -->|无| Shell[渲染 workspace shell + skeleton]
ApplySnapshot --> Shell
Shell --> Detail[getSession historyLimit=40]
Detail --> Command[agent_runtime_get_session]
Command --> Store[SessionStore tail window + turns/items]
Store --> ThreadRead[构建 thread_read / queue snapshots / history cursor]
ThreadRead --> HydrateMessages[优先 hydrate messages]
HydrateMessages --> RenderLight[MessageList 轻量渲染最近消息]
RenderLight --> DeferTimeline[延迟 timeline/tool/artifact detail]
DeferTimeline --> Idle[空闲时补过程层]
RenderLight --> HistoryCursor{history_truncated?}
HistoryCursor -->|是| LoadMore[显示加载更早历史入口]
HistoryCursor -->|否| Done[完成恢复]
```
关键约束：
- 不等待全量历史再挂载 UI。
- 不在打开旧会话时同时触发无 `historyLimit` 的 `getSession`。
- sidebar list 刷新应低优先级，不能抢 session detail 主链。
- 非活跃 tab 只保留 snapshot，不构建完整 MessageList/timeline。
- full history 只能分页，不回退到 `historyLimit: 0` 的默认路径。
## 4. Queue / Steer 流程
```mermaid
flowchart TB
Input([当前 turn 运行中用户继续输入]) --> Mode{用户选择模式}
Mode -->|Queue| QueueSubmit[submitTurn queueIfBusy=true]
Mode -->|Steer| SteerSubmit[submitTurn steer/current-turn intent]
QueueSubmit --> QueueAdded[queue_added]
QueueAdded --> TaskCapsule[Task capsule 显示排队数量和下一条摘要]
TaskCapsule --> QueuePanel[QueuedTurnsPanel 支持 promote/remove/edit]
SteerSubmit --> PendingPreview[pending steer preview]
PendingPreview --> Runtime[当前 runtime 消费 steer]
Runtime --> Status[queue_started / queue_removed / runtime_status]
Status --> UI[Task layer + Process layer 更新]
```
UI 规则：
- Queue 表示“本轮结束后执行”，Steer 表示“影响当前执行”。
- 两者必须有不同视觉和文案。
- queue 入口不应该触发新 tab 全量恢复。
- queue 操作应以 `queued_turn_id` 为稳定键，避免列表重排导致操作错位。
## 5. Action Required / Human-in-the-loop 流程
```mermaid
flowchart TB
Runtime[Agent Runtime] --> ActionEvent[action_required]
ActionEvent --> ThreadItem[approval_request / request_user_input item]
ActionEvent --> Frontend[前端 pendingActions]
Frontend --> Promote{是否提升到输入区 A2UI?}
Promote -->|是| InputA2UI[Inputbar / bottom pending form]
Promote -->|否| InlineCard[消息内 action card]
InputA2UI --> UserDecision[用户批准/拒绝/填写]
InlineCard --> UserDecision
UserDecision --> Respond[respondAgentRuntimeAction]
Respond --> Command[agent_runtime_respond_action]
Command --> RuntimeResume[Runtime 继续执行]
RuntimeResume --> Events[后续 text/tool/artifact/done]
```
UI 规则：
- `needs_input` 和 `plan_ready` 是 task capsule 的高优先级状态。
- 高风险动作必须显示明确操作对象、影响范围和确认按钮。
- action card 完成后应收起为历史摘要，不继续占据首屏。
- replay pending request 应从 `thread_read.pending_requests` 进入，不从文本猜测。
## 6. Artifact 流程
```mermaid
flowchart TB
Runtime[Agent Runtime / Tool] --> ArtifactEvent[artifact_snapshot 或 write_artifact event]
ArtifactEvent --> Timeline[AgentTimelineRecorder 写入 file_artifact item]
ArtifactEvent --> ArtifactService[ArtifactDocument / ArtifactOps]
ArtifactService --> Files[.lime/artifacts]
ArtifactEvent --> Frontend[前端 artifact signal]
Frontend --> MessagePreview[消息内简短 preview/card]
Frontend --> Workbench[Canvas / Artifact Workbench 自动选择或提示]
Workbench --> UserEdit[用户编辑 / diff / 导出]
```
UI 规则：
- artifact 主体不长期留在正文。
- 消息内 card 只做摘要和打开入口。
- artifact 版本、路径、metadata 应来自 artifact service，不由前端拼路径。
- `artifact_snapshot` 同时属于 process evidence 和 artifact delivery。
## 7. Evidence / Review 流程
```mermaid
flowchart TB
User[用户或系统触发导出] --> Export[agent_runtime_export_evidence_pack]
Export --> Detail[读取 session detail + thread_read]
Detail --> Timeline[读取 timeline]
Detail --> Artifacts[读取 artifact metadata]
Timeline --> EvidenceService[runtime_evidence_pack_service]
Artifacts --> EvidenceService
EvidenceService --> Files[.lime/harness/sessions/<session>/evidence]
Files --> HarnessPanel[HarnessStatusPanel 展示]
HarnessPanel --> Review[导出 review decision template]
Review --> ReviewService[runtime_review_decision_service]
ReviewService --> ReviewFiles[.lime/harness/sessions/<session>/review]
```
UI 规则：
- evidence/review 是证据层，不应阻塞聊天流式输出。
- 导出动作应显示后台任务状态，完成后给 capsule/harness 入口。
- evidence 的 summary、timeline、artifact、verification 必须同源，不允许前端伪造通过状态。
## 8. 慢点排查流程
### 8.1 旧会话恢复慢
```mermaid
flowchart LR
UserClick[click old session] --> UIStart[frontend switchTopic start]
UIStart --> GetSession[runtimeGetSession.start]
GetSession --> BackendLog[agent_runtime_get_session total/detail/projection/dto]
BackendLog --> FrontendHydrate[finalize detail / hydrate messages]
FrontendHydrate --> MessageCompute[MessageList timeline/group memo]
MessageCompute --> Paint[first stable paint]
```
需要同时记录：
- click -> shell rendered。
- click -> `runtimeGetSession.start`。
- `runtimeGetSession.start` -> success。
- 后端 `detail_ms/projection_ms/dto_ms`。
- detail success -> messages rendered。
- messages rendered -> historical timeline idle completed。
### 8.2 首字慢
```mermaid
flowchart LR
Send[send click] --> Bind[event listener bound]
Bind --> Submit[submit invoke start]
Submit --> Accepted[submit accepted]
Accepted --> FirstEvent[first AgentEvent]
FirstEvent --> FirstStatus[first runtime_status]
FirstStatus --> FirstText[first text_delta]
FirstText --> FirstPaint[first text paint]
```
需要同时记录：
- listener bound 时间。
- submit invoke 耗时。
- first event 时间。
- first runtime status 时间。
- first text delta 时间。
- delta flush queue depth。
- oldest unrendered delta age。
- catch-up mode transition。
### 8.3 CPU / 内存飙高
优先检查：
1. 是否多个 tab 同时全量恢复。
2. 是否非活跃会话仍在构建 `MessageList` 和 timeline。
3. 是否 `getSession(historyLimit: 0)` 或无 limit 请求被触发。
4. 是否 sidebar list 与 session detail 抢同一 invoke 通道。
5. 是否 streaming text 逐字动画在 backlog 很大时仍慢速推进。
6. 是否 tool output / artifact preview 被一次性塞入正文渲染。
docs/roadmap/agentui/lime-agentui-implementation-roadmap.md
+303
View File
@@ -0,0 +1,303 @@
# Lime AgentUI 实施路线图
> 状态：实施路线图
> 更新时间：2026-04-30
> 目标：把 AgentUI 架构落成可分批交付的工程任务，优先解决旧会话恢复慢、首字慢、tab 卡顿、流式重复吐字和过程信息噪声。
## 1. 阶段目标
| 阶段 | 目标 | 用户体感 |
| --- | --- | --- |
| P0 | 稳住当前对话主链和性能热点 | 旧会话能快速显示，首字前有可信反馈，流式不重复吐字 |
| P1 | 建立 task capsule 与 tab 管理 | 多会话、多后台任务不拖慢主页面 |
| P2 | 强化 artifact/workbench/evidence 分层 | 最终产物和证据离开正文，进入可编辑/可审计工作台 |
| P3 | 抽象 AgentUI 子系统 | `AgentChatWorkspace` 从巨石入口变成稳定 shell |
## 2. P0：体验止血
### 2.1 旧会话渐进恢复
目标：
- 点击历史会话后立即显示 shell、tab、标题和 skeleton。
- `getSession(historyLimit=40)` 返回后先 hydrate messages。
- timeline/tool/artifact detail 延迟到 idle 或用户展开。
- 禁止打开旧会话时触发无 limit detail。
代码入口：
- `src/components/agent/chat/hooks/useAgentSession.ts`
- `src/components/agent/chat/components/MessageList.tsx`
- `src/lib/api/agentRuntime/sessionClient.ts`
- `src-tauri/src/commands/aster_agent_cmd/command_api/runtime_api.rs`
验收：
- 旧会话 click -> shell paint 不依赖 getSession 完成。
- detail 请求带 `historyLimit`。
- MessageList 首屏不等待完整 timeline。
- Playwright E2E 能打开两个历史会话并来回切换。
### 2.2 首字慢分段日志
目标：
- 发送后立刻显示 submitted/routing/preparing。
- 记录 listener bound、submit invoke、first event、first status、first text delta、first paint。
- 后端记录 submit accepted、runtime start、provider request start、first provider event。
代码入口：
- `agentStreamSubmitExecution.ts`
- `agentStreamTurnEventBinding.ts`
- `agentStreamRuntimeHandler.ts`
- `runtime_turn.rs`
- `event_converter.rs`
验收：
- 首字慢时日志能判断慢在 invoke、queue、provider、event bridge、前端 flush 还是渲染。
- Playwright E2E 发送真实 prompt 后能采集首字链路。
- 首字前 UI 不再表现为鼠标 loading + 页面无反馈。
### 2.3 流式重复吐字防线
目标：
- `thinking_delta`、`text_delta`、`final_done` 严格分型。
- `final_done` 只 reconcile，不二次 append 完整正文。
- `<think>`、工具日志、过程 status 不进入最终 Markdown 正文。
代码入口：
- `agentStreamRuntimeHandler.ts`
- `StreamingRenderer.tsx`
- `processDisplayText.ts`
- `StreamingRenderer.test.tsx`
- `agentStreamRuntimeHandler.test.ts`
验收：
- 深度思考模型不会把 thinking 和 final answer 混合重复显示。
- 同一段最终答复不会出现“先流式一次，完成后又整段追加一次”。
- 历史恢复不会把已完成 thinking 当正文重放。
### 2.4 Streaming backlog catch-up
目标：
- 大量 delta 到达时自动进入 catch-up，避免逐字动画拖慢。
- 指标记录 queue depth、oldest unrendered age、flush count。
- MessageList / Markdown 轻量模式对历史和 backlog 区分处理。
代码入口：
- `StreamingRenderer.tsx`
- `agentStreamRuntimeHandler.ts`
- `MarkdownRenderer.tsx`
- `MessageList.tsx`
验收：
- 长回答中途不会出现 UI 越吐越慢。
- CPU 不因逐字动画长期满载。
- catch-up 触发和恢复可从日志看到。
## 3. P1：Tab 与 Task Capsule
### 3.1 浏览器式 tab 管理
目标：
- 新建对话是新增 tab，不跳出当前界面。
- 打开历史会话是新增或激活 tab，不导致已有 tab 全量重渲染。
- 非活跃 tab 只保留 snapshot。
- 关闭 tab 释放重型渲染对象。
代码入口：
- `AgentChatWorkspace.tsx`
- `useAgentSession.ts`
- `TaskCenterTabStrip.tsx`
- sidebar/topic 相关组件
验收：
- 连续打开多个历史会话不再 CPU/内存飙高。
- 可同时保留新建对话和多个旧会话。
- 切回旧 tab 先显示 snapshot，再后台刷新。
### 3.2 Task capsule strip
目标：
- running、queued、needs_input、plan_ready、failed、team/subagent 统一为胶囊。
- 普通 running 低调展示；needs_input/plan_ready/failed 抢注意力。
- 点击胶囊打开 task center 或 process drawer，不跳离上下文。
代码入口：
- `TaskCenterTabStrip.tsx`
- `QueuedTurnsPanel.tsx`
- `InputbarRuntimeStatusLine.tsx`
- `AgentRuntimeStrip.tsx`
- `thread_read` projection
验收：
- 当前 turn 状态不再只靠按钮 spinner。
- pending action 可从胶囊快速定位。
- queue 和 steer 的视觉、文案和行为区分清楚。
### 3.3 Sidebar list 降优先级
目标：
- 点击历史会话时 session detail 优先。
- recent list / archived list 刷新低优先级、可取消、可 debounce。
- list summary 不携带重型 detail。
代码入口：
- sidebar refresh 逻辑
- `sessionClient.ts`
- `session_runtime.rs`
验收：
- 打开历史会话时不因 sidebar list 刷新阻塞 detail。
- invoke 通道没有 list 与 getSession 串行抢占。
## 4. P2：Artifact / Evidence 分层
### 4.1 Artifact 从正文迁移到 Workbench
目标：
- `artifact_snapshot` 在正文只显示摘要卡。
- Workbench 自动选择最新关键 artifact 或显示建议。
- 大 artifact 按 preview 加载，不塞入 message。
代码入口：
- `StreamingRenderer.tsx`
- `AgentThreadTimelineArtifactCard.tsx`
- `useWorkspaceArtifactPreviewActions.ts`
- artifact services
验收：
- 长文档/代码/报告不会让消息列表卡顿。
- artifact 能从 timeline、message card、workbench 三处指向同一 id/path。
### 4.2 Evidence / Review 后台化
目标：
- evidence export 显示为后台任务。
- 完成后进入 harness panel，不阻塞当前聊天。
- review decision 和 replay 与 evidence pack 同源。
代码入口：
- `HarnessStatusPanel.tsx`
- `runtime_evidence_pack_service.rs`
- `runtime_review_decision_service.rs`
- `runtime_replay_case_service.rs`
验收：
- 导出期间仍可继续查看/发送消息。
- evidence/review 路径和状态来自后端返回，不由前端猜。
## 5. P3：结构收敛
### 5.1 拆分 AgentChatWorkspace
目标结构：
```text
AgentChatWorkspace
AgentWorkspaceShell
SessionChrome
ConversationPane
WorkbenchPane
ProcessDrawer
TaskCapsuleStrip
```
拆分原则：
- 每次只拆一个职责边界。
- 不改变用户行为时先补 snapshot/interaction 测试。
- 不把视觉重构和协议重构混在一刀。
### 5.2 拆分 useAgentSession
建议目标：
```text
useAgentSession
useTopicTabs
useSessionHistoryWindow
useRuntimeStream
useRuntimeProjection
useQueuedTurns
usePendingActions
```
收益：
- 性能日志能按子系统归因。
- 单元测试更容易覆盖 stream / history / queue。
- 后续 worker 化 timeline 不需要改整个 hook。
### 5.3 Worker / Virtualization
目标：
- timeline 构建可移到 worker 或 idle task。
- 历史 MessageList 可虚拟化。
- 大 tool/artifact 详情按需加载。
触发条件：
- 旧会话 messages > 200。
- threadItems > 500。
- 单条 assistant content > 24k。
- tool output preview > 64k。
## 6. 验证矩阵
| 改动类型 | 最低验证 |
| --- | --- |
| 文档变更 | markdown 格式检查、`git status --short` |
| 前端 UI/Hook | 受影响 `*.test.tsx` / `*.test.ts`，默认 `npm run verify:local` |
| 流式事件逻辑 | `agentStreamRuntimeHandler.test.ts`、`agentStreamTurnEventBinding.test.ts` |
| Tauri command / bridge | `npm run verify:local` + `npm run test:contracts` |
| GUI 主路径 | `npm run verify:local` + `npm run verify:gui-smoke` |
| 真实交互 | Playwright E2E：新建对话、打开两个历史会话、发送消息、queue/steer、pending action |
## 7. E2E 场景清单
优先补这些 Playwright 场景：
1. 新建对话：点击新任务后新增 tab，输入区立即可用。
2. 打开旧会话：历史消息快速显示，输入区吸顶/吸底行为符合目标设计。
3. 多历史会话：连续打开两个旧会话，二者都能切换，不卡死。
4. 发送消息：首字前出现 runtime status，首字后流式稳定。
5. 深度思考模型：thinking 折叠，不重复吐字，不污染 final answer。
6. Queue：当前 turn 运行中发送下一条，进入 queue capsule。
7. Steer：当前 turn 运行中修正当前任务，显示 pending steer。
8. Artifact：生成 artifact 后正文显示卡片，workbench 可打开。
9. Evidence：导出 evidence 后 harness panel 能打开文件。
## 8. 完成定义
AgentUI 下一阶段不能用“组件做出来了”作为完成标准。完成标准应同时满足：
1. 当前用户主路径更快：旧会话恢复、首字、tab 切换有日志和 E2E 证明。
2. UI 分层更清楚：正文、过程、任务、产物、证据不混在一个 Markdown 流里。
3. 后端事实源更稳定：summary、detail、timeline、artifact、evidence 各自有清晰接口。
4. 代码边界更小：新增能力不继续扩大 `AgentChatWorkspace` 和 `useAgentSession` 的职责。
5. 验证可重复：单元测试、GUI smoke、Playwright 覆盖核心流程。
docs/roadmap/agentui/lime-agentui-sequence-diagrams.md
+288
View File
@@ -0,0 +1,288 @@
# Lime AgentUI 时序图
> 状态：时序设计
> 更新时间：2026-04-30
> 目标：为实现、E2E、性能日志和故障排查提供端到端时序基线。
## 1. 新建会话并发送消息
```mermaid
sequenceDiagram
participant U as User
participant Home as HomeStartSurface / EmptyState
participant Shell as AgentChatWorkspace
participant Hook as useAgentSession
participant Bind as AgentEvent Binding
participant API as threadClient
participant Cmd as agent_runtime_submit_turn
participant Queue as runtime_queue
participant RT as Aster Runtime
participant State as Frontend State
participant UI as MessageList / StreamingRenderer
U->>Home: 新建对话并输入 prompt
Home->>Shell: create topic/session
Shell->>Hook: handleSend(prompt)
Hook->>State: append user message + assistant placeholder
Hook->>Bind: registerAgentStreamTurnEventBinding(eventName)
Hook->>API: submitAgentRuntimeTurn(request)
API->>Cmd: invoke agent_runtime_submit_turn
Cmd->>Queue: submit_runtime_turn
Queue-->>Cmd: accepted / enqueued
Cmd-->>State: early runtime_status
Queue->>RT: start turn
RT-->>Bind: turn_started
Bind->>State: currentTurnId / threadTurns
RT-->>Bind: thinking_delta
Bind->>State: contentParts.thinking
RT-->>Bind: text_delta
Bind->>State: buffered text flush
State->>UI: render first text
RT-->>Bind: tool_start / tool_end
Bind->>State: threadItems + tool parts
RT-->>Bind: artifact_snapshot
Bind->>State: artifact signal
RT-->>Bind: done / final_done
Bind->>State: reconcileFinalContentParts
State->>UI: completed message
```
验收点：
- `registerAgentStreamTurnEventBinding` 早于 submit invoke。
- 首个 `runtime_status` 能在首个 `text_delta` 前显示。
- `final_done` 不导致完整答案重复追加。
## 2. 打开旧会话渐进恢复
```mermaid
sequenceDiagram
participant U as User
participant Tabs as TopicTabs / Sidebar
participant Hook as useAgentSession
participant Cache as Session Snapshot Cache
participant API as sessionClient
participant Cmd as agent_runtime_get_session
participant Store as SessionStore
participant Projection as ThreadRead Projection
participant ML as MessageList
participant Timeline as AgentThreadTimeline
U->>Tabs: click old session
Tabs->>Hook: switchTopic(sessionId)
Hook->>Cache: read cached snapshot
alt snapshot exists
Cache-->>Hook: title + last messages + status
Hook->>ML: render cached shell/light messages
else no snapshot
Hook->>ML: render shell + skeleton
end
Hook->>API: getSession(sessionId, historyLimit=40)
API->>Cmd: agent_runtime_get_session
Cmd->>Store: get_runtime_session_detail_with_history_page
Store-->>Cmd: messages tail + turns + items
Cmd->>Projection: build thread_read + queue snapshots + cursor
Projection-->>Cmd: session detail
Cmd-->>API: AsterSessionDetail
API-->>Hook: normalized detail
Hook->>ML: hydrate messages first
ML-->>U: stable recent history visible
Hook->>Timeline: defer historical timeline to idle
Timeline-->>U: process detail becomes available
```
验收点：
- `switchTopic` 后 shell 立即出现。
- `historyLimit=40` 是默认恢复路径。
- timeline 的历史构建不阻塞最近消息可读。
- 非活跃 tab 不触发完整 timeline 渲染。
## 3. 运行中 Queue 后续输入
```mermaid
sequenceDiagram
participant U as User
participant Input as Inputbar
participant Hook as useAgentSession
participant API as threadClient
participant Cmd as agent_runtime_submit_turn
participant Queue as runtime_queue
participant State as Frontend State
participant Task as Task Capsule / QueuedTurnsPanel
U->>Input: 当前 turn 运行中输入 follow-up
Input->>Hook: send mode = queue
Hook->>API: submitAgentRuntimeTurn(queueIfBusy=true)
API->>Cmd: agent_runtime_submit_turn
Cmd->>Queue: enqueue turn
Queue-->>State: queue_added
State->>Task: show queued count + preview
U->>Task: promote/remove queued turn
Task->>API: promoteQueuedTurn / removeQueuedTurn
API->>Cmd: queue mutation command
Cmd->>Queue: mutate queue
Queue-->>State: queue_removed / queue_started
State->>Task: update capsule
```
验收点：
- queue 不创建假消息正文。
- queue 操作不会触发旧会话全量恢复。
- 运行中任务和排队任务视觉不同。
## 4. 运行中 Steer 当前任务
```mermaid
sequenceDiagram
participant U as User
participant Input as Inputbar
participant Hook as useAgentSession
participant API as threadClient
participant Cmd as agent_runtime_submit_turn
participant RT as Aster Runtime
participant State as Frontend State
participant UI as Process / Capsule
U->>Input: 当前 turn 运行中输入修正
Input->>Hook: send mode = steer
Hook->>State: show pending steer preview
Hook->>API: submitAgentRuntimeTurn(steer intent)
API->>Cmd: agent_runtime_submit_turn
Cmd->>RT: deliver steer to running turn
RT-->>State: runtime_status steer_received
State->>UI: update current task status
RT-->>State: subsequent text/tool/action events
```
验收点：
- steer 文案明确“影响当前任务”。
- pending steer 可取消或至少可见。
- steer 后状态进入当前 turn，而不是排队 turn。
## 5. Action Required / 权限确认
```mermaid
sequenceDiagram
participant RT as Aster Runtime
participant Bind as AgentEvent Binding
participant State as Frontend State
participant UI as Action Card / Input A2UI
participant User as User
participant API as threadClient
participant Cmd as agent_runtime_respond_action
RT-->>Bind: action_required(requestId, actionType, scope, data)
Bind->>State: pendingActions + threadItems.approval_request
State->>UI: show needs input / approval CTA
User->>UI: approve / reject / fill form
UI->>API: respondAgentRuntimeAction
API->>Cmd: agent_runtime_respond_action
Cmd-->>RT: resolve request
RT-->>Bind: runtime_status / text_delta / tool / done
Bind->>State: mark action resolved
State->>UI: collapse card to summary
```
验收点：
- `needs_input` 在 task capsule 中可见。
- action 完成后不能继续以 pending 状态吸顶。
- 高风险动作必须保留审批结果摘要。
## 6. Artifact Snapshot 到 Workbench
```mermaid
sequenceDiagram
participant RT as Aster Runtime / Tool
participant Bind as AgentEvent Binding
participant Timeline as AgentTimelineRecorder
participant ArtifactSvc as Artifact Services
participant State as Frontend State
participant Msg as Message Preview
participant Workbench as Artifact Workbench
RT-->>Bind: artifact_snapshot(artifactId, path, metadata)
RT-->>Timeline: record file_artifact item
RT-->>ArtifactSvc: persist document / snapshot
ArtifactSvc-->>State: artifact metadata available
Bind->>State: add artifact signal to current turn
State->>Msg: render compact artifact card
State->>Workbench: select or suggest artifact
Workbench->>ArtifactSvc: read preview / version / diff
```
验收点：
- 聊天正文只显示 artifact 摘要卡。
- Workbench 是 artifact 主编辑面。
- timeline 和 evidence 能追到同一个 artifact id/path。
## 7. Evidence Pack 与 Review Decision
```mermaid
sequenceDiagram
participant User as User
participant Harness as HarnessStatusPanel
participant API as export client
participant Cmd as agent_runtime_export_evidence_pack
participant Store as SessionStore
participant Timeline as Timeline DB
participant Evidence as runtime_evidence_pack_service
participant Review as runtime_review_decision_service
participant Files as .lime/harness
User->>Harness: export evidence
Harness->>API: exportEvidencePack(sessionId)
API->>Cmd: agent_runtime_export_evidence_pack
Cmd->>Store: load session detail
Cmd->>Timeline: load turns/items
Cmd->>Evidence: build summary/runtime/timeline/artifacts
Evidence->>Files: write evidence pack
Files-->>Harness: show evidence files
User->>Harness: export review decision
Harness->>Review: build template from evidence/replay/handoff
Review->>Files: write review decision
```
验收点：
- evidence 导出不阻塞当前流式 turn。
- review decision 只保存人工审核，不自动批准或自动应用修复。
- UI 展示的证据路径来自后端返回值。
## 8. 多 Tab / 多历史会话
```mermaid
sequenceDiagram
participant U as User
participant Tabs as Tab Manager
participant Snapshot as Tab Snapshot Store
participant Hook as useAgentSession
participant API as sessionClient
participant UI as Active Workspace
U->>Tabs: open session A
Tabs->>Hook: activate A
Hook->>API: getSession(A, historyLimit=40)
API-->>Hook: detail A
Hook->>Snapshot: save A lightweight snapshot
U->>Tabs: open session B
Tabs->>Hook: freeze A render state
Hook->>Snapshot: keep A summary only
Tabs->>Hook: activate B
Hook->>API: getSession(B, historyLimit=40)
API-->>Hook: detail B
U->>Tabs: switch back A
Tabs->>Snapshot: restore A snapshot immediately
Snapshot-->>UI: show A shell
Hook->>API: optional refresh A detail in background
```
验收点：
- 打开第二个历史会话不会让第一个历史会话继续全量渲染。
- tab 切换先恢复 snapshot，再后台刷新。
- 关闭 tab 释放 MessageList/timeline 重对象。
docs/roadmap/agentui/lime-agentui-target-architecture.md
+194
View File
@@ -0,0 +1,194 @@
# Lime AgentUI 目标架构
> 状态：目标架构
> 更新时间：2026-04-30
> 适用范围：`src/components/agent/chat` 主工作区、`src/lib/api/agentRuntime` 前端协议层、`src-tauri` agent runtime 主链。
## 1. 架构原则
Lime 的下一代 AgentUI 应是一个桌面工作台，而不是单列聊天页。它应当同时回答：
1. 用户当前目标是什么。
2. Agent 正在执行哪一步。
3. 哪些工具、权限、文件、网页、子代理和队列参与了执行。
4. 哪些内容是最终产物。
5. 结果能否被验证、复盘、重放和继续编辑。
因此目标 UI 分为五层：
| 层 | 用户问题 | 主要 UI | 主要事实源 |
| --- | --- | --- | --- |
| 对话层 Conversation | 我和 Agent 说了什么，最终答复是什么 | `MessageList`、`StreamingRenderer`、用户消息、助手正文 | `messages`、`contentParts`、`text_delta` |
| 过程层 Process | Agent 现在在做什么，卡在哪里 | runtime strip、thinking、tool step、timeline | `runtime_status`、`thinking_delta`、`threadItems`、`threadTurns` |
| 任务层 Task | 还有哪些后台任务、排队输入、子代理 | capsule、task center、queue panel、team board | `queuedTurns`、`childSubagentSessions`、`thread_read` |
| 产物层 Artifact | 最终交付物在哪里继续编辑 | canvas、artifact preview、workbench | `artifact_snapshot`、ArtifactDocument、`.lime/artifacts` |
| 证据层 Evidence | 这次执行是否可靠，如何复盘 | harness panel、evidence pack、review decision、replay | `thread_read`、timeline、evidence/review services |
## 2. 总体架构图
```mermaid
flowchart TB
User[用户] --> Shell[AgentChatWorkspace 工作台壳]
Shell --> Composer[输入层 Inputbar]
Shell --> Conversation[对话层 MessageList / StreamingRenderer]
Shell --> Process[过程层 RuntimeStrip / Timeline / Tool Step]
Shell --> Task[任务层 Capsule / Queue / Team / Task Center]
Shell --> Artifact[产物层 Canvas / Artifact Workbench]
Shell --> Evidence[证据层 Harness / Evidence / Review]
Composer --> RuntimeApi[agentRuntime API clients]
RuntimeApi --> Protocol[AgentEvent / Session DTO / ThreadRead]
RuntimeApi --> Tauri[Tauri Commands agent_runtime_*]
Tauri --> RuntimeQueue[Runtime Queue]
Tauri --> SessionStore[Session Store]
Tauri --> TimelineRecorder[AgentTimelineRecorder]
Tauri --> ArtifactServices[Artifact Services]
Tauri --> EvidenceServices[Evidence / Review Services]
RuntimeQueue --> Aster[Aster Agent Runtime]
Aster --> Events[AgentEvent stream]
Events --> Protocol
Events --> TimelineRecorder
Events --> ConversationState[前端 messages / contentParts]
Events --> ProcessState[前端 threadItems / turns / status]
Events --> TaskState[前端 queuedTurns / subagents]
SessionStore --> Db[(SQLite session / timeline)]
TimelineRecorder --> Db
ArtifactServices --> ArtifactFiles[.lime/artifacts]
EvidenceServices --> HarnessFiles[.lime/harness]
ConversationState --> Conversation
ProcessState --> Process
TaskState --> Task
ArtifactFiles --> Artifact
HarnessFiles --> Evidence
```
## 3. UI 布局目标
下一阶段建议把 `AgentChatWorkspace` 从“组件巨石”逐步收敛为稳定布局骨架：
```text
AgentChatWorkspace
AgentWorkspaceShell
SessionChrome
TopicTabs
TaskCapsuleStrip
RuntimeHealthIndicator
AgentMainLayout
ConversationPane
MessageList
StreamingRenderer
Inputbar
WorkbenchPane
ArtifactCanvas
TaskCenter
EvidencePanel
ProcessDrawer
AgentThreadTimeline
ToolDetail
RuntimeDiagnostics
```
这不是要求一次性重写，而是给后续拆分一个归宿。当前可以继续复用 `AgentChatWorkspace.tsx`，但新增能力应尽量下沉到更小的 selector、hook、panel，而不是继续把状态和渲染堆到主文件。
## 4. 五层 UI 的职责边界
### 4.1 对话层
对话层只负责高信噪比协作：
- 用户消息。
- 助手最终答复。
- 必要的流式占位。
- 与最终答复强相关的引用、预览和操作。
对话层不应承担：
- 全量 tool output。
- 全量 thinking 原文。
- 后台队列管理。
- 证据包文件树。
- artifact 编辑器主体。
### 4.2 过程层
过程层负责把“Agent 是否活着、正在做什么”变成低噪声状态：
- 首事件前显示可信阶段：submitted、routing、preparing、waiting provider。
- 首字前显示 `runtime_status`，不要只给鼠标 loading。
- thinking 默认折叠，完成后显示摘要。
- tool 默认压缩为 step，点击进入详情。
- 历史会话默认延迟渲染 timeline。
过程层的关键是分型，不是堆文本。
### 4.3 任务层
任务层负责多会话、多 turn、多子代理的压缩索引：
- `running`：低调胶囊，不抢注意力。
- `queued`：显示队列长度和下一条摘要。
- `needs_input`：高优先级 CTA。
- `plan_ready`：高优先级 CTA，进入计划审批。
- `failed`：可恢复错误胶囊，点击打开诊断。
- `team/subagent`：显示数量、状态和焦点会话。
任务层应学习浏览器 tab 的管理方式：可快速打开、切换、关闭、懒加载、后台冻结，避免多个旧会话同时全量恢复导致 CPU 和内存飙高。
### 4.4 产物层
产物层承接最终交付：
- 文档、代码、报告、图片、视频、网页、表单等不应长期塞在聊天正文里。
- `artifact_snapshot` 应生成 artifact card，并自动同步到 workbench。
- artifact 的编辑、预览、diff、保存、导出由 workbench 负责。
- 聊天消息只保留摘要、解释和打开入口。
### 4.5 证据层
证据层负责证明执行可信：
- `thread_read` 负责待处理请求、last outcome、incident、queue、interrupt。
- timeline 负责过程证据。
- evidence pack 负责导出 `summary.md`、`runtime.json`、`timeline.json`、`artifacts.json`。
- review decision 负责人工审核闭环。
- replay 负责复现失败场景。
证据层默认不出现在首屏正文里，但必须能从任务、timeline 或 harness panel 快速进入。
## 5. 状态所有权
| 状态 | 所有权建议 | 当前关键入口 |
| --- | --- | --- |
| 当前会话、topic、messages | `useAgentSession` 继续作为临时聚合层，逐步拆 selector | `src/components/agent/chat/hooks/useAgentSession.ts` |
| 流式事件绑定 | 独立 stream controller | `agentStreamSubmitExecution.ts`、`agentStreamTurnEventBinding.ts` |
| 流式 reducer | 独立 runtime event reducer | `agentStreamRuntimeHandler.ts` |
| 消息渲染投影 | MessageList selector / memo | `MessageList.tsx`、`threadTimelineView.ts`、`messageTurnGrouping.ts` |
| 队列与 capsule | Task state selector | `queuedTurns`、`TaskCenterTabStrip.tsx`、`QueuedTurnsPanel.tsx` |
| artifact 工作台 | Workbench controller | `useWorkspaceArtifactPreviewActions.ts`、artifact services |
| evidence/review | Harness controller | `HarnessStatusPanel.tsx`、runtime evidence/review services |
## 6. 性能目标
| 场景 | 目标体验 | 架构要求 |
| --- | --- | --- |
| 打开旧会话 | 立即显示 shell 和最近消息骨架，随后渐进 hydrate | `getSession(historyLimit: 40)`，timeline 延迟，full history 分页 |
| 首字前等待 | 用户看到明确阶段，而不是应用卡死 | 先发 `runtime_status`，前端显示 runtime strip |
| 大量 text_delta | 流式平滑，catch-up 时不逐字拖慢 | delta buffer、flush 指标、catch-up mode |
| 多 tab / 多历史会话 | 非活跃 tab 不全量渲染 | tab snapshot、懒加载、后台冻结、关闭释放 |
| 大 tool output | 正文不卡，详情可查 | tool output 截断、offload、timeline 详情 |
| evidence/review | 导出不阻塞聊天 | 后台任务 + 完成胶囊 + workbench 入口 |
## 7. 非目标
短期不做这些事：
- 不新增第二套 event bus。
- 不把所有 timeline 迁到前端本地推断。
- 不把 artifact 编辑器塞回 `MessageList`。
- 不为了视觉新鲜感引入和 Lime 现有设计语言冲突的大面积渐变、半透明主表面或嵌套卡片。
- 不一次性重写整个 `AgentChatWorkspace`；先用 selector、panel、controller 拆出稳定边界。
docs/roadmap/voice/README.md
+96
View File
@@ -0,0 +1,96 @@
# Lime 离线语音模型路线图
> 状态：current planning source
> 更新时间：2026-04-30
> 目标：把截图参考里的“语音模型”能力收敛为 Lime 可实现的离线语音主线：limecore 下发 SenseVoice Small 下载地址、本地 ASR 转写、Fn 按住说话、测试转写与转写历史。
## 1. 本路线图回答什么
本目录统一回答下面几类问题：
1. Lime 如何接入 SenseVoice Small 离线 ASR，而不是继续只依赖云端 ASR 凭证。
2. 离线模型如何通过 limecore 模型清单与对象存储/CDN 按需下载，避免安装包内置大文件。
3. Fn 按住录音、松开转写的快捷键体验如何落到 macOS 桌面主链。
4. 语音设置页如何区分语音输入 ASR、语音润色 LLM 与配音/TTS 服务模型。
5. 新能力如何接回现有 `voice_command_service -> voice_asr_service -> voice-core` 主链，而不是新造第二套语音系统。
## 2. 固定结论
### 2.1 模型不内置进安装包
SenseVoice Small、Silero VAD、Whisper、GGUF 等大模型文件都不进入 App 初始安装包。
安装包只包含：
1. 模型清单读取
2. 下载与校验能力
3. 本地模型状态管理
4. 已安装模型的推理运行时
5. 缺模型时的明确引导
模型必须由用户显式触发下载。后台不能静默拉取几百 MB 的模型文件。
### 2.2 P0 只产品化 SenseVoice Small
P0 只把截图参考中的 `SenseVoice Small` 做成主路径：
1. 通过 `sherpa-onnx` 运行本地 ASR。
2. 通过 limecore `voice-model-catalog` 获取对象存储/CDN 下载地址。
3. 支持中、粤、英、日、韩自动识别。
4. 默认启用 ITN，把语音识别结果归一成更适合输入的文本。
已有 `WhisperLocal` 能力作为现有代码资产保留，但不作为 P0 的产品化下载模型。
### 2.3 Fn 是专门输入模式，不是普通快捷键字符串
现有 `tauri_plugin_global_shortcut` 适合 `CommandOrControl+Shift+V` 这类组合键。Fn 在 macOS 上属于特殊硬件/系统层按键，不能简单塞进现有 shortcut 字符串解析。
P0 固定为：
1. Apple 键盘上优先提供 Fn 按住说话体验。
2. 无法捕获 Fn 或权限不足时，降级到现有自定义快捷键。
3. 第三方键盘默认走自定义快捷键，不承诺 Fn。
## 3. 先读顺序
1. [sensevoice-small-integration.md](./sensevoice-small-integration.md)
2. [fn-dictation-shortcut.md](./fn-dictation-shortcut.md)
## 4. 当前 Lime 事实源
当前仓库已经有语音主链基础：
1. `src-tauri/crates/voice-core` 负责录音、转写核心类型、云端 ASR 客户端与可选 Whisper 本地识别。
2. `src-tauri/crates/services/src/voice_asr_service.rs` 统一管理 ASR 服务。
3. `src-tauri/crates/services/src/voice_command_service.rs` 封装转写、润色和输出流程。
4. `src-tauri/src/voice/shortcut.rs` 负责普通全局快捷键 press/release。
5. `src/components/settings-v2/agent/voice/index.tsx` 已有语音输入、语音处理、语音服务模型三段设置。
6. `src/components/agent/chat/components/Inputbar/hooks/useInputbarDictation.ts` 已把输入栏语音听写接到现有转写主链。
后续实现必须复用这些入口，不新增平行 ASR 流程。
## 5. 当前实现进度
2026-04-30 已推进到可验证主链：
1. `SenseVoice Small` 模型目录、安装状态、下载、删除、设为默认已接到设置页“语音模型”卡片。
2. 模型文件仍按需下载到用户数据目录，不进入 App 安装包；下载地址优先来自 limecore `GET /api/v1/public/tenants/:tenantId/client/voice-model-catalog`，对象存储/CDN 域名由 `server.voiceModelAssetBaseUrl` 管理。
3. `voice_asr_service` 已把 `SenseVoiceLocal` 接到 `voice_core::SenseVoiceTranscriber`，通过 `sherpa-onnx` 执行 non-streaming 本地转写。
4. macOS Fn 按住录音已落第一刀：原生监听 Fn press/release，失败时保留普通快捷键 fallback。
5. 已补“已安装模型后的 WAV 测试转写”入口：设置页可原生选择或手动输入本机 16-bit PCM WAV 路径后，通过 `voice_models_test_transcribe_file` 复用 `voice_asr_service` 真实本地推理链路。
6. 仍未做 P1 能力：视频抽音、实时录音测试、VAD 分段、转写历史、`trigger_mode` / `fn_shortcut_enabled` 配置分流。
## 6. 当前必须避免的误区
1. 把 SenseVoice Small 打进安装包，导致桌面包体积膨胀。
2. 把 Fn 当成普通 shortcut 字符串，导致配置看似成功但实际不可用。
3. 把语音输入 ASR 和配音/TTS 的“语音服务模型”混在同一个设置语义里。
4. 因为接入 `sherpa-onnx` 就绕过 `voice_asr_service` 主链。
5. 缺模型时自动下载，造成不可预期流量和等待。
6. 在 Tauri 客户端保存对象存储 Access Key 或让客户端生成签名 URL；签名或公开域名必须由 limecore 管理。
## 7. 这一步如何服务主线
这套文档的直接主线收益是：
**把 Lime 语音输入从“云端 ASR 凭证 + 普通快捷键”推进到“按需下载本地模型 + Fn 按住说话 + 可测试可删除”的 current 语音主链。**
docs/roadmap/voice/fn-dictation-shortcut.md
+266
View File
@@ -0,0 +1,266 @@
# Fn 语音输入快捷键方案
> 状态：current planning source
> 更新时间：2026-04-30
> 目标：定义截图参考中的 Fn 按住录音、松开转写体验，并明确它与现有普通全局快捷键的边界。
## 1. 固定目标
P0 目标：
1. Apple 键盘上支持 Fn 按住录音。
2. 松开 Fn 后停止录音并触发转写。
3. Fn 不可用时，稳定降级到自定义快捷键。
4. 设置页明确提示 Fn 的系统限制和第三方键盘处理方式。
5. 现有 `CommandOrControl+Shift+V` 类普通快捷键继续工作。
非目标：
1. 不承诺 Windows 上支持 Fn。
2. 不承诺所有第三方键盘支持 Fn。
3. 不把 Fn 写入现有 shortcut 字符串解析器。
4. 不做系统键盘设置自动修改。
## 2. 现有实现边界
当前语音快捷键在 `src-tauri/src/voice/shortcut.rs` 中实现：
1. 使用 `tauri_plugin_global_shortcut`。
2. 通过字符串解析 `Shortcut`。
3. `Pressed` 时打开语音浮窗。
4. `Released` 时发送停止录音事件。
这条路径适合普通组合键：
```text
CommandOrControl+Shift+V
Alt+Space
Control+Option+V
```
Fn 不应进入这条路径。原因：
1. Fn 在 macOS 上经常不是普通 keyboard accelerator。
2. 系统可能把 Fn/地球仪键绑定到输入法、表情、听写或其他系统行为。
3. 第三方键盘对 Fn 的上报方式不统一。
4. 直接注册可能出现“配置成功但无事件”的假成功。
## 3. 配置模型
语音输入配置新增触发模式：
```rust
pub enum VoiceShortcutTriggerMode {
FnHold,
CustomShortcut,
}
```
配置字段：
```rust
pub struct VoiceInputConfig {
pub trigger_mode: VoiceShortcutTriggerMode,
pub shortcut: String,
pub fn_shortcut_enabled: bool,
}
```
默认策略：
| 平台 / 设备 | 默认模式 |
| --- | --- |
| macOS + Apple 键盘 | `FnHold` |
| macOS + 未知键盘 | `CustomShortcut` |
| Windows | `CustomShortcut` |
| Linux | `CustomShortcut` |
`shortcut` 字段继续保留，作为 fallback 与用户自定义快捷键。
## 4. macOS Fn 主路径
### 4.0 当前实现进度
2026-04-30 已落地第一刀：
1. `src-tauri/src/voice/fn_shortcut.rs` 通过 macOS `NSEventMaskFlagsChanged` 监听 Fn 修饰键状态。
2. Fn 按下复用 `open_floating_window_with_voice`，Fn 松开复用 `send_voice_stop_event`，不新增第二套语音窗口流程。
3. 监听随语音输入启用/停用注册和注销；普通语音快捷键仍作为 fallback 保留。
4. 运行时状态已接入 `get_voice_shortcut_runtime_status` 和设置页 Fn 状态展示。
5. 尚未实现本方案里的 `trigger_mode` / `fn_shortcut_enabled` 配置分流；当前是“启用语音输入时同时注册普通快捷键与 macOS Fn 监听”的最小可用形态。
### 4.1 监听模块
新增 macOS-only 模块，例如：
```text
src-tauri/src/voice/fn_shortcut_macos.rs
```
职责：
1. 启动 Fn 事件监听。
2. 判断 press / release。
3. 去抖，避免重复打开语音窗。
4. 把事件转发给现有语音窗口函数。
5. 暴露状态给设置页。
事件行为：
| 事件 | 行为 |
| --- | --- |
| Fn pressed | 打开语音浮窗并开始录音 |
| Fn repeated | 忽略 |
| Fn released | 发送停止录音事件 |
| 监听失败 | 标记 Fn 不可用，降级自定义快捷键 |
### 4.2 权限与系统设置
Fn 监听可能需要系统辅助功能、输入监听或键盘相关权限。实现时必须提供可解释状态：
| 状态 | 含义 | UI 提示 |
| --- | --- | --- |
| `available` | Fn 可监听 | 正常显示 Fn |
| `permission_required` | 缺少权限 | 引导到系统设置授权 |
| `system_reserved` | 系统占用 Fn/地球仪键 | 提示改系统“按地球仪键”为“什么都不做” |
| `unsupported_keyboard` | 当前键盘不支持 | 建议使用自定义快捷键 |
| `unknown_error` | 其他错误 | 保留诊断信息 |
UI 文案沿用截图含义：
> Fn 键仅支持 Apple 键盘（内置或妙控键盘），第三方键盘请点击左侧按钮自定义快捷键。建议在「系统设置 -> 键盘 -> 按地球仪键」中设为「什么都不做」以避免干扰。
### 4.3 降级策略
Fn 不可用时：
1. 不阻断语音功能。
2. 自动显示自定义快捷键配置。
3. 保留现有 `shortcut` 注册路径。
4. 运行时状态说明“Fn 当前不可用，已使用自定义快捷键”。
## 5. 普通快捷键兼容
`src-tauri/src/voice/shortcut.rs` 继续负责普通快捷键。
需要调整：
1. 注册时根据 `trigger_mode` 分流。
2. `FnHold` 模式只启动 macOS Fn 监听。
3. `CustomShortcut` 模式继续注册 `shortcut` 字符串。
4. 切换模式时先注销旧监听，再注册新监听。
5. 如果 Fn 注册失败，尝试注册 `shortcut` fallback。
不改动：
1. 现有 press 打开语音窗逻辑。
2. 现有 release 停止录音事件。
3. 翻译快捷键的普通 shortcut 路径。
翻译模式 P0 不绑定 Fn。Fn 只作为主语音输入触发键。
## 6. 前端设置页
设置页顶部新增 Fn 控制区：
1. Fn 胶囊按钮。
2. 启用开关。
3. 当前运行状态。
4. 系统限制提示。
5. 自定义快捷键 fallback 入口。
状态展示：
| 状态 | UI |
| --- | --- |
| Fn 可用 | 显示绿色可用状态 |
| 权限缺失 | 显示警告和打开系统设置按钮 |
| 系统占用 | 显示系统设置建议 |
| 第三方键盘 | 显示自定义快捷键入口 |
| fallback 生效 | 显示当前 fallback 快捷键 |
交互规则：
1. 用户关闭 Fn 开关后，直接进入 `CustomShortcut`。
2. 用户打开 Fn 开关但不可用时，不覆盖现有可用 shortcut。
3. 保存失败时恢复旧设置。
## 7. 命令与合同
若新增 Tauri command，必须同步四侧：
1. 前端 `safeInvoke(...)`
2. Rust `tauri::generate_handler!`
3. `agentCommandCatalog`
4. `mockPriorityCommands` / `defaultMocks`
建议命令：
| 命令 | 作用 |
| --- | --- |
| `voice_shortcut_get_runtime_status` | 获取 Fn 与普通快捷键运行状态 |
| `voice_shortcut_set_trigger_mode` | 切换 Fn / 自定义快捷键模式 |
| `voice_shortcut_probe_fn` | 探测 Fn 是否可监听 |
| `voice_shortcut_open_keyboard_settings` | 打开系统键盘设置或权限设置 |
运行时状态需要结构化返回：
```json
{
"triggerMode": "fn_hold",
"fnStatus": "permission_required",
"fallbackShortcut": "CommandOrControl+Shift+V",
"registeredShortcut": null,
"message": "需要授权输入监听权限"
}
```
## 8. 测试计划
Rust：
1. `FnHold` 模式不调用普通 shortcut parser。
2. `CustomShortcut` 模式仍注册普通快捷键。
3. Fn 注册失败时 fallback 到 `shortcut`。
4. 切换模式时旧监听被注销。
5. repeated press 不重复打开窗口。
前端：
1. Fn 可用状态展示。
2. 权限缺失提示。
3. 第三方键盘 fallback 提示。
4. 用户关闭 Fn 后显示自定义快捷键。
5. 保存失败恢复旧配置。
GUI：
```bash
npm run verify:gui-smoke
```
合同：
```bash
npm run test:contracts
```
收口：
```bash
npm run verify:local
```
## 9. 验收标准
1. Apple 键盘上按住 Fn 能启动录音，松开后转写。
2. Fn 不可用时，用户能清楚知道原因和 fallback。
3. 第三方键盘用户仍能使用自定义快捷键完成同一语音输入流程。
4. 现有输入栏听写、悬浮语音窗、翻译快捷键不被破坏。
5. 设置页不会展示“Fn 已启用”但运行时实际没有监听的假状态。
## 10. 这一步如何服务主线
Fn 快捷键的主线收益是：
**把语音输入从“需要记住组合键”推进到“按住说话”的低摩擦桌面交互，同时保留普通快捷键作为可靠兜底。**
docs/roadmap/voice/sensevoice-small-integration.md
+346
View File
@@ -0,0 +1,346 @@
# SenseVoice Small 离线 ASR 接入方案
> 状态：current planning source
> 更新时间：2026-04-30
> 目标：定义 SenseVoice Small 在 Lime 中的模型分发、配置、运行时接入、UI 状态和验证口径。
## 1. 固定目标
P0 交付一个可真实使用的本地语音输入模型：
1. 用户在设置页看到 `SenseVoice Small` 本地模型卡。
2. 未安装时，用户手动点击下载。
3. 下载完成后，模型状态变为已安装。
4. 用户可选择音频文件、视频文件或实时录音测试转写。
5. 语音输入主链可把录音交给 SenseVoice Small 本地转写。
6. 用户可以删除本地模型缓存。
非目标：
1. 不在 P0 同时产品化多个离线 ASR 模型。
2. 不把模型文件打包进 App。
3. 不重写现有语音润色、输出和输入栏听写链路。
4. 不在语音模块内实现独立 LLM 调用栈。
## 2. 技术选型
SenseVoice Small 通过 `sherpa-onnx` 接入。
调研结论：
1. `sherpa-onnx` 支持本地离线 ASR、VAD、TTS 等语音任务。
2. SenseVoice Small 有 ONNX INT8 模型包，适合桌面端按需下载。
3. 模型包主文件为 `model.int8.onnx` 与 `tokens.txt`。
4. 支持普通话、粤语、英语、日语、韩语。
5. `use_itn = true` 可做逆文本归一化，更适合输入场景。
6. `silero_vad.onnx` 可作为可选文件，用于长音频和实时录音分段。
参考源：
1. Context7: `/k2-fsa/sherpa-onnx`
2. `sherpa-onnx` Tauri 示例：`tauri-examples/non-streaming-speech-recognition-from-file`
3. `sherpa` SenseVoice 预训练模型文档
4. Context7: `/modelscope/modelscope`，用于确认 ModelScope `snapshot_download` / `modelscope download` 文件下载能力。
5. ModelScope: `https://modelscope.cn/models/iic/SenseVoiceSmall-onnx`，用于确认阿里系 SenseVoice Small ONNX 源。
6. 阿里云 OSS 文档：`https://help.aliyun.com/zh/oss/developer-reference/`，用于确认公开读对象和服务端签名 URL 的分发方式。
## 3. 模型分发
### 3.1 limecore / 对象存储模型清单
limecore control-plane-svc 提供公开客户端接口：
```text
GET /api/v1/public/tenants/:tenantId/client/voice-model-catalog
```
固定结论：
1. 模型对象放在 `/Users/coso/Documents/dev/ai/limecloud/limecore` 对应后端管理范围内，由 control-plane-svc 下发目录。
2. 实际大文件放在对象存储或 CDN 公开域名后；P0 可用阿里云 OSS、Cloudflare R2 或等价静态文件服务，客户端只消费下发 URL。
3. 对象存储 Access Key、Secret、签名逻辑不得进入 Tauri 客户端。
4. 若需要私有桶，后续由 limecore 生成短期 presigned GET URL；P0 先支持公开域名 / 自定义域名。
5. Lime 客户端保留同一组 `model_id`、必需文件与安装目录，不再把 GitHub Release URL 作为产品事实源。
首个模型条目：
```json
{
"version": 1,
"assetBaseURL": "https://models.example.com",
"items": [
{
"id": "sensevoice-small-int8-2024-07-17",
"name": "SenseVoice Small INT8",
"provider": "FunAudioLLM / sherpa-onnx",
"description": "本地离线 ASR，支持中文、英文、日文、韩文和粤语；客户端按需下载到用户数据目录，不随安装包内置。",
"version": "2024-07-17",
"languages": ["zh", "en", "ja", "ko", "yue"],
"runtime": "sherpa-onnx",
"bundled": false,
"sizeBytes": 262144000,
"requiredFiles": ["model.int8.onnx", "tokens.txt", "silero_vad.onnx"],
"download": {
"archive": {
"downloadPath": "voice/sensevoice-small-int8-2024-07-17/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17.tar.bz2",
"downloadUrl": "https://models.example.com/voice/sensevoice-small-int8-2024-07-17/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17.tar.bz2",
"sha256": "<server-provided-sha256>"
},
"vad": {
"modelId": "silero-vad-onnx",
"downloadPath": "voice/silero-vad-onnx/silero_vad.onnx",
"downloadUrl": "https://models.example.com/voice/silero-vad-onnx/silero_vad.onnx"
}
}
}
]
}
```
字段语义：
| 字段 | 含义 |
| --- | --- |
| `id` | 本地安装记录与配置引用的稳定 ID |
| `name` | UI 展示名 |
| `runtime` | 运行时类型，P0 为 `sherpa-onnx` |
| `download.archive.downloadUrl` | limecore 拼装后的模型归档下载地址 |
| `download.archive.sha256` | 可选压缩包校验值；存在时客户端必须校验 |
| `download.vad.downloadUrl` | Silero VAD 文件下载地址 |
| `bundled` | 是否内置到客户端；P0 固定为 `false` |
| `requiredFiles` | 解压后必须存在的文件 |
limecore 配置：
```yaml
server:
voiceModelAssetBaseUrl: "https://models.example.com"
```
可用环境变量：
```bash
SERVER_VOICE_MODEL_ASSET_BASE_URL="https://models.example.com"
VOICE_MODEL_ASSET_BASE_URL="https://models.example.com"
```
阿里系来源调研结论：
1. ModelScope 上存在 `iic/SenseVoiceSmall-onnx`，属于阿里系官方模型源，可通过 `modelscope download`、`snapshot_download` 或 `https://modelscope.cn/api/v1/models/iic/SenseVoiceSmall-onnx/resolve/master/<file>` 下载文件。
2. 该源文件是 `model_quant.onnx`、`tokens.json`、`config.yaml`，不是 sherpa-onnx release 的 `model.int8.onnx`、`tokens.txt` 归档形态。
3. P0 为了不扩散安装器复杂度，推荐在 limecore 侧把可运行形态重新打包并托管到阿里云 OSS / CDN 或 R2；若要直连 ModelScope，需新增 direct-file 安装模式，并把 `tokens.json` 转换为 sherpa-onnx 可读的 `tokens.txt`。
### 3.2 本地存储
模型下载到 Lime 统一数据目录下：
```text
<lime-data-dir>/models/voice/sensevoice-small-int8-2024-07-17/
```
实现要求：
1. 目录必须通过统一目录封装获取，禁止硬编码 macOS 或 Windows 路径。
2. 下载临时文件放在同一数据目录的临时子目录，成功校验后原子移动。
3. 下载完成先校验压缩包 sha256，再校验解压后的必需文件。
4. 校验失败标记为 `corrupted`，不能标记为 `installed`。
5. 删除模型只删除本地模型目录和安装记录，不删除用户的语音配置。
### 3.3 安装状态
客户端至少维护下面状态：
| 状态 | 含义 | UI 行为 |
| --- | --- | --- |
| `not_installed` | 未下载 | 显示下载模型 |
| `downloading` | 下载中 | 显示进度，禁用测试 |
| `verifying` | 校验中 | 禁用删除与测试 |
| `installed` | 已安装 | 可设为默认、测试、删除 |
| `corrupted` | 文件缺失或校验失败 | 显示重新下载 |
| `failed` | 下载失败 | 显示重试 |
## 4. Rust 运行时接入
### 4.0 当前实现进度
2026-04-30 已落地到真实推理主链：
1. `voice-core` 新增 `SenseVoiceTranscriber`，通过 `sherpa-onnx` offline recognizer 加载 `model.int8.onnx` 和 `tokens.txt`。
2. `voice_asr_service` 的 `SenseVoiceLocal` 分支已从占位错误切换为本地转写，输入仍复用现有 `AudioData`。
3. 录音采样率不是 16kHz 时，会通过 sherpa-onnx `LinearResampler` 转为 16kHz mono float samples。
4. 模型文件仍只来自设置页显式下载目录，不进入应用安装包；运行时缺文件会提示回到“设置 -> 语音模型”下载。
5. `voice_models_test_transcribe_file` 已提供已安装模型后的 WAV 文件测试转写入口，读取本机 16-bit PCM WAV 后复用 `AsrService::transcribe`，不新造第二套推理路径。
6. `voice_models_download` 优先使用前端从 limecore `voice-model-catalog` 取得的对象存储/CDN 下载 URL；无前端目录时才读取 `LIME_VOICE_MODEL_CATALOG_URL` / `LIME_VOICE_MODEL_ASSET_BASE_URL` 这类运行时配置。
7. P0 仍是 non-streaming decode；VAD 文件随模型状态校验保留，但本轮推理路径未做分段 VAD。
### 4.1 配置类型
配置层新增本地 ASR provider：
```rust
pub enum AsrProviderType {
WhisperLocal,
SenseVoiceLocal,
Xunfei,
Baidu,
OpenAI,
}
```
新增 SenseVoice 配置：
```rust
pub struct SenseVoiceLocalConfig {
pub model_id: String,
pub model_dir: Option<String>,
pub use_itn: bool,
pub num_threads: u16,
pub vad_model_id: Option<String>,
}
```
默认值：
1. `model_id = "sensevoice-small-int8-2024-07-17"`
2. 识别语言沿用 `AsrCredentialEntry.language`，设为默认模型时写入 `auto`
3. `use_itn = true`
4. `num_threads = 4`
5. `vad_model_id = "silero-vad-onnx"`
### 4.2 服务层
`voice_asr_service.rs` 增加 `SenseVoiceLocal` 分支：
```text
AsrService::transcribe
-> credential.provider == SenseVoiceLocal
-> transcribe_sensevoice_local(...)
-> voice_core::SenseVoiceTranscriber
```
失败口径：
1. 未安装模型：返回“本地 SenseVoice Small 尚未安装，请先在设置 -> 语音模型中下载”。
2. 文件损坏：返回“模型文件校验失败，请重新下载”。
3. 运行时依赖不可用：返回可诊断错误，并提示使用自定义快捷键或云端 ASR 不是解决方向。
### 4.3 voice-core
`voice-core` 新增 `SenseVoiceTranscriber`：
1. 输入接收现有 `AudioData`。
2. 转换为 16kHz mono float samples。
3. 创建 sherpa-onnx offline recognizer。
4. `language = auto`。
5. `use_itn = true`。
6. 输出复用现有 `TranscribeResult`。
P0 只做 non-streaming decode。实时录音体验仍按当前录音完成后转写，不做流式字幕。
### 4.4 sherpa-onnx 集成策略
优先顺序：
1. 优先验证 Rust crate / C API 直连方案。
2. 若动态库打包风险高，使用 sidecar 进程封装 sherpa-onnx 调用。
3. 不管采用哪种方式，对上都只暴露 `voice_core::SenseVoiceTranscriber`，避免 UI 或服务层感知运行时细节。
需要在实现前做最小 spike：
1. macOS aarch64 本地加载模型并转写 wav。
2. Windows x64 本地加载模型并转写 wav。
3. Tauri 打包后动态库可找到。
4. 无模型时错误稳定。
## 5. 前端设置页
设置页新增“语音模型”视图，参考截图组织：
1. 顶部：语音输入快捷键，显示 Fn 模式开关与说明。
2. 模型卡：`SenseVoice Small`、`本地`、简介、大小、安装状态。
3. 操作：下载模型、删除模型、设为默认。
4. 测试转写：P0 已提供原生选择或手动输入本机 WAV 路径；视频文件抽音和实时录音测试保留为 P1。
5. 历史：所有转写历史入口。
文案边界：
1. `语音模型` 指语音输入 ASR 模型。
2. `语音处理` 指转写后的 LLM 润色与翻译。
3. `语音服务模型` 指配音/TTS 生成模型。
这三个概念不能混用。
## 6. 命令与合同
若新增 Tauri command，必须同步四侧：
1. 前端 `safeInvoke(...)`
2. Rust `tauri::generate_handler!`
3. `agentCommandCatalog`
4. `mockPriorityCommands` / `defaultMocks`
建议命令：
| 命令 | 作用 |
| --- | --- |
| `voice_models_list_catalog` | 获取服务端模型清单 |
| `voice_models_get_install_state` | 获取本地安装状态 |
| `voice_models_download` | 下载并校验模型；可接收前端透传的后端目录项 |
| `voice_models_delete` | 删除本地模型 |
| `voice_models_test_transcribe_file` | 文件测试转写 |
下载与删除都必须返回结构化状态，不能只返回字符串。
## 7. 测试计划
Rust：
1. 模型清单解析与平台过滤。
2. sha256 校验失败。
3. 必需文件缺失。
4. 未安装模型时转写失败。
5. 短音频沿用现有错误语义。
6. 已安装模型路径解析。
前端：
1. 模型卡显示未安装、下载中、已安装、损坏、失败。
2. 未安装时测试按钮禁用。
3. 已安装时选择或输入 WAV 路径并点击测试，会调用 `voice_models_test_transcribe_file` 并显示转写结果、采样率与时长。
4. 下载进度展示。
5. 删除模型需要确认。
6. 设为默认后保存 ASR 配置。
合同：
```bash
npm run test:contracts
```
GUI：
```bash
npm run verify:gui-smoke
```
收口：
```bash
npm run verify:local
```
## 8. 验收标准
1. 新安装 App 不包含 SenseVoice 模型文件。
2. 用户能从设置页显式下载模型。
3. 断网但已安装模型时，语音转写可工作。
4. 删除模型后，转写不再假装可用，并提示重新下载。
5. 输入栏听写和悬浮语音窗消费同一条 ASR 主链。
6. 已安装模型后，可以用本机 16-bit PCM WAV 文件做测试转写，证明模型目录和本地推理链路可用。
7. 日志能区分下载失败、模型损坏、运行时加载失败和识别失败。
## 9. 这一步如何服务主线
SenseVoice Small 接入的主线收益是：
**让 Lime 的语音输入在无云端 ASR 凭证、无网络的情况下仍能完成转写，同时不牺牲安装包体积。**
docs/roadmap/warp/README.md
+32 -22
View File
@@ -1,7 +1,7 @@
# Warp 对照下的 Lime 多模态管理路线图
> 状态：current planning source
> 更新时间：2026-04-29
> 状态：current planning source
> 更新时间：2026-04-30
> 目标：吸收 Warp 开源客户端在 Agent Harness、Execution Profile、Artifact、Attachment、Task Index 与 Cloud/Local 分层上的可借鉴原则，把 Lime 的多模态能力收敛成统一运行合同，而不是继续按 `@` 命令和单点 viewer 分散扩张。
## 1. 本路线图回答什么
@@ -143,6 +143,10 @@ Lime 的 artifact graph 不应只有 `document` / `file`。
通用文件只作为兜底，不作为多模态默认主结果。
当前 `browser_session` / `browser_snapshot` 不新建平行 task 协议；Browser Assist tool timeline 先通过 evidence `snapshotIndex.browserActionIndex` 进入可查询索引层，并已在 Harness evidence panel 暴露摘要、通过最小 `browser_replay_viewer` 打开复盘。后续完整交互回放、权限 profile 与截图/DOM/network 深层展开继续消费同一事实源。
当前 `transcript` 已绑定到底层 `audio_transcription` contract；`@转写 / @transcribe / @Audio Extractor` 只是上层入口，前端、Rust metadata、`transcription_generate` task file、CLI 回退入口与 `lime-transcription-worker` 会保留同一份 `audio_transcription` runtime contract snapshot。当前闭环已经能写入 `.lime/tasks/transcription_generate/*.json`，在 payload 下生成 `transcript.pending`，通过 OpenAI-compatible transcription provider seam 回写 `transcript.completed/failed`，并把 transcript 状态/路径/来源/语言/格式/Provider 错误纳入 `list_media_task_artifacts`、聊天任务卡、`.lime/runtime/transcription-generate/*.md` 运行时文档、Evidence Pack `snapshotIndex.transcriptIndex` 与 Replay / grader。第四十三刀已让运行时文档读取 `.lime/runtime/transcripts/*` 文本内容，打开任务卡即可看到可复制校对的转写文本；第四十四刀继续解析 JSON / SRT / VTT transcript 的时间轴与说话人，并在聊天轻卡和运行时文档中展示可逐段编辑校对的段落表；第四十五刀复用 ArtifactDocument 保存链路，保存校对稿时写入 `transcriptCorrection*` / `transcriptSegmentsCorrected` metadata，并明确不改写原始 ASR 输出文件；第四十六刀补上 viewer 内“校对稿已保存”状态卡与 `transcriptCorrectionDiffSummary`，让原文/校对稿的文本长度、段落、说话人数差异可见。后续仍需要更专用的逐段 transcript viewer 交互、更多 ASR adapter 与本地离线 ASR 执行器。
## 4. 目录文档分工
1. [runtime-fact-map.md](./runtime-fact-map.md)
@@ -151,40 +155,46 @@ Lime 的 artifact graph 不应只有 `document` / `file`。
- Phase 1 `ModalityRuntimeContract` 字段语义与机器校验入口。
3. [capability-matrix.md](./capability-matrix.md)
- Phase 2 多模态能力矩阵、模型角色槽位与 capability gap 口径。
4. [implementation-plan.md](./implementation-plan.md)
4. [execution-profile.md](./execution-profile.md)
- Phase 3 / Phase 5 `ModalityExecutionProfile`、executor adapter registry 与治理守卫。
5. [artifact-graph.md](./artifact-graph.md)
- Phase 4 领域化产物图，明确 artifact kind、truth source、viewer、evidence 与 task index 映射。
6. [implementation-plan.md](./implementation-plan.md)
- 分阶段开发计划、改动面、验收输出和验证入口。
5. [evolution-guide.md](./evolution-guide.md)
7. [evolution-guide.md](./evolution-guide.md)
- Lime 自下而上的演进总图、泳道图、阶段门禁和每轮收口模板。
6. [acceptance.md](./acceptance.md)
8. [acceptance.md](./acceptance.md)
- 关键场景验收标准，防止路线图停留在抽象层。
当前机器可检查事实源：
1. `src/lib/governance/modalityRuntimeContracts.json`
2. `src/lib/governance/modalityCapabilityMatrix.json`
3. `scripts/check-modality-runtime-contracts.mjs`
4. `npm run governance:modality-contracts`
3. `src/lib/governance/modalityArtifactGraph.json`
4. `src/lib/governance/modalityExecutionProfiles.json`
5. `src/lib/governance/modalityExecutionProfiles.ts`
6. `scripts/check-modality-runtime-contracts.mjs`
7. `npm run governance:modality-contracts`
后续如果继续推进，再按需新增：
1. `artifact-graph.md`
2. `limecore-integration.md`
3. `browser-computer-use.md`
4. `migration-map.md`
1. `limecore-integration.md`
2. `browser-computer-use.md`
3. `migration-map.md`
## 5. 分阶段总览
| 阶段 | 目标 | 主产物 |
| --- | --- | --- |
| Phase 0 | 盘点底层运行事实源 | runtime fact map |
| Phase 1 | 建底层运行合同 schema | `ModalityRuntimeContract` + governance check |
| Phase 2 | 扩展模型能力矩阵 | modality capability matrix + routing evidence |
| Phase 3 | 建统一 execution profile | model roles + permission profile + tenant override |
| Phase 4 | 领域化 artifact graph | domain artifact kinds + viewer mapping |
| Phase 5 | 建 executor / Browser typed action 边界 | executor adapter + browser evidence |
| Phase 6 | LimeCore 目录与策略接线 | cloud catalog + model offer + Gateway/Scene policy |
| Phase 7 | 绑定上层入口 | `@` / button / scene launch mapping |
| Phase 8 | 任务索引与复盘 | modality task index + audit + replay hooks |
| 阶段 | 目标 | 主产物 |
| ------- | --------------------------------------- | -------------------------------------------------- |
| Phase 0 | 盘点底层运行事实源 | runtime fact map |
| Phase 1 | 建底层运行合同 schema | `ModalityRuntimeContract` + governance check |
| Phase 2 | 扩展模型能力矩阵 | modality capability matrix + routing evidence |
| Phase 3 | 建统一 execution profile | `modalityExecutionProfiles.json` + profile / policy guard |
| Phase 4 | 领域化 artifact graph | domain artifact kinds + viewer mapping |
| Phase 5 | 建 executor / Browser typed action 边界 | executor adapter registry + browser evidence |
| Phase 6 | LimeCore 目录与策略接线 | cloud catalog + model offer + Gateway/Scene policy |
| Phase 7 | 绑定上层入口 | `@` / button / scene launch mapping |
| Phase 8 | 任务索引与复盘 | modality task index + audit + replay hooks |
## 6. 当前必须避免的误区
docs/roadmap/warp/acceptance.md
+13 -9
View File
@@ -1,7 +1,7 @@
# Warp 对照多模态管理验收标准
> 状态：current planning source
> 更新时间：2026-04-29
> 状态：current planning source
> 更新时间：2026-04-30
> 目标：为 [implementation-plan.md](./implementation-plan.md) 提供可验证场景，避免“参考 Warp”停留在架构口号。
## 1. 总体验收口径
@@ -32,6 +32,7 @@
1. 上层入口不得直接创建任务、写 artifact 或决定 viewer。
2. 上层入口不得直接决定模型和权限。
3. 上层入口只能补 launch metadata，并绑定到底层 contract。
4. 找不到 execution profile 或 executor adapter 的 current contract 不能继续绑定入口。
## 2. `@配图` 验收
@@ -62,7 +63,7 @@
4. 执行动作是 typed browser/computer action。
5. 每次关键动作产生 observation：screenshot、DOM、network 或 URL state。
6. timeline 展示真实 browser tool 过程。
7. evidence pack 导出 browser trace。
7. evidence pack 导出 browser trace，并在 `snapshotIndex.browserActionIndex` 中提供 action/session/URL/observation 可查询摘要；Harness evidence panel 能展示该摘要，并能打开最小 `browser_replay_viewer` 复盘最近浏览器动作。
8. 禁用 browser_control 时给出阻断或询问，不回退 WebSearch 假装完成。
禁止：
@@ -93,17 +94,19 @@
必须证明：
1. `@配音` 走 `service_scene_launch(scene_key=voice_runtime)` 或后续同构 contract。
2. `@转写` 走 transcription contract，不和普通文件读取混淆。
2. `@转写` 走 `audio_transcription` contract，不和普通文件读取混淆；前端与 Rust metadata 都保留同一份 runtime contract snapshot。
3. 模型路由分别识别 `voice_generation` / `audio_transcription`。
4. 媒体上传、读取和生成权限进入 profile。
5. 输出区分 `audio_task`、`audio_output`、`transcript`。
6. 音频 artifact 带时长、mime、来源、任务状态。
7. evidence pack 能导出媒体任务与产物。
7. evidence pack 能导出媒体任务与产物；`audio_transcription` 必须把 `transcription_generate` task 与 `transcript` 子产物纳入同一 `snapshotIndex.transcriptIndex`，Replay / grader 能检查 transcript 状态、来源、语言、格式与失败码。
8. 当前转写 worker 必须通过同一 task artifact 回写 `transcript.completed` 或 `transcript.failed`；成功时 transcript 文件落在 `.lime/runtime/transcripts/*`，失败时保留 provider/source/contract 错误码，聊天任务卡与 `.lime/runtime/transcription-generate/*.md` 运行时文档必须优先消费 `list_media_task_artifacts` 的 `transcript_*` snapshot，并在完成态读取 `transcript_path` 指向的文本内容用于内部校对；如果 transcript 文件是 JSON / SRT / VTT，还必须解析时间轴与说话人并在同一运行时文档里展示可逐段编辑校对的段落表。用户保存校对稿时必须落回同一 ArtifactDocument 新版本，并写入 `transcriptCorrection*` / `transcriptSegmentsCorrected` / `transcriptCorrectionDiffSummary` metadata，同时在 viewer 中展示“校对稿已保存”状态；原始 ASR 输出文件保持不可变，不能另写普通文件卡。
禁止：
1. 回流旧本地 TTS 测试命令作为 current。
2. 把音频输出只当通用文件卡。
3. 前端直连 ASR、普通文件读取或 generic file transcript 绕过 `audio_transcription` 合同。
## 6. `@搜索` / `@深搜` / `@研报` 验收
@@ -179,10 +182,11 @@
每次推进本路线图，至少选择相关命令：
1. 文档/contract 改动：`npm run harness:doc-freshness` 或等价文档检查。
2. command/runtime contract 改动：`npm run test:contracts`。
3. UI 可见改动：相关 `*.test.tsx` 与 `npm run verify:gui-smoke`。
4. 模型路由改动：相关 Rust / TS 路由测试与 thread read 断言。
5. LimeCore 接口改动：同步改 LimeCore OpenAPI、SDK、类型与客户端消费测试。
2. contract / profile / artifact graph 改动：`npm run governance:modality-contracts`。
3. command/runtime contract 改动：`npm run test:contracts`。
4. UI 可见改动：相关 `*.test.tsx` 与 `npm run verify:gui-smoke`。
5. 模型路由改动：相关 Rust / TS 路由测试与 thread read 断言。
6. LimeCore 接口改动：同步改 LimeCore OpenAPI、SDK、类型与客户端消费测试。
最终收口前，至少跑：
docs/roadmap/warp/artifact-graph.md
+212
View File
@@ -0,0 +1,212 @@
# 领域化 Artifact Graph
> 状态：current planning source
> 更新时间：2026-04-30
> 目标：把 Warp 路线图 Phase 4 从“artifact kind 列表”推进成可检查事实源，约束多模态结果必须落到领域化产物、viewer、evidence 与 task index，而不是继续退回普通文件卡或聊天文本。
## 1. 事实源
当前机器可检查事实源：
1. Graph：`src/lib/governance/modalityArtifactGraph.json`
2. Contract：`src/lib/governance/modalityRuntimeContracts.json`
3. Check：`scripts/check-modality-runtime-contracts.mjs`
4. npm 入口：`npm run governance:modality-contracts`
本文件解释字段语义；JSON graph 是校验输入。
## 2. 固定原则
1. Artifact kind 描述底层产物，不描述 `@` 命令。
2. Contract 引用的每个 `artifact_kinds` 必须存在于 artifact graph。
3. Contract 的 `truth_source`、`viewer_surface`、`evidence_events` 必须能与对应 artifact kind 对上。
4. `generic_file` 只允许作为兜底，不允许成为图片、音频、浏览器、PDF、报告、PPT、网页等多模态主结果。
5. Viewer 只消费 artifact graph 或 runtime truth source，不从 UI 状态反推执行事实。
6. Evidence / replay / task index 消费同一组 artifact kind，不另建第二套恢复事实源。
## 3. Graph 字段
| 字段 | 说明 |
| ----------------------- | -------------------------------------------------------------------------------- |
| `kind` | artifact kind 主键，例如 `audio_output` |
| `lifecycle` | `current` / `compat` / `deprecated` / `dead` |
| `modality` | `text` / `image` / `audio` / `video` / `browser` / `document` / `code` / `mixed` |
| `implementation_status` | `current` / `partial` / `planned`，描述实现成熟度，不改变生命周期 |
| `truth_sources` | 产物事实源，例如 `audio_task_artifact`、`runtime_timeline_event` |
| `viewer_surfaces` | 允许消费该产物的 viewer surface |
| `evidence_events` | Evidence pack 至少能解释到的 runtime event |
| `task_index_fields` | 进入任务索引 / 复盘 / 诊断的最小字段 |
| `current_contracts` | 当前引用该 artifact kind 的 contract |
| `notes` | 当前缺口或边界说明 |
## 4. 当前 Artifact Graph
| artifact kind | 实现状态 | 当前合同 | viewer | 主缺口 |
| ----------------------- | -------------- | ------------------------------------------------- | -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `image_task` | current | `image_generation` | `image_workbench` | 继续把更多图片子入口收敛到同一 task graph |
| `image_output` | current | `image_generation` | `image_workbench` | 防止普通文件卡重复展示 |
| `audio_task` | current | `voice_generation` | `audio_player` | 后续接 execution profile / provider policy |
| `audio_output` | current | `voice_generation` | `audio_player` | 后续接 Provider 设置修复入口与 LimeCore offer |
| `transcript` | partial | `audio_transcription` | `document_viewer` / 后续 `transcript_viewer` | 已有 `transcription_generate` task writer、`lime-transcription-worker`、`transcript.completed/failed` 回写、媒体任务索引、聊天任务卡、可编辑校对的运行时文档 viewer、JSON/SRT/VTT 时间轴与说话人段落展示、ArtifactDocument 版本化校对稿保存、校对稿状态/差异摘要、Evidence `transcriptIndex` 与 Replay 检查；还缺更专用的逐段 transcript viewer 交互与更多 ASR adapter |
| `browser_session` | partial | `browser_control` | `browser_replay_viewer` | entry binding 已回挂 registry；`snapshotIndex.browserActionIndex` 已进入 Harness evidence 面板与最小复盘 viewer，还缺权限 profile 与完整交互回放 |
| `browser_snapshot` | partial | `browser_control` | `browser_replay_viewer` | observation / screenshot 已进 evidence 索引、Harness 摘要与最小复盘 viewer；截图/DOM/network 深层展开仍需补齐 |
| `pdf_extract` | partial | `pdf_extract` | `document_viewer` | 还缺页码/引用恢复与专属 PDF artifact viewer |
| `report_document` | partial | `pdf_extract` / `web_research` / `text_transform` | `document_viewer` / `report_viewer` | 还缺 report viewer 与来源索引闭环 |
| `presentation_document` | planned | 无 | `presentation_viewer` / `document_viewer` | 还缺 presentation contract |
| `webpage_artifact` | planned | `web_research` | `webpage_viewer` / `document_viewer` | 还缺 webpage artifact / viewer 分离 |
| `generic_file` | current compat | `text_transform` | `generic_file_viewer` / `document_viewer` | 只能兜底，不能继续承载多模态主结果 |
## 5. 运行关系图
```mermaid
flowchart TB
Contract["ModalityRuntimeContract"] --> ArtifactKind["artifact_kinds"]
Contract --> Truth["truth_source"]
Contract --> Viewer["viewer_surface"]
Contract --> Evidence["evidence_events"]
ArtifactKind --> Graph["modalityArtifactGraph.json"]
Graph --> Domain["domain artifact"]
Graph --> Index["task index fields"]
Graph --> Replay["evidence / replay"]
Graph --> Surface["viewer mapping"]
Domain --> Surface
Domain --> Index
Domain --> Replay
```
读法：
1. Contract 声明“会产出什么 kind”。
2. Artifact graph 声明“这个 kind 的事实源、viewer、evidence 和索引字段是什么”。
3. Viewer、task index、replay 都从 artifact graph 消费同一事实，不允许各自猜。
## 6. `voice_generation` 样板
当前最接近闭环的样板是：
```mermaid
sequenceDiagram
participant Entry as @配音 entry
participant Contract as voice_generation contract
participant Task as audio_task artifact
participant Worker as lime-audio-worker
participant Output as audio_output
participant Index as media task index
participant Viewer as audio task viewer
participant Evidence as evidence / replay
Entry->>Contract: harness.service_scene_launch + contract snapshot
Contract->>Task: create audio_generate task
Task->>Worker: standard audio task payload
Worker-->>Task: running / failed / completed
Worker-->>Output: audio_path / mime / duration or provider error
Task-->>Index: audio_output_* snapshot
Index-->>Viewer: restore task card and runtime document
Task-->>Evidence: audioOutput snapshot
Evidence-->>Viewer: replay / review can explain output
```
这条样板已经证明：
1. `audio_task` 与 `audio_output` 不再是普通文件卡。
2. 失败态保留 `audio_provider_*` 错误码，不伪造音频路径。
3. 媒体任务索引可直接恢复任务卡与 viewer。
4. Evidence / replay 能区分 `audio_output.completed` 和 provider 失败。
## 7. `browser_control` 索引闭环
本轮补齐的最小 browser index 不新增 browser task 协议，只消费已有 Browser Assist tool timeline metadata：
```mermaid
sequenceDiagram
participant Entry as @浏览器 entry
participant Contract as browser_control contract
participant Tool as mcp__lime-browser__*
participant Trace as browser_action_trace
participant Index as snapshotIndex.browserActionIndex
participant Harness as Harness evidence panel
participant Viewer as browser_replay_viewer
participant Replay as replay / grader
Entry->>Contract: harness.browser_assist + entry_source
Contract->>Tool: Browser Assist 决策 typed action
Tool-->>Trace: action / backend / session / target / result
Trace-->>Index: browser_session / browser_snapshot summary
Index-->>Harness: action / session / URL / observation 摘要
Harness-->>Viewer: browser_assist Artifact + browserActionIndex
Index-->>Viewer: 最近动作 / URL / session / screenshot
Index-->>Replay: actionCount / sessionCount / lastUrl / observationCount / screenshotCount
```
索引字段先固定为：
1. `actionCount`、`statusCounts`、`actionCounts`：判断是否真的产生 browser action。
2. `sessionIds`、`targetIds`、`profileKeys`、`backendCounts`：定位执行环境和 backend。
3. `lastUrl`：快速恢复最近页面位置。
4. `observationCount`、`screenshotCount`：区分普通 session action 与 snapshot/observation。
5. `items[].artifactKind`：在同一索引内区分 `browser_session` 与 `browser_snapshot`。
这一步把 `browser_control` 从 raw trace 人工扫描推进到可检索诊断层，并让 Harness evidence panel 能直接展示 Browser Assist 摘要、打开最小 `browser_replay_viewer`；后续完整交互回放、权限 profile 可视化与截图/DOM/network 深层展开仍消费同一份 index，不能另起 browser task 文件协议。
## 8. `audio_transcription` Transcript 索引闭环
本轮把 `audio_transcription` 从入口合同推进到最小 transcript task / index / evidence / replay 闭环：
```mermaid
sequenceDiagram
participant Entry as @转写 entry
participant Contract as audio_transcription contract
participant Skill as Skill(transcription_generate)
participant Task as transcription_generate task
participant Transcript as transcript.pending
participant Index as media task index
participant Evidence as snapshotIndex.transcriptIndex
participant Replay as replay / grader
participant Viewer as document_viewer / transcript_viewer
Entry->>Contract: harness.transcription_skill_launch + contract snapshot
Contract->>Skill: Agent 首刀执行转写技能
Skill->>Task: lime_create_transcription_task / CLI fallback
Task-->>Transcript: payload.transcript(status=pending)
Task->>Transcript: lime-transcription-worker -> completed/failed
Task-->>Index: transcript status/path/source/language/outputFormat/errorCode
Task-->>Evidence: transcription_task.modality_runtime_contract
Evidence-->>Replay: transcriptIndex + failure modes
Index-->>Viewer: transcript snapshot + transcript file text/segments -> 任务卡 / 可编辑校对文档 / 时间轴段落表
Viewer-->>Viewer: 保存校对稿 -> ArtifactDocument 新版本 + transcriptCorrection metadata
Viewer-->>Viewer: 校对稿状态卡 + transcriptCorrectionDiffSummary
```
当前已经固定的索引字段：
1. `transcript_count`、`transcript_statuses`、`transcript_error_codes`：判断转写产物是否已进入统一媒体任务索引。
2. `items[].transcript_status`、`transcript_path`、`transcript_source_url`、`transcript_source_path`：定位 transcript 状态、输出路径与来源。
3. `items[].transcript_language`、`transcript_output_format`：保留用户请求或 Provider 返回的语言和输出格式。
4. `snapshotIndex.transcriptIndex.items[]`：在 evidence / replay 中携带 provider、model、worker、errorCode 与 retryable。
这一步把 transcript 从 pending 事实推进到可执行 worker 与用户可见恢复闭环：OpenAI-compatible provider 成功时写入 `.lime/runtime/transcripts/*` 并回写 `transcript.completed`；Provider / source / contract 失败时写入 `transcript.failed` 与明确错误码；聊天区恢复层优先消费 `list_media_task_artifacts` 的 `transcript_*` snapshot，同步任务卡与 `.lime/runtime/transcription-generate/*.md` 运行时文档，只有索引缺失时才回退读取单个 task artifact。第四十三刀继续读取 `transcript_path` 指向的文本文件，把 transcript 内容嵌入运行时文档的 `code_block`，使用户能在 Lime 内部复制和校对文本。第四十四刀继续从 JSON verbose transcript、SRT 与 VTT 解析 `start/end/speaker/text` 段落，把时间轴预览展示到聊天轻卡，并把“转写时间轴（可逐段编辑校对）”表格写入同一个运行时文档。第四十五刀复用 ArtifactDocument 编辑保存链路：用户保存转写文本或时间轴校对结果时，只写回同一运行时文档的新版本和 `transcriptCorrection*` / `transcriptSegmentsCorrected` metadata，不改写原始 ASR 输出文件。第四十六刀继续把保存状态显性化：保存后运行时文档会插入/更新“校对稿已保存”状态卡，并写入 `transcriptCorrectionDiffSummary`，用于展示原文与校对稿的文本长度、段落和说话人数差异。更专用的逐段 transcript viewer、更多 ASR adapter 与本地离线 ASR 后续继续消费同一份 `transcription_generate` task file 和 `transcriptIndex`，不能另起 `generic_file` 或前端直连 ASR 旁路。
## 9. 未完成主线
下一批必须继续补：
1. `browser_session` / `browser_snapshot`：Harness evidence panel 与最小 `browser_replay_viewer` 已能展示摘要；继续把权限 profile、截图/DOM/network 深层展开与完整交互回放接到 `snapshotIndex.browserActionIndex`，形成可操作复盘闭环。
2. `pdf_extract`：页码、引用、来源文件与 viewer 恢复还未形成稳定 artifact graph。
3. `report_document`：研报 / 竞品 / 分析仍缺 report viewer 与来源索引。
4. `transcript`：task writer、OpenAI-compatible ASR worker、完成/失败态回写、媒体任务索引、聊天任务卡、可编辑校对运行时文档 viewer、JSON/SRT/VTT 时间轴与说话人段落展示、ArtifactDocument 版本化校对稿保存、校对稿状态/差异摘要、Evidence `transcriptIndex` 与 Replay 检查已补齐；继续补更专用的逐段 transcript viewer、更多 ASR adapter 与本地离线 ASR，不允许由 `generic_file` 冒充。
5. `presentation_document` / `webpage_artifact`：还缺对应 contract、artifact writer 与 viewer mapping。
6. `generic_file`：继续收口为 compat fallback，不能作为新增多模态能力的默认输出。
## 10. 机器守卫
`npm run governance:modality-contracts` 现在必须检查：
1. 每个已声明 artifact kind 都在 graph 中有定义。
2. 每个 contract 引用的 artifact kind 都能找到 viewer intersection。
3. 每个 contract 引用的 artifact kind 都能找到 truth source intersection。
4. 每个 contract 引用的 artifact kind 都能找到 evidence event intersection。
5. Graph 中的 current / partial artifact 至少声明 `task_id`、`contract_key`、`artifact_kind`、`status` 这组 task index 字段。
这样 Phase 4 不再只停留在文档表格，而是会阻止新 contract 继续把多模态结果丢给未知 viewer、未知 evidence 或 `generic_file` 旁路。
docs/roadmap/warp/contract-schema.md
+10 -2
View File
@@ -10,6 +10,8 @@
- Registry：`src/lib/governance/modalityRuntimeContracts.json`
- Capability Matrix：`src/lib/governance/modalityCapabilityMatrix.json`
- Artifact Graph：`src/lib/governance/modalityArtifactGraph.json`
- Execution Profiles：`src/lib/governance/modalityExecutionProfiles.json`
- Check：`scripts/check-modality-runtime-contracts.mjs`
- npm 入口：`npm run governance:modality-contracts`
@@ -24,6 +26,8 @@
5. `required_capabilities` 必须全部出现在 [capability-matrix.md](./capability-matrix.md)。
6. `routing_slot` 必须出现在 capability matrix 的 `model_roles`。
7. contract 不允许引用不存在的 artifact kind、viewer surface、permission key 或 evidence event。
8. contract 引用的 artifact kind 必须能在 artifact graph 中找到 truth source / viewer / evidence 交集。
9. current contract 必须能在 [execution-profile.md](./execution-profile.md) 中找到 profile 与 executor adapter 覆盖。
## 3. 顶层结构
@@ -107,13 +111,15 @@ Phase 7 才允许大量补 entry binding。
## 7. 首批 contract
首批 registry 先覆盖底层能力；`image_generation` 已作为第一条 vertical slice 进入 Phase 7 entry binding：
首批 registry 先覆盖底层能力；进入 current 的 contract 必须继续被 capability matrix、artifact graph、execution profile 与 executor adapter registry 同步覆盖：
1. `image_generation`
2. `browser_control`
3. `pdf_extract`
4. `voice_generation`
5. `web_research`
5. `audio_transcription`
6. `web_research`
7. `text_transform`
这些 contract 用于验证 schema 和治理守卫；后续 vertical slice 继续按“先底层、后 entry binding”的顺序推进。
@@ -132,3 +138,5 @@ npm run governance:modality-contracts
5. executor binding 声明能力与 failure mapping。
6. entry binding 不携带底层事实源字段。
7. entry binding 必须声明 `entry_source`，且 `launch_metadata_path` 必须留在 `harness.*`。
8. `artifact_kinds` 必须存在于 artifact graph，并与 contract 的 truth source、viewer、evidence 至少各有一个交集。
9. current contract 必须被 execution profile 覆盖，且 `executor_binding` 必须能解析到已声明 executor adapter。
docs/roadmap/warp/evolution-guide.md
+32 -26
View File
@@ -1,7 +1,7 @@
# Lime 多模态演进迭代指南
> 状态：current planning source
> 更新时间：2026-04-29
> 状态：current planning source
> 更新时间：2026-04-30
> 目标：把 Warp / ClaudeCode 参考转成 Lime 可持续演进的工程节奏，明确每一阶段先交付什么、如何验收、何时允许接上层入口。
## 1. 演进原则
@@ -148,6 +148,8 @@ sequenceDiagram
participant Browser as Browser Executor
participant Obs as Observation
participant Evidence as Evidence
participant Index as browserActionIndex
participant Harness as Harness evidence panel
participant Viewer as Browser replay viewer
participant Entry as @浏览器 binding
@@ -156,7 +158,10 @@ sequenceDiagram
Runtime->>Browser: typed action
Browser-->>Obs: screenshot / DOM / URL / network
Obs-->>Evidence: browser trace
Evidence-->>Viewer: replay material
Evidence-->>Index: action / session / URL / observation summary
Index-->>Harness: visible evidence summary
Harness-->>Viewer: browser_assist artifact preview
Index-->>Viewer: replay material
Runtime-->>Entry: 允许绑定 @浏览器
```
@@ -164,17 +169,17 @@ sequenceDiagram
## 6. 阶段门禁
| 阶段 | 允许进入下一阶段的条件 | 不允许的捷径 |
| --- | --- | --- |
| P0 Fact Map | 每个底层事实源有 owner、读写方、持久化和 evidence 关系 | 先从 `@` 命令盘点开始 |
| P1 Contract | Contract 以底层能力为主键，引用真实 truth source / artifact / viewer | 用入口名当 contract 主键 |
| P2 Capability | 路由能输出候选、唯一候选、候选为空和能力缺口 | 只看 provider/model id |
| P3 Profile | 模型、权限、租户策略、用户锁定能合并解释 | skill 内部临时判断权限 |
| P4 Artifact | domain artifact 能被 viewer 和 evidence 共同消费 | 所有结果写成 generic file |
| P5 Executor | executor 声明 progress/cancel/resume/artifact/failure mapping | 自由 Bash 或裸 CLI |
| P6 LimeCore | catalog/policy/offer/audit 能约束本地执行 | LimeCore 默认代跑所有入口 |
| P7 Entry | 入口只提交 metadata 并绑定 contract | 入口直建 task / artifact / viewer |
| P8 Index | task index 能按 contract、entry、modality、artifact 查询 | 只靠聊天线程恢复 |
| 阶段 | 允许进入下一阶段的条件 | 不允许的捷径 |
| ------------- | -------------------------------------------------------------------- | --------------------------------- |
| P0 Fact Map | 每个底层事实源有 owner、读写方、持久化和 evidence 关系 | 先从 `@` 命令盘点开始 |
| P1 Contract | Contract 以底层能力为主键，引用真实 truth source / artifact / viewer | 用入口名当 contract 主键 |
| P2 Capability | 路由能输出候选、唯一候选、候选为空和能力缺口 | 只看 provider/model id |
| P3 Profile | `modalityExecutionProfiles.json` 覆盖 current contracts；模型、权限、租户策略、用户锁定能合并解释 | skill 内部临时判断权限 |
| P4 Artifact | domain artifact 能被 viewer 和 evidence 共同消费 | 所有结果写成 generic file |
| P5 Executor | executor adapter registry 对齐 contract 绑定、支持位、产物、权限与 failure mapping | 自由 Bash 或裸 CLI |
| P6 LimeCore | catalog/policy/offer/audit 能约束本地执行 | LimeCore 默认代跑所有入口 |
| P7 Entry | 入口只提交 metadata 并绑定 contract | 入口直建 task / artifact / viewer |
| P8 Index | task index 能按 contract、entry、modality、artifact 查询 | 只靠聊天线程恢复 |
## 7. 迭代选择流程
@@ -206,17 +211,17 @@ flowchart TB
## 8. 文档产物演进表
| 阶段 | 文档产物 | 代码产物 | 验证入口 |
| --- | --- | --- | --- |
| P0 | `runtime-fact-map.md` | 无或只读脚本 | 链接/owner 一致性检查 |
| P1 | `contract-schema.md` | schema / governance check | contract check |
| P2 | capability matrix doc | routing 类型和证据 | 路由单测 / thread read 断言 |
| P3 | execution profile doc | profile merge / policy source | 权限与降级测试 |
| P4 | `artifact-graph.md` | artifact kind / viewer mapping | artifact/viewer 回归 |
| P5 | executor adapter doc | executor registry / browser action | executor / browser trace 测试 |
| P6 | `limecore-integration.md` | catalog/policy SDK 消费 | Lime + LimeCore contract test |
| P7 | entry binding inventory | `@` / button / scene binding | acceptance 场景 |
| P8 | task index doc | index / replay / audit query | evidence pack / replay 测试 |
| 阶段 | 文档产物 | 代码产物 | 验证入口 |
| ---- | -------------------------------------------------- | ------------------------------------------------- | ------------------------------------------ |
| P0 | `runtime-fact-map.md` | 无或只读脚本 | 链接/owner 一致性检查 |
| P1 | `contract-schema.md` | schema / governance check | contract check |
| P2 | capability matrix doc | routing 类型和证据 | 路由单测 / thread read 断言 |
| P3 | `execution-profile.md` | `modalityExecutionProfiles.json` / profile merge / policy source | profile registry 守卫 + 权限与降级测试 |
| P4 | `artifact-graph.md` + `modalityArtifactGraph.json` | artifact kind / viewer / evidence / index mapping | artifact graph 守卫 + artifact/viewer 回归 |
| P5 | `execution-profile.md` 的 executor adapter 章节 | executor adapter registry / browser action | adapter registry 守卫 + executor / browser trace 测试 |
| P6 | `limecore-integration.md` | catalog/policy SDK 消费 | Lime + LimeCore contract test |
| P7 | entry binding inventory | `@` / button / scene binding | acceptance 场景 |
| P8 | task index doc | index / replay / audit query | evidence pack / replay 测试 |
## 9. 版本演进目标
@@ -228,7 +233,8 @@ flowchart TB
1. `runtime-fact-map.md` 存在。
2. `contract-schema.md` 存在。
3. 至少一个 contract 完成 capability/profile/artifact/evidence 字段。
3. `capability-matrix.md`、`execution-profile.md`、`artifact-graph.md` 存在。
4. 至少一个 contract 完成 capability/profile/artifact/evidence 字段。
### V1：单能力闭环
docs/roadmap/warp/execution-profile.md
+154
View File
@@ -0,0 +1,154 @@
# ModalityExecutionProfile 与 Executor Adapter
> 状态：current planning source
> 更新时间：2026-04-30
> 目标：把 Warp 路线图 Phase 3 / Phase 5 从散文约束推进成可机器检查的 profile 与 executor adapter registry，确保每个 current 多模态合同都能解释模型角色、权限、执行器、产物策略、LimeCore 策略引用和失败映射。
## 1. 事实源
当前机器可检查事实源：
1. Profile registry：`src/lib/governance/modalityExecutionProfiles.json`
2. Contract registry：`src/lib/governance/modalityRuntimeContracts.json`
3. Capability matrix：`src/lib/governance/modalityCapabilityMatrix.json`
4. Artifact graph：`src/lib/governance/modalityArtifactGraph.json`
5. TS resolver：`src/lib/governance/modalityExecutionProfiles.ts`
6. Check：`scripts/check-modality-runtime-contracts.mjs`
7. npm 入口：`npm run governance:modality-contracts`
本文件解释字段语义；JSON registry 是校验输入。当前已建立治理事实源与前端 TS resolver，所有 current contract 的 launch metadata 可以携带 `execution_profile` 与 `executor_adapter` 快照；仍不新增 Tauri command、bridge、mock 或具体 Skill / ServiceSkill / Browser executor 分支。
## 2. 固定原则
1. Execution profile 描述底层运行策略，不描述 `@` 命令。
2. 每个 `current` contract 必须被至少一个 execution profile 覆盖。
3. Contract 的 `routing_slot`、`permission_profile_keys`、`limecore_policy_refs`、`artifact_kinds` 与 `executor_binding` 必须能在 profile / adapter registry 中找到对应声明。
4. Executor adapter 的 `adapter_key` 必须等于 `executor_kind:binding_key`，并与 contract 的 `executor_binding` 对齐。
5. Adapter 的 progress / cancel / resume / artifact 支持位必须和 contract 一致，不能伪造能力。
6. `generic_file` 只能作为 compat fallback；新增多模态主结果必须优先落到领域化 artifact kind。
7. LimeCore 在本阶段只作为 catalog / policy / offer / audit 引用，不是默认执行器。
## 3. Profile 字段
| 字段 | 说明 |
| --- | --- |
| `profile_key` | profile 主键，例如 `audio_transcription_profile` |
| `lifecycle` | `current` / `compat` / `deprecated` / `dead` |
| `supported_contracts` | 该 profile 覆盖的底层合同 |
| `model_role_slots` | 对应 capability matrix 的模型角色槽位 |
| `permission_profile_keys` | 运行前必须合并解释的权限面 |
| `executor_adapter_keys` | 允许调用的 executor adapter |
| `artifact_policy.write_mode` | 产物写入模式，例如 `domain_task_artifact` |
| `artifact_policy.artifact_kinds` | 允许写出的领域产物 |
| `artifact_policy.viewer_surfaces` | 允许消费该产物的 viewer surface |
| `limecore_policy_refs` | LimeCore 控制面引用，例如 `model_catalog`、`provider_offer`、`tenant_feature_flags` |
| `user_lock_policy` | 用户显式模型锁定的处理规则 |
| `fallback_behavior` | 权限、能力、执行器或来源失败时的降级 / 阻断口径 |
| `evidence_events` | profile 决策至少需要解释到的 evidence event |
| `audit_fields` | 后续 thread read / audit / evidence 需要携带的字段 |
| `notes` | 当前边界与后续缺口 |
## 4. Executor Adapter 字段
| 字段 | 说明 |
| --- | --- |
| `adapter_key` | `executor_kind:binding_key`，例如 `skill:transcription_generate` |
| `lifecycle` | 生命周期分类 |
| `executor_kind` | `skill` / `tool` / `service_skill` / `browser` / `gateway` / `scene_cloud` / `local_cli` |
| `binding_key` | 执行器在 runtime 中的绑定名 |
| `supported_contracts` | 该 adapter 允许服务的底层合同 |
| `supports_progress` | 是否能报告进度 |
| `supports_cancel` | 是否能取消 |
| `supports_resume` | 是否能恢复 |
| `supports_artifact` | 是否能写标准 artifact |
| `artifact_output_kinds` | adapter 可以写出的 artifact kind |
| `permission_requirements` | adapter 所需权限 |
| `credential_requirements` | adapter 所需凭证或云控制面引用 |
| `failure_mapping` | 失败必须映射到的标准原因 |
| `evidence_events` | adapter 执行至少需要解释到的 evidence event |
| `notes` | 当前实现边界 |
## 5. 当前覆盖
| contract | profile | executor adapter | artifact policy | 当前说明 |
| --- | --- | --- | --- | --- |
| `image_generation` | `image_generation_profile` | `skill:image_generate` | `image_task` / `image_output` | 图片生成只能写标准 image task/output，不回退 legacy CLI |
| `browser_control` | `browser_control_profile` | `browser:browser_assist` | `browser_session` / `browser_snapshot` | 浏览器动作必须保留 typed action 与 observation trace，不降级 WebSearch |
| `pdf_extract` | `pdf_extract_profile` | `skill:pdf_read` | `pdf_extract` / `report_document` | PDF 读取必须保留文件读取证据，页码/引用 viewer 后续补齐 |
| `voice_generation` | `voice_generation_profile` | `service_skill:voice_runtime` | `audio_task` / `audio_output` | 本地 ServiceSkill/worker 写音频任务，不把 LimeCore 当默认执行器 |
| `audio_transcription` | `audio_transcription_profile` | `skill:transcription_generate` | `transcript` | 转写固定走 transcription task / transcriptIndex，不走 frontend ASR 或 generic_file |
| `web_research` | `web_research_profile` | `skill:research` | `report_document` / `webpage_artifact` | 联网研究保留搜索来源与报告型产物，来源索引后续继续补 |
| `text_transform` | `text_transform_profile` | `skill:text_transform` | `report_document` / `generic_file` | `generic_file` 只保留为 compat fallback，主结果继续向 document viewer 收敛 |
## 6. 决策流程
```mermaid
flowchart TB
Contract["ModalityRuntimeContract"] --> Profile["ModalityExecutionProfile"]
Matrix["Capability Matrix\nmodel_roles"] --> Profile
Policy["LimeCore policy refs\nmodel_catalog / offer / tenant flags"] --> Profile
UserLock["User explicit model lock"] --> Profile
Profile --> Decision{"allow / ask / deny"}
Decision -->|allow| Adapter["Executor Adapter"]
Decision -->|ask| Ask["ask_user_question"]
Decision -->|deny| Gap["permission / capability / policy gap"]
Adapter --> Artifact["Domain artifact"]
Artifact --> Evidence["evidence / replay / task index"]
Evidence --> Viewer["viewer surface"]
```
读法：
1. Contract 声明底层能力需求。
2. Profile 合并模型角色、权限、LimeCore 策略、用户锁定与 fallback。
3. 前端 launch metadata 先携带 profile / adapter 快照，后续 Rust runtime preflight 再消费同一事实源执行 allow / ask / deny。
4. Adapter 只在 profile 允许后执行，并按声明写 domain artifact。
5. Evidence / task index / viewer 消费同一执行与产物事实。
## 7. Browser adapter 时序
```mermaid
sequenceDiagram
participant Contract as browser_control contract
participant Profile as browser_control_profile
participant Adapter as browser:browser_assist
participant Action as Typed browser action
participant Observation as screenshot / DOM / URL / network
participant Evidence as browserActionIndex
participant Viewer as browser_replay_viewer
Contract->>Profile: routing_slot + browser_control permission + gateway_policy
Profile-->>Contract: allow / ask / deny
Contract->>Adapter: executor_binding browser:browser_assist
Adapter->>Action: plan typed action
Action-->>Observation: collect observation
Observation-->>Evidence: browser_session / browser_snapshot
Evidence-->>Viewer: replay summary and latest artifacts
```
固定约束：如果 profile 未允许 `browser_control`，不得把浏览器任务改写成普通 WebSearch；如果 adapter 没有 observation，就不能写 `browser_snapshot`。
## 8. 机器守卫
`npm run governance:modality-contracts` 现在必须检查：
1. `modalityExecutionProfiles.json` 存在且 `version/status/owner` 正确。
2. Profile / adapter 主键唯一。
3. Profile 引用的 contract、model role、permission、adapter、artifact、viewer、LimeCore policy 与 evidence event 必须存在。
4. 每个 current contract 必须被 profile 覆盖。
5. 每个 current contract 的 `executor_binding` 必须存在于 `executor_adapters`。
6. Adapter 的支持位、artifact output、permission requirements、failure mapping 必须覆盖 contract 声明。
7. Profile 的 model role、permission、LimeCore policy 与 artifact policy 必须覆盖 contract 声明。
这一步把 Phase 3 / Phase 5 从“应该有 profile / executor adapter”推进成会阻断错误 contract 的机器事实源。
## 9. 未完成主线
后续继续补：
1. Rust / Agent 运行时真实 `ExecutionProfile` merge：把 `modalityExecutionProfiles.json` 或其生成快照接入 `TaskProfile`、权限判断、用户模型锁定和 thread read。
2. LimeCore policy snapshot：把 `model_catalog`、`provider_offer`、`tenant_feature_flags`、`gateway_policy` 的实际命中值写回 profile decision。
3. GUI / evidence 可视化：在 Harness evidence、任务卡或 viewer 中展示 profile allow / ask / deny、adapter key 和 policy gap。
4. Executor registry 运行时化：让 Skill / ServiceSkill / Browser / Gateway 的 adapter 能从同一事实源生成执行前检查。
5. Task index 统一层：把 `executor_kind`、`adapter_key`、`profile_key`、`policy_snapshot` 纳入统一查询字段。
6. LimeCore Phase 6：继续接目录、offer、Gateway/Scene policy 与 audit，不把 LimeCore 扩张成默认 executor。
docs/roadmap/warp/implementation-plan.md
+12 -4
View File
@@ -1,7 +1,7 @@
# Warp 对照多模态管理实施计划
> 状态：current planning source
> 更新时间：2026-04-29
> 状态：current planning source
> 更新时间：2026-04-30
> 目标：把 [README.md](./README.md) 的路线图拆成自下而上的执行阶段，先建设底层多模态运行合同，再把 `@` 命令、按钮和 Scene 这类上层入口绑定上来。
## 0. 排序修正
@@ -168,6 +168,8 @@
## Phase 3：ModalityExecutionProfile
当前落点：见 [execution-profile.md](./execution-profile.md)、`src/lib/governance/modalityExecutionProfiles.json` 与 `src/lib/governance/modalityExecutionProfiles.ts`；最小 profile / executor adapter registry 与前端 launch metadata 快照已落地，真实 Rust runtime policy merge、tenant policy snapshot 与 GUI/evidence 可视化仍待继续。
### 目标
建立 Warp `AIExecutionProfile` 式的 Lime profile，把模型角色、权限策略、执行器策略合并到一个解释层。
@@ -220,9 +222,12 @@
2. 租户禁止媒体上传时，图片/音频 contract 给出可解释降级。
3. 本地 explicit model lock 优先于自动优化，但能力不匹配时必须给出阻断原因。
4. Profile 决策写入 thread read 与 evidence，不只停留在设置页。
5. 每个 current contract 都必须被 `modalityExecutionProfiles.json` 覆盖，且 profile 的模型角色、权限、LimeCore policy、artifact policy 与 contract 对齐。
## Phase 4：领域化 Artifact Graph
当前落点：见 [artifact-graph.md](./artifact-graph.md) 与 `src/lib/governance/modalityArtifactGraph.json`。
### 目标
让多模态结果拥有领域 kind、恢复路径和 viewer 映射；上层入口只消费 artifact graph。
@@ -247,7 +252,7 @@
1. `generic_file` 只做兜底。
2. 二进制图片输出不自动镜像成普通文件卡。
3. viewer 只消费 artifact graph，不自己猜类型。
4. evidence pack 导出 artifact kind 和关联键。
4. evidence pack 导出 artifact kind、关联键与可查询索引；`browser_control` 当前索引事实源是 `snapshotIndex.browserActionIndex`。
### 验收
@@ -259,6 +264,8 @@
## Phase 5：Executor Adapter 与 Browser typed action
当前落点：见 [execution-profile.md](./execution-profile.md) 与 `src/lib/governance/modalityExecutionProfiles.json` 的 `executor_adapters`；最小 adapter registry 已覆盖 `image_generation`、`browser_control`、`pdf_extract`、`voice_generation`、`audio_transcription`、`web_research`、`text_transform`，前端 runtime contract snapshot 已携带 `executor_adapter`，Rust runtime adapter preflight 仍待继续。
### 目标
先定义执行器边界，再允许上层入口调用它们；借鉴 Warp harness 和 computer use，但不把外部 CLI 作为 current 捷径。
@@ -293,8 +300,9 @@ Browser Assist 必须收成：
1. 浏览器操作不会退回普通 WebSearch。
2. 每次浏览器动作都有 action summary 与 observation。
3. 高风险动作按 profile 决定自动、询问或阻断。
4. evidence pack 能导出 browser trace。
4. evidence pack 能导出 browser trace，并在 `browserActionIndex` 汇总 action/session/URL/observation/screenshot；Harness evidence panel 能展示同一索引摘要，并能打开最小 `browser_replay_viewer`。
5. `local_cli` adapter 不支持 progress/resume/artifact 时必须显式标注，不可伪造 current 能力。
6. 每个 current contract 的 `executor_binding` 必须能解析到 `executor_kind:binding_key` adapter，且 adapter 的 progress/cancel/resume/artifact 支持位、产物、权限与失败映射必须覆盖 contract。
## Phase 6：LimeCore 目录与策略接线
package-lock.json
Generated
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "lime",
"version": "1.24.0",
"version": "1.25.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "lime",
"version": "1.24.0",
"version": "1.25.0",
"dependencies": {
"@babel/standalone": "^7.29.0",
"@fabianlars/tauri-plugin-oauth": "^2",
package.json
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "lime",
"private": true,
"version": "1.24.0",
"version": "1.25.0",
"type": "module",
"engines": {
"node": ">=22.0.0"
packages/lime-cli-npm/README.md
+1 -1
View File
@@ -112,7 +112,7 @@ npm run build:release -- \
```bash
npm run build:release -- \
--target-triple "aarch64-apple-darwin" \
--version "1.24.0" \
--version "1.25.0" \
--out-dir "./dist"
```
packages/lime-cli-npm/package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@limecloud/lime-cli",
"version": "1.24.0",
"version": "1.25.0",
"description": "Lime 官方任务 CLI",
"bin": {
"lime": "scripts/run.js"
scripts/check-modality-runtime-contracts.mjs
+552 -4
View File
@@ -7,10 +7,15 @@ import process from "node:process";
const CONTRACT_PATH = "src/lib/governance/modalityRuntimeContracts.json";
const CAPABILITY_MATRIX_PATH =
"src/lib/governance/modalityCapabilityMatrix.json";
const ARTIFACT_GRAPH_PATH = "src/lib/governance/modalityArtifactGraph.json";
const EXECUTION_PROFILE_PATH =
"src/lib/governance/modalityExecutionProfiles.json";
const REQUIRED_DOCS = [
"docs/roadmap/warp/runtime-fact-map.md",
"docs/roadmap/warp/contract-schema.md",
"docs/roadmap/warp/capability-matrix.md",
"docs/roadmap/warp/execution-profile.md",
"docs/roadmap/warp/artifact-graph.md",
"docs/roadmap/warp/evolution-guide.md",
];
@@ -105,6 +110,24 @@ const FAILURE_REASONS = new Set([
"file_unavailable",
"source_unavailable",
]);
const PROFILE_WRITE_MODES = new Set([
"domain_task_artifact",
"domain_document_artifact",
"runtime_observation_trace",
"timeline_backed_document_artifact",
"document_or_compat_file_artifact",
]);
const ARTIFACT_IMPLEMENTATION_STATUSES = new Set([
"current",
"partial",
"planned",
]);
const REQUIRED_ARTIFACT_INDEX_FIELDS = new Set([
"task_id",
"contract_key",
"artifact_kind",
"status",
]);
const ENTRY_KINDS = new Set([
"command",
"button_action",
@@ -431,7 +454,515 @@ function validateCapabilityMatrix(matrix) {
};
}
function validateContractRegistry(registry, matrixRefs) {
function validateArtifactGraph(graph) {
const errors = [];
pushIf(errors, graph.version !== 1, "artifactGraph.version must be 1");
pushIf(
errors,
graph.status !== "current",
"artifactGraph.status must be current",
);
pushIf(
errors,
!isNonEmptyString(graph.owner),
"artifactGraph.owner must be set",
);
const artifactMap = collectUniqueObjects(
errors,
graph.artifact_kinds,
"kind",
"artifactGraph.artifact_kinds",
);
for (const kind of ARTIFACT_KINDS) {
pushIf(
errors,
!artifactMap.has(kind),
`artifactGraph.artifact_kinds is missing known artifact kind: ${kind}`,
);
}
for (const [kind, artifact] of artifactMap.entries()) {
pushIf(
errors,
!ARTIFACT_KINDS.has(kind),
`artifact ${kind}.kind is unknown: ${kind}`,
);
pushIf(
errors,
!LIFECYCLES.has(artifact.lifecycle),
`artifact ${kind}.lifecycle is unknown: ${String(artifact.lifecycle)}`,
);
pushIf(
errors,
!MODALITIES.has(artifact.modality),
`artifact ${kind}.modality is unknown: ${String(artifact.modality)}`,
);
pushIf(
errors,
!ARTIFACT_IMPLEMENTATION_STATUSES.has(artifact.implementation_status),
`artifact ${kind}.implementation_status is unknown: ${String(artifact.implementation_status)}`,
);
validateStringArray(errors, kind, "truth_sources", artifact.truth_sources);
validateEnumArray(
errors,
kind,
"viewer_surfaces",
artifact.viewer_surfaces,
VIEWER_SURFACES,
);
validateEnumArray(
errors,
kind,
"evidence_events",
artifact.evidence_events,
EVIDENCE_EVENTS,
);
validateStringArray(
errors,
kind,
"task_index_fields",
artifact.task_index_fields,
);
validateStringArray(
errors,
kind,
"current_contracts",
artifact.current_contracts,
{ allowEmpty: true },
);
pushIf(
errors,
!isNonEmptyString(artifact.notes),
`artifact ${kind}.notes must be set`,
);
if (
artifact.implementation_status === "current" ||
artifact.implementation_status === "partial"
) {
const indexFields = new Set(
Array.isArray(artifact.task_index_fields)
? artifact.task_index_fields
: [],
);
for (const requiredField of REQUIRED_ARTIFACT_INDEX_FIELDS) {
pushIf(
errors,
!indexFields.has(requiredField),
`artifact ${kind}.task_index_fields must include ${requiredField}`,
);
}
}
}
return {
errors,
artifactMap,
};
}
function hasIntersection(left, right) {
if (!Array.isArray(left) || !Array.isArray(right)) {
return false;
}
const rightSet = new Set(right);
return left.some((value) => rightSet.has(value));
}
function validateContractArtifactGraph(errors, contract, artifactGraphRefs) {
const contractKey = contract.contract_key;
if (!Array.isArray(contract.artifact_kinds)) {
return;
}
for (const artifactKind of contract.artifact_kinds) {
const artifact = artifactGraphRefs.artifactMap.get(artifactKind);
if (!artifact) {
errors.push(`${contractKey}.artifact_kinds references artifact not defined in graph: ${artifactKind}`);
continue;
}
pushIf(
errors,
Array.isArray(artifact.current_contracts) &&
artifact.current_contracts.length > 0 &&
!artifact.current_contracts.includes(contractKey),
`${contractKey}.artifact_kinds contains ${artifactKind}, but artifactGraph.${artifactKind}.current_contracts does not include ${contractKey}`,
);
pushIf(
errors,
!hasIntersection(contract.truth_source, artifact.truth_sources),
`${contractKey}.artifact_kinds ${artifactKind} has no truth_source intersection with artifact graph`,
);
pushIf(
errors,
!hasIntersection(contract.viewer_surface, artifact.viewer_surfaces),
`${contractKey}.artifact_kinds ${artifactKind} has no viewer_surface intersection with artifact graph`,
);
pushIf(
errors,
!hasIntersection(contract.evidence_events, artifact.evidence_events),
`${contractKey}.artifact_kinds ${artifactKind} has no evidence_events intersection with artifact graph`,
);
}
}
function resolveExecutorAdapterKey(executor) {
if (!isPlainObject(executor)) {
return null;
}
if (!isNonEmptyString(executor.executor_kind) || !isNonEmptyString(executor.binding_key)) {
return null;
}
return `${executor.executor_kind}:${executor.binding_key}`;
}
function validateSubset(errors, label, actualValues, requiredValues) {
const actualSet = new Set(Array.isArray(actualValues) ? actualValues : []);
for (const requiredValue of requiredValues || []) {
pushIf(
errors,
!actualSet.has(requiredValue),
`${label} must include ${requiredValue}`,
);
}
}
function validateArtifactPolicy(errors, label, policy, artifactGraphRefs) {
pushIf(errors, !isPlainObject(policy), `${label}.artifact_policy must be an object`);
if (!isPlainObject(policy)) {
return;
}
pushIf(
errors,
!PROFILE_WRITE_MODES.has(policy.write_mode),
`${label}.artifact_policy.write_mode is unknown: ${String(policy.write_mode)}`,
);
validateEnumArray(
errors,
label,
"artifact_policy.artifact_kinds",
policy.artifact_kinds,
ARTIFACT_KINDS,
);
if (Array.isArray(policy.artifact_kinds)) {
for (const artifactKind of policy.artifact_kinds) {
pushIf(
errors,
!artifactGraphRefs.artifactMap.has(artifactKind),
`${label}.artifact_policy.artifact_kinds references artifact not defined in graph: ${artifactKind}`,
);
}
}
validateEnumArray(
errors,
label,
"artifact_policy.viewer_surfaces",
policy.viewer_surfaces,
VIEWER_SURFACES,
);
}
function validateExecutionProfiles(registry, profiles, matrixRefs, artifactGraphRefs) {
const errors = [];
pushIf(errors, profiles.version !== 1, "executionProfiles.version must be 1");
pushIf(
errors,
profiles.status !== "current",
"executionProfiles.status must be current",
);
pushIf(
errors,
!isNonEmptyString(profiles.owner),
"executionProfiles.owner must be set",
);
const contractMap = collectUniqueObjects(
errors,
registry.contracts,
"contract_key",
"runtimeContracts.contracts",
);
const profileMap = collectUniqueObjects(
errors,
profiles.profiles,
"profile_key",
"executionProfiles.profiles",
);
const adapterMap = collectUniqueObjects(
errors,
profiles.executor_adapters,
"adapter_key",
"executionProfiles.executor_adapters",
);
for (const [adapterKey, adapter] of adapterMap.entries()) {
pushIf(
errors,
!LIFECYCLES.has(adapter.lifecycle),
`executor adapter ${adapterKey}.lifecycle is unknown: ${String(adapter.lifecycle)}`,
);
pushIf(
errors,
!EXECUTOR_KINDS.has(adapter.executor_kind),
`executor adapter ${adapterKey}.executor_kind is unknown: ${String(adapter.executor_kind)}`,
);
pushIf(
errors,
!isNonEmptyString(adapter.binding_key),
`executor adapter ${adapterKey}.binding_key must be set`,
);
pushIf(
errors,
isNonEmptyString(adapter.executor_kind) &&
isNonEmptyString(adapter.binding_key) &&
adapterKey !== `${adapter.executor_kind}:${adapter.binding_key}`,
`executor adapter ${adapterKey}.adapter_key must equal executor_kind:binding_key`,
);
validateEnumArray(
errors,
adapterKey,
"supported_contracts",
adapter.supported_contracts,
new Set(contractMap.keys()),
);
for (const fieldName of [
"supports_progress",
"supports_cancel",
"supports_resume",
"supports_artifact",
]) {
pushIf(
errors,
typeof adapter[fieldName] !== "boolean",
`executor adapter ${adapterKey}.${fieldName} must be boolean`,
);
}
validateEnumArray(
errors,
adapterKey,
"artifact_output_kinds",
adapter.artifact_output_kinds,
ARTIFACT_KINDS,
);
validateEnumArray(
errors,
adapterKey,
"permission_requirements",
adapter.permission_requirements,
PERMISSIONS,
);
validateStringArray(
errors,
adapterKey,
"credential_requirements",
adapter.credential_requirements,
{ allowEmpty: true },
);
validateEnumArray(
errors,
adapterKey,
"failure_mapping",
adapter.failure_mapping,
FAILURE_REASONS,
);
validateEnumArray(
errors,
adapterKey,
"evidence_events",
adapter.evidence_events,
EVIDENCE_EVENTS,
);
pushIf(
errors,
!isNonEmptyString(adapter.notes),
`executor adapter ${adapterKey}.notes must be set`,
);
}
const supportedContractsByProfile = new Set();
for (const [profileKey, profile] of profileMap.entries()) {
pushIf(
errors,
!LIFECYCLES.has(profile.lifecycle),
`execution profile ${profileKey}.lifecycle is unknown: ${String(profile.lifecycle)}`,
);
validateEnumArray(
errors,
profileKey,
"supported_contracts",
profile.supported_contracts,
new Set(contractMap.keys()),
);
validateEnumArray(
errors,
profileKey,
"model_role_slots",
profile.model_role_slots,
matrixRefs.modelRoleSlots,
);
validateEnumArray(
errors,
profileKey,
"permission_profile_keys",
profile.permission_profile_keys,
PERMISSIONS,
);
validateEnumArray(
errors,
profileKey,
"executor_adapter_keys",
profile.executor_adapter_keys,
new Set(adapterMap.keys()),
);
validateArtifactPolicy(errors, profileKey, profile.artifact_policy, artifactGraphRefs);
validateEnumArray(
errors,
profileKey,
"limecore_policy_refs",
profile.limecore_policy_refs,
LIMECORE_POLICY_REFS,
);
pushIf(
errors,
!isNonEmptyString(profile.user_lock_policy),
`execution profile ${profileKey}.user_lock_policy must be set`,
);
validateStringArray(
errors,
profileKey,
"fallback_behavior",
profile.fallback_behavior,
);
validateEnumArray(
errors,
profileKey,
"evidence_events",
profile.evidence_events,
EVIDENCE_EVENTS,
);
validateStringArray(errors, profileKey, "audit_fields", profile.audit_fields);
pushIf(
errors,
!isNonEmptyString(profile.notes),
`execution profile ${profileKey}.notes must be set`,
);
for (const contractKey of profile.supported_contracts || []) {
const contract = contractMap.get(contractKey);
if (!contract) {
continue;
}
supportedContractsByProfile.add(contractKey);
const adapterKey = resolveExecutorAdapterKey(contract.executor_binding);
validateSubset(
errors,
`execution profile ${profileKey}.model_role_slots`,
profile.model_role_slots,
[contract.routing_slot],
);
validateSubset(
errors,
`execution profile ${profileKey}.permission_profile_keys`,
profile.permission_profile_keys,
contract.permission_profile_keys,
);
validateSubset(
errors,
`execution profile ${profileKey}.limecore_policy_refs`,
profile.limecore_policy_refs,
contract.limecore_policy_refs,
);
validateSubset(
errors,
`execution profile ${profileKey}.artifact_policy.artifact_kinds`,
profile.artifact_policy?.artifact_kinds,
contract.artifact_kinds,
);
validateSubset(
errors,
`execution profile ${profileKey}.artifact_policy.viewer_surfaces`,
profile.artifact_policy?.viewer_surfaces,
contract.viewer_surface,
);
if (adapterKey) {
validateSubset(
errors,
`execution profile ${profileKey}.executor_adapter_keys`,
profile.executor_adapter_keys,
[adapterKey],
);
}
}
}
for (const [contractKey, contract] of contractMap.entries()) {
if (contract.lifecycle !== "current") {
continue;
}
pushIf(
errors,
!supportedContractsByProfile.has(contractKey),
`current contract ${contractKey} must be covered by an execution profile`,
);
const adapterKey = resolveExecutorAdapterKey(contract.executor_binding);
const adapter = adapterKey ? adapterMap.get(adapterKey) : null;
pushIf(
errors,
!adapter,
`current contract ${contractKey}.executor_binding must be defined in executionProfiles.executor_adapters: ${String(adapterKey)}`,
);
if (!adapter) {
continue;
}
validateSubset(
errors,
`executor adapter ${adapterKey}.supported_contracts`,
adapter.supported_contracts,
[contractKey],
);
for (const fieldName of [
"supports_progress",
"supports_cancel",
"supports_resume",
"supports_artifact",
]) {
pushIf(
errors,
adapter[fieldName] !== contract.executor_binding[fieldName],
`current contract ${contractKey}.executor_binding.${fieldName} must match executor adapter ${adapterKey}`,
);
}
validateSubset(
errors,
`executor adapter ${adapterKey}.artifact_output_kinds`,
adapter.artifact_output_kinds,
contract.artifact_kinds,
);
validateSubset(
errors,
`executor adapter ${adapterKey}.permission_requirements`,
adapter.permission_requirements,
contract.permission_profile_keys,
);
validateSubset(
errors,
`executor adapter ${adapterKey}.failure_mapping`,
adapter.failure_mapping,
contract.executor_binding.failure_mapping,
);
}
return {
errors,
profileCount: profileMap.size,
adapterCount: adapterMap.size,
};
}
function validateContractRegistry(registry, matrixRefs, artifactGraphRefs) {
const errors = [];
pushIf(errors, registry.version !== 1, "registry.version must be 1");
pushIf(errors, registry.status !== "current", "registry.status must be current");
@@ -508,6 +1039,7 @@ function validateContractRegistry(registry, matrixRefs) {
}
pushIf(errors, !isNonEmptyString(contract.owner_surface), `${contractKey}.owner_surface must be set`);
validateEntryBindings(errors, contract);
validateContractArtifactGraph(errors, contract, artifactGraphRefs);
}
return errors;
@@ -523,7 +1055,7 @@ function validateRequiredDocs() {
});
}
function renderSuccess(registry, matrix) {
function renderSuccess(registry, matrix, graph, profileReport) {
const currentCount = registry.contracts.filter(
(contract) => contract.lifecycle === "current",
).length;
@@ -533,19 +1065,35 @@ function renderSuccess(registry, matrix) {
` current: ${currentCount}`,
` capabilities: ${matrix.capabilities.length}`,
` model roles: ${matrix.model_roles.length}`,
` artifact kinds: ${graph.artifact_kinds.length}`,
` execution profiles: ${profileReport.profileCount}`,
` executor adapters: ${profileReport.adapterCount}`,
` registry: ${CONTRACT_PATH}`,
` matrix: ${CAPABILITY_MATRIX_PATH}`,
` graph: ${ARTIFACT_GRAPH_PATH}`,
` profiles: ${EXECUTION_PROFILE_PATH}`,
].join("\n");
}
function main() {
const registry = readJson(CONTRACT_PATH);
const matrix = readJson(CAPABILITY_MATRIX_PATH);
const graph = readJson(ARTIFACT_GRAPH_PATH);
const profiles = readJson(EXECUTION_PROFILE_PATH);
const matrixReport = validateCapabilityMatrix(matrix);
const graphReport = validateArtifactGraph(graph);
const profileReport = validateExecutionProfiles(
registry,
profiles,
matrixReport,
graphReport,
);
const errors = [
...validateRequiredDocs(),
...matrixReport.errors,
...validateContractRegistry(registry, matrixReport),
...graphReport.errors,
...validateContractRegistry(registry, matrixReport, graphReport),
...profileReport.errors,
];
if (errors.length > 0) {
@@ -556,7 +1104,7 @@ function main() {
process.exit(1);
}
console.log(renderSuccess(registry, matrix));
console.log(renderSuccess(registry, matrix, graph, profileReport));
}
main();
scripts/lib/harness-eval-history-record.test.ts
+3 -3
View File
@@ -287,7 +287,7 @@ describe("harness-eval-history-record", () => {
},
),
).not.toThrow();
}, 30_000);
}, 60_000);
it("应记录本地 summary 历史，并生成非 seed 的 trend / cleanup", () => {
const tempRoot = createTempRoot();
@@ -442,7 +442,7 @@ describe("harness-eval-history-record", () => {
),
),
).toBe(true);
}, 30_000);
}, 90_000);
it("并发记录 history 时不应覆盖已有样本", async () => {
const tempRoot = createTempRoot();
@@ -478,5 +478,5 @@ describe("harness-eval-history-record", () => {
expect(historyFiles).toHaveLength(2);
expect(firstResult.recordedSummaryPath).not.toBe(secondResult.recordedSummaryPath);
}, 30_000);
}, 60_000);
});
scripts/release-updater-manifest.test.mjs
+11 -11
View File
@@ -303,7 +303,7 @@ describe("GitHub release asset staging", () => {
"arm-sig",
);
writeFile(
path.join(assetsDir, "aarch64-apple-darwin", "Lime_1.24.0_aarch64.dmg"),
path.join(assetsDir, "aarch64-apple-darwin", "Lime_1.25.0_aarch64.dmg"),
);
writeFile(path.join(assetsDir, "x86_64-apple-darwin", "Lime.app.tar.gz"));
writeFile(
@@ -311,7 +311,7 @@ describe("GitHub release asset staging", () => {
"x64-sig",
);
writeFile(
path.join(assetsDir, "x86_64-apple-darwin", "Lime_1.24.0_x64.dmg"),
path.join(assetsDir, "x86_64-apple-darwin", "Lime_1.25.0_x64.dmg"),
);
writeFile(latestPath, "{}");
@@ -319,29 +319,29 @@ describe("GitHub release asset staging", () => {
assetsDir,
extraAssets: [latestPath],
outDir,
version: "v1.24.0",
version: "v1.25.0",
});
expect(copied.map((item) => item.name).sort()).toEqual(
[
"Lime_1.24.0_aarch64.app.tar.gz",
"Lime_1.24.0_aarch64.app.tar.gz.sig",
"Lime_1.24.0_aarch64.dmg",
"Lime_1.24.0_x64.app.tar.gz",
"Lime_1.24.0_x64.app.tar.gz.sig",
"Lime_1.24.0_x64.dmg",
"Lime_1.25.0_aarch64.app.tar.gz",
"Lime_1.25.0_aarch64.app.tar.gz.sig",
"Lime_1.25.0_aarch64.dmg",
"Lime_1.25.0_x64.app.tar.gz",
"Lime_1.25.0_x64.app.tar.gz.sig",
"Lime_1.25.0_x64.dmg",
"latest.json",
].sort(),
);
expect(
fs.readFileSync(
path.join(outDir, "Lime_1.24.0_aarch64.app.tar.gz.sig"),
path.join(outDir, "Lime_1.25.0_aarch64.app.tar.gz.sig"),
"utf8",
),
).toBe("arm-sig");
expect(
fs.readFileSync(
path.join(outDir, "Lime_1.24.0_x64.app.tar.gz.sig"),
path.join(outDir, "Lime_1.25.0_x64.app.tar.gz.sig"),
"utf8",
),
).toBe("x64-sig");
src-tauri/Cargo.lock
Generated
+81 -30
View File
@@ -2381,7 +2381,7 @@ dependencies = [
"dtoa-short",
"itoa",
"matches",
"phf 0.8.0",
"phf 0.10.1",
"proc-macro2",
"quote",
"smallvec",
@@ -2397,7 +2397,7 @@ dependencies = [
"cssparser-macros",
"dtoa-short",
"itoa",
"phf 0.8.0",
"phf 0.11.3",
"smallvec",
]
@@ -4338,7 +4338,7 @@ dependencies = [
"js-sys",
"log",
"wasm-bindgen",
"windows-core 0.56.0",
"windows-core 0.62.2",
]
[[package]]
@@ -5065,7 +5065,7 @@ dependencies = [
[[package]]
name = "lime"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"anyhow",
"arboard",
@@ -5075,7 +5075,9 @@ dependencies = [
"axum 0.7.9",
"axum-server",
"base64 0.22.1",
"block",
"bytes",
"bzip2",
"chrono",
"chrono-tz",
"cocoa",
@@ -5168,7 +5170,7 @@ dependencies = [
[[package]]
name = "lime-agent"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"anyhow",
"aster-core",
@@ -5197,7 +5199,7 @@ dependencies = [
[[package]]
name = "lime-browser-runtime"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"chrono",
"futures",
@@ -5214,7 +5216,7 @@ dependencies = [
[[package]]
name = "lime-cli"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"clap",
"lime-core",
@@ -5226,7 +5228,7 @@ dependencies = [
[[package]]
name = "lime-config"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"async-trait",
"lime-core",
@@ -5242,7 +5244,7 @@ dependencies = [
[[package]]
name = "lime-core"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"aster-models",
"async-trait",
@@ -5295,7 +5297,7 @@ dependencies = [
[[package]]
name = "lime-gateway"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"aes",
"axum 0.7.9",
@@ -5325,7 +5327,7 @@ dependencies = [
[[package]]
name = "lime-infra"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"chrono",
"dashmap 5.5.3",
@@ -5345,7 +5347,7 @@ dependencies = [
[[package]]
name = "lime-mcp"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"async-trait",
"dirs 5.0.1",
@@ -5361,7 +5363,7 @@ dependencies = [
[[package]]
name = "lime-media-runtime"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -5392,7 +5394,7 @@ dependencies = [
[[package]]
name = "lime-processor"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"async-trait",
"lime-core",
@@ -5411,7 +5413,7 @@ dependencies = [
[[package]]
name = "lime-providers"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"anyhow",
"async-stream",
@@ -5466,7 +5468,7 @@ dependencies = [
[[package]]
name = "lime-server"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"aster-core",
"async-stream",
@@ -5510,7 +5512,7 @@ dependencies = [
[[package]]
name = "lime-server-utils"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"axum 0.7.9",
"futures",
@@ -5525,7 +5527,7 @@ dependencies = [
[[package]]
name = "lime-services"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"anyhow",
"aster-core",
@@ -5568,7 +5570,7 @@ dependencies = [
[[package]]
name = "lime-skills"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"async-trait",
"dirs 5.0.1",
@@ -5586,7 +5588,7 @@ dependencies = [
[[package]]
name = "lime-websocket"
version = "1.24.0"
version = "1.25.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -6265,7 +6267,7 @@ version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff32365de1b6743cb203b710788263c44a03de03802daf96092f2da4fe6ba4d7"
dependencies = [
"proc-macro-crate 1.3.1",
"proc-macro-crate 3.4.0",
"proc-macro2",
"quote",
"syn 2.0.117",
@@ -7012,9 +7014,7 @@ version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3dfb61232e34fcb633f43d12c58f83c1df82962dcdfa565a4e866ffc17dafe12"
dependencies = [
"phf_macros 0.8.0",
"phf_shared 0.8.0",
"proc-macro-hack",
]
[[package]]
@@ -7023,7 +7023,9 @@ version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fabbf1ead8a5bcbc20f5f8b939ee3f5b0f6f281b6ad3468b84656b658b455259"
dependencies = [
"phf_macros 0.10.0",
"phf_shared 0.10.0",
"proc-macro-hack",
]
[[package]]
@@ -7127,12 +7129,12 @@ dependencies = [
[[package]]
name = "phf_macros"
version = "0.8.0"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f6fde18ff429ffc8fe78e2bf7f8b7a5a5a6e2a8b58bc5a9ac69198bbda9189c"
checksum = "58fdf3184dd560f160dd73922bea2d5cd6e8f064bf4b13110abd81b03697b4e0"
dependencies = [
"phf_generator 0.8.0",
"phf_shared 0.8.0",
"phf_generator 0.10.0",
"phf_shared 0.10.0",
"proc-macro-hack",
"proc-macro2",
"quote",
@@ -7533,7 +7535,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a56d757972c98b346a9b766e3f02746cde6dd1cd1d1d563472929fdd74bec4d"
dependencies = [
"anyhow",
"itertools 0.12.1",
"itertools 0.14.0",
"proc-macro2",
"quote",
"syn 2.0.117",
@@ -8256,6 +8258,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c665f33d38cea657d9614f766881e4d510e0eda4239891eea56b4cadcf01801b"
dependencies = [
"aws-lc-rs",
"log",
"once_cell",
"ring",
"rustls-pki-types",
@@ -8619,7 +8622,7 @@ dependencies = [
"sentry-panic",
"sentry-tracing",
"tokio",
"ureq",
"ureq 3.2.0",
]
[[package]]
@@ -9032,7 +9035,29 @@ version = "3.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b1fdf65dd6331831494dd616b30351c38e96e45921a27745cf98490458b90bb"
dependencies = [
"dirs 4.0.0",
"dirs 6.0.0",
]
[[package]]
name = "sherpa-onnx"
version = "1.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071b3daa7bf126fe47caf8f1f7fcd5af6a4ff40e67f410503c81de55b7608b20"
dependencies = [
"serde",
"serde_json",
"sherpa-onnx-sys",
]
[[package]]
name = "sherpa-onnx-sys"
version = "1.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c231bdd33c5b31cbf42996e426d186d453b3fc0574f194ecc661bfae7aea1114"
dependencies = [
"bzip2",
"tar",
"ureq 2.12.1",
]
[[package]]
@@ -10951,6 +10976,22 @@ version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1"
[[package]]
name = "ureq"
version = "2.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02d1a66277ed75f640d608235660df48c8e3c19f3b4edb6a263315626cc3c01d"
dependencies = [
"base64 0.22.1",
"flate2",
"log",
"once_cell",
"rustls 0.23.36",
"rustls-pki-types",
"url",
"webpki-roots 0.26.11",
]
[[package]]
name = "ureq"
version = "3.2.0"
@@ -11124,6 +11165,7 @@ dependencies = [
"serde",
"serde_json",
"sha2",
"sherpa-onnx",
"thiserror 1.0.69",
"tokio",
"tokio-tungstenite 0.24.0",
@@ -11436,6 +11478,15 @@ version = "0.25.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5f20c57d8d7db6d3b86154206ae5d8fba62dd39573114de97c2cb0578251f8e1"
[[package]]
name = "webpki-roots"
version = "0.26.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "521bc38abb08001b01866da9f51eb7c5d647a19260e00054a8c7fd5f9e57f7a9"
dependencies = [
"webpki-roots 1.0.6",
]
[[package]]
name = "webpki-roots"
version = "1.0.6"
src-tauri/Cargo.toml
+11 -3
View File
@@ -9,7 +9,7 @@ exclude = [
resolver = "2"
[workspace.package]
version = "1.24.0"
version = "1.25.0"
edition = "2021"
authors = ["coso"]
repository = "https://github.com/aiclientproxy/lime"
@@ -95,6 +95,7 @@ tar = "0.4"
fs2 = "0.4"
indexmap = { version = "2", features = ["serde"] }
zip = "0.6"
bzip2 = "0.4"
dashmap = "5"
notify = { version = "6", default-features = false, features = ["macos_fsevent"] }
parking_lot = "0.12"
@@ -182,6 +183,9 @@ features = [
version = "0.52"
# macOS 平台依赖
[workspace.dependencies.block]
version = "0.1"
[workspace.dependencies.cocoa]
version = "0.26"
@@ -193,7 +197,7 @@ version = "2.4"
[package]
name = "lime"
version = "1.24.0"
version = "1.25.0"
description = "AI API Proxy Desktop App"
authors = ["you"]
edition = "2021"
@@ -295,6 +299,7 @@ tar.workspace = true
fs2.workspace = true
indexmap.workspace = true
zip.workspace = true
bzip2.workspace = true
dashmap.workspace = true
notify.workspace = true
parking_lot.workspace = true
@@ -343,6 +348,7 @@ winreg.workspace = true
# macOS specific dependencies for browser interceptor
[target.'cfg(target_os = "macos")'.dependencies]
block.workspace = true
cocoa.workspace = true
objc.workspace = true
@@ -351,8 +357,10 @@ proptest.workspace = true
tempfile.workspace = true
[features]
default = ["custom-protocol"]
default = ["custom-protocol", "local-sensevoice"]
custom-protocol = ["tauri/custom-protocol"]
# 本地 Whisper 语音识别（编译很慢，CI 默认不启用）
local-whisper = ["lime-services/local-whisper"]
# 本地 SenseVoice Small 语音识别（模型按需下载，不进入安装包）
local-sensevoice = ["lime-services/local-sensevoice"]
notification = [] # 预留特性：系统通知功能
src-tauri/capabilities/default.json
+2 -1
View File
@@ -21,6 +21,7 @@
"core:window:allow-set-title",
"core:window:allow-inner-size",
"core:window:allow-outer-size",
"core:window:allow-start-dragging",
"core:webview:default",
"core:app:default",
"core:resources:default",
@@ -40,4 +41,4 @@
"global-shortcut:allow-is-registered",
"autostart:default"
]
}
}
src-tauri/crates/agent/src/credential_bridge.rs
+184 -4
View File
@@ -408,11 +408,12 @@ fn should_disable_provider_default_fast_model(config: &AsterProviderConfig) -> b
/// - `https://open.bigmodel.cn/api/paas/v4` -> (`https://open.bigmodel.cn`, `api/paas/v4`)
/// - `https://localhost:8080/v1` -> (`https://localhost:8080`, `v1`)
fn split_url_host_and_path(url: &str) -> (String, String) {
let url = strip_url_query_fragment(url);
// 找到 scheme 之后的 authority 部分
let after_scheme = if let Some(pos) = url.find("://") {
pos + 3
} else {
return (url.to_string(), String::new());
return (url, String::new());
};
// 找到 authority 之后的第一个 /（即路径开始）
@@ -450,6 +451,95 @@ fn resolve_anthropic_env_key(config: &AsterProviderConfig) -> &'static str {
}
}
const LIME_TENANT_HEADER: &str = "X-Lime-Tenant-ID";
const LIME_TENANT_PARAM: &str = "lime_tenant_id";
const OPENAI_CUSTOM_HEADERS_ENV: &str = "OPENAI_CUSTOM_HEADERS";
fn normalize_lime_tenant_id(value: &str) -> Option<String> {
let tenant_id = value.trim();
if tenant_id.is_empty() {
return None;
}
tenant_id
.chars()
.all(|ch| ch.is_ascii_alphanumeric() || matches!(ch, '-' | '_'))
.then(|| tenant_id.to_string())
}
fn parse_lime_tenant_id_from_pairs(value: &str) -> Option<String> {
value.split('&').find_map(|pair| {
let mut parts = pair.splitn(2, '=');
let key = parts.next()?.trim();
let value = parts.next().unwrap_or_default();
(key == LIME_TENANT_PARAM)
.then(|| normalize_lime_tenant_id(value))
.flatten()
})
}
fn lime_tenant_id_from_api_host(api_host: &str) -> Option<String> {
let trimmed = api_host.trim();
let query = trimmed
.find('?')
.map(|pos| trimmed[pos + 1..].split('#').next().unwrap_or_default());
let fragment = trimmed.find('#').map(|pos| &trimmed[pos + 1..]);
query
.and_then(parse_lime_tenant_id_from_pairs)
.or_else(|| fragment.and_then(parse_lime_tenant_id_from_pairs))
}
fn strip_url_query_fragment(value: &str) -> String {
let trimmed = value.trim();
let end = [trimmed.find('?'), trimmed.find('#')]
.into_iter()
.flatten()
.min()
.unwrap_or(trimmed.len());
trimmed[..end].trim_end_matches('/').to_string()
}
fn parse_openai_custom_headers_env(value: &str) -> Vec<(String, String)> {
value
.split(',')
.filter_map(|header| {
let mut parts = header.splitn(2, '=');
let key = parts.next()?.trim();
if key.is_empty() {
return None;
}
let value = parts.next().unwrap_or_default().trim();
Some((key.to_string(), value.to_string()))
})
.collect()
}
fn update_openai_lime_tenant_custom_header(tenant_id: Option<&str>) {
let existing = std::env::var(OPENAI_CUSTOM_HEADERS_ENV).unwrap_or_default();
let mut headers = parse_openai_custom_headers_env(&existing)
.into_iter()
.filter(|(key, _)| !key.eq_ignore_ascii_case(LIME_TENANT_HEADER))
.collect::<Vec<_>>();
if let Some(tenant_id) = tenant_id.and_then(normalize_lime_tenant_id) {
headers.push((LIME_TENANT_HEADER.to_string(), tenant_id));
}
if headers.is_empty() {
std::env::remove_var(OPENAI_CUSTOM_HEADERS_ENV);
return;
}
let value = headers
.into_iter()
.map(|(key, value)| format!("{key}={value}"))
.collect::<Vec<_>>()
.join(",");
std::env::set_var(OPENAI_CUSTOM_HEADERS_ENV, value);
}
fn set_provider_env_vars(config: &AsterProviderConfig) {
tracing::info!(
"[CredentialBridge] set_provider_env_vars: provider_name={}, has_api_key={}, base_url={:?}",
@@ -491,6 +581,9 @@ fn set_provider_env_vars(config: &AsterProviderConfig) {
} else {
std::env::remove_var("OPENAI_FORCE_RESPONSES_API");
}
if config.base_url.is_none() {
update_openai_lime_tenant_custom_header(None);
}
}
// 设置 base_url
@@ -499,18 +592,22 @@ fn set_provider_env_vars(config: &AsterProviderConfig) {
if let Some(base_url) = &config.base_url {
match config.provider_name.as_str() {
"openai" => {
let tenant_id = lime_tenant_id_from_api_host(base_url);
update_openai_lime_tenant_custom_header(tenant_id.as_deref());
// 解析 base_url，将路径部分拆分到 OPENAI_BASE_PATH。
// Responses 模式也复用这条前缀推导，避免出现 /v1/v1/responses。
// 例如 https://open.bigmodel.cn/api/paas/v4
// -> OPENAI_HOST = https://open.bigmodel.cn
// -> OPENAI_BASE_PATH = api/paas/v4/chat/completions
let (host_part, path_part) = split_url_host_and_path(base_url);
let sanitized_base_url = strip_url_query_fragment(base_url);
let (host_part, path_part) = split_url_host_and_path(&sanitized_base_url);
if path_part.is_empty() {
// 无路径部分（如 https://api.openai.com），直接设置
std::env::set_var("OPENAI_HOST", base_url);
std::env::set_var("OPENAI_HOST", &sanitized_base_url);
// 清除可能残留的 OPENAI_BASE_PATH，使用 Aster 默认值
std::env::remove_var("OPENAI_BASE_PATH");
tracing::info!("[CredentialBridge] 设置 OPENAI_HOST={}", base_url);
tracing::info!("[CredentialBridge] 设置 OPENAI_HOST={}", sanitized_base_url);
} else {
// base_url 包含路径，需要拆分
let base_path = format!("{}/chat/completions", path_part);
@@ -606,6 +703,12 @@ fn map_provider_type_to_aster_with_api_type(
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{Mutex, MutexGuard, OnceLock};
fn env_lock() -> MutexGuard<'static, ()> {
static LOCK: OnceLock<Mutex<()>> = OnceLock::new();
LOCK.get_or_init(|| Mutex::new(())).lock().unwrap()
}
#[test]
fn test_map_provider_type_to_aster_with_api_type() {
@@ -631,6 +734,7 @@ mod tests {
#[test]
fn test_set_provider_env_vars_openai_codex_responses_splits_prefixed_base_url() {
let _env_guard = env_lock();
std::env::remove_var("OPENAI_HOST");
std::env::remove_var("OPENAI_BASE_PATH");
std::env::remove_var("OPENAI_FORCE_RESPONSES_API");
@@ -665,6 +769,7 @@ mod tests {
#[test]
fn test_set_provider_env_vars_openai_codex_responses_normalizes_v1_base_url() {
let _env_guard = env_lock();
std::env::remove_var("OPENAI_HOST");
std::env::remove_var("OPENAI_BASE_PATH");
std::env::remove_var("OPENAI_FORCE_RESPONSES_API");
@@ -814,10 +919,84 @@ mod tests {
let (host, path) = split_url_host_and_path("https://api.openai.com/");
assert_eq!(host, "https://api.openai.com");
assert_eq!(path, "");
// 查询参数和 fragment 只用于附加元数据，不应进入真实请求 URL
let (host, path) =
split_url_host_and_path("https://llm.limeai.run/openai?lime_tenant_id=tenant-0001#x=1");
assert_eq!(host, "https://llm.limeai.run");
assert_eq!(path, "openai");
}
#[test]
fn test_set_provider_env_vars_openai_lime_hub_adds_tenant_header_from_fragment() {
let _env_guard = env_lock();
std::env::remove_var("OPENAI_HOST");
std::env::remove_var("OPENAI_BASE_PATH");
std::env::set_var(
OPENAI_CUSTOM_HEADERS_ENV,
"X-Other=1,X-Lime-Tenant-ID=stale",
);
let config = AsterProviderConfig {
provider_name: "openai".to_string(),
provider_selector: Some("lime-hub".to_string()),
model_name: "gpt-5.5".to_string(),
api_key: Some("test-key".to_string()),
base_url: Some("https://llm.limeai.run#lime_tenant_id=tenant-0001".to_string()),
credential_uuid: "test-uuid".to_string(),
force_responses_api: false,
toolshim: false,
toolshim_model: None,
};
set_provider_env_vars(&config);
assert_eq!(
std::env::var("OPENAI_HOST").ok().as_deref(),
Some("https://llm.limeai.run")
);
assert!(std::env::var("OPENAI_BASE_PATH").is_err());
assert_eq!(
std::env::var(OPENAI_CUSTOM_HEADERS_ENV).ok().as_deref(),
Some("X-Other=1,X-Lime-Tenant-ID=tenant-0001")
);
std::env::remove_var(OPENAI_CUSTOM_HEADERS_ENV);
}
#[test]
fn test_set_provider_env_vars_openai_without_lime_tenant_clears_stale_header() {
let _env_guard = env_lock();
std::env::set_var(
OPENAI_CUSTOM_HEADERS_ENV,
"X-Lime-Tenant-ID=stale,X-Other=1",
);
let config = AsterProviderConfig {
provider_name: "openai".to_string(),
provider_selector: Some("openai".to_string()),
model_name: "gpt-4o".to_string(),
api_key: Some("test-key".to_string()),
base_url: Some("https://api.openai.com/v1".to_string()),
credential_uuid: "test-uuid".to_string(),
force_responses_api: false,
toolshim: false,
toolshim_model: None,
};
set_provider_env_vars(&config);
assert_eq!(
std::env::var(OPENAI_CUSTOM_HEADERS_ENV).ok().as_deref(),
Some("X-Other=1")
);
std::env::remove_var(OPENAI_CUSTOM_HEADERS_ENV);
}
#[test]
fn test_set_provider_env_vars_anthropic_sets_host_and_base_url() {
let _env_guard = env_lock();
std::env::remove_var("ANTHROPIC_API_KEY");
std::env::remove_var("ANTHROPIC_AUTH_TOKEN");
@@ -852,6 +1031,7 @@ mod tests {
#[test]
fn test_set_provider_env_vars_official_anthropic_keeps_api_key_env() {
let _env_guard = env_lock();
std::env::remove_var("ANTHROPIC_API_KEY");
std::env::remove_var("ANTHROPIC_AUTH_TOKEN");
src-tauri/crates/aster-rust/crates/aster/src/providers/anthropic.rs
+88 -6
View File
@@ -7,6 +7,7 @@ use serde_json::Value;
use std::io;
use tokio::pin;
use tokio_util::io::StreamReader;
use url::{form_urlencoded, Url};
use super::api_client::{ApiClient, ApiResponse, AuthMethod};
use super::base::{ConfigKey, MessageStream, ModelInfo, Provider, ProviderMetadata, ProviderUsage};
@@ -36,6 +37,8 @@ const ANTHROPIC_KNOWN_MODELS: &[&str] = &[
const ANTHROPIC_DOC_URL: &str = "https://docs.anthropic.com/en/docs/about-claude/models";
const ANTHROPIC_API_VERSION: &str = "2023-06-01";
const LIME_TENANT_HEADER: &str = "X-Lime-Tenant-ID";
const LIME_TENANT_PARAM: &str = "lime_tenant_id";
fn normalize_anthropic_host(host: &str) -> String {
host.trim().trim_end_matches('/').to_ascii_lowercase()
@@ -62,6 +65,61 @@ fn should_use_bearer_auth_for_anthropic_host(host: &str) -> bool {
.any(|needle| normalized_host.contains(needle))
}
fn parse_config_url(host: &str) -> Option<Url> {
let trimmed = host.trim();
if trimmed.is_empty() {
return None;
}
Url::parse(trimmed)
.or_else(|_| Url::parse(&format!("https://{trimmed}")))
.ok()
}
fn strip_host_url_metadata(host: &str) -> String {
if let Some(mut url) = parse_config_url(host) {
url.set_query(None);
url.set_fragment(None);
return url.to_string().trim_end_matches('/').to_string();
}
let trimmed = host.trim();
let end = [trimmed.find('?'), trimmed.find('#')]
.into_iter()
.flatten()
.min()
.unwrap_or(trimmed.len());
trimmed[..end].trim_end_matches('/').to_string()
}
fn normalize_lime_tenant_id(value: &str) -> Option<String> {
let tenant_id = value.trim();
if tenant_id.is_empty() {
return None;
}
tenant_id
.chars()
.all(|ch| ch.is_ascii_alphanumeric() || matches!(ch, '-' | '_'))
.then(|| tenant_id.to_string())
}
fn parse_lime_tenant_id_from_pairs(value: &str) -> Option<String> {
form_urlencoded::parse(value.as_bytes()).find_map(|(key, value)| {
(key == LIME_TENANT_PARAM)
.then(|| normalize_lime_tenant_id(&value))
.flatten()
})
}
fn lime_tenant_id_from_host(host: &str) -> Option<String> {
let url = parse_config_url(host)?;
url.query()
.and_then(parse_lime_tenant_id_from_pairs)
.or_else(|| url.fragment().and_then(parse_lime_tenant_id_from_pairs))
}
fn resolve_anthropic_auth(host: &str, secret: String) -> AuthMethod {
if should_use_bearer_auth_for_anthropic_host(host) {
AuthMethod::BearerToken(secret)
@@ -74,16 +132,21 @@ fn resolve_anthropic_auth(host: &str, secret: String) -> AuthMethod {
}
fn build_anthropic_api_client(host: &str, auth_secret: String) -> Result<ApiClient> {
let should_add_x_api_key_header = should_use_bearer_auth_for_anthropic_host(host);
let auth = resolve_anthropic_auth(host, auth_secret.clone());
let sanitized_host = strip_host_url_metadata(host);
let should_add_x_api_key_header = should_use_bearer_auth_for_anthropic_host(&sanitized_host);
let auth = resolve_anthropic_auth(&sanitized_host, auth_secret.clone());
let mut api_client = ApiClient::new(host.to_string(), auth)?
let mut api_client = ApiClient::new(sanitized_host, auth)?
.with_header("anthropic-version", ANTHROPIC_API_VERSION)?;
if should_add_x_api_key_header {
api_client = api_client.with_header("x-api-key", &auth_secret)?;
}
if let Some(tenant_id) = lime_tenant_id_from_host(host) {
api_client = api_client.with_header(LIME_TENANT_HEADER, &tenant_id)?;
}
Ok(api_client)
}
@@ -106,7 +169,8 @@ impl AnthropicProvider {
.get_param("ANTHROPIC_HOST")
.or_else(|_| config.get_param("ANTHROPIC_BASE_URL"))
.unwrap_or_else(|_| "https://api.anthropic.com".to_string());
let api_key: String = if should_use_bearer_auth_for_anthropic_host(&host) {
let sanitized_host = strip_host_url_metadata(&host);
let api_key: String = if should_use_bearer_auth_for_anthropic_host(&sanitized_host) {
config
.get_secret("ANTHROPIC_AUTH_TOKEN")
.or_else(|_| config.get_secret("ANTHROPIC_API_KEY"))?
@@ -116,7 +180,7 @@ impl AnthropicProvider {
.or_else(|_| config.get_secret("ANTHROPIC_AUTH_TOKEN"))?
};
let automatic_prompt_cache = supports_automatic_prompt_cache_for_host(&host);
let automatic_prompt_cache = supports_automatic_prompt_cache_for_host(&sanitized_host);
let api_client = build_anthropic_api_client(&host, api_key)?;
Ok(Self {
@@ -137,7 +201,8 @@ impl AnthropicProvider {
.get_secret(&config.api_key_env)
.map_err(|_| anyhow::anyhow!("Missing API key: {}", config.api_key_env))?;
let automatic_prompt_cache = supports_automatic_prompt_cache_for_host(&config.base_url);
let sanitized_host = strip_host_url_metadata(&config.base_url);
let automatic_prompt_cache = supports_automatic_prompt_cache_for_host(&sanitized_host);
let api_client = build_anthropic_api_client(&config.base_url, api_key)?;
Ok(Self {
@@ -469,6 +534,23 @@ mod tests {
);
}
#[test]
fn test_lime_tenant_fragment_adds_custom_header_to_api_client() {
let api_client = build_anthropic_api_client(
"https://llm.limeai.run#lime_tenant_id=tenant-0001",
"k1".to_string(),
)
.expect("api client should build");
let headers = api_client.default_headers_for_test();
assert_eq!(
headers
.get(LIME_TENANT_HEADER)
.and_then(|value| value.to_str().ok()),
Some("tenant-0001")
);
}
#[test]
fn test_official_host_keeps_single_auth_header_mode() {
let api_client = build_anthropic_api_client("https://api.anthropic.com", "k2".to_string())
src-tauri/crates/aster-rust/crates/aster/src/providers/formats/openai.rs
+55 -4
View File
@@ -429,8 +429,16 @@ pub fn validate_tool_schemas(tools: &mut [Value]) {
}
/// Ensures that the given JSON value follows the expected JSON Schema structure.
fn ensure_valid_json_schema(schema: &mut Value) {
pub(crate) fn ensure_valid_json_schema(schema: &mut Value) {
if let Some(params_obj) = schema.as_object_mut() {
if params_obj
.get("type")
.and_then(|t| t.as_str())
.is_some_and(|t| t == "array")
{
params_obj.entry("items").or_insert_with(|| json!({}));
}
// Check if this is meant to be an object type schema
let is_object_type = params_obj
.get("type")
@@ -448,15 +456,31 @@ fn ensure_valid_json_schema(schema: &mut Value) {
if let Some(properties) = params_obj.get_mut("properties") {
if let Some(properties_obj) = properties.as_object_mut() {
for (_key, prop) in properties_obj.iter_mut() {
if prop.is_object()
&& prop.get("type").and_then(|t| t.as_str()) == Some("object")
{
if prop.is_object() {
ensure_valid_json_schema(prop);
}
}
}
}
}
if let Some(items) = params_obj.get_mut("items") {
ensure_valid_json_schema(items);
}
for keyword in ["oneOf", "anyOf", "allOf"] {
if let Some(variants) = params_obj.get_mut(keyword).and_then(Value::as_array_mut) {
for variant in variants.iter_mut() {
ensure_valid_json_schema(variant);
}
}
}
if let Some(additional_properties) = params_obj.get_mut("additionalProperties") {
if additional_properties.is_object() {
ensure_valid_json_schema(additional_properties);
}
}
}
}
@@ -857,6 +881,33 @@ mod tests {
let mut tools = vec![original_schema.clone()];
validate_tool_schemas(&mut tools);
assert_eq!(tools[0], original_schema);
// Test case 4: Nested oneOf array schema should get items for OpenAI-compatible APIs.
let mut tools = vec![json!({
"type": "function",
"function": {
"name": "test_func",
"description": "test description",
"parameters": {
"type": "object",
"properties": {
"message": {
"oneOf": [
{ "type": "string" },
{ "type": "array" }
]
}
},
"required": ["message"]
}
}
})];
validate_tool_schemas(&mut tools);
assert_eq!(
tools[0]["function"]["parameters"]["properties"]["message"]["oneOf"][1]["items"],
json!({})
);
}
const OPENAI_TOOL_USE_RESPONSE: &str = r#"{
src-tauri/crates/aster-rust/crates/aster/src/providers/formats/openai_responses.rs
+6 -1
View File
@@ -450,7 +450,7 @@ pub fn create_responses_request(
}
if !tools.is_empty() {
let tools_spec: Vec<Value> = tools
let mut tools_spec: Vec<Value> = tools
.iter()
.map(|tool| {
json!({
@@ -461,6 +461,11 @@ pub fn create_responses_request(
})
})
.collect();
for tool in tools_spec.iter_mut() {
if let Some(parameters) = tool.get_mut("parameters") {
super::openai::ensure_valid_json_schema(parameters);
}
}
payload
.as_object_mut()
src-tauri/crates/aster-rust/crates/aster/src/providers/openai.rs
+27
View File
@@ -654,6 +654,7 @@ impl EmbeddingCapable for OpenAiProvider {
mod tests {
use super::OpenAiProvider;
use crate::conversation::message::Message;
use crate::model::ModelConfig;
use crate::session::TurnContextOverride;
use std::collections::HashMap;
@@ -675,6 +676,32 @@ mod tests {
std::env::remove_var("OPENAI_FORCE_RESPONSES_API");
}
#[tokio::test]
#[serial_test::serial]
async fn test_from_env_applies_openai_custom_headers() {
std::env::set_var("OPENAI_API_KEY", "sk-test");
std::env::set_var("OPENAI_HOST", "https://llm.limeai.run");
std::env::remove_var("OPENAI_BASE_PATH");
std::env::set_var("OPENAI_CUSTOM_HEADERS", "X-Lime-Tenant-ID=tenant-0001");
let provider = OpenAiProvider::from_env(ModelConfig::new("gpt-4o").unwrap())
.await
.expect("provider should build");
assert_eq!(
provider
.api_client
.default_headers_for_test()
.get("X-Lime-Tenant-ID")
.and_then(|value| value.to_str().ok()),
Some("tenant-0001")
);
std::env::remove_var("OPENAI_API_KEY");
std::env::remove_var("OPENAI_HOST");
std::env::remove_var("OPENAI_CUSTOM_HEADERS");
}
#[test]
fn test_responses_path_from_base_path_reuses_existing_prefix() {
assert_eq!(
src-tauri/crates/aster-rust/crates/aster/src/tools/agent_control.rs
+27 -1
View File
@@ -472,7 +472,7 @@ impl Tool for SendInputTool {
"oneOf": [
{ "type": "string" },
{ "type": "object" },
{ "type": "array" },
{ "type": "array", "items": {} },
{ "type": "number" },
{ "type": "boolean" },
{ "type": "null" }
@@ -1181,6 +1181,32 @@ mod tests {
));
}
#[test]
fn test_send_message_array_schema_declares_items() {
let tool = SendInputTool::new(Arc::new(|_request| {
Box::pin(async move {
Ok(SendInputResponse {
submission_id: "unused".to_string(),
extra: BTreeMap::new(),
})
})
}));
let schema = tool.input_schema();
let message_variants = schema["properties"]["message"]["oneOf"]
.as_array()
.expect("message oneOf should be an array");
let array_variant = message_variants
.iter()
.find(|variant| variant["type"] == "array")
.expect("message should accept arrays");
assert!(
array_variant.get("items").is_some(),
"OpenAI-compatible tool schemas require array schemas to declare items"
);
}
#[tokio::test]
async fn test_send_message_tool_accepts_current_surface() {
let tool = SendInputTool::new(Arc::new(|request| {
src-tauri/crates/core/src/config/mod.rs
+9 -9
View File
@@ -37,14 +37,14 @@ pub use types::{
MultiSearchEngineEntryConfig, NativeAgentConfig, NavigationConfig, OpenAIAsrConfig,
PairingSettings, ProviderConfig, ProviderModelsConfig, ProvidersConfig, QuotaExceededConfig,
RateLimitSettings, RemoteManagementConfig, ResponseCacheSettings, RetrySettings, RoutingConfig,
ScreenshotChatConfig, SearchEngine, ServerConfig, ShellEnvironmentImportConfig, TaskSchedule,
TelegramAccountConfig, TelegramBotConfig, TelegramGroupConfig, TelegramTopicConfig, TlsConfig,
ToolCallingConfig, ToolExecutionOverrideConfig, ToolExecutionPolicyConfig,
ToolExecutionRestrictionProfileConfig, ToolExecutionSandboxProfileConfig,
ToolExecutionWarningPolicyConfig, UpdateCheckConfig, UserProfile, VertexApiKeyEntry,
VertexModelAlias, VoiceInputConfig, VoiceInstruction, VoiceOutputConfig, VoiceOutputMode,
VoiceProcessorConfig, WebSearchConfig, WebSearchProvider, WechatAccountConfig, WechatBotConfig,
WechatGroupConfig, WhisperLocalConfig, WhisperModelSize, WorkspacePreferencesConfig,
WorkspaceSandboxConfig, XunfeiConfig, DEFAULT_API_KEY,
ScreenshotChatConfig, SearchEngine, SenseVoiceLocalConfig, ServerConfig,
ShellEnvironmentImportConfig, TaskSchedule, TelegramAccountConfig, TelegramBotConfig,
TelegramGroupConfig, TelegramTopicConfig, TlsConfig, ToolCallingConfig,
ToolExecutionOverrideConfig, ToolExecutionPolicyConfig, ToolExecutionRestrictionProfileConfig,
ToolExecutionSandboxProfileConfig, ToolExecutionWarningPolicyConfig, UpdateCheckConfig,
UserProfile, VertexApiKeyEntry, VertexModelAlias, VoiceInputConfig, VoiceInstruction,
VoiceOutputConfig, VoiceOutputMode, VoiceProcessorConfig, WebSearchConfig, WebSearchProvider,
WechatAccountConfig, WechatBotConfig, WechatGroupConfig, WhisperLocalConfig, WhisperModelSize,
WorkspacePreferencesConfig, WorkspaceSandboxConfig, XunfeiConfig, DEFAULT_API_KEY,
};
pub use yaml::{load_config, save_config, ConfigError, ConfigManager, YamlService};
src-tauri/crates/core/src/config/types.rs
+51
View File
@@ -49,6 +49,8 @@ pub enum AsrProviderType {
/// 本地 Whisper（离线）
#[default]
WhisperLocal,
/// 本地 SenseVoice Small（离线）
SenseVoiceLocal,
/// 讯飞语音识别
Xunfei,
/// 百度语音识别
@@ -96,6 +98,9 @@ pub struct AsrCredentialEntry {
/// Whisper 本地配置（仅 WhisperLocal）
#[serde(default, skip_serializing_if = "Option::is_none")]
pub whisper_config: Option<WhisperLocalConfig>,
/// SenseVoice 本地配置（仅 SenseVoiceLocal）
#[serde(default, skip_serializing_if = "Option::is_none")]
pub sensevoice_config: Option<SenseVoiceLocalConfig>,
/// 讯飞配置（仅 Xunfei）
#[serde(default, skip_serializing_if = "Option::is_none")]
pub xunfei_config: Option<XunfeiConfig>,
@@ -122,6 +127,50 @@ pub struct WhisperLocalConfig {
pub model_path: Option<String>,
}
/// SenseVoice 本地配置
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct SenseVoiceLocalConfig {
/// 模型 ID
#[serde(default = "default_sensevoice_model_id")]
pub model_id: String,
/// 模型目录（可选，默认使用应用数据目录）
#[serde(default, skip_serializing_if = "Option::is_none")]
pub model_dir: Option<String>,
/// 是否启用逆文本标准化
#[serde(default = "default_sensevoice_use_itn")]
pub use_itn: bool,
/// 推理线程数
#[serde(default = "default_sensevoice_num_threads")]
pub num_threads: u16,
/// VAD 模型 ID（可选）
#[serde(default, skip_serializing_if = "Option::is_none")]
pub vad_model_id: Option<String>,
}
fn default_sensevoice_model_id() -> String {
"sensevoice-small-int8-2024-07-17".to_string()
}
fn default_sensevoice_use_itn() -> bool {
true
}
fn default_sensevoice_num_threads() -> u16 {
4
}
impl Default for SenseVoiceLocalConfig {
fn default() -> Self {
Self {
model_id: default_sensevoice_model_id(),
model_dir: None,
use_itn: default_sensevoice_use_itn(),
num_threads: default_sensevoice_num_threads(),
vad_model_id: Some("silero-vad-onnx".to_string()),
}
}
}
/// 讯飞语音配置
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct XunfeiConfig {
@@ -3384,6 +3433,7 @@ mod unit_tests {
model: WhisperModelSize::Base,
model_path: None,
}),
sensevoice_config: None,
xunfei_config: None,
baidu_config: None,
openai_config: None,
@@ -3437,6 +3487,7 @@ mod unit_tests {
disabled: false,
language: "zh".to_string(),
whisper_config: None,
sensevoice_config: None,
xunfei_config: Some(XunfeiConfig {
app_id: "test_app_id".to_string(),
api_key: "test_api_key".to_string(),
src-tauri/crates/core/src/database/dao/agent_timeline.rs
+92 -3
View File
@@ -6,17 +6,49 @@ use rusqlite::{params, Connection};
use serde::{Deserialize, Serialize};
const HISTORY_FILE_ARTIFACT_INLINE_CONTENT_BYTES_LIMIT: usize = 16 * 1024;
const HISTORY_ITEM_INLINE_OUTPUT_BYTES_LIMIT: usize = 16 * 1024;
fn history_item_payload_json_projection_sql() -> String {
format!(
"CASE
WHEN NOT json_valid(payload_json) THEN payload_json
WHEN item_type = 'file_artifact'
AND length(payload_json) > {limit}
AND json_valid(payload_json)
AND length(payload_json) > {file_limit}
THEN json_remove(payload_json, '$.content')
WHEN item_type = 'tool_call'
AND json_type(payload_json, '$.output') = 'text'
AND length(json_extract(payload_json, '$.output')) > {output_limit}
THEN json_set(
payload_json,
'$.output',
substr(json_extract(payload_json, '$.output'), 1, {output_limit})
|| char(10) || char(10)
|| '[历史输出已截断，完整输出未随首屏加载。]'
)
WHEN item_type = 'command_execution'
AND json_type(payload_json, '$.aggregated_output') = 'text'
AND length(json_extract(payload_json, '$.aggregated_output')) > {output_limit}
THEN json_set(
payload_json,
'$.aggregated_output',
substr(json_extract(payload_json, '$.aggregated_output'), 1, {output_limit})
|| char(10) || char(10)
|| '[历史输出已截断，完整输出未随首屏加载。]'
)
WHEN item_type = 'web_search'
AND json_type(payload_json, '$.output') = 'text'
AND length(json_extract(payload_json, '$.output')) > {output_limit}
THEN json_set(
payload_json,
'$.output',
substr(json_extract(payload_json, '$.output'), 1, {output_limit})
|| char(10) || char(10)
|| '[历史输出已截断，完整输出未随首屏加载。]'
)
ELSE payload_json
END",
limit = HISTORY_FILE_ARTIFACT_INLINE_CONTENT_BYTES_LIMIT,
file_limit = HISTORY_FILE_ARTIFACT_INLINE_CONTENT_BYTES_LIMIT,
output_limit = HISTORY_ITEM_INLINE_OUTPUT_BYTES_LIMIT,
)
}
@@ -876,6 +908,63 @@ mod tests {
));
}
#[test]
fn item_tail_query_should_truncate_large_tool_output() {
let conn = setup_conn();
let turn = AgentThreadTurn {
id: "turn-tool-output".to_string(),
thread_id: "thread-1".to_string(),
prompt_text: "生成封面".to_string(),
status: AgentThreadTurnStatus::Completed,
started_at: "2026-03-13T06:00:00Z".to_string(),
completed_at: Some("2026-03-13T06:00:10Z".to_string()),
error_message: None,
created_at: "2026-03-13T06:00:00Z".to_string(),
updated_at: "2026-03-13T06:00:10Z".to_string(),
};
AgentTimelineDao::create_turn(&conn, &turn).unwrap();
let large_output = format!("封面工具输出开始\n{}", "大结果".repeat(20_000));
AgentTimelineDao::upsert_item(
&conn,
&AgentThreadItem {
id: "item-large-tool-output".to_string(),
thread_id: "thread-1".to_string(),
turn_id: turn.id,
sequence: 1,
status: AgentThreadItemStatus::Completed,
started_at: "2026-03-13T06:00:01Z".to_string(),
completed_at: Some("2026-03-13T06:00:02Z".to_string()),
updated_at: "2026-03-13T06:00:02Z".to_string(),
payload: AgentThreadItemPayload::ToolCall {
tool_name: "social_generate_cover_image".to_string(),
arguments: Some(serde_json::json!({ "prompt": "咖啡封面" })),
output: Some(large_output.clone()),
success: Some(true),
error: None,
metadata: None,
},
},
)
.unwrap();
let full_items = AgentTimelineDao::list_items_by_thread(&conn, "thread-1").unwrap();
let tail_items = AgentTimelineDao::list_items_by_thread_tail(&conn, "thread-1", 1).unwrap();
assert!(matches!(
&full_items[0].payload,
AgentThreadItemPayload::ToolCall { output: Some(output), .. }
if output == &large_output
));
assert!(matches!(
&tail_items[0].payload,
AgentThreadItemPayload::ToolCall { output: Some(output), .. }
if output.starts_with("封面工具输出开始")
&& output.contains("历史输出已截断")
&& output.len() < large_output.len()
));
}
#[test]
fn update_turn_status_should_persist_terminal_state() {
let conn = setup_conn();
src-tauri/crates/core/src/database/system_providers.rs
+1 -1
View File
@@ -293,7 +293,7 @@ pub fn get_system_providers() -> Vec<SystemProviderDef> {
id: "sensenova",
name: "SenseNova",
provider_type: ApiProviderType::Openai,
api_host: "https://api.sensenova.cn/compatible-mode/v1",
api_host: "https://api.sensenova.cn/compatible-mode/v2",
group: ProviderGroup::Chinese,
sort_order: 29,
api_version: None,
src-tauri/crates/lime-cli/src/main.rs
+65
View File
@@ -14,6 +14,12 @@ use lime_media_runtime::{
};
use serde_json::{json, Value};
const AUDIO_TRANSCRIPTION_CONTRACT_KEY: &str = "audio_transcription";
const AUDIO_TRANSCRIPTION_MODALITY: &str = "audio";
const AUDIO_TRANSCRIPTION_ROUTING_SLOT: &str = "audio_transcription_model";
const AUDIO_TRANSCRIPTION_REQUIRED_CAPABILITIES: &[&str] =
&["text_generation", "audio_transcription"];
use crate::catalog::{
find_skill_entry, find_task_entry, ROOT_AFTER_HELP, SKILL_AFTER_HELP, SKILL_ENTRIES,
TASK_AFTER_HELP, TASK_ENTRIES,
@@ -967,6 +973,21 @@ fn create_transcription_task(args: TranscriptionGenerateArgs) -> Result<Value, M
"source_url 或 source_path 至少需要提供一个".to_string(),
));
}
let transcript = json!({
"kind": "transcript",
"status": "pending",
"source_url": source_url,
"source_path": source_path,
"language": args.language.as_deref(),
"output_format": args.output_format.as_deref(),
"speaker_labels": args.speaker_labels,
"timestamps": args.timestamps,
"provider_id": args.provider_id.as_deref(),
"model": args.model.as_deref(),
"modality_contract_key": AUDIO_TRANSCRIPTION_CONTRACT_KEY,
"modality": AUDIO_TRANSCRIPTION_MODALITY,
"routing_slot": AUDIO_TRANSCRIPTION_ROUTING_SLOT,
});
let output = write_task_artifact(
&workspace_root,
@@ -987,6 +1008,26 @@ fn create_transcription_task(args: TranscriptionGenerateArgs) -> Result<Value, M
"project_id": args.project_id,
"content_id": args.content_id,
"entry_source": args.entry_source,
"modality_contract_key": AUDIO_TRANSCRIPTION_CONTRACT_KEY,
"modality": AUDIO_TRANSCRIPTION_MODALITY,
"required_capabilities": AUDIO_TRANSCRIPTION_REQUIRED_CAPABILITIES,
"routing_slot": AUDIO_TRANSCRIPTION_ROUTING_SLOT,
"runtime_contract": {
"contract_key": AUDIO_TRANSCRIPTION_CONTRACT_KEY,
"modality": AUDIO_TRANSCRIPTION_MODALITY,
"required_capabilities": AUDIO_TRANSCRIPTION_REQUIRED_CAPABILITIES,
"routing_slot": AUDIO_TRANSCRIPTION_ROUTING_SLOT,
"executor_binding": {
"executor_kind": "skill",
"binding_key": "transcription_generate"
},
"truth_source": ["transcript_artifact", "runtime_timeline_event"],
"artifact_kinds": ["transcript"],
"viewer_surface": ["transcript_viewer", "document_viewer"],
"owner_surface": "agent_runtime"
},
"requested_target": "transcript",
"transcript": transcript,
}),
task_write_options(&args.output),
)?;
@@ -1376,6 +1417,30 @@ mod tests {
assert_eq!(output["record"]["payload"]["output_format"], "srt");
assert_eq!(output["record"]["payload"]["speaker_labels"], true);
assert_eq!(output["record"]["payload"]["timestamps"], true);
assert_eq!(
output["record"]["payload"]["modality_contract_key"],
AUDIO_TRANSCRIPTION_CONTRACT_KEY
);
assert_eq!(
output["record"]["payload"]["routing_slot"],
AUDIO_TRANSCRIPTION_ROUTING_SLOT
);
assert_eq!(
output["record"]["payload"]["runtime_contract"]["contract_key"],
AUDIO_TRANSCRIPTION_CONTRACT_KEY
);
assert_eq!(
output["record"]["payload"]["transcript"]["kind"],
"transcript"
);
assert_eq!(
output["record"]["payload"]["transcript"]["status"],
"pending"
);
assert_eq!(
output["record"]["payload"]["transcript"]["source_path"],
"/tmp/interview.wav"
);
}
#[test]
src-tauri/crates/providers/src/providers/claude_custom.rs
+115 -8
View File
@@ -4,11 +4,15 @@ use lime_core::database::dao::api_key_provider::{
};
use lime_core::models::anthropic::AnthropicMessagesRequest;
use lime_core::models::openai::{ChatCompletionRequest, ContentPart, MessageContent};
use reqwest::Client;
use reqwest::{Client, RequestBuilder};
use serde::{Deserialize, Serialize};
use serde_json::{json, Value};
use std::error::Error;
use std::time::Duration;
use url::{form_urlencoded, Url};
const LIME_TENANT_HEADER: &str = "X-Lime-Tenant-ID";
const LIME_TENANT_PARAM: &str = "lime_tenant_id";
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, Default)]
#[serde(rename_all = "snake_case")]
@@ -140,7 +144,7 @@ impl ClaudeCustomProvider {
/// 构建完整的 API URL
/// 智能处理用户输入的 base_url，无论是否带 /v1 都能正确工作
fn build_url(&self, endpoint: &str) -> String {
let base = self.get_base_url();
let base = Self::strip_config_url_metadata(&self.get_base_url());
let base = base.trim_end_matches('/');
// 如果用户输入了带 /v1 的 URL，直接拼接 endpoint
@@ -152,6 +156,65 @@ impl ClaudeCustomProvider {
}
}
fn parse_config_url(base_url: &str) -> Option<Url> {
let trimmed = base_url.trim();
if trimmed.is_empty() {
return None;
}
Url::parse(trimmed)
.or_else(|_| Url::parse(&format!("https://{trimmed}")))
.ok()
}
fn strip_config_url_metadata(base_url: &str) -> String {
if let Some(mut url) = Self::parse_config_url(base_url) {
url.set_query(None);
url.set_fragment(None);
return url.to_string().trim_end_matches('/').to_string();
}
let trimmed = base_url.trim();
let end = [trimmed.find('?'), trimmed.find('#')]
.into_iter()
.flatten()
.min()
.unwrap_or(trimmed.len());
trimmed[..end].trim_end_matches('/').to_string()
}
fn normalize_lime_tenant_id(value: &str) -> Option<String> {
let tenant_id = value.trim();
if tenant_id.is_empty() {
return None;
}
tenant_id
.chars()
.all(|ch| ch.is_ascii_alphanumeric() || matches!(ch, '-' | '_'))
.then(|| tenant_id.to_string())
}
fn parse_lime_tenant_id_from_pairs(value: &str) -> Option<String> {
form_urlencoded::parse(value.as_bytes()).find_map(|(key, value)| {
(key == LIME_TENANT_PARAM)
.then(|| Self::normalize_lime_tenant_id(&value))
.flatten()
})
}
fn lime_tenant_id_from_base_url(&self) -> Option<String> {
let base_url = self.config.base_url.as_deref()?;
let url = Self::parse_config_url(base_url)?;
url.query()
.and_then(Self::parse_lime_tenant_id_from_pairs)
.or_else(|| {
url.fragment()
.and_then(Self::parse_lime_tenant_id_from_pairs)
})
}
/// 将 OpenAI 图片 URL 格式转换为 Claude 图片格式
///
/// 支持两种格式：
@@ -378,14 +441,13 @@ impl ClaudeCustomProvider {
}
fn effective_runtime_spec(&self) -> ProviderRuntimeSpec {
infer_managed_runtime_spec(self.config.provider_type, &self.get_base_url())
infer_managed_runtime_spec(
self.config.provider_type,
&Self::strip_config_url_metadata(&self.get_base_url()),
)
}
fn apply_runtime_headers(
&self,
request: reqwest::RequestBuilder,
api_key: &str,
) -> reqwest::RequestBuilder {
fn apply_runtime_headers(&self, request: RequestBuilder, api_key: &str) -> RequestBuilder {
let api_key = api_key.trim();
let runtime_spec = self.effective_runtime_spec();
let auth_value = runtime_spec
@@ -412,6 +474,10 @@ impl ClaudeCustomProvider {
request = request.header(*name, *value);
}
if let Some(tenant_id) = self.lime_tenant_id_from_base_url() {
request = request.header(LIME_TENANT_HEADER, tenant_id);
}
request
}
@@ -1272,6 +1338,23 @@ mod tests {
assert_eq!(spec.auth_prefix, None);
}
#[test]
fn test_lime_tenant_id_from_base_url_fragment() {
let provider = ClaudeCustomProvider::with_config(
"test-key".to_string(),
Some("https://llm.limeai.run#lime_tenant_id=tenant-0001".to_string()),
);
assert_eq!(
provider.lime_tenant_id_from_base_url().as_deref(),
Some("tenant-0001")
);
assert_eq!(
provider.build_url("messages"),
"https://llm.limeai.run/v1/messages"
);
}
#[test]
fn test_apply_runtime_headers_adds_dual_auth_headers_for_minimax_host() {
let provider = ClaudeCustomProvider::with_config(
@@ -1310,6 +1393,30 @@ mod tests {
);
}
#[test]
fn test_apply_runtime_headers_adds_lime_tenant_header_from_fragment() {
let provider = ClaudeCustomProvider::with_config(
"test-key".to_string(),
Some("https://llm.limeai.run#lime_tenant_id=tenant-0001".to_string()),
);
let request = provider
.apply_runtime_headers(
reqwest::Client::new().post("https://example.com"),
"test-key",
)
.build()
.expect("构建请求失败");
assert_eq!(
request
.headers()
.get(LIME_TENANT_HEADER)
.and_then(|value| value.to_str().ok()),
Some("tenant-0001")
);
}
#[test]
fn test_apply_runtime_headers_keeps_official_anthropic_single_auth_header() {
let provider = ClaudeCustomProvider::with_config(
src-tauri/crates/providers/src/providers/openai_custom.rs
+73
View File
@@ -58,6 +58,8 @@ impl OpenAICustomProvider {
}
fn normalize_openai_request_payload(&self, payload: &mut serde_json::Value) {
self.normalize_provider_specific_request_payload(payload);
let model_name = payload
.get("model")
.and_then(|value| value.as_str())
@@ -151,6 +153,41 @@ impl OpenAICustomProvider {
}
}
fn normalize_provider_specific_request_payload(&self, payload: &mut serde_json::Value) {
if !self.uses_sensenova_compatible_api() {
return;
}
let Some(object) = payload.as_object_mut() else {
return;
};
if object.contains_key("max_completion_tokens") {
object.remove("max_tokens");
return;
}
if let Some(max_tokens) = object.remove("max_tokens") {
object.insert("max_completion_tokens".to_string(), max_tokens);
}
}
fn uses_sensenova_compatible_api(&self) -> bool {
self.config
.base_url
.as_deref()
.and_then(Self::parse_config_url)
.is_some_and(|url| {
url.host_str()
.map(|host| host.eq_ignore_ascii_case("api.sensenova.cn"))
.unwrap_or(false)
&& url
.path()
.trim_end_matches('/')
.eq_ignore_ascii_case("/compatible-mode/v2")
})
}
fn maybe_log_protocol_mismatch_hint(url: &str, status: StatusCode) {
if (status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN)
&& url.contains("/api/anthropic")
@@ -943,6 +980,42 @@ mod tests {
);
}
#[test]
fn test_normalize_openai_request_payload_uses_sensenova_token_field() {
let provider = OpenAICustomProvider::with_config(
"sk-test".to_string(),
Some("https://api.sensenova.cn/compatible-mode/v2".to_string()),
);
let mut payload = serde_json::json!({
"model": "SenseChat-5",
"messages": [{"role":"user","content":"hi"}],
"max_tokens": 64
});
provider.normalize_openai_request_payload(&mut payload);
assert!(payload.get("max_tokens").is_none());
assert_eq!(payload["max_completion_tokens"], serde_json::json!(64));
}
#[test]
fn test_normalize_openai_request_payload_keeps_standard_token_field() {
let provider = OpenAICustomProvider::with_config(
"sk-test".to_string(),
Some("https://api.deepseek.com".to_string()),
);
let mut payload = serde_json::json!({
"model": "deepseek-chat",
"messages": [{"role":"user","content":"hi"}],
"max_tokens": 64
});
provider.normalize_openai_request_payload(&mut payload);
assert_eq!(payload["max_tokens"], serde_json::json!(64));
assert!(payload.get("max_completion_tokens").is_none());
}
#[test]
fn test_normalize_openai_request_payload_clears_reasoning_before_latest_user() {
let provider = OpenAICustomProvider::default();
src-tauri/crates/services/Cargo.toml
+1
View File
@@ -8,6 +8,7 @@ repository.workspace = true
[features]
default = []
local-whisper = ["voice-core/local-whisper"]
local-sensevoice = ["voice-core/local-sensevoice"]
[dependencies]
# 项目内 crate
src-tauri/crates/services/src/api_key_provider_service.rs
+108 -6
View File
@@ -31,6 +31,10 @@ use std::collections::{HashMap, HashSet};
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::RwLock;
const SENSENOVA_OLD_OPENAI_COMPATIBLE_API_HOST: &str =
"https://api.sensenova.cn/compatible-mode/v1";
const SENSENOVA_OPENAI_COMPATIBLE_API_HOST: &str = "https://api.sensenova.cn/compatible-mode/v2";
// ============================================================================
// 连接测试结果
// ============================================================================
@@ -50,7 +54,10 @@ pub struct ConnectionTestResult {
#[cfg(test)]
mod tests {
use super::ApiKeyProviderService;
use super::{
ApiKeyProviderService, SENSENOVA_OLD_OPENAI_COMPATIBLE_API_HOST,
SENSENOVA_OPENAI_COMPATIBLE_API_HOST,
};
use base64::{engine::general_purpose::STANDARD as BASE64, Engine};
use chrono::Utc;
use lime_core::database::dao::api_key_provider::ApiProviderType;
@@ -419,6 +426,74 @@ data: [DONE]\n";
assert_eq!(persisted.provider.provider_type, ApiProviderType::Openai);
}
#[test]
fn test_initialize_system_providers_updates_legacy_sensenova_default_host() {
let db = init_test_database();
let service = ApiKeyProviderService::new();
service
.initialize_system_providers(&db)
.expect("初始化系统 Provider 失败");
{
let conn = db.lock().expect("获取数据库锁失败");
let mut provider = ApiKeyProviderDao::get_provider_by_id(&conn, "sensenova")
.expect("读取 SenseNova Provider 失败")
.expect("SenseNova Provider 应存在");
provider.api_host = SENSENOVA_OLD_OPENAI_COMPATIBLE_API_HOST.to_string();
ApiKeyProviderDao::update_provider(&conn, &provider)
.expect("写入旧 SenseNova 地址失败");
}
service
.initialize_system_providers(&db)
.expect("重新初始化系统 Provider 失败");
let persisted = service
.get_provider(&db, "sensenova")
.expect("读取 SenseNova Provider 失败")
.expect("SenseNova Provider 应存在");
assert_eq!(
persisted.provider.api_host,
SENSENOVA_OPENAI_COMPATIBLE_API_HOST
);
}
#[test]
fn test_initialize_system_providers_keeps_user_custom_sensenova_host() {
let db = init_test_database();
let service = ApiKeyProviderService::new();
service
.initialize_system_providers(&db)
.expect("初始化系统 Provider 失败");
{
let conn = db.lock().expect("获取数据库锁失败");
let mut provider = ApiKeyProviderDao::get_provider_by_id(&conn, "sensenova")
.expect("读取 SenseNova Provider 失败")
.expect("SenseNova Provider 应存在");
provider.api_host = "https://proxy.example.com/sensenova".to_string();
ApiKeyProviderDao::update_provider(&conn, &provider)
.expect("写入自定义 SenseNova 地址失败");
}
service
.initialize_system_providers(&db)
.expect("重新初始化系统 Provider 失败");
let persisted = service
.get_provider(&db, "sensenova")
.expect("读取 SenseNova Provider 失败")
.expect("SenseNova Provider 应存在");
assert_eq!(
persisted.provider.api_host,
"https://proxy.example.com/sensenova"
);
}
#[test]
fn test_add_custom_provider_should_force_known_anthropic_compatible_host_to_automatic() {
let db = init_test_database();
@@ -1900,27 +1975,54 @@ impl ApiKeyProviderService {
let conn = lime_core::database::lock_db(db)?;
let system_providers = get_system_providers();
let mut inserted_count = 0;
let mut updated_count = 0;
for def in &system_providers {
// 检查是否已存在
let existing =
ApiKeyProviderDao::get_provider_by_id(&conn, def.id).map_err(|e| e.to_string())?;
if existing.is_none() {
// 插入新的系统 Provider
let provider = to_api_key_provider(def);
ApiKeyProviderDao::insert_provider(&conn, &provider).map_err(|e| e.to_string())?;
inserted_count += 1;
match existing {
Some(mut provider) => {
if Self::should_update_system_provider_default_api_host(&provider, def.api_host)
{
provider.api_host = def.api_host.to_string();
provider.updated_at = Utc::now();
ApiKeyProviderDao::update_provider(&conn, &provider)
.map_err(|e| e.to_string())?;
updated_count += 1;
}
}
None => {
// 插入新的系统 Provider
let provider = to_api_key_provider(def);
ApiKeyProviderDao::insert_provider(&conn, &provider)
.map_err(|e| e.to_string())?;
inserted_count += 1;
}
}
}
if inserted_count > 0 {
tracing::info!("初始化了 {} 个系统 Provider", inserted_count);
}
if updated_count > 0 {
tracing::info!("同步了 {} 个系统 Provider 默认地址", updated_count);
}
Ok(inserted_count)
}
fn should_update_system_provider_default_api_host(
provider: &ApiKeyProvider,
next_api_host: &str,
) -> bool {
provider.id == "sensenova"
&& provider.is_system
&& provider.api_host.trim_end_matches('/') == SENSENOVA_OLD_OPENAI_COMPATIBLE_API_HOST
&& next_api_host == SENSENOVA_OPENAI_COMPATIBLE_API_HOST
}
/// 获取所有 Provider（包含 API Keys）
/// 首次调用时会自动初始化系统 Provider
pub fn get_all_providers(&self, db: &DbConnection) -> Result<Vec<ProviderWithKeys>, String> {
src-tauri/crates/services/src/voice_asr_service.rs
+161 -4
View File
@@ -23,9 +23,9 @@
//! let text = AsrService::transcribe(&credential, &audio_data, 16000).await?;
//! ```
#[cfg(feature = "local-whisper")]
use std::path::PathBuf;
use lime_core::app_paths;
#[cfg(feature = "local-whisper")]
use lime_core::config::WhisperModelSize;
use lime_core::config::{AsrCredentialEntry, AsrProviderType};
@@ -34,6 +34,10 @@ use super::voice_config_service;
use voice_core::asr_client::{AsrClient, BaiduClient, OpenAIWhisperClient, XunfeiClient};
use voice_core::types::AudioData;
const SENSEVOICE_MODEL_FILE: &str = "model.int8.onnx";
const SENSEVOICE_TOKENS_FILE: &str = "tokens.txt";
const SENSEVOICE_VAD_FILE: &str = "silero_vad.onnx";
/// ASR 服务
pub struct AsrService;
@@ -56,9 +60,16 @@ impl AsrService {
audio_data: &[u8],
sample_rate: u32,
) -> Result<String, String> {
// 如果是本地 Whisper，直接调用
if matches!(credential.provider, AsrProviderType::WhisperLocal) {
return Self::transcribe_whisper_local(credential, audio_data, sample_rate).await;
// 本地服务直接调用，不走云端回退链。
match credential.provider {
AsrProviderType::WhisperLocal => {
return Self::transcribe_whisper_local(credential, audio_data, sample_rate).await;
}
AsrProviderType::SenseVoiceLocal => {
return Self::transcribe_sensevoice_local(credential, audio_data, sample_rate)
.await;
}
_ => {}
}
// 云端服务：先尝试云端，失败则回退到本地 Whisper
@@ -73,6 +84,7 @@ impl AsrService {
Self::transcribe_xunfei(credential, audio_data, sample_rate).await
}
AsrProviderType::WhisperLocal => unreachable!(), // 已在上面处理
AsrProviderType::SenseVoiceLocal => unreachable!(), // 已在上面处理
};
// 云端成功，直接返回
@@ -178,6 +190,114 @@ impl AsrService {
Err("本地 Whisper 功能未启用。请使用云端 ASR 服务（OpenAI、百度、讯飞）".to_string())
}
/// 本地 SenseVoice 识别。
#[cfg(feature = "local-sensevoice")]
async fn transcribe_sensevoice_local(
credential: &AsrCredentialEntry,
audio_data: &[u8],
sample_rate: u32,
) -> Result<String, String> {
let config = credential
.sensevoice_config
.as_ref()
.ok_or("SenseVoice 本地配置缺失")?;
let model_dir = Self::get_sensevoice_model_dir(config)?;
let model_path = model_dir.join(SENSEVOICE_MODEL_FILE);
let tokens_path = model_dir.join(SENSEVOICE_TOKENS_FILE);
let vad_path = model_dir.join(SENSEVOICE_VAD_FILE);
Self::ensure_sensevoice_required_files(&[
(&model_path, SENSEVOICE_MODEL_FILE),
(&tokens_path, SENSEVOICE_TOKENS_FILE),
(&vad_path, SENSEVOICE_VAD_FILE),
])?;
let audio = Self::build_audio_data(audio_data, sample_rate)?;
if !audio.is_valid() {
return Err("录音时间过短（需要至少 0.5 秒）".to_string());
}
let language = if credential.language.trim().is_empty() {
"auto".to_string()
} else {
credential.language.clone()
};
let use_itn = config.use_itn;
let num_threads = config.num_threads;
tokio::task::spawn_blocking(move || {
let transcriber = voice_core::SenseVoiceTranscriber::new(
model_path,
tokens_path,
&language,
use_itn,
num_threads,
)
.map_err(|error| format!("SenseVoice 模型加载失败: {error}"))?;
let result = transcriber
.transcribe(&audio)
.map_err(|error| format!("SenseVoice 识别失败: {error}"))?;
Ok::<_, String>(result.text)
})
.await
.map_err(|error| format!("SenseVoice 识别任务执行失败: {error}"))?
}
/// 本地 SenseVoice 识别（未启用 local-sensevoice feature 时的 stub）
#[cfg(not(feature = "local-sensevoice"))]
async fn transcribe_sensevoice_local(
_credential: &AsrCredentialEntry,
_audio_data: &[u8],
_sample_rate: u32,
) -> Result<String, String> {
Err(
"本地 SenseVoice 功能未启用。请启用 local-sensevoice 构建特性，或使用云端 ASR 服务"
.to_string(),
)
}
fn get_sensevoice_model_dir(
config: &lime_core::config::SenseVoiceLocalConfig,
) -> Result<PathBuf, String> {
if let Some(model_dir) = config.model_dir.as_ref() {
let trimmed = model_dir.trim();
if !trimmed.is_empty() {
return Ok(PathBuf::from(trimmed));
}
}
Ok(app_paths::preferred_data_dir()?
.join("models")
.join("voice")
.join(&config.model_id))
}
fn ensure_sensevoice_required_files(files: &[(&PathBuf, &str)]) -> Result<(), String> {
let missing_files = files
.iter()
.filter_map(
|(path, name)| {
if path.is_file() {
None
} else {
Some(*name)
}
},
)
.collect::<Vec<_>>();
if missing_files.is_empty() {
Ok(())
} else {
Err(format!(
"本地 SenseVoice Small 尚未安装或文件不完整，请先在设置 -> 语音模型中下载；缺失文件: {}",
missing_files.join(", ")
))
}
}
/// 获取 Whisper 模型文件路径
#[cfg(feature = "local-whisper")]
fn get_whisper_model_path(model_size: &WhisperModelSize) -> Result<PathBuf, String> {
@@ -308,3 +428,40 @@ impl AsrService {
Ok(audio)
}
}
#[cfg(test)]
mod tests {
use super::*;
use lime_core::config::SenseVoiceLocalConfig;
#[test]
fn sensevoice_model_dir_prefers_explicit_config_path() {
let config = SenseVoiceLocalConfig {
model_dir: Some("/tmp/lime-sensevoice".to_string()),
..SenseVoiceLocalConfig::default()
};
let path = AsrService::get_sensevoice_model_dir(&config).expect("model dir");
assert_eq!(path, PathBuf::from("/tmp/lime-sensevoice"));
}
#[test]
fn sensevoice_required_files_reports_missing_names() {
let temp = tempfile::tempdir().expect("tempdir");
let model_path = temp.path().join(SENSEVOICE_MODEL_FILE);
std::fs::write(&model_path, b"mock").expect("write model");
let tokens_path = temp.path().join(SENSEVOICE_TOKENS_FILE);
let vad_path = temp.path().join(SENSEVOICE_VAD_FILE);
let error = AsrService::ensure_sensevoice_required_files(&[
(&model_path, SENSEVOICE_MODEL_FILE),
(&tokens_path, SENSEVOICE_TOKENS_FILE),
(&vad_path, SENSEVOICE_VAD_FILE),
])
.expect_err("missing files should fail");
assert!(error.contains(SENSEVOICE_TOKENS_FILE));
assert!(error.contains(SENSEVOICE_VAD_FILE));
}
}
src-tauri/crates/services/src/voice_config_service.rs
+1
View File
@@ -156,6 +156,7 @@ pub fn resolve_output_mode(mode: Option<&str>) -> Result<VoiceOutputMode, String
pub fn asr_provider_name(provider: AsrProviderType) -> &'static str {
match provider {
AsrProviderType::WhisperLocal => "本地 Whisper",
AsrProviderType::SenseVoiceLocal => "SenseVoice Small 本地",
AsrProviderType::OpenAI => "OpenAI Whisper",
AsrProviderType::Baidu => "百度语音",
AsrProviderType::Xunfei => "讯飞语音",
src-tauri/crates/voice-core/Cargo.toml
+5
View File
@@ -10,6 +10,8 @@ license = "MIT"
default = []
# 本地 Whisper 识别（编译很慢，CI 默认不启用）
local-whisper = ["dep:whisper-rs"]
# 本地 SenseVoice Small 识别（模型仍按需下载，不进入安装包）
local-sensevoice = ["dep:sherpa-onnx"]
[dependencies]
# 音频录制
@@ -18,6 +20,9 @@ cpal = "0.15"
# Whisper 本地识别（可选，编译耗时）
whisper-rs = { version = "0.12", optional = true }
# SenseVoice 本地识别（可选，链接 sherpa-onnx 运行时，不内置模型文件）
sherpa-onnx = { version = "1.13", default-features = false, features = ["static"], optional = true }
# WAV 处理
hound = "3.5"
src-tauri/crates/voice-core/src/lib.rs
+4
View File
@@ -8,6 +8,8 @@ pub mod device;
pub mod error;
pub mod output;
pub mod recorder;
#[cfg(feature = "local-sensevoice")]
pub mod sensevoice_transcriber;
pub mod text_polish;
pub mod threaded_recorder;
#[cfg(feature = "local-whisper")]
@@ -18,6 +20,8 @@ pub use device::{list_audio_devices, AudioDeviceInfo};
pub use error::{Result, VoiceError};
pub use output::OutputHandler;
pub use recorder::AudioRecorder;
#[cfg(feature = "local-sensevoice")]
pub use sensevoice_transcriber::SenseVoiceTranscriber;
pub use threaded_recorder::{RecordingCommand, RecordingResponse, RecordingService};
#[cfg(feature = "local-whisper")]
pub use transcriber::WhisperTranscriber;
src-tauri/crates/voice-core/src/sensevoice_transcriber.rs
+158
View File
@@ -0,0 +1,158 @@
//! SenseVoice Small 本地语音识别模块。
//!
//! 模型文件由上层按需下载；这里只负责把已安装模型接入 sherpa-onnx
//! offline recognizer。
use std::path::{Path, PathBuf};
use sherpa_onnx::{
LinearResampler, OfflineRecognizer, OfflineRecognizerConfig, OfflineSenseVoiceModelConfig,
};
use crate::error::{Result, VoiceError};
use crate::types::{AudioData, Segment, TranscribeResult};
const SENSEVOICE_SAMPLE_RATE: i32 = 16_000;
/// SenseVoice 识别器。
pub struct SenseVoiceTranscriber {
recognizer: OfflineRecognizer,
language: String,
}
impl SenseVoiceTranscriber {
/// 创建新的 SenseVoice 识别器。
pub fn new(
model_path: PathBuf,
tokens_path: PathBuf,
language: &str,
use_itn: bool,
num_threads: u16,
) -> Result<Self> {
ensure_file_exists(&model_path, "SenseVoice 模型文件")?;
ensure_file_exists(&tokens_path, "SenseVoice tokens 文件")?;
let language = normalize_language(language);
let mut config = OfflineRecognizerConfig::default();
config.model_config.sense_voice = OfflineSenseVoiceModelConfig {
model: Some(model_path.to_string_lossy().to_string()),
language: Some(language.clone()),
use_itn,
};
config.model_config.tokens = Some(tokens_path.to_string_lossy().to_string());
config.model_config.num_threads = i32::from(num_threads.max(1));
let recognizer = OfflineRecognizer::create(&config).ok_or_else(|| {
VoiceError::TranscriberError("创建 sherpa-onnx SenseVoice 识别器失败".to_string())
})?;
Ok(Self {
recognizer,
language,
})
}
/// 识别音频。
pub fn transcribe(&self, audio: &AudioData) -> Result<TranscribeResult> {
if !audio.is_valid() {
return Err(VoiceError::RecordingTooShort);
}
let samples = prepare_samples(audio)?;
if samples.is_empty() {
return Err(VoiceError::AudioFormatError(
"SenseVoice 输入音频为空".to_string(),
));
}
let stream = self.recognizer.create_stream();
stream.accept_waveform(SENSEVOICE_SAMPLE_RATE, &samples);
self.recognizer.decode(&stream);
let result = stream.get_result().ok_or_else(|| {
VoiceError::TranscriberError("读取 sherpa-onnx SenseVoice 识别结果失败".to_string())
})?;
let text = result.text.trim().to_string();
let segments = if text.is_empty() {
Vec::new()
} else {
vec![Segment {
start: 0.0,
end: audio.duration_secs,
text: text.clone(),
}]
};
Ok(TranscribeResult {
text,
language: (self.language != "auto").then(|| self.language.clone()),
confidence: None,
segments,
})
}
}
fn ensure_file_exists(path: &Path, label: &str) -> Result<()> {
if path.is_file() {
Ok(())
} else {
Err(VoiceError::TranscriberError(format!(
"{label}不存在: {}",
path.display()
)))
}
}
fn normalize_language(language: &str) -> String {
let normalized = language.trim();
if normalized.is_empty() {
"auto".to_string()
} else {
normalized.to_string()
}
}
fn prepare_samples(audio: &AudioData) -> Result<Vec<f32>> {
let samples = audio
.samples
.iter()
.map(|sample| *sample as f32 / i16::MAX as f32)
.collect::<Vec<_>>();
if audio.sample_rate == SENSEVOICE_SAMPLE_RATE as u32 {
return Ok(samples);
}
let resampler = LinearResampler::create(audio.sample_rate as i32, SENSEVOICE_SAMPLE_RATE)
.ok_or_else(|| {
VoiceError::AudioFormatError(format!(
"创建采样率转换器失败: {} -> {}",
audio.sample_rate, SENSEVOICE_SAMPLE_RATE
))
})?;
Ok(resampler.resample(&samples, true))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn normalize_empty_language_to_auto() {
assert_eq!(normalize_language(""), "auto");
assert_eq!(normalize_language(" "), "auto");
assert_eq!(normalize_language("zh"), "zh");
}
#[test]
fn prepare_samples_keeps_16khz_length() {
let audio = AudioData::new(vec![0, i16::MAX, i16::MIN], 16_000, 1);
let samples = prepare_samples(&audio).expect("prepare samples");
assert_eq!(samples.len(), 3);
assert_eq!(samples[0], 0.0);
assert!(samples[1] > 0.99);
assert!(samples[2] < -0.99);
}
}
src-tauri/src/app/runner.rs
+7
View File
@@ -1507,6 +1507,7 @@ pub fn run() {
commands::video_generation_cmd::cancel_video_generation_task,
commands::media_task_cmd::create_image_generation_task_artifact,
commands::media_task_cmd::create_audio_generation_task_artifact,
commands::media_task_cmd::complete_audio_generation_task_artifact,
commands::media_task_cmd::get_media_task_artifact,
commands::media_task_cmd::list_media_task_artifacts,
commands::media_task_cmd::cancel_media_task_artifact,
@@ -1596,6 +1597,12 @@ pub fn run() {
commands::asr_cmd::delete_asr_credential,
commands::asr_cmd::set_default_asr_credential,
commands::asr_cmd::test_asr_credential,
commands::voice_model_cmd::voice_models_list_catalog,
commands::voice_model_cmd::voice_models_get_install_state,
commands::voice_model_cmd::voice_models_download,
commands::voice_model_cmd::voice_models_delete,
commands::voice_model_cmd::voice_models_set_default,
commands::voice_model_cmd::voice_models_test_transcribe_file,
// External Tools commands (Codex CLI 等外部工具)
commands::external_tools_cmd::check_codex_cli_status,
commands::external_tools_cmd::open_codex_cli_login,
src-tauri/src/commands/agent_cmd.rs
+45 -1
View File
@@ -21,6 +21,7 @@ use aster::conversation::message::Message;
use futures::StreamExt;
use lime_agent::merge_system_prompt_with_runtime_agents;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use tauri::{AppHandle, State};
use uuid::Uuid;
@@ -172,6 +173,20 @@ fn build_title_generation_system_prompt(title_kind: &str) -> String {
}
}
fn force_direct_answer_tool_surface(metadata: &mut Option<Value>) {
let Some(Value::Object(root)) = metadata else {
return;
};
let Some(Value::Object(runtime_metadata)) = root.get_mut("lime_runtime") else {
return;
};
runtime_metadata.insert(
"tool_surface".to_string(),
Value::String("direct_answer".to_string()),
);
}
async fn generate_title_with_agent(
agent_state: &AsterAgentState,
db: &DbConnection,
@@ -204,7 +219,7 @@ async fn generate_title_with_agent(
)),
None => Some(build_title_generation_system_prompt(title_kind)),
};
let auxiliary_runtime_metadata = build_auxiliary_runtime_metadata(
let mut auxiliary_runtime_metadata = build_auxiliary_runtime_metadata(
provider_scope.resolution(),
if title_kind == "image_task" {
"auxiliary_generation_topic"
@@ -228,6 +243,7 @@ async fn generate_title_with_agent(
},
&["当前为内部标题生成辅助任务，只会使用一条已解析的 provider/model 路线。"],
);
force_direct_answer_tool_surface(&mut auxiliary_runtime_metadata);
let session_config = build_auxiliary_session_config_with_turn_context(
session_id,
system_prompt,
@@ -466,3 +482,31 @@ pub async fn agent_generate_title(
Ok(result)
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn force_direct_answer_tool_surface_marks_auxiliary_runtime() {
let mut metadata = Some(json!({
"lime_runtime": {
"task_profile": {
"kind": "title_generation"
}
}
}));
force_direct_answer_tool_surface(&mut metadata);
assert_eq!(
metadata
.as_ref()
.and_then(|value| value.get("lime_runtime"))
.and_then(|value| value.get("tool_surface"))
.and_then(Value::as_str),
Some("direct_answer")
);
}
}
src-tauri/src/commands/asr_cmd.rs
+67 -1
View File
@@ -4,12 +4,18 @@
use crate::config::{
load_config, save_config, AsrCredentialEntry, AsrProviderType, BaiduConfig, OpenAIAsrConfig,
WhisperLocalConfig, XunfeiConfig,
SenseVoiceLocalConfig, WhisperLocalConfig, XunfeiConfig,
};
use lime_core::app_paths;
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
use tauri::command;
use uuid::Uuid;
const SENSEVOICE_MODEL_FILE: &str = "model.int8.onnx";
const SENSEVOICE_TOKENS_FILE: &str = "tokens.txt";
const SENSEVOICE_VAD_FILE: &str = "silero_vad.onnx";
/// 获取所有 ASR 凭证
#[command]
pub async fn get_asr_credentials() -> Result<Vec<AsrCredentialEntry>, String> {
@@ -32,6 +38,8 @@ pub struct AddAsrCredentialRequest {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub whisper_config: Option<WhisperLocalConfig>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub sensevoice_config: Option<SenseVoiceLocalConfig>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub xunfei_config: Option<XunfeiConfig>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub baidu_config: Option<BaiduConfig>,
@@ -68,6 +76,7 @@ pub async fn add_asr_credential(
disabled: entry.disabled,
language: entry.language,
whisper_config: entry.whisper_config,
sensevoice_config: entry.sensevoice_config,
xunfei_config: entry.xunfei_config,
baidu_config: entry.baidu_config,
openai_config: entry.openai_config,
@@ -189,6 +198,27 @@ pub async fn test_asr_credential(id: String) -> Result<TestResult, String> {
message: "本地 Whisper 已就绪".to_string(),
})
}
AsrProviderType::SenseVoiceLocal => {
let Some(sensevoice_config) = credential.sensevoice_config.as_ref() else {
return Ok(TestResult {
success: false,
message: "SenseVoice 本地配置缺失".to_string(),
});
};
match resolve_sensevoice_model_dir(sensevoice_config)
.and_then(|model_dir| ensure_sensevoice_model_files(&model_dir))
{
Ok(()) => Ok(TestResult {
success: true,
message: "SenseVoice Small 本地模型已就绪".to_string(),
}),
Err(error) => Ok(TestResult {
success: false,
message: error,
}),
}
}
AsrProviderType::Xunfei => {
// TODO: 实现讯飞 API 测试
if credential.xunfei_config.is_some() {
@@ -234,6 +264,42 @@ pub async fn test_asr_credential(id: String) -> Result<TestResult, String> {
}
}
fn resolve_sensevoice_model_dir(config: &SenseVoiceLocalConfig) -> Result<PathBuf, String> {
if let Some(model_dir) = config.model_dir.as_ref() {
let trimmed = model_dir.trim();
if !trimmed.is_empty() {
return Ok(PathBuf::from(trimmed));
}
}
Ok(app_paths::preferred_data_dir()?
.join("models")
.join("voice")
.join(&config.model_id))
}
fn ensure_sensevoice_model_files(model_dir: &Path) -> Result<(), String> {
let required_files = [
SENSEVOICE_MODEL_FILE,
SENSEVOICE_TOKENS_FILE,
SENSEVOICE_VAD_FILE,
];
let missing_files = required_files
.iter()
.filter(|file_name| !model_dir.join(file_name).is_file())
.copied()
.collect::<Vec<_>>();
if missing_files.is_empty() {
Ok(())
} else {
Err(format!(
"SenseVoice Small 本地模型文件不完整，请先在设置 -> 语音模型中下载；缺失文件: {}",
missing_files.join(", ")
))
}
}
/// 测试结果
#[derive(serde::Serialize)]
pub struct TestResult {
src-tauri/src/commands/aster_agent_cmd/mod.rs
+1
View File
@@ -510,6 +510,7 @@ pub(crate) use tool_runtime::{
pub(crate) use transcription_skill_launch::{
append_transcription_skill_launch_session_permissions,
merge_system_prompt_with_transcription_skill_launch,
prepare_transcription_skill_launch_request_metadata,
prune_transcription_skill_launch_detour_tools_from_registry,
};
pub(crate) use translation_skill_launch::{
src-tauri/src/commands/aster_agent_cmd/runtime_turn.rs
+29 -3
View File
@@ -232,7 +232,7 @@ fn quoted_absolute_path_regex() -> &'static Regex {
fn unix_absolute_path_regex() -> &'static Regex {
static REGEX: OnceLock<Regex> = OnceLock::new();
REGEX.get_or_init(|| {
Regex::new(r#"(?P<path>/[^\s"'“”‘’,;:(){}\[\]<>]+)"#)
Regex::new(r#"(?P<path>/[^\s"'“”‘’,;:，。；：、()（）{}\[\]【】<>《》]+)"#)
.expect("unix absolute path regex should compile")
})
}
@@ -240,7 +240,7 @@ fn unix_absolute_path_regex() -> &'static Regex {
fn windows_absolute_path_regex() -> &'static Regex {
static REGEX: OnceLock<Regex> = OnceLock::new();
REGEX.get_or_init(|| {
Regex::new(r#"(?P<path>[A-Za-z]:\\[^\s"'“”‘’,;:(){}\[\]<>]+)"#)
Regex::new(r#"(?P<path>[A-Za-z]:\\[^\s"'“”‘’,;:，。；：、()（）{}\[\]【】<>《》]+)"#)
.expect("windows absolute path regex should compile")
})
}
@@ -1554,6 +1554,8 @@ async fn prepare_runtime_turn_request(
prepare_translation_skill_launch_request_metadata(request.metadata.as_ref());
request.metadata =
prepare_analysis_skill_launch_request_metadata(request.metadata.as_ref());
request.metadata =
prepare_transcription_skill_launch_request_metadata(request.metadata.as_ref());
request.metadata =
prepare_typesetting_skill_launch_request_metadata(request.metadata.as_ref());
request.metadata = prepare_webpage_skill_launch_request_metadata(request.metadata.as_ref());
@@ -5822,8 +5824,11 @@ mod tests {
#[test]
fn resolve_fast_chat_tool_surface_mode_should_use_local_workspace_for_explicit_local_path() {
let temp_dir = tempfile::TempDir::new().expect("create temp dir");
let repo_dir = temp_dir.path().join("claudecode");
std::fs::create_dir_all(&repo_dir).expect("create repo dir");
let request = build_runtime_turn_test_request(
"请读取并分析项目 /Users/coso/Documents/dev/js/claudecode",
&format!("请读取并分析项目 {}", repo_dir.display()),
None,
);
let policy = lime_agent::resolve_request_tool_policy(Some(false), false);
@@ -5875,6 +5880,27 @@ mod tests {
assert!(paths.contains(&plain_expected));
}
#[test]
fn extract_explicit_local_focus_paths_from_message_should_stop_at_chinese_punctuation() {
let temp_dir = tempfile::TempDir::new().expect("create temp dir");
let package_json = temp_dir.path().join("package.json");
std::fs::write(&package_json, r#"{"name":"lime"}"#).expect("write package json");
let message = format!(
"必须先调用 Read 工具读取 {}，并确认 JSON 里的 name 字段。",
package_json.display()
);
let paths = extract_explicit_local_focus_paths_from_message(&message);
let expected = package_json
.canonicalize()
.expect("canonicalize package json")
.to_string_lossy()
.to_string();
assert_eq!(paths, vec![expected]);
}
#[test]
fn merge_system_prompt_with_explicit_local_path_focus_should_append_focus_guidance() {
let temp_dir = tempfile::TempDir::new().expect("create temp dir");
src-tauri/src/commands/aster_agent_cmd/tests.rs
+64
View File
@@ -4824,6 +4824,13 @@ mod tests {
assert!(merged.contains("不要先走 ToolSearch / WebSearch / Read / Glob / Grep"));
assert!(merged.contains("应立即改为直调 Skill(transcription_generate)"));
assert!(merged.contains("不要伪造“转写已完成”"));
assert!(merged.contains(
"modality_contract_key=audio_transcription, modality=audio, routing_slot=audio_transcription_model"
));
assert!(merged.contains("当前合同所需能力：text_generation, audio_transcription"));
assert!(merged.contains("frontend_direct_asr"));
assert!(merged.contains("runtime_contract(JSON)"));
assert!(merged.contains("contract_key\":\"audio_transcription"));
assert!(merged.contains("当前任务已经显式进入转写技能主链"));
}
@@ -5496,6 +5503,63 @@ mod tests {
);
}
#[test]
fn test_prepare_transcription_skill_launch_request_metadata_sets_contract_fields() {
let metadata = serde_json::json!({
"harness": {
"transcription_skill_launch": {
"skill_name": "transcription_generate",
"kind": "transcription_task",
"transcription_task": {
"source_url": "https://example.com/interview.mp4"
}
}
}
});
let prepared = prepare_transcription_skill_launch_request_metadata(Some(&metadata))
.expect("prepared metadata");
let harness = prepared
.get("harness")
.and_then(serde_json::Value::as_object)
.expect("harness");
let launch = harness
.get("transcription_skill_launch")
.and_then(serde_json::Value::as_object)
.expect("transcription_skill_launch");
assert_eq!(
launch
.get("modality_contract_key")
.and_then(serde_json::Value::as_str),
Some("audio_transcription")
);
let transcription_task = launch
.get("transcription_task")
.and_then(serde_json::Value::as_object)
.expect("transcription_task");
assert_eq!(
transcription_task
.get("modality_contract_key")
.and_then(serde_json::Value::as_str),
Some("audio_transcription")
);
assert_eq!(
transcription_task
.get("routing_slot")
.and_then(serde_json::Value::as_str),
Some("audio_transcription_model")
);
assert_eq!(
transcription_task
.get("runtime_contract")
.and_then(|value| value.get("executor_binding"))
.and_then(|value| value.get("binding_key"))
.and_then(serde_json::Value::as_str),
Some("transcription_generate")
);
}
#[test]
fn test_prepare_research_skill_launch_request_metadata_sets_workbench_chat_mode() {
let metadata = serde_json::json!({
src-tauri/src/commands/aster_agent_cmd/tool_runtime/creation_tools.rs
+72 -31
View File
@@ -2,14 +2,17 @@ use super::*;
use crate::agent_tools::catalog::LIME_CREATE_AUDIO_TASK_TOOL_NAME;
use crate::commands::media_task_cmd::{
create_audio_generation_task_artifact_inner, create_image_generation_task_artifact_inner,
finalize_audio_generation_task_creation, finalize_image_generation_task_creation,
create_transcription_task_artifact_inner, finalize_audio_generation_task_creation,
finalize_image_generation_task_creation, finalize_transcription_task_creation,
CreateAudioGenerationTaskArtifactRequest, CreateImageGenerationTaskArtifactRequest,
ImageStoryboardSlotInput,
CreateTranscriptionTaskArtifactRequest, ImageStoryboardSlotInput,
};
use crate::commands::modality_runtime_contracts::{
image_generation_required_capabilities, IMAGE_GENERATION_CONTRACT_KEY,
IMAGE_GENERATION_MODALITY, IMAGE_GENERATION_ROUTING_SLOT, VOICE_GENERATION_CONTRACT_KEY,
VOICE_GENERATION_MODALITY, VOICE_GENERATION_ROUTING_SLOT,
audio_transcription_required_capabilities, image_generation_required_capabilities,
AUDIO_TRANSCRIPTION_CONTRACT_KEY, AUDIO_TRANSCRIPTION_MODALITY,
AUDIO_TRANSCRIPTION_ROUTING_SLOT, IMAGE_GENERATION_CONTRACT_KEY, IMAGE_GENERATION_MODALITY,
IMAGE_GENERATION_ROUTING_SLOT, VOICE_GENERATION_CONTRACT_KEY, VOICE_GENERATION_MODALITY,
VOICE_GENERATION_ROUTING_SLOT,
};
use lime_media_runtime::{
write_task_artifact, MediaTaskType, TaskRelationships, TaskType, TaskWriteOptions,
@@ -684,6 +687,18 @@ struct TranscriptionTaskInput {
#[serde(default)]
entry_source: Option<String>,
#[serde(default)]
modality_contract_key: Option<String>,
#[serde(default)]
modality: Option<String>,
#[serde(default)]
required_capabilities: Vec<String>,
#[serde(default)]
routing_slot: Option<String>,
#[serde(default)]
runtime_contract: Option<serde_json::Value>,
#[serde(default)]
requested_target: Option<String>,
#[serde(default)]
output_path: Option<String>,
}
@@ -1332,10 +1347,11 @@ fn submit_audio_generation_task_record(
input: AudioTaskInput,
) -> Result<ToolResult, ToolError> {
let request = build_audio_generation_task_request(context, input);
let project_root_path = request.project_root_path.trim().to_string();
let output = create_audio_generation_task_artifact_inner(request)
.map_err(|error| ToolError::execution_failed(format!("创建音频任务失败: {error}")))?;
finalize_audio_generation_task_creation(Some(app_handle), &output);
finalize_audio_generation_task_creation(Some(app_handle), project_root_path.as_str(), &output);
let serialized = serde_json::to_string_pretty(&output)
.unwrap_or_else(|_| serde_json::json!(&output).to_string());
@@ -1449,31 +1465,56 @@ impl Tool for LimeCreateTranscriptionTaskTool {
));
}
let payload = serde_json::json!({
"prompt": input.prompt,
"raw_text": input.raw_text,
"source_url": source_url,
"source_path": source_path,
"language": input.language,
"output_format": input.output_format,
"speaker_labels": input.speaker_labels,
"timestamps": input.timestamps,
"provider_id": input.provider_id,
"model": input.model,
"session_id": input.session_id,
"project_id": input.project_id,
"content_id": input.content_id,
"entry_source": input.entry_source
});
submit_creation_task_record(
&self.app_handle,
context,
TaskType::TranscriptionGenerate,
input.title,
payload,
None,
input.output_path.as_deref(),
)
let output =
create_transcription_task_artifact_inner(CreateTranscriptionTaskArtifactRequest {
project_root_path: context.working_directory.to_string_lossy().to_string(),
prompt: input.prompt,
title: input.title,
raw_text: input.raw_text,
source_url: source_url.map(ToString::to_string),
source_path: source_path.map(ToString::to_string),
language: input.language,
output_format: input.output_format,
speaker_labels: input.speaker_labels,
timestamps: input.timestamps,
provider_id: input.provider_id,
model: input.model,
session_id: input.session_id,
project_id: input.project_id,
content_id: input.content_id,
entry_source: input.entry_source,
modality_contract_key: input
.modality_contract_key
.or_else(|| Some(AUDIO_TRANSCRIPTION_CONTRACT_KEY.to_string())),
modality: input
.modality
.or_else(|| Some(AUDIO_TRANSCRIPTION_MODALITY.to_string())),
required_capabilities: if input.required_capabilities.is_empty() {
audio_transcription_required_capabilities()
} else {
input.required_capabilities
},
routing_slot: input
.routing_slot
.or_else(|| Some(AUDIO_TRANSCRIPTION_ROUTING_SLOT.to_string())),
runtime_contract: input.runtime_contract,
requested_target: input
.requested_target
.or_else(|| Some("transcript".to_string())),
output_path: input.output_path,
})
.map_err(ToolError::execution_failed)?;
finalize_transcription_task_creation(
Some(&self.app_handle),
context.working_directory.to_string_lossy().as_ref(),
&output,
);
let serialized = serde_json::to_string_pretty(&output)
.unwrap_or_else(|_| serde_json::json!(&output).to_string());
Ok(media_cli_bridge::attach_media_task_metadata(
ToolResult::success(serialized),
&output,
))
}
}
src-tauri/src/commands/aster_agent_cmd/tool_runtime/media_cli_bridge.rs
+60 -5
View File
@@ -100,6 +100,13 @@ pub(crate) fn emit_media_creation_task_event(app_handle: &AppHandle, output: &Me
"anchor_text": read_payload_string(payload_record, &["anchor_text", "anchorText"]),
"requested_count": read_output_result_u64(output, &["requested_count", "requestedCount"]),
"received_count": read_output_result_u64(output, &["received_count", "receivedCount"]),
"source_text": read_payload_string(payload_record, &["source_text", "sourceText"]),
"voice": read_payload_string(payload_record, &["voice"]),
"voice_style": read_payload_string(payload_record, &["voice_style", "voiceStyle"]),
"target_language": read_payload_string(payload_record, &["target_language", "targetLanguage"]),
"audio_path": read_payload_string(payload_record, &["audio_path", "audioPath"]),
"mime_type": read_payload_string(payload_record, &["mime_type", "mimeType"]),
"duration_ms": read_payload_u64(payload_record, &["duration_ms", "durationMs"]),
});
if let Err(error) = app_handle.emit(CREATION_TASK_EVENT_NAME, &payload) {
@@ -167,14 +174,16 @@ pub(crate) fn attach_media_task_metadata(
)
.with_metadata(
"provider_id",
serde_json::json!(read_output_result_string(
output,
&["provider_id", "providerId"]
)),
serde_json::json!(
read_output_result_string(output, &["provider_id", "providerId"]).or_else(|| {
read_payload_string(&output.record.payload, &["provider_id", "providerId"])
})
),
)
.with_metadata(
"model",
serde_json::json!(read_output_result_string(output, &["model"])),
serde_json::json!(read_output_result_string(output, &["model"])
.or_else(|| read_payload_string(&output.record.payload, &["model"]))),
)
.with_metadata(
"requested_count",
@@ -190,6 +199,52 @@ pub(crate) fn attach_media_task_metadata(
&["received_count", "receivedCount"]
)),
)
.with_metadata(
"source_text",
serde_json::json!(read_payload_string(
&output.record.payload,
&["source_text", "sourceText"]
)),
)
.with_metadata(
"voice",
serde_json::json!(read_payload_string(&output.record.payload, &["voice"])),
)
.with_metadata(
"voice_style",
serde_json::json!(read_payload_string(
&output.record.payload,
&["voice_style", "voiceStyle"]
)),
)
.with_metadata(
"target_language",
serde_json::json!(read_payload_string(
&output.record.payload,
&["target_language", "targetLanguage"]
)),
)
.with_metadata(
"audio_path",
serde_json::json!(read_payload_string(
&output.record.payload,
&["audio_path", "audioPath"]
)),
)
.with_metadata(
"mime_type",
serde_json::json!(read_payload_string(
&output.record.payload,
&["mime_type", "mimeType"]
)),
)
.with_metadata(
"duration_ms",
serde_json::json!(read_payload_u64(
&output.record.payload,
&["duration_ms", "durationMs"]
)),
)
.with_metadata("artifact_paths", serde_json::json!(output.artifact_paths()))
}
src-tauri/src/commands/aster_agent_cmd/transcription_skill_launch.rs
+164
View File
@@ -1,4 +1,9 @@
use super::*;
use crate::commands::modality_runtime_contracts::{
audio_transcription_required_capabilities, audio_transcription_runtime_contract,
insert_audio_transcription_contract_fields, AUDIO_TRANSCRIPTION_CONTRACT_KEY,
AUDIO_TRANSCRIPTION_MODALITY, AUDIO_TRANSCRIPTION_ROUTING_SLOT,
};
const TRANSCRIPTION_SKILL_LAUNCH_PROMPT_MARKER: &str = "<<LIME_TRANSCRIPTION_SKILL_LAUNCH_HINT>>";
const TRANSCRIPTION_SKILL_LAUNCH_DETOUR_DENY_PATTERNS: &[&str] = &[
@@ -25,6 +30,65 @@ fn extract_object_string(
.map(str::to_string)
}
fn extract_harness_nested_object_mut<'a>(
value: &'a mut serde_json::Value,
keys: &[&str],
) -> Option<&'a mut serde_json::Map<String, serde_json::Value>> {
let root = value.as_object_mut()?;
let harness = if root.contains_key("harness") {
root.get_mut("harness")
.and_then(serde_json::Value::as_object_mut)?
} else {
root
};
for key in keys.iter().copied() {
let exists = harness
.get(key)
.and_then(serde_json::Value::as_object)
.is_some();
if exists {
return harness
.get_mut(key)
.and_then(serde_json::Value::as_object_mut);
}
}
None
}
fn ensure_harness_workbench_chat_mode(value: &mut serde_json::Value, launch_keys: &[&str]) {
let Some(root) = value.as_object_mut() else {
return;
};
let harness = if root.contains_key("harness") {
match root
.get_mut("harness")
.and_then(serde_json::Value::as_object_mut)
{
Some(harness) => harness,
None => return,
}
} else {
root
};
let has_launch = launch_keys.iter().any(|key| {
harness
.get(*key)
.and_then(serde_json::Value::as_object)
.is_some()
});
if !has_launch {
return;
}
harness.insert(
"chat_mode".to_string(),
serde_json::Value::String("workbench".to_string()),
);
}
fn truncate_prompt_text(value: String, max_chars: usize) -> String {
let total_chars = value.chars().count();
if total_chars <= max_chars {
@@ -35,6 +99,68 @@ fn truncate_prompt_text(value: String, max_chars: usize) -> String {
format!("{truncated}...(已截断，原始长度 {total_chars} 字)")
}
fn extract_object_string_array(
object: &serde_json::Map<String, serde_json::Value>,
keys: &[&str],
) -> Vec<String> {
keys.iter()
.filter_map(|key| object.get(*key))
.find_map(serde_json::Value::as_array)
.map(|values| {
values
.iter()
.filter_map(serde_json::Value::as_str)
.map(str::trim)
.filter(|value| !value.is_empty())
.map(str::to_string)
.collect::<Vec<_>>()
})
.unwrap_or_default()
}
fn extract_object_value(
object: &serde_json::Map<String, serde_json::Value>,
keys: &[&str],
) -> Option<serde_json::Value> {
keys.iter()
.filter_map(|key| object.get(*key))
.find(|value| !value.is_null())
.cloned()
}
fn ensure_audio_transcription_contract_metadata(
launch: &mut serde_json::Map<String, serde_json::Value>,
) {
insert_audio_transcription_contract_fields(launch);
let transcription_task = if launch.contains_key("transcription_task") {
launch.get_mut("transcription_task")
} else {
launch.get_mut("transcriptionTask")
};
if let Some(transcription_task) = transcription_task.and_then(serde_json::Value::as_object_mut)
{
insert_audio_transcription_contract_fields(transcription_task);
}
}
pub(crate) fn prepare_transcription_skill_launch_request_metadata(
request_metadata: Option<&serde_json::Value>,
) -> Option<serde_json::Value> {
let mut metadata = request_metadata.cloned()?;
ensure_harness_workbench_chat_mode(
&mut metadata,
&["transcription_skill_launch", "transcriptionSkillLaunch"],
);
if let Some(launch) = extract_harness_nested_object_mut(
&mut metadata,
&["transcription_skill_launch", "transcriptionSkillLaunch"],
) {
ensure_audio_transcription_contract_metadata(launch);
}
Some(metadata)
}
pub(crate) fn merge_system_prompt_with_transcription_skill_launch(
base_prompt: Option<String>,
request_metadata: Option<&serde_json::Value>,
@@ -162,6 +288,29 @@ fn build_transcription_skill_launch_system_prompt(
let model = extract_object_string(transcription_task, &["model"]);
let entry_source = extract_object_string(transcription_task, &["entry_source", "entrySource"])
.unwrap_or_else(|| "at_transcription_command".to_string());
let modality_contract_key = extract_object_string(
transcription_task,
&["modality_contract_key", "modalityContractKey"],
)
.unwrap_or_else(|| AUDIO_TRANSCRIPTION_CONTRACT_KEY.to_string());
let modality = extract_object_string(transcription_task, &["modality"])
.unwrap_or_else(|| AUDIO_TRANSCRIPTION_MODALITY.to_string());
let routing_slot = extract_object_string(transcription_task, &["routing_slot", "routingSlot"])
.unwrap_or_else(|| AUDIO_TRANSCRIPTION_ROUTING_SLOT.to_string());
let required_capabilities = {
let values = extract_object_string_array(
transcription_task,
&["required_capabilities", "requiredCapabilities"],
);
if values.is_empty() {
audio_transcription_required_capabilities()
} else {
values
}
};
let runtime_contract =
extract_object_value(transcription_task, &["runtime_contract", "runtimeContract"])
.unwrap_or_else(audio_transcription_runtime_contract);
let args_payload = serde_json::json!({
"user_input": raw_text
.clone()
@@ -196,6 +345,21 @@ fn build_transcription_skill_launch_system_prompt(
"- 如果某个通用搜索/读文件工具因为 session policy 被拒绝，不要重复同类调用；应立即改为直调 Skill(transcription_generate)。".to_string(),
"- Skill 执行后，优先沿 transcription_generate skill 的 Bash / task file 主链提交异步任务；只有 Skill 明确不可用时，才允许直接回退到 lime_create_transcription_task。".to_string(),
"- 不要伪造“转写已完成”；在 task file 真正返回结果前，只能汇报任务已提交、排队或执行中。".to_string(),
format!(
"- 当前底层运行合同：modality_contract_key={modality_contract_key}, modality={modality}, routing_slot={routing_slot}；`@转写` 只是 audio_transcription 的上层入口，首刀仍走 Skill(transcription_generate)。"
),
format!(
"- 当前合同所需能力：{}；不得退回 frontend_direct_asr、generic_file_transcript、tool_search_before_transcription_skill 或 web_search_before_transcription_skill。",
required_capabilities.join(", ")
),
format!(
"- 当前 runtime_contract(JSON)：{}",
truncate_prompt_text(
serde_json::to_string(&runtime_contract)
.unwrap_or_else(|_| "{}".to_string()),
4_000,
)
),
format!("- 当前转写任务上下文(JSON)：{transcription_task_json}"),
format!("- 当前入口来源：{entry_source}。"),
];
src-tauri/src/commands/media_task_cmd.rs
+3325 -8
View File
File diff suppressed because it is too large Load Diff
src-tauri/src/commands/mod.rs
+1
View File
@@ -58,6 +58,7 @@ pub mod unified_memory_cmd;
pub mod update_cmd;
pub mod usage_stats_cmd;
pub mod video_generation_cmd;
pub mod voice_model_cmd;
pub mod voice_test_cmd;
pub mod websocket_cmd;
pub mod webview_cmd;
src-tauri/src/commands/modality_runtime_contracts.rs
+115
View File
@@ -28,6 +28,12 @@ pub(crate) const VOICE_GENERATION_ROUTING_SLOT: &str = "voice_generation_model";
pub(crate) const VOICE_GENERATION_EXECUTOR_BINDING_KEY: &str = "voice_runtime";
pub(crate) const VOICE_GENERATION_REQUIRED_CAPABILITIES: &[&str] =
&["text_generation", "voice_generation"];
pub(crate) const AUDIO_TRANSCRIPTION_CONTRACT_KEY: &str = "audio_transcription";
pub(crate) const AUDIO_TRANSCRIPTION_MODALITY: &str = "audio";
pub(crate) const AUDIO_TRANSCRIPTION_ROUTING_SLOT: &str = "audio_transcription_model";
pub(crate) const AUDIO_TRANSCRIPTION_EXECUTOR_BINDING_KEY: &str = "transcription_generate";
pub(crate) const AUDIO_TRANSCRIPTION_REQUIRED_CAPABILITIES: &[&str] =
&["text_generation", "audio_transcription"];
pub(crate) const WEB_RESEARCH_CONTRACT_KEY: &str = "web_research";
pub(crate) const WEB_RESEARCH_MODALITY: &str = "mixed";
pub(crate) const WEB_RESEARCH_ROUTING_SLOT: &str = "report_generation_model";
@@ -144,6 +150,13 @@ pub(crate) fn voice_generation_required_capabilities() -> Vec<String> {
.collect()
}
pub(crate) fn audio_transcription_required_capabilities() -> Vec<String> {
AUDIO_TRANSCRIPTION_REQUIRED_CAPABILITIES
.iter()
.map(|value| (*value).to_string())
.collect()
}
pub(crate) fn web_research_required_capabilities() -> Vec<String> {
WEB_RESEARCH_REQUIRED_CAPABILITIES
.iter()
@@ -256,6 +269,68 @@ pub(crate) fn normalize_voice_generation_required_capabilities(
Ok(expected)
}
fn normalize_expected_audio_transcription_contract_string(
value: Option<String>,
expected: &str,
field_name: &str,
) -> Result<String, String> {
match normalize_optional_contract_string(value) {
Some(value) if value == expected => Ok(expected.to_string()),
Some(value) => Err(format!(
"转写任务 {field_name} 必须是 {expected}，收到 {value}"
)),
None => Ok(expected.to_string()),
}
}
pub(crate) fn normalize_audio_transcription_contract_key(
value: Option<String>,
) -> Result<String, String> {
normalize_expected_audio_transcription_contract_string(
value,
AUDIO_TRANSCRIPTION_CONTRACT_KEY,
"modality_contract_key",
)
}
pub(crate) fn normalize_audio_transcription_modality(
value: Option<String>,
) -> Result<String, String> {
normalize_expected_audio_transcription_contract_string(
value,
AUDIO_TRANSCRIPTION_MODALITY,
"modality",
)
}
pub(crate) fn normalize_audio_transcription_routing_slot(
value: Option<String>,
) -> Result<String, String> {
normalize_expected_audio_transcription_contract_string(
value,
AUDIO_TRANSCRIPTION_ROUTING_SLOT,
"routing_slot",
)
}
pub(crate) fn normalize_audio_transcription_required_capabilities(
values: Vec<String>,
) -> Result<Vec<String>, String> {
let expected = audio_transcription_required_capabilities();
for value in values {
let normalized = value.trim();
if normalized.is_empty() {
continue;
}
if !expected.iter().any(|item| item == normalized) {
return Err(format!(
"转写任务 required_capabilities 包含不属于 audio_transcription contract 的能力: {normalized}"
));
}
}
Ok(expected)
}
pub(crate) fn image_generation_runtime_contract() -> Value {
json!({
"contract_key": IMAGE_GENERATION_CONTRACT_KEY,
@@ -324,6 +399,23 @@ pub(crate) fn voice_generation_runtime_contract() -> Value {
})
}
pub(crate) fn audio_transcription_runtime_contract() -> Value {
json!({
"contract_key": AUDIO_TRANSCRIPTION_CONTRACT_KEY,
"modality": AUDIO_TRANSCRIPTION_MODALITY,
"required_capabilities": AUDIO_TRANSCRIPTION_REQUIRED_CAPABILITIES,
"routing_slot": AUDIO_TRANSCRIPTION_ROUTING_SLOT,
"executor_binding": {
"executor_kind": "skill",
"binding_key": AUDIO_TRANSCRIPTION_EXECUTOR_BINDING_KEY
},
"truth_source": ["transcript_artifact", "runtime_timeline_event"],
"artifact_kinds": ["transcript"],
"viewer_surface": ["transcript_viewer", "document_viewer"],
"owner_surface": "agent_runtime"
})
}
pub(crate) fn web_research_runtime_contract() -> Value {
json!({
"contract_key": WEB_RESEARCH_CONTRACT_KEY,
@@ -506,6 +598,29 @@ pub(crate) fn insert_voice_generation_contract_fields(record: &mut Map<String, V
);
}
pub(crate) fn insert_audio_transcription_contract_fields(record: &mut Map<String, Value>) {
record.insert(
"modality_contract_key".to_string(),
Value::String(AUDIO_TRANSCRIPTION_CONTRACT_KEY.to_string()),
);
record.insert(
"modality".to_string(),
Value::String(AUDIO_TRANSCRIPTION_MODALITY.to_string()),
);
record.insert(
"required_capabilities".to_string(),
json!(AUDIO_TRANSCRIPTION_REQUIRED_CAPABILITIES),
);
record.insert(
"routing_slot".to_string(),
Value::String(AUDIO_TRANSCRIPTION_ROUTING_SLOT.to_string()),
);
record.insert(
"runtime_contract".to_string(),
audio_transcription_runtime_contract(),
);
}
pub(crate) fn insert_web_research_contract_fields(record: &mut Map<String, Value>) {
record.insert(
"modality_contract_key".to_string(),
src-tauri/src/commands/voice_model_cmd.rs
+1075
View File
File diff suppressed because it is too large Load Diff
src-tauri/src/dev_bridge.rs
+3
View File
@@ -267,6 +267,9 @@ async fn stream_events(
listener_id,
};
// 先刷新一个 SSE 注释，避免浏览器 EventSource 在首个业务事件到达前一直不触发 open。
yield Ok::<SseEvent, Infallible>(SseEvent::default().comment("ready"));
while let Some(payload) = rx.recv().await {
yield Ok::<SseEvent, Infallible>(SseEvent::default().data(payload));
}
src-tauri/src/dev_bridge/dispatcher.rs
+81
View File
@@ -832,6 +832,87 @@ mod tests {
assert!(error.to_string().contains("模型注册服务未初始化"));
}
#[tokio::test]
async fn api_key_provider_write_flow_is_bridged() {
let state = make_test_state();
let created = handle_command(
&state,
"add_custom_api_key_provider",
Some(serde_json::json!({
"request": {
"name": "SenseNova E2E",
"type": "openai",
"api_host": "https://api.sensenova.cn/compatible-mode/v2"
}
})),
)
.await
.expect("custom provider should be created through bridge");
let provider_id = created["id"].as_str().unwrap().to_string();
handle_command(
&state,
"update_api_key_provider",
Some(serde_json::json!({
"id": provider_id,
"request": {
"enabled": true,
"custom_models": ["SenseChat-5"]
}
})),
)
.await
.expect("custom provider should be updated through bridge");
let key = handle_command(
&state,
"add_api_key",
Some(serde_json::json!({
"request": {
"provider_id": provider_id,
"api_key": "sk-bridge-test",
"alias": "bridge"
}
})),
)
.await
.expect("api key should be added through bridge");
assert_eq!(key["provider_id"], created["id"]);
assert_ne!(key["api_key_masked"], "sk-bridge-test");
let provider = handle_command(
&state,
"get_api_key_provider",
Some(serde_json::json!({
"id": created["id"].as_str().unwrap()
})),
)
.await
.expect("custom provider should be readable through bridge");
assert_eq!(provider["id"], created["id"]);
assert_eq!(
provider["api_host"],
"https://api.sensenova.cn/compatible-mode/v2"
);
assert_eq!(provider["custom_models"][0], "SenseChat-5");
assert_eq!(provider["api_keys"].as_array().unwrap().len(), 1);
let deleted = handle_command(
&state,
"delete_custom_api_key_provider",
Some(serde_json::json!({
"id": created["id"].as_str().unwrap()
})),
)
.await
.expect("custom provider should be deleted through bridge");
assert_eq!(deleted, serde_json::json!(true));
}
#[tokio::test]
async fn test_api_key_provider_connection_is_bridged() {
let state = make_test_state();
src-tauri/src/dev_bridge/dispatcher/providers.rs
+183 -44
View File
@@ -1,5 +1,8 @@
use super::{args_or_default, get_string_arg, parse_nested_arg, require_app_handle};
use crate::connect::RelayRegistry;
use crate::database::dao::api_key_provider::{
ApiKeyEntry, ApiKeyProvider, ApiProviderPromptCacheMode, ApiProviderType,
};
use crate::dev_bridge::DevBridgeState;
use serde_json::Value as JsonValue;
use std::sync::Arc;
@@ -22,59 +25,67 @@ fn api_key_provider_with_keys_to_display(
provider_with_keys: &crate::database::dao::api_key_provider::ProviderWithKeys,
service: &lime_services::api_key_provider_service::ApiKeyProviderService,
) -> crate::commands::api_key_provider_cmd::ProviderWithKeysDisplay {
let api_keys = provider_with_keys
let api_keys: Vec<crate::commands::api_key_provider_cmd::ApiKeyDisplay> = provider_with_keys
.api_keys
.iter()
.map(|key| {
let masked = match service.decrypt_api_key(&key.api_key_encrypted) {
Ok(decrypted) => mask_api_key_for_display(&decrypted),
Err(_) => "****".to_string(),
};
crate::commands::api_key_provider_cmd::ApiKeyDisplay {
id: key.id.clone(),
provider_id: key.provider_id.clone(),
api_key_masked: masked,
alias: key.alias.clone(),
enabled: key.enabled,
usage_count: key.usage_count,
error_count: key.error_count,
last_used_at: key.last_used_at.map(|value| value.to_rfc3339()),
created_at: key.created_at.to_rfc3339(),
}
})
.map(|key| api_key_to_display(key, service))
.collect();
crate::commands::api_key_provider_cmd::ProviderWithKeysDisplay {
provider: crate::commands::api_key_provider_cmd::ProviderDisplay {
id: provider_with_keys.provider.id.clone(),
name: provider_with_keys.provider.name.clone(),
provider_type: provider_with_keys
.provider
.effective_provider_type()
.to_string(),
api_host: provider_with_keys.provider.api_host.clone(),
is_system: provider_with_keys.provider.is_system,
group: provider_with_keys.provider.group.to_string(),
enabled: provider_with_keys.provider.enabled,
sort_order: provider_with_keys.provider.sort_order,
api_version: provider_with_keys.provider.api_version.clone(),
project: provider_with_keys.provider.project.clone(),
location: provider_with_keys.provider.location.clone(),
region: provider_with_keys.provider.region.clone(),
custom_models: provider_with_keys.provider.custom_models.clone(),
prompt_cache_mode: provider_with_keys
.provider
.effective_prompt_cache_mode()
.map(|mode| mode.to_string()),
api_key_count: provider_with_keys.api_keys.len(),
created_at: provider_with_keys.provider.created_at.to_rfc3339(),
updated_at: provider_with_keys.provider.updated_at.to_rfc3339(),
},
provider: api_key_provider_to_display(&provider_with_keys.provider, api_keys.len()),
api_keys,
}
}
fn api_key_provider_to_display(
provider: &ApiKeyProvider,
api_key_count: usize,
) -> crate::commands::api_key_provider_cmd::ProviderDisplay {
crate::commands::api_key_provider_cmd::ProviderDisplay {
id: provider.id.clone(),
name: provider.name.clone(),
provider_type: provider.effective_provider_type().to_string(),
api_host: provider.api_host.clone(),
is_system: provider.is_system,
group: provider.group.to_string(),
enabled: provider.enabled,
sort_order: provider.sort_order,
api_version: provider.api_version.clone(),
project: provider.project.clone(),
location: provider.location.clone(),
region: provider.region.clone(),
custom_models: provider.custom_models.clone(),
prompt_cache_mode: provider
.effective_prompt_cache_mode()
.map(|mode| mode.to_string()),
api_key_count,
created_at: provider.created_at.to_rfc3339(),
updated_at: provider.updated_at.to_rfc3339(),
}
}
fn api_key_to_display(
key: &ApiKeyEntry,
service: &lime_services::api_key_provider_service::ApiKeyProviderService,
) -> crate::commands::api_key_provider_cmd::ApiKeyDisplay {
let masked = match service.decrypt_api_key(&key.api_key_encrypted) {
Ok(decrypted) => mask_api_key_for_display(&decrypted),
Err(_) => "****".to_string(),
};
crate::commands::api_key_provider_cmd::ApiKeyDisplay {
id: key.id.clone(),
provider_id: key.provider_id.clone(),
api_key_masked: masked,
alias: key.alias.clone(),
enabled: key.enabled,
usage_count: key.usage_count,
error_count: key.error_count,
last_used_at: key.last_used_at.map(|value| value.to_rfc3339()),
created_at: key.created_at.to_rfc3339(),
}
}
async fn relay_registry(state: &DevBridgeState) -> Option<Arc<RelayRegistry>> {
let state_guard = state.connect_state.read().await;
state_guard
@@ -161,6 +172,134 @@ pub(super) async fn try_handle(
serde_json::json!([])
}
}
"get_api_key_provider" => {
let args = args_or_default(args);
let id = get_string_arg(&args, "id", "providerId")?;
let db = state
.db
.as_ref()
.ok_or_else(|| "Database not initialized".to_string())?;
let provider = state.api_key_provider_service.get_provider(db, &id)?;
serde_json::to_value(provider.map(|provider| {
api_key_provider_with_keys_to_display(
&provider,
state.api_key_provider_service.as_ref(),
)
}))?
}
"add_custom_api_key_provider" => {
let args = args_or_default(args);
let request = parse_nested_arg::<
crate::commands::api_key_provider_cmd::AddCustomProviderRequest,
>(&args, "request")?;
let provider_type: ApiProviderType = request
.provider_type
.parse()
.map_err(|e: String| format!("无效的 Provider 类型: {e}"))?;
let prompt_cache_mode = request
.prompt_cache_mode
.map(|mode| mode.parse::<ApiProviderPromptCacheMode>())
.transpose()
.map_err(|e: String| format!("无效的 Prompt Cache 模式: {e}"))?;
let db = state
.db
.as_ref()
.ok_or_else(|| "Database not initialized".to_string())?;
let provider = state.api_key_provider_service.add_custom_provider(
db,
request.name,
provider_type,
request.api_host,
request.api_version,
request.project,
request.location,
request.region,
prompt_cache_mode,
)?;
serde_json::to_value(api_key_provider_to_display(&provider, 0))?
}
"update_api_key_provider" => {
let args = args_or_default(args);
let id = get_string_arg(&args, "id", "providerId")?;
let request = parse_nested_arg::<
crate::commands::api_key_provider_cmd::UpdateProviderRequest,
>(&args, "request")?;
let provider_type: Option<ApiProviderType> = request
.provider_type
.map(|value| value.parse())
.transpose()
.map_err(|e: String| format!("无效的 Provider 类型: {e}"))?;
let prompt_cache_mode = request
.prompt_cache_mode
.map(|mode| mode.parse::<ApiProviderPromptCacheMode>())
.transpose()
.map_err(|e: String| format!("无效的 Prompt Cache 模式: {e}"))?;
let db = state
.db
.as_ref()
.ok_or_else(|| "Database not initialized".to_string())?;
let provider = state.api_key_provider_service.update_provider(
db,
&id,
request.name,
provider_type,
request.api_host,
request.enabled,
request.sort_order,
request.api_version,
request.project,
request.location,
request.region,
prompt_cache_mode,
request.custom_models,
)?;
let api_key_count = state
.api_key_provider_service
.get_provider(db, &id)?
.map(|provider| provider.api_keys.len())
.unwrap_or(0);
serde_json::to_value(api_key_provider_to_display(&provider, api_key_count))?
}
"add_api_key" => {
let args = args_or_default(args);
let request = parse_nested_arg::<
crate::commands::api_key_provider_cmd::AddApiKeyRequest,
>(&args, "request")?;
let db = state
.db
.as_ref()
.ok_or_else(|| "Database not initialized".to_string())?;
let key = state.api_key_provider_service.add_api_key(
db,
&request.provider_id,
&request.api_key,
request.alias,
)?;
serde_json::to_value(api_key_to_display(
&key,
state.api_key_provider_service.as_ref(),
))?
}
"delete_custom_api_key_provider" => {
let args = args_or_default(args);
let id = get_string_arg(&args, "id", "providerId")?;
let db = state
.db
.as_ref()
.ok_or_else(|| "Database not initialized".to_string())?;
serde_json::to_value(
state
.api_key_provider_service
.delete_custom_provider(db, &id)?,
)?
}
"get_system_provider_catalog" => {
let catalog = crate::commands::api_key_provider_cmd::get_system_provider_catalog()
.map_err(|e| format!("获取系统 Provider Catalog 失败: {e}"))?;
src-tauri/src/dev_bridge/dispatcher/voice.rs
+49 -1
View File
@@ -1,4 +1,7 @@
use super::{args_or_default, get_string_arg, parse_nested_arg, require_app_handle};
use super::{
args_or_default, get_string_arg, parse_nested_arg, parse_optional_nested_arg,
require_app_handle,
};
use crate::commands::asr_cmd::AddAsrCredentialRequest;
use crate::config::AsrCredentialEntry;
use crate::dev_bridge::DevBridgeState;
@@ -64,6 +67,51 @@ pub(super) async fn try_handle(
let id = get_string_arg(&args, "id", "id")?;
serde_json::to_value(crate::commands::asr_cmd::test_asr_credential(id).await?)?
}
"voice_models_list_catalog" => serde_json::to_value(
crate::commands::voice_model_cmd::voice_models_list_catalog().await?,
)?,
"voice_models_get_install_state" => {
let args = args_or_default(args);
let model_id = get_string_arg(&args, "modelId", "model_id")?;
serde_json::to_value(
crate::commands::voice_model_cmd::voice_models_get_install_state(model_id).await?,
)?
}
"voice_models_download" => {
let args = args_or_default(args);
let model_id = get_string_arg(&args, "modelId", "model_id")?;
let catalog_entry = parse_optional_nested_arg(&args, "catalogEntry")?
.or(parse_optional_nested_arg(&args, "catalog_entry")?);
serde_json::to_value(
crate::commands::voice_model_cmd::voice_models_download(model_id, catalog_entry)
.await?,
)?
}
"voice_models_delete" => {
let args = args_or_default(args);
let model_id = get_string_arg(&args, "modelId", "model_id")?;
serde_json::to_value(
crate::commands::voice_model_cmd::voice_models_delete(model_id).await?,
)?
}
"voice_models_set_default" => {
let args = args_or_default(args);
let model_id = get_string_arg(&args, "modelId", "model_id")?;
serde_json::to_value(
crate::commands::voice_model_cmd::voice_models_set_default(model_id).await?,
)?
}
"voice_models_test_transcribe_file" => {
let args = args_or_default(args);
let model_id = get_string_arg(&args, "modelId", "model_id")?;
let file_path = get_string_arg(&args, "filePath", "file_path")?;
serde_json::to_value(
crate::commands::voice_model_cmd::voice_models_test_transcribe_file(
model_id, file_path,
)
.await?,
)?
}
"get_voice_input_config" => {
serde_json::to_value(crate::voice::commands::get_voice_input_config().await?)?
}
src-tauri/src/services/runtime_evidence_pack_service.rs
+1215 -20
View File
File diff suppressed because it is too large Load Diff
src-tauri/src/services/runtime_replay_case_service.rs
+582 -14
View File
@@ -7,8 +7,8 @@
use crate::agent::SessionDetail;
use crate::commands::aster_agent_cmd::AgentRuntimeThreadReadModel;
use crate::commands::modality_runtime_contracts::{
BROWSER_CONTROL_CONTRACT_KEY, PDF_EXTRACT_CONTRACT_KEY, TEXT_TRANSFORM_CONTRACT_KEY,
VOICE_GENERATION_CONTRACT_KEY, WEB_RESEARCH_CONTRACT_KEY,
AUDIO_TRANSCRIPTION_CONTRACT_KEY, BROWSER_CONTROL_CONTRACT_KEY, PDF_EXTRACT_CONTRACT_KEY,
TEXT_TRANSFORM_CONTRACT_KEY, VOICE_GENERATION_CONTRACT_KEY, WEB_RESEARCH_CONTRACT_KEY,
};
use crate::services::runtime_evidence_pack_service::{
export_runtime_evidence_pack, RuntimeEvidencePackExportResult,
@@ -694,6 +694,10 @@ fn build_success_criteria(
"浏览器控制任务必须继续走 Browser Assist / `mcp__lime-browser__*` 主链，不能用 WebSearch 或普通聊天替代真实浏览器动作。"
.to_string(),
);
criteria.push(
"Browser Assist 证据必须能在 `snapshotIndex.browserActionIndex` 中定位 action、session、URL 或 observation 摘要，不能只靠人工扫描 raw snapshots。"
.to_string(),
);
}
if modality_contract_has_pdf_extract(modality_runtime_contracts) {
criteria.push(
@@ -706,6 +710,28 @@ fn build_success_criteria(
"配音任务必须继续走 `service_scene_launch(scene_key=voice_runtime)` / 本地 ServiceSkill runtime 主链，不能用 `legacy_tts_test_command`、伪造云端已提交、普通聊天文本或通用文件卡替代真实语音生成合同。"
.to_string(),
);
if modality_contract_has_voice_audio_output_completed(modality_runtime_contracts) {
criteria.push(
"如果 replay 沿用已有音频任务，必须保留 `audio_output.completed` 与真实 `audio_path`，不能退回只展示文案或通用文件卡。"
.to_string(),
);
}
}
if modality_contract_has_audio_transcription(modality_runtime_contracts) {
criteria.push(
"转写任务必须继续走 `Skill(transcription_generate)` / `transcription_generate` task file 主链，不能用 `frontend_direct_asr`、`generic_file_transcript`、普通文件读取或普通聊天伪造 transcript。"
.to_string(),
);
criteria.push(
"转写证据必须能在 `snapshotIndex.transcriptIndex` 中定位 transcript 状态、来源、语言或输出格式，不能只靠人工扫描 raw task JSON。"
.to_string(),
);
if modality_contract_has_transcript_completed(modality_runtime_contracts) {
criteria.push(
"如果 replay 沿用已有转写结果，必须保留 `transcript.completed` 与真实 `transcriptPath`，不能退回只展示 Markdown 文本。"
.to_string(),
);
}
}
if modality_contract_has_web_research(modality_runtime_contracts) {
criteria.push(
@@ -778,6 +804,14 @@ fn build_blocking_checks(
.to_string(),
);
}
if modality_contract_has_browser_control(modality_runtime_contracts)
&& !modality_contract_has_browser_action_index(modality_runtime_contracts)
{
checks.push(
"`browser_control` 合同缺少 `snapshotIndex.browserActionIndex`；除非 replay 重新导出可查询的 browser session/snapshot 索引，否则不能判 PASS。"
.to_string(),
);
}
if modality_contract_has_pdf_extract(modality_runtime_contracts)
&& !modality_contract_has_pdf_skill_trace(modality_runtime_contracts)
{
@@ -794,6 +828,40 @@ fn build_blocking_checks(
.to_string(),
);
}
if modality_contract_has_voice_audio_output_failed(modality_runtime_contracts) {
checks.push(format!(
"`voice_generation` 的 audio_output 已失败：{}；除非 replay 修复 provider / model / API Key 并产出新的 audio_output.completed，否则不能判 PASS。",
format_text_list(
&modality_contract_voice_audio_output_error_codes(modality_runtime_contracts),
"未记录 audio_output errorCode"
)
));
}
if modality_contract_has_audio_transcription(modality_runtime_contracts)
&& !modality_contract_has_audio_transcription_task_trace(modality_runtime_contracts)
{
checks.push(
"`audio_transcription` 合同缺少 transcription_generate task/Skill trace；除非 replay 重新产生 Skill(transcription_generate) 或 transcription task artifact 证据，否则不能判 PASS。"
.to_string(),
);
}
if modality_contract_has_audio_transcription(modality_runtime_contracts)
&& !modality_contract_has_transcript_index(modality_runtime_contracts)
{
checks.push(
"`audio_transcription` 合同缺少 `snapshotIndex.transcriptIndex`；除非 replay 重新导出可查询的 transcript 索引，否则不能判 PASS。"
.to_string(),
);
}
if modality_contract_has_transcript_failed(modality_runtime_contracts) {
checks.push(format!(
"`audio_transcription` 的 transcript 已失败：{}；除非 replay 修复 provider / model / 输入源并产出新的 transcript.completed，否则不能判 PASS。",
format_text_list(
&modality_contract_transcript_error_codes(modality_runtime_contracts),
"未记录 transcript errorCode"
)
));
}
if modality_contract_has_web_research(modality_runtime_contracts)
&& !modality_contract_has_web_research_skill_trace(modality_runtime_contracts)
{
@@ -867,6 +935,10 @@ fn build_modality_contract_checks(modality_runtime_contracts: &Value) -> Vec<Str
"确认 evidence 中存在 `browser_action_trace` 或 `browser_action_requested` 快照，能证明浏览器执行器真实被调用。"
.to_string(),
);
checks.push(
"确认 `snapshotIndex.browserActionIndex` 已汇总 browser session / snapshot 的 action、session、URL、observation 与 screenshot 计数。"
.to_string(),
);
}
if modality_contract_has_pdf_extract(modality_runtime_contracts) {
checks.push(
@@ -883,11 +955,55 @@ fn build_modality_contract_checks(modality_runtime_contracts: &Value) -> Vec<Str
"确认 `voice_generation` replay 使用 `service_scene_launch(scene_key=voice_runtime)` / 本地 ServiceSkill runtime，并保留 voice_runtime service scene trace 或 audio_task/audio_output 产物证据。"
.to_string(),
);
if modality_contract_has_voice_audio_output_completed(modality_runtime_contracts) {
checks.push(
"确认 `audio_generate` task artifact 中的 `audio_output.completed`、`audio_path`、provider / model 仍能从 evidence 回溯，不能只把音频结果降级为 Markdown 文本。"
.to_string(),
);
}
if modality_contract_has_voice_audio_output_failed(modality_runtime_contracts) {
checks.push(format!(
"确认 replay 对 `audio_output.failed` 的处理仍保留 Provider 错误码：{}，不能静默回退 legacy TTS 或伪造音频路径。",
format_text_list(
&modality_contract_voice_audio_output_error_codes(modality_runtime_contracts),
"未记录 audio_output errorCode"
)
));
}
checks.push(
"确认没有把配音降级为 `legacy_tts_test_command`、`fake_cloud_scene_submitted`、普通聊天文案、通用文件卡或只展示文本结果。"
.to_string(),
);
}
if modality_contract_has_audio_transcription(modality_runtime_contracts) {
checks.push(
"确认 `audio_transcription` replay 使用 `Skill(transcription_generate)`，并保留 transcription_generate task artifact / transcript 产物证据。"
.to_string(),
);
checks.push(
"确认 `snapshotIndex.transcriptIndex` 已汇总 transcript 状态、来源、语言、输出格式与 provider / model。"
.to_string(),
);
if modality_contract_has_transcript_completed(modality_runtime_contracts) {
checks.push(
"确认 `transcription_generate` task artifact 中的 `transcript.completed`、`transcriptPath`、provider / model 仍能从 evidence 回溯，不能只把转写结果降级为 Markdown 文本。"
.to_string(),
);
}
if modality_contract_has_transcript_failed(modality_runtime_contracts) {
checks.push(format!(
"确认 replay 对 `transcript.failed` 的处理仍保留 Provider 错误码：{}，不能静默回退 frontend ASR 或伪造 transcriptPath。",
format_text_list(
&modality_contract_transcript_error_codes(modality_runtime_contracts),
"未记录 transcript errorCode"
)
));
}
checks.push(
"确认没有把转写降级为 `frontend_direct_asr`、`generic_file_transcript`、`tool_search_before_transcription_skill`、普通文件读取或普通聊天摘要。"
.to_string(),
);
}
if modality_contract_has_web_research(modality_runtime_contracts) {
checks.push(
"确认 `web_research` replay 使用 `Skill(research)`、`Skill(site_search)` 或 `Skill(report_generate)`，并保留真实 search_query / lime_site_* 工具时间线。"
@@ -981,6 +1097,9 @@ fn infer_replay_suite_tags(
if modality_contract_has_browser_action_trace(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "browser-action-trace");
}
if modality_contract_has_browser_action_index(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "browser-action-index");
}
}
if modality_contract_has_pdf_extract(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "pdf-extract");
@@ -995,6 +1114,28 @@ fn infer_replay_suite_tags(
if modality_contract_has_voice_generation_service_trace(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "voice-generation-trace");
}
if modality_contract_has_voice_audio_output_completed(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "audio-output-completed");
}
if modality_contract_has_voice_audio_output_failed(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "audio-output-failed");
}
}
if modality_contract_has_audio_transcription(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "audio-transcription");
push_unique_text_tag(&mut tags, "transcription-generate");
if modality_contract_has_audio_transcription_task_trace(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "transcription-task-trace");
}
if modality_contract_has_transcript_index(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "transcript-index");
}
if modality_contract_has_transcript_completed(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "transcript-completed");
}
if modality_contract_has_transcript_failed(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "transcript-failed");
}
}
if modality_contract_has_web_research(modality_runtime_contracts) {
push_unique_text_tag(&mut tags, "web-research");
@@ -1081,6 +1222,11 @@ fn infer_replay_failure_modes(
{
push_unique_text_tag(&mut failure_modes, "browser_control_missing_action_trace");
}
if modality_contract_has_browser_control(modality_runtime_contracts)
&& !modality_contract_has_browser_action_index(modality_runtime_contracts)
{
push_unique_text_tag(&mut failure_modes, "browser_control_missing_action_index");
}
if modality_contract_has_pdf_extract(modality_runtime_contracts)
&& !modality_contract_has_pdf_skill_trace(modality_runtime_contracts)
{
@@ -1091,6 +1237,33 @@ fn infer_replay_failure_modes(
{
push_unique_text_tag(&mut failure_modes, "voice_generation_missing_service_trace");
}
if modality_contract_has_voice_audio_output_failed(modality_runtime_contracts) {
push_unique_text_tag(&mut failure_modes, "voice_generation_audio_output_failed");
for error_code in
modality_contract_voice_audio_output_error_codes(modality_runtime_contracts)
{
push_unique_owned_tag(&mut failure_modes, error_code);
}
}
if modality_contract_has_audio_transcription(modality_runtime_contracts)
&& !modality_contract_has_audio_transcription_task_trace(modality_runtime_contracts)
{
push_unique_text_tag(&mut failure_modes, "audio_transcription_missing_task_trace");
}
if modality_contract_has_audio_transcription(modality_runtime_contracts)
&& !modality_contract_has_transcript_index(modality_runtime_contracts)
{
push_unique_text_tag(
&mut failure_modes,
"audio_transcription_missing_transcript_index",
);
}
if modality_contract_has_transcript_failed(modality_runtime_contracts) {
push_unique_text_tag(&mut failure_modes, "audio_transcription_transcript_failed");
for error_code in modality_contract_transcript_error_codes(modality_runtime_contracts) {
push_unique_owned_tag(&mut failure_modes, error_code);
}
}
if modality_contract_has_web_research(modality_runtime_contracts)
&& !modality_contract_has_web_research_skill_trace(modality_runtime_contracts)
{
@@ -1165,6 +1338,17 @@ fn modality_contract_has_browser_action_trace(modality_runtime_contracts: &Value
})
}
fn modality_contract_has_browser_action_index(modality_runtime_contracts: &Value) -> bool {
modality_runtime_contracts
.pointer("/snapshotIndex/browserActionIndex/actionCount")
.and_then(Value::as_u64)
.is_some_and(|count| count > 0)
|| modality_runtime_contracts
.pointer("/snapshotIndex/browserActionIndex/items")
.and_then(Value::as_array)
.is_some_and(|items| !items.is_empty())
}
fn modality_contract_has_pdf_extract(modality_runtime_contracts: &Value) -> bool {
modality_contract_snapshots(modality_runtime_contracts)
.iter()
@@ -1241,6 +1425,191 @@ fn modality_contract_has_voice_generation_service_trace(
})
}
fn modality_contract_voice_audio_outputs(modality_runtime_contracts: &Value) -> Vec<&Value> {
modality_contract_snapshots(modality_runtime_contracts)
.into_iter()
.filter(|snapshot| {
snapshot
.get("contractKey")
.and_then(Value::as_str)
.map(|value| value == VOICE_GENERATION_CONTRACT_KEY)
.unwrap_or(false)
})
.filter_map(|snapshot| snapshot.get("audioOutput"))
.filter(|value| value.is_object())
.collect()
}
fn modality_contract_has_voice_audio_output_completed(modality_runtime_contracts: &Value) -> bool {
modality_contract_voice_audio_outputs(modality_runtime_contracts)
.iter()
.any(|audio_output| {
let completed = audio_output
.get("status")
.and_then(Value::as_str)
.map(|value| value == "completed")
.unwrap_or(false);
let has_audio_path = audio_output
.get("audioPath")
.and_then(Value::as_str)
.and_then(|value| normalize_optional_text(Some(value.to_string())))
.is_some();
completed && has_audio_path
})
}
fn modality_contract_has_voice_audio_output_failed(modality_runtime_contracts: &Value) -> bool {
modality_contract_voice_audio_outputs(modality_runtime_contracts)
.iter()
.any(|audio_output| {
audio_output
.get("status")
.and_then(Value::as_str)
.map(|value| value == "failed")
.unwrap_or(false)
|| audio_output
.get("errorCode")
.and_then(Value::as_str)
.and_then(|value| normalize_optional_text(Some(value.to_string())))
.is_some()
})
}
fn modality_contract_voice_audio_output_error_codes(
modality_runtime_contracts: &Value,
) -> Vec<String> {
let mut error_codes = Vec::new();
for audio_output in modality_contract_voice_audio_outputs(modality_runtime_contracts) {
if let Some(error_code) = audio_output
.get("errorCode")
.and_then(Value::as_str)
.and_then(|value| normalize_optional_text(Some(value.to_string())))
{
push_unique_owned_tag(&mut error_codes, error_code);
}
}
error_codes
}
fn modality_contract_has_audio_transcription(modality_runtime_contracts: &Value) -> bool {
modality_contract_snapshots(modality_runtime_contracts)
.iter()
.any(|snapshot| {
snapshot
.get("contractKey")
.and_then(Value::as_str)
.map(|value| value == AUDIO_TRANSCRIPTION_CONTRACT_KEY)
.unwrap_or(false)
})
}
fn modality_contract_has_audio_transcription_task_trace(
modality_runtime_contracts: &Value,
) -> bool {
modality_contract_snapshots(modality_runtime_contracts)
.iter()
.any(|snapshot| {
let is_audio_transcription = snapshot
.get("contractKey")
.and_then(Value::as_str)
.map(|value| value == AUDIO_TRANSCRIPTION_CONTRACT_KEY)
.unwrap_or(false);
if !is_audio_transcription {
return false;
}
snapshot
.get("source")
.and_then(Value::as_str)
.map(|value| {
value.contains("transcription_task")
|| value.contains("transcription_skill_trace")
|| value.contains("transcript")
})
.unwrap_or(false)
|| snapshot
.get("routingEvent")
.and_then(Value::as_str)
.map(|value| value == "executor_invoked")
.unwrap_or(false)
})
}
fn modality_contract_has_transcript_index(modality_runtime_contracts: &Value) -> bool {
modality_runtime_contracts
.pointer("/snapshotIndex/transcriptIndex/transcriptCount")
.and_then(Value::as_u64)
.is_some_and(|count| count > 0)
|| modality_runtime_contracts
.pointer("/snapshotIndex/transcriptIndex/items")
.and_then(Value::as_array)
.is_some_and(|items| !items.is_empty())
}
fn modality_contract_transcripts(modality_runtime_contracts: &Value) -> Vec<&Value> {
modality_contract_snapshots(modality_runtime_contracts)
.into_iter()
.filter(|snapshot| {
snapshot
.get("contractKey")
.and_then(Value::as_str)
.map(|value| value == AUDIO_TRANSCRIPTION_CONTRACT_KEY)
.unwrap_or(false)
})
.filter_map(|snapshot| snapshot.get("transcript"))
.filter(|value| value.is_object())
.collect()
}
fn modality_contract_has_transcript_completed(modality_runtime_contracts: &Value) -> bool {
modality_contract_transcripts(modality_runtime_contracts)
.iter()
.any(|transcript| {
let completed = transcript
.get("status")
.and_then(Value::as_str)
.map(|value| value == "completed")
.unwrap_or(false);
let has_transcript_path = transcript
.get("transcriptPath")
.and_then(Value::as_str)
.and_then(|value| normalize_optional_text(Some(value.to_string())))
.is_some();
completed && has_transcript_path
})
}
fn modality_contract_has_transcript_failed(modality_runtime_contracts: &Value) -> bool {
modality_contract_transcripts(modality_runtime_contracts)
.iter()
.any(|transcript| {
transcript
.get("status")
.and_then(Value::as_str)
.map(|value| value == "failed")
.unwrap_or(false)
|| transcript
.get("errorCode")
.and_then(Value::as_str)
.and_then(|value| normalize_optional_text(Some(value.to_string())))
.is_some()
})
}
fn modality_contract_transcript_error_codes(modality_runtime_contracts: &Value) -> Vec<String> {
let mut error_codes = Vec::new();
for transcript in modality_contract_transcripts(modality_runtime_contracts) {
if let Some(error_code) = transcript
.get("errorCode")
.and_then(Value::as_str)
.and_then(|value| normalize_optional_text(Some(value.to_string())))
{
push_unique_owned_tag(&mut error_codes, error_code);
}
}
error_codes
}
fn modality_contract_has_web_research(modality_runtime_contracts: &Value) -> bool {
modality_contract_snapshots(modality_runtime_contracts)
.iter()
@@ -2011,28 +2380,43 @@ mod tests {
},
"audio_output": {
"kind": "audio_output",
"status": "pending",
"audio_path": null,
"status": "completed",
"audio_path": ".lime/runtime/audio/task-audio-1.mp3",
"mime_type": "audio/mpeg",
"duration_ms": null,
"duration_ms": 128000,
"source_text": "请为这段文案生成温暖旁白",
"voice": "warm_narrator"
"voice": "warm_narrator",
"provider_id": "limecore",
"model": "voice-pro"
}
},
"status": "pending_submit",
"normalized_status": "pending",
"status": "succeeded",
"normalized_status": "succeeded",
"created_at": "2026-04-30T10:00:00Z",
"updated_at": null,
"updated_at": "2026-04-30T10:00:05Z",
"submitted_at": null,
"started_at": null,
"completed_at": null,
"started_at": "2026-04-30T10:00:01Z",
"completed_at": "2026-04-30T10:00:05Z",
"cancelled_at": null,
"idempotency_key": null,
"retry_count": 0,
"source_task_id": null,
"result": null,
"result": {
"kind": "audio_generation_result",
"status": "completed",
"audio_output": {
"kind": "audio_output",
"status": "completed",
"audio_path": ".lime/runtime/audio/task-audio-1.mp3",
"mime_type": "audio/mpeg",
"duration_ms": 128000,
"provider_id": "limecore",
"model": "voice-pro"
}
},
"last_error": null,
"current_attempt_id": "attempt-audio-1",
"current_attempt_worker_id": "lime-audio-worker",
"attempts": [],
"relationships": {},
"progress": {},
@@ -2246,9 +2630,31 @@ mod tests {
"routing_slot": "browser_reasoning_model",
"runtime_contract": {
"contract_key": BROWSER_CONTROL_CONTRACT_KEY,
"routing_slot": "browser_reasoning_model"
"routing_slot": "browser_reasoning_model",
"executor_binding": {
"executor_kind": "browser_action",
"binding_key": "lime_browser_mcp"
}
},
"entry_source": "at_browser_command"
"entry_source": "at_browser_command",
"action": "navigate",
"selected_backend": "cdp_direct",
"result": {
"success": true,
"action": "navigate",
"request_id": "browser-request-1",
"session_id": "browser-session-1",
"target_id": "target-1",
"data": {
"browser_session": {
"session_id": "browser-session-1",
"profile_key": "general_browser_assist",
"target_id": "target-1",
"target_title": "Example",
"target_url": "https://example.com/"
}
}
}
})),
},
});
@@ -2283,6 +2689,22 @@ mod tests {
.and_then(Value::as_str),
Some("browser_action_requested")
);
assert_eq!(
input
.pointer(
"/runtimeContext/modalityRuntimeContracts/snapshotIndex/browserActionIndex/actionCount"
)
.and_then(Value::as_u64),
Some(1)
);
assert_eq!(
input
.pointer(
"/runtimeContext/modalityRuntimeContracts/snapshotIndex/browserActionIndex/items/0/sessionId"
)
.and_then(Value::as_str),
Some("browser-session-1")
);
let suite_tags = input
.pointer("/classification/suiteTags")
.and_then(Value::as_array)
@@ -2292,6 +2714,7 @@ mod tests {
"browser-control",
"browser-assist",
"browser-action-trace",
"browser-action-index",
] {
assert!(suite_tags
.iter()
@@ -2303,11 +2726,13 @@ mod tests {
assert!(expected.contains("mcp__lime-browser__*"));
assert!(expected.contains("WebSearch"));
assert!(expected.contains("browser_action_trace"));
assert!(expected.contains("browserActionIndex"));
assert!(expected.contains("\"requiresHumanReview\": false"));
let grader = fs::read_to_string(grader_path).expect("grader");
assert!(grader.contains("多模态运行合同检查"));
assert!(grader.contains("browser_action_requested"));
assert!(grader.contains("browserActionIndex"));
assert!(grader.contains("WebSearch"));
let links =
@@ -2319,6 +2744,12 @@ mod tests {
.and_then(Value::as_str),
Some("browser_action_trace.modality_runtime_contract")
);
assert_eq!(
links
.pointer("/modalityRuntimeContracts/snapshotIndex/browserActionIndex/lastUrl")
.and_then(Value::as_str),
Some("https://example.com/")
);
}
#[test]
@@ -2599,6 +3030,9 @@ mod tests {
let expected_path = temp_dir
.path()
.join(".lime/harness/sessions/session-1/replay/expected.json");
let grader_path = temp_dir
.path()
.join(".lime/harness/sessions/session-1/replay/grader.md");
let links_path = temp_dir
.path()
.join(".lime/harness/sessions/session-1/replay/evidence-links.json");
@@ -2618,6 +3052,26 @@ mod tests {
.and_then(Value::as_str),
Some("audio_generate")
);
assert_eq!(
input
.pointer("/runtimeContext/modalityRuntimeContracts/snapshots/0/audioOutput/status")
.and_then(Value::as_str),
Some("completed")
);
assert_eq!(
input
.pointer(
"/runtimeContext/modalityRuntimeContracts/snapshots/0/audioOutput/audioPath"
)
.and_then(Value::as_str),
Some(".lime/runtime/audio/task-audio-1.mp3")
);
assert_eq!(
input
.pointer("/runtimeContext/modalityRuntimeContracts/snapshotIndex/audioOutputIndex/outputCount")
.and_then(Value::as_u64),
Some(1)
);
let suite_tags = input
.pointer("/classification/suiteTags")
.and_then(Value::as_array)
@@ -2625,6 +3079,9 @@ mod tests {
assert!(suite_tags
.iter()
.any(|item| item.as_str() == Some("voice-generation-trace")));
assert!(suite_tags
.iter()
.any(|item| item.as_str() == Some("audio-output-completed")));
let failure_modes = input
.pointer("/classification/failureModes")
.and_then(Value::as_array)
@@ -2635,8 +3092,13 @@ mod tests {
let expected = fs::read_to_string(expected_path).expect("expected");
assert!(expected.contains("audio_task/audio_output"));
assert!(expected.contains("audio_output.completed"));
assert!(expected.contains("\"requiresHumanReview\": false"));
let grader = fs::read_to_string(grader_path).expect("grader");
assert!(grader.contains("audio_output.completed"));
assert!(grader.contains("audio_path"));
let links =
serde_json::from_str::<Value>(fs::read_to_string(links_path).expect("links").as_str())
.expect("parse links");
@@ -2646,6 +3108,112 @@ mod tests {
.and_then(Value::as_str),
Some("audio_task.modality_runtime_contract")
);
assert_eq!(
links
.pointer(
"/modalityRuntimeContracts/snapshotIndex/audioOutputIndex/items/0/audioPath"
)
.and_then(Value::as_str),
Some(".lime/runtime/audio/task-audio-1.mp3")
);
}
#[test]
fn should_classify_voice_audio_output_provider_failure_for_replay() {
let detail = build_detail();
let thread_read = build_thread_read();
let modality_runtime_contracts = json!({
"snapshotCount": 1,
"snapshots": [{
"contractKey": VOICE_GENERATION_CONTRACT_KEY,
"source": "audio_task.modality_runtime_contract",
"routingEvent": "executor_invoked",
"audioOutput": {
"status": "failed",
"errorCode": "audio_provider_unconfigured",
"errorMessage": "未找到可用的 voice_generation provider/API Key。",
"retryable": true
}
}]
});
let failure_modes =
infer_replay_failure_modes(&detail, &thread_read, &modality_runtime_contracts);
assert!(failure_modes
.iter()
.any(|item| item == "voice_generation_audio_output_failed"));
assert!(failure_modes
.iter()
.any(|item| item == "audio_provider_unconfigured"));
let blocking_checks = build_blocking_checks(&thread_read, &[], &modality_runtime_contracts);
assert!(blocking_checks
.iter()
.any(|item| item.contains("audio_provider_unconfigured")));
let contract_checks = build_modality_contract_checks(&modality_runtime_contracts);
assert!(contract_checks
.iter()
.any(|item| item.contains("audio_output.failed")));
assert!(contract_checks
.iter()
.any(|item| item.contains("audio_provider_unconfigured")));
}
#[test]
fn should_classify_audio_transcription_transcript_failure_for_replay() {
let detail = build_detail();
let thread_read = build_thread_read();
let modality_runtime_contracts = json!({
"snapshotCount": 1,
"snapshotIndex": {
"transcriptIndex": {
"transcriptCount": 1,
"statusCounts": [{ "status": "failed", "count": 1 }],
"errorCodes": ["transcription_provider_unconfigured"],
"items": [{
"status": "failed",
"errorCode": "transcription_provider_unconfigured"
}]
}
},
"snapshots": [{
"contractKey": AUDIO_TRANSCRIPTION_CONTRACT_KEY,
"source": "transcription_task.modality_runtime_contract",
"routingEvent": "executor_invoked",
"transcript": {
"status": "failed",
"errorCode": "transcription_provider_unconfigured",
"errorMessage": "未找到可用的 audio_transcription provider/API Key。",
"retryable": true
}
}]
});
let failure_modes =
infer_replay_failure_modes(&detail, &thread_read, &modality_runtime_contracts);
assert!(failure_modes
.iter()
.any(|item| item == "audio_transcription_transcript_failed"));
assert!(failure_modes
.iter()
.any(|item| item == "transcription_provider_unconfigured"));
let blocking_checks = build_blocking_checks(&thread_read, &[], &modality_runtime_contracts);
assert!(blocking_checks
.iter()
.any(|item| item.contains("transcription_provider_unconfigured")));
assert!(!blocking_checks
.iter()
.any(|item| item.contains("缺少 `snapshotIndex.transcriptIndex`")));
let contract_checks = build_modality_contract_checks(&modality_runtime_contracts);
assert!(contract_checks
.iter()
.any(|item| item.contains("transcript.failed")));
assert!(contract_checks
.iter()
.any(|item| item.contains("frontend_direct_asr")));
}
#[test]
src-tauri/src/voice/commands.rs
+14
View File
@@ -27,6 +27,10 @@ pub struct VoiceShortcutRuntimeStatus {
pub registered_shortcut: Option<String>,
pub translate_shortcut_registered: bool,
pub registered_translate_shortcut: Option<String>,
pub fn_supported: bool,
pub fn_registered: bool,
pub fn_fallback_shortcut: Option<String>,
pub fn_note: String,
}
/// 获取所有可用的麦克风设备
@@ -44,11 +48,17 @@ pub async fn get_voice_input_config() -> Result<VoiceInputConfig, String> {
/// 获取语音快捷键运行时状态
#[command]
pub async fn get_voice_shortcut_runtime_status() -> Result<VoiceShortcutRuntimeStatus, String> {
let fn_status = super::fn_shortcut::runtime_status();
Ok(VoiceShortcutRuntimeStatus {
shortcut_registered: super::shortcut::is_registered(),
registered_shortcut: super::shortcut::get_current(),
translate_shortcut_registered: super::shortcut::is_translate_registered(),
registered_translate_shortcut: super::shortcut::get_current_translate(),
fn_supported: fn_status.supported,
fn_registered: fn_status.registered,
fn_fallback_shortcut: super::shortcut::get_current()
.or_else(|| Some("CommandOrControl+Shift+V".to_string())),
fn_note: fn_status.note,
})
}
@@ -63,6 +73,8 @@ pub async fn save_voice_input_config(
let new_enabled = voice_config.enabled;
if old_enabled && new_enabled {
super::register_fn_shortcut_if_supported(&app);
if old_config.shortcut != voice_config.shortcut {
super::shortcut::update(&app, &voice_config.shortcut)?;
}
@@ -100,8 +112,10 @@ pub async fn save_voice_input_config(
} else if old_enabled && !new_enabled {
super::shortcut::unregister(&app)?;
let _ = super::shortcut::unregister_translate(&app);
let _ = super::fn_shortcut::unregister();
} else if !old_enabled && new_enabled {
super::shortcut::register(&app, &voice_config.shortcut)?;
super::register_fn_shortcut_if_supported(&app);
if let Some(translate_shortcut) =
normalize_shortcut(voice_config.translate_shortcut.clone())
src-tauri/src/voice/fn_shortcut.rs
+294
View File
@@ -0,0 +1,294 @@
//! macOS Fn 按住录音监听。
//!
//! Fn 不走 `tauri_plugin_global_shortcut` 的字符串解析路径；macOS 下通过
//! NSEvent 的 FlagsChanged 事件做一个窄桥接，其他平台明确报告不支持。
#[derive(Debug, Clone)]
pub struct FnShortcutRuntimeStatus {
pub supported: bool,
pub registered: bool,
pub note: String,
}
#[cfg(target_os = "macos")]
#[allow(deprecated, unexpected_cfgs)]
mod platform {
use super::FnShortcutRuntimeStatus;
use block::{ConcreteBlock, RcBlock};
use cocoa::appkit::{NSEvent, NSEventMask, NSEventModifierFlags};
use cocoa::base::{id, nil};
use objc::{class, msg_send, sel, sel_impl};
use parking_lot::Mutex;
use std::os::raw::c_uchar;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::OnceLock;
use tauri::AppHandle;
use tracing::{debug, error, info, warn};
type GlobalEventBlock = RcBlock<(id,), ()>;
type LocalEventBlock = RcBlock<(id,), id>;
static REGISTRATION: OnceLock<Mutex<Option<FnShortcutRegistration>>> = OnceLock::new();
static IS_REGISTERED: AtomicBool = AtomicBool::new(false);
static FN_IS_DOWN: AtomicBool = AtomicBool::new(false);
static LAST_ERROR: OnceLock<Mutex<Option<String>>> = OnceLock::new();
#[link(name = "ApplicationServices", kind = "framework")]
extern "C" {
fn AXIsProcessTrusted() -> c_uchar;
}
struct ObjcMonitor(id);
// AppKit monitor token 和 Block 都只在 register/unregister 边界持有；
// 回调由 AppKit 调度，Rust 侧不跨线程解引用这些裸指针。
unsafe impl Send for ObjcMonitor {}
struct GlobalBlock {
_inner: GlobalEventBlock,
}
unsafe impl Send for GlobalBlock {}
struct LocalBlock {
_inner: LocalEventBlock,
}
unsafe impl Send for LocalBlock {}
struct FnShortcutRegistration {
global_monitor: Option<ObjcMonitor>,
local_monitor: Option<ObjcMonitor>,
_global_block: GlobalBlock,
_local_block: LocalBlock,
}
fn registration() -> &'static Mutex<Option<FnShortcutRegistration>> {
REGISTRATION.get_or_init(|| Mutex::new(None))
}
fn last_error() -> &'static Mutex<Option<String>> {
LAST_ERROR.get_or_init(|| Mutex::new(None))
}
fn set_last_error(value: Option<String>) {
*last_error().lock() = value;
}
fn accessibility_trusted() -> bool {
// SAFETY: AXIsProcessTrusted 无参数、无副作用，返回 CoreServices Boolean。
unsafe { AXIsProcessTrusted() != 0 }
}
fn handle_flags_changed_event(app: &AppHandle, event: id) {
if event == nil {
return;
}
let fn_down = unsafe {
event
.modifierFlags()
.contains(NSEventModifierFlags::NSFunctionKeyMask)
};
let was_down = FN_IS_DOWN.swap(fn_down, Ordering::SeqCst);
match (was_down, fn_down) {
(false, true) => {
info!("[语音输入] Fn 按下");
if let Err(error) = crate::screenshot::window::open_floating_window_with_voice(app)
{
error!("[语音输入] Fn 打开语音窗口失败: {}", error);
}
}
(true, false) => {
info!("[语音输入] Fn 释放，发送停止录音事件");
if let Err(error) = crate::screenshot::window::send_voice_stop_event(app) {
error!("[语音输入] Fn 发送停止录音事件失败: {}", error);
}
}
_ => {
debug!("[语音输入] 忽略重复 Fn 修饰键状态: {}", fn_down);
}
}
}
unsafe fn add_global_monitor(block: &GlobalEventBlock) -> id {
let mask = NSEventMask::NSFlagsChangedMask.bits();
msg_send![
class!(NSEvent),
addGlobalMonitorForEventsMatchingMask: mask
handler: &**block
]
}
unsafe fn add_local_monitor(block: &LocalEventBlock) -> id {
let mask = NSEventMask::NSFlagsChangedMask.bits();
msg_send![
class!(NSEvent),
addLocalMonitorForEventsMatchingMask: mask
handler: &**block
]
}
unsafe fn remove_monitor(monitor: ObjcMonitor) {
if monitor.0 != nil {
let _: () = msg_send![class!(NSEvent), removeMonitor: monitor.0];
let _: () = msg_send![monitor.0, release];
}
}
unsafe fn retain_monitor(monitor: id) -> Option<ObjcMonitor> {
if monitor == nil {
None
} else {
let retained: id = msg_send![monitor, retain];
Some(ObjcMonitor(retained))
}
}
pub fn register(app: &AppHandle) -> Result<(), String> {
if IS_REGISTERED.load(Ordering::SeqCst) {
return Ok(());
}
let app_for_global = app.clone();
let global_block = ConcreteBlock::new(move |event: id| {
handle_flags_changed_event(&app_for_global, event);
})
.copy();
let app_for_local = app.clone();
let local_block = ConcreteBlock::new(move |event: id| -> id {
handle_flags_changed_event(&app_for_local, event);
event
})
.copy();
let global_monitor = unsafe { add_global_monitor(&global_block) };
let local_monitor = unsafe { add_local_monitor(&local_block) };
if global_monitor == nil && local_monitor == nil {
let error = "macOS Fn 事件监听注册失败，NSEvent 未返回 monitor token".to_string();
set_last_error(Some(error.clone()));
return Err(error);
}
*registration().lock() = Some(FnShortcutRegistration {
global_monitor: unsafe { retain_monitor(global_monitor) },
local_monitor: unsafe { retain_monitor(local_monitor) },
_global_block: GlobalBlock {
_inner: global_block,
},
_local_block: LocalBlock {
_inner: local_block,
},
});
FN_IS_DOWN.store(false, Ordering::SeqCst);
IS_REGISTERED.store(true, Ordering::SeqCst);
set_last_error(None);
if accessibility_trusted() {
info!("[语音输入] Fn 按住录音监听已注册");
} else {
warn!("[语音输入] Fn 监听已注册；跨应用捕获可能需要 macOS 辅助功能或输入监听权限");
}
Ok(())
}
pub fn unregister() -> Result<(), String> {
let registration = registration().lock().take();
if let Some(registration) = registration {
if let Some(monitor) = registration.global_monitor {
unsafe { remove_monitor(monitor) };
}
if let Some(monitor) = registration.local_monitor {
unsafe { remove_monitor(monitor) };
}
}
IS_REGISTERED.store(false, Ordering::SeqCst);
FN_IS_DOWN.store(false, Ordering::SeqCst);
info!("[语音输入] Fn 按住录音监听已注销");
Ok(())
}
pub fn is_supported() -> bool {
true
}
pub fn is_registered() -> bool {
IS_REGISTERED.load(Ordering::SeqCst)
}
pub fn runtime_status() -> FnShortcutRuntimeStatus {
let registered = is_registered();
let note = if let Some(error) = last_error().lock().clone() {
format!("Fn 监听注册失败：{error}；已使用普通语音快捷键回退。")
} else if registered && accessibility_trusted() {
"macOS Fn 监听已注册：按住 Fn 开始录音，松开 Fn 停止并转写。".to_string()
} else if registered {
"macOS Fn 监听已注册；若跨应用按 Fn 无响应，请在系统设置中授予 Lime 辅助功能/输入监听权限，或使用普通快捷键回退。"
.to_string()
} else {
"macOS 支持 Fn 监听；当前语音输入未启用或监听尚未注册，已使用普通语音快捷键回退。"
.to_string()
};
FnShortcutRuntimeStatus {
supported: true,
registered,
note,
}
}
}
#[cfg(not(target_os = "macos"))]
mod platform {
use super::FnShortcutRuntimeStatus;
use tauri::AppHandle;
pub fn register(_app: &AppHandle) -> Result<(), String> {
Err("Fn 按住录音当前仅支持 macOS；已使用普通语音快捷键回退。".to_string())
}
pub fn unregister() -> Result<(), String> {
Ok(())
}
pub fn is_supported() -> bool {
false
}
pub fn is_registered() -> bool {
false
}
pub fn runtime_status() -> FnShortcutRuntimeStatus {
FnShortcutRuntimeStatus {
supported: false,
registered: false,
note: "Fn 按住录音当前仅支持 macOS；已使用普通语音快捷键回退。".to_string(),
}
}
}
pub fn register(app: &tauri::AppHandle) -> Result<(), String> {
platform::register(app)
}
pub fn unregister() -> Result<(), String> {
platform::unregister()
}
pub fn is_supported() -> bool {
platform::is_supported()
}
pub fn is_registered() -> bool {
platform::is_registered()
}
pub fn runtime_status() -> FnShortcutRuntimeStatus {
platform::runtime_status()
}
src-tauri/src/voice/mod.rs
+16
View File
@@ -9,6 +9,7 @@
pub mod asr_service;
pub mod commands;
pub mod config;
pub mod fn_shortcut;
pub mod output_service;
pub mod processor;
pub mod recording_service;
@@ -17,6 +18,17 @@ pub mod window;
use tauri::AppHandle;
pub(crate) fn register_fn_shortcut_if_supported(app: &AppHandle) {
if !fn_shortcut::is_supported() {
tracing::info!("[语音输入] 当前平台不支持 Fn 按住录音，使用普通快捷键回退");
return;
}
if let Err(error) = fn_shortcut::register(app) {
tracing::warn!("[语音输入] Fn 按住录音监听注册失败: {}", error);
}
}
/// 初始化语音输入模块
pub fn init(app: &AppHandle) -> Result<(), String> {
// 加载配置
@@ -30,6 +42,7 @@ pub fn init(app: &AppHandle) -> Result<(), String> {
// 注册全局快捷键
shortcut::register(app, &config.shortcut)?;
register_fn_shortcut_if_supported(app);
// 注册翻译快捷键（如果配置了）
if let Some(translate_shortcut) = &config.translate_shortcut {
@@ -62,6 +75,9 @@ pub fn cleanup(app: &AppHandle) -> Result<(), String> {
// 注销翻译快捷键
let _ = shortcut::unregister_translate(app);
// 注销 Fn 监听
let _ = fn_shortcut::unregister();
// 关闭悬浮窗口
window::close_voice_window(app)?;
src-tauri/tauri.conf.headless.json
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Lime",
"version": "1.24.0",
"version": "1.25.0",
"identifier": "com.limecloud.lime.headless",
"build": {
"beforeDevCommand": "npm run dev:web-bridge",
src-tauri/tauri.conf.json
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Lime",
"version": "1.24.0",
"version": "1.25.0",
"identifier": "com.limecloud.lime",
"build": {
"beforeDevCommand": "node scripts/start-tauri-dev-server.mjs",
src/App.tsx
+70 -26
View File
@@ -10,7 +10,6 @@
import React, { Suspense, lazy, useState, useCallback } from "react";
import styled from "styled-components";
import { getCurrentWindow } from "@tauri-apps/api/window";
import { withI18nPatch } from "./i18n/withI18nPatch";
import { AppPageContent } from "./components/AppPageContent";
import { SplashScreen } from "./components/SplashScreen";
@@ -47,6 +46,8 @@ import { resolveWebsiteOpenNavigation } from "./lib/deepLink/websiteLaunch";
import { toast } from "sonner";
import { SettingsTabs } from "./types/settings";
import { hasTauriInvokeCapability } from "./lib/tauri-runtime";
import { shouldReserveMacWindowControls } from "./lib/windowControls";
import { startWindowDragFromMouseEvent } from "./lib/windowDrag";
const AppContainer = styled.div`
display: flex;
@@ -66,15 +67,42 @@ const MainContent = styled.main<{ $withSidebarGap?: boolean }>`
background: var(--lime-app-bg, hsl(var(--background)));
`;
const WindowDragRegion = styled.div`
const WINDOW_DRAG_TOP_HEIGHT = 30;
const WINDOW_DRAG_EDGE_WIDTH = 8;
const WINDOW_DRAG_DEFAULT_SAFE_LEFT = 160;
const WINDOW_DRAG_MAC_SAFE_LEFT = 92;
const WindowDragLayer = styled.div`
position: fixed;
top: 0;
left: 160px;
right: 0;
height: 28px;
inset: 0;
z-index: 1000;
background: transparent;
pointer-events: none;
`;
const WindowTopDragRegion = styled.div<{ $reserveMacWindowControls?: boolean }>`
position: absolute;
top: 0;
left: ${({ $reserveMacWindowControls }) =>
$reserveMacWindowControls
? `${WINDOW_DRAG_MAC_SAFE_LEFT}px`
: `${WINDOW_DRAG_DEFAULT_SAFE_LEFT}px`};
right: 0;
height: ${WINDOW_DRAG_TOP_HEIGHT}px;
pointer-events: auto;
user-select: none;
app-region: drag;
-webkit-app-region: drag;
`;
const WindowSideDragRegion = styled.div<{ $side: "left" | "right" }>`
position: absolute;
top: ${WINDOW_DRAG_TOP_HEIGHT}px;
bottom: 0;
${({ $side }) => $side}: 0;
width: ${WINDOW_DRAG_EDGE_WIDTH}px;
pointer-events: auto;
user-select: none;
app-region: drag;
-webkit-app-region: drag;
`;
@@ -116,6 +144,7 @@ const pageLoadingFallback = (
function AppContent() {
const hasTauriDesktopRuntime = hasTauriInvokeCapability();
const reserveMacWindowControls = shouldReserveMacWindowControls();
const [showSplash, setShowSplash] = useState(true);
const {
currentPage,
@@ -297,21 +326,10 @@ function AppContent() {
}, [completeOnboarding]);
const handleWindowDragStart = useCallback(
async (event: React.MouseEvent<HTMLDivElement>) => {
if (!hasTauriDesktopRuntime || event.button !== 0) {
return;
}
event.preventDefault();
try {
const currentWindow = getCurrentWindow();
await currentWindow.startDragging();
} catch (error) {
console.warn("[窗口] 主窗口拖拽启动失败:", error);
}
(event: React.MouseEvent<HTMLElement>) => {
void startWindowDragFromMouseEvent(event, { source: "app_shell" });
},
[hasTauriDesktopRuntime],
[],
);
if (showSplash) {
@@ -335,10 +353,26 @@ function AppContent() {
<ComponentDebugProvider>
<AppContainer>
{hasTauriDesktopRuntime ? (
<WindowDragRegion
data-tauri-drag-region
onMouseDown={handleWindowDragStart}
/>
<WindowDragLayer aria-hidden="true">
<WindowTopDragRegion
$reserveMacWindowControls={reserveMacWindowControls}
data-tauri-drag-region
data-lime-window-drag-region
onMouseDown={handleWindowDragStart}
/>
<WindowSideDragRegion
$side="left"
data-tauri-drag-region
data-lime-window-drag-region
onMouseDown={handleWindowDragStart}
/>
<WindowSideDragRegion
$side="right"
data-tauri-drag-region
data-lime-window-drag-region
onMouseDown={handleWindowDragStart}
/>
</WindowDragLayer>
) : null}
{shouldShowAppSidebar && (
<AppSidebar
@@ -347,9 +381,19 @@ function AppContent() {
requestedPage={requestedPage}
requestedPageParams={requestedPageParams}
onNavigate={handleNavigate}
onStartWindowDrag={handleWindowDragStart}
/>
)}
<MainContent $withSidebarGap={shouldAddMainContentGap}>
<MainContent
$withSidebarGap={shouldAddMainContentGap}
data-lime-window-drag-region
onMouseDown={(event) => {
void startWindowDragFromMouseEvent(event, {
allowDescendantTargets: false,
source: "main_content",
});
}}
>
<Suspense fallback={pageLoadingFallback}>
<AppPageContent
currentPage={currentPage}
src/components/AppSidebar.test.tsx
+746 -36
View File
@@ -8,6 +8,7 @@ import { AppSidebar } from "./AppSidebar";
import {
TASK_CENTER_CREATE_DRAFT_TASK_EVENT,
TASK_CENTER_OPEN_TASK_EVENT,
TASK_CENTER_PREFETCH_TASK_EVENT,
} from "@/components/agent/chat/taskCenterDraftTaskEvents";
import { LIME_COLOR_SCHEME_STORAGE_KEY } from "@/lib/appearance/colorSchemes";
import { LIME_THEME_STORAGE_KEY } from "@/lib/appearance/themeMode";
@@ -24,6 +25,7 @@ const {
mockSubscribeAppConfigChanged,
mockListAgentRuntimeSessions,
mockUpdateAgentRuntimeSession,
mockDeleteAgentRuntimeSession,
mockSetI18nLanguage,
mockScheduleMinimumDelayIdleTask,
mockLogoutClient,
@@ -44,6 +46,7 @@ const {
mockSubscribeAppConfigChanged: vi.fn(),
mockListAgentRuntimeSessions: vi.fn(),
mockUpdateAgentRuntimeSession: vi.fn(),
mockDeleteAgentRuntimeSession: vi.fn(),
mockSetI18nLanguage: vi.fn(),
mockScheduleMinimumDelayIdleTask: vi.fn((task: () => void) => {
task();
@@ -82,6 +85,7 @@ vi.mock("@/lib/api/pluginUI", () => ({
}));
vi.mock("@/lib/api/agentRuntime", () => ({
deleteAgentRuntimeSession: mockDeleteAgentRuntimeSession,
listAgentRuntimeSessions: mockListAgentRuntimeSessions,
updateAgentRuntimeSession: mockUpdateAgentRuntimeSession,
}));
@@ -164,6 +168,41 @@ async function flushEffects(times = 1) {
}
}
function setInputValue(input: HTMLInputElement, value: string) {
const prototype = Object.getPrototypeOf(input) as HTMLInputElement;
const valueSetter = Object.getOwnPropertyDescriptor(prototype, "value")?.set;
if (valueSetter) {
valueSetter.call(input, value);
} else {
input.value = value;
}
input.dispatchEvent(new Event("input", { bubbles: true }));
}
async function openConversationMenu(title: string) {
await act(async () => {
document
.querySelector<HTMLButtonElement>(
`button[aria-label="打开 ${title} 操作菜单"]`,
)
?.click();
await Promise.resolve();
});
return document.body.querySelector<HTMLElement>(
'[data-testid="app-sidebar-conversation-menu"]',
);
}
async function clickConversationMenuItem(testId: string) {
await act(async () => {
document.body
.querySelector<HTMLButtonElement>(`[data-testid="${testId}"]`)
?.click();
await Promise.resolve();
});
}
async function openAccountMenu(container: HTMLElement) {
await act(async () => {
container
@@ -247,6 +286,7 @@ describe("AppSidebar", () => {
mockGetPluginsForSurface.mockResolvedValue([]);
mockListAgentRuntimeSessions.mockResolvedValue([]);
mockUpdateAgentRuntimeSession.mockResolvedValue(undefined);
mockDeleteAgentRuntimeSession.mockResolvedValue(undefined);
mockLogoutClient.mockResolvedValue(undefined);
mockGetConfiguredOemCloudTarget.mockReturnValue({
baseUrl: "https://user.limeai.run",
@@ -1037,13 +1077,14 @@ describe("AppSidebar", () => {
}
});
it("任务中心内点击已有会话应先通知本地标签栏，再继续导航", async () => {
it("任务中心内点击已有会话应交给本地标签栏，不重复触发导航", async () => {
const onNavigate = vi.fn();
const receivedDetails: unknown[] = [];
const listener = (event: Event) => {
receivedDetails.push(
event instanceof CustomEvent ? event.detail : undefined,
);
event.preventDefault();
};
window.addEventListener(TASK_CENTER_OPEN_TASK_EVENT, listener);
mockListAgentRuntimeSessions.mockResolvedValue([
@@ -1084,19 +1125,462 @@ describe("AppSidebar", () => {
source: "sidebar",
},
]);
expect(onNavigate).toHaveBeenCalledWith(
"agent",
expect.objectContaining({
agentEntry: "claw",
projectId: "project-1",
initialSessionId: "session-target",
}),
);
expect(onNavigate).not.toHaveBeenCalled();
} finally {
window.removeEventListener(TASK_CENTER_OPEN_TASK_EVENT, listener);
}
});
it("新建任务首页点击已有会话应进入对应历史对话", async () => {
const onNavigate = vi.fn();
mockListAgentRuntimeSessions.mockResolvedValue([
{
id: "session-target",
name: "目标历史会话",
created_at: 1713000000,
updated_at: 1713000600,
archived_at: null,
workspace_id: "project-1",
messages_count: 3,
},
]);
const container = mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "new-task",
projectId: "project-1",
} as AgentPageParams,
onNavigate,
});
await flushEffects(2);
await act(async () => {
container
.querySelector<HTMLButtonElement>('button[title="目标历史会话"]')
?.click();
await Promise.resolve();
});
expect(onNavigate).toHaveBeenCalledWith(
"agent",
expect.objectContaining({
agentEntry: "claw",
projectId: "project-1",
initialSessionId: "session-target",
}),
);
});
it("任务中心内悬停已有会话应延迟通知本地预取旧会话", async () => {
vi.useFakeTimers();
const receivedDetails: unknown[] = [];
const listener = (event: Event) => {
receivedDetails.push(
event instanceof CustomEvent ? event.detail : undefined,
);
};
window.addEventListener(TASK_CENTER_PREFETCH_TASK_EVENT, listener);
mockListAgentRuntimeSessions.mockResolvedValue([
{
id: "session-prefetch",
name: "可预取历史会话",
created_at: 1713000000,
updated_at: 1713000600,
archived_at: null,
workspace_id: "project-1",
messages_count: 3,
},
]);
try {
const container = mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "claw",
projectId: "project-1",
initialSessionId: "session-current",
} as AgentPageParams,
});
await flushEffects(2);
await act(async () => {
container
.querySelector<HTMLButtonElement>('button[title="可预取历史会话"]')
?.focus();
await Promise.resolve();
});
expect(receivedDetails).toEqual([]);
act(() => {
vi.advanceTimersByTime(920);
});
expect(receivedDetails).toEqual([
{
sessionId: "session-prefetch",
workspaceId: "project-1",
source: "conversation_shelf",
},
]);
} finally {
vi.useRealTimers();
window.removeEventListener(TASK_CENTER_PREFETCH_TASK_EVENT, listener);
}
});
it("点击已有会话时不应先触发旧会话预取抢占切换链路", async () => {
vi.useFakeTimers();
const receivedPrefetchDetails: unknown[] = [];
const listener = (event: Event) => {
receivedPrefetchDetails.push(
event instanceof CustomEvent ? event.detail : undefined,
);
};
window.addEventListener(TASK_CENTER_PREFETCH_TASK_EVENT, listener);
mockListAgentRuntimeSessions.mockResolvedValue([
{
id: "session-click",
name: "立即打开历史会话",
created_at: 1713000000,
updated_at: 1713000600,
archived_at: null,
workspace_id: "project-1",
messages_count: 3,
},
]);
try {
const onNavigate = vi.fn();
const container = mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "claw",
projectId: "project-1",
initialSessionId: "session-current",
} as AgentPageParams,
onNavigate,
});
await flushEffects(2);
const button = container.querySelector<HTMLButtonElement>(
'button[title="立即打开历史会话"]',
);
await act(async () => {
button?.focus();
button?.click();
await Promise.resolve();
});
act(() => {
vi.advanceTimersByTime(200);
});
expect(receivedPrefetchDetails).toEqual([]);
expect(onNavigate).not.toHaveBeenCalled();
} finally {
vi.useRealTimers();
window.removeEventListener(TASK_CENTER_PREFETCH_TASK_EVENT, listener);
}
});
it("搜索按钮应打开标题搜索弹窗，并按会话标题过滤结果", async () => {
mockListAgentRuntimeSessions.mockResolvedValue([
{
id: "session-agent",
name: "写一篇AI Agent的公众号",
created_at: 1713000000,
updated_at: 1713000600,
archived_at: null,
workspace_id: "project-1",
messages_count: 3,
},
{
id: "session-daily",
name: "啊啊啊啊啊",
created_at: 1712900000,
updated_at: 1712900600,
archived_at: null,
workspace_id: "project-1",
messages_count: 1,
},
]);
const container = mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "new-task",
projectId: "project-1",
} as AgentPageParams,
});
await flushEffects(2);
await act(async () => {
container
.querySelector<HTMLButtonElement>(
'[data-testid="app-sidebar-search-button"]',
)
?.click();
await Promise.resolve();
});
const dialog = document.body.querySelector<HTMLElement>(
'[data-testid="app-sidebar-search-dialog"]',
);
expect(dialog).not.toBeNull();
expect(dialog?.textContent).toContain("新建对话");
expect(dialog?.textContent).toContain("写一篇AI Agent的公众号");
expect(dialog?.textContent).toContain("啊啊啊啊啊");
const input = document.body.querySelector<HTMLInputElement>(
'[data-testid="app-sidebar-search-input"]',
);
expect(input).not.toBeNull();
await act(async () => {
setInputValue(input as HTMLInputElement, "Agent");
await Promise.resolve();
});
expect(dialog?.textContent).toContain("匹配结果");
expect(dialog?.textContent).toContain("写一篇AI Agent的公众号");
expect(dialog?.textContent).not.toContain("啊啊啊啊啊");
await act(async () => {
setInputValue(input as HTMLInputElement, "不存在");
await Promise.resolve();
});
expect(dialog?.textContent).toContain("没有匹配的对话标题");
});
it("搜索弹窗应支持查看更多对话并展示后续结果", async () => {
const sessions = Array.from({ length: 12 }, (_, index) => {
const number = String(index + 1).padStart(2, "0");
return {
id: `session-${number}`,
name: `对话 ${number}`,
created_at: 1713000000 + index,
updated_at: 1713000600 + index,
archived_at: null,
workspace_id: "project-1",
messages_count: 1,
};
});
mockListAgentRuntimeSessions.mockResolvedValue(sessions);
const container = mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "new-task",
projectId: "project-1",
} as AgentPageParams,
});
await flushEffects(2);
await act(async () => {
container
.querySelector<HTMLButtonElement>(
'[data-testid="app-sidebar-search-button"]',
)
?.click();
await Promise.resolve();
});
let dialog = document.body.querySelector<HTMLElement>(
'[data-testid="app-sidebar-search-dialog"]',
);
expect(dialog?.textContent).toContain("查看更多对话");
expect(dialog?.textContent).toContain("对话 12");
expect(dialog?.textContent).not.toContain("对话 02");
await act(async () => {
dialog
?.querySelector<HTMLButtonElement>(
'[data-testid="app-sidebar-search-more"]',
)
?.click();
await Promise.resolve();
});
await flushEffects(2);
dialog = document.body.querySelector<HTMLElement>(
'[data-testid="app-sidebar-search-dialog"]',
);
expect(dialog?.textContent).toContain("对话 02");
expect(dialog?.textContent).not.toContain("查看更多对话");
expect(mockListAgentRuntimeSessions).toHaveBeenLastCalledWith({
limit: 21,
workspaceId: "project-1",
});
});
it("搜索结果点击应复用会话导航并关闭弹窗", async () => {
const onNavigate = vi.fn();
mockListAgentRuntimeSessions.mockResolvedValue([
{
id: "session-target",
name: "目标历史会话",
created_at: 1713000000,
updated_at: 1713000600,
archived_at: null,
workspace_id: "project-1",
messages_count: 3,
},
]);
const container = mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
projectId: "project-1",
} as AgentPageParams,
onNavigate,
});
await flushEffects(2);
await act(async () => {
container
.querySelector<HTMLButtonElement>(
'[data-testid="app-sidebar-search-button"]',
)
?.click();
await Promise.resolve();
});
const dialog = document.body.querySelector<HTMLElement>(
'[data-testid="app-sidebar-search-dialog"]',
);
await act(async () => {
dialog
?.querySelector<HTMLButtonElement>('button[title="目标历史会话"]')
?.click();
await Promise.resolve();
});
expect(onNavigate).toHaveBeenCalledWith(
"agent",
expect.objectContaining({
agentEntry: "claw",
projectId: "project-1",
initialSessionId: "session-target",
}),
);
expect(
document.body.querySelector('[data-testid="app-sidebar-search-dialog"]'),
).toBeNull();
});
it("搜索弹窗的新建对话入口应复用现有新建导航", async () => {
const onNavigate = vi.fn();
mockListAgentRuntimeSessions.mockResolvedValue([]);
const container = mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
projectId: "project-1",
} as AgentPageParams,
onNavigate,
});
await flushEffects(2);
await act(async () => {
container
.querySelector<HTMLButtonElement>(
'[data-testid="app-sidebar-search-button"]',
)
?.click();
await Promise.resolve();
});
await act(async () => {
document.body
.querySelector<HTMLButtonElement>(
'[data-testid="app-sidebar-search-new-conversation"]',
)
?.click();
await Promise.resolve();
});
expect(onNavigate).toHaveBeenCalledWith(
"agent",
expect.objectContaining({
agentEntry: "new-task",
projectId: "project-1",
}),
);
expect(
document.body.querySelector('[data-testid="app-sidebar-search-dialog"]'),
).toBeNull();
});
it("Meta/Ctrl + K 应打开搜索弹窗，Escape 应关闭弹窗", async () => {
mockListAgentRuntimeSessions.mockResolvedValue([]);
mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "new-task",
projectId: "project-1",
} as AgentPageParams,
});
await flushEffects(2);
expect(
document.body.querySelector('[data-testid="app-sidebar-search-dialog"]'),
).toBeNull();
await act(async () => {
window.dispatchEvent(
new KeyboardEvent("keydown", {
key: "k",
metaKey: true,
bubbles: true,
cancelable: true,
}),
);
await Promise.resolve();
});
await flushEffects();
expect(
document.body.querySelector('[data-testid="app-sidebar-search-dialog"]'),
).not.toBeNull();
await act(async () => {
document.dispatchEvent(
new KeyboardEvent("keydown", {
key: "Escape",
bubbles: true,
cancelable: true,
}),
);
await Promise.resolve();
});
expect(
document.body.querySelector('[data-testid="app-sidebar-search-dialog"]'),
).toBeNull();
await act(async () => {
window.dispatchEvent(
new KeyboardEvent("keydown", {
key: "k",
ctrlKey: true,
bubbles: true,
cancelable: true,
}),
);
await Promise.resolve();
});
expect(
document.body.querySelector('[data-testid="app-sidebar-search-dialog"]'),
).not.toBeNull();
});
it("claw 页面不应再外露左侧工作台一级入口", async () => {
const onNavigate = vi.fn();
mockListAgentRuntimeSessions.mockResolvedValue([
@@ -1181,7 +1665,7 @@ describe("AppSidebar", () => {
expect(container.textContent).not.toContain("归档会话");
expect(mockListAgentRuntimeSessions).toHaveBeenCalledTimes(1);
expect(mockListAgentRuntimeSessions).toHaveBeenCalledWith({
limit: 37,
limit: 11,
workspaceId: "project-1",
});
@@ -1194,7 +1678,7 @@ describe("AppSidebar", () => {
const recentConversationList = container.querySelector(
'[data-testid="app-sidebar-recent-conversations"]',
);
const archivedToggle = container.querySelector(
const archivedToggle = conversationShelf?.querySelector(
'button[aria-expanded="false"]',
) as HTMLButtonElement | null;
@@ -1231,7 +1715,7 @@ describe("AppSidebar", () => {
).toBe("auto");
expect(mockListAgentRuntimeSessions).toHaveBeenCalledWith({
archivedOnly: true,
limit: 17,
limit: 9,
workspaceId: "project-1",
});
});
@@ -1280,7 +1764,7 @@ describe("AppSidebar", () => {
}),
);
expect(mockListAgentRuntimeSessions).toHaveBeenCalledWith({
limit: 37,
limit: 11,
workspaceId: "project-1",
});
});
@@ -1315,7 +1799,7 @@ describe("AppSidebar", () => {
expect(container.querySelector('button[title="会话 24"]')).toBeNull();
expect(container.textContent).toContain("查看更多对话");
expect(mockListAgentRuntimeSessions).toHaveBeenCalledWith({
limit: 37,
limit: 11,
workspaceId: "project-1",
});
@@ -1359,7 +1843,7 @@ describe("AppSidebar", () => {
expect(mounted.container.textContent).not.toContain("正在加载对话");
expect(mockListAgentRuntimeSessions).toHaveBeenCalledTimes(1);
expect(mockListAgentRuntimeSessions).toHaveBeenCalledWith({
limit: 37,
limit: 11,
workspaceId: "project-1",
});
@@ -1384,7 +1868,7 @@ describe("AppSidebar", () => {
expect(mounted.container.textContent).not.toContain("正在加载对话");
expect(mockListAgentRuntimeSessions).toHaveBeenCalledTimes(2);
expect(mockListAgentRuntimeSessions).toHaveBeenCalledWith({
limit: 37,
limit: 11,
workspaceId: "project-2",
});
});
@@ -1446,12 +1930,178 @@ describe("AppSidebar", () => {
expect(mockListAgentRuntimeSessions).toHaveBeenCalledTimes(2);
expect(mockListAgentRuntimeSessions).toHaveBeenLastCalledWith({
limit: 37,
limit: 11,
workspaceId: "project-1",
});
});
it("点击导航栏归档动作时应走统一 session update 命令", async () => {
it("点击会话菜单归档动作时应走统一 session update 命令", async () => {
mockListAgentRuntimeSessions.mockResolvedValue([
{
id: "session-recent",
name: "最近会话",
created_at: 1714000000,
updated_at: 1714000600,
archived_at: null,
workspace_id: "project-1",
},
]);
mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "claw",
projectId: "project-1",
} as AgentPageParams,
});
await flushEffects(2);
const menu = await openConversationMenu("最近会话");
expect(menu).not.toBeNull();
expect(menu?.textContent).toContain("重命名");
expect(menu?.textContent).toContain("收藏");
expect(menu?.textContent).toContain("归档");
expect(menu?.textContent).toContain("多选");
expect(menu?.textContent).toContain("删除");
expect(mockListAgentRuntimeSessions).toHaveBeenCalledWith({
limit: 11,
workspaceId: "project-1",
});
await clickConversationMenuItem("app-sidebar-conversation-menu-archive");
expect(mockUpdateAgentRuntimeSession).toHaveBeenCalledWith({
session_id: "session-recent",
archived: true,
});
});
it("归档会话菜单应展示恢复动作并走统一 session update 命令", async () => {
mockListAgentRuntimeSessions.mockImplementation(
async (options?: {
archivedOnly?: boolean;
includeArchived?: boolean;
limit?: number;
workspaceId?: string;
}) =>
options?.archivedOnly
? [
{
id: "session-archived",
name: "归档会话",
created_at: 1713000000,
updated_at: 1713000600,
archived_at: 1713003600,
workspace_id: "project-1",
},
]
: [],
);
const container = mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "claw",
projectId: "project-1",
} as AgentPageParams,
});
await flushEffects(2);
await act(async () => {
container
.querySelector<HTMLButtonElement>('button[aria-expanded="false"]')
?.click();
await Promise.resolve();
});
await flushEffects(2);
const menu = await openConversationMenu("归档会话");
expect(menu).not.toBeNull();
expect(menu?.textContent).toContain("重命名");
expect(menu?.textContent).toContain("收藏");
expect(menu?.textContent).toContain("恢复");
expect(menu?.textContent).toContain("多选");
expect(menu?.textContent).toContain("删除");
expect(menu?.textContent).not.toContain("归档");
await clickConversationMenuItem("app-sidebar-conversation-menu-archive");
expect(mockUpdateAgentRuntimeSession).toHaveBeenCalledWith({
session_id: "session-archived",
archived: false,
});
});
it("会话菜单应支持重命名并同步更新 session 名称", async () => {
vi.spyOn(window, "prompt").mockReturnValue("重命名后的会话");
mockListAgentRuntimeSessions.mockResolvedValue([
{
id: "session-recent",
name: "最近会话",
created_at: 1714000000,
updated_at: 1714000600,
archived_at: null,
workspace_id: "project-1",
},
]);
mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "claw",
projectId: "project-1",
} as AgentPageParams,
});
await flushEffects(2);
await openConversationMenu("最近会话");
await clickConversationMenuItem("app-sidebar-conversation-menu-rename");
await flushEffects(2);
expect(window.prompt).toHaveBeenCalledWith("重命名对话", "最近会话");
expect(mockUpdateAgentRuntimeSession).toHaveBeenCalledWith({
session_id: "session-recent",
name: "重命名后的会话",
});
expect(mockToastSuccess).toHaveBeenCalledWith("已重命名对话");
});
it("会话菜单应支持删除并在执行前要求确认", async () => {
vi.spyOn(window, "confirm").mockReturnValue(true);
mockListAgentRuntimeSessions.mockResolvedValue([
{
id: "session-recent",
name: "最近会话",
created_at: 1714000000,
updated_at: 1714000600,
archived_at: null,
workspace_id: "project-1",
},
]);
mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "claw",
projectId: "project-1",
} as AgentPageParams,
});
await flushEffects(2);
await openConversationMenu("最近会话");
await clickConversationMenuItem("app-sidebar-conversation-menu-delete");
await flushEffects(2);
expect(window.confirm).toHaveBeenCalledWith(
"确定要删除“最近会话”吗？删除后无法恢复。",
);
expect(mockDeleteAgentRuntimeSession).toHaveBeenCalledWith(
"session-recent",
);
expect(mockToastSuccess).toHaveBeenCalledWith("已删除对话");
});
it("会话菜单的收藏与多选应提供即时反馈", async () => {
mockListAgentRuntimeSessions.mockResolvedValue([
{
id: "session-recent",
@@ -1472,25 +2122,40 @@ describe("AppSidebar", () => {
});
await flushEffects(2);
const archiveButton = container.querySelector(
'button[aria-label="归档 最近会话"]',
) as HTMLButtonElement | null;
await openConversationMenu("最近会话");
await clickConversationMenuItem("app-sidebar-conversation-menu-favorite");
expect(archiveButton).not.toBeNull();
expect(mockListAgentRuntimeSessions).toHaveBeenCalledWith({
limit: 37,
workspaceId: "project-1",
});
expect(
container.querySelector(
'[data-testid="app-sidebar-conversation-favorite-badge"]',
),
).not.toBeNull();
const favoriteMenu = await openConversationMenu("最近会话");
expect(favoriteMenu?.textContent).toContain("取消收藏");
await clickConversationMenuItem(
"app-sidebar-conversation-menu-multiselect",
);
expect(
container.querySelector(
'[data-testid="app-sidebar-conversation-multiselect-toolbar"]',
)?.textContent,
).toContain("已选择 1 个对话");
await act(async () => {
archiveButton?.click();
container
.querySelector<HTMLButtonElement>('button[title="最近会话"]')
?.click();
await Promise.resolve();
});
expect(mockUpdateAgentRuntimeSession).toHaveBeenCalledWith({
session_id: "session-recent",
archived: true,
});
expect(
container.querySelector(
'[data-testid="app-sidebar-conversation-multiselect-toolbar"]',
)?.textContent,
).toContain("已选择 0 个对话");
});
it("显式开启后应显示可选系统扩展入口", async () => {
@@ -1669,14 +2334,22 @@ describe("AppSidebar", () => {
expect(popover?.textContent).toContain("浅色");
expect(popover?.textContent).toContain("深色");
expect(popover?.textContent).toContain("跟随系统");
expect(popover?.textContent).toContain("Lime 经典");
expect(popover?.textContent).toContain("森林");
expect(popover?.textContent).toContain("海雾");
expect(popover?.textContent).toContain("砂岩");
expect(popover?.textContent).toContain("随机");
expect(popover?.textContent).toContain("墨绿");
expect(popover?.textContent).toContain("自然");
expect(popover?.textContent).toContain("海洋");
expect(popover?.textContent).toContain("复古");
expect(popover?.textContent).toContain("霓虹");
expect(popover?.textContent).toContain("青柠");
expect(popover?.textContent).toContain("黄昏");
expect(popover?.textContent).toContain("极简");
expect(popover?.textContent).toContain("活力");
expect(popover?.textContent).toContain("文艺");
expect(popover?.textContent).toContain("奢华");
await act(async () => {
container
.querySelector<HTMLButtonElement>('button[aria-label="切换配色为海雾"]')
.querySelector<HTMLButtonElement>('button[aria-label="切换配色为海洋"]')
?.click();
await Promise.resolve();
});
@@ -1698,6 +2371,43 @@ describe("AppSidebar", () => {
expect(document.documentElement.classList.contains("dark")).toBe(true);
});
it("外观弹层的随机配色应持久化到一个真实预设", async () => {
const randomSpy = vi.spyOn(Math, "random").mockReturnValue(0);
try {
const container = mountSidebarContainer({
currentPage: "agent",
currentPageParams: {
agentEntry: "new-task",
} as AgentPageParams,
});
await flushEffects(2);
await act(async () => {
container
.querySelector<HTMLButtonElement>('button[aria-label="快速切换外观"]')
?.click();
await Promise.resolve();
});
await act(async () => {
container
.querySelector<HTMLButtonElement>('button[aria-label="随机切换配色"]')
?.click();
await Promise.resolve();
});
expect(localStorage.getItem(LIME_COLOR_SCHEME_STORAGE_KEY)).toBe(
"lime-forest",
);
expect(document.documentElement.dataset.limeColorScheme).toBe(
"lime-forest",
);
} finally {
randomSpy.mockRestore();
}
});
it("桌宠入口开启后，进入 companion 视图应高亮桌宠", async () => {
mockGetConfig.mockResolvedValue({
navigation: {
src/components/AppSidebar.tsx
+707 -18
View File
@@ -12,6 +12,7 @@ import {
useRef,
useCallback,
type ReactElement,
type MouseEvent as ReactMouseEvent,
} from "react";
import styled from "styled-components";
import {
@@ -27,10 +28,13 @@ import {
Languages,
LogIn,
LogOut,
MessageSquare,
MessageSquarePlus,
Monitor,
Moon,
Palette,
RefreshCw,
Shuffle,
Sun,
Search,
PanelLeftClose,
@@ -54,10 +58,12 @@ import {
buildHomeAgentParams,
} from "@/lib/workspace/navigation";
import {
notifyTaskCenterTaskPrefetch,
notifyTaskCenterTaskOpen,
requestTaskCenterDraftTask,
} from "@/components/agent/chat/taskCenterDraftTaskEvents";
import {
deleteAgentRuntimeSession,
listAgentRuntimeSessions,
updateAgentRuntimeSession,
type AsterSessionInfo,
@@ -81,6 +87,10 @@ import { hasTauriInvokeCapability } from "@/lib/tauri-runtime";
import { LIME_BRAND_LOGO_SRC, LIME_BRAND_NAME } from "@/lib/branding";
import { scheduleMinimumDelayIdleTask } from "@/lib/utils/scheduleMinimumDelayIdleTask";
import { AppSidebarConversationShelf } from "@/components/app-sidebar/AppSidebarConversationShelf";
import {
formatSidebarSessionMeta,
resolveSidebarSessionTitle,
} from "@/components/app-sidebar/sidebarSessionFormatting";
import { shouldReserveMacWindowControls } from "@/lib/windowControls";
import {
clearStoredOemCloudSessionState,
@@ -149,6 +159,7 @@ interface AppSidebarProps {
requestedPage?: Page;
requestedPageParams?: PageParams;
onNavigate: (page: Page, params?: PageParams) => void;
onStartWindowDrag?: (event: ReactMouseEvent<HTMLElement>) => void;
}
type SidebarNavItem = SidebarNavItemDefinition;
@@ -158,8 +169,8 @@ const SIDEBAR_PLUGIN_CENTER_NAV_ITEM_ID = "plugins";
const SIDEBAR_PLUGIN_IDLE_TIMEOUT_MS = 1200;
const SIDEBAR_PLUGIN_BROWSER_IDLE_TIMEOUT_MS = 6000;
const SIDEBAR_RECENT_SESSION_PAGE_SIZE = 10;
const SIDEBAR_RECENT_SESSION_PREFETCH_LIMIT = 37;
const SIDEBAR_ARCHIVED_SESSION_PAGE_SIZE = 8;
const SIDEBAR_SEARCH_RESULT_LIMIT = 8;
const SIDEBAR_SESSION_ENTRY_REFRESH_DEFER_MS = 12_000;
const APP_SIDEBAR_LANGUAGE_OPTIONS: Array<{
@@ -184,7 +195,7 @@ function buildSidebarSessionRequestLimit(
pageSize: number,
): number {
const normalizedVisibleCount = Math.max(visibleCount, pageSize);
return normalizedVisibleCount + pageSize + 1;
return normalizedVisibleCount + 1;
}
function splitSidebarSessionResult(params: {
@@ -196,7 +207,7 @@ function splitSidebarSessionResult(params: {
hasMore: boolean;
} {
const { sessions, visibleCount, pageSize } = params;
const targetCount = Math.max(visibleCount, pageSize) + pageSize;
const targetCount = Math.max(visibleCount, pageSize);
return {
sessions: sessions.slice(0, targetCount),
hasMore: sessions.length > targetCount,
@@ -496,6 +507,292 @@ const SearchButton = styled.button<{ $collapsed?: boolean }>`
}
`;
const SidebarSearchSurface = styled.div`
display: flex;
min-height: min(620px, calc(100vh - 96px));
max-height: calc(100vh - 96px);
flex-direction: column;
overflow: hidden;
border-radius: 28px;
border: 1px solid var(--lime-card-subtle-border, rgba(226, 240, 226, 0.9));
background: var(--lime-surface, #ffffff);
color: var(--lime-text, #1a3b2b);
box-shadow:
0 32px 80px rgba(15, 23, 42, 0.18),
0 1px 0 rgba(255, 255, 255, 0.78) inset;
`;
const SidebarSearchHeader = styled.div`
display: flex;
align-items: center;
gap: 12px;
padding: 22px 26px 18px;
color: var(--lime-text-muted, #6b826b);
svg {
width: 22px;
height: 22px;
flex-shrink: 0;
}
`;
const SidebarSearchInput = styled.input`
min-width: 0;
flex: 1;
border: none;
background: transparent;
color: var(--lime-text-strong, #0f172a);
font-size: 19px;
font-weight: 650;
outline: none;
&::placeholder {
color: var(--lime-text-soft, #9aa89a);
font-weight: 600;
}
`;
const SidebarSearchShortcut = styled.div`
display: inline-flex;
align-items: center;
gap: 4px;
flex-shrink: 0;
`;
const SidebarSearchKey = styled.kbd`
min-width: 28px;
height: 28px;
border-radius: 9px;
border: 1px solid var(--lime-card-subtle-border, rgba(226, 240, 226, 0.86));
background: var(--lime-muted-surface, #f5f8f3);
color: var(--lime-text-muted, #6b826b);
display: inline-flex;
align-items: center;
justify-content: center;
padding: 0 7px;
font-family: inherit;
font-size: 13px;
font-weight: 800;
line-height: 1;
box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.76);
`;
const SidebarSearchCloseButton = styled.button`
width: 34px;
height: 34px;
border: none;
border-radius: 12px;
display: inline-flex;
align-items: center;
justify-content: center;
background: transparent;
color: var(--lime-text-muted, #6b826b);
cursor: pointer;
transition:
background-color 0.16s ease,
color 0.16s ease;
&:hover {
background: var(--lime-surface-hover, #f4fdf4);
color: var(--lime-text-strong, #0f172a);
}
svg {
width: 22px;
height: 22px;
}
`;
const SidebarSearchDivider = styled.div`
height: 1px;
margin: 0 26px;
background: var(--lime-divider-subtle, rgba(226, 240, 226, 0.86));
`;
const SidebarSearchBody = styled.div`
display: flex;
min-height: 0;
flex: 1;
flex-direction: column;
gap: 16px;
overflow-y: auto;
padding: 20px 26px 28px;
&::-webkit-scrollbar {
width: 6px;
}
&::-webkit-scrollbar-track {
background: transparent;
}
&::-webkit-scrollbar-thumb {
border-radius: 999px;
background: var(--lime-divider-strong, rgba(180, 196, 180, 0.7));
}
`;
const SidebarSearchCreateButton = styled.button`
min-height: 58px;
width: 100%;
border: 1px solid transparent;
border-radius: 18px;
background: var(--lime-surface-hover, #f4f7f2);
color: var(--lime-text-strong, #0f172a);
display: flex;
align-items: center;
gap: 12px;
padding: 0 18px;
cursor: pointer;
text-align: left;
transition:
border-color 0.16s ease,
background-color 0.16s ease,
transform 0.16s ease;
&:hover {
border-color: var(--lime-card-subtle-border, #bbf7d0);
background: var(--lime-surface, #ffffff);
transform: translateY(-1px);
}
svg {
width: 22px;
height: 22px;
flex-shrink: 0;
color: var(--lime-text-muted, #6b826b);
}
`;
const SidebarSearchCreateText = styled.span`
flex: 1;
min-width: 0;
font-size: 16px;
font-weight: 760;
`;
const SidebarSearchEnterHint = styled.span`
flex-shrink: 0;
color: var(--lime-text-soft, #9aa89a);
font-size: 20px;
font-weight: 800;
`;
const SidebarSearchSectionLabel = styled.div`
padding: 0 16px;
color: var(--lime-text-soft, #9aa89a);
font-size: 13px;
font-weight: 760;
`;
const SidebarSearchResultList = styled.div`
display: flex;
flex-direction: column;
gap: 4px;
`;
const SidebarSearchResultButton = styled.button<{ $active?: boolean }>`
width: 100%;
min-height: 54px;
border: 1px solid
${({ $active }) =>
$active
? "var(--lime-card-subtle-border, #bbf7d0)"
: "transparent"};
border-radius: 16px;
background: ${({ $active }) =>
$active ? "var(--lime-surface-hover, #f4fdf4)" : "transparent"};
color: var(--lime-text, #1a3b2b);
display: flex;
align-items: center;
gap: 12px;
padding: 0 16px;
cursor: pointer;
text-align: left;
transition:
border-color 0.16s ease,
background-color 0.16s ease,
color 0.16s ease;
&:hover {
border-color: var(--lime-card-subtle-border, rgba(187, 247, 208, 0.92));
background: var(--lime-surface-hover, #f4fdf4);
color: var(--lime-text-strong, #0f172a);
}
svg {
width: 20px;
height: 20px;
flex-shrink: 0;
color: var(--lime-text-muted, #6b826b);
}
`;
const SidebarSearchResultTitle = styled.span`
flex: 1;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-size: 15px;
font-weight: 700;
`;
const SidebarSearchResultMeta = styled.span`
flex-shrink: 0;
color: var(--lime-text-soft, #9aa89a);
font-size: 14px;
font-weight: 650;
`;
const SidebarSearchEmptyState = styled.div`
display: flex;
min-height: 180px;
align-items: center;
justify-content: center;
border-radius: 18px;
border: 1px dashed var(--lime-card-subtle-border, rgba(226, 240, 226, 0.9));
color: var(--lime-text-muted, #6b826b);
background: var(--lime-muted-surface, #f8faf7);
font-size: 14px;
font-weight: 680;
`;
const SidebarSearchMoreButton = styled.button`
min-height: 42px;
border: 1px solid var(--lime-card-subtle-border, rgba(226, 240, 226, 0.86));
border-radius: 14px;
background: var(--lime-surface, #ffffff);
color: var(--lime-text-muted, #6b826b);
display: inline-flex;
align-items: center;
justify-content: center;
gap: 7px;
cursor: pointer;
font-size: 13px;
font-weight: 760;
transition:
border-color 0.16s ease,
background-color 0.16s ease,
color 0.16s ease;
&:hover:not(:disabled) {
border-color: var(--lime-card-subtle-border, #bbf7d0);
background: var(--lime-surface-hover, #f4fdf4);
color: var(--lime-text-strong, #0f172a);
}
&:disabled {
cursor: not-allowed;
opacity: 0.62;
}
svg {
width: 15px;
height: 15px;
}
`;
const MenuScroll = styled.div`
flex: 1;
min-height: 0;
@@ -964,8 +1261,10 @@ const AppearancePopover = styled.div`
left: calc(100% + 12px);
bottom: -2px;
z-index: 70;
width: 228px;
max-width: min(228px, calc(100vw - 24px));
width: 252px;
max-width: min(252px, calc(100vw - 24px));
max-height: min(560px, calc(100vh - 24px));
overflow-y: auto;
border-radius: 18px;
border: 1px solid var(--lime-card-subtle-border, rgba(226, 240, 226, 0.92));
background: var(--lime-card-subtle, var(--lime-surface, #ffffff));
@@ -1127,6 +1426,43 @@ const ColorSchemeList = styled.div`
gap: 6px;
`;
const RandomColorSchemeButton = styled.button`
grid-column: 1 / -1;
min-height: 40px;
border-radius: 13px;
border: 1px solid var(--lime-card-subtle-border, rgba(226, 240, 226, 0.82));
background: var(--lime-surface, #ffffff);
color: var(--lime-text, #1a3b2b);
cursor: pointer;
display: inline-flex;
align-items: center;
justify-content: center;
gap: 7px;
padding: 8px 10px;
font-size: 12px;
font-weight: 750;
transition:
border-color 0.16s ease,
background 0.16s ease,
color 0.16s ease;
&:hover {
border-color: var(--lime-card-subtle-border, #bbf7d0);
background: var(
--lime-chrome-tab-hover,
var(--lime-surface-hover, #f4fdf4)
);
color: var(--lime-text-strong, #0f172a);
}
svg {
width: 14px;
height: 14px;
flex-shrink: 0;
color: var(--lime-brand-strong, #166534);
}
`;
const ColorSchemeButton = styled.button<{ $active?: boolean }>`
display: flex;
position: relative;
@@ -1824,6 +2160,23 @@ function buildVisibleSidebarSessions(params: {
return [...visibleSessions.slice(0, Math.max(limit - 1, 0)), currentSession];
}
function normalizeSidebarSearchText(value: string): string {
return value.trim().toLocaleLowerCase();
}
function matchesSidebarSessionTitle(
session: AsterSessionInfo,
normalizedQuery: string,
): boolean {
if (!normalizedQuery) {
return true;
}
return normalizeSidebarSearchText(
resolveSidebarSessionTitle(session),
).includes(normalizedQuery);
}
function resolveAccountDisplayName(
sessionState: OemCloudStoredSessionState | null,
): string {
@@ -1954,6 +2307,7 @@ export function AppSidebar({
requestedPage,
requestedPageParams,
onNavigate,
onStartWindowDrag,
}: AppSidebarProps) {
const activePage = requestedPage ?? currentPage;
const activePageParams = requestedPageParams ?? currentPageParams;
@@ -2033,6 +2387,8 @@ export function AppSidebar({
const [inviteLoading, setInviteLoading] = useState(false);
const [inviteError, setInviteError] = useState<string | null>(null);
const [inviteReloadKey, setInviteReloadKey] = useState(0);
const [sidebarSearchOpen, setSidebarSearchOpen] = useState(false);
const [sidebarSearchQuery, setSidebarSearchQuery] = useState("");
const { setLanguage: setI18nLanguage } = useI18nPatch();
const [enabledNavItems, setEnabledNavItems] = useState<string[]>(
@@ -2064,6 +2420,7 @@ export function AppSidebar({
useState(SIDEBAR_ARCHIVED_SESSION_PAGE_SIZE);
const [archivedSessionsCollapsed, setArchivedSessionsCollapsed] =
useState(true);
const sidebarSearchInputRef = useRef<HTMLInputElement | null>(null);
const appearanceControlRef = useRef<HTMLDivElement | null>(null);
const accountControlRef = useRef<HTMLDivElement | null>(null);
const reserveWindowControls = shouldReserveMacWindowControls();
@@ -2077,6 +2434,56 @@ export function AppSidebar({
currentSessionId,
);
const openSidebarSearchDialog = useCallback(() => {
setAccountMenuOpen(false);
setLanguageMenuOpen(false);
setAppearancePopoverOpen(false);
setSidebarSearchOpen(true);
}, []);
const closeSidebarSearchDialog = useCallback(() => {
setSidebarSearchOpen(false);
setSidebarSearchQuery("");
}, []);
useEffect(() => {
if (typeof window === "undefined") {
return;
}
const handleSearchShortcut = (event: KeyboardEvent) => {
if (
event.key.toLowerCase() !== "k" ||
(!event.metaKey && !event.ctrlKey)
) {
return;
}
event.preventDefault();
openSidebarSearchDialog();
};
window.addEventListener("keydown", handleSearchShortcut);
return () => {
window.removeEventListener("keydown", handleSearchShortcut);
};
}, [openSidebarSearchDialog]);
useEffect(() => {
if (!sidebarSearchOpen || typeof window === "undefined") {
return;
}
const focusTimer = window.setTimeout(() => {
sidebarSearchInputRef.current?.focus();
sidebarSearchInputRef.current?.select();
}, 0);
return () => {
window.clearTimeout(focusTimer);
};
}, [sidebarSearchOpen]);
useEffect(() => {
const loadNavConfig = async () => {
try {
@@ -2499,7 +2906,8 @@ export function AppSidebar({
!collapsed &&
!(activePage === "agent" && activeAgentPageParams?.immersiveHome);
const shouldLoadWorkspaceScopedConversations =
shouldShowConversationList && Boolean(currentProjectId);
(shouldShowConversationList || sidebarSearchOpen) &&
Boolean(currentProjectId);
const shouldShowSessionLoadingState =
sidebarSessionsLoading && sidebarSessions.length === 0;
const shouldShowArchivedSessionLoadingState =
@@ -2522,12 +2930,10 @@ export function AppSidebar({
}, [currentProjectId]);
const recentSessionRequestLimit = useMemo(() => {
const requestLimit = buildSidebarSessionRequestLimit(
return buildSidebarSessionRequestLimit(
recentSessionsVisibleCount,
SIDEBAR_RECENT_SESSION_PAGE_SIZE,
);
return Math.max(requestLimit, SIDEBAR_RECENT_SESSION_PREFETCH_LIMIT);
}, [recentSessionsVisibleCount]);
const archivedSessionRequestLimit = useMemo(
() =>
@@ -2810,6 +3216,45 @@ export function AppSidebar({
const hasMoreArchivedSidebarSessions =
archivedSessionEntriesHasMore ||
archivedSessionsVisibleCount < archivedSidebarSessions.length;
const normalizedSidebarSearchQuery = useMemo(
() => normalizeSidebarSearchText(sidebarSearchQuery),
[sidebarSearchQuery],
);
const sidebarSearchResultLimit = Math.max(
recentSessionsVisibleCount,
SIDEBAR_SEARCH_RESULT_LIMIT,
);
const sidebarSearchMatchedSessions = useMemo(() => {
if (!normalizedSidebarSearchQuery) {
return recentSidebarSessions;
}
return recentSidebarSessions.filter((session) =>
matchesSidebarSessionTitle(session, normalizedSidebarSearchQuery),
);
}, [normalizedSidebarSearchQuery, recentSidebarSessions]);
const sidebarSearchResultSessions = useMemo(() => {
if (!normalizedSidebarSearchQuery) {
return buildVisibleSidebarSessions({
sessions: recentSidebarSessions,
currentSessionId,
limit: sidebarSearchResultLimit,
});
}
return sidebarSearchMatchedSessions.slice(0, sidebarSearchResultLimit);
}, [
currentSessionId,
normalizedSidebarSearchQuery,
recentSidebarSessions,
sidebarSearchMatchedSessions,
sidebarSearchResultLimit,
]);
const sidebarSearchHasQuery = normalizedSidebarSearchQuery.length > 0;
const sidebarSearchHasMoreResults = sidebarSearchHasQuery
? sidebarSessionsHasMore ||
sidebarSearchResultLimit < sidebarSearchMatchedSessions.length
: hasMoreRecentSidebarSessions;
const isActive = (item: SidebarNavItem): boolean => {
if (!item.page) {
@@ -2950,6 +3395,7 @@ export function AppSidebar({
workspaceId: session.workspace_id ?? currentProjectId ?? null,
source: "sidebar",
});
return;
}
const targetParams = buildClawAgentParams({
@@ -2976,7 +3422,19 @@ export function AppSidebar({
onNavigate(target.page, target.rawParams);
};
const handleNavigateToNewTask = () => {
const handlePrefetchConversation = (session: AsterSessionInfo) => {
if (!isAgentWorkspace) {
return;
}
notifyTaskCenterTaskPrefetch({
sessionId: session.id,
workspaceId: session.workspace_id ?? currentProjectId ?? null,
source: "conversation_shelf",
});
};
const handleNavigateToNewTask = useCallback(() => {
if (tryOpenTaskCenterDraftFromSidebar()) {
return;
}
@@ -3002,8 +3460,66 @@ export function AppSidebar({
requestedNavigationTargetRef.current = target;
onNavigate(target.page, target.rawParams);
}, [currentProjectId, onNavigate, tryOpenTaskCenterDraftFromSidebar]);
const handleSidebarSearchCreateConversation = () => {
closeSidebarSearchDialog();
handleNavigateToNewTask();
};
const handleSidebarSearchNavigateToConversation = (
session: AsterSessionInfo,
) => {
closeSidebarSearchDialog();
handleNavigateToConversation(session);
};
const handleRenameConversation = useCallback(
async (session: AsterSessionInfo) => {
const currentTitle = resolveSidebarSessionTitle(session);
const nextTitle = window.prompt("重命名对话", currentTitle)?.trim();
if (!nextTitle || nextTitle === currentTitle) {
return;
}
const nextUpdatedAt = Math.floor(Date.now() / 1000);
const nextSession = {
...session,
name: nextTitle,
updated_at: nextUpdatedAt,
} satisfies AsterSessionInfo;
setSidebarSessionActionId(session.id);
setSidebarSessions((current) =>
sortSidebarSessions(
current.map((item) => (item.id === session.id ? nextSession : item)),
),
);
setArchivedSessionEntries((current) =>
sortSidebarSessions(
current.map((item) => (item.id === session.id ? nextSession : item)),
),
);
try {
await updateAgentRuntimeSession({
session_id: session.id,
name: nextTitle,
});
toast.success("已重命名对话");
await refreshSidebarSessions();
} catch (error) {
console.error("重命名会话失败:", error);
toast.error("重命名失败，请稍后重试");
await refreshSidebarSessions();
} finally {
setSidebarSessionActionId((current) =>
current === session.id ? null : current,
);
}
},
[refreshSidebarSessions],
);
const handleToggleSessionArchive = useCallback(
async (session: AsterSessionInfo, archived: boolean) => {
const nextUpdatedAt = Math.floor(Date.now() / 1000);
@@ -3048,6 +3564,45 @@ export function AppSidebar({
[refreshSidebarSessions],
);
const handleDeleteConversation = useCallback(
async (session: AsterSessionInfo) => {
const title = resolveSidebarSessionTitle(session);
const confirmed = window.confirm(
`确定要删除“${title}”吗？删除后无法恢复。`,
);
if (!confirmed) {
return;
}
setSidebarSessionActionId(session.id);
setSidebarSessions((current) =>
current.filter((item) => item.id !== session.id),
);
setArchivedSessionEntries((current) =>
current.filter((item) => item.id !== session.id),
);
try {
await deleteAgentRuntimeSession(session.id);
toast.success("已删除对话");
if (currentSessionId === session.id) {
handleNavigateToNewTask();
} else {
await refreshSidebarSessions();
}
} catch (error) {
console.error("删除会话失败:", error);
toast.error("删除失败，请稍后重试");
await refreshSidebarSessions();
} finally {
setSidebarSessionActionId((current) =>
current === session.id ? null : current,
);
}
},
[currentSessionId, handleNavigateToNewTask, refreshSidebarSessions],
);
const currentColorScheme = getLimeColorScheme(colorSchemeId);
const currentThemeLabel =
LIME_THEME_MODE_OPTIONS.find((option) => option.id === themeState.themeMode)
@@ -3092,6 +3647,16 @@ export function AppSidebar({
[],
);
const handleRandomColorScheme = useCallback(() => {
const candidates = LIME_COLOR_SCHEMES.filter(
(scheme) => scheme.id !== colorSchemeId,
);
const nextScheme =
candidates[Math.floor(Math.random() * candidates.length)] ??
LIME_COLOR_SCHEMES[0];
handleColorSchemeChange(nextScheme.id);
}, [colorSchemeId, handleColorSchemeChange]);
const handleAccountMenuNavigate = useCallback(
(params: PageParams) => {
setAccountMenuOpen(false);
@@ -3240,7 +3805,9 @@ export function AppSidebar({
$themeMode={themeState.effectiveThemeMode}
$reserveWindowControls={reserveWindowControls}
data-testid="app-sidebar"
data-lime-window-drag-region
data-window-controls-reserved={String(reserveWindowControls)}
onMouseDown={onStartWindowDrag}
>
<HeaderArea $collapsed={collapsed} data-testid="app-sidebar-header">
<HeaderTopRow $collapsed={collapsed}>
@@ -3301,16 +3868,12 @@ export function AppSidebar({
{maybeWrapWithTooltip(
<SearchButton
$collapsed={collapsed}
onClick={() =>
onNavigate(
"agent",
buildHomeAgentParams({
projectId: currentProjectId ?? undefined,
}),
)
}
onClick={openSidebarSearchDialog}
title="搜索任务"
aria-label="搜索任务"
aria-haspopup="dialog"
aria-expanded={sidebarSearchOpen ? true : undefined}
data-testid="app-sidebar-search-button"
>
<Search size={14} />
<span>搜索任务</span>
@@ -3337,6 +3900,9 @@ export function AppSidebar({
actionSessionId={sidebarSessionActionId}
onCreateConversation={handleNavigateToNewTask}
onNavigateToConversation={handleNavigateToConversation}
onPrefetchConversation={handlePrefetchConversation}
onRenameConversation={handleRenameConversation}
onDeleteConversation={handleDeleteConversation}
onToggleArchive={(session, archived) => {
void handleToggleSessionArchive(session, archived);
}}
@@ -3461,6 +4027,15 @@ export function AppSidebar({
<AppearanceGroup>
<AppearanceGroupLabel>配色</AppearanceGroupLabel>
<ColorSchemeList>
<RandomColorSchemeButton
type="button"
aria-label="随机切换配色"
title="随机切换一个颜色主题"
onClick={handleRandomColorScheme}
>
<Shuffle />
<span>随机</span>
</RandomColorSchemeButton>
{LIME_COLOR_SCHEMES.map((scheme) => {
const active = scheme.id === colorSchemeId;
return (
@@ -3810,6 +4385,120 @@ export function AppSidebar({
</AccountActionSlot>
</FooterArea>
</Container>
<Modal
isOpen={sidebarSearchOpen}
onClose={closeSidebarSearchDialog}
className="border-none bg-transparent p-0 shadow-none"
maxWidth="max-w-[832px]"
showCloseButton={false}
>
<SidebarSearchSurface data-testid="app-sidebar-search-dialog">
<SidebarSearchHeader>
<Search aria-hidden="true" />
<SidebarSearchInput
ref={sidebarSearchInputRef}
value={sidebarSearchQuery}
onChange={(event) => setSidebarSearchQuery(event.target.value)}
placeholder="搜索对话标题"
aria-label="搜索对话标题"
data-testid="app-sidebar-search-input"
/>
<SidebarSearchShortcut aria-hidden="true">
<SidebarSearchKey>⌘</SidebarSearchKey>
<SidebarSearchKey>K</SidebarSearchKey>
</SidebarSearchShortcut>
<SidebarSearchCloseButton
type="button"
aria-label="关闭搜索弹窗"
onClick={closeSidebarSearchDialog}
>
<X />
</SidebarSearchCloseButton>
</SidebarSearchHeader>
<SidebarSearchDivider />
<SidebarSearchBody>
<SidebarSearchCreateButton
type="button"
onClick={handleSidebarSearchCreateConversation}
data-testid="app-sidebar-search-new-conversation"
>
<MessageSquarePlus />
<SidebarSearchCreateText>新建对话</SidebarSearchCreateText>
<SidebarSearchEnterHint aria-hidden="true">
↵
</SidebarSearchEnterHint>
</SidebarSearchCreateButton>
<SidebarSearchSectionLabel>
{sidebarSearchHasQuery ? "匹配结果" : "最近"}
</SidebarSearchSectionLabel>
{shouldShowSessionLoadingState ? (
<SidebarSearchEmptyState role="status">
正在加载对话
</SidebarSearchEmptyState>
) : sidebarSearchResultSessions.length > 0 ? (
<SidebarSearchResultList>
{sidebarSearchResultSessions.map((session) => {
const title = resolveSidebarSessionTitle(session);
const isCurrentConversation = currentSessionId === session.id;
return (
<SidebarSearchResultButton
key={session.id}
type="button"
$active={isCurrentConversation}
aria-current={
isCurrentConversation ? "page" : undefined
}
title={title}
data-testid="app-sidebar-search-result"
onClick={() =>
handleSidebarSearchNavigateToConversation(session)
}
>
<MessageSquare />
<SidebarSearchResultTitle>
{title}
</SidebarSearchResultTitle>
<SidebarSearchResultMeta>
{formatSidebarSessionMeta(session)}
</SidebarSearchResultMeta>
</SidebarSearchResultButton>
);
})}
</SidebarSearchResultList>
) : (
<SidebarSearchEmptyState role="status">
{!currentProjectId
? "请先选择项目工作区"
: sidebarSearchHasQuery
? "没有匹配的对话标题"
: "还没有最近对话"}
</SidebarSearchEmptyState>
)}
{sidebarSearchHasMoreResults ? (
<SidebarSearchMoreButton
type="button"
disabled={sidebarSessionsLoading}
onClick={() =>
setRecentSessionsVisibleCount(
(current) => current + SIDEBAR_RECENT_SESSION_PAGE_SIZE,
)
}
data-testid="app-sidebar-search-more"
>
{sidebarSessionsLoading
? "正在加载..."
: sidebarSearchHasQuery
? "查看更多匹配结果"
: "查看更多对话"}
<ChevronDown />
</SidebarSearchMoreButton>
) : null}
</SidebarSearchBody>
</SidebarSearchSurface>
</Modal>
<Modal
isOpen={inviteDialogOpen}
onClose={() => setInviteDialogOpen(false)}
src/components/agent/chat/AgentChatWorkspace.tsx
+135 -40
View File
@@ -184,6 +184,8 @@ import {
import { useWorkspaceImageWorkbenchEventRuntime } from "./workspace/useWorkspaceImageWorkbenchEventRuntime";
import { buildImageSkillLaunchRequestMetadata } from "./workspace/imageSkillLaunch";
import { useWorkspaceImageTaskPreviewRuntime } from "./workspace/useWorkspaceImageTaskPreviewRuntime";
import { useWorkspaceAudioTaskPreviewRuntime } from "./workspace/useWorkspaceAudioTaskPreviewRuntime";
import { useWorkspaceTranscriptionTaskPreviewRuntime } from "./workspace/useWorkspaceTranscriptionTaskPreviewRuntime";
import { useWorkspaceVideoTaskPreviewRuntime } from "./workspace/useWorkspaceVideoTaskPreviewRuntime";
import { useWorkspaceVideoTaskActionRuntime } from "./workspace/useWorkspaceVideoTaskActionRuntime";
import { useWorkspaceSessionRestore } from "./workspace/useWorkspaceSessionRestore";
@@ -209,7 +211,10 @@ import { useWorkspaceGeneralWorkbenchDocumentPersistenceRuntime } from "./worksp
import { useWorkspaceServiceSkillEntryActions } from "./workspace/useWorkspaceServiceSkillEntryActions";
import { useWorkspaceSceneAppEntryActions } from "./workspace/useWorkspaceSceneAppEntryActions";
import { useWorkspaceArtifactViewModeControl } from "./workspace/useWorkspaceArtifactViewModeControl";
import { useWorkspaceInitialSessionNavigation } from "./workspace/useWorkspaceInitialSessionNavigation";
import {
rememberInitialSessionNavigationStart,
useWorkspaceInitialSessionNavigation,
} from "./workspace/useWorkspaceInitialSessionNavigation";
import { WorkspaceGeneralWorkbenchSidebar } from "./workspace/WorkspaceGeneralWorkbenchSidebar";
import { GeneralWorkbenchHarnessDialogSection } from "./workspace/WorkspaceHarnessDialogs";
import { WorkspaceShellScene } from "./workspace/WorkspaceShellScene";
@@ -219,6 +224,7 @@ import {
} from "./components/TaskCenterTabStrip";
import {
subscribeTaskCenterDraftTaskRequests,
subscribeTaskCenterTaskPrefetchRequests,
subscribeTaskCenterTaskOpenRequests,
} from "./taskCenterDraftTaskEvents";
import type { GeneralWorkbenchFollowUpActionPayload } from "./components/generalWorkbenchSidebarContract";
@@ -2049,6 +2055,7 @@ export function AgentChatWorkspace({
createFreshSession,
ensureSession = async () => null,
switchTopic: originalSwitchTopic,
prefetchTopic = async () => false,
loadFullSessionHistory = async () => false,
deleteTopic,
renameTopic,
@@ -2071,6 +2078,10 @@ export function AgentChatWorkspace({
: undefined,
getSyncedSessionRecentPreferences,
});
const topicById = useMemo(
() => new Map(topics.map((topic) => [topic.id, topic])),
[topics],
);
activeSessionIdRef.current = sessionId;
const clawSidebarEntryResetKey = shouldAutoCollapseClassicClawSidebar
? JSON.stringify({
@@ -3535,23 +3546,23 @@ export function AgentChatWorkspace({
rememberProjectId,
getRememberedProjectId,
loadTopicBoundProjectId: (topicId) =>
topics.find((topic) => topic.id === topicId)?.workspaceId ||
topicById.get(topicId)?.workspaceId ||
loadPersistedSessionWorkspaceId(topicId) ||
loadPersistedProjectId(`agent_session_workspace_${topicId}`),
resetTopicLocalState,
});
const resolveInitialSessionSwitch = useCallback(
(topicId: string) => {
const topic = topics.find((item) => item.id === topicId);
const topic = topicById.get(topicId);
return {
allowDetachedSession: true,
forceRefresh: true,
forceRefresh: topic?.statusReason === "workspace_error",
...(shouldResumeTaskSession(topic)
? { resumeSessionStartHooks: true }
: {}),
};
},
[topics],
[topicById],
);
useWorkspaceInitialSessionNavigation({
initialSessionId,
@@ -3663,7 +3674,9 @@ export function AgentChatWorkspace({
);
setTaskCenterDraftTabs((current) => (current.length > 0 ? [] : current));
setActiveTaskCenterDraftTabId(null);
setTaskCenterLocalSessionOverride(null);
if (agentEntry !== "new-task") {
setTaskCenterLocalSessionOverride(null);
}
return;
}
@@ -3671,7 +3684,7 @@ export function AgentChatWorkspace({
return;
}
const hasTopicMatch = topics.some((topic) => topic.id === sessionId);
const hasTopicMatch = topicById.has(sessionId);
if (hasTopicMatch) {
setTaskCenterDetachedTopicId((current) =>
current === sessionId ? null : current,
@@ -3682,7 +3695,7 @@ export function AgentChatWorkspace({
setTaskCenterDetachedTopicId((current) =>
current === sessionId ? current : sessionId,
);
}, [agentEntry, normalizedInitialSessionId, sessionId, topics]);
}, [agentEntry, normalizedInitialSessionId, sessionId, topicById]);
useEffect(() => {
if (
@@ -3758,8 +3771,7 @@ export function AgentChatWorkspace({
}
const shouldWaitForInitialSessionTopic =
normalizedInitialSessionId &&
!topics.some((topic) => topic.id === normalizedInitialSessionId);
normalizedInitialSessionId && !topicById.has(normalizedInitialSessionId);
if (shouldWaitForInitialSessionTopic) {
return;
}
@@ -3800,11 +3812,12 @@ export function AgentChatWorkspace({
taskCenterLocalSessionOverride?.sessionId,
taskCenterDetachedTopicId,
taskCenterWorkspaceId,
topicById,
topics,
]);
useEffect(() => {
if (agentEntry !== "claw") {
if (agentEntry !== "claw" && agentEntry !== "new-task") {
return;
}
@@ -4136,6 +4149,7 @@ export function AgentChatWorkspace({
(hasDisplayMessages ||
isThemeWorkbench ||
(!shouldUseCompactGeneralWorkbench && isBootstrapDispatchPending) ||
isSessionHydrating ||
isSending ||
queuedTurns.length > 0));
const shouldRestoreImageTasksFromWorkspace = !(
@@ -4428,7 +4442,7 @@ export function AgentChatWorkspace({
replaceOpenTabs?: boolean;
},
) => {
const topic = topics.find((item) => item.id === topicId);
const topic = topicById.get(topicId);
const topicWorkspaceId = normalizeProjectId(
topic?.workspaceId ??
loadPersistedSessionWorkspaceId(topicId) ??
@@ -4445,18 +4459,9 @@ export function AgentChatWorkspace({
: undefined;
const wasOpenInTaskCenter =
taskCenterOpenTabIdsRef.current.includes(topicId);
setTaskCenterTransitionTopicId(topicId);
setTaskCenterDetachedTopicId(null);
setActiveTaskCenterDraftTabId(null);
if (options?.replaceOpenTabs === true) {
replaceTaskCenterOpenTabs(topicId, topicWorkspaceId);
} else if (agentEntry === "claw") {
upsertTaskCenterOpenTab(topicId, topicWorkspaceId);
}
markTaskCenterLocalSessionOverride(topicId);
const switchResult = await switchTopic(topicId, switchOptions);
if (switchResult !== "success" && switchResult !== "deferred") {
const shouldMaintainTaskCenterTab =
agentEntry === "claw" || agentEntry === "new-task";
const rollbackPendingOpen = () => {
if (!wasOpenInTaskCenter && options?.replaceOpenTabs !== true) {
setTaskCenterOpenTabMap((currentMap) =>
updateTaskCenterTabIdsForWorkspace(
@@ -4473,12 +4478,47 @@ export function AgentChatWorkspace({
setTaskCenterTransitionTopicId((current) =>
current === topicId ? null : current,
);
};
setTaskCenterTransitionTopicId(topicId);
setTaskCenterDetachedTopicId(null);
setActiveTaskCenterDraftTabId(null);
if (options?.replaceOpenTabs === true) {
replaceTaskCenterOpenTabs(topicId, topicWorkspaceId);
} else if (shouldMaintainTaskCenterTab) {
upsertTaskCenterOpenTab(topicId, topicWorkspaceId);
}
markTaskCenterLocalSessionOverride(topicId);
rememberInitialSessionNavigationStart(topicId);
const switchResult = await switchTopic(topicId, switchOptions);
if (switchResult === "busy") {
scheduleMinimumDelayIdleTask(
() => {
void switchTopic(topicId, switchOptions)
.then((retryResult) => {
if (retryResult !== "success" && retryResult !== "deferred") {
rollbackPendingOpen();
}
})
.catch(() => {
rollbackPendingOpen();
});
},
{
minimumDelayMs: 120,
idleTimeoutMs: 600,
},
);
return;
}
if (switchResult !== "success" && switchResult !== "deferred") {
rollbackPendingOpen();
return;
}
if (options?.replaceOpenTabs === true) {
replaceTaskCenterOpenTabs(topicId, topicWorkspaceId);
} else {
if (agentEntry === "claw") {
if (shouldMaintainTaskCenterTab) {
upsertTaskCenterOpenTab(topicId, topicWorkspaceId);
}
}
@@ -4490,9 +4530,9 @@ export function AgentChatWorkspace({
setActiveTaskCenterDraftTabId,
switchTopic,
taskCenterWorkspaceId,
topics,
setTaskCenterDetachedTopicId,
setTaskCenterTransitionTopicId,
topicById,
upsertTaskCenterOpenTab,
],
);
@@ -4503,6 +4543,7 @@ export function AgentChatWorkspace({
setTaskCenterDetachedTopicId(topicId);
setTaskCenterTransitionTopicId(topicId);
markTaskCenterLocalSessionOverride(topicId);
rememberInitialSessionNavigationStart(topicId);
const switchResult = await switchTopic(topicId, {
allowDetachedSession: true,
});
@@ -4684,20 +4725,54 @@ export function AgentChatWorkspace({
});
}, [agentEntry, handleOpenTaskCenterNewTaskPage]);
useEffect(() => {
if (agentEntry !== "claw") {
return subscribeTaskCenterTaskPrefetchRequests(
({ sessionId: requestedSessionId, workspaceId }) => {
const requestedWorkspaceId = normalizeProjectId(workspaceId);
if (
requestedWorkspaceId &&
taskCenterWorkspaceId &&
requestedWorkspaceId !== normalizeProjectId(taskCenterWorkspaceId)
) {
return;
}
void prefetchTopic(requestedSessionId);
},
);
}, [prefetchTopic, taskCenterWorkspaceId]);
useEffect(() => {
if (agentEntry !== "claw" && agentEntry !== "new-task") {
return;
}
return subscribeTaskCenterTaskOpenRequests(
({ sessionId: requestedSessionId, workspaceId }) => {
setTaskCenterTransitionTopicId(requestedSessionId);
setTaskCenterDetachedTopicId(null);
setActiveTaskCenterDraftTabId(null);
markTaskCenterLocalSessionOverride(requestedSessionId);
upsertTaskCenterOpenTab(requestedSessionId, workspaceId);
const requestedWorkspaceId = normalizeProjectId(workspaceId);
if (
requestedWorkspaceId &&
requestedWorkspaceId !== normalizeProjectId(taskCenterWorkspaceId)
) {
setTaskCenterTransitionTopicId(requestedSessionId);
setTaskCenterDetachedTopicId(null);
setActiveTaskCenterDraftTabId(null);
markTaskCenterLocalSessionOverride(requestedSessionId);
upsertTaskCenterOpenTab(requestedSessionId, requestedWorkspaceId);
deferTopicSwitch(requestedSessionId, requestedWorkspaceId);
return;
}
void handleOpenTaskTopic(requestedSessionId);
},
);
}, [agentEntry, markTaskCenterLocalSessionOverride, upsertTaskCenterOpenTab]);
}, [
agentEntry,
deferTopicSwitch,
handleOpenTaskTopic,
markTaskCenterLocalSessionOverride,
taskCenterWorkspaceId,
upsertTaskCenterOpenTab,
]);
const taskCenterPreviewTopicId = useMemo(
() =>
resolveTaskCenterPreviewTopicId({
@@ -4800,7 +4875,7 @@ export function AgentChatWorkspace({
isPinned: false,
}));
const topicItems = taskCenterVisibleTabIds
.map((topicId) => topics.find((topic) => topic.id === topicId))
.map((topicId) => topicById.get(topicId))
.filter((topic): topic is NonNullable<typeof topic> => Boolean(topic))
.map((topic) => ({
id: topic.id,
@@ -4825,8 +4900,13 @@ export function AgentChatWorkspace({
taskCenterDraftTabs,
taskCenterPreviewTopicId,
taskCenterVisibleTabIds,
topics,
]);
topicById,
]);
const shouldRenderTaskCenterTabStrip =
agentEntry === "claw" ||
(agentEntry === "new-task" &&
taskCenterLocalSessionOverride !== null &&
taskCenterTabItems.length > 0);
useEffect(() => {
if (
agentEntry !== "claw" ||
@@ -4857,7 +4937,7 @@ export function AgentChatWorkspace({
}
const currentSessionIsKnownTopic = Boolean(
sessionId && topics.some((topic) => topic.id === sessionId),
sessionId && topicById.has(sessionId),
);
if (
!normalizedInitialSessionId &&
@@ -4913,10 +4993,11 @@ export function AgentChatWorkspace({
taskCenterTransitionTopicId,
taskCenterVisibleTabIds,
taskCenterWorkspaceId,
topicById,
topics,
]);
const taskCenterTabsNode = useMemo(() => {
if (agentEntry !== "claw") {
if (!shouldRenderTaskCenterTabStrip) {
return null;
}
@@ -4938,13 +5019,13 @@ export function AgentChatWorkspace({
/>
);
}, [
agentEntry,
handleCloseTaskCenterTab,
handleOpenTaskCenterNewTaskPage,
handleToggleCanvas,
handleSwitchTaskTopic,
isThemeWorkbench,
layoutMode,
shouldRenderTaskCenterTabStrip,
taskCenterTabItems,
]);
const browserWorkspaceHomeTabsNode = useMemo(() => {
@@ -6485,6 +6566,16 @@ export function AgentChatWorkspace({
messages,
setChatMessages,
});
useWorkspaceAudioTaskPreviewRuntime({
projectRootPath: project?.rootPath || null,
messages,
setChatMessages,
});
useWorkspaceTranscriptionTaskPreviewRuntime({
projectRootPath: project?.rootPath || null,
messages,
setChatMessages,
});
useWorkspaceVideoTaskActionRuntime({
projectId,
contentId,
@@ -6504,6 +6595,7 @@ export function AgentChatWorkspace({
hasUnconsumedInitialDispatch,
isPreparingSend,
isSending,
isSessionHydrating,
queuedTurnCount: queuedTurns.length,
});
@@ -6559,6 +6651,7 @@ export function AgentChatWorkspace({
isBootstrapDispatchPending,
isPreparingSend,
isSending,
isSessionHydrating,
isSpecializedThemeMode,
isThemeWorkbench,
layoutMode,
@@ -7175,7 +7268,9 @@ export function AgentChatWorkspace({
handleResumeRecentSession,
handleOpenSceneAppsDirectory,
taskCenterTabsNode:
agentEntry === "claw" ? taskCenterTabsNode : browserWorkspaceHomeTabsNode,
shouldRenderTaskCenterTabStrip
? taskCenterTabsNode
: browserWorkspaceHomeTabsNode,
suppressNavbarUtilityActions: suppressHomeNavbarUtilityActions,
hideHistoryToggle,
showChatPanel: effectiveShowChatPanel,
src/components/agent/chat/components/EmptyState.test.tsx
+182 -175
View File
@@ -9,7 +9,6 @@ import type { UnifiedMemory } from "@/lib/api/unifiedMemory";
import type { ServiceSkillHomeItem } from "../service-skills/types";
import type { SceneAppEntryCardItem } from "../sceneappEntryTypes";
import type { InputCapabilitySelection } from "../skill-selection/inputCapabilitySelection";
import { recordServiceSkillUsage } from "../service-skills/storage";
import { recordSlashEntryUsage } from "../skill-selection/slashEntryUsage";
import {
buildCuratedTaskLaunchPrompt,
@@ -89,7 +88,8 @@ vi.mock("@/lib/api/unifiedMemory", () => ({
}));
vi.mock("@/lib/api/skillCatalog", async (importOriginal) => {
const actual = await importOriginal<typeof import("@/lib/api/skillCatalog")>();
const actual =
await importOriginal<typeof import("@/lib/api/skillCatalog")>();
return {
...actual,
getSkillCatalog: mockGetSkillCatalog,
@@ -397,6 +397,27 @@ async function openHomeMoreSkillsDrawer(
return drawer;
}
function queryHomeDrawerButton(
drawer: HTMLElement,
testId: string,
): HTMLButtonElement | null {
return drawer.querySelector(
`[data-testid="${testId}"]`,
) as HTMLButtonElement | null;
}
function expectHomeDrawerButton(
drawer: HTMLElement,
testId: string,
): HTMLButtonElement {
const button = queryHomeDrawerButton(drawer, testId);
expect(button).toBeTruthy();
if (!button) {
throw new Error(`expected home drawer button ${testId}`);
}
return button;
}
function createGithubSearchServiceSkill(): ServiceSkillHomeItem {
return {
id: "github-repo-radar",
@@ -509,7 +530,7 @@ describe("EmptyState", () => {
expect(container.textContent).not.toContain("新建任务");
});
it("通用首页应展示轻量起手入口与下滑第二屏", async () => {
it("通用首页应挂载轻量起手区与下滑第二屏", async () => {
const container = renderEmptyState({
activeTheme: "general",
onLaunchBrowserAssist: vi.fn(),
@@ -526,16 +547,21 @@ describe("EmptyState", () => {
expect(
container.querySelector('[data-testid="home-starter-chips"]'),
).toBeTruthy();
expect(container.textContent).toContain("帮我想选题");
expect(container.textContent).toContain("写第一版");
expect(container.textContent).toContain("拆解爆款");
expect(container.textContent).toContain("改成我的风格");
expect(container.textContent).toContain("转成视频脚本");
expect(container.textContent).toContain("复盘账号");
expect(container.textContent).toContain("引导帮助");
expect(container.textContent).toContain("写作");
expect(container.textContent).toContain("PPT");
expect(container.textContent).toContain("调研报告");
expect(container.textContent).toContain("更多做法");
expect(
container.querySelector('[data-testid="home-scroll-cue"]'),
).toBeTruthy();
container.querySelector('[data-testid="home-guide-cards"]'),
).toBeNull();
const scrollCue = container.querySelector(
'[data-testid="home-scroll-cue"]',
);
expect(scrollCue).toBeTruthy();
expect(scrollCue?.textContent).toContain(
"向下滑，看看 Lime 可以帮你做什么",
);
expect(
container.querySelector('[data-testid="home-second-screen"]'),
).toBeTruthy();
@@ -544,43 +570,96 @@ describe("EmptyState", () => {
).toBeTruthy();
expect(container.textContent).toContain("你可以从这些任务开始");
expect(container.textContent).toContain("每日趋势摘要");
expect(container.textContent).toContain("内容主稿生成");
expect(container.textContent).toContain("拆解一条爆款内容");
expect(container.textContent).toContain("长文转多平台发布稿");
expect(container.textContent).toContain("脚本转口播/字幕稿");
expect(container.textContent).toContain("复盘这个账号/项目");
expect(container.textContent).toContain("复制轮播帖");
expect(container.textContent).toContain("复制视频脚本");
expect(container.textContent).not.toContain("先开始这一轮");
expect(container.textContent).not.toContain("其他起手结果");
expect(container.textContent).not.toContain("也可以直接按做法开工");
expect(container.textContent).not.toContain("首选结果");
expect(container.textContent).not.toContain("更多结果");
expect(container.textContent).not.toContain("快捷做法");
expect(container.textContent).not.toContain("任务拆分冒烟测试");
expect(container.textContent).not.toContain("推荐方案");
expect(container.textContent).not.toContain("通用对话");
const recommendationItems = Array.from(
container.querySelectorAll(
'[data-testid^="home-gallery-entry-recommended-"], [data-testid^="home-gallery-entry-service-skill-"]',
),
).map((element) => element.textContent?.trim() ?? "");
expect(recommendationItems.length).toBeGreaterThanOrEqual(8);
expect(recommendationItems.join(" ")).toContain("每日趋势摘要");
expect(recommendationItems.join(" ")).toContain("内容主稿生成");
expect(recommendationItems.join(" ")).toContain("拆解一条爆款内容");
expect(recommendationItems.join(" ")).toContain("长文转多平台发布稿");
expect(recommendationItems.join(" ")).toContain("脚本转口播/字幕稿");
expect(recommendationItems.join(" ")).toContain("复盘这个账号/项目");
expect(recommendationItems.join(" ")).toContain("复制轮播帖");
expect(recommendationItems.join(" ")).toContain("复制视频脚本");
expect(
container.querySelector('[data-testid^="entry-continuation-"]'),
).toBeNull();
});
it("点击引导帮助后进入可关闭的帮助模式，关闭后恢复默认起手入口", async () => {
const container = renderEmptyState({
activeTheme: "general",
serviceSkills: [],
});
await act(async () => {
await Promise.resolve();
});
const guideTrigger = container.querySelector(
'[data-testid="home-guide-help-trigger"]',
) as HTMLButtonElement | null;
expect(guideTrigger).toBeTruthy();
act(() => {
guideTrigger?.click();
});
expect(
container.querySelector('[data-testid="home-guide-help-active-badge"]'),
).toBeTruthy();
expect(
container.querySelector('[data-testid="home-guide-cards"]'),
).toBeTruthy();
expect(
container.querySelector('[data-testid="home-starter-chips"]'),
).toBeNull();
expect(container.querySelector("textarea")?.placeholder).toContain(
"想了解什么？试试",
);
const closeGuide = container.querySelector(
'[data-testid="home-guide-help-active-badge"] button',
) as HTMLButtonElement | null;
act(() => {
closeGuide?.click();
});
expect(
container.querySelector('[data-testid="home-guide-help-active-badge"]'),
).toBeNull();
expect(
container.querySelector('[data-testid="home-guide-cards"]'),
).toBeNull();
expect(
container.querySelector('[data-testid="home-starter-chips"]'),
).toBeTruthy();
expect(
container.querySelector('[data-testid="home-input-tab-suggestion"]'),
).toBeTruthy();
});
it("点击首屏起手任务只挂载输入区能力，不弹出启动表单", async () => {
const container = renderEmptyState({
activeTheme: "general",
serviceSkills: [],
});
await act(async () => {
await Promise.resolve();
});
const starter = container.querySelector(
'[data-testid="entry-recommended-daily-trend-briefing"]',
) as HTMLButtonElement | null;
expect(starter).toBeTruthy();
act(() => {
starter?.click();
});
expect(document.body.textContent).not.toContain(
"开始这一步前，我先确认几件事。",
);
expect(
container.querySelector('[data-testid="curated-task-badge"]'),
).toBeTruthy();
expect(container.textContent).toContain("每日趋势摘要");
});
it("保存判断结论后不应打乱首屏固定起手入口", async () => {
const projectId = "project-review-recommendation";
const container = renderEmptyState({
@@ -627,7 +706,7 @@ describe("EmptyState", () => {
).map((element) => element.getAttribute("data-testid"));
expect(starterItems).toEqual(initialStarterItems);
expect(starterItems[0]).toBe("entry-recommended-daily-trend-briefing");
expect(starterItems[0]).toBe("entry-recommended-social-post-starter");
expect(
container.querySelector('[data-testid="entry-review-feedback-banner"]'),
).toBeNull();
@@ -797,24 +876,10 @@ describe("EmptyState", () => {
});
const drawer = await openHomeMoreSkillsDrawer(container);
expect(drawer.textContent).toContain("最近使用");
const continuationItems = Array.from(
drawer.querySelectorAll('[data-testid^="home-drawer-"]'),
).map((element) => element.getAttribute("data-testid"));
expect(continuationItems).toContain(
const recentTemplateButton = expectHomeDrawerButton(
drawer,
"home-drawer-entry-recommended-social-post-starter",
);
expect(continuationItems).toContain(
"home-drawer-entry-service-skill-content-iteration-flow",
);
const recentTemplateButton = drawer.querySelector(
'[data-testid="home-drawer-entry-recommended-social-post-starter"]',
) as HTMLButtonElement | null;
expect(recentTemplateButton).toBeTruthy();
expect(recentTemplateButton?.textContent).toContain("内容主稿生成");
await act(async () => {
recentTemplateButton?.click();
@@ -871,11 +936,10 @@ describe("EmptyState", () => {
}),
);
const recentMethodButton = drawer.querySelector(
'[data-testid="home-drawer-entry-service-skill-content-iteration-flow"]',
) as HTMLButtonElement | null;
expect(recentMethodButton).toBeTruthy();
expect(recentMethodButton?.textContent).toContain("内容迭代整理");
const recentMethodButton = expectHomeDrawerButton(
drawer,
"home-drawer-entry-service-skill-content-iteration-flow",
);
act(() => {
recentMethodButton?.click();
@@ -908,11 +972,10 @@ describe("EmptyState", () => {
});
const drawer = await openHomeMoreSkillsDrawer(container);
expect(drawer.textContent).toContain("最近使用");
const sceneContinuationButton = drawer.querySelector(
'[data-testid="home-drawer-entry-service-skill-project-insight-flow"]',
) as HTMLButtonElement | null;
expect(sceneContinuationButton).toBeTruthy();
const sceneContinuationButton = expectHomeDrawerButton(
drawer,
"home-drawer-entry-service-skill-project-insight-flow",
);
act(() => {
sceneContinuationButton?.click();
@@ -959,14 +1022,10 @@ describe("EmptyState", () => {
});
const drawer = await openHomeMoreSkillsDrawer(container);
expect(drawer.textContent).toContain("最近使用");
const installedSkillButton = drawer.querySelector(
'[data-testid="home-drawer-entry-installed-skill-content-playbook"]',
) as HTMLButtonElement | null;
expect(installedSkillButton).toBeTruthy();
expect(installedSkillButton?.textContent).toContain("内容主稿方法");
expect(installedSkillButton?.textContent).toContain("继续优化这套内容主稿");
const installedSkillButton = expectHomeDrawerButton(
drawer,
"home-drawer-entry-installed-skill-content-playbook",
);
await act(async () => {
installedSkillButton?.click();
@@ -996,11 +1055,12 @@ describe("EmptyState", () => {
await Promise.resolve();
});
expect(
container.querySelector(
'[data-testid="home-drawer-entry-installed-skill-content-playbook"]',
),
).toBeNull();
const drawer = await openHomeMoreSkillsDrawer(container);
const installedSkillButton = expectHomeDrawerButton(
drawer,
"home-drawer-entry-installed-skill-content-playbook",
);
expect(installedSkillButton.title).toContain("本地补充技能");
await act(async () => {
recordSlashEntryUsage({
@@ -1012,54 +1072,11 @@ describe("EmptyState", () => {
await Promise.resolve();
});
const drawer = await openHomeMoreSkillsDrawer(container);
const installedSkillButton = drawer.querySelector(
'[data-testid="home-drawer-entry-installed-skill-content-playbook"]',
) as HTMLButtonElement | null;
expect(installedSkillButton).toBeTruthy();
expect(drawer.textContent).toContain("最近使用");
expect(installedSkillButton?.textContent).toContain("内容主稿方法");
expect(installedSkillButton?.textContent).toContain("继续完善这套内容方法");
});
it("最近使用的 service skill 也应显影上次补过的关键信息", async () => {
act(() => {
recordServiceSkillUsage({
skillId: "github-repo-radar",
runnerType: "instant",
slotValues: {
repository_query: "AI Agent 监控",
},
});
});
const onSelectServiceSkill = vi.fn<(skill: ServiceSkillHomeItem) => void>();
const recentMethod: ServiceSkillHomeItem = {
...createGithubSearchServiceSkill(),
defaultExecutorBinding: "agent_turn",
siteCapabilityBinding: undefined,
recentUsedAt: 1_900_000_000_000,
isRecent: true,
};
const container = renderEmptyState({
activeTheme: "general",
onSelectServiceSkill,
serviceSkills: [recentMethod],
});
await act(async () => {
await Promise.resolve();
});
const drawer = await openHomeMoreSkillsDrawer(container);
const recentMethodButton = drawer.querySelector(
'[data-testid="home-drawer-entry-service-skill-github-repo-radar"]',
) as HTMLButtonElement | null;
expect(recentMethodButton).toBeTruthy();
expect(recentMethodButton?.textContent).toContain(
"复用 GitHub 登录态检索项目。",
const updatedInstalledSkillButton = expectHomeDrawerButton(
drawer,
"home-drawer-entry-installed-skill-content-playbook",
);
expect(updatedInstalledSkillButton.title).toContain("继续完善这套内容方法");
});
it("首页主体不再重复展示项目选择器入口", async () => {
@@ -1084,7 +1101,7 @@ describe("EmptyState", () => {
).toBeNull();
});
it("通用首页应把补充入口收成轻量按钮，不再展开支撑能力说明", async () => {
it("通用首页不应把浏览器连接提前放到首屏补充入口", async () => {
const container = renderEmptyState({
activeTheme: "general",
onLaunchBrowserAssist: vi.fn(),
@@ -1096,19 +1113,14 @@ describe("EmptyState", () => {
expect(
container.querySelector('[data-testid="home-supplemental-actions"]'),
).toBeTruthy();
).toBeNull();
expect(
container.querySelector('[data-testid="entry-connect-browser"]'),
).toBeTruthy();
expect(container.textContent).toContain("连接浏览器");
).toBeNull();
expect(container.textContent).not.toContain("连接浏览器");
expect(container.textContent).not.toContain(
"需要网页登录时，也可以先把浏览器接上。",
);
expect(container.textContent).not.toContain("查看支撑能力");
expect(container.textContent).not.toContain("收起支撑能力");
expect(container.textContent).not.toContain("重复任务可持续复用");
expect(container.textContent).not.toContain("复杂任务可拆分并行推进");
expect(container.textContent).not.toContain("网页登录与网页执行");
expect(
container.querySelector('[data-testid="entry-capability-toggle"]'),
).toBeNull();
@@ -1182,7 +1194,7 @@ describe("EmptyState", () => {
expect(onOpenMemoryWorkbench).not.toHaveBeenCalled();
});
it("点击补充入口里的连接浏览器应触发浏览器接入", async () => {
it("浏览器接入不再通过首屏补充入口直接触发", async () => {
const onLaunchBrowserAssist = vi.fn();
const container = renderEmptyState({
activeTheme: "general",
@@ -1196,13 +1208,9 @@ describe("EmptyState", () => {
const mediaButton = container.querySelector(
'[data-testid="entry-connect-browser"]',
) as HTMLButtonElement | null;
expect(mediaButton).toBeTruthy();
expect(mediaButton).toBeNull();
act(() => {
mediaButton?.click();
});
expect(onLaunchBrowserAssist).toHaveBeenCalledTimes(1);
expect(onLaunchBrowserAssist).not.toHaveBeenCalled();
});
it("点击每日趋势摘要应开启联网搜索并记录最近使用", async () => {
@@ -1221,7 +1229,7 @@ describe("EmptyState", () => {
});
const card = container.querySelector(
'[data-testid="entry-recommended-daily-trend-briefing"]',
'[data-testid="home-gallery-entry-recommended-daily-trend-briefing"]',
) as HTMLDivElement | null;
expect(card).toBeTruthy();
@@ -1279,7 +1287,7 @@ describe("EmptyState", () => {
});
const card = container.querySelector(
'[data-testid="entry-recommended-daily-trend-briefing"]',
'[data-testid="home-gallery-entry-recommended-daily-trend-briefing"]',
) as HTMLDivElement | null;
expect(card).toBeTruthy();
@@ -1360,7 +1368,7 @@ describe("EmptyState", () => {
});
const card = container.querySelector(
'[data-testid="entry-recommended-social-post-starter"]',
'[data-testid="home-gallery-entry-recommended-social-post-starter"]',
) as HTMLDivElement | null;
expect(card).toBeTruthy();
@@ -1417,7 +1425,7 @@ describe("EmptyState", () => {
});
const card = container.querySelector(
'[data-testid="entry-recommended-longform-multiplatform-rewrite"]',
'[data-testid="home-gallery-entry-recommended-longform-multiplatform-rewrite"]',
) as HTMLDivElement | null;
expect(card).toBeTruthy();
@@ -1761,7 +1769,7 @@ describe("EmptyState", () => {
});
const templateButton = container.querySelector(
'[data-testid="entry-recommended-daily-trend-briefing"]',
'[data-testid="home-gallery-entry-recommended-daily-trend-briefing"]',
) as HTMLButtonElement | null;
expect(templateButton).toBeTruthy();
@@ -1884,7 +1892,7 @@ describe("EmptyState", () => {
});
const templateButton = container.querySelector(
'[data-testid="entry-recommended-daily-trend-briefing"]',
'[data-testid="home-gallery-entry-recommended-daily-trend-briefing"]',
) as HTMLButtonElement | null;
expect(templateButton).toBeTruthy();
@@ -2008,7 +2016,7 @@ describe("EmptyState", () => {
});
const templateButton = container.querySelector(
'[data-testid="entry-recommended-daily-trend-briefing"]',
'[data-testid="home-gallery-entry-recommended-daily-trend-briefing"]',
) as HTMLButtonElement | null;
expect(templateButton).toBeTruthy();
@@ -2052,7 +2060,7 @@ describe("EmptyState", () => {
});
const templateButton = container.querySelector(
'[data-testid="entry-recommended-account-project-review"]',
'[data-testid="home-gallery-entry-recommended-account-project-review"]',
) as HTMLButtonElement | null;
expect(templateButton).toBeTruthy();
expect(container.textContent).not.toContain("当前判断：先补复核与修复");
@@ -2115,11 +2123,15 @@ describe("EmptyState", () => {
});
const templateButton = container.querySelector(
'[data-testid="entry-recommended-daily-trend-briefing"]',
'[data-testid="home-gallery-entry-recommended-daily-trend-briefing"]',
) as HTMLButtonElement | null;
await act(async () => {
await Promise.resolve();
});
expect(templateButton).toBeTruthy();
expect(templateButton?.textContent).toContain("帮我想选题");
expect(templateButton?.textContent).toContain("每日趋势摘要");
expect(container.textContent).not.toContain("当前结果基线：AI 内容周报");
expect(container.textContent).not.toContain("当前判断：先补复核与修复");
expect(container.textContent).not.toContain("当前卡点：复核阻塞");
@@ -2254,7 +2266,7 @@ describe("EmptyState", () => {
});
const templateButton = container.querySelector(
'[data-testid="entry-recommended-daily-trend-briefing"]',
'[data-testid="home-gallery-entry-recommended-daily-trend-briefing"]',
) as HTMLButtonElement | null;
expect(templateButton).toBeTruthy();
@@ -2388,7 +2400,7 @@ describe("EmptyState", () => {
);
});
it("通用对话默认占位文案应强调本轮任务起手", async () => {
it("通用对话默认展示 Ribbi 式 Tab 起手建议", async () => {
const container = renderEmptyState({
activeTheme: "general",
serviceSkills: [],
@@ -2398,8 +2410,11 @@ describe("EmptyState", () => {
await Promise.resolve();
});
const textarea = container.querySelector("textarea");
expect(textarea?.getAttribute("placeholder")).toContain("先说这轮要做什么");
expect(
container.querySelector('[data-testid="home-input-tab-suggestion"]')
?.textContent,
).toContain("tab");
expect(container.textContent).toContain("帮我整理一下会议纪要");
});
it("点击高级设置中的地球按钮应切换联网搜索开关", async () => {
@@ -2564,7 +2579,7 @@ describe("EmptyState", () => {
expect(
container.querySelector('[data-testid="home-start-surface"]'),
).toBeTruthy();
expect(container.textContent).toContain("帮我想选题");
expect(container.textContent).toContain("调研报告");
});
it("通用首页发送时不应自动注入任何历史默认 skill", async () => {
@@ -2759,7 +2774,7 @@ describe("EmptyState", () => {
expect(onSend).toHaveBeenCalledWith("整理最近发布计划", "react", undefined);
});
it("通用主题应提供浏览器协助入口并触发启动回调", async () => {
it("通用主题不应在首屏直接展示浏览器协助入口", async () => {
const onLaunchBrowserAssist = vi.fn();
const container = renderEmptyState({
activeTheme: "general",
@@ -2772,16 +2787,12 @@ describe("EmptyState", () => {
const launchButton = container.querySelector(
'[data-testid="entry-connect-browser"]',
) as HTMLButtonElement | null;
expect(launchButton).toBeTruthy();
expect(launchButton).toBeNull();
act(() => {
launchButton?.click();
});
expect(onLaunchBrowserAssist).toHaveBeenCalledTimes(1);
expect(onLaunchBrowserAssist).not.toHaveBeenCalled();
});
it("通用主题应渲染 SceneApp 面板并触发启动回调", async () => {
it("通用主题应把 SceneApp 放到第二屏任务库而不是首屏补充入口", async () => {
const onLaunchSceneApp = vi.fn();
const container = renderEmptyState({
activeTheme: "general",
@@ -2795,13 +2806,13 @@ describe("EmptyState", () => {
expect(container.textContent).toContain("短视频编排");
expect(
container.querySelector('[data-testid="home-supplemental-actions"]'),
).toBeTruthy();
).toBeNull();
expect(
container.querySelector('[data-testid="sceneapps-home-directory"]'),
).toBeNull();
const launchButton = container.querySelector(
'[data-testid="sceneapp-launch-story-video-suite"]',
'[data-testid="home-gallery-entry-sceneapp-story-video-suite"]',
) as HTMLButtonElement | null;
expect(launchButton).toBeTruthy();
@@ -2812,7 +2823,7 @@ describe("EmptyState", () => {
expect(onLaunchSceneApp).toHaveBeenCalledWith("story-video-suite");
});
it("通用主题应提供进入 SceneApp 目录页的入口", async () => {
it("通用主题不应把 SceneApp 目录页入口提前放到首屏", async () => {
const onOpenSceneAppsDirectory = vi.fn();
const container = renderEmptyState({
activeTheme: "general",
@@ -2826,13 +2837,9 @@ describe("EmptyState", () => {
const browseButton = Array.from(container.querySelectorAll("button")).find(
(button) => button.textContent?.includes("查看全部做法"),
);
expect(browseButton).toBeTruthy();
expect(browseButton).toBeFalsy();
act(() => {
browseButton?.dispatchEvent(new MouseEvent("click", { bubbles: true }));
});
expect(onOpenSceneAppsDirectory).toHaveBeenCalledTimes(1);
expect(onOpenSceneAppsDirectory).not.toHaveBeenCalled();
});
it("仅有目录入口但没有可直接收益时，不应默认渲染更多起手方式", async () => {
src/components/agent/chat/components/EmptyState.tsx
+131 -60
View File
@@ -9,8 +9,10 @@ import styled, { keyframes } from "styled-components";
import { getConfig } from "@/lib/api/appConfig";
import {
getSkillCatalog,
listSkillCatalogEntries,
listSkillCatalogSceneEntries,
subscribeSkillCatalogChanged,
type SkillCatalogEntry,
type SkillCatalogSceneEntry,
} from "@/lib/api/skillCatalog";
import type { CreationMode } from "./types";
@@ -88,12 +90,19 @@ import {
import { HomeSkillGallery } from "../home/HomeSkillGallery";
import {
buildHomeGalleryItems,
buildHomeGuideCards,
buildHomeInputSuggestions,
buildHomeSkillItems,
buildHomeSkillSections,
buildHomeStarterChips,
} from "../home/buildHomeSkillSurface";
import { HOME_COMPOSER_PLACEHOLDER } from "../home/homeSurfaceCopy";
import {
HOME_COMPOSER_PLACEHOLDER,
HOME_GUIDE_HELP_CONTEXT_LABEL,
HOME_GUIDE_HELP_PLACEHOLDER,
} from "../home/homeSurfaceCopy";
import type {
HomeGuideCard,
HomeSkillSurfaceItem,
HomeStarterChip,
} from "../home/homeSurfaceTypes";
@@ -198,7 +207,7 @@ const ComposerGlowFrame = styled.div`
const ScrollCue = styled.a`
position: absolute;
left: 50%;
bottom: clamp(0.62rem, 1.8vh, 1.15rem);
bottom: clamp(0.7rem, 1.9vh, 1.25rem);
z-index: 8;
display: grid;
width: min(680px, calc(100% - 2rem));
@@ -209,9 +218,9 @@ const ScrollCue = styled.a`
gap: 0.9rem;
transform: translateX(-50%);
padding: 0.35rem 0;
color: var(--lime-text-muted, rgb(100 116 139));
font-size: 12px;
font-weight: 680;
color: var(--lime-brand-strong, rgb(47 83 60));
font-size: 13px;
font-weight: 760;
line-height: 1;
text-decoration: none;
white-space: nowrap;
@@ -240,7 +249,20 @@ const ScrollCueLine = styled.span`
const ScrollCueText = styled.span`
display: inline-flex;
align-items: center;
gap: 0.22rem;
gap: 0.32rem;
border-radius: 999px;
border: 1px solid rgba(187, 247, 208, 0.86);
background:
linear-gradient(
180deg,
rgba(255, 255, 255, 0.94),
rgba(240, 253, 244, 0.88)
),
var(--lime-surface, #fff);
padding: 0.42rem 0.78rem;
box-shadow:
0 10px 28px rgba(15, 23, 42, 0.055),
inset 0 1px 0 rgba(255, 255, 255, 0.92);
`;
const ScrollCueArrow = styled.span`
@@ -443,10 +465,7 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
onImportSkill,
onRefreshSkills,
onLaunchBrowserAssist,
browserAssistLoading = false,
featuredSceneApps = [],
sceneAppsLoading = false,
sceneAppLaunchingId = null,
onLaunchSceneApp,
canResumeRecentSceneApp = false,
onResumeRecentSceneApp,
@@ -994,10 +1013,19 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
return presets;
}, []);
const homeStarterChips = useMemo(() => buildHomeStarterChips(), []);
const [homeCatalogEntries, setHomeCatalogEntries] = useState<
SkillCatalogEntry[]
>([]);
const [homeCatalogSceneEntries, setHomeCatalogSceneEntries] = useState<
SkillCatalogSceneEntry[]
>([]);
const [guideHelpActive, setGuideHelpActive] = useState(false);
useEffect(() => {
if (!isGeneralTheme) {
setGuideHelpActive(false);
}
}, [isGeneralTheme]);
useEffect(() => {
let cancelled = false;
@@ -1007,6 +1035,10 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
if (cancelled) {
return;
}
const entries = listSkillCatalogEntries(catalog).filter((entry) =>
(entry.surfaceScopes ?? []).includes("home"),
);
setHomeCatalogEntries(entries);
setHomeCatalogSceneEntries(
listSkillCatalogSceneEntries(catalog).filter((entry) =>
(entry.surfaceScopes ?? []).includes("home"),
@@ -1014,6 +1046,7 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
);
} catch {
if (!cancelled) {
setHomeCatalogEntries([]);
setHomeCatalogSceneEntries([]);
}
}
@@ -1034,6 +1067,27 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
() => listFeaturedHomeServiceSkills(serviceSkills ?? [], { limit: 6 }),
[serviceSkills],
);
const homeStarterChips = useMemo(
() => buildHomeStarterChips(homeCatalogEntries),
[homeCatalogEntries],
);
const homeInputSuggestions = useMemo(
() => buildHomeInputSuggestions(homeCatalogEntries),
[homeCatalogEntries],
);
const homeGuideCards = useMemo(
() => buildHomeGuideCards(homeCatalogEntries),
[homeCatalogEntries],
);
const guideHelpStarterLabel = useMemo(
() =>
homeStarterChips.find((chip) => chip.launchKind === "toggle_guide")
?.label,
[homeStarterChips],
);
const guideHelpLabel = guideHelpStarterLabel
? `Lime ${guideHelpStarterLabel}`
: HOME_GUIDE_HELP_CONTEXT_LABEL;
const homeSkillItems = useMemo(() => {
void slashEntryUsageVersion;
@@ -1148,15 +1202,65 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
onOpenSceneAppsDirectory?.();
return;
}
if (chip.launchKind === "prefill_prompt") {
setGuideHelpActive(false);
const prompt = chip.prompt?.trim();
if (prompt) {
setInput(prompt);
}
return;
}
const targetItem = chip.targetItemId
? homeSkillItems.find((item) => item.id === chip.targetItemId)
: null;
if (targetItem) {
if (targetItem.launchKind === "curated_task_launcher") {
const template = findCuratedTaskTemplateById(targetItem.id);
if (!template) {
return;
}
const prefill = resolveCuratedTaskTemplateLaunchPrefill(template);
setGuideHelpActive(false);
setActiveCapability({
kind: "curated_task",
task: template,
launchInputValues: prefill?.inputValues,
referenceMemoryIds:
prefill?.referenceMemoryIds ??
effectiveDefaultCuratedTaskReferenceMemoryIds,
referenceEntries:
prefill?.referenceEntries ??
effectiveDefaultCuratedTaskReferenceEntries,
});
if (prefill?.hint) {
toast.info(prefill.hint);
}
return;
}
setGuideHelpActive(false);
handleSelectHomeSkillItem(targetItem);
}
},
[handleSelectHomeSkillItem, homeSkillItems, onOpenSceneAppsDirectory],
[
effectiveDefaultCuratedTaskReferenceEntries,
effectiveDefaultCuratedTaskReferenceMemoryIds,
handleSelectHomeSkillItem,
homeSkillItems,
onOpenSceneAppsDirectory,
setInput,
],
);
const handleSelectHomeGuideCard = useCallback(
(card: HomeGuideCard) => {
setGuideHelpActive(true);
const prompt = card.prompt.trim();
if (prompt) {
setInput(prompt);
}
},
[setInput],
);
const recentSessionLinkLabel = useMemo(() => {
@@ -1197,66 +1301,23 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
});
}
featuredSceneApps.slice(0, 2).forEach((item) => {
actions.push({
id: `sceneapp-${item.id}`,
label:
sceneAppLaunchingId === item.id
? `${item.title} 准备中…`
: item.title,
title: `${item.businessLabel} · ${item.summary}`,
testId: `sceneapp-launch-${item.id}`,
onSelect: () => {
if (item.disabledReason || sceneAppLaunchingId === item.id) {
return;
}
void onLaunchSceneApp?.(item.id);
},
});
});
if (onOpenSceneAppsDirectory) {
actions.push({
id: "sceneapps-directory",
label: sceneAppsLoading ? "做法加载中" : "查看全部做法",
testId: "entry-sceneapps-directory",
onSelect: onOpenSceneAppsDirectory,
});
}
if (onLaunchBrowserAssist) {
actions.push({
id: "connect-browser",
label: browserAssistLoading ? "浏览器连接准备中" : "连接浏览器",
testId: "entry-connect-browser",
onSelect: () => {
void onLaunchBrowserAssist();
},
});
}
return actions;
}, [
browserAssistLoading,
canResumeRecentSceneApp,
featuredSceneApps,
onLaunchBrowserAssist,
onLaunchSceneApp,
onOpenSceneAppsDirectory,
onResumeRecentSceneApp,
onResumeRecentSession,
recentSessionLinkLabel,
recentSessionLinkTitle,
recentSessionTitle,
sceneAppLaunchingId,
sceneAppsLoading,
]);
const composerPanel = (
<ComposerGlowFrame>
<EmptyStateComposerPanel
input={input}
placeholder={getPlaceholder()}
placeholder={
guideHelpActive ? HOME_GUIDE_HELP_PLACEHOLDER : getPlaceholder()
}
onSend={handleSend}
activeTheme={activeTheme}
providerType={providerType}
@@ -1330,6 +1391,12 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
onFileSelect={handleFileSelect}
onPaste={handlePaste}
onRemoveImage={handleRemoveImage}
inputSuggestions={
hasAutoLaunchSiteSkill || guideHelpActive ? [] : homeInputSuggestions
}
guideHelpActive={guideHelpActive}
guideHelpLabel={guideHelpLabel}
onClearGuideHelp={() => setGuideHelpActive(false)}
/>
</ComposerGlowFrame>
);
@@ -1352,9 +1419,13 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
const homeStartSurfacePanel = (
<HomeStartSurface
starterChips={homeStarterChips}
guideCards={homeGuideCards}
guideOpen={guideHelpActive}
sections={homeSkillSections}
supplementalActions={homeSupplementalActions}
onGuideOpenChange={setGuideHelpActive}
onSelectStarterChip={handleSelectHomeStarterChip}
onSelectGuideCard={handleSelectHomeGuideCard}
onSelectSkillItem={handleSelectHomeSkillItem}
/>
);
@@ -1389,12 +1460,12 @@ export const EmptyState: React.FC<EmptyStateProps> = ({
<ScrollCue
href="#home-skill-gallery-screen"
data-testid="home-scroll-cue"
aria-label="向下滚动或点击以查看示例"
aria-label="向下滑，看看 Lime 可以帮你做什么"
>
<ScrollCueLine aria-hidden />
<ScrollCueText>
向下滚动或点击以查看示例
<ScrollCueArrow aria-hidden>⌄</ScrollCueArrow>
向下滑，看看 Lime 可以帮你做什么
<ScrollCueArrow aria-hidden>↓</ScrollCueArrow>
</ScrollCueText>
<ScrollCueLine aria-hidden />
</ScrollCue>
src/components/agent/chat/components/EmptyStateComposerPanel.test.tsx
+113
View File
@@ -339,6 +339,119 @@ describe("EmptyStateComposerPanel", () => {
expect(composer?.className).toContain("floating-composer");
});
it("输入为空时展示 Tab 起手建议，按 Tab 后填入当前建议", async () => {
const container = renderPanel({
inputSuggestions: [
{
id: "suggestion-email",
label: "帮我写一封工作邮件",
prompt: "请帮我写一封工作邮件。",
order: 10,
},
],
});
expect(
container.querySelector('[data-testid="home-input-tab-suggestion"]')
?.textContent,
).toContain("帮我写一封工作邮件");
const textarea = container.querySelector("textarea");
await act(async () => {
textarea?.dispatchEvent(
new KeyboardEvent("keydown", {
key: "Tab",
bubbles: true,
cancelable: true,
}),
);
await Promise.resolve();
});
expect(
(container.querySelector("textarea") as HTMLTextAreaElement).value,
).toBe("请帮我写一封工作邮件。");
expect(
container.querySelector('[data-testid="home-input-tab-suggestion"]'),
).toBeNull();
});
it("Shift+Tab 保持焦点切换，不填入起手建议", () => {
const container = renderPanel({
inputSuggestions: [
{
id: "suggestion-email",
label: "帮我写一封工作邮件",
prompt: "请帮我写一封工作邮件。",
order: 10,
},
],
});
const textarea = container.querySelector("textarea");
act(() => {
textarea?.dispatchEvent(
new KeyboardEvent("keydown", {
key: "Tab",
shiftKey: true,
bubbles: true,
cancelable: true,
}),
);
});
expect(
(container.querySelector("textarea") as HTMLTextAreaElement).value,
).toBe("");
});
it("引导帮助模式应展示可关闭的上下文 badge 并隐藏 Tab 起手建议", () => {
const onClearGuideHelp = vi.fn();
const container = renderPanel({
guideHelpActive: true,
guideHelpLabel: "Lime 引导帮助",
onClearGuideHelp,
inputSuggestions: [
{
id: "suggestion-meeting",
label: "帮我整理一下会议纪要",
prompt: "帮我整理一下会议纪要。",
order: 10,
},
],
});
expect(
container.querySelector('[data-testid="home-guide-help-active-badge"]')
?.textContent,
).toContain("Lime 引导帮助");
expect(
container.querySelector('[data-testid="home-guide-help-toolbar-badge"]')
?.textContent,
).toContain("引导帮助");
expect(
container.querySelector('[data-testid="home-input-tab-suggestion"]'),
).toBeNull();
const closeButton = container.querySelector(
'[data-testid="home-guide-help-active-badge"] button',
) as HTMLButtonElement | null;
act(() => {
closeButton?.click();
});
expect(onClearGuideHelp).toHaveBeenCalledTimes(1);
const toolbarCloseButton = container.querySelector(
'[data-testid="home-guide-help-toolbar-badge"]',
) as HTMLButtonElement | null;
act(() => {
toolbarCloseButton?.click();
});
expect(onClearGuideHelp).toHaveBeenCalledTimes(2);
});
it("存在当前带入的灵感时，应在输入区顶部展示被带入的参考对象", () => {
const container = renderPanel({
creationReplaySurface: {
src/components/agent/chat/components/EmptyStateComposerPanel.tsx
+149 -1
View File
@@ -1,5 +1,12 @@
import React, { useEffect, useMemo, useRef, useState } from "react";
import { ChevronDown, ChevronUp, Globe, Settings2 } from "lucide-react";
import {
ChevronDown,
ChevronUp,
Globe,
Lightbulb,
Settings2,
X,
} from "lucide-react";
import {
Select,
SelectContent,
@@ -48,6 +55,7 @@ import type {
import type { CuratedTaskTemplateItem } from "../utils/curatedTaskTemplates";
import type { CuratedTaskReferenceEntry } from "../utils/curatedTaskReferenceSelection";
import type { CreationReplaySurfaceModel } from "../utils/creationReplaySurface";
import type { HomeInputSuggestion } from "../home/homeSurfaceTypes";
import { getProviderLabel } from "@/lib/constants/providerMappings";
interface EmptyStateComposerPanelProps {
@@ -99,6 +107,60 @@ interface EmptyStateComposerPanelProps {
onFileSelect: (event: React.ChangeEvent<HTMLInputElement>) => void;
onPaste?: (event: React.ClipboardEvent<HTMLTextAreaElement>) => void;
onRemoveImage?: (index: number) => void;
inputSuggestions?: HomeInputSuggestion[];
guideHelpActive?: boolean;
guideHelpLabel?: string;
onClearGuideHelp?: () => void;
}
function GuideHelpBadge({
label,
onClear,
}: {
label: string;
onClear: () => void;
}) {
return (
<div
data-testid="home-guide-help-active-badge"
className="mx-1 mt-1 inline-flex w-fit max-w-full items-center gap-1.5 rounded-full border border-emerald-200/80 bg-emerald-50/90 px-2.5 py-1.5 text-xs font-semibold text-emerald-900 shadow-sm shadow-emerald-950/5"
title={label}
>
<Lightbulb className="h-3 w-3" strokeWidth={1.9} />
<span className="truncate">{label}</span>
<button
type="button"
onClick={onClear}
className="ml-0.5 text-emerald-800/70 hover:opacity-70"
aria-label={`关闭${label}`}
title="关闭引导帮助"
>
<X className="h-3 w-3" />
</button>
</div>
);
}
function GuideHelpToolbarBadge({
label,
onClear,
}: {
label: string;
onClear: () => void;
}) {
return (
<button
type="button"
data-testid="home-guide-help-toolbar-badge"
className="inline-flex min-h-8 max-w-full items-center gap-1.5 rounded-full border border-emerald-200/80 bg-emerald-50/90 px-3 text-xs font-semibold text-emerald-900 shadow-sm shadow-emerald-950/5 transition hover:bg-white hover:text-emerald-950"
title="关闭引导帮助"
onClick={onClear}
>
<Lightbulb className="h-3.5 w-3.5" strokeWidth={1.9} />
<span className="truncate">{label.replace(/^Lime\s+/, "")}</span>
<X className="h-3.5 w-3.5 opacity-70" aria-hidden />
</button>
);
}
export function EmptyStateComposerPanel({
@@ -148,6 +210,10 @@ export function EmptyStateComposerPanel({
onFileSelect,
onPaste,
onRemoveImage,
inputSuggestions = [],
guideHelpActive = false,
guideHelpLabel = "Lime 引导帮助",
onClearGuideHelp,
}: EmptyStateComposerPanelProps) {
const [draftInput, setDraftInput] = useState(input);
const textareaRef = useRef<HTMLTextAreaElement>(null);
@@ -200,6 +266,71 @@ export function EmptyStateComposerPanel({
teamSuggestion.shouldSuggest &&
dismissedSuggestionKey !== suggestionKey;
const shouldShowTeamSelector = isGeneralTheme && subagentEnabled;
const sortedInputSuggestions = useMemo(
() =>
[...inputSuggestions].sort((left, right) => {
if (left.order !== right.order) {
return left.order - right.order;
}
return left.label.localeCompare(right.label, "zh-CN");
}),
[inputSuggestions],
);
const [inputSuggestionIndex, setInputSuggestionIndex] = useState(0);
const shouldShowInputSuggestion =
sortedInputSuggestions.length > 0 &&
!isLoading &&
!disabled &&
draftInput.trim().length === 0 &&
pendingImages.length === 0 &&
!guideHelpActive &&
!activeBuiltinCommand &&
!activeRuntimeScene &&
!activeCuratedTask &&
!activeSkill &&
!creationReplaySurface;
const activeInputSuggestion =
shouldShowInputSuggestion && sortedInputSuggestions.length > 0
? sortedInputSuggestions[
inputSuggestionIndex % sortedInputSuggestions.length
]
: null;
useEffect(() => {
if (inputSuggestionIndex < sortedInputSuggestions.length) {
return;
}
setInputSuggestionIndex(0);
}, [inputSuggestionIndex, sortedInputSuggestions.length]);
useEffect(() => {
if (!shouldShowInputSuggestion || sortedInputSuggestions.length <= 1) {
return;
}
const timer = window.setInterval(() => {
setInputSuggestionIndex(
(current) => (current + 1) % sortedInputSuggestions.length,
);
}, 3500);
return () => window.clearInterval(timer);
}, [shouldShowInputSuggestion, sortedInputSuggestions.length]);
const handleAcceptInputSuggestion = (suggestion: {
label: string;
prompt: string;
testId?: string;
}) => {
setDraftInput(suggestion.prompt);
window.requestAnimationFrame(() => {
textareaRef.current?.focus();
textareaRef.current?.setSelectionRange(
suggestion.prompt.length,
suggestion.prompt.length,
);
});
};
const handleEnableTeamSuggestion = () => {
onSubagentEnabledChange?.(true);
@@ -238,6 +369,7 @@ export function EmptyStateComposerPanel({
};
const topExtra =
guideHelpActive ||
activeBuiltinCommand ||
activeRuntimeScene ||
activeCuratedTask ||
@@ -245,6 +377,13 @@ export function EmptyStateComposerPanel({
creationReplaySurface ||
shouldShowTeamSuggestion ? (
<>
{guideHelpActive ? (
<GuideHelpBadge
label={guideHelpLabel}
onClear={onClearGuideHelp ?? (() => undefined)}
/>
) : null}
{activeBuiltinCommand ? (
<BuiltinCommandBadge
command={activeBuiltinCommand}
@@ -354,6 +493,13 @@ export function EmptyStateComposerPanel({
)}
</MetaToggleButton>
{guideHelpActive ? (
<GuideHelpToolbarBadge
label={guideHelpLabel}
onClear={onClearGuideHelp ?? (() => undefined)}
/>
) : null}
{!showAdvancedControls && currentModelSummary ? (
<Badge
variant="outline"
@@ -511,6 +657,8 @@ export function EmptyStateComposerPanel({
topExtra={topExtra}
leftExtra={leftExtra}
showMetaTools={showAdvancedControls}
inputSuggestion={activeInputSuggestion}
onAcceptInputSuggestion={handleAcceptInputSuggestion}
/>
</>
);
src/components/agent/chat/components/HarnessStatusPanel.test.tsx
+88
View File
@@ -2,6 +2,10 @@ import { act, type ComponentProps } from "react";
import { createRoot, type Root } from "react-dom/client";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { AgentRuntimeToolInventory } from "@/lib/api/agentRuntime";
import {
areLightweightRenderersRegistered,
registerLightweightRenderers,
} from "@/components/artifact/renderers";
import { HarnessStatusPanel } from "./HarnessStatusPanel";
import type { HarnessSessionState } from "../utils/harnessState";
@@ -144,6 +148,18 @@ function findButtonByText(text: string): HTMLButtonElement | null {
) as HTMLButtonElement | null;
}
async function flushUntilTextAppears(text: string): Promise<void> {
for (let index = 0; index < 20; index += 1) {
if (document.body.textContent?.includes(text)) {
return;
}
await act(async () => {
await Promise.resolve();
await new Promise((resolve) => window.setTimeout(resolve, 0));
});
}
}
function createToolInventory(): AgentRuntimeToolInventory {
return {
request: {
@@ -610,6 +626,10 @@ beforeEach(() => {
}
).IS_REACT_ACT_ENVIRONMENT = true;
if (!areLightweightRenderersRegistered()) {
registerLightweightRenderers();
}
originalClipboard = navigator.clipboard;
Object.defineProperty(navigator, "clipboard", {
configurable: true,
@@ -949,6 +969,56 @@ describe("HarnessStatusPanel", () => {
"Artifact 校验已恢复 1 个产物，fallback 0 次。",
],
},
modality_runtime_contracts: {
snapshot_count: 2,
snapshot_index: {
browser_action_index: {
action_count: 2,
session_count: 1,
observation_count: 1,
screenshot_count: 1,
last_url: "https://example.com/",
session_ids: ["browser-session-1"],
target_ids: ["target-1"],
profile_keys: ["general_browser_assist"],
status_counts: [{ status: "completed", count: 2 }],
artifact_kind_counts: [
{ artifact_kind: "browser_session", count: 1 },
{ artifact_kind: "browser_snapshot", count: 1 },
],
action_counts: [
{ action: "navigate", count: 1 },
{ action: "get_page_info", count: 1 },
],
backend_counts: [{ backend: "lime_extension_bridge", count: 1 }],
items: [
{
artifact_kind: "browser_session",
action: "navigate",
status: "completed",
success: true,
session_id: "browser-session-1",
target_id: "target-1",
backend: "cdp_direct",
last_url: "https://example.com/",
},
{
artifact_kind: "browser_snapshot",
action: "get_page_info",
status: "completed",
success: true,
session_id: "browser-session-1",
target_id: "target-1",
entry_source: "at_browser_agent_command",
backend: "lime_extension_bridge",
last_url: "https://example.com/",
observation_available: true,
screenshot_available: true,
},
],
},
},
},
},
artifacts: [
{
@@ -992,6 +1062,24 @@ describe("HarnessStatusPanel", () => {
expect(document.body.textContent).toContain("阻塞失败");
expect(document.body.textContent).toContain("验证失败焦点");
expect(document.body.textContent).toContain("已恢复结果");
expect(document.body.textContent).toContain("Browser Assist 索引");
expect(document.body.textContent).toContain("https://example.com/");
expect(document.body.textContent).toContain("browser_snapshot");
expect(document.body.textContent).toContain("get_page_info");
expect(document.body.textContent).toContain("observation / screenshot");
const replayButton = document.body.querySelector(
'button[aria-label="打开 Browser Assist 复盘"]',
) as HTMLButtonElement | null;
expect(replayButton).not.toBeNull();
await act(async () => {
replayButton?.click();
await Promise.resolve();
});
await flushUntilTextAppears("最近浏览器动作");
expect(document.body.textContent).toContain("最近浏览器动作");
expect(document.body.textContent).toContain("browser_replay_viewer");
expect(document.body.textContent).toContain("当前已知缺口");
expect(document.body.textContent).toContain(
".lime/harness/sessions/session-evidence-1/evidence/summary.md",
src/components/agent/chat/components/HarnessStatusPanel.tsx
+297 -1
View File
@@ -37,6 +37,8 @@ import { toast } from "sonner";
import type { StepStatus } from "@/lib/workspace/workbenchContract";
import type {
AgentRuntimeAnalysisHandoff,
AgentRuntimeEvidenceBrowserActionIndex,
AgentRuntimeEvidenceBrowserActionItem,
AgentRuntimeEvidencePack,
AgentRuntimeHandoffBundle,
AgentRuntimeSaveReviewDecisionRequest,
@@ -60,6 +62,8 @@ import {
saveAgentRuntimeReviewDecision,
} from "@/lib/api/agentRuntime";
import { getMcpInnerToolName } from "@/lib/api/mcp";
import { ArtifactRenderer } from "@/components/artifact/ArtifactRenderer";
import type { Artifact } from "@/lib/artifact/types";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
import {
@@ -192,6 +196,7 @@ interface PreviewDialogState {
displayName: string;
content?: string;
preview?: string;
artifact?: Artifact;
error?: string;
isBinary: boolean;
size?: number;
@@ -490,6 +495,44 @@ function formatEvidenceArtifactKindLabel(
}
}
function formatBrowserActionArtifactKindLabel(kind?: string): string {
switch (kind?.trim()) {
case "browser_session":
return "browser_session";
case "browser_snapshot":
return "browser_snapshot";
default:
return kind?.trim() || "未知产物";
}
}
function formatBrowserActionStatusLabel(
item: AgentRuntimeEvidenceBrowserActionItem,
): string {
if (item.success === true && !item.status) {
return "成功";
}
if (item.success === false && !item.status) {
return "失败";
}
switch (item.status?.trim()) {
case "completed":
case "success":
case "succeeded":
return "成功";
case "failed":
case "error":
return "失败";
case "running":
return "执行中";
case "pending":
return "待处理";
default:
return item.status?.trim() || "未知状态";
}
}
function formatReplayArtifactKindLabel(
kind: AgentRuntimeReplayCase["artifacts"][number]["kind"],
): string {
@@ -1596,6 +1639,209 @@ function InventoryStatCard({
);
}
function BrowserActionIndexSummarySection({
index,
onOpenReplay,
}: {
index: AgentRuntimeEvidenceBrowserActionIndex;
onOpenReplay?: () => void;
}) {
if (index.action_count <= 0 && index.items.length === 0) {
return null;
}
const recentItems = index.items.slice(-3).reverse();
const latestUrl =
index.last_url ||
recentItems.find((item) => item.last_url)?.last_url ||
"暂无 URL";
return (
<div className="rounded-xl border border-sky-200 bg-sky-50/80 p-3">
<div className="flex items-center gap-2 text-sm font-medium text-sky-950">
<Eye className="h-4 w-4 text-sky-700" />
<span>Browser Assist 索引</span>
</div>
<p className="mt-1 text-xs text-sky-800">
来自 modalityRuntimeContracts.snapshotIndex.browserActionIndex，复盘
browser_session / browser_snapshot 的执行证据。
</p>
{onOpenReplay ? (
<div className="mt-3">
<Button
type="button"
size="sm"
variant="outline"
className="gap-2 bg-background"
onClick={onOpenReplay}
aria-label="打开 Browser Assist 复盘"
>
<Eye className="h-4 w-4" />
打开复盘
</Button>
</div>
) : null}
<div className="mt-3 grid gap-2 sm:grid-cols-2 xl:grid-cols-4">
<InventoryStatCard
title="浏览器动作"
value={`${index.action_count}`}
hint="browser_control action"
/>
<InventoryStatCard
title="会话"
value={`${index.session_count}`}
hint={
index.profile_keys.length > 0
? `profile ${index.profile_keys.slice(0, 2).join(" / ")}`
: "session / target"
}
/>
<InventoryStatCard
title="观察 / 截图"
value={`${index.observation_count} / ${index.screenshot_count}`}
hint="observation / screenshot"
/>
<InventoryStatCard
title="最近 URL"
value={latestUrl === "暂无 URL" ? latestUrl : "已记录"}
hint={latestUrl}
/>
</div>
{recentItems.length > 0 ? (
<div className="mt-3 space-y-2">
{recentItems.map((item, indexInList) => {
const itemKey = [
item.request_id,
item.session_id,
item.action,
indexInList,
]
.filter(Boolean)
.join(":");
return (
<div
key={itemKey}
className="rounded-lg border border-sky-200/80 bg-background/85 p-2.5"
>
<div className="flex flex-wrap items-center gap-2">
<span className="text-sm font-medium text-foreground">
{item.action || item.tool_name || "browser action"}
</span>
<Badge variant="outline">
{formatBrowserActionArtifactKindLabel(item.artifact_kind)}
</Badge>
<Badge
variant={
item.success === false ? "destructive" : "secondary"
}
>
{formatBrowserActionStatusLabel(item)}
</Badge>
{item.backend ? (
<Badge variant="outline">{item.backend}</Badge>
) : null}
</div>
<div className="mt-2 space-y-1 text-xs text-muted-foreground">
{item.last_url ? (
<div className="break-all">
URL：
<span className="ml-1 font-mono text-foreground">
{item.last_url}
</span>
</div>
) : null}
<div className="flex flex-wrap gap-x-3 gap-y-1">
{item.session_id ? (
<span>
session：
<span className="ml-1 font-mono text-foreground">
{item.session_id}
</span>
</span>
) : null}
{item.target_id ? (
<span>
target：
<span className="ml-1 font-mono text-foreground">
{item.target_id}
</span>
</span>
) : null}
{item.entry_source ? (
<span>
entry：
<span className="ml-1 font-mono text-foreground">
{item.entry_source}
</span>
</span>
) : null}
</div>
</div>
</div>
);
})}
</div>
) : null}
</div>
);
}
function buildBrowserReplayArtifact(
evidencePack: AgentRuntimeEvidencePack,
index: AgentRuntimeEvidenceBrowserActionIndex,
): Artifact {
const timestamp = Date.parse(evidencePack.exported_at);
return {
id: `browser-replay:${evidencePack.session_id}`,
type: "browser_assist",
title: "Browser Assist 复盘",
content: "",
status: "complete",
meta: {
browserActionIndex: {
actionCount: index.action_count,
sessionCount: index.session_count,
observationCount: index.observation_count,
screenshotCount: index.screenshot_count,
lastUrl: index.last_url,
sessionIds: index.session_ids,
targetIds: index.target_ids,
profileKeys: index.profile_keys,
items: index.items.map((item) => ({
artifactKind: item.artifact_kind,
toolName: item.tool_name,
action: item.action,
status: item.status,
success: item.success,
sessionId: item.session_id,
targetId: item.target_id,
profileKey: item.profile_key,
backend: item.backend,
requestId: item.request_id,
lastUrl: item.last_url,
title: item.title,
entrySource: item.entry_source,
observationAvailable: item.observation_available,
screenshotAvailable: item.screenshot_available,
})),
},
modalityContractKey: "browser_control",
viewerSurface: "browser_replay_viewer",
evidencePackRoot: evidencePack.pack_relative_root,
sessionId: index.session_ids[0] || evidencePack.session_id,
profileKey: index.profile_keys[0],
targetId: index.target_ids[0],
url: index.last_url,
},
position: { start: 0, end: 0 },
createdAt: Number.isFinite(timestamp) ? timestamp : Date.now(),
updatedAt: Number.isFinite(timestamp) ? timestamp : Date.now(),
};
}
function Section({
sectionKey,
title,
@@ -2526,6 +2772,7 @@ export function HarnessStatusPanel({
displayName: path ? getFileName(path) : title,
content: content?.trim() || preview?.trim(),
preview,
artifact: undefined,
error:
content?.trim() || preview?.trim()
? undefined
@@ -2635,6 +2882,25 @@ export function HarnessStatusPanel({
}
}, [previewDialog.content]);
const openBrowserReplayPreview = useCallback(
(
pack: AgentRuntimeEvidencePack,
index: AgentRuntimeEvidenceBrowserActionIndex,
) => {
setPreviewDialog({
open: true,
title: "Browser Assist 复盘",
description:
"来自 evidence browserActionIndex 的 browser_session / browser_snapshot 复盘。",
displayName: "browser_replay_viewer",
artifact: buildBrowserReplayArtifact(pack, index),
isBinary: false,
loading: false,
});
},
[],
);
const handleOpenPathValue = useCallback(
async (path: string) => {
const normalizedPath = path.trim();
@@ -3408,6 +3674,28 @@ export function HarnessStatusPanel({
</div>
</div>
{evidencePack.observability_summary
?.modality_runtime_contracts?.snapshot_index
?.browser_action_index
? (() => {
const browserActionIndex =
evidencePack.observability_summary
.modality_runtime_contracts.snapshot_index
.browser_action_index;
return (
<BrowserActionIndexSummarySection
index={browserActionIndex}
onOpenReplay={() =>
openBrowserReplayPreview(
evidencePack,
browserActionIndex,
)
}
/>
);
})()
: null}
{evidencePack.observability_summary
?.verification_summary ? (
<HarnessVerificationSummarySection
@@ -6454,7 +6742,15 @@ export function HarnessStatusPanel({
</div>
<ScrollArea className="max-h-[60vh] rounded-xl border border-border bg-muted/30">
{previewDialog.isBinary ? (
{previewDialog.artifact ? (
<div className="h-[58vh] min-h-[360px] bg-background">
<ArtifactRenderer
artifact={previewDialog.artifact}
tone="light"
hideToolbar
/>
</div>
) : previewDialog.isBinary ? (
<div className="flex items-center gap-2 px-4 py-6 text-sm text-muted-foreground">
<HardDriveDownload className="h-4 w-4" />
该文件为二进制内容，暂不支持文本预览。
src/components/agent/chat/components/Inputbar/components/InputbarCore.tsx
+85 -4
View File
@@ -5,6 +5,9 @@ import {
InputBarContainer,
InputColumn,
InputIconButton,
InputSuggestionKeycap,
InputSuggestionLayer,
InputSuggestionText,
MainRow,
MetaSlot,
StyledTextarea,
@@ -78,6 +81,16 @@ interface InputbarCoreProps {
onPromoteQueuedTurn?: (queuedTurnId: string) => void | Promise<boolean>;
onRemoveQueuedTurn?: (queuedTurnId: string) => void | Promise<boolean>;
showMetaTools?: boolean;
inputSuggestion?: {
label: string;
prompt: string;
testId?: string;
} | null;
onAcceptInputSuggestion?: (suggestion: {
label: string;
prompt: string;
testId?: string;
}) => void;
}
export const InputbarCore: React.FC<InputbarCoreProps> = ({
@@ -105,6 +118,8 @@ export const InputbarCore: React.FC<InputbarCoreProps> = ({
onPromoteQueuedTurn,
onRemoveQueuedTurn,
showMetaTools = true,
inputSuggestion = null,
onAcceptInputSuggestion,
}) => {
const [isTextareaExpanded, setIsTextareaExpanded] = useState(false);
const inputBarContainerRef = useRef<HTMLDivElement | null>(null);
@@ -184,6 +199,57 @@ export const InputbarCore: React.FC<InputbarCoreProps> = ({
: dictationEnabled || !voiceConfigLoaded
? "开始语音输入"
: "语音输入未启用";
const shouldShowInputSuggestion =
Boolean(inputSuggestion) && text.trim().length === 0 && !disabled;
const handleInputSuggestionKeyDown = useCallback(
(event: React.KeyboardEvent<HTMLTextAreaElement>) => {
if (
!inputSuggestion ||
event.key !== "Tab" ||
event.shiftKey ||
event.metaKey ||
event.ctrlKey ||
event.altKey ||
text.trim().length > 0 ||
disabled
) {
return;
}
const nativeEvent = event.nativeEvent as KeyboardEvent & {
isComposing?: boolean;
};
if (
nativeEvent.isComposing ||
nativeEvent.key === "Process" ||
nativeEvent.keyCode === 229
) {
return;
}
event.preventDefault();
const acceptedText = inputSuggestion.prompt;
if (onAcceptInputSuggestion) {
onAcceptInputSuggestion(inputSuggestion);
} else {
setText(acceptedText);
}
window.requestAnimationFrame(() => {
const textarea = resolvedTextareaRef.current;
textarea?.focus();
textarea?.setSelectionRange(acceptedText.length, acceptedText.length);
});
},
[
disabled,
inputSuggestion,
onAcceptInputSuggestion,
resolvedTextareaRef,
setText,
text,
],
);
const handleRemoveImageMouseDown = useCallback(
(event: React.MouseEvent<HTMLButtonElement>) => {
@@ -215,6 +281,7 @@ export const InputbarCore: React.FC<InputbarCoreProps> = ({
isLoading={isLoading}
disabled={disabled}
onPaste={onPaste}
onKeyDown={handleInputSuggestionKeyDown}
isFullscreen={isFullscreen}
fillHeightWhenFullscreen
hasAdditionalContent={pendingImages.length > 0}
@@ -224,10 +291,12 @@ export const InputbarCore: React.FC<InputbarCoreProps> = ({
allowSendWhileLoading
rows={isTextareaExpanded ? 7 : isFloatingVariant ? 3 : 1}
placeholder={
placeholder ||
(isFullscreen
? "全屏编辑模式，按 ESC 退出，Enter 发送"
: "在这里输入消息, 按 Enter 发送")
shouldShowInputSuggestion
? ""
: placeholder ||
(isFullscreen
? "全屏编辑模式，按 ESC 退出，Enter 发送"
: "在这里输入消息, 按 Enter 发送")
}
>
{({ textareaProps, textareaRef, isPrimaryDisabled, onPrimaryAction }) => {
@@ -295,6 +364,18 @@ export const InputbarCore: React.FC<InputbarCoreProps> = ({
<ImagePlus size={14} />
</InputIconButton>
<InputColumn>
{shouldShowInputSuggestion && inputSuggestion ? (
<InputSuggestionLayer
className={textareaClassName}
data-testid="home-input-tab-suggestion"
title="按 Tab 使用这条起手建议"
>
<InputSuggestionText>
{inputSuggestion.label}
</InputSuggestionText>
<InputSuggestionKeycap>tab</InputSuggestionKeycap>
</InputSuggestionLayer>
) : null}
<StyledTextarea
ref={textareaRef}
{...textareaProps}
src/components/agent/chat/components/Inputbar/styles.ts
+52
View File
@@ -106,9 +106,61 @@ export const InputColumn = styled.div`
min-width: 0;
display: flex;
flex-direction: column;
position: relative;
`;
export const InputSuggestionLayer = styled.div`
position: absolute;
top: 4px;
right: 0;
left: 0;
z-index: 0;
display: flex;
min-width: 0;
align-items: center;
gap: 9px;
color: hsl(var(--muted-foreground) / 0.58);
font-size: 14px;
line-height: 1.5;
pointer-events: none;
&.floating-composer {
top: 4px;
font-size: 17px;
line-height: 1.78;
}
&.floating-composer.floating-collapsed {
top: 8px;
line-height: 1.35;
}
`;
export const InputSuggestionText = styled.span`
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
`;
export const InputSuggestionKeycap = styled.span`
display: inline-flex;
min-height: 25px;
flex: 0 0 auto;
align-items: center;
justify-content: center;
border-radius: 999px;
background: rgba(148, 163, 184, 0.11);
padding: 0 0.62rem;
color: hsl(var(--muted-foreground) / 0.82);
font-size: 0.82em;
font-weight: 760;
letter-spacing: 0.01em;
`;
export const StyledTextarea = styled.textarea`
position: relative;
z-index: 1;
flex: 1;
padding: 4px 0 0;
border-radius: 0;
src/components/agent/chat/components/MarkdownRenderer.tsx
+13 -6
View File
@@ -502,8 +502,12 @@ interface MarkdownRendererProps {
showBlockActions?: boolean;
/** 引用当前正文块 */
onQuoteContent?: (content: string) => void;
/** 历史恢复等冷路径可用轻量模式，避开高成本 HTML/Katex/语法高亮。 */
renderMode?: MarkdownRenderMode;
}
export type MarkdownRenderMode = "standard" | "light";
function normalizeCodeLanguage(language: string): string {
const normalized = language.trim().toLowerCase();
if (!normalized) {
@@ -567,6 +571,7 @@ export const MarkdownRenderer: React.FC<MarkdownRendererProps> = memo(
isStreaming = false,
showBlockActions = false,
onQuoteContent,
renderMode = "standard",
}) => {
const [copied, setCopied] = React.useState<string | null>(null);
const [bundleImageOverrides, setBundleImageOverrides] = React.useState<
@@ -577,6 +582,8 @@ export const MarkdownRenderer: React.FC<MarkdownRendererProps> = memo(
const selectionSnapshotRef = React.useRef<string | null>(null);
const useLightweightStreamingRender =
isStreaming && content.length >= STREAMING_LIGHT_RENDER_THRESHOLD;
const useLightweightMarkdownRender =
renderMode === "light" || useLightweightStreamingRender;
const debouncedStreamingContent = useDebouncedValue(
content,
useLightweightStreamingRender
@@ -592,13 +599,13 @@ export const MarkdownRenderer: React.FC<MarkdownRendererProps> = memo(
const remarkPlugins = React.useMemo(
() =>
useLightweightStreamingRender ? [remarkGfm] : [remarkGfm, remarkMath],
[useLightweightStreamingRender],
useLightweightMarkdownRender ? [remarkGfm] : [remarkGfm, remarkMath],
[useLightweightMarkdownRender],
);
const rehypePlugins = React.useMemo(
() => (useLightweightStreamingRender ? [] : [rehypeRaw, rehypeKatex]),
[useLightweightStreamingRender],
() => (useLightweightMarkdownRender ? [] : [rehypeRaw, rehypeKatex]),
[useLightweightMarkdownRender],
);
const hasRemoteImageReferences = React.useMemo(
() => /https?:\/\//i.test(content),
@@ -995,7 +1002,7 @@ export const MarkdownRenderer: React.FC<MarkdownRendererProps> = memo(
<ReactMarkdown
remarkPlugins={remarkPlugins}
rehypePlugins={rehypePlugins}
skipHtml={useLightweightStreamingRender}
skipHtml={useLightweightMarkdownRender}
components={{
// 使用 pre 组件来处理代码块，以便更好地控制 a2ui 的渲染
pre({ children, ...props }: any) {
@@ -1068,7 +1075,7 @@ export const MarkdownRenderer: React.FC<MarkdownRendererProps> = memo(
);
}
if (useLightweightStreamingRender) {
if (useLightweightMarkdownRender) {
return (
<pre {...props}>
<code className={className}>{codeContent}</code>
src/components/agent/chat/components/MessageList.test.tsx
+902 -8
View File
File diff suppressed because it is too large Load Diff
src/components/agent/chat/components/MessageList.tsx
+829 -164
View File
File diff suppressed because it is too large Load Diff
src/components/agent/chat/components/StreamingRenderer.test.tsx
+50
View File
@@ -20,15 +20,18 @@ const mockMarkdownRenderer = vi.fn(
content,
showBlockActions,
onQuoteContent,
renderMode,
}: {
content: string;
showBlockActions?: boolean;
onQuoteContent?: (content: string) => void;
renderMode?: "standard" | "light";
}) => (
<div
data-testid="markdown-renderer"
data-show-block-actions={showBlockActions ? "yes" : "no"}
data-has-on-quote-content={onQuoteContent ? "yes" : "no"}
data-render-mode={renderMode || "standard"}
>
{content}
</div>
@@ -50,6 +53,7 @@ vi.mock("./MarkdownRenderer", () => ({
content: string;
showBlockActions?: boolean;
onQuoteContent?: (content: string) => void;
renderMode?: "standard" | "light";
}) => mockMarkdownRenderer(props),
}));
@@ -107,6 +111,7 @@ afterEach(() => {
});
mounted.container.remove();
}
vi.useRealTimers();
vi.clearAllMocks();
});
@@ -135,6 +140,7 @@ function renderHarness(props: {
suppressProcessFlow?: boolean;
showContentBlockActions?: boolean;
onQuoteContent?: (content: string) => void;
markdownRenderMode?: "standard" | "light";
}) {
const container = document.createElement("div");
document.body.appendChild(container);
@@ -239,6 +245,36 @@ describe("StreamingRenderer", () => {
expect(parseAIResponseMock).not.toHaveBeenCalled();
});
it("流式纯文本首帧应立即显示前缀，避免等待下一帧才吐字", () => {
const fullText = "这是第一段流式输出，应该马上可见。";
const { container } = renderHarness({
content: fullText,
isStreaming: true,
});
const renderedText = container.textContent || "";
expect(renderedText.length).toBeGreaterThan(0);
expect(fullText.startsWith(renderedText)).toBe(true);
expect(renderedText.length).toBeLessThan(fullText.length);
});
it("流式正文积压较多时应快速追上最新目标文本", () => {
vi.useFakeTimers();
const fullText = "流式输出".repeat(80);
const { container } = renderHarness({
content: fullText,
isStreaming: true,
});
act(() => {
vi.advanceTimersByTime(100);
});
const renderedText = container.textContent || "";
expect(renderedText.length).toBeGreaterThan(120);
expect(fullText.startsWith(renderedText)).toBe(true);
});
it("开启正文块操作时应向 MarkdownRenderer 透传引用/复制能力", () => {
const onQuoteContent = vi.fn();
@@ -257,6 +293,20 @@ describe("StreamingRenderer", () => {
);
});
it("历史恢复轻量模式应向 MarkdownRenderer 透传 light 渲染模式", () => {
renderHarness({
content: "这是历史会话正文\n\n```ts\nconsole.log('heavy')\n```",
markdownRenderMode: "light",
});
expect(mockMarkdownRenderer).toHaveBeenCalledWith(
expect.objectContaining({
content: expect.stringContaining("这是历史会话正文"),
renderMode: "light",
}),
);
});
it("交错内容重复渲染时应复用已缓存解析结果", () => {
const structuredText = '<write_file path="demo.md">hello</write_file>';
parseAIResponseMock.mockImplementation((content: string) => {
src/components/agent/chat/components/StreamingRenderer.tsx
+88 -9
View File
@@ -9,7 +9,7 @@ import React, { memo, useMemo, useState, useEffect, useRef } from "react";
import { cn } from "@/lib/utils";
import { ChevronDown, ExternalLink, FileText } from "lucide-react";
import { useDebouncedValue } from "@/lib/artifact/hooks/useDebouncedValue";
import { MarkdownRenderer } from "./MarkdownRenderer";
import { MarkdownRenderer, type MarkdownRenderMode } from "./MarkdownRenderer";
import { A2UITaskCard, A2UITaskLoadingCard } from "./A2UITaskCard";
import { ActionRequestA2UIPreviewCard } from "./ActionRequestA2UIPreviewCard";
import { InlineToolProcessStep } from "./InlineToolProcessStep";
@@ -52,6 +52,42 @@ import { resolveToolProcessNarrative } from "../utils/toolProcessSummary";
const STRUCTURED_CONTENT_HINT_RE = /<a2ui|```\s*a2ui|<write_file|<document/i;
const STRUCTURED_PARSE_CACHE_LIMIT = 64;
const STREAMING_STRUCTURED_PARSE_DEBOUNCE_MS = 48;
const STREAMING_TEXT_LARGE_BACKLOG_CHARS = 240;
const STREAMING_TEXT_MEDIUM_BACKLOG_CHARS = 80;
const STREAMING_TEXT_SMALL_BACKLOG_CHARS = 24;
const STREAMING_TEXT_INITIAL_VISIBLE_CHARS = 12;
function resolveStreamingTextStepSize(
pendingChars: number,
elapsedMs: number,
charInterval: number,
): number {
const timedStep = Math.max(1, Math.floor(elapsedMs / charInterval));
if (pendingChars > STREAMING_TEXT_LARGE_BACKLOG_CHARS) {
return Math.max(timedStep, Math.ceil(pendingChars * 0.5));
}
if (pendingChars > STREAMING_TEXT_MEDIUM_BACKLOG_CHARS) {
return Math.max(timedStep, Math.ceil(pendingChars * 0.3));
}
if (pendingChars > STREAMING_TEXT_SMALL_BACKLOG_CHARS) {
return Math.max(timedStep, 8);
}
return timedStep;
}
function resolveInitialStreamingDisplayText(text: string, isStreaming: boolean) {
if (!isStreaming || !text || hasStructuredContentHint(text)) {
return isStreaming ? "" : text;
}
return Array.from(text)
.slice(0, STREAMING_TEXT_INITIAL_VISIBLE_CHARS)
.join("");
}
// ============ 思考内容组件 ============
@@ -311,6 +347,7 @@ interface PlanAwareMarkdownOptions {
renderProposedPlanBlocks?: boolean;
showBlockActions?: boolean;
onQuoteContent?: (content: string) => void;
markdownRenderMode?: MarkdownRenderMode;
}
function renderPlanAwareMarkdown(
@@ -326,6 +363,7 @@ function renderPlanAwareMarkdown(
renderProposedPlanBlocks = true,
showBlockActions = false,
onQuoteContent,
markdownRenderMode = "standard",
}: PlanAwareMarkdownOptions,
) {
if (!renderProposedPlanBlocks) {
@@ -345,6 +383,7 @@ function renderPlanAwareMarkdown(
isStreaming={isStreaming}
showBlockActions={showBlockActions}
onQuoteContent={onQuoteContent}
renderMode={markdownRenderMode}
/>
);
}
@@ -373,6 +412,7 @@ function renderPlanAwareMarkdown(
isStreaming={isStreaming}
showBlockActions={showBlockActions}
onQuoteContent={onQuoteContent}
renderMode={markdownRenderMode}
/>
),
);
@@ -411,6 +451,8 @@ interface StreamingTextProps {
showBlockActions?: boolean;
/** 引用当前正文块 */
onQuoteContent?: (content: string) => void;
/** Markdown 渲染模式；历史恢复可使用 light 降低首帧成本。 */
markdownRenderMode?: MarkdownRenderMode;
}
/**
@@ -436,14 +478,21 @@ const StreamingText: React.FC<StreamingTextProps> = memo(
onCodeBlockClick,
showBlockActions = false,
onQuoteContent,
markdownRenderMode = "standard",
}) => {
const [displayText, setDisplayText] = useState("");
const displayIndexRef = useRef(0);
const initialDisplayText = resolveInitialStreamingDisplayText(
text,
isStreaming,
);
const [displayText, setDisplayText] = useState(() => initialDisplayText);
const displayIndexRef = useRef(initialDisplayText.length);
const animationRef = useRef<number | null>(null);
const prevTextRef = useRef("");
const prevTextRef = useRef(isStreaming ? "" : text);
const targetTextRef = useRef(text);
const parseCacheRef = useRef<Map<string, ParseResult>>(new Map());
useEffect(() => {
targetTextRef.current = text;
// 如果不是流式输出，直接显示完整文本
if (!isStreaming) {
// 调试：确认非流式时是否正确设置完整文本
@@ -463,6 +512,19 @@ const StreamingText: React.FC<StreamingTextProps> = memo(
return;
}
if (
!text.startsWith(prevTextRef.current) ||
displayIndexRef.current > text.length
) {
const seededText = resolveInitialStreamingDisplayText(
text,
isStreaming,
);
displayIndexRef.current = seededText.length;
prevTextRef.current = "";
setDisplayText(seededText);
}
// 检测文本是否有新增
if (text.length <= prevTextRef.current.length) {
prevTextRef.current = text;
@@ -483,23 +545,31 @@ const StreamingText: React.FC<StreamingTextProps> = memo(
const elapsed = currentTime - lastTime;
if (elapsed >= charInterval) {
// 计算这一帧应该显示多少个字符
const charsToAdd = Math.max(1, Math.floor(elapsed / charInterval));
const targetText = targetTextRef.current;
const pendingChars = Math.max(
0,
targetText.length - displayIndexRef.current,
);
const charsToAdd = resolveStreamingTextStepSize(
pendingChars,
elapsed,
charInterval,
);
const newIndex = Math.min(
displayIndexRef.current + charsToAdd,
text.length,
targetText.length,
);
if (newIndex > displayIndexRef.current) {
displayIndexRef.current = newIndex;
setDisplayText(text.slice(0, newIndex));
setDisplayText(targetText.slice(0, newIndex));
}
lastTime = currentTime;
}
// 继续动画直到追上目标
if (displayIndexRef.current < text.length) {
if (displayIndexRef.current < targetTextRef.current.length) {
animationRef.current = requestAnimationFrame(animate);
} else {
animationRef.current = null;
@@ -570,6 +640,7 @@ const StreamingText: React.FC<StreamingTextProps> = memo(
isStreaming,
showBlockActions,
onQuoteContent,
markdownRenderMode,
});
}
@@ -627,6 +698,7 @@ const StreamingText: React.FC<StreamingTextProps> = memo(
isStreaming,
showBlockActions,
onQuoteContent,
markdownRenderMode,
});
}
}
@@ -913,6 +985,8 @@ interface StreamingRendererProps {
suppressProcessFlow?: boolean;
showContentBlockActions?: boolean;
onQuoteContent?: (content: string) => void;
/** Markdown 渲染模式；历史恢复可使用 light 降低首帧成本。 */
markdownRenderMode?: MarkdownRenderMode;
}
/**
@@ -953,6 +1027,7 @@ export const StreamingRenderer: React.FC<StreamingRendererProps> = memo(
suppressProcessFlow = false,
showContentBlockActions = false,
onQuoteContent,
markdownRenderMode = "standard",
}) => {
const shouldRenderInlineActionRequest = React.useCallback(
(request: ActionRequired) =>
@@ -1368,6 +1443,7 @@ export const StreamingRenderer: React.FC<StreamingRendererProps> = memo(
onCodeBlockClick={onCodeBlockClick}
showBlockActions={showContentBlockActions}
onQuoteContent={onQuoteContent}
markdownRenderMode={markdownRenderMode}
/>
);
}
@@ -1383,6 +1459,7 @@ export const StreamingRenderer: React.FC<StreamingRendererProps> = memo(
onA2UISubmit,
onCodeBlockClick,
onQuoteContent,
markdownRenderMode,
renderA2UIInline,
renderProposedPlanBlocks,
renderWriteFileIndicator,
@@ -1434,6 +1511,7 @@ export const StreamingRenderer: React.FC<StreamingRendererProps> = memo(
onCodeBlockClick={onCodeBlockClick}
showBlockActions={showContentBlockActions}
onQuoteContent={onQuoteContent}
markdownRenderMode={markdownRenderMode}
/>
);
},
@@ -1450,6 +1528,7 @@ export const StreamingRenderer: React.FC<StreamingRendererProps> = memo(
onQuoteContent,
renderParsedResultParts,
renderProposedPlanBlocks,
markdownRenderMode,
showContentBlockActions,
shouldCollapseCodeBlock,
shouldShowCursor,
src/components/agent/chat/components/TaskMessagePreview.tsx
+166 -3
View File
@@ -8,10 +8,15 @@ import {
Mic,
PlayCircle,
Search,
Volume2,
} from "lucide-react";
import { emitVideoWorkbenchTaskAction } from "@/lib/videoWorkbenchEvents";
import { cn } from "@/lib/utils";
import type { MessageTaskPreview } from "../types";
import {
countTranscriptSpeakers,
formatTranscriptSegmentRange,
} from "../utils/transcriptSegments";
interface TaskMessagePreviewProps {
preview: MessageTaskPreview;
@@ -20,6 +25,8 @@ interface TaskMessagePreviewProps {
function resolveTaskLabel(preview: MessageTaskPreview): string {
switch (preview.kind) {
case "audio_generate":
return "配音生成";
case "video_generate":
return "视频生成";
case "broadcast_generate":
@@ -95,6 +102,23 @@ function resolveDescription(preview: MessageTaskPreview): string {
}
}
if (preview.kind === "audio_generate") {
switch (preview.status) {
case "complete":
case "partial":
return preview.audioUrl?.trim()
? "音频结果已同步，打开查看即可继续预览与管理任务。"
: "配音任务已完成，正在同步音频结果。";
case "failed":
return "配音生成失败，请调整文本、音色或模型后重试。";
case "cancelled":
return "配音任务已经取消，当前不会继续生成音频。";
case "running":
default:
return "配音任务已写入统一 audio_task/audio_output 协议，工作区会继续同步结果。";
}
}
switch (preview.status) {
case "complete":
case "partial":
@@ -151,6 +175,28 @@ function buildMetaItems(preview: MessageTaskPreview): string[] {
if (preview.model?.trim()) {
items.push(preview.model.trim());
}
if (
(preview.kind === "audio_generate" ||
preview.kind === "transcription_generate") &&
preview.errorCode?.trim()
) {
items.push(`错误码: ${preview.errorCode.trim()}`);
}
if (preview.kind === "transcription_generate") {
if (preview.language?.trim()) {
items.push(preview.language.trim());
}
if (preview.outputFormat?.trim()) {
items.push(preview.outputFormat.trim());
}
if (preview.transcriptSegments && preview.transcriptSegments.length > 0) {
items.push(`${preview.transcriptSegments.length} 段时间轴`);
const speakerCount = countTranscriptSpeakers(preview.transcriptSegments);
if (speakerCount > 0) {
items.push(`${speakerCount} 位说话人`);
}
}
}
return items.filter((item) => item.trim().length > 0);
}
@@ -158,6 +204,8 @@ function resolveGenericTaskIcon(
preview: Exclude<MessageTaskPreview, { kind: "video_generate" }>,
) {
switch (preview.kind) {
case "audio_generate":
return Volume2;
case "broadcast_generate":
return Mic;
case "modal_resource_search":
@@ -175,6 +223,38 @@ function renderGenericTaskMedia(
preview: Exclude<MessageTaskPreview, { kind: "video_generate" }>,
Icon: ReturnType<typeof resolveGenericTaskIcon>,
) {
if (preview.kind === "audio_generate") {
const playableAudioUrl = preview.audioUrl?.trim();
return (
<div className="flex h-20 w-28 shrink-0 flex-col justify-between rounded-[18px] border border-sky-100 bg-[radial-gradient(circle_at_top,rgba(14,165,233,0.18),transparent_48%),linear-gradient(180deg,#f8fafc,#eef6ff)] p-2 text-sky-700">
<div className="flex items-center justify-between gap-2">
<Volume2 className="h-5 w-5" />
<span className="rounded-full border border-sky-200 bg-white px-2 py-0.5 text-[10px] font-semibold text-sky-700">
audio
</span>
</div>
{playableAudioUrl ? (
<audio
controls
src={playableAudioUrl}
className="h-7 w-full"
aria-label="配音任务音频预览"
/>
) : (
<div className="flex items-end gap-1 text-sky-400">
{[14, 22, 11, 28, 18].map((height, index) => (
<span
key={`${preview.taskId}-wave-${index}`}
className="w-1.5 rounded-full bg-sky-300"
style={{ height }}
/>
))}
</div>
)}
</div>
);
}
if (
preview.kind === "modal_resource_search" &&
preview.imageCandidates &&
@@ -214,6 +294,68 @@ function renderGenericTaskMedia(
);
}
function renderTaskFailureDetails(
preview: Exclude<MessageTaskPreview, { kind: "video_generate" }>,
) {
if (
(preview.kind !== "audio_generate" &&
preview.kind !== "transcription_generate") ||
preview.status !== "failed"
) {
return null;
}
const errorCode = preview.errorCode?.trim();
const errorMessage = preview.errorMessage?.trim();
if (!errorCode && !errorMessage) {
return null;
}
return (
<div className="rounded-2xl border border-rose-100 bg-rose-50/80 px-3 py-2 text-xs leading-5 text-rose-700">
{errorCode ? (
<div className="font-semibold">错误码：{errorCode}</div>
) : null}
{errorMessage ? <div>{errorMessage}</div> : null}
</div>
);
}
function renderTranscriptionSegmentSummary(
preview: Exclude<MessageTaskPreview, { kind: "video_generate" }>,
) {
if (
preview.kind !== "transcription_generate" ||
!preview.transcriptSegments ||
preview.transcriptSegments.length === 0
) {
return null;
}
return (
<div className="rounded-2xl border border-slate-200 bg-slate-50 px-3 py-2">
<div className="mb-1.5 text-[11px] font-semibold text-slate-500">
时间轴预览
</div>
<div className="space-y-1.5">
{preview.transcriptSegments.slice(0, 2).map((segment) => (
<div
key={`${preview.taskId}-${segment.id}`}
className="grid grid-cols-[72px_minmax(0,1fr)] gap-2 text-xs leading-5"
>
<span className="font-medium text-slate-500">
{formatTranscriptSegmentRange(segment)}
</span>
<span className="line-clamp-2 text-slate-700">
{segment.speaker ? `${segment.speaker}：` : ""}
{segment.text}
</span>
</div>
))}
</div>
</div>
);
}
function renderVideoTaskPreview(
preview: Extract<MessageTaskPreview, { kind: "video_generate" }>,
onOpen?: (preview: MessageTaskPreview) => void,
@@ -364,6 +506,11 @@ function renderGenericTaskPreview(
) {
const Icon = resolveGenericTaskIcon(preview);
const media = renderGenericTaskMedia(preview, Icon);
const titleText =
preview.kind === "audio_generate" ||
preview.kind === "transcription_generate"
? preview.prompt || preview.title?.trim() || resolveTaskLabel(preview)
: preview.title?.trim() || preview.prompt || resolveTaskLabel(preview);
return (
<button
type="button"
@@ -403,15 +550,16 @@ function renderGenericTaskPreview(
<div className="min-w-0 flex-1 space-y-2.5 py-1">
<div className="space-y-1.5">
<div className="line-clamp-2 text-sm font-semibold leading-6 text-slate-900">
{preview.title?.trim() ||
preview.prompt ||
resolveTaskLabel(preview)}
{titleText}
</div>
<p className="text-sm leading-6 text-slate-600">
{resolveDescription(preview)}
</p>
</div>
{renderTaskFailureDetails(preview)}
{renderTranscriptionSegmentSummary(preview)}
{metaItems.length > 0 ? (
<div className="flex flex-wrap gap-2">
{metaItems.map((item) => (
@@ -430,6 +578,21 @@ function renderGenericTaskPreview(
任务文件: {preview.artifactPath.trim()}
</div>
) : null}
{(preview.kind === "audio_generate" ||
preview.kind === "transcription_generate") &&
preview.taskFilePath?.trim() ? (
<div className="truncate text-xs text-slate-500">
源任务: {preview.taskFilePath.trim()}
</div>
) : null}
{preview.kind === "transcription_generate" &&
preview.transcriptPath?.trim() ? (
<div className="truncate text-xs text-slate-500">
转写结果: {preview.transcriptPath.trim()}
</div>
) : null}
</div>
</div>
</div>
src/components/agent/chat/home/HomeGuideCards.tsx
+112
View File
@@ -0,0 +1,112 @@
import styled from "styled-components";
import type { HomeGuideCard } from "./homeSurfaceTypes";
const GuideGrid = styled.div`
display: grid;
width: min(1520px, 100%);
min-width: 0;
grid-template-columns: repeat(4, minmax(0, 1fr));
gap: 0.68rem;
margin: 0.35rem auto 0;
@media (max-width: 1180px) {
grid-template-columns: repeat(2, minmax(0, 1fr));
}
@media (max-width: 720px) {
grid-template-columns: 1fr;
}
`;
const GuideButton = styled.button`
position: relative;
display: flex;
min-height: 104px;
min-width: 0;
flex-direction: column;
justify-content: center;
gap: 0.52rem;
border-radius: 18px;
border: 1px solid var(--lime-surface-border, rgba(226, 232, 240, 0.92));
background: color-mix(
in srgb,
var(--lime-surface, #fff) 90%,
var(--lime-surface-soft, #f4f8f2) 10%
);
padding: 1rem 1.1rem;
text-align: left;
box-shadow: 0 10px 24px rgba(15, 23, 42, 0.035);
outline: none;
transition:
border-color 160ms ease,
box-shadow 160ms ease,
transform 160ms ease;
&:hover {
transform: translateY(-1px);
border-color: var(--lime-surface-border-strong, rgba(148, 163, 184, 0.72));
box-shadow: 0 14px 30px rgba(15, 23, 42, 0.07);
}
&:focus-visible {
border-color: var(--lime-composer-border-focus, rgba(74, 222, 128, 0.7));
box-shadow:
0 0 0 3px var(--lime-focus-ring, rgba(74, 222, 128, 0.18)),
0 14px 30px rgba(15, 23, 42, 0.07);
}
`;
const GuideCopy = styled.span`
display: grid;
width: 100%;
min-width: 0;
gap: 0.46rem;
`;
const GuideTitle = styled.span`
color: var(--lime-text-strong, rgb(15 23 42));
font-size: 15px;
font-weight: 780;
line-height: 1.28;
`;
const GuideSummary = styled.span`
display: -webkit-box;
overflow: hidden;
color: var(--lime-text-muted, rgb(100 116 139));
font-size: 12px;
font-weight: 520;
line-height: 1.45;
-webkit-box-orient: vertical;
-webkit-line-clamp: 2;
`;
interface HomeGuideCardsProps {
cards: HomeGuideCard[];
onSelect: (card: HomeGuideCard) => void;
}
export function HomeGuideCards({ cards, onSelect }: HomeGuideCardsProps) {
if (cards.length === 0) {
return null;
}
return (
<GuideGrid data-testid="home-guide-cards" aria-label="首页引导帮助">
{cards.map((card) => (
<GuideButton
key={card.id}
type="button"
data-testid={card.testId}
title={`${card.title} · ${card.summary}`}
onClick={() => onSelect(card)}
>
<GuideCopy>
<GuideTitle>{card.title}</GuideTitle>
<GuideSummary>{card.summary}</GuideSummary>
</GuideCopy>
</GuideButton>
))}
</GuideGrid>
);
}
src/components/agent/chat/home/HomeMoreSkillsDrawer.test.tsx
+109
View File
@@ -0,0 +1,109 @@
import React from "react";
import { act } from "react";
import { createRoot, type Root } from "react-dom/client";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { HomeMoreSkillsDrawer } from "./HomeMoreSkillsDrawer";
import type {
HomeSkillSection,
HomeSkillSurfaceItem,
} from "./homeSurfaceTypes";
const mountedRoots: Array<{ root: Root; container: HTMLDivElement }> = [];
beforeEach(() => {
(
globalThis as typeof globalThis & {
IS_REACT_ACT_ENVIRONMENT?: boolean;
}
).IS_REACT_ACT_ENVIRONMENT = true;
});
function createItem(id: string, title: string): HomeSkillSurfaceItem {
return {
id,
title,
summary: `${title} 摘要`,
category: "social",
sourceKind: "curated_task",
launchKind: "curated_task_launcher",
coverToken: "trend",
isRecent: false,
isRecommended: true,
usedAt: null,
testId: `entry-recommended-${id}`,
};
}
function renderDrawer(
sections: HomeSkillSection[],
open = true,
onSelectItem = vi.fn(),
) {
const container = document.createElement("div");
document.body.appendChild(container);
const root = createRoot(container);
mountedRoots.push({ root, container });
act(() => {
root.render(
<HomeMoreSkillsDrawer
open={open}
sections={sections}
onSelectItem={onSelectItem}
/>,
);
});
return { container, onSelectItem };
}
afterEach(() => {
while (mountedRoots.length > 0) {
const mounted = mountedRoots.pop();
if (!mounted) break;
act(() => {
mounted.root.unmount();
});
mounted.container.remove();
}
});
describe("HomeMoreSkillsDrawer", () => {
it("关闭时不渲染抽屉", () => {
const { container } = renderDrawer(
[
{
id: "social",
title: "社交媒体",
items: [createItem("draft", "写第一版")],
},
],
false,
);
expect(
container.querySelector('[data-testid="home-more-skills-drawer"]'),
).toBeNull();
});
it("按分组渲染 compact 做法列表并触发选择", () => {
const item = createItem("draft", "写第一版");
const { container, onSelectItem } = renderDrawer([
{ id: "recent", title: "最近使用", items: [item] },
]);
expect(container.textContent).toContain("最近使用");
expect(container.textContent).toContain("/写第一版");
const button = container.querySelector(
'[data-testid="home-drawer-entry-recommended-draft"]',
) as HTMLButtonElement | null;
expect(button).toBeTruthy();
act(() => {
button?.click();
});
expect(onSelectItem).toHaveBeenCalledWith(item);
});
});
src/components/agent/chat/home/HomeSkillGallery.test.tsx
+85
View File
@@ -0,0 +1,85 @@
import React from "react";
import { act } from "react";
import { createRoot, type Root } from "react-dom/client";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { HomeSkillGallery } from "./HomeSkillGallery";
import type { HomeSkillSurfaceItem } from "./homeSurfaceTypes";
const mountedRoots: Array<{ root: Root; container: HTMLDivElement }> = [];
beforeEach(() => {
(
globalThis as typeof globalThis & {
IS_REACT_ACT_ENVIRONMENT?: boolean;
}
).IS_REACT_ACT_ENVIRONMENT = true;
});
function createItem(): HomeSkillSurfaceItem {
return {
id: "daily-trend-briefing",
title: "每日趋势摘要",
summary: "先收一版内容趋势。",
category: "social",
sourceKind: "curated_task",
launchKind: "curated_task_launcher",
coverToken: "trend",
isRecent: false,
isRecommended: true,
usedAt: null,
testId: "entry-recommended-daily-trend-briefing",
};
}
function renderGallery(items: HomeSkillSurfaceItem[], onSelectItem = vi.fn()) {
const container = document.createElement("div");
document.body.appendChild(container);
const root = createRoot(container);
mountedRoots.push({ root, container });
act(() => {
root.render(<HomeSkillGallery items={items} onSelectItem={onSelectItem} />);
});
return { container, onSelectItem };
}
afterEach(() => {
while (mountedRoots.length > 0) {
const mounted = mountedRoots.pop();
if (!mounted) break;
act(() => {
mounted.root.unmount();
});
mounted.container.remove();
}
});
describe("HomeSkillGallery", () => {
it("空列表不渲染第二屏任务库", () => {
const { container } = renderGallery([]);
expect(
container.querySelector('[data-testid="home-skill-gallery"]'),
).toBeNull();
});
it("渲染第二屏任务卡并触发选择", () => {
const item = createItem();
const { container, onSelectItem } = renderGallery([item]);
expect(container.textContent).toContain("你可以从这些任务开始");
expect(container.textContent).toContain("每日趋势摘要");
const button = container.querySelector(
'[data-testid="home-gallery-entry-recommended-daily-trend-briefing"]',
) as HTMLButtonElement | null;
expect(button).toBeTruthy();
act(() => {
button?.click();
});
expect(onSelectItem).toHaveBeenCalledWith(item);
});
});

Some files were not shown because too many files have changed in this diff Show More