mirror of
https://github.com/dataelement/bisheng.git
synced 2026-09-01 15:32:50 +08:00
feat(linsight): restore tenant skill runtime via copy-time provisioning (F035 Fork X)
Re-enable the F035 task-mode Skill runtime that was disabled 2026-06-16. Instead of the dormant TenantSkillsMiddleware runtime whitelist, use a copy-time gate (Fork X): at task startup materialize_session_skills copies the run's allowed bundles (governance-enabled ∩ user-selected) into the session workspace /skills/ subtree, then a plain deepagents SkillsMiddleware (FilesystemBackend over the workspace cache, which is dir-aware unlike the MinIO WorkspaceBackend.ls) enumerates them; the model reads the same /skills/<name>/SKILL.md paths back through the workspace. The copy is the whitelist gate, so unselected skills never reach the agent — no per-run active_skills config. - skill_provisioning.materialize_session_skills (new) + SkillStore.read_bytes - thread skills: LinsightQuestionSubmitSchema + linsight_session_version.skills (JSON col + migration f035_linsight_skills) + workbench_impl + task_exec._create_agent - agent_factory attaches SkillsMiddleware when skills_present - TenantSkillsMiddleware retired to dormant (docstring); whitelist semantics moved here - fix(client): selecting a skill in /c task-mode no longer toggles task mode OFF - tests: test_skill_provisioning (gate, byte fidelity, cross-tenant, enumeration loop) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,128 @@
|
||||
# 灵思任务模式 Skill 能力恢复方案
|
||||
|
||||
- **关联 Feature**:F035 `035-linsight-task-mode`(本方案是其 Skill 运行时能力的收尾)
|
||||
- **关联契约**:[release-contract](../../../features/v2.6.0/release-contract.md)(F035:`LinsightSkill` / 110 段 11050–11069 / `linsight_skill` 表)
|
||||
- **关联设计**:F035 `design.md §7`(Skill 存储与中间件,原始设计——部分已演进/过时,详见 §1)、`spec.md AC-2/AC-3`
|
||||
- **文档性质**:历史背景沉淀 + 需求 + 方案思路。**先文档对齐,暂不开发。**
|
||||
- **一句话**:F035 迁移时把流程知识从动态 SOP 改为可热插拔的静态 Skill,但 **Skill 的运行时注入入口在 2026-06-16 被临时关闭**;本方案厘清「原设计 → 实现演进 → 禁用 → 现状再核查」全脉络,论证恢复无架构级阻塞,并给出落地路径与工作量。
|
||||
|
||||
> ✅ **实现状态(2026-06-24,代码已落地 + 单测绿,端到端实测待跑)**:按 **Option 1 / Fork X(复制时过滤,最简)** 实现。
|
||||
> - 落点:`skill_provisioning.materialize_session_skills`(复制闸门)+ `SkillStore.read_bytes` + schema/`linsight_session_version.skills` 列 + migration `f035_linsight_skills` + `task_exec._create_agent` 触发复制 + `agent_factory.create_linsight_agent(skills_present=...)` 装配 `SkillsMiddleware`(枚举 backend=指向工作区缓存的 `FilesystemBackend`)。
|
||||
> - **未采用** `active_skills` run-config 穿透与 `TenantSkillsMiddleware` 运行时白名单(语义迁至复制闸门;该子类保留为休眠 fallback)。
|
||||
> - **§1.5 两条静态推断已校正并单测确认**:① `SkillsMiddleware` 不注册文件工具 → 双 backend 不 shadow(deepagents 0.6.8 源码 + 装配共存确认);② 原生 `skills=` 无法用——`SkillsMiddleware` 靠 `ls` 的 `is_dir` 目录项枚举(`skills.py:674`),而 MinIO 版 `WorkspaceBackend.ls` 只返回文件项 → 改用「复制进工作区 + `FilesystemBackend` 枚举缓存」;`test_skill_provisioning.py::TestEnumerationLoop` 实跑确认复制后被真实 `SkillsMiddleware` 枚举到、且注入路径经 `normalize_workspace_path` 解析回工作区同一物理文件(AC-R2 路径闭环)。
|
||||
> - **待办**:D1 端到端实测(真实 MinIO + 模型 + 交付物落 `output/` 验 shadow 不复现)、D3 DM8/MySQL `alembic upgrade head` 实库回归。
|
||||
|
||||
---
|
||||
|
||||
## 一、历史背景与演进脉络
|
||||
|
||||
### 1.1 Skill 在 F035 中的定位
|
||||
F035 把灵思自研 ReAct 内核替换为 deepagents,流程知识从**运行时动态生成的 SOP** 迁移为**可热插拔的静态 Skill**(progressive disclosure:先看 name/description,命中再 `read_file` 读正文)。Skill 分两类:
|
||||
- **built-in**(`SKILLS_ROOT/built-in/`):内核能力,始终生效、前端不暴露、不经 API。
|
||||
- **租户自定义**(`SKILLS_ROOT/data/skills/{tenant_id}/`):前端唯一可见可管理的一类,可 CRUD + 启停,按租户隔离。
|
||||
|
||||
### 1.2 原始设计(design §7.2)——双中间件 + 顺序硬约束
|
||||
原设计用**两个**中间件:
|
||||
|
||||
| 中间件 | 职责 |
|
||||
|---|---|
|
||||
| `SkillsMiddleware` | progressive disclosure,加载两类 skill 的 frontmatter 注入 prompt |
|
||||
| `SkillWhitelistMiddleware` | 按本轮 `active_skills` 过滤**租户自定义** skill;built-in 始终放行 |
|
||||
|
||||
顺序硬约束:`SkillWhitelistMiddleware → SkillsMiddleware → GenerativeUIMiddleware`。
|
||||
`active_skills` 契约(C3):`["a","b"]`=勾选白名单 / `[]`=全禁租户技能 / `None`=全放行(仅留给非 UI 调用方,产品前端始终下发显式列表)。
|
||||
|
||||
### 1.3 实现期演进(deviation D8)——合并为单 subclass
|
||||
实际实现没有按双中间件落地,而是合并成**一个 subclass** `TenantSkillsMiddleware(SkillsMiddleware)`(`skill_middleware.py:54-109`),在 `before_agent`/`abefore_agent` 里直接过滤 `skills_metadata`。代码注释明示这是 **deviation D8**:单 subclass 取代 §7.2 的双中间件拆分,消除了对其它中间件的顺序依赖。白名单逻辑(built-in 始终放行、租户技能需 `enabled` + 命中 `active_skills`)写在 `_skill_allowed()` 中,已完整实现。
|
||||
|
||||
### 1.4 临时禁用(2026-06-16,commit `4ce0c496f`;注释 `285d6c20e`)
|
||||
`agent_factory.create_linsight_agent()` 的 `middleware=` 列表**不再注入** `TenantSkillsMiddleware`,`make_skills_middleware()` 无生产调用方。代码注释(`agent_factory.py:385-392` / `skill_middleware.py:1-32`)记录**两条独立原因**:
|
||||
|
||||
1. **Workspace filesystem shadow bug**:`TenantSkillsMiddleware` 带了**自己独立的** `FilesystemBackend(SKILLS_ROOT, virtual_mode)`。装在工作区 `FilesystemMiddleware` 之后,被认为会**遮蔽** agent 的 `write_file/read_file`,导致交付物落进技能库而非工作区 `output/`,最终工作区空、产不出结果文档。
|
||||
2. **`active_skills` 白名单从未生效**:per-run 白名单键从未写进 run config(`task_exec.py` 三处 config 只设 `thread_id`),第二道门即使开了也是 no-op。
|
||||
|
||||
当时取舍:**技能可选、交付物核心,先保交付物**,Skill 留作 forward-compatible WIP。
|
||||
|
||||
### 1.5 现状再核查(基于当前 deepagents 0.6.8,静态代码分析)
|
||||
对当前安装版本(`deepagents==0.6.8`,pin `>=0.6.3`)逐行核查后,**原「shadow 阻塞」的前提已不成立**:
|
||||
|
||||
- 0.6.8 的 `SkillsMiddleware` **不注册任何文件工具**——它只在 `before_agent` 用自带 backend 读 SKILL.md 元数据、在 `wrap_model_call` 把技能清单注入 system prompt。文件工具的**唯一来源**是 scaffolding 级的 `FilesystemMiddleware`(`graph.py:206/213`,不可剔除)。
|
||||
- deepagents 原生 `skills=` 参数收的是**同一个 backend 下的路径前缀**(如 `/skills/user/`)——官方设计本就让 Skill 与工作区**共用一个 backend**,根本不产生 shadow。
|
||||
|
||||
→ **结论**:当年那条注释的心智模型对应的是更早的版本/自建独立 backend 的组合。在 0.6.8 下,把 `TenantSkillsMiddleware` 加回 `middleware=` 列表**不会**再注册第二套文件工具、**不会** shadow 工作区。原「阻塞性架构问题」需要的是一次廉价的端到端实测复核,而非大改。
|
||||
|
||||
**但暴露出一个真实、有界的遗留缺口**(不是 shadow,而是原设计 §7.1 的隐含隐患):技能 bundle 存在独立的 `SKILLS_ROOT`,而模型读正文/附属文件用的是**工作区** backend 的 `read_file`。design §7.1 写「模型按 SKILL.md 中的相对路径 `read_file` 附属文件」、§5 写「Skill 独立走磁盘、两套后端互不混用」——但 progressive disclosure 注入 prompt 的技能路径是 `SKILLS_ROOT` 内的路径,工作区 `read_file` 解析不到。**模型能看到技能 name/description,却读不到技能正文**。这才是恢复真正要解决的「命名空间」问题,且有成熟解法(§3.2)。
|
||||
|
||||
> ⚠️ 该断裂为**静态分析结论**,尚未端到端实跑验证;恢复方案 D 块(§3.3)需先实测复核再据此定方案。
|
||||
|
||||
---
|
||||
|
||||
## 二、需求
|
||||
|
||||
### 2.1 恢复目标
|
||||
让 **租户自定义 Skill 端到端可用**:终端用户在任务模式勾选已启用技能 → 后端把勾选项作为 `active_skills` 下发 → 中间件按治理 `enabled` + 本轮白名单过滤加载 → 模型能读到技能正文并据其执行 → 交付物仍正确落工作区 `output/`。
|
||||
|
||||
### 2.2 验收标准(对齐 F035)
|
||||
| # | 验收点 | 来源 |
|
||||
|---|---|---|
|
||||
| AC-R1 | 勾选的租户技能在运行时被加载,未勾选/停用的不加载(白名单二元:`[names]` / `[]`) | design §7.2 |
|
||||
| AC-R2 | 模型能 `read_file` 到技能正文与 bundle 附属文件(progressive disclosure 闭环) | 本方案 §1.5 缺口 |
|
||||
| AC-R3 | 启用技能后,任务交付物仍正确落工作区 `output/`(shadow 不复现) | 本方案 §1.4 原因1 |
|
||||
| AC-R4 | 跨租户隔离:A 租户技能不被 B 租户加载 | design §6 / §7.6 |
|
||||
| AC-R5 | 关联 F035 AC-2「Skill 稳定触发、命中率」可在恢复后重新评测 | spec AC-2 |
|
||||
|
||||
### 2.3 范围红线(非目标)
|
||||
- ❌ **built-in 内置技能的设计与编写**:中间件对「无技能」优雅处理,built-in 缺省不影响租户技能恢复;属独立增量。
|
||||
- ❌ 技能热更新策略、管理页↔选择器实时同步(WebSocket)、file-memory 超期兜底等产品增强。
|
||||
- ❌ 不碰 F035 N1–N7 既有红线(自动挂载 / learning loop / marketplace / Interpreter Skill / 双引擎共存)。
|
||||
|
||||
---
|
||||
|
||||
## 三、方案思路
|
||||
|
||||
### 3.1 现状盘点(哪些已就绪、哪些缺)
|
||||
| 组件 | 状态 | 锚点 |
|
||||
|---|---|---|
|
||||
| 平台端技能管理 UI(列表/新建/上传/启停/详情) | ✅ live | `platform/.../components/LinSight/skill/` |
|
||||
| 客户端技能选择器 + 提交 payload 带 `skills:[name]` | ✅ live | `client/.../Linsight/Input/SkillSelector.tsx`、`TaskModeInput.tsx` |
|
||||
| 后端 CRUD/上传/启停 API(10 端点) | ✅ live | `linsight/api/endpoints/skill.py` |
|
||||
| `linsight_skill` 表 + `SkillStore` 磁盘层 | ✅ live | migration 2026-06-11、`skill_store.py` |
|
||||
| `TenantSkillsMiddleware`(含白名单过滤) | ⚠️ 已实现未装配 | `skill_middleware.py`(DISABLED 注释) |
|
||||
| 后端接收 `skills` 字段 | ❌ 缺 | 提交端点 schema / `session_version` 模型均无此字段,前端发了被丢 |
|
||||
| `active_skills` 写入 run config | ❌ 缺 | `task_exec.py` 三处 config 只设 `thread_id`(约 L301/L389/L780) |
|
||||
| 技能正文对模型可读(跨 backend) | ❌ 缺 | §1.5 缺口 |
|
||||
| built-in 内置技能文件 | ❌ 仓库无(非恢复必需) | `SKILLS_ROOT/built-in/` 为空 |
|
||||
|
||||
要点:**前端 + 管理/存储后端已 100% 就绪**,缺的全在「把已上传的租户技能在运行时喂进 agent」这一段。
|
||||
|
||||
### 3.2 关键技术判断:命名空间缺口的本质与三种解法
|
||||
原设计 §7.1/§5 坚持「Skill 与工作区两套后端互不混用、Skill 独立走磁盘」。在 0.6.8 下这反而让模型读不到技能正文(§1.5)。解法(恢复时三选一):
|
||||
|
||||
- **Option 1(推荐,最简、零 deepagents 内核改动)**:任务启动时把「本租户 `enabled` 且本轮 `active_skills` 命中」的技能 bundle 复制进**当前会话工作区**的 `/skills/` 子树,中间件 sources 指向工作区内该子树(或直接用原生 `skills=`)。技能体积小(≤10MB/个),复制开销可忽略;治理过滤仍由 `TenantSkillsMiddleware` 子类完成。← 修正原设计「两套 backend 互不混用」的取向。
|
||||
- **Option 2**:实现 overlay backend,`/skills/**` 路由到 `SKILLS_ROOT`、其余到工作区。更优雅、免复制,但要实现 `BackendProtocol` 包装,多 1–2 天。
|
||||
- **Option 3**:让 `SkillsMiddleware` 非渐进地把技能正文直接注入 prompt(放弃 progressive disclosure)。仅适合技能少且小,不推荐。
|
||||
|
||||
### 3.3 改动块(全在后端 `linsight`,按依赖排序)
|
||||
- **A. 技能正文对模型可读**:落 §3.2 的 Option 1(或 2)。关键文件 `agent_factory.py`(backend/sources 组装)、`skill_store.py`(复制源路径,`builtin_dir/tenant_dir/skill_dir` 已现成)。
|
||||
- **B. `skills` 字段穿透到 run config**:① 提交/启动端点 request schema 增加 `skills: list[str]`(接住前端已发字段);② 随会话带到 worker(持久 `session_version` 或随 Redis 队列 payload);③ `task_exec.py` 三处(含 resume 路径)`config.configurable` 注入 `active_skills`(契约同 design §7.2:`[]`=禁用全部自定义技能,缺键=不约束仅兜底)。关键文件:`linsight/api/` 提交端点 + `domain/schemas/` + `task_exec.py`。
|
||||
- **C. 装配中间件**:`create_linsight_agent()` 调 `make_skills_middleware(tenant_id)`(`session_model` 自带 tenant_id)加进 `middlewares`,恢复 system prompt 对技能的条件化介绍。`make_skills_middleware` 已实现,基本是「解开禁用 + 接好 backend/sources」。
|
||||
- **D. 实测复核 + 测试**:先实测复核 §1.5 的两个论断(shadow 不复现 / 正文可读断裂是否真实),再据此定 A 块方案;复用 `test/linsight/test_skill_middleware.py`(白名单已覆盖),补 agent 装配级集成测试 + 跨租户隔离用例。
|
||||
|
||||
### 3.4 与原设计 §7 的差异修正(须在落地时回链标注)
|
||||
1. **§7.2 双中间件 → 单 subclass**:已发生(deviation D8),design §7.2 的「两个中间件 + `SkillWhitelistMiddleware → SkillsMiddleware → GenerativeUIMiddleware` 顺序」描述为历史,实际以 `TenantSkillsMiddleware` 单类为准。
|
||||
2. **§7.1/§5「两套 backend 互不混用」修正**:为闭合 progressive disclosure,恢复时技能需对工作区 `read_file` 可达(Option 1 复制进工作区,或 Option 2 overlay)。
|
||||
|
||||
---
|
||||
|
||||
## 四、工作量与风险
|
||||
- **难度**:中等,**无架构级阻塞**。当年的「shadow 阻塞」在 0.6.8 下已消解;真正要做的命名空间缺口有 1 天级成熟解法。
|
||||
- **核心恢复工作量(A+B+C+D,仅租户自定义技能)**:约 **5–9 人天**,单人 **1–2 周** wall-clock。
|
||||
- **不在范围(勿混入估算)**:built-in 编写、热更新、实时同步——产品增强,非「恢复入口」必需。
|
||||
- **风险**:① A 块若选 Option 2 overlay 略增成本;② 多节点部署须满足 design §7.1 的 `SKILLS_ROOT` 共享卷约束;③ 多租户隔离 + DM8 双库回归须在 D 块补测;④ §1.5 缺口为静态结论,须先实测复核。
|
||||
|
||||
## 五、验证方式(恢复实施后)
|
||||
1. 本地起前后端 + 连 test 中间件;平台端建/传一个技能并启用。
|
||||
2. 客户端任务模式勾选该技能发起任务 → 后端日志确认 `active_skills` 进 config、中间件加载到该技能。
|
||||
3. 让任务产出交付物 → 确认文件落会话工作区 `output/`(不在技能库),即 shadow 不复现(AC-R3)。
|
||||
4. 让 prompt 命中技能 → 确认模型成功 `read_file` 技能正文并按其指引执行(AC-R2)。
|
||||
5. 三态白名单 + 跨租户隔离用 `test/linsight/` 集成测试守护(AC-R1/AC-R4)。
|
||||
@@ -561,6 +561,8 @@ Linsight 独立 Worker 多进程,**子进程不继承 API 进程的租户上
|
||||
|
||||
## 7. Skill 存储与中间件
|
||||
|
||||
> ⚠️ **现状更新(2026-06-24)**:Skill 运行时注入于 2026-06-16 临时禁用(交付物管道优先);§7.2 的「双中间件」已演进为单 subclass `TenantSkillsMiddleware`(deviation D8)。当年「shadow 阻塞」在 deepagents 0.6.8 下已不成立,恢复无架构级阻塞。完整脉络(原设计 → 演进 → 禁用 → 再核查)、遗留缺口与恢复方案见《[灵思任务模式 Skill 能力恢复方案](../../../docs/PRD/2.6%20灵思%20deepagents%20迁移%20PRD/灵思任务模式%20Skill%20能力恢复方案.md)》。**✅ 2026-06-24 已按 Fork X(复制时过滤)恢复(代码+单测,端到端待跑)**:任务启动把命中技能复制进工作区 `/skills/`,`SkillsMiddleware` 经指向缓存的 `FilesystemBackend` 枚举;`TenantSkillsMiddleware` 单 subclass 转休眠,白名单语义迁至 `skill_provisioning`。详见恢复方案「实现状态」。
|
||||
|
||||
### 7.1 目录结构
|
||||
|
||||
```
|
||||
|
||||
@@ -24,6 +24,8 @@
|
||||
| AC-7 | 多租户隔离、权限、DM8 双库回归全部通过 | PRD §4.7(FR-7.x,含 §4.7.7 任务模式菜单权限) | design §6 |
|
||||
| AC-8 | 自研 `bisheng_langchain/linsight` ReAct 内核及 SOP 动态生成链路代码下线 | PRD §3.4 | design §1/§8.6 |
|
||||
|
||||
> 📌 **AC-2 运行时现状(2026-06-24)**:Skill **运行时注入**于 2026-06-16 临时禁用——管理链路(AC-3)端到端仍可用,运行时触发(AC-2)待恢复。无架构级阻塞,恢复方案与现状再核查见《[灵思任务模式 Skill 能力恢复方案](../../../docs/PRD/2.6%20灵思%20deepagents%20迁移%20PRD/灵思任务模式%20Skill%20能力恢复方案.md)》。**✅ 2026-06-24 运行时注入已按 Fork X(复制时过滤)恢复(代码+单测绿)**;AC-2 命中率评测待端到端实测后重跑。
|
||||
|
||||
## 3. 边界情况
|
||||
|
||||
全部边界与异常表在 PRD 各节(§4.1.9 / §4.2.5 / §4.3.6 / §4.4.5 / §4.5.10 / §4.6.5 / §4.7.6)与 design 各 §x.x 边界异常表,此处不复制。**范围红线(非目标)**:不做 Skill 自动挂载/learning loop/marketplace/Interpreter Skill/双引擎共存/邀请码(PRD §1.3 N1–N7);不做旧灵思模式兼容(一次性切换)。
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
"""F035: add linsight_session_version.skills column (per-run skill selection).
|
||||
|
||||
Task mode persists the skill names the user picked for a run so the worker can
|
||||
copy the matched (governance-enabled ∩ selected) skill bundles into the session
|
||||
workspace at startup, and so resume/continue (which reload this row) materialize
|
||||
the same skills. One nullable JSON column carries the name list.
|
||||
|
||||
Nullable for backward compatibility: sessions created before this column existed
|
||||
read as NULL → no skills. ``JsonType`` keeps DM8/MySQL compatible.
|
||||
|
||||
Revision ID: f035_linsight_skills
|
||||
Revises: f035_linsight_status_varchar
|
||||
Create Date: 2026-06-24
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from typing import Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
from bisheng.core.database.dialect_helpers import JsonType, column_exists
|
||||
|
||||
revision: str = "f035_linsight_skills"
|
||||
down_revision: Union[str, Sequence[str], None] = "f035_linsight_status_varchar"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
_TABLE = "linsight_session_version"
|
||||
_COLUMN = "skills"
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not column_exists(conn, _TABLE, _COLUMN):
|
||||
op.add_column(
|
||||
_TABLE,
|
||||
sa.Column(_COLUMN, JsonType, nullable=True, comment="Selected skill names for this run"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if column_exists(conn, _TABLE, _COLUMN):
|
||||
op.drop_column(_TABLE, _COLUMN)
|
||||
@@ -81,6 +81,13 @@ class LinsightSessionVersionBase(SQLModelSerializable):
|
||||
# F035: per-task execution model id chosen at submit time (nullable; falls
|
||||
# back to the tenant linsight_default_model_id when empty).
|
||||
model: str | None = Field(None, description="Per-task execution model id", sa_type=Text, nullable=True)
|
||||
# F035 Track D: skill names the user picked for this run, persisted so resume /
|
||||
# continue (which reload this row) materialize the same skills. The worker copies
|
||||
# the matched bundles into the workspace at startup; unselected ones never load.
|
||||
# Nullable for back-compat with sessions created before this column existed.
|
||||
skills: list[str] | None = Field(
|
||||
None, description="Selected skill names for this run", sa_column=Column(JsonType, nullable=True)
|
||||
)
|
||||
sop: str | None = Field(None, description="SOPContents", sa_type=Text, nullable=True)
|
||||
output_result: dict | None = Field(None, description="Output Results", sa_column=Column(JsonType, nullable=True))
|
||||
status: SessionVersionStatusEnum = Field(
|
||||
|
||||
@@ -48,6 +48,12 @@ class LinsightQuestionSubmitSchema(BaseModel):
|
||||
# F035: per-task selected execution model id; None falls back to the tenant
|
||||
# ``linsight_default_model_id`` at resolve time (agent_factory._resolve_model).
|
||||
model: str | None = Field(None, description="Per-task selected execution model id")
|
||||
# F035 Track D: skill names the user picked for this run. The worker copies the
|
||||
# matched (governance-enabled ∩ selected) bundles into the session workspace at
|
||||
# startup (skill_provisioning.materialize_session_skills) — the copy IS the
|
||||
# whitelist gate, so unselected skills never reach the agent. [] disables all;
|
||||
# None (non-UI callers) means "no per-run constraint" → every enabled skill.
|
||||
skills: list[str] | None = Field(None, description="Selected skill names for this run")
|
||||
# F035: continue an existing session (a follow-up round in the same 会话).
|
||||
# None creates a brand-new session.
|
||||
session_id: str | None = Field(None, description="Existing session id to continue; None creates a new session")
|
||||
|
||||
@@ -1,18 +1,21 @@
|
||||
"""Skill loading + per-run whitelist middleware for the deepagents kernel.
|
||||
|
||||
⚠️ CURRENTLY DISABLED / NOT WIRED (F035, 2026-06-16). The production agent
|
||||
assembly (``agent_factory.create_linsight_agent``) injects only
|
||||
``_ToolExclusionMiddleware`` — this middleware is intentionally NOT in that list,
|
||||
and ``make_skills_middleware`` has no production caller. Two independent reasons:
|
||||
(1) its own ``FilesystemBackend`` (below) shadows the workspace file tools, so
|
||||
deliverables would land in the skills store instead of ``output/`` (same hazard
|
||||
documented in ``agent_factory``); (2) the per-run whitelist key ``active_skills``
|
||||
is never written into the run config (``task_exec`` sets only ``thread_id``), so
|
||||
the second gate would be a no-op even if re-enabled. Re-enable preconditions
|
||||
(Track D Phase 2): give skills a file-tool namespace separate from the workspace
|
||||
AND thread ``active_skills`` into the worker run config. Kept as forward-compatible
|
||||
WIP — the Skill *management* CRUD/upload layer (skill_service / skill_store /
|
||||
``/api/v1/linsight/skill``) is live and independent of this runtime injection.
|
||||
⚠️ DORMANT / SUPERSEDED (F035, 2026-06-24). This subclass is no longer wired into
|
||||
``agent_factory.create_linsight_agent`` and ``make_skills_middleware`` has no
|
||||
production caller. The Skill runtime was restored via **Fork X (copy-time gate)**
|
||||
instead: at task startup ``skill_provisioning.materialize_session_skills`` copies
|
||||
only the ``governance-enabled ∩ user-selected`` bundles into the session workspace
|
||||
``/skills/`` subtree, then a plain ``deepagents.SkillsMiddleware`` (backed by a
|
||||
``FilesystemBackend`` over the workspace cache) enumerates them. Because the copy
|
||||
is the whitelist gate, the model physically cannot see an unselected skill — so
|
||||
the per-run ``active_skills`` config key and the runtime ``_skill_allowed`` filter
|
||||
below are NOT needed and NOT threaded. The whitelist *semantics* (built-in always
|
||||
on; tenant skills need ``enabled`` + per-run selection; ``[]`` disables all) now
|
||||
live in ``materialize_session_skills`` and are covered by its tests.
|
||||
|
||||
This class is kept (not deleted) for reference and as a fallback should we ever
|
||||
move filtering back to runtime. The Skill *management* CRUD/upload layer
|
||||
(skill_service / skill_store / ``/api/v1/linsight/skill``) is live and orthogonal.
|
||||
|
||||
F035 Track D (design §7.2, deviation D8): deepagents 0.6.x has no native
|
||||
whitelist hook — ``SkillsMiddleware`` only loads sources. We subclass it and
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
"""Copy a run's selected skill bundles into the session workspace (F035, Fork X).
|
||||
|
||||
deepagents' ``SkillsMiddleware`` discovers a skill from the *directory* entries an
|
||||
``ls`` returns (``is_dir=True``) and the model reads the body via the workspace
|
||||
``read_file`` tool. Two facts make a naive wiring impossible:
|
||||
|
||||
* the session ``WorkspaceBackend`` lists MinIO recursively and returns only
|
||||
*file* entries (``is_dir=False``) — deepagents' native ``skills=`` param
|
||||
pointed at it would discover zero skills;
|
||||
* skill bundles live in a separate ``SKILLS_ROOT`` the workspace ``read_file``
|
||||
cannot reach.
|
||||
|
||||
So at task startup we copy the bundles this run is allowed to use into the
|
||||
workspace ``/skills/`` subtree. ``WorkspaceBackend.aupload_files`` write-throughs
|
||||
to both MinIO and the local cache, after which a plain ``SkillsMiddleware`` backed
|
||||
by a ``FilesystemBackend`` over that cache can enumerate them (real on-disk dirs,
|
||||
``is_dir``-aware) and the model reads the very same ``/skills/<name>/SKILL.md``
|
||||
paths back through the workspace backend.
|
||||
|
||||
The copy IS the whitelist gate (Fork X): only ``enabled (DB governance) ∩
|
||||
selected (this run)`` bundles are materialized, so the model physically cannot
|
||||
see a skill it was not granted — no per-run config key, no runtime filter. This
|
||||
replaces the dormant ``TenantSkillsMiddleware`` runtime whitelist.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from bisheng.linsight.domain.models.linsight_skill import LinsightSkillDao
|
||||
from bisheng.linsight.domain.services.skill_store import SkillStore
|
||||
|
||||
WORKSPACE_SKILLS_DIR = "skills"
|
||||
"""Workspace subtree the copied bundles live under (``/skills/<name>/...``)."""
|
||||
|
||||
|
||||
async def materialize_session_skills(
|
||||
backend,
|
||||
tenant_id: int,
|
||||
selected: list[str] | None,
|
||||
store: SkillStore | None = None,
|
||||
) -> list[str]:
|
||||
"""Copy allowed skill bundles into the workspace ``/skills/`` subtree.
|
||||
|
||||
Args:
|
||||
backend: the session ``WorkspaceBackend`` (write-throughs to MinIO+cache).
|
||||
tenant_id: owning tenant; scopes the on-disk bundle source path.
|
||||
selected: skill names picked for this run. ``[]`` = none (all disabled);
|
||||
``None`` = unconstrained (non-UI fallback — copy every enabled skill);
|
||||
``["a", "b"]`` = exactly those that are also governance-enabled.
|
||||
store: skill disk store (injectable for tests).
|
||||
|
||||
Returns:
|
||||
The skill names actually materialized. Empty when nothing matched — the
|
||||
caller then skips attaching the skills middleware entirely.
|
||||
"""
|
||||
# [] = the UI explicitly disabled every skill for this run; copy nothing.
|
||||
if selected == []:
|
||||
return []
|
||||
|
||||
store = store or SkillStore()
|
||||
# Governance gate, scoped to the current tenant (LinsightSkillDao.list_enabled
|
||||
# uses strict_tenant_filter); the worker has already restored tenant context.
|
||||
enabled = {skill.name for skill in await LinsightSkillDao.list_enabled()}
|
||||
wanted = enabled if selected is None else {name for name in selected if name in enabled}
|
||||
if not wanted:
|
||||
return []
|
||||
|
||||
copied: list[str] = []
|
||||
for name in sorted(wanted):
|
||||
try:
|
||||
pairs = [
|
||||
(
|
||||
f"/{WORKSPACE_SKILLS_DIR}/{name}/{entry['path']}",
|
||||
store.read_bytes(tenant_id, name, entry["path"]),
|
||||
)
|
||||
for entry in store.list_files(tenant_id, name)
|
||||
]
|
||||
if not pairs:
|
||||
logger.warning("linsight skill %r (tenant %s) has no files on disk; skipping", name, tenant_id)
|
||||
continue
|
||||
responses = await backend.aupload_files(pairs)
|
||||
failed = [r for r in responses if getattr(r, "error", None)]
|
||||
if failed:
|
||||
logger.warning("linsight skill %r copy had failures, not advertising: %s", name, failed)
|
||||
continue
|
||||
copied.append(name)
|
||||
except Exception:
|
||||
# Best-effort: one malformed/unreadable bundle must never abort the task.
|
||||
logger.exception("failed to materialize linsight skill %r (tenant %s)", name, tenant_id)
|
||||
logger.info(
|
||||
"linsight skill provisioning: tenant=%s selected=%r enabled=%s -> materialized %s",
|
||||
tenant_id,
|
||||
selected,
|
||||
sorted(enabled),
|
||||
copied,
|
||||
)
|
||||
return copied
|
||||
@@ -220,6 +220,19 @@ class SkillStore:
|
||||
raise FileNotFoundError(str(target))
|
||||
return target.read_text(encoding="utf-8", errors="replace")
|
||||
|
||||
def read_bytes(self, tenant_id: int, name: str, rel: str) -> bytes:
|
||||
"""Read a bundle file as raw bytes (binary-safe).
|
||||
|
||||
``read_text`` decodes utf-8 with ``errors="replace"`` and is lossy for
|
||||
binary assets (images, fonts) bundled alongside SKILL.md. The skill
|
||||
copy-into-workspace path (skill_provisioning) needs faithful bytes, so it
|
||||
reads through here instead.
|
||||
"""
|
||||
target = self.skill_dir(tenant_id, name) / _safe_rel_path(rel)
|
||||
if not target.is_file():
|
||||
raise FileNotFoundError(str(target))
|
||||
return target.read_bytes()
|
||||
|
||||
def list_files(self, tenant_id: int, name: str) -> list[dict]:
|
||||
"""Bundle file tree as [{path, size}], SKILL.md first, then sorted."""
|
||||
base = self.skill_dir(tenant_id, name)
|
||||
|
||||
@@ -227,6 +227,7 @@ class LinsightWorkbenchImpl:
|
||||
knowledge_space_ids=submit_obj.knowledge_space_ids,
|
||||
files=processed_files,
|
||||
model=submit_obj.model,
|
||||
skills=submit_obj.skills,
|
||||
)
|
||||
linsight_session_version = await LinsightSessionVersionDao.insert_one(linsight_session_version)
|
||||
|
||||
|
||||
@@ -593,10 +593,18 @@ class LinsightWorkflowTask:
|
||||
resume path passes a Redis-backed ``checkpointer`` so the parked
|
||||
interrupt checkpoint (thread_id = session_version_id) is located.
|
||||
"""
|
||||
from bisheng.linsight.domain.services.skill_provisioning import materialize_session_skills
|
||||
from bisheng.linsight.domain.services.workspace_backend import WorkspaceBackend
|
||||
|
||||
minio = await get_minio_storage()
|
||||
backend = WorkspaceBackend(svid=session_model.id, minio=minio, file_dir=self.file_dir)
|
||||
# F035 Fork X: copy this run's allowed skill bundles into the workspace
|
||||
# /skills/ subtree (governance-enabled ∩ user-selected — the copy IS the
|
||||
# whitelist gate). Re-runs harmlessly on resume/continue since this builds a
|
||||
# fresh agent each time. skills_present gates attaching the skills middleware.
|
||||
copied_skills = await materialize_session_skills(
|
||||
backend, session_model.tenant_id, getattr(session_model, "skills", None)
|
||||
)
|
||||
return await create_linsight_agent(
|
||||
session_model=session_model,
|
||||
tools=tools,
|
||||
@@ -605,6 +613,7 @@ class LinsightWorkflowTask:
|
||||
svid=session_model.id,
|
||||
checkpointer=checkpointer,
|
||||
backend=backend,
|
||||
skills_present=bool(copied_skills),
|
||||
)
|
||||
|
||||
async def _seed_workspace_from_previous(self, session_model: LinsightSessionVersion) -> None:
|
||||
|
||||
@@ -0,0 +1,200 @@
|
||||
"""F035 Fork X — skill copy-time gate + workspace enumeration loop.
|
||||
|
||||
``materialize_session_skills`` is the whitelist gate that replaced the dormant
|
||||
``TenantSkillsMiddleware`` runtime filter: it copies only the
|
||||
``governance-enabled ∩ user-selected`` bundles into the session workspace
|
||||
``/skills/`` subtree. These tests pin the C3 contract semantics (moved here from
|
||||
``test_skill_middleware``):
|
||||
|
||||
- ``selected=["a"]`` → only governance-enabled selected names copied;
|
||||
- ``selected=[]`` → nothing copied (UI disabled all);
|
||||
- ``selected=None`` → every enabled skill copied (non-UI fallback);
|
||||
- a DB-disabled skill is never copied even if selected;
|
||||
- bundle bytes (incl. binary assets) are copied losslessly;
|
||||
- the on-disk source is tenant-scoped (cross-tenant read yields nothing).
|
||||
|
||||
The final test runs the full loop: copy → real deepagents ``SkillsMiddleware``
|
||||
enumerates the subtree → the injected path is the one the workspace ``read_file``
|
||||
would resolve back to (``normalize_workspace_path`` closes the cross-backend loop).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from bisheng.linsight.domain.services import skill_provisioning
|
||||
from bisheng.linsight.domain.services.skill_provisioning import WORKSPACE_SKILLS_DIR, materialize_session_skills
|
||||
from bisheng.linsight.domain.services.skill_store import SkillStore
|
||||
|
||||
TENANT = 1
|
||||
OTHER_TENANT = 2
|
||||
|
||||
# A real PNG header — invalid UTF-8, so read_text(errors="replace") would corrupt it.
|
||||
BINARY_ASSET = b"\x89PNG\r\n\x1a\n\x00\x01\xff\xfe\xfd\x00template"
|
||||
|
||||
|
||||
def _write_skill(base: Path, name: str, *, assets: dict[str, bytes] | None = None) -> None:
|
||||
d = base / name
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
(d / "SKILL.md").write_text(
|
||||
f"---\nname: {name}\ndescription: desc of {name}\n---\n\n# {name}\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
for rel, data in (assets or {}).items():
|
||||
target = d / rel
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(data)
|
||||
|
||||
|
||||
class _Resp:
|
||||
def __init__(self, path: str, error: str | None = None):
|
||||
self.path = path
|
||||
self.error = error
|
||||
|
||||
|
||||
class _CacheBackend:
|
||||
"""Minimal WorkspaceBackend stand-in: aupload_files write-throughs to a dir.
|
||||
|
||||
Mirrors ``WorkspaceBackend._cache_write`` (strip leading ``/``, write bytes),
|
||||
so a FilesystemBackend rooted at ``file_dir`` sees exactly what the real
|
||||
write-through cache would after the copy.
|
||||
"""
|
||||
|
||||
def __init__(self, file_dir: Path):
|
||||
self.file_dir = Path(file_dir)
|
||||
self.uploaded: list[str] = []
|
||||
|
||||
async def aupload_files(self, files):
|
||||
out = []
|
||||
for raw_path, data in files:
|
||||
rel = raw_path.lstrip("/")
|
||||
target = self.file_dir / rel
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(data)
|
||||
self.uploaded.append(raw_path)
|
||||
out.append(_Resp("/" + rel))
|
||||
return out
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def store(tmp_path) -> SkillStore:
|
||||
s = SkillStore(root=tmp_path / "skills_root")
|
||||
_write_skill(s.tenant_dir(TENANT), "biao-shu-zhuan-xie")
|
||||
_write_skill(s.tenant_dir(TENANT), "he-tong-shen-yue")
|
||||
_write_skill(s.tenant_dir(TENANT), "ting-yong-ji-neng") # exists on disk but DB-disabled
|
||||
return s
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def backend(tmp_path) -> _CacheBackend:
|
||||
return _CacheBackend(tmp_path / "workspace_cache")
|
||||
|
||||
|
||||
class _EnabledSkill:
|
||||
"""Stand-in for a LinsightSkill row (only ``.name`` is read)."""
|
||||
|
||||
def __init__(self, name: str):
|
||||
self.name = name
|
||||
|
||||
|
||||
def _patch_enabled(monkeypatch, names: set[str]) -> None:
|
||||
async def _fake_list_enabled():
|
||||
return [_EnabledSkill(n) for n in names]
|
||||
|
||||
monkeypatch.setattr(skill_provisioning.LinsightSkillDao, "list_enabled", _fake_list_enabled)
|
||||
|
||||
|
||||
ENABLED = {"biao-shu-zhuan-xie", "he-tong-shen-yue"} # ting-yong-ji-neng disabled in DB
|
||||
|
||||
|
||||
def _copied_rel_paths(backend: _CacheBackend) -> set[str]:
|
||||
return set(backend.uploaded)
|
||||
|
||||
|
||||
class TestGate:
|
||||
async def test_selected_subset_copies_only_those(self, monkeypatch, store, backend):
|
||||
_patch_enabled(monkeypatch, ENABLED)
|
||||
copied = await materialize_session_skills(backend, TENANT, ["biao-shu-zhuan-xie"], store=store)
|
||||
assert copied == ["biao-shu-zhuan-xie"]
|
||||
assert _copied_rel_paths(backend) == {"/skills/biao-shu-zhuan-xie/SKILL.md"}
|
||||
|
||||
async def test_empty_selection_copies_nothing(self, monkeypatch, store, backend):
|
||||
_patch_enabled(monkeypatch, ENABLED)
|
||||
copied = await materialize_session_skills(backend, TENANT, [], store=store)
|
||||
assert copied == []
|
||||
assert backend.uploaded == []
|
||||
|
||||
async def test_none_selection_copies_all_enabled(self, monkeypatch, store, backend):
|
||||
_patch_enabled(monkeypatch, ENABLED)
|
||||
copied = await materialize_session_skills(backend, TENANT, None, store=store)
|
||||
assert copied == sorted(ENABLED)
|
||||
assert _copied_rel_paths(backend) == {
|
||||
"/skills/biao-shu-zhuan-xie/SKILL.md",
|
||||
"/skills/he-tong-shen-yue/SKILL.md",
|
||||
}
|
||||
|
||||
async def test_db_disabled_skill_never_copied_even_if_selected(self, monkeypatch, store, backend):
|
||||
_patch_enabled(monkeypatch, ENABLED)
|
||||
copied = await materialize_session_skills(
|
||||
backend, TENANT, ["ting-yong-ji-neng", "biao-shu-zhuan-xie"], store=store
|
||||
)
|
||||
assert copied == ["biao-shu-zhuan-xie"]
|
||||
|
||||
async def test_unknown_selected_name_ignored(self, monkeypatch, store, backend):
|
||||
_patch_enabled(monkeypatch, ENABLED)
|
||||
copied = await materialize_session_skills(backend, TENANT, ["does-not-exist"], store=store)
|
||||
assert copied == []
|
||||
|
||||
|
||||
class TestByteFidelity:
|
||||
async def test_binary_asset_copied_losslessly(self, monkeypatch, tmp_path, backend):
|
||||
store = SkillStore(root=tmp_path / "skills_root")
|
||||
_write_skill(store.tenant_dir(TENANT), "with-asset", assets={"templates/logo.png": BINARY_ASSET})
|
||||
_patch_enabled(monkeypatch, {"with-asset"})
|
||||
|
||||
copied = await materialize_session_skills(backend, TENANT, ["with-asset"], store=store)
|
||||
assert copied == ["with-asset"]
|
||||
# The binary asset round-trips byte-identical (read_bytes, not lossy read_text).
|
||||
cached = backend.file_dir / "skills" / "with-asset" / "templates" / "logo.png"
|
||||
assert cached.read_bytes() == BINARY_ASSET
|
||||
|
||||
|
||||
class TestCrossTenant:
|
||||
async def test_other_tenant_cannot_read_disk_bundle(self, monkeypatch, store, backend):
|
||||
# DAO gate is tenant-scoped in production (strict_tenant_filter); here even if
|
||||
# the name were "enabled", the on-disk source path is keyed by tenant_id, so a
|
||||
# different tenant resolves an empty bundle and copies nothing.
|
||||
_patch_enabled(monkeypatch, {"biao-shu-zhuan-xie"})
|
||||
copied = await materialize_session_skills(backend, OTHER_TENANT, ["biao-shu-zhuan-xie"], store=store)
|
||||
assert copied == []
|
||||
assert backend.uploaded == []
|
||||
|
||||
|
||||
class TestEnumerationLoop:
|
||||
async def test_copied_skill_is_enumerated_and_path_resolves(self, monkeypatch, store, backend):
|
||||
"""Full Fork X loop: copy → real SkillsMiddleware enumerates → path consistency."""
|
||||
from deepagents.backends.filesystem import FilesystemBackend
|
||||
from deepagents.middleware.skills import SkillsMiddleware
|
||||
|
||||
from bisheng.linsight.domain.services.workspace_backend import normalize_workspace_path
|
||||
|
||||
_patch_enabled(monkeypatch, ENABLED)
|
||||
await materialize_session_skills(backend, TENANT, ["biao-shu-zhuan-xie"], store=store)
|
||||
|
||||
# Enumerate via a FilesystemBackend over the same cache dir the copy wrote to —
|
||||
# this is exactly what agent_factory attaches when skills_present is True.
|
||||
mw = SkillsMiddleware(
|
||||
backend=FilesystemBackend(root_dir=str(backend.file_dir), virtual_mode=True),
|
||||
sources=[(f"/{WORKSPACE_SKILLS_DIR}/", "Skills")],
|
||||
)
|
||||
update = mw.before_agent({}, MagicMock(), {"configurable": {}})
|
||||
skills = {s["name"]: s for s in update["skills_metadata"]}
|
||||
assert "biao-shu-zhuan-xie" in skills
|
||||
|
||||
injected_path = skills["biao-shu-zhuan-xie"]["path"]
|
||||
# The path the model is told to read_file is the same one the workspace
|
||||
# backend resolves back to the copied bundle (cross-backend loop closed).
|
||||
assert normalize_workspace_path(injected_path) == "skills/biao-shu-zhuan-xie/SKILL.md"
|
||||
@@ -590,13 +590,18 @@ const AiChatInput = memo(
|
||||
onChange={(next) => {
|
||||
setDailySkills(next);
|
||||
close();
|
||||
// Picking a skill enters task mode. On the daily
|
||||
// welcome page do it in place (callback); otherwise
|
||||
// fall back to navigating to the linsight landing.
|
||||
if (onToggleTaskMode) {
|
||||
onToggleTaskMode();
|
||||
} else {
|
||||
navigate('/linsight/new');
|
||||
// Picking a skill ENTERS task mode only when not
|
||||
// already in it. onToggleTaskMode is a toggle, so
|
||||
// calling it while task mode is already active flips
|
||||
// it OFF — selecting a skill must never exit task
|
||||
// mode. On the daily welcome page enter in place
|
||||
// (callback); otherwise navigate to the linsight landing.
|
||||
if (!taskMode) {
|
||||
if (onToggleTaskMode) {
|
||||
onToggleTaskMode();
|
||||
} else {
|
||||
navigate('/linsight/new');
|
||||
}
|
||||
}
|
||||
}}
|
||||
/>
|
||||
|
||||
Reference in New Issue
Block a user