diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000..62f46b99a --- /dev/null +++ b/docs/README.md @@ -0,0 +1,130 @@ +# IoT DC3 文档导航(docs/) + +> 本文件是 `docs/` 目录的中文导读与分类索引:13 篇文档分三类,逐篇给出摘要、状态与阅读建议。 +> 仓库工程规范见根目录 [AGENTS.md](../AGENTS.md)。目录内全部文档均为简体中文。 + +## 总览 + +| 分类 | 文档 | 状态 | +|------|------|------| +| 选型指南 | [db-dialects.md](./db-dialects.md) 关系库选型 | 有效 | +| 选型指南 | [tsdb-stores.md](./tsdb-stores.md) 时序存储选型 | 有效 | +| 选型指南 | [mq-brokers.md](./mq-brokers.md) 消息中间件选型 | 有效 | +| 设计/架构 | [design/relational-r2dbc.md](./design/relational-r2dbc.md) R2DBC 关系访问层 | 已批准(2026-08-28),待实施 | +| 设计/架构 | [design/tsdb-abstraction.md](./design/tsdb-abstraction.md) 时序存储 Port | 已落地 | +| 设计/架构 | [design/mq-abstraction.md](./design/mq-abstraction.md) 消息中间件 Port | 已落地 | +| 设计/架构 | [design/storage-abstraction.md](./design/storage-abstraction.md) 存储抽象总纲 | 部分被取代(§3/§4) | +| 设计/架构 | [design/mcp-runtime-overhaul.md](./design/mcp-runtime-overhaul.md) MCP 运行时重构 | 提案,未实施 | +| 设计/架构 | [design/dc3-client-sdk.md](./design/dc3-client-sdk.md) 客户端 SDK | 提案(phase 0 已落地) | +| 设计/架构 | [design/frontend-three-terminal-ux.md](./design/frontend-three-terminal-ux.md) 前端三终端 UX ADR | ADR | +| 分析对比 | [comparison-dgiot.md](./comparison-dgiot.md) DG-IoT 对比借鉴 | 开放评审,未排期 | +| 分析对比 | [comparison-supos-free.md](./comparison-supos-free.md) supOS-Free 对比借鉴 | 开放评审,未排期 | +| 分析对比 | [dc3-thing-model-review.md](./dc3-thing-model-review.md) 物模型代码级核实 | 事实判定(非路线图) | + +--- + +## 一、选型指南(部署 / 运维决策用) + +三篇对应三个可插拔维度:关系库、时序库、消息中间件。**部署换什么、门槛是什么、能力差异是什么,先看这里。** + +### 1. [关系库选型指南](./db-dialects.md) + +- **回答的问题**:PostgreSQL / MySQL / MariaDB 三方言怎么选,差异到底在哪。 +- **关键结论**:PostgreSQL 默认;MySQL 硬门槛 ≥8.0(窗口函数、CTE、SKIP LOCKED);MariaDB ≥10.6。 +- **最有用的一张表**:"现状矩阵"——upsert 三种方言形态(`ON CONFLICT` / `AS new` 行别名 / `VALUES(col)`)、RETURNING 已弃用、 + 咨询锁(`pg_advisory_xact_lock` 事务级 vs `GET_LOCK` 会话级须 try/finally)、`operate_time` 触发器 vs 列属性、三方言契约套件 8/8。 + +### 2. [时序存储选型指南](./tsdb-stores.md) + +- **回答的问题**:TimescaleDB / TDengine / InfluxDB / IoTDB 四库,哪些能力是真支持、哪些是如实拒绝。 +- **关键结论**:矩阵按适配器**实际声明**发布而非预估——如 IoTDB 无法按 driver 分组计数(driver 是 measurement 非路径层)、 + InfluxDB 无精确 PERCENTILE(门面精算)、直方图能力两家声明 false(面板零桶降级)。 +- **默认**:TimescaleDB(内嵌 PG,部署最简)。 + +### 3. [消息 broker 选型指南](./mq-brokers.md) + +- **回答的问题**:内部异步面选哪个 broker;南向设备面 MQTT 怎么与之共处("两平面一原则")。 +- **关键结论**:RabbitMQ 默认;Kafka / Pulsar / ActiveMQ / MQTT 5 已认证;RocketMQ 实验性(未认证,自担风险)。 +- **注意**:能力矩阵里"延迟消息"多数是本地回退(fallback)而非原生;Kafka 适配器不配 SASL/TLS。 + +--- + +## 二、设计与架构文档(design/) + +### 存储演进主线(取代链,先看这张图再读文档) + +```text +storage-abstraction.md(2026-08-17 总纲) + ├─ §1-2 三层存储模型、§6 统一配置面 .......... 仍有效(总纲骨架) + ├─ §3 关系方言机制(MyBatis databaseId 双方言)──已落地 R1/R2,后继──→ relational-r2dbc.md(R2DBC 重写,已批准) + └─ §4 时序存储 Port .......................... 已被取代──→ tsdb-abstraction.md(已落地) +``` + +### 4. [基于 Spring Data R2DBC 的关系访问层设计](./design/relational-r2dbc.md) + +- **一句话**:用 Spring Data R2DBC 替换 MyBatis-Plus + JDBC,auth 中心试点,目标是 GraalVM native + 端到端响应式 + 方言自由。 +- **读什么**:§3 事实清单(F1–F15,全部经代码核实);§5 决策表 **D1–D19**(每方言每服务模块、TCK 纪律、编译期租户隔离、 + Facade/gRPC 阻塞边界、timestamptz 编解码、雪花 workerId 分配);§12 P0–P5 阶段计划(P2–P3 为 go/no-go 闸门)。 +- **注意**:分页信封是一次性硬切换(D8,不搞兼容);auth 的 HTTP 分页契约会被有意破坏。 + +### 5. [时序存储抽象(TSDB Port)](./design/tsdb-abstraction.md) + +- **一句话**:把位值时序存储抽成 Port,`dc3-tsdb-core` + 四适配器(timescale/tdengine/influxdb/iotdb)+ TCK,Phase 1–3 已落地并 TCK 认证。 +- **读什么**:port 必须承载的语义(§4)、核心 API(§6);能力差异的落地细节对应 [tsdb-stores.md](./tsdb-stores.md)。 + +### 6. [消息中间件抽象(MQ Port)](./design/mq-abstraction.md) + +- **一句话**:center ↔ driver 异步平面的 broker 抽象,`dc3-mq-core` + 六适配器(rocketmq 实验)+ TCK,已交付。 +- **读什么**:核心 API(§6)、订阅模式与目的地映射(§7)、逐 broker 的硬语义(§8:延迟消息、死信、MQTT 两平面分离)。 + +### 7. [存储抽象总纲](./design/storage-abstraction.md) *部分被取代* + +- **一句话**:整个"可插拔存储"叙事的起点;关系/时序两半已分别演进出后继文档,**只有 §1–2(三层模型)、§5(向量占位)、§6(统一配置面)仍是现行参考**。 +- **读法**:当历史背景或总纲骨架读,不要按 §3/§4 实施。 + +### 8. [MCP 运行时全面重构:内聚的授权契约](./design/mcp-runtime-overhaul.md) *提案* + +- **一句话**:现状一次 `tools/call` 要三次网关→auth 冗余往返且有正确性缺口;目标一次内聚往返、响应式非阻塞、真实 input schema、异步审计。 +- **读什么**:§2.3 的三冗余一缺口(问题陈述最精彩)、§5 目标契约、§9 备选方案。 +- **关联**:实施依赖 relational-r2dbc 的 auth 响应式改造节奏。 + +### 9. [dc3-sdk:框架无关客户端 SDK](./design/dc3-client-sdk.md) *提案(phase 0 已落地)* + +- **一句话**:网关 HTTP 契约现在被 dc3-web(30 个 api 模块)和 dc3-cli(手写 fetch)重复实现,未来 native app 会是第三份; + 目标一个 TypeScript SDK 实现一次,三端消费。 +- **读什么**:边界规则(可 lint 强制)、迁移计划;依赖前端 ADR 的边界纪律。 + +### 10. [前端三终端 UX 架构(ADR)](./design/frontend-three-terminal-ux.md) + +- **一句话**:dc3-web 从桌面优先(1280px 硬底线)转向桌面/平板/手机三终端架构,用公理化推导替代零散 `@media` 补丁。 +- **读什么**:公理、分层模型、断点契约、边界复用纪律——前端任何新页面开工前都该过一遍。 + +--- + +## 三、分析与对比(参考性,未进实施排期) + +### 11. [DG-IoT 与 IoT DC3 对比借鉴](./comparison-dgiot.md) + +- **一句话**:跨项目对比提炼可借鉴点,并澄清双方物模型差异;开放评审中,**不是承诺路线图**。 + +### 12. [supOS-Free 与 IoT DC3 对比借鉴](./comparison-supos-free.md) + +- **一句话**:同上,对标 supOS-Free;结论与 storage/mq 抽象设计互相印证。 + +### 13. [DC3 物模型(Profile)代码级核实与最终判定](./dc3-thing-model-review.md) + +- **一句话**:用 grep 全仓 + 逐文件读源码的方式回答"DC3 的 Profile 是否等价于物模型",证据落到类/方法/表名,给出最终判定。 +- **价值**:回答"DC3 有没有物模型"这一常见问题的权威出处。 + +--- + +## 按任务找文档(推荐阅读路径) + +| 你想做什么 | 按顺序读 | +|------------|----------| +| 部署 / 换库 / 换 broker | [db-dialects](./db-dialects.md) → [tsdb-stores](./tsdb-stores.md) → [mq-brokers](./mq-brokers.md) | +| 理解存储层现状与演进 | [storage-abstraction §1–2](./design/storage-abstraction.md) → [tsdb-abstraction](./design/tsdb-abstraction.md) → [db-dialects](./db-dialects.md) → [relational-r2dbc](./design/relational-r2dbc.md) | +| 参与 auth R2DBC 迁移 | [relational-r2dbc 全文](./design/relational-r2dbc.md)(重点 §5 决策表 D1–D19)→ [db-dialects](./db-dialects.md) → 根目录 AGENTS.md | +| 前端 / 客户端开发 | [frontend-three-terminal-ux](./design/frontend-three-terminal-ux.md) → [dc3-client-sdk](./design/dc3-client-sdk.md) | +| MCP / AI 集成 | [mcp-runtime-overhaul](./design/mcp-runtime-overhaul.md) → [relational-r2dbc §5 D13](./design/relational-r2dbc.md)(OAuth/MCP 聚合拆分) | +| 了解项目定位 / 竞品差异 | [comparison-dgiot](./comparison-dgiot.md) → [comparison-supos-free](./comparison-supos-free.md) → [dc3-thing-model-review](./dc3-thing-model-review.md) | diff --git a/docs/design/dc3-client-sdk.md b/docs/design/dc3-client-sdk.md index 78eb99a9b..d0d8e3911 100644 --- a/docs/design/dc3-client-sdk.md +++ b/docs/design/dc3-client-sdk.md @@ -1,26 +1,25 @@ -# dc3-sdk: Framework-Agnostic Client SDK (Design) +# dc3-sdk：框架无关的客户端 SDK（设计） -Status: proposed (phase 0 landed) · Scope: web / cli / future app · Drivers: A1 (device-independent semantics), boundary -discipline in docs/design/frontend-three-terminal-ux.md +状态：提案（phase 0 已落地） · 范围：web / cli / 未来 app · 驱动因素：A1（设备无关语义）、 +docs/design/frontend-three-terminal-ux.md 中的边界纪律 -## Problem +## 问题 -The HTTP gateway contract is currently implemented twice: +HTTP 网关契约目前被实现了两次： -- dc3-web/src/api/ (30 modules) — thin axios wrappers, coupled to - `@/config/axios` (Element Plus notifications, router-guard 401 flow) and Vite `@/` aliases. -- dc3-cli/src/core/client.ts — hand-rolled fetch calls for auth (token/salt, token/generate, token/cancel). +- dc3-web/src/api/（30 个模块）——薄 axios 封装，与 + `@/config/axios`（Element Plus 通知、路由守卫 401 流程）及 Vite `@/` 别名耦合。 +- dc3-cli/src/core/client.ts——为认证（token/salt、token/generate、token/cancel）手写的 fetch 调用。 -A native app (React Native) would need a third implementation. Every new endpoint, header, and error-code interpretation -must be re-implemented per client — a standing violation of A1. +原生 app（React Native）则需要第三套实现。每个新端点、请求头与错误码解释都必须逐客户端重复实现—— +这是对 A1 的持续性违反。 -## Goal +## 目标 -One TypeScript SDK that implements the gateway contract exactly once and is consumed by web, CLI, and future app -clients. The SDK is framework-agnostic: -no Vue, no Element Plus, no vue-i18n, no axios hard-dependency. +一个 TypeScript SDK，将网关契约精确实现一次，供 web、CLI 与未来的 app 客户端共同消费。该 SDK 框架无关： +无 Vue、无 Element Plus、无 vue-i18n、不硬依赖 axios。 -## Package layout +## 包结构 ```text dc3-sdk/ @@ -36,7 +35,7 @@ dc3-sdk/ └── package.json ``` -### Core contracts +### 核心契约 ```ts // HTTP transport abstraction — web injects axios, RN/node injects fetch. @@ -60,53 +59,51 @@ interface SdkOptions { } ``` -Auth/session: SDK exposes salt/generate/cancel/change-password and a -`TokenStore` abstraction (web: httpOnly cookie — no-op; CLI: memory/file; app: Keychain via host adapter). The SDK never -decides storage policy. +认证/会话：SDK 暴露 salt/generate/cancel/change-password 以及 +`TokenStore` 抽象（web：httpOnly cookie——空实现；CLI：内存/文件；app：经宿主适配器访问 Keychain）。存储策略 +永远不由 SDK 决定。 -### Semantic layer (L1) portability +### 语义层（L1）可移植性 -Entity schemas already became framework-free in this iteration: +实体 schema 在本次迭代中已经去框架化： -- `EntityListConfig` and friends carry labels as translated strings produced by a - `Translator = (key, params?) => string` — Vue's ComposerTranslation satisfies it structurally; 16 config modules no - longer import vue-i18n. -- Next step: move `EntityListConfig` type into the SDK's types layer so web and app render the same schema (configs stay - in the host apps; the shape is shared). +- `EntityListConfig` 及相关类型以翻译后的字符串承载标签，字符串由 + `Translator = (key, params?) => string` 生成——Vue 的 ComposerTranslation 在结构上满足该签名；16 个配置模块 + 不再 import vue-i18n。 +- 下一步：把 `EntityListConfig` 类型移入 SDK 的 types 层，使 web 与 app 渲染同一份 schema + （配置仍留在宿主应用中，共享的是形状）。 -Future option: backend-served JSON schemas via the OpenAPI export (`make openapi`) replace hand-authored configs without -client changes. +未来选项：通过 OpenAPI 导出（`make openapi`）由后端下发 JSON schema，在不改动客户端的前提下取代手写配置。 -## Boundary rules (lint-enforceable) +## 边界规则（可用 lint 强制） -Inside dc3-sdk, imports of the following are forbidden: +在 dc3-sdk 内部，禁止 import 以下内容： - vue / vue-router / pinia / element-plus / vue-i18n -- `@/config/*` host infrastructure -- any browser-only global except behind `typeof window` guards +- `@/config/*` 宿主基础设施 +- 任何仅浏览器才有的全局对象，除非位于 `typeof window` 守卫之后 -Enforce with eslint `no-restricted-imports` / `import/no-extraneous-dependencies` -rules in the SDK package config. +在 SDK 包配置中用 eslint `no-restricted-imports` / `import/no-extraneous-dependencies` +规则强制执行。 -## Migration plan +## 迁移计划 -1. **Phase 0 (landed)**: de-framework L1 configs — `Translator` type, zero vue-i18n imports in the 16 entity config - modules. -2. **Phase 1**: scaffold `dc3-sdk` package (pnpm workspace), port - `PageQuery/PageResult`, `R` envelope, error taxonomy; vitest with mock adapter. -3. **Phase 2**: port the 30 API wrapper modules; contract tests against the backend OpenAPI spec (or recorded fixtures). -4. **Phase 3**: dc3-web consumes the SDK — delete `src/api`, keep the axios adapter + interceptors as the host glue; run - full e2e (608 unit tests + Playwright) before removing anything. -5. **Phase 4**: dc3-cli swaps its hand-rolled fetch auth for the SDK (removes the duplicated salt/generate/cancel flow). -6. **Phase 5**: app client consumes the SDK with a fetch/Keychain adapter. +1. **Phase 0（已落地）**：L1 配置去框架化——引入 `Translator` 类型，16 个实体配置模块零 vue-i18n import。 +2. **Phase 1**：搭建 `dc3-sdk` 包（pnpm workspace），移植 + `PageQuery/PageResult`、`R` 信封、错误分类体系；使用 mock 适配器的 vitest。 +3. **Phase 2**：移植 30 个 API 封装模块；针对后端 OpenAPI 规范（或录制的 fixture）做契约测试。 +4. **Phase 3**：dc3-web 接入 SDK——删除 `src/api`，保留 axios 适配器 + 拦截器作为宿主粘合层；删除任何内容之前 + 先跑完整 e2e（608 个单元测试 + Playwright）。 +5. **Phase 4**：dc3-cli 用 SDK 替换其手写的 fetch 认证（移除重复的 salt/generate/cancel 流程）。 +6. **Phase 5**：app 客户端通过 fetch/Keychain 适配器消费 SDK。 -## Verification +## 验证 -- SDK unit tests run in node with zero DOM/Vue imports (CI assertion: - `import ... from 'vue'` absent in SDK source). -- Web regression gate: pnpm check + lint + full vitest + Playwright e2e. -- CLI gate: dc3-cli vitest suites keep passing against a mock adapter. +- SDK 单元测试在 node 中运行，零 DOM/Vue import（CI 断言： + SDK 源码中不存在 `import ... from 'vue'`）。 +- Web 回归门禁：pnpm check + lint + 完整 vitest + Playwright e2e。 +- CLI 门禁：dc3-cli 的 vitest 套件针对 mock 适配器保持通过。 -## Revision +## 修订记录 -2026-08: v1 — proposal with Phase 0 landed (Translator de-frameworking, 16 modules). +2026-08：v1——提案，Phase 0 已落地（Translator 去框架化，16 个模块）。 diff --git a/docs/design/frontend-three-terminal-ux.md b/docs/design/frontend-three-terminal-ux.md index 1c1a3f9c0..e3555657f 100644 --- a/docs/design/frontend-three-terminal-ux.md +++ b/docs/design/frontend-three-terminal-ux.md @@ -1,63 +1,56 @@ -# Frontend Three-Terminal UX Architecture (ADR) +# 前端三终端 UX 架构（ADR） -Status: accepted · Scope: dc3-web (and future dc3 app client) · Drivers: first-principles UX engineering +状态：已接受 · 范围：dc3-web（以及未来的 dc3 app 客户端）· 驱动因素：第一性原理 UX 工程 -## Context +## 背景 -dc3-web is a Vue 3 management console for IoT operators. Its experience targets three device classes (desktop, tablet, -mobile) and must later accommodate a native app as a fourth host of the mobile experience. The legacy codebase was -desktop-first: a hard `min-width: 1280px` floor on -`body` and the layout shell, fixed header columns, a fixed 220px settings aside, and no JavaScript breakpoint system. +dc3-web 是面向 IoT 运维人员的 Vue 3 管理控制台。其体验面向三类设备（桌面、平板、移动），并且后续必须容纳一个原生 app， +作为移动体验的第四个宿主。遗留代码库是桌面优先的：`body` 与布局壳上硬性的 `min-width: 1280px` 下限、固定的表头列、 +固定的 220px 设置侧栏（aside），而且没有 JavaScript 断点系统。 -Ad hoc responsive fixes (a few `@media` blocks, `el-col` responsive props on ten pages) proved that patches do not -scale: every new page reintroduces the desktop assumptions. This ADR replaces patching with an architecture derived from -first principles. +临时性的响应式修补（少量 `@media` 块、十个页面上的 `el-col` 响应式属性）已经证明补丁不可扩展：每个新页面都会重新引入 +桌面假设。本 ADR 用一套从第一性原理推导出的架构取代打补丁。 -## First principles +## 第一性原理 -Experience quality reduces to four physical quantities, valid on every device: +体验质量可以归结为四个物理量，在任何设备上都成立： -| Element | Question | Budget | +| 要素 | 问题 | 预算 | |------------|--------------------------------------------------------|-------------------| -| Perceive | Does the user notice key state instantly? | < 100ms | -| Understand | Is the information hierarchy readable at first glance? | first screen < 3s | -| Act | Can the user complete a task with minimal motor cost? | fewest steps | -| Feedback | Does every action answer visibly and predictably? | < 300ms | +| 感知 | 用户能否即刻注意到关键状态？ | < 100ms | +| 理解 | 信息层级是否一眼可读？ | 首屏 < 3s | +| 操作 | 用户能否以最小的动作成本完成任务？ | 步骤最少 | +| 反馈 | 每个操作是否可见且可预期地得到回应？ | < 300ms | -Devices differ in exactly four variables: input modality (mouse vs thumb), viewport geometry, usage context (long -sessions vs alarm response), and compute/bandwidth. The product is an operations console: state monitoring plus -high-consequence actions over long sessions. +设备之间的差异恰好落在四个变量上：输入模态（鼠标 vs 拇指）、视口几何、使用情境（长时间会话 vs 告警响应），以及算力/带宽。 +这个产品是一个运维管理控制台：长时间会话中的状态监控，加上高后果操作。 -From these, the architecture derives seven axioms. Every axiom is a rule with a falsifiable violation signature so it -can be enforced in review and CI. +由此，该架构推导出七条公理。每条公理都是一条带有可证伪违反特征的规则，从而可以在评审与 CI 中加以强制。 -## Axioms +## 公理 -**A1. Content semantics are device-independent.** Entity models, operation models, and schemas are authored once and -consumed by every presentation host. *Violation signature:* a second copy of a field definition for a different device. +**A1. 内容语义与设备无关。**实体模型、操作模型与 schema 只编写一次，由每个呈现宿主消费。*违反特征：*为另一台设备复制出 +第二份字段定义。 -**A2. Presentation is rebuilt per device class, not shrunk.** Layout follows container geometry via fluid primitives -(`minmax`, `auto-fit`, wrapping flex); coarse device-class switches are the only job of breakpoints. *Violation -signature:* a growing pile of one-off `@media` patches. +**A2. 呈现按设备类重建，而不是缩放。**布局经流式原语（`minmax`、`auto-fit`、可换行 flex）跟随容器几何；粗糙的设备类切换 +才是断点的唯一职责。*违反特征：*一次性 `@media` 补丁越堆越多。 -**A3. Input capability decides interaction mode.** Hover, right-click, and inline editing are mouse language; 44x44 -targets, bottom sheets, and swipes are thumb language. Pointer capability (fine/coarse, hover) is a runtime property, -not a media-query guess — a tablet with a keyboard still deserves mouse interactions. *Violation signature:* hover-only -actions, or touch targets below 44px. +**A3. 输入能力决定交互模式。**悬停、右键与行内编辑是鼠标语言；44x44 目标、底部抽屉与滑动是拇指语言。指针能力 +（fine/coarse、悬停）是运行期属性，不是媒体查询的猜测——带键盘的平板仍然配得上鼠标交互。*违反特征：*只有悬停才能触发的 +操作，或低于 44px 的触控目标。 -**A4. Feedback latency is the perceived product.** 100ms synchronous feedback, <1s skeletons, optimistic updates beyond -that. *Violation signature:* a button that does nothing until the network answers. +**A4. 反馈时延就是被感知的产品。**100ms 同步反馈、<1s 骨架屏，再往上是乐观更新。*违反特征：*在网络应答之前毫无反应的 +按钮。 -**A5. Consistency carries the user's memory.** Tokens -> components -> page templates, each with a single source of -truth; no hardcoded colors, radii, or magic widths. *Violation signature:* non-token color/spacing literals in SCSS. +**A5. 一致性承载用户的记忆。**Token -> 组件 -> 页面模板，各自拥有单一事实来源；没有硬编码颜色、圆角或魔法宽度。*违反 +特征：*SCSS 中非 token 的颜色/间距字面量。 -**A6. State is part of the experience.** Preferences (theme, density, locale), navigation position, and draft forms -survive device and session boundaries. *Violation signature:* user context reset on device switch or refresh. +**A6. 状态是体验的一部分。**偏好（主题、密度、语言）、导航位置与草稿表单要跨越设备与会话边界存活。*违反特征：*切换设备或 +刷新后用户上下文被重置。 -**A7. Accessibility is the quality baseline.** Contrast, keyboard focus, reduced motion, and zoom are non-negotiable — -they double as the spec for outdoor/site inspection scenarios. +**A7. 无障碍是质量基线。**对比度、键盘焦点、减弱动效与缩放没有商量余地——它们同时也是户外/现场巡检场景的规格。 -## Layer model +## 分层模型 ```text L1 Semantic entity model + operation model + schemas (summary/detail) device-independent, single truth @@ -67,82 +60,76 @@ L4 Patterns 4 page templates: monitor / list / detail / edit — three physica L5 Quality measurement -> gates -> regression -> revision (closed loop) ``` -Each layer depends only on the one below it. Changes propagate downward, never sideways. +每一层只依赖紧邻的下一层。变更只向下传播，绝不横向扩散。 -## Boundary discipline (reuse contract) +## 边界纪律（复用契约） -The web UI shell (L3/L4) is not reusable by a native app; the layers below the rendering boundary are. To keep that -reuse real, the following are forbidden in L1/L2 and in a future shared `dc3-sdk` package: +Web UI 壳（L3/L4）对原生 app 不可复用；渲染边界之下的层则可以复用。为了让这种复用真实成立，以下内容在 L1/L2 以及未来 +共享的 `dc3-sdk` 包中一律禁止： -- imports of `@/config/*` Vue/Element Plus infrastructure (axios instance, - `ElMessage`, vue-i18n `ComposerTranslation`); -- Element Plus or Vue types in entity schema definitions (labels are i18n keys, not translation functions); -- device-specific values in token sources (tokens are host-independent values; SCSS/CSS are one rendering of them). +- 导入 `@/config/*` 的 Vue/Element Plus 基础设施（axios 实例、`ElMessage`、vue-i18n `ComposerTranslation`）； +- 实体 schema 定义中出现 Element Plus 或 Vue 类型（label 是 i18n 键，不是翻译函数）； +- token 源中出现设备特定值（token 是与宿主无关的值；SCSS/CSS 只是它们的一种渲染）。 -Enforcement: ADR review + lint rules (no non-token literals in SCSS, no framework imports below L2). +强制手段：ADR 评审 + lint 规则（SCSS 中不得出现非 token 字面量，L2 之下不得导入框架）。 -## Breakpoint contract +## 断点契约 -Single contract, aligned with Element Plus `el-col` semantics (A5): +单一契约，与 Element Plus `el-col` 语义对齐（A5）： -| Tier | Range | Terminal | -|------|---------------|------------------------| -| xs | < 768px | mobile | -| sm | 768 - 991px | tablet | -| md | 992 - 1199px | tablet / small desktop | -| lg | 1200 - 1919px | desktop | -| xl | >= 1920px | wide desktop | +| 档位 | 范围 | 终端 | +|------|---------------|-------------------------| +| xs | < 768px | 手机 | +| sm | 768 - 991px | 平板 | +| md | 992 - 1199px | 平板 / 小桌面 | +| lg | 1200 - 1919px | 桌面 | +| xl | >= 1920px | 宽屏桌面 | -JavaScript (`useBreakpoint`) and CSS must both read this contract — no second, hand-rolled breakpoint set anywhere. +JavaScript（`useBreakpoint`）与 CSS 必须都读取这一契约——任何地方都不允许出现第二套手搓断点。 -## Decisions (derived, not chosen) +## 决策（推导而得，而非任意挑选） -1. Mobile navigation: drawer menu, not bottom tabs — the task distribution is long-tail (home/alarms frequent, 40+ - settings pages rare), which bottom tabs model badly. -2. Mobile tables: summary card lists driven by the L1 summary schema, not horizontal scrolling. Cross-row comparison — - the table's purpose — is destroyed by horizontal panning; a sticky first column is the accepted interim state. -3. Dark mode: in scope, as a product of L2 tokens, for long-session eye load and sharper alarm contrast, not as - decoration. -4. Visual layer: tokenized Element Plus, no bespoke component library — bespoke UI violates A5 economics (maintenance - grows with component count). +1. 移动端导航：抽屉菜单，而非底部标签——任务分布是长尾的（首页/告警高频，40+ 个设置页面低频），底部标签对这种分布建模 + 很差。 +2. 移动端表格：由 L1 摘要 schema 驱动的摘要卡片列表，而非横向滚动。跨行比较——表格的本意——会被横向平移摧毁；粘性首列 + 是被接受的过渡状态。 +3. 深色模式：在范围内，作为 L2 token 的产物，为的是长会话的眼部负担与更锐利的告警对比，而非装饰。 +4. 视觉层：token 化的 Element Plus，不做专属组件库——专属 UI 违背 A5 的经济学（维护成本随组件数量增长）。 -## Acceptance criteria +## 验收标准 -- No page-level horizontal scroll from 360px to 2560px (table containers exempt); - `document.documentElement.scrollWidth <= window.innerWidth`. -- Lighthouse mobile: perf >= 90, CLS <= 0.1, LCP <= 2.5s. -- 100% touch targets >= 44x44; dialogs full-screen below 768px. -- Zero hardcoded color/spacing literals in SCSS (token lint gate). -- Playwright runs desktop, tablet, and mobile viewport projects; axe scan clean; visual regression on critical pages. +- 360px 到 2560px 之间无页面级横向滚动（表格容器豁免）； + `document.documentElement.scrollWidth <= window.innerWidth`。 +- Lighthouse 移动端：perf >= 90、CLS <= 0.1、LCP <= 2.5s。 +- 100% 的触控目标 >= 44x44；768px 以下对话框全屏。 +- SCSS 中零硬编码颜色/间距字面量（token lint 闸门）。 +- Playwright 运行桌面、平板与移动视口项目；axe 扫描干净；关键页面执行视觉回归。 -## Revision +## 修订记录 -2026-08: v1 — adopted with the L2 token/breakpoint/theme foundation (dc3-web src/styles/tokens.scss, theme.scss, -src/composables/useBreakpoint.ts, src/store/modules/app.ts). +2026-08：v1 —— 随 L2 token/断点/主题基座一起采纳（dc3-web src/styles/tokens.scss、theme.scss、 +src/composables/useBreakpoint.ts、src/store/modules/app.ts）。 -2026-08: v2 — shell three-terminal forms shipped: shared NavMenu (horizontal ellipsis / vertical drawer), Settings -sidebar menu extracted and hosted in aside (desktop) / collapsed rail (tablet) / drawer (mobile), responsive login -panels, compact mobile pagination in ToolCard. Verified via artifacts/viewport-check.mjs: zero page-level overflow and -correct per-terminal DOM at 1440/834/390px viewports against the mock build. +2026-08：v2 —— 布局壳三终端形态发布：共享 NavMenu（水平省略 / 垂直抽屉）、Settings 侧栏菜单抽取出来并托管为 +aside（桌面）/ 折叠栏（平板）/ 抽屉（移动）、响应式登录面板、ToolCard 内的移动端紧凑分页。经 artifacts/viewport-check.mjs +验证：在 1440/834/390px 视口下对照 mock 构建零页面级溢出，且各终端 DOM 正确。 -2026-08: v3 — L1 de-frameworking (dc3-client-sdk Phase 0): Translator contract replaces vue-i18n ComposerTranslation -across all 16 entity config modules; dc3-sdk extraction design documented in docs/design/dc3-client-sdk.md. +2026-08：v3 —— L1 去框架化（dc3-client-sdk Phase 0）：全部 16 个实体配置模块中 Translator 契约取代 vue-i18n +ComposerTranslation；dc3-sdk 抽取设计记录于 docs/design/dc3-client-sdk.md。 -2026-08: v4 — measurement gate shipped: Playwright gains chromium-desktop (1440x900), chromium-tablet (834x1112, touch), -and chromium-mobile (393x851, touch) projects; tests/e2e/specs/responsive.spec.ts gates the A2 overflow criterion -(scrollWidth <= clientWidth on login/home/settings), A3 shell adaptation (menu strip vs drawer, aside vs drawer), and an -A7 accessible-name smoke probe. All 12 gate tests green against the mock build. CI (ci-web.yml) runs the gate -automatically via pnpm test:e2e with chromium only. Fixes recorded: playwright 1.61.1/1.62.0 version split aligned to -1.62.0; ToolCard refresh/sort icon buttons gained aria-labels. Lighthouse budget and axe-core scans remain CI follow-ups -(no new deps this iteration). +2026-08：v4 —— 度量闸门发布：Playwright 新增 chromium-desktop（1440x900）、chromium-tablet（834x1112，触控）与 +chromium-mobile（393x851，触控）项目；tests/e2e/specs/responsive.spec.ts 为以下内容设闸：A2 溢出判据 +（login/home/settings 上 scrollWidth <= clientWidth）、A3 布局壳适配（菜单条 vs 抽屉、aside vs 抽屉），以及 A7 可访问名称 +冒烟探测。对照 mock 构建全部 12 个闸门测试通过。CI（ci-web.yml）经 pnpm test:e2e 且仅用 chromium 自动运行该闸门。已记录 +修复：playwright 1.61.1/1.62.0 的版本分裂对齐到 1.62.0；ToolCard 刷新/排序图标按钮补上 aria-label。Lighthouse 预算与 +axe-core 扫描仍是 CI 后续事项（本轮不新增依赖）。 -2026-08: v5 — contract hardening + first acceptance gaps closed. (1) Breakpoint contract now literal-free: the last -hand-rolled width thresholds (Home 1024/1280/640, alarm Overview 1024/1280/640, AgenticAssistant 900) replaced with -$breakpoint-* tokens, mapping single-column collapses to sm-max, 3-col stat grids to md-max, and 1-col to xs-max. tests/guardrails/breakpoint-contract.test.ts enforces A5: any @media (min/max-width) in src must reference a -$breakpoint-* token. (2) A3 dialog criterion shipped: theme.scss re-shapes .el-dialog to the viewport (calc (100vw - -16px), capped height, scrolling body) below xs-max, overriding Element Plus inline widths; gated by a mobile-only -Playwright test (dialog never wider than viewport, zero page overflow). (3) A2 gate coverage extended beyond the shell: -nine template-sweep routes (monitor/list/detail/history families — alarm overview, device, driver, profile, label, -point_value, alarm/point, event/command history) now gated on every terminal, 48 gate tests total. Still open: L4 -template rollout to the remaining ~100 views (mobile summary-card lists blocked on the L1 summary schema decision), -Lighthouse budget + axe-core in CI, token lint. +2026-08：v5 —— 契约加固 + 首批验收缺口闭合。(1) 断点契约现已去字面量：最后一批手搓宽度阈值（Home 1024/1280/640、告警 +Overview 1024/1280/640、AgenticAssistant 900）替换为 $breakpoint-* token，把单列折叠映射到 sm-max、3 列统计网格映射到 +md-max、1 列映射到 xs-max。tests/guardrails/breakpoint-contract.test.ts 强制 A5：src 中任何 @media (min/max-width) 必须 +引用 $breakpoint-* token。(2) A3 对话框判据落地：theme.scss 在 xs-max 以下将 .el-dialog 重塑为贴合视口 +（calc (100vw - 16px)、限高、主体滚动），覆盖 Element Plus 的内联宽度；由一个仅移动端的 Playwright 测试设闸（对话框永不 +宽于视口、页面零溢出）。(3) A2 闸门覆盖扩展到布局壳之外：九条模板横扫路由（monitor/list/detail/history 家族——告警概览、 +设备、驱动、profile、label、point_value、alarm/point、事件/命令历史）现已在每个终端设闸，闸门测试合计 48 个。仍待解决： +L4 模板推广到其余约 100 个视图（移动端摘要卡片列表受阻于 L1 摘要 schema 决策）、CI 中的 Lighthouse 预算 + axe-core、 +token lint。 diff --git a/docs/design/mcp-runtime-overhaul.md b/docs/design/mcp-runtime-overhaul.md index cdf83cbcb..4b2bed23f 100644 --- a/docs/design/mcp-runtime-overhaul.md +++ b/docs/design/mcp-runtime-overhaul.md @@ -1,125 +1,122 @@ -# Design: MCP Runtime Overhaul — Cohesive Authorization Contract +# 设计：MCP 运行时全面重构 —— 内聚的授权契约 | | | |----------------|-------------------------------------------------------------------------------------------------------------------------------------------------| -| **Status** | Proposed — not yet implemented | -| **Date** | 2026-08-18 | -| **Scope** | MCP runtime plane: `mcp_runtime.proto`, `McpRuntimeFacade`, `McpGatewayController`, auth-side `OAuthMcpRuntimeServiceImpl` + `McpRuntimeServer` | -| **Target** | One cohesive gateway→auth round-trip per MCP method; reactive, non-blocking; real input schemas; async audit | -| **Related** | [`mq-abstraction.md`](./mq-abstraction.md) — broker port used for the async audit channel, if adopted | -| **Discussion** | open for review before implementation starts | +| **状态** | 提案 —— 尚未实施 | +| **日期** | 2026-08-18 | +| **范围** | MCP 运行时平面：`mcp_runtime.proto`、`McpRuntimeFacade`、`McpGatewayController`，auth 侧的 `OAuthMcpRuntimeServiceImpl` + `McpRuntimeServer` | +| **目标** | 每个 MCP 方法一次内聚的 gateway→auth 往返；响应式、非阻塞；真实 input schema；异步审计 | +| **相关** | [`mq-abstraction.md`](./mq-abstraction.md) —— 异步审计通道（若采用）所用的 broker port | +| **讨论** | 实施启动前开放评审 | -## 1. Summary +## 1. 摘要 -The MCP runtime that sits behind `McpGatewayController` (the gateway's MCP JSON-RPC resource server) currently reaches -the auth center through a **synchronous facade with five fine-grained RPCs**, which forces the gateway to behave like a -remote database client: it first calls -`Introspect` to recover the token context, then spreads `tenantId` / `principalId` / -`connectionId` across `ResolveTool` and `AuthorizeToolCall`, and finally calls `Audit`. One `tools/call` therefore costs -**4 gRPC round-trips, ~8–10 DB queries, one blocking-thread hop per call**, and — as a correctness gap — returns tools -whose `inputSchema` is a static empty envelope instead of the real JSON schema. +位于 `McpGatewayController`（网关的 MCP JSON-RPC 资源服务器）背后的 MCP 运行时，目前通过一个**带有五个细粒度 RPC 的 +同步 Facade** 访问 auth 中心，这迫使网关表现得像一个远程数据库客户端：它先调用 +`Introspect` 取回令牌上下文，再把 `tenantId` / `principalId` / +`connectionId` 摊到 `ResolveTool` 和 `AuthorizeToolCall` 两个调用里，最后调用 `Audit`。因此一次 `tools/call` 的代价是 +**4 次 gRPC 往返、约 8–10 次 DB 查询、每次调用一次阻塞线程跳转**，并且——作为一个正确性缺口——返回的工具 +`inputSchema` 是静态的空壳信封，而不是真实的 JSON schema。 -This document proposes a **breaking, non-compatible overhaul** of that runtime contract: +本文档提出对这一运行时契约进行**破坏性、不兼容的全面重构**： -- **3 RPCs instead of 5**: `ListTools`, `CallTool`, `Audit`. -- **The bearer token is the only input** for `ListTools` / `CallTool`; the auth center is the single place that - understands a token, so it does verification + visibility + authorization in one decision inside the same call. -- `CallTool` returns the **decision, the resolved tool, and the principal context** together, so the gateway no longer - needs a separate introspection round-trip before forwarding to a backend. -- The facade becomes **reactive** (`Mono`/future stubs); the `blocking() + boundedElastic` hop disappears. -- `inputSchema` is carried end-to-end so external agents see real tool parameters. -- `Audit` is decoupled from the call path (fire-and-forget or broker event). +- **3 个 RPC 取代 5 个**：`ListTools`、`CallTool`、`Audit`。 +- `ListTools` / `CallTool` **以 bearer token 作为唯一输入**；auth 中心是唯一理解令牌的地方， + 因此它在同一次调用内部以一个决策完成校验 + 可见性 + 授权。 +- `CallTool` 一次性返回**决策、解析出的工具与主体上下文**，网关在转发给后端之前不再需要单独的自省往返。 +- Facade 变为**响应式**（`Mono`/future stub）；`blocking() + boundedElastic` 跳转消失。 +- `inputSchema` 端到端透传，外部 agent 看到的是真实的工具参数。 +- `Audit` 与调用路径解耦（fire-and-forget 或 broker 事件）。 -## 2. Background — how the MCP runtime works today +## 2. 背景 —— MCP 运行时今天如何工作 -Verified against the tree on 2026-08-18. File references are exact. +已对照 2026-08-18 的代码树核实。文件引用均为确切路径。 -### 2.1 The contract today +### 2.1 现行契约 -`dc3-api/dc3-api-auth/src/main/protobuf/api/common/auth/mcp_runtime.proto` declares five RPCs: +`dc3-api/dc3-api-auth/src/main/protobuf/api/common/auth/mcp_runtime.proto` 声明了五个 RPC： -| RPC | Purpose | Called per tools/call? | +| RPC | 用途 | 每次 tools/call 都调用？ | |---------------------|-----------------------------------------------------------------------------|------------------------| -| `Introspect` | Validate the OAuth bearer token, return tenant/principal/connection context | yes (every request) | -| `ListTools` | List tools visible to the connection | no (tools/list only) | -| `ResolveTool` | Resolve one tool to its backend invocation metadata | yes | -| `AuthorizeToolCall` | Enforce high-risk confirmation + idempotency, return a decision | yes | -| `Audit` | Store one audit record | yes | +| `Introspect` | 校验 OAuth bearer token，返回租户/主体/连接上下文 | 是（每个请求） | +| `ListTools` | 列出该连接可见的工具 | 否（仅 tools/list） | +| `ResolveTool` | 把一个工具解析为其后端调用元数据 | 是 | +| `AuthorizeToolCall` | 强制高风险确认 + 幂等，返回决策 | 是 | +| `Audit` | 存储一条审计记录 | 是 | -`McpRuntimeFacade` (`dc3-common-facade-api`) mirrors this as **synchronous** methods: -`introspect(String)`, `listTools(...)`, `resolveTool(...)`, `authorizeToolCall(...)`, -`audit(...)`. Its gRPC implementation (`McpRuntimeGrpcFacade`) uses an injected blocking stub; -`GrpcFacadeSupport.call` only adds a deadline. Connection reuse is fine — the stub is a shared bean — the problem is -*what* is called and *how often*. +`McpRuntimeFacade`（`dc3-common-facade-api`）以**同步**方法镜像这一契约： +`introspect(String)`、`listTools(...)`、`resolveTool(...)`、`authorizeToolCall(...)`、 +`audit(...)`。它的 gRPC 实现（`McpRuntimeGrpcFacade`）使用注入的阻塞 stub； +`GrpcFacadeSupport.call` 只增加了一个 deadline。连接复用没有问题——stub 是共享 bean——问题在于 +*调用的是什么*以及*调用频率*。 -### 2.2 One `tools/call`, step by step +### 2.2 一次 `tools/call` 的逐步分解 -The gateway path is `McpGatewayController.mcp(...)` → `dispatch(...)` → -`McpGatewayClient.callTool(...)`. Each hop and its verified cost: +网关路径是 `McpGatewayController.mcp(...)` → `dispatch(...)` → +`McpGatewayClient.callTool(...)`。每一跳及其核实过的成本： -| Step | Where | Cost | +| 步骤 | 位置 | 成本 | |---------------------|----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| ① introspect | `OAuthMcpRuntimeServiceImpl.introspect` | 1× local JWT verify (`verifyWith(publicKey())`) + **4 DB queries**: `selectAuthorizationByAccessTokenJti`, `selectConnectionById`, `principalManager.getById`, `tenantMembershipService.isTenantMember` | -| ② resolveTool | `resolveVisibleTool` | `selectVisibleToolByName` (1) + `updateConnectionLastUsed` (1) | -| ③ authorizeToolCall | `authorizeToolCall` → re-runs `resolveVisibleTool` | **repeats** `selectVisibleToolByName` + `updateConnectionLastUsed` (2 more); only HIGH risk then queries the confirmation ticket | -| ④ invokeBackend | gateway `WebClient` | 1 HTTP forward (the actual business call) | -| ⑤ audit | `audit` → `insert` | 1 DB write (post-hoc, failure swallowed) | +| ① introspect | `OAuthMcpRuntimeServiceImpl.introspect` | 1× 本地 JWT 校验（`verifyWith(publicKey())`）+ **4 次 DB 查询**：`selectAuthorizationByAccessTokenJti`、`selectConnectionById`、`principalManager.getById`、`tenantMembershipService.isTenantMember` | +| ② resolveTool | `resolveVisibleTool` | `selectVisibleToolByName`（1 次）+ `updateConnectionLastUsed`（1 次） | +| ③ authorizeToolCall | `authorizeToolCall` → 重新执行 `resolveVisibleTool` | **重复执行** `selectVisibleToolByName` + `updateConnectionLastUsed`（再多 2 次）；仅 HIGH 风险随后查询确认票据 | +| ④ invokeBackend | 网关 `WebClient` | 1 次 HTTP 转发（真正的业务调用） | +| ⑤ audit | `audit` → `insert` | 1 次 DB 写入（事后执行，失败被吞掉） | -Plus **4 gateway→auth gRPC round-trips** (`Introspect`, `ResolveTool`, `AuthorizeToolCall`, -`Audit`), each wrapped in `blocking(...)` which subscribes on `boundedElastic` — one blocking thread per in-flight call. +再加上 **4 次 gateway→auth gRPC 往返**（`Introspect`、`ResolveTool`、`AuthorizeToolCall`、 +`Audit`），每次都包在 `blocking(...)` 里并在 `boundedElastic` 上订阅——每个在途调用占用一个阻塞线程。 -### 2.3 Three redundancies and one correctness gap +### 2.3 三处冗余与一个正确性缺口 -1. **Introspection is fully re-run on every request, with no cache.** The four DB queries cannot be dropped (OAuth - introspection must check revocation, principal enablement, and tenant membership), but the same token pays them again - on every tool call in an agent loop. -2. **Resolve and authorize duplicate the visibility query.** `authorizeToolCall` explicitly re-runs `resolveVisibleTool` - ("Re-run the full visibility/whitelist/scope check"). The result is identical; `selectVisibleToolByName` and - `updateConnectionLastUsed` each execute twice. -3. **`updateConnectionLastUsed` is non-critical telemetry executed synchronously — twice.** -4. **Correctness gap: `inputSchema` is lost.** `GrpcMcpToolDefinitionDTO` has **no** - `input_schema` field (the proto comment says "excluding the static JSON schema envelope"), and - `McpRuntimeGrpcFacade.toDTO` hard-codes `DEFAULT_INPUT_SCHEMA`. An external agent therefore receives an empty schema - envelope for every tool instead of the real parameters that - `McpOpenApiAggregator` already computed and `OAuthMcpRuntimeServiceImpl.inputSchemaOf` already reads from `tool_ext`. +1. **自省在每个请求上完整重跑，没有任何缓存。** 这四次 DB 查询不能省略（OAuth + 自省必须检查吊销、主体启用状态与租户成员关系），但在 agent 循环里，同一个令牌在每次工具调用时都要再付一遍。 +2. **resolve 与 authorize 重复执行可见性查询。** `authorizeToolCall` 显式重跑 `resolveVisibleTool` + （"重新执行完整的可见性/白名单/scope 检查"）。结果完全相同；`selectVisibleToolByName` 与 + `updateConnectionLastUsed` 各执行两次。 +3. **`updateConnectionLastUsed` 是非关键遥测，却同步执行——而且执行两次。** +4. **正确性缺口：`inputSchema` 丢失。** `GrpcMcpToolDefinitionDTO` **没有** + `input_schema` 字段（proto 注释写明"不含静态 JSON schema 信封"），并且 + `McpRuntimeGrpcFacade.toDTO` 硬编码了 `DEFAULT_INPUT_SCHEMA`。因此外部 agent 对每个工具收到的都是空的 schema + 信封，而不是 `McpOpenApiAggregator` 已经算出、`OAuthMcpRuntimeServiceImpl.inputSchemaOf` 已经从 `tool_ext` + 读出的真实参数。 -## 3. Goals / Non-Goals +## 3. 目标 / 非目标 -**Goals** +**目标** -- Reduce one `tools/call` to **2 gateway→auth round-trips** (`CallTool` + async `Audit`). -- Make the token the single input to `ListTools` / `CallTool`; auth is the only token authority. -- Eliminate the duplicate visibility query (`selectVisibleToolByName` once, not twice). -- Make the runtime **reactive end-to-end**; remove `blocking()` + `boundedElastic`. -- Carry the **real `inputSchema`** through `tools/list` so external agents see true parameters. -- Decouple audit from the call path. -- Drop `Introspect`, `ResolveTool`, `AuthorizeToolCall` as gateway-facing RPCs — **no backward compatibility**, no - dual-mode shim. +- 把一次 `tools/call` 降为 **2 次 gateway→auth 往返**（`CallTool` + 异步 `Audit`）。 +- 让令牌成为 `ListTools` / `CallTool` 的唯一输入；auth 是唯一的令牌权威。 +- 消除重复的可见性查询（`selectVisibleToolByName` 一次，而非两次）。 +- 让运行时**端到端响应式**；移除 `blocking()` + `boundedElastic`。 +- 让**真实 `inputSchema`** 随 `tools/list` 透传，外部 agent 看到真实参数。 +- 将审计与调用路径解耦。 +- 移除面向网关的 `Introspect`、`ResolveTool`、`AuthorizeToolCall` RPC——**不保留向后兼容**，也不做 + 双模式 shim。 -**Non-Goals** +**非目标** -- No change to the MCP wire protocol the client sees (still JSON-RPC 2.0 + OAuth bearer). -- No change to the management plane (`McpManagementController`) or the frontend MCP settings pages. -- No change to `McpOpenApiAggregator` / `dc3_api` tool-catalog generation. -- No change to how backend services authenticate downstream principal headers (HMAC + JSON header stay as-is). +- 客户端看到的 MCP 线上协议不变（仍是 JSON-RPC 2.0 + OAuth bearer）。 +- 管理平面（`McpManagementController`）与前端 MCP 设置页不变。 +- `McpOpenApiAggregator` / `dc3_api` 工具目录的生成方式不变。 +- 后端服务认证下游 principal 头的方式不变（HMAC + JSON header 维持原样）。 -## 4. Design principles +## 4. 设计原则 -1. **Decision, not lookup.** The gateway asks "may this token call this tool, and where does it go?" - in one request; it does not assemble the answer from three lookups. -2. **Auth owns the token.** `ListTools` and `CallTool` take the raw token; auth parses, verifies, and resolves context - internally. The gateway never reconstructs tenant/principal context from claims. -3. **One authoritative visibility check.** Visibility + risk + confirmation + idempotency are decided in one place, - once, inside `CallTool`. -4. **Reactive throughout.** Facade returns `Mono`; gRPC uses future/async stubs; gateway stays on the WebFlux event - loop. -5. **Telemetry is off-path.** Audit is fire-and-forget (or a broker event); it can never delay or fail a call. +1. **要决策，不是查询。** 网关在一个请求里问"这个令牌能否调用这个工具、它该转发到哪里"； + 而不是从三次查询拼出答案。 +2. **auth 拥有令牌。** `ListTools` 与 `CallTool` 接收原始令牌；auth 在内部解析、校验并还原上下文。 + 网关绝不从 claims 重建租户/主体上下文。 +3. **唯一一次权威可见性检查。** 可见性 + 风险 + 确认 + 幂等在同一处、只此一次、于 `CallTool` + 内部裁决。 +4. **全程响应式。** Facade 返回 `Mono`；gRPC 使用 future/异步 stub；网关留在 WebFlux 事件 + 循环上。 +5. **遥测在路径之外。** 审计是 fire-and-forget（或 broker 事件）；它永远不能延迟或使一次调用失败。 -## 5. Target contract +## 5. 目标契约 ### 5.1 Protobuf -`mcp_runtime.proto` shrinks to three RPCs. Message names keep the existing `Grpc` convention. +`mcp_runtime.proto` 收缩为三个 RPC。消息名沿用既有的 `Grpc` 命名约定。 ```proto service McpRuntimeApi { @@ -135,14 +132,14 @@ service McpRuntimeApi { } ``` -Removed: `Introspect`, `ResolveTool`, `AuthorizeToolCall` and their request/response messages -(`GrpcMcpIntrospectRequest`, `GrpcRMcpIntrospectDTO`, `GrpcMcpIntrospectDTO`, -`GrpcMcpToolListRequest`, `GrpcMcpToolResolveRequest`, `GrpcRMcpToolResolveDTO`, -`GrpcMcpToolResolveDTO`, `GrpcMcpToolAuthorizeRequest`, `GrpcRMcpToolAuthorizeDTO`). The shared enums -(`GrpcMcpRiskLevel`, `GrpcMcpDecision`, `GrpcMcpPrincipalType`, -`GrpcMcpAuditStatus`) stay. +移除：`Introspect`、`ResolveTool`、`AuthorizeToolCall` 及其请求/响应消息 +（`GrpcMcpIntrospectRequest`、`GrpcRMcpIntrospectDTO`、`GrpcMcpIntrospectDTO`、 +`GrpcMcpToolListRequest`、`GrpcMcpToolResolveRequest`、`GrpcRMcpToolResolveDTO`、 +`GrpcMcpToolResolveDTO`、`GrpcMcpToolAuthorizeRequest`、`GrpcRMcpToolAuthorizeDTO`）。共享枚举 +（`GrpcMcpRiskLevel`、`GrpcMcpDecision`、`GrpcMcpPrincipalType`、 +`GrpcMcpAuditStatus`）保留。 -New / changed messages: +新增 / 变更的消息： ```proto message GrpcMcpListToolsRequest { @@ -193,9 +190,8 @@ message GrpcMcpCallToolDTO { } ``` -`GrpcMcpToolResolveDTO` is retained (it is still the `tool` sub-message inside -`GrpcMcpCallToolDTO`) and gains `input_schema` so the gateway can forward a tool's schema to a backend when a future -backend needs it. +`GrpcMcpToolResolveDTO` 保留（它仍是 `GrpcMcpCallToolDTO` 内部的 `tool` 子消息），并新增 +`input_schema`，以便未来的后端需要时，网关可以把工具的 schema 转发给后端。 ### 5.2 Facade @@ -207,41 +203,40 @@ public interface McpRuntimeFacade { } ``` -- `McpRuntimeGrpcFacade` switches from `McpRuntimeApiBlockingStub` to the async - `McpRuntimeApiStub` (future→`Mono`) and drops `GrpcFacadeSupport.call` in favor of reactive error translation. -- New DTOs `McpCallToolRequestDTO` / `McpCallToolResponseDTO` mirror the proto; the response carries - `McpPrincipalContextDTO` in place of the old `McpIntrospectResponseDTO`. +- `McpRuntimeGrpcFacade` 从 `McpRuntimeApiBlockingStub` 切换到异步的 + `McpRuntimeApiStub`（future→`Mono`），并去掉 `GrpcFacadeSupport.call`，改用响应式错误转换。 +- 新 DTO `McpCallToolRequestDTO` / `McpCallToolResponseDTO` 与 proto 一一对应；响应以 + `McpPrincipalContextDTO` 取代旧的 `McpIntrospectResponseDTO`。 -### 5.3 Auth service +### 5.3 Auth 服务 -`OAuthMcpRuntimeServiceImpl` replaces `introspect` + `resolveVisibleTool` + -`authorizeToolCall` with two cohesive operations: +`OAuthMcpRuntimeServiceImpl` 用两个内聚操作取代 `introspect` + `resolveVisibleTool` + +`authorizeToolCall`： -- `listTools(token)`: `parseAccessToken` → active-authorization check → connection/principal/ membership checks → - `listVisibleTools` → `toolToMcp` (now with real `inputSchema`). -- `callTool(request)`: same token verification, then **one** `selectVisibleToolByName` + visibility/scope/risk decision, - then the confirmation/idempotency gate (HIGH risk only), returning decision + resolved tool + principal context. +- `listTools(token)`：`parseAccessToken` → 有效授权检查 → 连接/主体/成员关系检查 → + `listVisibleTools` → `toolToMcp`（现在带真实 `inputSchema`）。 +- `callTool(request)`：同样的令牌校验，然后**一次** `selectVisibleToolByName` + 可见性/scope/风险决策， + 随后是确认/幂等闸门（仅 HIGH 风险），返回决策 + 解析出的工具 + 主体上下文。 -`McpRuntimeServer` mirrors this: three gRPC methods, `toGrpc` builders for the new messages, and -`inputSchema` serialization out of `tool_ext` (reusing the existing `inputSchemaOf` logic). +`McpRuntimeServer` 与之对应：三个 gRPC 方法、为新消息准备的 +`toGrpc` 构建器，以及从 `tool_ext` 序列化 `inputSchema`（复用既有的 `inputSchemaOf` 逻辑）。 -### 5.4 Gateway +### 5.4 网关 -`McpGatewayController` dispatch simplifies to: +`McpGatewayController` 的分发简化为： -- `tools/list` → `mcpRuntimeFacade.listTools(token)` → JSON-RPC result. -- `tools/call` → `mcpRuntimeFacade.callTool(request(token, toolName, digest, confirmId, - idempotencyKey, client meta))`; on `AUTHORIZED`, forward to the backend using - `tool.serviceName/apiPath/httpMethod` and build `X_AUTH_PRINCIPAL` from `principal`; on - `CONFIRM_REQUIRED`, return the confirm prompt; on `REJECTED`, return the denial. -- `audit` → fire-and-forget (no longer awaited before returning). +- `tools/list` → `mcpRuntimeFacade.listTools(token)` → JSON-RPC 结果。 +- `tools/call` → `mcpRuntimeFacade.callTool(request(token, toolName, digest, confirmId, idempotencyKey, client meta))`； + `AUTHORIZED` 时使用 + `tool.serviceName/apiPath/httpMethod` 转发到后端，并由 `principal` 构建 `X_AUTH_PRINCIPAL`； + `CONFIRM_REQUIRED` 时返回确认提示；`REJECTED` 时返回拒绝。 +- `audit` → fire-and-forget（返回前不再等待）。 -The `blocking(...)` helper, the `toLong(context.getTenantId())` scattering, and the standalone introspection call all -disappear. +`blocking(...)` 辅助方法、`toLong(context.getTenantId())` 的四处散布，以及独立的自省调用全部消失。 -## 6. Request flows +## 6. 请求流程 -### 6.1 Before (today, one tools/call) +### 6.1 之前（今天，一次 tools/call） ```text client → gateway ─ Introspect ──────────────→ auth (JWT + 4 DB) @@ -252,7 +247,7 @@ client → gateway ─ Introspect ──────────────→ gateway ←─ decision / tool / context ── (assembled from 3 responses) ``` -### 6.2 After (target, one tools/call) +### 6.2 之后（目标，一次 tools/call） ```text client → gateway ─ CallTool(token, tool, digest, confirm, key, client meta) ─→ auth @@ -263,64 +258,63 @@ client → gateway ─ CallTool(token, tool, digest, confirm, key, client meta) gateway ←─ decision + tool + principal ── (one response) ``` -## 7. Migration plan +## 7. 迁移计划 -Breaking, no compatibility shim. Each phase must leave the tree compiling and tests green. +破坏性变更，无兼容 shim。每个阶段结束时代码树必须可编译、测试保持绿色。 -| Phase | Change | Files | +| 阶段 | 变更 | 文件 | |-------|-----------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------| -| 1 | Rewrite `mcp_runtime.proto` (3 RPCs, new messages, `input_schema`), regenerate stubs | `dc3-api/dc3-api-auth/.../mcp_runtime.proto` | -| 2 | Reactive facade + gRPC impl (`Mono`, async stub) | `McpRuntimeFacade`, `McpRuntimeGrpcFacade`, new request/response DTOs | -| 3 | Auth service + server: merge introspect/resolve/authorize into `listTools`/`callTool`; real `inputSchema` | `OAuthMcpRuntimeServiceImpl`, `McpRuntimeServer` | -| 4 | Gateway: token-direct dispatch, drop `blocking()`, async audit | `McpGatewayController` | -| 5 | Rewrite affected tests | `McpRuntimeServerTest`, `McpGatewayControllerTest`, `OAuthMcpRuntimeServiceImplTest`, facade tests | +| 1 | 重写 `mcp_runtime.proto`（3 个 RPC、新消息、`input_schema`），重新生成 stub | `dc3-api/dc3-api-auth/.../mcp_runtime.proto` | +| 2 | 响应式 Facade + gRPC 实现（`Mono`、异步 stub） | `McpRuntimeFacade`、`McpRuntimeGrpcFacade`、新的请求/响应 DTO | +| 3 | Auth 服务 + 服务端：把 introspect/resolve/authorize 合并进 `listTools`/`callTool`；真实 `inputSchema` | `OAuthMcpRuntimeServiceImpl`、`McpRuntimeServer` | +| 4 | 网关：凭令牌直接分发，去掉 `blocking()`，异步审计 | `McpGatewayController` | +| 5 | 重写受影响的测试 | `McpRuntimeServerTest`、`McpGatewayControllerTest`、`OAuthMcpRuntimeServiceImplTest`、Facade 测试 | -Frontend is **untouched** (the MCP settings pages use `McpManagementController`, not this gateway runtime). +前端**完全不动**（MCP 设置页使用的是 `McpManagementController`，不是这个网关运行时）。 -## 8. Verification +## 8. 验证 -- `mvn -s .mvn/settings.xml -q -DskipTests compile` after each phase. -- `mvn -s .mvn/settings.xml test -pl dc3-common/dc3-common-auth -am` and - `-pl dc3-common/dc3-common-gateway -am` for the rewritten tests. -- Contract assertions to preserve: - - one `tools/call` = one `CallTool` RPC + one async `Audit` (assert via test doubles), - - `selectVisibleToolByName` invoked exactly once per call, - - `tools/list` returns non-default `inputSchema` for a fixture tool, - - HIGH-risk still issues `CONFIRM_REQUIRED` + confirmId, idempotency key still deduplicates. -- `pnpm check` in `dc3-web` to confirm no frontend regression. +- 每个阶段之后运行 `mvn -s .mvn/settings.xml -q -DskipTests compile`。 +- 用 `mvn -s .mvn/settings.xml test -pl dc3-common/dc3-common-auth -am` 与 + `-pl dc3-common/dc3-common-gateway -am` 运行重写后的测试。 +- 需要保留的契约断言： + - 一次 `tools/call` = 一次 `CallTool` RPC + 一次异步 `Audit`（通过测试替身断言）， + - `selectVisibleToolByName` 每次调用恰好执行一次， + - `tools/list` 为夹具工具返回非默认 `inputSchema`， + - HIGH 风险仍产生 `CONFIRM_REQUIRED` + confirmId，幂等键仍去重。 +- 在 `dc3-web` 中运行 `pnpm check`，确认前端无回归。 -## 9. Alternatives considered +## 9. 已考虑的备选方案 -1. **Cache introspection instead of reshaping the contract** (keeps 5 RPCs). Rejected: it papers over the synchronous - facade, the duplicate visibility query, and the missing schema; the round-trip count and the gateway's - context-scattering remain. -2. **Local JWT verification in the gateway + short-TTL jti deny-list.** Removes the introspection round-trip entirely, - but moves token authority into the gateway and adds a revocation-propagation channel. Rejected as the primary design - because it blurs the auth boundary; it can be layered later as a pure optimization on top of the 3-RPC contract. -3. **Keep `ResolveTool` and only merge the rest.** Rejected: resolve is a sub-step of the call decision, not an - independent operation; returning the tool inside `CallTool` is strictly simpler. +1. **缓存自省结果而不重塑契约**（保留 5 个 RPC）。已否决：它只是掩盖同步 + Facade、重复的可见性查询和缺失的 schema；往返次数与网关的上下文散布依然存在。 +2. **网关本地 JWT 校验 + 短 TTL jti 拒绝列表。** 彻底消除自省往返， + 但把令牌权威移进了网关，并引入一条吊销传播通道。因其模糊 auth 边界而否决作为主设计； + 日后可以作为纯优化叠加在 3-RPC 契约之上。 +3. **保留 `ResolveTool`，只合并其余部分。** 已否决：resolve 是调用决策的一个子步骤，不是独立操作； + 把工具放进 `CallTool` 的返回里严格更简单。 -## 10. Open questions +## 10. 开放问题 -1. **Audit transport.** Fire-and-forget `Mono` (smaller change) vs a RabbitMQ event through the MQ abstraction (fully - off-path, but depends on broker availability). Leaning broker event, to be confirmed. -2. **`inputSchema` in `GrpcMcpToolResolveDTO`.** Should the forwarded backend ever need the schema, or is schema only a - `tools/list` concern? Current proposal adds it defensively. -3. **Token verification cost.** With 5→3 RPCs, the 4 introspection DB queries still run once per - `CallTool`. A short-TTL `jti → context` cache inside auth (invalidated on revoke/disable) is a follow-up, not part of - this contract overhaul. +1. **审计传输。** fire-and-forget 的 `Mono`（改动更小）还是通过 MQ 抽象发 RabbitMQ 事件（完全在路径之外， + 但依赖 broker 可用性）。倾向 broker 事件，待确认。 +2. **`GrpcMcpToolResolveDTO` 中的 `inputSchema`。** 被转发的后端是否终究需要 schema， + 还是 schema 只是 `tools/list` 的关注点？当前提案防御性地加上了它。 +3. **令牌校验成本。** RPC 从 5 收敛到 3 之后，4 次自省 DB 查询在每次 + `CallTool` 时仍会执行。auth 内部的短 TTL `jti → context` 缓存（在吊销/禁用时失效）是后续工作， + 不属于本次契约重构。 -## 11. Appendix — current call-site inventory +## 11. 附录 —— 现有调用点盘点 -Verified 2026-08-18. +已于 2026-08-18 核实。 -| Artifact | Location | Notes | +| 构件 | 位置 | 说明 | |---------------|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------| -| Proto | `dc3-api/dc3-api-auth/src/main/protobuf/api/common/auth/mcp_runtime.proto` | 5 RPCs, 260 lines | -| Facade | `dc3-common-facade-api/.../McpRuntimeFacade.java` | 5 synchronous methods | -| gRPC impl | `dc3-common-facade-grpc/.../McpRuntimeGrpcFacade.java` | blocking stub + `GrpcFacadeSupport` | -| Gateway | `dc3-common-gateway/.../McpGatewayController.java` | JSON-RPC dispatch, `blocking()`+boundedElastic, `invokeBackend`, `audit` | -| Auth service | `dc3-common-auth/.../OAuthMcpRuntimeServiceImpl.java` | `introspect`, `resolveVisibleTool`, `authorizeToolCall` | -| gRPC server | `dc3-common-auth/.../McpRuntimeServer.java` | 5 server methods | -| Schema source | `dc3-common-auth/.../tool/McpOpenApiAggregator.java` | `inputSchema` computed but not carried over gRPC | -| Tests | `McpRuntimeServerTest`, `McpGatewayControllerTest`, `OAuthMcpRuntimeServiceImplTest` | rewrite in Phase 5 | +| Proto | `dc3-api/dc3-api-auth/src/main/protobuf/api/common/auth/mcp_runtime.proto` | 5 个 RPC，260 行 | +| Facade | `dc3-common-facade-api/.../McpRuntimeFacade.java` | 5 个同步方法 | +| gRPC 实现 | `dc3-common-facade-grpc/.../McpRuntimeGrpcFacade.java` | 阻塞 stub + `GrpcFacadeSupport` | +| 网关 | `dc3-common-gateway/.../McpGatewayController.java` | JSON-RPC 分发、`blocking()`+boundedElastic、`invokeBackend`、`audit` | +| Auth 服务 | `dc3-common-auth/.../OAuthMcpRuntimeServiceImpl.java` | `introspect`、`resolveVisibleTool`、`authorizeToolCall` | +| gRPC 服务端 | `dc3-common-auth/.../McpRuntimeServer.java` | 5 个服务端方法 | +| Schema 来源 | `dc3-common-auth/.../tool/McpOpenApiAggregator.java` | 已计算 `inputSchema` 但未随 gRPC 传递 | +| 测试 | `McpRuntimeServerTest`、`McpGatewayControllerTest`、`OAuthMcpRuntimeServiceImplTest` | 在第 5 阶段重写 | diff --git a/docs/design/mq-abstraction.md b/docs/design/mq-abstraction.md index ccc1ac636..7a851f524 100644 --- a/docs/design/mq-abstraction.md +++ b/docs/design/mq-abstraction.md @@ -1,161 +1,145 @@ -# Design: Pluggable Message Broker Abstraction (MQ Port) +# 设计：可插拔消息中间件抽象（MQ Port） -| | | -|----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **Status** | Delivered: port + five adapters (rabbitmq / kafka / activemq / mqtt / pulsar) TCK-certified; rocketmq adapter experimental (opt-in TCK, not yet certified) | -| **Date** | 2026-08-17, revised 2026-08-19 (re-verified after commit `956de3dd3`; MQTT decoupling) | -| **Scope** | `dc3-common-*` messaging layer (center ↔ driver async plane) | -| **Target** | RabbitMQ (default), Kafka, RocketMQ, Pulsar, ActiveMQ (Artemis / Classic), MQTT 5 (EMQX / HiveMQ / NanoMQ / …) | -| **Related** | [`storage-abstraction.md`](./storage-abstraction.md) — relational + time-series pluggability, same profile/TCK mechanism | -| **Discussion** | open for review before Phase 1 starts | +| | | +|------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **状态** | 已交付：Port + 五个适配器（rabbitmq / kafka / activemq / mqtt / pulsar）已通过 TCK 认证；rocketmq 适配器为实验性（TCK 可选启用，尚未认证） | +| **日期** | 2026-08-17，修订于 2026-08-19（提交 `956de3dd3` 后复核；MQTT 解耦） | +| **范围** | `dc3-common-*` 消息层（中心↔驱动 异步平面） | +| **目标** | RabbitMQ（默认）、Kafka、RocketMQ、Pulsar、ActiveMQ（Artemis / Classic）、MQTT 5（EMQX / HiveMQ / NanoMQ / …） | +| **相关** | [`storage-abstraction.md`](./storage-abstraction.md) —— 关系 + 时序可插拔，同一套 profile/TCK 机制 | +| **讨论** | Phase 1 启动前开放评审 | -## 1. Summary +## 1. 摘要 -IoT DC3 currently talks to RabbitMQ through its native Spring AMQP API (`RabbitTemplate`, -`@RabbitListener`, `Channel`, `CorrelationData`) from three business modules. As a globally distributed open-source -project we want deployers — and the community — to be able to run the platform on whichever broker their environment -already standardizes on. +IoT DC3 目前在三个业务模块中通过 Spring AMQP 原生 API（`RabbitTemplate`、`@RabbitListener`、`Channel`、 +`CorrelationData`）与 RabbitMQ 交互。作为一个全球分布的开源项目，我们希望部署者——以及社区——能够把平台运行在 +其环境已经标准化的任意中间件上。 -The proposal: introduce a thin **port** (`dc3-mq-core`) that abstracts the *semantics* the platform actually uses — -logical topics, load-balanced vs broadcast subscription, delayed delivery, dead-lettering, batch consumption with -transactional acknowledgement, bounded redelivery, publisher confirmation — plus one adapter module per broker. RabbitMQ -comes first as a byte-for-byte wire-compatible migration of the existing code; a broker-neutral contract-test suite -(TCK) makes community adapters for other brokers possible with a clear acceptance bar. One of the adapters is **MQTT -5**, so the internal plane can also run over whichever MQTT broker the deployment already operates. +本方案：引入一个薄 **Port**（`dc3-mq-core`），抽象平台实际使用的*语义*——逻辑主题、负载均衡订阅与广播订阅、 +延迟投递、死信、带事务性确认的批量消费、有界重投、发布者确认——并为每个中间件提供一个适配器模块。RabbitMQ +率先落地，作为现有代码逐字节线级兼容的迁移；一套中间件中立（broker-neutral）的契约测试套件（TCK）以明确的 +验收门槛让社区为其他中间件贡献适配器成为可能。其中一个适配器是 **MQTT 5**，因此内部平面也可以运行在部署环境 +已在运营的任意 MQTT 中间件之上。 -The design deliberately does **not** try to model broker topology (exchanges, bindings, partitions). It models the two -subscription modes every mainstream broker can express, and negotiates everything else through declared capabilities -with documented fallbacks. +本设计刻意 **不** 去建模中间件拓扑（exchange、binding、分区）。它只建模每个主流中间件都能表达的两种订阅模式， +其余一切通过声明的能力进行协商，并配有文档化的回退(fallback)。 -MQTT plays two distinct roles in the platform and both must treat the broker as a free selection: the **device-access -plane** (vendor-neutral by protocol already) and — via the new adapter — the **internal async plane** itself. §7.1 draws -the boundary. +MQTT 在平台中扮演两个不同的角色，二者都必须把中间件视为自由选择：**南向设备面**（协议层面本就厂商中立）以及 +——通过新适配器——**内部异步平面**本身。§7.1 划定了这一边界。 -Two recent platform changes (2026-08-18, commit `956de3dd3` — lease-fenced durable telemetry) -strengthen rather than invalidate this design: the driver now owns durability through a mandatory SQLite outbox in front -of the broker, and point-value ingestion is transactional and idempotent (schema-versioned payloads with `messageId` -dedupe and fencing tokens). Both reduce how much the port has to demand from the broker. +两项近期的平台变更（2026-08-18，提交 `956de3dd3`——租约 fencing 的持久化遥测）是加强而非否定本设计：驱动现在 +通过中间件前置的强制性 SQLite 发件箱(outbox) 掌控持久性，且点位值摄入是事务性且幂等的（带 schema 版本的载荷， +配 `messageId` 去重与 fencing token）。两者都降低了 Port 对中间件的要求。 -## 2. Background — how RabbitMQ is used today +## 2. 现状 —— RabbitMQ 目前的使用方式 -Accurate as of 2026-08-19, re-verified after commit `956de3dd3` (2026-08-18). All numbers verified against the tree. +信息截至 2026-08-19 准确，在提交 `956de3dd3`（2026-08-18）之后复核。所有数字均已对照代码树核实。 -**Topology** (names centralized in -`dc3-common/dc3-common-constant/.../driver/RabbitConstant.java`): +**拓扑**（名称集中定义于 +`dc3-common/dc3-common-constant/.../driver/RabbitConstant.java`）： -- **15 declared topic exchanges** (`dc3.e.`): state, alarm, metadata, point_command, value, mqtt, - state_timeout_delay, state_timeout_check, command, command_result, command_dead, event (declared in - `dc3-common-rabbitmq/.../ExchangeConfig.java`) plus point_value_dead, point_command_dead, point_command_result - (declared in - `dc3-common-data/.../DataTopicConfig.java`). Two of them are dead weight: `dc3.e.mqtt` is declared but has no binding, - producer, or consumer, and the `register` exchange/queue constants in `RabbitConstant` are declared and referenced - nowhere — both are Phase 1 cleanup candidates (§12; routing through an MQTT broker belongs to the §7.1 adapter, not a - bespoke exchange). -- **19 declared queue definitions** — 16 center-side (load-balanced and dead-letter) in - `DataTopicConfig` and 3 driver-side patterns in `dc3-common-driver/.../DriverTopicConfig.java`, the latter - instantiated per driver client. The driver-side command queues carry a lease-coupled `x-expires` (§8.8); the metadata - queue is auto-delete with 30 s TTL. -- Routing keys `dc3.r..`; environment prefix via the `dc3.rabbit.tag` - system property. +- **15 个已声明的 topic exchange**（`dc3.e.`）：state、alarm、metadata、point_command、value、mqtt、 + state_timeout_delay、state_timeout_check、command、command_result、command_dead、event（声明于 + `dc3-common-rabbitmq/.../ExchangeConfig.java`），外加 point_value_dead、point_command_dead、point_command_result + （声明于 + `dc3-common-data/.../DataTopicConfig.java`）。其中两个是无用的累赘：`dc3.e.mqtt` 已声明但没有任何 binding、 + 生产者或消费者，`RabbitConstant` 中的 `register` exchange/queue 常量声明了却无处引用——两者都是 Phase 1 的 + 清理候选（§12；经 MQTT 中间件路由属于 §7.1 适配器的职责，而非定制 exchange）。 +- **19 个已声明的队列定义** —— `DataTopicConfig` 中 16 个中心侧（负载均衡与死信）队列， + `dc3-common-driver/.../DriverTopicConfig.java` 中 3 个驱动侧模式，后者按每个驱动客户端实例化。驱动侧命令队列 + 携带与租约耦合的 `x-expires`（§8.8）；元数据队列为 auto-delete，TTL 30 秒。 +- 路由键 `dc3.r..`；环境前缀通过 `dc3.rabbit.tag` 系统属性指定。 -**Producers** (14 raw `rabbitTemplate.convertAndSend` call sites in main code; the driver's 8 business send methods -funnel through 6 raw sites): +**生产者**（主代码中 14 处裸 `rabbitTemplate.convertAndSend` 调用点；驱动的 8 个业务发送方法汇入其中 6 处裸调用点）： -| Side | Wrapped behind interface? | Sites | +| 侧 | 是否有接口封装？ | 调用点 | |----------------|---------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------| -| Driver | Yes — `DriverSenderService` → `DriverSenderServiceImpl` | 8 sends (6 raw call sites) + `BufferServiceImpl` outbox republish | -| Data center | No — direct `RabbitTemplate` | `CommandHistoryServiceImpl`, `PointCommandServiceImpl`, `DriverStateServiceImpl`, `NotifyTaskSender`, `EntityStateExpiryScanner` | -| Manager center | No — direct `RabbitTemplate` | `MetadataEventListener` | +| 驱动 | 是 —— `DriverSenderService` → `DriverSenderServiceImpl` | 8 个发送（6 处裸调用点）+ `BufferServiceImpl` 发件箱(outbox) 重发布 | +| 数据中心 | 否 —— 直接使用 `RabbitTemplate` | `CommandHistoryServiceImpl`、`PointCommandServiceImpl`、`DriverStateServiceImpl`、`NotifyTaskSender`、`EntityStateExpiryScanner` | +| 管理中心 | 否 —— 直接使用 `RabbitTemplate` | `MetadataEventListener` | -**Consumers** — 16 `@RabbitListener` methods, all manual-ack via `Channel` + -`RabbitAckUtil`, split across three container factories (latency-tuned default, high-throughput, and — since -`956de3dd3` — the batch-enabled -`pointValueRabbitListenerContainerFactory` for the point-value path): 13 in `dc3-common-data` -(`data/rabbit/*` receivers, `NotifyWorker`, `EntityStateExpiryScanner`) and 3 in -`dc3-common-driver` (`receiver/rabbit/*`). The point-value **dead** queue intentionally has no listener: since -`956de3dd3` it is a consumer-less quarantine (§8.2). +**消费者** —— 16 个 `@RabbitListener` 方法，全部通过 `Channel` + `RabbitAckUtil` 手动确认，分布于三个容器工厂 +（低延迟调优的默认工厂、高吞吐工厂，以及——自 `956de3dd3` 起——点位值路径启用批量的 +`pointValueRabbitListenerContainerFactory`）：13 个位于 `dc3-common-data` +（`data/rabbit/*` 接收器、`NotifyWorker`、`EntityStateExpiryScanner`），3 个位于 +`dc3-common-driver`（`receiver/rabbit/*`）。点位值 **死信** 队列刻意没有监听器：自 +`956de3dd3` 起它是无消费者的隔离区（§8.2）。 -**Module coupling** — `dc3-common-data`, `dc3-common-driver`, `dc3-common-manager` and -`dc3-common-facade-local-manager` depend directly on `dc3-common-rabbitmq`, so the Spring AMQP API leaks into every -business module. A parallel module, `dc3-common-mqtt` -(Spring Integration MQTT + Eclipse Paho v3 client), carries the driver-side MQTT device-access plane; only -`dc3-driver-mqtt` depends on it, and it contains no broker-vendor code — EMQX is a deployment default, not a code -dependency (§7.1). +**模块耦合** —— `dc3-common-data`、`dc3-common-driver`、`dc3-common-manager` 与 +`dc3-common-facade-local-manager` 直接依赖 `dc3-common-rabbitmq`，因此 Spring AMQP API 泄漏进每个 +业务模块。另一个并行模块 `dc3-common-mqtt` +（Spring Integration MQTT + Eclipse Paho v3 客户端）承载驱动侧的 MQTT 南向设备面；只有 +`dc3-driver-mqtt` 依赖它，且其中不含任何中间件厂商代码——EMQX 是部署默认值，不是代码 +依赖（§7.1）。 -**Eight RabbitMQ-specific semantics** the abstraction must carry (the hard part — plain send/receive is easy): +**抽象必须承载的八项 RabbitMQ 特有语义**（难点所在——普通的收发很容易）： -| # | Semantic | Current implementation | +| # | 语义 | 当前实现 | |---|--------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 1 | Delayed messages | TTL + DLX: 45 s driver-timeout check, 10 s device-scan tick (`DriverStateServiceImpl`, `EntityStateExpiryScanner`, delay/check exchanges in `DataTopicConfig`) | -| 2 | Per-instance broadcast | Driver metadata sync via auto-delete queue (TTL 30 s) named per driver client (`DriverTopicConfig`) | -| 3 | Dead-letter queues | point_value (live-queue TTL 7 d → DLX; dead queue is a consumer-less quarantine since `956de3dd3`), point_command, command DLX + dedicated dead receivers (`PointCommandDeadReceiver`, `CommandDeadReceiver`) | -| 4 | Manual ack + bounded back-pressure | manual ack everywhere; the point-value path acks a whole broker batch only after the PostgreSQL transaction commits (`PointValueReceiver` + `PointValueRabbitConfig`); back-pressure is bounded by prefetch ≥ batchSize with the broker as durable buffer (the old data-side ingest buffer and its buffer-full `nack(requeue=true)` were removed in `956de3dd3`) | -| 5 | Durable outbox + publisher confirm | driver persists every point value to a **mandatory SQLite outbox** (WAL, `synchronous=FULL`) before publishing; the confirm future and a retry scheduler delete or republish (`BufferServiceImpl`, `PointValueBuffer`) | -| 6 | Queue TTL as retention guard | state/alarm 30 s, event/command_result 60 s, notify_task 24 h, point_value 7 d, driver-side command queues 30 s (`DataTopicConfig`, `DriverTopicConfig`) | -| 7 | Batch consumption | consumer-side batching on the point-value path: batch listener + consumer batch, batch-level bounded retry with backoff, exhaustion rejects the whole batch to DLX (`PointValueRabbitConfig`) | -| 8 | Lease-coupled per-instance queue lifecycle | driver command queues expire (`x-expires` from `driver.lease.queue-expires`) with the driver lease; metadata broadcast queues are auto-delete with 30 s TTL (`DriverTopicConfig`) | +| 1 | 延迟消息 | TTL + DLX：45 秒驱动超时检查、10 秒设备扫描 tick（`DriverStateServiceImpl`、`EntityStateExpiryScanner`，delay/check exchange 见 `DataTopicConfig`） | +| 2 | 每实例广播 | 驱动元数据同步，通过按驱动客户端命名的 auto-delete 队列（TTL 30 秒）（`DriverTopicConfig`） | +| 3 | 死信队列 | point_value（活队列 TTL 7 天 → DLX；死队列自 `956de3dd3` 起为无消费者的隔离区）、point_command、command DLX + 专用死信接收器（`PointCommandDeadReceiver`、`CommandDeadReceiver`） | +| 4 | 手动确认 + 有界背压 | 到处手动确认；点位值路径仅在 PostgreSQL 事务提交后才确认整个中间件批次（`PointValueReceiver` + `PointValueRabbitConfig`）；背压以 prefetch ≥ batchSize 为界，中间件充当持久缓冲（旧的数据侧摄入缓冲及其缓冲满时的 `nack(requeue=true)` 已在 `956de3dd3` 移除） | +| 5 | 持久发件箱(outbox) + 发布者确认 | 驱动在发布前把每个点位值持久化到 **强制性 SQLite 发件箱(outbox)**（WAL、`synchronous=FULL`）；确认 future 与重试调度器负责删除或重发布（`BufferServiceImpl`、`PointValueBuffer`） | +| 6 | 队列 TTL 作为保留守卫 | state/alarm 30 秒、event/command_result 60 秒、notify_task 24 小时、point_value 7 天、驱动侧命令队列 30 秒（`DataTopicConfig`、`DriverTopicConfig`） | +| 7 | 批量消费 | 点位值路径上的消费者侧批量：批量监听器 + 消费者批量，批次级有界退避重试，耗尽后将整个批次拒绝进 DLX（`PointValueRabbitConfig`） | +| 8 | 与租约耦合的每实例队列生命周期 | 驱动命令队列随驱动租约过期（`x-expires` 取自 `driver.lease.queue-expires`）；元数据广播队列为 auto-delete，TTL 30 秒（`DriverTopicConfig`） | -Two load-bearing facts make the whole abstraction feasible: +两个承重事实使整个抽象可行： -- **The delayed-timeout paths already treat the database lease as the source of truth** - (`leaseVersion` idempotency check in `DriverTimeoutCheckReceiver`). The MQ delay is only a *trigger*, so a broker - without native delayed delivery can fall back to a local scheduler without breaking correctness. -- **At-least-once + idempotent consumers is now implemented, not assumed**: point-value payloads carry a schema-v1 - envelope (`schemaVersion`, `messageId`, `sequence`, - `fencingToken`, `driverNode`) validated on ingest, and Data-Center ingestion is transactional and idempotent - (`956de3dd3`). +- **延迟超时路径已经把数据库租约当作事实来源** + （`DriverTimeoutCheckReceiver` 中的 `leaseVersion` 幂等检查）。MQ 延迟只是一个 *触发器*，因此不具备原生延迟投递的 + 中间件可以回退(fallback) 到本地调度器而不破坏正确性。 +- **至少一次 + 幂等消费者现在已实现，而非假设**：点位值载荷携带 schema-v1 + 信封（`schemaVersion`、`messageId`、`sequence`、 + `fencingToken`、`driverNode`）并在摄入时校验，数据中心摄入是事务性且幂等的 + （`956de3dd3`）。 -Cross-cutting concerns that already exist and must survive the abstraction unchanged: -JSON payload convention (Jackson) — formalized on the point-value path by the schema-v1 envelope, `X-Request-Id` MDC -propagation across the broker hop (`MdcRequestIdMessagePostProcessor` / `MdcRequestIdListenerAdvice`), persistent -delivery, tenant-scoped payloads. +已经存在且必须在抽象中原样保留的横切关注点： +JSON 载荷约定（Jackson）——在点位值路径上由 schema-v1 信封固化，`X-Request-Id` 的 MDC 跨中间件跳传播 +（`MdcRequestIdMessagePostProcessor` / `MdcRequestIdListenerAdvice`），持久化投递，租户作用域载荷。 -## 3. Goals / Non-Goals +## 3. 目标 / 非目标 -**Goals** +**目标** -- Business modules (`dc3-common-data`, `dc3-common-driver`, `dc3-common-manager`) compile against a broker-neutral API - with **zero** broker classes on the compile classpath. -- One adapter module per broker; exactly one adapter active at runtime, selected by - `dc3.mq.type` (default `rabbitmq`). -- The MQTT broker is a free deployment selection in both of its roles: the device-access plane stays vendor-neutral by - protocol (EMQX is a compose default, not a code dependency), and the internal plane may optionally run over any MQTT 5 - broker via the - `dc3-mq-mqtt` adapter (§7.1). -- Existing RabbitMQ deployments keep working through the migration: physical exchange/queue/routing-key names - byte-for-byte identical, no forced redeploy of the whole fleet at once. -- The point-value path's throughput semantics survive the abstraction: batch delivery with post-commit batch - acknowledgement and bounded redelivery. -- Third-party driver authors remain unaffected: `DriverSenderService` signature does not change. -- Community-contributed adapters have a mechanical acceptance bar (the TCK). +- 业务模块（`dc3-common-data`、`dc3-common-driver`、`dc3-common-manager`）面向中间件中立 API 编译，编译类路径上 + **零** 中间件类。 +- 每个中间件一个适配器模块；运行时恰好激活一个适配器，由 + `dc3.mq.type` 选择（默认 `rabbitmq`）。 +- MQTT 中间件在其两个角色中都是自由的部署选择：南向设备面在协议层面保持厂商中立（EMQX 是 compose 默认值， + 不是代码依赖），内部平面可选地通过 + `dc3-mq-mqtt` 适配器运行在任意 MQTT 5 中间件之上（§7.1）。 +- 现有 RabbitMQ 部署在迁移期间持续可用：物理 exchange/queue/路由键名称逐字节一致，不强制一次性重新部署全部实例。 +- 点位值路径的吞吐语义在抽象之后保持不变：批量投递、提交后批次确认、有界重投。 +- 第三方驱动作者不受影响：`DriverSenderService` 签名不变。 +- 社区贡献的适配器有一个机械化的验收门槛（TCK）。 -**Non-Goals** +**非目标** -- Exactly-once delivery or cross-broker distributed transactions. The contract is **at-least-once + idempotent - consumers** — formally implemented on the point-value path (messageId dedupe, fencing tokens) and required by - convention elsewhere. -- Multi-broker topologies (bridging RabbitMQ to Kafka etc.). -- Rewriting the device-access MQTT plane (`dc3-common-mqtt`, `dc3-driver-mqtt`) — those are protocol drivers, not the - internal async plane. §7.1 records the boundary and the vendor-neutrality guardrails instead. -- Hiding performance/ordering differences between brokers (documented, not erased). +- 恰好一次投递或跨中间件分布式事务。契约是 **至少一次 + 幂等 + 消费者**——在点位值路径上正式实现（messageId 去重、fencing token），其余路径按约定要求。 +- 多中间件拓扑（把 RabbitMQ 桥接到 Kafka 等）。 +- 重写南向设备面的 MQTT 平面（`dc3-common-mqtt`、`dc3-driver-mqtt`）——它们是协议驱动，不是 + 内部异步平面。§7.1 改为记录边界与厂商中立护栏。 +- 隐藏中间件之间的性能/顺序差异（文档化，而非抹平）。 -## 4. Design principles +## 4. 设计原则 -1. **Abstract semantics, not broker features.** The unifying level is *logical topic + subscription mode*. - Exchange/binding/partition/tag models are adapter internals. The two subscription modes (`LOAD_BALANCE`, `BROADCAST`) - are the lowest common semantics all five target brokers can express. -2. **Capability negotiation with graceful degradation, not lowest common denominator.** - The adapter declares what it supports; the core applies documented fallbacks (e.g. delayed message → local scheduler) - and logs the negotiated result at startup. We never dumb the API down to what JMS can do. -3. **One envelope format.** Payload is always JSON bytes; type info and trace context ride in standardized headers - (`dc3-type`, `X-Request-Id`, `tenant-id`), and payload-level schema versioning is already in production on the - point-value path (schema-v1 envelope). Serialization lives in the API layer — no reliance on broker-side type - converters (today's `__TypeId__` header is a Spring AMQP internal). This keeps the wire format identical across - brokers. +1. **抽象语义，而非中间件特性。** 统一层次是 *逻辑主题 + 订阅模式*。 + exchange/binding/分区/tag 模型属于适配器内部。两种订阅模式（`LOAD_BALANCE`、`BROADCAST`） + 是全部五个目标中间件都能表达的最低公共语义。 +2. **能力协商与优雅降级，而非最低公共分母。** + 适配器声明自己支持什么；核心应用文档化的回退(fallback)（如延迟消息 → 本地调度器）， + 并在启动时记录协商结果。我们绝不把 API 降到 JMS 能做到的水平。 +3. **单一信封格式。** 载荷始终是 JSON 字节；类型信息与追踪上下文搭载在标准化 header + （`dc3-type`、`X-Request-Id`、`tenant-id`）中，且载荷级 schema 版本化已在点位值路径上生产使用 + （schema-v1 信封）。序列化位于 API 层——不依赖中间件侧的类型转换器（今天的 `__TypeId__` header 是 Spring AMQP + 内部机制）。这保证线级格式在各中间件间完全一致。 -## 5. Module layout +## 5. 模块布局 -Mirrors the existing facade-module organization (contract module + per-transport implementations): +镜像现有的 facade 模块组织方式（契约模块 + 每种传输一个实现）： ``` dc3-mq/ # top-level aggregator: the broker-selection family @@ -173,11 +157,11 @@ dc3-mq/ # top-level aggregator: the broker-selection fa └── dc3-mq-tck/ # Broker-neutral contract test suite (Testcontainers) ``` -(Restructured 2026-08-20 out of `dc3-common` into the top-level `dc3-mq` aggregator — the family is a deployment -selection, not shared plumbing; Java packages stay -`io.github.pnoker.common.mq.*`.) +（2026-08-20 重组：从 `dc3-common` 迁出，成为顶层 `dc3-mq` 聚合模块——该家族是部署 +选择，不是共享基础设施；Java 包名保持 +`io.github.pnoker.common.mq.*`。） -Dependency direction after migration: +迁移后的依赖方向： ``` dc3-common-data ─┐ @@ -187,11 +171,11 @@ dc3-common-manager ─┘ ▲ dc3-mq-rabbitmq / -kafka / -rocketmq / -pulsar / -activemq / -mqtt ``` -Adapters are selected by `dc3.mq.type` and activated via -`@ConditionalOnProperty(prefix = "dc3.mq", name = "type", havingValue = "...")`, the same pattern `dc3.facade.mode` uses -for facade transports. +适配器由 `dc3.mq.type` 选择，并通过 +`@ConditionalOnProperty(prefix = "dc3.mq", name = "type", havingValue = "...")` 激活——与 `dc3.facade.mode` 为 +facade 传输使用的模式相同。 -## 6. Core API +## 6. 核心 API ```java package io.github.pnoker.common.mq; @@ -285,383 +269,372 @@ public record BrokerCapabilities( ) {} ``` -Business-side shape after migration — a single-message receiver becomes: +迁移后的业务侧形态——单消息接收器变为： ```java @Dc3Listener(topic = MqTopic.COMMAND, mode = LOAD_BALANCE, profile = LATENCY) public void commandReceive(MqReceived message, Acknowledgment ack) { ... } ``` -and the batch point-value receiver keeps its throughput semantics: +批量点位值接收器保持其吞吐语义： ```java @Dc3Listener(topic = MqTopic.POINT_VALUE, mode = LOAD_BALANCE, profile = THROUGHPUT, delivery = BATCH) public void pointValueReceive(List> messages, Acknowledgment ack) { ... } ``` -batchSize, prefetch and the retry bounds bind from configuration (`BatchConsumerProperties` -today) rather than annotation literals, so ops can tune them per deployment. +batchSize、prefetch 与重试上界从配置绑定（今天是 `BatchConsumerProperties`）而非注解字面量， +以便运维按部署调优。 -`Channel`, `Message`, delivery tags and `RabbitAckUtil` disappear from business code. The -`@Dc3Listener` annotation is processed by `dc3-mq-core` core, which registers the subscription with the active adapter — -mirroring how `@RabbitListener` is processed today, minus the broker API. +`Channel`、`Message`、投递标签与 `RabbitAckUtil` 从业务代码中消失。The +`@Dc3Listener` 注解由 `dc3-mq-core` 核心处理，它向激活的适配器注册订阅—— +镜像今天 `@RabbitListener` 的处理方式，只是去掉了中间件 API。 -Poison handling within a batch is batch-granular — matching today's behavior, where one invalid message causes the whole -batch to retry and then dead-letter (§8.7). Whether the port should eventually offer per-message extraction is open -question §13.7. +批次内的毒消息处理是批次粒度的——与今天的行为一致：一条无效消息导致整个 +批次重试然后死信（§8.7）。Port 最终是否应提供按消息提取是开放 +问题 §13.7。 -## 7. Subscription modes and destination mapping +## 7. 订阅模式与目的地映射 -The two subscription modes are the central modeling decision: +两种订阅模式是核心建模决策： -- **LOAD_BALANCE** — competing consumers, each message handled once platform-wide. Used by point values, states, alarms, - commands, events, notify tasks. -- **BROADCAST** — every running instance gets its own copy. Used by driver metadata sync (each driver process must - refresh its local cache) and the device-scan tick fan-out. +- **LOAD_BALANCE** —— 竞争消费者，每条消息在全平台只被处理一次。用于点位值、状态、告警、 + 命令、事件、通知任务。 +- **BROADCAST** —— 每个运行实例都获得自己的副本。用于驱动元数据同步（每个驱动进程必须 + 刷新本地缓存）与设备扫描 tick 扇出。 -Physical mapping per broker (namespace = today's `dc3.rabbit.tag` environment prefix): +各中间件的物理映射（namespace = 今天的 `dc3.rabbit.tag` 环境前缀）： -| Semantic | RabbitMQ (adapter keeps current names) | Kafka | RocketMQ | Pulsar | ActiveMQ (JMS) | MQTT 5 | +| 语义 | RabbitMQ（适配器保持现有名称） | Kafka | RocketMQ | Pulsar | ActiveMQ (JMS) | MQTT 5 | |---------------------|-----------------------------------------------------------|------------------------------------------------|-----------------------------------------------|--------------------------------------|----------------------------------------------------|------------------------------------------------------| -| LOAD_BALANCE | shared durable queue bound `rk.*` | consumer group on topic `dc3.` | CLUSTERING consume mode | Shared subscription `dc3-` | JMS Queue | shared subscription `$share//dc3/` | -| BROADCAST | per-instance auto-delete queue (current design) | per-instance `groupId = group + instanceId` | BROADCASTING consume mode | Exclusive subscription per instance | JMS Topic, unshared durable subscriber | normal subscription, persistent session per instance | -| Partition key | routing-key suffix `.` | record key → partition | message key (routing only, no ordering) | key (routing only, shared sub) | `dc3-partition-key` property | `dc3-partition-key` user property | -| Key-pattern routing | broker topic bindings | client-side router (`KeyMatcher`/`KeyRoutes`) | client-side router | client-side router | client-side router | client-side router | -| Batch | consumer batch (`setBatchListener`, prefetch ≥ batchSize) | poll loop (`max.poll.records`) | batch consumption (`consumeMessageBatchSize`) | batch receive API | adapter drains within a short window (synthesized) | adapter drains within a short window (synthesized) | -| Instance expiry | `x-expires` / auto-delete + TTL (§8.8) | ❌ offsets persist — documented cleanup policy | subscription group config | subscription expiry policy | no direct equivalent — documented cleanup policy | session expiry interval ⚠️ (§13.8) | -| Namespace | name prefix (today) | topic prefix | namespace | Pulsar tenant/namespace (native fit) | destination prefix | topic prefix (`dc3/`, slash-separated) | +| LOAD_BALANCE | 绑定 `rk.*` 的共享持久队列 | topic `dc3.` 上的消费者组 | CLUSTERING 消费模式 | 共享订阅 `dc3-` | JMS Queue | 共享订阅 `$share//dc3/` | +| BROADCAST | 每实例 auto-delete 队列（现有设计） | 每实例 `groupId = group + instanceId` | BROADCASTING 消费模式 | 每实例独占订阅 | JMS Topic，非共享持久订阅 | 普通订阅，每实例持久会话 | +| 分区键 | 路由键后缀 `.` | record key → 分区 | 消息 key（仅路由，不保证顺序） | key（仅路由，共享订阅） | `dc3-partition-key` 属性 | `dc3-partition-key` 用户属性 | +| Key 模式路由 | 中间件 topic binding | 客户端侧路由器（`KeyMatcher`/`KeyRoutes`） | 客户端侧路由器 | 客户端侧路由器 | 客户端侧路由器 | 客户端侧路由器 | +| 批量 | 消费者批量（`setBatchListener`，prefetch ≥ batchSize） | poll 循环（`max.poll.records`） | 批量消费（`consumeMessageBatchSize`） | 批量接收 API | 适配器在短窗口内排空（合成） | 适配器在短窗口内排空（合成） | +| 实例过期 | `x-expires` / auto-delete + TTL（§8.8） | ❌ offset 持久存在 —— 文档化清理策略 | 订阅组配置 | 订阅过期策略 | 无直接等价物 —— 文档化清理策略 | 会话过期间隔 ⚠️（§13.8） | +| 命名空间 | 名称前缀（今天） | topic 前缀 | namespace | Pulsar tenant/namespace（原生契合） | destination 前缀 | topic 前缀（`dc3/`，斜杠分隔） | -Ordering note (documented, not hidden): RabbitMQ today guarantees nothing; Kafka with -`partitionKey = driver service` gives per-driver ordering — strictly stronger than the current semantics, and it matters -for point-value ingestion order. +顺序说明（文档化，而非隐藏）：RabbitMQ 今天不提供任何顺序保证；Kafka 以 +`partitionKey = driver service` 提供每驱动顺序——严格强于当前语义，且它对 +点位值摄入顺序很重要。 -### 7.1 The MQTT boundary — two planes, one principle +### 7.1 MQTT 边界 —— 两平面一原则 -MQTT appears in two distinct roles in the platform. Both must treat the MQTT broker as a free, swappable selection — -same principle as the rest of this design: +MQTT 在平台中以两个不同角色出现。二者都必须把 MQTT 中间件视为自由、可替换的选择—— +与本设计其余部分遵循同一原则： -1. **Device-access plane (southbound)** — `dc3-common-mqtt` + `dc3-driver-mqtt`. This is protocol access, not the - internal async plane, and it is already vendor-neutral by construction: a standard Eclipse Paho v3 client (MQTT - 3.1.1) configured entirely through `dc3.driver.mqtt.*` (URL, auth, TLS material, topic prefix). EMQX is the - compose-bundled default (`docker-compose-optional.yml`), nothing more — Mosquitto, HiveMQ, NanoMQ or VerneMQ drop in - via configuration alone. Guardrails that keep it that way: no broker-vendor management API, rule engine, or - plugin-specific code in Java modules; vendor specifics live in deploy configuration only. The client library pins the - *protocol version* (3.1.1 today), not the vendor — upgrading to an MQTT 5 client is an independent, optional library - change. -2. **Internal async plane (optional broker)** — the `dc3-mq-mqtt` adapter. MQTT 5 shared subscriptions express - LOAD_BALANCE; a normal subscription per instance expresses BROADCAST; QoS 1 gives per-message ack and PUBACK - confirmation (§8.4). This lets a deployment consolidate: one MQTT broker serving both planes, or a smaller stack - without RabbitMQ at all. The gaps fall back through the standard mechanisms (delay → local scheduler, DLQ → explicit - topic, batch → synthesized windowing), and the TCK — including group durability with all instances down (§11 case - 13) — is the acceptance bar, because shared-subscription retention semantics vary between brokers (§13.8). +1. **南向设备面（southbound）** —— `dc3-common-mqtt` + `dc3-driver-mqtt`。这是协议接入，不是 + 内部异步平面，且它在构造上已厂商中立：标准 Eclipse Paho v3 客户端（MQTT + 3.1.1），完全通过 `dc3.driver.mqtt.*`（URL、认证、TLS 材料、topic 前缀）配置。EMQX 是 + compose 捆绑的默认值（`docker-compose-optional.yml`），仅此而已——Mosquitto、HiveMQ、NanoMQ 或 VerneMQ 仅凭 + 配置即可替换。保持这一性质的护栏：Java 模块中不含任何中间件厂商管理 API、规则引擎或 + 插件专属代码；厂商细节只存在于部署配置中。客户端库锁定的是 + *协议版本*（今天是 3.1.1），不是厂商——升级到 MQTT 5 客户端是一次独立的、可选的库 + 变更。 +2. **内部异步平面（可选中间件）** —— `dc3-mq-mqtt` 适配器。MQTT 5 共享订阅表达 + LOAD_BALANCE；每实例普通订阅表达 BROADCAST；QoS 1 提供逐消息确认与 PUBACK + 确认（§8.4）。这让部署得以整合：一个 MQTT 中间件同时服务两个平面，或彻底去掉 + RabbitMQ 的更小技术栈。能力缺口通过标准机制回退(fallback)（延迟 → 本地调度器、DLQ → 显式 + topic、批量 → 合成窗口），而 TCK——包括全体实例宕机时的组持久性（§11 用例 + 13）——是验收门槛，因为共享订阅的保留语义因中间件而异（§13.8）。 -The two planes stay independent: running the internal plane over MQTT changes nothing for the device-access drivers, and -swapping the device-access broker changes nothing for the internal plane. +两个平面保持独立：内部平面改跑 MQTT 对南向设备面的驱动毫无影响，替换南向设备面的中间件对内部平面也毫无影响。 -## 8. Hard semantics, broker by broker +## 8. 逐中间件梳理的硬语义 -### 8.1 Delayed messages +### 8.1 延迟消息 -| Broker | Mechanism | +| 中间件 | 机制 | |----------|------------------------------------------------------------------------| -| RabbitMQ | current TTL + DLX pattern (unchanged); delayed-message plugin optional | -| RocketMQ | delay levels / arbitrary timing (5.x) | -| Pulsar | delayed delivery API | -| ActiveMQ | JMS scheduled delivery (`AMQ_SCHEDULED_DELAY`) | -| Kafka | **none** → API-layer fallback: local `ScheduledExecutor` re-send | -| MQTT | **none** → same local scheduler fallback | +| RabbitMQ | 现有 TTL + DLX 模式（不变）；延迟消息插件可选 | +| RocketMQ | 延迟级别 / 任意定时（5.x） | +| Pulsar | 延迟投递 API | +| ActiveMQ | JMS 定时投递（`AMQ_SCHEDULED_DELAY`） | +| Kafka | **无** → API 层回退(fallback)：本地 `ScheduledExecutor` 重发 | +| MQTT | **无** → 同样的本地调度器回退(fallback) | -Fallback safety: consumers of delayed messages (`DriverTimeoutCheckReceiver`, device scan) are already idempotent -against the DB lease (`leaseVersion`). Multiple or late triggers are harmless. The fallback runs in whichever instance -sent the message; startup logs state `delayedMessage=false → local scheduler fallback active`. +回退(fallback) 安全性：延迟消息的消费者（`DriverTimeoutCheckReceiver`、设备扫描）已经针对数据库租约 +（`leaseVersion`）幂等。多次或迟到触发无害。回退(fallback) 运行在发送该消息的实例中；启动日志输出 +`delayedMessage=false → local scheduler fallback active`。 -### 8.2 Dead-letter queues +### 8.2 死信队列 -`reject(false)` is the single entry point. The adapter maps it to: +`reject(false)` 是唯一入口。适配器把它映射为： -| Broker | Mechanism | +| 中间件 | 机制 | |----------|-------------------------------------------------------------------------| -| RabbitMQ | current DLX wiring (point_value / point_command / command) | -| Kafka | explicit `dc3..dlq` topic written by the adapter on exhaustion | -| RocketMQ | built-in `%RETRY%group` / `%DLQ%group`, surfaced as logical dead-letter | -| Pulsar | DLQ policy + `maxRedeliverCount` | -| ActiveMQ | JMS redelivery policy + DLQ | -| MQTT | explicit `dc3/.dlq` topic written by the adapter on exhaustion | +| RabbitMQ | 现有 DLX 接线（point_value / point_command / command） | +| Kafka | 重试耗尽时由适配器写入显式 `dc3..dlq` topic | +| RocketMQ | 内建 `%RETRY%group` / `%DLQ%group`，以逻辑死信形式呈现 | +| Pulsar | DLQ 策略 + `maxRedeliverCount` | +| ActiveMQ | JMS 重投策略 + DLQ | +| MQTT | 重试耗尽时由适配器写入显式 `dc3/.dlq` topic | -The dedicated dead receivers (`CommandDeadReceiver`, `PointCommandDeadReceiver`) re-declare their subscriptions against -the logical dead-letter topics. The point-value dead queue has **no consumer by design** since `956de3dd3` — it is a -quarantine for poison batches and 7-day-unconsumed values; the port models it as a dead-letter topic without a -subscription. Its retention is an open question (§13.6). The 7-day TTL on the **live** point-value queue becomes a topic -retention policy attribute — see §8.6. +专用死信接收器（`CommandDeadReceiver`、`PointCommandDeadReceiver`）针对逻辑死信主题重新声明订阅。点位值死信队列自 +`956de3dd3` 起 **设计上就没有消费者**——它是毒批次与 7 天未消费值的隔离区；Port 把它建模为没有订阅的死信主题。 +其保留策略是开放问题（§13.6）。 **活** 点位值队列上的 7 天 TTL 变为主题 +保留策略属性——见 §8.6。 -### 8.3 Acknowledgement, retry and back-pressure +### 8.3 确认、重试与背压 -| Operation | RabbitMQ | Kafka | RocketMQ | Pulsar | ActiveMQ | MQTT 5 | +| 操作 | RabbitMQ | Kafka | RocketMQ | Pulsar | ActiveMQ | MQTT 5 | |--------------------------------------|------------------------------------------------------------|-----------------------------------|-----------------------------|---------------------------|-----------------------|-----------------------------------------------| -| `ack()` | basicAck (single or batch-multiple) | offset commit (batched) | CONSUME_SUCCESS | ack | acknowledge | PUBACK (QoS 1) | -| `reject(true)` | basicNack requeue | seek back, no commit | RECONSUME_LATER | negative ack / redelivery | rollback | adapter-level retry loop (broker has no nack) | -| `reject(false)` | basicReject → DLX | write to `.dlq` topic | built-in retry→DLQ | DLQ policy | redelivery-policy DLQ | publish to `.dlq` topic | -| bounded retry (batch retry settings) | stateless retry advice + recoverer (today's batch factory) | in-memory retry loop, then `.dlq` | `%RETRY%group` (native fit) | redelivery backoff + DLQ | redelivery policy | adapter in-memory loop | -| back-pressure | prefetch ≥ batchSize, broker as durable buffer | consumer `pause()`/`resume()` | suspend current queue | flow control / queue size | session recover | receive maximum (MQTT 5 flow control) | +| `ack()` | basicAck（单条或批量 multiple） | offset 提交（批量） | CONSUME_SUCCESS | ack | acknowledge | PUBACK（QoS 1） | +| `reject(true)` | basicNack 重新入队 | seek 回退，不提交 | RECONSUME_LATER | negative ack / 重投 | rollback | 适配器级重试循环（中间件没有 nack） | +| `reject(false)` | basicReject → DLX | 写入 `.dlq` topic | 内建 retry→DLQ | DLQ 策略 | 重投策略 DLQ | 发布到 `.dlq` topic | +| 有界重试（批量重试配置） | 无状态重试 advice + recoverer（今天的批量工厂） | 内存重试循环，然后 `.dlq` | `%RETRY%group`（原生契合） | 重投退避 + DLQ | 重投策略 | 适配器内存循环 | +| 背压 | prefetch ≥ batchSize，中间件充当持久缓冲 | 消费者 `pause()`/`resume()` | 挂起当前队列 | 流控 / 队列大小 | 会话恢复 | receive maximum（MQTT 5 流控） | -The bounded-retry row matters: since `956de3dd3` the point-value factory wraps delivery in a stateless retry advice with -exponential backoff, and exhaustion rejects the whole batch to the DLX. The shared batch-consumer retry settings -(`BatchConsumerProperties`, bound from `dc3.data.point.batch.*`) drive that bound on every adapter instead of -per-adapter improvisation — Kafka needs an in-memory loop, RocketMQ gets it natively from `%RETRY%group`. +有界重试这一行很关键：自 `956de3dd3` 起，点位值工厂用带指数退避的无状态重试 advice 包装投递， +耗尽后把整个批次拒绝进 DLX。共享的批量消费者重试配置 +（`BatchConsumerProperties`，从 `dc3.data.point.batch.*` 绑定）在所有适配器上驱动这一上界，而不是 +各适配器各自即兴发挥——Kafka 需要内存循环，RocketMQ 则从 `%RETRY%group` 原生获得。 -The old buffer-full `nack(requeue=true)` pattern is gone from the codebase (the data-side ingest buffer was removed; -`NotifyWorker` deliberately acks failures as FAILED instead of requeueing). Back-pressure is now "broker as durable -buffer + bounded prefetch", which maps cleanly to every target broker. +旧的缓冲满 `nack(requeue=true)` 模式已从代码库中移除（数据侧摄入缓冲已删除； +`NotifyWorker` 刻意把失败按 FAILED 确认而不是重新入队）。背压现在是“中间件充当持久 +缓冲 + 有界 prefetch”，能干净地映射到每个目标中间件。 -Contract documented for consumers: **at-least-once**, redelivery possible, idempotency required. `redeliveryCount` on -`MqReceived` is best-effort (exact on Kafka via retry topic count, RocketMQ reconsume times, Pulsar redelivery count; -approximate elsewhere). +面向消费者的契约文档：**至少一次**，可能重投，要求幂等。`MqReceived` 上的 +`redeliveryCount` 是尽力而为（Kafka 经重试 topic 计数、RocketMQ reconsume 次数、Pulsar 重投计数为精确值； +其余为近似值）。 -### 8.4 Publisher confirmation and the driver outbox +### 8.4 发布者确认与驱动发件箱(outbox) -Since `956de3dd3` the driver already owns durability: every point value is persisted to a **mandatory SQLite outbox** -(WAL, `synchronous=FULL`, startup validation, missing configuration fails fast) *before* `convertAndSend`, and +自 `956de3dd3` 起驱动已掌控持久性：每个点位值都在 `convertAndSend` *之前* 持久化到 **强制性 SQLite 发件箱(outbox)** +（WAL、`synchronous=FULL`、启动校验、缺配置即快速失败），并且 `BufferServiceImpl` + -`PointValueBuffer` run a claim-before-publish retry scheduler over pending rows. +`PointValueBuffer` 对待处理行运行先认领后发布的重试调度器。 -After migration, the outbox itself stays driver-side and broker-neutral — only the final -`convertAndSend` + `CorrelationData`-future plumbing swaps for `sendAsync` + -`SendConfirmation`. Broker mapping: RabbitMQ publisher confirms; Kafka `acks=all` future; RocketMQ send result; Pulsar -send future; MQTT QoS 1 PUBACK (per-message confirm — the strongest of the non-RabbitMQ set); **JMS has no -confirmation** → best-effort (fire-and-forget with failure-driven republish from the outbox), marked in the capability -matrix. +迁移后，发件箱(outbox) 本身仍留在驱动侧且中间件中立——只有最后的 +`convertAndSend` + `CorrelationData`-future 管线换成 `sendAsync` + +`SendConfirmation`。中间件映射：RabbitMQ 发布者确认；Kafka `acks=all` future；RocketMQ 发送结果；Pulsar +发送 future；MQTT QoS 1 PUBACK（逐消息确认——非 RabbitMQ 阵营中最强的）；**JMS 没有确认** +→ 尽力而为（发后即忘，失败时从发件箱(outbox) 重发布），在能力矩阵中标注。 -Consequence worth stating explicitly: **publisher confirm is no longer load-bearing for durability**. Because the outbox -owns persistence, `publisherConfirm=false` brokers are fully acceptable — confirm becomes a latency/republish-frequency -optimization, not a correctness requirement. The ActiveMQ adapter is therefore a first-class citizen, not a degraded -mode. +值得明确指出的结论：**发布者确认不再是持久性的承重机制**。因为发件箱(outbox) +负责持久化，`publisherConfirm=false` 的中间件完全可接受——确认变成延迟/重发布频率的 +优化，而非正确性要求。因此 ActiveMQ 适配器是一等公民，不是降级 +模式。 -### 8.5 Envelope, tracing, tenant context +### 8.5 信封、链路追踪与租户上下文 -- `dc3-type` header carries the payload class name; the API layer deserializes — replaces Spring AMQP's `__TypeId__`. -- Payload-level schema versioning is already in production on the point-value path (schema-v1: `schemaVersion`, - `messageId`, `sequence`, `fencingToken`, `driverNode`, validated in `PointValueReceiver`); the port adopts the same - convention — schema fields live in the payload, transport hints in headers. -- `X-Request-Id` propagation (currently `MdcRequestIdMessagePostProcessor` / - `MdcRequestIdListenerAdvice`) moves into `dc3-mq-core` core; all brokers support string headers. -- Tenant id remains inside the payload (and mirrored as a header for operational filtering); tenant scoping rules are - unaffected by the broker choice. +- `dc3-type` header 携带载荷类名；由 API 层反序列化——取代 Spring AMQP 的 `__TypeId__`。 +- 载荷级 schema 版本化已在点位值路径上生产使用（schema-v1：`schemaVersion`、 + `messageId`、`sequence`、`fencingToken`、`driverNode`，在 `PointValueReceiver` 中校验）；Port 采用同一 + 约定——schema 字段放在载荷中，传输提示放在 header 中。 +- `X-Request-Id` 传播（当前为 `MdcRequestIdMessagePostProcessor` / + `MdcRequestIdListenerAdvice`）移入 `dc3-mq-core` 核心；所有中间件都支持字符串 header。 +- 租户 id 留在载荷内（并镜像为一个 header 便于运维过滤）；租户作用域规则不受中间件选择影响。 -### 8.6 Queue-level message TTL as retention policy +### 8.6 队列级消息 TTL 作为保留策略 -A RabbitMQ-specific usage the port must model: queues carry per-queue message TTL as a retention/expiry guard — -state/alarm 30 s (stale lifecycle events are worthless), event/command_result 60 s, notify_task 24 h (runaway-backlog -guard when outbound channels are stuck), point_value 7 d (then dead-lettered, §8.2), driver-side command queues 30 s. +Port 必须建模的一种 RabbitMQ 特有用法：队列携带每队列消息 TTL 作为保留/过期守卫—— +state/alarm 30 秒（过期的生命周期事件毫无价值）、event/command_result 60 秒、notify_task 24 小时（出站通道卡死时 +的失控积压守卫）、point_value 7 天（随后死信，§8.2）、驱动侧命令队列 30 秒。 -This becomes a `retention` attribute on the topic registry, with per-broker semantics documented rather than identical: -RabbitMQ TTL drops per message from the head of the queue; Kafka/RocketMQ/Pulsar retention is time-based log retention -(drops whole old segments — effectively the same "don't keep forever" guarantee); JMS TTL and MQTT 5 message expiry are -per message. The distinction only matters for exact expiry timing, which no consumer in the codebase relies on (expired -state events are simply dropped, and the DB remains the source of truth). +这变成主题注册表上的一个 `retention` 属性，各中间件语义文档化而非完全一致： +RabbitMQ TTL 从队头逐消息丢弃；Kafka/RocketMQ/Pulsar 的保留是基于时间的日志保留 +（丢弃整个旧段——实际上等同于“不永久保存”的保证）；JMS TTL 与 MQTT 5 消息过期是 +逐消息的。区别只在精确过期时点上才有影响，代码库中没有任何消费者依赖它（过期的 +状态事件直接丢弃，数据库仍是事实来源）。 -### 8.7 Batch consumption and transactional ack +### 8.7 批量消费与事务性确认 -The point-value path — the platform's highest-volume stream — runs on consumer-side batching since `956de3dd3`: +点位值路径——平台吞吐量最大的流——自 `956de3dd3` 起运行在消费者侧批量之上： -- `PointValueRabbitConfig` declares a third container factory: batch listener + consumer batch, `prefetch ≥ batchSize`, - bounded retry with backoff, exhaustion → whole batch to the point-value DLX. -- `PointValueReceiver` receives `List`, validates the schema-v1 envelope per message, persists history + latest - projections in **one PostgreSQL transaction**, and only then `basicAck(lastTag, multiple=true)` — ack and commit are - atomic from the consumer's perspective. +- `PointValueRabbitConfig` 声明第三个容器工厂：批量监听器 + 消费者批量、`prefetch ≥ batchSize`、 + 带退避的有界重试、耗尽 → 整批进入点位值 DLX。 +- `PointValueReceiver` 接收 `List`，逐消息校验 schema-v1 信封，在 **单个 PostgreSQL 事务** 中持久化历史 + + 最新值投影，然后才 `basicAck(lastTag, multiple=true)`——从消费者视角看，确认与提交是 + 原子的。 -| Broker | Batch mechanism | +| 中间件 | 批量机制 | |----------|------------------------------------------------------------------------------------------------------------------------------------| -| RabbitMQ | consumer batch (`setBatchListener` + `setConsumerBatchEnabled`), current design | -| Kafka | poll loop with `max.poll.records`; offsets committed after the handler returns | -| RocketMQ | batch consumption (`consumeMessageBatchSize`) | -| Pulsar | batch receive API | -| ActiveMQ | no native consumer batch → adapter drains available messages within a short window (synthesized, capability `batchDelivery=false`) | -| MQTT | no native consumer batch → same adapter-side windowing (synthesized, `batchDelivery=false`) | +| RabbitMQ | 消费者批量（`setBatchListener` + `setConsumerBatchEnabled`），现有设计 | +| Kafka | 带 `max.poll.records` 的 poll 循环；处理器返回后提交 offset | +| RocketMQ | 批量消费（`consumeMessageBatchSize`） | +| Pulsar | 批量接收 API | +| ActiveMQ | 无原生消费者批量 → 适配器在短窗口内排空可用消息（合成，能力 `batchDelivery=false`） | +| MQTT | 无原生消费者批量 → 同样的适配器侧窗口（合成，`batchDelivery=false`） | -Port model: `DeliveryMode.BATCH` + `MqBatchListener` + the shared batch retry settings. The TCK verifies that `ack()` -after batch processing commits every message in the batch and that retry exhaustion dead-letters rather than drops -(§11). +Port 模型：`DeliveryMode.BATCH` + `MqBatchListener` + 共享批量重试配置。TCK 验证批量处理后的 `ack()` +会提交批内每条消息，且重试耗尽走死信而非丢弃 +（§11）。 -Known trade-off (documented, not hidden): one poison message retries and dead-letters the **whole batch** today. -Batch-level granularity is the Spring default and keeps the cross-broker story simple; per-message extraction is open -question §13.7. +已知取舍（文档化，而非隐藏）：今天一条毒消息会导致 **整个批次** 重试并死信。 +批次级粒度是 Spring 默认行为，也让跨中间件的叙事保持简单；按消息提取是开放 +问题 §13.7。 -### 8.8 Per-instance queue lifecycle (lease-coupled expiry) +### 8.8 每实例队列生命周期（租约耦合的过期） -Driver-side subscriptions are per-instance and must not outlive their driver: +驱动侧订阅是每实例的，且不得比其驱动存活更久： -- command/point-command queues carry `x-expires = driver.lease.queue-expires` — a dead driver's queue disappears with - its lease instead of silently accumulating commands that would be delivered to a stale instance on restart; -- the metadata broadcast queue is auto-delete with 30 s message TTL, bounding staleness from dead instances. +- command/point-command 队列携带 `x-expires = driver.lease.queue-expires`——失效驱动的队列随其租约消失， + 而不是默默堆积命令、在重启后投递给一个过期实例； +- 元数据广播队列是 auto-delete，消息 TTL 30 秒，限制失效实例造成的陈旧。 -Port model: `instanceTtl` on `SubscriptionSpec` + capability `subscriptionExpiry`. RabbitMQ maps to `x-expires` -/auto-delete natively; RocketMQ/Pulsar have subscription-level equivalents; Kafka offsets simply persist (stale groups -are cosmetic — documented cleanup policy via admin tooling); JMS has no direct equivalent (documented cleanup policy); -MQTT 5 maps to the session expiry interval of the persistent session. Neither fallback affects correctness, because -command senders already validate driver lease/ownership before dispatch. +Port 模型：`SubscriptionSpec` 上的 `instanceTtl` + 能力 `subscriptionExpiry`。RabbitMQ 原生映射到 `x-expires` +/auto-delete；RocketMQ/Pulsar 有订阅级等价物；Kafka offset 直接持久存在（过期组只是观感问题——通过管理工具 +文档化清理策略）；JMS 没有直接等价物（文档化清理策略）； +MQTT 5 映射到持久会话的会话过期间隔。两种回退(fallback) 都不影响正确性，因为 +命令发送方在分发前已经校验驱动租约/所有权。 -## 9. Capability matrix (published, per adapter) +## 9. 能力矩阵（按适配器发布） -Implementation status (revised 2026-08-25): five adapters — rabbitmq, kafka, activemq (Artemis), mqtt 5 and pulsar — are -implemented and certified by the TCK against live brokers. The **rocketmq adapter is experimental and not yet -certified**: its contract suite is opt-in (`TCK_ROCKETMQ_NAMESRV`) and self-describes as not-yet-certified, so it ships -for evaluation, not as a production selection. Certified columns reflect the implemented behavior (e.g. rabbit delays -arbitrary messages through the port fallback, rocketmq delay levels would quantize; the rocketmq classic client replays -topic backlog for brand-new consumer groups regardless of consumeFromWhere, so that adapter seeds fresh groups to the -latest offset and warms up not-yet-created topics on subscribe — the warm-up probe carries a marker property no business -listener ever sees; pulsar subscriptions start at the latest position natively, which matches the fresh-queue semantics -without seeding). +实现状态（2026-08-25 修订）：五个适配器——rabbitmq、kafka、activemq（Artemis）、mqtt 5 与 pulsar—— +已实现并通过 TCK 对照真实中间件完成认证。**rocketmq 适配器为实验性且尚未认证**：其契约套件为可选启用 +（`TCK_ROCKETMQ_NAMESRV`），且自述为尚未认证，因此它面向评估发布，不是生产选择。“已认证”列反映的是已实现的行为 +（例如 rabbit 经 Port 回退(fallback) 延迟任意消息，rocketmq 延迟级别会量化；rocketmq classic 客户端对全新消费者组 +会无视 consumeFromWhere 重放 topic 积压，因此该适配器把新组播种到最新 offset，并在订阅时预热尚未创建的 +topic——预热探针携带业务监听器永远看不到的标记属性；pulsar 订阅原生从最新位置开始，无需播种即符合新队列语义）。 -| Capability | RabbitMQ ✅ | Kafka ✅ | ActiveMQ ✅ | MQTT 5 ✅ | RocketMQ ⚠️ experimental | Pulsar ✅ | +| 能力 | RabbitMQ ✅ | Kafka ✅ | ActiveMQ ✅ | MQTT 5 ✅ | RocketMQ ⚠️ 实验性 | Pulsar ✅ | |--------------------------|-----------------------|-----------------------|-----------------------------------------|-----------------------------|------------------------------------|-----------------------| -| Delayed message | fallback* | ❌ → local fallback | ✅ JMS scheduled | ❌ → local fallback | fallback (levels quantize) | ✅ native | -| Native DLQ | DLX + quarantine | adapter `.dlq` topic | adapter `.dlq` queue | adapter `/dlq` topic | adapter `-dlq` topic | adapter `.dlq` topic | -| Broadcast | ✅ per-instance queue | ✅ (instance groups) | ✅ topic consumer | ✅ plain filter | ✅ BROADCASTING | ✅ | -| Per-message ack | ✅ | offset (approx) | ✅ client ack | ✅ QoS 1 | ✅ | ✅ | -| Publisher confirm | ✅ confirms | ✅ (acks=all) | ❌ best-effort (outbox covers it, §8.4) | ✅ (PUBACK) | ✅ sync send | ✅ | -| Batch delivery | ✅ native | ✅ native | ⚠️ synthesized | ⚠️ synthesized | ✅ consumer batch | ✅ batch receive | -| Per-key ordering | ❌ | ✅ | ❌ | ❌ | ❌ (keys carried for routing only) | ❌ (shared sub) | -| Key-pattern routing | ✅ broker bindings | ✅ client-side router | ✅ client-side router | ✅ client-side router | ✅ client-side router | ✅ client-side router | -| Subscription expiry | ✅ x-expires | ❌ documented | ❌ documented | ⚠️ session expiry | ❌ documented | ❌ documented | -| Group durability offline | ✅ durable queue | ✅ log retention | ✅ durable subscription | ⚠️ broker-dependent (§13.8) | ✅ offsets | ✅ | -| Retention | queue TTL | retention config | subscription retention | broker-dependent | retention | retention/TTL | +| 延迟消息 | 回退(fallback)* | ❌ → 本地回退(fallback) | ✅ JMS 定时 | ❌ → 本地回退(fallback) | 回退(fallback)（级别量化） | ✅ 原生 | +| 原生 DLQ | DLX + 隔离区 | 适配器 `.dlq` topic | 适配器 `.dlq` 队列 | 适配器 `/dlq` topic | 适配器 `-dlq` topic | 适配器 `.dlq` topic | +| 广播 | ✅ 每实例队列 | ✅（实例组） | ✅ topic 消费者 | ✅ 普通过滤 | ✅ BROADCASTING | ✅ | +| 逐消息确认 | ✅ | offset（近似） | ✅ 客户端确认 | ✅ QoS 1 | ✅ | ✅ | +| 发布者确认 | ✅ confirms | ✅（acks=all） | ❌ 尽力而为（发件箱(outbox) 兜底，§8.4） | ✅（PUBACK） | ✅ 同步发送 | ✅ | +| 批量投递 | ✅ 原生 | ✅ 原生 | ⚠️ 合成 | ⚠️ 合成 | ✅ 消费者批量 | ✅ 批量接收 | +| 按 key 顺序 | ❌ | ✅ | ❌ | ❌ | ❌（key 仅用于路由） | ❌（共享订阅） | +| Key 模式路由 | ✅ 中间件 binding | ✅ 客户端侧路由器 | ✅ 客户端侧路由器 | ✅ 客户端侧路由器 | ✅ 客户端侧路由器 | ✅ 客户端侧路由器 | +| 订阅过期 | ✅ x-expires | ❌ 文档化 | ❌ 文档化 | ⚠️ 会话过期 | ❌ 文档化 | ❌ 文档化 | +| 组离线持久性 | ✅ 持久队列 | ✅ 日志保留 | ✅ 持久订阅 | ⚠️ 取决于中间件（§13.8） | ✅ offset | ✅ | +| 保留 | 队列 TTL | 保留配置 | 订阅保留 | 取决于中间件 | 保留 | 保留/TTL | -\* rabbit intrinsic TTL+DLX delays (STATE_TIMEOUT / DEVICE_SCAN) work server-side as before; arbitrary per-message -delays use the port fallback (capability false). +\* rabbit 固有的 TTL+DLX 延迟（STATE_TIMEOUT / DEVICE_SCAN）照旧在服务端生效；任意逐消息 +延迟使用 Port 回退(fallback)（能力为 false）。 -This table is user-facing documentation ("which broker should I pick?") and the startup negotiation log summarizes it -per deployment. +这张表是面向用户的文档（“我该选哪个中间件？”），启动协商日志按部署对它做摘要。 -## 10. Alternatives considered +## 10. 已考虑的备选方案 -**Spring Cloud Stream** — gives official RabbitMQ/Kafka binders for free, but: the RocketMQ binder is -Alibaba-maintained, the Pulsar binder has thin community coverage, ActiveMQ has no modern binder; and its functional -programming model fights our per-driver dynamic subscriptions, TTL+DLX delays, manual-ack back-pressure, batch-ack and -confirm-outbox patterns. Rejected as the port layer — though an individual adapter may internally build on SCStream -later; the port is the contract we control. +**Spring Cloud Stream** —— 免费提供官方 RabbitMQ/Kafka binder，但是：RocketMQ binder 由 +Alibaba 维护，Pulsar binder 社区覆盖薄弱，ActiveMQ 没有现代 binder；且其函数式 +编程模型与我们按驱动的动态订阅、TTL+DLX 延迟、手动确认背压、批量确认与 +确认-发件箱(outbox) 模式相冲突。作为 Port 层被否决——不过单个适配器以后可以在内部基于 SCStream +构建；Port 是我们掌控的契约。 -**JMS 2.0 as the core API** — rejected. Lowest common denominator: no delayed delivery standard, weak DLQ semantics, no -publisher confirmation, broadcast/group semantics map poorly, no batch consumption. JMS remains the implementation -technology of the ActiveMQ adapter only. +**以 JMS 2.0 作为核心 API** —— 否决。最低公共分母：没有延迟投递标准、DLQ 语义弱、没有 +发布者确认、广播/组语义映射不佳、没有批量消费。JMS 仅保留为 ActiveMQ 适配器的实现 +技术。 -**Apache Camel as the port** — rejected. An integration framework as the contract layer inverts the dependency (the -platform's core messaging on a routing DSL), drags a large dependency graph into every service, and still leaves the -DC3-specific semantics (batch post-commit ack, lease-coupled subscription expiry) to be modeled somewhere. Adapters stay -plain client libraries. +**以 Apache Camel 作为 Port** —— 否决。把集成框架当作契约层颠倒了依赖（平台的核心消息建立在路由 DSL +之上），给每个服务拖入庞大的依赖图，而且 DC3 特有语义（批量提交后确认、租约耦合的订阅过期）仍需要在某处建模。 +适配器保持纯客户端库。 -**Do nothing / RabbitMQ-only** — rejected for a globally distributed project: Kafka and RocketMQ are the default choices -in many enterprise environments (especially CN ecosystem for RocketMQ), and "bring your own broker" is a recurring -community ask for on-prem integration. +**什么都不做 / 仅支持 RabbitMQ** —— 对一个全球分布的项目而言被否决：Kafka 与 RocketMQ 是许多企业环境的默认选择 +（RocketMQ 尤其在国内生态），“自带中间件”是社区在本地化集成中反复提出的诉求。 -## 11. TCK — the community extension mechanism +## 11. TCK —— 社区扩展机制 -`dc3-mq-tck` contains one broker-neutral contract suite executed against each adapter via Testcontainers (rabbitmq, -kafka, pulsar, artemis, an MQTT 5 broker — EMQX or NanoMQ; the rocketmq suite is opt-in via -`TCK_ROCKETMQ_NAMESRV` and not yet certified): +`dc3-mq-tck` 包含一套中间件中立的契约套件，通过 Testcontainers 对每个适配器执行（rabbitmq、 +kafka、pulsar、artemis、一个 MQTT 5 中间件——EMQX 或 NanoMQ；rocketmq 套件通过 +`TCK_ROCKETMQ_NAMESRV` 可选启用，尚未认证）： -1. send → receive (round-trip, envelope fidelity, headers, `dc3-type` deserialization) -2. LOAD_BALANCE: exactly one consumer receives each message across 2 instances -3. BROADCAST: both instances receive each message -4. delay: message not delivered before the deadline, delivered after -5. `reject(false)` → dead-letter topic receives the message -6. `reject(true)` → redelivery observed (at-least-once) -7. `sendAsync` confirmation fires with the correct outcome -8. requestId header survives the hop (MDC restored) -9. back-pressure: rejected/full-path message is not lost -10. BATCH delivery: batch callback receives ≥ 1 messages; `ack()` commits the whole batch (no redelivery after restart) -11. Bounded retry: observed attempts ≤ the configured bound, then dead-letter — never silent drop -12. instanceTtl: where `subscriptionExpiry=true`, an idle per-instance subscription is removed after the TTL - (timing-tolerant assertion) -13. LOAD_BALANCE with no live instance: messages published while the entire group is down are retained and delivered - when an instance starts (queue-level durability; MQTT shared-subscription brokers vary here — §13.8) +1. send → receive（往返、信封保真、header、`dc3-type` 反序列化） +2. LOAD_BALANCE：跨 2 个实例时每条消息恰好被一个消费者接收 +3. BROADCAST：两个实例都接收每条消息 +4. delay：截止时间之前不投递，之后投递 +5. `reject(false)` → 死信主题收到消息 +6. `reject(true)` → 观察到重投（至少一次） +7. `sendAsync` 确认以正确结果触发 +8. requestId header 在跳转后仍存在（MDC 恢复） +9. 背压：被拒绝/经完整路径的消息不丢失 +10. BATCH 投递：批量回调收到 ≥ 1 条消息；`ack()` 提交整个批次（重启后无重投） +11. 有界重试：观察到的尝试次数 ≤ 配置上界，然后死信——绝不静默丢弃 +12. instanceTtl：在 `subscriptionExpiry=true` 的地方，空闲的每实例订阅在 TTL 之后被移除 + （对时序宽容的断言） +13. LOAD_BALANCE 且无存活实例：全组宕机期间发布的消息被保留，并在实例启动时投递 + （队列级持久性；MQTT 共享订阅中间件在此表现不一——§13.8） -**An adapter passes the TCK ⇒ it is compliant.** This is the acceptance bar for community adapters (EMQX-bridged -transports, Redis Streams, SQS…). The existing -`RabbitDeliveryIT` / `RabbitTestHarness` assertions migrate into this suite; the E2E suite keeps running unchanged -against the rabbitmq adapter as the migration gate. +**适配器通过 TCK 即视为合规。** 这是社区适配器（EMQX 桥接 +传输、Redis Streams、SQS…）的验收门槛。现有 +`RabbitDeliveryIT` / `RabbitTestHarness` 断言迁入本套件；E2E 套件在迁移门槛期间继续原样运行于 +rabbitmq 适配器之上。 -## 12. Migration plan +## 12. 迁移计划 -Wire compatibility is the invariant: no existing RabbitMQ deployment should notice the refactor. +线级兼容是不变量：任何现有 RabbitMQ 部署都不应察觉这次重构。 -- **Phase 1 — extract the port, RabbitMQ adapter moves unchanged.** - Create `dc3-mq-core` + `dc3-mq-rabbitmq`. Move `RabbitConfig`, - `ExchangeConfig`, `DataTopicConfig`, `DriverTopicConfig`, `PointValueRabbitConfig`, MDC propagation, ack helpers into - the adapter with **identical physical names**. Convert the 14 raw producer call sites (8 driver send methods, outbox - republish, 6 center-side sends) and 16 listeners to the new API (`@Dc3Listener`), including the batch point-value - receiver. Business-module poms swap `dc3-common-rabbitmq` → `dc3-mq-core` + - `dc3-mq-rabbitmq`. *Gate: existing E2E (`RabbitDeliveryIT` etc.) runs unmodified and green.* -- **Phase 2 — TCK + Kafka adapter.** Highest global demand. Partition-key ordering per driver is a documented upgrade - over the RabbitMQ baseline. -- **Phase 3 — RocketMQ (CN ecosystem demand), Pulsar, ActiveMQ/Artemis, MQTT 5.** The MQTT adapter lets a deployment run - both planes (device access + internal async) on one broker — or drop RabbitMQ entirely where an MQTT broker is already - operated. Publish the capability matrix, `dc3.mq.type` compose profiles per broker. -- **Throughout** — `DriverSenderService` interface is untouched; third-party driver JARs compiled against it keep - working. +- **Phase 1 —— 抽取 Port，RabbitMQ 适配器原样搬迁。** + 创建 `dc3-mq-core` + `dc3-mq-rabbitmq`。把 `RabbitConfig`、 + `ExchangeConfig`、`DataTopicConfig`、`DriverTopicConfig`、`PointValueRabbitConfig`、MDC 传播、确认辅助类以 + **完全相同的物理名称** 移入适配器。把 14 处裸生产者调用点（8 个驱动发送方法、发件箱(outbox) + 重发布、6 处中心侧发送）与 16 个监听器转换到新 API（`@Dc3Listener`），包括批量点位值 + 接收器。业务模块 pom 把 `dc3-common-rabbitmq` 换成 `dc3-mq-core` + + `dc3-mq-rabbitmq`。*门槛：现有 E2E（`RabbitDeliveryIT` 等）不改一行、全绿通过。* +- **Phase 2 —— TCK + Kafka 适配器。** 全球需求最高。按驱动的分区键顺序是文档化的、超越 + RabbitMQ 基线的升级。 +- **Phase 3 —— RocketMQ（国内生态需求）、Pulsar、ActiveMQ/Artemis、MQTT 5。** MQTT 适配器让部署可以在一个中间件上 + 运行两个平面（南向设备面 + 内部异步）——或在已运营 MQTT 中间件的地方彻底去掉 + RabbitMQ。发布能力矩阵、按中间件的 `dc3.mq.type` compose profile。 +- **贯穿始终** —— `DriverSenderService` 接口不动；针对它编译的第三方驱动 JAR 持续可用。 -Suggested cleanups riding along Phase 1: +随 Phase 1 一并进行的建议清理： -- `RabbitConstant` moves out of `dc3-common-constant`'s `constant/driver` package: logical names become `MqTopic` in the - port; physical names become adapter-private (`RabbitNames`). -- Delete dead topology: the `register` exchange/queue/routing constants (declared and referenced nowhere) and the unused - `dc3.e.mqtt` exchange + `QUEUE_MQTT` (no binding, no producer, no consumer). Re-introduce only when a real consumer - exists. -- Packages `data/rabbit`, `receiver/rabbit` → `data/mq/listener`, `driver/mq/listener`. -- `dc3.rabbit.tag` → `dc3.mq.namespace` (rabbitmq adapter maps it to the legacy property for compatibility). +- `RabbitConstant` 移出 `dc3-common-constant` 的 `constant/driver` 包：逻辑名称变成 Port 中的 `MqTopic`； + 物理名称变成适配器私有（`RabbitNames`）。 +- 删除死拓扑：`register` exchange/queue/路由常量（声明了却无处引用）与未使用的 + `dc3.e.mqtt` exchange + `QUEUE_MQTT`（无 binding、无生产者、无消费者）。只在真实消费者 + 存在时再引入。 +- 包 `data/rabbit`、`receiver/rabbit` → `data/mq/listener`、`driver/mq/listener`。 +- `dc3.rabbit.tag` → `dc3.mq.namespace`（rabbitmq 适配器为兼容把它映射到旧属性）。 -## 13. Open questions +## 13. 开放问题 -1. **Pulsar tenancy** — map `dc3.mq.namespace` to Pulsar tenant/namespace natively, or flatten to topic prefix? (Native - is cleaner but requires tenant provisioning docs.) -2. **Envelope evolution** — align `dc3-type`/headers with CloudEvents 1.0 attribute names for third-party - interoperability? Cheap now, breaking later; leaning yes for `type` / - `traceparent`. The schema-v1 payload envelope strengthens the case: transport headers and payload schema should - evolve on one roadmap. -3. **Kafka back-pressure semantics** — `pause()` needs a resume signal the current - `Acknowledgment` API doesn't express; may need `Acknowledgment.defer()` or a listener-container-level hook. Decide - during Phase 2 TCK work. -4. **BROADCAST queue TTLs** — the 30 s auto-delete TTL on metadata queues bounds staleness on dead instances; - non-RabbitMQ brokers express this via subscription expiry or heartbeat — confirm per-adapter strategy in the TCK. -5. **Single-process mode (`dc3-center-single`)** — confirm whether it should get an in-process/no-broker adapter - (`dc3-mq-local`) for the smallest deployments, reusing the facade `local` precedent. -6. **Point-value dead-queue retention** — the quarantine queue has no consumer and no TTL; a stuck deployment grows it - without bound. Decide: queue TTL, size-capped alerting, or a minimal auditing consumer. -7. **Batch poison granularity** — today one poison message dead-letters the whole batch. Per-message extraction - (dead-letter only the offender, ack the rest) is friendlier but cross-broker messier (Kafka needs a reprocess loop). - Decide what the TCK mandates; batch-granular is the safe default. -8. **MQTT shared-subscription group semantics** — MQTT 5 standardizes `$share` delivery to one *online* member but is - silent on retention while no member is online (broker-specific). TCK case 13 decides compliance per broker; the - capability matrix must call the behavior out for deployers weighing the MQTT adapter. +1. **Pulsar 租户模型** —— 把 `dc3.mq.namespace` 原生映射到 Pulsar tenant/namespace，还是压平为 topic 前缀？（原生 + 更干净，但需要租户开通文档。） +2. **信封演进** —— 为了第三方互操作，把 `dc3-type`/header 对齐到 CloudEvents 1.0 属性名？现在改便宜、以后改是 + 破坏性的；对 `type` / + `traceparent` 倾向采纳。schema-v1 载荷信封强化了这一理由：传输 header 与载荷 schema 应 + 在同一路线图上演进。 +3. **Kafka 背压语义** —— `pause()` 需要一个当前 + `Acknowledgment` API 无法表达的恢复信号；可能需要 `Acknowledgment.defer()` 或监听器容器级钩子。在 + Phase 2 的 TCK 工作中决定。 +4. **BROADCAST 队列 TTL** —— 元数据队列 30 秒 auto-delete TTL 限制了失效实例造成的陈旧； + 非 RabbitMQ 中间件通过订阅过期或心跳表达——在 TCK 中确认各适配器策略。 +5. **单进程模式（`dc3-center-single`）** —— 确认是否应为最小部署提供一个进程内/无中间件适配器 + （`dc3-mq-local`），复用 facade `local` 的先例。 +6. **点位值死信队列保留** —— 隔离队列既无消费者也无 TTL；卡住的部署会让它 + 无界增长。需要决定：队列 TTL、按容量上限告警，或一个最小审计消费者。 +7. **批量毒消息粒度** —— 今天一条毒消息会把整个批次死信。按消息提取 + （只死信问题消息、确认其余）更友好，但跨中间件更麻烦（Kafka 需要重处理循环）。 + 决定 TCK 强制什么；批次粒度是安全默认值。 +8. **MQTT 共享订阅组语义** —— MQTT 5 把 `$share` 的投递标准化为投给一个 *在线* 成员，但对没有成员在线时的保留 + 保持沉默（因中间件而异）。TCK 用例 13 逐中间件判定合规； + 能力矩阵必须向权衡 MQTT 适配器的部署者明确指出该行为。 -## 14. Appendix — current call-site inventory (Phase 1 checklist) +## 14. 附录 —— 当前调用点清单（Phase 1 检查清单） -Producers (14 raw call sites, main code): +生产者（14 处裸调用点，主代码）： -| Module | Class | Sends to | +| 模块 | 类 | 发往 | |---------|----------------------------------------------------|---------------------------------------------------------------------------| -| driver | `DriverSenderServiceImpl` (8 methods, 6 raw sites) | STATE, ALARM ×2, POINT_VALUE, POINT_COMMAND_RESULT, COMMAND_RESULT, EVENT | -| driver | `BufferServiceImpl` | POINT_VALUE (outbox republish + retry scheduler) | -| data | `DriverStateServiceImpl` | STATE_TIMEOUT_DELAY (45 s) | -| data | `EntityStateExpiryScanner` | STATE_TIMEOUT_DELAY (scan tick), DEVICE_SCAN | -| data | `CommandHistoryServiceImpl` | COMMAND | -| data | `PointCommandServiceImpl` | POINT_COMMAND | -| data | `NotifyTaskSender` | NOTIFY_TASK (via alarm exchange) | -| manager | `MetadataEventListener` | METADATA | +| driver | `DriverSenderServiceImpl`（8 个方法，6 处裸调用点） | STATE、ALARM ×2、POINT_VALUE、POINT_COMMAND_RESULT、COMMAND_RESULT、EVENT | +| driver | `BufferServiceImpl` | POINT_VALUE（发件箱(outbox) 重发布 + 重试调度器） | +| data | `DriverStateServiceImpl` | STATE_TIMEOUT_DELAY（45 秒） | +| data | `EntityStateExpiryScanner` | STATE_TIMEOUT_DELAY（扫描 tick）、DEVICE_SCAN | +| data | `CommandHistoryServiceImpl` | COMMAND | +| data | `PointCommandServiceImpl` | POINT_COMMAND | +| data | `NotifyTaskSender` | NOTIFY_TASK（经 alarm exchange） | +| manager | `MetadataEventListener` | METADATA | -Consumers (16): +消费者（16 个）： -| Module | Listener | Topic / role | Mode | +| 模块 | 监听器 | 主题 / 角色 | 模式 | |--------|-------------------------------------------------------|-------------------------|--------------------------------------------------------| -| data | `DriverStateReceiver`, `DeviceStateReceiver` | STATE | LOAD_BALANCE | -| data | `DriverAlarmReceiver`, `DeviceAlarmReceiver` | ALARM | LOAD_BALANCE | -| data | `PointValueReceiver` (BATCH, post-commit ack) | POINT_VALUE | LOAD_BALANCE | -| data | `PointCommandResultReceiver`, `CommandResultReceiver` | result topics | LOAD_BALANCE | -| data | `PointCommandDeadReceiver`, `CommandDeadReceiver` | dead letters | LOAD_BALANCE | +| data | `DriverStateReceiver`、`DeviceStateReceiver` | STATE | LOAD_BALANCE | +| data | `DriverAlarmReceiver`、`DeviceAlarmReceiver` | ALARM | LOAD_BALANCE | +| data | `PointValueReceiver`（BATCH，提交后确认） | POINT_VALUE | LOAD_BALANCE | +| data | `PointCommandResultReceiver`、`CommandResultReceiver` | 结果主题 | LOAD_BALANCE | +| data | `PointCommandDeadReceiver`、`CommandDeadReceiver` | 死信 | LOAD_BALANCE | | data | `EventReportReceiver` | EVENT | LOAD_BALANCE | -| data | `DriverTimeoutCheckReceiver` | delay check | LOAD_BALANCE | +| data | `DriverTimeoutCheckReceiver` | 延迟检查 | LOAD_BALANCE | | data | `NotifyWorker` | NOTIFY_TASK | LOAD_BALANCE | -| data | `EntityStateExpiryScanner` (listener) | device scan | LOAD_BALANCE | +| data | `EntityStateExpiryScanner`（监听器） | 设备扫描 | LOAD_BALANCE | | driver | `MetadataReceiver` | METADATA | **BROADCAST** | -| driver | `CommandReceiver`, `PointCommandReceiver` | COMMAND / POINT_COMMAND | LOAD_BALANCE (per-service queue, lease-coupled expiry) | +| driver | `CommandReceiver`、`PointCommandReceiver` | COMMAND / POINT_COMMAND | LOAD_BALANCE（每服务队列，租约耦合过期） | -The point-value dead queue deliberately has no listener (quarantine, §8.2). +点位值死信队列刻意没有监听器（隔离区，§8.2）。 diff --git a/docs/design/relational-r2dbc.md b/docs/design/relational-r2dbc.md index 6e8558547..52aea0519 100644 --- a/docs/design/relational-r2dbc.md +++ b/docs/design/relational-r2dbc.md @@ -1,82 +1,77 @@ -# Design: Relational Access Layer on Spring Data R2DBC (per-dialect, per-service modules) +# 设计：基于 Spring Data R2DBC 的关系访问层（每方言、每服务模块） -| | | -|----------------|------------------------------------------------------------------------------------------------------------------------------| -| **Status** | Approved 2026-08-28 — §5 decisions confirmed (D14 deferred to the P1 TCK spike; D18/D19 added at first review) | -| **Date** | 2026-08-28 | -| **Scope** | relational persistence layer: `dc3-common-auth` + `dc3-db` family first; `dc3-common-manager` / `dc3-common-data` follow | -| **Target** | replace MyBatis-Plus + JDBC with Spring Data R2DBC; end-to-end reactive; GraalVM native-image ready | -| **Supersedes** | `storage-abstraction.md` §3 (relational dialect mechanism — "MyBatis is the port"); its TSDB half already moved to `tsdb-abstraction.md` | -| **Related** | `mq-abstraction.md`, `tsdb-abstraction.md` — the family pattern this design extends to the relational layer | +| | | +|------------|------------------------------------------------------------------------------------------------------------------------------| +| **状态** | 已批准 2026-08-28 —— §5 决策已确认（D14 推迟至 P1 TCK spike；D18/D19 在首次评审时增补） | +| **日期** | 2026-08-28 | +| **范围** | 关系持久层：`dc3-common-auth` + `dc3-db` 家族先行；`dc3-common-manager` / `dc3-common-data` 随后跟进 | +| **目标** | 用 Spring Data R2DBC 取代 MyBatis-Plus + JDBC；端到端响应式；GraalVM native image 就绪 | +| **取代** | `storage-abstraction.md` §3（关系方言机制——“MyBatis 即 Port”）；其 TSDB 一半已移入 `tsdb-abstraction.md` | +| **相关** | `mq-abstraction.md`、`tsdb-abstraction.md` —— 本设计将该家族模式扩展到关系层 | -## 1. Motivation +## 1. 动机 -Three drivers, in priority order: +三个驱动因素，按优先级排序： -1. **GraalVM native images.** `native-maven-plugin` 1.1.10 is already managed in the root POM, but MyBatis-Plus blocks the - path: runtime mapper proxies, reflection-driven bean wiring, and jsqlparser-based SQL rewriting are all hostile to - closed-world compilation. Spring Data R2DBC ships native support (hints built in since Boot 3; drivers are - Netty-based). The dependency swap is the enabling step for native builds of every center service. -2. **True end-to-end reactive.** Controllers are WebFlux today, but Service → Manager → Mapper is synchronous blocking, - bridged by `BaseController.async(...)` onto `Schedulers.boundedElastic()` with manual `TenantContextHolder` - set/clear. A reactive persistence layer removes the bridge, the thread-pool hop, and the ThreadLocal lifecycle - hazard in one move. -3. **Dialect freedom.** The current dialect mechanism is MyBatis `databaseId` forks inside shared XML (42 statements - across 6 files), gated by a "portable-first" discipline. R2DBC has no `databaseId`; the natural shape is one - implementation module per dialect holding native SQL — the same contract/adapter/TCK structure the MQ and TSDB - families already use (`RepositoryService` port in `dc3-common-repository`, adapters in `dc3-tsdb-*`, TCK in - `dc3-tsdb-tck`). +1. **GraalVM native image。** `native-maven-plugin` 1.1.10 已在根 POM 中托管版本，但 MyBatis-Plus 挡住了这条路：运行期 + mapper 代理、反射驱动的 bean 装配、基于 jsqlparser 的 SQL 改写，全都与封闭世界（closed-world）编译相抵触。Spring Data + R2DBC 自带 native 支持（Boot 3 起内置 hints；驱动基于 Netty）。这次依赖替换是每个 center 服务得以 native 构建的前置一步。 +2. **真正的端到端响应式。** 控制器今天已是 WebFlux，但 Service → Manager → Mapper 链路是同步阻塞的，靠 + `BaseController.async(...)` 桥接到 `Schedulers.boundedElastic()`，并手工 set/clear `TenantContextHolder`。响应式持久层 + 一步移除这座桥、这次线程池跳转和 ThreadLocal 生命周期隐患。 +3. **方言自由。** 当前的方言机制是共享 XML 内按 MyBatis `databaseId` 分叉（42 条语句散布在 6 个文件中），靠“可移植优先” + 的纪律把关。R2DBC 没有 `databaseId`；自然的形态是每个方言一个实现模块、承载原生 SQL——与 MQ 和 TSDB 家族已在使用的 + 契约/适配器/TCK 结构相同（`dc3-common-repository` 中的 `RepositoryService` Port、`dc3-tsdb-*` 中的适配器、 + `dc3-tsdb-tck` 中的 TCK）。 -Weighting, stated plainly: driver 1 is the hard requirement. Driver 2, for auth itself, is an architectural-consistency -investment — auth is a management plane (login/CRUD, human-driven QPS) and no latency or throughput gain is claimed from -removing the boundedElastic hop; its payoffs (native image, no ThreadLocal lifecycle hazard, readiness for the -high-concurrency data/manager wave) arrive later. The cost — rewriting 12 services plus the OAuth/MCP runtime as -reactive chains, with harder debugging — is accepted and bounded by the P2/P3 go/no-go gate, the TCK, and e2e. +权重，直说：驱动因素 1 是硬性要求。对 auth 本身而言，驱动因素 2 是一笔架构一致性投资——auth 是管理面（登录/CRUD、人为 +驱动的 QPS），移除 boundedElastic 跳换并不宣称带来时延或吞吐收益；它的回报（native image、消除 ThreadLocal 生命周期隐患、 +为高并发的 data/manager 浪潮做好准备）要到后面才兑现。代价——把 12 个 service 加上 OAuth/MCP 运行时重写为响应式链路、 +调试更难——已被接受，并由 P2/P3 的 go/no-go 闸门、TCK 与 e2e 加以约束。 -What does **not** change: deploy-time selection via `dc3.db.type` (postgres | mysql | mariadb), the initdb seed SQL -(executed by container entrypoints, independent of the access layer), table schemas, and the HTTP/gRPC surface. +**不会**改变的部分：经 `dc3.db.type`（postgres | mysql | mariadb）的部署期选择、initdb 种子 SQL（由容器 entrypoint 执行， +与访问层无关）、表结构，以及 HTTP/gRPC 契约面。 -## 2. Goals / Non-goals +## 2. 目标 / 非目标 -**Goals** +**目标** -- auth center is the pilot: fully R2DBC, zero MyBatis on its classpath, native image compiles and boots. -- One module per (dialect × service), holding that dialect's native SQL — no shared XML, no `databaseId`. -- Repository behavior TCK: same suite green on PostgreSQL, MySQL, MariaDB via Testcontainers. -- Tenant isolation preserved with **stronger** guarantees than today (see §6). -- gRPC contracts unchanged; HTTP contracts unchanged **except** the deliberately redesigned pagination - envelope (D8 — nested `page` object; one-step cutover, no compatibility window). +- 鉴权中心(auth)是试点：全面 R2DBC，classpath 上零 MyBatis，native image 可编译、可启动。 +- 每个（方言 × 服务）一个模块，承载该方言的原生 SQL——没有共享 XML，没有 `databaseId`。 +- Repository 行为 TCK：同一套套件经 Testcontainers 在 PostgreSQL、MySQL、MariaDB 上全绿。 +- 租户隔离得以保留，且保证**强于**今天（见 §6）。 +- gRPC 契约不变；HTTP 契约不变，**唯独**刻意重设计的分页信封除外（D8——嵌套 `page` 对象；一步切换，无兼容窗口）。 -**Non-goals** +**非目标** -- Runtime hot-swap of the database engine (unchanged: deploy-time selection, restart to switch). -- Rewriting manager/data in this track — they follow the same pattern later; their MyBatis stack is untouched. -- MyBatis-Plus feature parity. Only the features actually used by auth are rebuilt (inventory in §3). -- ORM-level schema management. initdb stays authoritative; no Flyway/Liquibase introduced. +- 数据库引擎的运行期热替换（不变：部署期选择，重启切换）。 +- 在本轨中重写 manager/data——它们稍后按同一模式跟进；其 MyBatis 技术栈原封不动。 +- 与 MyBatis-Plus 的功能对齐。只重建 auth 实际用到的功能（盘点见 §3）。 +- ORM 级的 schema 管理。initdb 保持权威；不引入 Flyway/Liquibase。 -## 3. Current-state facts that shape the design +## 3. 塑造设计的现状事实 -Verified inventory (2026-08-28), each fact constrains a decision: +已核实的盘点（2026-08-28），每条事实都约束一项决策： -| # | Fact | Consequence | +| # | 事实 | 推论 | |---|------|-------------| -| F1 | Of 16 Manager interfaces in `dc3-common-auth`, **15 are empty** (`extends IService`, zero methods). Only `IdentityAuditLogManager` has a custom method. | The contract is a blank page — define it from what Services actually need, not from `IService`. | -| F2 | Service layer leans on `IService` + `LambdaQueryWrapper` (fuzzy queries, `page(...)`, `getOne(wrapper)`, `wrapper.apply("1 = 0")`, two-step membership→`IN principalIds` tenant filtering for tenant-less tables). | Dynamic-query logic moves into named repository methods taking the existing Query BOs (D7). | -| F3 | `OAuthMcpMapper` is a plain hand-written mapper (30 methods, no `BaseMapper`), injected **directly by biz layer** (`OAuthMcpRuntimeServiceImpl`), writing 7 tables; all `databaseId` forks for auth live in its XML. `ResourceRegistryLockMapper` (advisory lock) is also hand-written. | Already repository-shaped; migration is SQL relocation + reactive signatures. It is the largest single aggregate (D13). | -| F4 | `TenantContextHolder` is ThreadLocal (in `dc3-common-constant`); written in exactly 3 entry kinds: `BaseController.async`, local facades, gRPC servers (manual set/clear). Consumer is `TenantLineHandlerImpl` (fail-closed: no tenant → `TenantNotScopedException`). Hand-written XML SQL already carries explicit `tenant_id = #{tenantId}` — the interceptor only covers wrapper paths. | Explicit tenant parameters in the new contract are a continuation of the existing pattern, not new discipline (D5). ThreadLocal stays for MyBatis-era modules only. | -| F5 | 11 tables have no `tenant_id` column (whitelist in `TenantLineHandlerImpl`): `dc3_tenant`, `dc3_principal`, `dc3_user`, `dc3_local_credential`, `dc3_external_identity`, `dc3_resource`, `dc3_role_resource_bind`, `dc3_api`, `dc3_mcp_tool_catalog`, `dc3_mcp_connection_tool`, `dc3_menu`. | Tenant-less repositories are named as such in the contract; no runtime whitelist needed (D6). | -| F6 | IDs are application-side snowflake everywhere: `@TableId(type = ASSIGN_ID)` (60+ DOs) plus explicit `IdWorker.getId()` in the OAuth/MCP path. No DB sequences. | Swap in a small first-party snowflake generator; no schema change (D10). | -| F7 | Logical delete via `@TableLogic` (47 DOs, `deleted` flag; partial unique indexes like `... WHERE deleted = 0`). | Explicit `deleted = 0` predicates in implementations; contract documents the semantics (D9). | -| F8 | `create_time` / `operate_time` are purely DB-side (PG trigger functions / MySQL `ON UPDATE CURRENT_TIMESTAMP`); updates deliberately null out `operate_time` and re-read. | Inserts exclude time columns; keep re-read pattern; zero schema change (D11). | -| F9 | 19 `@Transactional(rollbackFor = Exception.class)` sites in auth (9 in `OAuthMcpRuntimeServiceImpl`, 3 in `ResourceRegistrySyncServiceImpl`, 3 in `MenuServiceImpl`, 3 in `ServiceAccountServiceImpl`). Plus one unmanaged 3-table write in `UserController.add/delete` (principal → user → membership) — a pre-existing defect. | Reactive transactions with `R2dbcTransactionManager`; fix the UserController gap during migration (D12). | -| F10 | Pagination is one uniform pattern: `Page` + `PageUtil.page(Pages)` in 12 ServiceImpls; HTTP returns MyBatis-Plus `Page` (a leaked framework type — internal fields serialize into the JSON); gRPC uses proto `GrpcPage` (data/driver contracts only — auth's gRPC surface has no pagination); OAuth/MCP does manual `limit/offset` + `count`. | Redesign the envelope (D8): frontend consumption is funneled through `PageResult` in `types/common.ts` + `usePagedList`/`useEntityListPage` + `mock/response.ts`, reading only `records/total/size/current` — one coordinated frontend switch. | -| F11 | SQL exceptions have no dedicated translation; business uniqueness is check-then-insert (`getOne` → `DuplicateException`); fallback advice returns 500. | Keep check-then-insert as primary; add a thin `R2dbcException` mapping helper (D17). | -| F12 | `dc3-common-dal` (label/group tables) is only used by manager; auth merely imports `DictionaryBO` from it, and `DictionaryForAuthService` reads `dc3_tenant` via `TenantManager`. data/agentic depend on dal in POM but reference it zero times in code. | auth pilot does **not** touch `dc3-common-dal`. Scope shrinks. | -| F13 | Datasource per center service is a single `master` route (dynamic-datasource), URL `currentSchema=dc3_auth` (PG) / per-service database (MySQL); Hikari max 64. | R2DBC needs one connection factory + `r2dbc-pool`; dynamic-datasource drops out of auth entirely. | -| F14 | Selection mechanism today: `dc3.db.type` → `@ConditionalOnProperty` per dialect jar + EnvironmentPostProcessor profile activation; adapter jars are opt-in Maven deps (auth ships postgres by default; mysql/mariadb only in TCK). | Same mechanism reused; no new selection concept (D3). | -| F15 | Existing `dc3-db-{postgres,mysql,mariadb}` jars carry MyBatis-specific config (pagination `DbType`, timestamptz TypeHandler, driver-class-name). manager/data still need them during transition. | Leave the existing dialect modules untouched; R2DBC config lives in the new per-service dialect modules (D3). | +| F1 | `dc3-common-auth` 的 16 个 Manager 接口中，**15 个是空的**（`extends IService`，零方法）。只有 `IdentityAuditLogManager` 拥有自定义方法。 | 契约是一张白纸——按 Service 实际需要来定义它，而不是按 `IService`。 | +| F2 | Service 层依赖 `IService` + `LambdaQueryWrapper`（模糊查询、`page(...)`、`getOne(wrapper)`、`wrapper.apply("1 = 0")`、针对无租户表的两步 membership→`IN principalIds` 租户过滤）。 | 动态查询逻辑移入具名 repository 方法，接收既有 Query BO（D7）。 | +| F3 | `OAuthMcpMapper` 是纯手写 mapper（30 个方法，无 `BaseMapper`），被 biz 层**直接注入**（`OAuthMcpRuntimeServiceImpl`），写 7 张表；auth 的全部 `databaseId` 分叉都位于其 XML 中。`ResourceRegistryLockMapper`（咨询锁）同样是手写的。 | 它已经是 Repository 的形状；迁移即 SQL 搬家 + 响应式签名。它是最大的单一聚合（D13）。 | +| F4 | `TenantContextHolder` 基于 ThreadLocal（位于 `dc3-common-constant`）；写入点恰好有 3 类入口：`BaseController.async`、本地 Facade、gRPC 服务器（手工 set/clear）。消费方是 `TenantLineHandlerImpl`（fail-closed：无租户 → `TenantNotScopedException`）。手写 XML SQL 已携带显式 `tenant_id = #{tenantId}`——拦截器只覆盖 wrapper 路径。 | 新契约中的显式租户参数是既有模式的延续，不是新纪律（D5）。ThreadLocal 只留给 MyBatis 时代的模块。 | +| F5 | 11 张表没有 `tenant_id` 列（`TenantLineHandlerImpl` 中的白名单）：`dc3_tenant`、`dc3_principal`、`dc3_user`、`dc3_local_credential`、`dc3_external_identity`、`dc3_resource`、`dc3_role_resource_bind`、`dc3_api`、`dc3_mcp_tool_catalog`、`dc3_mcp_connection_tool`、`dc3_menu`。 | 无租户 repository 在契约中以名称标明；无需运行期白名单（D6）。 | +| F6 | ID 处处都是应用侧雪花 ID：`@TableId(type = ASSIGN_ID)`（60+ 个 DO），OAuth/MCP 路径另有显式 `IdWorker.getId()`。无数据库序列。 | 换成一个小的第一方雪花 ID 生成器；无 schema 变更（D10）。 | +| F7 | 经 `@TableLogic` 的逻辑删除（47 个 DO，`deleted` 标志；形如 `... WHERE deleted = 0` 的部分唯一索引）。 | 实现中写显式 `deleted = 0` 谓词；契约将该语义文档化（D9）。 | +| F8 | `create_time` / `operate_time` 纯属数据库侧（PG 触发器函数 / MySQL `ON UPDATE CURRENT_TIMESTAMP`）；更新时故意把 `operate_time` 置空并回读。 | 插入排除时间列；保留回读模式；零 schema 变更（D11）。 | +| F9 | auth 中有 19 处 `@Transactional(rollbackFor = Exception.class)`（`OAuthMcpRuntimeServiceImpl` 9 处、`ResourceRegistrySyncServiceImpl` 3 处、`MenuServiceImpl` 3 处、`ServiceAccountServiceImpl` 3 处）。另有 `UserController.add/delete` 中一处不受管理的 3 表写（principal → user → membership）——既有缺陷。 | 用 `R2dbcTransactionManager` 做响应式事务；迁移期间修复 UserController 缺口（D12）。 | +| F10 | 分页是统一的一种模式：12 个 ServiceImpl 中 `Page` + `PageUtil.page(Pages)`；HTTP 返回 MyBatis-Plus `Page`（泄漏的框架类型——内部字段序列化进 JSON）；gRPC 用 proto `GrpcPage`（仅 data/driver 契约——auth 的 gRPC 契约面没有分页）；OAuth/MCP 手工 `limit/offset` + `count`。 | 重新设计信封（D8）：前端消费收拢到 `types/common.ts` 的 `PageResult` + `usePagedList`/`useEntityListPage` + `mock/response.ts`，只读 `records/total/size/current`——一次协调好的前端切换。 | +| F11 | SQL 异常没有专门转译；业务唯一性靠 check-then-insert（`getOne` → `DuplicateException`）；兜底 advice 返回 500。 | check-then-insert 仍为主路径；增加一个薄的 `R2dbcException` 映射助手（D17）。 | +| F12 | `dc3-common-dal`（label/group 表）只有 manager 在用；auth 仅从中导入 `DictionaryBO`，`DictionaryForAuthService` 经 `TenantManager` 读 `dc3_tenant`。data/agentic 在 POM 中依赖 dal，但代码中零引用。 | auth 试点**不**触碰 `dc3-common-dal`。范围缩小。 | +| F13 | 每个 center 服务的数据源是单一 `master` 路由（dynamic-datasource），URL 为 `currentSchema=dc3_auth`（PG）/ 每服务一个数据库（MySQL）；Hikari max 64。 | R2DBC 只需一个连接工厂 + `r2dbc-pool`；dynamic-datasource 从 auth 中完全退出。 | +| F14 | 今天的选择机制：`dc3.db.type` → 每方言 jar 的 `@ConditionalOnProperty` + EnvironmentPostProcessor profile 激活；适配器 jar 是可选的 Maven 依赖（auth 默认自带 postgres；mysql/mariadb 只在 TCK 中）。 | 复用同一机制；没有新的选择概念（D3）。 | +| F15 | 既有 `dc3-db-{postgres,mysql,mariadb}` jar 携带 MyBatis 专属配置（分页 `DbType`、timestamptz TypeHandler、driver-class-name）。过渡期 manager/data 仍需要它们。 | 既有方言模块原样保留；R2DBC 配置放进新的每服务方言模块（D3）。 | -## 4. Target architecture +## 4. 目标架构 ```text dc3-common/dc3-common-auth @@ -97,185 +92,167 @@ dc3-db/dc3-db-r2dbc-core ← new, pure types, ZERO Spring/R2DBC deps: manager/data's MyBatis stack until their own migration) ``` -Dependency direction: `dc3-db-auth-{dialect} → dc3-common-auth (contract) + dc3-db-r2dbc-core (pure types) + dc3-db-r2dbc-boot + spring-data-r2dbc + dialect r2dbc driver`. The contract module depends only on `dc3-db-r2dbc-core` and never sees Spring Data, driver types, or MyBatis — the core/boot split is what keeps D2 true while D4 still lets contracts use `PageResult`. +依赖方向：`dc3-db-auth-{dialect} → dc3-common-auth (contract) + dc3-db-r2dbc-core (pure types) + dc3-db-r2dbc-boot + spring-data-r2dbc + dialect r2dbc driver`。契约模块只依赖 `dc3-db-r2dbc-core`，永远接触不到 Spring Data、驱动类型或 MyBatis——core/boot 拆分正是让 D2 保持成立、同时 D4 仍允许契约使用 `PageResult` 的关键。 -Selection reuses F14 exactly: each `dc3-db-auth-{dialect}` auto-configuration is `@ConditionalOnProperty(prefix = "dc3.db", name = "type", havingValue = "{dialect}")` (postgres `matchIfMissing = true`), contributes its `ConnectionFactory`-based config via its own profile yml, and a missing-adapter mismatch fails fast at boot — same as `MybatisPlusConfig` does today. +选择机制精确复用 F14：每个 `dc3-db-auth-{dialect}` 自动配置都是 `@ConditionalOnProperty(prefix = "dc3.db", name = "type", havingValue = "{dialect}")`（postgres 为 `matchIfMissing = true`），经由自己的 profile yml 提供基于 `ConnectionFactory` 的配置；适配器缺失或不匹配时启动即快速失败——与今天 `MybatisPlusConfig` 的做法相同。 -## 5. Decision checklist +## 5. 决策清单 -Each entry: context → options → **recommendation** → consequence. Status column is the review tracker. +每条决策：背景 → 备选项 → **建议** → 后果。状态列是评审跟踪表。 -| # | Decision | Recommendation | Status | -|---|----------|----------------|--------| -| D1 | **Contract home** — where repository interfaces live. Options: (a) `dc3-common-auth` `repository` package; (b) `dc3-db-core` as a contract hub. | **(a)** — follows the TSDB precedent (`RepositoryService` lives in `dc3-common-repository`, adapters depend on it). Keeps `dc3-db-core` from becoming a business omnibus; implementation → contract dependency is clean. | **Confirmed 2026-08-28** | -| D2 | **Contract style** — plain `Mono`/`Flux` interfaces vs extending Spring Data repository types. | **Plain interfaces, zero framework types.** Dialect modules may internally extend `ReactiveCrudRepository` and adapt. TCK tests the contract, not Spring Data. | **Confirmed 2026-08-28** | -| D3 | **Module layout** — nested under existing dialect modules vs flat under `dc3-db`; if flat, which naming axis. | **Flat, service-first**: `dc3-db/dc3-db-auth-{postgres,mysql,mariadb}` (later `dc3-db-manager-*`, `dc3-db-data-*`). Service-first because day-to-day work opens the three dialect implementations of one contract side by side, `dc3-center-auth`'s dependency list shows a single prefix, and TCK contracts are organized per business module. Existing `dc3-db-*` modules untouched (F15). Nesting rejected: `dc3-db-postgres` is a jar serving a retiring MyBatis transition and would need aggregator/inheritance contortions. | **Confirmed 2026-08-28** | -| D4 | **Shared R2DBC infrastructure** — new `dc3-db-r2dbc-core` vs scattering helpers. | **Two thin modules, split by dependency weight**: `dc3-db-r2dbc-core` holds pure types only — first-party snowflake (MP-compatible shape: timestamp+worker+sequence, so IDs stay trend-ordered across the transition), `PageResult`/`PageInfo`, `R2dbcException` → business exception mapping — with zero Spring/R2DBC deps, so the contract module may depend on it without violating D2; `dc3-db-r2dbc-boot` holds the Spring-dependent shared helpers (ConnectionFactory/pool auto-config conventions, custom conversions, D19 codec, observability) where spring-data-r2dbc is a compile dep. No MyBatis anywhere. | **Confirmed 2026-08-28** (amended: core/boot split so D2 and D4 stop contradicting) | -| D5 | **Tenant passing** — explicit `tenantId` parameters vs Reactor Context propagation vs keeping ThreadLocal. | **Explicit parameters** (first argument, `Long tenantId`). Compile-time fail-closed — stronger than today's runtime exception; continues what hand-written SQL already does (F4); kills the ThreadLocal lifecycle hazard on the R2DBC path. All three entry kinds already hold `tenantId` explicitly (Controller via security context, gRPC via request field, local facade via parameter) — the change deletes the ThreadLocal detour rather than adding parameter threading. `TenantContextHolder` remains untouched for manager/data's MyBatis stack. | **Confirmed 2026-08-28** | -| D6 | **Tenant-less surfaces** — how `runIgnore`/whitelist semantics carry over. | **Named method variants** on tenant-less repositories (F5 tables): no `runIgnore`, no global whitelist. System/login paths call methods whose names say they are unscoped (e.g. `findByLoginName`), and every tenant-scoped method simply does not exist without a tenant argument. | **Confirmed 2026-08-28** | -| D7 | **Dynamic queries** — port `LambdaQueryWrapper` semantics how? | **Named repository methods per Query BO** (`Mono> listByQuery(Long tenantId, UserQuery q)`); each dialect renders its own SQL. No criteria-builder port — the 12 fuzzyQuery methods are finite, reviewable SQL. | **Confirmed 2026-08-28** | -| D8 | **Pagination** — neutral type and envelope shape. | **Redesigned, one-step cutover, no compatibility window (not MP-compatible): nested page object.** `PageResult = { records: T[], page: PageInfo }`, `PageInfo = { current, size, total, pages }`, both in `dc3-db-r2dbc-core` (D4). Why nested instead of a flat MP-shaped neutral type (flat would be zero-cost today but pays forever): a paged response keeps exactly two top-level keys — payload and metadata — so future pagination metadata (sort echo, approximate-total flags, cursors) extends `PageInfo` without ever touching the payload or accreting new top-level fields, which is precisely the accretion mechanism that leaked MP's internal fields in the first place (F10); `PageInfo` is reusable wherever only counts matter; frontend gets one generic `PageResult` with typed metadata. `orders` are not echoed back (the caller knows what it sent). `PageUtil`'s clamping/default-ordering moves into the repository layer. Why no compatibility layer: the break is paid exactly once — a dual-format window would keep the leaked shape alive through the manager/data migrations and force a second break later; the shape changes here, at the moment the persistence layer is rewritten anyway. Bounded cutover: dc3-web (four funnel touchpoints — `types/common.ts`, `usePagedList`, `useEntityListPage`, `mock/response.ts` — now reading `records` + `page.*`), openapi snapshots, and e2e switch in the same change set; external REST integrators get a documented breaking change via release notes and regenerated OpenAPI. auth's gRPC contracts carry no pagination, so protos are untouched. `PageResult`/`PageInfo` become the project-wide envelope as manager/data migrate. | **Confirmed 2026-08-28** (re-review: one-step cutover, compatibility explicitly rejected) | -| D9 | **Logical delete** — how `@TableLogic` semantics survive. | **Explicit SQL predicates** (`deleted = 0` in every read, `SET deleted = 1` in deletes) + Javadoc on each contract method stating whether deleted rows are filtered. Partial unique indexes (`WHERE deleted = 0`) continue to enforce uniqueness. | **Confirmed 2026-08-28** | -| D10 | **ID generation** — MP `IdWorker` is unavailable post-migration. | **First-party snowflake in `dc3-db-r2dbc-core`**, same bit layout as MP `ASSIGN_ID`; called explicitly before insert (the `OAuthMcpRuntimeServiceImpl` pattern already does exactly this — generalize it). **WorkerId allocation is a correctness invariant, not a detail**: explicit env assignment (`DC3_DB_WORKER_ID`-style; StatefulSet ordinal / fixed per replica), deterministic IP-hash fallback for single-node dev, and startup detection that refuses a duplicate workerId+datacenter pair within one deployment; the TCK adds a two-instance uniqueness contract. The assigned space must also not collide with the MP `IdWorker` instances still writing the same tables from `dc3-center-single` during the dual-stack transition. | **Confirmed 2026-08-28** (amended: workerId allocation rule) | -| D11 | **Timestamps** — who writes `create_time`/`operate_time`. | **DB stays authoritative** (F8): inserts exclude both columns, updates never set `operate_time`, business code re-reads after update where it needs the trigger value. Zero schema change, zero behavior change. | **Confirmed 2026-08-28** | -| D12 | **Transactions** — reactive transaction strategy. | **`@Transactional(rollbackFor = Exception.class)` on reactive methods + `R2dbcTransactionManager`** auto-configured per dialect module. All 19 existing sites carry over (F9); the unmanaged `UserController.add/delete` 3-table write is wrapped in a service-level transaction as part of the migration (defect fix, noted in §11). | **Confirmed 2026-08-28** | -| D13 | **OAuth/MCP surface split** — one `OAuthMcpRepository` mirroring the 30-method mapper vs per-aggregate repositories. | **Per-aggregate split**: `OAuthClientRepository`, `OAuthAuthorizationRepository`, `McpConnectionRepository` (+ `dc3_mcp_connection_tool` — `replaceConnectionTools` is a connection aggregate operation), `McpToolCatalogRepository`, `McpAuditLogRepository` (+ `dc3_mcp_tool_confirmation`; exact home decided at implementation by the aggregate-root rule). The 7 tables (F3) map to 5 focused contracts; dialect-forked statements (JSON casts, upsert idioms) land in each dialect module. Reactive transactions bind to the `ConnectionFactory`, not the repository, so biz-level `@Transactional` composes across repositories freely. | **Confirmed 2026-08-28** | -| D14 | **MySQL driver** — `r2dbc-mysql` (community) vs `r2dbc-mariadb` (official) against MySQL servers. | **Decided by TCK spike, both candidates wired**: MySQL has no official R2DBC driver; the community driver is well-regarded but volunteer-maintained, while the official MariaDB driver also speaks the MySQL protocol (caveats: `caching_sha2_password`, JSON codec). Gate: whichever passes the full TCK on `mysql:8.4` stays; record the rationale here. MariaDB dialect uses `r2dbc-mariadb` (no contest). | Pending — P1 spike | -| D15 | **Query style inside dialect modules.** | **Spring Data interfaces + `@Query` native SQL as the default; `DatabaseClient` for batch upserts and connection-bound operations** (advisory locks — see §8). Free choice per statement, invisible above the contract (D2). | **Confirmed 2026-08-28** | -| D16 | **Coexistence & cutover** — MyBatis removal timing for auth. | **Hard cutover per service**: auth ships R2DBC-only (drops `dc3-db-core`, dynamic-datasource, `mapping/*.xml`, `@MapperScan`); `dc3-center-single` runs both stacks against the same database until manager/data migrate (two pools during transition — accepted, bounded). | **Confirmed 2026-08-28** | -| D17 | **Error translation** — SQL exception mapping. | **Thin helper in `dc3-db-r2dbc-core`**: `R2dbcDataIntegrityViolationException` → existing `DuplicateException`/`BusinessException` mapping, applied in dialect implementations at natural points. Check-then-insert stays the primary UX path (F11); the mapping is a backstop so constraint violations no longer surface as 500s. | **Confirmed 2026-08-28** | -| D18 | **Facade & gRPC server boundary** — auth's non-HTTP seams, absent from earlier drafts. Auth services are also exposed via 7 blocking facade contracts (`TokenFacade.checkValid` returns `boolean`, plus Permission/User/Tenant/LocalCredential/ResourceRegistry/McpRuntime), 7 gRPC servers (`grpc/*Server.java` — `StreamObserver` callbacks wrapping services in `TenantContextHolder.runIgnore`), and the local facade impls; consumers are the gateway (blocking gRPC stub) and every WebFlux service's security chain via `FacadePermissionProvider`. Options: (a) make facade contracts reactive now; (b) keep blocking contracts and bridge explicitly. | **(b) — blocking edge, explicit and temporary.** Facade contracts stay synchronous in this track; the reactive→blocking bridge lives only in auth's gRPC servers and local facade impls, executed on gRPC executor / boundedElastic threads (never an event loop — guard against event-loop `block()` errors), and `runIgnore` wrappers become unscoped method variants (§6). This keeps the pilot's blast radius bounded — option (a) would drag the gateway `FilterServiceImpl`, `dc3-common-facade-grpc`, and shared `dc3-common-web` into the auth track. "True end-to-end reactive" holds controller→repository inside auth; the edge stays synchronous by decision until the facade wave lands with manager/data, at which point `TokenGrpcFacade`, `FilterServiceImpl`, and `FacadePermissionProvider` simplify. In-process gRPC tests keep the blocking `GrpcInProcessExtension` harness. | **Confirmed 2026-08-28** | -| D19 | **Timestamp codec** — who converts `timestamptz` ↔ `LocalDateTime` (successor to `TimestamptzLocalDateTimeTypeHandler`). The R2DBC PostgreSQL driver maps `timestamptz` natively to `OffsetDateTime`, not `LocalDateTime`; MySQL/MariaDB `DATETIME` needs no conversion. | **First-party `R2dbcCustomConversions` in `dc3-db-r2dbc-boot`** (D4 split): registers UTC converters (OffsetDateTime ↔ LocalDateTime) for the PG dialect, preserving F8/D11 semantics and the §9 round-trip contract; MySQL/MariaDB modules use default mappings. Decided explicitly because it is the top R2DBC-PG migration pitfall and would otherwise surface as a red TCK item with no owner. | **Confirmed 2026-08-28** | +| # | 决策 | 建议 | 状态 | +|---|------|------|------| +| D1 | **契约归属**——Repository 接口放在哪里。备选项：(a) `dc3-common-auth` 的 `repository` 包；(b) `dc3-db-core` 作为契约枢纽。 | **(a)**——沿袭 TSDB 先例（`RepositoryService` 放在 `dc3-common-repository`，适配器依赖它）。避免 `dc3-db-core` 变成业务大杂烩；实现 → 契约的依赖方向干净。 | **已确认 2026-08-28** | +| D2 | **契约风格**——纯 `Mono`/`Flux` 接口，还是扩展 Spring Data repository 类型。 | **纯接口，零框架类型。**方言模块可以在内部扩展 `ReactiveCrudRepository` 并做适配。TCK 测的是契约，不是 Spring Data。 | **已确认 2026-08-28** | +| D3 | **模块布局**——嵌在既有方言模块之下，还是在 `dc3-db` 下平铺；若平铺，按哪个命名轴。 | **平铺、服务优先**：`dc3-db/dc3-db-auth-{postgres,mysql,mariadb}`（后续 `dc3-db-manager-*`、`dc3-db-data-*`）。服务优先，因为日常工作会把同一契约的三个方言实现并排打开，`dc3-center-auth` 的依赖列表呈现单一前缀，TCK 契约也按业务模块组织。既有 `dc3-db-*` 模块不动（F15）。否决嵌套：`dc3-db-postgres` 是一个服务于行将退役的 MyBatis 过渡期的 jar，嵌套将需要聚合器/继承层面的别扭扭曲。 | **已确认 2026-08-28** | +| D4 | **共享 R2DBC 基础设施**——新建 `dc3-db-r2dbc-core`，还是把助手散落各处。 | **两个薄模块，按依赖重量拆分**：`dc3-db-r2dbc-core` 只放纯类型——第一方雪花 ID（与 MP 兼容的形状：timestamp+worker+sequence，保证 ID 在过渡期保持趋势有序）、`PageResult`/`PageInfo`、`R2dbcException` → 业务异常映射——零 Spring/R2DBC 依赖，契约模块可以依赖它而不违反 D2；`dc3-db-r2dbc-boot` 放依赖 Spring 的共享助手（ConnectionFactory/连接池自动配置约定、自定义转换、D19 编解码、可观测性），spring-data-r2dbc 只在这里是编译期依赖。任何地方都不碰 MyBatis。 | **已确认 2026-08-28**（修订：core/boot 拆分，使 D2 与 D4 不再互相矛盾） | +| D5 | **租户传递**——显式 `tenantId` 参数，还是 Reactor Context 传播，还是保留 ThreadLocal。 | **显式参数**（第一个参数，`Long tenantId`）。编译期 fail-closed——比今天的运行期异常更强；延续手写 SQL 已有的做法（F4）；消灭 R2DBC 路径上的 ThreadLocal 生命周期隐患。三类入口本就显式持有 `tenantId`（Controller 经安全上下文，gRPC 经请求字段，本地 Facade 经参数）——这一改动是删除 ThreadLocal 的绕路，而不是新增一路参数传递。`TenantContextHolder` 为 manager/data 的 MyBatis 技术栈原样保留。 | **已确认 2026-08-28** | +| D6 | **无租户契约面**——`runIgnore`/白名单语义如何延续。 | 无租户 repository（F5 各表）上采用**具名方法变体**：没有 `runIgnore`，没有全局白名单。系统/登录路径调用的方法，名字本身就说明其无作用域（如 `findByLoginName`）；每个租户作用域方法在没有租户参数的情况下干脆不存在。 | **已确认 2026-08-28** | +| D7 | **动态查询**——如何移植 `LambdaQueryWrapper` 语义？ | **每个 Query BO 对应具名 repository 方法**（`Mono> listByQuery(Long tenantId, UserQuery q)`）；每个方言渲染自己的 SQL。不做 criteria-builder 式移植——12 个 fuzzyQuery 方法是有限、可评审的 SQL。 | **已确认 2026-08-28** | +| D8 | **分页**——中立类型与信封形状。 | **重新设计、一步切换、无兼容窗口（不与 MP 兼容）：嵌套 page 对象。**`PageResult = { records: T[], page: PageInfo }`、`PageInfo = { current, size, total, pages }`，二者都放在 `dc3-db-r2dbc-core`（D4）。为什么嵌套而非扁平的 MP 形状中立类型（扁平今天零成本，却要永远偿还）：分页响应恰好保持两个顶层键——载荷与元数据——未来的分页元数据（排序回显、近似总数标志、游标）只扩展 `PageInfo`，永不触碰载荷、也不累积新的顶层字段，而累积正是当初把 MP 内部字段泄漏出来的机制（F10）；凡是只关心计数的地方 `PageInfo` 都可复用；前端得到一个泛型 `PageResult`，元数据带类型。`orders` 不回显（调用方知道自己发了什么）。`PageUtil` 的钳制/默认排序移入 repository 层。为什么不设兼容层：这次破坏恰好只付一次——双格式窗口会让泄漏的形状在 manager/data 迁移期间继续存活，并迫使日后二次破坏；形状就在此处改变，何况持久层本来就在此刻重写。切换范围有界：dc3-web（四个收拢触点——`types/common.ts`、`usePagedList`、`useEntityListPage`、`mock/response.ts`——现改为读取 `records` + `page.*`）、openapi 快照与 e2e 在同一变更集内切换；外部 REST 集成方经由发布说明与重新生成的 OpenAPI 获得书面记录的破坏性变更。auth 的 gRPC 契约不带分页，proto 因此不动。随着 manager/data 迁移，`PageResult`/`PageInfo` 成为项目级信封。 | **已确认 2026-08-28**（复审：一步切换，明确否决兼容方案） | +| D9 | **逻辑删除**——`@TableLogic` 语义如何存续。 | **显式 SQL 谓词**（每次读取带 `deleted = 0`，删除用 `SET deleted = 1`）+ 每个契约方法的 Javadoc 注明是否过滤已删除行。部分唯一索引（`WHERE deleted = 0`）继续保障唯一性。 | **已确认 2026-08-28** | +| D10 | **ID 生成**——迁移后 MP `IdWorker` 不可用。 | **`dc3-db-r2dbc-core` 中的第一方雪花 ID**，位布局与 MP `ASSIGN_ID` 相同；插入前显式调用（`OAuthMcpRuntimeServiceImpl` 的既有模式已经正是这么做的——把它推广开来）。**workerId 分配是正确性不变量，不是细节**：显式环境变量分配（`DC3_DB_WORKER_ID` 风格；StatefulSet 序号 / 每副本固定值）、单节点开发用确定性的 IP-hash 兜底，以及启动检测——同一部署内拒绝重复的 workerId+datacenter 组合；TCK 增加双实例唯一性契约。双栈过渡期，已分配空间还必须不与 `dc3-center-single` 中仍在写同一批表的 MP `IdWorker` 实例冲突。 | **已确认 2026-08-28**（修订：workerId 分配规则） | +| D11 | **时间戳**——`create_time`/`operate_time` 由谁写。 | **数据库保持权威**（F8）：插入排除两列，更新绝不设置 `operate_time`，需要触发器值的业务代码在更新后回读。零 schema 变更，零行为变更。 | **已确认 2026-08-28** | +| D12 | **事务**——响应式事务策略。 | **响应式方法上的 `@Transactional(rollbackFor = Exception.class)` + `R2dbcTransactionManager`**，由每个方言模块自动配置。既有 19 处全部平移（F9）；`UserController.add/delete` 那处不受管理的 3 表写作为迁移的一部分包进 service 级事务（缺陷修复，见 §11）。 | **已确认 2026-08-28** | +| D13 | **OAuth/MCP 契约面拆分**——一个镜像 30 方法 mapper 的 `OAuthMcpRepository`，还是按聚合拆分 repository。 | **按聚合拆分**：`OAuthClientRepository`、`OAuthAuthorizationRepository`、`McpConnectionRepository`（+ `dc3_mcp_connection_tool`——`replaceConnectionTools` 是 connection 聚合的操作）、`McpToolCatalogRepository`、`McpAuditLogRepository`（+ `dc3_mcp_tool_confirmation`；确切归属在实现时按聚合根规则决定）。7 张表（F3）映射为 5 个聚焦契约；方言分叉语句（JSON cast、upsert 惯用法）落在各自方言模块。响应式事务绑定的是 `ConnectionFactory` 而非 repository，因此 biz 级 `@Transactional` 可以自由跨 repository 组合。 | **已确认 2026-08-28** | +| D14 | **MySQL 驱动**——对 MySQL 服务器用 `r2dbc-mysql`（社区）还是 `r2dbc-mariadb`（官方）。 | **由 TCK spike 决定，两个候选都接好线**：MySQL 没有官方 R2DBC 驱动；社区驱动口碑良好但由志愿者维护，官方 MariaDB 驱动同样说 MySQL 协议（注意点：`caching_sha2_password`、JSON 编解码）。闸门：在 `mysql:8.4` 上通过完整 TCK 的那个留下；理由记录于此。MariaDB 方言用 `r2dbc-mariadb`（没有悬念）。 | 待定 —— P1 spike | +| D15 | **方言模块内部的查询风格。** | **默认用 Spring Data 接口 + `@Query` 原生 SQL；批量 upsert 与连接绑定操作用 `DatabaseClient`**（咨询锁——见 §8）。逐语句自由选择，在契约层之上不可见（D2）。 | **已确认 2026-08-28** | +| D16 | **共存与切换**——auth 移除 MyBatis 的时机。 | **每服务硬切换**：auth 只随 R2DBC 发布（去掉 `dc3-db-core`、dynamic-datasource、`mapping/*.xml`、`@MapperScan`）；`dc3-center-single` 在 manager/data 迁移完之前对同一数据库同时运行两套技术栈（过渡期两个连接池——已接受，有界）。 | **已确认 2026-08-28** | +| D17 | **错误转译**——SQL 异常映射。 | **`dc3-db-r2dbc-core` 中的薄助手**：`R2dbcDataIntegrityViolationException` → 既有 `DuplicateException`/`BusinessException` 的映射，在方言实现中的自然位置应用。check-then-insert 仍是主要 UX 路径（F11）；该映射是兜底，使约束冲突不再以 500 的形式暴露。 | **已确认 2026-08-28** | +| D18 | **Facade 与 gRPC 服务器边界**——auth 的非 HTTP 接缝，早期草案中缺席。auth 服务还经由 7 个阻塞的 Facade 契约（`TokenFacade.checkValid` 返回 `boolean`，另有 Permission/User/Tenant/LocalCredential/ResourceRegistry/McpRuntime）、7 个 gRPC 服务器（`grpc/*Server.java`——`StreamObserver` 回调把服务包进 `TenantContextHolder.runIgnore`）以及本地 Facade 实现暴露；消费方是网关（阻塞 gRPC stub）和每个 WebFlux 服务经 `FacadePermissionProvider` 的安全链。备选项：(a) 现在就将 Facade 契约改为响应式；(b) 保留阻塞契约并显式架桥。 | **(b)——阻塞边缘，显式且临时。**Facade 契约在本轨保持同步；响应式→阻塞桥只存在于 auth 的 gRPC 服务器与本地 Facade 实现中，在 gRPC executor / boundedElastic 线程上执行（绝不在事件循环上——防范事件循环上的 `block()` 错误），`runIgnore` 包装改为无作用域方法变体（§6）。这使试点的爆炸半径有界——选项 (a) 会把网关的 `FilterServiceImpl`、`dc3-common-facade-grpc` 和共享的 `dc3-common-web` 拖进 auth 轨道。“真正的端到端响应式”在 auth 内部覆盖 controller→repository；边缘经决策保持同步，直到 Facade 浪潮随 manager/data 落地，届时 `TokenGrpcFacade`、`FilterServiceImpl`、`FacadePermissionProvider` 得以简化。进程内 gRPC 测试继续使用阻塞的 `GrpcInProcessExtension` 装置。 | **已确认 2026-08-28** | +| D19 | **时间戳编解码**——`timestamptz` ↔ `LocalDateTime` 由谁转换（`TimestamptzLocalDateTimeTypeHandler` 的后继）。R2DBC PostgreSQL 驱动原生把 `timestamptz` 映射为 `OffsetDateTime`，而非 `LocalDateTime`；MySQL/MariaDB 的 `DATETIME` 无需转换。 | **`dc3-db-r2dbc-boot` 中的第一方 `R2dbcCustomConversions`**（D4 拆分）：为 PG 方言注册 UTC 转换器（OffsetDateTime ↔ LocalDateTime），保持 F8/D11 语义与 §9 的往返契约；MySQL/MariaDB 模块使用默认映射。显式决策，因为它是 R2DBC-PG 迁移的头号陷阱，否则会以一条无主的红色 TCK 项浮出水面。 | **已确认 2026-08-28** | -## 6. Tenant isolation on R2DBC +## 6. R2DBC 上的租户隔离 -The contract makes tenant scope a **type-level property**: +契约把租户作用域变成**类型级属性**： -- Every repository for a tenant-owned table exposes only `tenantId`-carrying methods — code that lacks a tenant cannot - compile a query, which upgrades today's runtime `TenantNotScopedException` to a compile-time guarantee. -- Repositories over the 11 tenant-less tables (F5) have no tenant parameters at all; cross-tenant reads on those tables - (e.g. user → memberships) go through explicit two-step service logic exactly as today. -- System paths (startup sync, expiry scanners, login-before-context, MCP runtime) use unscoped method variants with - explicit names — replacing `TenantContextHolder.runIgnore(...)` threading on the auth path. -- TCK ships **negative tests**: cross-tenant reads must return empty/absent on every list/get; tenant-less repositories - must not gain tenant filters; constraint violations on cross-tenant writes stay mapped (D17). +- 租户拥有表的每个 repository 只暴露携带 `tenantId` 的方法——缺租户的代码无法编译出查询，这把今天的运行期 + `TenantNotScopedException` 升级为编译期保证。 +- 覆盖 11 张无租户表（F5）的 repository 完全没有租户参数；对这些表的跨租户读取（如 user → memberships）照今天一样经过 + 显式的两步 service 逻辑。 +- 系统路径（启动同步、过期扫描器、上下文之前的登录、MCP 运行时）使用名字明示的无作用域方法变体——取代 auth 路径上的 + `TenantContextHolder.runIgnore(...)` 穿线。 +- TCK 提供**否定测试**：跨租户读取在每个 list/get 上必须返回空/不存在；无租户 repository 不得长出租户过滤；跨租户写上的 + 约束冲突保持可映射（D17）。 -`TenantContextHolder`, `TenantLineHandlerImpl` and the tenant interceptor are **not modified** — they keep serving -manager/data until their migration removes them. +`TenantContextHolder`、`TenantLineHandlerImpl` 与租户拦截器**不做修改**——它们继续服务 manager/data，直到其迁移移除它们。 -## 7. Repository contract conventions +## 7. Repository 契约约定 -- Package `io.github.pnoker.common.auth.repository`; one interface per aggregate; parameter objects are existing - BOs/Query objects and domain DOs. -- Method naming follows the project's CRUD verb policy (`add/delete/update/getById/list...`; `select*` reserved for - persistence-flavored reads — same rule as AGENTS.md, applied to the new layer). -- Return shapes: `Mono` for 0..1, `Flux` for many, `Mono>` for paged; writes return `Mono` - or the re-read entity where the business needs DB-computed values (F8 pattern). -- Transactions are declared in services/biz (as today, F9); repositories stay transaction-unaware. -- No ORM annotations are required on DOs by the contract; dialect modules may keep private `@Table` row classes and - map to domain DOs, or annotate shared DOs directly if clean — decided per aggregate during implementation, TCK is - indifferent (D2 keeps this an implementation detail). +- 包 `io.github.pnoker.common.auth.repository`；每个聚合一个接口；参数对象是既有 BO/Query 对象与领域 DO。 +- 方法命名遵循项目 CRUD 动词策略（`add/delete/update/getById/list...`；`select*` 保留给持久化风味的读取——与 AGENTS.md + 同一规则，应用于新层）。 +- 返回形状：`Mono` 表示 0..1，`Flux` 表示多个，`Mono>` 表示分页；写入返回 `Mono`，或在业务 + 需要数据库计算值时返回回读实体（F8 模式）。 +- 事务在 services/biz 中声明（同今天，F9）；repository 对事务无感知。 +- 契约不要求 DO 上有 ORM 注解；方言模块可以保留私有 `@Table` 行类再映射到领域 DO，若干净也可直接注解共享 DO——实现期间 + 按聚合决定，TCK 不关心（D2 使这一点保持为实现细节）。 -## 8. Dialect implementation conventions +## 8. 方言实现约定 -Per dialect module (`dc3-db-auth-postgres` as the reference): +每个方言模块（以 `dc3-db-auth-postgres` 为参照）： -- **Config**: `application-auth-{dialect}.yml` providing `spring.r2dbc.*` (URL template from the same - `DC3_DB_*` env family — PG `options=search_path=dc3_auth`, URL-encoded, same search-path semantics as today's - JDBC URL; MySQL/MariaDB per-service database) and `r2dbc-pool` sizing equivalent to today's Hikari settings (F13). - Profile activated by the module's EnvironmentPostProcessor, mirroring F14. Observability parity is part of the - config contract: r2dbc-pool Micrometer metrics and a connection health indicator, registered by - `dc3-db-r2dbc-boot`, replacing Hikari's metrics. -- **Standard CRUD**: Spring Data repository interfaces extending `ReactiveCrudRepository` where the shape fits. -- **Dialect-locked statements** — the statements that motivated per-dialect modules. Each dialect module writes its - native form; the table below is the migration inventory (auth-relevant): +- **配置**：`application-auth-{dialect}.yml` 提供 `spring.r2dbc.*`（URL 模板来自同一 `DC3_DB_*` 环境变量家族——PG 为 + `options=search_path=dc3_auth`，URL 编码，与今天 JDBC URL 相同的 search-path 语义；MySQL/MariaDB 为每服务一个数据库） + 与等价于今天 Hikari 设置的 `r2dbc-pool` 容量（F13）。Profile 由模块的 EnvironmentPostProcessor 激活，镜像 F14。 + 可观测性对等是配置契约的一部分：r2dbc-pool 的 Micrometer 指标与连接健康指示器，由 `dc3-db-r2dbc-boot` 注册，取代 + Hikari 的指标。 +- **标准 CRUD**：形状合适处使用扩展 `ReactiveCrudRepository` 的 Spring Data repository 接口。 +- **方言锁定语句**——正是它们催生了每方言模块。每个方言模块写出自己的原生形式；下表是迁移盘点（与 auth 相关的）： - | Concern | PostgreSQL | MySQL 8 | MariaDB | Note | - |---------|------------|---------|---------|------| - | Upsert | `INSERT ... ON CONFLICT ... DO UPDATE` | `ON DUPLICATE KEY UPDATE ... AS new` (alias form; `VALUES()` removed in 8.4) | `ON DUPLICATE KEY UPDATE ... VALUES(col)` | returning rows differs: PG `RETURNING` vs re-select (established TCK pattern) | - | Advisory lock | `pg_advisory_xact_lock(hashtext(?))` in-tx | `GET_LOCK(?, 10)` **session-level — must run on one pooled connection and pair with `RELEASE_LOCK`** | same as MySQL | R2DBC hazard: use connection-bound execution (`Mono.usingWhen` / `Connection` API) for the MySQL/MariaDB form | - | JSON columns | native JSON codec | `CAST(? AS JSON)` binding (no backslash escapes / use bind params) | no `CAST AS JSON` — plain parameter text | contract type is `String`; conversion is dialect-internal | - | String ops in catalog queries | `||`, `regexp_replace(..., 'g')` | `CONCAT`, global-by-default replace | as MySQL | from current `OAuthMcpMapper` forks | - | Pagination | `LIMIT ? OFFSET ?` | `LIMIT ?, ?` | as MySQL | count query per repository method | + | 关注点 | PostgreSQL | MySQL 8 | MariaDB | 说明 | + |--------|------------|---------|---------|------| + | upsert | `INSERT ... ON CONFLICT ... DO UPDATE` | `ON DUPLICATE KEY UPDATE ... AS new`（别名形式；`VALUES()` 在 8.4 中移除） | `ON DUPLICATE KEY UPDATE ... VALUES(col)` | 返回行的方式不同：PG `RETURNING` vs 再查询（既有 TCK 模式） | + | 咨询锁 | `pg_advisory_xact_lock(hashtext(?))` 事务内 | `GET_LOCK(?, 10)` **会话级——必须在单条池化连接上运行并与 `RELEASE_LOCK` 成对** | 同 MySQL | R2DBC 风险点：MySQL/MariaDB 形式须用连接绑定执行（`Mono.usingWhen` / `Connection` API） | + | JSON 列 | 原生 JSON 编解码 | `CAST(? AS JSON)` 绑定（不得反斜杠转义 / 用绑定参数） | 无 `CAST AS JSON`——普通参数文本 | 契约类型是 `String`；转换在方言内部完成 | + | 目录查询中的字符串操作 | `||`、`regexp_replace(..., 'g')` | `CONCAT`、默认全局替换 | 同 MySQL | 来自当前 `OAuthMcpMapper` 分叉 | + | 分页 | `LIMIT ? OFFSET ?` | `LIMIT ?, ?` | 同 MySQL | 每个 repository 方法一条 count 查询 | -- **Lock × transaction composition rule** (binds the advisory-lock row above): `pg_advisory_xact_lock` participates in - the surrounding `@Transactional` and auto-releases at commit; the MySQL/MariaDB `GET_LOCK`/`RELEASE_LOCK` pair runs - on one pooled connection via `Mono.usingWhen` (release wired to cancellation as well) and **must not execute inside - a Spring-managed transaction** — the transaction binds its own connection, and lock-connection + tx-connection under - pool pressure can starve into deadlock. Lock repository methods are transaction-exempt by contract; the TCK ships a - negative test for the pairing. -- **Snowflake IDs** are generated app-side before insert (D10) — no `RETURNING id` dependency, all dialects identical. -- **Native hints**: dialect modules register `RuntimeHints` for their row classes (`@RegisterReflectionForBinding`) - and contribute driver-specific hints; verified by the native smoke test (§10). +- **锁 × 事务组合规则**（约束上表咨询锁一行）：`pg_advisory_xact_lock` 参与外层 `@Transactional` 并在提交时自动释放； + MySQL/MariaDB 的 `GET_LOCK`/`RELEASE_LOCK` 对经 `Mono.usingWhen` 在单条池化连接上运行（释放同时挂接到取消），且**绝不 + 在 Spring 管理的事务内执行**——事务绑定自己的连接，池压力下锁连接 + 事务连接可能互相饿死成死锁。锁 repository 方法在 + 契约上豁免事务；TCK 为该配对提供否定测试。 +- **雪花 ID** 在插入前由应用侧生成（D10）——不依赖 `RETURNING id`，各方言完全一致。 +- **native hints**：方言模块为其行类注册 `RuntimeHints`（`@RegisterReflectionForBinding`），并贡献驱动专属 hints；由 + native 冒烟测试验证（§10）。 -## 9. TCK 2.0 — repository behavior contracts +## 9. TCK 2.0 —— repository 行为契约 -`dc3-db-tck` gains a second suite alongside the existing mapper contracts (which remain until manager/data migrate): +`dc3-db-tck` 在既有 mapper 契约（保留至 manager/data 迁移完）之外新增第二套套件： -- One abstract contract test per repository interface; three concrete subclasses (Postgres / MySQL / MariaDB) on - Testcontainers, fixtures identical to today's images (`postgres`, `mysql:8.4`, `mariadb:10.11`, same initdb seeds). -- Coverage classes: CRUD + logical-delete semantics; fuzzy/paged listing equivalence (incl. default `create_time DESC` - ordering and `PageUtil` clamping); upsert idempotency under the three dialect idioms; advisory-lock acquire/release - (including the MySQL/MariaDB same-connection requirement); tenant negative tests (§6); JSON round-trip; UTC - timestamp round-trip (TIMESTAMPTZ ↔ `LocalDateTime`, the current `TimestamptzLocalDateTimeTypeHandler` contract). -- Gate: new mapper/repository work must extend the contract suite — this is the discipline mechanism replacing - `databaseId` routing. +- 每个 repository 接口一个抽象契约测试；三个具体子类（Postgres / MySQL / MariaDB）跑在 Testcontainers 上，夹具与今天的 + 镜像一致（`postgres`、`mysql:8.4`、`mariadb:10.11`，同样的 initdb 种子）。 +- 覆盖类别：CRUD + 逻辑删除语义；模糊/分页列表等价性（含默认 `create_time DESC` 排序与 `PageUtil` 钳制）；三种方言惯用法 + 下的 upsert 幂等性；咨询锁获取/释放（含 MySQL/MariaDB 同连接要求）；租户否定测试（§6）；JSON 往返；UTC 时间戳往返 + （TIMESTAMPTZ ↔ `LocalDateTime`，即现有 `TimestamptzLocalDateTimeTypeHandler` 契约）。 +- 闸门：新的 mapper/repository 工作必须扩展契约套件——这就是取代 `databaseId` 路由的纪律机制。 -## 10. Native verification +## 10. Native 验证 -The pilot's exit criterion, in order: +试点的退出判据，按顺序： -1. `dc3-center-auth` builds with `mvn -Pnative` (plugin already managed in the root POM) against the R2DBC stack. -2. The native binary boots against a Testcontainers PostgreSQL, serves a representative endpoint slice - (login → token → one tenant-scoped list → one MCP tool-catalog query), and passes the TCK contract suite run - natively (allowed to be a subset — record which). -3. Image size / RSS / startup time are recorded in this document as the baseline for manager/data to beat. +1. `dc3-center-auth` 以 `mvn -Pnative`（插件已在根 POM 托管）针对 R2DBC 技术栈完成构建。 +2. native 二进制对 Testcontainers PostgreSQL 启动，服务一段有代表性的端点切片（登录 → token → 一次租户作用域列表 → + 一次 MCP 工具目录查询），并以 native 方式跑通 TCK 契约套件（允许是子集——记录是哪些）。 +3. 镜像大小 / RSS / 启动时间记录在本文档中，作为 manager/data 要超越的基线。 -If any blocking native issue surfaces (driver hints, reflection misses), it is a design-level finding: fix in -`dc3-db-r2dbc-core`/dialect modules, never by re-adding MyBatis to auth. +若浮现任何阻塞性 native 问题（驱动 hints、反射遗漏），那是设计级发现：在 `dc3-db-r2dbc-core`/方言模块中修复，绝不能用 +给 auth 重新加回 MyBatis 的方式解决。 -## 11. Transition and coexistence +## 11. 过渡与共存 -- **auth cutover is hard** (D16): one commit removes `dc3-db-core`, `dynamic-datasource`, `@MapperScan`, - `mapping/*.xml`, the 16 Manager shells, and `TenantContextHolder` usage from the auth path. No long-lived - dual-persistence inside auth. -- **`dc3-center-single`** temporarily carries both stacks (auth on R2DBC, manager/data on MyBatis) against the same - database — two pools, accepted and bounded by the transition. -- **Facade/gRPC edge stays synchronous through the transition (D18)**: the gateway and manager/data keep consuming - blocking facade contracts; the reactive→blocking bridge is confined to auth's gRPC servers and local facade impls - on non-event-loop threads, and is retired when the facade wave lands with manager/data. -- **Uncommitted MariaDB XML forks in the worktree** (`OAuthMcpMapper.xml` etc.): land or drop them independently of - this design — their *semantic* findings (CAST-AS-JSON absence, `AS new` vs `VALUES()`, GET_LOCK pairing) are already - folded into §8 and survive the XML's deletion. -- **Defect fix riding along** (D12): `UserController.add/delete`'s unmanaged principal→user→membership write gets a - service-level reactive transaction during rewrite. -- **Common reuse across services**: when manager/data migrate, they copy the module pattern - (`dc3-db-manager-{dialect}`, ...); `dc3-db-r2dbc-core` and TCK infrastructure are shared from day one. +- **auth 的切换是硬切换**（D16）：一个提交从 auth 路径移除 `dc3-db-core`、`dynamic-datasource`、`@MapperScan`、 + `mapping/*.xml`、16 个 Manager 壳与 `TenantContextHolder` 的使用。auth 内部不搞长期的双持久化。 +- **`dc3-center-single`** 临时同时承载两套技术栈（auth 在 R2DBC 上，manager/data 在 MyBatis 上）指向同一数据库——两个 + 连接池，已接受并由过渡期限定。 +- **Facade/gRPC 边缘在整个过渡期保持同步（D18）**：网关与 manager/data 继续消费阻塞的 Facade 契约；响应式→阻塞桥仅限于 + auth 的 gRPC 服务器与本地 Facade 实现、位于非事件循环线程，并在 Facade 浪潮随 manager/data 落地时退役。 +- **工作区中未提交的 MariaDB XML 分叉**（`OAuthMcpMapper.xml` 等）：无论落地还是丢弃都与本设计相互独立——它们的*语义* + 发现（缺 CAST-AS-JSON、`AS new` vs `VALUES()`、GET_LOCK 配对）已经吸收进 §8，XML 删除后依然留存。 +- **顺路修复的缺陷**（D12）：`UserController.add/delete` 中不受管理的 principal→user→membership 写入在重写时补上 + service 级响应式事务。 +- **跨服务复用公共部分**：manager/data 迁移时复制该模块模式（`dc3-db-manager-{dialect}`，……）；`dc3-db-r2dbc-core` 与 + TCK 基础设施从第一天起就是共享的。 -## 12. Phased plan and gates +## 12. 分阶段计划与闸门 -| Phase | Content | Gate | -|-------|---------|------| -| P0 | Design sign-off (this document; all §5 decisions Confirmed). | Approved doc committed. **Done 2026-08-28** (D14 deferred to its spike). | -| P1 | Skeletons: `dc3-db-r2dbc-core` (snowflake + workerId rule (D10), `PageResult`, exception mapping), `dc3-db-r2dbc-boot` (auto-config, D19 codec), module shells ×3, TCK harness extension, auto-config + fail-fast selection wiring. | `make test` green; empty contract wired end-to-end on PG. | -| P2 | **Vertical slice**: `TenantRepository` (pure CRUD + paging) and `ResourceRegistryLockRepository` (advisory lock, most dialect-sensitive) — contract → 3 dialects → TCK → reactive service → controller. | TCK 3/3 green; PG dev stack boots and serves tenant CRUD. | -| P3 | Native spike on the slice: `dc3-center-auth` `-Pnative` compile + boot + endpoint smoke. | §10 criteria on the slice. | -| P4 | Rollout by aggregate: users/principals/credentials → roles/binds/memberships → menus/APIs/resources + registry sync → service accounts/audit → OAuth/MCP (largest, last). Each aggregate lands with its TCK contracts and its migrated StepVerifier tests (Appendix A). | Full TCK green ×3; auth E2E (`dc3-e2e`) green. | -| P5 | Auth MyBatis removal (D16), `dc3-center-auth` native image as the deliverable, docs updated (`db-dialects.md`, this file's status), MySQL driver decision (D14) recorded. | Native boot; zero `mybatis` strings on auth classpath. | +| 阶段 | 内容 | 闸门 | +|------|------|------| +| P0 | 设计签核（本文档；§5 全部决策为已确认）。 | 已批准文档完成提交。**2026-08-28 完成**（D14 推迟至其 spike）。 | +| P1 | 骨架：`dc3-db-r2dbc-core`（雪花 ID + workerId 规则（D10）、`PageResult`、异常映射）、`dc3-db-r2dbc-boot`（自动配置、D19 编解码）、模块壳 ×3、TCK 装置扩展、自动配置 + fail-fast 选择接线。 | `make test` 全绿；空契约在 PG 上端到端接通。 | +| P2 | **垂直切片**：`TenantRepository`（纯 CRUD + 分页）与 `ResourceRegistryLockRepository`（咨询锁，方言最敏感）——契约 → 3 个方言 → TCK → 响应式 service → controller。 | TCK 3/3 全绿；PG 开发栈可启动并提供租户 CRUD。 | +| P3 | 在切片上做 native spike：`dc3-center-auth` `-Pnative` 编译 + 启动 + 端点冒烟。 | 切片满足 §10 判据。 | +| P4 | 按聚合推广：users/principals/credentials → roles/binds/memberships → menus/APIs/resources + registry sync → service accounts/audit → OAuth/MCP（最大，最后）。每个聚合连同其 TCK 契约与其迁移后的 StepVerifier 测试（附录 A）一起落地。 | 完整 TCK ×3 全绿；auth E2E（`dc3-e2e`）全绿。 | +| P5 | auth 移除 MyBatis（D16）、`dc3-center-auth` native image 作为交付物、文档更新（`db-dialects.md`、本文件状态）、记录 MySQL 驱动决策（D14）。 | native 启动；auth classpath 上零 `mybatis` 字样。 | -P2–P3 are the go/no-go point: if the slice proves the pattern, P4 is mechanical; if it doesn't, only the slice is -thrown away. +P2–P3 是 go/no-go 节点：如果切片证明了该模式，P4 就是机械劳动；如果证明不了，扔掉的只有切片。 -## 13. Open questions +## 13. 待解问题 -1. **D14 driver data** — actual TCK results for `r2dbc-mysql` vs `r2dbc-mariadb`-against-MySQL (P1 spike output). -2. **Row-class strategy** (§7): annotated shared DOs vs private row classes per dialect — pick one convention during - P2 and record it here. -3. **Connection pool sizing under native/R2DBC** — whether 64 max (F13) is still right for reactive demand patterns; - measure during P3. -4. **`dc3-e2e` coverage** — which auth E2E flows must exist before P4 rollout is allowed to start. -5. **History of `storage-abstraction.md`** — mark §3 superseded by this document (one-line status edit) or retire the - whole file once manager/data land (its TSDB half is already superseded). -6. **Observability parity** — exact r2dbc-pool metric set, health-indicator wiring, and the slow-query logging - convention to be implemented in `dc3-db-r2dbc-boot` (P2); measure against the Hikari baseline from F13. +1. **D14 驱动数据**——`r2dbc-mysql` 与 `r2dbc-mariadb`-against-MySQL 的真实 TCK 结果（P1 spike 产出）。 +2. **行类策略**（§7）：直接注解共享 DO，还是每方言私有行类——P2 期间定下一个约定并记录于此。 +3. **native/R2DBC 下的连接池容量**——max 64（F13）对响应式需求模式是否仍然合适；P3 期间测量。 +4. **`dc3-e2e` 覆盖**——允许 P4 推广开始之前必须存在哪些 auth E2E 流程。 +5. **`storage-abstraction.md` 的处置**——标记 §3 被本文档取代（一行状态编辑），或在 manager/data 落地后退役整个文件 + （其 TSDB 一半已被取代）。 +6. **可观测性对等**——`dc3-db-r2dbc-boot`（P2）中要实现的确切 r2dbc-pool 指标集、健康指示器接线与慢查询日志约定；对照 + F13 的 Hikari 基线测量。 -## Appendix A — auth migration inventory +## 附录 A —— auth 迁移盘点 -- **Mappers (18)**: 16 `BaseMapper` shells (map 1:1 to 15 empty Managers + `IdentityAuditLogManager`) → standard - repository contracts; `OAuthMcpMapper` (30 methods, 7 tables) → 5 aggregate repositories (D13); - `ResourceRegistryLockMapper` (advisory lock) → `ResourceRegistryLockRepository`. -- **`@Transactional` sites (19)**: enumerated in F9; all carry to D12. -- **Dialect-forked statements in auth**: `ResourceRegistryLockMapper.xml` (lock, 3 forms), `OAuthMcpMapper.xml` - (JSON casts ×4, catalog string ops ×2, upsert idioms ×2) — semantics table in §8. -- **Services to rewrite reactive**: 12 paged/fuzzy Services + `OAuthMcpRuntimeServiceImpl` + - `ResourceRegistrySyncServiceImpl` + `DictionaryForAuthService` (reads `dc3_tenant`, F12). -- **Facade & gRPC servers (the edge, D18)**: the 7 facade contracts stay blocking; the 7 gRPC servers - (`TokenServer`, `PermissionServer`, `UserServer`, `TenantServer`, `LocalCredentialServer`, - `ResourceRegistryServer`, `McpRuntimeServer`) and the 7 local facade impls gain the explicit reactive→blocking - bridge (non-event-loop threads) and swap `runIgnore` wrappers for unscoped method variants (§6). -- **Tests (17 files)**: blocking service/biz, controller, and gRPC tests (e.g. `TokenServiceImplTest`, - `OAuthMcpRuntimeServiceImplTest`, `TokenServerTest`, `McpRuntimeServerTest`) rewritten with `StepVerifier`; - in-process gRPC tests keep `GrpcInProcessExtension` against the D18 blocking edge; ThreadLocal-based context - tests migrate to explicit-parameter assertions. -- **Not in scope**: `dc3-common-dal` (F12), all manager/data/agentic DAL, DDL/initdb, `dc3-db-core` and existing - dialect modules (F15). +- **Mapper（18 个）**：16 个 `BaseMapper` 壳（与 15 个空 Manager + `IdentityAuditLogManager` 一一对应）→ 标准 repository + 契约；`OAuthMcpMapper`（30 个方法，7 张表）→ 5 个聚合 repository（D13）；`ResourceRegistryLockMapper`（咨询锁）→ + `ResourceRegistryLockRepository`。 +- **`@Transactional` 位置（19 处）**：F9 已列举；全部平移至 D12。 +- **auth 中的方言分叉语句**：`ResourceRegistryLockMapper.xml`（锁，3 种形式）、`OAuthMcpMapper.xml`（JSON cast ×4、目录 + 字符串操作 ×2、upsert 惯用法 ×2）——语义表见 §8。 +- **待重写为响应式的 Service**：12 个分页/模糊 Service + `OAuthMcpRuntimeServiceImpl` + `ResourceRegistrySyncServiceImpl` + + `DictionaryForAuthService`（读 `dc3_tenant`，F12）。 +- **Facade 与 gRPC 服务器（边缘，D18）**：7 个 Facade 契约保持阻塞；7 个 gRPC 服务器（`TokenServer`、`PermissionServer`、 + `UserServer`、`TenantServer`、`LocalCredentialServer`、`ResourceRegistryServer`、`McpRuntimeServer`）与 7 个本地 Facade + 实现加上显式响应式→阻塞桥（非事件循环线程），并把 `runIgnore` 包装换成无作用域方法变体（§6）。 +- **测试（17 个文件）**：阻塞式 service/biz、controller 与 gRPC 测试（如 `TokenServiceImplTest`、 + `OAuthMcpRuntimeServiceImplTest`、`TokenServerTest`、`McpRuntimeServerTest`）以 `StepVerifier` 重写；进程内 gRPC 测试 + 针对 D18 阻塞边缘继续使用 `GrpcInProcessExtension`；基于 ThreadLocal 的上下文测试迁移为显式参数断言。 +- **不在范围内**：`dc3-common-dal`（F12）、manager/data/agentic 的全部 DAL、DDL/initdb、`dc3-db-core` 与既有方言模块 + （F15）。 diff --git a/docs/design/storage-abstraction.md b/docs/design/storage-abstraction.md index 7fde3e6a5..257e8e7db 100644 --- a/docs/design/storage-abstraction.md +++ b/docs/design/storage-abstraction.md @@ -1,41 +1,36 @@ -# Design: Pluggable Storage Abstraction (Relational Dialect + Time-Series Port) +# 设计：可插拔存储抽象（关系库方言 + 时序 Port） -| | | -|----------------|---------------------------------------------------------------------------------------------------------------------------------| -| **Status** | Implemented — R1/R2 landed (dual-dialect seed + forks + contract suite; selection guide at [db-dialects.md](../db-dialects.md)) | -| **Date** | 2026-08-17 | -| **Revised** | 2026-08-19 — inventory refreshed after the driver-lease commit (956de3dd3); §6.1 added | -| **Scope** | persistence layer: relational core + point-value time-series store | -| **Target** | Relational: PostgreSQL (default), MySQL 8 — Time-series: TimescaleDB (default), TDengine, InfluxDB, IoTDB | -| **Related** | [`mq-abstraction.md`](./mq-abstraction.md) — the third pluggable dimension | -| **Discussion** | open for review before implementation starts | +| | | +|------------|---------------------------------------------------------------------------------------------------------------------------------| +| **状态** | 已实现 —— R1/R2 已落地（双方言种子 + 方言分支 + 契约套件；选型指南见 [db-dialects.md](../db-dialects.md)） | +| **日期** | 2026-08-17 | +| **修订** | 2026-08-19 —— 驱动租约提交（956de3dd3）之后刷新盘点；新增 §6.1 | +| **范围** | 持久层：关系库核心 + 位值时序存储 | +| **目标** | 关系库：PostgreSQL（默认）、MySQL 8 —— 时序：TimescaleDB（默认）、TDengine、InfluxDB、IoTDB | +| **相关** | [`mq-abstraction.md`](./mq-abstraction.md) —— 第三个可插拔维度 | +| **讨论** | 实施启动前开放评审 | -## 1. Summary +## 1. 摘要 -IoT DC3 should let a deployer pick the relational database, the time-series store, and the message broker -**independently** — any combination must be a supported deployment, e.g. -`MySQL + TDengine + RocketMQ` for one company, `PostgreSQL + TimescaleDB + Kafka` for another. This document designs the -two storage dimensions; the broker dimension is covered by the companion MQ design. +IoT DC3 应当允许部署者**独立地**选择关系数据库、时序存储与消息中间件——任意组合都必须是一种受支持的部署形态，例如 +一家公司用 `MySQL + TDengine + RocketMQ`，另一家用 `PostgreSQL + TimescaleDB + Kafka`。本文设计其中两个存储维度； +消息中间件维度由姊妹篇 MQ 设计覆盖。 -The key structural insight: today TimescaleDB runs as an **extension inside the main PostgreSQL instance** (single `dc3` -database, single datasource, `dc3_point_value` -hypertable in the `dc3_history` schema), so "replace the time-series store" currently has no deployment boundary to swap -at. But the code is closer to ready than the deployment: -a `RepositoryService` port already exists (`dc3-common-repository`) and carries almost the entire point-value surface — -but the dashboard read surface still bypasses it. The work is therefore: +关键的结构性判断：如今 TimescaleDB 作为**扩展运行在主 PostgreSQL 实例内部**（单一 `dc3` 数据库、单一数据源， +`dc3_history` schema 中的 `dc3_point_value` +超表），因此"替换时序存储"目前在部署上没有可供切换的边界。但代码比部署更接近就绪： +一个 `RepositoryService` port（`dc3-common-repository`）已经存在，承载了几乎整个位值操作面—— +但看板读取面仍在绕开它。因此要做的工作是： -- **Relational layer** — not a new port: MyBatis *is* the port. What is needed is a **dialect mechanism**: a - dialect-neutral infra module, portable-first SQL policy, a - `databaseId` fork for the genuinely dialect-bound statements (9 today, across 4 mappers — concentrated in the - driver-lease subsystem), and per-dialect seed SQL. Only 7 of 51 mapper files contain PostgreSQL-specific syntax today. -- **Time-series layer** — promote the existing `RepositoryService` port to a real swappable boundary: extract the - implementation into per-store modules, fold the dashboard read surface (≈10 bypass statements) into the port, - neutralize the SQL-flavored parts of the interface (pagination, aggregate semantics), and add per-store adapters. -- Both layers share the same mechanism family as the MQ design: **profiles + capability matrix + TCK**, giving the - project one consistent pluggability story: - `dc3.facade.mode` (today) → `dc3.db.type`, `dc3.repository.type`, `dc3.mq.type`. +- **关系库层** —— 不需要新的 port：MyBatis *就是* port。需要的是一套**方言机制**：一个方言中立的基建模块、可移植优先的 + SQL 策略、针对真正绑定方言语句的 `databaseId` 方言分支（当前 9 条，分布在 4 个 mapper——集中于驱动租约子系统）， + 以及按方言划分的种子 SQL。当前 51 个 mapper 文件中只有 7 个包含 PostgreSQL 特有语法。 +- **时序层** —— 把既有的 `RepositoryService` port 提升为真正可切换的边界：把实现抽取到按库划分的模块、把看板读取面 + （约 10 条旁路语句）收编进 port、中和接口中 SQL 风味的部分（分页、聚合语义），并增加按库的适配器。 +- 两个层与 MQ 设计共享同一机制家族：**profile + 能力矩阵 + TCK**，让项目拥有一个一致的可插拔叙事： + `dc3.facade.mode`（现状）→ `dc3.db.type`、`dc3.repository.type`、`dc3.mq.type`。 -## 2. The three-layer storage model +## 2. 三层存储模型 ``` ┌─────────────────────────────────────────────────────────────────┐ @@ -52,57 +47,56 @@ but the dashboard read surface still bypasses it. The work is therefore: └─────────────────────────────────────────────────────────────────┘ ``` -Orthogonality matrix — every cell is a valid deployment: +正交性矩阵——每个单元格都是一种有效部署： | | TimescaleDB (in-PG) | TDengine | InfluxDB | IoTDB | |---------------------|---------------------|----------|----------|-------| -| **PostgreSQL core** | ✅ today's default | ✅ | ✅ | ✅ | -| **MySQL core** | n/a (PG-only) | ✅ | ✅ | ✅ | +| **PostgreSQL 核心** | ✅ 当前默认 | ✅ | ✅ | ✅ | +| **MySQL 核心** | 不适用（仅 PG） | ✅ | ✅ | ✅ | -Note the bottom-left cell: with an external time-series store, the relational database loses its only heavy -PostgreSQL-only feature (the hypertable), and `timescale` collapses into `postgres`. The two migration tracks are -independent and mutually simplifying. +注意左下角单元格：时序存储一旦外置，关系数据库就失去了它唯一重度依赖 PostgreSQL 的特性（超表）， +`timescale` 也随之坍缩为 `postgres`。两条迁移轨道相互独立，且互为简化。 -## 3. Layer 1 — Relational dialect +## 3. 第 1 层 —— 关系库方言 -### 3.1 Current coupling (verified inventory) +### 3.1 现状耦合（已核实的盘点） -**51 mapper XML files** across `dc3-common-auth` (18), `dc3-common-dal` (4), -`dc3-common-data` (11), `dc3-common-manager` (18). **7 contain PostgreSQL-specific syntax** — the 2026-08-18 -driver-lease commit (`956de3dd3`) added one file (`DriverLeaseMapper`) and converted another (`PointValueMapper`: its +**51 个 mapper XML 文件**，分布于 `dc3-common-auth`（18）、`dc3-common-dal`（4）、 +`dc3-common-data`（11）、`dc3-common-manager`（18）。其中 **7 个包含 PostgreSQL 特有语法**——2026-08-18 的 +驱动租约提交（`956de3dd3`）新增了一个文件（`DriverLeaseMapper`），并改造了另一个（`PointValueMapper`：其 `DISTINCT ON` -batch-latest statement was replaced by `ON CONFLICT` upserts): +批量最新值语句已被 `ON CONFLICT` upsert 取代）： -| File | Syntax | Semantics | +| 文件 | 语法 | 语义 | |------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------| -| `dc3-common-data/.../EntityStateMapper.xml` | 2 stmts: `ON CONFLICT ... RETURNING` (`upsertEntityState`, + `json_build_object`), `UPDATE ... FROM ... RETURNING` (`claimExpiredDevices`) | entity-state upsert + expired-lease claim, both returning rows (driver heartbeat hot path) | -| `dc3-common-data/.../PointValueMapper.xml` | 2 stmts: `ON CONFLICT DO NOTHING ... RETURNING`, `ON CONFLICT DO UPDATE` | idempotent batch insert + latest-value upsert (durable telemetry path) | -| `dc3-common-data/.../DashboardMapper.xml` | `time_bucket(INTERVAL ...)` | dashboard time-series aggregation (TimescaleDB) — belongs to Layer 2, see §4.2 | -| `dc3-common-data/.../AlertMapper.xml` | `COUNT(*) FILTER (WHERE)` ×3, `generate_series` + `date_trunc` | alert stats + calendar-day spine | -| `dc3-common-auth/.../ResourceRegistryLockMapper.xml` | `pg_advisory_xact_lock(hashtext())` | registry distributed lock | -| `dc3-common-auth/.../OAuthMcpMapper.xml` | `::text` casts | parameter null-check idiom | -| `dc3-common-manager/.../DriverLeaseMapper.xml` | 4 stmts: `pg_advisory_xact_lock`, `ON CONFLICT ... DO UPDATE ... RETURNING` ×3 | driver lease acquire / renew / device-claim (hot path, new in `956de3dd3`) | +| `dc3-common-data/.../EntityStateMapper.xml` | 2 条语句：`ON CONFLICT ... RETURNING`（`upsertEntityState`，含 `json_build_object`）、`UPDATE ... FROM ... RETURNING`（`claimExpiredDevices`） | 实体状态 upsert + 过期租约认领，均返回行（驱动心跳热路径） | +| `dc3-common-data/.../PointValueMapper.xml` | 2 条语句：`ON CONFLICT DO NOTHING ... RETURNING`、`ON CONFLICT DO UPDATE` | 幂等批量插入 + 最新值 upsert（持久遥测路径） | +| `dc3-common-data/.../DashboardMapper.xml` | `time_bucket(INTERVAL ...)` | 看板时序聚合（TimescaleDB）——属于第 2 层，见 §4.2 | +| `dc3-common-data/.../AlertMapper.xml` | `COUNT(*) FILTER (WHERE)` ×3、`generate_series` + `date_trunc` | 告警统计 + 日历日骨架 | +| `dc3-common-auth/.../ResourceRegistryLockMapper.xml` | `pg_advisory_xact_lock(hashtext())` | 注册表分布式锁 | +| `dc3-common-auth/.../OAuthMcpMapper.xml` | `::text` 类型转换 | 参数判空惯用法 | +| `dc3-common-manager/.../DriverLeaseMapper.xml` | 4 条语句：`pg_advisory_xact_lock`、`ON CONFLICT ... DO UPDATE ... RETURNING` ×3 | 驱动租约获取 / 续期 / 设备认领（热路径，`956de3dd3` 新增） | -**Infrastructure module** `dc3-common-postgres` — hardcodes `org.postgresql.Driver`, -`PaginationInnerInterceptor(DbType.POSTGRE_SQL)`, `TimestamptzLocalDateTimeTypeHandler`, and the `postgres` profile -activation. Every business module depends on it. +**基建模块** `dc3-common-postgres` —— 硬编码了 `org.postgresql.Driver`、 +`PaginationInnerInterceptor(DbType.POSTGRE_SQL)`、`TimestamptzLocalDateTimeTypeHandler`，以及 `postgres` profile +的激活。每个业务模块都依赖它。 -**DDL specifics** (seed `dc3/dependencies/postgres/initdb/`): `TIMESTAMPTZ` columns throughout; 5 schemas (`dc3_auth`, -`dc3_data`, `dc3_manager`, `dc3_history`, -`dc3_agentic`) resolved via `search_path` — mapper table names are conveniently **unqualified**; `JSON` columns -(portable type, not `jsonb`), defaulted as -`'{}'::JSON` — MySQL has no literal JSON defaults, its seed uses expression defaults (`DEFAULT ('{}')`, requires -8.0.13+). **plpgsql trigger functions** are additional MySQL-incompatible seed logic: a trivial `update_operate_time()` -per schema (MySQL: -`ON UPDATE CURRENT_TIMESTAMP`), and — from the driver-lease commit — three -`track_driver_device_revision_*` statement-level triggers using transition tables and -`INSERT ... ON CONFLICT` (MySQL has neither statement-level triggers nor transition tables; this revision-counter logic -must move into the application layer or be re-expressed). Extensions installed: `timescaledb` (used — Layer 2), `vector` -(installed, **no columns use it**), `age` (installed; no `cypher()` usage exists in any mapper — see open questions). +**DDL 细节**（种子位于 `dc3/dependencies/postgres/initdb/`）：全库使用 `TIMESTAMPTZ` 列；5 个 schema（`dc3_auth`、 +`dc3_data`、`dc3_manager`、`dc3_history`、 +`dc3_agentic`）通过 `search_path` 解析——mapper 表名因此得以**不带限定**；`JSON` 列 +（可移植类型，而非 `jsonb`），默认值为 +`'{}'::JSON` —— MySQL 不支持字面量 JSON 默认值，其种子改用表达式默认值（`DEFAULT ('{}')`，要求 +8.0.13+）。**plpgsql 触发器函数**是另一部分与 MySQL 不兼容的种子逻辑：每个 schema 一个简单的 `update_operate_time()` +（MySQL 对应： +`ON UPDATE CURRENT_TIMESTAMP`），以及——来自驱动租约提交——三个 +使用过渡表（transition table）和 `INSERT ... ON CONFLICT` 的 `track_driver_device_revision_*` 语句级触发器 +（MySQL 既没有语句级触发器，也没有过渡表；这套修订计数逻辑必须上移到应用层或重新表达）。已安装的扩展：`timescaledb` +（在用——第 2 层）、`vector` +（已安装，**没有任何列使用它**）、`age`（已安装；任何 mapper 中都不存在 `cypher()` 用法——见开放问题）。 -### 3.2 Design +### 3.2 设计 -**Module split** — same family as facade/MQ profiles: +**模块拆分** —— 与 facade/MQ profile 同一家族： ``` dc3-common-postgres ──► dc3-common-jdbc # dialect-neutral: TenantLineHandler, @@ -111,93 +105,89 @@ dc3-common-postgres ──► dc3-common-jdbc # dialect-neutral: TenantLine dc3-db-mysql # driver, DbType, DATETIME(6) mapping ``` -Selected by `dc3.db.type` via `@ConditionalOnProperty`, the established -`dc3.facade.mode` pattern. +由 `dc3.db.type` 通过 `@ConditionalOnProperty` 选择，沿用既有的 +`dc3.facade.mode` 模式。 -**Portable-first SQL policy.** The default action for a PG-specific statement is a portable rewrite that performs -equivalently on PostgreSQL — the fork is reserved for statements where portability costs semantics or performance: +**可移植优先的 SQL 策略。** 对 PG 特有语句的默认处理，是改写为一种在 PostgreSQL 上表现相当的等价可移植形式—— +方言分支仅保留给那些可移植化会损失语义或性能的语句： -| PG-only syntax | Portable rewrite (runs on both) | Fork needed? | +| PG 特有语法 | 可移植改写（两库皆可运行） | 是否需要方言分支？ | |---------------------------------|--------------------------------------|----------------------------------------------------------------------| -| `COUNT(*) FILTER (WHERE x)` | `SUM(CASE WHEN x THEN 1 ELSE 0 END)` | no | -| `::text` casts | drop / rewrite null-check idiom | no | -| `generate_series` date spine | recursive CTE (MySQL 8 ✔, PG ✔) | no | -| `ON CONFLICT ... RETURNING` | — | **fork**: `INSERT ... ON DUPLICATE KEY UPDATE` + same-tx re-select | -| `UPDATE ... FROM ... RETURNING` | — | **fork**: MySQL has no `RETURNING`; multi-table `UPDATE` + re-select | -| `pg_advisory_xact_lock` | — | **fork**: `GET_LOCK()` / lock-table unique constraint | -| `time_bucket(...)` | — | **moves to Layer 2** (TSDB adapter concern) | +| `COUNT(*) FILTER (WHERE x)` | `SUM(CASE WHEN x THEN 1 ELSE 0 END)` | 否 | +| `::text` 类型转换 | 去掉 / 改写参数判空惯用法 | 否 | +| `generate_series` 日期骨架 | 递归 CTE（MySQL 8 ✔、PG ✔） | 否 | +| `ON CONFLICT ... RETURNING` | — | **方言分支**：`INSERT ... ON DUPLICATE KEY UPDATE` + 同事务 re-select | +| `UPDATE ... FROM ... RETURNING` | — | **方言分支**：MySQL 没有 `RETURNING`；多表 `UPDATE` + re-select | +| `pg_advisory_xact_lock` | — | **方言分支**：`GET_LOCK()` / 锁表唯一约束 | +| `time_bucket(...)` | — | **移入第 2 层**（TSDB 适配器关注点） | -(`DISTINCT ON` → `ROW_NUMBER()` was a planned portable rewrite until 2026-08-18; the statement it applied to was -replaced by `ON CONFLICT` upserts and is now a fork — see §3.1.) +（`DISTINCT ON` → `ROW_NUMBER()` 曾是计划中的可移植改写，直到 2026-08-18；它所作用的语句已被 +`ON CONFLICT` upsert 取代，如今成为一条方言分支——见 §3.1。） -Forks use MyBatis' native `databaseIdProvider`: ``。可移植改写完成后的预期分支面： +**9 条语句、分布在 4 个 mapper**—— +`EntityStateMapper` ×2、`PointValueMapper` ×2、`DriverLeaseMapper` ×4、 +`ResourceRegistryLockMapper` ×1。驱动租约子系统（租约获取/续期、持久遥测、修订触发器） +如今是最大的单一 PG 惯用语法簇；其语义可移植，只是表达方式不可移植。 -**Schema → database.** The 5 PG schemas map to 5 MySQL databases; because mapper table names are unqualified, the -connection's selected database plays the role `search_path` -plays today. No mapper changes. +**schema → 数据库。** 5 个 PG schema 映射为 5 个 MySQL 数据库；由于 mapper 表名不带限定， +连接所选定的数据库就扮演了如今 `search_path` +的角色。mapper 无需任何改动。 -**Time semantics.** `TIMESTAMPTZ` → `DATETIME(6)` plus an explicit **store-and-transfer-in-UTC** convention (already -implicit in -`TimestamptzLocalDateTimeTypeHandler`); documented and asserted in the TCK. +**时间语义。** `TIMESTAMPTZ` → `DATETIME(6)`，外加一条显式的**以 UTC 存储与传输**的约定（在 +`TimestamptzLocalDateTimeTypeHandler` 中已是隐式做法）；该约定写入文档并在 TCK 中以断言锁定。 -**Seed SQL.** Per-dialect directories, backend-owned as today: -`dc3/dependencies/postgres/initdb/` (existing) and `dc3/dependencies/mysql/initdb/`. The `dc3_point_value` DDL and all -TimescaleDB statements move to the `timescale` -repository adapter (§4) — with an external TSDB the relational seed contains no extension DDL at all. Seed-side -divergences beyond column types: the plpgsql trigger functions (§3.1) — `update_operate_time()` maps to -`ON UPDATE CURRENT_TIMESTAMP`, the driver-device revision triggers need re-implementation (application layer or MySQL -row-level triggers) — and `'{}'::JSON` defaults map to expression defaults. +**种子 SQL。** 按方言划分目录，与今天一样由后端仓库维护： +`dc3/dependencies/postgres/initdb/`（既有）与 `dc3/dependencies/mysql/initdb/`。`dc3_point_value` 的 DDL 与全部 +TimescaleDB 语句移入 `timescale` +仓储适配器（§4）——时序库外置后，关系种子中完全不包含任何扩展 DDL。列类型之外的种子侧差异：plpgsql 触发器函数 +（§3.1）——`update_operate_time()` 映射为 +`ON UPDATE CURRENT_TIMESTAMP`，驱动设备修订触发器需要重新实现（应用层或 MySQL +行级触发器）——以及 `'{}'::JSON` 默认值映射为表达式默认值。 -**Hard requirement:** MySQL 8.0+ (window functions, CTEs, recursive CTEs). MariaDB 10.5+ follows for free; older MySQL -is out of scope. +**硬性要求：** MySQL 8.0+（窗口函数、CTE、递归 CTE）。MariaDB 10.5+ 顺带即可支持；更老的 MySQL 不在范围内。 -### 3.3 Relational TCK +### 3.3 关系库 TCK -`make test-it` (DAL integration tests) executed against two Testcontainers images (`postgres` and `mysql:8`) with -identical fixtures; identical results = compliant. CI runs both — this is also the discipline mechanism that keeps new -mappers portable or explicitly forked. +`make test-it`（DAL 集成测试）针对两个 Testcontainers 镜像（`postgres` 与 `mysql:8`）以完全相同的夹具执行； +结果一致即视为合规。CI 两个都跑——这也是确保新增 mapper 保持可移植、或显式走分支的纪律机制。 -## 4. Layer 2 — Time-series store (`RepositoryService` port) +## 4. 第 2 层 —— 时序存储（`RepositoryService` port） -> **Superseded 2026-08-20**: the time-series track now has its own full design at -> [`tsdb-abstraction.md`](./tsdb-abstraction.md) (standalone-store targets, TCK gate, -> latest-value-stays-relational decision). The sketch below is kept for history. +> **已于 2026-08-20 被取代**：时序轨道现在拥有自己的完整设计，见 +> [`tsdb-abstraction.md`](./tsdb-abstraction.md)（独立存储目标、TCK 门槛、 +> 最新值留在关系库的决策）。下面的草稿仅作历史记录保留。 -### 4.0 (historical sketch) +### 4.0（历史草稿） -### 4.1 What exists today +### 4.1 今天的现状 -- **Port**: `dc3-common-repository` defines `RepositoryService` — 8 operations that cover the entire point-value domain: - `savePointValue(s)`, - `listHistoryPointValue`, `selectLatestPointValue`, `listLatestPointValues`, - `listPagePointValue`, `aggregateInWindow`, `samplesInWindow` — plus - `RepositoryStrategyFactory` (named-strategy registry) and `ActiveRepositoryProfileConfig`. -- **Single implementation**: `PostgresRepositoryServiceImpl` (inside - `dc3-common-data`), registered under `StrategyConstant.Storage.POSTGRES`; treats the TimescaleDB hypertable as a plain - PG table via `PointValueManager`/`PointValueMapper`. -- **Flow**: `PointValueReceiver` (MQ) → `PointValueIngestBuffer` → - `PointValueServiceImpl` → `PointValueLocalCacheService` (latest-value cache) + - `RepositoryService` (persistence). Alarm long-window evaluation reads via - `RepositoryWindowDataSource` → the same port. -- **Deployment**: TimescaleDB is an in-PG extension — single instance, single datasource, no swap boundary. +- **Port**：`dc3-common-repository` 定义了 `RepositoryService`——覆盖位值全域的 8 个操作： + `savePointValue(s)`、 + `listHistoryPointValue`、`selectLatestPointValue`、`listLatestPointValues`、 + `listPagePointValue`、`aggregateInWindow`、`samplesInWindow`——外加 + `RepositoryStrategyFactory`（命名策略注册表）与 `ActiveRepositoryProfileConfig`。 +- **单一实现**：`PostgresRepositoryServiceImpl`（位于 + `dc3-common-data` 内），以 `StrategyConstant.Storage.POSTGRES` 注册；通过 `PointValueManager`/`PointValueMapper` + 把 TimescaleDB 超表当作普通 PG 表处理。 +- **数据流**：`PointValueReceiver`（MQ）→ `PointValueIngestBuffer` → + `PointValueServiceImpl` → `PointValueLocalCacheService`（最新值缓存）+ + `RepositoryService`（持久化）。告警长窗口评估通过 + `RepositoryWindowDataSource` 读取——同样走这个 port。 +- **部署形态**：TimescaleDB 是 PG 内的扩展——单实例、单数据源，没有可切换的边界。 -### 4.2 Gaps to close before the store is truly swappable +### 4.2 存储真正可换之前需要补齐的缺口 -1. **The dashboard read surface bypasses the port.** The data-side - `DashboardMapper.xml` reads `dc3_point_value` directly in **9 of its 10 statements** - (`countInRange`, `countTotal`, `timeseries`, `top`, `latestStream`, - `latencyHistogram`, `hourlyActivity`, `silentSources`, `coverageGapItems`); the manager-side `DashboardMapper.xml` - adds a 10th (`FROM dc3_history.dc3_point_value`). Only `timeseries` uses `time_bucket` — the others are portable SQL, - but portable or not they all break the moment the point-value store is external. And several are richer than a - bucketed aggregation (latency histogram bins, silent-source detection, coverage gaps, activity grid): designing the - neutral read port — or re-expressing these dashboards on top of port primitives — is the largest single T1 workload, - not a one-statement move (open question §8). -2. **Extract the implementation** into per-store modules; selection by - `dc3.repository.type`: +1. **看板读取面绕开了 port。** 数据侧 + `DashboardMapper.xml` 的 **10 条语句中有 9 条**直接读 `dc3_point_value` + （`countInRange`、`countTotal`、`timeseries`、`top`、`latestStream`、 + `latencyHistogram`、`hourlyActivity`、`silentSources`、`coverageGapItems`）；manager 侧的 `DashboardMapper.xml` + 又加了第 10 条（`FROM dc3_history.dc3_point_value`）。只有 `timeseries` 用到 `time_bucket`——其余都是可移植 SQL， + 但无论可移植与否，位值存储一旦外置它们全会失效。而且其中几条比分桶聚合更复杂（延迟直方图分箱、静默源检测、 + 覆盖缺口、活跃度网格）：设计中立的读取 port——或基于 port 原语重新表达这些看板——是 T1 中最大的一块单独工作量， + 不是挪一条语句了事（开放问题 §8）。 +2. **把实现抽取**为按库划分的模块；由 + `dc3.repository.type` 选择： ``` dc3-repository-timescale # today's PostgresRepositoryServiceImpl, moved @@ -206,44 +196,41 @@ mappers portable or explicitly forked. dc3-repository-iotdb # Apache IoTDB adapter ``` -3. **Neutralize the SQL-flavored surface** of the port: `Page` (a MyBatis-Plus type) - and SQL aggregate semantics leak through. Design target: a cursor-friendly pagination abstraction (time-descending - cursor; offset optional capability) and a store-neutral aggregate enum — TDengine `INTERVAL`, InfluxDB 3 SQL and - IoTDB all express AVG/MIN/MAX/SUM/COUNT windows, so the enum maps cleanly. -4. **Data-model mapping** per adapter — tenant isolation rides the store's native dimension (tenant safety rule - preserved): +3. **中和 port 的 SQL 风味表面**：`Page`（一个 MyBatis-Plus 类型） + 与 SQL 聚合语义从 port 泄漏出去。设计目标：一个对游标友好的分页抽象（按时间降序的游标；offset 作为可选能力） + 与一个库中立的聚合枚举——TDengine `INTERVAL`、InfluxDB 3 SQL 与 + IoTDB 都能表达 AVG/MIN/MAX/SUM/COUNT 窗口，因此该枚举可以干净映射。 +4. 每个适配器的**数据模型映射**——租户隔离依托各存储的原生维度（租户安全规则得以保留）： - | Concept | Timescale | TDengine 3 | InfluxDB 3 | IoTDB | + | 概念 | Timescale | TDengine 3 | InfluxDB 3 | IoTDB | |---------|-----------|------------|------------|-------| - | series identity | (tenant_id, device_id, point_id) PK | super table + tags `tenant/device/point` | measurement + tags | path `tenant.device.point` | - | tenant scoping | WHERE tenant_id | tag filter (indexed) | tag filter | path prefix filter | - | latest value | index / `DISTINCT ON` | `LAST()` | `last()` / SQL | `last` query | - | window aggregate | `time_bucket` | `INTERVAL(...)` | SQL `date_bin` | group-by-time window | - | retention | drop-chunks policy / compression | `KEEP` per database | retention policy | TTL | - | write path | SQL batch | schemaless / STABLE batch insert | line protocol / SQL | session batch insert | + | 序列标识 | (tenant_id, device_id, point_id) 主键 | 超级表 + 标签 `tenant/device/point` | measurement + 标签 | 路径 `tenant.device.point` | + | 租户作用域 | WHERE tenant_id | 标签过滤（有索引） | 标签过滤 | 路径前缀过滤 | + | 最新值 | 索引 / `DISTINCT ON` | `LAST()` | `last()` / SQL | `last` 查询 | + | 窗口聚合 | `time_bucket` | `INTERVAL(...)` | SQL `date_bin` | 按时间分组的窗口 | + | 保留策略 | drop-chunks 策略 / 压缩 | 每数据库 `KEEP` | retention policy | TTL | + | 写入路径 | SQL 批量 | schemaless / STABLE 批量插入 | line protocol / SQL | session 批量插入 | -5. **Retention & lifecycle** as declared capabilities (timescale compression, TDengine `KEEP`, Influx retention, IoTDB - TTL) rather than hidden DDL. -6. **Deployment topology**: an external TSDB is a second datasource with its own compose service and profile - (`dc3/dependencies//`, image per store, mirroring the broker-per-adapter pattern). With an external store the - main PG no longer needs TimescaleDB at all. +5. **保留与生命周期**作为显式声明的能力（timescale 压缩、TDengine `KEEP`、Influx retention、IoTDB + TTL），而不是隐藏的 DDL。 +6. **部署拓扑**：外置 TSDB 是第二个数据源，拥有自己的 compose 服务与 profile + （`dc3/dependencies//`，每个库一个镜像，与每个适配器配一个 broker 的模式一致）。有了外置存储， + 主 PG 完全不再需要 TimescaleDB。 -### 4.3 Time-series TCK +### 4.3 时序 TCK -One suite, N containers: write a fixed fixture of point values, then assert +一套套件，N 个容器：写入固定的位值夹具，然后断言 `listHistoryPointValue` / `listLatestPointValues` / `aggregateInWindow` / -`samplesInWindow` return equivalent results; tenant-isolation negative tests (cross-tenant reads return nothing); -retention test (expired data disappears). Passing the TCK is the acceptance bar for community stores (e.g. someone -bringing Cassandra or ClickHouse adapters later). +`samplesInWindow` 返回等价结果；租户隔离反向测试（跨租户读取不返回任何数据）； +保留测试（过期数据消失）。通过 TCK 是社区存储的验收门槛（例如日后有人带来 Cassandra 或 ClickHouse 适配器）。 -## 5. Layer 3 — Vector (placeholder, capability-negotiated) +## 5. 第 3 层 —— 向量（占位，按能力协商） -`pgvector` is installed but unused — no `vector` columns exist. When the Agentic Center adds embeddings, the decision -point arrives: vector search as a **PostgreSQL-only capability** (feature-flagged via capability negotiation, MySQL -profile degrades to external store or no vector search) or as a fourth pluggable dimension. Not designed now; recorded -so the choice is conscious when it happens. +`pgvector` 已安装但未使用——不存在任何 `vector` 列。当 Agentic Center 引入 embedding 时，决策点就会到来： +向量检索是作为 **PostgreSQL 专属能力**（通过能力协商加特性开关，MySQL profile 退化为外置存储或无向量检索）， +还是作为第四个可插拔维度。现在不做设计；记录在此，是为了届时这是一个有意识的选择。 -## 6. Configuration surface (unified across all pluggable dimensions) +## 6. 配置面（跨所有可插拔维度统一） ```yaml dc3: @@ -257,113 +244,101 @@ dc3: type: rabbitmq # see mq-abstraction.md ``` -One mechanism for all: `@ConditionalOnProperty` profile selection, capability matrix published per dimension, startup -log summarizing the negotiated combination. Compose stacks declare per-store services behind profiles so `make up` -assembles whatever combination is configured. +所有维度共用一个机制：`@ConditionalOnProperty` 的 profile 选择、按维度发布能力矩阵、启动日志汇总协商出的组合。 +Compose 栈在 profile 背后声明按库划分的服务，`make up` 即可组装出所配置的任意组合。 -### 6.1 What "pluggable" means — and what it deliberately does not +### 6.1 "可插拔"意味着什么——以及它刻意不意味着什么 -**Deploy-time selection is the goal.** Any combination of the three dimensions is chosen when the stack is assembled; -`dc3.*.type` is read once at startup. Switching an existing deployment means changing the property and restarting the -affected services — for a fresh deployment that is the whole story, no data moves. +**部署期选择就是目标。** 三个维度的任意组合都在组装栈时确定； +`dc3.*.type` 在启动时读取一次。切换既有部署意味着修改配置项并重启受影响的服务——对全新部署而言这就是全部，无需迁移任何数据。 -**Runtime hot-swap is explicitly not a goal**, for three structural reasons: +**运行时热替换明确不是目标**，理由有三个结构性原因： -1. **Data gravity** — the engine holds the data. Swapping PostgreSQL for MySQL under live data without moving it yields - an empty store; moving the data is a *migration*, not a plug event. No abstraction layer can change that. -2. **Startup binding** — connection pools, `SqlSessionFactory` and `databaseId` - resolution (relational), TSDB clients and ingest-buffer flush points (time-series) - are all established at boot. -3. **Dialect correctness** — every forked statement is parsed and routed for the engine that will execute it; a runtime - dialect flip would bypass that guarantee. +1. **数据引力** —— 数据在引擎手里。在真实数据之下把 PostgreSQL 换成 MySQL 而不搬数据，得到的只是一个空库； + 搬数据是一次*迁移*，不是一次插拔动作。任何抽象层都改变不了这一点。 +2. **启动期绑定** —— 连接池、`SqlSessionFactory` 与 `databaseId` + 解析（关系库），TSDB 客户端与摄入缓冲刷写点（时序） + 都在启动时建立。 +3. **方言正确性** —— 每条分支语句都是为将执行它的那个引擎解析和路由的；运行时翻转方言会绕开这一保证。 -**What the port architecture does enable is online migration with cutover** ("warm swap"). For the time-series dimension -the `RepositoryStrategyFactory` can register two adapters simultaneously: dual-write window (fan out writes) → backfill -history from the old store → equivalence-check against the TCK fixtures → flip reads → drain → retire the old store. The -MQ-buffered ingest path (`PointValueReceiver` → -`PointValueIngestBuffer`) means no in-flight data is lost while a store is briefly unavailable. For the relational -dimension the equivalent is standard online-migration tooling (logical replication / CDC / pgloader) — supported by, but -outside, this design. +**port 架构真正能带来的是带切换（cutover）的在线迁移**（"温替换"）。对时序维度， +`RepositoryStrategyFactory` 可以同时注册两个适配器：双写窗口（写入扇出）→ 从旧库回填历史 → 用 TCK 夹具做等价校验 +→ 切换读取 → 排空 → 退役旧库。MQ 缓冲的摄入路径（`PointValueReceiver` → +`PointValueIngestBuffer`）意味着存储短暂不可用时不会有在途数据丢失。对关系维度，对应物是标准的在线迁移工具 +（逻辑复制 / CDC / pgloader）——得到本设计的支持，但在本设计范围之外。 -**Adding a store is "hot" in the only sense that matters**: adapter modules are additive — new module + one config -property, zero core changes; the TCK is the bar. +**新增一个存储在唯一重要的意义上是"热"的**：适配器模块是纯增量的——新模块 + 一个配置项，核心零改动； +TCK 就是门槛。 -## 7. Migration plan +## 7. 迁移计划 -The two tracks are independent and can interleave: +两条轨道相互独立，可以交错进行： -- **R1 — relational hygiene (zero behavior change).** Portable SQL rewrites (`FILTER`→`SUM(CASE)` ×3, `generate_series` - →recursive CTE, drop `::text`), split - `dc3-common-postgres` → `dc3-common-jdbc` + `dc3-db-postgres`. *Gate: existing E2E and `make test-it` green, - unchanged.* -- **R2 — MySQL dialect.** Fork the 9 dialect-bound statements across 4 mappers via - `databaseId`; re-implement the seed trigger functions (`update_operate_time` → - `ON UPDATE CURRENT_TIMESTAMP`, driver-device revision triggers → application layer); write `initdb/mysql/` (expression - JSON defaults), add `dc3-db-mysql`, stand up the dual-dialect DAL TCK in CI. -- **T1 — consolidate the TSDB boundary.** Move `PostgresRepositoryServiceImpl` → - `dc3-repository-timescale`; fold the 10-statement dashboard read surface into the port (spike the read-primitives - shape first — open question §8.2); move - `dc3_point_value` DDL + `time_bucket` out of the relational seed. *Gate: no query outside the TSDB adapter references - `dc3_point_value`.* -- **T2 — port surface cleanup + TDengine adapter.** Cursor pagination, neutral aggregate enum; TDengine 3 adapter as the - first external-store proof, plus the time-series TCK. -- **T3 — community stores.** InfluxDB / IoTDB adapters, explicitly framed as community-sized tasks with the TCK as the - bar (same framing as MQ adapters). +- **R1 —— 关系库卫生（零行为变化）。** 可移植 SQL 改写（`FILTER`→`SUM(CASE)` ×3、`generate_series` + →递归 CTE、去掉 `::text`），把 + `dc3-common-postgres` 拆分为 `dc3-common-jdbc` + `dc3-db-postgres`。*门槛：既有 E2E 与 `make test-it` 保持绿色、结果不变。* +- **R2 —— MySQL 方言。** 通过 `databaseId` 对 4 个 mapper 中 9 条绑定方言的语句做分支；重新实现种子触发器函数 + （`update_operate_time` → + `ON UPDATE CURRENT_TIMESTAMP`，驱动设备修订触发器 → 应用层）；编写 `initdb/mysql/`（表达式 + JSON 默认值），新增 `dc3-db-mysql`，在 CI 中建立双方言 DAL TCK。 +- **T1 —— 收拢 TSDB 边界。** 把 `PostgresRepositoryServiceImpl` 迁至 + `dc3-repository-timescale`；把 10 条语句的看板读取面收编进 port（先做一次 spike 摸清读取原语的形状——开放问题 §8.2）；把 + `dc3_point_value` DDL + `time_bucket` 移出关系种子。*门槛：TSDB 适配器之外没有任何查询再引用 + `dc3_point_value`。* +- **T2 —— port 表面清理 + TDengine 适配器。** 游标分页、中立聚合枚举；TDengine 3 适配器作为首个外置存储的验证， + 外加时序 TCK。 +- **T3 —— 社区存储。** InfluxDB / IoTDB 适配器，明确定位为社区体量的任务，以 TCK 为门槛 + （与 MQ 适配器采用相同的定位方式）。 -## 8. Open questions +## 8. 开放问题 -1. **AGE** — installed and loaded at bootstrap, but no `cypher()` query exists in the codebase, and AGENTS.md describes - it as a pillar extension. Keep (documented as reserved for future graph features) or drop from the base image? Either - way the docs and the image should agree. -2. **Dashboard read-port home and shape** — the bypass surface is 10 statements (§4.2), several richer than bucketed - aggregation (latency histogram, silent sources, coverage gaps). Options: (a) extend `RepositoryService` with - composable read primitives (count-in-range, bucketed aggregate, latest-stream) and re-express the analytic dashboards - on top; (b) a separate read-only dashboard port carrying the full statement semantics. Leaning: (a) — a port that - encodes every dashboard shape would leak the current UI into the contract, and the primitives compose; but the - re-expression cost for `latencyHistogram` / - `silentSources` / `coverageGapItems` needs a spike before T1 commits to it. -3. **Pagination semantics** — pure time cursor, or cursor + optional offset for small-result UI pages? Affects the - frontend history view contract. -4. **Latest-value read path** — `PointValueLocalCacheService` already caches latest values; confirm whether port-level - `selectLatest*` is only a cold-start fallback (if so, adapters can implement it simply and optimize for write - throughput). -5. **InfluxDB version** — 3.x (SQL) is the natural target; whether a 1.8/2.x contributor adapter is worth accepting is a - community question. -6. **Phase ordering** — if T1/T2 land before R2, the MySQL fork shrinks (hypertable already gone); if R2 lands first, - MySQL ships with plain-table `dc3_point_value` - and T2 migrates it. Decide by community demand signals. +1. **AGE** —— 启动时已安装并加载，但代码库中不存在任何 `cypher()` 查询，而 AGENTS.md 将其描述为支柱性扩展。 + 保留（记录为预留给未来图功能）还是从基础镜像中移除？无论哪种，文档与镜像应当一致。 +2. **看板读取 port 的归属与形状** —— 旁路面共 10 条语句（§4.2），其中几条比分桶聚合更复杂 + （延迟直方图、静默源、覆盖缺口）。选项：(a) 用可组合的读取原语（范围计数、分桶聚合、最新流）扩展 + `RepositoryService`，并在此基础上重新表达分析型看板；(b) 另设一个只读的看板 port，承载完整的语句语义。 + 倾向：(a) —— 一个把每种看板形状都编码进去的 port 会把当前 UI 泄漏进契约，而原语是可组合的； + 但在 T1 承诺之前，`latencyHistogram` / + `silentSources` / `coverageGapItems` 的重新表达成本需要先做一次 spike。 +3. **分页语义** —— 纯时间游标，还是游标 + 可选 offset（用于小结果集的 UI 分页）？影响前端历史视图契约。 +4. **最新值读取路径** —— `PointValueLocalCacheService` 已经缓存最新值；需确认 port 层的 + `selectLatest*` 是否只是冷启动回退（如果是，适配器可以简单实现它，并针对写吞吐做优化）。 +5. **InfluxDB 版本** —— 3.x（SQL）是自然的目标；1.8/2.x 的贡献者适配器是否值得接受，是社区问题。 +6. **阶段顺序** —— 若 T1/T2 先于 R2 落地，MySQL 分支会缩小（超表已消失）；若 R2 先落地， + MySQL 将随普通表形态的 `dc3_point_value` + 一起发布，由 T2 完成迁移。根据社区需求信号决定。 -## 9. Appendix — inventory (migration checklists) +## 9. 附录 —— 盘点（迁移清单） -PG-specific mapper statements (R1/R2 checklist; refreshed 2026-08-19): +PG 特有的 mapper 语句（R1/R2 清单；2026-08-19 刷新）： -| Mapper | Statement | Action | +| Mapper | 语句 | 处理动作 | |------------------------------|------------------------------------------------------------------------|----------------------------------------------------------------| -| `AlertMapper` ×3 | `COUNT(*) FILTER` | portable rewrite (`SUM(CASE)`) | -| `AlertMapper` | `generate_series` + `date_trunc` | recursive CTE (unify) | -| `OAuthMcpMapper` | `::text` casts | drop / rewrite | -| `EntityStateMapper` | `upsertEntityState` — `ON CONFLICT ... RETURNING` | **fork** (pg / mysql) | -| `EntityStateMapper` | `claimExpiredDevices` — `UPDATE ... FROM ... RETURNING` | **fork** (pg / mysql) | -| `PointValueMapper` | idempotent batch insert — `ON CONFLICT DO NOTHING ... RETURNING` | **fork** (pg / mysql); statement moves into TSDB adapter at T1 | -| `PointValueMapper` | latest-value upsert — `ON CONFLICT DO UPDATE` | **fork** (pg / mysql); moves into TSDB adapter at T1 | -| `DriverLeaseMapper` | `pg_advisory_xact_lock` acquire | **fork** (pg / mysql) | -| `DriverLeaseMapper` ×3 | lease renew / device-claim — `ON CONFLICT ... DO UPDATE ... RETURNING` | **fork** (pg / mysql) | -| `ResourceRegistryLockMapper` | `pg_advisory_xact_lock(hashtext())` | **fork** (pg / mysql) | -| `DashboardMapper` (data) | `time_bucket` + 8 further direct `dc3_point_value` reads | **T1**: fold into TSDB port | -| `DashboardMapper` (manager) | direct `dc3_history.dc3_point_value` read | **T1**: fold into TSDB port | +| `AlertMapper` ×3 | `COUNT(*) FILTER` | 可移植改写（`SUM(CASE)`） | +| `AlertMapper` | `generate_series` + `date_trunc` | 递归 CTE（统一） | +| `OAuthMcpMapper` | `::text` 类型转换 | 去掉 / 改写 | +| `EntityStateMapper` | `upsertEntityState` —— `ON CONFLICT ... RETURNING` | **方言分支**（pg / mysql） | +| `EntityStateMapper` | `claimExpiredDevices` —— `UPDATE ... FROM ... RETURNING` | **方言分支**（pg / mysql） | +| `PointValueMapper` | 幂等批量插入 —— `ON CONFLICT DO NOTHING ... RETURNING` | **方言分支**（pg / mysql）；语句在 T1 移入 TSDB 适配器 | +| `PointValueMapper` | 最新值 upsert —— `ON CONFLICT DO UPDATE` | **方言分支**（pg / mysql）；在 T1 移入 TSDB 适配器 | +| `DriverLeaseMapper` | `pg_advisory_xact_lock` 获取 | **方言分支**（pg / mysql） | +| `DriverLeaseMapper` ×3 | 租约续期 / 设备认领 —— `ON CONFLICT ... DO UPDATE ... RETURNING` | **方言分支**（pg / mysql） | +| `ResourceRegistryLockMapper` | `pg_advisory_xact_lock(hashtext())` | **方言分支**（pg / mysql） | +| `DashboardMapper`（数据侧） | `time_bucket` + 另有 8 条直读 `dc3_point_value` | **T1**：收编进 TSDB port | +| `DashboardMapper`（manager 侧） | 直读 `dc3_history.dc3_point_value` | **T1**：收编进 TSDB port | -Seed-side R2 items: plpgsql `update_operate_time()` triggers (→ `ON UPDATE -CURRENT_TIMESTAMP`), 3× `track_driver_device_revision_*` statement-level triggers (→ application layer), `'{}'::JSON` -defaults (→ expression defaults). +种子侧 R2 事项：plpgsql `update_operate_time()` 触发器（→ `ON UPDATE +CURRENT_TIMESTAMP`）、3 个 `track_driver_device_revision_*` 语句级触发器（→ 应用层）、`'{}'::JSON` +默认值（→ 表达式默认值）。 -TSDB boundary checklist (T1): +TSDB 边界清单（T1）： -- `RepositoryService` / `RepositoryStrategyFactory` / `ActiveRepositoryProfileConfig` — keep, promote to shared - contract. +- `RepositoryService` / `RepositoryStrategyFactory` / `ActiveRepositoryProfileConfig` —— 保留，提升为共享 + 契约。 - `PostgresRepositoryServiceImpl` + `PointValueManager`/`PointValueMapper` - point-value statements → `dc3-repository-timescale`. -- `dc3_point_value` DDL, hypertable/compression DDL, seed data → timescale adapter. -- Verify zero remaining `dc3_point_value` references outside TSDB adapters (`grep -r dc3_point_value` gate). + 的位值语句 → `dc3-repository-timescale`。 +- `dc3_point_value` DDL、超表/压缩 DDL、种子数据 → timescale 适配器。 +- 验证 TSDB 适配器之外不再残留任何 `dc3_point_value` 引用（`grep -r dc3_point_value` 门槛）。 **R1/R2 实施记录（2026-08-24）**：关系轨道全部落地——R1 可移植改写 + 模块拆分 （中立基建 + 顶层 `dc3-db` 家族——后按家族一致性迁为 `dc3-db/dc3-db-core`，与 dc3-mq-core/dc3-tsdb-core 同构）；R2 MySQL 方言（databaseId fork、RETURNING 解耦为 diff --git a/docs/mq-brokers.md b/docs/mq-brokers.md index bf7e001e6..2797262db 100644 --- a/docs/mq-brokers.md +++ b/docs/mq-brokers.md @@ -1,88 +1,81 @@ -# Message Broker Selection +# 消息 broker 选型指南 -IoT DC3's internal async plane (point values, commands, states, alarms, events, notify tasks between the center services -and the drivers) runs behind a broker-neutral messaging port: the application code is identical on every broker, and the -deployment picks one with a single setting plus one dependency. +IoT DC3 的内部异步平面（中心服务与驱动之间的点位值、命令、状态、告警、事件、通知任务）运行在 broker 中立的消息 +Port 后面：应用代码在每个 broker 上完全相同，部署侧只需一个配置项加一个依赖即可选定 broker。 -> Design background: [`docs/design/mq-abstraction.md`](./design/mq-abstraction.md) +> 设计背景：[`docs/design/mq-abstraction.md`](./design/mq-abstraction.md) -## How to pick a broker +## 如何选型 broker -1. Set `DC3_MQ_TYPE` (maps to `dc3.mq.type`) — see the table below. Default is - `rabbitmq`, so existing deployments change nothing. -2. Provide the connection settings for that broker (table below). -3. That is all in terms of configuration — nothing else in the stack is broker-specific. Note that only the `rabbitmq` - adapter ships in the standard `dc3` images; running another broker additionally requires adding the corresponding - `dc3-mq-*` adapter dependency to the deployment (step 1 of the selection is still just the type + connection - settings). +1. 设置 `DC3_MQ_TYPE`（映射到 `dc3.mq.type`）——见下表。默认值为 + `rabbitmq`，因此既有部署无需任何改动。 +2. 提供该 broker 的连接配置（见下表）。 +3. 配置层面仅此而已——技术栈中没有其他任何部分与特定 broker 相关。注意标准 `dc3` 镜像只随附 + `rabbitmq` 适配器；运行其他 broker 还需在部署中额外加入对应的 + `dc3-mq-*` 适配器依赖（选型第 1 步仍然只是类型 + 连接配置）。 -The internal plane and the device-access plane are independent: switching the internal broker never affects the MQTT -drivers or the EMQX-based device access. +内部平面与南向设备面相互独立：更换内部 broker 绝不会影响 MQTT 驱动或基于 EMQX 的设备接入。 -## Device-access plane (southbound MQTT) +## 南向设备面（MQTT 南向接入） -The device-access MQTT stack needs no adapter layer at all — MQTT is an open wire protocol and the driver uses a -standard Paho client, so the broker is selected by one URL (`dc3.driver.mqtt.url`): EMQX, Mosquitto, HiveMQ, NanoMQ, -VerneMQ ... are drop-in replacements with zero code change. The compose default (`dc3-emqx`) is a deployment choice, not -a dependency; vendor-specific features (management APIs, rule engines) must not leak into driver code, and -`MqttVendorNeutralityIT` keeps that property verified mechanically — the same client code round-trips against two -different broker vendors, differing only in the URL. +南向设备面的 MQTT 技术栈完全不需要适配器层——MQTT 是开放的线上协议，驱动使用标准 Paho 客户端，broker 只由一个 +URL（`dc3.driver.mqtt.url`）选定：EMQX、Mosquitto、HiveMQ、NanoMQ、VerneMQ …… 都可以零代码改动直接替换。 +Compose 默认值（`dc3-emqx`）是一种部署选择，而非依赖；厂商特有功能（管理 API、规则引擎）不得泄漏进驱动代码， +`MqttVendorNeutralityIT` 以机械化方式持续验证这一性质——同一份客户端代码对两家不同的 broker 厂商做往返验证， +唯一差异只是 URL。 -One dialect note: the device-access client speaks MQTT 3.1.1, which every mainstream broker supports and which covers -device access (publish/subscribe, QoS, retained, TLS, X.509). MQTT 5-only features such as shared subscriptions are used -exclusively by the internal-plane adapter above. +一条方言说明：南向设备面客户端使用 MQTT 3.1.1——所有主流 broker 都支持该版本，且足以覆盖设备接入 +（发布/订阅、QoS、保留消息、TLS、X.509）。共享订阅等 MQTT 5 独有特性仅由上文的内部平面适配器使用。 -## Certified brokers +## 已认证的 broker -Every adapter below except RocketMQ passes the same broker-neutral contract suite (`dc3-mq-tck`) against a live broker. -The RocketMQ adapter is **experimental / not yet certified**: its contract suite is opt-in (`TCK_ROCKETMQ_NAMESRV`) and -self-describes as not-yet-certified — run it against your own broker before relying on it. +下表中的适配器除 RocketMQ 外，均已在真实 broker 上通过同一套 broker 中立契约套件（`dc3-mq-tck`）。 +RocketMQ 适配器为**实验性 / 尚未认证**：其契约套件需显式开启（`TCK_ROCKETMQ_NAMESRV`），并且自述为尚未认证—— +在依赖它之前，请先针对你自己的 broker 运行一遍。 -| Broker | `DC3_MQ_TYPE` | Connection settings | Notes | -|-------------------------------------------|---------------|---------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------| -| RabbitMQ (default) | `rabbitmq` | `RABBITMQ_HOST` / `RABBITMQ_PORT` / `RABBITMQ_USERNAME` / `RABBITMQ_PASSWORD` (+ SSL) | Wire-compatible with all pre-port deployments; publisher confirms + returns | -| Kafka | `kafka` | `spring.kafka.bootstrap-servers` | `acks=all`; new consumer groups start at latest; per-key ordering; plain connections only — the adapter configures no SASL/TLS | -| RocketMQ (experimental) | `rocketmq` | `dc3.mq.rocketmq.name-server-address` | CLUSTERING/BROADCASTING modes; sync-send confirmation; uncertified — see above | -| Pulsar | `pulsar` | `dc3.mq.pulsar.service-url` | Shared/exclusive subscriptions; native batch receive | -| ActiveMQ (Artemis / Classic) | `activemq` | `dc3.mq.activemq.url` (+ user/password) | JMS 2.0; shared durable subscriptions; scheduled delays; no publisher confirm — the driver outbox covers durability | -| MQTT 5 (EMQX / HiveMQ / NanoMQ / VerneMQ) | `mqtt` | `dc3.mq.mqtt.host` / `dc3.mq.mqtt.port` | Shared subscriptions (MQTT 5); QoS 1; lets one broker serve both the device plane and the internal plane | +| Broker | `DC3_MQ_TYPE` | 连接配置 | 说明 | +|--------------------------------------------|---------------|---------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------| +| RabbitMQ（默认） | `rabbitmq` | `RABBITMQ_HOST` / `RABBITMQ_PORT` / `RABBITMQ_USERNAME` / `RABBITMQ_PASSWORD`（+ SSL） | 与 Port 引入之前的所有部署线上兼容；发布者确认 + returns | +| Kafka | `kafka` | `spring.kafka.bootstrap-servers` | `acks=all`；新消费者组从 latest 起始；按 key 有序；仅明文连接——适配器不配置任何 SASL/TLS | +| RocketMQ（实验性） | `rocketmq` | `dc3.mq.rocketmq.name-server-address` | CLUSTERING/BROADCASTING 模式；同步发送确认；未认证——见上文 | +| Pulsar | `pulsar` | `dc3.mq.pulsar.service-url` | 共享/独占订阅；原生批量接收 | +| ActiveMQ（Artemis / Classic） | `activemq` | `dc3.mq.activemq.url`（+ user/password） | JMS 2.0；共享持久订阅；定时延迟；无发布者确认——驱动发件箱(outbox)保障持久性 | +| MQTT 5（EMQX / HiveMQ / NanoMQ / VerneMQ） | `mqtt` | `dc3.mq.mqtt.host` / `dc3.mq.mqtt.port` | 共享订阅（MQTT 5）；QoS 1；可让一个 broker 同时服务设备面与内部平面 | -## Capability matrix +## 能力矩阵 -| Capability | RabbitMQ | Kafka | RocketMQ | Pulsar | ActiveMQ | MQTT 5 | +| 能力 | RabbitMQ | Kafka | RocketMQ | Pulsar | ActiveMQ | MQTT 5 | |--------------------------|--------------------|-----------------------|------------------------------------|-----------------------------|------------------------------------------|-----------------------| -| Delayed message | fallback* | ❌ → local fallback | fallback (levels quantize) | fallback (native available) | ✅ JMS scheduled | ❌ → local fallback | -| Native DLQ | DLX + quarantine | adapter `.dlq` topic | adapter `-dlq` topic | adapter `.dlq` topic | adapter `.dlq` queue | adapter `/dlq` topic | -| Broadcast | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| Per-message ack | ✅ | offset (approx) | ✅ | ✅ | ✅ | ✅ (QoS 1) | -| Publisher confirm | ✅ | ✅ (acks=all) | ✅ sync send | ✅ (message id) | ❌ best-effort (driver outbox covers it) | ✅ (PUBACK) | -| Batch delivery | ✅ native | ✅ native | ✅ native | ✅ native | ⚠️ synthesized | ⚠️ synthesized | -| Per-key ordering | ❌ | ✅ | ❌ (keys carried for routing only) | ❌ (shared sub) | ❌ | ❌ | -| Key-pattern routing | ✅ broker bindings | ✅ client-side router | ✅ client-side router | ✅ client-side router | ✅ client-side router | ✅ client-side router | -| Subscription expiry | ✅ x-expires | ❌ documented | ❌ documented | ❌ documented | ❌ documented | ⚠️ session expiry | -| Group durability offline | ✅ durable queue | ✅ log retention | ✅ offsets | ✅ durable subscription | ✅ durable subscription | ⚠️ broker-dependent | -| Retention | queue TTL | retention config | retention | retention | subscription retention | broker-dependent | +| 延迟消息 | 回退(fallback)* | ❌ → 本地回退(fallback) | 回退(fallback)（按级别量化） | 回退(fallback)（原生能力可用） | ✅ JMS 定时投递 | ❌ → 本地回退(fallback) | +| 原生死信队列 | DLX + 隔离区 | 适配器 `.dlq` 主题 | 适配器 `-dlq` 主题 | 适配器 `.dlq` 主题 | 适配器 `.dlq` 队列 | 适配器 `/dlq` 主题 | +| 广播 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 逐消息 ack | ✅ | offset（近似） | ✅ | ✅ | ✅ | ✅（QoS 1） | +| 发布者确认 | ✅ | ✅ (acks=all) | ✅ 同步发送 | ✅（message id） | ❌ 尽力而为（驱动发件箱(outbox)兜底） | ✅ (PUBACK) | +| 批量投递 | ✅ 原生 | ✅ 原生 | ✅ 原生 | ✅ 原生 | ⚠️ 合成 | ⚠️ 合成 | +| 按 key 有序 | ❌ | ✅ | ❌（key 仅用于路由） | ❌（共享订阅） | ❌ | ❌ | +| key 模式路由 | ✅ broker 绑定 | ✅ 客户端路由器 | ✅ 客户端路由器 | ✅ 客户端路由器 | ✅ 客户端路由器 | ✅ 客户端路由器 | +| 订阅过期 | ✅ x-expires | ❌ 文档注明不支持 | ❌ 文档注明不支持 | ❌ 文档注明不支持 | ❌ 文档注明不支持 | ⚠️ 会话过期 | +| 消费组离线持久性 | ✅ 持久化队列 | ✅ 日志保留 | ✅ offsets | ✅ 持久订阅 | ✅ 持久订阅 | ⚠️ 取决于 broker | +| 保留 | queue TTL | 保留配置 | 保留 | 保留 | 订阅保留 | 取决于 broker | -\* RabbitMQ's intrinsic TTL+DLX delays (driver lease timeout, device scan tick) work server-side exactly as before; -arbitrary per-message delays use the port's local scheduler fallback on every broker for uniform behavior. +\* RabbitMQ 固有的 TTL+DLX 延迟（驱动租约超时、设备扫描 tick）仍在服务端按原样工作；任意逐消息延迟则统一使用 +Port 的本地调度器回退(fallback)，以保证所有 broker 行为一致。 -The full reasoning (semantics per broker, fallbacks, the TCK as the acceptance bar for community adapters) lives in the -design document linked above. +完整论证（各 broker 的语义、回退(fallback)策略、作为社区适配器验收门槛的 TCK）见上方链接的设计文档。 -## Notes +## 说明 -- **Key-pattern routing**: only RabbitMQ evaluates `SubscriptionSpec.keyPattern` broker-side (topic-exchange bindings). - Every other adapter runs a client-side topic router in front of one consumer per (topic, group): the message's - partition key is matched against each listener's pattern with Rabbit topic-binding wildcard semantics (`*` = one word, - `#` = zero or more words, blank pattern = everything; see `KeyMatcher` in `dc3-mq-core`), several matching listeners - in one JVM round-robin, and a key matching no listener in that JVM is acknowledged and skipped — Rabbit's - unroutable-drop semantics. Cross-JVM load balancing stays the broker's job via the shared consumer group. -- **Kafka security**: the adapter builds its producer/consumer with `acks=all` but no SASL/TLS settings — plain - connections only. Secured Kafka clusters need a user-provided `KafkaTemplate`/consumer config until the adapter grows - a security-properties bridge. -- **Driver outbox**: point-value durability is owned by the driver's SQLite outbox on every broker, so brokers without - publisher confirmation remain fully safe. -- **At-least-once**: the delivery contract is at-least-once with idempotent consumers on every broker; consumers must - not assume ordering (except where the matrix notes per-key ordering). -- **Community adapters**: pass the contract suite in `dc3-mq-tck` against your broker to certify a new adapter (Redis - Streams, SQS, ...). +- **key 模式路由**：只有 RabbitMQ 在 broker 侧求值 `SubscriptionSpec.keyPattern`（topic 交换机绑定）。 + 其余所有适配器都在每个 (topic, group) 的单个消费者前面运行一个客户端主题路由器：消息的分区 key 按 Rabbit + topic 绑定通配符语义（`*` = 一个词，`#` = 零个或多个词，空白模式 = 全部；见 `dc3-mq-core` 中的 + `KeyMatcher`）与各监听器的模式进行匹配；同一 JVM 内多个匹配的监听器以轮询方式分发；key 在该 JVM 中 + 匹配不到任何监听器时会被确认并跳过——即 Rabbit 的不可路由即丢弃语义。跨 JVM 负载均衡仍由 broker + 通过共享消费者组承担。 +- **Kafka 安全**：适配器构建生产者/消费者时带 `acks=all`，但不带任何 SASL/TLS 设置——仅明文连接。 + 在适配器具备安全属性桥接之前，启用了安全机制的 Kafka 集群需要用户自行提供 `KafkaTemplate`/消费者配置。 +- **驱动发件箱(outbox)**：点位值持久性在每个 broker 上都由驱动的 SQLite 发件箱(outbox)负责，因此不支持 + 发布者确认的 broker 依然完全安全。 +- **至少一次（at-least-once）**：投递契约在每个 broker 上都是至少一次、消费方幂等；消费方不得假设 + 有序性（矩阵中注明按 key 有序的场景除外）。 +- **社区适配器**：针对你的 broker 通过 `dc3-mq-tck` 中的契约套件，即可认证新的适配器 + （Redis Streams、SQS 等）。