The dc3-db aggregate mixed two naming axes and one axis-free module:
infrastructure used dc3-db-r2dbc-{core,runtime,postgres} (with "postgres"
sitting in a layer slot), the domain adapters used dc3-db-postgres-{domain}
(no "r2dbc" at all), and tck had no axis. PostgreSQL-only and R2DBC-only
are properties of the whole aggregate (dc3-db README, docs/db-dialects.md),
so the per-module prefixes carried no discriminating information — the
65 store classes already say R2dbc in their names.
New layout, one dc3-db-{role} pattern, aligned with the common-{domain} /
center-{domain} pairs they serve:
dc3-db-r2dbc-core -> dc3-db-core
dc3-db-r2dbc-runtime -> dc3-db-runtime (absorbs the dialect)
dc3-db-r2dbc-postgres -> (merged into dc3-db-runtime)
dc3-db-postgres-auth -> dc3-db-auth
dc3-db-postgres-manager -> dc3-db-manager
dc3-db-postgres-data -> dc3-db-data
dc3-db-postgres-agentic -> dc3-db-agentic
dc3-db-tck -> unchanged
The dialect module held a single 12-line AutoConfiguration registering
StandardR2dbcDialect("postgres", ...) plus the r2dbc-postgresql driver;
both move into dc3-db-runtime (package io.github.pnoker.db.runtime.dialect,
registered in the runtime AutoConfiguration.imports, driver dependency
transferred). Its six dependents now rely on the runtime they already
pulled in. Java packages rename in step — io.github.pnoker.db.r2dbc.core
-> io.github.pnoker.db.core, io.github.pnoker.db.r2dbc.runtime ->
io.github.pnoker.db.runtime, io.github.pnoker.db.postgres.{auth,manager,
data,agentic} -> io.github.pnoker.db.{auth,manager,data,agentic} — so no
trace of the old axis survives outside git history.
Also fixes the stale module layout in docs/design/relational-r2dbc.md §6
(it still described stores living in dc3-common-*/repository, the
pre-7fde11134 arrangement) and ships migrations/02-schema-fingerprint-
resync.sql for existing volumes still carrying the pre-documentation
fingerprint marker.
Verified: full-reactor compile, dc3-db aggregate tests (-am), all six
service images rebuilt through the multi-stage Dockerfile (container
Maven package), WSL stack recreated with every center healthy, and an
end-to-end login + manager list smoke test through the gateway (JWT
cookie, offset/limit/sort paging).
Co-Authored-By: Claude Code <noreply@anthropic.com>
Now that every token migration has landed, lock them in:
- radius-contract.test.ts: every border-radius in src/ must resolve to
the --dc3-radius-* scale (directly or through an Element Plus radius
variable). Geometric circles (50%/100%), inherit and single-corner 0
resets stay literal — hand-rolled px values are the exact drift this
cleared once.
- spacing-contract.test.ts: macro spacing (page edges, card walls,
section rhythm) must resolve to --dc3-space-* / --dc3-page-padding /
--dc3-gutter. Semantic tokens defined exactly once in theme.scss; a
whitelist of layout-bearing containers may only declare margin/
padding/gap with tokens; clamp() in spacing and el-row :gutter
values other than 8 are banned repo-wide (the JS prop cannot read
CSS vars, so the literal must mirror --dc3-gutter's desktop value);
a // spacing-exempt marker opts out line-by-line. Intra-component
micro spacing stays out of scope this round.
Document the v8 spacing-system decision in
docs/design/frontend-three-terminal-ux.md and add the A5 rules to
dc3/agents/frontend.md so agents follow the scales.
Co-Authored-By: Claude Code <noreply@anthropic.com>
docs/README.md still advertised a three-dialect selection guide that
db-dialects.md no longer contains, tsdb-abstraction.md documented a
MySQL upsert branch that no longer exists, and the r2dbc-postgres
README pointed at the removed dc3.db.type activation property.
R2dbcPageExecutor and R2dbcCursorExecutor had zero production
adopters: every store executes count/items and keyset paging inline
inside SpringR2dbcPageTransaction snapshots. Dropping the dead
executors leaves one pagination implementation instead of two,
and the migration handoff checklist no longer lists them.
Land the v7 three-terminal UX decision: drop the width/pointer-based
44px touch enlargement so one 32px density holds across desktop,
tablet, and mobile, and rebuild entity pages on a responsive record
list (ResponsiveRecordList + EntityMobileCard + per-entity mobile
summaries). Tighten the nginx security headers (CSP frame-ancestors
'none', X-Frame-Options DENY) to match the static e2e server.
Complete the dc3-web audit remediation, including the remaining LOW
items from docs/dc3-web-low-defects-handoff.md: streaming-safe
message loads in the agentic store, mock menu reparent cycle
rejection, paged-list page clamping, fetch/axios mock request
normalization, per-table attribute catalogs, post-flush alarm
watchers, rowKey-agnostic delete state, localized card text
fallbacks, and e2e-server port fail-fast.
Gate: vue-tsc, eslint, vitest 687 passed, responsive e2e 64 passed.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- migrate auth, manager, data, and agentic persistence to R2DBC
- hard cut pagination, facade, gateway, web, and CLI contracts
- remove legacy database and TSDB adapters and document validation gates
BREAKING CHANGE: platform persistence now requires PostgreSQL 18 with TimescaleDB and the new reactive API contracts.
- projection is additive per risk class: a mixed LOW/HIGH principal
keeps tools_call; MEDIUM (hidden-until-enabled) now accounted
- qualify high-risk 'no standing grant' to the default path so it no
longer contradicts the approved --approve-window escape hatch
- note phase-1 ordering: --oauth login can land before the gateway
resolver chain, behind a flag until REST accepts RS256
Revert the premature McpServer.vue change (implementation was not
requested). Record the onboarding UX contract in
token-unification-mcp-first-cli.md §4.1 instead, and rename the CLI
package back to dc3-cli per review.
Add design draft covering:
- verified state of the two parallel auth systems (HS256 login vs
RS256 OAuth/MCP) and their incompatible verdict points
- gateway TokenResolver chain over an extended X_AUTH_PRINCIPAL so
both tickets converge on one RBAC-backed permission model
- scope-as-projection of dc3_resource bindings, risk-layered TTLs
with step-up for high-risk calls
- CLI rebuild around the MCP/OAuth credential mechanism (supersedes
dc3-client-sdk.md Phase 4) with five open decision questions
Translate the seven remaining English documents under docs/ in place,
preserving all technical content, code blocks, identifiers, links, and
decision IDs (D1-D19, F1-F15, P0-P5). Add a categorized Chinese
navigation index (docs/README.md) with per-document summaries, the
storage supersession chain, and task-based reading paths.
Approved design for replacing MyBatis-Plus + JDBC with Spring Data R2DBC,
auth center as the pilot. Records the verified fact base (F1-F15), the
D1-D19 decision checklist (per-dialect repository modules, TCK discipline,
compile-time tenant scoping, facade/gRPC blocking edge, timestamptz codec,
workerId allocation rule), and the P0-P5 phased plan with the P2-P3
go/no-go gate. Supersedes storage-abstraction.md section 3 (relational
half); its TSDB half was already superseded.
- let TSDB failures propagate in the alarm window data source: swallowing
them made an outage read as no-data and silently missed alarms
- dedupe ingest batches by natural key before store upserts (same-row
writes poison timescale ON CONFLICT batches)
- turn unsupported countByDimension into a structured refusal (analytics)
or capability-honest empty panel (dashboard) instead of a raw 500
- pin LocalDateTime.now to DEFAULT_ZONEID on the tsdb read paths
- adapter fixes: influxdb/iotdb/tdengine/timescale store and auto-config
corrections, iotdb system properties, capability declarations on
TsdbStore, new timescale store unit tests
- derive pagination DbType from the selected dc3-db adapter with a fail-fast
when a non-default dc3.db.type lacks its adapter jar
- skip postgres profile activation for mariadb, not just mysql
- fork OAuthMcpMapper/AlertMapper statements for mysql and mariadb
(CAST AS JSON, ::text casts, ODKU value references)
- honor GET_LOCK verdict in resource registry sync instead of assuming
acquisition on every engine
- pg2mysql_seed.py: convert partial unique indexes to stored generated guard
columns instead of dropping them; add 99-iot-dc3-grants.sql
- add DC3_DB_URL/DC3_DB_HISTORY_URL per-service overrides in center configs
- cover the new mapper forks in the db TCK; fix AGPL headers in spring
metadata files
- add KeyMatcher/KeyRoutes: client-side topic router for brokers without
binding-level key filtering, with wildcard unit tests in dc3-mq-core
- stamp a dc3-partition-key header mirror for brokers with no native key
field (mqtt user property, jms string property)
- drop the unused RetryPolicy; batch redelivery is governed by the batch
consumer retry settings
- align adapter configurations, error messages, and the Pulsar contract
test with the dc3-mq-* module names
dc3-common-jdbc was the family's core in disguise — MyBatis-Plus wiring,
the tenant-line handler, the databaseIdProvider, the jdbc profile. It now
lives as dc3-db/dc3-db-core, matching the dc3-mq-core / dc3-tsdb-core
shape: one aggregator, port first, adapters after, tck last. Artifact
renamed, packages deliberately untouched (same convention as the mq
family move); twelve poms rewired, the family aggregator puts core
first, and the db-tck now depends on the core explicitly instead of
riding transitives.
dc3-common-sql stays where it is on purpose: it is not platform dialect
plumbing but the base class of the four JDBC *device drivers*
(mysql/oracle/postgresql/sqlserver drivers that connect to external
databases being collected from) and depends on the driver SDK — under
dc3-db it would claim a meaning it does not have.
Gates: db-tck 24/24 (postgres + mysql + mariadb), data 274/274,
full-repo compile green.
dc3-db-mariadb brings MariaDB 10.6+ behind dc3.db.type=mariadb with the
official mariadb-java-client and the same dual-dialect contract suite.
MariaDB shares the MySQL seed wholesale (utf8mb4_unicode_ci — the 0900
collation is MySQL-8-only and MariaDB rejects it outright, so the seed
translator and the hand-written 05 both moved to the portably-supported
collation, re-verified against mysql:8.4).
The mapper forks differ from the mysql twins in exactly one dimension,
verified against a live 10.11 container first: MariaDB never adopted
the AS-new ODKU row alias (syntax error on 10.11), so its 15 twins
reference VALUES(col) instead — everything else (GET_LOCK/RELEASE_LOCK,
FOR UPDATE SKIP LOCKED, recursive CTEs, INTERVAL ? SECOND, the
JSON_UNQUOTE(JSON_EXTRACT(...)) double-hop, REGEXP_REPLACE without the
g flag, DAYOFWEEK/HOUR units) proved compatible and rides the same
statement bodies.
Scope decision recorded in docs/db-dialects.md: DB2, H2 and HSQLDB are
explicitly out (dialect cost with zero demand; embedded test databases
unfit for multi-service concurrent IoT writes), SQLite equally unfit
for the multi-service shape (database-level write lock, no network
listener). Oracle and SQL Server stay on the roadmap with their real
obstacles spelled out (MERGE-heavy rewrite; no ARM64 mssql image for
local certification on Apple Silicon) — not started without a demand
signal.
Gates: db-tck 24/24 (postgres + mysql + mariadb, 8 each),
data/auth/manager suites green (274 in data), full-repo compile green.
R2c of docs/design/storage-abstraction.md §3. dc3-db-tck runs one set of
mapper-level assertions against both engines with the real mapper XML —
the databaseIdProvider routing is part of what gets certified, not just
the SQL: the fenced latest-value upsert's three-level tiebreak (fencing
token, then time, then sequence, then message id), the state upsert +
same-transaction re-select shape, the three-step expired-lease claim
(lock-and-read FOR UPDATE SKIP LOCKED, flip offline, derive the
post-update view), the lease upserts' row-local increments (token moves
only on ownership change), the row-level revision triggers (insert and
meaningful update bump, trivial update does not), and the tool-catalog
query's encoded-JSON double-hop.
Harness notes worth keeping: one SqlSessionFactory per database mirrors
the production @DS wiring (state/latest/leases/catalog span four
databases with unqualified table names); the mapper XML loads through
XMLMapperBuilder with the affectData mybatis-plus DTD extension
stripped; postgres registers the timestamptz type handler. Both
harnesses learned to distrust podman: log-following never fires, so
mysql waits on the listening TCP port and postgres polls a real JDBC
handshake (the port opens during initdb's local-only phase and closes
again for the restart). The postgres harness mounts seed files 01..07
individually — 00's age/pgvector extensions exist only in the
production base image — with 05 trimmed at runtime to the
dc3_point_latest projection (the hypertable half needs the base
image's TimescaleDB function set; the dialect contract doesn't touch
it).
The catalog fixture plants its api row through a parameterized insert:
the escaped-JSON value must not travel through SQL text (MySQL's
default backslash escapes eat the \" the value depends on) nor bind
uniformly (PG rejects varchar->json, MySQL rejects the binary
Types.OTHER binding) — the fixture branches on the engine.
Selection guide published at docs/db-dialects.md (dialect matrix,
swap procedure, the semantic decisions the suite locks); the storage
design's status header moves to Implemented with an R1/R2 closeout
note.
Gates: db-tck 16/16 (8 postgres + 8 mysql), data/auth/manager suites
green (274 in data), e2e 26/26, full-repo compile green.
Proposed (not implemented): collapse the 5 internal runtime RPCs to 3,
carry inputSchema end-to-end so tools/list stops shipping an empty
envelope, and unify the authorize/confirm/audit seam. Aligned with the
S19 analytics endpoints (nine REST tools) which already match its target
dispatch shape; the s19 work noted the empty-schema gap this proposal
closes.
Three research notes from the platform-comparison track: DG-IoT and
supOS-Free feature-by-feature comparisons against IoT DC3 with borrow
lists, and a code-level review of the DC3 profile (thing model) design
with a final verdict on its current shape.
Phase 3 of docs/design/tsdb-abstraction.md: the store family reaches four
certified adapters, the selection guide publishes the real capability
matrix, and external stores become first-class compose services.
dc3-tsdb-influxdb (3.11.2-core) talks the documented v3 HTTP APIs
directly — line protocol writes, query_sql CSV reads, zero client
dependencies. Integer fields carry the i suffix from the first write (a
bare number binds the column to Float64 forever and destroys ns
timestamps); timestamps cross the wire as CAST(... AS BIGINT)
nanoseconds because JSON renders them in scientific notation; PERCENTILE
is approximate-only and deleteRange does not exist in Core — both
declared honestly (false), the analytics facade computes exact
percentiles from bounded pulls.
dc3-tsdb-iotdb (2.0.10-standalone) maps series onto tree paths
root.dc3.t<tenant>.d<device>.p<point> — path nodes cannot be purely
numeric, so the design's raw-id mapping is amended. The server must run
with timestamp_precision=us and dn_rpc_address=0.0.0.0 (a two-line
properties override shipped under dc3/dependencies/iotdb); WHERE time
literals must use the 2026-08-20T12:59:59+00:00 form because bare epoch
numbers are parsed as milliseconds regardless of the server precision —
a microsecond literal silently matches nothing. Sessions disable
redirection: node discovery hands out internal cluster addresses that
break behind port mappings. GROUP BY LEVEL columns come back
wildcard-shaped and are parsed per column; COUNT rides the INT32 quality
measurement (never null); null measurements are omitted per record — a
null value in insertRecords makes the whole record vanish.
Capability matrix published in docs/tsdb-stores.md per adapter-declared
values (skip counts in the TCK are declared-unsupported cases, degraded
by facades, never wrong data). External store services (tdengine,
influxdb, iotdb) join the optional compose stack with healthchecks and
DC3_TSDB_* passthrough in the app stack.
Gates: timescale 24/24, tdengine 24/24 (2 skips), influxdb 24/24 (2
skips), iotdb 24/24 (3 skips); full-repo compile green.
The timescale adapter now owns the S16 lifecycle: raw 30 days (seed 05,
was 180) -> 1-minute tier 1 year (configurable via
dc3.tsdb.timescale.rollup.minute-keep-days) -> 1-hour tier forever.
bucketedAggregate and bucketedCount serve bucket widths at or above a
tier's granularity from that tier with exactly composable expressions
(AVG recombines as SUM(num_sum)/SUM(num_count), COUNT as
SUM(sample_count)); PERCENTILE and FIRST/LAST stay on the raw path — the
shared caggs carry textual first/last only, and tiered numeric first/last
would version-skew deployments created before such columns existed.
Single-window aggregate() stays raw by design: alarm windows must never
read through materialization lag.
The tiers are the observability pipeline's existing real-time continuous
aggregates (cagg_point_value_1m/1h, queried by Grafana) — one structure
serves both consumers; the adapter bootstraps the same names with an
identical shape via IF NOT EXISTS, so embedded seed boots, standalone
adapter boots and already-running deployments all converge on it without
rebuilding anything. Real-time mode (materialized_only=FALSE, no longer
the TS 2.13+ default) keeps tier reads correct immediately after an
append; the refresh policies only move aggregation work off the read
path. deleteRange refreshes both tiers afterwards.
Also fixed along the way: TDengine's bucketedAggregate PERCENTILE hit the
same "single table query" limit as aggregate() and now routes per-series
subtables.
Gates: timescale TCK 24/24 including the new rollup case (tiered
COUNT/AVG/LAST/bucketedCount and bucketed P50 all verified against raw
scans before any refresh materialized); TDengine TCK 24/24 with the same
case passing via honest raw degradation (2 declared-capability skips).
DataAnalyticsService (biz/analytics) composes TSDB port primitives with
relational metadata into the nine coarse-grained reads of
docs/design/tsdb-abstraction.md §9.7: latest values, raw/M4 history,
per-series stats with percentiles, period comparison, entity ranking
(activity or mean/max/min), least-squares trends, threshold exceedance
reports with merged intervals, Pearson correlation, and a tenant-level
quality overview (coverage, silence, quality-code census).
Exposed as nine /analytics/* REST endpoints. The MCP gateway's tools/call
dispatches over HTTP to catalog rows synthesized from OpenAPI snapshots,
so these endpoints (x-dc3-ai annotated, fully @Schema-documented — the
data annotation gate enforces it) become the MCP tool surface after
make openapi + refreshToolCatalog; no Java tool classes are needed.
Hard guarantees: the tenant id always comes from the authenticated
principal, never the body, and the facade re-validates it (S11 double
gate); scan volume is bounded (20 series, 5000 samples per series, 90-day
windows) with structured over-budget rejections and honest degradation
notes instead of silent truncation; ambiguous name resolution fails with
the candidate list so an agent can disambiguate on the next turn.
correlate is dual-path — the store primitive when declared, otherwise
Pearson over aligned bucket pulls (verified against TDengine's
capability=false, exact on a perfectly-correlated fixture).
Gates: dc3-common-data 274/274 including 12 new analytics tests;
dc3-center-data assembly compiles.
dc3-tsdb-tdengine maps the port onto TDengine 3.x: one supertable
point_value tagged by (tenant, device, point), one deterministic subtable
pv_<t>_<d>_<p> auto-created on first insert via USING TAGS, database
created with PRECISION 'us' and KEEP 180. Connection settings live under
dc3.tsdb.tdengine.* (REST driver, own Hikari pool); selected by
dc3.tsdb.type=tdengine. Like the broker family, only the default adapter
(timescale) ships inside consumers — switching deployments add this
dependency plus DC3_TSDB_TYPE/DC3_TSDB_TDENGINE_URL.
The supertable mapping grilled four real issues out of the port
surface, each fixed and locked by the TCK:
- Timestamps never touch string form. The REST driver serializes
Timestamp parameters in the client JVM zone while the server parses
them as UTC — every write would shift by the deployment zone and
cursor pagination would drift per page without end. Instants now
travel as epoch-micro integer literals and reads return
CAST(ts AS BIGINT) — verified symmetric against the image.
- AS value collides with a reserved word (agg_value now);
INTERVAL takes a bare number read in the database (micro) precision.
- PERCENTILE only runs on single tables, so single-series percentiles
query the deterministic subtable directly; tenant-wide percentile is
refused rather than approximated.
- REST readiness must be probed with POST /rest/sql + basic auth; the
GET path-style route 404s even on a healthy server.
Honest capabilities: latencyHistogram=false (dashboard degrades to
zero-filled bins via a capability check) and correlation=false (future
analytics facade computes from bucketed pulls); rollups stay NONE until
S16 lands stream computing. TCK result: 23 tests, 0 failures, 0 errors,
2 capability-gated skips; the timescale suite re-ran 23/23 unchanged.
The design doc's capability matrix now reflects the adapter's declared
values instead of the pre-implementation estimates.
Phase 1b of docs/design/tsdb-abstraction.md: every point-value read and
write now flows through the dc3-tsdb port, and dc3-common-repository is
deleted outright (no compat aliases).
Write path — new PointValueIngestService orchestration in dc3-common-data:
the stale-owner lease guard leaves the history INSERT's cross-schema join
and resolves the active owner per distinct device through the existing
DeviceFacade.getActiveOwner chain; a caffeine-backed ingest idempotency
window (dc3.data.ingest.*, marked only after both writes commit) absorbs
MQ redeliveries now that the message_id unique index is retired; the
batch lands via TsdbStore.append (natural series+time upsert, INGEST_ORDER
kept for the fenced dc3_point_latest upsert).
Read path — history via last(), page() resolves name/enable filters to
series keys through relational metadata (tenant-wide when unrestricted,
cursor offset emulation capped at 10k) with count() for totals; the alarm
window backend becomes TsdbWindowDataSource over aggregate()/history().
Dashboards — all nine data-side hypertable statements move to the S13
analytics facet (count/bucketedCount/countByDimension/latencyHistogram/
lastSeenPerSeries) plus the dc3_point_latest projection for latestStream;
the manager topology statement crosses into the data center via a new
ListSeriesVolumes RPC backed by the new S13-5 seriesCounts primitive (a
point shared by several devices needs per-series counts, which
single-dimension grouping cannot reconstruct); both cross-schema joins
are gone.
Port/adapter — seriesCounts added with a TCK case (23 total); series
filters compile to row-value IN lists chunked at 500 pairs; bootstrap
adds the quality column idempotently and retires uk_point_value_event so
existing deployments converge on the new upsert; the adapter binds the
application-provided tsdbDataSource bean (the history dynamic-datasource
entry) instead of the routing primary.
Timestamps — BO<->Instant conversion is pinned to
TimeConstant.DEFAULT_ZONEID (the platform canonical zone the latest
projection type handler already writes with); the design note's "lock
UTC" is amended accordingly, together with the lease-guard and S13-5
deviations recorded in §6.2.
Seed/deploy — 05-iot-dc3-history.sql gains the quality column and swaps
the unique index; compose carries DC3_TSDB_TYPE; make deploy publishes
dc3-tsdb.
Also fixed along the way: root pom never managed dc3-mq-core (latent,
masked by full-reactor builds); PointValueMapper ran unrouted on the
master datasource (now @DS("history")); latestStream name enrichment
looked up a Long-keyed map with String keys.
Gates: tsdb TCK 23/23, dc3-common-data 262/262, e2e 26/26 including the
unmodified PostgresHypertableIT; full reactor green except dc3-mq-tck's
kafka/pulsar container-startup flake, reproduced identically twice and
unrelated to this change.
The agentic center (sessions/messages/actions/model providers) and the
mcp gateway tools/call path already exist, but the only data surface
today is the human CRUD api (PointValueController latest/list/history)
— nothing answers analytic questions. An agent's access pattern is
different: exploratory, combinational, cross-device comparisons, and
it needs a few coarse tools with typed json-schema inputs returning
self-contained statistical conclusions, not raw sample pages.
Three-layer design (§9.7):
- MCP tool layer: nine coarse tools (query_latest, query_history,
compute_stats, compare_periods, rank_entities, trend_analysis,
threshold_report, correlate, data_quality_report) — deliberately
capped; typed inputs; audited into the existing dc3_action table
- DataAnalyticsFacade (new, in the data center): resolves names to
series keys via relational metadata, composes port primitives into
conclusions, enriches names, injects tenant context from the mcp
auth (ai can never cross tenants), enforces scan budgets with
structured downgrade advice
- TsdbStore port: principle 'no new ops for ai' — everything composes
from S5/S6/S7/S13/S14/S15 except one capability-gated primitive,
correlation(a, b, window, alignBucket), which the three sql stores
express natively and iotdb degrades to facade-side bucketed pearson
Resolves open question 11: threshold-duration analysis is served by
threshold_report's bucket-locate + precise-history + merge composition
instead of a state-duration port op. TCK to 23 cases; phase 2 carries
the facade + mcp wiring.
The extraction is also the moment to outgrow the current logic, not
just replicate it. Five new capability groups, all default-off so the
phase-1 behavior-parity gate stays intact:
- S14 multi-series reads: history/last/aggregate take a unified
SeriesFilter (single series, series set, tenant-wide are one shape),
collapsing the single-series and analytics facets into one read API
- S15 chart-grade aggregates: FIRST/LAST join the enum (M4 rendering
over bucketed aggregates), PERCENTILE capability-gated
- S16 tiered retention + rollups: raw 30d -> 1m 1y -> 1h forever
(configurable), reads tier-transparent (bucketedAggregate serves from
the coarsest materialized tier), RollupSupport NATIVE (timescale
caggs, tdengine streams) / MANUAL (influx tasks) / NONE (iotdb
degrades to raw scan honestly); alarm windows always read raw
- S17 quality flag: OPC-UA-style int quality rides every sample now
(one int today, a whole-store migration later if skipped)
- S18 ops surface: listSeries for the migration CLI and coverage
audits, deadlines on every read, adapter-declared maxAppendBatch
with port-side chunking
API unified around the filter shape; TsdbCapabilities grows five
fields; TCK to 22 cases; matrix gains four rows; open questions to
eleven (rollup materialization set, state-duration analytics).
Re-verified every dashboard statement against the actual SQL and closed
the gaps the review found:
- S13 tenant-wide analytics facet: the dashboard statements are all
tenant-scoped (no per-series filter) — timeseries is tenant-wide
bucketed COUNT, top groups by device/point/driver, silentSources is
per-series lastSeen with a HAVING threshold, latencyHistogram bins a
derived expression, coverageGapItems joins dc3_manager cross-schema.
The single-series port API could not express any of these; the port
gains bucketedCount / countByDimension / lastSeenPerSeries /
latencyHistogram (capability-gated) with CursorPage and the supporting
records defined
- fixed dangling cross-references (§10.x -> §12.x) from the translation
- new §6.1 runtime surface: selection key, per-adapter config namespaces,
idempotent schema bootstrap responsibility, whole-batch append failure
semantics, health indicators, port-level metrics
- new §6.2 caller migration table incl. the LocalDateTime -> Instant
timezone hazard called out as a Phase 1 gate
- TCK extended to 16 cases (analytics facet coverage), capability matrix
gains the analytics and latency rows, open questions to nine (external
timescale topology, per-tenant retention)
Full Chinese rewrite of the tsdb port design (same structure and
content: verified inventory, S1-S12 semantics, port API, per-store
mappings, capability matrix, hard-parts arguments, 12-case TCK,
three-phase migration, seven open questions). Code identifiers, SQL and
enum names stay in English as is conventional.
Design the time-series store as the second pluggable family, following
the executed mq playbook (port + adapters + tck + dc3.tsdb.type).
Grounded in a verified inventory of today's TimescaleDB usage — the
lease-fenced transactional write path, the fencing-guarded latest
projection, the SQL-shaped RepositoryService, and the ten dashboard
statements that bypass the port (one cross-schema) — the design's two
load-bearing decisions are:
- the latest-value projection stays relational (dc3_point_latest): it is
OLTP state with a fencing-tuple guard that no TSDB expresses
uniformly; adapters store history only, which removes the hardest
cross-store semantic instead of negotiating it
- idempotency moves to an ingest-layer window; stores do natural
(series, timestamp) upsert with per-adapter duplicate policies
The port adds what the dashboards had to bypass for: bucketed
aggregation, cursor pagination and count; all ten bypassing statements
are mapped onto port primitives. Per-store mappings cover TimescaleDB,
TDengine 3, InfluxDB 3 (version strategy argued) and IoTDB, with a
capability matrix, a 12-case TCK, a three-phase migration (timescale
extraction first, behavior-identical) and seven open questions.
The broker-selection family (port + six certified adapters + tck) is a
deployment selection, not shared plumbing, so it moves out of dc3-common
into a top-level dc3-mq aggregator beside dc3-api — the same standing as
the gRPC contract layer, and a template for future pluggable families
(storage is next on the design board).
- dc3-common-mq -> dc3-mq/dc3-mq-core (port: API + core runtime)
- dc3-common-mq-{rabbitmq,kafka,rocketmq,pulsar,activemq,mqtt,tck} ->
dc3-mq/dc3-mq-{...}; no compatibility aliases — every coordinate is
renamed everywhere in one commit (business poms, coverage, root
dependency management)
- dc3-common sheds eight modules back to plain shared plumbing; the
device-access dc3-common-mqtt stays by design (protocol stack, not a
broker selection)
- Java packages deliberately stay io.github.pnoker.common.mq.* — the
facade modules set the precedent that packages do not track module
homes, keeping this a pom-only, zero-source refactor
- make deploy now publishes dc3-api + dc3-common + dc3-mq (third-party
driver authors compile against the port transitively)
- docs/mq-brokers.md and the design's module layout follow the new home
Verified: full-repo test-compile green; constant/data/driver/manager
unit suites unchanged and green (255/144/291); the rabbitmq contract
suite passes 12/12 from the new coordinates against a live broker,
proving adapter wiring survived the move.
Answer the broker-selection question for the device-access plane
mechanically: MQTT is an open wire protocol and the driver talks a
standard Paho client, so any conformant broker is a drop-in selection —
EMQX, Mosquitto, HiveMQ, NanoMQ, VerneMQ — via one URL
(dc3.driver.mqtt.url), no adapter layer needed.
MqttVendorNeutralityIT proves the property: the production option
builder (MqttUtil) plus the same client code performs a
publish/subscribe round trip against HiveMQ CE and Eclipse Mosquitto,
differing only in the broker URL. The compose default (dc3-emqx) is a
deployment choice; the guide's new device-access section documents the
URL-only selection, the no-vendor-code guardrail and the MQTT 3.1.1
dialect note (MQTT 5-only features belong to the internal-plane
adapter).
Full gated E2E now 26/26.
Close out the mq abstraction rollout:
- docs/mq-brokers.md: user-facing broker selection guide — how to pick
(DC3_MQ_TYPE + connection settings), the six certified brokers, the
capability matrix, and the at-least-once/outbox notes
- .env.example + dc3/env/dev.env: DC3_MQ_TYPE and the per-adapter
connection variables with safe rabbitmq defaults
- docker-compose.yml: DC3_MQ_TYPE and adapter endpoints flow through the
shared x-app-runtime-env anchor so every service sees the selection
- adapter configurations resolve connection settings from the canonical
dc3.mq.* property first, then the DC3_MQ_* environment variable, then
the transport-standard property (spring.kafka.bootstrap-servers),
making containerized selection deterministic
The gated E2E gate passes unmodified through the port: RabbitDeliveryIT
6/6, CommandCallE2eIT 4/4, EventReportE2eIT 3/3 (full suite 24/24
including postgres and harness ITs) against testcontainers brokers.
Phase 3 (4/4) and the final piece of the mq abstraction: every broker
named in the design is now a TCK-certified selection behind
dc3.mq.type. Pulsar suite 11 pass + 1 documented skip on the first run.
- topics map to persistent://public/default/dc3-<topic>, logical dead
topics to dc3-<topic>-dlq; LOAD_BALANCE rides a durable shared
subscription named after the consumer group (competing consumers,
retained while offline), BROADCAST rides an exclusive per-instance
subscription
- pulsar subscriptions start at the latest position natively, matching
the rabbit fresh-queue / kafka latest semantics with no seeding
- acknowledge(List<MessageId>) commits deliveries, negativeAcknowledge
requeues; reject(false) republishes to the -dlq topic and commits;
poison and exhausted batches dead-letter through the shared
synchronous bounded-retry semantics
- sends complete with the broker-issued message id (publisher
confirmation); batchReceive is the native batch (capability true);
arbitrary delays ride the port fallback like the other adapters
- tck harness against apachepulsar/pulsar standalone, or TCK_PULSAR_URL
for an externally managed broker
Phase 3 (3/4): RocketMQ adapter (classic client) for
dc3.mq.type=rocketmq, TCK 11 pass + 1 documented skip against a live
nameserver/broker pair.
The 5.x classic client has two behaviors that had blocked certification,
both now handled in the adapter:
- brand-new consumer groups replay the topic backlog regardless of
consumeFromWhere (LAST and TIMESTAMP both ignored), so fresh groups
are seeded explicitly: a short-lived pull consumer sets every queue
without a committed offset to the current max offset before the push
consumer starts, matching the rabbit fresh-queue / kafka latest
semantics; groups with a real offset trail are untouched
- subscribing to a topic that was never published to waits out the 30s
route refresh, so subscribe-time seeding also warms up missing topics
with a publish (the warm-up lands before the seeded offset, invisible
to every fresh group)
Further fixes surfaced by the suite: dead-letter republishes only the
standard envelope headers (RocketMQ system properties like
CONSUME_START_TIME are rejected on re-publish), and requeues request the
1s delay level so bounded retry exhaustion dead-letters within the
contract window instead of the 10s default.
Topics map to dc3-<topic> (no dots), dead letters to dc3-<topic>-dlq;
LOAD_BALANCE rides CLUSTERING consumer groups, BROADCASTING fans out per
instance; sync sends are the publisher confirmation; arbitrary delays go
through the port fallback (delay levels would quantize). The tck harness
is opt-in via TCK_ROCKETMQ_NAMESRV (testcontainers cannot manage the
nameserver/broker pair); a fresh-group isolation probe rides along in
the adapter module, also opt-in.
Rabbitmq, kafka, activemq and mqtt 5 adapters are implemented and
TCK-certified against live brokers; the matrix now reflects implemented
behavior including the group-durability row (mqtt documents the §13.8
variance) and the rabbit delay note (intrinsic TTL+DLX topics stay
server-side, arbitrary delays ride the port fallback). Rocketmq is
implemented but pending certification; pulsar pending.
Add the pluggable message broker ADR (docs/design/mq-abstraction.md):
- port + one adapter per broker, selected by dc3.mq.type (default rabbitmq)
- two subscription modes (LOAD_BALANCE / BROADCAST), batch consumption with
post-commit batch ack, bounded redelivery, lease-coupled instance expiry
- eight broker-specific semantics mapped per target broker with documented
fallbacks (delay -> local scheduler, JMS confirm -> driver outbox)
- capability matrix, 13-case TCK as the community adapter acceptance bar,
three-phase wire-compatible migration plan
- mqtt boundary (7.1): device-access plane stays vendor-neutral by protocol,
internal plane may run over any mqtt 5 broker via a dedicated adapter
Status: proposed; phase 1 (port + rabbitmq adapter + business migration)
implemented in the follow-up commits.
* chore(docs): move docs site to standalone pnoker/iot-dc3-docs repo
The VitePress docs site now lives in pnoker/iot-dc3-docs and serves docs.dc3.site from there. Remove the in-repo docs/ and all docs-only build/deploy wiring so nothing dangles.
- delete docs/ (source moved to pnoker/iot-dc3-docs)
- delete .github/workflows/docs.yml (deploy owned by iot-dc3-docs)
- drop npm /docs dependabot block; clean codeql/.gitignore docs paths
- remove Makefile docs-* targets
- relocate AI testing policy into dc3-web/tests/ so the required Web guardrail test is self-contained (was reading ../docs/...)
- rewire README image refs (7 READMEs) and in-repo doc links (README.ai.md, AGENTS.md, dc3/doc/README.md) to https://docs.dc3.site / the new repo
dc3/doc/{TITLE,CHANGE,USAGE}.md STAY - release tooling (make changelog, docker-ci) regenerates them; the docs repo snapshots them.
* fix(docs): repair pre-existing broken doc links found during migration
- 8 READMEs: /en|zh/introduction/technology-stack -> /en|zh/development/technology-stack (page lives under development/, not introduction/)
- dc3-web/tests/README.md: ../docs/test-debugging.md (missing) -> https://docs.dc3.site/en/frontend/test-debugging
* style(backend): reformat and rearrange java/xml sources
Apply code cleanup across dc3-common and dc3-driver: whitespace and
indent normalization, import sorting, member rearrangement, and javadoc
normalization. No semantic changes; compilation verified.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* style(docs): reformat vitepress vue diagram components
Expand compact svg templates and normalize formatting across diagram
components, the seo module, and the theme stylesheet. No behavior
changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* style(docs): reformat markdown pages
Normalize markdown formatting. Restore the <script setup> component
imports that an earlier unused-import pass had wrongly stripped; every
<Diagram> tag now resolves to a backing import again.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* style: reformat point view and trim pom whitespace
Normalize whitespace in the point view and drop trailing whitespace in
pom.xml. No behavior changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
