Relocate all 64 R2dbc*Store implementations from dc3-common-{auth,manager,data,agentic} repositories into new dc3-db-postgres-{auth,manager,data,agentic} adapter modules (package io.github.pnoker.db.postgres.<domain>), each registered through its own AutoConfiguration. Reactive*Store ports and filter types stay in the business modules; services are untouched. Drops stale MyBatis wording from AGENTS.md and init runner javadoc.
- migrate auth, manager, data, and agentic persistence to R2DBC
- hard cut pagination, facade, gateway, web, and CLI contracts
- remove legacy database and TSDB adapters and document validation gates
BREAKING CHANGE: platform persistence now requires PostgreSQL 18 with TimescaleDB and the new reactive API contracts.
dc3-common-jdbc was the family's core in disguise — MyBatis-Plus wiring,
the tenant-line handler, the databaseIdProvider, the jdbc profile. It now
lives as dc3-db/dc3-db-core, matching the dc3-mq-core / dc3-tsdb-core
shape: one aggregator, port first, adapters after, tck last. Artifact
renamed, packages deliberately untouched (same convention as the mq
family move); twelve poms rewired, the family aggregator puts core
first, and the db-tck now depends on the core explicitly instead of
riding transitives.
dc3-common-sql stays where it is on purpose: it is not platform dialect
plumbing but the base class of the four JDBC *device drivers*
(mysql/oracle/postgresql/sqlserver drivers that connect to external
databases being collected from) and depends on the driver SDK — under
dc3-db it would claim a meaning it does not have.
Gates: db-tck 24/24 (postgres + mysql + mariadb), data 274/274,
full-repo compile green.
dc3-db-mariadb brings MariaDB 10.6+ behind dc3.db.type=mariadb with the
official mariadb-java-client and the same dual-dialect contract suite.
MariaDB shares the MySQL seed wholesale (utf8mb4_unicode_ci — the 0900
collation is MySQL-8-only and MariaDB rejects it outright, so the seed
translator and the hand-written 05 both moved to the portably-supported
collation, re-verified against mysql:8.4).
The mapper forks differ from the mysql twins in exactly one dimension,
verified against a live 10.11 container first: MariaDB never adopted
the AS-new ODKU row alias (syntax error on 10.11), so its 15 twins
reference VALUES(col) instead — everything else (GET_LOCK/RELEASE_LOCK,
FOR UPDATE SKIP LOCKED, recursive CTEs, INTERVAL ? SECOND, the
JSON_UNQUOTE(JSON_EXTRACT(...)) double-hop, REGEXP_REPLACE without the
g flag, DAYOFWEEK/HOUR units) proved compatible and rides the same
statement bodies.
Scope decision recorded in docs/db-dialects.md: DB2, H2 and HSQLDB are
explicitly out (dialect cost with zero demand; embedded test databases
unfit for multi-service concurrent IoT writes), SQLite equally unfit
for the multi-service shape (database-level write lock, no network
listener). Oracle and SQL Server stay on the roadmap with their real
obstacles spelled out (MERGE-heavy rewrite; no ARM64 mssql image for
local certification on Apple Silicon) — not started without a demand
signal.
Gates: db-tck 24/24 (postgres + mysql + mariadb, 8 each),
data/auth/manager suites green (274 in data), full-repo compile green.
R2c of docs/design/storage-abstraction.md §3. dc3-db-tck runs one set of
mapper-level assertions against both engines with the real mapper XML —
the databaseIdProvider routing is part of what gets certified, not just
the SQL: the fenced latest-value upsert's three-level tiebreak (fencing
token, then time, then sequence, then message id), the state upsert +
same-transaction re-select shape, the three-step expired-lease claim
(lock-and-read FOR UPDATE SKIP LOCKED, flip offline, derive the
post-update view), the lease upserts' row-local increments (token moves
only on ownership change), the row-level revision triggers (insert and
meaningful update bump, trivial update does not), and the tool-catalog
query's encoded-JSON double-hop.
Harness notes worth keeping: one SqlSessionFactory per database mirrors
the production @DS wiring (state/latest/leases/catalog span four
databases with unqualified table names); the mapper XML loads through
XMLMapperBuilder with the affectData mybatis-plus DTD extension
stripped; postgres registers the timestamptz type handler. Both
harnesses learned to distrust podman: log-following never fires, so
mysql waits on the listening TCP port and postgres polls a real JDBC
handshake (the port opens during initdb's local-only phase and closes
again for the restart). The postgres harness mounts seed files 01..07
individually — 00's age/pgvector extensions exist only in the
production base image — with 05 trimmed at runtime to the
dc3_point_latest projection (the hypertable half needs the base
image's TimescaleDB function set; the dialect contract doesn't touch
it).
The catalog fixture plants its api row through a parameterized insert:
the escaped-JSON value must not travel through SQL text (MySQL's
default backslash escapes eat the \" the value depends on) nor bind
uniformly (PG rejects varchar->json, MySQL rejects the binary
Types.OTHER binding) — the fixture branches on the engine.
Selection guide published at docs/db-dialects.md (dialect matrix,
swap procedure, the semantic decisions the suite locks); the storage
design's status header moves to Implemented with an R1/R2 closeout
note.
Gates: db-tck 16/16 (8 postgres + 8 mysql), data/auth/manager suites
green (274 in data), e2e 26/26, full-repo compile green.
R2a of docs/design/storage-abstraction.md §3. MySQL 8 becomes a
first-class relational deployment behind dc3.db.type=mysql:
- dc3-db-mysql module: mysql-connector-j, MYSQL pagination DbType, the
mysql profile (driver + jdbc-type-for-null), activated only when
dc3.db.type=mysql — the postgres profile and DbType stay the default
for unset values, so existing deployments are untouched. The neutral
jdbc module now exposes a VendorDatabaseIdProvider (postgres/mysql
aliases) so mapper XML can fork statements by engine.
- RETURNING decoupled where MySQL has no equivalent: the state upsert
and the lease-state upsert no longer RETURN rows — callers re-select
by the natural key in the same transaction (portable shape, one extra
keyed read on both engines). The expired-lease claim becomes three
portable steps in one TransactionTemplate: lock-and-read (FOR UPDATE
SKIP LOCKED works identically on both engines), flip offline (INTERVAL
expression is the only forked fragment), derive the post-update view
in Java from the locked rows — the update is deterministic per column.
- Sequences retired in favor of row-local increments: the assignment
version and the fencing token advance as col = col + 1 on every
change. Per-driver / per-device monotonicity is the property callers
and stale-writer checks rely on; the global sequences only ever
delivered it incidentally, and MySQL has no sequence objects.
- Forked statements (databaseId twins): the point-latest fencing upsert
(row-value tuple guard expanded into per-column IF chains — MySQL has
neither row comparisons nor WHERE on ON DUPLICATE KEY UPDATE), the
driver-instance upsert, both lease upserts, the driver lock (MySQL
serializes on the driver row itself: SELECT ... FOR UPDATE, scoped to
the transaction like the advisory lock it replaces), the registry
lock (GET_LOCK + an explicitly paired advisoryUnlock — a no-op on
postgres; the sync service wraps its critical section in try/finally),
the tool-catalog candidate query (|| -> CONCAT, regexp_replace flag,
and the (ext ->> 'content')::json double-hop re-expressed through
JSON_UNQUOTE(JSON_EXTRACT(...)) — the content value is a JSON-encoded
string, not a nested object, verified against live data), four alert
statements' ->> extractions, hourly counts (DATE_FORMAT hour bucket),
and the activity heatmap (DAYOFWEEK/HOUR units, dow 0 = Sunday).
- Portable rewrites that needed no fork: token_metadata::TEXT ->
CONCAT(token_metadata, '') (both engines return text; only PG JDBC
needs the nudge past PGobject), typeDistribution's GROUP BY now
references the renamed bucket_key alias (the bare `key` was a MySQL
reserved-word landmine).
Gates: data/auth/manager suites green (274 in data), e2e 26/26.
MySQL seed and the dual-dialect DAL TCK follow in R2b/R2c.
R1 of docs/design/storage-abstraction.md §3 — zero behavior change:
Portable SQL rewrites (verified against a live PostgreSQL before landing):
- COUNT(*) FILTER (WHERE) x3 -> COALESCE(SUM(CASE WHEN ... THEN 1 ELSE 0
END), 0); the COALESCE keeps the empty-set result at 0 where bare SUM
would return NULL.
- dailyTrend's generate_series + date_trunc calendar spine -> a
RECURSIVE CTE day ladder (CAST(day + INTERVAL '1' DAY AS DATE) keeps
the column type stable on both engines — the bare date+interval form
widens to timestamp on PostgreSQL and fails the recursive term's type
check, caught live); day rendering via CONCAT instead of ::text.
- OAuthMcpMapper: '{} '::JSON literals -> '{}' (unknown literals coerce
to the JSON column on both engines), NULL::BIGINT -> NULL in the
UNION arm, #{riskLevel}::text parameter casts dropped (the text column
compares untyped parameters fine on both).
- The reserved-word alias AS key (a MySQL landmine hiding inside three
otherwise-portable statements) renamed to bucket_key / bucketKey
across both Dashboard mappers and BucketRow.
Module split: dc3-common-postgres -> dc3-common-jdbc (MyBatis-Plus
wiring, tenant-line handler, generator utils, the new dialect-neutral
jdbc profile carrying the shared mybatis-plus/hikari plumbing) + the
top-level dc3-db family's dc3-db-postgres (driver, timestamptz type
handler, postgres profile delta, and the pagination DbType bean the
neutral config now injects — defaulting to PostgreSQL so a bare
jdbc-only classpath behaves exactly as before). Packages deliberately
unchanged, mirroring the mq/tsdb family moves; all nine dependent poms
rewired; make deploy publishes the dc3-db family.
The jdbc module's interceptor integration test now pins an explicit
embedded H2 datasource — the module carries no driver on its main
classpath, so context inference would fail without it.
Gates: data/auth/manager/jdbc/db-postgres unit suites green (274 in
data), full-repo compile green, e2e 26/26.
pnoker