sweep every module to satisfy the new method-name rule, renaming
about 40 methods together with their interface, implementation, test
and cli call sites: store find() lookups become get(), analytics
queryLatest/queryHistory become getLatest/getHistory, sql helper
query() becomes spec()/sql(), jwks fetch() becomes load(), and the
cursor record component queryDigest becomes requestDigest
no behavior changes; full unit suite (5278 tests) and both quality
gates stay green
the client_credentials grant could never complete: the authorization
insert left principal_type null, state/code/refresh columns were
declared not null for a grant that never sets them, and the connection
grant type silently fell back to authorization_code because builders
match on the enum value, not the name
- stamp the service-account principal type on the cc authorization
- bind explicit nulls for grant-specific columns in insertauthorization
- make state_hash, authorization_code_hash and the refresh hashes
nullable in seed ddl and resync the embedded schema fingerprint
Backfill the one-line Javadoc required by check_documentation on public
top-level types and non-override public methods (1305 sites across 313
files), matching the existing single-line contract style: each doc states
the behaviour contract without @param/@return tags.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Add README files for the four dc3-db-postgres-* modules, state the
artifactId in the r2dbc module READMEs, list the new modules in the
dc3-db README, and document the CallTool RPC in dc3-api-auth.
Co-Authored-By: Claude Code <noreply@anthropic.com>
7fde11134 moved the R2dbc store adapters into per-domain postgres modules
but left the tck suite importing them from dc3-common, breaking test
compilation. Add the four module dependencies and update the imports.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Relocate all 64 R2dbc*Store implementations from dc3-common-{auth,manager,data,agentic} repositories into new dc3-db-postgres-{auth,manager,data,agentic} adapter modules (package io.github.pnoker.db.postgres.<domain>), each registered through its own AutoConfiguration. Reactive*Store ports and filter types stay in the business modules; services are untouched. Drops stale MyBatis wording from AGENTS.md and init runner javadoc.
- migrate auth, manager, data, and agentic persistence to R2DBC
- hard cut pagination, facade, gateway, web, and CLI contracts
- remove legacy database and TSDB adapters and document validation gates
BREAKING CHANGE: platform persistence now requires PostgreSQL 18 with TimescaleDB and the new reactive API contracts.
- derive pagination DbType from the selected dc3-db adapter with a fail-fast
when a non-default dc3.db.type lacks its adapter jar
- skip postgres profile activation for mariadb, not just mysql
- fork OAuthMcpMapper/AlertMapper statements for mysql and mariadb
(CAST AS JSON, ::text casts, ODKU value references)
- honor GET_LOCK verdict in resource registry sync instead of assuming
acquisition on every engine
- pg2mysql_seed.py: convert partial unique indexes to stored generated guard
columns instead of dropping them; add 99-iot-dc3-grants.sql
- add DC3_DB_URL/DC3_DB_HISTORY_URL per-service overrides in center configs
- cover the new mapper forks in the db TCK; fix AGPL headers in spring
metadata files
- add READMEs for the dc3-mq core, six broker adapters, and the contract suite
- add READMEs for the dc3-tsdb core, four store adapters, and the contract suite
- add READMEs for the dc3-db core, three dialect adapters, and the contract suite
Follow-up review of the comment pass found three spots where new
javadoc landed next to pre-existing blocks:
- AnalyticsModel: SeriesSelector/TimeRange/HistoryPoint carried the same
doc twice (the batch injector did not notice the originals).
- MybatisPlusConfig: an orphaned copy of the old mybatisPlusInterceptor
doc sat above databaseIdProvider's own javadoc.
- EntityStateMapper: the parameter javadoc of the retired
claimExpiredDevices method was left dangling above
selectExpiredForClaim's doc.
All three removed; the original mybatisPlusInterceptor doc survives
(the one-line summary I added earlier was replaced by restoring the
full contract note in place). Verified by a repository-wide scan for
adjacent javadoc pairs — now zero. data suite green, compile green.
A comment-health audit driven by an AST scanner (regex scanners kept
false-positiving on annotation-separated javadoc; the AST pass with
constructor and @Override-implementation exemptions is what produced a
trustworthy worklist) ended with a much smaller true gap than the raw
numbers suggested: class-level javadoc is already at 100% repository-wide.
What actually gets filled here:
- AnalyticsModel: all 30 nested records of the S19 analytics facet get
maintainer-voice javadoc (the @Schema descriptions serve the API side;
these serve code readers).
- The whole entity/ext family (29 files, 47 nested classes): every
Content/Ui/Validation/Security/AppliesTo/Template/Dedup/... schema
class states what it actually holds — read from the fields, not
templated.
- Adapter families: the @Bean methods of all six MQ adapters, all four
TSDB adapters and the three DB dialect adapters get one-line
maintainer docs (what binds to what, what is overridable).
- Nested support types across data/agentic/constant/public/facade:
NotifyBindKey, CacheTuning, Credential, dashboard VOs' Item/BucketVO,
ExpireListener (plus its null-doc placeholder javadoc replaced with a
real contract), agentic chat/stream VOs' choice/delta/usage shapes,
tool-context records, RequestHeader envelopes, FleetSummary,
visualization Type/Scale codes.
- PointValueSampleConverter's boundary methods now state the port-side
contract (quality default injection point, batch order preservation).
Comment drift fixed: the class javadoc of the can/mqtt/opc-da driver
skeletons claimed "see TODO markers in method bodies" — those markers
do not exist; the claims are corrected to plain work-in-progress notes.
The genuine TODOs (zigbee API verification, lwm2m lookup, ethernet-ip
CIP framing, CAN JNI) stay: they mark real unimplemented behavior.
Vendored-code boundary: the opc-da module carries a ported
org.openscada library (46 files, package org.openscada.*). Its trivial
accessors stay bare on purpose — filler comments on get/set pairs are
exactly the kind of noise this pass exists to remove — but the
non-obvious JIStruct wire-decoder methods (getStruct/fromStruct
FILETIME + three DO wrappers) now document the COM bridge shape.
Gates: full-repo compile green; data/model/constant/agentic/public
test suites green.
dc3-common-jdbc was the family's core in disguise — MyBatis-Plus wiring,
the tenant-line handler, the databaseIdProvider, the jdbc profile. It now
lives as dc3-db/dc3-db-core, matching the dc3-mq-core / dc3-tsdb-core
shape: one aggregator, port first, adapters after, tck last. Artifact
renamed, packages deliberately untouched (same convention as the mq
family move); twelve poms rewired, the family aggregator puts core
first, and the db-tck now depends on the core explicitly instead of
riding transitives.
dc3-common-sql stays where it is on purpose: it is not platform dialect
plumbing but the base class of the four JDBC *device drivers*
(mysql/oracle/postgresql/sqlserver drivers that connect to external
databases being collected from) and depends on the driver SDK — under
dc3-db it would claim a meaning it does not have.
Gates: db-tck 24/24 (postgres + mysql + mariadb), data 274/274,
full-repo compile green.
dc3-db-mariadb brings MariaDB 10.6+ behind dc3.db.type=mariadb with the
official mariadb-java-client and the same dual-dialect contract suite.
MariaDB shares the MySQL seed wholesale (utf8mb4_unicode_ci — the 0900
collation is MySQL-8-only and MariaDB rejects it outright, so the seed
translator and the hand-written 05 both moved to the portably-supported
collation, re-verified against mysql:8.4).
The mapper forks differ from the mysql twins in exactly one dimension,
verified against a live 10.11 container first: MariaDB never adopted
the AS-new ODKU row alias (syntax error on 10.11), so its 15 twins
reference VALUES(col) instead — everything else (GET_LOCK/RELEASE_LOCK,
FOR UPDATE SKIP LOCKED, recursive CTEs, INTERVAL ? SECOND, the
JSON_UNQUOTE(JSON_EXTRACT(...)) double-hop, REGEXP_REPLACE without the
g flag, DAYOFWEEK/HOUR units) proved compatible and rides the same
statement bodies.
Scope decision recorded in docs/db-dialects.md: DB2, H2 and HSQLDB are
explicitly out (dialect cost with zero demand; embedded test databases
unfit for multi-service concurrent IoT writes), SQLite equally unfit
for the multi-service shape (database-level write lock, no network
listener). Oracle and SQL Server stay on the roadmap with their real
obstacles spelled out (MERGE-heavy rewrite; no ARM64 mssql image for
local certification on Apple Silicon) — not started without a demand
signal.
Gates: db-tck 24/24 (postgres + mysql + mariadb, 8 each),
data/auth/manager suites green (274 in data), full-repo compile green.
R2c of docs/design/storage-abstraction.md §3. dc3-db-tck runs one set of
mapper-level assertions against both engines with the real mapper XML —
the databaseIdProvider routing is part of what gets certified, not just
the SQL: the fenced latest-value upsert's three-level tiebreak (fencing
token, then time, then sequence, then message id), the state upsert +
same-transaction re-select shape, the three-step expired-lease claim
(lock-and-read FOR UPDATE SKIP LOCKED, flip offline, derive the
post-update view), the lease upserts' row-local increments (token moves
only on ownership change), the row-level revision triggers (insert and
meaningful update bump, trivial update does not), and the tool-catalog
query's encoded-JSON double-hop.
Harness notes worth keeping: one SqlSessionFactory per database mirrors
the production @DS wiring (state/latest/leases/catalog span four
databases with unqualified table names); the mapper XML loads through
XMLMapperBuilder with the affectData mybatis-plus DTD extension
stripped; postgres registers the timestamptz type handler. Both
harnesses learned to distrust podman: log-following never fires, so
mysql waits on the listening TCP port and postgres polls a real JDBC
handshake (the port opens during initdb's local-only phase and closes
again for the restart). The postgres harness mounts seed files 01..07
individually — 00's age/pgvector extensions exist only in the
production base image — with 05 trimmed at runtime to the
dc3_point_latest projection (the hypertable half needs the base
image's TimescaleDB function set; the dialect contract doesn't touch
it).
The catalog fixture plants its api row through a parameterized insert:
the escaped-JSON value must not travel through SQL text (MySQL's
default backslash escapes eat the \" the value depends on) nor bind
uniformly (PG rejects varchar->json, MySQL rejects the binary
Types.OTHER binding) — the fixture branches on the engine.
Selection guide published at docs/db-dialects.md (dialect matrix,
swap procedure, the semantic decisions the suite locks); the storage
design's status header moves to Implemented with an R1/R2 closeout
note.
Gates: db-tck 16/16 (8 postgres + 8 mysql), data/auth/manager suites
green (274 in data), e2e 26/26, full-repo compile green.
R2a of docs/design/storage-abstraction.md §3. MySQL 8 becomes a
first-class relational deployment behind dc3.db.type=mysql:
- dc3-db-mysql module: mysql-connector-j, MYSQL pagination DbType, the
mysql profile (driver + jdbc-type-for-null), activated only when
dc3.db.type=mysql — the postgres profile and DbType stay the default
for unset values, so existing deployments are untouched. The neutral
jdbc module now exposes a VendorDatabaseIdProvider (postgres/mysql
aliases) so mapper XML can fork statements by engine.
- RETURNING decoupled where MySQL has no equivalent: the state upsert
and the lease-state upsert no longer RETURN rows — callers re-select
by the natural key in the same transaction (portable shape, one extra
keyed read on both engines). The expired-lease claim becomes three
portable steps in one TransactionTemplate: lock-and-read (FOR UPDATE
SKIP LOCKED works identically on both engines), flip offline (INTERVAL
expression is the only forked fragment), derive the post-update view
in Java from the locked rows — the update is deterministic per column.
- Sequences retired in favor of row-local increments: the assignment
version and the fencing token advance as col = col + 1 on every
change. Per-driver / per-device monotonicity is the property callers
and stale-writer checks rely on; the global sequences only ever
delivered it incidentally, and MySQL has no sequence objects.
- Forked statements (databaseId twins): the point-latest fencing upsert
(row-value tuple guard expanded into per-column IF chains — MySQL has
neither row comparisons nor WHERE on ON DUPLICATE KEY UPDATE), the
driver-instance upsert, both lease upserts, the driver lock (MySQL
serializes on the driver row itself: SELECT ... FOR UPDATE, scoped to
the transaction like the advisory lock it replaces), the registry
lock (GET_LOCK + an explicitly paired advisoryUnlock — a no-op on
postgres; the sync service wraps its critical section in try/finally),
the tool-catalog candidate query (|| -> CONCAT, regexp_replace flag,
and the (ext ->> 'content')::json double-hop re-expressed through
JSON_UNQUOTE(JSON_EXTRACT(...)) — the content value is a JSON-encoded
string, not a nested object, verified against live data), four alert
statements' ->> extractions, hourly counts (DATE_FORMAT hour bucket),
and the activity heatmap (DAYOFWEEK/HOUR units, dow 0 = Sunday).
- Portable rewrites that needed no fork: token_metadata::TEXT ->
CONCAT(token_metadata, '') (both engines return text; only PG JDBC
needs the nudge past PGobject), typeDistribution's GROUP BY now
references the renamed bucket_key alias (the bare `key` was a MySQL
reserved-word landmine).
Gates: data/auth/manager suites green (274 in data), e2e 26/26.
MySQL seed and the dual-dialect DAL TCK follow in R2b/R2c.
R1 of docs/design/storage-abstraction.md §3 — zero behavior change:
Portable SQL rewrites (verified against a live PostgreSQL before landing):
- COUNT(*) FILTER (WHERE) x3 -> COALESCE(SUM(CASE WHEN ... THEN 1 ELSE 0
END), 0); the COALESCE keeps the empty-set result at 0 where bare SUM
would return NULL.
- dailyTrend's generate_series + date_trunc calendar spine -> a
RECURSIVE CTE day ladder (CAST(day + INTERVAL '1' DAY AS DATE) keeps
the column type stable on both engines — the bare date+interval form
widens to timestamp on PostgreSQL and fails the recursive term's type
check, caught live); day rendering via CONCAT instead of ::text.
- OAuthMcpMapper: '{} '::JSON literals -> '{}' (unknown literals coerce
to the JSON column on both engines), NULL::BIGINT -> NULL in the
UNION arm, #{riskLevel}::text parameter casts dropped (the text column
compares untyped parameters fine on both).
- The reserved-word alias AS key (a MySQL landmine hiding inside three
otherwise-portable statements) renamed to bucket_key / bucketKey
across both Dashboard mappers and BucketRow.
Module split: dc3-common-postgres -> dc3-common-jdbc (MyBatis-Plus
wiring, tenant-line handler, generator utils, the new dialect-neutral
jdbc profile carrying the shared mybatis-plus/hikari plumbing) + the
top-level dc3-db family's dc3-db-postgres (driver, timestamptz type
handler, postgres profile delta, and the pagination DbType bean the
neutral config now injects — defaulting to PostgreSQL so a bare
jdbc-only classpath behaves exactly as before). Packages deliberately
unchanged, mirroring the mq/tsdb family moves; all nine dependent poms
rewired; make deploy publishes the dc3-db family.
The jdbc module's interceptor integration test now pins an explicit
embedded H2 datasource — the module carries no driver on its main
classpath, so context inference would fail without it.
Gates: data/auth/manager/jdbc/db-postgres unit suites green (274 in
data), full-repo compile green, e2e 26/26.
pnoker