- migrate auth, manager, data, and agentic persistence to R2DBC
- hard cut pagination, facade, gateway, web, and CLI contracts
- remove legacy database and TSDB adapters and document validation gates
BREAKING CHANGE: platform persistence now requires PostgreSQL 18 with TimescaleDB and the new reactive API contracts.
R1 of docs/design/storage-abstraction.md §3 — zero behavior change:
Portable SQL rewrites (verified against a live PostgreSQL before landing):
- COUNT(*) FILTER (WHERE) x3 -> COALESCE(SUM(CASE WHEN ... THEN 1 ELSE 0
END), 0); the COALESCE keeps the empty-set result at 0 where bare SUM
would return NULL.
- dailyTrend's generate_series + date_trunc calendar spine -> a
RECURSIVE CTE day ladder (CAST(day + INTERVAL '1' DAY AS DATE) keeps
the column type stable on both engines — the bare date+interval form
widens to timestamp on PostgreSQL and fails the recursive term's type
check, caught live); day rendering via CONCAT instead of ::text.
- OAuthMcpMapper: '{} '::JSON literals -> '{}' (unknown literals coerce
to the JSON column on both engines), NULL::BIGINT -> NULL in the
UNION arm, #{riskLevel}::text parameter casts dropped (the text column
compares untyped parameters fine on both).
- The reserved-word alias AS key (a MySQL landmine hiding inside three
otherwise-portable statements) renamed to bucket_key / bucketKey
across both Dashboard mappers and BucketRow.
Module split: dc3-common-postgres -> dc3-common-jdbc (MyBatis-Plus
wiring, tenant-line handler, generator utils, the new dialect-neutral
jdbc profile carrying the shared mybatis-plus/hikari plumbing) + the
top-level dc3-db family's dc3-db-postgres (driver, timestamptz type
handler, postgres profile delta, and the pagination DbType bean the
neutral config now injects — defaulting to PostgreSQL so a bare
jdbc-only classpath behaves exactly as before). Packages deliberately
unchanged, mirroring the mq/tsdb family moves; all nine dependent poms
rewired; make deploy publishes the dc3-db family.
The jdbc module's interceptor integration test now pins an explicit
embedded H2 datasource — the module carries no driver on its main
classpath, so context inference would fail without it.
Gates: data/auth/manager/jdbc/db-postgres unit suites green (274 in
data), full-repo compile green, e2e 26/26.
Phase 1b of docs/design/tsdb-abstraction.md: every point-value read and
write now flows through the dc3-tsdb port, and dc3-common-repository is
deleted outright (no compat aliases).
Write path — new PointValueIngestService orchestration in dc3-common-data:
the stale-owner lease guard leaves the history INSERT's cross-schema join
and resolves the active owner per distinct device through the existing
DeviceFacade.getActiveOwner chain; a caffeine-backed ingest idempotency
window (dc3.data.ingest.*, marked only after both writes commit) absorbs
MQ redeliveries now that the message_id unique index is retired; the
batch lands via TsdbStore.append (natural series+time upsert, INGEST_ORDER
kept for the fenced dc3_point_latest upsert).
Read path — history via last(), page() resolves name/enable filters to
series keys through relational metadata (tenant-wide when unrestricted,
cursor offset emulation capped at 10k) with count() for totals; the alarm
window backend becomes TsdbWindowDataSource over aggregate()/history().
Dashboards — all nine data-side hypertable statements move to the S13
analytics facet (count/bucketedCount/countByDimension/latencyHistogram/
lastSeenPerSeries) plus the dc3_point_latest projection for latestStream;
the manager topology statement crosses into the data center via a new
ListSeriesVolumes RPC backed by the new S13-5 seriesCounts primitive (a
point shared by several devices needs per-series counts, which
single-dimension grouping cannot reconstruct); both cross-schema joins
are gone.
Port/adapter — seriesCounts added with a TCK case (23 total); series
filters compile to row-value IN lists chunked at 500 pairs; bootstrap
adds the quality column idempotently and retires uk_point_value_event so
existing deployments converge on the new upsert; the adapter binds the
application-provided tsdbDataSource bean (the history dynamic-datasource
entry) instead of the routing primary.
Timestamps — BO<->Instant conversion is pinned to
TimeConstant.DEFAULT_ZONEID (the platform canonical zone the latest
projection type handler already writes with); the design note's "lock
UTC" is amended accordingly, together with the lease-guard and S13-5
deviations recorded in §6.2.
Seed/deploy — 05-iot-dc3-history.sql gains the quality column and swaps
the unique index; compose carries DC3_TSDB_TYPE; make deploy publishes
dc3-tsdb.
Also fixed along the way: root pom never managed dc3-mq-core (latent,
masked by full-reactor builds); PointValueMapper ran unrouted on the
master datasource (now @DS("history")); latestStream name enrichment
looked up a Long-keyed map with String keys.
Gates: tsdb TCK 23/23, dc3-common-data 262/262, e2e 26/26 including the
unmodified PostgresHypertableIT; full reactor green except dc3-mq-tck's
kafka/pulsar container-startup flake, reproduced identically twice and
unrelated to this change.
The broker-selection family (port + six certified adapters + tck) is a
deployment selection, not shared plumbing, so it moves out of dc3-common
into a top-level dc3-mq aggregator beside dc3-api — the same standing as
the gRPC contract layer, and a template for future pluggable families
(storage is next on the design board).
- dc3-common-mq -> dc3-mq/dc3-mq-core (port: API + core runtime)
- dc3-common-mq-{rabbitmq,kafka,rocketmq,pulsar,activemq,mqtt,tck} ->
dc3-mq/dc3-mq-{...}; no compatibility aliases — every coordinate is
renamed everywhere in one commit (business poms, coverage, root
dependency management)
- dc3-common sheds eight modules back to plain shared plumbing; the
device-access dc3-common-mqtt stays by design (protocol stack, not a
broker selection)
- Java packages deliberately stay io.github.pnoker.common.mq.* — the
facade modules set the precedent that packages do not track module
homes, keeping this a pom-only, zero-source refactor
- make deploy now publishes dc3-api + dc3-common + dc3-mq (third-party
driver authors compile against the port transitively)
- docs/mq-brokers.md and the design's module layout follow the new home
Verified: full-repo test-compile green; constant/data/driver/manager
unit suites unchanged and green (255/144/291); the rabbitmq contract
suite passes 12/12 from the new coordinates against a live broker,
proving adapter wiring survived the move.
Complete phase 1 of the mq abstraction design: business code compiles
against the broker-neutral port with zero amqp classes, and the legacy
dc3-common-rabbitmq module is deleted.
- dc3-common-driver: 8 sender methods plus the sqlite outbox now publish
through MessageSender (outbox keeps durability; legacy physical routing
keys in pending rows are normalized on republish); metadata/command/
point-command receivers become @Dc3Listener subscriptions with
placeholder-driven per-instance queues; DriverTopicConfig dissolved into
adapter subscribe-time declarations
- dc3-common-data: command and point-command dispatch use sendConfirmed
with the correlation-id header; state timeout, device scan tick, notify
tasks and metadata events send through logical topics; all 13 receivers
including the batch point-value consumer migrate to @Dc3Listener;
DataTopicConfig, PointValueRabbitConfig and the duplicated batch
properties class move into the port/adapter (same config keys)
- dc3-common-manager / facade-local-manager: metadata fan-out via the port
- dead-letter consumers read the correlation id from the standardized
header the adapter mirrors onto the amqp property
- 14 test classes adapted to the new listener/sender shapes; module poms
swap dc3-common-rabbitmq for dc3-common-mq + dc3-common-mq-rabbitmq
Verified: unit suites green for constant (291 incl. boundary guard),
data (165), driver (144), manager (255); full-repo main and test compile
green. Container-backed E2E (RabbitDeliveryIT) intentionally not run here.
- Replace obsolete nexus-staging-maven-plugin with central-publishing-
maven-plugin in dc3-api and dc3-common
- Remove 20 redundant version declarations from dc3-coverage dependencies
- Fix snmp4j version from non-existent 3.8.3 to available 3.7.4
Introduce two reactor modules that exist purely to host repository-wide
test concerns:
- dc3-coverage (packaging=pom) runs jacoco:report-aggregate against all
contributing modules and applies a tree-wide jacoco:check rule. The
aggregator excludes generated sources, deployable Application classes,
POJO entity packages and the third-party modbus4j sero utilities so
the percentage stays anchored on real business logic. Coverage gates
start at 0% and tighten in later stages as more tests land.
- dc3-e2e (packaging=jar) reserves a place for the docker-compose driven
end-to-end suite delivered in stage S8. Unit and integration phases
are skipped by default and only the e2e profile turns them on.
Both modules are wired into the root reactor.
pnoker