356 Commits

Author SHA1 Message Date
pnoker fb97274691 build(deps): bump spring boot to 4.1.1 and refresh dependency stack
- Spring Boot 4.0.7 -> 4.1.1, Spring Cloud 2025.1.2 -> 2025.1.3,
  Spring AI 2.0.0 -> 2.0.1, springdoc 3.0.3 -> 3.1.0
- Migrate gRPC starters to the Boot-owned spring-boot-starter-grpc-client/server
  line (org.springframework.grpc starter stops at 1.0.3 and mixes badly with the
  spring-grpc-core 1.1.1 managed by the Boot 4.1 BOM), and move the
  spring.grpc.channels.* YAML keys to the new spring.grpc.channel.* format
- Refresh driver protocol libraries (modbus4j 3.1.0, jSerialComm 2.11.4,
  bacnet4j 6.1.1, gurux-dlms 4.0.96, j60870 1.8.0, snmp4j 3.13.1) and drop the
  flow-control overrides removed from the modbus4j SerialPortWrapper interface
- Refresh jackson 2.x/3.x, protobuf, okhttp, micrometer, oshi, sqlite-jdbc,
  swagger-core, instancio, mockito, logstash-logback-encoder 9.0 and the
  graalvm native plugin; keep commons-io (broken timestamp-version candidate)
  and j-interop 2.0.4 (legacy major) pinned

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-27 07:47:23 +08:00
pnoker 5a89b39ec1 refactor(build): move the relational core into the db family as dc3-db-core
dc3-common-jdbc was the family's core in disguise — MyBatis-Plus wiring,
the tenant-line handler, the databaseIdProvider, the jdbc profile. It now
lives as dc3-db/dc3-db-core, matching the dc3-mq-core / dc3-tsdb-core
shape: one aggregator, port first, adapters after, tck last. Artifact
renamed, packages deliberately untouched (same convention as the mq
family move); twelve poms rewired, the family aggregator puts core
first, and the db-tck now depends on the core explicitly instead of
riding transitives.

dc3-common-sql stays where it is on purpose: it is not platform dialect
plumbing but the base class of the four JDBC *device drivers*
(mysql/oracle/postgresql/sqlserver drivers that connect to external
databases being collected from) and depends on the driver SDK — under
dc3-db it would claim a meaning it does not have.

Gates: db-tck 24/24 (postgres + mysql + mariadb), data 274/274,
full-repo compile green.
2026-08-24 20:19:06 +08:00
pnoker 1bbc60ffa6 build(db): manage the dc3-db-mariadb artifact in the root pom 2026-08-24 20:04:09 +08:00
pnoker 9f91984c01 feat(db): add the mysql dialect — dc3-db-mysql, statement forks, portability rewrites
R2a of docs/design/storage-abstraction.md §3. MySQL 8 becomes a
first-class relational deployment behind dc3.db.type=mysql:

- dc3-db-mysql module: mysql-connector-j, MYSQL pagination DbType, the
  mysql profile (driver + jdbc-type-for-null), activated only when
  dc3.db.type=mysql — the postgres profile and DbType stay the default
  for unset values, so existing deployments are untouched. The neutral
  jdbc module now exposes a VendorDatabaseIdProvider (postgres/mysql
  aliases) so mapper XML can fork statements by engine.

- RETURNING decoupled where MySQL has no equivalent: the state upsert
  and the lease-state upsert no longer RETURN rows — callers re-select
  by the natural key in the same transaction (portable shape, one extra
  keyed read on both engines). The expired-lease claim becomes three
  portable steps in one TransactionTemplate: lock-and-read (FOR UPDATE
  SKIP LOCKED works identically on both engines), flip offline (INTERVAL
  expression is the only forked fragment), derive the post-update view
  in Java from the locked rows — the update is deterministic per column.

- Sequences retired in favor of row-local increments: the assignment
  version and the fencing token advance as col = col + 1 on every
  change. Per-driver / per-device monotonicity is the property callers
  and stale-writer checks rely on; the global sequences only ever
  delivered it incidentally, and MySQL has no sequence objects.

- Forked statements (databaseId twins): the point-latest fencing upsert
  (row-value tuple guard expanded into per-column IF chains — MySQL has
  neither row comparisons nor WHERE on ON DUPLICATE KEY UPDATE), the
  driver-instance upsert, both lease upserts, the driver lock (MySQL
  serializes on the driver row itself: SELECT ... FOR UPDATE, scoped to
  the transaction like the advisory lock it replaces), the registry
  lock (GET_LOCK + an explicitly paired advisoryUnlock — a no-op on
  postgres; the sync service wraps its critical section in try/finally),
  the tool-catalog candidate query (|| -> CONCAT, regexp_replace flag,
  and the (ext ->> 'content')::json double-hop re-expressed through
  JSON_UNQUOTE(JSON_EXTRACT(...)) — the content value is a JSON-encoded
  string, not a nested object, verified against live data), four alert
  statements' ->> extractions, hourly counts (DATE_FORMAT hour bucket),
  and the activity heatmap (DAYOFWEEK/HOUR units, dow 0 = Sunday).

- Portable rewrites that needed no fork: token_metadata::TEXT ->
  CONCAT(token_metadata, '') (both engines return text; only PG JDBC
  needs the nudge past PGobject), typeDistribution's GROUP BY now
  references the renamed bucket_key alias (the bare `key` was a MySQL
  reserved-word landmine).

Gates: data/auth/manager suites green (274 in data), e2e 26/26.
MySQL seed and the dual-dialect DAL TCK follow in R2b/R2c.
2026-08-24 17:06:31 +08:00
pnoker c6509762a6 refactor(db): split relational infra into a neutral jdbc module and a postgres dialect family
R1 of docs/design/storage-abstraction.md §3 — zero behavior change:

Portable SQL rewrites (verified against a live PostgreSQL before landing):
- COUNT(*) FILTER (WHERE) x3 -> COALESCE(SUM(CASE WHEN ... THEN 1 ELSE 0
  END), 0); the COALESCE keeps the empty-set result at 0 where bare SUM
  would return NULL.
- dailyTrend's generate_series + date_trunc calendar spine -> a
  RECURSIVE CTE day ladder (CAST(day + INTERVAL '1' DAY AS DATE) keeps
  the column type stable on both engines — the bare date+interval form
  widens to timestamp on PostgreSQL and fails the recursive term's type
  check, caught live); day rendering via CONCAT instead of ::text.
- OAuthMcpMapper: '{} '::JSON literals -> '{}' (unknown literals coerce
  to the JSON column on both engines), NULL::BIGINT -> NULL in the
  UNION arm, #{riskLevel}::text parameter casts dropped (the text column
  compares untyped parameters fine on both).
- The reserved-word alias AS key (a MySQL landmine hiding inside three
  otherwise-portable statements) renamed to bucket_key / bucketKey
  across both Dashboard mappers and BucketRow.

Module split: dc3-common-postgres -> dc3-common-jdbc (MyBatis-Plus
wiring, tenant-line handler, generator utils, the new dialect-neutral
jdbc profile carrying the shared mybatis-plus/hikari plumbing) + the
top-level dc3-db family's dc3-db-postgres (driver, timestamptz type
handler, postgres profile delta, and the pagination DbType bean the
neutral config now injects — defaulting to PostgreSQL so a bare
jdbc-only classpath behaves exactly as before). Packages deliberately
unchanged, mirroring the mq/tsdb family moves; all nine dependent poms
rewired; make deploy publishes the dc3-db family.

The jdbc module's interceptor integration test now pins an explicit
embedded H2 datasource — the module carries no driver on its main
classpath, so context inference would fail without it.

Gates: data/auth/manager/jdbc/db-postgres unit suites green (274 in
data), full-repo compile green, e2e 26/26.
2026-08-24 16:44:47 +08:00
pnoker d83ead1f64 feat(tsdb): complete the store lineup — influxdb 3 and iotdb adapters, published capability matrix
Phase 3 of docs/design/tsdb-abstraction.md: the store family reaches four
certified adapters, the selection guide publishes the real capability
matrix, and external stores become first-class compose services.

dc3-tsdb-influxdb (3.11.2-core) talks the documented v3 HTTP APIs
directly — line protocol writes, query_sql CSV reads, zero client
dependencies. Integer fields carry the i suffix from the first write (a
bare number binds the column to Float64 forever and destroys ns
timestamps); timestamps cross the wire as CAST(... AS BIGINT)
nanoseconds because JSON renders them in scientific notation; PERCENTILE
is approximate-only and deleteRange does not exist in Core — both
declared honestly (false), the analytics facade computes exact
percentiles from bounded pulls.

dc3-tsdb-iotdb (2.0.10-standalone) maps series onto tree paths
root.dc3.t<tenant>.d<device>.p<point> — path nodes cannot be purely
numeric, so the design's raw-id mapping is amended. The server must run
with timestamp_precision=us and dn_rpc_address=0.0.0.0 (a two-line
properties override shipped under dc3/dependencies/iotdb); WHERE time
literals must use the 2026-08-20T12:59:59+00:00 form because bare epoch
numbers are parsed as milliseconds regardless of the server precision —
a microsecond literal silently matches nothing. Sessions disable
redirection: node discovery hands out internal cluster addresses that
break behind port mappings. GROUP BY LEVEL columns come back
wildcard-shaped and are parsed per column; COUNT rides the INT32 quality
measurement (never null); null measurements are omitted per record — a
null value in insertRecords makes the whole record vanish.

Capability matrix published in docs/tsdb-stores.md per adapter-declared
values (skip counts in the TCK are declared-unsupported cases, degraded
by facades, never wrong data). External store services (tdengine,
influxdb, iotdb) join the optional compose stack with healthchecks and
DC3_TSDB_* passthrough in the app stack.

Gates: timescale 24/24, tdengine 24/24 (2 skips), influxdb 24/24 (2
skips), iotdb 24/24 (3 skips); full-repo compile green.
2026-08-24 16:06:44 +08:00
pnoker 20ae3fbf33 feat(tsdb): add the tdengine adapter — second store certified against the contract suite
dc3-tsdb-tdengine maps the port onto TDengine 3.x: one supertable
point_value tagged by (tenant, device, point), one deterministic subtable
pv_<t>_<d>_<p> auto-created on first insert via USING TAGS, database
created with PRECISION 'us' and KEEP 180. Connection settings live under
dc3.tsdb.tdengine.* (REST driver, own Hikari pool); selected by
dc3.tsdb.type=tdengine. Like the broker family, only the default adapter
(timescale) ships inside consumers — switching deployments add this
dependency plus DC3_TSDB_TYPE/DC3_TSDB_TDENGINE_URL.

The supertable mapping grilled four real issues out of the port
surface, each fixed and locked by the TCK:

- Timestamps never touch string form. The REST driver serializes
  Timestamp parameters in the client JVM zone while the server parses
  them as UTC — every write would shift by the deployment zone and
  cursor pagination would drift per page without end. Instants now
  travel as epoch-micro integer literals and reads return
  CAST(ts AS BIGINT) — verified symmetric against the image.
- AS value collides with a reserved word (agg_value now);
  INTERVAL takes a bare number read in the database (micro) precision.
- PERCENTILE only runs on single tables, so single-series percentiles
  query the deterministic subtable directly; tenant-wide percentile is
  refused rather than approximated.
- REST readiness must be probed with POST /rest/sql + basic auth; the
  GET path-style route 404s even on a healthy server.

Honest capabilities: latencyHistogram=false (dashboard degrades to
zero-filled bins via a capability check) and correlation=false (future
analytics facade computes from bucketed pulls); rollups stay NONE until
S16 lands stream computing. TCK result: 23 tests, 0 failures, 0 errors,
2 capability-gated skips; the timescale suite re-ran 23/23 unchanged.

The design doc's capability matrix now reflects the adapter's declared
values instead of the pre-implementation estimates.
2026-08-21 09:01:24 +08:00
pnoker daea5de87d feat(tsdb): add the time-series store family — port, timescale adapter, contract suite
First vertical slice of the tsdb abstraction (docs/design/
tsdb-abstraction.md phase 1): the dc3-tsdb top-level family lands with
the S19-final port and a fully certified TimescaleDB adapter.

dc3-tsdb-core — the port, zero store dependencies:
- TsdbModel: SeriesKey, the unified SeriesFilter (single series / series
  set / tenant-wide as one shape), PointValueSample with both timestamps
  (S9) and the quality flag (S17), AggregateFunction incl. FIRST/LAST
  (M4) and gated PERCENTILE, cursor records, analytics records
- TsdbStore SPI: append / last / cursor history / aggregate /
  bucketedAggregate / count, the S13 analytics facet
  (bucketedCount, countByDimension, lastSeenPerSeries,
  latencyHistogram), listSeries, deleteRange, correlation; every read
  carries TsdbDeadline

dc3-tsdb-timescale — the reference adapter:
- unnest single-statement batch append (one round trip) with natural
  upsert on (series, deviceTime); batches chunked at the declared
  maxAppendBatch
- read paths through SeriesFilter-shaped SQL: ROW_NUMBER per-series
  last-N, global (create_time, message_id) descending cursor history,
  time_bucket bucketed aggregates, percentile_cont, aligned-bucket
  corr() correlation, CASE-binned latency histogram
- idempotent schema bootstrap incl. initial-chunk priming (a sentinel
  row at a fixed early instant forces TimescaleDB's initial chunk
  creation at bootstrap with a controlled boundary)
- capability negotiation logged at startup; rollupSupport NONE in this
  extraction (S16 continuous aggregates arrive with phase 2)

dc3-tsdb-tck — the 22-case contract suite on Testcontainers:
append-readback fidelity (every field incl. both timestamps and
quality), newest-first last-N with exact limit, cursor pagination
without skip or duplicate, NULL-skipping aggregates, epoch-anchored
bucket boundaries, series and tenant-wide counts, duplicate-timestamp
last-write-wins, backfill acceptance, cross-tenant isolation,
microsecond precision, 5k-sample burst, the four analytics ops,
multi-series isolation, FIRST/LAST M4, percentile tolerance, quality
round-trip, deadline-bounded reads, and known-correlation detection.

Two debugging lessons worth recording (both fixed and TCK-locked):
- Spring's RowCallbackHandler fires once PER ROW; a while(rs.next())
  inside it silently skips every other row — the original cause of all
  'vanishing row' symptoms, initially misattributed to TimescaleDB
- TimescaleDB sizes the initial hypertable chunk around the first
  inserted row; priming at bootstrap avoids boundary anomalies

Timescale contract suite: 22/22 green against timescale-ha:pg18.
2026-08-20 20:01:06 +08:00
pnoker d003291f81 refactor(build): extract the mq family into a top-level dc3-mq aggregator
The broker-selection family (port + six certified adapters + tck) is a
deployment selection, not shared plumbing, so it moves out of dc3-common
into a top-level dc3-mq aggregator beside dc3-api — the same standing as
the gRPC contract layer, and a template for future pluggable families
(storage is next on the design board).

- dc3-common-mq -> dc3-mq/dc3-mq-core (port: API + core runtime)
- dc3-common-mq-{rabbitmq,kafka,rocketmq,pulsar,activemq,mqtt,tck} ->
  dc3-mq/dc3-mq-{...}; no compatibility aliases — every coordinate is
  renamed everywhere in one commit (business poms, coverage, root
  dependency management)
- dc3-common sheds eight modules back to plain shared plumbing; the
  device-access dc3-common-mqtt stays by design (protocol stack, not a
  broker selection)
- Java packages deliberately stay io.github.pnoker.common.mq.* — the
  facade modules set the precedent that packages do not track module
  homes, keeping this a pom-only, zero-source refactor
- make deploy now publishes dc3-api + dc3-common + dc3-mq (third-party
  driver authors compile against the port transitively)
- docs/mq-brokers.md and the design's module layout follow the new home

Verified: full-repo test-compile green; constant/data/driver/manager
unit suites unchanged and green (255/144/291); the rabbitmq contract
suite passes 12/12 from the new coordinates against a live broker,
proving adapter wiring survived the move.
2026-08-20 12:23:09 +08:00
pnoker cc655699fd feat(mq): add pulsar adapter, completing broker coverage
Phase 3 (4/4) and the final piece of the mq abstraction: every broker
named in the design is now a TCK-certified selection behind
dc3.mq.type. Pulsar suite 11 pass + 1 documented skip on the first run.

- topics map to persistent://public/default/dc3-<topic>, logical dead
  topics to dc3-<topic>-dlq; LOAD_BALANCE rides a durable shared
  subscription named after the consumer group (competing consumers,
  retained while offline), BROADCAST rides an exclusive per-instance
  subscription
- pulsar subscriptions start at the latest position natively, matching
  the rabbit fresh-queue / kafka latest semantics with no seeding
- acknowledge(List<MessageId>) commits deliveries, negativeAcknowledge
  requeues; reject(false) republishes to the -dlq topic and commits;
  poison and exhausted batches dead-letter through the shared
  synchronous bounded-retry semantics
- sends complete with the broker-issued message id (publisher
  confirmation); batchReceive is the native batch (capability true);
  arbitrary delays ride the port fallback like the other adapters
- tck harness against apachepulsar/pulsar standalone, or TCK_PULSAR_URL
  for an externally managed broker
2026-08-20 10:56:04 +08:00
pnoker fb703f5b64 feat(mq): certify the rocketmq adapter on the contract suite
Phase 3 (3/4): RocketMQ adapter (classic client) for
dc3.mq.type=rocketmq, TCK 11 pass + 1 documented skip against a live
nameserver/broker pair.

The 5.x classic client has two behaviors that had blocked certification,
both now handled in the adapter:

- brand-new consumer groups replay the topic backlog regardless of
  consumeFromWhere (LAST and TIMESTAMP both ignored), so fresh groups
  are seeded explicitly: a short-lived pull consumer sets every queue
  without a committed offset to the current max offset before the push
  consumer starts, matching the rabbit fresh-queue / kafka latest
  semantics; groups with a real offset trail are untouched
- subscribing to a topic that was never published to waits out the 30s
  route refresh, so subscribe-time seeding also warms up missing topics
  with a publish (the warm-up lands before the seeded offset, invisible
  to every fresh group)

Further fixes surfaced by the suite: dead-letter republishes only the
standard envelope headers (RocketMQ system properties like
CONSUME_START_TIME are rejected on re-publish), and requeues request the
1s delay level so bounded retry exhaustion dead-letters within the
contract window instead of the 10s default.

Topics map to dc3-<topic> (no dots), dead letters to dc3-<topic>-dlq;
LOAD_BALANCE rides CLUSTERING consumer groups, BROADCASTING fans out per
instance; sync sends are the publisher confirmation; arbitrary delays go
through the port fallback (delay levels would quantize). The tck harness
is opt-in via TCK_ROCKETMQ_NAMESRV (testcontainers cannot manage the
nameserver/broker pair); a fresh-group isolation probe rides along in
the adapter module, also opt-in.
2026-08-19 22:42:00 +08:00
pnoker 0250f9edea feat(mq): add mqtt 5 adapter passing the contract suite
Phase 3 (2/4): MQTT 5 adapter (hivemq client) for dc3.mq.type=mqtt,
working against any MQTT 5 broker with shared subscriptions (EMQX,
HiveMQ, NanoMQ, VerneMQ). TCK 10 pass + 2 documented skips.

- topics map to dc3/<topic> (slash style), logical dead topics to
  dc3/<topic>/dlq; LOAD_BALANCE rides $share/<group> shared
  subscriptions, BROADCAST rides a plain filter, and every subscription
  gets its own client session so broadcast instances are independent
- QoS 1 with manual acknowledgements: ack acknowledges the publish,
  PUBACK doubles as the publisher confirmation (capability true)
- MQTT has no server nack: reject(true) is approximated by bounded
  client-side redelivery, reject(false) republishes to the /dlq topic
  and acknowledges; poison and exhausted messages dead-letter
- delays go through the port fallback and batches are synthesized
  (capabilities false)
- the no-consumer durability case is disabled for this adapter with the
  design §13.8 reference: MQTT 5 leaves retention for an offline shared
  subscription to the broker (HiveMQ CE drops; EMQX-class brokers may
  retain) — the capability matrix documents the variance
- tck harness against hivemq-ce, or TCK_MQTT_HOST/TCK_MQTT_PORT for an
  externally managed broker
2026-08-19 21:37:55 +08:00
pnoker 65aa6d5956 feat(mq): add activemq adapter passing the contract suite
Phase 3 (1/4): ActiveMQ (Artemis / Classic, JMS 2.0) adapter for
dc3.mq.type=activemq, TCK 11 pass + 1 documented skip.

- live topics map to JMS topics: LOAD_BALANCE rides a shared durable
  subscription named after the consumer group (competing consumers share
  it, offline messages retained), BROADCAST rides a plain per-instance
  consumer — one publish fans out to every subscription like an exchange
- dead-letter destinations are queues; reject(false) republishes to
  dc3.<topic>.dlq and acknowledges, reject(true) recovers the session
- delays use JMS scheduled delivery natively (capability true); batches
  are synthesized by draining the consumer with the shared synchronous
  bounded-retry semantics; exhaustion dead-letters instead of dropping
- JMS property names must be java identifiers, so the dashed standard
  headers ride underscored and are restored on read
- best-effort publisher confirmation (capability false): the synchronous
  persistent send returning is the reported signal
- tck harness against apache/activemq-artemis, or TCK_ARTEMIS_URL for an
  externally managed broker; subscriptionExpiry documented false
2026-08-19 21:31:04 +08:00
pnoker b8aee2cf69 feat(mq): add kafka adapter and broker-neutral contract suite
Phase 2 of the mq abstraction design: the acceptance bar (TCK) plus the
kafka adapter, both passing the full contract suite against live brokers.

dc3-common-mq-tck (contract suite, reusable by community adapters):
- abstract 12-case suite covering round-trip envelope fidelity, load
  balance exactly-once across instances, broadcast fan-out, delay via the
  port fallback, reject-to-dead-letter, reject-with-requeue redelivery,
  sendAsync confirmation, 100-message burst, batch ack committing the
  whole batch, retry exhaustion dead-lettering instead of dropping,
  durability while no consumer runs, and per-instance subscription expiry
- rabbitmq and kafka harnesses on testcontainers 2.x, skipped gracefully
  without a container runtime; kafka harness also honors TCK_KAFKA_BOOTSTRAP
  for externally managed brokers (testcontainers 2.0.5 configures
  apache/kafka 3.9.0 with a nonroutable advertised listener on some runtimes)
- per-run unique consumer groups plus post-subscription settle keep runs
  isolated on log-based brokers with auto.offset.reset=latest

dc3-common-mq-kafka (adapter, dc3.mq.type=kafka):
- topics map to dc3.<topic>; logical dead topics map to dc3.<topic>.dlq so
  rejects and dead-letter subscriptions land on the same topic
- partition key becomes the record key (per-key ordering); LOAD_BALANCE
  rides consumer groups, BROADCAST uses per-instance group ids
- auto.offset.reset=latest mirrors the rabbit fresh-queue semantics: a new
  group only sees messages published after it joins
- batch delivery with synchronous bounded retry and backoff mirroring the
  rabbit stateless retry advice; exhaustion republishes the whole batch to
  the dead-letter topic and commits instead of dropping silently (spring's
  DefaultErrorHandler committed exhausted batches without publishing)
- non-poison failures on single deliveries nack for redelivery; poison
  messages are republished to the dead-letter topic and acknowledged

rabbit adapter adjustments surfaced by the suite: delayedMessage capability
is now false (only the intrinsic TTL+DLX topics delay server-side; arbitrary
delays go through the port fallback), POINT_VALUE_DEAD is subscribable for
dead-letter auditing, spec instanceTtl overrides the configured queue
expiry, subscriptions with a named group get a group-suffixed copy of the
platform-shared queue (blank group keeps the pre-port names), and shared
topology moved to a descriptor table

Verified: RabbitMQ 12/12 and Kafka 11+1-skipped against live brokers via
podman; constant/data/driver/manager unit suites unchanged and green.
2026-08-19 20:52:27 +08:00
pnoker 28efa5c8fc refactor(mq): migrate business modules onto the messaging port
Complete phase 1 of the mq abstraction design: business code compiles
against the broker-neutral port with zero amqp classes, and the legacy
dc3-common-rabbitmq module is deleted.

- dc3-common-driver: 8 sender methods plus the sqlite outbox now publish
  through MessageSender (outbox keeps durability; legacy physical routing
  keys in pending rows are normalized on republish); metadata/command/
  point-command receivers become @Dc3Listener subscriptions with
  placeholder-driven per-instance queues; DriverTopicConfig dissolved into
  adapter subscribe-time declarations
- dc3-common-data: command and point-command dispatch use sendConfirmed
  with the correlation-id header; state timeout, device scan tick, notify
  tasks and metadata events send through logical topics; all 13 receivers
  including the batch point-value consumer migrate to @Dc3Listener;
  DataTopicConfig, PointValueRabbitConfig and the duplicated batch
  properties class move into the port/adapter (same config keys)
- dc3-common-manager / facade-local-manager: metadata fan-out via the port
- dead-letter consumers read the correlation id from the standardized
  header the adapter mirrors onto the amqp property
- 14 test classes adapted to the new listener/sender shapes; module poms
  swap dc3-common-rabbitmq for dc3-common-mq + dc3-common-mq-rabbitmq

Verified: unit suites green for constant (291 incl. boundary guard),
data (165), driver (144), manager (255); full-repo main and test compile
green. Container-backed E2E (RabbitDeliveryIT) intentionally not run here.
2026-08-19 18:47:50 +08:00
pnoker f0c6e9bf25 feat(mq): add broker-neutral messaging port and rabbitmq adapter
Introduce the pluggable message broker layer from the mq abstraction
design. Existing rabbitmq deployments keep their topology byte-for-byte.

dc3-common-mq (port, zero broker dependencies):
- MqTopic logical destinations, JSON envelope with standardized headers
  (dc3-type, X-Request-Id via MDC/OpenTelemetry, correlation id)
- MessageSender send/sendAsync/sendConfirmed plus local-scheduler delay
  fallback for brokers without native delayed delivery
- @Dc3Listener annotation processor: resolves payload types from method
  signatures, resolves ${...} placeholders in keyPattern/group, restores
  the request id into the MDC around every invocation
- SubscriptionSpec (mode, profile, delivery, keyPattern, group) and
  BrokerCapabilities negotiation logged at startup

dc3-common-mq-rabbitmq (adapter, dc3.mq.type=rabbitmq, default):
- RabbitNames replicates the pre-port physical names exactly, including
  the historical tag inconsistency between routing-key families
- RabbitTopology declares exchanges/queues/bindings programmatically with
  identical TTL, dead-letter and binding-argument details; driver-side
  per-instance queues are declared on subscribe with lease-coupled x-expires
- one listener container per subscription: latency 2/8/10, throughput
  4/32/100, batch consumer replicated from the point-value factory
  (prefetch >= batchSize, batch-level bounded retry, exhaustion dead-letters)
- stamps legacy __TypeId__ for rolling upgrades and mirrors the correlation
  id onto the AMQP property for dead-letter consumers

Shared enums (MqTopic, SubscriptionMode, ConsumptionProfile, DeliveryMode,
OrderingGuarantee) live in dc3-common-constant per the module boundary
guard. The legacy dc3-common-rabbitmq module is retired in the next
commit; both coexist so this commit stays independently buildable.
2026-08-19 18:47:24 +08:00
pnoker dddf2e8690 test: enforce truthful coverage and behavioral quality 2026-08-18 23:24:58 +08:00
pnoker 8f6520dd79 chore: refresh dependencies and project documentation 2026-08-18 08:14:26 +08:00
Henry Zhang 73ff48bcb9 style: reformat codebase (backend java/xml, docs, web) (#175)
* style(backend): reformat and rearrange java/xml sources

Apply code cleanup across dc3-common and dc3-driver: whitespace and
indent normalization, import sorting, member rearrangement, and javadoc
normalization. No semantic changes; compilation verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style(docs): reformat vitepress vue diagram components

Expand compact svg templates and normalize formatting across diagram
components, the seo module, and the theme stylesheet. No behavior
changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style(docs): reformat markdown pages

Normalize markdown formatting. Restore the <script setup> component
imports that an earlier unused-import pass had wrongly stripped; every
<Diagram> tag now resolves to a backing import again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style: reformat point view and trim pom whitespace

Normalize whitespace in the point view and drop trailing whitespace in
pom.xml. No behavior changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 12:13:36 +08:00
Henry Zhang 5e0003f019 build(deps): bump mybatis-plus 3.5.16 → 3.5.17 + adapt IService/ServiceImpl imports (#154)
mybatis-plus 3.5.17 fixes the JDK 9+ split-package conflict (#7027) by
relocating service classes from com.baomidou.mybatisplus.extension.service
to com.baomidou.mybatisplus.spring.service. Verified via the 3.5.17 jar:
only IService and ServiceImpl moved — Page, JacksonTypeHandler, chain
wrappers, Db, interceptors all stay in extension. Updated 112 imports.
Supersedes dependabot #141 (which failed compile on the same move).

Verified: mvn compile clean across every IService module (bacnet4j driver
excluded only due to a local mirror cache miss, unrelated to this change).
2026-08-03 13:31:22 +08:00
dependabot[bot] 231a44e3cb build(deps): bump com.squareup.okhttp3:okhttp-bom from 5.3.2 to 5.4.0 (#142)
Bumps [com.squareup.okhttp3:okhttp-bom](https://github.com/square/okhttp) from 5.3.2 to 5.4.0.
- [Changelog](https://github.com/lysine-dev/okhttp/blob/master/CHANGELOG.md)
- [Commits](https://github.com/square/okhttp/compare/parent-5.3.2...parent-5.4.0)

---
updated-dependencies:
- dependency-name: com.squareup.okhttp3:okhttp-bom
  dependency-version: 5.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Henry Zhang <pnokers@icloud.com>
2026-08-03 12:22:39 +08:00
dependabot[bot] e18dbef8d9 build(deps): bump tools.jackson.core:jackson-databind (#136)
Bumps [tools.jackson.core:jackson-databind](https://github.com/FasterXML/jackson) from 3.1.4 to 3.1.5.
- [Commits](https://github.com/FasterXML/jackson/commits)

---
updated-dependencies:
- dependency-name: tools.jackson.core:jackson-databind
  dependency-version: 3.1.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Henry Zhang <pnokers@icloud.com>
2026-08-03 12:05:53 +08:00
pnoker f019e2066b feat(driver): introduce sqlite-jdbc dependency and buffer config properties
Add sqlite-jdbc 3.53.2.0 to parent pom dependencyManagement; add sqlite-jdbc and HikariCP to dc3-common-driver pom; add BufferProperties nested group to DriverProperties; add BUFFER_REPUBLISH_SCHEDULE_JOB constant to ScheduleConstant
2026-07-26 00:13:29 +08:00
pnoker db14f0f089 build(deps): add opentelemetry bom and tracing dependencies
Replace micrometer-tracing-bridge-brave with the full OpenTelemetry stack
(bridge-otel, OTLP exporter, SDK, trace SDK, propagators, API, context).
Add OTel BOM and semconv to the parent POM for centralized version control.
Add OTel API dependencies to dc3-common-rabbitmq and dc3-common-facade-grpc
for trace context propagation.
2026-07-23 23:19:48 +08:00
dependabot[bot] f5d893dcb6 build(deps): bump swagger-core.version from 2.2.47 to 2.2.52 (#108)
Bumps `swagger-core.version` from 2.2.47 to 2.2.52.

Updates `io.swagger.core.v3:swagger-core-jakarta` from 2.2.47 to 2.2.52

Updates `io.swagger.core.v3:swagger-annotations-jakarta` from 2.2.47 to 2.2.52

Updates `io.swagger.core.v3:swagger-models-jakarta` from 2.2.47 to 2.2.52

Updates `io.swagger.core.v3:swagger-annotations` from 2.2.47 to 2.2.52

Updates `io.swagger.core.v3:swagger-models` from 2.2.47 to 2.2.52

---
updated-dependencies:
- dependency-name: io.swagger.core.v3:swagger-core-jakarta
  dependency-version: 2.2.52
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: io.swagger.core.v3:swagger-annotations-jakarta
  dependency-version: 2.2.52
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: io.swagger.core.v3:swagger-models-jakarta
  dependency-version: 2.2.52
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: io.swagger.core.v3:swagger-annotations
  dependency-version: 2.2.52
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: io.swagger.core.v3:swagger-models
  dependency-version: 2.2.52
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-19 20:30:13 +08:00
dependabot[bot] 5f6ac1732d build(deps): bump jakarta.xml.bind-api from 4.0.2 to 4.0.5
Bumps [jakarta.xml.bind:jakarta.xml.bind-api](https://github.com/jakartaee/jaxb-api) from 4.0.2 to 4.0.5.
- [Release notes](https://github.com/jakartaee/jaxb-api/releases)
- [Commits](https://github.com/jakartaee/jaxb-api/compare/4.0.2...4.0.5)

---
updated-dependencies:
- dependency-name: jakarta.xml.bind:jakarta.xml.bind-api
  dependency-version: 4.0.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-28 23:08:13 +08:00
dependabot[bot] 70e193c596 build(deps): bump oshi-core from 7.1.0 to 7.3.2
Bumps [com.github.oshi:oshi-core](https://github.com/oshi/oshi) from 7.1.0 to 7.3.2.
- [Release notes](https://github.com/oshi/oshi/releases)
- [Changelog](https://github.com/oshi/oshi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/oshi/oshi/compare/oshi-parent-7.1.0...oshi-parent-7.3.2)

---
updated-dependencies:
- dependency-name: com.github.oshi:oshi-core
  dependency-version: 7.3.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-28 22:35:59 +08:00
pnoker a620afb087 build(deps): upgrade spring-ai to 2.0.0 ga and drop milestone repository 2026-06-23 19:34:21 +08:00
pnoker 46f8243c07 test(driver): add postgresql testcontainers integration test for jdbc io
Add the first driver integration test: spins a real PostgreSQL container and
verifies the shared JDBC read path returns stored values and that the write
path binds the point value as a parameter — a SQL-injection payload is stored
verbatim and cannot drop the table, confirming the parameterised-write fix
end-to-end. Set the Failsafe classesDirectory so integration tests run against
the plain compiled classes instead of the Spring Boot repackaged fat jar.
2026-06-22 18:49:49 +08:00
pnoker c2c9a690a4 fix(common): correctly convert PEM certificates and keys in X509Util
Replace the unsafe (T) cast of PemReader.readPemObject() — which threw
ClassCastException at runtime — with explicit conversion: X.509 certs via
CertificateFactory and private keys (PKCS#1/PKCS#8, encrypted or plain) via
BouncyCastle's PEMParser. Add the bcpkix dependency and unit tests covering
the encrypted-key path used by MQTT TLS.
2026-06-22 16:09:50 +08:00
pnoker 51a3847205 feat(openapi): wire up springdoc OpenAPI docs and fix security chain loading
Make the REST API documentation actually reachable end-to-end:

- Upgrade springdoc to 3.0.3 (Spring Boot 4 / Spring Framework 7 compatible)
  and pin swagger-core 2.2.47 to resolve the Spring AI conflict.
- Register SpringDocConfig and WebFluxSecurityConfig as @AutoConfiguration so
  they load (they sit in io.github.pnoker.common.config, which centers do not
  component-scan); add the global OpenAPI info + X-Auth security schemes.
- Add per-module GroupedOpenApi beans (auth/manager/data/agentic) and gateway
  aggregation routes + swagger-ui.urls for a single federated Swagger UI.
- Fix the security chain: guard WebFluxSecurityConfig off the gateway, supply a
  permissive default PermissionProvider, register the @perm bean, and correct
  the token-path whitelist (base-path is stripped before the chain).
- Whitelist doc paths; disable docs in the production profile.
- Register swagger-ui webjar resources (default static mappings are disabled).
2026-06-09 15:49:54 +08:00
pnoker a07d3f29ea feat(doc): add springdoc-openapi infrastructure
- Add springdoc-openapi 2.8.6 to parent POM dependency management
- Add springdoc-openapi-starter-webflux-ui to dc3-common-web
- Add springdoc-openapi-starter-common to facade, dal, repository modules
- Add SpringDocConfig with IoT DC3 API metadata (title, contact, license)
2026-06-08 17:00:00 +08:00
pnoker f58e8dd52f chore(deps): upgrade Spring AI to 2.0.0-M8 and migrate tool API
- Bump spring-ai.version from 2.0.0-M6 to 2.0.0-M8
- Migrate ChatClientConfig: suppressToolCallStreaming() → streamToolCallResponses(false)
- Migrate AgenticPromptBuilder: toolContext() → tools(toolSpec → toolSpec.context())
  and toolCallbacks() → tools(toolSpec → toolSpec.callbacks())
2026-06-08 16:02:00 +08:00
pnoker 6884022dc9 chore(pom): bump dependency versions
- surefire / failsafe / jacoco: 3.5.2 → 3.5.6 / 0.8.12 → 0.8.15
- graalvm native: 1.1.0 → 1.1.1
- jaxb: 4.0.8 → 4.0.9
- jna: 5.18.1 → 5.19.0
- tools.jackson: 3.1.3 → 3.1.4
- maven-dependency-plugin: 3.10.0 → 3.11.0
2026-06-07 01:11:18 +08:00
pnoker 5730f7c137 feat: update pom version 2026-06-06 13:00:39 +08:00
pnoker b5ef7a8501 fix(pom): clean up dependency management and plugin configuration
- Replace obsolete nexus-staging-maven-plugin with central-publishing-
  maven-plugin in dc3-api and dc3-common
- Remove 20 redundant version declarations from dc3-coverage dependencies
- Fix snmp4j version from non-existent 3.8.3 to available 3.7.4
2026-06-06 00:40:18 +08:00
pnoker 26f6e91a64 feat(driver): add 17 protocol drivers to extend IoT connectivity coverage
## Common Foundation
- dc3-common-sql: AbstractJdbcDriverCustomService base class with HikariCP
  pool management and JDBC template methods

## Serial & General-Purpose (3)
- Serial: RS232/RS485/RS422 with configurable frame parsing (CRC16/XOR)
- HTTP: REST Client using Spring WebClient with JSON path extraction
- TCP/UDP Raw: generic socket driver with pluggable frame protocol

## Database Drivers (4)
- MySQL, PostgreSQL, Oracle (SID/ServiceName), SQL Server

## IoT & Wireless Protocols (3)
- LwM2M: Eclipse Leshan 2.0.0-M14 with CoAP server and device observation
- Zigbee: Z-Smart Systems 1.4.16.1 with Telegesis dongle support
- BLE: Sputnikdev Bluetooth Manager 1.5.3 with TinyB transport

## Building Automation & Smart Metering (2)
- BACnet IP: BACnet4J 6.0.1 with ReadProperty/WriteProperty support
- DLMS/COSEM: Gurux DLMS 4.0.79 for smart metering (TCP/Serial transport)

## Industrial & Power Protocols (4)
- IEC 104: OpenMUC j60870 1.7.2 for power telecontrol
- SNMP: SNMP4J 3.7.4 for network device monitoring
- EtherNet/IP: CIP Data Table Read/Write for Rockwell AB PLCs
- FINS: Omron PLC memory area read/write over TCP

## Automotive & Fieldbus (1)
- CAN Bus: SocketCAN interface with standard/extended frame support

## Build
- Register all 17 driver modules and dc3-common-sql in parent POMs
- Add version properties and dependencyManagement entries
2026-06-02 09:54:13 +08:00
Vickey 29f1594072 feat(driver): add Mitsubishi Melsec MC and SL651 hydrological telemetry drivers
feat: new driver modules
- dc3-driver-melsec: Mitsubishi Melsec MC (1E/3E/4E frame) driver via iot-communication
- dc3-driver-sl651: SL651-2014 hydrological telemetry server-side driver via iot-communication
- Bump iot-communication from 1.5.4 to 1.5.6 in dc3-driver-plcs7
- Relocate ias-releases repository from root pom.xml to dc3-driver/pom.xml

fix: code review fixes across driver implementations
- PlcS7PointVariable: only include bit offset in address when bit access and offset > 0
- OpcDaDriverCustomServiceImpl: disconnect server on device delete/update
- OpcUaDriverCustomServiceImpl: disconnect client on device delete/update
- ModbusTcpDriverCustomServiceImpl: remove duplicate FLOAT case in write value switch

refactor: eliminate magic numbers, consolidate statics, and normalize log formatting
- NettyServerHandler: replace magic numbers with named constants
- OpcUaDriverCustomServiceImpl: replace magic timeout numbers with named constants
- ModbusFactory: consolidate static block into private static final field
- VirtualDriverCustomServiceImpl: replace new Random() with ThreadLocalRandom
- KeyLoader: reorder fields so statics precede instance fields
- Normalize log format from string concat to {} placeholders across multiple drivers
- MqttDriverCustomServiceImpl: downgrade QoS fallback log warn→debug
- Remove unused imports
2026-05-26 10:00:00 +08:00
Vickey 27bcf1383b fix: address code review — security, thread safety, performance, and null safety
Security (CRITICAL):
- KeyUtil: read JWT signing key from DC3_SECURITY_KEY env/property
- UserPasswordServiceImpl: read default password from DC3_SECURITY_DEFAULT_PASSWORD env
- KeyLoader: read OPC-UA keystore password from OPCUA_KEYSTORE_PASSWORD env
- WebFilterConfig: return 401 on malformed X-Auth-User header
- AlgorithmConstant: document deprecated hardcoded constants

Thread safety (HIGH):
- Fix TOCTOU races in 4 driver connectors (computeIfAbsent)
- Fix PlcS7 lock leak, OpcUa connect timeout (5s)
- CoapClientManager: synchronize setURI to prevent URI race
- PointValueJob, MqttScheduleJob: add @DisallowConcurrentExecution
- SystemHealthServiceImpl: preserve interrupt flag
- WindowSampleBuffer: local AtomicInteger → int

Performance (MEDIUM):
- EntityStateExpiryScanner: batch alarm saves (saveBatch)
- ImportDeviceServiceImpl: batch config saves
- ResourceRegistrySyncServiceImpl: batch-load nodes, eliminate N+1 COUNTs
- DriverSenderServiceImpl: debug-gate hot-path logging
- PointServiceImpl: stream().count() → size()

Null safety (LOW):
- RegexUtil: null guards on isName/isPhone/isMail/isPassword/isHost
- HostUtil: null-check getNetworkInterfaces() return
- TimeUtil: log parse failures instead of silent null

Infrastructure:
- Add spring-boot-starter-cache + @EnableCaching CacheConfig
- Replace embedded modbus4j/plc-s7 jars with external Maven dependencies
- Add dc3-driver-modbus-rtu module
2026-05-26 08:40:25 +08:00
Vickey 918b3d5e8e feat: update pom version to 2026.5.22 2026-05-26 00:30:11 +08:00
Vickey fd3af64634 feat: add layered architecture diagram and reposition project narrative
- Add IoT DC3 layered architecture diagram (SVG + HTML viewer)
  with Application / Management / Data / Driver layers,
  Security & Load and Platform & System sidebars,
  Data→Driver connection flows, and color-coded sections
- Reposition README descriptions across all languages (en, zh, ja, vi)
  and pom.xml to AI-ready narrative: "Connects devices, collects
  data organized for AI, and orchestrates the closed loop"
- Add README.ai.md with Chinese multi-paragraph AI narrative
  and closed-loop flowchart
- Optimize package.json metadata (name, license, scripts)
2026-05-24 13:35:57 +08:00
pnoker 9007cbc22c style: apply consistent code formatting across all modules and docs 2026-05-22 00:24:22 +08:00
pnoker 5a3ab1440a test: improve backend test coverage governance 2026-05-19 18:19:12 +08:00
pnoker 7f5da1a0f8 chore: bump version to 2026.5.18 across all modules 2026-05-18 00:24:22 +08:00
pnoker 737caa178c build: update root pom and banner version to 2026.5.17 2026-05-17 19:04:31 +08:00
pnoker 019daf62ce feat: do someting test and code review 2026-05-16 00:56:04 +08:00
pnoker 67f1f7c2c0 build(agentic): upgrade Spring AI from 2.0.0-M5 to 2.0.0-M6
Bump spring-ai-bom in root and agentic module POMs. API is fully
compatible — no code changes required. Full project build and 89
agentic module tests pass.
2026-05-15 01:53:21 +08:00
pnoker 9123b480f4 test(e2e): stabilize infrastructure harness 2026-05-14 21:29:08 +08:00
pnoker ade8579369 fix(test-infra): propagate test bom into dc3-common dependency management
Maven dependencyManagement does not flow across the iot-dc3 root reactor
into dc3-common because dc3-common inherits from the externally
published dc3-parent. As a result, dc3-common-test could not resolve
JUnit 5 / Testcontainers / Instancio dependencies and dc3-e2e could not
resolve dc3-common-test itself.

Mirror the relevant slice of test-toolchain version management into
dc3-common/pom.xml (junit-bom + testcontainers-bom imports plus pinned
versions for AssertJ, Awaitility, Instancio, REST-assured and WireMock)
and add the missing dc3-common-test version entry to the iot-dc3 root
dependencyManagement so the e2e module can import it.

Verified locally with mvn -B clean compile -pl dc3-common/dc3-common-test -am,
which now resolves all 9 source files and produces
dc3-common-test-2026.5.5.jar.
2026-05-13 13:17:55 +08:00
Vickey c47e5f0a37 test(infra): add coverage and end-to-end aggregator modules
Introduce two reactor modules that exist purely to host repository-wide
test concerns:

- dc3-coverage (packaging=pom) runs jacoco:report-aggregate against all
  contributing modules and applies a tree-wide jacoco:check rule. The
  aggregator excludes generated sources, deployable Application classes,
  POJO entity packages and the third-party modbus4j sero utilities so
  the percentage stays anchored on real business logic. Coverage gates
  start at 0% and tighten in later stages as more tests land.
- dc3-e2e (packaging=jar) reserves a place for the docker-compose driven
  end-to-end suite delivered in stage S8. Unit and integration phases
  are skipped by default and only the e2e profile turns them on.

Both modules are wired into the root reactor.
2026-05-13 13:02:26 +08:00