- add ble, can, dlt645, dnp3, iec61850, kafka, knx, lorawan, mbus, redis, and zigbee to the k8s driver list, kustomization, helm values, and compose service stacks
- regenerate k8s driver deployments and preserve the listening-virtual ports in the generator
- replace the per-driver named volumes with one shared driver_data volume isolated by mount path, matching the scale/swarm stacks
- correct scaling docs: compose scale/swarm drivers stay at one replica because replicas would share one SQLite outbox file
R2b of docs/design/storage-abstraction.md §3:
- PostgreSQL seed: the three driver-lease sequences retire. The fencing
token and assignment version default to 1 and advance row-locally —
per-device / per-driver monotonicity is the property the checks rely
on, which the global sequences only delivered incidentally, and MySQL
has no sequence objects. The three statement-level revision triggers
(transition tables + INSERT .. ON CONFLICT, none of which MySQL can
express) collapse into one row-level function with row-local +1 upserts
— a shape both engines share.
- MySQL seed under dc3/dependencies/mysql/initdb/ (00-06), derived by
dc3/dependencies/mysql/pg2mysql_seed.py: schemas become databases,
TIMESTAMPTZ becomes DATETIME(6), TEXT/JSON literal defaults become
expression defaults (MySQL 8 refuses literal defaults on BLOB/TEXT/
JSON), operate-time plpgsql triggers become the ON UPDATE column
attribute (explicit SETs keep winning), the revision family becomes
row-level DELIMITER triggers, keyed TEXT columns widen to VARCHAR(191),
partial indexes drop, and the embedded-JSON seed values load under
NO_BACKSLASH_ESCAPES (MySQL string literals eat the backslashes the
escaped-JSON content values depend on; PostgreSQL does not). 05 is
hand-maintained: only the dc3_point_latest projection — a MySQL core
requires an external time-series store, so the hypertable and cagg DDL
have no counterpart by design.
- Compose: the optional stack gains a mysql:8.4 service (utf8mb4, seed
mounted at docker-entrypoint-initdb.d, healthcheck); the app stack
passes DC3_DB_TYPE through.
Verified by loading all seven files into a fresh mysql:8.4 container —
zero errors, five databases, table counts aligned with the PostgreSQL
seed (manager 24 incl. common, auth 24, data 12, agentic 6, history 1),
and the revision trigger proven live: insert bumps the owning driver's
revision to 1, an enable_flag flip bumps it again to 2.
Phase 3 of docs/design/tsdb-abstraction.md: the store family reaches four
certified adapters, the selection guide publishes the real capability
matrix, and external stores become first-class compose services.
dc3-tsdb-influxdb (3.11.2-core) talks the documented v3 HTTP APIs
directly — line protocol writes, query_sql CSV reads, zero client
dependencies. Integer fields carry the i suffix from the first write (a
bare number binds the column to Float64 forever and destroys ns
timestamps); timestamps cross the wire as CAST(... AS BIGINT)
nanoseconds because JSON renders them in scientific notation; PERCENTILE
is approximate-only and deleteRange does not exist in Core — both
declared honestly (false), the analytics facade computes exact
percentiles from bounded pulls.
dc3-tsdb-iotdb (2.0.10-standalone) maps series onto tree paths
root.dc3.t<tenant>.d<device>.p<point> — path nodes cannot be purely
numeric, so the design's raw-id mapping is amended. The server must run
with timestamp_precision=us and dn_rpc_address=0.0.0.0 (a two-line
properties override shipped under dc3/dependencies/iotdb); WHERE time
literals must use the 2026-08-20T12:59:59+00:00 form because bare epoch
numbers are parsed as milliseconds regardless of the server precision —
a microsecond literal silently matches nothing. Sessions disable
redirection: node discovery hands out internal cluster addresses that
break behind port mappings. GROUP BY LEVEL columns come back
wildcard-shaped and are parsed per column; COUNT rides the INT32 quality
measurement (never null); null measurements are omitted per record — a
null value in insertRecords makes the whole record vanish.
Capability matrix published in docs/tsdb-stores.md per adapter-declared
values (skip counts in the TCK are declared-unsupported cases, degraded
by facades, never wrong data). External store services (tdengine,
influxdb, iotdb) join the optional compose stack with healthchecks and
DC3_TSDB_* passthrough in the app stack.
Gates: timescale 24/24, tdengine 24/24 (2 skips), influxdb 24/24 (2
skips), iotdb 24/24 (3 skips); full-repo compile green.
dc3-tsdb-tdengine maps the port onto TDengine 3.x: one supertable
point_value tagged by (tenant, device, point), one deterministic subtable
pv_<t>_<d>_<p> auto-created on first insert via USING TAGS, database
created with PRECISION 'us' and KEEP 180. Connection settings live under
dc3.tsdb.tdengine.* (REST driver, own Hikari pool); selected by
dc3.tsdb.type=tdengine. Like the broker family, only the default adapter
(timescale) ships inside consumers — switching deployments add this
dependency plus DC3_TSDB_TYPE/DC3_TSDB_TDENGINE_URL.
The supertable mapping grilled four real issues out of the port
surface, each fixed and locked by the TCK:
- Timestamps never touch string form. The REST driver serializes
Timestamp parameters in the client JVM zone while the server parses
them as UTC — every write would shift by the deployment zone and
cursor pagination would drift per page without end. Instants now
travel as epoch-micro integer literals and reads return
CAST(ts AS BIGINT) — verified symmetric against the image.
- AS value collides with a reserved word (agg_value now);
INTERVAL takes a bare number read in the database (micro) precision.
- PERCENTILE only runs on single tables, so single-series percentiles
query the deterministic subtable directly; tenant-wide percentile is
refused rather than approximated.
- REST readiness must be probed with POST /rest/sql + basic auth; the
GET path-style route 404s even on a healthy server.
Honest capabilities: latencyHistogram=false (dashboard degrades to
zero-filled bins via a capability check) and correlation=false (future
analytics facade computes from bucketed pulls); rollups stay NONE until
S16 lands stream computing. TCK result: 23 tests, 0 failures, 0 errors,
2 capability-gated skips; the timescale suite re-ran 23/23 unchanged.
The design doc's capability matrix now reflects the adapter's declared
values instead of the pre-implementation estimates.
Phase 1b of docs/design/tsdb-abstraction.md: every point-value read and
write now flows through the dc3-tsdb port, and dc3-common-repository is
deleted outright (no compat aliases).
Write path — new PointValueIngestService orchestration in dc3-common-data:
the stale-owner lease guard leaves the history INSERT's cross-schema join
and resolves the active owner per distinct device through the existing
DeviceFacade.getActiveOwner chain; a caffeine-backed ingest idempotency
window (dc3.data.ingest.*, marked only after both writes commit) absorbs
MQ redeliveries now that the message_id unique index is retired; the
batch lands via TsdbStore.append (natural series+time upsert, INGEST_ORDER
kept for the fenced dc3_point_latest upsert).
Read path — history via last(), page() resolves name/enable filters to
series keys through relational metadata (tenant-wide when unrestricted,
cursor offset emulation capped at 10k) with count() for totals; the alarm
window backend becomes TsdbWindowDataSource over aggregate()/history().
Dashboards — all nine data-side hypertable statements move to the S13
analytics facet (count/bucketedCount/countByDimension/latencyHistogram/
lastSeenPerSeries) plus the dc3_point_latest projection for latestStream;
the manager topology statement crosses into the data center via a new
ListSeriesVolumes RPC backed by the new S13-5 seriesCounts primitive (a
point shared by several devices needs per-series counts, which
single-dimension grouping cannot reconstruct); both cross-schema joins
are gone.
Port/adapter — seriesCounts added with a TCK case (23 total); series
filters compile to row-value IN lists chunked at 500 pairs; bootstrap
adds the quality column idempotently and retires uk_point_value_event so
existing deployments converge on the new upsert; the adapter binds the
application-provided tsdbDataSource bean (the history dynamic-datasource
entry) instead of the routing primary.
Timestamps — BO<->Instant conversion is pinned to
TimeConstant.DEFAULT_ZONEID (the platform canonical zone the latest
projection type handler already writes with); the design note's "lock
UTC" is amended accordingly, together with the lease-guard and S13-5
deviations recorded in §6.2.
Seed/deploy — 05-iot-dc3-history.sql gains the quality column and swaps
the unique index; compose carries DC3_TSDB_TYPE; make deploy publishes
dc3-tsdb.
Also fixed along the way: root pom never managed dc3-mq-core (latent,
masked by full-reactor builds); PointValueMapper ran unrouted on the
master datasource (now @DS("history")); latestStream name enrichment
looked up a Long-keyed map with String keys.
Gates: tsdb TCK 23/23, dc3-common-data 262/262, e2e 26/26 including the
unmodified PostgresHypertableIT; full reactor green except dc3-mq-tck's
kafka/pulsar container-startup flake, reproduced identically twice and
unrelated to this change.
Close out the mq abstraction rollout:
- docs/mq-brokers.md: user-facing broker selection guide — how to pick
(DC3_MQ_TYPE + connection settings), the six certified brokers, the
capability matrix, and the at-least-once/outbox notes
- .env.example + dc3/env/dev.env: DC3_MQ_TYPE and the per-adapter
connection variables with safe rabbitmq defaults
- docker-compose.yml: DC3_MQ_TYPE and adapter endpoints flow through the
shared x-app-runtime-env anchor so every service sees the selection
- adapter configurations resolve connection settings from the canonical
dc3.mq.* property first, then the DC3_MQ_* environment variable, then
the transport-standard property (spring.kafka.bootstrap-servers),
making containerized selection deterministic
The gated E2E gate passes unmodified through the port: RabbitDeliveryIT
6/6, CommandCallE2eIT 4/4, EventReportE2eIT 3/3 (full suite 24/24
including postgres and harness ITs) against testcontainers brokers.
Use PostgreSQL leases and fencing for distributed ownership, require a durable SQLite outbox before RabbitMQ publication, and make Data Center ingestion transactional and idempotent.
BREAKING CHANGE: drivers require mandatory durable outbox configuration and use lease-aware ownership and telemetry contracts.
28 driver application.yml add dc3.driver.buffer; Dockerfile each stage adds dc3/data dir and VOLUME; docker-compose main/dev add driver_data named volume
Merge the standalone iot-dc3-web repository into iot-dc3 as dc3-web/,
creating a single monorepo with unified versioning, CI/CD, and release
process.
- subtree-add iot-dc3-web under dc3-web/ with full commit history
- relocate nginx configs and SSL certs to dc3/dependencies/
- merge CI workflows (ci-backend.yml, ci-web.yml, docker-ci-web.yml)
- merge Issue/PR templates and dependabot config
- host frontend docs under docs/zh/frontend/
- share .husky/ hooks via root-level package.json (pnpm)
- unify versioning: single dc3/bin/tag.sh (vYYYY.M.P) for the whole repo
- update Dockerfile, docker-compose.yml, Makefile for new paths
- remove redundant directories from dc3-web/ (.claude, .github, dc3, docs, bin)
Co-Authored-By: Claude <noreply@anthropic.com>
Set AUTH_HMAC_SECRET default to "io.github.pnoker.dc3" across .env
templates, compose files, and Java config properties so that X-Auth-User
header signing is active out of the box without manual configuration.
Parametrize JDK/JRE base images via build args in Dockerfile, replace
hardcoded GC/dump env vars with per-service path variables, fix APM
port. Simplify Makefile registry options to global|cn, add SERVICES
and GROUP selectors for targeted compose operations.
Add dc3-web frontend service to docker-compose.yml with nginx reverse
proxy as the sole user-facing entry point. Remove backend port mappings
since web now proxies to gateway.
Merge docker-compose-elasticsearch.yml and docker-compose-grafana.yml
into a single docker-compose-optional.yml that includes EMQX, ELK+APM,
and Prometheus+Grafana stacks.
Expose AGENTIC_MEMORY_ENABLED, AGENTIC_TOOL_CALLING_ENABLED,
AGENTIC_MEMORY_MAX_MESSAGES, and AGENTIC_ATTACHMENT_STORAGE_PATH
in development and production compose files, with defaults in dev.env.
Also mark OpenAI base-url config as deprecated in application-agentic.yml.
The HMAC signing feature was dormant because no deployment config
set the shared secret. Add AUTH_HMAC_SECRET to .env.example, dev
env files, both compose stacks (all 12 services), and document it
in ENVIRONMENT.md.
Standardized the use of double quotes in docker-compose files for consistency. Added new docker-compose files for optional and Aliyun configurations to support different deployment environments.
Updated the version from 2025.2.4 to 2025.2.5 in pom.xml files and various Java classes to reflect the latest release. This change ensures consistency across the codebase and aligns with the new version of the IoT DC3 Platform.
Updated the version from 2025.2.2 to 2025.2.4 in all pom.xml files, Java classes, and CI/CD configurations to reflect the latest release version. This change ensures consistency across the entire project.
pnoker