337 Commits
Author SHA1 Message Date
pnoker 372d75b0c9 refactor(core): rename find/query/fetch methods to get per crud law
sweep every module to satisfy the new method-name rule, renaming
about 40 methods together with their interface, implementation, test
and cli call sites: store find() lookups become get(), analytics
queryLatest/queryHistory become getLatest/getHistory, sql helper
query() becomes spec()/sql(), jwks fetch() becomes load(), and the
cursor record component queryDigest becomes requestDigest

no behavior changes; full unit suite (5278 tests) and both quality
gates stay green
2026-09-18 09:53:51 +08:00
pnoker e4523f993b fix(api): accept snake_case params on history lookup endpoints
- command_history/get_by_record_id now binds the declared wire name
  record_id instead of the raw parameter name recordId, matching the
  event_history twin and the frontend/CLI clients
- point_command_history/get_by_command_id likewise binds command_id,
  and the accepted statusUri built by point_command write now uses it
- openapi-data snapshot updated for both query params
- cli command history and web mock handlers aligned to the same names
- web mock registers command_history under the data service prefix
2026-09-18 09:44:36 +08:00
pnokerandClaude Code b984580d4e docs(javadoc): backfill public API Javadoc
Backfill the one-line Javadoc required by check_documentation on public
top-level types and non-override public methods (1305 sites across 313
files), matching the existing single-line contract style: each doc states
the behaviour contract without @param/@return tags.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-03 12:55:54 +08:00
pnoker 7fde111341 refactor(db): move r2dbc store adapters into per-domain postgres modules
Relocate all 64 R2dbc*Store implementations from dc3-common-{auth,manager,data,agentic} repositories into new dc3-db-postgres-{auth,manager,data,agentic} adapter modules (package io.github.pnoker.db.postgres.<domain>), each registered through its own AutoConfiguration. Reactive*Store ports and filter types stay in the business modules; services are untouched. Drops stale MyBatis wording from AGENTS.md and init runner javadoc.
2026-09-03 09:01:32 +08:00
pnoker 66aab58f20 build: standardize repository tooling and quality gates 2026-09-02 20:51:10 +08:00
pnoker 51edc7c198 refactor(common): rename resource registrar module 2026-09-01 23:42:00 +08:00
pnoker 0c10f71951 build: eliminate Maven package warnings 2026-09-01 23:41:21 +08:00
pnoker eae86b58fc refactor(storage)!: unify platform persistence on reactive postgresql
- migrate auth, manager, data, and agentic persistence to R2DBC
- hard cut pagination, facade, gateway, web, and CLI contracts
- remove legacy database and TSDB adapters and document validation gates

BREAKING CHANGE: platform persistence now requires PostgreSQL 18 with TimescaleDB and the new reactive API contracts.
2026-09-01 22:03:16 +08:00
pnokerandClaude c18077d7e2 fix(data): restore transaction management with the tsdb datasource
The tsdbDataSource bean breaks by-type injection: the dynamic routing
dataSource carries no @Primary, so a candidate-eligible second
DataSource throws NoUniqueBeanDefinitionException in consumers that
cannot take a qualifier (MyBatis-Plus sqlSessionFactory, Quartz, the
health check). Mark it defaultCandidate = false so those consumers see
exactly one candidate again while the TSDB adapter keeps resolving it
through @Qualifier("tsdbDataSource").

The same second bean makes @ConditionalOnSingleCandidate(DataSource)
fail, so DataSourceTransactionManagerAutoConfiguration silently backs
off and no TransactionTemplate is auto-configured. Declare the
DataSourceTransactionManager explicitly, bound to the routing
datasource with local-transaction semantics: switching routes inside
an open transaction does not take effect.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-27 09:39:31 +08:00
pnoker a7ca489b9b style(repo): reformat java, xml, yaml, markdown and frontend sources 2026-08-25 21:27:49 +08:00
pnoker d90021ce57 fix(tsdb): capability and consistency fixes across store adapters
- let TSDB failures propagate in the alarm window data source: swallowing
  them made an outage read as no-data and silently missed alarms
- dedupe ingest batches by natural key before store upserts (same-row
  writes poison timescale ON CONFLICT batches)
- turn unsupported countByDimension into a structured refusal (analytics)
  or capability-honest empty panel (dashboard) instead of a raw 500
- pin LocalDateTime.now to DEFAULT_ZONEID on the tsdb read paths
- adapter fixes: influxdb/iotdb/tdengine/timescale store and auto-config
  corrections, iotdb system properties, capability declarations on
  TsdbStore, new timescale store unit tests
2026-08-25 19:27:51 +08:00
pnoker 7610b8dadd fix(db): harden mysql/mariadb dialects and seed conversion
- derive pagination DbType from the selected dc3-db adapter with a fail-fast
  when a non-default dc3.db.type lacks its adapter jar
- skip postgres profile activation for mariadb, not just mysql
- fork OAuthMcpMapper/AlertMapper statements for mysql and mariadb
  (CAST AS JSON, ::text casts, ODKU value references)
- honor GET_LOCK verdict in resource registry sync instead of assuming
  acquisition on every engine
- pg2mysql_seed.py: convert partial unique indexes to stored generated guard
  columns instead of dropping them; add 99-iot-dc3-grants.sql
- add DC3_DB_URL/DC3_DB_HISTORY_URL per-service overrides in center configs
- cover the new mapper forks in the db TCK; fix AGPL headers in spring
  metadata files
2026-08-25 19:27:44 +08:00
pnoker 769d1cb977 style: apply repository-wide formatting
- expand single-line Javadocs to multi-line blocks, rewrap Markdown prose, and reflow POM, YAML, and TypeScript formatting
- formatting only: no Java code lines changed, no build-file semantics changed
- verified: backend compile and check_documentation.py pass, pnpm check and pnpm lint:check pass, and the dc3 YAML files parse
2026-08-25 13:35:00 +08:00
pnoker 0ec9a636b1 docs(backend): complete the Javadoc and comment sweep
- fill the missing Javadoc on public methods, records, and types across the dc3-common, dc3-mq, and dc3-tsdb core modules
- document about 350 public constants and enum constants in dc3-common-constant, including the MCP, RabbitMQ, agentic, manager, auth, and data surfaces
- document exception constructors and replace the lombok @RequiredArgsConstructor with explicit documented constructors in ThreadPoolConfig and QuartzService
- translate the remaining non-English section comments in the TSDB store adapters
- verified: check_documentation.py passes, javadoc doclint=all reports zero warnings across all aggregators, and the touched modules compile
2026-08-25 13:29:16 +08:00
pnoker ee44de8544 docs(comments): remove orphaned and duplicated javadoc blocks
Follow-up review of the comment pass found three spots where new
javadoc landed next to pre-existing blocks:

- AnalyticsModel: SeriesSelector/TimeRange/HistoryPoint carried the same
  doc twice (the batch injector did not notice the originals).
- MybatisPlusConfig: an orphaned copy of the old mybatisPlusInterceptor
  doc sat above databaseIdProvider's own javadoc.
- EntityStateMapper: the parameter javadoc of the retired
  claimExpiredDevices method was left dangling above
  selectExpiredForClaim's doc.

All three removed; the original mybatisPlusInterceptor doc survives
(the one-line summary I added earlier was replaced by restoring the
full contract note in place). Verified by a repository-wide scan for
adjacent javadoc pairs — now zero. data suite green, compile green.
2026-08-24 21:27:20 +08:00
pnoker 1fa2d6f0cf docs(comments): fill the real comment gaps across the repository
A comment-health audit driven by an AST scanner (regex scanners kept
false-positiving on annotation-separated javadoc; the AST pass with
constructor and @Override-implementation exemptions is what produced a
trustworthy worklist) ended with a much smaller true gap than the raw
numbers suggested: class-level javadoc is already at 100% repository-wide.

What actually gets filled here:

- AnalyticsModel: all 30 nested records of the S19 analytics facet get
  maintainer-voice javadoc (the @Schema descriptions serve the API side;
  these serve code readers).
- The whole entity/ext family (29 files, 47 nested classes): every
  Content/Ui/Validation/Security/AppliesTo/Template/Dedup/... schema
  class states what it actually holds — read from the fields, not
  templated.
- Adapter families: the @Bean methods of all six MQ adapters, all four
  TSDB adapters and the three DB dialect adapters get one-line
  maintainer docs (what binds to what, what is overridable).
- Nested support types across data/agentic/constant/public/facade:
  NotifyBindKey, CacheTuning, Credential, dashboard VOs' Item/BucketVO,
  ExpireListener (plus its null-doc placeholder javadoc replaced with a
  real contract), agentic chat/stream VOs' choice/delta/usage shapes,
  tool-context records, RequestHeader envelopes, FleetSummary,
  visualization Type/Scale codes.
- PointValueSampleConverter's boundary methods now state the port-side
  contract (quality default injection point, batch order preservation).

Comment drift fixed: the class javadoc of the can/mqtt/opc-da driver
skeletons claimed "see TODO markers in method bodies" — those markers
do not exist; the claims are corrected to plain work-in-progress notes.
The genuine TODOs (zigbee API verification, lwm2m lookup, ethernet-ip
CIP framing, CAN JNI) stay: they mark real unimplemented behavior.

Vendored-code boundary: the opc-da module carries a ported
org.openscada library (46 files, package org.openscada.*). Its trivial
accessors stay bare on purpose — filler comments on get/set pairs are
exactly the kind of noise this pass exists to remove — but the
non-obvious JIStruct wire-decoder methods (getStruct/fromStruct
FILETIME + three DO wrappers) now document the COM bridge shape.

Gates: full-repo compile green; data/model/constant/agentic/public
test suites green.
2026-08-24 21:23:39 +08:00
pnoker 5a89b39ec1 refactor(build): move the relational core into the db family as dc3-db-core
dc3-common-jdbc was the family's core in disguise — MyBatis-Plus wiring,
the tenant-line handler, the databaseIdProvider, the jdbc profile. It now
lives as dc3-db/dc3-db-core, matching the dc3-mq-core / dc3-tsdb-core
shape: one aggregator, port first, adapters after, tck last. Artifact
renamed, packages deliberately untouched (same convention as the mq
family move); twelve poms rewired, the family aggregator puts core
first, and the db-tck now depends on the core explicitly instead of
riding transitives.

dc3-common-sql stays where it is on purpose: it is not platform dialect
plumbing but the base class of the four JDBC *device drivers*
(mysql/oracle/postgresql/sqlserver drivers that connect to external
databases being collected from) and depends on the driver SDK — under
dc3-db it would claim a meaning it does not have.

Gates: db-tck 24/24 (postgres + mysql + mariadb), data 274/274,
full-repo compile green.
2026-08-24 20:19:06 +08:00
pnoker de656d293c feat(db): add the mariadb adapter — third engine certified, 24/24 across the family
dc3-db-mariadb brings MariaDB 10.6+ behind dc3.db.type=mariadb with the
official mariadb-java-client and the same dual-dialect contract suite.
MariaDB shares the MySQL seed wholesale (utf8mb4_unicode_ci — the 0900
collation is MySQL-8-only and MariaDB rejects it outright, so the seed
translator and the hand-written 05 both moved to the portably-supported
collation, re-verified against mysql:8.4).

The mapper forks differ from the mysql twins in exactly one dimension,
verified against a live 10.11 container first: MariaDB never adopted
the AS-new ODKU row alias (syntax error on 10.11), so its 15 twins
reference VALUES(col) instead — everything else (GET_LOCK/RELEASE_LOCK,
FOR UPDATE SKIP LOCKED, recursive CTEs, INTERVAL ? SECOND, the
JSON_UNQUOTE(JSON_EXTRACT(...)) double-hop, REGEXP_REPLACE without the
g flag, DAYOFWEEK/HOUR units) proved compatible and rides the same
statement bodies.

Scope decision recorded in docs/db-dialects.md: DB2, H2 and HSQLDB are
explicitly out (dialect cost with zero demand; embedded test databases
unfit for multi-service concurrent IoT writes), SQLite equally unfit
for the multi-service shape (database-level write lock, no network
listener). Oracle and SQL Server stay on the roadmap with their real
obstacles spelled out (MERGE-heavy rewrite; no ARM64 mssql image for
local certification on Apple Silicon) — not started without a demand
signal.

Gates: db-tck 24/24 (postgres + mysql + mariadb, 8 each),
data/auth/manager suites green (274 in data), full-repo compile green.
2026-08-24 20:01:48 +08:00
pnoker 0ddb60f31c test(db): add the dual-dialect contract suite — postgres and mysql, 16/16
R2c of docs/design/storage-abstraction.md §3. dc3-db-tck runs one set of
mapper-level assertions against both engines with the real mapper XML —
the databaseIdProvider routing is part of what gets certified, not just
the SQL: the fenced latest-value upsert's three-level tiebreak (fencing
token, then time, then sequence, then message id), the state upsert +
same-transaction re-select shape, the three-step expired-lease claim
(lock-and-read FOR UPDATE SKIP LOCKED, flip offline, derive the
post-update view), the lease upserts' row-local increments (token moves
only on ownership change), the row-level revision triggers (insert and
meaningful update bump, trivial update does not), and the tool-catalog
query's encoded-JSON double-hop.

Harness notes worth keeping: one SqlSessionFactory per database mirrors
the production @DS wiring (state/latest/leases/catalog span four
databases with unqualified table names); the mapper XML loads through
XMLMapperBuilder with the affectData mybatis-plus DTD extension
stripped; postgres registers the timestamptz type handler. Both
harnesses learned to distrust podman: log-following never fires, so
mysql waits on the listening TCP port and postgres polls a real JDBC
handshake (the port opens during initdb's local-only phase and closes
again for the restart). The postgres harness mounts seed files 01..07
individually — 00's age/pgvector extensions exist only in the
production base image — with 05 trimmed at runtime to the
dc3_point_latest projection (the hypertable half needs the base
image's TimescaleDB function set; the dialect contract doesn't touch
it).

The catalog fixture plants its api row through a parameterized insert:
the escaped-JSON value must not travel through SQL text (MySQL's
default backslash escapes eat the \" the value depends on) nor bind
uniformly (PG rejects varchar->json, MySQL rejects the binary
Types.OTHER binding) — the fixture branches on the engine.

Selection guide published at docs/db-dialects.md (dialect matrix,
swap procedure, the semantic decisions the suite locks); the storage
design's status header moves to Implemented with an R1/R2 closeout
note.

Gates: db-tck 16/16 (8 postgres + 8 mysql), data/auth/manager suites
green (274 in data), e2e 26/26, full-repo compile green.
2026-08-24 19:13:24 +08:00
pnoker 9f91984c01 feat(db): add the mysql dialect — dc3-db-mysql, statement forks, portability rewrites
R2a of docs/design/storage-abstraction.md §3. MySQL 8 becomes a
first-class relational deployment behind dc3.db.type=mysql:

- dc3-db-mysql module: mysql-connector-j, MYSQL pagination DbType, the
  mysql profile (driver + jdbc-type-for-null), activated only when
  dc3.db.type=mysql — the postgres profile and DbType stay the default
  for unset values, so existing deployments are untouched. The neutral
  jdbc module now exposes a VendorDatabaseIdProvider (postgres/mysql
  aliases) so mapper XML can fork statements by engine.

- RETURNING decoupled where MySQL has no equivalent: the state upsert
  and the lease-state upsert no longer RETURN rows — callers re-select
  by the natural key in the same transaction (portable shape, one extra
  keyed read on both engines). The expired-lease claim becomes three
  portable steps in one TransactionTemplate: lock-and-read (FOR UPDATE
  SKIP LOCKED works identically on both engines), flip offline (INTERVAL
  expression is the only forked fragment), derive the post-update view
  in Java from the locked rows — the update is deterministic per column.

- Sequences retired in favor of row-local increments: the assignment
  version and the fencing token advance as col = col + 1 on every
  change. Per-driver / per-device monotonicity is the property callers
  and stale-writer checks rely on; the global sequences only ever
  delivered it incidentally, and MySQL has no sequence objects.

- Forked statements (databaseId twins): the point-latest fencing upsert
  (row-value tuple guard expanded into per-column IF chains — MySQL has
  neither row comparisons nor WHERE on ON DUPLICATE KEY UPDATE), the
  driver-instance upsert, both lease upserts, the driver lock (MySQL
  serializes on the driver row itself: SELECT ... FOR UPDATE, scoped to
  the transaction like the advisory lock it replaces), the registry
  lock (GET_LOCK + an explicitly paired advisoryUnlock — a no-op on
  postgres; the sync service wraps its critical section in try/finally),
  the tool-catalog candidate query (|| -> CONCAT, regexp_replace flag,
  and the (ext ->> 'content')::json double-hop re-expressed through
  JSON_UNQUOTE(JSON_EXTRACT(...)) — the content value is a JSON-encoded
  string, not a nested object, verified against live data), four alert
  statements' ->> extractions, hourly counts (DATE_FORMAT hour bucket),
  and the activity heatmap (DAYOFWEEK/HOUR units, dow 0 = Sunday).

- Portable rewrites that needed no fork: token_metadata::TEXT ->
  CONCAT(token_metadata, '') (both engines return text; only PG JDBC
  needs the nudge past PGobject), typeDistribution's GROUP BY now
  references the renamed bucket_key alias (the bare `key` was a MySQL
  reserved-word landmine).

Gates: data/auth/manager suites green (274 in data), e2e 26/26.
MySQL seed and the dual-dialect DAL TCK follow in R2b/R2c.
2026-08-24 17:06:31 +08:00
pnoker c6509762a6 refactor(db): split relational infra into a neutral jdbc module and a postgres dialect family
R1 of docs/design/storage-abstraction.md §3 — zero behavior change:

Portable SQL rewrites (verified against a live PostgreSQL before landing):
- COUNT(*) FILTER (WHERE) x3 -> COALESCE(SUM(CASE WHEN ... THEN 1 ELSE 0
  END), 0); the COALESCE keeps the empty-set result at 0 where bare SUM
  would return NULL.
- dailyTrend's generate_series + date_trunc calendar spine -> a
  RECURSIVE CTE day ladder (CAST(day + INTERVAL '1' DAY AS DATE) keeps
  the column type stable on both engines — the bare date+interval form
  widens to timestamp on PostgreSQL and fails the recursive term's type
  check, caught live); day rendering via CONCAT instead of ::text.
- OAuthMcpMapper: '{} '::JSON literals -> '{}' (unknown literals coerce
  to the JSON column on both engines), NULL::BIGINT -> NULL in the
  UNION arm, #{riskLevel}::text parameter casts dropped (the text column
  compares untyped parameters fine on both).
- The reserved-word alias AS key (a MySQL landmine hiding inside three
  otherwise-portable statements) renamed to bucket_key / bucketKey
  across both Dashboard mappers and BucketRow.

Module split: dc3-common-postgres -> dc3-common-jdbc (MyBatis-Plus
wiring, tenant-line handler, generator utils, the new dialect-neutral
jdbc profile carrying the shared mybatis-plus/hikari plumbing) + the
top-level dc3-db family's dc3-db-postgres (driver, timestamptz type
handler, postgres profile delta, and the pagination DbType bean the
neutral config now injects — defaulting to PostgreSQL so a bare
jdbc-only classpath behaves exactly as before). Packages deliberately
unchanged, mirroring the mq/tsdb family moves; all nine dependent poms
rewired; make deploy publishes the dc3-db family.

The jdbc module's interceptor integration test now pins an explicit
embedded H2 datasource — the module carries no driver on its main
classpath, so context inference would fail without it.

Gates: data/auth/manager/jdbc/db-postgres unit suites green (274 in
data), full-repo compile green, e2e 26/26.
2026-08-24 16:44:47 +08:00
pnoker 8163aefde6 feat(data): add the s19 ai analytics facet — nine conclusion-shaped tools over the tsdb port
DataAnalyticsService (biz/analytics) composes TSDB port primitives with
relational metadata into the nine coarse-grained reads of
docs/design/tsdb-abstraction.md §9.7: latest values, raw/M4 history,
per-series stats with percentiles, period comparison, entity ranking
(activity or mean/max/min), least-squares trends, threshold exceedance
reports with merged intervals, Pearson correlation, and a tenant-level
quality overview (coverage, silence, quality-code census).

Exposed as nine /analytics/* REST endpoints. The MCP gateway's tools/call
dispatches over HTTP to catalog rows synthesized from OpenAPI snapshots,
so these endpoints (x-dc3-ai annotated, fully @Schema-documented — the
data annotation gate enforces it) become the MCP tool surface after
make openapi + refreshToolCatalog; no Java tool classes are needed.

Hard guarantees: the tenant id always comes from the authenticated
principal, never the body, and the facade re-validates it (S11 double
gate); scan volume is bounded (20 series, 5000 samples per series, 90-day
windows) with structured over-budget rejections and honest degradation
notes instead of silent truncation; ambiguous name resolution fails with
the candidate list so an agent can disambiguate on the next turn.
correlate is dual-path — the store primitive when declared, otherwise
Pearson over aligned bucket pulls (verified against TDengine's
capability=false, exact on a perfectly-correlated fixture).

Gates: dc3-common-data 274/274 including 12 new analytics tests;
dc3-center-data assembly compiles.
2026-08-21 11:53:37 +08:00
pnoker 20ae3fbf33 feat(tsdb): add the tdengine adapter — second store certified against the contract suite
dc3-tsdb-tdengine maps the port onto TDengine 3.x: one supertable
point_value tagged by (tenant, device, point), one deterministic subtable
pv_<t>_<d>_<p> auto-created on first insert via USING TAGS, database
created with PRECISION 'us' and KEEP 180. Connection settings live under
dc3.tsdb.tdengine.* (REST driver, own Hikari pool); selected by
dc3.tsdb.type=tdengine. Like the broker family, only the default adapter
(timescale) ships inside consumers — switching deployments add this
dependency plus DC3_TSDB_TYPE/DC3_TSDB_TDENGINE_URL.

The supertable mapping grilled four real issues out of the port
surface, each fixed and locked by the TCK:

- Timestamps never touch string form. The REST driver serializes
  Timestamp parameters in the client JVM zone while the server parses
  them as UTC — every write would shift by the deployment zone and
  cursor pagination would drift per page without end. Instants now
  travel as epoch-micro integer literals and reads return
  CAST(ts AS BIGINT) — verified symmetric against the image.
- AS value collides with a reserved word (agg_value now);
  INTERVAL takes a bare number read in the database (micro) precision.
- PERCENTILE only runs on single tables, so single-series percentiles
  query the deterministic subtable directly; tenant-wide percentile is
  refused rather than approximated.
- REST readiness must be probed with POST /rest/sql + basic auth; the
  GET path-style route 404s even on a healthy server.

Honest capabilities: latencyHistogram=false (dashboard degrades to
zero-filled bins via a capability check) and correlation=false (future
analytics facade computes from bucketed pulls); rollups stay NONE until
S16 lands stream computing. TCK result: 23 tests, 0 failures, 0 errors,
2 capability-gated skips; the timescale suite re-ran 23/23 unchanged.

The design doc's capability matrix now reflects the adapter's declared
values instead of the pre-implementation estimates.
2026-08-21 09:01:24 +08:00
pnoker edadfa0a06 feat(tsdb): rewire the data center onto the tsdb port and retire the repository module
Phase 1b of docs/design/tsdb-abstraction.md: every point-value read and
write now flows through the dc3-tsdb port, and dc3-common-repository is
deleted outright (no compat aliases).

Write path — new PointValueIngestService orchestration in dc3-common-data:
the stale-owner lease guard leaves the history INSERT's cross-schema join
and resolves the active owner per distinct device through the existing
DeviceFacade.getActiveOwner chain; a caffeine-backed ingest idempotency
window (dc3.data.ingest.*, marked only after both writes commit) absorbs
MQ redeliveries now that the message_id unique index is retired; the
batch lands via TsdbStore.append (natural series+time upsert, INGEST_ORDER
kept for the fenced dc3_point_latest upsert).

Read path — history via last(), page() resolves name/enable filters to
series keys through relational metadata (tenant-wide when unrestricted,
cursor offset emulation capped at 10k) with count() for totals; the alarm
window backend becomes TsdbWindowDataSource over aggregate()/history().

Dashboards — all nine data-side hypertable statements move to the S13
analytics facet (count/bucketedCount/countByDimension/latencyHistogram/
lastSeenPerSeries) plus the dc3_point_latest projection for latestStream;
the manager topology statement crosses into the data center via a new
ListSeriesVolumes RPC backed by the new S13-5 seriesCounts primitive (a
point shared by several devices needs per-series counts, which
single-dimension grouping cannot reconstruct); both cross-schema joins
are gone.

Port/adapter — seriesCounts added with a TCK case (23 total); series
filters compile to row-value IN lists chunked at 500 pairs; bootstrap
adds the quality column idempotently and retires uk_point_value_event so
existing deployments converge on the new upsert; the adapter binds the
application-provided tsdbDataSource bean (the history dynamic-datasource
entry) instead of the routing primary.

Timestamps — BO<->Instant conversion is pinned to
TimeConstant.DEFAULT_ZONEID (the platform canonical zone the latest
projection type handler already writes with); the design note's "lock
UTC" is amended accordingly, together with the lease-guard and S13-5
deviations recorded in §6.2.

Seed/deploy — 05-iot-dc3-history.sql gains the quality column and swaps
the unique index; compose carries DC3_TSDB_TYPE; make deploy publishes
dc3-tsdb.

Also fixed along the way: root pom never managed dc3-mq-core (latent,
masked by full-reactor builds); PointValueMapper ran unrouted on the
master datasource (now @DS("history")); latestStream name enrichment
looked up a Long-keyed map with String keys.

Gates: tsdb TCK 23/23, dc3-common-data 262/262, e2e 26/26 including the
unmodified PostgresHypertableIT; full reactor green except dc3-mq-tck's
kafka/pulsar container-startup flake, reproduced identically twice and
unrelated to this change.
2026-08-20 21:58:01 +08:00
pnoker d003291f81 refactor(build): extract the mq family into a top-level dc3-mq aggregator
The broker-selection family (port + six certified adapters + tck) is a
deployment selection, not shared plumbing, so it moves out of dc3-common
into a top-level dc3-mq aggregator beside dc3-api — the same standing as
the gRPC contract layer, and a template for future pluggable families
(storage is next on the design board).

- dc3-common-mq -> dc3-mq/dc3-mq-core (port: API + core runtime)
- dc3-common-mq-{rabbitmq,kafka,rocketmq,pulsar,activemq,mqtt,tck} ->
  dc3-mq/dc3-mq-{...}; no compatibility aliases — every coordinate is
  renamed everywhere in one commit (business poms, coverage, root
  dependency management)
- dc3-common sheds eight modules back to plain shared plumbing; the
  device-access dc3-common-mqtt stays by design (protocol stack, not a
  broker selection)
- Java packages deliberately stay io.github.pnoker.common.mq.* — the
  facade modules set the precedent that packages do not track module
  homes, keeping this a pom-only, zero-source refactor
- make deploy now publishes dc3-api + dc3-common + dc3-mq (third-party
  driver authors compile against the port transitively)
- docs/mq-brokers.md and the design's module layout follow the new home

Verified: full-repo test-compile green; constant/data/driver/manager
unit suites unchanged and green (255/144/291); the rabbitmq contract
suite passes 12/12 from the new coordinates against a live broker,
proving adapter wiring survived the move.
2026-08-20 12:23:09 +08:00
pnoker 28efa5c8fc refactor(mq): migrate business modules onto the messaging port
Complete phase 1 of the mq abstraction design: business code compiles
against the broker-neutral port with zero amqp classes, and the legacy
dc3-common-rabbitmq module is deleted.

- dc3-common-driver: 8 sender methods plus the sqlite outbox now publish
  through MessageSender (outbox keeps durability; legacy physical routing
  keys in pending rows are normalized on republish); metadata/command/
  point-command receivers become @Dc3Listener subscriptions with
  placeholder-driven per-instance queues; DriverTopicConfig dissolved into
  adapter subscribe-time declarations
- dc3-common-data: command and point-command dispatch use sendConfirmed
  with the correlation-id header; state timeout, device scan tick, notify
  tasks and metadata events send through logical topics; all 13 receivers
  including the batch point-value consumer migrate to @Dc3Listener;
  DataTopicConfig, PointValueRabbitConfig and the duplicated batch
  properties class move into the port/adapter (same config keys)
- dc3-common-manager / facade-local-manager: metadata fan-out via the port
- dead-letter consumers read the correlation id from the standardized
  header the adapter mirrors onto the amqp property
- 14 test classes adapted to the new listener/sender shapes; module poms
  swap dc3-common-rabbitmq for dc3-common-mq + dc3-common-mq-rabbitmq

Verified: unit suites green for constant (291 incl. boundary guard),
data (165), driver (144), manager (255); full-repo main and test compile
green. Container-backed E2E (RabbitDeliveryIT) intentionally not run here.
2026-08-19 18:47:50 +08:00
pnoker 90a7db4d6b merge: integrate upstream deployment docs and validation targets 2026-08-19 14:30:34 +08:00
pnoker 51f0efd2d9 style(javadoc): document non-override public methods 2026-08-19 00:05:58 +08:00
pnoker aec6ebfc76 refactor(logging): standardize application logging 2026-08-18 08:14:52 +08:00
pnoker 956de3dd38 feat(driver)!: enforce lease-fenced durable telemetry
Use PostgreSQL leases and fencing for distributed ownership, require a durable SQLite outbox before RabbitMQ publication, and make Data Center ingestion transactional and idempotent.

BREAKING CHANGE: drivers require mandatory durable outbox configuration and use lease-aware ownership and telemetry contracts.
2026-08-18 00:50:57 +08:00
pnoker c87f31be9a refactor!: emit identifiers as strings on the HTTP contract
BREAKING CHANGE: REST VO/DTO identifier fields (id and all *Id) now serialize as JSON strings. Internal Long identifiers (DO/BO/gRPC/RabbitMQ) are unchanged; MapStruct converts at the boundary.
2026-08-17 21:37:37 +08:00
hongyuan.zhang 6b37cb5ccd docs(readme): refresh module documentation 2026-08-11 22:58:30 +08:00
pnoker 50c6de9438 docs: rewrap project guides and README files
Prose rewrapping and table reflow from a project-wide markdown pass;
no content changes. Also reverts two formatter mistakes: restores
MD5(password) in dc3-web/AGENTS.md and drops spurious spaces after
CJK punctuation in README.zh.md / README.ja.md.

Covers root guides (AGENTS/CONTRIBUTING/SECURITY/README.*), module
READMEs across dc3-api/center/common/driver/e2e/gateway, the nginx
guide, dc3/doc/{README,TITLE}.md, and dc3-web docs.
2026-08-09 18:44:55 +08:00
pnoker 8201d75de6 style(java): reformat auth/agentic/data sources and tests
Pure formatting from a project-wide pass (import ordering, whitespace
alignment); no behavior change.

- dc3-common-agentic: AgenticRunEvent ternary indent
- dc3-common-auth: TokenController import order; TokenServiceImplTest
  and McpRuntimeServerTest comma/import spacing
- dc3-common-data: PointValueController parameter indent
- dc3-common-public: KeyUtilTest comma spacing

Verified: auth + public unit tests green (TokenServiceImplTest 18,
McpRuntimeServerTest 4, KeyUtilTest 9); data + agentic compile clean.
2026-08-09 18:42:56 +08:00
Henry ZhangandClaude Fable 5 7c643a8e80 fix(api): dc3-api contract overhaul (auth JWT, history, presence, decision enum) (#193)
* fix(api): correct inverted enable_flag comment and document time unit

- tenant.proto enable_flag comment said "1 enabled, 0 disabled" but the
  actual EnableFlagEnum is ENABLE=0, DISABLE=1 — callers reading the
  comment would invert the flag. Correct it.
- base.proto create_time/operate_time are epoch milliseconds
  (LocalDateTimeUtil/Instant.toEpochMilli); document the unit so callers
  don't guess seconds vs millis.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(api): carry timestamp in point-value history; explicit bool presence

#5 Point-value history: the whole chain (proto -> gRPC server ->
PointValueService -> repository -> facades -> controller -> agentic tool)
returned List<String> and dropped create_time even though the DB row
already carried it. Switch to PointValueBO / FacadePointValueBO end to
end so callers receive each value with its timestamp.

#6 Field presence: GrpcR.ok and McpIntrospectDTO.active are now optional
bool, so "unset" is distinguishable from false on the wire.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(auth): unify JWT signing key, drop salt/password from token contract

The JWT signing key was per-principal — securityKey + ":" + salt, where
salt was the credential salt also carried through the gateway and the
token RPC. That makes the JWT key weak and widens salt exposure to every
gateway hop. Sign with the single configured securityKey (HS256) instead
and remove salt from the JWT path entirely:

- KeyUtil.generateToken/parserToken drop the salt parameter; the HMAC
  key is now just securityKey (must be >= 32 bytes for HS256).
- TokenService.generateToken/checkValid drop the salt parameter.
- token.proto GrpcLoginQuery drops salt and password — CheckValid only
  ever needed tenant, name and token.
- gateway FilterService, TokenGrpcFacade, TokenController adapted; the
  request header no longer feeds salt into validation.

The login-handshake salt (POST /token/salt, used to salt the password
client-side before verifyPassword) is unchanged — only the JWT signing
salt is removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(api): type MCP authorize decision as proto enum

The authorize-tool-call decision was a free-form string, so callers
could only compare it against literals with no compile-time check.
Introduce GrpcMcpDecision (MCP_DECISION_UNSPECIFIED / AUTHORIZED /
CONFIRM_REQUIRED / REJECTED) and type GrpcMcpToolAuthorizeDTO.decision
with it. Enum names match the existing auth-side literals, so the
mapping is a direct valueOf / name().

- McpRuntimeServer builds the enum via toGrpcDecision (unknown -> UNSPECIFIED).
- McpRuntimeGrpcFacade maps enum -> BO string via name().
- gateway unchanged: it receives the BO (string decision) through the facade.

The other string MCP fields (risk_level, status, grant_type,
principal_type) stay strings for now — their value sets are wider and
partly lowercase (OAuth grant_type), so they are a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(facade): align TokenLocalFacade with 3-arg checkValid (CI compile fix)

TokenLocalFacade (local impl) was missed when TokenFacade.checkValid
dropped salt; CI compiles facade-local-auth independently and caught
it. Align it + its test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 16:36:35 +08:00
Henry ZhangandClaude Fable 5 73ff48bcb9 style: reformat codebase (backend java/xml, docs, web) (#175)
* style(backend): reformat and rearrange java/xml sources

Apply code cleanup across dc3-common and dc3-driver: whitespace and
indent normalization, import sorting, member rearrangement, and javadoc
normalization. No semantic changes; compilation verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style(docs): reformat vitepress vue diagram components

Expand compact svg templates and normalize formatting across diagram
components, the seo module, and the theme stylesheet. No behavior
changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style(docs): reformat markdown pages

Normalize markdown formatting. Restore the <script setup> component
imports that an earlier unused-import pass had wrongly stripped; every
<Diagram> tag now resolves to a backing import again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style: reformat point view and trim pom whitespace

Normalize whitespace in the point view and drop trailing whitespace in
pom.xml. No behavior changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 12:13:36 +08:00
pnoker 056bd9ae21 Merge remote-tracking branch 'origin/develop' into chore/promote-develop-main-3 2026-08-03 21:57:03 +08:00
Henry Zhang 00c67406b0 refactor(quality): fix unused-param/confusing-sig/local-temp/js-comparison (12) (#163)
- unused-parameter (7 private/non-override): drop unused params in
  Snmp getConnector/buildTarget, Sl651 handleSl651Message,
  PointCommandValidator.validateWriteValue, TypedValueConverter
  finiteFloat/finiteDouble (+ cascade callers). 3 interface default
  methods kept (contract).
- confusing-method-signature (1): AuthenticGatewayFilterFactory
  <Object> → dedicated Config type; apply(Object)→apply(Config)
- local-temp-file-info-disclosure (1): FileUtil creates temp dirs
  with owner-only rwx------ via PosixFilePermissions (Windows fallback)
- js/comparison-between-incompatible-types (1): isNull drop redundant
  val === undefined (== null loose already covers it)

Behavior-preserving. Compile + test-compile clean.
2026-08-03 21:55:13 +08:00
Henry Zhang 4e2c5bc770 refactor(quality): fix 27 medium-risk CodeQL alerts (null/exception/boundary/exposure) (#162)
- non-null-boxed (7): GroupServiceImpl extract Byte→byte; DashboardServiceImpl
  Long→long for primitive id fields
- uncaught-number-format (7): try-catch parse failures in FINS/EthernetIP/BACnet/
  Dashboard (fall back to 0 / skip, matching existing pattern); remove dead
  unreachable outer catch in BACnet
- dereferenced-null (4): DeviceMetadata expand MapUtils.isEmpty to explicit
  null check for short-circuit clarity
- ignored-error-status (4): ConcurrentLinkedQueue.offer→add (unbounded, offer
  always true; add's boolean is "modified", not error)
- index-out-of-bounds (2): orderedMap(Object...) guard odd-length → IAE
- internal-rep-exposure (3): CoverageGapVO/Pages/DriverMetadata return
  unmodifiable views + addX/removeX mutators; migrate all callers + tests

Behavior-preserving. Compile clean (except bacnet4j pre-existing mirror miss).
2026-08-03 21:49:22 +08:00
Henry Zhang 8531e3b872 refactor(quality): migrate 23 deprecated API calls (#161)
- commons-lang3 3.20: StringUtils.equals/equalsIgnoreCase/contains/
  startsWith/removeEnd → Strings.CS/CI (16 calls)
- Jackson 2.19+: JsonNode.fields() → properties() (4, McpOpenApiAggregator)
- Spring AI 2.0: ToolCallAdvisor → ToolCallingAdvisor;
  toolCallbacks(provider) → tools(provider) (3, + tests)

Behavior-preserving. Compile clean.
2026-08-03 21:49:08 +08:00
Henry Zhang 852f0682d3 refactor(quality): fix 21 low-risk CodeQL quality alerts (#160)
- local-variable-is-never-read (3): dead host/port/database locals in
  AbstractJdbcDriverCustomService (subclasses read config themselves)
- unused-container (2): unused `valid` set in AlarmRuleTriggerServiceImpl;
  unused driverSenderService field/ctor param in CanDriver (+ test)
- unknown-javadoc-parameter (12): dangling ctor Javadoc on
  @RequiredArgsConstructor classes / stale @param
- call-to-object-tostring (3): @ToString on MessageHeader (root cause);
  field-log MetadataEventDTO instead of hash
- constant-comparison (1): DriverInitRunner == → >= (defensive)

No behavior change. Compile clean (agentic/can/data/auth/mqtt/driver/sql).
2026-08-03 21:49:03 +08:00
Henry Zhang 5e0003f019 build(deps): bump mybatis-plus 3.5.16 → 3.5.17 + adapt IService/ServiceImpl imports (#154)
mybatis-plus 3.5.17 fixes the JDK 9+ split-package conflict (#7027) by
relocating service classes from com.baomidou.mybatisplus.extension.service
to com.baomidou.mybatisplus.spring.service. Verified via the 3.5.17 jar:
only IService and ServiceImpl moved — Page, JacksonTypeHandler, chain
wrappers, Db, interceptors all stay in extension. Updated 112 imports.
Supersedes dependabot #141 (which failed compile on the same move).

Verified: mvn compile clean across every IService module (bacnet4j driver
excluded only due to a local mirror cache miss, unrelated to this change).
2026-08-03 13:31:22 +08:00
pnoker a1876727d1 refactor(data): remove PointValueJob, hand ingestion to buffer
Point-value ingestion is now driven by PointValueIngestBuffer workers, so the Quartz 5s tick job is removed; ScheduleForDataServiceImpl only registers the hourly maintenance job; HourlyJobForData test migrated to its own file.
2026-07-27 01:34:34 +08:00
pnoker 6293bbdd19 refactor(data): route point value ingestion through bounded buffer
PointValueReceiver offers every message to PointValueIngestBuffer (ack on accept, nack-requeue when full) and switches to the high-throughput listener container factory; drops the speed-threshold dual path.
2026-07-27 01:34:34 +08:00
pnoker f8ddcf5877 feat(data): add PointValueIngestBuffer with bounded queue and backpressure
Bounded ArrayBlockingQueue plus worker threads draining on a size-or-time trigger; offer returns false when full so the receiver can nack-requeue; a failed save re-queues the batch for retry. PointValueServiceImpl.savePointValuesToRepository wraps failures into RepositoryException so the buffer can re-queue instead of silently dropping.
2026-07-27 01:34:33 +08:00
pnoker c02fc3b3fb refactor(data): extend PointBatchProperties for bounded ingest buffer
Replace speed/interval threshold with queueCapacity, batchSize, flushIntervalMillis and workerCount for the new bounded ingest buffer.
2026-07-27 01:34:32 +08:00
pnokerandClaude 9bd4dd303f refactor(bo): annotate inherited Lombok getters with @Override and add @ToString
Mark @Getter with onMethod_={@Override} on BOs/enums whose accessors override a base-class getter, and add @ToString to the ChatCompletion VOs and Pages.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-16 20:03:41 +08:00
pnoker 10dd261e2d docs(data): document notify and rule state service impl helpers
- NotifyChannelBindServiceImpl: fuzzyQuery, checkDuplicate,
  requireReferences, getDOById
- NotifyChannelServiceImpl: fuzzyQuery, checkDuplicate, getDOById
- NotifyHistoryServiceImpl: fuzzyQuery, getDOById
- RuleStateServiceImpl: fuzzyQuery, checkDuplicate (rule+type+entity+
  fingerprint), getDOById
2026-07-14 17:52:39 +08:00
pnoker 239b2bc000 docs(data): document scanner, state-flip and notify adapter helpers
- EntityStateExpiryScanner: document scanExpiredDevices,
  buildOfflineAlarm, completeExpiredDevice
- DeviceStateServiceImpl/DriverStateServiceImpl: document isFlip and
  online (ONLINE+MAINTAIN count as the online side)
- FeishuBotNotifyChannelAdapter: document buildFeishuMessage
  (type dispatch) and sign (HMAC-SHA256)
- MessageRenderServiceImpl: document selectTemplate and missingVariables
2026-07-14 17:48:04 +08:00
pnoker 3a2de55f4b docs(data): document alarm engine private methods
- ConditionEvaluator: document evaluateNumeric/evaluateText/isNumericOperator
  operator-dispatch helpers
- RuleNotificationServiceImpl: document persistRuleState (state-machine
  transition), loadState, fingerprint (dedup key resolution)
- NotifyWorker: document dispatch and loadChannel
- RuleAlarmPersistenceServiceImpl: document persistEntityAlarm and
  getFiringAlarmId
- WindowedRuleEvaluator: document evaluate/evaluateAggregate/evaluateFold
  window-mode dispatch
2026-07-14 17:44:13 +08:00