sweep every module to satisfy the new method-name rule, renaming
about 40 methods together with their interface, implementation, test
and cli call sites: store find() lookups become get(), analytics
queryLatest/queryHistory become getLatest/getHistory, sql helper
query() becomes spec()/sql(), jwks fetch() becomes load(), and the
cursor record component queryDigest becomes requestDigest
no behavior changes; full unit suite (5278 tests) and both quality
gates stay green
Backfill the one-line Javadoc required by check_documentation on public
top-level types and non-override public methods (1305 sites across 313
files), matching the existing single-line contract style: each doc states
the behaviour contract without @param/@return tags.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- migrate auth, manager, data, and agentic persistence to R2DBC
- hard cut pagination, facade, gateway, web, and CLI contracts
- remove legacy database and TSDB adapters and document validation gates
BREAKING CHANGE: platform persistence now requires PostgreSQL 18 with TimescaleDB and the new reactive API contracts.
- fill the missing Javadoc on public methods, records, and types across the dc3-common, dc3-mq, and dc3-tsdb core modules
- document about 350 public constants and enum constants in dc3-common-constant, including the MCP, RabbitMQ, agentic, manager, auth, and data surfaces
- document exception constructors and replace the lombok @RequiredArgsConstructor with explicit documented constructors in ThreadPoolConfig and QuartzService
- translate the remaining non-English section comments in the TSDB store adapters
- verified: check_documentation.py passes, javadoc doclint=all reports zero warnings across all aggregators, and the touched modules compile
A comment-health audit driven by an AST scanner (regex scanners kept
false-positiving on annotation-separated javadoc; the AST pass with
constructor and @Override-implementation exemptions is what produced a
trustworthy worklist) ended with a much smaller true gap than the raw
numbers suggested: class-level javadoc is already at 100% repository-wide.
What actually gets filled here:
- AnalyticsModel: all 30 nested records of the S19 analytics facet get
maintainer-voice javadoc (the @Schema descriptions serve the API side;
these serve code readers).
- The whole entity/ext family (29 files, 47 nested classes): every
Content/Ui/Validation/Security/AppliesTo/Template/Dedup/... schema
class states what it actually holds — read from the fields, not
templated.
- Adapter families: the @Bean methods of all six MQ adapters, all four
TSDB adapters and the three DB dialect adapters get one-line
maintainer docs (what binds to what, what is overridable).
- Nested support types across data/agentic/constant/public/facade:
NotifyBindKey, CacheTuning, Credential, dashboard VOs' Item/BucketVO,
ExpireListener (plus its null-doc placeholder javadoc replaced with a
real contract), agentic chat/stream VOs' choice/delta/usage shapes,
tool-context records, RequestHeader envelopes, FleetSummary,
visualization Type/Scale codes.
- PointValueSampleConverter's boundary methods now state the port-side
contract (quality default injection point, batch order preservation).
Comment drift fixed: the class javadoc of the can/mqtt/opc-da driver
skeletons claimed "see TODO markers in method bodies" — those markers
do not exist; the claims are corrected to plain work-in-progress notes.
The genuine TODOs (zigbee API verification, lwm2m lookup, ethernet-ip
CIP framing, CAN JNI) stay: they mark real unimplemented behavior.
Vendored-code boundary: the opc-da module carries a ported
org.openscada library (46 files, package org.openscada.*). Its trivial
accessors stay bare on purpose — filler comments on get/set pairs are
exactly the kind of noise this pass exists to remove — but the
non-obvious JIStruct wire-decoder methods (getStruct/fromStruct
FILETIME + three DO wrappers) now document the COM bridge shape.
Gates: full-repo compile green; data/model/constant/agentic/public
test suites green.
Phase 1b of docs/design/tsdb-abstraction.md: every point-value read and
write now flows through the dc3-tsdb port, and dc3-common-repository is
deleted outright (no compat aliases).
Write path — new PointValueIngestService orchestration in dc3-common-data:
the stale-owner lease guard leaves the history INSERT's cross-schema join
and resolves the active owner per distinct device through the existing
DeviceFacade.getActiveOwner chain; a caffeine-backed ingest idempotency
window (dc3.data.ingest.*, marked only after both writes commit) absorbs
MQ redeliveries now that the message_id unique index is retired; the
batch lands via TsdbStore.append (natural series+time upsert, INGEST_ORDER
kept for the fenced dc3_point_latest upsert).
Read path — history via last(), page() resolves name/enable filters to
series keys through relational metadata (tenant-wide when unrestricted,
cursor offset emulation capped at 10k) with count() for totals; the alarm
window backend becomes TsdbWindowDataSource over aggregate()/history().
Dashboards — all nine data-side hypertable statements move to the S13
analytics facet (count/bucketedCount/countByDimension/latencyHistogram/
lastSeenPerSeries) plus the dc3_point_latest projection for latestStream;
the manager topology statement crosses into the data center via a new
ListSeriesVolumes RPC backed by the new S13-5 seriesCounts primitive (a
point shared by several devices needs per-series counts, which
single-dimension grouping cannot reconstruct); both cross-schema joins
are gone.
Port/adapter — seriesCounts added with a TCK case (23 total); series
filters compile to row-value IN lists chunked at 500 pairs; bootstrap
adds the quality column idempotently and retires uk_point_value_event so
existing deployments converge on the new upsert; the adapter binds the
application-provided tsdbDataSource bean (the history dynamic-datasource
entry) instead of the routing primary.
Timestamps — BO<->Instant conversion is pinned to
TimeConstant.DEFAULT_ZONEID (the platform canonical zone the latest
projection type handler already writes with); the design note's "lock
UTC" is amended accordingly, together with the lease-guard and S13-5
deviations recorded in §6.2.
Seed/deploy — 05-iot-dc3-history.sql gains the quality column and swaps
the unique index; compose carries DC3_TSDB_TYPE; make deploy publishes
dc3-tsdb.
Also fixed along the way: root pom never managed dc3-mq-core (latent,
masked by full-reactor builds); PointValueMapper ran unrouted on the
master datasource (now @DS("history")); latestStream name enrichment
looked up a Long-keyed map with String keys.
Gates: tsdb TCK 23/23, dc3-common-data 262/262, e2e 26/26 including the
unmodified PostgresHypertableIT; full reactor green except dc3-mq-tck's
kafka/pulsar container-startup flake, reproduced identically twice and
unrelated to this change.
Use PostgreSQL leases and fencing for distributed ownership, require a durable SQLite outbox before RabbitMQ publication, and make Data Center ingestion transactional and idempotent.
BREAKING CHANGE: drivers require mandatory durable outbox configuration and use lease-aware ownership and telemetry contracts.
BREAKING CHANGE: REST VO/DTO identifier fields (id and all *Id) now serialize as JSON strings. Internal Long identifiers (DO/BO/gRPC/RabbitMQ) are unchanged; MapStruct converts at the boundary.
- EventAttributeExt/CommandAttributeExt: change @Schema example from
Chinese placeholder to 'Enter text'
- dc3-web/package.json: change description to English
Part of the x-dc3-ai annotation-standard rollout. Complete method/param javadoc (TokenService, RepositoryService, RuleEngine, ...), flag work-in-progress utilities and drivers (X509Util, MQTT/CAN/DLMS/... ) with explicit WARNING notes, and remove redundant inline comments / blank lines from query/VO/ext/constant classes plus mapper indentation. Comment/doc only, no behavior change.
OAuthClientRegistration request/response are REST controller bodies
(@RequestBody and Mono<R<...>> on POST /client/register) that never cross
a gRPC or messaging boundary, so the DTO suffix was misplaced. Move them
into auth.entity.vo (project VO convention is per-module) and drop the
suffix. Kept as plain Serializable rather than extending BaseVO to avoid
injecting audit fields into the OAuth-spec wire payload.
Replace the McpConnectionToolsReplace/McpToolCatalogList request DTOs (in
dc3-common-model) with McpConnectionToolsReplaceVO/McpToolCatalogQueryVO and a
full set of Mcp/OAuth VOs in dc3-common-auth, wired through MapStruct builders
for VO<->BO mapping, per the project DO/BO/VO convention. Update the
PointValueMapper WindowAggregateRequest reference to WindowAggregateQuery.
Standardize description wording, add scalar-field examples, mark required
fields via requiredMode, and drop examples from object/collection fields.
Spans VO/Query/ext/controller plus related BO/DTO/response types.
Tool authorization (HIGH-risk confirmation):
- add AuthorizeToolCall RPC; the gateway calls it before tool execution
- sign/consume confirmation tickets with idempotency-key dedup
- propagate x-dc3-ai metadata (riskLevel/destructive/openWorld/idempotent) from the scan through the registry to the tool catalog
OAuth hardening:
- rotate refresh tokens and revoke the authorization on replay via previous_refresh_token_hash
Password lifecycle:
- require password change on login when flagged or expired (R20303/R20304)
- expose a public POST /token/change_password endpoint
OpenAPI aggregation:
- switch McpOpenApiAggregator to classpath static snapshots, dropping the runtime fetch and McpAggregatorProperties
Finish moving io.swagger.v3.oas.annotations imports to alphabetical position
in remaining auth controllers. Add @Schema descriptions to SessionExt,
BaseVO, and Pages for richer OpenAPI schema generation.
Align CommandHistory/PointCommandHistory with the EventHistory pattern:
DB stores the enum index (SMALLINT via @EnumValue), gRPC carries int32,
JSON keeps enum name. Removes the lone string-code outlier in the schema.
- DO status/type/source: String code -> enum types
- VO type/source: String -> enum (status already enum); JSON stays name
- Builder: drop ofCode @AfterMapping (DO and VO now share enum type)
- RabbitMQ DTOs (CommandCall/PointCommandResult): status/source -> enum
- driver sendResult signatures: String status -> PointCommandStatusEnum
- gRPC command_history.proto status/source: string -> int32; server uses
getIndex()/ofIndex() to match EventHistoryServer
- tests updated to assert enum values
- Update test references to the renamed *TypeEnum/RuleStatusEnum.
- Add the new builder/PermissionProvider constructor args to the affected
service-impl test setups.
- Fix pre-existing failures unrelated to this work: String-vs-Long assertions
(DriverStatusServiceImplTest), save->saveBatch (EntityStateExpiryScannerTest),
List.of(null) NPE (AlarmRulePipelineServiceImplTest), and the cache/batch
refactor expectations (PointValueServiceImplTest).
Align the DO/BO/VO model and enums with the project domain-modeling rules:
- Rename 24 multi-valued *FlagEnum to *TypeEnum (closed classifications) and
RuleStateFlagEnum to RuleStatusEnum (state machine); keep true 0/1 toggles as
*FlagEnum (EnableFlagEnum/DefaultFlagEnum/ConfirmFlagEnum). Add the new
ConfirmFlagEnum.
- Give every enum the consistent triplet + ofIndex/ofCode/ofName (fill the two
Agentic enums) and switch underscore codes to hyphen-case.
- Replace bare Byte/Integer/String flag fields on VOs with their domain enums
(EventHistory, AlertItem, ...); centralize index/code<->enum conversion in the
MapStruct builders instead of services; drop ad-hoc (byte) enum casts.
- Stop leaking DO out of the data history services/controllers: the command /
point-command / event history services now return VO, converting inside the
service via the builder.
- Add @Schema (description + example/required, verified enum constants) across
VO/BO/Query DTOs and the entity/ext nested beans; @Schema the R<T> wrapper;
exclude credential fields from @ToString.
- Rename LocalCacheService→LocalCacheImpl, PointValueLocalCacheService→
PointValueLocalCache (concrete classes must not use *Service suffix)
- Rename LogsType→LogsTypeEnum, WindowMode→WindowModeEnum (all enums
must end with Enum suffix)
- Rename ResponseEnum.text→remark, ResourceTypeFlagEnum.text→remark
(consistent field naming across all 41+ enums)
- Rename RwFlagEnum constants: R→READ_ONLY, W→WRITE_ONLY, RW→READ_WRITE
- Update all callers, imports, and test files across 50+ files
Introduce CommandAttribute/EventAttribute with config variants:
- REST controllers, MyBatis mappers, services, and DAL managers
- gRPC client/server builders and proto message definitions
- DTOs, BOs, DOs, VOs for type-safe data transfer layer
fix: add explicit ::text casts in json_build_object parameters
PostgreSQL cannot infer parameter types inside json_build_object()
calls. Add ::text casts to #{stateExtType} and #{stateDescription}
to resolve "could not determine data type of parameter $10" errors.
refactor: restructure driver gRPC client and metadata layer
- Rename GrpcStubConfig to DriverClientStubConfig with dedicated DTOs
- Extract attribute builders from inline gRPC conversion logic
- Consolidate driver metadata, register service, and command handling
- Update device state heartbeat and command result event handling
docs: add multilingual architecture diagram and sync READMEs
- Translate architecture SVG into zh, ja, vi variants
- Update README architecture image references per language
- Add missing service-level shortcut sections to ja/vi READMEs
P0: Replace content JSON-string with sealed PointCommandPayload + record DTOs.
P0: Accept write() boolean return, suppress echo on failure.
P1: Return commandId from POST endpoints, add RESTful GET /{commandId}.
P1: Add driver-online check via EntityStateMapper, PointCommandValidator.
P2: Add DeviceLockManager for per-device serialized command execution.
P3: Add POST /list API with pagination, commandId idempotency support.
- Replace two duplicate enums with single EntityStatusEnum (same index/code)
- Rename DB columns: state_flag→entity_state_flag, state_ext→entity_state_ext
- Add stateDescription field to DeviceStateDTO/DriverStateDTO for diagnostics
- Add description field to DeviceHealthState/DriverHealthState
- UPSERT SQL now writes stateDescription into entity_state_ext on heartbeat
- Remove EntityStateStatus interface (no longer needed with single enum)
- Update MQ layer, heartbeat services, status queries, gRPC, and all tests
Phase 2-4 of the point command chain redesign (point-command.md):
- Add DLX (dead letter exchange/queue) for failed commands
- Add result exchange/queue and PointCommandResultReceiver
- Add PointCommandDeadReceiver for DLX message processing
- Add CommandDedupCache (Caffeine 5-min TTL) for driver-side idempotency
- Add PointCommandResultDTO for driver-to-center result receipts
- Persist commands (PENDING->SENT) with commandId/CorrelationData
- Enhance validateCommandScope: device/point enable flag, rwFlag for write
- Add GET /point_command/get_by_command_id query endpoint
- Update PointCommandDTO to carry commandId and tenantId
- Update PointCommandReceiver: dedup check, result sending, bounded retry
Replace Caffeine LocalCacheService + @Scheduled scanner with
lease-based timeout using RabbitMQ TTL + DLX:
- Driver: each heartbeat publishes a 45s delayed check message;
DriverTimeoutCheckReceiver performs secondary lease_version and
expire_time verification before marking offline.
- Device: a self-sustaining 10s tick queue triggers batch scanning
of expired device leases in EntityStateExpiryScanner.
- Remove OfflineExpiryListener and LocalCacheService from state chain.
- Heartbeat services now only write dc3_entity_state (source of truth).
Phase 4 lifts the LAST-only gate so users can save AVG/MIN/MAX/SUM/
COUNT/ALL/ANY rules ahead of the runtime evaluator landing. RuleExt's
window block already declares a mode + ISO-8601 duration + minSamples,
but nothing parsed those fields — save validation rejected anything
non-LAST and runtime ignored the duration entirely.
Add three pieces:
- WindowMode enum (dc3-common-model) — typed mode value with case-
insensitive ofCode(), plus reducesToScalar() / requiresDuration()
classifiers used by the future evaluator.
- WindowSpec record — parsed view {mode, duration, minSamples, valid,
reason} with last() / ok() / invalid() factories.
- WindowSpecParser — turns RuleExt.Window into WindowSpec, applying
the rules consistently (null window = LAST, blank mode = LAST,
duration parse failure = invalid, non-positive duration on
aggregation modes = invalid). Future runtime evaluator + this
validator both call through the same parser.
RuleServiceImpl.validateWindowMode now routes through the parser:
unknown modes, malformed durations, and zero/negative durations on
aggregation modes get UnSupportException at save time. AVG with PT3M
is now accepted at save (the runtime path still no-ops for non-LAST
until the evaluator wiring lands).
Existing rejection tests flipped: AVG with PT3M is now accepted, the
rejection paths are exercised through unknown mode, zero duration,
and malformed duration. WindowSpecParserTest covers parser behavior
in isolation.
Outbound notification used to call adapter.send synchronously inside
the @Transactional rule pipeline, so a slow webhook or feishu adapter
would stretch the rule transaction and starve the rabbit consumers
that feed it. Move the dispatch onto its own queue:
- New dc3.q.notify.task queue bound to the alarm exchange with routing
key dc3.r.notify.task.<channel-type>; 24h TTL on the queue caps any
runaway backlog when an adapter is fully wedged.
- NotifyTaskDTO carries the rendered MessagePayload plus the PENDING
notify_history row id so the worker can mutate it in place.
- NotifyTaskSender publishes tasks; RuleNotificationServiceImpl writes
the PENDING row, advances last_notify_time for rate-limit debouncing,
and hands the task to the sender — no blocking I/O remains in the
notification entrypoint.
- NotifyWorker consumes the queue, looks up the channel/adapter, and
stamps the history row with SUCCESS / FAILED / SKIPPED. Failures
re-publish the task with retryCount++ until MAX_ATTEMPTS (3); the
history row carries RETRYING in between. We deliberately don't use
basicNack(requeue=true) — RabbitMQ would put the message back at the
head of the queue and the worker would tight-loop on it.
- No DLQ wired; terminal FAILED rows are the operator audit trail.
Add tests for both the sender (routing-key shape, history-id required)
and the worker (success/failed/retrying/terminal/skipped paths). Tests
now total 138 in dc3-common-data.
Extends the existing point/device metadata refresh flow to drivers:
- MetadataEvent gains an optional targetServices set so manager-side
publishers can scope a metadata event to specific driver services
instead of broadcasting to every listener.
- New DRIVER value on MetadataTypeEnum, plus a new GetById RPC on
driver_driver.proto so drivers can re-fetch their registered metadata
without going through the full registration handshake again.
- DriverClient.refreshMetadata(driverId) calls the new RPC and reapplies
the returned device ids and attribute maps via a shared applyMetadata
helper.
- MetadataReceiver now handles MetadataTypeEnum.DRIVER:
DELETE -> clear DriverMetadata, DeviceMetadata, and PointMetadata
caches and flip status to OFFLINE
ADD/UPDATE -> driverClient.refreshMetadata(id)
- Manager-side MetadataEventListener forwards events to RabbitMQ keyed by
the target services in the event, falling back to broadcast when none
are specified.
- DeviceServiceImpl, DriverAttributeServiceImpl, PointAttributeServiceImpl,
and PointServiceImpl populate targetServices using the relevant driver
service names so changes only wake the drivers that own the affected
entity.
- DriverDriverServer.getById exposes the new RPC; new DriverDriverServerTest
covers it.
- MetadataReceiverTest gains driverUpdate / driverDelete cases;
MetadataEventListenerTest covers the per-service routing.
Unify device/driver event tables into a single entity_alarm table with
alarm_target_type_flag (POINT/DEVICE/DRIVER), alarm_type_flag, and
alarm_source_flag enums. Split RabbitMQ exchanges into state exchange
for heartbeats and alarm exchange for persistence/rule triggers. Add
point alarm source throughout the frontend.
pnoker