273 Commits
Author SHA1 Message Date
pnoker 66aab58f20 build: standardize repository tooling and quality gates 2026-09-02 20:51:10 +08:00
pnoker eae86b58fc refactor(storage)!: unify platform persistence on reactive postgresql
- migrate auth, manager, data, and agentic persistence to R2DBC
- hard cut pagination, facade, gateway, web, and CLI contracts
- remove legacy database and TSDB adapters and document validation gates

BREAKING CHANGE: platform persistence now requires PostgreSQL 18 with TimescaleDB and the new reactive API contracts.
2026-09-01 22:03:16 +08:00
pnoker 1fa2d6f0cf docs(comments): fill the real comment gaps across the repository
A comment-health audit driven by an AST scanner (regex scanners kept
false-positiving on annotation-separated javadoc; the AST pass with
constructor and @Override-implementation exemptions is what produced a
trustworthy worklist) ended with a much smaller true gap than the raw
numbers suggested: class-level javadoc is already at 100% repository-wide.

What actually gets filled here:

- AnalyticsModel: all 30 nested records of the S19 analytics facet get
  maintainer-voice javadoc (the @Schema descriptions serve the API side;
  these serve code readers).
- The whole entity/ext family (29 files, 47 nested classes): every
  Content/Ui/Validation/Security/AppliesTo/Template/Dedup/... schema
  class states what it actually holds — read from the fields, not
  templated.
- Adapter families: the @Bean methods of all six MQ adapters, all four
  TSDB adapters and the three DB dialect adapters get one-line
  maintainer docs (what binds to what, what is overridable).
- Nested support types across data/agentic/constant/public/facade:
  NotifyBindKey, CacheTuning, Credential, dashboard VOs' Item/BucketVO,
  ExpireListener (plus its null-doc placeholder javadoc replaced with a
  real contract), agentic chat/stream VOs' choice/delta/usage shapes,
  tool-context records, RequestHeader envelopes, FleetSummary,
  visualization Type/Scale codes.
- PointValueSampleConverter's boundary methods now state the port-side
  contract (quality default injection point, batch order preservation).

Comment drift fixed: the class javadoc of the can/mqtt/opc-da driver
skeletons claimed "see TODO markers in method bodies" — those markers
do not exist; the claims are corrected to plain work-in-progress notes.
The genuine TODOs (zigbee API verification, lwm2m lookup, ethernet-ip
CIP framing, CAN JNI) stay: they mark real unimplemented behavior.

Vendored-code boundary: the opc-da module carries a ported
org.openscada library (46 files, package org.openscada.*). Its trivial
accessors stay bare on purpose — filler comments on get/set pairs are
exactly the kind of noise this pass exists to remove — but the
non-obvious JIStruct wire-decoder methods (getStruct/fromStruct
FILETIME + three DO wrappers) now document the COM bridge shape.

Gates: full-repo compile green; data/model/constant/agentic/public
test suites green.
2026-08-24 21:23:39 +08:00
pnoker 51f0efd2d9 style(javadoc): document non-override public methods 2026-08-19 00:05:58 +08:00
pnoker aec6ebfc76 refactor(logging): standardize application logging 2026-08-18 08:14:52 +08:00
pnoker 956de3dd38 feat(driver)!: enforce lease-fenced durable telemetry
Use PostgreSQL leases and fencing for distributed ownership, require a durable SQLite outbox before RabbitMQ publication, and make Data Center ingestion transactional and idempotent.

BREAKING CHANGE: drivers require mandatory durable outbox configuration and use lease-aware ownership and telemetry contracts.
2026-08-18 00:50:57 +08:00
hongyuan.zhang 6b37cb5ccd docs(readme): refresh module documentation 2026-08-11 22:58:30 +08:00
pnoker 50c6de9438 docs: rewrap project guides and README files
Prose rewrapping and table reflow from a project-wide markdown pass;
no content changes. Also reverts two formatter mistakes: restores
MD5(password) in dc3-web/AGENTS.md and drops spurious spaces after
CJK punctuation in README.zh.md / README.ja.md.

Covers root guides (AGENTS/CONTRIBUTING/SECURITY/README.*), module
READMEs across dc3-api/center/common/driver/e2e/gateway, the nginx
guide, dc3/doc/{README,TITLE}.md, and dc3-web docs.
2026-08-09 18:44:55 +08:00
pnoker 7bf615b3b1 feat(driver): configure driver buffer and mount container volume
28 driver application.yml add dc3.driver.buffer; Dockerfile each stage adds dc3/data dir and VOLUME; docker-compose main/dev add driver_data named volume
2026-07-26 00:13:31 +08:00
pnokerandClaude fe28d41803 refactor(config): drop redundant YAML blocks covered by shared profile bases
Remove configuration duplicated verbatim by the auto-activated shared profile bases: the dc3-center-manager gRPC channel in several drivers, the management.* block in center/gateway services, the gateway swagger-ui.path, duplicate logging sub-packages, and dead dc3.driver.mqtt config in dc3-center-data (no dc3-common-mqtt dependency). spring.env/spring.group are kept (consumed by RabbitmqEnvironmentConfig/WebFluxSecurityConfig/HmacAuthConfig). Effective configuration unchanged; validated via YAML parse + Maven process-resources.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-16 19:45:16 +08:00
pnoker d18bf4d03d style: fix table alignment and whitespace in README files 2026-06-26 13:17:38 +08:00
pnoker 7320b99183 docs(driver): fix attribute classification and flag WIP mqtt driver
Move misclassified point attributes out of the driver-attribute tables
(modbus-tcp Slave ID, virtual Tag, listening-virtual), drop plcs7's
fabricated Data Length and add the required PLC Type, document
command/event attributes, and flag the mqtt driver as a WIP skeleton.
2026-06-22 16:49:29 +08:00
Vickey 1b00e74346 test: align tests with enum rename and fix pre-existing failures
- Update test references to the renamed *TypeEnum/RuleStatusEnum.
- Add the new builder/PermissionProvider constructor args to the affected
  service-impl test setups.
- Fix pre-existing failures unrelated to this work: String-vs-Long assertions
  (DriverStatusServiceImplTest), save->saveBatch (EntityStateExpiryScannerTest),
  List.of(null) NPE (AlarmRulePipelineServiceImplTest), and the cache/batch
  refactor expectations (PointValueServiceImplTest).
2026-06-09 15:52:12 +08:00
pnoker 72515d9f31 refactor(model): standardize domain enums and object boundaries
Align the DO/BO/VO model and enums with the project domain-modeling rules:

- Rename 24 multi-valued *FlagEnum to *TypeEnum (closed classifications) and
  RuleStateFlagEnum to RuleStatusEnum (state machine); keep true 0/1 toggles as
  *FlagEnum (EnableFlagEnum/DefaultFlagEnum/ConfirmFlagEnum). Add the new
  ConfirmFlagEnum.
- Give every enum the consistent triplet + ofIndex/ofCode/ofName (fill the two
  Agentic enums) and switch underscore codes to hyphen-case.
- Replace bare Byte/Integer/String flag fields on VOs with their domain enums
  (EventHistory, AlertItem, ...); centralize index/code<->enum conversion in the
  MapStruct builders instead of services; drop ad-hoc (byte) enum casts.
- Stop leaking DO out of the data history services/controllers: the command /
  point-command / event history services now return VO, converting inside the
  service via the builder.
- Add @Schema (description + example/required, verified enum constants) across
  VO/BO/Query DTOs and the entity/ext nested beans; @Schema the R<T> wrapper;
  exclude credential fields from @ToString.
2026-06-09 15:51:35 +08:00
pnoker be88efc379 feat(driver): add DriverValidator SPI for protocol-level sandbox validation
Introduce DriverValidator SPI with validate(), validateDevice(), validatePoint(),
and simulate() default methods. All 28 protocol drivers now implement per-driver
and per-point configuration validation, checking that required attributes are
present before attempting device I/O.

- Add ValidationReport DTO with structured AttributeIssue / IssueLevel reporting
- Add DriverValidator SPI interface with sensible defaults
- Integrate DriverValidator into DriverCustomService aggregate SPI
- Implement validation for all 28 drivers (Modbus TCP, OPC UA, BACnet, MQTT, etc.)
- Add validation contract tests for Modbus TCP driver
2026-06-07 01:11:07 +08:00
pnoker 5730f7c137 feat: update pom version 2026-06-06 13:00:39 +08:00
pnoker 80c45456b9 refactor(driver): replace log string concatenation with parameterized placeholders
Replace string concatenation in SLF4J log calls with {} placeholders
across 13 driver implementations. Parameterized logging avoids string
construction when the log level is disabled, reducing CPU overhead.

Affected: listening-virtual (6 files), virtual, modbus-tcp, modbus-rtu,
opc-da, opc-ua, mqtt, coap.
2026-06-05 23:24:54 +08:00
pnoker f5743a89fc security(auth): remove hardcoded default keys, secrets, and password fallbacks
Remove DEFAULT_KEY and DEFAULT_PASSWORD constants from AlgorithmConstant.
Require DC3_SECURITY_KEY env var at startup — fail with IllegalStateException
if not configured instead of silently falling back to a well-known string.

Remove hardcoded default HMAC secret from HmacAuthProperties. Log a warning
when request signing is disabled in WebFilterConfig.

Require DC3_SECURITY_DEFAULT_PASSWORD for password reset operations instead
of falling back to the hardcoded 'dc3dc3dc3'.

Replace all YAML password fallback values (dc3dc3dc3) with empty defaults so
misconfigured deployments fail at connection time with a clear error rather
than silently connecting with a well-known password.
2026-06-05 22:35:50 +08:00
pnoker 60cd16d8bd fix(driver): invalidate failing connectors and tighten driver null-safety
Drop and dispose protocol connectors when read/write fails so the next
call rebuilds a fresh client for Modbus RTU/TCP, OPC DA, OPC UA, Melsec,
and PLC S7. Replace silent zero defaults with UnSupportException for
unknown Modbus function codes and reject null/empty config in CoAP and
listening-virtual paths. Make Netty TCP/UDP handlers Spring-managed
beans, release inbound ByteBuf references, unregister channels on
inactive, await write flush with a timeout, and bound-check every
configured byte slice. Skip MQTT/CoAP point values that lack
device/point identity and isolate event reporting failures.
2026-05-26 22:01:26 +08:00
Vickey 29f1594072 feat(driver): add Mitsubishi Melsec MC and SL651 hydrological telemetry drivers
feat: new driver modules
- dc3-driver-melsec: Mitsubishi Melsec MC (1E/3E/4E frame) driver via iot-communication
- dc3-driver-sl651: SL651-2014 hydrological telemetry server-side driver via iot-communication
- Bump iot-communication from 1.5.4 to 1.5.6 in dc3-driver-plcs7
- Relocate ias-releases repository from root pom.xml to dc3-driver/pom.xml

fix: code review fixes across driver implementations
- PlcS7PointVariable: only include bit offset in address when bit access and offset > 0
- OpcDaDriverCustomServiceImpl: disconnect server on device delete/update
- OpcUaDriverCustomServiceImpl: disconnect client on device delete/update
- ModbusTcpDriverCustomServiceImpl: remove duplicate FLOAT case in write value switch

refactor: eliminate magic numbers, consolidate statics, and normalize log formatting
- NettyServerHandler: replace magic numbers with named constants
- OpcUaDriverCustomServiceImpl: replace magic timeout numbers with named constants
- ModbusFactory: consolidate static block into private static final field
- VirtualDriverCustomServiceImpl: replace new Random() with ThreadLocalRandom
- KeyLoader: reorder fields so statics precede instance fields
- Normalize log format from string concat to {} placeholders across multiple drivers
- MqttDriverCustomServiceImpl: downgrade QoS fallback log warn→debug
- Remove unused imports
2026-05-26 10:00:00 +08:00
Vickey 918b3d5e8e feat: update pom version to 2026.5.22 2026-05-26 00:30:11 +08:00
Vickey dedb03fe74 feat: add command/event attribute configuration system
Introduce CommandAttribute/EventAttribute with config variants:
- REST controllers, MyBatis mappers, services, and DAL managers
- gRPC client/server builders and proto message definitions
- DTOs, BOs, DOs, VOs for type-safe data transfer layer

fix: add explicit ::text casts in json_build_object parameters

PostgreSQL cannot infer parameter types inside json_build_object()
calls. Add ::text casts to #{stateExtType} and #{stateDescription}
to resolve "could not determine data type of parameter $10" errors.

refactor: restructure driver gRPC client and metadata layer

- Rename GrpcStubConfig to DriverClientStubConfig with dedicated DTOs
- Extract attribute builders from inline gRPC conversion logic
- Consolidate driver metadata, register service, and command handling
- Update device state heartbeat and command result event handling

docs: add multilingual architecture diagram and sync READMEs

- Translate architecture SVG into zh, ja, vi variants
- Update README architecture image references per language
- Add missing service-level shortcut sections to ja/vi READMEs
2026-05-25 00:00:00 +08:00
Vickey 0defdb5b61 refactor: extract driver protocol strings to config and constants
Replace hardcoded PROTOCOL constants in 8 driver CustomServiceImpl classes
with @Value("${dc3.driver.code}") for runtime configurability. Extract
transport-layer protocol strings (netty/tcp/udp) in 5 Netty handler/server
classes to PROTOCOL constants.
2026-05-24 03:53:18 +08:00
Vickey 47760888bd feat: unify DeviceStatusEnum and DriverStatusEnum into EntityStatusEnum
- Replace two duplicate enums with single EntityStatusEnum (same index/code)
- Rename DB columns: state_flag→entity_state_flag, state_ext→entity_state_ext
- Add stateDescription field to DeviceStateDTO/DriverStateDTO for diagnostics
- Add description field to DeviceHealthState/DriverHealthState
- UPSERT SQL now writes stateDescription into entity_state_ext on heartbeat
- Remove EntityStateStatus interface (no longer needed with single enum)
- Update MQ layer, heartbeat services, status queries, gRPC, and all tests
2026-05-22 22:35:02 +08:00
pnoker 436b3b01bb feat: refine device and driver health state handling 2026-05-22 18:29:24 +08:00
pnoker 0471bd904e feat: add device health timeout state handling 2026-05-22 16:44:58 +08:00
pnoker 91eef8cbcf refactor(drivers,tests): switch to constructor injection via @RequiredArgsConstructor
Last batch of the cross-module refactor away from @Resource field
injection.

Production code:
- dc3-driver/* (8 driver modules): listening-virtual (4 files: Netty
  handlers + driver custom impl), virtual, mqtt (2), coap (2),
  modbus-tcp, plcs7, opc-da, opc-ua. Fields become private final;
  jakarta.annotation.Resource imports are removed.
- NettyTcpServer/NettyUdpServer pass null to NettyTcpServerHandler /
  NettyUdpServerHandler when adding the manually-created handler to the
  Netty pipeline. Those handler instances are pure stubs that delegate
  to the Spring-managed instance via the existing static self-reference,
  so a null collaborator field on the stub is functionally equivalent to
  the prior no-op default-constructor instance.

Test adaptations:
- 26 tests across dc3-common-mqtt, dc3-common-data, dc3-common-auth,
  dc3-common-manager, dc3-common-facade-{local-manager,local-auth,
  local-data}, and dc3-driver/* swap the no-args + reflection setup for
  a direct constructor call that passes the Lombok-generated parameter
  list. Reflection setters and the `injectField` helper are removed
  where they only existed for refactor-affected fields; reflection
  helpers that touch unrelated state (static factories, private cache
  maps used as test seams) are kept.
- 3 dc3-common-driver tests (DeviceCommandReceiverTest,
  MetadataReceiverTest, DriverRegisterServiceImplTest) get the same
  treatment. MetadataReceiverTest also gains a DriverClient mock to
  match the now-five-arg constructor.
- VirtualDriverCustomServiceImplTest and SessionServiceImplTest drop
  their reflection-only @BeforeEach blocks: with final fields, Mockito
  @InjectMocks now uses constructor injection automatically and matches
  collaborators by type.
- PointServiceImplTest gains a missing DriverService @Mock so that
  Mockito constructor injection can populate every parameter.
2026-05-21 14:57:13 +08:00
pnoker 7f5da1a0f8 chore: bump version to 2026.5.18 across all modules 2026-05-18 00:24:22 +08:00
pnoker 994ccba991 docs: update READMEs and project documentation
Refresh module READMEs, AGENTS.md, CONTRIBUTING.md, and dc3/doc/
guides to reflect the unified Dockerfile, consolidated compose stacks,
and updated build instructions.
2026-05-17 19:03:21 +08:00
pnoker 105c63603e build: consolidate per-service Dockerfiles into unified multi-target build
Replace 14 individual Dockerfiles with a single unified Dockerfile at
the repo root using multi-stage BuildKit targets. Maven build runs
inside the builder stage so no host JDK is required.
2026-05-17 19:02:03 +08:00
pnoker 996518186a build: bump version to 2026.5.17
Update parent version and dc3.version from 2026.5.5 to 2026.5.17
across all Maven modules.
2026-05-17 19:01:44 +08:00
pnoker 7c853ac8bc chore: correct @since tag to project inception date
Update @since from 2022.1.0 to 2016.10.1 and @version from 2026.5.5
to 2026.5.17 across all Java source files to reflect the actual
project start date.
2026-05-17 19:01:28 +08:00
pnoker 019daf62ce feat: do someting test and code review 2026-05-16 00:56:04 +08:00
pnoker cb457c37cf refactor(driver): rename RValue/WValue and add CalculatedPointValue
Driver SPI used the cryptic RValue/WValue names for read/write payload
envelopes. Rename them to ReadPointValue/WritePointValue so the public
contract reads naturally, add a sibling CalculatedPointValue for values
that need their derived/scaled form to flow alongside the raw sample,
and pull the value-type conversion utility into a dedicated
support/TypedValueConverter helper instead of leaving it inline.

DriverCustomService.read/write signatures and the seven driver
implementations (modbus-tcp, mqtt, opc-da, opc-ua, plcs7, virtual,
listening-virtual) move to the new types in lockstep so every consumer
of the SDK keeps compiling. Tests are renamed and resequenced to match,
and the driver authoring guide reflects the new identifiers.
2026-05-15 16:10:18 +08:00
pnoker 74e16bfb30 style: format dc3-common, dc3-driver and dc3-e2e test code 2026-05-15 00:28:35 +08:00
Vickey 22bd526819 test(driver): cover virtual and listening-virtual driver custom services
VirtualDriverCustomServiceImplTest (9 tests):
  - initial is no-op and does not touch collaborators
  - schedule sends ONLINE for every attached device with 25s TTL
  - schedule is silent when no devices are registered
  - device and point metadata events are observed without side effects
  - read returns constant string for STRING points
  - read returns parseable boolean for BOOLEAN points
  - read returns numeric value in [0, 100) for other point types
  - write returns false unconditionally

ListeningVirtualDriverCustomServiceImplTest (7 tests):
  - initial dispatches both TCP and UDP listeners through the thread pool
  - schedule sends ONLINE for every attached device with 25s TTL
  - device and point metadata events are observed without side effects
  - read returns null because the listener is passive
  - write flushes value through the channel when device channel is registered
  - write is a no-op (returns true) when no channel is mapped for the device

16 GREEN.
2026-05-14 10:59:19 +08:00
pnoker 252c55b83d refactor(driver): rename DriverCustomServiceImpl to protocol-specific class names
Rename the generic DriverCustomServiceImpl in each driver module to a
protocol-specific name (e.g., ModbusTcpDriverCustomServiceImpl) so
that static analysis tools can scan every driver module without
filtering duplicate fully-qualified class names. Update
DRIVER-AUTHORING.md to reflect the new convention.
2026-05-10 11:25:55 +08:00
Vickey 00692d125b refactor(driver): replace System.out with logger and improve structured logging
- Replace System.out.println/print with slf4j logger calls in S7/PLC
  driver (Nodave, S7BaseConnection, PDU, PLCinterface, S7Connection)
- Add structured log messages with protocol, deviceId, and operation
  context to modbus-tcp, mqtt, opc-ua, opc-da, and virtual drivers
- Add null checks and guard clauses in Netty message handlers
- Add keystore lifecycle logging in OPC-UA KeyLoader
2026-05-10 10:48:08 +08:00
pnoker f586f0cd1c docs: align markdown tables and wrap long lines 2026-05-09 23:50:58 +08:00
pnoker bfb9d19999 refactor(i18n): standardize user-facing text in english 2026-05-09 20:59:51 +08:00
pnoker 50c18422d3 refactor(config): standardize dc3 property prefixes 2026-05-09 20:45:22 +08:00
pnoker e47299a327 chore: remove format and format-check targets from Makefile 2026-05-07 23:55:32 +08:00
pnoker db456d61fe style: apply spring-javaformat 0.0.47 and fix checkstyle violations
- Apply Spring Java Format to all 1087 Java source files
- Fix star imports (java.lang.annotation, org.quartz, java.util.concurrent, etc.)
- Fix missing trailing newlines across codebase
- Fix redundant public modifiers on interface methods
- Fix multiple variable declarations
- Fix if-statements missing braces in JsonUtil, RegexUtil, RequestUtil
2026-05-07 16:53:16 +08:00
pnoker 95c40b2387 chore: bump version to 2026.5.5
Roll every pom, per-module README, Spring banner, and @version /
@since javadoc tag from 2026.4.30 to the new 2026.5.5 release line.
2026-05-05 00:56:09 +08:00
Vickey 200beeaa71 chore: update version to 2026.4.30 in multiple files 2026-04-30 16:06:30 +08:00
pnoker a38ec90601 feat: add gRPC client configuration and exception handling for ResponseStatusException 2026-04-29 19:02:37 +08:00
pnoker 1a52a12b23 fix: downgrade JRE version in Dockerfile from 25 to 21 2026-04-29 16:25:23 +08:00
pnoker c5a9feeeb9 feat: add .env file for Docker Compose configuration and update usage instructions 2026-04-29 10:27:57 +08:00
pnoker e87cff607e fix: correct JAVA_OPS syntax and update Maven command in Dockerfile 2026-04-19 14:07:19 +08:00
pnoker ea17c86387 chore: update version to 2026.4.19 across all modules and adjust branch settings in codeql.yml 2026-04-19 13:45:21 +08:00