122 Commits
Author SHA1 Message Date
pnoker eae86b58fc refactor(storage)!: unify platform persistence on reactive postgresql
- migrate auth, manager, data, and agentic persistence to R2DBC
- hard cut pagination, facade, gateway, web, and CLI contracts
- remove legacy database and TSDB adapters and document validation gates

BREAKING CHANGE: platform persistence now requires PostgreSQL 18 with TimescaleDB and the new reactive API contracts.
2026-09-01 22:03:16 +08:00
pnoker 769d1cb977 style: apply repository-wide formatting
- expand single-line Javadocs to multi-line blocks, rewrap Markdown prose, and reflow POM, YAML, and TypeScript formatting
- formatting only: no Java code lines changed, no build-file semantics changed
- verified: backend compile and check_documentation.py pass, pnpm check and pnpm lint:check pass, and the dc3 YAML files parse
2026-08-25 13:35:00 +08:00
pnoker 1fa2d6f0cf docs(comments): fill the real comment gaps across the repository
A comment-health audit driven by an AST scanner (regex scanners kept
false-positiving on annotation-separated javadoc; the AST pass with
constructor and @Override-implementation exemptions is what produced a
trustworthy worklist) ended with a much smaller true gap than the raw
numbers suggested: class-level javadoc is already at 100% repository-wide.

What actually gets filled here:

- AnalyticsModel: all 30 nested records of the S19 analytics facet get
  maintainer-voice javadoc (the @Schema descriptions serve the API side;
  these serve code readers).
- The whole entity/ext family (29 files, 47 nested classes): every
  Content/Ui/Validation/Security/AppliesTo/Template/Dedup/... schema
  class states what it actually holds — read from the fields, not
  templated.
- Adapter families: the @Bean methods of all six MQ adapters, all four
  TSDB adapters and the three DB dialect adapters get one-line
  maintainer docs (what binds to what, what is overridable).
- Nested support types across data/agentic/constant/public/facade:
  NotifyBindKey, CacheTuning, Credential, dashboard VOs' Item/BucketVO,
  ExpireListener (plus its null-doc placeholder javadoc replaced with a
  real contract), agentic chat/stream VOs' choice/delta/usage shapes,
  tool-context records, RequestHeader envelopes, FleetSummary,
  visualization Type/Scale codes.
- PointValueSampleConverter's boundary methods now state the port-side
  contract (quality default injection point, batch order preservation).

Comment drift fixed: the class javadoc of the can/mqtt/opc-da driver
skeletons claimed "see TODO markers in method bodies" — those markers
do not exist; the claims are corrected to plain work-in-progress notes.
The genuine TODOs (zigbee API verification, lwm2m lookup, ethernet-ip
CIP framing, CAN JNI) stay: they mark real unimplemented behavior.

Vendored-code boundary: the opc-da module carries a ported
org.openscada library (46 files, package org.openscada.*). Its trivial
accessors stay bare on purpose — filler comments on get/set pairs are
exactly the kind of noise this pass exists to remove — but the
non-obvious JIStruct wire-decoder methods (getStruct/fromStruct
FILETIME + three DO wrappers) now document the COM bridge shape.

Gates: full-repo compile green; data/model/constant/agentic/public
test suites green.
2026-08-24 21:23:39 +08:00
pnoker edadfa0a06 feat(tsdb): rewire the data center onto the tsdb port and retire the repository module
Phase 1b of docs/design/tsdb-abstraction.md: every point-value read and
write now flows through the dc3-tsdb port, and dc3-common-repository is
deleted outright (no compat aliases).

Write path — new PointValueIngestService orchestration in dc3-common-data:
the stale-owner lease guard leaves the history INSERT's cross-schema join
and resolves the active owner per distinct device through the existing
DeviceFacade.getActiveOwner chain; a caffeine-backed ingest idempotency
window (dc3.data.ingest.*, marked only after both writes commit) absorbs
MQ redeliveries now that the message_id unique index is retired; the
batch lands via TsdbStore.append (natural series+time upsert, INGEST_ORDER
kept for the fenced dc3_point_latest upsert).

Read path — history via last(), page() resolves name/enable filters to
series keys through relational metadata (tenant-wide when unrestricted,
cursor offset emulation capped at 10k) with count() for totals; the alarm
window backend becomes TsdbWindowDataSource over aggregate()/history().

Dashboards — all nine data-side hypertable statements move to the S13
analytics facet (count/bucketedCount/countByDimension/latencyHistogram/
lastSeenPerSeries) plus the dc3_point_latest projection for latestStream;
the manager topology statement crosses into the data center via a new
ListSeriesVolumes RPC backed by the new S13-5 seriesCounts primitive (a
point shared by several devices needs per-series counts, which
single-dimension grouping cannot reconstruct); both cross-schema joins
are gone.

Port/adapter — seriesCounts added with a TCK case (23 total); series
filters compile to row-value IN lists chunked at 500 pairs; bootstrap
adds the quality column idempotently and retires uk_point_value_event so
existing deployments converge on the new upsert; the adapter binds the
application-provided tsdbDataSource bean (the history dynamic-datasource
entry) instead of the routing primary.

Timestamps — BO<->Instant conversion is pinned to
TimeConstant.DEFAULT_ZONEID (the platform canonical zone the latest
projection type handler already writes with); the design note's "lock
UTC" is amended accordingly, together with the lease-guard and S13-5
deviations recorded in §6.2.

Seed/deploy — 05-iot-dc3-history.sql gains the quality column and swaps
the unique index; compose carries DC3_TSDB_TYPE; make deploy publishes
dc3-tsdb.

Also fixed along the way: root pom never managed dc3-mq-core (latent,
masked by full-reactor builds); PointValueMapper ran unrouted on the
master datasource (now @DS("history")); latestStream name enrichment
looked up a Long-keyed map with String keys.

Gates: tsdb TCK 23/23, dc3-common-data 262/262, e2e 26/26 including the
unmodified PostgresHypertableIT; full reactor green except dc3-mq-tck's
kafka/pulsar container-startup flake, reproduced identically twice and
unrelated to this change.
2026-08-20 21:58:01 +08:00
pnoker b61d6350a3 docs(repository): align documentation with executable sources 2026-08-19 00:06:08 +08:00
hongyuan.zhang 6b37cb5ccd docs(readme): refresh module documentation 2026-08-11 22:58:30 +08:00
pnoker 50c6de9438 docs: rewrap project guides and README files
Prose rewrapping and table reflow from a project-wide markdown pass;
no content changes. Also reverts two formatter mistakes: restores
MD5(password) in dc3-web/AGENTS.md and drops spurious spaces after
CJK punctuation in README.zh.md / README.ja.md.

Covers root guides (AGENTS/CONTRIBUTING/SECURITY/README.*), module
READMEs across dc3-api/center/common/driver/e2e/gateway, the nginx
guide, dc3/doc/{README,TITLE}.md, and dc3-web docs.
2026-08-09 18:44:55 +08:00
Henry ZhangandClaude Fable 5 7c643a8e80 fix(api): dc3-api contract overhaul (auth JWT, history, presence, decision enum) (#193)
* fix(api): correct inverted enable_flag comment and document time unit

- tenant.proto enable_flag comment said "1 enabled, 0 disabled" but the
  actual EnableFlagEnum is ENABLE=0, DISABLE=1 — callers reading the
  comment would invert the flag. Correct it.
- base.proto create_time/operate_time are epoch milliseconds
  (LocalDateTimeUtil/Instant.toEpochMilli); document the unit so callers
  don't guess seconds vs millis.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(api): carry timestamp in point-value history; explicit bool presence

#5 Point-value history: the whole chain (proto -> gRPC server ->
PointValueService -> repository -> facades -> controller -> agentic tool)
returned List<String> and dropped create_time even though the DB row
already carried it. Switch to PointValueBO / FacadePointValueBO end to
end so callers receive each value with its timestamp.

#6 Field presence: GrpcR.ok and McpIntrospectDTO.active are now optional
bool, so "unset" is distinguishable from false on the wire.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(auth): unify JWT signing key, drop salt/password from token contract

The JWT signing key was per-principal — securityKey + ":" + salt, where
salt was the credential salt also carried through the gateway and the
token RPC. That makes the JWT key weak and widens salt exposure to every
gateway hop. Sign with the single configured securityKey (HS256) instead
and remove salt from the JWT path entirely:

- KeyUtil.generateToken/parserToken drop the salt parameter; the HMAC
  key is now just securityKey (must be >= 32 bytes for HS256).
- TokenService.generateToken/checkValid drop the salt parameter.
- token.proto GrpcLoginQuery drops salt and password — CheckValid only
  ever needed tenant, name and token.
- gateway FilterService, TokenGrpcFacade, TokenController adapted; the
  request header no longer feeds salt into validation.

The login-handshake salt (POST /token/salt, used to salt the password
client-side before verifyPassword) is unchanged — only the JWT signing
salt is removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(api): type MCP authorize decision as proto enum

The authorize-tool-call decision was a free-form string, so callers
could only compare it against literals with no compile-time check.
Introduce GrpcMcpDecision (MCP_DECISION_UNSPECIFIED / AUTHORIZED /
CONFIRM_REQUIRED / REJECTED) and type GrpcMcpToolAuthorizeDTO.decision
with it. Enum names match the existing auth-side literals, so the
mapping is a direct valueOf / name().

- McpRuntimeServer builds the enum via toGrpcDecision (unknown -> UNSPECIFIED).
- McpRuntimeGrpcFacade maps enum -> BO string via name().
- gateway unchanged: it receives the BO (string decision) through the facade.

The other string MCP fields (risk_level, status, grant_type,
principal_type) stay strings for now — their value sets are wider and
partly lowercase (OAuth grant_type), so they are a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(facade): align TokenLocalFacade with 3-arg checkValid (CI compile fix)

TokenLocalFacade (local impl) was missed when TokenFacade.checkValid
dropped salt; CI compiles facade-local-auth independently and caught
it. Align it + its test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 16:36:35 +08:00
pnokerandClaude 77a20466bc refactor(api): reorganize proto files from center to common package
- Move proto files from api/center/ to api/common/ for auth, data, manager modules
- Update java_package and java_outer_classname in moved proto files
- Add GrpcDriverQuery message to shared query.proto in dc3-common-api
- Move GrpcDriverQuery proto from common.driver to common package
- Update Java imports referencing relocated proto messages
- Remove redundant pom.xml metadata (developers, licenses, scm, version)
  from api submodules — inherited from parent

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-09 12:43:48 +08:00
pnoker d18bf4d03d style: fix table alignment and whitespace in README files 2026-06-26 13:17:38 +08:00
pnoker abd826bf6e docs(api): correct gRPC RPC names, import paths, and missing services
Align the four api READMEs with the actual protos: Select*->Get*/List*
per the CRUD verb convention, fix api/common->api/center import paths
for auth/data, drop the fabricated manager SelectByServiceName RPC, and
document the previously omitted auth/data/manager services.
2026-06-22 16:49:07 +08:00
Vickey b3ee76f893 refactor(data): unify command history status/type/source to enum index
Align CommandHistory/PointCommandHistory with the EventHistory pattern:
DB stores the enum index (SMALLINT via @EnumValue), gRPC carries int32,
JSON keeps enum name. Removes the lone string-code outlier in the schema.

- DO status/type/source: String code -> enum types
- VO type/source: String -> enum (status already enum); JSON stays name
- Builder: drop ofCode @AfterMapping (DO and VO now share enum type)
- RabbitMQ DTOs (CommandCall/PointCommandResult): status/source -> enum
- driver sendResult signatures: String status -> PointCommandStatusEnum
- gRPC command_history.proto status/source: string -> int32; server uses
  getIndex()/ofIndex() to match EventHistoryServer
- tests updated to assert enum values
2026-06-09 18:05:40 +08:00
pnoker a7b19611a3 refactor: rename proto RPCs and facade methods to follow conventions
Proto RPC renames:
- command_history.proto: Call→CallCommand, List→ListByPage
- event_history.proto: Report→ReportEvent, List→ListByPage
- point_value.proto: LastValue→GetLastValue, HistoryValue→ListHistoryValues

Facade interface renames:
- StatusHealthFacade: selectDeviceStatusesByIds→listDeviceStatusesByIds,
  selectDeviceStatusesByProfileId→listDeviceStatusesByProfileId,
  selectDriverStatusesByIds→listDriverStatusesByIds

Update all gRPC server implementations, client stubs, and test files.
2026-06-06 00:03:20 +08:00
Vickey 918b3d5e8e feat: update pom version to 2026.5.22 2026-05-26 00:30:11 +08:00
Vickey dedb03fe74 feat: add command/event attribute configuration system
Introduce CommandAttribute/EventAttribute with config variants:
- REST controllers, MyBatis mappers, services, and DAL managers
- gRPC client/server builders and proto message definitions
- DTOs, BOs, DOs, VOs for type-safe data transfer layer

fix: add explicit ::text casts in json_build_object parameters

PostgreSQL cannot infer parameter types inside json_build_object()
calls. Add ::text casts to #{stateExtType} and #{stateDescription}
to resolve "could not determine data type of parameter $10" errors.

refactor: restructure driver gRPC client and metadata layer

- Rename GrpcStubConfig to DriverClientStubConfig with dedicated DTOs
- Extract attribute builders from inline gRPC conversion logic
- Consolidate driver metadata, register service, and command handling
- Update device state heartbeat and command result event handling

docs: add multilingual architecture diagram and sync READMEs

- Translate architecture SVG into zh, ja, vi variants
- Update README architecture image references per language
- Add missing service-level shortcut sections to ja/vi READMEs
2026-05-25 00:00:00 +08:00
pnoker 3b378e854b refactor: align data history models and constants 2026-05-24 01:07:02 +08:00
pnoker 150ba33244 feat: add custom command call API, event report API with RabbitMQ and gRPC
- Add dc3_command_record and dc3_event_record DTOs, DOs, mappers, managers
- Implement CommandRecordService: call flow with validation, RabbitMQ dispatch,
  result/dead receivers, full lifecycle (PENDING→SENT→SUCCESS/FAILED/TIMEOUT/DEAD)
- Implement EventReportService: report flow with validation and persistence
- Add REST controllers: /command_record (call/get/list), /event_report (report/get/list)
- Add RabbitMQ exchanges/queues/bindings for command dispatch, result, dead letter
- Add event exchange for driver→center event reporting
- Add gRPC CommandRecordApi and EventReportApi with proto definitions and servers
- Fix PointCommandValidator parameter type (String→PointExt)
2026-05-23 18:26:28 +08:00
pnoker 7f5da1a0f8 chore: bump version to 2026.5.18 across all modules 2026-05-18 00:24:22 +08:00
pnoker 994ccba991 docs: update READMEs and project documentation
Refresh module READMEs, AGENTS.md, CONTRIBUTING.md, and dc3/doc/
guides to reflect the unified Dockerfile, consolidated compose stacks,
and updated build instructions.
2026-05-17 19:03:21 +08:00
pnoker 996518186a build: bump version to 2026.5.17
Update parent version and dc3.version from 2026.5.5 to 2026.5.17
across all Maven modules.
2026-05-17 19:01:44 +08:00
pnoker 20bb57df17 refactor(agentic): split chat orchestration and structure tool results 2026-05-16 16:23:48 +08:00
pnoker ca0393ae7b feat(agentic): expand platform tool capabilities 2026-05-14 23:45:30 +08:00
pnoker e47299a327 chore: remove format and format-check targets from Makefile 2026-05-07 23:55:32 +08:00
pnoker 6992c20e5e feat(api): expand point_value.proto with HistoryValue, ReadCommand, and WriteCommand RPCs
Add three new RPC methods to PointValueApi: HistoryValue for querying
historical point values, ReadCommand and WriteCommand for device control.
Define corresponding request/response message types.
2026-05-07 00:09:08 +08:00
pnoker 95c40b2387 chore: bump version to 2026.5.5
Roll every pom, per-module README, Spring banner, and @version /
@since javadoc tag from 2026.4.30 to the new 2026.5.5 release line.
2026-05-05 00:56:09 +08:00
Vickey 200beeaa71 chore: update version to 2026.4.30 in multiple files 2026-04-30 16:06:30 +08:00
pnoker c5a9feeeb9 feat: add .env file for Docker Compose configuration and update usage instructions 2026-04-29 10:27:57 +08:00
pnoker ea17c86387 chore: update version to 2026.4.19 across all modules and adjust branch settings in codeql.yml 2026-04-19 13:45:21 +08:00
pnoker a1514061b4 fix: revert proto package to api.common.* to fix build breakage
Proto3 type resolution walks up the package hierarchy. After changing
package api.common.auth/data/manager to api.center.*, unqualified
references like GrpcR/GrpcBase (defined in api.common) can no longer
be resolved, causing PROTOC compile errors.

Revert package declarations to api.common.auth/data/manager while
keeping java_package as io.github.pnoker.api.center.* — java_package
is independent from proto package and controls Java output only.
2026-04-18 02:05:36 +08:00
pnoker 0833a88115 fix: complete dc3-api proto optimization
1. manager_point.proto: add missing SelectById RPC and GrpcRPointDTO
   response message to match driver_point.proto symmetry

2. Rename java_outer_classname to include module prefix to avoid
   confusion between driver and manager modules:
   - driver: DeviceProto→DriverDeviceProto, DriverProto→DriverRegistrationProto,
     EntityProto→DriverEntityProto, PointProto→DriverPointProto,
     QueryProto→DriverQueryProto, QueryPageProto→DriverQueryPageProto
   - manager: DeviceProto→ManagerDeviceProto, DriverProto→ManagerDriverProto,
     PointProto→ManagerPointProto, QueryProto→ManagerQueryProto,
     QueryPageProto→ManagerQueryPageProto

3. Reorganize proto files into subdirectories matching proto package:
   - auth:    src/main/protobuf/api/center/auth/
   - data:    src/main/protobuf/api/center/data/
   - driver:  src/main/protobuf/api/common/driver/
   - manager: src/main/protobuf/api/center/manager/
   Update all intra-module imports to use full absolute paths
2026-04-18 01:51:07 +08:00
pnoker 4ebde7cec0 fix: optimize dc3-api pom element order and proto files
- dc3-api/pom.xml: move <profiles> to after <build> per Maven POM
  standard element ordering; move <dependencyManagement> before
  <dependencies>
- driver_device.proto: rename driverConfigs/pointConfigs to
  driver_configs/point_configs (proto3 snake_case convention)
- auth/data/manager protos: align package declaration from
  api.common.xxx to api.center.xxx to match java_package option
- driver_entity.proto, driver_query.proto, driver_query_page.proto,
  manager_query_page.proto: remove trailing blank lines
2026-04-18 01:41:47 +08:00
pnoker d102e18b00 chore: fix inceptionYear and scm connection in all pom.xml
- Fix <inceptionYear> from '2016-2022' to '2016' in 20 pom.xml files
- Fix <scm> connection/developerConnection to use scm:git: prefix in 22 pom.xml files
2026-04-18 01:31:07 +08:00
pnoker e57793ec5f chore: bump version to 2026.4.18.3 and optimize pom.xml dependencies
- Bump all module versions from 2025.11.1 to 2026.4.18.3
- Remove orphaned dc3-common-influx dependency and property from root pom.xml
- Standardize dependency comment style in dc3-common/pom.xml
- Merge gRPC dependency groups into single section in dc3-common-api
- Restructure Spring Integration MQTT and Eclipse Paho groups in dc3-common-mqtt
- Fix incorrect comment 'DC3 Common Exception' to 'DC3 Common Related' in dc3-common-postgres
- Add missing blank line before </dependencies> in dc3-common-driver
- Remove stray blank line in root pom.xml dependencyManagement
2026-04-18 01:05:30 +08:00
Vickey 025a7b348f chore: update copyright headers and minor proto field comments across modules 2026-01-07 15:19:43 +08:00
Vickey 8a22940eb9 Merge remote-tracking branch 'origin/develop' into develop
# Conflicts:
#	dc3-common/dc3-common-constant/src/main/java/io/github/pnoker/common/constant/common/PrefixConstant.java
#	dc3-common/dc3-common-constant/src/main/java/io/github/pnoker/common/constant/common/QueryWrapperConstant.java
#	dc3-common/dc3-common-constant/src/main/java/io/github/pnoker/common/constant/common/RequestConstant.java
#	dc3-common/dc3-common-constant/src/main/java/io/github/pnoker/common/constant/common/SuffixConstant.java
#	dc3-common/dc3-common-constant/src/main/java/io/github/pnoker/common/constant/common/SymbolConstant.java
#	dc3-common/dc3-common-constant/src/main/java/io/github/pnoker/common/constant/common/TimeConstant.java
#	dc3-common/dc3-common-constant/src/main/java/io/github/pnoker/common/enums/DeviceEventTypeEnum.java
#	dc3-common/dc3-common-constant/src/main/java/io/github/pnoker/common/enums/DeviceStatusEnum.java
#	dc3-common/dc3-common-constant/src/main/java/io/github/pnoker/common/enums/EnableFlagEnum.java
2026-01-07 15:19:02 +08:00
zzi666 578739f338 refactor(proto): simplify proto comments, remove redundant README files, and unify format 2025-12-31 14:15:29 +08:00
zzi666 7c6b9e3ea3 Merge branch 'develop' of github.com:pnoker/iot-dc3 into develop 2025-12-31 10:07:38 +08:00
Vickey 557152226c docs(api): add README files for auth, data, and driver API modules 2025-12-28 19:23:20 +08:00
Vickey 0e419bab05 feat: update pom 2025-11-11 13:22:24 +08:00
zzi666 eafe3e14a4 refactor: standardize and translate comments across Protobuf files and environment script for clarity and consistency 2025-11-06 16:37:35 +08:00
zzi666 73fe41091e refactor: standardize descriptions, comments, and documentation across POM files and Protobuf definitions 2025-10-24 20:07:28 +08:00
zzi666 137a0f89e0 refactor: standardize protobuf annotations and document translations 2025-10-24 15:45:50 +08:00
Vickey a9e8d05631 feat: update version 2025-10-05 17:16:37 +08:00
Vickey 7302107c56 chore: bump all module versions to 2025.9.2 2025-09-11 17:02:24 +08:00
Vickey 2051d42847 refactor(common): remove hutool dependency and replace usages with native utilities 2025-09-11 14:57:02 +08:00
Vickey 8cc7365293 build: bump all module versions to 2025.6.6 and update compose image tags 2025-08-26 22:38:53 +08:00
Vickey 022bab4fbd chore: switch license declaration from Apache-2.0 to AGPL-3.0 across all poms 2025-08-26 22:16:35 +08:00
Vickey a39d2fb533 chore: migrate license headers from Apache 2.0 to AGPL-3.0 across all source files 2025-08-26 22:01:24 +08:00
Vickey a04ff5e612 refactor(data): remove origin time field from point value and implement postgres pagination queries 2025-08-23 18:11:42 +08:00
Vickey 4d147af22e feat(common-driver): extend protobuf schemas and refactor driver service implementations 2025-08-09 13:23:18 +08:00