processResponseCommand used get+remove (non-atomic) to retrieve ResponseFuture
from responseTable. scanResponseTable used iterator.remove(). If the response
arrived between scanResponseTable's remove and processResponseCommand's get,
the response would be logged as 'not matched any request' and silently dropped.
The callback would fire with timeout instead of success, even though the broker
had successfully processed the request.
Fix: Use ConcurrentHashMap.remove(key) in both methods. This is atomic:
either processResponseCommand gets the future (and executes success callback),
or scanResponseTable gets it (and executes timeout callback), but never both
and never neither.
Co-authored-by: wangjiahua.wjh <wangjiahua.wjh@alibaba-inc.com>
* chore: bump DLedger to 0.3.3.4 (maintenance line)
Update the DLedger dependency from 0.3.2 to 0.3.3.4, the latest
release on the 0.3.x maintenance line published on Maven Central.
The 0.3.3.4 release is API-compatible with 0.3.2 and only carries
dependency hygiene fixes (aligning fastjson2 and excluding fastjson1
from its rocketmq-remoting transitive path).
Also update the Bazel WORKSPACE artifact coordinate to keep the two
build systems in sync.
* chore: fully remove fastjson1 from the build
DLedger 0.3.3.4 already excludes com.alibaba:fastjson from its
rocketmq-remoting dependency and its bytecode only references
fastjson2, but rocketmq still declared fastjson1 itself, so it kept
leaking onto every module compile classpath through rocketmq-common.
- drop the com.alibaba:fastjson property and dependencyManagement
entry from the root pom
- drop the unused com.alibaba:fastjson dependency from rocketmq-common
(no main source imports com.alibaba.fastjson.*)
- drop the fastjson1 pin from WORKSPACE and the fastjson1 dep from the
remoting test target in remoting/BUILD.bazel
- migrate RemotingSerializableCompatTest to fastjson2: use the
fastjson2 JSONField annotation (which is what the protocol classes
actually carry) and round trip through RemotingSerializable instead
of com.alibaba.fastjson.JSON
fastjson1 wire-format coverage is retained by the frozen fastjson1
payload already asserted in testCompatibilityCheckWithBitSet.
* fix(bazel): exclude rocketmq-remoting from coursier resolution
dledger 0.3.3.4 depends on rocketmq-remoting:5.5.0, which Coursier
fetches from Maven Central. That artifact transitively brings
grpc-netty-shaded:1.53.0 with a strict [1.53.0] range on grpc-core,
conflicting with the WORKSPACE-declared grpc 1.47.0 artifacts.
Since rocketmq-remoting is built from source in this repo (//remoting),
exclude it from external resolution — matching what PR #10947 already
does for the DLedger 0.4.x line.
---------
Co-authored-by: 通融 <rongtong.jrt@alibaba-inc.com>
- Broker: add PopLiteLongPollingService to NotificationProcessor for lite consumer notification polling
- Broker: LiteEventDispatcher notifies both PopLiteMessageProcessor and NotificationProcessor polling services
- Broker: add hasEvents(clientId) API to LiteEventDispatcher for message availability check
- Proxy: unify lite/normal pop paths into single popMessage call, route by ProxyContext.isLiteConsumer()
- Proxy: register LITE_SIMPLE_CONSUMER in ClientActivity and GrpcClientSettingsManager lifecycle
- Proxy: pass liteTopic property only for lite consumers in filter and response writer
- Proxy: remove standalone popLiteMessage from MessagingProcessor/ConsumerProcessor
- Remoting: add isLiteConsumer and clientId fields to NotificationRequestHeader
- Add wildcard (*) subscription support for liteTopic
- Implement consume suspend mechanism with invalid scan count threshold
- Refactor subscriber query interface with SubscriberWrapper for flexible retrieval
- Add wildcard client cache with 30s TTL for performance optimization
- Update related components and enhance test coverage
Change-Id: I4ecaceec7daa2f4364d911437007df98dc49d542
* Use fastjson2 in all modules
* Update test
* Update test
* Update test
* Add serialization compatibility test tool class
* Update RemotingSerializableCompatTest.java
* Update RemotingSerializableCompatTest.java
* Update RemotingSerializableCompatTest.java
* Update BitSet problem
* Update
* Update
* Update test
* Update test
* Update BUILD.bazel
* Update BUILD.bazel
* Update test
* Update BitSet problem
* Add test
* Add compat test
* merge develop
* Update test
* merge develop
* refactor: convert RemotingMetricsManager from static to instance-based
- Remove all static global instance related code
- Convert all static methods to instance methods
- Remove static wrapper methods for backward compatibility
- Add constructor and getter/setter methods for instance variables
- This ensures each broker instance manages its own metrics in multi-broker scenarios
* refactor: convert PopMetricsManager from static to instance-based
- Remove all static global instance related code
- Convert all static methods to instance methods
- Remove static wrapper methods for backward compatibility
- Add constructor and getter/setter methods for instance variables
- This ensures each broker instance manages its own Pop metrics in multi-broker scenarios
* refactor: implement instance-based metrics management for multi-broker scenarios
- Add RemotingMetricsManager instance variable to NettyRemotingAbstract
- Add setter/getter methods for RemotingMetricsManager in NettyRemotingAbstract
- Update writeResponse method to use instance-based metrics recording
- Add writeResponse method to RemotingServer interface
- Update BrokerController to set RemotingMetricsManager on NettyRemotingServer instances
- Update InnerBrokerController to set RemotingMetricsManager on SubRemotingServer instances
- Update all processor classes to use instance-based writeResponse calls
- Update RocksDBStoreMetricsManager integration in DefaultStoreMetricsManager
- Remove unused imports and clean up code
This ensures each broker instance manages its own metrics independently,
solving the static variable conflict issue in multi-broker processes.
* refactor: fix test failures caused by metrics manager refactoring
- Fix ChangeInvisibleTimeProcessorTest by using @Spy instead of @Mock for BrokerController
- Inject BrokerMetricsManager into BrokerController using reflection
- Remove unnecessary mock stubbings
- Update all processors to use new static writeResponse method
- Fix BrokerMetricsManager initialization issues in AckMessageProcessorTest and PopReviveServiceTest
These changes ensure all related tests pass after refactoring metrics managers from static variables to instance variables.
* fix: resolve checkstyle errors and ensure compilation success
- Remove duplicate import in PopMessageProcessor.java
- Clean up unused imports in ChangeInvisibleTimeProcessorTest.java
- Clean up unused imports in AckMessageProcessorTest.java
- Ensure all modules compile successfully without checkstyle violations
These changes complete the metrics manager refactoring from static variables to instance variables, resolving statistical information conflicts in multi-broker processes.
* Fix PopReviveServiceTest by adding proper PopMetricsManager mock
- Add PopMetricsManager mock to PopReviveServiceTest
- Fix NullPointerException in reviveRetry method
- Ensure rePutCK method is called correctly when retry fails
- Test now passes successfully with proper metrics manager setup
* Remove debug code from PopReviveService
- Remove debug logging statements from reviveMsgFromCk method
- Clean up test output for better readability
* fix
* fix
* fix: prevent broker startup failure after power outage
- Add atomic file backup mechanism in persist() methods
- Delete corrupted config files during startup to avoid bak file pollution
- Add directory sync to ensure file operations visibility
Fixes: TimerMetrics#persist, TransactionMetrics#persist, ConfigManager#persist
* remove "Shutdown" implements
* empty commit
* add SuppressWarnings
* fix ut
* fix ut
* pass rocksdb ut when isMac()
* pass rocksdb ut when isMac()
* pass rocksdb ut when isMac()
* pass rocksdb ut when isMac()
* pass RocksdbGroupConfigTransferTest ut when isWindows()
* pass Rocksdb ut when isMac()
---------
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com>
* Implement paged retrieval of Topic and SubscriptionGroup information
- Implemented pagination logic to support data retrieval by sequence number and maximum count.
- Added data version checking to ensure the retrieved data is the latest.
- Optimized the result structure to include total count and current page data.
- Added unit tests
* BrokerOuterAPI
* add timeout & log
---------
Co-authored-by: xiaoming.lt <xiaoming.lt@alibaba-inc.com>
Jiahua Wang