Commit Graph
78 Commits
Author SHA1 Message Date
rongtongandRongtongJin 89e021e871 [ISSUE #9898] Remove AbstractBrokerRunnable and replace with Runnable
Change-Id: I94104151c452d09cbe195a3e1126a473b662a337

Co-authored-by: RongtongJin <user@example.com>
2025-12-05 14:03:54 +08:00
yx9o d7e27d6d69 [ISSUE #9396] Use fastjson2 in all modules (#9397)
* Use fastjson2 in all modules

* Update test

* Update test

* Update test

* Add serialization compatibility test tool class

* Update RemotingSerializableCompatTest.java

* Update RemotingSerializableCompatTest.java

* Update RemotingSerializableCompatTest.java

* Update BitSet problem

* Update

* Update

* Update test

* Update test

* Update BUILD.bazel

* Update BUILD.bazel

* Update test

* Update BitSet problem

* Add test

* Add compat test

* merge develop

* Update test

* merge develop
2025-12-04 19:09:08 +08:00
rongtong 7d7502126c [ISSUE #9719] Fix Static Variable Conflicts in Multi-Broker Metrics Management (#9720)
* refactor: convert RemotingMetricsManager from static to instance-based

- Remove all static global instance related code
- Convert all static methods to instance methods
- Remove static wrapper methods for backward compatibility
- Add constructor and getter/setter methods for instance variables
- This ensures each broker instance manages its own metrics in multi-broker scenarios

* refactor: convert PopMetricsManager from static to instance-based

- Remove all static global instance related code
- Convert all static methods to instance methods
- Remove static wrapper methods for backward compatibility
- Add constructor and getter/setter methods for instance variables
- This ensures each broker instance manages its own Pop metrics in multi-broker scenarios

* refactor: implement instance-based metrics management for multi-broker scenarios

- Add RemotingMetricsManager instance variable to NettyRemotingAbstract
- Add setter/getter methods for RemotingMetricsManager in NettyRemotingAbstract
- Update writeResponse method to use instance-based metrics recording
- Add writeResponse method to RemotingServer interface
- Update BrokerController to set RemotingMetricsManager on NettyRemotingServer instances
- Update InnerBrokerController to set RemotingMetricsManager on SubRemotingServer instances
- Update all processor classes to use instance-based writeResponse calls
- Update RocksDBStoreMetricsManager integration in DefaultStoreMetricsManager
- Remove unused imports and clean up code

This ensures each broker instance manages its own metrics independently,
solving the static variable conflict issue in multi-broker processes.

* refactor: fix test failures caused by metrics manager refactoring

- Fix ChangeInvisibleTimeProcessorTest by using @Spy instead of @Mock for BrokerController
- Inject BrokerMetricsManager into BrokerController using reflection
- Remove unnecessary mock stubbings
- Update all processors to use new static writeResponse method
- Fix BrokerMetricsManager initialization issues in AckMessageProcessorTest and PopReviveServiceTest

These changes ensure all related tests pass after refactoring metrics managers from static variables to instance variables.

* fix: resolve checkstyle errors and ensure compilation success

- Remove duplicate import in PopMessageProcessor.java
- Clean up unused imports in ChangeInvisibleTimeProcessorTest.java
- Clean up unused imports in AckMessageProcessorTest.java
- Ensure all modules compile successfully without checkstyle violations

These changes complete the metrics manager refactoring from static variables to instance variables, resolving statistical information conflicts in multi-broker processes.

* Fix PopReviveServiceTest by adding proper PopMetricsManager mock

- Add PopMetricsManager mock to PopReviveServiceTest
- Fix NullPointerException in reviveRetry method
- Ensure rePutCK method is called correctly when retry fails
- Test now passes successfully with proper metrics manager setup

* Remove debug code from PopReviveService

- Remove debug logging statements from reviveMsgFromCk method
- Clean up test output for better readability

* fix

* fix
2025-09-22 09:53:52 +08:00
rongtong 73e8fdbdb8 [ISSUE #9699] Optimize shutdown process and resource management (#9700)
* Optimize shutdown process and resource management

- Improve BrokerController shutdown flow for graceful shutdown
- Optimize BrokerStartup startup and shutdown logic
- Enhance ClientHousekeepingService resource cleanup
- Improve shutdown handling for various processors
- Optimize resource management in storage layer components
- Enhance lifecycle management for statistics manager
- Improve shutdown flow for timer components

* Fix this.popMessageProcessor.getPopLongPollingService() not shutdown

* Fix test shutdown state transition issue

- Add proper null checks and exception handling in test cleanup
- Prevent IllegalStateException during test teardown
- Ensure graceful test cleanup without state conflicts

* Fix DefaultMessageStoreCleanFilesTest can not pass

* Fix CombineConsumeQueueStoreTest can not pass

* Polish the code

* Polish the code

* Ignore flaky test first
2025-09-17 19:21:49 +08:00
rongtong bb45b8f5f7 [ISSUE #9634] Improve BrokerContainer extensibility and module structure (#9635)
* Initially optimize the broker container structure

* refactor: Improve BrokerContainer extensibility and logging

This commit enhances the BrokerContainer module to improve code structure
and logging capabilities:

Key improvements:
- Polish the code structure to make BrokerContainer more extensible
- Improve container logging configuration and management
- Enhance BrokerBootHook for better hook management
- Update BrokerContainer and BrokerContainerProcessor for improved functionality
- Remove unused BrokerLogbackConfigurator to reduce complexity
- Update BrokerStartup and BrokerController for better container integration

Modified files:
- broker/src/main/java/org/apache/rocketmq/broker/BrokerController.java
- broker/src/main/java/org/apache/rocketmq/broker/BrokerStartup.java
- container/src/main/java/org/apache/rocketmq/container/BrokerBootHook.java
- container/src/main/java/org/apache/rocketmq/container/BrokerContainer.java
- container/src/main/java/org/apache/rocketmq/container/BrokerContainerProcessor.java
- container/src/main/java/org/apache/rocketmq/container/BrokerContainerStartup.java
- container/src/main/java/org/apache/rocketmq/container/logback/BrokerLogbackConfigurator.java (removed)

This refactoring improves the overall maintainability and extensibility
of the container module while maintaining backward compatibility.

* test: Add unit tests for BrokerContainer extensibility improvements

This commit adds comprehensive unit tests for the BrokerContainer
extensibility improvements introduced in this branch:

Key test coverage:
- BrokerBootHook system extensibility and proper hook execution
- Container configuration accessibility and management
- Container initialization and lifecycle management
- BrokerContainerProcessor integration
- Startup and shutdown sequence robustness
- Extension points and customization capabilities

These tests ensure that the improved BrokerContainer architecture
maintains backward compatibility while providing enhanced extensibility
for future development and customization.

* Fix testBrokerGracefulShutdown can not pass

* Fix BrokerContainerExtensibilityTest can not pass

* fix(container): add missing auth module dependency

- Add //auth dependency to container target in BUILD.bazel
- Add //auth dependency to tests target in BUILD.bazel
- Fixes compilation errors for AuthConfig class usage
- Resolves Bazel build failures in container module

Fixes: symbol not found org.apache.rocketmq.auth.config.AuthConfig
2025-09-09 16:12:43 +08:00
cvictoryandcvictory 1c23a8094a [ISSUE #9663] Adopt Maven's revision mechanism (requires Maven >= 3.5.0) (#9665)
Co-authored-by: cvictory <shengli.caosl@alibaba-inc.com>
2025-09-04 09:51:02 +08:00
Xu Chengxin be67d61223 [ISSUE #9447] Avoiding possible resource leaks in InputStream (#9459) 2025-06-30 14:31:04 +08:00
WJ66880 f4e2a4a0e3 [ISSUE #9335] Use MixAll.ROCKETMQ_HOME_DIR for home directory (#9474) 2025-06-18 10:28:35 +08:00
Crazylychee 4f9d292ee5 [ISSUE #9439] Add escape for win in the method returning broker configuration (#9440) 2025-06-05 17:09:13 +08:00
lizhimins aecb97ec44 [maven-release-plugin] prepare for next development iteration (#9389) 2025-05-07 16:34:04 +08:00
lizhimins 29d90a4ff9 [maven-release-plugin] prepare release rocketmq-all-5.3.3 (#9388) 2025-05-07 09:48:32 +08:00
lizhimins 47fe6b2ba6 [maven-release-plugin] prepare for next development iteration (#9232) 2025-03-08 17:20:49 +08:00
lizhimins 2baaf044ea [maven-release-plugin] prepare release rocketmq-all-5.3.2 (#9231) 2025-03-08 14:07:17 +08:00
Quan 14156b4f0a [ISSUE #9191] Provide the ability to replace the remoting layer implementation for Proxy and Broker (#9192)
* remoting replacement for proxy and broker

* add unit test

* fix code style
2025-02-24 12:27:30 +08:00
jiao jiananandjiaoja 93d7e80a97 [ISSUE #8950] Remove Redundant nullcheck of configPath (#8951)
Co-authored-by: jiaoja <jiaoja@asiainfo.com>
2024-12-02 10:03:16 +08:00
lizhimins daf3d1a666 [maven-release-plugin] prepare for next development iteration (#8750) 2024-09-26 09:33:52 +08:00
lizhimins b122c8109d [maven-release-plugin] prepare release rocketmq-all-5.3.1 (#8749) 2024-09-25 23:24:56 +08:00
lizhimins 5dc0c6a8b5 [maven-release-plugin] prepare for next development iteration (#8382) 2024-07-09 01:30:43 +08:00
lizhimins 7ee50ec32e [maven-release-plugin] prepare release rocketmq-all-5.3.0 (#8379) 2024-07-08 23:15:33 +08:00
yx9o b96d6b98e8 [ISSUE #8285] Add more test coverage for BrokerPreOnlineService (#8286) 2024-06-12 09:08:49 +08:00
dingshuangxi888 e1339ac8a0 [ISSUE #7560] [RIP-68] Support RocketMQ ACL 2.0 (#7725)
[ISSUE #7560] [RIP-68] Support RocketMQ ACL 2.0 (#7725)
2024-03-18 14:07:51 +08:00
rongtong f41160210a [maven-release-plugin] prepare for next development iteration (#7828) 2024-02-12 11:07:03 +08:00
rongtong 281a7b32f0 [maven-release-plugin] prepare release rocketmq-all-5.2.0 (#7827) 2024-02-11 17:53:50 +08:00
rongtong 430ee0a755 Add validation in broker container configure updating command. (#7587) 2023-11-28 16:11:14 +08:00
lk f90c55341c [ISSUE #7497] Extract the frequency of calling updateNamesrvAddr into a configuration (#7498)
merge
2023-10-24 16:13:36 +08:00
Zhouxiang Zhan 2043dd5034 [ISSUE #7493] Introduce a new event NettyEventType.ACTIVE (#7494)
* [ISSUE #7493] Introduce a new event NettyEventType.ACTIVE for ChannelEventListener

* introduce a new event NettyEventType.ACTIVE,

* implement channelActive interface for NettyRemotingClient#NettyConnectManageHandler

* add onChannelActive for ChannelEventListener interface.

* Move send heartbeat to onChannelActive
2023-10-24 10:29:43 +08:00
rongtong 82b2f8eefa AddBroker removes parsing configuration from body (#7472) 2023-10-18 13:51:47 +08:00
lk 73b3fde837 [maven-release-plugin] prepare for next development iteration (#7379) 2023-09-22 10:08:59 +08:00
lk 1a681bdf9b [maven-release-plugin] prepare release rocketmq-all-5.1.4 (#7377) 2023-09-21 19:58:29 +08:00
guyinyou c100d815d7 [ISSUE #7328] Convergent thread pool creation (#7329)
* Convergence thread pool creation to facilitate subsequent iteration management

* Convergence thread pool creation in ThreadPoolMonitor.java

* fix unit test

* Convergence ThreadPool constructor

* Convergence ScheduledThreadPool constructor

* remove unused import

* Convergence ScheduledThreadPool constructor

* remove unused import

---------
2023-09-11 10:13:56 +08:00
rongtong e369d7deac [maven-release-plugin] prepare for next development iteration (#6939) 2023-06-24 14:31:42 +08:00
rongtong f7425f52f9 [maven-release-plugin] prepare release rocketmq-all-5.1.3 (#6938) 2023-06-24 12:23:11 +08:00
Guocheng TangandRongtongJin 5d6ee6384f [ISSUE #6910] Extract the interval of fetchNameServerAddr
Co-authored-by: RongtongJin <jinrongtong16@mails.ucas.ac.cn>
2023-06-23 14:21:29 +08:00
rongtong ad4cc957fb [maven-release-plugin] prepare for next development iteration (#6895) 2023-06-12 19:12:25 +08:00
rongtong 749967baa8 [maven-release-plugin] prepare release rocketmq-all-5.1.2 (#6892) 2023-06-12 17:44:39 +08:00
rongtong 8aea26ef8f [maven-release-plugin] prepare for next development iteration 2023-05-15 19:48:34 +08:00
rongtong e244ebb99a [maven-release-plugin] prepare release rocketmq-all-5.1.1 (#6763) 2023-05-15 18:51:01 +08:00
zhouxiang 0f878b5157 [maven-release-plugin] prepare for next development iteration 2023-02-16 12:35:58 +08:00
zhouxiang b8e9334d2a [maven-release-plugin] prepare release rocketmq-all-5.1.0 2023-02-16 12:35:58 +08:00
zhouxiang 83d6778747 [ISSUE #6030] Update version to 5.1.0-SNAPSHOT 2023-02-16 12:35:58 +08:00
Aaron Ai ff8c4c856d [ISSUE #5890] Fix dledger logging (#5959)
* Fix dledger logging

* Add bridge into store module
2023-02-03 14:12:23 +08:00
Oliver 37f83124b3 [ISSUE #5900] Fix the problem of can not remove the broker successfully when a shutdown failure (#5901) 2023-01-18 10:23:46 +08:00
Oliver e662715e13 [ISSUE #5897] Polish exception info when broker has already been added to broker container (#5898) 2023-01-17 16:13:38 +08:00
Aaron Ai ffb4a3664e Apply MDC in broker container mode (#5587) 2022-11-24 19:32:07 +08:00
Aaron Ai 4469fb51dd Update group id of rocketmq logging (#5564) 2022-11-22 11:31:21 +08:00
Aaron Ai a68078fef8 [ISSUE #5484] Replace logging module with the shaded logback (#5540)
* Replace logging module with the shaded logback

* Add license header

* Fix bazel compile issue
2022-11-21 16:30:15 +08:00
yukon 506b563f55 Revert "[ISSUE #5484] Replace Logging Module with Shaded Logback (#5524)" (#5537)
This reverts commit 809ff6b3e0.
2022-11-17 20:04:17 +08:00
Aaron AiandLi Zhanhui 809ff6b3e0 [ISSUE #5484] Replace Logging Module with Shaded Logback (#5524)
* Remove ClientLogger

* WIP

* WIP

* Rename logger to log

* Make it compile

* WIP

* Fix bazel to use shaded slf4j

* Fix DeleteExpiredCommitLogSubCommandTest

* Fix pom.xml duplication

* Fix maven deps

* Add test logback configuration file

* Fix unit test output

* Fix logback configuration file

* All logging are made on top of slf4j

* Fix test log configuration file name

* Fix test log configuration file name for test module

* All logging are shaded slf4j targeted

* DLedger has an explicit dependency on slf4j

* Fix DLedger

* Fix DLeader issue

* Move logback configuration files to each module

Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
2022-11-17 00:38:28 +08:00
SSpiritsandLi Zhanhui ec5d3236f6 invert dependency between remoting and common module (#5503)
* invert dependency between remoting and common module

* fix bazel

* Fix Bazel warning of indirect dependency

* remove duplicate class ServiceThread

* fix conflict

* revert delete class ServiceThread because of introduced by dledger

Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
2022-11-11 13:34:31 +08:00
NowinkeyandZhanhui Li 53e0847528 [ISSUE #5204] Enable checkstyle for test code (all module) (#5297)
* delete the checkstyle plugin for child modules

* [ISSUE #5204] Enable checkstyle for test code (all module)

* Exclude generated files

Co-authored-by: Zhanhui Li <lizhanhui@apache.org>
2022-10-14 10:58:48 +08:00