7a08b4cb68
[RIP-82] Implement Timer message, transaction message, and index based on RocksDB ( #9787 )
...
* add isWakeCommitWhenPutMessage for AIO
* Implemented Timer message, Transaction Message, Index based on rocksdb
* add timeline roll interval can be config
* optimize the code
* optimize the code
* add sone logic for recover commitlog
* optimize the code
* update
* update
* optimize the code
* update
* update
* optimize the code
* optimize the code
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
Change-Id: I700798ef625187ac32c16c8ea5aae5f2debfdddf
* update
Change-Id: I1d2dfa24317d2ad7c0ca41d90cd2b3c1ba260b0b
* update
Change-Id: I974e30eb909722ea2a17a14e505759bf20f4f860
* update
Change-Id: Ica01fbd08a2d60e9a0f4ab54b41ae4768cc4fa5d
* update
Change-Id: I764ce6cea3159b025bf1e6e65b60e20514cecb4f
* update
Change-Id: I0cc735896e1266934293ec06cb4f07746a1894a1
* update
Change-Id: I6538b6a60631c62e162bf511ae20185a07b72508
* update
Change-Id: I03f83b9e8199f3595ca28471df4a2e94671fbaae
* update
Change-Id: I5b19b671fa0b40cd2aba63f1d0eb654b92775cb4
* update the code
Change-Id: I22788d112b7f214310719e2d3ebad839ef818e67
* update
Change-Id: I7da89fe635b58ac5026095a8e484fa46f68521e7
* update the code
Change-Id: I5eade6c31803b8cf05c1a4bb83c226d57f4980c7
* update
Change-Id: Id67f600fad1a876926cd505dba047cc82011160c
* update
Change-Id: Ia4b0e2a21aa5e12570a80713432fd48ddfb210e6
* optimize the code
Change-Id: Idf8080c7b17c25e14fe34ef6bad1e1150dab58d6
* update
Change-Id: I904f0120a728b4eb87226159f9c0cde3d18ef26a
* update code
Change-Id: I21653d6bcad905daa8f10d12298d1d27535af9d9
* update
Change-Id: I712b9cde9a18c730fd020ea76f05560e594a9edd
* update
Change-Id: Iafe10c007e9b5057212e2dcfb21671f970b1d344
* update
Change-Id: I40c1c7f54852ee033e0caa22573436ee505a27b8
* optimize the code
Change-Id: Iae006b01608df4f363c63a0ae876dbf63cd56fbd
---------
Co-authored-by: drizzle.zk <drizzle.zk@alibaba-inc.com >
Co-authored-by: zhouli11 <04081337@163.com >
2025-12-11 15:13:56 +08:00
f73fe6c4fb
[ISSUE #9885 ] Fix tiered store cache count and bytes metrics ( #9886 )
2025-12-11 10:27:08 +08:00
1f387b286b
Fix ACL 2.0: correct consumer group extraction from retry topic in SEND_MESSAGE/SEND_MESSAGE_V2/SEND_BATCH_MESSAGE ( #9911 )
2025-12-10 15:58:10 +08:00
89e021e871
[ISSUE #9898 ] Remove AbstractBrokerRunnable and replace with Runnable
...
Change-Id: I94104151c452d09cbe195a3e1126a473b662a337
Co-authored-by: RongtongJin <user@example.com >
2025-12-05 14:03:54 +08:00
1bb2168bf3
Fix controller mode HAService removeConnection ( #9897 )
...
* fix controller mode HAService removeConnection
* fix store test
2025-12-05 11:28:07 +08:00
cbfd37b251
Add some log for gRPC route ( #9896 )
2025-12-05 10:19:27 +08:00
d7e27d6d69
[ISSUE #9396 ] Use fastjson2 in all modules ( #9397 )
...
* Use fastjson2 in all modules
* Update test
* Update test
* Update test
* Add serialization compatibility test tool class
* Update RemotingSerializableCompatTest.java
* Update RemotingSerializableCompatTest.java
* Update RemotingSerializableCompatTest.java
* Update BitSet problem
* Update
* Update
* Update test
* Update test
* Update BUILD.bazel
* Update BUILD.bazel
* Update test
* Update BitSet problem
* Add test
* Add compat test
* merge develop
* Update test
* merge develop
2025-12-04 19:09:08 +08:00
47c6e89589
[ISSUE #9894 ] Update README for RocketMQ 5.3.4 and minor fixes ( #9895 )
2025-12-04 10:05:03 +08:00
1a3c922fdf
Defer the shutdown behavior of popConsumerService to prevent JVM crash ( #9889 )
...
Change-Id: Iaa23996d94f81ed3250b8d528338e8502a35cada
Co-authored-by: RongtongJin <user@example.com >
2025-12-03 14:15:22 +08:00
9f58198abd
[ISSUE #9883 ] Remove unnecessary MessageFormat.format ( #9884 )
...
* [ISSUE #9883 ] Remove unnecessary MessageFormat.format
* Update import
2025-12-03 13:51:13 +08:00
bca6393261
[ISSUE #9891 ] Optimize pop orderly implementation to facilitate expansion ( #9892 )
...
Change-Id: Ie2faff9942027a510c4550a9f2bf2df8c6283137
2025-12-02 16:30:15 +08:00
63d20eb92a
[ISSUE #9734 ] Set revision to 5.3.4 in maven pom ( #9876 )
...
Change-Id: I30234c0c2953bea1e09283610bee6851a89d34c0
Signed-off-by: terrance.lzm <terrance.lzm@alibaba-inc.com >
2025-11-27 11:32:42 +08:00
b77f2ab3a4
[ISSUE #9734 ] Prepare to release Apache RocketMQ 5.3.4 ( #9861 )
...
Change-Id: I2a9e9b99223ed2f98b7611619bff369d83b3e1ea
Signed-off-by: terrance.lzm <terrance.lzm@alibaba-inc.com >
2025-11-26 14:07:50 +08:00
e87f9cbb08
[ISSUE #9875 ] Optimize the RocksDB config shutdown logic when useSingleRocksDBForAllConfigs is set to true to prevent JVM crashes. ( #9874 )
...
* Optimize the RocksDB config shutdown logic when useSingleRocksDBForAllConfigs is set to true to prevent JVM crashes.
Change-Id: I309e8d13b6adc46d68146c05ffd7e026e2852ad8
* Fix bug
Change-Id: Ie577e32f65a3902dd60d654f80a8e7eda5790fbf
---------
Co-authored-by: RongtongJin <user@example.com >
2025-11-26 11:21:01 +08:00
e984023d31
[ISSUE #9816 ] Fix calculate consumer lag with opentelemetry ( #9873 )
2025-11-25 19:52:21 +08:00
f33f626114
[ISSUE #9663 ] Add flatten-maven-plugin plugin ( #9866 )
...
Change-Id: Ic339c1e6622fbf8d48a0257ef532831d268325a2
2025-11-24 19:28:51 +08:00
12774c64be
[ISSUE #9779 ] fix DirectBuffer will cause error at 9+ JDK version. ( #9801 )
...
* fix(DirectBuffer): issue#9779,fix the promblem of 'DirectBuffer will cause error at 9+ JDK version'.
Change-Id: I4657ecc401046a3b0d29b466ee68845f45d34105
* fix(DirectBuffer): issue#9779,fix the promblem of 'DirectBuffer will cause error at 9+ JDK version'.
Change-Id: Iff0880f694cccfa86b812b81260dfe09e4763fa9
2025-11-24 18:00:38 +08:00
736f027c9c
[ISSUE #9870 ] Ensure metadata provider cache executors are shutdown correctly ( #9871 )
...
* [ISSUE #9870 ] Ensure metadata provider cache executors are shutdown correctly
* Improve test comments
2025-11-24 10:34:43 +08:00
5892de6d14
[ISSUE #9862 ] [Bug] Fix the issue of missing bornTime in POP requests when broker ACL is enabled ( #9863 )
2025-11-20 20:09:43 +08:00
ccf973081e
[ISSUE #9789 ] LitePullConsumer supports manually adding subscription reported in Heartbeat ( #9790 )
2025-11-20 19:53:21 +08:00
a68a5bfd8c
[ISSUE #9809 ] Fix NPE in getAcl when subject is null ( #9810 )
2025-11-20 11:50:50 +08:00
963c0e28ff
[ISSUE #9741 ] Optimize authorization whitelist lookup efficiency ( #9760 )
...
* [ISSUE #9741 ] Optimize authorization whitelist lookup efficiency
* Remove unused import
* Fix get authorization metadata provider.
2025-11-20 10:31:46 +08:00
8eabe68acd
[ISSUE #9849 ] Correct ACL cache loader error log in ClusterMetadataService ( #9850 )
2025-11-20 10:30:06 +08:00
7acfe4dca1
[ISSUE #9737 ] Fix client memory leak on connection failure ( #9740 )
2025-11-19 16:22:07 +08:00
d08f787771
[ISSUE #9807 ] Optimize log.warn Time Consumption in the send Method ( #9808 )
2025-11-19 16:05:36 +08:00
d750461f3c
[ISSUE #9857 ] Optimize the performance of the notification by removing unnecessary retry checks ( #9858 )
...
Change-Id: Iddb8841defd4a33e65080c2451df466d045e4a23
2025-11-18 20:58:36 +08:00
9c3e7fe273
[ISSUE #9847 ] Reduce lock contention on the HandleData object to prevent threads from hanging ( #9848 )
2025-11-14 15:11:40 +08:00
6c798d64f0
[ISSUE #9593 ] Optimize Deflater resource release ( #9594 )
2025-11-14 13:44:15 +08:00
69e2524c45
Fix resource leak in IndexStoreFile shutdown ( #9840 )
...
* Fix resource leak in IndexStoreFile shutdown
Add cleanResources() calls after shutdown() for both mappedFile and compactMappedFile to ensure proper cleanup of memory-mapped buffers and file channels.
Change-Id: I2716b4e3b0cd281e89a9d5a00a389dc6048de3e7
Co-developed-by: Cursor <noreply@cursor.com >
* Add a FileNotFoundException check to allow breaking out of the loop after shutdown
Change-Id: Icc0063544ef91de8b2bd96a80f3829a6922cb0e6
* Add a FileNotFoundException check to allow breaking out of the loop after shutdown.
Change-Id: I53a7d2800775d80c1db8feb159c5c258b3517f09
---------
Co-authored-by: RongtongJin <user@example.com >
Co-authored-by: ShannonDing <dinglei@apache.org >
2025-11-14 11:32:35 +08:00
2017630383
[ISSUE #9834 ] Support writeWithoutMmap in IndexStoreFile ( #9835 )
...
* Use FileChannel for writing when writeWithoutMmap is enabled in IndexStoreFile
Change-Id: I23ac2a4dfb8286cd8c3e51aeeb2d54d91136bc03
Co-developed-by: Cursor <noreply@cursor.com >
* Use FileChannel for writing when writeWithoutMmap is enabled in IndexStoreFile
Change-Id: I6a541d13c81a16f39b88ac91ce770717f60d64ff
Co-developed-by: Cursor <noreply@cursor.com >
* Remove unnecessary rewind() call in IndexStoreFile when using FileChannel
Change-Id: Id6deca99d34736761fd1d937f5eb5e75506ba1cc
Co-developed-by: Cursor <noreply@cursor.com >
* Add parameterized test for writeWithoutMmap in IndexStoreFileTest
Change-Id: I1ce92e4f728a018ff7bd3160995f395b480345ea
Co-developed-by: Cursor <noreply@cursor.com >
* Fix compactToNewFile to read data from file when using FileChannel
Change-Id: I9058b08f1bbcf646b5e7913260a5eefe853b9b16
Co-developed-by: Cursor <noreply@cursor.com >
* Fix newBuffer position and limit in compactToNewFile
Change-Id: I1a0100b6159d636a8db1124e3de416b3a2e62bc5
Co-developed-by: Cursor <noreply@cursor.com >
* Update test parameter order in IndexStoreFileTest
Change-Id: Ie01656574c1866badf726ead58a8ad2ca3d01cf7
Co-developed-by: Cursor <noreply@cursor.com >
---------
Co-authored-by: RongtongJin <user@example.com >
2025-11-13 17:25:31 +08:00
e60b67afe7
[ISSUE #9838 ] IndexStoreService use forceShutdown when disk is not writable ( #9839 )
...
* Update store and tieredstore implementation
Change-Id: I29c931148110c4238ce9a256ab2bf69d7b6139a8
Co-developed-by: Cursor <noreply@cursor.com >
* Fix UT NPE
Change-Id: I699ec538118166743308af9e78da50f2aa1b56b3
---------
Co-authored-by: RongtongJin <user@example.com >
2025-11-13 17:25:09 +08:00
3b78ab0960
[ISSUE #9632 ] Fix Pop Long-polling Not Awakened for V1 Retry Messages ( #9828 )
2025-11-12 20:01:23 +08:00
7a1a9501ca
[ISSUE# 9841] Improve Resource Management in TimerWheel to Prevent File Handle Leaks ( #9842 )
...
* fix: fix file handle leak in TimerWheel
- Move RandomAccessFile and FileChannel to instance variables
- Close FileChannel in shutdown() method instead of constructor finally block
- This ensures proper resource cleanup and prevents file handle leaks
Change-Id: If6f0bdd555e5f6fdd5c44ca8404f7918b45cf67b
Co-developed-by: Cursor <noreply@cursor.com >
* fix
Change-Id: Icf8f6c316208f8e38421013f2d7b7028f56d5259
* fix
Change-Id: I9ed022b94fca31f005e71afe92a0374265f6ea11
* fix
Change-Id: Ibfd499652378710ccc7b5632370a77ab5dc4b8f3
---------
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2025-11-12 15:22:59 +08:00
e2be0f259d
fix controller mode scanControllerAddress ( #9845 )
...
* Enhancing code robustness
* fix Checkstyle
2025-11-12 13:39:20 +08:00
f0d3b9c8c6
[ISSUE #9794 ] Change JDK prerequisite from 1.7+ to 1.8+ in documentation ( #9795 )
2025-11-12 11:19:12 +08:00
12f8db1e23
[ISSUE #9821 ] Fix getBrokerConfig no response issue for empty parameters ( #9822 )
...
* [ISSUE #9821 ] Fix getBrokerConfig no response issue for empty parameters
* Update
* Update test
2025-11-12 11:18:40 +08:00
e4170f561c
Add timestamp query message offset to abstract class ( #9836 )
...
Change-Id: I78091629c0b3f7ac438a82aa326a76970e1a80df
Signed-off-by: terrance.lzm <terrance.lzm@alibaba-inc.com >
2025-11-11 15:23:22 +08:00
29a62c46c9
[ISSUE #9735 ] Add TimerWheel snapshot mechanism for reliable recovery ( #9736 )
...
* feat(store): add timer wheel snapshot mechanism for recovery
- Add snapshot backup and recovery functionality to TimerWheel
- Add configuration options for snapshot control with backward compatibility
- Implement atomic flush operations with synchronization locks
- Add snapshot file management (creation, cleanup, selection)
- Support recovery from snapshot files to avoid rebuilding TimerWheel
- Add TimerFlushService.flush() method for manual flush operations
This addresses the issue where discrete TimerWheel state cannot be
recovered from TimerLog alone, ensuring data consistency and
improving recovery performance.
Configuration changes:
- timerWheelSnapshotFlush: enable/disable snapshot functionality (default: false)
- timerWheelDefaultFlush: enable/disable default flush behavior (default: true)
- timerWheelSnapshotIntervalMs: snapshot creation interval (default: 10s)
Change-Id: I801ce7ca6330d02aefcae905e59e1e2b7642ebf5
* fix
Change-Id: I97968a6b87d55c73ce871b3522c67b8b543d6eb2
* fix
Change-Id: Id751e63ddd20ca6223d83d8057623f19e54fa3ca
* fix
Change-Id: I8099c89ebded4f5b66eb98537396afc75b147583
* fix
Change-Id: I1610f1ebba7b05de8517111155f2116025c65e1e
* fix
Change-Id: Iad1171bf7805102f0a87b9623de048fc8bbe9fce
* fix
Change-Id: Ic1ef006f709006a0f30c734cb4426d3aea57eccd
* Acquire the lock only when the timerWheelSnapshotFlush is true
Change-Id: I1c7c2895efebfef0185411c0630bbd2da23ce3c7
* fix
Change-Id: I73e9ecd37f25bda6d6d2f5191bd11ebb8d02c39e
---------
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
Co-authored-by: RongtongJin <jinrongtong16@mails.ucas.ac.cn >
2025-11-10 16:54:30 +08:00
607263ba38
upgrade OpenTelemetry to version 1.44.1 ( #9820 )
...
Change-Id: I1d05cbfbd09d4064b73be68e8c108c7466bed498
2025-11-10 13:57:28 +08:00
35a747452c
[ISSUE #9802 ] Fix the issue of delete logic in tiered storage index service ( #9803 )
2025-11-10 10:29:53 +08:00
e7d930f585
Fixed error message of CommandUtil.fetchMasterAddrByBrokerName ( #9825 )
...
Co-authored-by: TheRealHaui <michael.hausegger@hausegger.tech >
2025-11-08 16:08:08 +08:00
455305a9d4
[ISSUE #9813 ] Fix buffer fetch thread pool starvation in tiered storage ( #9818 )
2025-11-07 11:51:34 +08:00
736d2a860e
[ISSUE #9816 ] Fix concurrent modify opentelemetry record in calculate consumer lag ( #9817 )
2025-11-07 11:47:53 +08:00
440341b604
[ISSUE #9811 ] Fixed the issue of excessive loop iterations in reput service ( #9812 )
2025-11-06 11:37:39 +08:00
6b91c7d4a2
[ISSUE #9798 ] Reduce unnecessary thread switching in sending message via proxy ( #9799 )
2025-11-06 10:21:02 +08:00
d7c8bb7341
[ISSUE #9784 ] Fix the policy comparator to prioritize DENY over ALLOW ( #9785 )
2025-11-03 14:35:13 +08:00
204b251239
Correct method name from updateDateVersion to updateDataVersion ( #9788 )
2025-10-30 11:11:32 +08:00
bea086feb9
[ISSUE #9773 ] Fix core dump when shutdown broker twice
...
Change-Id: I31338a675c8af1657d3e2675a5783b1afc33d7ab
2025-10-27 19:08:52 +08:00
e262890216
[ISSUE #9776 ] Make SharedByteBuffer size configurable via MessageStoreConfig.maxMessageSize ( #9775 )
...
* Make SharedByteBuffer size configurable via MessageStoreConfig.maxMessageSize
Change-Id: Ie3c291ba10b84963fb3ba0af90afa323d9b955ff
* Fix checkstyle
Change-Id: I75f9f767e30f33fc2ea4ceafd59b9d950875c765
* Fix UTs
Change-Id: I57b3c904d37558e4301394fc1dd4188b0866718b
* Fix UTs
Change-Id: I87775116926d3f5271eb13f0a86c0a40446ae432
* Fix bugs
Change-Id: Ib76596b91621b59d1e189642d081d259880f9ac8
* Fix comments
Change-Id: I7bdd3b9f24172afe77a7023b9aa4109dc271c27b
* refactor: make SharedByteBufferManager buffer count configurable
Change-Id: Ia97908f7e96f23542e8acf1a2cc6c1407d3d8e87
2025-10-24 13:36:52 +08:00
2eb21293cf
[ISSUE #9773 ] Implement Shared RocksDB Instance for Broker Configs ( #9774 )
2025-10-23 15:31:41 +08:00
Drizzle