b43a61f3df
Fix flaky store HA and DLedger tests
2026-05-29 16:11:32 +08:00
9e2d877055
[ISSUE #10375 ] Fix race condition between deleteTopic and FlushConsumeQueueService by removing getLifeCycle indirection ( #10376 )
2026-05-25 23:13:40 +08:00
7e5d22dff8
[ISSUE #10334 ] Make native CqCompactionFilter shim self-contained without rocksdbjni dependency ( #10371 )
2026-05-25 10:07:16 +08:00
61c493c068
[ISSUE #10334 ] Support native CqCompactionFilter with cross-platform JNI shim ( #10335 )
2026-05-21 10:12:14 +08:00
0752ddf89b
[ISSUE #10270 ] Make Pop RocksDB BlockCache size configurable via MessageStoreConfig ( #10271 )
...
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
2026-04-22 17:24:38 +08:00
343daa9fb5
[ISSUE #10173 ] Improve PopLite: rename RocksDB CQ path and schedule autoClean ( #10242 )
2026-04-07 17:51:59 +08:00
2c2cc921ba
[ISSUE #10173 ] Support LMQ in CombineConsumeQueueStore without migration to RocksDB CQ ( #10174 )
2026-04-03 14:29:42 +08:00
d66cfa9d7a
[ISSUE #10165 ] Prevent RocksDB metrics from being overwritten after timer engine switch ( #10166 )
2026-03-30 16:48:33 +08:00
e5d33722d9
[ISSUE #10223 ] Not query the index of system topics in tiered storage ( #10224 )
2026-03-30 13:34:53 +08:00
932588d3e1
[ISSUE #10201 ] Optimize queryOffset method overloads in IndexService ( #10202 )
2026-03-25 15:24:21 +08:00
ebf1595418
[ISSUE #10103 ] Improve batch polling efficiency in pollIndexRecord method ( #10104 )
2026-03-17 16:22:28 +08:00
628230d22c
[ISSUE #10011 ] Optimize accelerated recovery process and refactor code ( #10012 )
...
* When IndexRocksDBEnable or TransRocksDBEnable are enabled, we need to take these two offsets into account to accelerate recovery.
* Add UTs
* Refactor the code based on the review comments
* Revert "[ISSUE #8127 ]Optimize the metric calculation logic of the time wheel"
* Remove useless import
* Refactor Code
* Refactor Code
* Refactor Code
* Refactor Code
* Refactor Code
* Implement accelerated recovery for the file-based ConsumeQueue.
* Implement accelerated recovery for the file-based ConsumeQueue.
Change-Id: Ieac45d0582f2f83d977aeb8e6f5084268b7f8752
* Implement accelerated recovery for the file-based ConsumeQueue.
* Ignore testTruncateCQ UT
---------
Co-authored-by: RongtongJin <user@example.com >
2026-02-28 10:30:15 +08:00
7be720250e
[ISSUE #10108 ] Broker startup fail in recover ( #10109 )
2026-02-24 13:56:23 +08:00
b8f8856868
[ISSUE #10105 ] Fix ClassCastException in getLocks() method ( #10106 )
2026-02-23 14:43:45 +08:00
c197e50f8d
[ISSUE #10079 ] FlushConsumeQueueService: always flush store checkpoint after CQ flush ( #10080 )
...
Change-Id: I57c0922bb81c2d43359867e82a92fdf2deab7ad7
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2026-02-09 17:38:03 +08:00
1754cece4b
Restore the lost PR 3460 ( #10067 )
2026-02-06 15:30:15 +08:00
26480ddc87
[ISSUE #10073 ] Fix StoreCheckpoint logicsMsgTimestamp to reflect flushed consume queue storetime ( #10074 )
...
* Fix StoreCheckpoint: set logicsMsgTimestamp only after CQ flush, use logicsMsgTempTimestamp for in-memory storetime
Change-Id: I6085bf6efaef84168ece31d080481717465f2b13
* rename to tmpLogicsMsgTimestamp
Change-Id: Ia65ca06751f765bdc2bf053c58e08789f4b2fb22
---------
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2026-02-05 22:34:12 +08:00
10c650c487
[ISSUE #10034 ] Optimizing cq iterator and calculating lag ( #10056 )
2026-02-02 21:01:46 +08:00
f80753f0da
[ISSUE #10050 ] Support ChangeInvisibleTime without incrementing message reconsume times ( #10051 )
2026-02-02 14:06:57 +08:00
7583fda113
fix(store): close all consume queue file handles on ConsumeQueueStore shutdown ( #10060 )
...
* fix(store): close all consume queue file handles on ConsumeQueueStore shutdown
* remove implementation
---------
Co-authored-by: RongtongJin <user@example.com >
2026-02-02 11:41:35 +08:00
24ca9e46b5
[ISSUE #10043 ] Make TimerMessageReputService thread pool configurable and shutdown gracefully ( #10044 )
2026-01-30 10:04:40 +08:00
c7892a694a
[ISSUE #9852 ] Print full message when CRC not found in properties ( #9853 )
...
When the error 'failed to check message CRC, not found CRC in properties' occurs,
now it will print the full message content including topic, properties map,
properties length, and full message hex for debugging purposes.
Change-Id: I26d977e13802c5e69d01f16512e40d121dcff354
Co-developed-by: Cursor <noreply@cursor.com >
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2026-01-26 10:02:17 +08:00
90b0906ae6
[ISSUE #10038 ] Fix checkCommitLogOffsetOnRecover to skip validation for BLANK_MAGIC_CODE messages ( #10039 )
...
Change-Id: If1913db5155fe4135f2dbd1b58417b3afc6748c1
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2026-01-26 10:01:49 +08:00
236ede218a
Fix: avoid extra padding when message end position is already page-aligned ( #10036 )
...
Change-Id: I3e7fad9c4b194b20015414bcceb830760df68fea
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2026-01-21 17:19:34 +08:00
4aa6cdd99e
[ISSUE #10019 ] Revert "[ISSUE #8127 ]Optimize the metric calculation logic of the time wheel" ( #10020 )
...
* Revert "[ISSUE #8127 ]Optimize the metric calculation logic of the time wheel"
* Revert "[ISSUE #8127 ]Optimize the metric calculation logic of the time wheel"
* Revert "[ISSUE #8127 ]Optimize the metric calculation logic of the time wheel"
---------
Co-authored-by: RongtongJin <user@example.com >
2026-01-21 14:15:00 +08:00
94ba5e14f8
[ISSUE #10031 ] Add PreprocessHandler interface in AllocateMappedFileService
...
Change-Id: I4e81916a79f89c095ffb7b860c8ccd49e88c76ea
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2026-01-21 10:07:51 +08:00
36adf1251f
[ISSUE #10017 ] Validate commitlog offset in recoverAbnormally to prevent processing … ( #10018 )
...
* validate commitlog offset in recoverAbnormally to prevent processing old file data that passes CRC checks
Change-Id: If4b1881f82d26ce8d374472d73ec9ce3d51ba643
* fix
Change-Id: Idc4bf7ec476cc9b6529619c2aa9afd6a980b819c
* add checkCommitLogOffsetOnRecover in MessageStoreConfig
Change-Id: Iac9afbb8b3ffb03fa15890decaf502afbfa44cf9
---------
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2026-01-20 14:44:22 +08:00
ee104114ac
[ISSUE #10015 ] Optimize writeWithoutMmap. add page alignment to avoid read-modify-write
...
Change-Id: I41ae3b71a4803295c2487cdf8f5e458764b64ebc
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2026-01-16 17:59:02 +08:00
1b6a919bde
[RIP-83] Lite Topic: A New Message Model ( #9800 )
...
Co-authored-by: imzs <imzs@foxmail.com >
Co-authored-by: zhouli11 <04081337@163.com >
2026-01-16 17:20:22 +08:00
4b187c6e11
[ISSUE #9945 ] Use UniqueKey as the TimerDelKey value when no namespace is appended ( #9951 )
2025-12-24 13:48:47 +08:00
541fdad0ff
[ISSUE #9947 ] Fix TimerMessageStore.checkAndReviseMetrics throws BufferUnderflowException ( #9948 )
2025-12-24 11:27:59 +08:00
79e700312a
[ISSUE #9945 ] Use UniqueKey as TimerDelKey by default
2025-12-24 10:00:01 +08:00
5132258db9
Fix epoch truncate ( #9942 )
2025-12-24 09:31:42 +08:00
fe5c17cb8b
Optimized the function naming ( #9935 )
...
* add isWakeCommitWhenPutMessage for AIO
* optimzie the Function name
Change-Id: Id91e3eb9c4488fb9804fb2c105082657e66c44c0
* optimized the function naming
Change-Id: Ifc482f91220ff328e5c5425a57a04ac627e8d469
---------
Co-authored-by: drizzle.zk <drizzle.zk@alibaba-inc.com >
2025-12-19 16:03:49 +08:00
9f9cab88ba
[ISSUE #9396 ] replace fastjson with fastjson2 ( #9927 )
...
* remove fastjson dependency
* remove fastjson
* revert
* revert
2025-12-16 19:11:01 +08:00
7a08b4cb68
[RIP-82] Implement Timer message, transaction message, and index based on RocksDB ( #9787 )
...
* add isWakeCommitWhenPutMessage for AIO
* Implemented Timer message, Transaction Message, Index based on rocksdb
* add timeline roll interval can be config
* optimize the code
* optimize the code
* add sone logic for recover commitlog
* optimize the code
* update
* update
* optimize the code
* update
* update
* optimize the code
* optimize the code
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
Change-Id: I700798ef625187ac32c16c8ea5aae5f2debfdddf
* update
Change-Id: I1d2dfa24317d2ad7c0ca41d90cd2b3c1ba260b0b
* update
Change-Id: I974e30eb909722ea2a17a14e505759bf20f4f860
* update
Change-Id: Ica01fbd08a2d60e9a0f4ab54b41ae4768cc4fa5d
* update
Change-Id: I764ce6cea3159b025bf1e6e65b60e20514cecb4f
* update
Change-Id: I0cc735896e1266934293ec06cb4f07746a1894a1
* update
Change-Id: I6538b6a60631c62e162bf511ae20185a07b72508
* update
Change-Id: I03f83b9e8199f3595ca28471df4a2e94671fbaae
* update
Change-Id: I5b19b671fa0b40cd2aba63f1d0eb654b92775cb4
* update the code
Change-Id: I22788d112b7f214310719e2d3ebad839ef818e67
* update
Change-Id: I7da89fe635b58ac5026095a8e484fa46f68521e7
* update the code
Change-Id: I5eade6c31803b8cf05c1a4bb83c226d57f4980c7
* update
Change-Id: Id67f600fad1a876926cd505dba047cc82011160c
* update
Change-Id: Ia4b0e2a21aa5e12570a80713432fd48ddfb210e6
* optimize the code
Change-Id: Idf8080c7b17c25e14fe34ef6bad1e1150dab58d6
* update
Change-Id: I904f0120a728b4eb87226159f9c0cde3d18ef26a
* update code
Change-Id: I21653d6bcad905daa8f10d12298d1d27535af9d9
* update
Change-Id: I712b9cde9a18c730fd020ea76f05560e594a9edd
* update
Change-Id: Iafe10c007e9b5057212e2dcfb21671f970b1d344
* update
Change-Id: I40c1c7f54852ee033e0caa22573436ee505a27b8
* optimize the code
Change-Id: Iae006b01608df4f363c63a0ae876dbf63cd56fbd
---------
Co-authored-by: drizzle.zk <drizzle.zk@alibaba-inc.com >
Co-authored-by: zhouli11 <04081337@163.com >
2025-12-11 15:13:56 +08:00
89e021e871
[ISSUE #9898 ] Remove AbstractBrokerRunnable and replace with Runnable
...
Change-Id: I94104151c452d09cbe195a3e1126a473b662a337
Co-authored-by: RongtongJin <user@example.com >
2025-12-05 14:03:54 +08:00
1bb2168bf3
Fix controller mode HAService removeConnection ( #9897 )
...
* fix controller mode HAService removeConnection
* fix store test
2025-12-05 11:28:07 +08:00
d7e27d6d69
[ISSUE #9396 ] Use fastjson2 in all modules ( #9397 )
...
* Use fastjson2 in all modules
* Update test
* Update test
* Update test
* Add serialization compatibility test tool class
* Update RemotingSerializableCompatTest.java
* Update RemotingSerializableCompatTest.java
* Update RemotingSerializableCompatTest.java
* Update BitSet problem
* Update
* Update
* Update test
* Update test
* Update BUILD.bazel
* Update BUILD.bazel
* Update test
* Update BitSet problem
* Add test
* Add compat test
* merge develop
* Update test
* merge develop
2025-12-04 19:09:08 +08:00
12774c64be
[ISSUE #9779 ] fix DirectBuffer will cause error at 9+ JDK version. ( #9801 )
...
* fix(DirectBuffer): issue#9779,fix the promblem of 'DirectBuffer will cause error at 9+ JDK version'.
Change-Id: I4657ecc401046a3b0d29b466ee68845f45d34105
* fix(DirectBuffer): issue#9779,fix the promblem of 'DirectBuffer will cause error at 9+ JDK version'.
Change-Id: Iff0880f694cccfa86b812b81260dfe09e4763fa9
2025-11-24 18:00:38 +08:00
e60b67afe7
[ISSUE #9838 ] IndexStoreService use forceShutdown when disk is not writable ( #9839 )
...
* Update store and tieredstore implementation
Change-Id: I29c931148110c4238ce9a256ab2bf69d7b6139a8
Co-developed-by: Cursor <noreply@cursor.com >
* Fix UT NPE
Change-Id: I699ec538118166743308af9e78da50f2aa1b56b3
---------
Co-authored-by: RongtongJin <user@example.com >
2025-11-13 17:25:09 +08:00
7a1a9501ca
[ISSUE# 9841] Improve Resource Management in TimerWheel to Prevent File Handle Leaks ( #9842 )
...
* fix: fix file handle leak in TimerWheel
- Move RandomAccessFile and FileChannel to instance variables
- Close FileChannel in shutdown() method instead of constructor finally block
- This ensures proper resource cleanup and prevents file handle leaks
Change-Id: If6f0bdd555e5f6fdd5c44ca8404f7918b45cf67b
Co-developed-by: Cursor <noreply@cursor.com >
* fix
Change-Id: Icf8f6c316208f8e38421013f2d7b7028f56d5259
* fix
Change-Id: I9ed022b94fca31f005e71afe92a0374265f6ea11
* fix
Change-Id: Ibfd499652378710ccc7b5632370a77ab5dc4b8f3
---------
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
2025-11-12 15:22:59 +08:00
e4170f561c
Add timestamp query message offset to abstract class ( #9836 )
...
Change-Id: I78091629c0b3f7ac438a82aa326a76970e1a80df
Signed-off-by: terrance.lzm <terrance.lzm@alibaba-inc.com >
2025-11-11 15:23:22 +08:00
29a62c46c9
[ISSUE #9735 ] Add TimerWheel snapshot mechanism for reliable recovery ( #9736 )
...
* feat(store): add timer wheel snapshot mechanism for recovery
- Add snapshot backup and recovery functionality to TimerWheel
- Add configuration options for snapshot control with backward compatibility
- Implement atomic flush operations with synchronization locks
- Add snapshot file management (creation, cleanup, selection)
- Support recovery from snapshot files to avoid rebuilding TimerWheel
- Add TimerFlushService.flush() method for manual flush operations
This addresses the issue where discrete TimerWheel state cannot be
recovered from TimerLog alone, ensuring data consistency and
improving recovery performance.
Configuration changes:
- timerWheelSnapshotFlush: enable/disable snapshot functionality (default: false)
- timerWheelDefaultFlush: enable/disable default flush behavior (default: true)
- timerWheelSnapshotIntervalMs: snapshot creation interval (default: 10s)
Change-Id: I801ce7ca6330d02aefcae905e59e1e2b7642ebf5
* fix
Change-Id: I97968a6b87d55c73ce871b3522c67b8b543d6eb2
* fix
Change-Id: Id751e63ddd20ca6223d83d8057623f19e54fa3ca
* fix
Change-Id: I8099c89ebded4f5b66eb98537396afc75b147583
* fix
Change-Id: I1610f1ebba7b05de8517111155f2116025c65e1e
* fix
Change-Id: Iad1171bf7805102f0a87b9623de048fc8bbe9fce
* fix
Change-Id: Ic1ef006f709006a0f30c734cb4426d3aea57eccd
* Acquire the lock only when the timerWheelSnapshotFlush is true
Change-Id: I1c7c2895efebfef0185411c0630bbd2da23ce3c7
* fix
Change-Id: I73e9ecd37f25bda6d6d2f5191bd11ebb8d02c39e
---------
Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com >
Co-authored-by: RongtongJin <jinrongtong16@mails.ucas.ac.cn >
2025-11-10 16:54:30 +08:00
440341b604
[ISSUE #9811 ] Fixed the issue of excessive loop iterations in reput service ( #9812 )
2025-11-06 11:37:39 +08:00
204b251239
Correct method name from updateDateVersion to updateDataVersion ( #9788 )
2025-10-30 11:11:32 +08:00
e262890216
[ISSUE #9776 ] Make SharedByteBuffer size configurable via MessageStoreConfig.maxMessageSize ( #9775 )
...
* Make SharedByteBuffer size configurable via MessageStoreConfig.maxMessageSize
Change-Id: Ie3c291ba10b84963fb3ba0af90afa323d9b955ff
* Fix checkstyle
Change-Id: I75f9f767e30f33fc2ea4ceafd59b9d950875c765
* Fix UTs
Change-Id: I57b3c904d37558e4301394fc1dd4188b0866718b
* Fix UTs
Change-Id: I87775116926d3f5271eb13f0a86c0a40446ae432
* Fix bugs
Change-Id: Ib76596b91621b59d1e189642d081d259880f9ac8
* Fix comments
Change-Id: I7bdd3b9f24172afe77a7023b9aa4109dc271c27b
* refactor: make SharedByteBufferManager buffer count configurable
Change-Id: Ia97908f7e96f23542e8acf1a2cc6c1407d3d8e87
2025-10-24 13:36:52 +08:00
4c460387e7
[ISSUE #9752 ] Fix time dequeue latency unit to milliseconds ( #9753 )
2025-10-15 15:47:05 +08:00
4c06b15b8b
[ISSUE #9739 ] Revert skipped test cases in macos environment ( #9749 )
2025-10-13 20:58:33 +08:00
b1a7ac80dd
fix get dledger mod storepath ( #9746 )
...
Co-authored-by: 赵 <zhaoze@harmonycloud.cn >
2025-10-09 14:03:11 +08:00
RongtongJin