Commit Graph
8676 Commits
Author SHA1 Message Date
Zhanhui Li 8505482c0b fix: avoid memory overhead when there is large number of LMQ ConsumeQueue (#8956) 2024-11-20 14:55:30 +08:00
hqbfzandwanghuaiyuan c13f051eb8 Improve IO for asynchronous delivery processes (#8954)
Co-authored-by: wanghuaiyuan <wanghuaiyuan@xiaomi.com>
2024-11-20 11:13:12 +08:00
yuz10 ae7179d75e [ISSUE #8765] fix low performance of delay message when enable rocksdb consume queue (#8766)
* #7538 fix wrong cachedMsgSize if msg body is changed in consumer callback
* [ISSUE #8765] fix low performance of delay message when enable rocksdb consume queue
* remove prefetch
2024-11-20 11:10:01 +08:00
Ji Juntao 78575af0b8 [ISSUE #8935] Fix behind metrics unit error in timer message store (#8936) 2024-11-19 17:35:10 +08:00
rongtong bf2f2a7774 Fix incorrect path for exportMetadataInRocksDBCommand (#8941) 2024-11-18 17:25:20 +08:00
hqbfzandwanghuaiyuan c961edd2fd [ISSUE #8945] Remove unnecessary operations from the critical section
Co-authored-by: wanghuaiyuan <wanghuaiyuan@xiaomi.com>
2024-11-18 16:46:34 +08:00
rongtong 163451ed30 Support for Persisting LMQ Consumer Offsets in Config V1 Using RocksDB (#8939) 2024-11-18 14:30:01 +08:00
rongtong 4e8a5ca48f Add incGroupAckNums and incGroupCkNums to LmqBrokerStatsManager (#8943) 2024-11-18 14:23:00 +08:00
yx9o 2e1ca053d2 [ISSUE #8901] Add more test coverage for RpcClientImpl (#8902) 2024-11-18 10:19:48 +08:00
jiao jiananandjiaoja 4574647a3a [ISSUE #8909] Move nullcheck ahead (#8910)
Co-authored-by: jiaoja <jiaoja@asiainfo.com>
2024-11-16 09:07:55 +08:00
dingshuangxi888 797024f3f8 fix update user not valid bug (#8926) 2024-11-15 10:35:37 +08:00
Drizzleanddrizzle.zk 60e68dadd9 [ISSUE #8921] Add isWakeCommitWhenPutMessage for AIO
Co-authored-by: drizzle.zk <drizzle.zk@alibaba-inc.com>
2024-11-14 19:42:25 +08:00
dingshuangxi888 87b97f271c [ISSUE #8917]Topic route return none permission message queues for gRPC client (#8919)
* When the queue lacks permission, return one queue to allow the client to upload a heartbeat for gRPC Topic route interface.
2024-11-14 19:24:19 +08:00
qianye 66ba4566f5 close channel when receive go away twice (#8862)
close channel when receive go away twice (#8862)
2024-11-14 14:07:20 +08:00
mawen12 2bbc852361 [ISSUE #8906] Handle string toUpperCase outside the loop 2024-11-12 19:29:21 +08:00
TianMing2018 7eb16fffec fixed typo of RocketMQ_Example.md (#8905)
fixed typo worlds 'finalString'
2024-11-11 10:00:26 +08:00
mawen12 dbd2791eca [ISSUE #8889] handle namespace outside the loop (#8890)
* handle namespace outside the loop

* fix checkstyle check failed
2024-11-08 09:03:10 +08:00
dingshuangxi888 851473443e [ISSUE #8882] Change the compare method for acl signature to improve the security. (#8883)
* Change the compare method for acl signature to improve the security.

* Change the compare method for acl signature to improve the security.
2024-11-07 19:04:54 +08:00
Thomas Lee e281697b8e [ISSUE #8808] Resolve unsupported 'UseBiasedLocking' VM Option for JDK21 (#8809) 2024-11-06 19:30:18 +08:00
rongtong 43f3c2b61b [ISSUE #8885] Resolve the issue of inaccurate CK number statistics (#8886) 2024-11-06 10:15:42 +08:00
Zhanhui Li 311b831c02 [ISSUE #8829] Keep data version while reload and XXXConfigManagerV2 turns off sync
Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
2024-11-06 10:12:24 +08:00
yuz10 5600684eb4 [ISSUE #8725] clean DefaultMQPushConsumer after start fail (#8726)
* [ISSUE #8725]clean DefaultMQPushConsumer after start fail

* clean DefaultLitePullConsumerImpl after start fail
2024-10-30 20:08:28 +08:00
Crazywen fe8077250b [ISSUE #8875] Fix HAConnection leak 2024-10-30 16:42:30 +08:00
Zhanhui Li b974362b37 Revert "fix RequestCode overflowed issues:8868 (#8871)" (#8874)
This reverts commit 327abe5d6e.
2024-10-30 14:20:00 +08:00
Zhanhui Li dd62ed0f3b [ISSUE #8892] Add test cases to config manager v2 (#8873)
* fix: add unit test for TopicConfigManagerV2

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: add unit test cases for config manager v2

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* chore: add copyright header

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

---------

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
2024-10-29 20:07:49 +08:00
zzjcoolandzhijiezheng 327abe5d6e fix RequestCode overflowed issues:8868 (#8871)
Co-authored-by: zhijiezheng <zhijiezheng@tencent.com>
2024-10-29 19:44:20 +08:00
LetLetMe 0b24768500 [ISSUE #8822] Double write cq, reduce unnecessary switches (#8823)
* Reduce unnecessary switches
2024-10-29 19:09:35 +08:00
zhaohaiandzh378814 a96a14f734 update LanguageCode (#8872)
Co-authored-by: zh378814 <wb-zh378814@alibaba-inc.com>
2024-10-29 19:07:14 +08:00
yx9o ecb45bb90d [ISSUE #8852] Add more test coverage for ClientMetadata (#8853)
* [ISSUE #8852] Add more test coverage for ClientMetadata

* Update

* Update

* Update
2024-10-28 16:32:58 +08:00
Zhanhui Li c8938e0c6c [ISSUE #8829] feat: provide ConfigManagerV2 to make best uses of RocksDB (#8856)
* feat: provide ConfigManagerV2 to make best uses of RocksDB

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: release RocksDB objects using try-with-resource

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

---------

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
2024-10-28 10:00:33 +08:00
hqbfzandwanghuaiyuan 95b88ff8fc [ISSUE #8442][RIP-70-3] Extract adaptive lock mechanism (#8663)
* extract the adaptive lock

* extract the adaptive lock

* feat(): perfect the adaptive lock

* feat(): perfect the adaptive lock

* Optimized code type

* Optimized code type

* Optimized code type

* fix fail test

* Optimize the adaptive locking mechanism logic

* Optimize the adaptive locking mechanism logic

* feat:Adaptive locking mechanism adjustment

* feat:Adaptive locking mechanism adjustment

* feat:Adaptive locking mechanism adjustment

* Optimize the adaptive locking mechanism logic

* Optimize the adaptive locking mechanism logic

* Optimize the adaptive locking mechanism logic

* feat:Supports the hot activation of ABS locks

* feat:Supports the hot activation of ABS locks

* feat:Supports the hot activation of ABS locks

* feat:Supports the hot activation of ABS locks

* Optimize code style

* Optimize code style

* Optimize code style

* Optimize code style

* Optimize code style

* Optimize code style

* Updated the locking mechanism name

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* delete unused import

* Optimize the logic of switching to spin locks

* Revert "Optimize the logic of switching to spin locks"

This reverts commit 1d7bac5c2f.

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimize the logic of switching to spin locks

* Optimized locking logic

* Optimized locking logic

* Optimized locking logic

* fix test

* fix test

* fix test

* fix test

* Optimize code style

* Optimize code style

* fix test

* fix test

* optimize client rebalancing logic

---------

Co-authored-by: wanghuaiyuan <wanghuaiyuan@xiaomi.com>
2024-10-23 19:17:37 +08:00
Zhanhui Li 738c9f3a95 [ISSUE #8829]feat: add test case to RocksDBConsumeQueueOffsetTable (#8857)
* feat: add test case to RocksDBConsumeQueueOffsetTable, verifying forEach works properly for long topic-name

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: change OpenJDK distribution from adopt to corretto as the previous one is not updated anymore

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: unify set-java version and distribution

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

---------

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
2024-10-23 19:17:05 +08:00
Liu Shengzhong 2956f6d46b [ISSUE #8772] Remove lock mq step in broadcasting mode rebalancing (#8774) 2024-10-23 10:29:56 +08:00
mawen12 7f8667adde use correct log method (#8851) 2024-10-23 09:57:35 +08:00
Zhanhui Li b86059c7c1 Support LMQ dispatch in case if Consume Queue Store is RocksDB-based (#8842)
* feat: support LMQ dispatch

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: introduce group-commit for batch insertion of RocksDB KV pairs

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: propagate store error to broker module

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* chore: fix all Bazel warning and errors

Signed-off-by: Zhanhui Li <lizhanhui@gmail.com>

* fix: remove unnecessary batch-ops when writing RocksDB using atomic flush

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: find a writable directory for RocksDB logs

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* chore: clean up ConfigHelperTest

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: truncate consume queues in case commit log records are truncated

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: truncate LMQ max offsets

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: correct truncate boundary of consume queues

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: correct MessageExt encoding

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* chore: remove unused import

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

---------

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
Signed-off-by: Zhanhui Li <lizhanhui@gmail.com>
2024-10-23 09:56:37 +08:00
Zhanhui Li d2fd068be7 fix: registerProducer should not be affected by concurrent scanNotAct… (#8847)
* fix: registerProducer should not be affected by concurrent scanNotActiveChannel

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* chore: fix code format and make CI pass

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

---------

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
2024-10-23 09:37:17 +08:00
mawen12 c033c3e85f [ISSUE #8848] Fix log typo 2024-10-23 08:29:35 +08:00
dinglei a024dc81da update netty version to 4.1.114 to fix CVE-2023-34462 (#8832) 2024-10-21 13:40:49 +08:00
Zhanhui Li 07f13fd883 fix: remove unnecessary synchronized to improve concurrency (#8840)
Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
2024-10-21 09:25:55 +08:00
Zhanhui Li 1e0b08f56a fix: make ConsumeQueueStore bottom most compression type configurable (#8841)
Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
2024-10-18 16:50:13 +08:00
Zhanhui Li 49e23e1e0b fix: atomic flush incorrect use and clean up code (#8830)
Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
2024-10-18 14:30:38 +08:00
mawen12 01f9848b74 fix variables match annotation (@RocketMQResource) (#8821) 2024-10-18 09:18:41 +08:00
Ji Juntao 031687dbd7 [ISSUE #8835] When ck is in the buffer, incomplete ack will lead to message duplication. (#8836)
* add brokerName in ackMsg

* add brokerName in ackMsg
2024-10-17 19:16:14 +08:00
rongtong 4f5f705f16 [ISSUE #8780] Implement asynchronous storage of ack/ck messages in pop consume to enhance performance (#8727)
* Pop consume asynchronization

* Pass UTs and ITs

* Pass the checkstyle

* Fix LocalGrpcIT can not pass

* Fix the UT can not pass

* Simplify duplicate methods in EscapeBridge
2024-10-16 13:39:27 +08:00
yx9o a7dc023589 [ISSUE #8824] Fix IllegalStateException caused by logical errors (#8825) 2024-10-16 11:00:51 +08:00
LetLetMe 2fa7513a2b [ISSUE #8764] Implement consume lag estimation in cq rocksdb store (#8800) 2024-10-15 14:14:31 +08:00
luozongle01 2355a5f784 [ISSUE #8816] Fix log typo. (#8817) 2024-10-14 09:13:17 +08:00
小陈 0c4064dc24 [ISSUE #8810] Fix independent execution of e2e and benchmark deployments (#8812)
* Fix e2e and benchmark tests dependency issues

* Update job naming

* Fix bug

* Update
2024-10-11 16:43:07 +08:00
luozongle01 11f0002a3c [ISSUE #8806] fix autoBatch bug when connecting multiple RocketMQ clusters. (#8807) 2024-10-11 15:29:18 +08:00
Lei Zhiyuan e75554d5a8 [ISSUE #8804] clean offset when remove group offset 2024-10-10 09:53:06 +08:00