opensource/rocketmq
3
0
Fork 0
mirror of https://github.com/apache/rocketmq.git synced 2026-09-17 17:17:55 +08:00
Code Issues Packages Projects Releases Wiki Activity
8,499 Commits 73 Branches 40 Tags
release-5.3.0
Commit Graph
72 Commits
Author SHA1 Message Date
lizhimins 7ee50ec32e [maven-release-plugin] prepare release rocketmq-all-5.3.0 (#8379) 2024-07-08 23:15:33 +08:00
lizhimins 155bcbf23c [ISSUE #8197] Support fast filter message in tiered storage (#8198) 2024-06-06 15:50:52 +08:00
yuz10andyuzhou 1b42515093 [ISSUE #8129] Support topic reserved time in tiered storage (#8130) 
Co-authored-by: yuzhou <yuzhou4@huawei.com>
 2024-05-22 14:09:00 +08:00
yuz10 159a603219 [ISSUE #5923] Fix tiered store README.md error (#8115) 2024-05-10 14:16:07 +08:00
bxfjbandzhaoyuhan ad02756078 [ISSUE #8049] fix tiered store delete empty topic NPE (#8050) 
Co-authored-by: zhaoyuhan <zhaoyuhan@xiaomi.com>
 2024-05-08 15:39:49 +08:00
yuz10 634092a16f [ISSUE #5923] Fix tiered store README.md error about Configuration (#8110) 2024-05-08 13:59:46 +08:00
lizhimins 0f0324a7dd [ISSUE #8076] Fix correct min cq offset when delete tiered storage CommitLog (#8082) 2024-05-07 09:52:48 +08:00
lizhimins 093cb843ea [ISSUE #7974] Add repeatedly read same offset log to find unexpected situations (#7975) 2024-03-27 18:51:01 +08:00
AYueandayue d9c75ff1f9 [ISSUE #7872] Fix Tiered Store Query Message Async return different view each time (#7874) 
Co-authored-by: ayue <ericyu0421@163.com>
 2024-03-26 15:12:38 +08:00
lizhimins dde4fcc3f6 Add interface comment for query offset operation with boundary type (#7965) 2024-03-26 11:20:22 +08:00
lizhimins b6efbb11ca [ISSUE #7878] Fix query message offset return wrong offset with boundary type (#7962) 2024-03-25 18:55:05 +08:00
dingshuangxi888 f2dd5b06fe [ISSUE #7943] Add bazel config for auth module and fix bazel test (#7944) 2024-03-20 14:38:01 +08:00
lizhimins 026a910b26 [ISSUE #7878] Performance Improvement and Bug Fixes for the Tiered Storage Module (#7899) 
Performance Improvement and Bug Fixes for the Tiered Storage Module
 2024-03-18 10:46:36 +08:00
rongtong f41160210a [maven-release-plugin] prepare for next development iteration (#7828) 2024-02-12 11:07:03 +08:00
rongtong 281a7b32f0 [maven-release-plugin] prepare release rocketmq-all-5.2.0 (#7827) 2024-02-11 17:53:50 +08:00
lizhimins 50a92a2474 [ISSUE #7585] Always return duplicate buffer when filter message and fix log format (#7654) 2023-12-13 19:50:19 +08:00
lizhimins c2c29c2435 [ISSUE #7545] Fix set mapped file to null cause file can not destroy (#7612) 2023-12-04 16:13:07 +08:00
lizhimins aec1055830 [ISSUE #7585] Support message filtering in rocketmq tiered storage (#7594) 2023-12-04 16:12:42 +08:00
lizhiminsandbareheadtom 63130f51e8 [ISSUE #7545] [RIP-65] Support efficient random index for massive messages (#7546) 
Support efficient random index for massive messages

Co-authored-by: bareheadtom <1983697019@qq.com>
 2023-11-21 13:57:44 +08:00
lizhimins 4791d9a1f1 [ISSUE #5923] Revert "Fix tiered store README.md error about Configuration (#7436)" (#7557) 
This reverts commit 70dc93abbc.
 2023-11-14 17:59:47 +08:00
Zhanhui Li 15d32db03b [ISSUE #7547] Let consumer be aware of message queue assignment change (#7548) 
* let consumer be aware of message queue assignment change

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* add unit test for DefaultMQPushConsumer#setMessageQueueListener

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix: bazel build warnings

Signed-off-by: Zhanhui Li <lizhanhui@apache.org>

* fix: set MixCommitlogTest test size as medium

Signed-off-by: Zhanhui Li <lizhanhui@apache.org>

* allow cache bazel test results

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix code style issue by removing unused imports

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* fix #7552

Signed-off-by: Zhanhui Li <lizhanhui@apache.org>

---------

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
Signed-off-by: Zhanhui Li <lizhanhui@apache.org>
 2023-11-13 09:44:25 +08:00
yangguodongandyangguodong.cn 70dc93abbc Fix tiered store README.md error about Configuration (#7436) 
* Fix tiered store README.md error about Configuration

* Fix change tieredStoreFilePath to tieredStoreFilepath

* revert README.md change

---------

Co-authored-by: yangguodong.cn <yangguodong.cn@bytedance.com>
 2023-11-09 11:14:54 +08:00
guyinyouandguyinyou 91349f30b9 [ISSUE #7437] Add the CRC check of commitlog (#7468) 
* Added CRC32 check for full data

* add unit test

* add MessageExtBrokerInnerTest

* fix codestyle

* fix codestyle

---------

Co-authored-by: guyinyou <guyinyou.gyy@alibaba-inc.com>
 2023-10-25 14:54:00 +08:00
fujian-zfjandRongtongJin f565654351 [ISSUE #7064] [RIP-66-2] Support KV(RocksDB) Storage for ConsumeQueue (#7120) 
* typo int readme[ecosystem]

* consumequue support rocksdb

* fix rocksdb test

* fix rocksdb test

* remove unused method

* split into two tables

* CqUnit in Rocksdb [phyOffset, bodySize, tagHashCode, msgStoreTime]

* fix build.baze in tieredMessageStore

* skip RocksDBMessageStoreTest bazel

* skip RocksDBMessageStoreTest bazel

* Rocksdb TimerMessageStore

* fix unit test bazel

* fix store build.bazel

* fix store build.bazel

* optimize

* optimize

* polish

* build bytebuffer pair inner

* remove unused code

* DataConverter.CHARSET_UTF8

* fix comment

* fix comment

* rebuild test

* rebuild test

* optimize

* rebuild test

* polish

* polish

* rebuild test

* merge develop

* rebuild test

* rebuild test

* fix getConsumeQueue not find cq

* fix test

* rocksdb new version

* rebuild test

* fix bug

* Resolve conflicts after merging develop

* fix updateCqOffset

* fix recoverAbnormally

* fix recoverAbnormally

* polish

* polish

* polish

* remove exception in cleanunusedTopic

* remove exception in cleanunusedTopic

* remove exception in cleanunusedTopic

---------

Co-authored-by: RongtongJin <jinrongtong16@mails.ucas.ac.cn>
 2023-10-14 11:04:04 +08:00
shriVATSA54 c36bb78e85 [ISSUE 7313] Enhancement Optimization Method name (#7420) 
* Enhancment/method_name/#7313/

* Enhancment/method_name/#7313/

* Enhancment/method_name/#7313/
 2023-10-07 14:52:39 +08:00
lk 73b3fde837 [maven-release-plugin] prepare for next development iteration (#7379) 2023-09-22 10:08:59 +08:00
lk 1a681bdf9b [maven-release-plugin] prepare release rocketmq-all-5.1.4 (#7377) 2023-09-21 19:58:29 +08:00
Jixiang Jin b8610d87bb Replace loggingMetricExporter with OtlpJsonLoggingMetricExporter. (#7373) 
* Replace loggingMetricExporter with OtlpJsonLoggingMetricExporter.

* Fix bazel workspace

* Move OtlpJsonLoggingMetricExporter to proxy and controller.

* Fix Bazel imports.
 2023-09-21 16:21:44 +08:00
lizhimins 831fcc76cd [ISSUE #7363] Fix get message from tiered storage return incorrect next pull offset (#7365) 2023-09-19 10:04:04 +08:00
lizhimins fd32dae2ab [ISSUE #6633] Not clear uninitialized files and fix metadata recover (#7342) 2023-09-12 19:58:08 +08:00
yuz10 1a8e7cb17c [ISSUE #7345] Fix wrong result of searchOffset in tiered storage 2023-09-12 19:33:41 +08:00
guyinyou c100d815d7 [ISSUE #7328] Convergent thread pool creation (#7329) 
* Convergence thread pool creation to facilitate subsequent iteration management

* Convergence thread pool creation in ThreadPoolMonitor.java

* fix unit test

* Convergence ThreadPool constructor

* Convergence ScheduledThreadPool constructor

* remove unused import

* Convergence ScheduledThreadPool constructor

* remove unused import

---------
 2023-09-11 10:13:56 +08:00
lizhimins e11e29419f [ISSUE #7308] Adding topic blacklist and filter in tiered storage module (#7310) 2023-09-06 20:42:24 +08:00
lizhimins c78061bf6c [ISSUE#7280] Fix and refactor handle commit exception in tiered storage (#7281) 
* refactor handle commit exception

* refactor handle commit exception

* fix handle commit exception
 2023-08-30 22:29:51 +08:00
lizhimins 9bb73b9a38 [#ISSUE 7222] Bug fix and refactoring of the Indexfile in tiered storage (#7224) 2023-08-23 09:46:27 +08:00
Jixiang Jin 50d1050437 To config the cardinalityLimit for openTelemetry metrics exporting and fix logging config for metrics (#7196) 2023-08-16 21:15:00 +08:00
lizhimins b2deef179d [ISSUE #7144] Accelerate the recovery speed of the tiered storage module (#7145) 2023-08-10 10:38:47 +08:00
fujian-zfjandLi Zhanhui 3a6ef0400c [ISSUE #7064] [RIP-66-1] Support KV(RocksDB) Storage for Metadata (#7092) 
* typo int readme[ecosystem]

* rocksdb metadata

* add unit test

* fix testOffsetPersistInMemory

* fix unit test

* fix unit test

* remove unused import

* move RocksDBOffsetSerialize to broker moudle

* Fix bazel build scripts

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* Flag QueryMsgByKeyIT as flaky as it fails at frequency: 5 out of 32

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* change public to private of some inner method

---------

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
 2023-08-05 00:32:11 +08:00
koado 1fe5d62334 [ISSUE #7074] Allow a BoundaryType to be specified when retrieving offset based on the timestamp (#7082) 
* add new interface for searching offset with boundary type

* format code

* fix failed test

* unify two BoundaryType class

* add interface getOffsetInQueueByTime(long timestamp, BoundaryType boundaryTYpe) in ConsumeQueueInterface

* fix AdminBrokerProcessorTest unnecessary Mockito stubbings
 2023-08-03 11:40:16 +08:00
lizhimins a1bf49d5d0 [ISSUE #7093] Avoid dispatch tasks too much cause dispatch task failed (#7094) 
* Avoid dispatch tasks too much cause dispatch task failed

* set schedule task async
 2023-08-01 15:56:34 +08:00
lizhimins 2c5808b9fd [#ISSUE 7035] Fix correct min offset behavior in tiered storage (#7038) 2023-07-19 20:14:02 +08:00
lizhimins 33cb22e1c0 [ISSUE #7018] fix append in tiered storage when message offset incorrect (#7019) 
* fix append in tiered storage when message offset incorrect
 2023-07-13 15:37:24 +08:00
lizhimins 79967c00b2 [ISSUE #6933] Optimize delete topic in tiered storage (#6973) 2023-06-30 15:54:32 +08:00
lizhimins c96a0b5665 [ISSUE #6933] Support delete expired or damaged file in tiered storage and optimize fetch code (#6952) 
If cq dispatch smaller than local store min offset, do self-healing logic for storage and rebuild automatically
 2023-06-27 17:53:43 +08:00
rongtong e369d7deac [maven-release-plugin] prepare for next development iteration (#6939) 2023-06-24 14:31:42 +08:00
rongtong f7425f52f9 [maven-release-plugin] prepare release rocketmq-all-5.1.3 (#6938) 2023-06-24 12:23:11 +08:00
lizhimins 9899be15e7 [ISSUE #6933] Support recreate file if local cq and tiered storage offset not match 2023-06-23 15:37:57 +08:00
rongtong 1f0cdc88f8 Temporarily ignoring testQueryMessageAsync testing on the Linux platform (#6930) 2023-06-22 19:26:07 +08:00
rongtong ad4cc957fb [maven-release-plugin] prepare for next development iteration (#6895) 2023-06-12 19:12:25 +08:00
rongtong 749967baa8 [maven-release-plugin] prepare release rocketmq-all-5.1.2 (#6892) 2023-06-12 17:44:39 +08:00