* typo int readme[ecosystem]
* rocksdb metadata
* add unit test
* fix testOffsetPersistInMemory
* fix unit test
* fix unit test
* remove unused import
* move RocksDBOffsetSerialize to broker moudle
* Fix bazel build scripts
Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
* Flag QueryMsgByKeyIT as flaky as it fails at frequency: 5 out of 32
Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
* change public to private of some inner method
---------
Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
* support s3 backend in tiered storage
* refactor(tieredstorage): Unify all object storage configuration properties
1. Unify all object storage configuration properties
* refactor(tieredstorage): replace some lambda function with more simple expression
1. replace some lambda function with more simple expression
* style(tieredstorage): perfect comments on ChunkMetadata
1. perfect comments on ChunkMetadata
* refactor(tieredstorage): perfect lambda expression
1. perfect lambda expression
* fix(tieredstorage): fix unmatched config attributes in brokerS3.conf
1. fix unmatched config attributes in brokerS3.conf
* feat(tieredstorage): More context in logging output
1. More context in logging output
* fix(tieredstorage): fix wrong concurrently put
1. fix wrong concurrently put
* test(tieredstorage): add UT to verify TieredFileSegmentInputStream
1. add UT to verify TieredFileSegmentInputStream
* refactor(tieredstorage): better code placement
1. better code placement
* refactor(tieredstorage): refactor TieredFileSegmentInputStream for better understandability
1. refactor TieredFileSegmentInputStream for better understandability
* feat(tieredstorage): support `reset` of TieredFileSegmentInputStream
1. support `reset` of TieredFileSegmentInputStream
* fix(tieredstorage): fix wrong position when failed in `S3FileSegment#commit0`
1. fix wrong position when failed in `S3FileSegment#commit0`
* fix(tieredstorage): fix still have upload buffer when already seal the segment
1. fix still have upload buffer when already seal the segment
* test(tieredstorage): fix wrong assertion
1. fix wrong assertion
* feat(tieredstorage): support switch to enable merge chunks into segment
1. support switch to enable merge chunks into segment
* feat(tieredstorage): add more debug log in TieredMessageStore
1. add more debug log in TieredMessageStore
* style(tieredstorage): use rmq code style
1. use rmq code style
* feat(tieredstorage): add metrics for S3 provider
1. add metrics for S3 provider
* fix(tieredstorage): resolve conflicts after rebasing master
1. resolve conflicts after rebasing master
Closes https://github.com/apache/rocketmq/issues/6624
* style(tieredstorage): change log level
1. change log level
Closes https://github.com/apache/rocketmq/issues/6154
* build(controller): build tieredstorage with bazel
1. build tieredstorage with bazel
* build(controller): build tieredstorage with bazel
1. build tieredstorage with bazel
* style(tieredstorage): change log level
1. change log level
Closes https://github.com/apache/rocketmq/issues/6154
* test(tieredstorage): ignore tests about S3Mock
1. ignore tests about S3Mock
* test(tieredstorage): ignore tests about S3Mock
1. ignore tests about S3Mock
* refactor: simplify getPID (#5962)
* [ISSUE #5923] Add example tiered storage backend service provider (#5926)
* implement example file segment
* add metrics
* add readme
* fix license
* fix tests
* fix links in README.md
* add comment to PosixFileSegment and mark as experimental
* fix test
* optimize image quality
* Remove the useless exception class: MQRedirectException #5963
* [ISSUE #5965] Fix lmqTopicQueueTable initialization (#5968)
* [ISSUE #5965] Fix lmqTopicQueueTable initialization
* [ISSUE #5965] Fix lmqTopicQueueTable initialization
* [ISSUE #5890] Fix dledger logging (#5959)
* Fix dledger logging
* Add bridge into store module
* [ISSUE #5860] Set the value of order when create or update topic (#5861)
* [ISSUE #5939]Adjust the MQClientInstance#sendHeartbeatToAllBroker catch code block log print level from info to warn (#5940)
* [ISSUE #5924] Optimize UtilAll#sleep method (#5925)
* [ISSUE #5924]Optimize UtilAll#sleep method
* polish code
* [ISSUE #5986] optimize the BrokerOuterAPITest class code
Co-authored-by: zhouyunpeng <2474138779@qq.com>
* [ISSUE #5971] Make the internal logs related to the dledger in the controller print to a file separately (#5972)
* Make the internal logs related to the dledger in the controller print to a file separately
* Make the internal logs related to the dledger in the controller print to a file separately
* [ISSUE #5969] Remvoe duplicate deleteUnusedStats in admin processor (#5973)
* [ISSUE #5847] Add checkBlock for hasMsgFromQueue
* [ISSUE #5983] Make consumer support flow control code better (#5984)
* When encountering the flow control code, pull it after 20ms instead of 3s
* When encountering the flow control code, pull it after 20ms instead of 3s
* [ISSUE #5896] feat:add pop consumer example (#5991)
* feat:add pop consumer
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* Update example/src/main/java/org/apache/rocketmq/example/simple/PushConsumer.java
Co-authored-by: Oliver <wqdyxnbd@163.com>
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
* feat:fix
---------
Co-authored-by: mahaitao617 <mahaitao617@mahaitao617deMacBook-Pro.local>
Co-authored-by: Oliver <wqdyxnbd@163.com>
* [ISSUE #5942] Fix the produce count include the quantity of the system topic(#5943)
* [ISSUE #5999] Fix the TopicQueueMappingUtils comments typo (#6000)
* [ISSUE #5996] Optimize the RemotingSerializable class code (#5998)
* simplified RemotingSerializable null check
* optimize the RemotingSerializable class code
* [ISSUE #5994] [RIP-46] add pop and timer metrics (#5995)
* add pop and timer metrics
* fix according to review comment
* test(broker): add ReplicasManagerRegisterTest to test the register process
1. add ReplicasManagerRegisterTest to test the register process
* chore(pom): modify pom.xml to replace mockito with powermock
1. modify pom.xml to replace mockito with powermock
* build(bazel): export powermock in bazel
1. export powermock in bazel
---------
Co-authored-by: Xinda <xdshent@gmail.com>
Co-authored-by: SSpirits <admin@lv5.moe>
Co-authored-by: loboxu <loboxu@tencent.com>
Co-authored-by: pingww <pingw002@gmail.com>
Co-authored-by: Aaron Ai <yangkun.ayk@gmail.com>
Co-authored-by: Slideee <yechun@corp.netease.com>
Co-authored-by: mxsm <ljbmxsm@gmail.com>
Co-authored-by: hardyfish <85128645+hardyfish@users.noreply.github.com>
Co-authored-by: zhouyunpeng <2474138779@qq.com>
Co-authored-by: rongtong <jinrongtong5@163.com>
Co-authored-by: zhiliatom <87265072+zhiliatom@users.noreply.github.com>
Co-authored-by: zhouxiang <zhouxiang.zzx@alibaba-inc.com>
Co-authored-by: mahaitao <15828010639@163.com>
Co-authored-by: mahaitao617 <mahaitao617@mahaitao617deMacBook-Pro.local>
Co-authored-by: Oliver <wqdyxnbd@163.com>
* Add long polling
* Change rocketmq-proto version to 2.0.2
* fix checkstyle
* Fix rocketmq-proto version
Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
* Change pollTime to timeRemaining
* fix test
Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
* Remove ClientLogger
* WIP
* WIP
* Rename logger to log
* Make it compile
* WIP
* Fix bazel to use shaded slf4j
* Fix DeleteExpiredCommitLogSubCommandTest
* Fix pom.xml duplication
* Fix maven deps
* Add test logback configuration file
* Fix unit test output
* Fix logback configuration file
* All logging are made on top of slf4j
* Fix test log configuration file name
* Fix test log configuration file name for test module
* All logging are shaded slf4j targeted
* DLedger has an explicit dependency on slf4j
* Fix DLedger
* Fix DLeader issue
* Move logback configuration files to each module
Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
* update org.yaml:snakeyaml 1.31 to 1.32
* Revert pom.xml format
* Revert space stripping
* Show failed tests errors in GitHub action logs
Co-authored-by: Zhanhui Li <lizhanhui@apache.org>
lk