Commit Graph
298 Commits
Author SHA1 Message Date
hiyoandLi Zhanhui 0faa0a2c7d [ISSUE #6414] polish MQClientAPIImpl.getDefaultTopicRouteInfoFromName… (#6452)
* [ISSUE #6414] polish MQClientAPIImpl.getDefaultTopicRouteInfoFromNameServer parameters

* Fix to make Bazel CI pass

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* Fix typo

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

---------

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
2023-03-23 20:30:21 +08:00
TheR1sing3un 19eafc80ff fix(broker): fix wrong test
1. fix wrong test
2023-03-14 09:40:15 +08:00
RongtongJin bbf54f8a71 Modify StorePathMetadata to StorePathBrokerIdentity 2023-03-14 09:40:15 +08:00
RongtongJin 3a141868b7 Fix some typos and log output 2023-03-14 09:40:15 +08:00
TheR1sing3un 5a2aea7997 refactor(controller): remove meaningless attribute MessageStoreConfig#storePathTempMetadata
1. remove meaningless attribute
`MessageStoreConfig#storePathTempMetadata`
2023-03-14 09:40:15 +08:00
TheR1sing3un 5460c77622 test(test): add AutoSwitchRoleIntegration#testBasicWorkWhenControllerShutdown
1. add AutoSwitchRoleIntegration#testBasicWorkWhenControllerShutdown
2023-03-14 09:40:15 +08:00
TheR1sing3un c494a43c52 refactor(controller): optimize some code suggested by reviewer
1. optimize some code suggested by reviewer
2023-03-14 09:40:15 +08:00
TheR1sing3un 46668b4eaf test(broker): add test to verify broker restart with changed address
1. add test to verify broker restart with changed address
2023-03-14 09:40:15 +08:00
+6 3a69aae701 [ISSUE #6023] Add a unit test to verify new register process in broker with controller mode (#6024)
* refactor: simplify getPID (#5962)

* [ISSUE #5923] Add example tiered storage backend service provider (#5926)

* implement example file segment

* add metrics

* add readme

* fix license

* fix tests

* fix links in README.md

* add comment to PosixFileSegment and mark as experimental

* fix test

* optimize image quality

* Remove the useless exception class: MQRedirectException #5963

* [ISSUE #5965] Fix lmqTopicQueueTable initialization (#5968)

* [ISSUE #5965] Fix lmqTopicQueueTable initialization

* [ISSUE #5965] Fix lmqTopicQueueTable initialization

* [ISSUE #5890] Fix dledger logging (#5959)

* Fix dledger logging

* Add bridge into store module

* [ISSUE #5860] Set the value of order when create or update topic (#5861)

* [ISSUE #5939]Adjust the MQClientInstance#sendHeartbeatToAllBroker catch code block log print level from info to warn (#5940)

* [ISSUE #5924] Optimize UtilAll#sleep method (#5925)

* [ISSUE #5924]Optimize UtilAll#sleep method

* polish code

* [ISSUE #5986] optimize the BrokerOuterAPITest class code

Co-authored-by: zhouyunpeng <2474138779@qq.com>

* [ISSUE #5971] Make the internal logs related to the dledger in the controller print to a file separately (#5972)

* Make the internal logs related to the dledger in the controller print to a file separately

* Make the internal logs related to the dledger in the controller print to a file separately

* [ISSUE #5969] Remvoe duplicate deleteUnusedStats in admin processor (#5973)

* [ISSUE #5847] Add checkBlock for hasMsgFromQueue

* [ISSUE #5983] Make consumer support flow control code better (#5984)

* When encountering the flow control code, pull it after 20ms instead of 3s

* When encountering the flow control code, pull it after 20ms instead of 3s

* [ISSUE #5896] feat:add pop consumer example (#5991)

* feat:add pop consumer

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* Update example/src/main/java/org/apache/rocketmq/example/simple/PushConsumer.java

Co-authored-by: Oliver <wqdyxnbd@163.com>

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

* feat:fix

---------

Co-authored-by: mahaitao617 <mahaitao617@mahaitao617deMacBook-Pro.local>
Co-authored-by: Oliver <wqdyxnbd@163.com>

* [ISSUE #5942] Fix the produce count include the quantity of the system topic(#5943)

* [ISSUE #5999] Fix the TopicQueueMappingUtils comments typo (#6000)

* [ISSUE #5996] Optimize the RemotingSerializable class code (#5998)

* simplified RemotingSerializable null check

* optimize the RemotingSerializable class code

* [ISSUE #5994] [RIP-46] add pop and timer metrics (#5995)

* add pop and timer metrics
* fix according to review comment

* test(broker): add ReplicasManagerRegisterTest to test the register process

1. add ReplicasManagerRegisterTest to test the register process

* chore(pom): modify pom.xml to replace mockito with powermock

1. modify pom.xml to replace mockito with powermock

* build(bazel): export powermock in bazel

1. export powermock in bazel

---------

Co-authored-by: Xinda <xdshent@gmail.com>
Co-authored-by: SSpirits <admin@lv5.moe>
Co-authored-by: loboxu <loboxu@tencent.com>
Co-authored-by: pingww <pingw002@gmail.com>
Co-authored-by: Aaron Ai <yangkun.ayk@gmail.com>
Co-authored-by: Slideee <yechun@corp.netease.com>
Co-authored-by: mxsm <ljbmxsm@gmail.com>
Co-authored-by: hardyfish <85128645+hardyfish@users.noreply.github.com>
Co-authored-by: zhouyunpeng <2474138779@qq.com>
Co-authored-by: rongtong <jinrongtong5@163.com>
Co-authored-by: zhiliatom <87265072+zhiliatom@users.noreply.github.com>
Co-authored-by: zhouxiang <zhouxiang.zzx@alibaba-inc.com>
Co-authored-by: mahaitao <15828010639@163.com>
Co-authored-by: mahaitao617 <mahaitao617@mahaitao617deMacBook-Pro.local>
Co-authored-by: Oliver <wqdyxnbd@163.com>
2023-03-14 09:40:15 +08:00
TheR1sing3un f843851788 fix(controller): fix some bug to pass AutoSwitchRoleIntegrationTest
1. fix some bug to pass AutoSwitchRoleIntegrationTest
2023-03-14 09:40:15 +08:00
TheR1sing3un ea71dc545a fix(controller): fix some bug about ignoring new-add event type
1. fix some bug about ignoring new-add event type
2023-03-14 09:40:15 +08:00
mxsm 543c118200 [ISSUE #6060] Optimize AutoSwitchRoleBase#nextPort method (#6068)
* [ISSUE #6060]Optimize AutoSwitchRoleBase#nextPort method

* polish code

* polish code style
2023-03-02 20:01:02 +08:00
hardyfish 33371e5d8e [ISSUE #6145] Modify Collections.sort usage (#6146)
* modify Collections.sort usage

* modify Collections.sort usage
2023-02-28 13:56:03 +08:00
rongtong b24682619c Ignore AutoSwitchRoleIntegrationTest temporarily because it is still flaky (#6144) 2023-02-21 14:01:37 +08:00
zhouxiang 0f878b5157 [maven-release-plugin] prepare for next development iteration 2023-02-16 12:35:58 +08:00
zhouxiang b8e9334d2a [maven-release-plugin] prepare release rocketmq-all-5.1.0 2023-02-16 12:35:58 +08:00
zhouxiang 83d6778747 [ISSUE #6030] Update version to 5.1.0-SNAPSHOT 2023-02-16 12:35:58 +08:00
hardyfish bf3344386b [ISSUE #6073] remove static reference 2023-02-16 08:37:34 +08:00
nowinkey 50651ef8dd Add integration test 2023-02-14 13:50:47 +08:00
nowinkey dcf84e4125 Add integration test and add enableBuildConsumeQueueConcurrently attribute to BrokerConfig class 2023-02-14 13:50:47 +08:00
Zhouxiang ZhanandLi Zhanhui 83c4a8f3f0 Support long polling in rocketmq proxy in the protocol (#5788)
* Add long polling

* Change rocketmq-proto version to 2.0.2

* fix checkstyle

* Fix rocketmq-proto version

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

* Change pollTime to timeRemaining

* fix test

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
2022-12-30 22:31:01 +08:00
SSpiritsandLi Zhanhui 9f8e85bb76 [ISSUE #5754] [RIP-57] Add asynchronous interfaces to MessageStore (#5755)
* [RIP-57] Add asynchronous interfaces to MessageStore

* fix unit test

* Modified according to review comments

* Remove unnecessary stubbing

* Add some other methods to MessageStore

* fix bazel test

* Skip failing test case

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>

Signed-off-by: Li Zhanhui <lizhanhui@gmail.com>
Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
2022-12-29 09:42:45 +08:00
xiaoyifang 4fdf91f446 [ISSUE #5726] VerifyDelay should break the loop when matched the delayed message. (#5727) 2022-12-18 15:23:16 +08:00
zhouxiang 23c8efd801 [ISSUE #5485] Fix GrpcBaseIT 2022-12-09 19:53:18 +08:00
SSpirits 774bc67520 optimize LagCalculationIT 2022-12-09 16:04:26 +08:00
rongtong cc6614c236 Fix bug that make static topic ITs unable to pass (#5666) 2022-12-08 17:52:51 +08:00
lizhimins 49fc6eded9 [ISSUE #5568] Support long length group and topic for pop mode consumption (#5611)
* [ISSUE #5568] Support long length group and topic for pop mode consumption
Co-authored-by: 斜阳 <terrance.lzm@alibaba-inc.com>
2022-12-07 10:26:50 +08:00
rongtong 59acc7d355 Try to make AutoSwitchRoleIntegrationTest more stable (#5638) 2022-12-04 12:17:43 +08:00
Zhanhui Li 322e00a475 [ISSUE #5617] Fix Bazel warning & dependency (#5618)
* Fix Bazel indirect deps warning

* Remove unused deps
2022-12-01 15:53:30 +08:00
SSpiritsandLi Zhanhui d3de48c806 [ISSUE #5605]Introduce tag estimation for lag calculation (#5606)
* Introduce tag estimation for lag calculation

Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>

* fixed according to review comments

Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
2022-11-30 14:52:59 +08:00
SSpirits 08dbe7ede9 fix flaky test (#5591) 2022-11-25 10:06:31 +08:00
Aaron Ai 4469fb51dd Update group id of rocketmq logging (#5564) 2022-11-22 11:31:21 +08:00
rongtong ee37e3a55f [ISSUE #5471] Fix AutoSwitchRoleIntegrationTest still experience random failures (#5475)
* Make AutoSwitchRoleIntegrationTest more stable

* Pass the checkstyle

* Pass the checkstyle

* Fix compatibility issues

* Simple optimization

* Remove useless import

* Pass the check style

* Pass the check style

* Modify DEFAULT_FILE_SIZE

* Modify DEFAULT_FILE_SIZE

* test

* Remove useless import

* Add more output to debug

* Use the new directory every time when start a broker

* Use the new directory every time when start a broker

* Remove console output

* Remove console output
2022-11-22 09:34:30 +08:00
Aaron Ai a68078fef8 [ISSUE #5484] Replace logging module with the shaded logback (#5540)
* Replace logging module with the shaded logback

* Add license header

* Fix bazel compile issue
2022-11-21 16:30:15 +08:00
yukon 506b563f55 Revert "[ISSUE #5484] Replace Logging Module with Shaded Logback (#5524)" (#5537)
This reverts commit 809ff6b3e0.
2022-11-17 20:04:17 +08:00
Aaron AiandLi Zhanhui 809ff6b3e0 [ISSUE #5484] Replace Logging Module with Shaded Logback (#5524)
* Remove ClientLogger

* WIP

* WIP

* Rename logger to log

* Make it compile

* WIP

* Fix bazel to use shaded slf4j

* Fix DeleteExpiredCommitLogSubCommandTest

* Fix pom.xml duplication

* Fix maven deps

* Add test logback configuration file

* Fix unit test output

* Fix logback configuration file

* All logging are made on top of slf4j

* Fix test log configuration file name

* Fix test log configuration file name for test module

* All logging are shaded slf4j targeted

* DLedger has an explicit dependency on slf4j

* Fix DLedger

* Fix DLeader issue

* Move logback configuration files to each module

Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
2022-11-17 00:38:28 +08:00
SSpiritsandLi Zhanhui ec5d3236f6 invert dependency between remoting and common module (#5503)
* invert dependency between remoting and common module

* fix bazel

* Fix Bazel warning of indirect dependency

* remove duplicate class ServiceThread

* fix conflict

* revert delete class ServiceThread because of introduced by dledger

Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
2022-11-11 13:34:31 +08:00
lizhiminsand斜阳 2113c16ebc [RIP-48] Support reset offset in server-side for pop message (#5457)
* [RIP-48] Support reset offset in server-side for pop message

* [RIP-48] Support reset offset in server-side for pop message

* no need remove error log

Co-authored-by: 斜阳 <terrance.lzm@alibaba-inc.com>
2022-11-07 13:55:03 +08:00
SSpiritsandLi Zhanhui 350e433acc implement broker stats metrics and request metrics (#5449)
* implement broker stats metrics and request metrics

* fix bazel

* PopBigMessageIT potentially needs more time

* Fix bazel deps warning

Co-authored-by: Li Zhanhui <lizhanhui@gmail.com>
2022-11-02 20:42:42 +08:00
lk 58c79c9f13 [ISSUE #5365] [RIP 51] Add notification mechanism for pop orderly (#5387)
* [ISSUE #5365] Add notification mechanism for pop orderly

* [ISSUE #5399] add guava dependency for the test of store in BUILD.bazel
2022-10-31 13:32:14 +08:00
lk d54f8cb1eb [ISSUE #5402] not to decompress the body of messages when get retry messages in PopReviveService (#5403) 2022-10-27 15:32:51 +08:00
a7c1d49454 [ISSUE #5356] [RIP 48] Introduces the concept of ready and inflight messages (#5357)
* Introduces the concept of ready and inflight messages

* update consumer progress subcommand

* fix consumer pull offset is less than consumer offset

* remove unused import

Co-authored-by: 斜阳 <terrance.lzm@alibaba-inc.com>
Co-authored-by: ShadowySpirits <admin@lv5.moe>
2022-10-26 15:41:41 +08:00
rongtong 0e3f8b4d72 Fix the issue that TransactionMessageIT can not pass (#5394) 2022-10-24 18:45:05 +08:00
caigy 4af193e325 [ISSUE #5012] Support escaping transactional messages (#5062)
* support escaping transactional messages

* add integration test

* increase the number of messages and size integration test

* reorder condition and remove sysout

* add mock in unit test

* fix checkstyle

* ignore integration test by default

* use PutMessageResult.isOk() to check the result of escaping messages
2022-10-24 14:28:00 +08:00
lk 03ffc45a8b [ISSUE #5364] Support changeInvisibleTime for pop orderly (#5367) 2022-10-24 10:44:06 +08:00
mxsm 06a5496933 [ISSUE #5324]Replace deprecated class PosixParser with DefaultParser for CLI (#5325)
* [ISSUE #5324]Replace deprecated class PosixParser with DefaultParser for CLI

* fix bazel compile UpdateAccessConfigSubCommandTest testExecute failures

* Fixed bazel compile UpdateAccessConfigSubCommandTest testExecute failures
2022-10-19 22:43:06 +08:00
lizhiminsand斜阳 d663bcf6aa [ISSUE #5292] [RIP-48] Support reset offset in server-side to improve the success rate (#5293)
* [RIP-48] Support reset offset in server side to improve the success rate

* [RIP-48] Support reset offset in server side to improve the success rate

* [RIP-48] Fix code style

* fix unit test

* no need put offset in reset command header

Co-authored-by: 斜阳 <terrance.lzm@alibaba-inc.com>
2022-10-14 14:02:08 +08:00
NowinkeyandZhanhui Li 53e0847528 [ISSUE #5204] Enable checkstyle for test code (all module) (#5297)
* delete the checkstyle plugin for child modules

* [ISSUE #5204] Enable checkstyle for test code (all module)

* Exclude generated files

Co-authored-by: Zhanhui Li <lizhanhui@apache.org>
2022-10-14 10:58:48 +08:00
rongtong 12e29df6db Fix the issue that AutoSwitchRoleIntegrationTest is flaky (#5277) 2022-10-11 18:44:22 +08:00
Zhanhui Li 94d8e55012 Fix NPE in CI (#5273) 2022-10-11 17:27:51 +08:00