145 Commits

Author SHA1 Message Date
TheR1sing3un de7cd0df10 [ISSUE#4959] Fix the logic when deal with a write event with empty events
* fix(controller): fix the logic when deal with a write event with empty events

1. fix the logic when deal with a write event with empty events

* style(controller): remove unused import

1. remove unused import
2022-09-01 19:48:05 +08:00
TheR1sing3un f043ca8c18 [ISSUE#4950] fix wrong typo about DLedger (#4951)
* style(controller): fix the wrong typo about DLedger

1. fix the wrong typo about DLedger

* style(controller): fix the wrong typo about DLedger

1. fix the wrong typo about DLedger
2022-08-31 20:11:17 +08:00
Aaron Ai f23fdbe016 Replace findbugs by spotbugs (#4931) 2022-08-30 17:45:56 +08:00
TheR1sing3un 7f25e453cd style(controller): fix the wrong typo about DLedger (#4930)
1. fix the wrong typo about DLedger
2022-08-30 13:38:26 +08:00
TheR1sing3un 6a8b1aa96b [ISSUE#4917] Optimize the retry logic in appending a no-op request when controller change to leader (#4918) 2022-08-30 08:43:03 +08:00
mxsm a961d1b44a [ISSUE #4817] Add a command to clear broker data from controller for CLI (#4823) 2022-08-27 19:42:55 +08:00
mxsm 64c18fb0db [ISSUE #4889]Fix controller test:DLedgerControllerTest#testChangeControllerLeader (#4892) 2022-08-26 09:06:02 +08:00
mxsm 5578808b44 [ISSUE #4873]Optimize controller module test cases execution time (#4882) 2022-08-25 14:43:44 +08:00
mxsm 0909aa91a2 [ISSUE #4857] Fix broker can't auto register to controller again (#4862) 2022-08-25 10:03:57 +08:00
Zhanhui Li 3da7ea9c1b [Infra Improvement] Build And Run Unit Tests in Parallel (#4878)
* Support build with Bazel (#4865)

* Support build with Bazel, fixing tests to make them capable of running in concurrency and hermetic.

* Make test cases capable of running in parallel. (#4874)
2022-08-24 20:53:30 +08:00
RongtongJin bbbb96e129 Revert "Support build with Bazel (#4865)"
This reverts commit ec8a93d2f7.
2022-08-23 14:29:46 +08:00
Zhanhui Li ec8a93d2f7 Support build with Bazel (#4865)
* Support build with Bazel, fixing tests to make them capable of running in concurrency and hermetic.
2022-08-23 13:11:53 +08:00
mxsm d117742e3c [ISSUE #4847]Optimize DefaultBrokerHeartbeatManager#onBrokerHeartbeat mehtod code logic (#4848) 2022-08-20 09:54:04 +08:00
TheR1sing3un c5f80713be [ISSUE #4813] Add elect policy for controller (#4809)
* feat(controller): add elect policy

1. add epoch and maxOffset in heartbeat.
2. refactor elect logic, now we
elect a new master by elect policy(can expand).
3. add some unit tests

* refactor(controller): refactor some code and format some code

1. refactor some code and format some code

* fix typo in ReplicasInfoManager

1. fix typo in ReplicasInfoManager

* fix wrong method call in ReplicasManagerTest#before

1.fix wrong method call in ReplicasManagerTest#before

* fix wrong call in ReplicasManagerTest#before

1. fix wrong call in ReplicasManagerTest#before

* fix(controller): fix the bug about ReElectMaster

1. fix the bug about ReElectMaster
2. fix some typo

* style(controller): fix typo by checkstyle

1. fix typo by checkstyle

* test(broker): fix invalid usage of "any()" in DLedegerControllerTest#before

1. fix invalid usage of "any()" in DLedegerControllerTest#before

* fix(controller): fix some wrong usage in test

1. fix some wrong usage in test
2. refactor some field type to avoid NPE
2022-08-16 08:41:51 +08:00
mxsm 06ca14a508 [ISSUE #4791] Add elect master command for admin CLI (#4798) 2022-08-11 10:06:46 +08:00
mxsm ce274265db [ISSUE #4794] Fix deserialize ElectMasterResponseHeader attribute value of brokerMemberGroup is null (#4796) 2022-08-08 17:11:14 +08:00
rongtong 0b6563b58f Add getControllerConfig and updateControllerConfig command for admin CLI (#4793) 2022-08-07 15:28:51 +08:00
mxsm c506036cb6 [ISSUE #4745] Check controllerDLegerPeers and controllerDLegerSelfId attributes before create DLedgerServer (#4768) 2022-08-04 11:16:17 +08:00
mxsm c6b3a66ecf [ISSUE #4706] fix console printed log format is incorrect (#4716) 2022-08-03 09:19:03 +08:00
mxsm bcd4a92e41 [ISSUE #4750] Fix ControllerRequestProcessor#processRequest handle CONTROLLER_GET_SYNC_STATE_DATA return incorrect (#4759) 2022-08-02 08:46:41 +08:00
mxsm b875b3309a [ISSUE #4709] fix method is on the same line as @Override annotation (#4717)
Co-authored-by: Aaron Ai <yangkun.ayk@gmail.com>
2022-08-01 20:42:34 +08:00
mxsm c489a70baa [ISSUE #4734]Fix DLedgerController startScheduling concurrency problem (#4735) 2022-07-30 20:54:06 +08:00
mxsm a58b2443a4 [ISSUE #4700]Optimize print information when the rocketmqHome variable not set (#4703) 2022-07-28 09:02:41 +08:00
RongtongJin e095914869 Change rocketmq version to 5.0.0-SNAPSHOT in controller pom.xml 2022-07-12 14:46:18 +08:00
RongtongJin 9dd34df413 Merge branch '5.0.0-beta-dledger-controller' of github.com:apache/rocketmq into 5.0.0-beta-dledger-controller
# Conflicts:
#	pom.xml
2022-07-12 11:50:56 +08:00
RongtongJin d47f7024fe Polish some pom.xml file and remove useless dependency 2022-07-12 11:17:11 +08:00
RongtongJin 4be6e2c1ee Upgrade dledger version to 0.2.7 2022-07-08 20:22:45 +08:00
RongtongJin ab4f7ddbcc Pass the CI after merge develop 2022-07-08 17:55:11 +08:00
RongtongJin 5550266765 1.Rename BrokerRegisterRequestHeader to RegisterBrokerToControllerRequestHeader
2.Make ChangeToMaster do all things even if it is already master
2022-06-29 10:49:16 +08:00
RongtongJin fd8dafd5f9 ReplicasInfoManager use controller config instead of enableElectUncleanMaster so that it can be modified at run time 2022-06-17 15:50:20 +08:00
rongtong d9273e70f6 Update the logic of confirmOffset and recover topicQueueTable to prevent message loss when broker role change (#4460)
* Update the logic of confirmOffset and recover topicQueueTable to prevent message loss

* Do not dispatch when reputFromOffset + size is greater than confirmOffset to prevent message loss

* Pass the UT

* Pass the UT
2022-06-15 15:53:10 +08:00
hzh0425 5eaeeabf51 [Summer of code] Let broker register to controller again when master not existed. (#4450)
* register to controller if master is null

* stop electmaster when the old master is active

* review

* fix test bugs for brokerAlivePredicate
2022-06-12 16:01:08 +08:00
hzh0425 6645d0a42b [Summer of code] Let controller become role state after append initial logs (#4442)
* fix bug

* fix bug
2022-06-10 13:59:37 +08:00
RongtongJin ebce387057 Add notifyBrokerRoleChanged configuration 2022-06-07 17:40:40 +08:00
hzh0425 98a283f5ae [Summer of code] Let controller inform broker that role changed. (#4424)
* add broker api --notifyBrokerRoleChanged --

* add broker api --notifyBrokerRoleChanged --

* let controller inform broker when role changed

* code reivew
2022-06-07 16:51:14 +08:00
rongtong e7bf51f4c2 Fix ha controller bugs (#4422)
* Fix bug that do not remove caughtUpTime in connectionCaughtUpTimeTable

* Polish the comment

* Remove replicas from syncStateSet if connection disconnect and ha service not shutdown
2022-06-06 20:53:08 +08:00
hzh0425 8515a8d091 [Summer of code] Reuse dledger remotingServer in controller mode. (#4409)
* reuse dledger remotingServer in Controller

* reuse dledger remotingServer in Namesrv when startup controller

* fix some bugs in controller

* trigger ci

* code review

* fix controllerManagerTest bug
2022-06-05 15:23:25 +08:00
rongtong f27800a134 Polish switching logic and auto switch ha code (#4406)
* Polish switching logic and auto switch ha code

* Make UT can pass

* Polish the code
2022-06-04 22:42:01 +08:00
hzh0425 05841188e2 [Summer of code] Add admin tool getControllerMetadata (#4405)
* add tool get controller metadata

* fix bug
2022-06-04 16:36:40 +08:00
RongtongJin addd307e01 Remove useless remotingClient in ControllerManager 2022-06-02 22:21:50 +08:00
RongtongJin 6f3b372e83 Polish naming convention of controller 2022-06-01 14:59:24 +08:00
hzh0425 e0121ca1da [Summer of code] Add admin tools for controller mode (#4388)
* add tool getSyncStateDataCommand

* get controller leaderAddr when execute command

* modify getControllerMetadata api

* code review

* code review

* add tool get brokerEpochCache

* init command

* set lastEpochEndOffset

* take maxPhyOffset in EpochCache
2022-06-01 11:05:48 +08:00
hzh0425 97e6cecbf9 [Summer of code] Fix some bugs in controller mode.
* modify pom for using dledger

* rename option

* 1.send heartbeat to haclient when get epochEntry failed to hold connection.
2.update lastCatchupTimeMs in time.

* fix some bugs
2022-05-26 20:44:34 +08:00
hzh0425 f905da411a [Summer of code] Let broker send heartbeat to controller (#4341)
* let broker send heartbeat to controller

* code remview

* code review

* fix bug

* add state in replicasmanager

* add brokerId when getReplicasInfo

* code review
2022-05-20 17:35:07 +08:00
hzh0425 4e9e4e3efb [Summer of code] Stand alone a new controller module (#4333)
* feature: add new module controller

* feature: add heartbeat manager

* feature: link requestProcessor and heartbeatmanager

* feature: add controllerManager and startup

* feature: remove namesrv's duplicate controller code

* review code
2022-05-18 20:16:43 +08:00