Commit Graph

8369 Commits

Author SHA1 Message Date
rongtong cb7fa3e0b0 Revise the measurement method of GROUP_GET_LATENCY to reveal its intended semantics (#7808) 2024-02-04 10:27:21 +08:00
yuz10 c833ff6e9a [ISSUE #7538] fix wrong cachedMsgSize if msg body is changed in consumer callback 2024-02-02 12:34:13 +08:00
rongtong 6616600fa5 [ISSUE #7791] Polish the code for jraft controller 2024-02-02 12:33:22 +08:00
bazelisky af1936d93a [ISSUE #7715] Update Bazel toolchain to make the CI work (#7714)
* Update bazel toolchain

Signed-off-by: Bazelisky <bazelisk@hotmail.com>

* fix a bad test case as it violates requirement of ConcurrentHashMap#computeIfAbsent

Signed-off-by: Bazelisky <bazelisk@hotmail.com>

* Exclude flaky tests from test suite

Signed-off-by: bazelisky <155456828+bazelisky@users.noreply.github.com>

* fix: format issue

Signed-off-by: Zhanhui Li <lizhanhui@gmail.com>

* fix: Remove unused import

* fix deps issue

Signed-off-by: Zhanhui Li <lizhanhui@gmail.com>

* Expose data property of java_test, allowing to specify run_file

Signed-off-by: Zhanhui Li <lizhanhui@gmail.com>

---------

Signed-off-by: Bazelisky <bazelisk@hotmail.com>
Signed-off-by: bazelisky <155456828+bazelisky@users.noreply.github.com>
Signed-off-by: Zhanhui Li <lizhanhui@gmail.com>
Co-authored-by: Zhanhui Li <lizhanhui@gmail.com>
2024-02-01 16:52:55 +08:00
rongtong 84285d13a7 Update copyright year (#7797) 2024-02-01 11:29:40 +08:00
landonchan90 06e22b4b42 [ISSUE #7765] Fix unit test testEstimateLag
Co-authored-by: landonchan90 <landaonChan@163.com>
2024-01-31 17:39:30 +08:00
YASH PATEL f70862e57b [ISSUE #5613] Change the broker default value configuration of 'useServerSideResetOffset' (#5613) (#7581)
Co-authored-by: RongtongJin <jinrongtong16@mails.ucas.ac.cn>
2024-01-31 03:35:03 +08:00
schopenhauerz 7b64f50137 [ISSUE #7446] Add config of maxFilterMessageSize (#7447)
* bugfix broker boot succes but get fail ip addr

bug:
broker ip addr(IPV4) get fail  after broker start up;
fix:
add compare ,continue  when ip is start with '0.' ;
like :
The broker[broker-a, 0.0.1.1:10911] boot success. serializeType=JSON and name server is 127.0.0.1:9876;

* add config  maxFilterMessageSize , not hard code

* Pass the code check style

---------

Co-authored-by: mingzhi.zhang <mingzhi.zhang@jiduauto.com>
Co-authored-by: RongtongJin <jinrongtong16@mails.ucas.ac.cn>
2024-01-31 03:34:27 +08:00
yulangz 8df53dfde8 [ISSUE #7300] jRaft-Controller Implemention (#7301)
* jRaft-Controller Implemention

* fix bazel build

* reformat code

* remove fury dependence

* fix bazel build

* resolve conflict

* clear code

* Optimize code style

* Optimize code style

* Optimize code style

* revert style only change
fix code style
rollback an unexpected modification.

* Update Producer.java

* chore: move raft startup to start method

* chore: use jraftconfig to collect all configs about jraft

* fix: fix wrong store path because init use string constant

* fix: fix CONTROLLER_NOT_LEADER error in follower

* chore: seperate jraft and controller log

* chore: fix conflict with develop

* chore: add comment to clear the filter logic

* feat: triggerElectMaster will retry when failed

* feat: when controller all restart, we use a timestamp to trace the first heartbeat, avoid to elect again

* fix: implements Serializable to enable snapshot serialize

* fix: use for loop to simple the elect retry

* chore: update jraft version

* chore: opt import

---------

Co-authored-by: leizhiyuan <leizhiyuan@gmail.com>
2024-01-29 20:45:20 +08:00
Poirot Hercule 178421386f [ISSUE #7786] Optimize the execution logic of tool.sh in the JRE environment 2024-01-29 09:40:19 +08:00
mxsm a0cb9d42e3 [ISSUE #7777] Optimize the logic of DefaultRequestProcessor#getTopicsByCluster to avoid unnecessary deserialization (#7778) 2024-01-29 08:34:29 +08:00
wyyl1 e104273fd0 [ISSUE #7785] Remove the redundant code 2024-01-28 20:06:18 +08:00
Ji Juntao 3d357bb3ed [ISSUE #7772] Ensuring broker protection capabilities when POP does not return ACK (#7773)
* add stop pop situation.

* refactor the imports

* modify the threadhold into threshold.

* checkstyle
2024-01-26 14:34:53 +08:00
Zhouxiang Zhan 04eec34978 Get namespace from header (#7782) 2024-01-24 20:18:59 +08:00
dingshuangxi888 92e058c823 [ISSUE #7774] Make the handle of ppv2 tlv more extendable (#7775)
* Fix ascii validate for ppv2 tls.
* make the handle of ppv2 tlv extendable.
2024-01-24 09:50:15 +08:00
Zhouxiang Zhan 5262358140 [ISSUE #7699] Refector NamespaceRpcHook (#7769)
* [ISSUE #7699] Refector NamespaceRpcHook

* fix
2024-01-23 13:56:26 +08:00
Ji Juntao 6d7513425c [RIP-46] Enhanced metrics for timing and transactional messages (#7500)
* add request codes' distribution and timing messages' distribution

* remove the requestCode distribution.

* add delay message latency distribution.

* add transaction metrics

* transaction metric of topics finished, v1.

* add the transaction metrics, to be tested.

* fix the judgement of putMessageResult

* optimize.

* add config.

* fix test case.

* add unit tests for transactionMetrics.

* remove chinese character

* add rocksdb metrics.

* add more rocksdb metrics.

* fix NPE

* avoid the total time is 0.

* add license

* remove useless import.
2024-01-22 16:56:05 +08:00
Lei Zhiyuan 920dc32d27 [ISSUE #7760] make timerSkipUnknownError can be set by config file 2024-01-19 09:59:23 +08:00
bxfjb 646e2a4942 [ISSUE #7355] fix dledger recover abnormally may lost consume queue of tail (#7599)
* fix dledger recover abnormally may lost consume queue of tail

* fix correct storeTimestampPosition when bornhost is v6

* fix correct SYSFLAG offset

---------

Co-authored-by: 赵宇晗 <zhaoyuhan@xiaomi.com>
2024-01-18 15:43:01 +08:00
道君 7a36d4d736 [ISSUE #7757] Use CompositeByteBuf to prevent memory copy. (#7694)
* Use CompositeByteBuf to prevent mem_copy.

* Fix code

* Add tests

* Remove useless UTs

* Remove unused imports.

---------

Co-authored-by: RongtongJin <jinrongtong16@mails.ucas.ac.cn>
2024-01-17 09:49:23 +08:00
Ao Qiao 1242a5873d [ISSUE #7495] Support Higher Java version in Windows (#7507)
* fix 7495
2024-01-15 17:49:40 +08:00
dingshuangxi888 89fdab4d58 [ISSUE #7752] Fix ppv2 tls ascii start with (byte)2 (#7753)
* Fix ascii validate for ppv2 tls.

* fix ppv2 tls ascii check

---------

Co-authored-by: ShuangxiDing <shuangxi.dsx@alibaba-inc.com>
2024-01-15 15:18:45 +08:00
mxsm 5e50badca1 [ISSUE #7747] Simplify code checks using Optional in ClientRequestProcessor#getRouteInfoByTopic method (#7748) 2024-01-15 11:44:00 +08:00
Lei Zhiyuan 50974adaaa [ISSUE #7658] Fix bug of timer message metrics with setDelayTimeMs 2024-01-12 11:43:26 +08:00
Zhouxiang Zhan b1d8d306a5 [ISSUE #7699] Add namespace v2 in client (#7700)
* Add namespace v2

* Add NamespaceRpcHook

* Refector extends header

* Use Boolean in request header to remove unnecessary encode

* Add unit test

* Add NamespaceRpcHookTest

* Remove GrpcConverter#wrapResourceWithNamespace

* Optimize readability of RpcRequestHeader
2024-01-12 10:46:15 +08:00
Zhouxiang Zhan d2b818d993 Revert "[ISSUE #7707] Refector Context with link node implementation (#7708)" (#7742)
This reverts commit 3f99b1e96b.
2024-01-12 10:24:07 +08:00
Zhouxiang Zhan 8a36471a19 [ISSUE #7543] Add enableRetryTopicV2 brokerConfig (#7734)
* Add enableRetryTopicV2
2024-01-11 11:11:53 +08:00
Zhouxiang Zhan 3f99b1e96b [ISSUE #7707] Refector Context with link node implementation (#7708)
* Refector Context with link node implement
2024-01-09 10:51:26 +08:00
zzl 6fce427e10 [ISSUE ##7719] Add more property check for dealy message when auto-batch chack (#7720) 2024-01-04 17:38:03 +08:00
Qinglong-Lee 806454bc5e [ISSUE #7710] Handle blank string for UtilAll#split to fix the bugs of ACL
Co-authored-by: liqinglong <liqinglong96536@billjc.com>
2024-01-02 19:07:45 +08:00
baijun44 9e4fd0e68a [ISSUE #7697] Fix can't open controller metricsExporter. (#7705) 2023-12-28 15:39:10 +08:00
dingshuangxi888 2c5bc1422b Fix ascii validate for ppv2 tls. (#7703)
Co-authored-by: ShuangxiDing <shuangxi.dsx@alibaba-inc.com>
2023-12-27 19:09:52 +08:00
Zhouxiang Zhan 59e8f9b66e [ISSUE #7644] Optimize client rebalance 2023-12-27 10:42:50 +08:00
Zhouxiang Zhan 620e6a2544 [ISSUE #7642] Add return value for sendHeartbeat related method
Add sendHeartbeatToBroker
2023-12-27 10:42:22 +08:00
dingshuangxi888 4bb4d78f1d [ISSUE #7686] The bornTime is not set when using the popMessage API in cluster mode. (#7687)
* The bornTime is not set when using the popMessage API in cluster mode.

---------

Co-authored-by: ShuangxiDing <shuangxi.dsx@alibaba-inc.com>
2023-12-22 15:55:00 +08:00
Ji Juntao 2c898c9b31 [ISSUE #7689] In Controller mode, messages may lost due to sharing the same cq offset (#7690)
* fix the reput bug.

* add more logs.

* refactor the method of compensating for HA.

* not modify the imports.

* refactor the log.

* refactor the log.
2023-12-21 14:41:51 +08:00
EvanMi f0a3e933b9 [ISSUE #7684] Fix iterator.remove() bug (#7682)
* bugfix: CopyOnWriteArray#listIterator do not support remove action when iterating

* add testcase

---------

Co-authored-by: mipengcheng3 <mipengcheng3@jd.com>
2023-12-20 14:34:53 +08:00
rongtong fbfc066695 Bump guava version from 31.1-jre to 32.0.1-jre (#7681) 2023-12-20 11:49:20 +08:00
cserwen c10121697a [ISSUE #7676] use clientDecode for consuming message directly (#7677)
Co-authored-by: dengzhiwen1 <dengzhiwen1@xiaomi.com>
2023-12-20 10:12:15 +08:00
mxsm 403ad6f66c [ISSUE #7679] Optimize the serialization of RemotingCommand processTimer property (#7683) 2023-12-20 08:45:08 +08:00
YASH PATEL 9dec4cf5fe [ISSUE #7669] map variable delayLevelTable changed to ConcurrentSkipListMap from ConcurrentHashMap (#7675) 2023-12-19 19:47:13 +08:00
rongtong 80c0330f75 ConfirmOffset directly takes the max offset when allAckInSyncStateSet is false (#7657) 2023-12-18 10:10:28 +08:00
Dongyuan Pan 086a726c5d [ISSUE #7659]Trim property 2023-12-18 08:29:54 +08:00
littleboy 7e1786732c [ISSUE #7626] Topic perm was mistakenly changed to 4 in dledger mode (#7661)
Signed-off-by: littleboy <2283985296@qq.com>
2023-12-15 17:49:47 +08:00
Zhouxiang Zhan 23ee0eaaef [ISSUE #7646] Optimize pull onException logging (#7647)
* Optimize SUBSCRIPTION_NOT_LATEST logging

* Add message queue in pullMessage onException
2023-12-15 16:49:44 +08:00
Zhouxiang Zhan 71a7a659be [ISSUE #7543] only call a single type of retry topic in pop (#7665)
* only call a single type of retry topic in pop
2023-12-15 16:09:11 +08:00
Zhouxiang Zhan 8e585d8767 [ISSUE #7543] Use "+" as the new separator for retry topic (#7655) 2023-12-15 16:08:43 +08:00
lizhimins 50a92a2474 [ISSUE #7585] Always return duplicate buffer when filter message and fix log format (#7654) 2023-12-13 19:50:19 +08:00
Zhouxiang Zhan a376fbcdb8 [ISSUE #7634] Introduce controllableOffset to prevent unnecessary suspension when OFFSET_ILLEGAL (#7635)
* Add controllableOffset to prevent unnecessary suspension when OFFSET_ILLEGAL
2023-12-12 16:52:12 +08:00
rongtong f7a6d0b73a Bump grpc from 1.50.0 to 1.53.0 (#7633) 2023-12-12 16:22:21 +08:00