mirror of
https://github.com/apache/rocketmq.git
synced 2026-08-30 18:10:44 +08:00
Finish code dump. Reviewed by @vongosling @stevenschew @vintagewang @lollipopjin
This commit is contained in:
yukon
+11
@@ -0,0 +1,11 @@
|
||||
*dependency-reduced-pom.xml
|
||||
.classpath
|
||||
.project
|
||||
.settings/
|
||||
target/
|
||||
devenv
|
||||
*.log*
|
||||
*.iml
|
||||
.idea/
|
||||
*.versionsBackup
|
||||
*bin
|
||||
@@ -0,0 +1,5 @@
|
||||
language: java
|
||||
jdk:
|
||||
- oraclejdk8
|
||||
- oraclejdk7
|
||||
- openjdk6
|
||||
@@ -0,0 +1,17 @@
|
||||
## How To Contribute
|
||||
|
||||
We are always very happy to have contributions, whether for trivial cleanups or big new features.
|
||||
We want to have high quality, well documented codes for each programming language. These, as well as the surrounding [ecosystem](https://github.com/rocketmq) of integration tools that people use with RocketMQ, are critical aspects of the project.
|
||||
|
||||
Nor is code the only way to contribute to the project. We strongly value documentation and gladly accept improvements to the documentation.
|
||||
|
||||
## Contributing code
|
||||
|
||||
To submit a change for inclusion, please do the following:
|
||||
#### If the change is non-trivial please include some unit tests that cover the new functionality.
|
||||
#### If you are introducing a completely new feature or API it is a good idea to start a wiki and get consensus on the basic design first.
|
||||
#### It is our job to follow up on patches in a timely fashion. Nag us if we aren't doing our job (sometimes we drop things).
|
||||
|
||||
## Becoming a Committer
|
||||
|
||||
We are always interested in adding new contributors. What we look for a series of contributions, good taste, and an ongoing interest in the project. If you are interested in becoming a committer, let one of the existing committers know and they can help guide you through the process.
|
||||
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (properties) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright 2016 Alibaba Group
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,8 @@
|
||||
RocketMQ
|
||||
Copyright 2016 Alibaba Group.
|
||||
|
||||
This product includes software developed at
|
||||
The Apache Software Foundation (http://www.apache.org/).
|
||||
|
||||
This distribution has producerGroup binary dependency on mysql-connector-java, which is available under gpl 2.0.
|
||||
The source code of mysql-connector-java can be found at http://dev.mysql.com/downloads/connector/reconsumeTimes/.
|
||||
@@ -0,0 +1,52 @@
|
||||
## RocketMQ -- Alibaba's MQ [](https://travis-ci.org/alibaba/RocketMQ)
|
||||
[](http://search.maven.org/#search%7Cga%7C1%7Ccom.alibaba.rocketmq)
|
||||
[](https://github.com/alibaba/RocketMQ/releases)
|
||||
[](https://www.apache.org/licenses/LICENSE-2.0.html)
|
||||
|
||||
**RocketMQ is a low latency, reliable, scalable, easy to use message oriented middleware born from alibaba massive messaging business.**
|
||||
|
||||
It offers a variety of features as follows:
|
||||
|
||||
* Pub/Sub and P2P messaging model
|
||||
* Reliable FIFO and strict sequential messaging in the same queue
|
||||
* Long pull queue model,also support push consumption style
|
||||
* Million message accumulation ability in single queue
|
||||
* Over a variety of messaging protocols.such as JMS,MQTT,HTTP etc.
|
||||
* Distributed high available deploy architecture, meets at least once message delivery semantics
|
||||
* Docker images for isolated testing and cloud Isolated clusters
|
||||
* Feature-rich administrative dashboard for configuration,metrics and monitoring [](https://www.aliyun.com/product/ons/?spm=5176.7960203.237031.135.67OdNO)
|
||||
* Message full-link tracking [![Cloud Charge] (https://img.shields.io/badge/cloud-charge-red.svg)](https://www.aliyun.com/product/ons/?spm=5176.7960203.237031.135.67OdNO)
|
||||
* Producer transaction message,making producer and local database transaction in one atomic operation [](https://www.aliyun.com/product/ons/?spm=5176.7960203.237031.135.67OdNO)
|
||||
* Message Schedule delivery,similar JMS2 spec's delivery delay [![Cloud Charge] (https://img.shields.io/badge/cloud-charge-red.svg)](https://www.aliyun.com/product/ons/?spm=5176.7960203.237031.135.67OdNO)
|
||||
|
||||
----------
|
||||
|
||||
## Learn it & Contact us
|
||||
* Homepage: <https://github.com/alibaba/RocketMQ>
|
||||
* Mailing list: <https://groups.google.com/forum/?hl=en#!forum/rocketmq>
|
||||
* Documentation: <https://github.com/alibaba/RocketMQ/wiki>
|
||||
* Wiki: <https://github.com/alibaba/RocketMQ/wiki>
|
||||
* Issues: <https://github.com/alibaba/RocketMQ/issues>
|
||||
* Ask: <http://stackoverflow.com/questions/tagged/rocketmq> or <https://www.quora.com/topic/RocketMQ>
|
||||
* [](https://gitter.im/alibaba/RocketMQ?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
|
||||
|
||||
|
||||
----------
|
||||
|
||||
## How can I develop with RocketMQ?
|
||||
* [Download the latest release package](https://github.com/alibaba/RocketMQ/releases)
|
||||
* [Get the latest document & Who using RocketMQ](https://github.com/alibaba/RocketMQ/issues/1)
|
||||
* [`Using RocketMQ in Aliyun`](http://www.aliyun.com/product/ons)
|
||||
* [`Docker images`](https://registry.hub.docker.com/u/vongosling/rocketmq/)
|
||||
* [Integrate with Alibaba JStorm](https://github.com/alibaba/jstorm)
|
||||
* [Integrate with Alibaba Canal](https://github.com/alibaba/canal)
|
||||
* [RocketMQ Community Projects](https://github.com/rocketmq)
|
||||
|
||||
----------
|
||||
|
||||
## Contributing
|
||||
We are always very happy to have contributions, whether for trivial cleanups,big new features or other material rewards. more details see [here](https://github.com/alibaba/RocketMQ/wiki/How-to-donate)
|
||||
|
||||
----------
|
||||
## License
|
||||
[Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.html) Copyright (C) 2010-2013 Alibaba Group Holding Limited
|
||||
@@ -0,0 +1,6 @@
|
||||
#!/bin/sh
|
||||
|
||||
#
|
||||
# $Id: consumer.sh 1831 2013-05-16 01:39:51Z shijia.wxr $
|
||||
#
|
||||
sh ./runclass.sh com.alibaba.rocketmq.example.benchmark.Consumer $@ &
|
||||
@@ -0,0 +1,6 @@
|
||||
#!/bin/sh
|
||||
|
||||
#
|
||||
# $Id: producer.sh 1831 2013-05-16 01:39:51Z shijia.wxr $
|
||||
#
|
||||
sh ./runclass.sh -Dcom.alibaba.rocketmq.client.sendSmartMsg=true com.alibaba.rocketmq.example.benchmark.Producer $@ &
|
||||
@@ -0,0 +1,32 @@
|
||||
#!/bin/sh
|
||||
|
||||
#
|
||||
# $Id: runserver.sh 1831 2013-05-16 01:39:51Z shijia.wxr $
|
||||
#
|
||||
|
||||
if [ $# -lt 1 ];
|
||||
then
|
||||
echo "USAGE: $0 classname opts"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BASE_DIR=$(dirname $0)/..
|
||||
CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}
|
||||
|
||||
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn256m -XX:PermSize=128m -XX:MaxPermSize=320m"
|
||||
JAVA_OPT="${JAVA_OPT} -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8 -XX:+DisableExplicitGC"
|
||||
JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails"
|
||||
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
|
||||
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${BASE_DIR}/lib"
|
||||
JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages"
|
||||
JAVA_OPT="${JAVA_OPT} -XX:+PerfDisableSharedMem"
|
||||
#JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n"
|
||||
JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"
|
||||
|
||||
if [ -z "$JAVA_HOME" ]; then
|
||||
JAVA_HOME=/usr/java
|
||||
fi
|
||||
|
||||
JAVA="$JAVA_HOME/bin/java"
|
||||
|
||||
$JAVA ${JAVA_OPT} $@
|
||||
@@ -0,0 +1,6 @@
|
||||
#!/bin/sh
|
||||
|
||||
#
|
||||
# $Id: producer.sh 1831 2013-05-16 01:39:51Z shijia.wxr $
|
||||
#
|
||||
sh ./runclass.sh com.alibaba.rocketmq.example.benchmark.TransactionProducer $@
|
||||
@@ -0,0 +1,128 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE module PUBLIC
|
||||
"-//Puppy Crawl//DTD Check Configuration 1.3//EN"
|
||||
"http://www.puppycrawl.com/dtds/configuration_1_3.dtd">
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
<!--Refer http://checkstyle.sourceforge.net/reports/google-java-style.html#s2.2-file-encoding -->
|
||||
<module name="Checker">
|
||||
|
||||
<property name="localeLanguage" value="en"/>
|
||||
|
||||
<!--To configure the check to report on the first instance in each file-->
|
||||
<module name="FileTabCharacter"/>
|
||||
|
||||
<!-- header -->
|
||||
<module name="RegexpHeader">
|
||||
<property name="header" value="/\*\*\nLicensed to the Apache.*"/>
|
||||
</module>
|
||||
|
||||
<module name="RegexpSingleline">
|
||||
<property name="format" value="System\.out\.println" />
|
||||
<property name="message" value="Prohibit invoking System.out.println in source code !" />
|
||||
</module>
|
||||
|
||||
<module name="RegexpSingleline">
|
||||
<property name="format" value="//FIXME"/>
|
||||
<property name="message" value="Recommended fix FIXME task !"/>
|
||||
</module>
|
||||
|
||||
<module name="RegexpSingleline">
|
||||
<property name="format" value="//TODO"/>
|
||||
<property name="message" value="Recommended fix TODO task !"/>
|
||||
</module>
|
||||
|
||||
<module name="RegexpSingleline">
|
||||
<property name="format" value="@alibaba"/>
|
||||
<property name="message" value="Recommended remove @alibaba keyword!"/>
|
||||
</module>
|
||||
|
||||
<module name="RegexpSingleline">
|
||||
<property name="format"
|
||||
value=".*[\u3400-\u4DB5\u4E00-\u9FA5\u9FA6-\u9FBB\uF900-\uFA2D\uFA30-\uFA6A\uFA70-\uFAD9\uFF00-\uFFEF\u2E80-\u2EFF\u3000-\u303F\u31C0-\u31EF]+.*"/>
|
||||
<property name="message" value="Not allow chinese character !"/>
|
||||
</module>
|
||||
|
||||
<module name="FileLength">
|
||||
<property name="max" value="2000"/>
|
||||
</module>
|
||||
|
||||
<module name="TreeWalker">
|
||||
|
||||
<module name="UnusedImports">
|
||||
<property name="processJavadoc" value="true"/>
|
||||
</module>
|
||||
<module name="RedundantImport"/>
|
||||
|
||||
<!--<module name="IllegalImport" />-->
|
||||
|
||||
<!--Checks that classes that override equals() also override hashCode()-->
|
||||
<module name="EqualsHashCode"/>
|
||||
<!--Checks for over-complicated boolean expressions. Currently finds code like if (topic == true), topic || true, !false, etc.-->
|
||||
<module name="SimplifyBooleanExpression"/>
|
||||
<module name="OneStatementPerLine"/>
|
||||
<module name="UnnecessaryParentheses"/>
|
||||
<!--Checks for over-complicated boolean return statements. For example the following code-->
|
||||
<module name="SimplifyBooleanReturn"/>
|
||||
|
||||
<!--Check that the default is after all the cases in producerGroup switch statement-->
|
||||
<module name="DefaultComesLast"/>
|
||||
<!--Detects empty statements (standalone ";" semicolon)-->
|
||||
<module name="EmptyStatement"/>
|
||||
<!--Checks that long constants are defined with an upper ell-->
|
||||
<module name="UpperEll"/>
|
||||
<module name="ConstantName">
|
||||
<property name="format" value="(^[A-Z][A-Z0-9]*(_[A-Z0-9]+)*$)|(^log$)"/>
|
||||
</module>
|
||||
<!--Checks that local, non-final variable names conform to producerGroup format specified by the format property-->
|
||||
<module name="LocalVariableName"/>
|
||||
<!--Validates identifiers for local, final variables, including catch parameters-->
|
||||
<module name="LocalFinalVariableName"/>
|
||||
<!--Validates identifiers for non-static fields-->
|
||||
<module name="MemberName"/>
|
||||
<!--Validates identifiers for class type parameters-->
|
||||
<module name="ClassTypeParameterName">
|
||||
<property name="format" value="^[A-Z0-9]*$"/>
|
||||
</module>
|
||||
<!--Validates identifiers for method type parameters-->
|
||||
<module name="MethodTypeParameterName">
|
||||
<property name="format" value="^[A-Z0-9]*$"/>
|
||||
</module>
|
||||
<module name="PackageName"/>
|
||||
<module name="ParameterName"/>
|
||||
<module name="StaticVariableName"/>
|
||||
<module name="TypeName"/>
|
||||
<!--Checks that there are no import statements that use the * notation-->
|
||||
<!--
|
||||
<module name="AvoidStarImport"/>
|
||||
-->
|
||||
|
||||
<!--whitespace-->
|
||||
<module name="GenericWhitespace"/>
|
||||
<module name="NoWhitespaceBefore"/>
|
||||
<module name="WhitespaceAfter"/>
|
||||
<module name="NoWhitespaceAfter"/>
|
||||
<module name="WhitespaceAround">
|
||||
<property name="allowEmptyConstructors" value="true"/>
|
||||
<property name="allowEmptyMethods" value="true"/>
|
||||
</module>
|
||||
<module name="Indentation"/>
|
||||
<module name="MethodParamPad"/>
|
||||
<module name="ParenPad"/>
|
||||
<module name="TypecastParenPad"/>
|
||||
</module>
|
||||
</module>
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-a
|
||||
brokerId=1
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=SLAVE
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-a
|
||||
brokerId=0
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=ASYNC_MASTER
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-b
|
||||
brokerId=1
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=SLAVE
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-b
|
||||
brokerId=0
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=ASYNC_MASTER
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-a
|
||||
brokerId=1
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=SLAVE
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-a
|
||||
brokerId=0
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=SYNC_MASTER
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-b
|
||||
brokerId=1
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=SLAVE
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-b
|
||||
brokerId=0
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=SYNC_MASTER
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-a
|
||||
brokerId=0
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=ASYNC_MASTER
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-b
|
||||
brokerId=0
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=ASYNC_MASTER
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,22 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
brokerClusterName=DefaultCluster
|
||||
brokerName=broker-a
|
||||
brokerId=0
|
||||
deleteWhen=04
|
||||
fileReservedTime=48
|
||||
brokerRole=ASYNC_MASTER
|
||||
flushDiskType=ASYNC_FLUSH
|
||||
@@ -0,0 +1,328 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<configuration>
|
||||
<appender name="DefaultAppender"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/broker_default.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/broker_default.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>10</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqBrokerAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/broker.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/broker.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>20</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>128MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqBrokerAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqBrokerAppender_inner"/>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqProtectionAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/protection.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/protection.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>10</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqProtectionAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqProtectionAppender_inner"/>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqWaterMarkAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/watermark.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/watermark.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>10</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqWaterMarkAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqWaterMarkAppender_inner"/>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqStoreAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/store.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/store.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>10</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>128MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqStoreAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqStoreAppender_inner"/>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqRemotingAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/remoting.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/remoting.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>10</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqRemotingAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqRemotingAppender_inner"/>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqStoreErrorAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/storeerror.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/storeerror.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>10</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqStoreErrorAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqStoreErrorAppender_inner"/>
|
||||
</appender>
|
||||
|
||||
|
||||
<appender name="RocketmqTransactionAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/transaction.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/transaction.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>10</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqTransactionAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqTransactionAppender_inner"/>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqRebalanceLockAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/lock.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/lock.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>5</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqRebalanceLockAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqRebalanceLockAppender_inner"/>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqStatsAppender"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/stats.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/stats.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>5</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqCommercialAppender"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/commercial.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/commercial.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>10</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>500MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
</appender>
|
||||
|
||||
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
|
||||
<append>true</append>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH\:mm\:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<logger name="RocketmqBroker" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqBrokerAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqProtection" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqProtectionAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqWaterMark" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqWaterMarkAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqCommon" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqBrokerAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqStore" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqStoreAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqStoreError" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqStoreErrorAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqTransaction" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqTransactionAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqRebalanceLock" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqRebalanceLockAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqRemoting" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqRemotingAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqStats" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqStatsAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqCommercial" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqCommercialAppender"/>
|
||||
</logger>
|
||||
|
||||
<root>
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="DefaultAppender"/>
|
||||
</root>
|
||||
</configuration>
|
||||
@@ -0,0 +1,91 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<configuration>
|
||||
<appender name="DefaultAppender"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/filtersrv_default.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/filtersrv_default.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>5</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqFiltersrvAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/filtersrv.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/filtersrv.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>5</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqFiltersrvAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqFiltersrvAppender_inner"/>
|
||||
<discardingThreshold>0</discardingThreshold>
|
||||
</appender>
|
||||
|
||||
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
|
||||
<append>true</append>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH\:mm\:ss,SSS} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<logger name="RocketmqFiltersrv" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqFiltersrvAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqCommon" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqFiltersrvAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqRemoting" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqFiltersrvAppender"/>
|
||||
</logger>
|
||||
|
||||
<root>
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="DefaultAppender"/>
|
||||
</root>
|
||||
</configuration>
|
||||
@@ -0,0 +1,91 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<configuration>
|
||||
<appender name="DefaultAppender"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/namesrv_default.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/namesrv_default.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>5</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqNamesrvAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/namesrv.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/namesrv.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>5</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqNamesrvAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqNamesrvAppender_inner"/>
|
||||
<discardingThreshold>0</discardingThreshold>
|
||||
</appender>
|
||||
|
||||
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
|
||||
<append>true</append>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH\:mm\:ss,SSS} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<logger name="RocketmqNamesrv" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqNamesrvAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqCommon" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqNamesrvAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqRemoting" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqNamesrvAppender"/>
|
||||
</logger>
|
||||
|
||||
<root>
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="DefaultAppender"/>
|
||||
</root>
|
||||
</configuration>
|
||||
@@ -0,0 +1,96 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<configuration>
|
||||
<appender name="DefaultAppender"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/tools_default.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/tools_default.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>5</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<appender name="RocketmqToolsAppender_inner"
|
||||
class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${user.home}/logs/rocketmqlogs/tools.log</file>
|
||||
<append>true</append>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/tools.%i.log
|
||||
</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>5</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy
|
||||
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>100MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
<appender name="RocketmqToolsAppender" class="ch.qos.logback.classic.AsyncAppender">
|
||||
<appender-ref ref="RocketmqToolsAppender_inner"/>
|
||||
</appender>
|
||||
|
||||
|
||||
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
|
||||
<append>true</append>
|
||||
<encoder>
|
||||
<pattern>%d{yyy-MM-dd HH\:mm\:ss,GMT+8} %p %t - %m%n</pattern>
|
||||
<charset class="java.nio.charset.Charset">UTF-8</charset>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<logger name="RocketmqTools" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqToolsAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqCommon" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqBrokerAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqStore" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqBrokerAppender"/>
|
||||
</logger>
|
||||
|
||||
<logger name="RocketmqRemoting" additivity="false">
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="RocketmqBrokerAppender"/>
|
||||
</logger>
|
||||
|
||||
<root>
|
||||
<level value="INFO"/>
|
||||
<appender-ref ref="DefaultAppender"/>
|
||||
</root>
|
||||
</configuration>
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
mvn -Dmaven.test.skip=true deploy
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
mvn -U eclipse:eclipse
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
REM Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
REM contributor license agreements. See the NOTICE file distributed with
|
||||
REM this work for additional information regarding copyright ownership.
|
||||
REM The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
REM (the "License"); you may not use this file except in compliance with
|
||||
REM the License. You may obtain a copy of the License at
|
||||
REM
|
||||
REM http://www.apache.org/licenses/LICENSE-2.0
|
||||
REM
|
||||
REM Unless required by applicable law or agreed to in writing, software
|
||||
REM distributed under the License is distributed on an "AS IS" BASIS,
|
||||
REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
REM See the License for the specific language governing permissions and
|
||||
REM limitations under the License.
|
||||
|
||||
mvn -Dmaven.test.skip=true clean package install assembly:assembly -U
|
||||
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
git pull
|
||||
|
||||
rm -rf target
|
||||
rm -f devenv
|
||||
if [ -z "$JAVA_HOME" ]; then
|
||||
JAVA_HOME=/usr/java
|
||||
fi
|
||||
export PATH=$JAVA_HOME/bin:$PATH
|
||||
mvn clean package install assembly:assembly -U
|
||||
|
||||
ln -s target/alibaba-rocketmq-broker/alibaba-rocketmq devenv
|
||||
@@ -0,0 +1,518 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
|
||||
|
||||
<parent>
|
||||
<groupId>org.apache</groupId>
|
||||
<artifactId>apache</artifactId>
|
||||
<version>18</version>
|
||||
</parent>
|
||||
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<inceptionYear>2012</inceptionYear>
|
||||
<groupId>com.alibaba.rocketmq</groupId>
|
||||
<artifactId>rocketmq-all</artifactId>
|
||||
<version>4.0.0-SNAPSHOT</version>
|
||||
<packaging>pom</packaging>
|
||||
<name>rocketmq-all ${project.version}</name>
|
||||
<url>http://rocketmq.incubator.apache.org/</url>
|
||||
|
||||
<prerequisites>
|
||||
<maven>3.2.5</maven>
|
||||
</prerequisites>
|
||||
|
||||
<scm>
|
||||
<url>https://git-wip-us.apache.org/repos/asf/incubator-rocketmq.git</url>
|
||||
<connection>scm:git:https://git-wip-us.apache.org/repos/asf/incubator-rocketmq.git</connection>
|
||||
<developerConnection>scm:git:https://git-wip-us.apache.org/repos/asf/incubator-rocketmq.git</developerConnection>
|
||||
</scm>
|
||||
|
||||
<mailingLists>
|
||||
<mailingList>
|
||||
<name>Development List</name>
|
||||
<subscribe>dev-subscribe@rocketmq.incubator.apache.org</subscribe>
|
||||
<unsubscribe>dev-unsubscribe@rocketmq.incubator.apache.org</unsubscribe>
|
||||
<post>dev@rocketmq.incubator.apache.org</post>
|
||||
</mailingList>
|
||||
<mailingList>
|
||||
<name>User List</name>
|
||||
<subscribe>users-subscribe@rocketmq.incubator.apache.org</subscribe>
|
||||
<unsubscribe>users-unsubscribe@rocketmq.incubator.apache.org</unsubscribe>
|
||||
<post>users@rocketmq.incubator.apache.org</post>
|
||||
</mailingList>
|
||||
<mailingList>
|
||||
<name>Commits List</name>
|
||||
<subscribe>commits-subscribe@rocketmq.incubator.apache.org</subscribe>
|
||||
<unsubscribe>commits-unsubscribe@rocketmq.incubator.apache.org</unsubscribe>
|
||||
<post>commits@rocketmq.incubator.apache.org</post>
|
||||
</mailingList>
|
||||
</mailingLists>
|
||||
|
||||
<developers>
|
||||
<developer>
|
||||
<id>vintagewang</id>
|
||||
<name>vintagewang</name>
|
||||
<roles>
|
||||
<role>architect</role>
|
||||
<role>committer</role>
|
||||
</roles>
|
||||
<email>vintagewang@apache.org</email>
|
||||
<url>https://github.com/vintagewang</url>
|
||||
<timezone>8</timezone>
|
||||
</developer>
|
||||
<developer>
|
||||
<id>vongosling</id>
|
||||
<name>vongosling</name>
|
||||
<roles>
|
||||
<role>architect</role>
|
||||
<role>committer</role>
|
||||
</roles>
|
||||
<email>vongosling@apache.org</email>
|
||||
<url>https://github.com/vongosling</url>
|
||||
<timezone>+8</timezone>
|
||||
</developer>
|
||||
<developer>
|
||||
<id>yukon</id>
|
||||
<name>Xinyu Zhou</name>
|
||||
<email>yukon@@apache.org</email>
|
||||
<url>https://github.com/zhouxinyu</url>
|
||||
<roles>
|
||||
<role>committer</role>
|
||||
</roles>
|
||||
<timezone>+8</timezone>
|
||||
</developer>
|
||||
<developer>
|
||||
<id>stevenschew</id>
|
||||
<name>Wei Zhou</name>
|
||||
<email>stevenschew@@apache.org</email>
|
||||
<url>https://github.com/zhouwei198732</url>
|
||||
<roles>
|
||||
<role>committer</role>
|
||||
</roles>
|
||||
<timezone>+8</timezone>
|
||||
</developer>
|
||||
<developer>
|
||||
<id>lollipop</id>
|
||||
<name>Jixiang Jin</name>
|
||||
<email>lollipop@apache.org</email>
|
||||
<url>https://github.com/jinjixiang</url>
|
||||
<roles>
|
||||
<role>committer</role>
|
||||
</roles>
|
||||
<timezone>+8</timezone>
|
||||
</developer>
|
||||
<developer>
|
||||
<id>lizhanhui</id>
|
||||
<name>Zhanhui Li</name>
|
||||
<email>lizhanhui@apache.org</email>
|
||||
<url>https://github.com/lizhanhui</url>
|
||||
<roles>
|
||||
<role>committer</role>
|
||||
</roles>
|
||||
<timezone>+8</timezone>
|
||||
</developer>
|
||||
</developers>
|
||||
|
||||
<licenses>
|
||||
<license>
|
||||
<name>Apache License, Version 2.0</name>
|
||||
<url>http://www.apache.org/licenses/LICENSE-2.0</url>
|
||||
<distribution>repo</distribution>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
<organization>
|
||||
<name>Apache Software Foundation</name>
|
||||
<url>http://www.apache.org</url>
|
||||
</organization>
|
||||
|
||||
<issueManagement>
|
||||
<system>jira</system>
|
||||
<url>https://issues.apache.org/jira/browse/RocketMQ</url>
|
||||
</issueManagement>
|
||||
|
||||
<properties>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<!--maven properties -->
|
||||
<maven.test.skip>false</maven.test.skip>
|
||||
<maven.javadoc.skip>true</maven.javadoc.skip>
|
||||
<!-- compiler settings properties -->
|
||||
<maven.compiler.source>1.7</maven.compiler.source>
|
||||
<maven.compiler.target>1.7</maven.compiler.target>
|
||||
</properties>
|
||||
|
||||
<modules>
|
||||
<module>rocketmq-client</module>
|
||||
<module>rocketmq-common</module>
|
||||
<module>rocketmq-broker</module>
|
||||
<module>rocketmq-tools</module>
|
||||
<module>rocketmq-store</module>
|
||||
<module>rocketmq-namesrv</module>
|
||||
<module>rocketmq-remoting</module>
|
||||
<module>rocketmq-example</module>
|
||||
<module>rocketmq-filtersrv</module>
|
||||
<module>rocketmq-srvutil</module>
|
||||
</modules>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.codehaus.mojo</groupId>
|
||||
<artifactId>versions-maven-plugin</artifactId>
|
||||
<version>2.2</version>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>com.github.vongosling</groupId>
|
||||
<artifactId>dependency-mediator-maven-plugin</artifactId>
|
||||
<version>1.0.2</version>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.codehaus.mojo</groupId>
|
||||
<artifactId>clirr-maven-plugin</artifactId>
|
||||
<version>2.7</version>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<artifactId>maven-enforcer-plugin</artifactId>
|
||||
<version>1.4.1</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>enforce-ban-circular-dependencies</id>
|
||||
<goals>
|
||||
<goal>enforce</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
<configuration>
|
||||
<rules>
|
||||
<banCircularDependencies/>
|
||||
</rules>
|
||||
<fail>true</fail>
|
||||
</configuration>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.codehaus.mojo</groupId>
|
||||
<artifactId>extra-enforcer-rules</artifactId>
|
||||
<version>1.0-beta-4</version>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<artifactId>maven-compiler-plugin</artifactId>
|
||||
<version>3.5.1</version>
|
||||
<configuration>
|
||||
<source>${maven.compiler.source}</source>
|
||||
<target>${maven.compiler.target}</target>
|
||||
<compilerVersion>${maven.compiler.source}</compilerVersion>
|
||||
<showDeprecation>true</showDeprecation>
|
||||
<showWarnings>true</showWarnings>
|
||||
</configuration>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<artifactId>maven-surefire-plugin</artifactId>
|
||||
<version>2.19.1</version>
|
||||
<configuration>
|
||||
<argLine>-Xms512m -Xmx1024m</argLine>
|
||||
<includes>
|
||||
<include>**/*Test.java</include>
|
||||
</includes>
|
||||
<excludes>
|
||||
<exclude>com/alibaba/rocketmq/remoting/ExceptionTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/remoting/SyncInvokeTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/remoting/NettyIdleTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/remoting/NettyConnectionTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/common/filter/PolishExprTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/common/protocol/MQProtosHelperTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/client/consumer/loadbalance/AllocateMessageQueueAveragelyTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/store/RecoverTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/broker/api/SendMessageTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/test/integration/*/*.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/test/integration/BaseTest.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/test/*/*.java</exclude>
|
||||
<exclude>com/alibaba/rocketmq/test/BaseTest.java</exclude>
|
||||
</excludes>
|
||||
</configuration>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<artifactId>maven-assembly-plugin</artifactId>
|
||||
<version>2.6</version>
|
||||
<configuration>
|
||||
<finalName>alibaba-rocketmq</finalName>
|
||||
<descriptors>
|
||||
<descriptor>release.xml</descriptor>
|
||||
</descriptors>
|
||||
</configuration>
|
||||
</plugin>
|
||||
|
||||
<!--
|
||||
<plugin>
|
||||
<artifactId>maven-assembly-plugin</artifactId>
|
||||
<configuration>
|
||||
<finalName>alibaba-rocketmq-client-java-${project.version}</finalName>
|
||||
<descriptors>
|
||||
<descriptor>release-client.xml</descriptor>
|
||||
</descriptors>
|
||||
</configuration>
|
||||
</plugin>
|
||||
-->
|
||||
|
||||
<plugin>
|
||||
<artifactId>maven-javadoc-plugin</artifactId>
|
||||
<version>2.10.4</version>
|
||||
<configuration>
|
||||
<charset>UTF-8</charset>
|
||||
</configuration>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>attach-javadocs</id>
|
||||
<goals>
|
||||
<goal>jar</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<artifactId>maven-source-plugin</artifactId>
|
||||
<version>3.0.1</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>attach-sources</id>
|
||||
<goals>
|
||||
<goal>jar</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<artifactId>maven-help-plugin</artifactId>
|
||||
<version>2.2</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>generate-effective-dependencies-pom</id>
|
||||
<phase>generate-resources</phase>
|
||||
<goals>
|
||||
<goal>effective-pom</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<output>${project.build.directory}/effective-pom/effective-dependencies.xml</output>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-checkstyle-plugin</artifactId>
|
||||
<version>2.17</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>verify</id>
|
||||
<phase>verify</phase>
|
||||
<configuration>
|
||||
<configLocation>checkstyle/checkstyle.xml</configLocation>
|
||||
<encoding>UTF-8</encoding>
|
||||
<consoleOutput>true</consoleOutput>
|
||||
<failsOnError>true</failsOnError>
|
||||
</configuration>
|
||||
<goals>
|
||||
<goal>check</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<artifactId>maven-resources-plugin</artifactId>
|
||||
<version>3.0.2</version>
|
||||
<configuration>
|
||||
<resources>
|
||||
<resource>
|
||||
<directory>src/main/resources</directory>
|
||||
<filtering>false</filtering>
|
||||
</resource>
|
||||
</resources>
|
||||
<encoding>${project.build.sourceEncoding}</encoding>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
<profiles>
|
||||
<profile>
|
||||
<id>jdk8</id>
|
||||
<activation>
|
||||
<jdk>[1.8,)</jdk>
|
||||
</activation>
|
||||
<!-- Disable doclint under JDK 8 -->
|
||||
<reporting>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<artifactId>maven-javadoc-plugin</artifactId>
|
||||
<version>2.10.4</version>
|
||||
<configuration>
|
||||
<additionalparam>-Xdoclint:none</additionalparam>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</reporting>
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<artifactId>maven-javadoc-plugin</artifactId>
|
||||
<version>2.10.4</version>
|
||||
<configuration>
|
||||
<additionalparam>-Xdoclint:none</additionalparam>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</profile>
|
||||
<profile>
|
||||
<id>release-sign-artifacts</id>
|
||||
<activation>
|
||||
<property>
|
||||
<name>performRelease</name>
|
||||
<value>true</value>
|
||||
</property>
|
||||
</activation>
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<artifactId>maven-gpg-plugin</artifactId>
|
||||
<version>1.6</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>sign-artifacts</id>
|
||||
<phase>verify</phase>
|
||||
<goals>
|
||||
<goal>sign</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</profile>
|
||||
</profiles>
|
||||
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-client</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-broker</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-common</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-store</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-namesrv</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-tools</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-remoting</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-qatest</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-filtersrv</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-srvutil</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>junit</groupId>
|
||||
<artifactId>junit</artifactId>
|
||||
<version>4.11</version>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.slf4j</groupId>
|
||||
<artifactId>slf4j-api</artifactId>
|
||||
<version>1.7.5</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>ch.qos.logback</groupId>
|
||||
<artifactId>logback-classic</artifactId>
|
||||
<version>1.0.13</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>ch.qos.logback</groupId>
|
||||
<artifactId>logback-core</artifactId>
|
||||
<version>1.0.13</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>commons-cli</groupId>
|
||||
<artifactId>commons-cli</artifactId>
|
||||
<version>1.2</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.netty</groupId>
|
||||
<artifactId>netty-all</artifactId>
|
||||
<version>4.0.36.Final</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.alibaba</groupId>
|
||||
<artifactId>fastjson</artifactId>
|
||||
<version>1.2.12</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.javassist</groupId>
|
||||
<artifactId>javassist</artifactId>
|
||||
<version>3.20.0-GA</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.apache.commons</groupId>
|
||||
<artifactId>commons-lang3</artifactId>
|
||||
<version>3.4</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>net.java.dev.jna</groupId>
|
||||
<artifactId>jna</artifactId>
|
||||
<version>4.2.2</version>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
</project>
|
||||
@@ -0,0 +1,52 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<assembly>
|
||||
<id>client</id>
|
||||
<baseDirectory>alibaba-rocketmq-client-java</baseDirectory>
|
||||
<formats>
|
||||
<format>dir</format>
|
||||
<format>tar.gz</format>
|
||||
</formats>
|
||||
<fileSets>
|
||||
<fileSet>
|
||||
<includes>
|
||||
<include>LICENSE</include>
|
||||
<include>NOTICE</include>
|
||||
</includes>
|
||||
</fileSet>
|
||||
</fileSets>
|
||||
|
||||
|
||||
<moduleSets>
|
||||
<moduleSet>
|
||||
<includes>
|
||||
<include>com.alibaba.rocketmq:rocketmq-client</include>
|
||||
</includes>
|
||||
<binaries>
|
||||
<outputDirectory>./</outputDirectory>
|
||||
<unpack>false</unpack>
|
||||
<dependencySets>
|
||||
<dependencySet>
|
||||
<outputDirectory>./</outputDirectory>
|
||||
</dependencySet>
|
||||
</dependencySets>
|
||||
</binaries>
|
||||
</moduleSet>
|
||||
</moduleSets>
|
||||
</assembly>
|
||||
+60
@@ -0,0 +1,60 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain producerGroup copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<assembly>
|
||||
<id>broker</id>
|
||||
<baseDirectory>alibaba-rocketmq</baseDirectory>
|
||||
<formats>
|
||||
<format>dir</format>
|
||||
<format>tar.gz</format>
|
||||
</formats>
|
||||
<fileSets>
|
||||
<fileSet>
|
||||
<includes>
|
||||
<include>bin/*</include>
|
||||
<include>conf/*</include>
|
||||
<include>benchmark/*</include>
|
||||
<include>LICENSE</include>
|
||||
<include>NOTICE</include>
|
||||
</includes>
|
||||
</fileSet>
|
||||
</fileSets>
|
||||
|
||||
|
||||
<moduleSets>
|
||||
<moduleSet>
|
||||
<includes>
|
||||
<include>com.alibaba.rocketmq:rocketmq-broker</include>
|
||||
<include>com.alibaba.rocketmq:rocketmq-tools</include>
|
||||
<include>com.alibaba.rocketmq:rocketmq-client</include>
|
||||
<include>com.alibaba.rocketmq:rocketmq-namesrv</include>
|
||||
<include>com.alibaba.rocketmq:rocketmq-filtersrv</include>
|
||||
<include>com.alibaba.rocketmq:rocketmq-example</include>
|
||||
</includes>
|
||||
<binaries>
|
||||
<outputDirectory>lib/</outputDirectory>
|
||||
<unpack>false</unpack>
|
||||
<dependencySets>
|
||||
<dependencySet>
|
||||
<outputDirectory>lib/</outputDirectory>
|
||||
</dependencySet>
|
||||
</dependencySets>
|
||||
</binaries>
|
||||
</moduleSet>
|
||||
</moduleSets>
|
||||
</assembly>
|
||||
@@ -0,0 +1,79 @@
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<parent>
|
||||
<groupId>com.alibaba.rocketmq</groupId>
|
||||
<artifactId>rocketmq-all</artifactId>
|
||||
<version>4.0.0-SNAPSHOT</version>
|
||||
</parent>
|
||||
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<packaging>jar</packaging>
|
||||
<artifactId>rocketmq-broker</artifactId>
|
||||
<name>rocketmq-broker ${project.version}</name>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>junit</groupId>
|
||||
<artifactId>junit</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<!--<dependency>-->
|
||||
<!--<groupId>commons-beanutils</groupId>-->
|
||||
<!--<artifactId>commons-beanutils</artifactId>-->
|
||||
<!--<version>1.9.2</version>-->
|
||||
<!--</dependency>-->
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-common</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-store</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-remoting</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-client</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-srvutil</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>ch.qos.logback</groupId>
|
||||
<artifactId>logback-classic</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>ch.qos.logback</groupId>
|
||||
<artifactId>logback-core</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.alibaba</groupId>
|
||||
<artifactId>fastjson</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.javassist</groupId>
|
||||
<artifactId>javassist</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</project>
|
||||
@@ -0,0 +1,773 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker;
|
||||
|
||||
import com.alibaba.rocketmq.broker.client.*;
|
||||
import com.alibaba.rocketmq.broker.client.net.Broker2Client;
|
||||
import com.alibaba.rocketmq.broker.client.rebalance.RebalanceLockManager;
|
||||
import com.alibaba.rocketmq.broker.filtersrv.FilterServerManager;
|
||||
import com.alibaba.rocketmq.broker.latency.BrokerFastFailure;
|
||||
import com.alibaba.rocketmq.broker.latency.BrokerFixedThreadPoolExecutor;
|
||||
import com.alibaba.rocketmq.broker.longpolling.NotifyMessageArrivingListener;
|
||||
import com.alibaba.rocketmq.broker.longpolling.PullRequestHoldService;
|
||||
import com.alibaba.rocketmq.broker.mqtrace.ConsumeMessageHook;
|
||||
import com.alibaba.rocketmq.broker.mqtrace.SendMessageHook;
|
||||
import com.alibaba.rocketmq.broker.offset.ConsumerOffsetManager;
|
||||
import com.alibaba.rocketmq.broker.out.BrokerOuterAPI;
|
||||
import com.alibaba.rocketmq.broker.plugin.MessageStoreFactory;
|
||||
import com.alibaba.rocketmq.broker.plugin.MessageStorePluginContext;
|
||||
import com.alibaba.rocketmq.broker.processor.*;
|
||||
import com.alibaba.rocketmq.broker.slave.SlaveSynchronize;
|
||||
import com.alibaba.rocketmq.broker.subscription.SubscriptionGroupManager;
|
||||
import com.alibaba.rocketmq.broker.topic.TopicConfigManager;
|
||||
import com.alibaba.rocketmq.common.*;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.constant.PermName;
|
||||
import com.alibaba.rocketmq.common.namesrv.RegisterBrokerResult;
|
||||
import com.alibaba.rocketmq.common.protocol.RequestCode;
|
||||
import com.alibaba.rocketmq.common.protocol.body.TopicConfigSerializeWrapper;
|
||||
import com.alibaba.rocketmq.common.stats.MomentStatsItem;
|
||||
import com.alibaba.rocketmq.remoting.RPCHook;
|
||||
import com.alibaba.rocketmq.remoting.RemotingServer;
|
||||
import com.alibaba.rocketmq.remoting.netty.*;
|
||||
import com.alibaba.rocketmq.store.DefaultMessageStore;
|
||||
import com.alibaba.rocketmq.store.MessageArrivingListener;
|
||||
import com.alibaba.rocketmq.store.MessageStore;
|
||||
import com.alibaba.rocketmq.store.config.BrokerRole;
|
||||
import com.alibaba.rocketmq.store.config.MessageStoreConfig;
|
||||
import com.alibaba.rocketmq.store.stats.BrokerStats;
|
||||
import com.alibaba.rocketmq.store.stats.BrokerStatsManager;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.*;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class BrokerController {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private static final Logger LOG_PROTECTION = LoggerFactory.getLogger(LoggerName.PROTECTION_LOGGER_NAME);
|
||||
private static final Logger LOG_WATER_MARK = LoggerFactory.getLogger(LoggerName.WATER_MARK_LOGGER_NAME);
|
||||
private final BrokerConfig brokerConfig;
|
||||
private final NettyServerConfig nettyServerConfig;
|
||||
private final NettyClientConfig nettyClientConfig;
|
||||
private final MessageStoreConfig messageStoreConfig;
|
||||
private final ConsumerOffsetManager consumerOffsetManager;
|
||||
private final ConsumerManager consumerManager;
|
||||
private final ProducerManager producerManager;
|
||||
private final ClientHousekeepingService clientHousekeepingService;
|
||||
private final PullMessageProcessor pullMessageProcessor;
|
||||
private final PullRequestHoldService pullRequestHoldService;
|
||||
private final MessageArrivingListener messageArrivingListener;
|
||||
private final Broker2Client broker2Client;
|
||||
private final SubscriptionGroupManager subscriptionGroupManager;
|
||||
private final ConsumerIdsChangeListener consumerIdsChangeListener;
|
||||
private final RebalanceLockManager rebalanceLockManager = new RebalanceLockManager();
|
||||
private final BrokerOuterAPI brokerOuterAPI;
|
||||
private final ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor(new ThreadFactoryImpl(
|
||||
"BrokerControllerScheduledThread"));
|
||||
private final SlaveSynchronize slaveSynchronize;
|
||||
private final BlockingQueue<Runnable> sendThreadPoolQueue;
|
||||
private final BlockingQueue<Runnable> pullThreadPoolQueue;
|
||||
private final BlockingQueue<Runnable> clientManagerThreadPoolQueue;
|
||||
private final BlockingQueue<Runnable> consumerManagerThreadPoolQueue;
|
||||
private final FilterServerManager filterServerManager;
|
||||
private final BrokerStatsManager brokerStatsManager;
|
||||
private final List<SendMessageHook> sendMessageHookList = new ArrayList<SendMessageHook>();
|
||||
private final List<ConsumeMessageHook> consumeMessageHookList = new ArrayList<ConsumeMessageHook>();
|
||||
private MessageStore messageStore;
|
||||
private RemotingServer remotingServer;
|
||||
private RemotingServer fastRemotingServer;
|
||||
private TopicConfigManager topicConfigManager;
|
||||
private ExecutorService sendMessageExecutor;
|
||||
private ExecutorService pullMessageExecutor;
|
||||
private ExecutorService adminBrokerExecutor;
|
||||
private ExecutorService clientManageExecutor;
|
||||
private ExecutorService consumerManageExecutor;
|
||||
private boolean updateMasterHAServerAddrPeriodically = false;
|
||||
private BrokerStats brokerStats;
|
||||
private InetSocketAddress storeHost;
|
||||
private BrokerFastFailure brokerFastFailure;
|
||||
private Configuration configuration;
|
||||
|
||||
public BrokerController(//
|
||||
final BrokerConfig brokerConfig, //
|
||||
final NettyServerConfig nettyServerConfig, //
|
||||
final NettyClientConfig nettyClientConfig, //
|
||||
final MessageStoreConfig messageStoreConfig //
|
||||
) {
|
||||
this.brokerConfig = brokerConfig;
|
||||
this.nettyServerConfig = nettyServerConfig;
|
||||
this.nettyClientConfig = nettyClientConfig;
|
||||
this.messageStoreConfig = messageStoreConfig;
|
||||
this.consumerOffsetManager = new ConsumerOffsetManager(this);
|
||||
this.topicConfigManager = new TopicConfigManager(this);
|
||||
this.pullMessageProcessor = new PullMessageProcessor(this);
|
||||
this.pullRequestHoldService = new PullRequestHoldService(this);
|
||||
this.messageArrivingListener = new NotifyMessageArrivingListener(this.pullRequestHoldService);
|
||||
this.consumerIdsChangeListener = new DefaultConsumerIdsChangeListener(this);
|
||||
this.consumerManager = new ConsumerManager(this.consumerIdsChangeListener);
|
||||
this.producerManager = new ProducerManager();
|
||||
this.clientHousekeepingService = new ClientHousekeepingService(this);
|
||||
this.broker2Client = new Broker2Client(this);
|
||||
this.subscriptionGroupManager = new SubscriptionGroupManager(this);
|
||||
this.brokerOuterAPI = new BrokerOuterAPI(nettyClientConfig);
|
||||
this.filterServerManager = new FilterServerManager(this);
|
||||
|
||||
if (this.brokerConfig.getNamesrvAddr() != null) {
|
||||
this.brokerOuterAPI.updateNameServerAddressList(this.brokerConfig.getNamesrvAddr());
|
||||
log.info("user specfied name server address: {}", this.brokerConfig.getNamesrvAddr());
|
||||
}
|
||||
|
||||
this.slaveSynchronize = new SlaveSynchronize(this);
|
||||
|
||||
this.sendThreadPoolQueue = new LinkedBlockingQueue<Runnable>(this.brokerConfig.getSendThreadPoolQueueCapacity());
|
||||
|
||||
this.pullThreadPoolQueue = new LinkedBlockingQueue<Runnable>(this.brokerConfig.getPullThreadPoolQueueCapacity());
|
||||
this.clientManagerThreadPoolQueue = new LinkedBlockingQueue<Runnable>(this.brokerConfig.getClientManagerThreadPoolQueueCapacity());
|
||||
this.consumerManagerThreadPoolQueue = new LinkedBlockingQueue<Runnable>(this.brokerConfig.getConsumerManagerThreadPoolQueueCapacity());
|
||||
|
||||
this.brokerStatsManager = new BrokerStatsManager(this.brokerConfig.getBrokerClusterName());
|
||||
this.setStoreHost(new InetSocketAddress(this.getBrokerConfig().getBrokerIP1(), this.getNettyServerConfig().getListenPort()));
|
||||
|
||||
this.brokerFastFailure = new BrokerFastFailure(this);
|
||||
this.configuration = new Configuration(
|
||||
log,
|
||||
BrokerPathConfigHelper.getBrokerConfigPath(),
|
||||
this.brokerConfig, this.nettyServerConfig, this.nettyClientConfig, this.messageStoreConfig
|
||||
);
|
||||
}
|
||||
|
||||
public BrokerConfig getBrokerConfig() {
|
||||
return brokerConfig;
|
||||
}
|
||||
|
||||
public NettyServerConfig getNettyServerConfig() {
|
||||
return nettyServerConfig;
|
||||
}
|
||||
|
||||
public BlockingQueue<Runnable> getPullThreadPoolQueue() {
|
||||
return pullThreadPoolQueue;
|
||||
}
|
||||
|
||||
public boolean initialize() throws CloneNotSupportedException {
|
||||
boolean result = true;
|
||||
|
||||
result = result && this.topicConfigManager.load();
|
||||
|
||||
result = result && this.consumerOffsetManager.load();
|
||||
result = result && this.subscriptionGroupManager.load();
|
||||
|
||||
if (result) {
|
||||
try {
|
||||
this.messageStore =
|
||||
new DefaultMessageStore(this.messageStoreConfig, this.brokerStatsManager, this.messageArrivingListener,
|
||||
this.brokerConfig);
|
||||
this.brokerStats = new BrokerStats((DefaultMessageStore) this.messageStore);
|
||||
//load plugin
|
||||
MessageStorePluginContext context = new MessageStorePluginContext(messageStoreConfig, brokerStatsManager, messageArrivingListener, brokerConfig);
|
||||
this.messageStore = MessageStoreFactory.build(context, this.messageStore);
|
||||
} catch (IOException e) {
|
||||
result = false;
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
result = result && this.messageStore.load();
|
||||
|
||||
if (result) {
|
||||
this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.clientHousekeepingService);
|
||||
NettyServerConfig fastConfig = (NettyServerConfig) this.nettyServerConfig.clone();
|
||||
fastConfig.setListenPort(nettyServerConfig.getListenPort() - 2);
|
||||
this.fastRemotingServer = new NettyRemotingServer(fastConfig, this.clientHousekeepingService);
|
||||
this.sendMessageExecutor = new BrokerFixedThreadPoolExecutor(
|
||||
this.brokerConfig.getSendMessageThreadPoolNums(),
|
||||
this.brokerConfig.getSendMessageThreadPoolNums(),
|
||||
1000 * 60,
|
||||
TimeUnit.MILLISECONDS,
|
||||
this.sendThreadPoolQueue,
|
||||
new ThreadFactoryImpl("SendMessageThread_"));
|
||||
|
||||
this.pullMessageExecutor = new BrokerFixedThreadPoolExecutor(
|
||||
this.brokerConfig.getPullMessageThreadPoolNums(),
|
||||
this.brokerConfig.getPullMessageThreadPoolNums(),
|
||||
1000 * 60,
|
||||
TimeUnit.MILLISECONDS,
|
||||
this.pullThreadPoolQueue,
|
||||
new ThreadFactoryImpl("PullMessageThread_"));
|
||||
|
||||
this.adminBrokerExecutor =
|
||||
Executors.newFixedThreadPool(this.brokerConfig.getAdminBrokerThreadPoolNums(), new ThreadFactoryImpl(
|
||||
"AdminBrokerThread_"));
|
||||
|
||||
this.clientManageExecutor = new ThreadPoolExecutor(
|
||||
this.brokerConfig.getClientManageThreadPoolNums(),
|
||||
this.brokerConfig.getClientManageThreadPoolNums(),
|
||||
1000 * 60,
|
||||
TimeUnit.MILLISECONDS,
|
||||
this.clientManagerThreadPoolQueue,
|
||||
new ThreadFactoryImpl("ClientManageThread_"));
|
||||
|
||||
this.consumerManageExecutor =
|
||||
Executors.newFixedThreadPool(this.brokerConfig.getConsumerManageThreadPoolNums(), new ThreadFactoryImpl(
|
||||
"ConsumerManageThread_"));
|
||||
|
||||
this.registerProcessor();
|
||||
|
||||
|
||||
// TODO remove in future
|
||||
final long initialDelay = UtilAll.computNextMorningTimeMillis() - System.currentTimeMillis();
|
||||
final long period = 1000 * 60 * 60 * 24;
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
BrokerController.this.getBrokerStats().record();
|
||||
} catch (Throwable e) {
|
||||
log.error("schedule record error.", e);
|
||||
}
|
||||
}
|
||||
}, initialDelay, period, TimeUnit.MILLISECONDS);
|
||||
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
BrokerController.this.consumerOffsetManager.persist();
|
||||
} catch (Throwable e) {
|
||||
log.error("schedule persist consumerOffset error.", e);
|
||||
}
|
||||
}
|
||||
}, 1000 * 10, this.brokerConfig.getFlushConsumerOffsetInterval(), TimeUnit.MILLISECONDS);
|
||||
|
||||
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
BrokerController.this.protectBroker();
|
||||
} catch (Exception e) {
|
||||
log.error("protectBroker error.", e);
|
||||
}
|
||||
}
|
||||
}, 3, 3, TimeUnit.MINUTES);
|
||||
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
BrokerController.this.printWaterMark();
|
||||
} catch (Exception e) {
|
||||
log.error("printWaterMark error.", e);
|
||||
}
|
||||
}
|
||||
}, 10, 1, TimeUnit.SECONDS);
|
||||
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
log.info("dispatch behind commit log {} bytes", BrokerController.this.getMessageStore().dispatchBehindBytes());
|
||||
} catch (Throwable e) {
|
||||
log.error("schedule dispatchBehindBytes error.", e);
|
||||
}
|
||||
}
|
||||
}, 1000 * 10, 1000 * 60, TimeUnit.MILLISECONDS);
|
||||
|
||||
if (this.brokerConfig.getNamesrvAddr() != null) {
|
||||
this.brokerOuterAPI.updateNameServerAddressList(this.brokerConfig.getNamesrvAddr());
|
||||
} else if (this.brokerConfig.isFetchNamesrvAddrByAddressServer()) {
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
BrokerController.this.brokerOuterAPI.fetchNameServerAddr();
|
||||
} catch (Throwable e) {
|
||||
log.error("ScheduledTask fetchNameServerAddr exception", e);
|
||||
}
|
||||
}
|
||||
}, 1000 * 10, 1000 * 60 * 2, TimeUnit.MILLISECONDS);
|
||||
}
|
||||
|
||||
if (BrokerRole.SLAVE == this.messageStoreConfig.getBrokerRole()) {
|
||||
if (this.messageStoreConfig.getHaMasterAddress() != null && this.messageStoreConfig.getHaMasterAddress().length() >= 6) {
|
||||
this.messageStore.updateHaMasterAddress(this.messageStoreConfig.getHaMasterAddress());
|
||||
this.updateMasterHAServerAddrPeriodically = false;
|
||||
} else {
|
||||
this.updateMasterHAServerAddrPeriodically = true;
|
||||
}
|
||||
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
BrokerController.this.slaveSynchronize.syncAll();
|
||||
} catch (Throwable e) {
|
||||
log.error("ScheduledTask syncAll slave exception", e);
|
||||
}
|
||||
}
|
||||
}, 1000 * 10, 1000 * 60, TimeUnit.MILLISECONDS);
|
||||
} else {
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
BrokerController.this.printMasterAndSlaveDiff();
|
||||
} catch (Throwable e) {
|
||||
log.error("schedule printMasterAndSlaveDiff error.", e);
|
||||
}
|
||||
}
|
||||
}, 1000 * 10, 1000 * 60, TimeUnit.MILLISECONDS);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
public void registerProcessor() {
|
||||
/**
|
||||
* SendMessageProcessor
|
||||
*/
|
||||
SendMessageProcessor sendProcessor = new SendMessageProcessor(this);
|
||||
sendProcessor.registerSendMessageHook(sendMessageHookList);
|
||||
sendProcessor.registerConsumeMessageHook(consumeMessageHookList);
|
||||
|
||||
this.remotingServer.registerProcessor(RequestCode.SEND_MESSAGE, sendProcessor, this.sendMessageExecutor);
|
||||
this.remotingServer.registerProcessor(RequestCode.SEND_MESSAGE_V2, sendProcessor, this.sendMessageExecutor);
|
||||
this.remotingServer.registerProcessor(RequestCode.CONSUMER_SEND_MSG_BACK, sendProcessor, this.sendMessageExecutor);
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.SEND_MESSAGE, sendProcessor, this.sendMessageExecutor);
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.SEND_MESSAGE_V2, sendProcessor, this.sendMessageExecutor);
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.CONSUMER_SEND_MSG_BACK, sendProcessor, this.sendMessageExecutor);
|
||||
/**
|
||||
* PullMessageProcessor
|
||||
*/
|
||||
this.remotingServer.registerProcessor(RequestCode.PULL_MESSAGE, this.pullMessageProcessor, this.pullMessageExecutor);
|
||||
this.pullMessageProcessor.registerConsumeMessageHook(consumeMessageHookList);
|
||||
|
||||
/**
|
||||
* QueryMessageProcessor
|
||||
*/
|
||||
NettyRequestProcessor queryProcessor = new QueryMessageProcessor(this);
|
||||
this.remotingServer.registerProcessor(RequestCode.QUERY_MESSAGE, queryProcessor, this.pullMessageExecutor);
|
||||
this.remotingServer.registerProcessor(RequestCode.VIEW_MESSAGE_BY_ID, queryProcessor, this.pullMessageExecutor);
|
||||
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.QUERY_MESSAGE, queryProcessor, this.pullMessageExecutor);
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.VIEW_MESSAGE_BY_ID, queryProcessor, this.pullMessageExecutor);
|
||||
|
||||
/**
|
||||
* ClientManageProcessor
|
||||
*/
|
||||
ClientManageProcessor clientProcessor = new ClientManageProcessor(this);
|
||||
this.remotingServer.registerProcessor(RequestCode.HEART_BEAT, clientProcessor, this.clientManageExecutor);
|
||||
this.remotingServer.registerProcessor(RequestCode.UNREGISTER_CLIENT, clientProcessor, this.clientManageExecutor);
|
||||
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.HEART_BEAT, clientProcessor, this.clientManageExecutor);
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.UNREGISTER_CLIENT, clientProcessor, this.clientManageExecutor);
|
||||
|
||||
/**
|
||||
* ConsumerManageProcessor
|
||||
*/
|
||||
ConsumerManageProcessor consumerManageProcessor = new ConsumerManageProcessor(this);
|
||||
this.remotingServer.registerProcessor(RequestCode.GET_CONSUMER_LIST_BY_GROUP, consumerManageProcessor, this.consumerManageExecutor);
|
||||
this.remotingServer.registerProcessor(RequestCode.UPDATE_CONSUMER_OFFSET, consumerManageProcessor, this.consumerManageExecutor);
|
||||
this.remotingServer.registerProcessor(RequestCode.QUERY_CONSUMER_OFFSET, consumerManageProcessor, this.consumerManageExecutor);
|
||||
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.GET_CONSUMER_LIST_BY_GROUP, consumerManageProcessor, this.consumerManageExecutor);
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.UPDATE_CONSUMER_OFFSET, consumerManageProcessor, this.consumerManageExecutor);
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.QUERY_CONSUMER_OFFSET, consumerManageProcessor, this.consumerManageExecutor);
|
||||
|
||||
|
||||
/**
|
||||
* EndTransactionProcessor
|
||||
*/
|
||||
this.remotingServer.registerProcessor(RequestCode.END_TRANSACTION, new EndTransactionProcessor(this), this.sendMessageExecutor);
|
||||
this.fastRemotingServer.registerProcessor(RequestCode.END_TRANSACTION, new EndTransactionProcessor(this), this.sendMessageExecutor);
|
||||
|
||||
/**
|
||||
* Default
|
||||
*/
|
||||
AdminBrokerProcessor adminProcessor = new AdminBrokerProcessor(this);
|
||||
this.remotingServer.registerDefaultProcessor(adminProcessor, this.adminBrokerExecutor);
|
||||
this.fastRemotingServer.registerDefaultProcessor(adminProcessor, this.adminBrokerExecutor);
|
||||
}
|
||||
|
||||
public BrokerStats getBrokerStats() {
|
||||
return brokerStats;
|
||||
}
|
||||
|
||||
public void setBrokerStats(BrokerStats brokerStats) {
|
||||
this.brokerStats = brokerStats;
|
||||
}
|
||||
|
||||
public void protectBroker() {
|
||||
if (this.brokerConfig.isDisableConsumeIfConsumerReadSlowly()) {
|
||||
final Iterator<Map.Entry<String, MomentStatsItem>> it = this.brokerStatsManager.getMomentStatsItemSetFallSize().getStatsItemTable().entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
final Map.Entry<String, MomentStatsItem> next = it.next();
|
||||
final long fallBehindBytes = next.getValue().getValue().get();
|
||||
if (fallBehindBytes > this.brokerConfig.getConsumerFallbehindThreshold()) {
|
||||
final String[] split = next.getValue().getStatsKey().split("@");
|
||||
final String group = split[2];
|
||||
LOG_PROTECTION.info("[PROTECT_BROKER] the consumer[{}] consume slowly, {} bytes, disable it", group, fallBehindBytes);
|
||||
this.subscriptionGroupManager.disableConsume(group);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public long headSlowTimeMills(BlockingQueue<Runnable> q) {
|
||||
long slowTimeMills = 0;
|
||||
final Runnable peek = q.peek();
|
||||
if (peek != null) {
|
||||
RequestTask rt = BrokerFastFailure.castRunnable(peek);
|
||||
slowTimeMills = this.messageStore.now() - rt.getCreateTimestamp();
|
||||
}
|
||||
|
||||
if (slowTimeMills < 0) slowTimeMills = 0;
|
||||
|
||||
return slowTimeMills;
|
||||
}
|
||||
|
||||
public long headSlowTimeMills4SendThreadPoolQueue() {
|
||||
return this.headSlowTimeMills(this.sendThreadPoolQueue);
|
||||
}
|
||||
|
||||
public long headSlowTimeMills4PullThreadPoolQueue() {
|
||||
return this.headSlowTimeMills(this.pullThreadPoolQueue);
|
||||
}
|
||||
|
||||
public void printWaterMark() {
|
||||
LOG_WATER_MARK.info("[WATERMARK] Send Queue Size: {} SlowTimeMills: {}", this.sendThreadPoolQueue.size(), headSlowTimeMills4SendThreadPoolQueue());
|
||||
LOG_WATER_MARK.info("[WATERMARK] Pull Queue Size: {} SlowTimeMills: {}", this.pullThreadPoolQueue.size(), headSlowTimeMills4PullThreadPoolQueue());
|
||||
}
|
||||
|
||||
public MessageStore getMessageStore() {
|
||||
return messageStore;
|
||||
}
|
||||
|
||||
public void setMessageStore(MessageStore messageStore) {
|
||||
this.messageStore = messageStore;
|
||||
}
|
||||
|
||||
private void printMasterAndSlaveDiff() {
|
||||
long diff = this.messageStore.slaveFallBehindMuch();
|
||||
|
||||
// XXX: warn and notify me
|
||||
log.info("slave fall behind master, how much, {} bytes", diff);
|
||||
}
|
||||
|
||||
public Broker2Client getBroker2Client() {
|
||||
return broker2Client;
|
||||
}
|
||||
|
||||
public ConsumerManager getConsumerManager() {
|
||||
return consumerManager;
|
||||
}
|
||||
|
||||
public ConsumerOffsetManager getConsumerOffsetManager() {
|
||||
return consumerOffsetManager;
|
||||
}
|
||||
|
||||
public MessageStoreConfig getMessageStoreConfig() {
|
||||
return messageStoreConfig;
|
||||
}
|
||||
|
||||
public ProducerManager getProducerManager() {
|
||||
return producerManager;
|
||||
}
|
||||
|
||||
public void setFastRemotingServer(RemotingServer fastRemotingServer) {
|
||||
this.fastRemotingServer = fastRemotingServer;
|
||||
}
|
||||
|
||||
public PullMessageProcessor getPullMessageProcessor() {
|
||||
return pullMessageProcessor;
|
||||
}
|
||||
|
||||
public PullRequestHoldService getPullRequestHoldService() {
|
||||
return pullRequestHoldService;
|
||||
}
|
||||
|
||||
public SubscriptionGroupManager getSubscriptionGroupManager() {
|
||||
return subscriptionGroupManager;
|
||||
}
|
||||
|
||||
public void shutdown() {
|
||||
if (this.brokerStatsManager != null) {
|
||||
this.brokerStatsManager.shutdown();
|
||||
}
|
||||
|
||||
if (this.clientHousekeepingService != null) {
|
||||
this.clientHousekeepingService.shutdown();
|
||||
}
|
||||
|
||||
if (this.pullRequestHoldService != null) {
|
||||
this.pullRequestHoldService.shutdown();
|
||||
}
|
||||
|
||||
if (this.remotingServer != null) {
|
||||
this.remotingServer.shutdown();
|
||||
}
|
||||
|
||||
if (this.fastRemotingServer != null) {
|
||||
this.fastRemotingServer.shutdown();
|
||||
}
|
||||
|
||||
if (this.messageStore != null) {
|
||||
this.messageStore.shutdown();
|
||||
}
|
||||
|
||||
this.scheduledExecutorService.shutdown();
|
||||
try {
|
||||
this.scheduledExecutorService.awaitTermination(5000, TimeUnit.MILLISECONDS);
|
||||
} catch (InterruptedException e) {
|
||||
}
|
||||
|
||||
this.unregisterBrokerAll();
|
||||
|
||||
if (this.sendMessageExecutor != null) {
|
||||
this.sendMessageExecutor.shutdown();
|
||||
}
|
||||
|
||||
if (this.pullMessageExecutor != null) {
|
||||
this.pullMessageExecutor.shutdown();
|
||||
}
|
||||
|
||||
if (this.adminBrokerExecutor != null) {
|
||||
this.adminBrokerExecutor.shutdown();
|
||||
}
|
||||
|
||||
if (this.brokerOuterAPI != null) {
|
||||
this.brokerOuterAPI.shutdown();
|
||||
}
|
||||
|
||||
this.consumerOffsetManager.persist();
|
||||
|
||||
if (this.filterServerManager != null) {
|
||||
this.filterServerManager.shutdown();
|
||||
}
|
||||
|
||||
if (this.brokerFastFailure != null) {
|
||||
this.brokerFastFailure.shutdown();
|
||||
}
|
||||
}
|
||||
|
||||
private void unregisterBrokerAll() {
|
||||
this.brokerOuterAPI.unregisterBrokerAll(
|
||||
this.brokerConfig.getBrokerClusterName(),
|
||||
this.getBrokerAddr(),
|
||||
this.brokerConfig.getBrokerName(),
|
||||
this.brokerConfig.getBrokerId());
|
||||
}
|
||||
|
||||
public String getBrokerAddr() {
|
||||
return this.brokerConfig.getBrokerIP1() + ":" + this.nettyServerConfig.getListenPort();
|
||||
}
|
||||
|
||||
public void start() throws Exception {
|
||||
if (this.messageStore != null) {
|
||||
this.messageStore.start();
|
||||
}
|
||||
|
||||
if (this.remotingServer != null) {
|
||||
this.remotingServer.start();
|
||||
}
|
||||
|
||||
if (this.fastRemotingServer != null) {
|
||||
this.fastRemotingServer.start();
|
||||
}
|
||||
|
||||
if (this.brokerOuterAPI != null) {
|
||||
this.brokerOuterAPI.start();
|
||||
}
|
||||
|
||||
if (this.pullRequestHoldService != null) {
|
||||
this.pullRequestHoldService.start();
|
||||
}
|
||||
|
||||
if (this.clientHousekeepingService != null) {
|
||||
this.clientHousekeepingService.start();
|
||||
}
|
||||
|
||||
if (this.filterServerManager != null) {
|
||||
this.filterServerManager.start();
|
||||
}
|
||||
|
||||
this.registerBrokerAll(true, false);
|
||||
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
BrokerController.this.registerBrokerAll(true, false);
|
||||
} catch (Throwable e) {
|
||||
log.error("registerBrokerAll Exception", e);
|
||||
}
|
||||
}
|
||||
}, 1000 * 10, 1000 * 30, TimeUnit.MILLISECONDS);
|
||||
|
||||
if (this.brokerStatsManager != null) {
|
||||
this.brokerStatsManager.start();
|
||||
}
|
||||
|
||||
if (this.brokerFastFailure != null) {
|
||||
this.brokerFastFailure.start();
|
||||
}
|
||||
}
|
||||
|
||||
public synchronized void registerBrokerAll(final boolean checkOrderConfig, boolean oneway) {
|
||||
TopicConfigSerializeWrapper topicConfigWrapper = this.getTopicConfigManager().buildTopicConfigSerializeWrapper();
|
||||
|
||||
if (!PermName.isWriteable(this.getBrokerConfig().getBrokerPermission())
|
||||
|| !PermName.isReadable(this.getBrokerConfig().getBrokerPermission())) {
|
||||
ConcurrentHashMap<String, TopicConfig> topicConfigTable = new ConcurrentHashMap<String, TopicConfig>();
|
||||
for (TopicConfig topicConfig : topicConfigWrapper.getTopicConfigTable().values()) {
|
||||
TopicConfig tmp =
|
||||
new TopicConfig(topicConfig.getTopicName(), topicConfig.getReadQueueNums(), topicConfig.getWriteQueueNums(),
|
||||
this.brokerConfig.getBrokerPermission());
|
||||
topicConfigTable.put(topicConfig.getTopicName(), tmp);
|
||||
}
|
||||
topicConfigWrapper.setTopicConfigTable(topicConfigTable);
|
||||
}
|
||||
|
||||
RegisterBrokerResult registerBrokerResult = this.brokerOuterAPI.registerBrokerAll(
|
||||
this.brokerConfig.getBrokerClusterName(),
|
||||
this.getBrokerAddr(),
|
||||
this.brokerConfig.getBrokerName(),
|
||||
this.brokerConfig.getBrokerId(),
|
||||
this.getHAServerAddr(),
|
||||
topicConfigWrapper,
|
||||
this.filterServerManager.buildNewFilterServerList(),
|
||||
oneway,
|
||||
this.brokerConfig.getRegisterBrokerTimeoutMills());
|
||||
|
||||
if (registerBrokerResult != null) {
|
||||
if (this.updateMasterHAServerAddrPeriodically && registerBrokerResult.getHaServerAddr() != null) {
|
||||
this.messageStore.updateHaMasterAddress(registerBrokerResult.getHaServerAddr());
|
||||
}
|
||||
|
||||
this.slaveSynchronize.setMasterAddr(registerBrokerResult.getMasterAddr());
|
||||
|
||||
if (checkOrderConfig) {
|
||||
this.getTopicConfigManager().updateOrderTopicConfig(registerBrokerResult.getKvTable());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public TopicConfigManager getTopicConfigManager() {
|
||||
return topicConfigManager;
|
||||
}
|
||||
|
||||
public void setTopicConfigManager(TopicConfigManager topicConfigManager) {
|
||||
this.topicConfigManager = topicConfigManager;
|
||||
}
|
||||
|
||||
public String getHAServerAddr() {
|
||||
return this.brokerConfig.getBrokerIP2() + ":" + this.messageStoreConfig.getHaListenPort();
|
||||
}
|
||||
|
||||
public RebalanceLockManager getRebalanceLockManager() {
|
||||
return rebalanceLockManager;
|
||||
}
|
||||
|
||||
public SlaveSynchronize getSlaveSynchronize() {
|
||||
return slaveSynchronize;
|
||||
}
|
||||
|
||||
public ExecutorService getPullMessageExecutor() {
|
||||
return pullMessageExecutor;
|
||||
}
|
||||
|
||||
public void setPullMessageExecutor(ExecutorService pullMessageExecutor) {
|
||||
this.pullMessageExecutor = pullMessageExecutor;
|
||||
}
|
||||
|
||||
public BlockingQueue<Runnable> getSendThreadPoolQueue() {
|
||||
return sendThreadPoolQueue;
|
||||
}
|
||||
|
||||
public FilterServerManager getFilterServerManager() {
|
||||
return filterServerManager;
|
||||
}
|
||||
|
||||
public BrokerStatsManager getBrokerStatsManager() {
|
||||
return brokerStatsManager;
|
||||
}
|
||||
|
||||
public List<SendMessageHook> getSendMessageHookList() {
|
||||
return sendMessageHookList;
|
||||
}
|
||||
|
||||
public void registerSendMessageHook(final SendMessageHook hook) {
|
||||
this.sendMessageHookList.add(hook);
|
||||
log.info("register SendMessageHook Hook, {}", hook.hookName());
|
||||
}
|
||||
|
||||
public List<ConsumeMessageHook> getConsumeMessageHookList() {
|
||||
return consumeMessageHookList;
|
||||
}
|
||||
|
||||
public void registerConsumeMessageHook(final ConsumeMessageHook hook) {
|
||||
this.consumeMessageHookList.add(hook);
|
||||
log.info("register ConsumeMessageHook Hook, {}", hook.hookName());
|
||||
}
|
||||
|
||||
public void registerServerRPCHook(RPCHook rpcHook) {
|
||||
getRemotingServer().registerRPCHook(rpcHook);
|
||||
}
|
||||
|
||||
public RemotingServer getRemotingServer() {
|
||||
return remotingServer;
|
||||
}
|
||||
|
||||
public void setRemotingServer(RemotingServer remotingServer) {
|
||||
this.remotingServer = remotingServer;
|
||||
}
|
||||
|
||||
public void registerClientRPCHook(RPCHook rpcHook) {
|
||||
this.getBrokerOuterAPI().registerRPCHook(rpcHook);
|
||||
}
|
||||
|
||||
public BrokerOuterAPI getBrokerOuterAPI() {
|
||||
return brokerOuterAPI;
|
||||
}
|
||||
|
||||
public InetSocketAddress getStoreHost() {
|
||||
return storeHost;
|
||||
}
|
||||
|
||||
public void setStoreHost(InetSocketAddress storeHost) {
|
||||
this.storeHost = storeHost;
|
||||
}
|
||||
|
||||
public Configuration getConfiguration() {
|
||||
return this.configuration;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker;
|
||||
|
||||
import java.io.File;
|
||||
|
||||
|
||||
public class BrokerPathConfigHelper {
|
||||
private static String brokerConfigPath = System.getProperty("user.home") + File.separator + "store"
|
||||
+ File.separator + "config" + File.separator + "broker.properties";
|
||||
|
||||
|
||||
public static String getBrokerConfigPath() {
|
||||
return brokerConfigPath;
|
||||
}
|
||||
|
||||
|
||||
public static void setBrokerConfigPath(String path) {
|
||||
brokerConfigPath = path;
|
||||
}
|
||||
|
||||
|
||||
public static String getTopicConfigPath(final String rootDir) {
|
||||
return rootDir + File.separator + "config" + File.separator + "topics.json";
|
||||
}
|
||||
|
||||
|
||||
public static String getConsumerOffsetPath(final String rootDir) {
|
||||
return rootDir + File.separator + "config" + File.separator + "consumerOffset.json";
|
||||
}
|
||||
|
||||
|
||||
public static String getSubscriptionGroupPath(final String rootDir) {
|
||||
return rootDir + File.separator + "config" + File.separator + "subscriptionGroup.json";
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,266 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker;
|
||||
|
||||
import ch.qos.logback.classic.LoggerContext;
|
||||
import ch.qos.logback.classic.joran.JoranConfigurator;
|
||||
import com.alibaba.rocketmq.common.BrokerConfig;
|
||||
import com.alibaba.rocketmq.common.MQVersion;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingUtil;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyClientConfig;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyServerConfig;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettySystemConfig;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import com.alibaba.rocketmq.srvutil.ServerUtil;
|
||||
import com.alibaba.rocketmq.store.config.BrokerRole;
|
||||
import com.alibaba.rocketmq.store.config.MessageStoreConfig;
|
||||
import org.apache.commons.cli.CommandLine;
|
||||
import org.apache.commons.cli.Option;
|
||||
import org.apache.commons.cli.Options;
|
||||
import org.apache.commons.cli.PosixParser;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.io.BufferedInputStream;
|
||||
import java.io.FileInputStream;
|
||||
import java.io.InputStream;
|
||||
import java.util.Properties;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class BrokerStartup {
|
||||
public static Properties properties = null;
|
||||
public static CommandLine commandLine = null;
|
||||
public static String configFile = null;
|
||||
public static Logger log;
|
||||
|
||||
public static void main(String[] args) {
|
||||
start(createBrokerController(args));
|
||||
}
|
||||
|
||||
public static BrokerController start(BrokerController controller) {
|
||||
try {
|
||||
controller.start();
|
||||
String tip = "The broker[" + controller.getBrokerConfig().getBrokerName() + ", "
|
||||
+ controller.getBrokerAddr() + "] boot success. serializeType=" + RemotingCommand.getSerializeTypeConfigInThisServer();
|
||||
|
||||
if (null != controller.getBrokerConfig().getNamesrvAddr()) {
|
||||
tip += " and name server is " + controller.getBrokerConfig().getNamesrvAddr();
|
||||
}
|
||||
|
||||
log.info(tip);
|
||||
return controller;
|
||||
} catch (Throwable e) {
|
||||
e.printStackTrace();
|
||||
System.exit(-1);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
public static BrokerController createBrokerController(String[] args) {
|
||||
System.setProperty(RemotingCommand.REMOTING_VERSION_KEY, Integer.toString(MQVersion.CURRENT_VERSION));
|
||||
|
||||
if (null == System.getProperty(NettySystemConfig.COM_ROCKETMQ_REMOTING_SOCKET_SNDBUF_SIZE)) {
|
||||
NettySystemConfig.socketSndbufSize = 131072;
|
||||
}
|
||||
|
||||
if (null == System.getProperty(NettySystemConfig.COM_ROCKETMQ_REMOTING_SOCKET_RCVBUF_SIZE)) {
|
||||
NettySystemConfig.socketRcvbufSize = 131072;
|
||||
}
|
||||
|
||||
try {
|
||||
//PackageConflictDetect.detectFastjson();
|
||||
Options options = ServerUtil.buildCommandlineOptions(new Options());
|
||||
commandLine = ServerUtil.parseCmdLine("mqbroker", args, buildCommandlineOptions(options),
|
||||
new PosixParser());
|
||||
if (null == commandLine) {
|
||||
System.exit(-1);
|
||||
}
|
||||
|
||||
final BrokerConfig brokerConfig = new BrokerConfig();
|
||||
final NettyServerConfig nettyServerConfig = new NettyServerConfig();
|
||||
final NettyClientConfig nettyClientConfig = new NettyClientConfig();
|
||||
nettyServerConfig.setListenPort(10911);
|
||||
final MessageStoreConfig messageStoreConfig = new MessageStoreConfig();
|
||||
|
||||
if (BrokerRole.SLAVE == messageStoreConfig.getBrokerRole()) {
|
||||
int ratio = messageStoreConfig.getAccessMessageInMemoryMaxRatio() - 10;
|
||||
messageStoreConfig.setAccessMessageInMemoryMaxRatio(ratio);
|
||||
}
|
||||
|
||||
if (commandLine.hasOption('p')) {
|
||||
MixAll.printObjectProperties(null, brokerConfig);
|
||||
MixAll.printObjectProperties(null, nettyServerConfig);
|
||||
MixAll.printObjectProperties(null, nettyClientConfig);
|
||||
MixAll.printObjectProperties(null, messageStoreConfig);
|
||||
System.exit(0);
|
||||
} else if (commandLine.hasOption('m')) {
|
||||
MixAll.printObjectProperties(null, brokerConfig, true);
|
||||
MixAll.printObjectProperties(null, nettyServerConfig, true);
|
||||
MixAll.printObjectProperties(null, nettyClientConfig, true);
|
||||
MixAll.printObjectProperties(null, messageStoreConfig, true);
|
||||
System.exit(0);
|
||||
}
|
||||
|
||||
if (commandLine.hasOption('c')) {
|
||||
String file = commandLine.getOptionValue('c');
|
||||
if (file != null) {
|
||||
configFile = file;
|
||||
InputStream in = new BufferedInputStream(new FileInputStream(file));
|
||||
properties = new Properties();
|
||||
properties.load(in);
|
||||
|
||||
parsePropertie2SystemEnv(properties);
|
||||
MixAll.properties2Object(properties, brokerConfig);
|
||||
MixAll.properties2Object(properties, nettyServerConfig);
|
||||
MixAll.properties2Object(properties, nettyClientConfig);
|
||||
MixAll.properties2Object(properties, messageStoreConfig);
|
||||
|
||||
BrokerPathConfigHelper.setBrokerConfigPath(file);
|
||||
in.close();
|
||||
}
|
||||
}
|
||||
|
||||
MixAll.properties2Object(ServerUtil.commandLine2Properties(commandLine), brokerConfig);
|
||||
|
||||
if (null == brokerConfig.getRocketmqHome()) {
|
||||
System.out.printf("Please set the " + MixAll.ROCKETMQ_HOME_ENV
|
||||
+ " variable in your environment to match the location of the RocketMQ installation");
|
||||
System.exit(-2);
|
||||
}
|
||||
|
||||
String namesrvAddr = brokerConfig.getNamesrvAddr();
|
||||
if (null != namesrvAddr) {
|
||||
try {
|
||||
String[] addrArray = namesrvAddr.split(";");
|
||||
if (addrArray != null) {
|
||||
for (String addr : addrArray) {
|
||||
RemotingUtil.string2SocketAddress(addr);
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
System.out.printf(
|
||||
"The Name Server Address[%s] illegal, please set it as follows, \"127.0.0.1:9876;192.168.0.1:9876\"%n",
|
||||
namesrvAddr);
|
||||
System.exit(-3);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
switch (messageStoreConfig.getBrokerRole()) {
|
||||
case ASYNC_MASTER:
|
||||
case SYNC_MASTER:
|
||||
brokerConfig.setBrokerId(MixAll.MASTER_ID);
|
||||
break;
|
||||
case SLAVE:
|
||||
if (brokerConfig.getBrokerId() <= 0) {
|
||||
System.out.printf("Slave's brokerId must be > 0");
|
||||
System.exit(-3);
|
||||
}
|
||||
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
messageStoreConfig.setHaListenPort(nettyServerConfig.getListenPort() + 1);
|
||||
LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory();
|
||||
JoranConfigurator configurator = new JoranConfigurator();
|
||||
configurator.setContext(lc);
|
||||
lc.reset();
|
||||
configurator.doConfigure(brokerConfig.getRocketmqHome() + "/conf/logback_broker.xml");
|
||||
log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
|
||||
MixAll.printObjectProperties(log, brokerConfig);
|
||||
MixAll.printObjectProperties(log, nettyServerConfig);
|
||||
MixAll.printObjectProperties(log, nettyClientConfig);
|
||||
MixAll.printObjectProperties(log, messageStoreConfig);
|
||||
|
||||
final BrokerController controller = new BrokerController(//
|
||||
brokerConfig, //
|
||||
nettyServerConfig, //
|
||||
nettyClientConfig, //
|
||||
messageStoreConfig);
|
||||
// remember all configs to prevent discard
|
||||
controller.getConfiguration().registerConfig(properties);
|
||||
|
||||
boolean initResult = controller.initialize();
|
||||
if (!initResult) {
|
||||
controller.shutdown();
|
||||
System.exit(-3);
|
||||
}
|
||||
|
||||
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
|
||||
private volatile boolean hasShutdown = false;
|
||||
private AtomicInteger shutdownTimes = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
synchronized (this) {
|
||||
log.info("shutdown hook was invoked, " + this.shutdownTimes.incrementAndGet());
|
||||
if (!this.hasShutdown) {
|
||||
this.hasShutdown = true;
|
||||
long begineTime = System.currentTimeMillis();
|
||||
controller.shutdown();
|
||||
long consumingTimeTotal = System.currentTimeMillis() - begineTime;
|
||||
log.info("shutdown hook over, consuming time total(ms): " + consumingTimeTotal);
|
||||
}
|
||||
}
|
||||
}
|
||||
}, "ShutdownHook"));
|
||||
|
||||
return controller;
|
||||
} catch (Throwable e) {
|
||||
e.printStackTrace();
|
||||
System.exit(-1);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
private static void parsePropertie2SystemEnv(Properties properties) {
|
||||
if (properties == null) {
|
||||
return;
|
||||
}
|
||||
String rmqAddressServerDomain = properties.getProperty("rmqAddressServerDomain", "jmenv.tbsite.net");
|
||||
String rmqAddressServerSubGroup = properties.getProperty("rmqAddressServerSubGroup", "nsaddr");
|
||||
System.setProperty("rocketmq.namesrv.domain", rmqAddressServerDomain);
|
||||
System.setProperty("rocketmq.namesrv.domain.subgroup", rmqAddressServerSubGroup);
|
||||
}
|
||||
|
||||
public static Options buildCommandlineOptions(final Options options) {
|
||||
Option opt = new Option("c", "configFile", true, "Broker config properties file");
|
||||
opt.setRequired(false);
|
||||
options.addOption(opt);
|
||||
|
||||
opt = new Option("p", "printConfigItem", false, "Print all config item");
|
||||
opt.setRequired(false);
|
||||
options.addOption(opt);
|
||||
|
||||
opt = new Option("m", "printImportantConfig", false, "Print important config item");
|
||||
opt.setRequired(false);
|
||||
options.addOption(opt);
|
||||
|
||||
return options;
|
||||
}
|
||||
}
|
||||
+115
@@ -0,0 +1,115 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.client;
|
||||
|
||||
import com.alibaba.rocketmq.remoting.protocol.LanguageCode;
|
||||
import io.netty.channel.Channel;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ClientChannelInfo {
|
||||
private final Channel channel;
|
||||
private final String clientId;
|
||||
private final LanguageCode language;
|
||||
private final int version;
|
||||
private volatile long lastUpdateTimestamp = System.currentTimeMillis();
|
||||
|
||||
|
||||
public ClientChannelInfo(Channel channel) {
|
||||
this(channel, null, null, 0);
|
||||
}
|
||||
|
||||
|
||||
public ClientChannelInfo(Channel channel, String clientId, LanguageCode language, int version) {
|
||||
this.channel = channel;
|
||||
this.clientId = clientId;
|
||||
this.language = language;
|
||||
this.version = version;
|
||||
}
|
||||
|
||||
|
||||
public Channel getChannel() {
|
||||
return channel;
|
||||
}
|
||||
|
||||
|
||||
public String getClientId() {
|
||||
return clientId;
|
||||
}
|
||||
|
||||
|
||||
public LanguageCode getLanguage() {
|
||||
return language;
|
||||
}
|
||||
|
||||
|
||||
public int getVersion() {
|
||||
return version;
|
||||
}
|
||||
|
||||
|
||||
public long getLastUpdateTimestamp() {
|
||||
return lastUpdateTimestamp;
|
||||
}
|
||||
|
||||
|
||||
public void setLastUpdateTimestamp(long lastUpdateTimestamp) {
|
||||
this.lastUpdateTimestamp = lastUpdateTimestamp;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public int hashCode() {
|
||||
final int prime = 31;
|
||||
int result = 1;
|
||||
result = prime * result + ((channel == null) ? 0 : channel.hashCode());
|
||||
result = prime * result + ((clientId == null) ? 0 : clientId.hashCode());
|
||||
result = prime * result + ((language == null) ? 0 : language.hashCode());
|
||||
result = prime * result + (int) (lastUpdateTimestamp ^ (lastUpdateTimestamp >>> 32));
|
||||
result = prime * result + version;
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public boolean equals(Object obj) {
|
||||
if (this == obj)
|
||||
return true;
|
||||
if (obj == null)
|
||||
return false;
|
||||
if (getClass() != obj.getClass())
|
||||
return false;
|
||||
ClientChannelInfo other = (ClientChannelInfo) obj;
|
||||
if (channel == null) {
|
||||
if (other.channel != null)
|
||||
return false;
|
||||
} else if (this.channel != other.channel) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return "ClientChannelInfo [channel=" + channel + ", clientId=" + clientId + ", language=" + language
|
||||
+ ", version=" + version + ", lastUpdateTimestamp=" + lastUpdateTimestamp + "]";
|
||||
}
|
||||
}
|
||||
+100
@@ -0,0 +1,100 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.client;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.common.ThreadFactoryImpl;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.remoting.ChannelEventListener;
|
||||
import io.netty.channel.Channel;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.ScheduledExecutorService;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ClientHousekeepingService implements ChannelEventListener {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private final BrokerController brokerController;
|
||||
|
||||
private ScheduledExecutorService scheduledExecutorService = Executors
|
||||
.newSingleThreadScheduledExecutor(new ThreadFactoryImpl("ClientHousekeepingScheduledThread"));
|
||||
|
||||
|
||||
public ClientHousekeepingService(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
|
||||
public void start() {
|
||||
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
ClientHousekeepingService.this.scanExceptionChannel();
|
||||
} catch (Exception e) {
|
||||
log.error("", e);
|
||||
}
|
||||
}
|
||||
}, 1000 * 10, 1000 * 10, TimeUnit.MILLISECONDS);
|
||||
}
|
||||
|
||||
private void scanExceptionChannel() {
|
||||
this.brokerController.getProducerManager().scanNotActiveChannel();
|
||||
this.brokerController.getConsumerManager().scanNotActiveChannel();
|
||||
this.brokerController.getFilterServerManager().scanNotActiveChannel();
|
||||
}
|
||||
|
||||
public void shutdown() {
|
||||
this.scheduledExecutorService.shutdown();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onChannelConnect(String remoteAddr, Channel channel) {
|
||||
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void onChannelClose(String remoteAddr, Channel channel) {
|
||||
this.brokerController.getProducerManager().doChannelCloseEvent(remoteAddr, channel);
|
||||
this.brokerController.getConsumerManager().doChannelCloseEvent(remoteAddr, channel);
|
||||
this.brokerController.getFilterServerManager().doChannelCloseEvent(remoteAddr, channel);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void onChannelException(String remoteAddr, Channel channel) {
|
||||
this.brokerController.getProducerManager().doChannelCloseEvent(remoteAddr, channel);
|
||||
this.brokerController.getConsumerManager().doChannelCloseEvent(remoteAddr, channel);
|
||||
this.brokerController.getFilterServerManager().doChannelCloseEvent(remoteAddr, channel);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void onChannelIdle(String remoteAddr, Channel channel) {
|
||||
this.brokerController.getProducerManager().doChannelCloseEvent(remoteAddr, channel);
|
||||
this.brokerController.getConsumerManager().doChannelCloseEvent(remoteAddr, channel);
|
||||
this.brokerController.getFilterServerManager().doChannelCloseEvent(remoteAddr, channel);
|
||||
}
|
||||
}
|
||||
+273
@@ -0,0 +1,273 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.client;
|
||||
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.consumer.ConsumeFromWhere;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.ConsumeType;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.MessageModel;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.SubscriptionData;
|
||||
import io.netty.channel.Channel;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.Map.Entry;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ConsumerGroupInfo {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private final String groupName;
|
||||
private final ConcurrentHashMap<String/* Topic */, SubscriptionData> subscriptionTable =
|
||||
new ConcurrentHashMap<String, SubscriptionData>();
|
||||
private final ConcurrentHashMap<Channel, ClientChannelInfo> channelInfoTable =
|
||||
new ConcurrentHashMap<Channel, ClientChannelInfo>(16);
|
||||
private volatile ConsumeType consumeType;
|
||||
private volatile MessageModel messageModel;
|
||||
private volatile ConsumeFromWhere consumeFromWhere;
|
||||
private volatile long lastUpdateTimestamp = System.currentTimeMillis();
|
||||
|
||||
|
||||
public ConsumerGroupInfo(String groupName, ConsumeType consumeType, MessageModel messageModel,
|
||||
ConsumeFromWhere consumeFromWhere) {
|
||||
this.groupName = groupName;
|
||||
this.consumeType = consumeType;
|
||||
this.messageModel = messageModel;
|
||||
this.consumeFromWhere = consumeFromWhere;
|
||||
}
|
||||
|
||||
|
||||
public ClientChannelInfo findChannel(final String clientId) {
|
||||
Iterator<Entry<Channel, ClientChannelInfo>> it = this.channelInfoTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<Channel, ClientChannelInfo> next = it.next();
|
||||
if (next.getValue().getClientId().equals(clientId)) {
|
||||
return next.getValue();
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
|
||||
public ConcurrentHashMap<String, SubscriptionData> getSubscriptionTable() {
|
||||
return subscriptionTable;
|
||||
}
|
||||
|
||||
|
||||
public ConcurrentHashMap<Channel, ClientChannelInfo> getChannelInfoTable() {
|
||||
return channelInfoTable;
|
||||
}
|
||||
|
||||
|
||||
public List<Channel> getAllChannel() {
|
||||
List<Channel> result = new ArrayList<Channel>();
|
||||
|
||||
result.addAll(this.channelInfoTable.keySet());
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
public List<String> getAllClientId() {
|
||||
List<String> result = new ArrayList<String>();
|
||||
|
||||
Iterator<Entry<Channel, ClientChannelInfo>> it = this.channelInfoTable.entrySet().iterator();
|
||||
|
||||
while (it.hasNext()) {
|
||||
Entry<Channel, ClientChannelInfo> entry = it.next();
|
||||
ClientChannelInfo clientChannelInfo = entry.getValue();
|
||||
result.add(clientChannelInfo.getClientId());
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
public void unregisterChannel(final ClientChannelInfo clientChannelInfo) {
|
||||
ClientChannelInfo old = this.channelInfoTable.remove(clientChannelInfo.getChannel());
|
||||
if (old != null) {
|
||||
log.info("unregister a consumer[{}] from consumerGroupInfo {}", this.groupName, old.toString());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public boolean doChannelCloseEvent(final String remoteAddr, final Channel channel) {
|
||||
final ClientChannelInfo info = this.channelInfoTable.remove(channel);
|
||||
if (info != null) {
|
||||
log.warn(
|
||||
"NETTY EVENT: remove not active channel[{}] from ConsumerGroupInfo groupChannelTable, consumer group: {}",
|
||||
info.toString(), groupName);
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
public boolean updateChannel(final ClientChannelInfo infoNew, ConsumeType consumeType,
|
||||
MessageModel messageModel, ConsumeFromWhere consumeFromWhere) {
|
||||
boolean updated = false;
|
||||
this.consumeType = consumeType;
|
||||
this.messageModel = messageModel;
|
||||
this.consumeFromWhere = consumeFromWhere;
|
||||
|
||||
ClientChannelInfo infoOld = this.channelInfoTable.get(infoNew.getChannel());
|
||||
if (null == infoOld) {
|
||||
ClientChannelInfo prev = this.channelInfoTable.put(infoNew.getChannel(), infoNew);
|
||||
if (null == prev) {
|
||||
log.info("new consumer connected, group: {} {} {} channel: {}", this.groupName, consumeType,
|
||||
messageModel, infoNew.toString());
|
||||
updated = true;
|
||||
}
|
||||
|
||||
infoOld = infoNew;
|
||||
} else {
|
||||
if (!infoOld.getClientId().equals(infoNew.getClientId())) {
|
||||
log.error("[BUG] consumer channel exist in broker, but clientId not equal. GROUP: {} OLD: {} NEW: {} ",
|
||||
this.groupName,
|
||||
infoOld.toString(),
|
||||
infoNew.toString());
|
||||
this.channelInfoTable.put(infoNew.getChannel(), infoNew);
|
||||
}
|
||||
}
|
||||
|
||||
this.lastUpdateTimestamp = System.currentTimeMillis();
|
||||
infoOld.setLastUpdateTimestamp(this.lastUpdateTimestamp);
|
||||
|
||||
return updated;
|
||||
}
|
||||
|
||||
|
||||
public boolean updateSubscription(final Set<SubscriptionData> subList) {
|
||||
boolean updated = false;
|
||||
|
||||
for (SubscriptionData sub : subList) {
|
||||
SubscriptionData old = this.subscriptionTable.get(sub.getTopic());
|
||||
if (old == null) {
|
||||
SubscriptionData prev = this.subscriptionTable.putIfAbsent(sub.getTopic(), sub);
|
||||
if (null == prev) {
|
||||
updated = true;
|
||||
log.info("subscription changed, add new topic, group: {} {}",
|
||||
this.groupName,
|
||||
sub.toString());
|
||||
}
|
||||
} else if (sub.getSubVersion() > old.getSubVersion()) {
|
||||
if (this.consumeType == ConsumeType.CONSUME_PASSIVELY) {
|
||||
log.info("subscription changed, group: {} OLD: {} NEW: {}",
|
||||
this.groupName,
|
||||
old.toString(),
|
||||
sub.toString()
|
||||
);
|
||||
}
|
||||
|
||||
this.subscriptionTable.put(sub.getTopic(), sub);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Iterator<Entry<String, SubscriptionData>> it = this.subscriptionTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<String, SubscriptionData> next = it.next();
|
||||
String oldTopic = next.getKey();
|
||||
|
||||
boolean exist = false;
|
||||
for (SubscriptionData sub : subList) {
|
||||
if (sub.getTopic().equals(oldTopic)) {
|
||||
exist = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!exist) {
|
||||
log.warn("subscription changed, group: {} remove topic {} {}",
|
||||
this.groupName,
|
||||
oldTopic,
|
||||
next.getValue().toString()
|
||||
);
|
||||
|
||||
it.remove();
|
||||
updated = true;
|
||||
}
|
||||
}
|
||||
|
||||
this.lastUpdateTimestamp = System.currentTimeMillis();
|
||||
|
||||
return updated;
|
||||
}
|
||||
|
||||
|
||||
public Set<String> getSubscribeTopics() {
|
||||
return subscriptionTable.keySet();
|
||||
}
|
||||
|
||||
|
||||
public SubscriptionData findSubscriptionData(final String topic) {
|
||||
return this.subscriptionTable.get(topic);
|
||||
}
|
||||
|
||||
|
||||
public ConsumeType getConsumeType() {
|
||||
return consumeType;
|
||||
}
|
||||
|
||||
|
||||
public void setConsumeType(ConsumeType consumeType) {
|
||||
this.consumeType = consumeType;
|
||||
}
|
||||
|
||||
|
||||
public MessageModel getMessageModel() {
|
||||
return messageModel;
|
||||
}
|
||||
|
||||
|
||||
public void setMessageModel(MessageModel messageModel) {
|
||||
this.messageModel = messageModel;
|
||||
}
|
||||
|
||||
|
||||
public String getGroupName() {
|
||||
return groupName;
|
||||
}
|
||||
|
||||
|
||||
public long getLastUpdateTimestamp() {
|
||||
return lastUpdateTimestamp;
|
||||
}
|
||||
|
||||
|
||||
public void setLastUpdateTimestamp(long lastUpdateTimestamp) {
|
||||
this.lastUpdateTimestamp = lastUpdateTimestamp;
|
||||
}
|
||||
|
||||
|
||||
public ConsumeFromWhere getConsumeFromWhere() {
|
||||
return consumeFromWhere;
|
||||
}
|
||||
|
||||
|
||||
public void setConsumeFromWhere(ConsumeFromWhere consumeFromWhere) {
|
||||
this.consumeFromWhere = consumeFromWhere;
|
||||
}
|
||||
}
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.client;
|
||||
|
||||
import io.netty.channel.Channel;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public interface ConsumerIdsChangeListener {
|
||||
public void consumerIdsChanged(final String group, final List<Channel> channels);
|
||||
}
|
||||
+187
@@ -0,0 +1,187 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.client;
|
||||
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.consumer.ConsumeFromWhere;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.ConsumeType;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.MessageModel;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.SubscriptionData;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingHelper;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingUtil;
|
||||
import io.netty.channel.Channel;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.HashSet;
|
||||
import java.util.Iterator;
|
||||
import java.util.Map.Entry;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ConsumerManager {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private static final long CHANNEL_EXPIRED_TIMEOUT = 1000 * 120;
|
||||
private final ConcurrentHashMap<String/* Group */, ConsumerGroupInfo> consumerTable =
|
||||
new ConcurrentHashMap<String, ConsumerGroupInfo>(1024);
|
||||
private final ConsumerIdsChangeListener consumerIdsChangeListener;
|
||||
|
||||
public ConsumerManager(final ConsumerIdsChangeListener consumerIdsChangeListener) {
|
||||
this.consumerIdsChangeListener = consumerIdsChangeListener;
|
||||
}
|
||||
|
||||
public ClientChannelInfo findChannel(final String group, final String clientId) {
|
||||
ConsumerGroupInfo consumerGroupInfo = this.consumerTable.get(group);
|
||||
if (consumerGroupInfo != null) {
|
||||
return consumerGroupInfo.findChannel(clientId);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
public SubscriptionData findSubscriptionData(final String group, final String topic) {
|
||||
ConsumerGroupInfo consumerGroupInfo = this.getConsumerGroupInfo(group);
|
||||
if (consumerGroupInfo != null) {
|
||||
return consumerGroupInfo.findSubscriptionData(topic);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
public ConsumerGroupInfo getConsumerGroupInfo(final String group) {
|
||||
return this.consumerTable.get(group);
|
||||
}
|
||||
|
||||
public int findSubscriptionDataCount(final String group) {
|
||||
ConsumerGroupInfo consumerGroupInfo = this.getConsumerGroupInfo(group);
|
||||
if (consumerGroupInfo != null) {
|
||||
return consumerGroupInfo.getSubscriptionTable().size();
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
public void doChannelCloseEvent(final String remoteAddr, final Channel channel) {
|
||||
Iterator<Entry<String, ConsumerGroupInfo>> it = this.consumerTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<String, ConsumerGroupInfo> next = it.next();
|
||||
ConsumerGroupInfo info = next.getValue();
|
||||
boolean removed = info.doChannelCloseEvent(remoteAddr, channel);
|
||||
if (removed) {
|
||||
if (info.getChannelInfoTable().isEmpty()) {
|
||||
ConsumerGroupInfo remove = this.consumerTable.remove(next.getKey());
|
||||
if (remove != null) {
|
||||
log.info("unregister consumer ok, no any connection, and remove consumer group, {}",
|
||||
next.getKey());
|
||||
}
|
||||
}
|
||||
|
||||
this.consumerIdsChangeListener.consumerIdsChanged(next.getKey(), info.getAllChannel());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public boolean registerConsumer(final String group, final ClientChannelInfo clientChannelInfo,
|
||||
ConsumeType consumeType, MessageModel messageModel, ConsumeFromWhere consumeFromWhere,
|
||||
final Set<SubscriptionData> subList, boolean isNotifyConsumerIdsChangedEnable) {
|
||||
|
||||
ConsumerGroupInfo consumerGroupInfo = this.consumerTable.get(group);
|
||||
if (null == consumerGroupInfo) {
|
||||
ConsumerGroupInfo tmp = new ConsumerGroupInfo(group, consumeType, messageModel, consumeFromWhere);
|
||||
ConsumerGroupInfo prev = this.consumerTable.putIfAbsent(group, tmp);
|
||||
consumerGroupInfo = prev != null ? prev : tmp;
|
||||
}
|
||||
|
||||
boolean r1 =
|
||||
consumerGroupInfo.updateChannel(clientChannelInfo, consumeType, messageModel,
|
||||
consumeFromWhere);
|
||||
boolean r2 = consumerGroupInfo.updateSubscription(subList);
|
||||
|
||||
if (r1 || r2) {
|
||||
if (isNotifyConsumerIdsChangedEnable) {
|
||||
this.consumerIdsChangeListener.consumerIdsChanged(group, consumerGroupInfo.getAllChannel());
|
||||
}
|
||||
}
|
||||
|
||||
return r1 || r2;
|
||||
}
|
||||
|
||||
public void unregisterConsumer(final String group, final ClientChannelInfo clientChannelInfo, boolean isNotifyConsumerIdsChangedEnable) {
|
||||
ConsumerGroupInfo consumerGroupInfo = this.consumerTable.get(group);
|
||||
if (null != consumerGroupInfo) {
|
||||
consumerGroupInfo.unregisterChannel(clientChannelInfo);
|
||||
if (consumerGroupInfo.getChannelInfoTable().isEmpty()) {
|
||||
ConsumerGroupInfo remove = this.consumerTable.remove(group);
|
||||
if (remove != null) {
|
||||
log.info("unregister consumer ok, no any connection, and remove consumer group, {}", group);
|
||||
}
|
||||
}
|
||||
if (isNotifyConsumerIdsChangedEnable) {
|
||||
this.consumerIdsChangeListener.consumerIdsChanged(group, consumerGroupInfo.getAllChannel());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public void scanNotActiveChannel() {
|
||||
Iterator<Entry<String, ConsumerGroupInfo>> it = this.consumerTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<String, ConsumerGroupInfo> next = it.next();
|
||||
String group = next.getKey();
|
||||
ConsumerGroupInfo consumerGroupInfo = next.getValue();
|
||||
ConcurrentHashMap<Channel, ClientChannelInfo> channelInfoTable =
|
||||
consumerGroupInfo.getChannelInfoTable();
|
||||
|
||||
Iterator<Entry<Channel, ClientChannelInfo>> itChannel = channelInfoTable.entrySet().iterator();
|
||||
while (itChannel.hasNext()) {
|
||||
Entry<Channel, ClientChannelInfo> nextChannel = itChannel.next();
|
||||
ClientChannelInfo clientChannelInfo = nextChannel.getValue();
|
||||
long diff = System.currentTimeMillis() - clientChannelInfo.getLastUpdateTimestamp();
|
||||
if (diff > CHANNEL_EXPIRED_TIMEOUT) {
|
||||
log.warn(
|
||||
"SCAN: remove expired channel from ConsumerManager consumerTable. channel={}, consumerGroup={}",
|
||||
RemotingHelper.parseChannelRemoteAddr(clientChannelInfo.getChannel()), group);
|
||||
RemotingUtil.closeChannel(clientChannelInfo.getChannel());
|
||||
itChannel.remove();
|
||||
}
|
||||
}
|
||||
|
||||
if (channelInfoTable.isEmpty()) {
|
||||
log.warn(
|
||||
"SCAN: remove expired channel from ConsumerManager consumerTable, all clear, consumerGroup={}",
|
||||
group);
|
||||
it.remove();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public HashSet<String> queryTopicConsumeByWho(final String topic) {
|
||||
HashSet<String> groups = new HashSet<String>();
|
||||
Iterator<Entry<String, ConsumerGroupInfo>> it = this.consumerTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<String, ConsumerGroupInfo> entry = it.next();
|
||||
ConcurrentHashMap<String, SubscriptionData> subscriptionTable =
|
||||
entry.getValue().getSubscriptionTable();
|
||||
if (subscriptionTable.containsKey(topic)) {
|
||||
groups.add(entry.getKey());
|
||||
}
|
||||
}
|
||||
return groups;
|
||||
}
|
||||
}
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.client;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import io.netty.channel.Channel;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class DefaultConsumerIdsChangeListener implements ConsumerIdsChangeListener {
|
||||
private final BrokerController brokerController;
|
||||
|
||||
|
||||
public DefaultConsumerIdsChangeListener(BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void consumerIdsChanged(String group, List<Channel> channels) {
|
||||
if (channels != null && brokerController.getBrokerConfig().isNotifyConsumerIdsChangedEnable()) {
|
||||
for (Channel chl : channels) {
|
||||
this.brokerController.getBroker2Client().notifyConsumerIdsChanged(chl, group);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+199
@@ -0,0 +1,199 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.client;
|
||||
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingHelper;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingUtil;
|
||||
import io.netty.channel.Channel;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.HashMap;
|
||||
import java.util.Iterator;
|
||||
import java.util.Map;
|
||||
import java.util.Map.Entry;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.locks.Lock;
|
||||
import java.util.concurrent.locks.ReentrantLock;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ProducerManager {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private static final long LOCK_TIMEOUT_MILLIS = 3000;
|
||||
private static final long CHANNEL_EXPIRED_TIMEOUT = 1000 * 120;
|
||||
private final Lock groupChannelLock = new ReentrantLock();
|
||||
private final HashMap<String /* group name */, HashMap<Channel, ClientChannelInfo>> groupChannelTable =
|
||||
new HashMap<String, HashMap<Channel, ClientChannelInfo>>();
|
||||
|
||||
|
||||
public ProducerManager() {
|
||||
}
|
||||
|
||||
|
||||
public HashMap<String, HashMap<Channel, ClientChannelInfo>> getGroupChannelTable() {
|
||||
HashMap<String /* group name */, HashMap<Channel, ClientChannelInfo>> newGroupChannelTable =
|
||||
new HashMap<String, HashMap<Channel, ClientChannelInfo>>();
|
||||
try {
|
||||
if (this.groupChannelLock.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
|
||||
try {
|
||||
newGroupChannelTable.putAll(groupChannelTable);
|
||||
} finally {
|
||||
groupChannelLock.unlock();
|
||||
}
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
log.error("", e);
|
||||
}
|
||||
return newGroupChannelTable;
|
||||
}
|
||||
|
||||
|
||||
public void scanNotActiveChannel() {
|
||||
try {
|
||||
if (this.groupChannelLock.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
|
||||
try {
|
||||
for (final Map.Entry<String, HashMap<Channel, ClientChannelInfo>> entry : this.groupChannelTable
|
||||
.entrySet()) {
|
||||
final String group = entry.getKey();
|
||||
final HashMap<Channel, ClientChannelInfo> chlMap = entry.getValue();
|
||||
|
||||
Iterator<Entry<Channel, ClientChannelInfo>> it = chlMap.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<Channel, ClientChannelInfo> item = it.next();
|
||||
// final Integer id = item.getKey();
|
||||
final ClientChannelInfo info = item.getValue();
|
||||
|
||||
long diff = System.currentTimeMillis() - info.getLastUpdateTimestamp();
|
||||
if (diff > CHANNEL_EXPIRED_TIMEOUT) {
|
||||
it.remove();
|
||||
log.warn(
|
||||
"SCAN: remove expired channel[{}] from ProducerManager groupChannelTable, producer group name: {}",
|
||||
RemotingHelper.parseChannelRemoteAddr(info.getChannel()), group);
|
||||
RemotingUtil.closeChannel(info.getChannel());
|
||||
}
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
this.groupChannelLock.unlock();
|
||||
}
|
||||
} else {
|
||||
log.warn("ProducerManager scanNotActiveChannel lock timeout");
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
log.error("", e);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public void doChannelCloseEvent(final String remoteAddr, final Channel channel) {
|
||||
if (channel != null) {
|
||||
try {
|
||||
if (this.groupChannelLock.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
|
||||
try {
|
||||
for (final Map.Entry<String, HashMap<Channel, ClientChannelInfo>> entry : this.groupChannelTable
|
||||
.entrySet()) {
|
||||
final String group = entry.getKey();
|
||||
final HashMap<Channel, ClientChannelInfo> clientChannelInfoTable =
|
||||
entry.getValue();
|
||||
final ClientChannelInfo clientChannelInfo =
|
||||
clientChannelInfoTable.remove(channel);
|
||||
if (clientChannelInfo != null) {
|
||||
log.info(
|
||||
"NETTY EVENT: remove channel[{}][{}] from ProducerManager groupChannelTable, producer group: {}",
|
||||
clientChannelInfo.toString(), remoteAddr, group);
|
||||
}
|
||||
|
||||
}
|
||||
} finally {
|
||||
this.groupChannelLock.unlock();
|
||||
}
|
||||
} else {
|
||||
log.warn("ProducerManager doChannelCloseEvent lock timeout");
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
log.error("", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public void registerProducer(final String group, final ClientChannelInfo clientChannelInfo) {
|
||||
try {
|
||||
ClientChannelInfo clientChannelInfoFound = null;
|
||||
|
||||
if (this.groupChannelLock.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
|
||||
try {
|
||||
HashMap<Channel, ClientChannelInfo> channelTable = this.groupChannelTable.get(group);
|
||||
if (null == channelTable) {
|
||||
channelTable = new HashMap<Channel, ClientChannelInfo>();
|
||||
this.groupChannelTable.put(group, channelTable);
|
||||
}
|
||||
|
||||
clientChannelInfoFound = channelTable.get(clientChannelInfo.getChannel());
|
||||
if (null == clientChannelInfoFound) {
|
||||
channelTable.put(clientChannelInfo.getChannel(), clientChannelInfo);
|
||||
log.info("new producer connected, group: {} channel: {}", group,
|
||||
clientChannelInfo.toString());
|
||||
}
|
||||
} finally {
|
||||
this.groupChannelLock.unlock();
|
||||
}
|
||||
|
||||
if (clientChannelInfoFound != null) {
|
||||
clientChannelInfoFound.setLastUpdateTimestamp(System.currentTimeMillis());
|
||||
}
|
||||
} else {
|
||||
log.warn("ProducerManager registerProducer lock timeout");
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
log.error("", e);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public void unregisterProducer(final String group, final ClientChannelInfo clientChannelInfo) {
|
||||
try {
|
||||
if (this.groupChannelLock.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
|
||||
try {
|
||||
HashMap<Channel, ClientChannelInfo> channelTable = this.groupChannelTable.get(group);
|
||||
if (null != channelTable && !channelTable.isEmpty()) {
|
||||
ClientChannelInfo old = channelTable.remove(clientChannelInfo.getChannel());
|
||||
if (old != null) {
|
||||
log.info("unregister a producer[{}] from groupChannelTable {}", group,
|
||||
clientChannelInfo.toString());
|
||||
}
|
||||
|
||||
if (channelTable.isEmpty()) {
|
||||
this.groupChannelTable.remove(group);
|
||||
log.info("unregister a producer group[{}] from groupChannelTable", group);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
this.groupChannelLock.unlock();
|
||||
}
|
||||
} else {
|
||||
log.warn("ProducerManager unregisterProducer lock timeout");
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
log.error("", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
+317
@@ -0,0 +1,317 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.client.net;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.client.ClientChannelInfo;
|
||||
import com.alibaba.rocketmq.broker.client.ConsumerGroupInfo;
|
||||
import com.alibaba.rocketmq.broker.pagecache.OneMessageTransfer;
|
||||
import com.alibaba.rocketmq.common.MQVersion;
|
||||
import com.alibaba.rocketmq.common.TopicConfig;
|
||||
import com.alibaba.rocketmq.common.UtilAll;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.message.MessageQueue;
|
||||
import com.alibaba.rocketmq.common.message.MessageQueueForC;
|
||||
import com.alibaba.rocketmq.common.protocol.RequestCode;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
import com.alibaba.rocketmq.common.protocol.body.GetConsumerStatusBody;
|
||||
import com.alibaba.rocketmq.common.protocol.body.ResetOffsetBody;
|
||||
import com.alibaba.rocketmq.common.protocol.body.ResetOffsetBodyForC;
|
||||
import com.alibaba.rocketmq.common.protocol.header.CheckTransactionStateRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.GetConsumerStatusRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.NotifyConsumerIdsChangedRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.ResetOffsetRequestHeader;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingHelper;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingSendRequestException;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingTimeoutException;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import com.alibaba.rocketmq.store.SelectMappedBufferResult;
|
||||
import io.netty.channel.Channel;
|
||||
import io.netty.channel.ChannelFuture;
|
||||
import io.netty.channel.ChannelFutureListener;
|
||||
import io.netty.channel.FileRegion;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Map.Entry;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class Broker2Client {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private final BrokerController brokerController;
|
||||
|
||||
public Broker2Client(BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
public void checkProducerTransactionState(
|
||||
final Channel channel,
|
||||
final CheckTransactionStateRequestHeader requestHeader,
|
||||
final SelectMappedBufferResult selectMappedBufferResult) {
|
||||
RemotingCommand request =
|
||||
RemotingCommand.createRequestCommand(RequestCode.CHECK_TRANSACTION_STATE, requestHeader);
|
||||
request.markOnewayRPC();
|
||||
|
||||
try {
|
||||
FileRegion fileRegion =
|
||||
new OneMessageTransfer(request.encodeHeader(selectMappedBufferResult.getSize()),
|
||||
selectMappedBufferResult);
|
||||
channel.writeAndFlush(fileRegion).addListener(new ChannelFutureListener() {
|
||||
@Override
|
||||
public void operationComplete(ChannelFuture future) throws Exception {
|
||||
selectMappedBufferResult.release();
|
||||
if (!future.isSuccess()) {
|
||||
log.error("invokeProducer failed,", future.cause());
|
||||
}
|
||||
}
|
||||
});
|
||||
} catch (Throwable e) {
|
||||
log.error("invokeProducer exception", e);
|
||||
selectMappedBufferResult.release();
|
||||
}
|
||||
}
|
||||
|
||||
public RemotingCommand callClient(final Channel channel,
|
||||
final RemotingCommand request
|
||||
) throws RemotingSendRequestException, RemotingTimeoutException, InterruptedException {
|
||||
return this.brokerController.getRemotingServer().invokeSync(channel, request, 10000);
|
||||
}
|
||||
|
||||
public void notifyConsumerIdsChanged(
|
||||
final Channel channel,
|
||||
final String consumerGroup) {
|
||||
if (null == consumerGroup) {
|
||||
log.error("notifyConsumerIdsChanged consumerGroup is null");
|
||||
return;
|
||||
}
|
||||
|
||||
NotifyConsumerIdsChangedRequestHeader requestHeader = new NotifyConsumerIdsChangedRequestHeader();
|
||||
requestHeader.setConsumerGroup(consumerGroup);
|
||||
RemotingCommand request =
|
||||
RemotingCommand.createRequestCommand(RequestCode.NOTIFY_CONSUMER_IDS_CHANGED, requestHeader);
|
||||
|
||||
try {
|
||||
this.brokerController.getRemotingServer().invokeOneway(channel, request, 10);
|
||||
} catch (Exception e) {
|
||||
log.error("notifyConsumerIdsChanged exception, " + consumerGroup, e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
public RemotingCommand resetOffset(String topic, String group, long timeStamp, boolean isForce) {
|
||||
return resetOffset(topic, group, timeStamp, isForce, false);
|
||||
}
|
||||
|
||||
public RemotingCommand resetOffset(String topic, String group, long timeStamp, boolean isForce,
|
||||
boolean isC) {
|
||||
final RemotingCommand response = RemotingCommand.createResponseCommand(null);
|
||||
|
||||
TopicConfig topicConfig = this.brokerController.getTopicConfigManager().selectTopicConfig(topic);
|
||||
if (null == topicConfig) {
|
||||
log.error("[reset-offset] reset offset failed, no topic in this broker. topic={}", topic);
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("[reset-offset] reset offset failed, no topic in this broker. topic=" + topic);
|
||||
return response;
|
||||
}
|
||||
|
||||
Map<MessageQueue, Long> offsetTable = new HashMap<MessageQueue, Long>();
|
||||
|
||||
for (int i = 0; i < topicConfig.getWriteQueueNums(); i++) {
|
||||
MessageQueue mq = new MessageQueue();
|
||||
mq.setBrokerName(this.brokerController.getBrokerConfig().getBrokerName());
|
||||
mq.setTopic(topic);
|
||||
mq.setQueueId(i);
|
||||
|
||||
long consumerOffset =
|
||||
this.brokerController.getConsumerOffsetManager().queryOffset(group, topic, i);
|
||||
if (-1 == consumerOffset) {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark(String.format("THe consumer group <%s> not exist", group));
|
||||
return response;
|
||||
}
|
||||
|
||||
long timeStampOffset;
|
||||
if (timeStamp == -1) {
|
||||
|
||||
timeStampOffset = this.brokerController.getMessageStore().getMaxOffsetInQuque(topic, i);
|
||||
} else {
|
||||
timeStampOffset = this.brokerController.getMessageStore().getOffsetInQueueByTime(topic, i, timeStamp);
|
||||
}
|
||||
|
||||
if (timeStampOffset < 0) {
|
||||
log.warn("reset offset is invalid. topic={}, queueId={}, timeStampOffset={}", topic, i, timeStampOffset);
|
||||
timeStampOffset = 0;
|
||||
}
|
||||
|
||||
if (isForce || timeStampOffset < consumerOffset) {
|
||||
offsetTable.put(mq, timeStampOffset);
|
||||
} else {
|
||||
offsetTable.put(mq, consumerOffset);
|
||||
}
|
||||
}
|
||||
|
||||
ResetOffsetRequestHeader requestHeader = new ResetOffsetRequestHeader();
|
||||
requestHeader.setTopic(topic);
|
||||
requestHeader.setGroup(group);
|
||||
requestHeader.setTimestamp(timeStamp);
|
||||
RemotingCommand request =
|
||||
RemotingCommand.createRequestCommand(RequestCode.RESET_CONSUMER_CLIENT_OFFSET, requestHeader);
|
||||
if (isC) {
|
||||
// c++ language
|
||||
ResetOffsetBodyForC body = new ResetOffsetBodyForC();
|
||||
List<MessageQueueForC> offsetList = convertOffsetTable2OffsetList(offsetTable);
|
||||
body.setOffsetTable(offsetList);
|
||||
request.setBody(body.encode());
|
||||
} else {
|
||||
// other language
|
||||
ResetOffsetBody body = new ResetOffsetBody();
|
||||
body.setOffsetTable(offsetTable);
|
||||
request.setBody(body.encode());
|
||||
}
|
||||
|
||||
ConsumerGroupInfo consumerGroupInfo =
|
||||
this.brokerController.getConsumerManager().getConsumerGroupInfo(group);
|
||||
|
||||
if (consumerGroupInfo != null && !consumerGroupInfo.getAllChannel().isEmpty()) {
|
||||
ConcurrentHashMap<Channel, ClientChannelInfo> channelInfoTable =
|
||||
consumerGroupInfo.getChannelInfoTable();
|
||||
for (Map.Entry<Channel, ClientChannelInfo> entry : channelInfoTable.entrySet()) {
|
||||
int version = entry.getValue().getVersion();
|
||||
if (version >= MQVersion.Version.V3_0_7_SNAPSHOT.ordinal()) {
|
||||
try {
|
||||
this.brokerController.getRemotingServer().invokeOneway(entry.getKey(), request, 5000);
|
||||
log.info("[reset-offset] reset offset success. topic={}, group={}, clientId={}",
|
||||
new Object[]{topic, group, entry.getValue().getClientId()});
|
||||
} catch (Exception e) {
|
||||
log.error("[reset-offset] reset offset exception. topic={}, group={}",
|
||||
new Object[]{topic, group}, e);
|
||||
}
|
||||
} else {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("the client does not support this feature. version="
|
||||
+ MQVersion.getVersionDesc(version));
|
||||
log.warn("[reset-offset] the client does not support this feature. version={}",
|
||||
RemotingHelper.parseChannelRemoteAddr(entry.getKey()), MQVersion.getVersionDesc(version));
|
||||
return response;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
String errorInfo =
|
||||
String.format("Consumer not online, so can not reset offset, Group: %s Topic: %s Timestamp: %d",
|
||||
requestHeader.getGroup(),
|
||||
requestHeader.getTopic(),
|
||||
requestHeader.getTimestamp());
|
||||
log.error(errorInfo);
|
||||
response.setCode(ResponseCode.CONSUMER_NOT_ONLINE);
|
||||
response.setRemark(errorInfo);
|
||||
return response;
|
||||
}
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
ResetOffsetBody resBody = new ResetOffsetBody();
|
||||
resBody.setOffsetTable(offsetTable);
|
||||
response.setBody(resBody.encode());
|
||||
return response;
|
||||
}
|
||||
|
||||
private List<MessageQueueForC> convertOffsetTable2OffsetList(Map<MessageQueue, Long> table) {
|
||||
List<MessageQueueForC> list = new ArrayList<MessageQueueForC>();
|
||||
for (Entry<MessageQueue, Long> entry : table.entrySet()) {
|
||||
MessageQueue mq = entry.getKey();
|
||||
MessageQueueForC tmp =
|
||||
new MessageQueueForC(mq.getTopic(), mq.getBrokerName(), mq.getQueueId(), entry.getValue());
|
||||
list.add(tmp);
|
||||
}
|
||||
return list;
|
||||
}
|
||||
|
||||
public RemotingCommand getConsumeStatus(String topic, String group, String originClientId) {
|
||||
final RemotingCommand result = RemotingCommand.createResponseCommand(null);
|
||||
|
||||
GetConsumerStatusRequestHeader requestHeader = new GetConsumerStatusRequestHeader();
|
||||
requestHeader.setTopic(topic);
|
||||
requestHeader.setGroup(group);
|
||||
RemotingCommand request =
|
||||
RemotingCommand.createRequestCommand(RequestCode.GET_CONSUMER_STATUS_FROM_CLIENT,
|
||||
requestHeader);
|
||||
|
||||
Map<String, Map<MessageQueue, Long>> consumerStatusTable =
|
||||
new HashMap<String, Map<MessageQueue, Long>>();
|
||||
ConcurrentHashMap<Channel, ClientChannelInfo> channelInfoTable =
|
||||
this.brokerController.getConsumerManager().getConsumerGroupInfo(group).getChannelInfoTable();
|
||||
if (null == channelInfoTable || channelInfoTable.isEmpty()) {
|
||||
result.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
result.setRemark(String.format("No Any Consumer online in the consumer group: [%s]", group));
|
||||
return result;
|
||||
}
|
||||
|
||||
for (Map.Entry<Channel, ClientChannelInfo> entry : channelInfoTable.entrySet()) {
|
||||
int version = entry.getValue().getVersion();
|
||||
String clientId = entry.getValue().getClientId();
|
||||
if (version < MQVersion.Version.V3_0_7_SNAPSHOT.ordinal()) {
|
||||
result.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
result.setRemark("the client does not support this feature. version="
|
||||
+ MQVersion.getVersionDesc(version));
|
||||
log.warn("[get-consumer-status] the client does not support this feature. version={}",
|
||||
RemotingHelper.parseChannelRemoteAddr(entry.getKey()), MQVersion.getVersionDesc(version));
|
||||
return result;
|
||||
} else if (UtilAll.isBlank(originClientId) || originClientId.equals(clientId)) {
|
||||
try {
|
||||
RemotingCommand response =
|
||||
this.brokerController.getRemotingServer().invokeSync(entry.getKey(), request, 5000);
|
||||
assert response != null;
|
||||
switch (response.getCode()) {
|
||||
case ResponseCode.SUCCESS: {
|
||||
if (response.getBody() != null) {
|
||||
GetConsumerStatusBody body =
|
||||
GetConsumerStatusBody.decode(response.getBody(),
|
||||
GetConsumerStatusBody.class);
|
||||
|
||||
consumerStatusTable.put(clientId, body.getMessageQueueTable());
|
||||
log.info(
|
||||
"[get-consumer-status] get consumer status success. topic={}, group={}, channelRemoteAddr={}",
|
||||
new Object[]{topic, group, clientId});
|
||||
}
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error(
|
||||
"[get-consumer-status] get consumer status exception. topic={}, group={}, offset={}",
|
||||
new Object[]{topic, group}, e);
|
||||
}
|
||||
|
||||
if (!UtilAll.isBlank(originClientId) && originClientId.equals(clientId)) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
result.setCode(ResponseCode.SUCCESS);
|
||||
GetConsumerStatusBody resBody = new GetConsumerStatusBody();
|
||||
resBody.setConsumerTable(consumerStatusTable);
|
||||
result.setBody(resBody.encode());
|
||||
return result;
|
||||
}
|
||||
}
|
||||
+281
@@ -0,0 +1,281 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.client.rebalance;
|
||||
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.message.MessageQueue;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.HashSet;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.locks.Lock;
|
||||
import java.util.concurrent.locks.ReentrantLock;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class RebalanceLockManager {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.REBALANCE_LOCK_LOGGER_NAME);
|
||||
private final static long REBALANCE_LOCK_MAX_LIVE_TIME = Long.parseLong(System.getProperty(
|
||||
"rocketmq.broker.rebalance.lockMaxLiveTime", "60000"));
|
||||
private final Lock lock = new ReentrantLock();
|
||||
private final ConcurrentHashMap<String/* group */, ConcurrentHashMap<MessageQueue, LockEntry>> mqLockTable =
|
||||
new ConcurrentHashMap<String, ConcurrentHashMap<MessageQueue, LockEntry>>(1024);
|
||||
|
||||
public boolean tryLock(final String group, final MessageQueue mq, final String clientId) {
|
||||
|
||||
if (!this.isLocked(group, mq, clientId)) {
|
||||
try {
|
||||
this.lock.lockInterruptibly();
|
||||
try {
|
||||
ConcurrentHashMap<MessageQueue, LockEntry> groupValue = this.mqLockTable.get(group);
|
||||
if (null == groupValue) {
|
||||
groupValue = new ConcurrentHashMap<MessageQueue, LockEntry>(32);
|
||||
this.mqLockTable.put(group, groupValue);
|
||||
}
|
||||
|
||||
LockEntry lockEntry = groupValue.get(mq);
|
||||
if (null == lockEntry) {
|
||||
lockEntry = new LockEntry();
|
||||
lockEntry.setClientId(clientId);
|
||||
groupValue.put(mq, lockEntry);
|
||||
log.info("tryLock, message queue not locked, I got it. Group: {} NewClientId: {} {}", //
|
||||
group, //
|
||||
clientId, //
|
||||
mq);
|
||||
}
|
||||
|
||||
if (lockEntry.isLocked(clientId)) {
|
||||
lockEntry.setLastUpdateTimestamp(System.currentTimeMillis());
|
||||
return true;
|
||||
}
|
||||
|
||||
String oldClientId = lockEntry.getClientId();
|
||||
|
||||
|
||||
if (lockEntry.isExpired()) {
|
||||
lockEntry.setClientId(clientId);
|
||||
lockEntry.setLastUpdateTimestamp(System.currentTimeMillis());
|
||||
log.warn(
|
||||
"tryLock, message queue lock expired, I got it. Group: {} OldClientId: {} NewClientId: {} {}", //
|
||||
group, //
|
||||
oldClientId, //
|
||||
clientId, //
|
||||
mq);
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
log.warn(
|
||||
"tryLock, message queue locked by other client. Group: {} OtherClientId: {} NewClientId: {} {}", //
|
||||
group, //
|
||||
oldClientId, //
|
||||
clientId, //
|
||||
mq);
|
||||
return false;
|
||||
} finally {
|
||||
this.lock.unlock();
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
log.error("putMessage exception", e);
|
||||
}
|
||||
} else {
|
||||
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private boolean isLocked(final String group, final MessageQueue mq, final String clientId) {
|
||||
ConcurrentHashMap<MessageQueue, LockEntry> groupValue = this.mqLockTable.get(group);
|
||||
if (groupValue != null) {
|
||||
LockEntry lockEntry = groupValue.get(mq);
|
||||
if (lockEntry != null) {
|
||||
boolean locked = lockEntry.isLocked(clientId);
|
||||
if (locked) {
|
||||
lockEntry.setLastUpdateTimestamp(System.currentTimeMillis());
|
||||
}
|
||||
|
||||
return locked;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
public Set<MessageQueue> tryLockBatch(final String group, final Set<MessageQueue> mqs,
|
||||
final String clientId) {
|
||||
Set<MessageQueue> lockedMqs = new HashSet<MessageQueue>(mqs.size());
|
||||
Set<MessageQueue> notLockedMqs = new HashSet<MessageQueue>(mqs.size());
|
||||
|
||||
|
||||
for (MessageQueue mq : mqs) {
|
||||
if (this.isLocked(group, mq, clientId)) {
|
||||
lockedMqs.add(mq);
|
||||
} else {
|
||||
notLockedMqs.add(mq);
|
||||
}
|
||||
}
|
||||
|
||||
if (!notLockedMqs.isEmpty()) {
|
||||
try {
|
||||
this.lock.lockInterruptibly();
|
||||
try {
|
||||
ConcurrentHashMap<MessageQueue, LockEntry> groupValue = this.mqLockTable.get(group);
|
||||
if (null == groupValue) {
|
||||
groupValue = new ConcurrentHashMap<MessageQueue, LockEntry>(32);
|
||||
this.mqLockTable.put(group, groupValue);
|
||||
}
|
||||
|
||||
|
||||
for (MessageQueue mq : notLockedMqs) {
|
||||
LockEntry lockEntry = groupValue.get(mq);
|
||||
if (null == lockEntry) {
|
||||
lockEntry = new LockEntry();
|
||||
lockEntry.setClientId(clientId);
|
||||
groupValue.put(mq, lockEntry);
|
||||
log.info(
|
||||
"tryLockBatch, message queue not locked, I got it. Group: {} NewClientId: {} {}", //
|
||||
group, //
|
||||
clientId, //
|
||||
mq);
|
||||
}
|
||||
|
||||
|
||||
if (lockEntry.isLocked(clientId)) {
|
||||
lockEntry.setLastUpdateTimestamp(System.currentTimeMillis());
|
||||
lockedMqs.add(mq);
|
||||
continue;
|
||||
}
|
||||
|
||||
String oldClientId = lockEntry.getClientId();
|
||||
|
||||
|
||||
if (lockEntry.isExpired()) {
|
||||
lockEntry.setClientId(clientId);
|
||||
lockEntry.setLastUpdateTimestamp(System.currentTimeMillis());
|
||||
log.warn(
|
||||
"tryLockBatch, message queue lock expired, I got it. Group: {} OldClientId: {} NewClientId: {} {}", //
|
||||
group, //
|
||||
oldClientId, //
|
||||
clientId, //
|
||||
mq);
|
||||
lockedMqs.add(mq);
|
||||
continue;
|
||||
}
|
||||
|
||||
|
||||
log.warn(
|
||||
"tryLockBatch, message queue locked by other client. Group: {} OtherClientId: {} NewClientId: {} {}", //
|
||||
group, //
|
||||
oldClientId, //
|
||||
clientId, //
|
||||
mq);
|
||||
}
|
||||
} finally {
|
||||
this.lock.unlock();
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
log.error("putMessage exception", e);
|
||||
}
|
||||
}
|
||||
|
||||
return lockedMqs;
|
||||
}
|
||||
|
||||
public void unlockBatch(final String group, final Set<MessageQueue> mqs, final String clientId) {
|
||||
try {
|
||||
this.lock.lockInterruptibly();
|
||||
try {
|
||||
ConcurrentHashMap<MessageQueue, LockEntry> groupValue = this.mqLockTable.get(group);
|
||||
if (null != groupValue) {
|
||||
for (MessageQueue mq : mqs) {
|
||||
LockEntry lockEntry = groupValue.get(mq);
|
||||
if (null != lockEntry) {
|
||||
if (lockEntry.getClientId().equals(clientId)) {
|
||||
groupValue.remove(mq);
|
||||
log.info("unlockBatch, Group: {} {} {}",
|
||||
group,
|
||||
mq,
|
||||
clientId);
|
||||
} else {
|
||||
log.warn("unlockBatch, but mq locked by other client: {}, Group: {} {} {}",
|
||||
lockEntry.getClientId(),
|
||||
group,
|
||||
mq,
|
||||
clientId);
|
||||
}
|
||||
} else {
|
||||
log.warn("unlockBatch, but mq not locked, Group: {} {} {}",
|
||||
group,
|
||||
mq,
|
||||
clientId);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
log.warn("unlockBatch, group not exist, Group: {} {}",
|
||||
group,
|
||||
clientId);
|
||||
}
|
||||
} finally {
|
||||
this.lock.unlock();
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
log.error("putMessage exception", e);
|
||||
}
|
||||
}
|
||||
|
||||
static class LockEntry {
|
||||
private String clientId;
|
||||
private volatile long lastUpdateTimestamp = System.currentTimeMillis();
|
||||
|
||||
|
||||
public String getClientId() {
|
||||
return clientId;
|
||||
}
|
||||
|
||||
|
||||
public void setClientId(String clientId) {
|
||||
this.clientId = clientId;
|
||||
}
|
||||
|
||||
|
||||
public long getLastUpdateTimestamp() {
|
||||
return lastUpdateTimestamp;
|
||||
}
|
||||
|
||||
|
||||
public void setLastUpdateTimestamp(long lastUpdateTimestamp) {
|
||||
this.lastUpdateTimestamp = lastUpdateTimestamp;
|
||||
}
|
||||
|
||||
public boolean isLocked(final String clientId) {
|
||||
boolean eq = this.clientId.equals(clientId);
|
||||
return eq && !this.isExpired();
|
||||
}
|
||||
|
||||
public boolean isExpired() {
|
||||
boolean expired =
|
||||
(System.currentTimeMillis() - this.lastUpdateTimestamp) > REBALANCE_LOCK_MAX_LIVE_TIME;
|
||||
|
||||
return expired;
|
||||
}
|
||||
}
|
||||
}
|
||||
+175
@@ -0,0 +1,175 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker.filtersrv;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.BrokerStartup;
|
||||
import com.alibaba.rocketmq.common.ThreadFactoryImpl;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingUtil;
|
||||
import io.netty.channel.Channel;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.Map.Entry;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.ScheduledExecutorService;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
|
||||
public class FilterServerManager {
|
||||
|
||||
public static final long FILTER_SERVER_MAX_IDLE_TIME_MILLS = 30000;
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private final ConcurrentHashMap<Channel, FilterServerInfo> filterServerTable =
|
||||
new ConcurrentHashMap<Channel, FilterServerInfo>(16);
|
||||
private final BrokerController brokerController;
|
||||
|
||||
private ScheduledExecutorService scheduledExecutorService = Executors
|
||||
.newSingleThreadScheduledExecutor(new ThreadFactoryImpl("FilterServerManagerScheduledThread"));
|
||||
|
||||
public FilterServerManager(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
public void start() {
|
||||
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
FilterServerManager.this.createFilterServer();
|
||||
} catch (Exception e) {
|
||||
log.error("", e);
|
||||
}
|
||||
}
|
||||
}, 1000 * 5, 1000 * 30, TimeUnit.MILLISECONDS);
|
||||
}
|
||||
|
||||
public void createFilterServer() {
|
||||
int more =
|
||||
this.brokerController.getBrokerConfig().getFilterServerNums() - this.filterServerTable.size();
|
||||
String cmd = this.buildStartCommand();
|
||||
for (int i = 0; i < more; i++) {
|
||||
FilterServerUtil.callShell(cmd, log);
|
||||
}
|
||||
}
|
||||
|
||||
private String buildStartCommand() {
|
||||
String config = "";
|
||||
if (BrokerStartup.configFile != null) {
|
||||
config = String.format("-c %s", BrokerStartup.configFile);
|
||||
}
|
||||
|
||||
if (this.brokerController.getBrokerConfig().getNamesrvAddr() != null) {
|
||||
config += String.format(" -n %s", this.brokerController.getBrokerConfig().getNamesrvAddr());
|
||||
}
|
||||
|
||||
if (RemotingUtil.isWindowsPlatform()) {
|
||||
return String.format("start /b %s\\bin\\mqfiltersrv.exe %s",
|
||||
this.brokerController.getBrokerConfig().getRocketmqHome(),
|
||||
config);
|
||||
} else {
|
||||
return String.format("sh %s/bin/startfsrv.sh %s",
|
||||
this.brokerController.getBrokerConfig().getRocketmqHome(),
|
||||
config);
|
||||
}
|
||||
}
|
||||
|
||||
public void shutdown() {
|
||||
this.scheduledExecutorService.shutdown();
|
||||
}
|
||||
|
||||
public void registerFilterServer(final Channel channel, final String filterServerAddr) {
|
||||
FilterServerInfo filterServerInfo = this.filterServerTable.get(channel);
|
||||
if (filterServerInfo != null) {
|
||||
filterServerInfo.setLastUpdateTimestamp(System.currentTimeMillis());
|
||||
} else {
|
||||
filterServerInfo = new FilterServerInfo();
|
||||
filterServerInfo.setFilterServerAddr(filterServerAddr);
|
||||
filterServerInfo.setLastUpdateTimestamp(System.currentTimeMillis());
|
||||
this.filterServerTable.put(channel, filterServerInfo);
|
||||
log.info("Receive a New Filter Server<{}>", filterServerAddr);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
*/
|
||||
public void scanNotActiveChannel() {
|
||||
|
||||
Iterator<Entry<Channel, FilterServerInfo>> it = this.filterServerTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<Channel, FilterServerInfo> next = it.next();
|
||||
long timestamp = next.getValue().getLastUpdateTimestamp();
|
||||
Channel channel = next.getKey();
|
||||
if ((System.currentTimeMillis() - timestamp) > FILTER_SERVER_MAX_IDLE_TIME_MILLS) {
|
||||
log.info("The Filter Server<{}> expired, remove it", next.getKey());
|
||||
it.remove();
|
||||
RemotingUtil.closeChannel(channel);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public void doChannelCloseEvent(final String remoteAddr, final Channel channel) {
|
||||
FilterServerInfo old = this.filterServerTable.remove(channel);
|
||||
if (old != null) {
|
||||
log.warn("The Filter Server<{}> connection<{}> closed, remove it", old.getFilterServerAddr(),
|
||||
remoteAddr);
|
||||
}
|
||||
}
|
||||
|
||||
public List<String> buildNewFilterServerList() {
|
||||
List<String> addr = new ArrayList<String>();
|
||||
Iterator<Entry<Channel, FilterServerInfo>> it = this.filterServerTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<Channel, FilterServerInfo> next = it.next();
|
||||
addr.add(next.getValue().getFilterServerAddr());
|
||||
}
|
||||
return addr;
|
||||
}
|
||||
|
||||
static class FilterServerInfo {
|
||||
private String filterServerAddr;
|
||||
private long lastUpdateTimestamp;
|
||||
|
||||
|
||||
public String getFilterServerAddr() {
|
||||
return filterServerAddr;
|
||||
}
|
||||
|
||||
|
||||
public void setFilterServerAddr(String filterServerAddr) {
|
||||
this.filterServerAddr = filterServerAddr;
|
||||
}
|
||||
|
||||
|
||||
public long getLastUpdateTimestamp() {
|
||||
return lastUpdateTimestamp;
|
||||
}
|
||||
|
||||
|
||||
public void setLastUpdateTimestamp(long lastUpdateTimestamp) {
|
||||
this.lastUpdateTimestamp = lastUpdateTimestamp;
|
||||
}
|
||||
}
|
||||
}
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker.filtersrv;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
|
||||
|
||||
public class FilterServerUtil {
|
||||
public static void callShell(final String shellString, final Logger log) {
|
||||
Process process = null;
|
||||
try {
|
||||
String[] cmdArray = splitShellString(shellString);
|
||||
process = Runtime.getRuntime().exec(cmdArray);
|
||||
process.waitFor();
|
||||
log.info("callShell: <{}> OK", shellString);
|
||||
} catch (Throwable e) {
|
||||
log.error("callShell: readLine IOException, " + shellString, e);
|
||||
} finally {
|
||||
if (null != process)
|
||||
process.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
private static String[] splitShellString(final String shellString) {
|
||||
String[] split = shellString.split(" ");
|
||||
return split;
|
||||
}
|
||||
}
|
||||
+115
@@ -0,0 +1,115 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.latency;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.common.ThreadFactoryImpl;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.remoting.netty.RequestTask;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingSysResponseCode;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.ScheduledExecutorService;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class BrokerFastFailure {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private final ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor(new ThreadFactoryImpl(
|
||||
"BrokerFastFailureScheduledThread"));
|
||||
private final BrokerController brokerController;
|
||||
|
||||
public BrokerFastFailure(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
public void start() {
|
||||
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
cleanExpiredRequest();
|
||||
}
|
||||
}, 1000, 10, TimeUnit.MILLISECONDS);
|
||||
}
|
||||
|
||||
private void cleanExpiredRequest() {
|
||||
while (this.brokerController.getMessageStore().isOSPageCacheBusy()) {
|
||||
try {
|
||||
if (!this.brokerController.getSendThreadPoolQueue().isEmpty()) {
|
||||
final Runnable runnable = this.brokerController.getSendThreadPoolQueue().poll(0, TimeUnit.SECONDS);
|
||||
if (null == runnable) {
|
||||
break;
|
||||
}
|
||||
|
||||
final RequestTask rt = castRunnable(runnable);
|
||||
rt.returnResponse(RemotingSysResponseCode.SYSTEM_BUSY, String.format("[PCBUSY_CLEAN_QUEUE]broker busy, start flow control for a while, period in queue: %sms, size of queue: %d", System.currentTimeMillis() - rt.getCreateTimestamp(), this.brokerController.getSendThreadPoolQueue().size()));
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
} catch (Throwable e) {
|
||||
}
|
||||
}
|
||||
|
||||
while (true) {
|
||||
try {
|
||||
if (!this.brokerController.getSendThreadPoolQueue().isEmpty()) {
|
||||
final Runnable runnable = this.brokerController.getSendThreadPoolQueue().peek();
|
||||
if (null == runnable) {
|
||||
break;
|
||||
}
|
||||
final RequestTask rt = castRunnable(runnable);
|
||||
if (rt.isStopRun()) {
|
||||
break;
|
||||
}
|
||||
|
||||
final long behind = System.currentTimeMillis() - rt.getCreateTimestamp();
|
||||
if (behind >= this.brokerController.getBrokerConfig().getWaitTimeMillsInSendQueue()) {
|
||||
if (this.brokerController.getSendThreadPoolQueue().remove(runnable)) {
|
||||
rt.setStopRun(true);
|
||||
rt.returnResponse(RemotingSysResponseCode.SYSTEM_BUSY, String.format("[TIMEOUT_CLEAN_QUEUE]broker busy, start flow control for a while, period in queue: %sms, size of queue: %d", behind, this.brokerController.getSendThreadPoolQueue().size()));
|
||||
}
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
} catch (Throwable e) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public static RequestTask castRunnable(final Runnable runnable) {
|
||||
try {
|
||||
FutureTaskExt object = (FutureTaskExt) runnable;
|
||||
return (RequestTask) object.getRunnable();
|
||||
} catch (Throwable e) {
|
||||
log.error(String.format("castRunnable exception, %s", runnable.getClass().getName()), e);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
public void shutdown() {
|
||||
this.scheduledExecutorService.shutdown();
|
||||
}
|
||||
}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker.latency;
|
||||
|
||||
import java.util.concurrent.*;
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class BrokerFixedThreadPoolExecutor extends ThreadPoolExecutor {
|
||||
public BrokerFixedThreadPoolExecutor(final int corePoolSize, final int maximumPoolSize, final long keepAliveTime, final TimeUnit unit, final BlockingQueue<Runnable> workQueue) {
|
||||
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
|
||||
}
|
||||
|
||||
public BrokerFixedThreadPoolExecutor(final int corePoolSize, final int maximumPoolSize, final long keepAliveTime, final TimeUnit unit, final BlockingQueue<Runnable> workQueue, final ThreadFactory threadFactory) {
|
||||
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory);
|
||||
}
|
||||
|
||||
public BrokerFixedThreadPoolExecutor(final int corePoolSize, final int maximumPoolSize, final long keepAliveTime, final TimeUnit unit, final BlockingQueue<Runnable> workQueue, final RejectedExecutionHandler handler) {
|
||||
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, handler);
|
||||
}
|
||||
|
||||
public BrokerFixedThreadPoolExecutor(final int corePoolSize, final int maximumPoolSize, final long keepAliveTime, final TimeUnit unit, final BlockingQueue<Runnable> workQueue, final ThreadFactory threadFactory, final RejectedExecutionHandler handler) {
|
||||
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected <T> RunnableFuture<T> newTaskFor(final Runnable runnable, final T value) {
|
||||
return new FutureTaskExt<T>(runnable, value);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker.latency;
|
||||
|
||||
import java.util.concurrent.Callable;
|
||||
import java.util.concurrent.FutureTask;
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class FutureTaskExt<V> extends FutureTask<V> {
|
||||
private final Runnable runnable;
|
||||
|
||||
public FutureTaskExt(final Callable<V> callable) {
|
||||
super(callable);
|
||||
this.runnable = null;
|
||||
}
|
||||
|
||||
public FutureTaskExt(final Runnable runnable, final V result) {
|
||||
super(runnable, result);
|
||||
this.runnable = runnable;
|
||||
}
|
||||
|
||||
public Runnable getRunnable() {
|
||||
return runnable;
|
||||
}
|
||||
}
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.longpolling;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ManyPullRequest {
|
||||
private final ArrayList<PullRequest> pullRequestList = new ArrayList<PullRequest>();
|
||||
|
||||
|
||||
public synchronized void addPullRequest(final PullRequest pullRequest) {
|
||||
this.pullRequestList.add(pullRequest);
|
||||
}
|
||||
|
||||
|
||||
public synchronized void addPullRequest(final List<PullRequest> many) {
|
||||
this.pullRequestList.addAll(many);
|
||||
}
|
||||
|
||||
|
||||
public synchronized List<PullRequest> cloneListAndClear() {
|
||||
if (!this.pullRequestList.isEmpty()) {
|
||||
List<PullRequest> result = (ArrayList<PullRequest>) this.pullRequestList.clone();
|
||||
this.pullRequestList.clear();
|
||||
return result;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
}
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker.longpolling;
|
||||
|
||||
import com.alibaba.rocketmq.store.MessageArrivingListener;
|
||||
|
||||
|
||||
public class NotifyMessageArrivingListener implements MessageArrivingListener {
|
||||
private final PullRequestHoldService pullRequestHoldService;
|
||||
|
||||
|
||||
public NotifyMessageArrivingListener(final PullRequestHoldService pullRequestHoldService) {
|
||||
this.pullRequestHoldService = pullRequestHoldService;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void arriving(String topic, int queueId, long logicOffset, long tagsCode) {
|
||||
this.pullRequestHoldService.notifyMessageArriving(topic, queueId, logicOffset, tagsCode);
|
||||
}
|
||||
}
|
||||
+74
@@ -0,0 +1,74 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.longpolling;
|
||||
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.SubscriptionData;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import io.netty.channel.Channel;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class PullRequest {
|
||||
private final RemotingCommand requestCommand;
|
||||
private final Channel clientChannel;
|
||||
private final long timeoutMillis;
|
||||
private final long suspendTimestamp;
|
||||
private final long pullFromThisOffset;
|
||||
private final SubscriptionData subscriptionData;
|
||||
|
||||
|
||||
public PullRequest(RemotingCommand requestCommand, Channel clientChannel, long timeoutMillis, long suspendTimestamp,
|
||||
long pullFromThisOffset, SubscriptionData subscriptionData) {
|
||||
this.requestCommand = requestCommand;
|
||||
this.clientChannel = clientChannel;
|
||||
this.timeoutMillis = timeoutMillis;
|
||||
this.suspendTimestamp = suspendTimestamp;
|
||||
this.pullFromThisOffset = pullFromThisOffset;
|
||||
this.subscriptionData = subscriptionData;
|
||||
}
|
||||
|
||||
|
||||
public RemotingCommand getRequestCommand() {
|
||||
return requestCommand;
|
||||
}
|
||||
|
||||
|
||||
public Channel getClientChannel() {
|
||||
return clientChannel;
|
||||
}
|
||||
|
||||
|
||||
public long getTimeoutMillis() {
|
||||
return timeoutMillis;
|
||||
}
|
||||
|
||||
|
||||
public long getSuspendTimestamp() {
|
||||
return suspendTimestamp;
|
||||
}
|
||||
|
||||
|
||||
public long getPullFromThisOffset() {
|
||||
return pullFromThisOffset;
|
||||
}
|
||||
|
||||
public SubscriptionData getSubscriptionData() {
|
||||
return subscriptionData;
|
||||
}
|
||||
}
|
||||
+169
@@ -0,0 +1,169 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.longpolling;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.common.ServiceThread;
|
||||
import com.alibaba.rocketmq.common.SystemClock;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.store.DefaultMessageFilter;
|
||||
import com.alibaba.rocketmq.store.MessageFilter;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class PullRequestHoldService extends ServiceThread {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private static final String TOPIC_QUEUEID_SEPARATOR = "@";
|
||||
private final BrokerController brokerController;
|
||||
private final SystemClock systemClock = new SystemClock();
|
||||
private final MessageFilter messageFilter = new DefaultMessageFilter();
|
||||
private ConcurrentHashMap<String/* topic@queueId */, ManyPullRequest> pullRequestTable =
|
||||
new ConcurrentHashMap<String, ManyPullRequest>(1024);
|
||||
|
||||
|
||||
public PullRequestHoldService(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
public void suspendPullRequest(final String topic, final int queueId, final PullRequest pullRequest) {
|
||||
String key = this.buildKey(topic, queueId);
|
||||
ManyPullRequest mpr = this.pullRequestTable.get(key);
|
||||
if (null == mpr) {
|
||||
mpr = new ManyPullRequest();
|
||||
ManyPullRequest prev = this.pullRequestTable.putIfAbsent(key, mpr);
|
||||
if (prev != null) {
|
||||
mpr = prev;
|
||||
}
|
||||
}
|
||||
|
||||
mpr.addPullRequest(pullRequest);
|
||||
}
|
||||
|
||||
private String buildKey(final String topic, final int queueId) {
|
||||
StringBuilder sb = new StringBuilder();
|
||||
sb.append(topic);
|
||||
sb.append(TOPIC_QUEUEID_SEPARATOR);
|
||||
sb.append(queueId);
|
||||
return sb.toString();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
log.info(this.getServiceName() + " service started");
|
||||
while (!this.isStopped()) {
|
||||
try {
|
||||
if (this.brokerController.getBrokerConfig().isLongPollingEnable()) {
|
||||
this.waitForRunning(5 * 1000);
|
||||
} else {
|
||||
this.waitForRunning(this.brokerController.getBrokerConfig().getShortPollingTimeMills());
|
||||
}
|
||||
|
||||
long beginLockTimestamp = this.systemClock.now();
|
||||
this.checkHoldRequest();
|
||||
long costTime = this.systemClock.now() - beginLockTimestamp;
|
||||
if (costTime > 5 * 1000) {
|
||||
log.info("[NOTIFYME] check hold request cost {} ms.", costTime);
|
||||
}
|
||||
} catch (Throwable e) {
|
||||
log.warn(this.getServiceName() + " service has exception. ", e);
|
||||
}
|
||||
}
|
||||
|
||||
log.info(this.getServiceName() + " service end");
|
||||
}
|
||||
|
||||
@Override
|
||||
public String getServiceName() {
|
||||
return PullRequestHoldService.class.getSimpleName();
|
||||
}
|
||||
|
||||
private void checkHoldRequest() {
|
||||
for (String key : this.pullRequestTable.keySet()) {
|
||||
String[] kArray = key.split(TOPIC_QUEUEID_SEPARATOR);
|
||||
if (kArray != null && 2 == kArray.length) {
|
||||
String topic = kArray[0];
|
||||
int queueId = Integer.parseInt(kArray[1]);
|
||||
final long offset = this.brokerController.getMessageStore().getMaxOffsetInQuque(topic, queueId);
|
||||
try {
|
||||
this.notifyMessageArriving(topic, queueId, offset);
|
||||
} catch (Throwable e) {
|
||||
log.error("check hold request failed. topic={}, queueId={}", topic, queueId, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public void notifyMessageArriving(final String topic, final int queueId, final long maxOffset) {
|
||||
notifyMessageArriving(topic, queueId, maxOffset, null);
|
||||
}
|
||||
|
||||
public void notifyMessageArriving(final String topic, final int queueId, final long maxOffset, final Long tagsCode) {
|
||||
String key = this.buildKey(topic, queueId);
|
||||
ManyPullRequest mpr = this.pullRequestTable.get(key);
|
||||
if (mpr != null) {
|
||||
List<PullRequest> requestList = mpr.cloneListAndClear();
|
||||
if (requestList != null) {
|
||||
List<PullRequest> replayList = new ArrayList<PullRequest>();
|
||||
|
||||
for (PullRequest request : requestList) {
|
||||
long newestOffset = maxOffset;
|
||||
if (newestOffset <= request.getPullFromThisOffset()) {
|
||||
newestOffset = this.brokerController.getMessageStore().getMaxOffsetInQuque(topic, queueId);
|
||||
}
|
||||
|
||||
Long tmp = tagsCode;
|
||||
if (newestOffset > request.getPullFromThisOffset()) {
|
||||
if (this.messageFilter.isMessageMatched(request.getSubscriptionData(), tmp)) {
|
||||
try {
|
||||
this.brokerController.getPullMessageProcessor().excuteRequestWhenWakeup(request.getClientChannel(),
|
||||
request.getRequestCommand());
|
||||
} catch (Throwable e) {
|
||||
log.error("execute request when wakeup failed.", e);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
if (System.currentTimeMillis() >= (request.getSuspendTimestamp() + request.getTimeoutMillis())) {
|
||||
try {
|
||||
this.brokerController.getPullMessageProcessor().excuteRequestWhenWakeup(request.getClientChannel(),
|
||||
request.getRequestCommand());
|
||||
} catch (Throwable e) {
|
||||
log.error("execute request when wakeup failed.", e);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
|
||||
replayList.add(request);
|
||||
}
|
||||
|
||||
if (!replayList.isEmpty()) {
|
||||
mpr.addPullRequest(replayList);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+172
@@ -0,0 +1,172 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.mqtrace;
|
||||
|
||||
import com.alibaba.rocketmq.store.stats.BrokerStatsManager;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
|
||||
public class ConsumeMessageContext {
|
||||
private String consumerGroup;
|
||||
private String topic;
|
||||
private Integer queueId;
|
||||
private String clientHost;
|
||||
private String storeHost;
|
||||
private Map<String, Long> messageIds;
|
||||
private int bodyLength;
|
||||
private boolean success;
|
||||
private String status;
|
||||
private Object mqTraceContext;
|
||||
|
||||
private String commercialOwner;
|
||||
private BrokerStatsManager.StatsType commercialRcvStats;
|
||||
private int commercialRcvTimes;
|
||||
private int commercialRcvSize;
|
||||
|
||||
|
||||
public String getConsumerGroup() {
|
||||
return consumerGroup;
|
||||
}
|
||||
|
||||
|
||||
public void setConsumerGroup(String consumerGroup) {
|
||||
this.consumerGroup = consumerGroup;
|
||||
}
|
||||
|
||||
|
||||
public String getTopic() {
|
||||
return topic;
|
||||
}
|
||||
|
||||
|
||||
public void setTopic(String topic) {
|
||||
this.topic = topic;
|
||||
}
|
||||
|
||||
|
||||
public Integer getQueueId() {
|
||||
return queueId;
|
||||
}
|
||||
|
||||
|
||||
public void setQueueId(Integer queueId) {
|
||||
this.queueId = queueId;
|
||||
}
|
||||
|
||||
|
||||
public String getClientHost() {
|
||||
return clientHost;
|
||||
}
|
||||
|
||||
|
||||
public void setClientHost(String clientHost) {
|
||||
this.clientHost = clientHost;
|
||||
}
|
||||
|
||||
|
||||
public String getStoreHost() {
|
||||
return storeHost;
|
||||
}
|
||||
|
||||
|
||||
public void setStoreHost(String storeHost) {
|
||||
this.storeHost = storeHost;
|
||||
}
|
||||
|
||||
|
||||
public Map<String, Long> getMessageIds() {
|
||||
return messageIds;
|
||||
}
|
||||
|
||||
|
||||
public void setMessageIds(Map<String, Long> messageIds) {
|
||||
this.messageIds = messageIds;
|
||||
}
|
||||
|
||||
|
||||
public boolean isSuccess() {
|
||||
return success;
|
||||
}
|
||||
|
||||
|
||||
public void setSuccess(boolean success) {
|
||||
this.success = success;
|
||||
}
|
||||
|
||||
|
||||
public String getStatus() {
|
||||
return status;
|
||||
}
|
||||
|
||||
|
||||
public void setStatus(String status) {
|
||||
this.status = status;
|
||||
}
|
||||
|
||||
|
||||
public Object getMqTraceContext() {
|
||||
return mqTraceContext;
|
||||
}
|
||||
|
||||
|
||||
public void setMqTraceContext(Object mqTraceContext) {
|
||||
this.mqTraceContext = mqTraceContext;
|
||||
}
|
||||
|
||||
|
||||
public int getBodyLength() {
|
||||
return bodyLength;
|
||||
}
|
||||
|
||||
|
||||
public void setBodyLength(int bodyLength) {
|
||||
this.bodyLength = bodyLength;
|
||||
}
|
||||
|
||||
public String getCommercialOwner() {
|
||||
return commercialOwner;
|
||||
}
|
||||
|
||||
public void setCommercialOwner(final String commercialOwner) {
|
||||
this.commercialOwner = commercialOwner;
|
||||
}
|
||||
|
||||
public BrokerStatsManager.StatsType getCommercialRcvStats() {
|
||||
return commercialRcvStats;
|
||||
}
|
||||
|
||||
public void setCommercialRcvStats(final BrokerStatsManager.StatsType commercialRcvStats) {
|
||||
this.commercialRcvStats = commercialRcvStats;
|
||||
}
|
||||
|
||||
public int getCommercialRcvTimes() {
|
||||
return commercialRcvTimes;
|
||||
}
|
||||
|
||||
public void setCommercialRcvTimes(final int commercialRcvTimes) {
|
||||
this.commercialRcvTimes = commercialRcvTimes;
|
||||
}
|
||||
|
||||
public int getCommercialRcvSize() {
|
||||
return commercialRcvSize;
|
||||
}
|
||||
|
||||
public void setCommercialRcvSize(final int commercialRcvSize) {
|
||||
this.commercialRcvSize = commercialRcvSize;
|
||||
}
|
||||
}
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.mqtrace;
|
||||
|
||||
public interface ConsumeMessageHook {
|
||||
String hookName();
|
||||
|
||||
|
||||
void consumeMessageBefore(final ConsumeMessageContext context);
|
||||
|
||||
|
||||
void consumeMessageAfter(final ConsumeMessageContext context);
|
||||
}
|
||||
+261
@@ -0,0 +1,261 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.mqtrace;
|
||||
|
||||
import com.alibaba.rocketmq.common.message.MessageType;
|
||||
import com.alibaba.rocketmq.store.stats.BrokerStatsManager;
|
||||
|
||||
import java.util.Properties;
|
||||
|
||||
|
||||
public class SendMessageContext {
|
||||
private String producerGroup;
|
||||
private String topic;
|
||||
private String msgId;
|
||||
private String originMsgId;
|
||||
private Integer queueId;
|
||||
private Long queueOffset;
|
||||
private String brokerAddr;
|
||||
private String bornHost;
|
||||
private int bodyLength;
|
||||
private int code;
|
||||
private String errorMsg;
|
||||
private String msgProps;
|
||||
private Object mqTraceContext;
|
||||
private Properties extProps;
|
||||
private String brokerRegionId;
|
||||
private String msgUniqueKey;
|
||||
private long bornTimeStamp;
|
||||
private MessageType msgType = MessageType.Trans_msg_Commit;
|
||||
private boolean isSuccess = false;
|
||||
//For Commercial
|
||||
private String commercialOwner;
|
||||
private BrokerStatsManager.StatsType commercialSendStats;
|
||||
private int commercialSendSize;
|
||||
private int commercialSendTimes;
|
||||
|
||||
public boolean isSuccess() {
|
||||
return isSuccess;
|
||||
}
|
||||
|
||||
public void setSuccess(final boolean success) {
|
||||
isSuccess = success;
|
||||
}
|
||||
|
||||
public MessageType getMsgType() {
|
||||
return msgType;
|
||||
}
|
||||
|
||||
public void setMsgType(final MessageType msgType) {
|
||||
this.msgType = msgType;
|
||||
}
|
||||
|
||||
public String getMsgUniqueKey() {
|
||||
return msgUniqueKey;
|
||||
}
|
||||
|
||||
public void setMsgUniqueKey(final String msgUniqueKey) {
|
||||
this.msgUniqueKey = msgUniqueKey;
|
||||
}
|
||||
|
||||
public long getBornTimeStamp() {
|
||||
return bornTimeStamp;
|
||||
}
|
||||
|
||||
public void setBornTimeStamp(final long bornTimeStamp) {
|
||||
this.bornTimeStamp = bornTimeStamp;
|
||||
}
|
||||
|
||||
public String getBrokerRegionId() {
|
||||
return brokerRegionId;
|
||||
}
|
||||
|
||||
public void setBrokerRegionId(final String brokerRegionId) {
|
||||
this.brokerRegionId = brokerRegionId;
|
||||
}
|
||||
|
||||
public String getProducerGroup() {
|
||||
return producerGroup;
|
||||
}
|
||||
|
||||
|
||||
public void setProducerGroup(String producerGroup) {
|
||||
this.producerGroup = producerGroup;
|
||||
}
|
||||
|
||||
|
||||
public String getTopic() {
|
||||
return topic;
|
||||
}
|
||||
|
||||
|
||||
public void setTopic(String topic) {
|
||||
this.topic = topic;
|
||||
}
|
||||
|
||||
|
||||
public String getMsgId() {
|
||||
return msgId;
|
||||
}
|
||||
|
||||
|
||||
public void setMsgId(String msgId) {
|
||||
this.msgId = msgId;
|
||||
}
|
||||
|
||||
|
||||
public String getOriginMsgId() {
|
||||
return originMsgId;
|
||||
}
|
||||
|
||||
|
||||
public void setOriginMsgId(String originMsgId) {
|
||||
this.originMsgId = originMsgId;
|
||||
}
|
||||
|
||||
|
||||
public Integer getQueueId() {
|
||||
return queueId;
|
||||
}
|
||||
|
||||
|
||||
public void setQueueId(Integer queueId) {
|
||||
this.queueId = queueId;
|
||||
}
|
||||
|
||||
|
||||
public Long getQueueOffset() {
|
||||
return queueOffset;
|
||||
}
|
||||
|
||||
|
||||
public void setQueueOffset(Long queueOffset) {
|
||||
this.queueOffset = queueOffset;
|
||||
}
|
||||
|
||||
|
||||
public String getBrokerAddr() {
|
||||
return brokerAddr;
|
||||
}
|
||||
|
||||
|
||||
public void setBrokerAddr(String brokerAddr) {
|
||||
this.brokerAddr = brokerAddr;
|
||||
}
|
||||
|
||||
|
||||
public String getBornHost() {
|
||||
return bornHost;
|
||||
}
|
||||
|
||||
|
||||
public void setBornHost(String bornHost) {
|
||||
this.bornHost = bornHost;
|
||||
}
|
||||
|
||||
|
||||
public int getBodyLength() {
|
||||
return bodyLength;
|
||||
}
|
||||
|
||||
|
||||
public void setBodyLength(int bodyLength) {
|
||||
this.bodyLength = bodyLength;
|
||||
}
|
||||
|
||||
|
||||
public int getCode() {
|
||||
return code;
|
||||
}
|
||||
|
||||
|
||||
public void setCode(int code) {
|
||||
this.code = code;
|
||||
}
|
||||
|
||||
|
||||
public String getErrorMsg() {
|
||||
return errorMsg;
|
||||
}
|
||||
|
||||
|
||||
public void setErrorMsg(String errorMsg) {
|
||||
this.errorMsg = errorMsg;
|
||||
}
|
||||
|
||||
|
||||
public String getMsgProps() {
|
||||
return msgProps;
|
||||
}
|
||||
|
||||
|
||||
public void setMsgProps(String msgProps) {
|
||||
this.msgProps = msgProps;
|
||||
}
|
||||
|
||||
|
||||
public Object getMqTraceContext() {
|
||||
return mqTraceContext;
|
||||
}
|
||||
|
||||
|
||||
public void setMqTraceContext(Object mqTraceContext) {
|
||||
this.mqTraceContext = mqTraceContext;
|
||||
}
|
||||
|
||||
|
||||
public Properties getExtProps() {
|
||||
return extProps;
|
||||
}
|
||||
|
||||
|
||||
public void setExtProps(Properties extProps) {
|
||||
this.extProps = extProps;
|
||||
}
|
||||
|
||||
public String getCommercialOwner() {
|
||||
return commercialOwner;
|
||||
}
|
||||
|
||||
public void setCommercialOwner(final String commercialOwner) {
|
||||
this.commercialOwner = commercialOwner;
|
||||
}
|
||||
|
||||
public BrokerStatsManager.StatsType getCommercialSendStats() {
|
||||
return commercialSendStats;
|
||||
}
|
||||
|
||||
public void setCommercialSendStats(final BrokerStatsManager.StatsType commercialSendStats) {
|
||||
this.commercialSendStats = commercialSendStats;
|
||||
}
|
||||
|
||||
public int getCommercialSendSize() {
|
||||
return commercialSendSize;
|
||||
}
|
||||
|
||||
public void setCommercialSendSize(final int commercialSendSize) {
|
||||
this.commercialSendSize = commercialSendSize;
|
||||
}
|
||||
|
||||
public int getCommercialSendTimes() {
|
||||
return commercialSendTimes;
|
||||
}
|
||||
|
||||
public void setCommercialSendTimes(final int commercialSendTimes) {
|
||||
this.commercialSendTimes = commercialSendTimes;
|
||||
}
|
||||
}
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.mqtrace;
|
||||
|
||||
public interface SendMessageHook {
|
||||
public String hookName();
|
||||
|
||||
|
||||
public void sendMessageBefore(final SendMessageContext context);
|
||||
|
||||
|
||||
public void sendMessageAfter(final SendMessageContext context);
|
||||
}
|
||||
+249
@@ -0,0 +1,249 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.offset;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.BrokerPathConfigHelper;
|
||||
import com.alibaba.rocketmq.common.ConfigManager;
|
||||
import com.alibaba.rocketmq.common.UtilAll;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingSerializable;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.*;
|
||||
import java.util.Map.Entry;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ConsumerOffsetManager extends ConfigManager {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private static final String TOPIC_GROUP_SEPARATOR = "@";
|
||||
|
||||
private ConcurrentHashMap<String/* topic@group */, ConcurrentHashMap<Integer, Long>> offsetTable =
|
||||
new ConcurrentHashMap<String, ConcurrentHashMap<Integer, Long>>(512);
|
||||
|
||||
private transient BrokerController brokerController;
|
||||
|
||||
|
||||
public ConsumerOffsetManager() {
|
||||
}
|
||||
|
||||
|
||||
public ConsumerOffsetManager(BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
|
||||
public void scanUnsubscribedTopic() {
|
||||
Iterator<Entry<String, ConcurrentHashMap<Integer, Long>>> it = this.offsetTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<String, ConcurrentHashMap<Integer, Long>> next = it.next();
|
||||
String topicAtGroup = next.getKey();
|
||||
String[] arrays = topicAtGroup.split(TOPIC_GROUP_SEPARATOR);
|
||||
if (arrays != null && arrays.length == 2) {
|
||||
String topic = arrays[0];
|
||||
String group = arrays[1];
|
||||
|
||||
if (null == brokerController.getConsumerManager().findSubscriptionData(group, topic)
|
||||
&& this.offsetBehindMuchThanData(topic, next.getValue())) {
|
||||
it.remove();
|
||||
log.warn("remove topic offset, {}", topicAtGroup);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
private boolean offsetBehindMuchThanData(final String topic, ConcurrentHashMap<Integer, Long> table) {
|
||||
Iterator<Entry<Integer, Long>> it = table.entrySet().iterator();
|
||||
boolean result = !table.isEmpty();
|
||||
|
||||
while (it.hasNext() && result) {
|
||||
Entry<Integer, Long> next = it.next();
|
||||
long minOffsetInStore = this.brokerController.getMessageStore().getMinOffsetInQuque(topic, next.getKey());
|
||||
long offsetInPersist = next.getValue();
|
||||
if (offsetInPersist > minOffsetInStore) {
|
||||
result = false;
|
||||
} else {
|
||||
result = true;
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
public Set<String> whichTopicByConsumer(final String group) {
|
||||
Set<String> topics = new HashSet<String>();
|
||||
|
||||
Iterator<Entry<String, ConcurrentHashMap<Integer, Long>>> it = this.offsetTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<String, ConcurrentHashMap<Integer, Long>> next = it.next();
|
||||
String topicAtGroup = next.getKey();
|
||||
String[] arrays = topicAtGroup.split(TOPIC_GROUP_SEPARATOR);
|
||||
if (arrays != null && arrays.length == 2) {
|
||||
if (group.equals(arrays[1])) {
|
||||
topics.add(arrays[0]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return topics;
|
||||
}
|
||||
|
||||
|
||||
public Set<String> whichGroupByTopic(final String topic) {
|
||||
Set<String> groups = new HashSet<String>();
|
||||
|
||||
Iterator<Entry<String, ConcurrentHashMap<Integer, Long>>> it = this.offsetTable.entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<String, ConcurrentHashMap<Integer, Long>> next = it.next();
|
||||
String topicAtGroup = next.getKey();
|
||||
String[] arrays = topicAtGroup.split(TOPIC_GROUP_SEPARATOR);
|
||||
if (arrays != null && arrays.length == 2) {
|
||||
if (topic.equals(arrays[0])) {
|
||||
groups.add(arrays[1]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return groups;
|
||||
}
|
||||
|
||||
|
||||
public void commitOffset(final String clientHost, final String group, final String topic, final int queueId, final long offset) {
|
||||
// topic@group
|
||||
String key = topic + TOPIC_GROUP_SEPARATOR + group;
|
||||
this.commitOffset(clientHost, key, queueId, offset);
|
||||
}
|
||||
|
||||
private void commitOffset(final String clientHost, final String key, final int queueId, final long offset) {
|
||||
ConcurrentHashMap<Integer, Long> map = this.offsetTable.get(key);
|
||||
if (null == map) {
|
||||
map = new ConcurrentHashMap<Integer, Long>(32);
|
||||
map.put(queueId, offset);
|
||||
this.offsetTable.put(key, map);
|
||||
} else {
|
||||
Long storeOffset = map.put(queueId, offset);
|
||||
if (storeOffset != null && offset < storeOffset) {
|
||||
log.warn("[NOTIFYME]update consumer offset less than store. clientHost={}, key={}, queueId={}, requestOffset={}, storeOffset={}", clientHost, key, queueId, offset, storeOffset);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public long queryOffset(final String group, final String topic, final int queueId) {
|
||||
// topic@group
|
||||
String key = topic + TOPIC_GROUP_SEPARATOR + group;
|
||||
ConcurrentHashMap<Integer, Long> map = this.offsetTable.get(key);
|
||||
if (null != map) {
|
||||
Long offset = map.get(queueId);
|
||||
if (offset != null)
|
||||
return offset;
|
||||
}
|
||||
|
||||
return -1;
|
||||
}
|
||||
|
||||
public String encode() {
|
||||
return this.encode(false);
|
||||
}
|
||||
|
||||
@Override
|
||||
public String configFilePath() {
|
||||
return BrokerPathConfigHelper.getConsumerOffsetPath(this.brokerController.getMessageStoreConfig().getStorePathRootDir());
|
||||
}
|
||||
|
||||
@Override
|
||||
public void decode(String jsonString) {
|
||||
if (jsonString != null) {
|
||||
ConsumerOffsetManager obj = RemotingSerializable.fromJson(jsonString, ConsumerOffsetManager.class);
|
||||
if (obj != null) {
|
||||
this.offsetTable = obj.offsetTable;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public String encode(final boolean prettyFormat) {
|
||||
return RemotingSerializable.toJson(this, prettyFormat);
|
||||
}
|
||||
|
||||
public ConcurrentHashMap<String, ConcurrentHashMap<Integer, Long>> getOffsetTable() {
|
||||
return offsetTable;
|
||||
}
|
||||
|
||||
|
||||
public void setOffsetTable(ConcurrentHashMap<String, ConcurrentHashMap<Integer, Long>> offsetTable) {
|
||||
this.offsetTable = offsetTable;
|
||||
}
|
||||
|
||||
|
||||
public Map<Integer, Long> queryMinOffsetInAllGroup(final String topic, final String filterGroups) {
|
||||
|
||||
Map<Integer, Long> queueMinOffset = new HashMap<Integer, Long>();
|
||||
Set<String> topicGroups = this.offsetTable.keySet();
|
||||
if (!UtilAll.isBlank(filterGroups)) {
|
||||
for (String group : filterGroups.split(",")) {
|
||||
Iterator<String> it = topicGroups.iterator();
|
||||
while (it.hasNext()) {
|
||||
if (group.equals(it.next().split(TOPIC_GROUP_SEPARATOR)[1])) {
|
||||
it.remove();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (Map.Entry<String, ConcurrentHashMap<Integer, Long>> offSetEntry : this.offsetTable.entrySet()) {
|
||||
String topicGroup = offSetEntry.getKey();
|
||||
String[] topicGroupArr = topicGroup.split(TOPIC_GROUP_SEPARATOR);
|
||||
if (topic.equals(topicGroupArr[0])) {
|
||||
for (Entry<Integer, Long> entry : offSetEntry.getValue().entrySet()) {
|
||||
long minOffset = this.brokerController.getMessageStore().getMinOffsetInQuque(topic, entry.getKey());
|
||||
if (entry.getValue() >= minOffset) {
|
||||
Long offset = queueMinOffset.get(entry.getKey());
|
||||
if (offset == null) {
|
||||
queueMinOffset.put(entry.getKey(), Math.min(Long.MAX_VALUE, entry.getValue()));
|
||||
} else {
|
||||
queueMinOffset.put(entry.getKey(), Math.min(entry.getValue(), offset));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
return queueMinOffset;
|
||||
}
|
||||
|
||||
|
||||
public Map<Integer, Long> queryOffset(final String group, final String topic) {
|
||||
// topic@group
|
||||
String key = topic + TOPIC_GROUP_SEPARATOR + group;
|
||||
return this.offsetTable.get(key);
|
||||
}
|
||||
|
||||
|
||||
public void cloneOffset(final String srcGroup, final String destGroup, final String topic) {
|
||||
ConcurrentHashMap<Integer, Long> offsets = this.offsetTable.get(topic + TOPIC_GROUP_SEPARATOR + srcGroup);
|
||||
if (offsets != null) {
|
||||
this.offsetTable.put(topic + TOPIC_GROUP_SEPARATOR + destGroup, new ConcurrentHashMap<Integer, Long>(offsets));
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,302 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.out;
|
||||
|
||||
import com.alibaba.rocketmq.client.exception.MQBrokerException;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.namesrv.RegisterBrokerResult;
|
||||
import com.alibaba.rocketmq.common.namesrv.TopAddressing;
|
||||
import com.alibaba.rocketmq.common.protocol.RequestCode;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
import com.alibaba.rocketmq.common.protocol.body.*;
|
||||
import com.alibaba.rocketmq.common.protocol.header.namesrv.RegisterBrokerRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.namesrv.RegisterBrokerResponseHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.namesrv.UnRegisterBrokerRequestHeader;
|
||||
import com.alibaba.rocketmq.remoting.RPCHook;
|
||||
import com.alibaba.rocketmq.remoting.RemotingClient;
|
||||
import com.alibaba.rocketmq.remoting.exception.*;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyClientConfig;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyRemotingClient;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.io.UnsupportedEncodingException;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
* @author manhong.yqd
|
||||
*/
|
||||
public class BrokerOuterAPI {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private final RemotingClient remotingClient;
|
||||
private final TopAddressing topAddressing = new TopAddressing(MixAll.WS_ADDR);
|
||||
private String nameSrvAddr = null;
|
||||
|
||||
public BrokerOuterAPI(final NettyClientConfig nettyClientConfig) {
|
||||
this(nettyClientConfig, null);
|
||||
}
|
||||
|
||||
public BrokerOuterAPI(final NettyClientConfig nettyClientConfig, RPCHook rpcHook) {
|
||||
this.remotingClient = new NettyRemotingClient(nettyClientConfig);
|
||||
this.remotingClient.registerRPCHook(rpcHook);
|
||||
}
|
||||
|
||||
public void start() {
|
||||
this.remotingClient.start();
|
||||
}
|
||||
|
||||
public void shutdown() {
|
||||
this.remotingClient.shutdown();
|
||||
}
|
||||
|
||||
public String fetchNameServerAddr() {
|
||||
try {
|
||||
String addrs = this.topAddressing.fetchNSAddr();
|
||||
if (addrs != null) {
|
||||
if (!addrs.equals(this.nameSrvAddr)) {
|
||||
log.info("name server address changed, old: " + this.nameSrvAddr + " new: " + addrs);
|
||||
this.updateNameServerAddressList(addrs);
|
||||
this.nameSrvAddr = addrs;
|
||||
return nameSrvAddr;
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("fetchNameServerAddr Exception", e);
|
||||
}
|
||||
return nameSrvAddr;
|
||||
}
|
||||
|
||||
public void updateNameServerAddressList(final String addrs) {
|
||||
List<String> lst = new ArrayList<String>();
|
||||
String[] addrArray = addrs.split(";");
|
||||
if (addrArray != null) {
|
||||
for (String addr : addrArray) {
|
||||
lst.add(addr);
|
||||
}
|
||||
|
||||
this.remotingClient.updateNameServerAddressList(lst);
|
||||
}
|
||||
}
|
||||
|
||||
public RegisterBrokerResult registerBrokerAll(
|
||||
final String clusterName,
|
||||
final String brokerAddr,
|
||||
final String brokerName,
|
||||
final long brokerId,
|
||||
final String haServerAddr,
|
||||
final TopicConfigSerializeWrapper topicConfigWrapper,
|
||||
final List<String> filterServerList,
|
||||
final boolean oneway,
|
||||
final int timeoutMills) {
|
||||
RegisterBrokerResult registerBrokerResult = null;
|
||||
|
||||
List<String> nameServerAddressList = this.remotingClient.getNameServerAddressList();
|
||||
if (nameServerAddressList != null) {
|
||||
for (String namesrvAddr : nameServerAddressList) {
|
||||
try {
|
||||
RegisterBrokerResult result = this.registerBroker(namesrvAddr, clusterName, brokerAddr, brokerName, brokerId,
|
||||
haServerAddr, topicConfigWrapper, filterServerList, oneway, timeoutMills);
|
||||
if (result != null) {
|
||||
registerBrokerResult = result;
|
||||
}
|
||||
|
||||
log.info("register broker to name server {} OK", namesrvAddr);
|
||||
} catch (Exception e) {
|
||||
log.warn("registerBroker Exception, " + namesrvAddr, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return registerBrokerResult;
|
||||
}
|
||||
|
||||
private RegisterBrokerResult registerBroker(
|
||||
final String namesrvAddr,
|
||||
final String clusterName,
|
||||
final String brokerAddr,
|
||||
final String brokerName,
|
||||
final long brokerId,
|
||||
final String haServerAddr,
|
||||
final TopicConfigSerializeWrapper topicConfigWrapper,
|
||||
final List<String> filterServerList,
|
||||
final boolean oneway,
|
||||
final int timeoutMills
|
||||
) throws RemotingCommandException, MQBrokerException, RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
|
||||
InterruptedException {
|
||||
RegisterBrokerRequestHeader requestHeader = new RegisterBrokerRequestHeader();
|
||||
requestHeader.setBrokerAddr(brokerAddr);
|
||||
requestHeader.setBrokerId(brokerId);
|
||||
requestHeader.setBrokerName(brokerName);
|
||||
requestHeader.setClusterName(clusterName);
|
||||
requestHeader.setHaServerAddr(haServerAddr);
|
||||
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.REGISTER_BROKER, requestHeader);
|
||||
|
||||
RegisterBrokerBody requestBody = new RegisterBrokerBody();
|
||||
requestBody.setTopicConfigSerializeWrapper(topicConfigWrapper);
|
||||
requestBody.setFilterServerList(filterServerList);
|
||||
request.setBody(requestBody.encode());
|
||||
|
||||
if (oneway) {
|
||||
try {
|
||||
this.remotingClient.invokeOneway(namesrvAddr, request, timeoutMills);
|
||||
} catch (RemotingTooMuchRequestException e) {
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
RemotingCommand response = this.remotingClient.invokeSync(namesrvAddr, request, timeoutMills);
|
||||
assert response != null;
|
||||
switch (response.getCode()) {
|
||||
case ResponseCode.SUCCESS: {
|
||||
RegisterBrokerResponseHeader responseHeader =
|
||||
(RegisterBrokerResponseHeader) response.decodeCommandCustomHeader(RegisterBrokerResponseHeader.class);
|
||||
RegisterBrokerResult result = new RegisterBrokerResult();
|
||||
result.setMasterAddr(responseHeader.getMasterAddr());
|
||||
result.setHaServerAddr(responseHeader.getHaServerAddr());
|
||||
result.setHaServerAddr(responseHeader.getHaServerAddr());
|
||||
if (response.getBody() != null) {
|
||||
result.setKvTable(KVTable.decode(response.getBody(), KVTable.class));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
throw new MQBrokerException(response.getCode(), response.getRemark());
|
||||
}
|
||||
|
||||
public void unregisterBrokerAll(
|
||||
final String clusterName,
|
||||
final String brokerAddr,
|
||||
final String brokerName,
|
||||
final long brokerId
|
||||
) {
|
||||
List<String> nameServerAddressList = this.remotingClient.getNameServerAddressList();
|
||||
if (nameServerAddressList != null) {
|
||||
for (String namesrvAddr : nameServerAddressList) {
|
||||
try {
|
||||
this.unregisterBroker(namesrvAddr, clusterName, brokerAddr, brokerName, brokerId);
|
||||
log.info("unregisterBroker OK, NamesrvAddr: {}", namesrvAddr);
|
||||
} catch (Exception e) {
|
||||
log.warn("unregisterBroker Exception, " + namesrvAddr, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public void unregisterBroker(
|
||||
final String namesrvAddr,
|
||||
final String clusterName,
|
||||
final String brokerAddr,
|
||||
final String brokerName,
|
||||
final long brokerId
|
||||
) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, InterruptedException, MQBrokerException {
|
||||
UnRegisterBrokerRequestHeader requestHeader = new UnRegisterBrokerRequestHeader();
|
||||
requestHeader.setBrokerAddr(brokerAddr);
|
||||
requestHeader.setBrokerId(brokerId);
|
||||
requestHeader.setBrokerName(brokerName);
|
||||
requestHeader.setClusterName(clusterName);
|
||||
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.UNREGISTER_BROKER, requestHeader);
|
||||
|
||||
RemotingCommand response = this.remotingClient.invokeSync(namesrvAddr, request, 3000);
|
||||
assert response != null;
|
||||
switch (response.getCode()) {
|
||||
case ResponseCode.SUCCESS: {
|
||||
return;
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
throw new MQBrokerException(response.getCode(), response.getRemark());
|
||||
}
|
||||
|
||||
public TopicConfigSerializeWrapper getAllTopicConfig(final String addr) throws RemotingConnectException, RemotingSendRequestException,
|
||||
RemotingTimeoutException, InterruptedException, MQBrokerException {
|
||||
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_ALL_TOPIC_CONFIG, null);
|
||||
|
||||
RemotingCommand response = this.remotingClient.invokeSync(MixAll.brokerVIPChannel(true, addr), request, 3000);
|
||||
assert response != null;
|
||||
switch (response.getCode()) {
|
||||
case ResponseCode.SUCCESS: {
|
||||
return TopicConfigSerializeWrapper.decode(response.getBody(), TopicConfigSerializeWrapper.class);
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
throw new MQBrokerException(response.getCode(), response.getRemark());
|
||||
}
|
||||
|
||||
public ConsumerOffsetSerializeWrapper getAllConsumerOffset(final String addr) throws InterruptedException, RemotingTimeoutException,
|
||||
RemotingSendRequestException, RemotingConnectException, MQBrokerException {
|
||||
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_ALL_CONSUMER_OFFSET, null);
|
||||
RemotingCommand response = this.remotingClient.invokeSync(addr, request, 3000);
|
||||
assert response != null;
|
||||
switch (response.getCode()) {
|
||||
case ResponseCode.SUCCESS: {
|
||||
return ConsumerOffsetSerializeWrapper.decode(response.getBody(), ConsumerOffsetSerializeWrapper.class);
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
throw new MQBrokerException(response.getCode(), response.getRemark());
|
||||
}
|
||||
|
||||
public String getAllDelayOffset(final String addr) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException,
|
||||
RemotingConnectException, MQBrokerException, UnsupportedEncodingException {
|
||||
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_ALL_DELAY_OFFSET, null);
|
||||
RemotingCommand response = this.remotingClient.invokeSync(addr, request, 3000);
|
||||
assert response != null;
|
||||
switch (response.getCode()) {
|
||||
case ResponseCode.SUCCESS: {
|
||||
return new String(response.getBody(), MixAll.DEFAULT_CHARSET);
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
throw new MQBrokerException(response.getCode(), response.getRemark());
|
||||
}
|
||||
|
||||
public SubscriptionGroupWrapper getAllSubscriptionGroupConfig(final String addr) throws InterruptedException, RemotingTimeoutException,
|
||||
RemotingSendRequestException, RemotingConnectException, MQBrokerException {
|
||||
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_ALL_SUBSCRIPTIONGROUP_CONFIG, null);
|
||||
RemotingCommand response = this.remotingClient.invokeSync(addr, request, 3000);
|
||||
assert response != null;
|
||||
switch (response.getCode()) {
|
||||
case ResponseCode.SUCCESS: {
|
||||
return SubscriptionGroupWrapper.decode(response.getBody(), SubscriptionGroupWrapper.class);
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
throw new MQBrokerException(response.getCode(), response.getRemark());
|
||||
}
|
||||
|
||||
public void registerRPCHook(RPCHook rpcHook) {
|
||||
remotingClient.registerRPCHook(rpcHook);
|
||||
}
|
||||
}
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.pagecache;
|
||||
|
||||
import com.alibaba.rocketmq.store.GetMessageResult;
|
||||
import io.netty.channel.FileRegion;
|
||||
import io.netty.util.AbstractReferenceCounted;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.ByteBuffer;
|
||||
import java.nio.channels.WritableByteChannel;
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ManyMessageTransfer extends AbstractReferenceCounted implements FileRegion {
|
||||
private final ByteBuffer byteBufferHeader;
|
||||
private final GetMessageResult getMessageResult;
|
||||
private long transfered; // the bytes which was transfered already
|
||||
|
||||
|
||||
public ManyMessageTransfer(ByteBuffer byteBufferHeader, GetMessageResult getMessageResult) {
|
||||
this.byteBufferHeader = byteBufferHeader;
|
||||
this.getMessageResult = getMessageResult;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public long position() {
|
||||
int pos = byteBufferHeader.position();
|
||||
List<ByteBuffer> messageBufferList = this.getMessageResult.getMessageBufferList();
|
||||
for (ByteBuffer bb : messageBufferList) {
|
||||
pos += bb.position();
|
||||
}
|
||||
return pos;
|
||||
}
|
||||
|
||||
@Override
|
||||
public long transfered() {
|
||||
return transfered;
|
||||
}
|
||||
|
||||
@Override
|
||||
public long count() {
|
||||
return byteBufferHeader.limit() + this.getMessageResult.getBufferTotalSize();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long transferTo(WritableByteChannel target, long position) throws IOException {
|
||||
if (this.byteBufferHeader.hasRemaining()) {
|
||||
transfered += target.write(this.byteBufferHeader);
|
||||
return transfered;
|
||||
} else {
|
||||
List<ByteBuffer> messageBufferList = this.getMessageResult.getMessageBufferList();
|
||||
for (ByteBuffer bb : messageBufferList) {
|
||||
if (bb.hasRemaining()) {
|
||||
transfered += target.write(bb);
|
||||
return transfered;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
public void close() {
|
||||
this.deallocate();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void deallocate() {
|
||||
this.getMessageResult.release();
|
||||
}
|
||||
}
|
||||
+79
@@ -0,0 +1,79 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.pagecache;
|
||||
|
||||
import com.alibaba.rocketmq.store.SelectMappedBufferResult;
|
||||
import io.netty.channel.FileRegion;
|
||||
import io.netty.util.AbstractReferenceCounted;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.ByteBuffer;
|
||||
import java.nio.channels.WritableByteChannel;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class OneMessageTransfer extends AbstractReferenceCounted implements FileRegion {
|
||||
private final ByteBuffer byteBufferHeader;
|
||||
private final SelectMappedBufferResult selectMappedBufferResult;
|
||||
private long transfered; // the bytes which was transfered already
|
||||
|
||||
|
||||
public OneMessageTransfer(ByteBuffer byteBufferHeader, SelectMappedBufferResult selectMappedBufferResult) {
|
||||
this.byteBufferHeader = byteBufferHeader;
|
||||
this.selectMappedBufferResult = selectMappedBufferResult;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public long position() {
|
||||
return this.byteBufferHeader.position() + this.selectMappedBufferResult.getByteBuffer().position();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long transfered() {
|
||||
return transfered;
|
||||
}
|
||||
|
||||
@Override
|
||||
public long count() {
|
||||
return this.byteBufferHeader.limit() + this.selectMappedBufferResult.getSize();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long transferTo(WritableByteChannel target, long position) throws IOException {
|
||||
if (this.byteBufferHeader.hasRemaining()) {
|
||||
transfered += target.write(this.byteBufferHeader);
|
||||
return transfered;
|
||||
} else if (this.selectMappedBufferResult.getByteBuffer().hasRemaining()) {
|
||||
transfered += target.write(this.selectMappedBufferResult.getByteBuffer());
|
||||
return transfered;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
public void close() {
|
||||
this.deallocate();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void deallocate() {
|
||||
this.selectMappedBufferResult.release();
|
||||
}
|
||||
}
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.pagecache;
|
||||
|
||||
import com.alibaba.rocketmq.store.QueryMessageResult;
|
||||
import io.netty.channel.FileRegion;
|
||||
import io.netty.util.AbstractReferenceCounted;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.ByteBuffer;
|
||||
import java.nio.channels.WritableByteChannel;
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class QueryMessageTransfer extends AbstractReferenceCounted implements FileRegion {
|
||||
private final ByteBuffer byteBufferHeader;
|
||||
private final QueryMessageResult queryMessageResult;
|
||||
private long transfered; // the bytes which was transfered already
|
||||
|
||||
|
||||
public QueryMessageTransfer(ByteBuffer byteBufferHeader, QueryMessageResult queryMessageResult) {
|
||||
this.byteBufferHeader = byteBufferHeader;
|
||||
this.queryMessageResult = queryMessageResult;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public long position() {
|
||||
int pos = byteBufferHeader.position();
|
||||
List<ByteBuffer> messageBufferList = this.queryMessageResult.getMessageBufferList();
|
||||
for (ByteBuffer bb : messageBufferList) {
|
||||
pos += bb.position();
|
||||
}
|
||||
return pos;
|
||||
}
|
||||
|
||||
@Override
|
||||
public long transfered() {
|
||||
return transfered;
|
||||
}
|
||||
|
||||
@Override
|
||||
public long count() {
|
||||
return byteBufferHeader.limit() + this.queryMessageResult.getBufferTotalSize();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long transferTo(WritableByteChannel target, long position) throws IOException {
|
||||
if (this.byteBufferHeader.hasRemaining()) {
|
||||
transfered += target.write(this.byteBufferHeader);
|
||||
return transfered;
|
||||
} else {
|
||||
List<ByteBuffer> messageBufferList = this.queryMessageResult.getMessageBufferList();
|
||||
for (ByteBuffer bb : messageBufferList) {
|
||||
if (bb.hasRemaining()) {
|
||||
transfered += target.write(bb);
|
||||
return transfered;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
public void close() {
|
||||
this.deallocate();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void deallocate() {
|
||||
this.queryMessageResult.release();
|
||||
}
|
||||
}
|
||||
+236
@@ -0,0 +1,236 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
/**
|
||||
*
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.plugin;
|
||||
|
||||
import com.alibaba.rocketmq.common.message.MessageExt;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.SubscriptionData;
|
||||
import com.alibaba.rocketmq.store.*;
|
||||
|
||||
import java.util.HashMap;
|
||||
import java.util.Set;
|
||||
|
||||
public abstract class AbstractPluginMessageStore implements MessageStore {
|
||||
protected MessageStore next = null;
|
||||
protected MessageStorePluginContext context;
|
||||
|
||||
public AbstractPluginMessageStore(MessageStorePluginContext context, MessageStore next) {
|
||||
this.next = next;
|
||||
this.context = context;
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getEarliestMessageTime() {
|
||||
return next.getEarliestMessageTime();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long lockTimeMills() {
|
||||
return next.lockTimeMills();
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean isOSPageCacheBusy() {
|
||||
return next.isOSPageCacheBusy();
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean isTransientStorePoolDeficient() {
|
||||
return next.isTransientStorePoolDeficient();
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean load() {
|
||||
return next.load();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void start() throws Exception {
|
||||
next.start();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void shutdown() {
|
||||
next.shutdown();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void destroy() {
|
||||
next.destroy();
|
||||
}
|
||||
|
||||
@Override
|
||||
public PutMessageResult putMessage(MessageExtBrokerInner msg) {
|
||||
return next.putMessage(msg);
|
||||
}
|
||||
|
||||
@Override
|
||||
public GetMessageResult getMessage(String group, String topic, int queueId, long offset,
|
||||
int maxMsgNums, SubscriptionData subscriptionData) {
|
||||
return next.getMessage(group, topic, queueId, offset, maxMsgNums, subscriptionData);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getMaxOffsetInQuque(String topic, int queueId) {
|
||||
return next.getMaxOffsetInQuque(topic, queueId);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getMinOffsetInQuque(String topic, int queueId) {
|
||||
return next.getMinOffsetInQuque(topic, queueId);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getCommitLogOffsetInQueue(String topic, int queueId, long cqOffset) {
|
||||
return next.getCommitLogOffsetInQueue(topic, queueId, cqOffset);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getOffsetInQueueByTime(String topic, int queueId, long timestamp) {
|
||||
return next.getOffsetInQueueByTime(topic, queueId, timestamp);
|
||||
}
|
||||
|
||||
@Override
|
||||
public MessageExt lookMessageByOffset(long commitLogOffset) {
|
||||
return next.lookMessageByOffset(commitLogOffset);
|
||||
}
|
||||
|
||||
@Override
|
||||
public SelectMappedBufferResult selectOneMessageByOffset(long commitLogOffset) {
|
||||
return next.selectOneMessageByOffset(commitLogOffset);
|
||||
}
|
||||
|
||||
@Override
|
||||
public SelectMappedBufferResult selectOneMessageByOffset(long commitLogOffset, int msgSize) {
|
||||
return next.selectOneMessageByOffset(commitLogOffset, msgSize);
|
||||
}
|
||||
|
||||
@Override
|
||||
public String getRunningDataInfo() {
|
||||
return next.getRunningDataInfo();
|
||||
}
|
||||
|
||||
@Override
|
||||
public HashMap<String, String> getRuntimeInfo() {
|
||||
return next.getRuntimeInfo();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getMaxPhyOffset() {
|
||||
return next.getMaxPhyOffset();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getMinPhyOffset() {
|
||||
return next.getMinPhyOffset();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getEarliestMessageTime(String topic, int queueId) {
|
||||
return next.getEarliestMessageTime(topic, queueId);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getMessageStoreTimeStamp(String topic, int queueId, long offset) {
|
||||
return next.getMessageStoreTimeStamp(topic, queueId, offset);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getMessageTotalInQueue(String topic, int queueId) {
|
||||
return next.getMessageTotalInQueue(topic, queueId);
|
||||
}
|
||||
|
||||
@Override
|
||||
public SelectMappedBufferResult getCommitLogData(long offset) {
|
||||
return next.getCommitLogData(offset);
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean appendToCommitLog(long startOffset, byte[] data) {
|
||||
return next.appendToCommitLog(startOffset, data);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void excuteDeleteFilesManualy() {
|
||||
next.excuteDeleteFilesManualy();
|
||||
}
|
||||
|
||||
@Override
|
||||
public QueryMessageResult queryMessage(String topic, String key, int maxNum, long begin,
|
||||
long end) {
|
||||
return next.queryMessage(topic, key, maxNum, begin, end);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void updateHaMasterAddress(String newAddr) {
|
||||
next.updateHaMasterAddress(newAddr);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long slaveFallBehindMuch() {
|
||||
return next.slaveFallBehindMuch();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long now() {
|
||||
return next.now();
|
||||
}
|
||||
|
||||
@Override
|
||||
public int cleanUnusedTopic(Set<String> topics) {
|
||||
return next.cleanUnusedTopic(topics);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void cleanExpiredConsumerQueue() {
|
||||
next.cleanExpiredConsumerQueue();
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean checkInDiskByConsumeOffset(String topic, int queueId, long consumeOffset) {
|
||||
return next.checkInDiskByConsumeOffset(topic, queueId, consumeOffset);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long dispatchBehindBytes() {
|
||||
return next.dispatchBehindBytes();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long flush() {
|
||||
return next.flush();
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean resetWriteOffset(long phyOffset) {
|
||||
return next.resetWriteOffset(phyOffset);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long getConfirmOffset() {
|
||||
return next.getConfirmOffset();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void setConfirmOffset(long phyOffset) {
|
||||
next.setConfirmOffset(phyOffset);
|
||||
}
|
||||
|
||||
}
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
/**
|
||||
*
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.plugin;
|
||||
|
||||
import com.alibaba.rocketmq.store.MessageStore;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.lang.reflect.Constructor;
|
||||
|
||||
public final class MessageStoreFactory {
|
||||
public final static MessageStore build(MessageStorePluginContext context, MessageStore messageStore)
|
||||
throws IOException {
|
||||
String plugin = context.getBrokerConfig().getMessageStorePlugIn();
|
||||
if (plugin != null && plugin.trim().length() != 0) {
|
||||
String[] pluginClasses = plugin.split(",");
|
||||
for (int i = pluginClasses.length - 1; i >= 0; --i) {
|
||||
String pluginClass = pluginClasses[i];
|
||||
try {
|
||||
@SuppressWarnings("unchecked")
|
||||
Class<AbstractPluginMessageStore> clazz = (Class<AbstractPluginMessageStore>) Class.forName(pluginClass);
|
||||
Constructor<AbstractPluginMessageStore> construct = clazz.getConstructor(MessageStorePluginContext.class, MessageStore.class);
|
||||
AbstractPluginMessageStore pluginMessageStore = (AbstractPluginMessageStore) construct.newInstance(context, messageStore);
|
||||
messageStore = pluginMessageStore;
|
||||
} catch (Throwable e) {
|
||||
throw new RuntimeException(String.format(
|
||||
"Initialize plugin's class %s not found!", pluginClass), e);
|
||||
}
|
||||
}
|
||||
}
|
||||
return messageStore;
|
||||
}
|
||||
}
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
/**
|
||||
*
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.plugin;
|
||||
|
||||
import com.alibaba.rocketmq.common.BrokerConfig;
|
||||
import com.alibaba.rocketmq.store.MessageArrivingListener;
|
||||
import com.alibaba.rocketmq.store.config.MessageStoreConfig;
|
||||
import com.alibaba.rocketmq.store.stats.BrokerStatsManager;
|
||||
|
||||
public class MessageStorePluginContext {
|
||||
private MessageStoreConfig messageStoreConfig;
|
||||
private BrokerStatsManager brokerStatsManager;
|
||||
private MessageArrivingListener messageArrivingListener;
|
||||
private BrokerConfig brokerConfig;
|
||||
|
||||
public MessageStorePluginContext(MessageStoreConfig messageStoreConfig,
|
||||
BrokerStatsManager brokerStatsManager, MessageArrivingListener messageArrivingListener,
|
||||
BrokerConfig brokerConfig) {
|
||||
super();
|
||||
this.messageStoreConfig = messageStoreConfig;
|
||||
this.brokerStatsManager = brokerStatsManager;
|
||||
this.messageArrivingListener = messageArrivingListener;
|
||||
this.brokerConfig = brokerConfig;
|
||||
}
|
||||
|
||||
public MessageStoreConfig getMessageStoreConfig() {
|
||||
return messageStoreConfig;
|
||||
}
|
||||
|
||||
public BrokerStatsManager getBrokerStatsManager() {
|
||||
return brokerStatsManager;
|
||||
}
|
||||
|
||||
public MessageArrivingListener getMessageArrivingListener() {
|
||||
return messageArrivingListener;
|
||||
}
|
||||
|
||||
public BrokerConfig getBrokerConfig() {
|
||||
return brokerConfig;
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
+332
@@ -0,0 +1,332 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.processor;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.mqtrace.SendMessageContext;
|
||||
import com.alibaba.rocketmq.broker.mqtrace.SendMessageHook;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.TopicConfig;
|
||||
import com.alibaba.rocketmq.common.TopicFilterType;
|
||||
import com.alibaba.rocketmq.common.constant.DBMsgConstants;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.constant.PermName;
|
||||
import com.alibaba.rocketmq.common.help.FAQUrl;
|
||||
import com.alibaba.rocketmq.common.message.MessageAccessor;
|
||||
import com.alibaba.rocketmq.common.message.MessageConst;
|
||||
import com.alibaba.rocketmq.common.message.MessageDecoder;
|
||||
import com.alibaba.rocketmq.common.protocol.RequestCode;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
import com.alibaba.rocketmq.common.protocol.header.SendMessageRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.SendMessageRequestHeaderV2;
|
||||
import com.alibaba.rocketmq.common.protocol.header.SendMessageResponseHeader;
|
||||
import com.alibaba.rocketmq.common.sysflag.MessageSysFlag;
|
||||
import com.alibaba.rocketmq.common.sysflag.TopicSysFlag;
|
||||
import com.alibaba.rocketmq.common.utils.ChannelUtil;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingHelper;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingCommandException;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyRequestProcessor;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import com.alibaba.rocketmq.store.MessageExtBrokerInner;
|
||||
import io.netty.channel.ChannelHandlerContext;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.net.InetSocketAddress;
|
||||
import java.net.SocketAddress;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Random;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public abstract class AbstractSendMessageProcessor implements NettyRequestProcessor {
|
||||
protected static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
|
||||
protected final static int DLQ_NUMS_PER_GROUP = 1;
|
||||
protected final BrokerController brokerController;
|
||||
protected final Random random = new Random(System.currentTimeMillis());
|
||||
protected final SocketAddress storeHost;
|
||||
private List<SendMessageHook> sendMessageHookList;
|
||||
|
||||
|
||||
public AbstractSendMessageProcessor(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
this.storeHost =
|
||||
new InetSocketAddress(brokerController.getBrokerConfig().getBrokerIP1(), brokerController
|
||||
.getNettyServerConfig().getListenPort());
|
||||
}
|
||||
|
||||
protected SendMessageContext buildMsgContext(ChannelHandlerContext ctx,
|
||||
SendMessageRequestHeader requestHeader) {
|
||||
if (!this.hasSendMessageHook()) {
|
||||
return null;
|
||||
}
|
||||
SendMessageContext mqtraceContext;
|
||||
mqtraceContext = new SendMessageContext();
|
||||
mqtraceContext.setProducerGroup(requestHeader.getProducerGroup());
|
||||
mqtraceContext.setTopic(requestHeader.getTopic());
|
||||
mqtraceContext.setMsgProps(requestHeader.getProperties());
|
||||
mqtraceContext.setBornHost(RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
|
||||
mqtraceContext.setBrokerAddr(this.brokerController.getBrokerAddr());
|
||||
mqtraceContext.setBrokerRegionId(this.brokerController.getBrokerConfig().getRegionId());
|
||||
mqtraceContext.setBornTimeStamp(requestHeader.getBornTimestamp());
|
||||
|
||||
Map<String, String> properties = MessageDecoder.string2messageProperties(requestHeader.getProperties());
|
||||
String uniqueKey = properties.get(MessageConst.PROPERTY_UNIQ_CLIENT_MESSAGE_ID_KEYIDX);
|
||||
properties.put(MessageConst.PROPERTY_MSG_REGION, this.brokerController.getBrokerConfig().getRegionId());
|
||||
properties.put(MessageConst.PROPERTY_TRACE_SWITCH, String.valueOf(this.brokerController.getBrokerConfig().isTraceOn()));
|
||||
requestHeader.setProperties(MessageDecoder.messageProperties2String(properties));
|
||||
|
||||
|
||||
if (uniqueKey == null) {
|
||||
uniqueKey = "";
|
||||
}
|
||||
mqtraceContext.setMsgUniqueKey(uniqueKey);
|
||||
return mqtraceContext;
|
||||
}
|
||||
|
||||
public boolean hasSendMessageHook() {
|
||||
return sendMessageHookList != null && !this.sendMessageHookList.isEmpty();
|
||||
}
|
||||
|
||||
protected MessageExtBrokerInner buildInnerMsg(final ChannelHandlerContext ctx,
|
||||
final SendMessageRequestHeader requestHeader, final byte[] body, TopicConfig topicConfig) {
|
||||
int queueIdInt = requestHeader.getQueueId();
|
||||
if (queueIdInt < 0) {
|
||||
queueIdInt = Math.abs(this.random.nextInt() % 99999999) % topicConfig.getWriteQueueNums();
|
||||
}
|
||||
int sysFlag = requestHeader.getSysFlag();
|
||||
|
||||
if (TopicFilterType.MULTI_TAG == topicConfig.getTopicFilterType()) {
|
||||
sysFlag |= MessageSysFlag.MULTI_TAGS_FLAG;
|
||||
}
|
||||
|
||||
MessageExtBrokerInner msgInner = new MessageExtBrokerInner();
|
||||
msgInner.setTopic(requestHeader.getTopic());
|
||||
msgInner.setBody(body);
|
||||
msgInner.setFlag(requestHeader.getFlag());
|
||||
MessageAccessor.setProperties(msgInner,
|
||||
MessageDecoder.string2messageProperties(requestHeader.getProperties()));
|
||||
msgInner.setPropertiesString(requestHeader.getProperties());
|
||||
msgInner.setTagsCode(MessageExtBrokerInner.tagsString2tagsCode(topicConfig.getTopicFilterType(),
|
||||
msgInner.getTags()));
|
||||
|
||||
msgInner.setQueueId(queueIdInt);
|
||||
msgInner.setSysFlag(sysFlag);
|
||||
msgInner.setBornTimestamp(requestHeader.getBornTimestamp());
|
||||
msgInner.setBornHost(ctx.channel().remoteAddress());
|
||||
msgInner.setStoreHost(this.getStoreHost());
|
||||
msgInner.setReconsumeTimes(requestHeader.getReconsumeTimes() == null ? 0 : requestHeader
|
||||
.getReconsumeTimes());
|
||||
return msgInner;
|
||||
}
|
||||
|
||||
public SocketAddress getStoreHost() {
|
||||
return storeHost;
|
||||
}
|
||||
|
||||
protected RemotingCommand msgContentCheck(final ChannelHandlerContext ctx,
|
||||
final SendMessageRequestHeader requestHeader, RemotingCommand request,
|
||||
final RemotingCommand response) {
|
||||
if (requestHeader.getTopic().length() > Byte.MAX_VALUE) {
|
||||
log.warn("putMessage message topic length too long " + requestHeader.getTopic().length());
|
||||
response.setCode(ResponseCode.MESSAGE_ILLEGAL);
|
||||
return response;
|
||||
}
|
||||
if (requestHeader.getProperties() != null && requestHeader.getProperties().length() > Short.MAX_VALUE) {
|
||||
log.warn("putMessage message properties length too long "
|
||||
+ requestHeader.getProperties().length());
|
||||
response.setCode(ResponseCode.MESSAGE_ILLEGAL);
|
||||
return response;
|
||||
}
|
||||
if (request.getBody().length > DBMsgConstants.MAX_BODY_SIZE) {
|
||||
log.warn(" topic {} msg body size {} from {}", requestHeader.getTopic(),
|
||||
request.getBody().length, ChannelUtil.getRemoteIp(ctx.channel()));
|
||||
response.setRemark("msg body must be less 64KB");
|
||||
response.setCode(ResponseCode.MESSAGE_ILLEGAL);
|
||||
return response;
|
||||
}
|
||||
return response;
|
||||
}
|
||||
|
||||
protected RemotingCommand msgCheck(final ChannelHandlerContext ctx,
|
||||
final SendMessageRequestHeader requestHeader, final RemotingCommand response) {
|
||||
if (!PermName.isWriteable(this.brokerController.getBrokerConfig().getBrokerPermission())
|
||||
&& this.brokerController.getTopicConfigManager().isOrderTopic(requestHeader.getTopic())) {
|
||||
response.setCode(ResponseCode.NO_PERMISSION);
|
||||
response.setRemark("the broker[" + this.brokerController.getBrokerConfig().getBrokerIP1()
|
||||
+ "] sending message is forbidden");
|
||||
return response;
|
||||
}
|
||||
if (!this.brokerController.getTopicConfigManager().isTopicCanSendMessage(requestHeader.getTopic())) {
|
||||
String errorMsg =
|
||||
"the topic[" + requestHeader.getTopic() + "] is conflict with system reserved words.";
|
||||
log.warn(errorMsg);
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark(errorMsg);
|
||||
return response;
|
||||
}
|
||||
|
||||
TopicConfig topicConfig =
|
||||
this.brokerController.getTopicConfigManager().selectTopicConfig(requestHeader.getTopic());
|
||||
if (null == topicConfig) {
|
||||
int topicSysFlag = 0;
|
||||
if (requestHeader.isUnitMode()) {
|
||||
if (requestHeader.getTopic().startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
|
||||
topicSysFlag = TopicSysFlag.buildSysFlag(false, true);
|
||||
} else {
|
||||
topicSysFlag = TopicSysFlag.buildSysFlag(true, false);
|
||||
}
|
||||
}
|
||||
|
||||
log.warn("the topic " + requestHeader.getTopic() + " not exist, producer: "
|
||||
+ ctx.channel().remoteAddress());
|
||||
topicConfig = this.brokerController.getTopicConfigManager().createTopicInSendMessageMethod(//
|
||||
requestHeader.getTopic(), //
|
||||
requestHeader.getDefaultTopic(), //
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel()), //
|
||||
requestHeader.getDefaultTopicQueueNums(), topicSysFlag);
|
||||
|
||||
if (null == topicConfig) {
|
||||
if (requestHeader.getTopic().startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
|
||||
topicConfig =
|
||||
this.brokerController.getTopicConfigManager().createTopicInSendMessageBackMethod(
|
||||
requestHeader.getTopic(), 1, PermName.PERM_WRITE | PermName.PERM_READ,
|
||||
topicSysFlag);
|
||||
}
|
||||
}
|
||||
|
||||
if (null == topicConfig) {
|
||||
response.setCode(ResponseCode.TOPIC_NOT_EXIST);
|
||||
response.setRemark("topic[" + requestHeader.getTopic() + "] not exist, apply first please!"
|
||||
+ FAQUrl.suggestTodo(FAQUrl.APPLY_TOPIC_URL));
|
||||
return response;
|
||||
}
|
||||
}
|
||||
|
||||
int queueIdInt = requestHeader.getQueueId();
|
||||
int idValid = Math.max(topicConfig.getWriteQueueNums(), topicConfig.getReadQueueNums());
|
||||
if (queueIdInt >= idValid) {
|
||||
String errorInfo = String.format("request queueId[%d] is illagal, %s Producer: %s",
|
||||
queueIdInt,
|
||||
topicConfig.toString(),
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
|
||||
|
||||
log.warn(errorInfo);
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark(errorInfo);
|
||||
|
||||
return response;
|
||||
}
|
||||
return response;
|
||||
}
|
||||
|
||||
public void registerSendMessageHook(List<SendMessageHook> sendMessageHookList) {
|
||||
this.sendMessageHookList = sendMessageHookList;
|
||||
}
|
||||
|
||||
protected void doResponse(ChannelHandlerContext ctx, RemotingCommand request,
|
||||
final RemotingCommand response) {
|
||||
if (!request.isOnewayRPC()) {
|
||||
try {
|
||||
ctx.writeAndFlush(response);
|
||||
} catch (Throwable e) {
|
||||
log.error("SendMessageProcessor process request over, but response failed", e);
|
||||
log.error(request.toString());
|
||||
log.error(response.toString());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public void executeSendMessageHookBefore(final ChannelHandlerContext ctx, final RemotingCommand request,
|
||||
SendMessageContext context) {
|
||||
if (hasSendMessageHook()) {
|
||||
for (SendMessageHook hook : this.sendMessageHookList) {
|
||||
try {
|
||||
final SendMessageRequestHeader requestHeader = parseRequestHeader(request);
|
||||
|
||||
if (null != requestHeader) {
|
||||
context.setProducerGroup(requestHeader.getProducerGroup());
|
||||
context.setTopic(requestHeader.getTopic());
|
||||
context.setBodyLength(request.getBody().length);
|
||||
context.setMsgProps(requestHeader.getProperties());
|
||||
context.setBornHost(RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
|
||||
context.setBrokerAddr(this.brokerController.getBrokerAddr());
|
||||
context.setQueueId(requestHeader.getQueueId());
|
||||
}
|
||||
|
||||
hook.sendMessageBefore(context);
|
||||
requestHeader.setProperties(context.getMsgProps());
|
||||
} catch (Throwable e) {
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
protected SendMessageRequestHeader parseRequestHeader(RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
|
||||
SendMessageRequestHeaderV2 requestHeaderV2 = null;
|
||||
SendMessageRequestHeader requestHeader = null;
|
||||
switch (request.getCode()) {
|
||||
case RequestCode.SEND_MESSAGE_V2:
|
||||
requestHeaderV2 =
|
||||
(SendMessageRequestHeaderV2) request
|
||||
.decodeCommandCustomHeader(SendMessageRequestHeaderV2.class);
|
||||
case RequestCode.SEND_MESSAGE:
|
||||
if (null == requestHeaderV2) {
|
||||
requestHeader =
|
||||
(SendMessageRequestHeader) request
|
||||
.decodeCommandCustomHeader(SendMessageRequestHeader.class);
|
||||
} else {
|
||||
requestHeader = SendMessageRequestHeaderV2.createSendMessageRequestHeaderV1(requestHeaderV2);
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
return requestHeader;
|
||||
}
|
||||
|
||||
public void executeSendMessageHookAfter(final RemotingCommand response, final SendMessageContext context) {
|
||||
if (hasSendMessageHook()) {
|
||||
for (SendMessageHook hook : this.sendMessageHookList) {
|
||||
try {
|
||||
if (response != null) {
|
||||
final SendMessageResponseHeader responseHeader =
|
||||
(SendMessageResponseHeader) response.readCustomHeader();
|
||||
context.setMsgId(responseHeader.getMsgId());
|
||||
context.setQueueId(responseHeader.getQueueId());
|
||||
context.setQueueOffset(responseHeader.getQueueOffset());
|
||||
context.setCode(response.getCode());
|
||||
context.setErrorMsg(response.getRemark());
|
||||
}
|
||||
hook.sendMessageAfter(context);
|
||||
} catch (Throwable e) {
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean rejectRequest() {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
+1212
File diff suppressed because it is too large
Load Diff
+164
@@ -0,0 +1,164 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.processor;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.client.ClientChannelInfo;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.constant.PermName;
|
||||
import com.alibaba.rocketmq.common.protocol.RequestCode;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
import com.alibaba.rocketmq.common.protocol.header.UnregisterClientRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.UnregisterClientResponseHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.ConsumerData;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.HeartbeatData;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.ProducerData;
|
||||
import com.alibaba.rocketmq.common.subscription.SubscriptionGroupConfig;
|
||||
import com.alibaba.rocketmq.common.sysflag.TopicSysFlag;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingHelper;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingCommandException;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyRequestProcessor;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import io.netty.channel.ChannelHandlerContext;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ClientManageProcessor implements NettyRequestProcessor {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private final BrokerController brokerController;
|
||||
|
||||
public ClientManageProcessor(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
@Override
|
||||
public RemotingCommand processRequest(ChannelHandlerContext ctx, RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
switch (request.getCode()) {
|
||||
case RequestCode.HEART_BEAT:
|
||||
return this.heartBeat(ctx, request);
|
||||
case RequestCode.UNREGISTER_CLIENT:
|
||||
return this.unregisterClient(ctx, request);
|
||||
default:
|
||||
break;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean rejectRequest() {
|
||||
return false;
|
||||
}
|
||||
|
||||
public RemotingCommand heartBeat(ChannelHandlerContext ctx, RemotingCommand request) {
|
||||
RemotingCommand response = RemotingCommand.createResponseCommand(null);
|
||||
HeartbeatData heartbeatData = HeartbeatData.decode(request.getBody(), HeartbeatData.class);
|
||||
ClientChannelInfo clientChannelInfo = new ClientChannelInfo(
|
||||
ctx.channel(),
|
||||
heartbeatData.getClientID(),
|
||||
request.getLanguage(),
|
||||
request.getVersion()
|
||||
);
|
||||
|
||||
for (ConsumerData data : heartbeatData.getConsumerDataSet()) {
|
||||
SubscriptionGroupConfig subscriptionGroupConfig =
|
||||
this.brokerController.getSubscriptionGroupManager().findSubscriptionGroupConfig(
|
||||
data.getGroupName());
|
||||
boolean isNotifyConsumerIdsChangedEnable = true;
|
||||
if (null != subscriptionGroupConfig) {
|
||||
isNotifyConsumerIdsChangedEnable = subscriptionGroupConfig.isNotifyConsumerIdsChangedEnable();
|
||||
int topicSysFlag = 0;
|
||||
if (data.isUnitMode()) {
|
||||
topicSysFlag = TopicSysFlag.buildSysFlag(false, true);
|
||||
}
|
||||
String newTopic = MixAll.getRetryTopic(data.getGroupName());
|
||||
this.brokerController.getTopicConfigManager().createTopicInSendMessageBackMethod(
|
||||
newTopic,
|
||||
subscriptionGroupConfig.getRetryQueueNums(),
|
||||
PermName.PERM_WRITE | PermName.PERM_READ, topicSysFlag);
|
||||
}
|
||||
|
||||
boolean changed = this.brokerController.getConsumerManager().registerConsumer(
|
||||
data.getGroupName(),
|
||||
clientChannelInfo,
|
||||
data.getConsumeType(),
|
||||
data.getMessageModel(),
|
||||
data.getConsumeFromWhere(),
|
||||
data.getSubscriptionDataSet(),
|
||||
isNotifyConsumerIdsChangedEnable
|
||||
);
|
||||
|
||||
if (changed) {
|
||||
log.info("registerConsumer info changed {} {}",
|
||||
data.toString(),
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel())
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (ProducerData data : heartbeatData.getProducerDataSet()) {
|
||||
this.brokerController.getProducerManager().registerProducer(data.getGroupName(),
|
||||
clientChannelInfo);
|
||||
}
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
return response;
|
||||
}
|
||||
|
||||
public RemotingCommand unregisterClient(ChannelHandlerContext ctx, RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
final RemotingCommand response =
|
||||
RemotingCommand.createResponseCommand(UnregisterClientResponseHeader.class);
|
||||
final UnregisterClientRequestHeader requestHeader =
|
||||
(UnregisterClientRequestHeader) request
|
||||
.decodeCommandCustomHeader(UnregisterClientRequestHeader.class);
|
||||
|
||||
ClientChannelInfo clientChannelInfo = new ClientChannelInfo(
|
||||
ctx.channel(),
|
||||
requestHeader.getClientID(),
|
||||
request.getLanguage(),
|
||||
request.getVersion());
|
||||
{
|
||||
final String group = requestHeader.getProducerGroup();
|
||||
if (group != null) {
|
||||
this.brokerController.getProducerManager().unregisterProducer(group, clientChannelInfo);
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
final String group = requestHeader.getConsumerGroup();
|
||||
if (group != null) {
|
||||
SubscriptionGroupConfig subscriptionGroupConfig =
|
||||
this.brokerController.getSubscriptionGroupManager().findSubscriptionGroupConfig(group);
|
||||
boolean isNotifyConsumerIdsChangedEnable = true;
|
||||
if (null != subscriptionGroupConfig) {
|
||||
isNotifyConsumerIdsChangedEnable = subscriptionGroupConfig.isNotifyConsumerIdsChangedEnable();
|
||||
}
|
||||
this.brokerController.getConsumerManager().unregisterConsumer(group, clientChannelInfo, isNotifyConsumerIdsChangedEnable);
|
||||
}
|
||||
}
|
||||
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
return response;
|
||||
}
|
||||
}
|
||||
+157
@@ -0,0 +1,157 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.processor;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.client.ConsumerGroupInfo;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.protocol.RequestCode;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
import com.alibaba.rocketmq.common.protocol.header.*;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingHelper;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingCommandException;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyRequestProcessor;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import io.netty.channel.ChannelHandlerContext;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ConsumerManageProcessor implements NettyRequestProcessor {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
|
||||
private final BrokerController brokerController;
|
||||
|
||||
|
||||
public ConsumerManageProcessor(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
@Override
|
||||
public RemotingCommand processRequest(ChannelHandlerContext ctx, RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
switch (request.getCode()) {
|
||||
case RequestCode.GET_CONSUMER_LIST_BY_GROUP:
|
||||
return this.getConsumerListByGroup(ctx, request);
|
||||
case RequestCode.UPDATE_CONSUMER_OFFSET:
|
||||
return this.updateConsumerOffset(ctx, request);
|
||||
case RequestCode.QUERY_CONSUMER_OFFSET:
|
||||
return this.queryConsumerOffset(ctx, request);
|
||||
default:
|
||||
break;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean rejectRequest() {
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
public RemotingCommand getConsumerListByGroup(ChannelHandlerContext ctx, RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
final RemotingCommand response =
|
||||
RemotingCommand.createResponseCommand(GetConsumerListByGroupResponseHeader.class);
|
||||
final GetConsumerListByGroupRequestHeader requestHeader =
|
||||
(GetConsumerListByGroupRequestHeader) request
|
||||
.decodeCommandCustomHeader(GetConsumerListByGroupRequestHeader.class);
|
||||
|
||||
ConsumerGroupInfo consumerGroupInfo =
|
||||
this.brokerController.getConsumerManager().getConsumerGroupInfo(
|
||||
requestHeader.getConsumerGroup());
|
||||
if (consumerGroupInfo != null) {
|
||||
List<String> clientIds = consumerGroupInfo.getAllClientId();
|
||||
if (!clientIds.isEmpty()) {
|
||||
GetConsumerListByGroupResponseBody body = new GetConsumerListByGroupResponseBody();
|
||||
body.setConsumerIdList(clientIds);
|
||||
response.setBody(body.encode());
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
return response;
|
||||
} else {
|
||||
log.warn("getAllClientId failed, {} {}", requestHeader.getConsumerGroup(),
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
|
||||
}
|
||||
} else {
|
||||
log.warn("getConsumerGroupInfo failed, {} {}", requestHeader.getConsumerGroup(),
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
|
||||
}
|
||||
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("no consumer for this group, " + requestHeader.getConsumerGroup());
|
||||
return response;
|
||||
}
|
||||
|
||||
private RemotingCommand updateConsumerOffset(ChannelHandlerContext ctx, RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
final RemotingCommand response =
|
||||
RemotingCommand.createResponseCommand(UpdateConsumerOffsetResponseHeader.class);
|
||||
final UpdateConsumerOffsetRequestHeader requestHeader =
|
||||
(UpdateConsumerOffsetRequestHeader) request
|
||||
.decodeCommandCustomHeader(UpdateConsumerOffsetRequestHeader.class);
|
||||
this.brokerController.getConsumerOffsetManager().commitOffset(RemotingHelper.parseChannelRemoteAddr(ctx.channel()), requestHeader.getConsumerGroup(),
|
||||
requestHeader.getTopic(), requestHeader.getQueueId(), requestHeader.getCommitOffset());
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
private RemotingCommand queryConsumerOffset(ChannelHandlerContext ctx, RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
final RemotingCommand response =
|
||||
RemotingCommand.createResponseCommand(QueryConsumerOffsetResponseHeader.class);
|
||||
final QueryConsumerOffsetResponseHeader responseHeader =
|
||||
(QueryConsumerOffsetResponseHeader) response.readCustomHeader();
|
||||
final QueryConsumerOffsetRequestHeader requestHeader =
|
||||
(QueryConsumerOffsetRequestHeader) request
|
||||
.decodeCommandCustomHeader(QueryConsumerOffsetRequestHeader.class);
|
||||
|
||||
long offset =
|
||||
this.brokerController.getConsumerOffsetManager().queryOffset(
|
||||
requestHeader.getConsumerGroup(), requestHeader.getTopic(), requestHeader.getQueueId());
|
||||
|
||||
|
||||
if (offset >= 0) {
|
||||
responseHeader.setOffset(offset);
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
} else {
|
||||
long minOffset =
|
||||
this.brokerController.getMessageStore().getMinOffsetInQuque(requestHeader.getTopic(),
|
||||
requestHeader.getQueueId());
|
||||
if (minOffset <= 0
|
||||
&& !this.brokerController.getMessageStore().checkInDiskByConsumeOffset(
|
||||
requestHeader.getTopic(), requestHeader.getQueueId(), 0)) {
|
||||
responseHeader.setOffset(0L);
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
} else {
|
||||
response.setCode(ResponseCode.QUERY_NOT_FOUND);
|
||||
response.setRemark("Not found, V3_0_6_SNAPSHOT maybe this group consumer boot first");
|
||||
}
|
||||
}
|
||||
|
||||
return response;
|
||||
}
|
||||
}
|
||||
+236
@@ -0,0 +1,236 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.processor;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.common.TopicFilterType;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.message.MessageAccessor;
|
||||
import com.alibaba.rocketmq.common.message.MessageConst;
|
||||
import com.alibaba.rocketmq.common.message.MessageDecoder;
|
||||
import com.alibaba.rocketmq.common.message.MessageExt;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
import com.alibaba.rocketmq.common.protocol.header.EndTransactionRequestHeader;
|
||||
import com.alibaba.rocketmq.common.sysflag.MessageSysFlag;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingHelper;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingCommandException;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyRequestProcessor;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import com.alibaba.rocketmq.store.MessageExtBrokerInner;
|
||||
import com.alibaba.rocketmq.store.MessageStore;
|
||||
import com.alibaba.rocketmq.store.PutMessageResult;
|
||||
import io.netty.channel.ChannelHandlerContext;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class EndTransactionProcessor implements NettyRequestProcessor {
|
||||
private static final Logger LOGGER = LoggerFactory.getLogger(LoggerName.TRANSACTION_LOGGER_NAME);
|
||||
private final BrokerController brokerController;
|
||||
|
||||
public EndTransactionProcessor(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
@Override
|
||||
public RemotingCommand processRequest(ChannelHandlerContext ctx, RemotingCommand request) throws RemotingCommandException {
|
||||
final RemotingCommand response = RemotingCommand.createResponseCommand(null);
|
||||
final EndTransactionRequestHeader requestHeader =
|
||||
(EndTransactionRequestHeader) request.decodeCommandCustomHeader(EndTransactionRequestHeader.class);
|
||||
|
||||
|
||||
if (requestHeader.getFromTransactionCheck()) {
|
||||
switch (requestHeader.getCommitOrRollback()) {
|
||||
case MessageSysFlag.TRANSACTION_NOT_TYPE: {
|
||||
LOGGER.warn("check producer[{}] transaction state, but it's pending status."
|
||||
+ "RequestHeader: {} Remark: {}",
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel()),
|
||||
requestHeader.toString(),
|
||||
request.getRemark());
|
||||
return null;
|
||||
}
|
||||
|
||||
case MessageSysFlag.TRANSACTION_COMMIT_TYPE: {
|
||||
LOGGER.warn("check producer[{}] transaction state, the producer commit the message."
|
||||
+ "RequestHeader: {} Remark: {}",
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel()),
|
||||
requestHeader.toString(),
|
||||
request.getRemark());
|
||||
|
||||
break;
|
||||
}
|
||||
|
||||
case MessageSysFlag.TRANSACTION_ROLLBACK_TYPE: {
|
||||
LOGGER.warn("check producer[{}] transaction state, the producer rollback the message."
|
||||
+ "RequestHeader: {} Remark: {}",
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel()),
|
||||
requestHeader.toString(),
|
||||
request.getRemark());
|
||||
break;
|
||||
}
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
} else {
|
||||
switch (requestHeader.getCommitOrRollback()) {
|
||||
case MessageSysFlag.TRANSACTION_NOT_TYPE: {
|
||||
LOGGER.warn("the producer[{}] end transaction in sending message, and it's pending status."
|
||||
+ "RequestHeader: {} Remark: {}",
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel()),
|
||||
requestHeader.toString(),
|
||||
request.getRemark());
|
||||
return null;
|
||||
}
|
||||
|
||||
case MessageSysFlag.TRANSACTION_COMMIT_TYPE: {
|
||||
break;
|
||||
}
|
||||
|
||||
case MessageSysFlag.TRANSACTION_ROLLBACK_TYPE: {
|
||||
LOGGER.warn("the producer[{}] end transaction in sending message, rollback the message."
|
||||
+ "RequestHeader: {} Remark: {}",
|
||||
RemotingHelper.parseChannelRemoteAddr(ctx.channel()),
|
||||
requestHeader.toString(),
|
||||
request.getRemark());
|
||||
break;
|
||||
}
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
final MessageExt msgExt = this.brokerController.getMessageStore().lookMessageByOffset(requestHeader.getCommitLogOffset());
|
||||
if (msgExt != null) {
|
||||
final String pgroupRead = msgExt.getProperty(MessageConst.PROPERTY_PRODUCER_GROUP);
|
||||
if (!pgroupRead.equals(requestHeader.getProducerGroup())) {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("the producer group wrong");
|
||||
return response;
|
||||
}
|
||||
|
||||
if (msgExt.getQueueOffset() != requestHeader.getTranStateTableOffset()) {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("the transaction state table offset wrong");
|
||||
return response;
|
||||
}
|
||||
|
||||
if (msgExt.getCommitLogOffset() != requestHeader.getCommitLogOffset()) {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("the commit log offset wrong");
|
||||
return response;
|
||||
}
|
||||
|
||||
MessageExtBrokerInner msgInner = this.endMessageTransaction(msgExt);
|
||||
msgInner.setSysFlag(MessageSysFlag.resetTransactionValue(msgInner.getSysFlag(), requestHeader.getCommitOrRollback()));
|
||||
|
||||
msgInner.setQueueOffset(requestHeader.getTranStateTableOffset());
|
||||
msgInner.setPreparedTransactionOffset(requestHeader.getCommitLogOffset());
|
||||
msgInner.setStoreTimestamp(msgExt.getStoreTimestamp());
|
||||
if (MessageSysFlag.TRANSACTION_ROLLBACK_TYPE == requestHeader.getCommitOrRollback()) {
|
||||
msgInner.setBody(null);
|
||||
}
|
||||
|
||||
final MessageStore messageStore = this.brokerController.getMessageStore();
|
||||
final PutMessageResult putMessageResult = messageStore.putMessage(msgInner);
|
||||
if (putMessageResult != null) {
|
||||
switch (putMessageResult.getPutMessageStatus()) {
|
||||
// Success
|
||||
case PUT_OK:
|
||||
case FLUSH_DISK_TIMEOUT:
|
||||
case FLUSH_SLAVE_TIMEOUT:
|
||||
case SLAVE_NOT_AVAILABLE:
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
break;
|
||||
// Failed
|
||||
case CREATE_MAPEDFILE_FAILED:
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("create maped file failed.");
|
||||
break;
|
||||
case MESSAGE_ILLEGAL:
|
||||
case PROPERTIES_SIZE_EXCEEDED:
|
||||
response.setCode(ResponseCode.MESSAGE_ILLEGAL);
|
||||
response.setRemark("the message is illegal, maybe msg body or properties length not matched. msg body length limit 128k, msg properties length limit 32k.");
|
||||
break;
|
||||
case SERVICE_NOT_AVAILABLE:
|
||||
response.setCode(ResponseCode.SERVICE_NOT_AVAILABLE);
|
||||
response.setRemark("service not available now.");
|
||||
break;
|
||||
case OS_PAGECACHE_BUSY:
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("OS page cache busy, please try another machine");
|
||||
break;
|
||||
case UNKNOWN_ERROR:
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("UNKNOWN_ERROR");
|
||||
break;
|
||||
default:
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("UNKNOWN_ERROR DEFAULT");
|
||||
break;
|
||||
}
|
||||
|
||||
return response;
|
||||
} else {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("store putMessage return null");
|
||||
}
|
||||
} else {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("find prepared transaction message failed");
|
||||
return response;
|
||||
}
|
||||
|
||||
return response;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean rejectRequest() {
|
||||
return false;
|
||||
}
|
||||
|
||||
private MessageExtBrokerInner endMessageTransaction(MessageExt msgExt) {
|
||||
MessageExtBrokerInner msgInner = new MessageExtBrokerInner();
|
||||
msgInner.setBody(msgExt.getBody());
|
||||
msgInner.setFlag(msgExt.getFlag());
|
||||
MessageAccessor.setProperties(msgInner, msgExt.getProperties());
|
||||
|
||||
TopicFilterType topicFilterType =
|
||||
(msgInner.getSysFlag() & MessageSysFlag.MULTI_TAGS_FLAG) == MessageSysFlag.MULTI_TAGS_FLAG ? TopicFilterType.MULTI_TAG
|
||||
: TopicFilterType.SINGLE_TAG;
|
||||
long tagsCodeValue = MessageExtBrokerInner.tagsString2tagsCode(topicFilterType, msgInner.getTags());
|
||||
msgInner.setTagsCode(tagsCodeValue);
|
||||
msgInner.setPropertiesString(MessageDecoder.messageProperties2String(msgExt.getProperties()));
|
||||
|
||||
msgInner.setSysFlag(msgExt.getSysFlag());
|
||||
msgInner.setBornTimestamp(msgExt.getBornTimestamp());
|
||||
msgInner.setBornHost(msgExt.getBornHost());
|
||||
msgInner.setStoreHost(msgExt.getStoreHost());
|
||||
msgInner.setReconsumeTimes(msgExt.getReconsumeTimes());
|
||||
|
||||
msgInner.setWaitStoreMsgOK(false);
|
||||
MessageAccessor.clearProperty(msgInner, MessageConst.PROPERTY_DELAY_TIME_LEVEL);
|
||||
|
||||
msgInner.setTopic(msgExt.getTopic());
|
||||
msgInner.setQueueId(msgExt.getQueueId());
|
||||
|
||||
return msgInner;
|
||||
}
|
||||
}
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.processor;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyRequestProcessor;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import io.netty.channel.ChannelHandlerContext;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ForwardRequestProcessor implements NettyRequestProcessor {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
|
||||
private final BrokerController brokerController;
|
||||
|
||||
|
||||
public ForwardRequestProcessor(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public RemotingCommand processRequest(ChannelHandlerContext ctx, RemotingCommand request) {
|
||||
return null;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean rejectRequest() {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
+540
@@ -0,0 +1,540 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.processor;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.client.ConsumerGroupInfo;
|
||||
import com.alibaba.rocketmq.broker.longpolling.PullRequest;
|
||||
import com.alibaba.rocketmq.broker.mqtrace.ConsumeMessageContext;
|
||||
import com.alibaba.rocketmq.broker.mqtrace.ConsumeMessageHook;
|
||||
import com.alibaba.rocketmq.broker.pagecache.ManyMessageTransfer;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.TopicConfig;
|
||||
import com.alibaba.rocketmq.common.TopicFilterType;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.constant.PermName;
|
||||
import com.alibaba.rocketmq.common.filter.FilterAPI;
|
||||
import com.alibaba.rocketmq.common.help.FAQUrl;
|
||||
import com.alibaba.rocketmq.common.message.MessageDecoder;
|
||||
import com.alibaba.rocketmq.common.message.MessageQueue;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
import com.alibaba.rocketmq.common.protocol.header.PullMessageRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.PullMessageResponseHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.MessageModel;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.SubscriptionData;
|
||||
import com.alibaba.rocketmq.common.protocol.topic.OffsetMovedEvent;
|
||||
import com.alibaba.rocketmq.common.subscription.SubscriptionGroupConfig;
|
||||
import com.alibaba.rocketmq.common.sysflag.PullSysFlag;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingHelper;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingUtil;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingCommandException;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyRequestProcessor;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import com.alibaba.rocketmq.store.GetMessageResult;
|
||||
import com.alibaba.rocketmq.store.MessageExtBrokerInner;
|
||||
import com.alibaba.rocketmq.store.PutMessageResult;
|
||||
import com.alibaba.rocketmq.store.config.BrokerRole;
|
||||
import com.alibaba.rocketmq.store.stats.BrokerStatsManager;
|
||||
import io.netty.channel.*;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.nio.ByteBuffer;
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class PullMessageProcessor implements NettyRequestProcessor {
|
||||
private static final Logger LOG = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private final BrokerController brokerController;
|
||||
private List<ConsumeMessageHook> consumeMessageHookList;
|
||||
|
||||
public PullMessageProcessor(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
@Override
|
||||
public RemotingCommand processRequest(final ChannelHandlerContext ctx, RemotingCommand request) throws RemotingCommandException {
|
||||
return this.processRequest(ctx.channel(), request, true);
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean rejectRequest() {
|
||||
return false;
|
||||
}
|
||||
|
||||
private RemotingCommand processRequest(final Channel channel, RemotingCommand request, boolean brokerAllowSuspend)
|
||||
throws RemotingCommandException {
|
||||
RemotingCommand response = RemotingCommand.createResponseCommand(PullMessageResponseHeader.class);
|
||||
final PullMessageResponseHeader responseHeader = (PullMessageResponseHeader) response.readCustomHeader();
|
||||
final PullMessageRequestHeader requestHeader =
|
||||
(PullMessageRequestHeader) request.decodeCommandCustomHeader(PullMessageRequestHeader.class);
|
||||
|
||||
|
||||
response.setOpaque(request.getOpaque());
|
||||
|
||||
if (LOG.isDebugEnabled()) {
|
||||
LOG.debug("receive PullMessage request command, " + request);
|
||||
}
|
||||
|
||||
|
||||
if (!PermName.isReadable(this.brokerController.getBrokerConfig().getBrokerPermission())) {
|
||||
response.setCode(ResponseCode.NO_PERMISSION);
|
||||
response.setRemark("the broker[" + this.brokerController.getBrokerConfig().getBrokerIP1() + "] pulling message is forbidden");
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
SubscriptionGroupConfig subscriptionGroupConfig =
|
||||
this.brokerController.getSubscriptionGroupManager().findSubscriptionGroupConfig(requestHeader.getConsumerGroup());
|
||||
if (null == subscriptionGroupConfig) {
|
||||
response.setCode(ResponseCode.SUBSCRIPTION_GROUP_NOT_EXIST);
|
||||
response.setRemark("subscription group not exist, " + requestHeader.getConsumerGroup() + " "
|
||||
+ FAQUrl.suggestTodo(FAQUrl.SUBSCRIPTION_GROUP_NOT_EXIST));
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
if (!subscriptionGroupConfig.isConsumeEnable()) {
|
||||
response.setCode(ResponseCode.NO_PERMISSION);
|
||||
response.setRemark("subscription group no permission, " + requestHeader.getConsumerGroup());
|
||||
return response;
|
||||
}
|
||||
|
||||
final boolean hasSuspendFlag = PullSysFlag.hasSuspendFlag(requestHeader.getSysFlag());
|
||||
final boolean hasCommitOffsetFlag = PullSysFlag.hasCommitOffsetFlag(requestHeader.getSysFlag());
|
||||
final boolean hasSubscriptionFlag = PullSysFlag.hasSubscriptionFlag(requestHeader.getSysFlag());
|
||||
|
||||
final long suspendTimeoutMillisLong = hasSuspendFlag ? requestHeader.getSuspendTimeoutMillis() : 0;
|
||||
|
||||
|
||||
TopicConfig topicConfig = this.brokerController.getTopicConfigManager().selectTopicConfig(requestHeader.getTopic());
|
||||
if (null == topicConfig) {
|
||||
LOG.error("the topic " + requestHeader.getTopic() + " not exist, consumer: " + RemotingHelper.parseChannelRemoteAddr(channel));
|
||||
response.setCode(ResponseCode.TOPIC_NOT_EXIST);
|
||||
response.setRemark(
|
||||
"topic[" + requestHeader.getTopic() + "] not exist, apply first please!" + FAQUrl.suggestTodo(FAQUrl.APPLY_TOPIC_URL));
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
if (!PermName.isReadable(topicConfig.getPerm())) {
|
||||
response.setCode(ResponseCode.NO_PERMISSION);
|
||||
response.setRemark("the topic[" + requestHeader.getTopic() + "] pulling message is forbidden");
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
if (requestHeader.getQueueId() < 0 || requestHeader.getQueueId() >= topicConfig.getReadQueueNums()) {
|
||||
String errorInfo = "queueId[" + requestHeader.getQueueId() + "] is illagal,Topic :" + requestHeader.getTopic()
|
||||
+ " topicConfig.readQueueNums: " + topicConfig.getReadQueueNums() + " consumer: " + channel.remoteAddress();
|
||||
LOG.warn(errorInfo);
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark(errorInfo);
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
SubscriptionData subscriptionData = null;
|
||||
if (hasSubscriptionFlag) {
|
||||
try {
|
||||
subscriptionData = FilterAPI.buildSubscriptionData(requestHeader.getConsumerGroup(), requestHeader.getTopic(),
|
||||
requestHeader.getSubscription());
|
||||
} catch (Exception e) {
|
||||
LOG.warn("parse the consumer's subscription[{}] failed, group: {}", requestHeader.getSubscription(), //
|
||||
requestHeader.getConsumerGroup());
|
||||
response.setCode(ResponseCode.SUBSCRIPTION_PARSE_FAILED);
|
||||
response.setRemark("parse the consumer's subscription failed");
|
||||
return response;
|
||||
}
|
||||
} else {
|
||||
ConsumerGroupInfo consumerGroupInfo =
|
||||
this.brokerController.getConsumerManager().getConsumerGroupInfo(requestHeader.getConsumerGroup());
|
||||
if (null == consumerGroupInfo) {
|
||||
LOG.warn("the consumer's group info not exist, group: {}", requestHeader.getConsumerGroup());
|
||||
response.setCode(ResponseCode.SUBSCRIPTION_NOT_EXIST);
|
||||
response.setRemark("the consumer's group info not exist" + FAQUrl.suggestTodo(FAQUrl.SAME_GROUP_DIFFERENT_TOPIC));
|
||||
return response;
|
||||
}
|
||||
|
||||
if (!subscriptionGroupConfig.isConsumeBroadcastEnable() //
|
||||
&& consumerGroupInfo.getMessageModel() == MessageModel.BROADCASTING) {
|
||||
response.setCode(ResponseCode.NO_PERMISSION);
|
||||
response.setRemark("the consumer group[" + requestHeader.getConsumerGroup() + "] can not consume by broadcast way");
|
||||
return response;
|
||||
}
|
||||
|
||||
subscriptionData = consumerGroupInfo.findSubscriptionData(requestHeader.getTopic());
|
||||
if (null == subscriptionData) {
|
||||
LOG.warn("the consumer's subscription not exist, group: {}, topic:{}", requestHeader.getConsumerGroup(), requestHeader.getTopic());
|
||||
response.setCode(ResponseCode.SUBSCRIPTION_NOT_EXIST);
|
||||
response.setRemark("the consumer's subscription not exist" + FAQUrl.suggestTodo(FAQUrl.SAME_GROUP_DIFFERENT_TOPIC));
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
if (subscriptionData.getSubVersion() < requestHeader.getSubVersion()) {
|
||||
LOG.warn("the broker's subscription is not latest, group: {} {}", requestHeader.getConsumerGroup(),
|
||||
subscriptionData.getSubString());
|
||||
response.setCode(ResponseCode.SUBSCRIPTION_NOT_LATEST);
|
||||
response.setRemark("the consumer's subscription not latest");
|
||||
return response;
|
||||
}
|
||||
}
|
||||
|
||||
final GetMessageResult getMessageResult =
|
||||
this.brokerController.getMessageStore().getMessage(requestHeader.getConsumerGroup(), requestHeader.getTopic(),
|
||||
requestHeader.getQueueId(), requestHeader.getQueueOffset(), requestHeader.getMaxMsgNums(), subscriptionData);
|
||||
if (getMessageResult != null) {
|
||||
response.setRemark(getMessageResult.getStatus().name());
|
||||
responseHeader.setNextBeginOffset(getMessageResult.getNextBeginOffset());
|
||||
responseHeader.setMinOffset(getMessageResult.getMinOffset());
|
||||
responseHeader.setMaxOffset(getMessageResult.getMaxOffset());
|
||||
|
||||
|
||||
if (getMessageResult.isSuggestPullingFromSlave()) {
|
||||
responseHeader.setSuggestWhichBrokerId(subscriptionGroupConfig.getWhichBrokerWhenConsumeSlowly());
|
||||
} else {
|
||||
responseHeader.setSuggestWhichBrokerId(MixAll.MASTER_ID);
|
||||
}
|
||||
|
||||
switch (this.brokerController.getMessageStoreConfig().getBrokerRole()) {
|
||||
case ASYNC_MASTER:
|
||||
case SYNC_MASTER:
|
||||
break;
|
||||
case SLAVE:
|
||||
if (!this.brokerController.getBrokerConfig().isSlaveReadEnable()) {
|
||||
response.setCode(ResponseCode.PULL_RETRY_IMMEDIATELY);
|
||||
responseHeader.setSuggestWhichBrokerId(MixAll.MASTER_ID);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
if (this.brokerController.getBrokerConfig().isSlaveReadEnable()) {
|
||||
// consume too slow ,redirect to another machine
|
||||
if (getMessageResult.isSuggestPullingFromSlave()) {
|
||||
responseHeader.setSuggestWhichBrokerId(subscriptionGroupConfig.getWhichBrokerWhenConsumeSlowly());
|
||||
}
|
||||
// consume ok
|
||||
else {
|
||||
responseHeader.setSuggestWhichBrokerId(subscriptionGroupConfig.getBrokerId());
|
||||
}
|
||||
} else {
|
||||
responseHeader.setSuggestWhichBrokerId(MixAll.MASTER_ID);
|
||||
}
|
||||
|
||||
switch (getMessageResult.getStatus()) {
|
||||
case FOUND:
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
break;
|
||||
case MESSAGE_WAS_REMOVING:
|
||||
response.setCode(ResponseCode.PULL_RETRY_IMMEDIATELY);
|
||||
break;
|
||||
case NO_MATCHED_LOGIC_QUEUE:
|
||||
case NO_MESSAGE_IN_QUEUE:
|
||||
if (0 != requestHeader.getQueueOffset()) {
|
||||
response.setCode(ResponseCode.PULL_OFFSET_MOVED);
|
||||
|
||||
// XXX: warn and notify me
|
||||
LOG.info("the broker store no queue data, fix the request offset {} to {}, Topic: {} QueueId: {} Consumer Group: {}", //
|
||||
requestHeader.getQueueOffset(), //
|
||||
getMessageResult.getNextBeginOffset(), //
|
||||
requestHeader.getTopic(), //
|
||||
requestHeader.getQueueId(), //
|
||||
requestHeader.getConsumerGroup()//
|
||||
);
|
||||
} else {
|
||||
response.setCode(ResponseCode.PULL_NOT_FOUND);
|
||||
}
|
||||
break;
|
||||
case NO_MATCHED_MESSAGE:
|
||||
response.setCode(ResponseCode.PULL_RETRY_IMMEDIATELY);
|
||||
break;
|
||||
case OFFSET_FOUND_NULL:
|
||||
response.setCode(ResponseCode.PULL_NOT_FOUND);
|
||||
break;
|
||||
case OFFSET_OVERFLOW_BADLY:
|
||||
response.setCode(ResponseCode.PULL_OFFSET_MOVED);
|
||||
// XXX: warn and notify me
|
||||
LOG.info("the request offset: " + requestHeader.getQueueOffset() + " over flow badly, broker max offset: "
|
||||
+ getMessageResult.getMaxOffset() + ", consumer: " + channel.remoteAddress());
|
||||
break;
|
||||
case OFFSET_OVERFLOW_ONE:
|
||||
response.setCode(ResponseCode.PULL_NOT_FOUND);
|
||||
break;
|
||||
case OFFSET_TOO_SMALL:
|
||||
response.setCode(ResponseCode.PULL_OFFSET_MOVED);
|
||||
LOG.info("the request offset too small. group={}, topic={}, requestOffset={}, brokerMinOffset={}, clientIp={}",
|
||||
requestHeader.getConsumerGroup(), requestHeader.getTopic(), requestHeader.getQueueOffset(),
|
||||
getMessageResult.getMinOffset(), channel.remoteAddress());
|
||||
break;
|
||||
default:
|
||||
assert false;
|
||||
break;
|
||||
}
|
||||
|
||||
if (this.hasConsumeMessageHook()) {
|
||||
ConsumeMessageContext context = new ConsumeMessageContext();
|
||||
context.setConsumerGroup(requestHeader.getConsumerGroup());
|
||||
context.setTopic(requestHeader.getTopic());
|
||||
context.setQueueId(requestHeader.getQueueId());
|
||||
|
||||
String owner = request.getExtFields().get(BrokerStatsManager.COMMERCIAL_OWNER);
|
||||
|
||||
switch (response.getCode()) {
|
||||
case ResponseCode.SUCCESS:
|
||||
|
||||
context.setCommercialRcvStats(BrokerStatsManager.StatsType.RCV_SUCCESS);
|
||||
context.setCommercialRcvTimes(getMessageResult.getMsgCount4Commercial());
|
||||
context.setCommercialRcvSize(getMessageResult.getBufferTotalSize());
|
||||
context.setCommercialOwner(owner);
|
||||
|
||||
break;
|
||||
case ResponseCode.PULL_NOT_FOUND:
|
||||
if (!brokerAllowSuspend) {
|
||||
|
||||
|
||||
context.setCommercialRcvStats(BrokerStatsManager.StatsType.RCV_EPOLLS);
|
||||
context.setCommercialRcvTimes(1);
|
||||
context.setCommercialOwner(owner);
|
||||
|
||||
}
|
||||
break;
|
||||
case ResponseCode.PULL_RETRY_IMMEDIATELY:
|
||||
case ResponseCode.PULL_OFFSET_MOVED:
|
||||
context.setCommercialRcvStats(BrokerStatsManager.StatsType.RCV_EPOLLS);
|
||||
context.setCommercialRcvTimes(1);
|
||||
context.setCommercialOwner(owner);
|
||||
break;
|
||||
default:
|
||||
assert false;
|
||||
break;
|
||||
}
|
||||
|
||||
this.executeConsumeMessageHookBefore(context);
|
||||
}
|
||||
|
||||
switch (response.getCode()) {
|
||||
case ResponseCode.SUCCESS:
|
||||
|
||||
this.brokerController.getBrokerStatsManager().incGroupGetNums(requestHeader.getConsumerGroup(), requestHeader.getTopic(),
|
||||
getMessageResult.getMessageCount());
|
||||
|
||||
this.brokerController.getBrokerStatsManager().incGroupGetSize(requestHeader.getConsumerGroup(), requestHeader.getTopic(),
|
||||
getMessageResult.getBufferTotalSize());
|
||||
|
||||
this.brokerController.getBrokerStatsManager().incBrokerGetNums(getMessageResult.getMessageCount());
|
||||
if (this.brokerController.getBrokerConfig().isTransferMsgByHeap()) {
|
||||
final long beginTimeMills = this.brokerController.getMessageStore().now();
|
||||
final byte[] r = this.readGetMessageResult(getMessageResult, requestHeader.getConsumerGroup(), requestHeader.getTopic(), requestHeader.getQueueId());
|
||||
this.brokerController.getBrokerStatsManager().incGroupGetLatency(requestHeader.getConsumerGroup(),
|
||||
requestHeader.getTopic(), requestHeader.getQueueId(),
|
||||
(int) (this.brokerController.getMessageStore().now() - beginTimeMills));
|
||||
response.setBody(r);
|
||||
} else {
|
||||
try {
|
||||
FileRegion fileRegion =
|
||||
new ManyMessageTransfer(response.encodeHeader(getMessageResult.getBufferTotalSize()), getMessageResult);
|
||||
channel.writeAndFlush(fileRegion).addListener(new ChannelFutureListener() {
|
||||
@Override
|
||||
public void operationComplete(ChannelFuture future) throws Exception {
|
||||
getMessageResult.release();
|
||||
if (!future.isSuccess()) {
|
||||
LOG.error("transfer many message by pagecache failed, " + channel.remoteAddress(), future.cause());
|
||||
}
|
||||
}
|
||||
});
|
||||
} catch (Throwable e) {
|
||||
LOG.error("transfer many message by pagecache exception", e);
|
||||
getMessageResult.release();
|
||||
}
|
||||
|
||||
response = null;
|
||||
}
|
||||
break;
|
||||
case ResponseCode.PULL_NOT_FOUND:
|
||||
|
||||
if (brokerAllowSuspend && hasSuspendFlag) {
|
||||
long pollingTimeMills = suspendTimeoutMillisLong;
|
||||
if (!this.brokerController.getBrokerConfig().isLongPollingEnable()) {
|
||||
pollingTimeMills = this.brokerController.getBrokerConfig().getShortPollingTimeMills();
|
||||
}
|
||||
|
||||
String topic = requestHeader.getTopic();
|
||||
long offset = requestHeader.getQueueOffset();
|
||||
int queueId = requestHeader.getQueueId();
|
||||
PullRequest pullRequest = new PullRequest(request, channel, pollingTimeMills,
|
||||
this.brokerController.getMessageStore().now(), offset, subscriptionData);
|
||||
this.brokerController.getPullRequestHoldService().suspendPullRequest(topic, queueId, pullRequest);
|
||||
response = null;
|
||||
break;
|
||||
}
|
||||
|
||||
|
||||
case ResponseCode.PULL_RETRY_IMMEDIATELY:
|
||||
break;
|
||||
case ResponseCode.PULL_OFFSET_MOVED:
|
||||
if (this.brokerController.getMessageStoreConfig().getBrokerRole() != BrokerRole.SLAVE
|
||||
|| this.brokerController.getMessageStoreConfig().isOffsetCheckInSlave()) {
|
||||
MessageQueue mq = new MessageQueue();
|
||||
mq.setTopic(requestHeader.getTopic());
|
||||
mq.setQueueId(requestHeader.getQueueId());
|
||||
mq.setBrokerName(this.brokerController.getBrokerConfig().getBrokerName());
|
||||
|
||||
OffsetMovedEvent event = new OffsetMovedEvent();
|
||||
event.setConsumerGroup(requestHeader.getConsumerGroup());
|
||||
event.setMessageQueue(mq);
|
||||
event.setOffsetRequest(requestHeader.getQueueOffset());
|
||||
event.setOffsetNew(getMessageResult.getNextBeginOffset());
|
||||
this.generateOffsetMovedEvent(event);
|
||||
LOG.warn(
|
||||
"PULL_OFFSET_MOVED:correction offset. topic={}, groupId={}, requestOffset={}, newOffset={}, suggestBrokerId={}",
|
||||
requestHeader.getTopic(), requestHeader.getConsumerGroup(), event.getOffsetRequest(), event.getOffsetNew(),
|
||||
responseHeader.getSuggestWhichBrokerId());
|
||||
} else {
|
||||
responseHeader.setSuggestWhichBrokerId(subscriptionGroupConfig.getBrokerId());
|
||||
response.setCode(ResponseCode.PULL_RETRY_IMMEDIATELY);
|
||||
LOG.warn("PULL_OFFSET_MOVED:none correction. topic={}, groupId={}, requestOffset={}, suggestBrokerId={}",
|
||||
requestHeader.getTopic(), requestHeader.getConsumerGroup(), requestHeader.getQueueOffset(),
|
||||
responseHeader.getSuggestWhichBrokerId());
|
||||
}
|
||||
|
||||
break;
|
||||
default:
|
||||
assert false;
|
||||
}
|
||||
} else {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("store getMessage return null");
|
||||
}
|
||||
|
||||
|
||||
boolean storeOffsetEnable = brokerAllowSuspend;
|
||||
storeOffsetEnable = storeOffsetEnable && hasCommitOffsetFlag;
|
||||
storeOffsetEnable = storeOffsetEnable
|
||||
&& this.brokerController.getMessageStoreConfig().getBrokerRole() != BrokerRole.SLAVE;
|
||||
if (storeOffsetEnable) {
|
||||
this.brokerController.getConsumerOffsetManager().commitOffset(RemotingHelper.parseChannelRemoteAddr(channel),
|
||||
requestHeader.getConsumerGroup(), requestHeader.getTopic(), requestHeader.getQueueId(), requestHeader.getCommitOffset());
|
||||
}
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
public boolean hasConsumeMessageHook() {
|
||||
return consumeMessageHookList != null && !this.consumeMessageHookList.isEmpty();
|
||||
}
|
||||
|
||||
public void executeConsumeMessageHookBefore(final ConsumeMessageContext context) {
|
||||
if (hasConsumeMessageHook()) {
|
||||
for (ConsumeMessageHook hook : this.consumeMessageHookList) {
|
||||
try {
|
||||
hook.consumeMessageBefore(context);
|
||||
} catch (Throwable e) {
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private byte[] readGetMessageResult(final GetMessageResult getMessageResult, final String group, final String topic, final int queueId) {
|
||||
final ByteBuffer byteBuffer = ByteBuffer.allocate(getMessageResult.getBufferTotalSize());
|
||||
|
||||
long storeTimestamp = 0;
|
||||
try {
|
||||
List<ByteBuffer> messageBufferList = getMessageResult.getMessageBufferList();
|
||||
for (ByteBuffer bb : messageBufferList) {
|
||||
|
||||
byteBuffer.put(bb);
|
||||
storeTimestamp = bb.getLong(MessageDecoder.MESSAGE_STORE_TIMESTAMP_POSTION);
|
||||
}
|
||||
} finally {
|
||||
getMessageResult.release();
|
||||
}
|
||||
|
||||
this.brokerController.getBrokerStatsManager().recordDiskFallBehindTime(group, topic, queueId, this.brokerController.getMessageStore().now() - storeTimestamp);
|
||||
return byteBuffer.array();
|
||||
}
|
||||
|
||||
private void generateOffsetMovedEvent(final OffsetMovedEvent event) {
|
||||
try {
|
||||
MessageExtBrokerInner msgInner = new MessageExtBrokerInner();
|
||||
msgInner.setTopic(MixAll.OFFSET_MOVED_EVENT);
|
||||
msgInner.setTags(event.getConsumerGroup());
|
||||
msgInner.setDelayTimeLevel(0);
|
||||
msgInner.setKeys(event.getConsumerGroup());
|
||||
msgInner.setBody(event.encode());
|
||||
msgInner.setFlag(0);
|
||||
msgInner.setPropertiesString(MessageDecoder.messageProperties2String(msgInner.getProperties()));
|
||||
msgInner.setTagsCode(MessageExtBrokerInner.tagsString2tagsCode(TopicFilterType.SINGLE_TAG, msgInner.getTags()));
|
||||
|
||||
msgInner.setQueueId(0);
|
||||
msgInner.setSysFlag(0);
|
||||
msgInner.setBornTimestamp(System.currentTimeMillis());
|
||||
msgInner.setBornHost(RemotingUtil.string2SocketAddress(this.brokerController.getBrokerAddr()));
|
||||
msgInner.setStoreHost(msgInner.getBornHost());
|
||||
|
||||
msgInner.setReconsumeTimes(0);
|
||||
|
||||
PutMessageResult putMessageResult = this.brokerController.getMessageStore().putMessage(msgInner);
|
||||
} catch (Exception e) {
|
||||
LOG.warn(String.format("generateOffsetMovedEvent Exception, %s", event.toString()), e);
|
||||
}
|
||||
}
|
||||
|
||||
public void excuteRequestWhenWakeup(final Channel channel, final RemotingCommand request) throws RemotingCommandException {
|
||||
Runnable run = new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
try {
|
||||
final RemotingCommand response = PullMessageProcessor.this.processRequest(channel, request, false);
|
||||
|
||||
if (response != null) {
|
||||
response.setOpaque(request.getOpaque());
|
||||
response.markResponseType();
|
||||
try {
|
||||
channel.writeAndFlush(response).addListener(new ChannelFutureListener() {
|
||||
@Override
|
||||
public void operationComplete(ChannelFuture future) throws Exception {
|
||||
if (!future.isSuccess()) {
|
||||
LOG.error("processRequestWrapper response to " + future.channel().remoteAddress() + " failed",
|
||||
future.cause());
|
||||
LOG.error(request.toString());
|
||||
LOG.error(response.toString());
|
||||
}
|
||||
}
|
||||
});
|
||||
} catch (Throwable e) {
|
||||
LOG.error("processRequestWrapper process request over, but response failed", e);
|
||||
LOG.error(request.toString());
|
||||
LOG.error(response.toString());
|
||||
}
|
||||
}
|
||||
} catch (RemotingCommandException e1) {
|
||||
LOG.error("excuteRequestWhenWakeup run", e1);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
this.brokerController.getPullMessageExecutor().submit(run);
|
||||
}
|
||||
|
||||
public void registerConsumeMessageHook(List<ConsumeMessageHook> sendMessageHookList) {
|
||||
this.consumeMessageHookList = sendMessageHookList;
|
||||
}
|
||||
}
|
||||
+178
@@ -0,0 +1,178 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.processor;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.pagecache.OneMessageTransfer;
|
||||
import com.alibaba.rocketmq.broker.pagecache.QueryMessageTransfer;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.protocol.RequestCode;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
import com.alibaba.rocketmq.common.protocol.header.QueryMessageRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.QueryMessageResponseHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.ViewMessageRequestHeader;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingCommandException;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyRequestProcessor;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import com.alibaba.rocketmq.store.QueryMessageResult;
|
||||
import com.alibaba.rocketmq.store.SelectMappedBufferResult;
|
||||
import io.netty.channel.ChannelFuture;
|
||||
import io.netty.channel.ChannelFutureListener;
|
||||
import io.netty.channel.ChannelHandlerContext;
|
||||
import io.netty.channel.FileRegion;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class QueryMessageProcessor implements NettyRequestProcessor {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
|
||||
private final BrokerController brokerController;
|
||||
|
||||
|
||||
public QueryMessageProcessor(final BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public RemotingCommand processRequest(ChannelHandlerContext ctx, RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
switch (request.getCode()) {
|
||||
case RequestCode.QUERY_MESSAGE:
|
||||
return this.queryMessage(ctx, request);
|
||||
case RequestCode.VIEW_MESSAGE_BY_ID:
|
||||
return this.viewMessageById(ctx, request);
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean rejectRequest() {
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
public RemotingCommand queryMessage(ChannelHandlerContext ctx, RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
final RemotingCommand response =
|
||||
RemotingCommand.createResponseCommand(QueryMessageResponseHeader.class);
|
||||
final QueryMessageResponseHeader responseHeader =
|
||||
(QueryMessageResponseHeader) response.readCustomHeader();
|
||||
final QueryMessageRequestHeader requestHeader =
|
||||
(QueryMessageRequestHeader) request
|
||||
.decodeCommandCustomHeader(QueryMessageRequestHeader.class);
|
||||
|
||||
|
||||
response.setOpaque(request.getOpaque());
|
||||
|
||||
|
||||
String isUniqueKey = request.getExtFields().get(MixAll.UNIQUE_MSG_QUERY_FLAG);
|
||||
if (isUniqueKey != null && isUniqueKey.equals("true")) {
|
||||
requestHeader.setMaxNum(this.brokerController.getMessageStoreConfig().getDefaultQueryMaxNum());
|
||||
}
|
||||
|
||||
final QueryMessageResult queryMessageResult =
|
||||
this.brokerController.getMessageStore().queryMessage(requestHeader.getTopic(),
|
||||
requestHeader.getKey(), requestHeader.getMaxNum(), requestHeader.getBeginTimestamp(),
|
||||
requestHeader.getEndTimestamp());
|
||||
assert queryMessageResult != null;
|
||||
|
||||
responseHeader.setIndexLastUpdatePhyoffset(queryMessageResult.getIndexLastUpdatePhyoffset());
|
||||
responseHeader.setIndexLastUpdateTimestamp(queryMessageResult.getIndexLastUpdateTimestamp());
|
||||
|
||||
|
||||
if (queryMessageResult.getBufferTotalSize() > 0) {
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
|
||||
try {
|
||||
FileRegion fileRegion =
|
||||
new QueryMessageTransfer(response.encodeHeader(queryMessageResult
|
||||
.getBufferTotalSize()), queryMessageResult);
|
||||
ctx.channel().writeAndFlush(fileRegion).addListener(new ChannelFutureListener() {
|
||||
@Override
|
||||
public void operationComplete(ChannelFuture future) throws Exception {
|
||||
queryMessageResult.release();
|
||||
if (!future.isSuccess()) {
|
||||
log.error("transfer query message by pagecache failed, ", future.cause());
|
||||
}
|
||||
}
|
||||
});
|
||||
} catch (Throwable e) {
|
||||
log.error("", e);
|
||||
queryMessageResult.release();
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
response.setCode(ResponseCode.QUERY_NOT_FOUND);
|
||||
response.setRemark("can not find message, maybe time range not correct");
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
public RemotingCommand viewMessageById(ChannelHandlerContext ctx, RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
final RemotingCommand response = RemotingCommand.createResponseCommand(null);
|
||||
final ViewMessageRequestHeader requestHeader =
|
||||
(ViewMessageRequestHeader) request.decodeCommandCustomHeader(ViewMessageRequestHeader.class);
|
||||
|
||||
|
||||
response.setOpaque(request.getOpaque());
|
||||
|
||||
final SelectMappedBufferResult selectMappedBufferResult =
|
||||
this.brokerController.getMessageStore().selectOneMessageByOffset(requestHeader.getOffset());
|
||||
if (selectMappedBufferResult != null) {
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
|
||||
try {
|
||||
FileRegion fileRegion =
|
||||
new OneMessageTransfer(response.encodeHeader(selectMappedBufferResult.getSize()),
|
||||
selectMappedBufferResult);
|
||||
ctx.channel().writeAndFlush(fileRegion).addListener(new ChannelFutureListener() {
|
||||
@Override
|
||||
public void operationComplete(ChannelFuture future) throws Exception {
|
||||
selectMappedBufferResult.release();
|
||||
if (!future.isSuccess()) {
|
||||
log.error("transfer one message by pagecache failed, ", future.cause());
|
||||
}
|
||||
}
|
||||
});
|
||||
} catch (Throwable e) {
|
||||
log.error("", e);
|
||||
selectMappedBufferResult.release();
|
||||
}
|
||||
|
||||
return null;
|
||||
} else {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("can not find message by the offset, " + requestHeader.getOffset());
|
||||
}
|
||||
|
||||
return response;
|
||||
}
|
||||
}
|
||||
+496
@@ -0,0 +1,496 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.processor;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.mqtrace.ConsumeMessageContext;
|
||||
import com.alibaba.rocketmq.broker.mqtrace.ConsumeMessageHook;
|
||||
import com.alibaba.rocketmq.broker.mqtrace.SendMessageContext;
|
||||
import com.alibaba.rocketmq.common.*;
|
||||
import com.alibaba.rocketmq.common.constant.PermName;
|
||||
import com.alibaba.rocketmq.common.help.FAQUrl;
|
||||
import com.alibaba.rocketmq.common.message.MessageAccessor;
|
||||
import com.alibaba.rocketmq.common.message.MessageConst;
|
||||
import com.alibaba.rocketmq.common.message.MessageDecoder;
|
||||
import com.alibaba.rocketmq.common.message.MessageExt;
|
||||
import com.alibaba.rocketmq.common.protocol.RequestCode;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
import com.alibaba.rocketmq.common.protocol.header.ConsumerSendMsgBackRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.SendMessageRequestHeader;
|
||||
import com.alibaba.rocketmq.common.protocol.header.SendMessageResponseHeader;
|
||||
import com.alibaba.rocketmq.common.subscription.SubscriptionGroupConfig;
|
||||
import com.alibaba.rocketmq.common.sysflag.MessageSysFlag;
|
||||
import com.alibaba.rocketmq.common.sysflag.TopicSysFlag;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingCommandException;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyRequestProcessor;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingCommand;
|
||||
import com.alibaba.rocketmq.store.MessageExtBrokerInner;
|
||||
import com.alibaba.rocketmq.store.PutMessageResult;
|
||||
import com.alibaba.rocketmq.store.config.StorePathConfigHelper;
|
||||
import com.alibaba.rocketmq.store.stats.BrokerStatsManager;
|
||||
import io.netty.channel.ChannelHandlerContext;
|
||||
|
||||
import java.net.SocketAddress;
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class SendMessageProcessor extends AbstractSendMessageProcessor implements NettyRequestProcessor {
|
||||
|
||||
private List<ConsumeMessageHook> consumeMessageHookList;
|
||||
|
||||
public SendMessageProcessor(final BrokerController brokerController) {
|
||||
super(brokerController);
|
||||
}
|
||||
|
||||
@Override
|
||||
public RemotingCommand processRequest(ChannelHandlerContext ctx, RemotingCommand request) throws RemotingCommandException {
|
||||
SendMessageContext mqtraceContext;
|
||||
switch (request.getCode()) {
|
||||
case RequestCode.CONSUMER_SEND_MSG_BACK:
|
||||
return this.consumerSendMsgBack(ctx, request);
|
||||
default:
|
||||
SendMessageRequestHeader requestHeader = parseRequestHeader(request);
|
||||
if (requestHeader == null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
mqtraceContext = buildMsgContext(ctx, requestHeader);
|
||||
this.executeSendMessageHookBefore(ctx, request, mqtraceContext);
|
||||
final RemotingCommand response = this.sendMessage(ctx, request, mqtraceContext, requestHeader);
|
||||
|
||||
this.executeSendMessageHookAfter(response, mqtraceContext);
|
||||
return response;
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean rejectRequest() {
|
||||
return this.brokerController.getMessageStore().isOSPageCacheBusy() ||
|
||||
this.brokerController.getMessageStore().isTransientStorePoolDeficient();
|
||||
}
|
||||
|
||||
private RemotingCommand consumerSendMsgBack(final ChannelHandlerContext ctx, final RemotingCommand request)
|
||||
throws RemotingCommandException {
|
||||
final RemotingCommand response = RemotingCommand.createResponseCommand(null);
|
||||
final ConsumerSendMsgBackRequestHeader requestHeader =
|
||||
(ConsumerSendMsgBackRequestHeader) request.decodeCommandCustomHeader(ConsumerSendMsgBackRequestHeader.class);
|
||||
|
||||
if (this.hasConsumeMessageHook() && !UtilAll.isBlank(requestHeader.getOriginMsgId())) {
|
||||
|
||||
ConsumeMessageContext context = new ConsumeMessageContext();
|
||||
context.setConsumerGroup(requestHeader.getGroup());
|
||||
context.setTopic(requestHeader.getOriginTopic());
|
||||
context.setCommercialRcvStats(BrokerStatsManager.StatsType.SEND_BACK);
|
||||
context.setCommercialRcvTimes(1);
|
||||
context.setCommercialOwner(request.getExtFields().get(BrokerStatsManager.COMMERCIAL_OWNER));
|
||||
|
||||
this.executeConsumeMessageHookAfter(context);
|
||||
}
|
||||
|
||||
|
||||
SubscriptionGroupConfig subscriptionGroupConfig =
|
||||
this.brokerController.getSubscriptionGroupManager().findSubscriptionGroupConfig(requestHeader.getGroup());
|
||||
if (null == subscriptionGroupConfig) {
|
||||
response.setCode(ResponseCode.SUBSCRIPTION_GROUP_NOT_EXIST);
|
||||
response.setRemark("subscription group not exist, " + requestHeader.getGroup() + " "
|
||||
+ FAQUrl.suggestTodo(FAQUrl.SUBSCRIPTION_GROUP_NOT_EXIST));
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
if (!PermName.isWriteable(this.brokerController.getBrokerConfig().getBrokerPermission())) {
|
||||
response.setCode(ResponseCode.NO_PERMISSION);
|
||||
response.setRemark("the broker[" + this.brokerController.getBrokerConfig().getBrokerIP1() + "] sending message is forbidden");
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
if (subscriptionGroupConfig.getRetryQueueNums() <= 0) {
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
return response;
|
||||
}
|
||||
|
||||
String newTopic = MixAll.getRetryTopic(requestHeader.getGroup());
|
||||
int queueIdInt = Math.abs(this.random.nextInt() % 99999999) % subscriptionGroupConfig.getRetryQueueNums();
|
||||
|
||||
|
||||
int topicSysFlag = 0;
|
||||
if (requestHeader.isUnitMode()) {
|
||||
topicSysFlag = TopicSysFlag.buildSysFlag(false, true);
|
||||
}
|
||||
|
||||
|
||||
TopicConfig topicConfig = this.brokerController.getTopicConfigManager().createTopicInSendMessageBackMethod(//
|
||||
newTopic, //
|
||||
subscriptionGroupConfig.getRetryQueueNums(), //
|
||||
PermName.PERM_WRITE | PermName.PERM_READ, topicSysFlag);
|
||||
if (null == topicConfig) {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("topic[" + newTopic + "] not exist");
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
if (!PermName.isWriteable(topicConfig.getPerm())) {
|
||||
response.setCode(ResponseCode.NO_PERMISSION);
|
||||
response.setRemark(String.format("the topic[%s] sending message is forbidden", newTopic));
|
||||
return response;
|
||||
}
|
||||
|
||||
MessageExt msgExt = this.brokerController.getMessageStore().lookMessageByOffset(requestHeader.getOffset());
|
||||
if (null == msgExt) {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("look message by offset failed, " + requestHeader.getOffset());
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
final String retryTopic = msgExt.getProperty(MessageConst.PROPERTY_RETRY_TOPIC);
|
||||
if (null == retryTopic) {
|
||||
MessageAccessor.putProperty(msgExt, MessageConst.PROPERTY_RETRY_TOPIC, msgExt.getTopic());
|
||||
}
|
||||
msgExt.setWaitStoreMsgOK(false);
|
||||
|
||||
|
||||
int delayLevel = requestHeader.getDelayLevel();
|
||||
|
||||
|
||||
int maxReconsumeTimes = subscriptionGroupConfig.getRetryMaxTimes();
|
||||
if (request.getVersion() >= MQVersion.Version.V3_4_9.ordinal()) {
|
||||
maxReconsumeTimes = requestHeader.getMaxReconsumeTimes();
|
||||
}
|
||||
|
||||
|
||||
if (msgExt.getReconsumeTimes() >= maxReconsumeTimes//
|
||||
|| delayLevel < 0) {
|
||||
newTopic = MixAll.getDLQTopic(requestHeader.getGroup());
|
||||
queueIdInt = Math.abs(this.random.nextInt() % 99999999) % DLQ_NUMS_PER_GROUP;
|
||||
|
||||
topicConfig = this.brokerController.getTopicConfigManager().createTopicInSendMessageBackMethod(newTopic, //
|
||||
DLQ_NUMS_PER_GROUP, //
|
||||
PermName.PERM_WRITE, 0
|
||||
);
|
||||
if (null == topicConfig) {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("topic[" + newTopic + "] not exist");
|
||||
return response;
|
||||
}
|
||||
} else {
|
||||
if (0 == delayLevel) {
|
||||
delayLevel = 3 + msgExt.getReconsumeTimes();
|
||||
}
|
||||
|
||||
msgExt.setDelayTimeLevel(delayLevel);
|
||||
}
|
||||
|
||||
MessageExtBrokerInner msgInner = new MessageExtBrokerInner();
|
||||
msgInner.setTopic(newTopic);
|
||||
msgInner.setBody(msgExt.getBody());
|
||||
msgInner.setFlag(msgExt.getFlag());
|
||||
MessageAccessor.setProperties(msgInner, msgExt.getProperties());
|
||||
msgInner.setPropertiesString(MessageDecoder.messageProperties2String(msgExt.getProperties()));
|
||||
msgInner.setTagsCode(MessageExtBrokerInner.tagsString2tagsCode(null, msgExt.getTags()));
|
||||
|
||||
msgInner.setQueueId(queueIdInt);
|
||||
msgInner.setSysFlag(msgExt.getSysFlag());
|
||||
msgInner.setBornTimestamp(msgExt.getBornTimestamp());
|
||||
msgInner.setBornHost(msgExt.getBornHost());
|
||||
msgInner.setStoreHost(this.getStoreHost());
|
||||
msgInner.setReconsumeTimes(msgExt.getReconsumeTimes() + 1);
|
||||
|
||||
String originMsgId = MessageAccessor.getOriginMessageId(msgExt);
|
||||
MessageAccessor.setOriginMessageId(msgInner, UtilAll.isBlank(originMsgId) ? msgExt.getMsgId() : originMsgId);
|
||||
|
||||
PutMessageResult putMessageResult = this.brokerController.getMessageStore().putMessage(msgInner);
|
||||
if (putMessageResult != null) {
|
||||
switch (putMessageResult.getPutMessageStatus()) {
|
||||
case PUT_OK:
|
||||
String backTopic = msgExt.getTopic();
|
||||
String correctTopic = msgExt.getProperty(MessageConst.PROPERTY_RETRY_TOPIC);
|
||||
if (correctTopic != null) {
|
||||
backTopic = correctTopic;
|
||||
}
|
||||
|
||||
this.brokerController.getBrokerStatsManager().incSendBackNums(requestHeader.getGroup(), backTopic);
|
||||
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
response.setRemark(null);
|
||||
|
||||
return response;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark(putMessageResult.getPutMessageStatus().name());
|
||||
return response;
|
||||
}
|
||||
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("putMessageResult is null");
|
||||
return response;
|
||||
}
|
||||
|
||||
private RemotingCommand sendMessage(final ChannelHandlerContext ctx, //
|
||||
final RemotingCommand request, //
|
||||
final SendMessageContext sendMessageContext, //
|
||||
final SendMessageRequestHeader requestHeader) throws RemotingCommandException {
|
||||
|
||||
final RemotingCommand response = RemotingCommand.createResponseCommand(SendMessageResponseHeader.class);
|
||||
final SendMessageResponseHeader responseHeader = (SendMessageResponseHeader) response.readCustomHeader();
|
||||
|
||||
|
||||
response.setOpaque(request.getOpaque());
|
||||
|
||||
response.addExtField(MessageConst.PROPERTY_MSG_REGION, this.brokerController.getBrokerConfig().getRegionId());
|
||||
response.addExtField(MessageConst.PROPERTY_TRACE_SWITCH, String.valueOf(this.brokerController.getBrokerConfig().isTraceOn()));
|
||||
|
||||
if (log.isDebugEnabled()) {
|
||||
log.debug("receive SendMessage request command, " + request);
|
||||
}
|
||||
|
||||
final long startTimstamp = this.brokerController.getBrokerConfig().getStartAcceptSendRequestTimeStamp();
|
||||
if (this.brokerController.getMessageStore().now() < startTimstamp) {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark(String.format("broker unable to service, until %s", UtilAll.timeMillisToHumanString2(startTimstamp)));
|
||||
return response;
|
||||
}
|
||||
|
||||
response.setCode(-1);
|
||||
super.msgCheck(ctx, requestHeader, response);
|
||||
if (response.getCode() != -1) {
|
||||
return response;
|
||||
}
|
||||
|
||||
final byte[] body = request.getBody();
|
||||
|
||||
int queueIdInt = requestHeader.getQueueId();
|
||||
TopicConfig topicConfig = this.brokerController.getTopicConfigManager().selectTopicConfig(requestHeader.getTopic());
|
||||
|
||||
if (queueIdInt < 0) {
|
||||
queueIdInt = Math.abs(this.random.nextInt() % 99999999) % topicConfig.getWriteQueueNums();
|
||||
}
|
||||
|
||||
int sysFlag = requestHeader.getSysFlag();
|
||||
|
||||
if (TopicFilterType.MULTI_TAG == topicConfig.getTopicFilterType()) {
|
||||
sysFlag |= MessageSysFlag.MULTI_TAGS_FLAG;
|
||||
}
|
||||
|
||||
String newTopic = requestHeader.getTopic();
|
||||
if (null != newTopic && newTopic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
|
||||
String groupName = newTopic.substring(MixAll.RETRY_GROUP_TOPIC_PREFIX.length());
|
||||
SubscriptionGroupConfig subscriptionGroupConfig =
|
||||
this.brokerController.getSubscriptionGroupManager().findSubscriptionGroupConfig(groupName);
|
||||
if (null == subscriptionGroupConfig) {
|
||||
response.setCode(ResponseCode.SUBSCRIPTION_GROUP_NOT_EXIST);
|
||||
response.setRemark(
|
||||
"subscription group not exist, " + groupName + " " + FAQUrl.suggestTodo(FAQUrl.SUBSCRIPTION_GROUP_NOT_EXIST));
|
||||
return response;
|
||||
}
|
||||
|
||||
|
||||
int maxReconsumeTimes = subscriptionGroupConfig.getRetryMaxTimes();
|
||||
if (request.getVersion() >= MQVersion.Version.V3_4_9.ordinal()) {
|
||||
maxReconsumeTimes = requestHeader.getMaxReconsumeTimes();
|
||||
}
|
||||
int reconsumeTimes = requestHeader.getReconsumeTimes() == null ? 0 : requestHeader.getReconsumeTimes();
|
||||
if (reconsumeTimes >= maxReconsumeTimes) {
|
||||
newTopic = MixAll.getDLQTopic(groupName);
|
||||
queueIdInt = Math.abs(this.random.nextInt() % 99999999) % DLQ_NUMS_PER_GROUP;
|
||||
topicConfig = this.brokerController.getTopicConfigManager().createTopicInSendMessageBackMethod(newTopic, //
|
||||
DLQ_NUMS_PER_GROUP, //
|
||||
PermName.PERM_WRITE, 0
|
||||
);
|
||||
if (null == topicConfig) {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("topic[" + newTopic + "] not exist");
|
||||
return response;
|
||||
}
|
||||
}
|
||||
}
|
||||
MessageExtBrokerInner msgInner = new MessageExtBrokerInner();
|
||||
msgInner.setTopic(newTopic);
|
||||
msgInner.setBody(body);
|
||||
msgInner.setFlag(requestHeader.getFlag());
|
||||
MessageAccessor.setProperties(msgInner, MessageDecoder.string2messageProperties(requestHeader.getProperties()));
|
||||
msgInner.setPropertiesString(requestHeader.getProperties());
|
||||
msgInner.setTagsCode(MessageExtBrokerInner.tagsString2tagsCode(topicConfig.getTopicFilterType(), msgInner.getTags()));
|
||||
|
||||
msgInner.setQueueId(queueIdInt);
|
||||
msgInner.setSysFlag(sysFlag);
|
||||
msgInner.setBornTimestamp(requestHeader.getBornTimestamp());
|
||||
msgInner.setBornHost(ctx.channel().remoteAddress());
|
||||
msgInner.setStoreHost(this.getStoreHost());
|
||||
msgInner.setReconsumeTimes(requestHeader.getReconsumeTimes() == null ? 0 : requestHeader.getReconsumeTimes());
|
||||
|
||||
if (this.brokerController.getBrokerConfig().isRejectTransactionMessage()) {
|
||||
String traFlag = msgInner.getProperty(MessageConst.PROPERTY_TRANSACTION_PREPARED);
|
||||
if (traFlag != null) {
|
||||
response.setCode(ResponseCode.NO_PERMISSION);
|
||||
response.setRemark(
|
||||
"the broker[" + this.brokerController.getBrokerConfig().getBrokerIP1() + "] sending transaction message is forbidden");
|
||||
return response;
|
||||
}
|
||||
}
|
||||
|
||||
PutMessageResult putMessageResult = this.brokerController.getMessageStore().putMessage(msgInner);
|
||||
if (putMessageResult != null) {
|
||||
boolean sendOK = false;
|
||||
|
||||
switch (putMessageResult.getPutMessageStatus()) {
|
||||
// Success
|
||||
case PUT_OK:
|
||||
sendOK = true;
|
||||
response.setCode(ResponseCode.SUCCESS);
|
||||
break;
|
||||
case FLUSH_DISK_TIMEOUT:
|
||||
response.setCode(ResponseCode.FLUSH_DISK_TIMEOUT);
|
||||
sendOK = true;
|
||||
break;
|
||||
case FLUSH_SLAVE_TIMEOUT:
|
||||
response.setCode(ResponseCode.FLUSH_SLAVE_TIMEOUT);
|
||||
sendOK = true;
|
||||
break;
|
||||
case SLAVE_NOT_AVAILABLE:
|
||||
response.setCode(ResponseCode.SLAVE_NOT_AVAILABLE);
|
||||
sendOK = true;
|
||||
break;
|
||||
|
||||
// Failed
|
||||
case CREATE_MAPEDFILE_FAILED:
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("create mapped file failed, server is busy or broken.");
|
||||
break;
|
||||
case MESSAGE_ILLEGAL:
|
||||
case PROPERTIES_SIZE_EXCEEDED:
|
||||
response.setCode(ResponseCode.MESSAGE_ILLEGAL);
|
||||
response.setRemark(
|
||||
"the message is illegal, maybe msg body or properties length not matched. msg body length limit 128k, msg properties length limit 32k.");
|
||||
break;
|
||||
case SERVICE_NOT_AVAILABLE:
|
||||
response.setCode(ResponseCode.SERVICE_NOT_AVAILABLE);
|
||||
response.setRemark(
|
||||
"service not available now, maybe disk full, " + diskUtil() + ", maybe your broker machine memory too small.");
|
||||
break;
|
||||
case OS_PAGECACHE_BUSY:
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("[PC_SYNCHRONIZED]broker busy, start flow control for a while");
|
||||
break;
|
||||
case UNKNOWN_ERROR:
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("UNKNOWN_ERROR");
|
||||
break;
|
||||
default:
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("UNKNOWN_ERROR DEFAULT");
|
||||
break;
|
||||
}
|
||||
|
||||
String owner = request.getExtFields().get(BrokerStatsManager.COMMERCIAL_OWNER);
|
||||
if (sendOK) {
|
||||
|
||||
this.brokerController.getBrokerStatsManager().incTopicPutNums(msgInner.getTopic());
|
||||
this.brokerController.getBrokerStatsManager().incTopicPutSize(msgInner.getTopic(),
|
||||
putMessageResult.getAppendMessageResult().getWroteBytes());
|
||||
this.brokerController.getBrokerStatsManager().incBrokerPutNums();
|
||||
|
||||
response.setRemark(null);
|
||||
|
||||
responseHeader.setMsgId(putMessageResult.getAppendMessageResult().getMsgId());
|
||||
responseHeader.setQueueId(queueIdInt);
|
||||
responseHeader.setQueueOffset(putMessageResult.getAppendMessageResult().getLogicsOffset());
|
||||
|
||||
|
||||
doResponse(ctx, request, response);
|
||||
|
||||
|
||||
if (hasSendMessageHook()) {
|
||||
sendMessageContext.setMsgId(responseHeader.getMsgId());
|
||||
sendMessageContext.setQueueId(responseHeader.getQueueId());
|
||||
sendMessageContext.setQueueOffset(responseHeader.getQueueOffset());
|
||||
|
||||
int wroteSize = putMessageResult.getAppendMessageResult().getWroteBytes();
|
||||
int incValue = (int) Math.ceil(wroteSize / BrokerStatsManager.SIZE_PER_COUNT);
|
||||
|
||||
sendMessageContext.setCommercialSendStats(BrokerStatsManager.StatsType.SEND_SUCCESS);
|
||||
sendMessageContext.setCommercialSendTimes(incValue);
|
||||
sendMessageContext.setCommercialSendSize(wroteSize);
|
||||
sendMessageContext.setCommercialOwner(owner);
|
||||
}
|
||||
return null;
|
||||
} else {
|
||||
if (hasSendMessageHook()) {
|
||||
int wroteSize = request.getBody().length;
|
||||
int incValue = (int) Math.ceil(wroteSize / BrokerStatsManager.SIZE_PER_COUNT);
|
||||
|
||||
sendMessageContext.setCommercialSendStats(BrokerStatsManager.StatsType.SEND_FAILURE);
|
||||
sendMessageContext.setCommercialSendTimes(incValue);
|
||||
sendMessageContext.setCommercialSendSize(wroteSize);
|
||||
sendMessageContext.setCommercialOwner(owner);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
response.setCode(ResponseCode.SYSTEM_ERROR);
|
||||
response.setRemark("store putMessage return null");
|
||||
}
|
||||
|
||||
return response;
|
||||
}
|
||||
|
||||
public boolean hasConsumeMessageHook() {
|
||||
return consumeMessageHookList != null && !this.consumeMessageHookList.isEmpty();
|
||||
}
|
||||
|
||||
public void executeConsumeMessageHookAfter(final ConsumeMessageContext context) {
|
||||
if (hasConsumeMessageHook()) {
|
||||
for (ConsumeMessageHook hook : this.consumeMessageHookList) {
|
||||
try {
|
||||
hook.consumeMessageAfter(context);
|
||||
} catch (Throwable e) {
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public SocketAddress getStoreHost() {
|
||||
return storeHost;
|
||||
}
|
||||
|
||||
private String diskUtil() {
|
||||
String storePathPhysic = this.brokerController.getMessageStoreConfig().getStorePathCommitLog();
|
||||
double physicRatio = UtilAll.getDiskPartitionSpaceUsedPercent(storePathPhysic);
|
||||
|
||||
String storePathLogis =
|
||||
StorePathConfigHelper.getStorePathConsumeQueue(this.brokerController.getMessageStoreConfig().getStorePathRootDir());
|
||||
double logisRatio = UtilAll.getDiskPartitionSpaceUsedPercent(storePathLogis);
|
||||
|
||||
String storePathIndex =
|
||||
StorePathConfigHelper.getStorePathIndex(this.brokerController.getMessageStoreConfig().getStorePathRootDir());
|
||||
double indexRatio = UtilAll.getDiskPartitionSpaceUsedPercent(storePathIndex);
|
||||
|
||||
return String.format("CL: %5.2f CQ: %5.2f INDEX: %5.2f", physicRatio, logisRatio, indexRatio);
|
||||
}
|
||||
|
||||
public void registerConsumeMessageHook(List<ConsumeMessageHook> consumeMessageHookList) {
|
||||
this.consumeMessageHookList = consumeMessageHookList;
|
||||
}
|
||||
}
|
||||
+158
@@ -0,0 +1,158 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.slave;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.subscription.SubscriptionGroupManager;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.protocol.body.ConsumerOffsetSerializeWrapper;
|
||||
import com.alibaba.rocketmq.common.protocol.body.SubscriptionGroupWrapper;
|
||||
import com.alibaba.rocketmq.common.protocol.body.TopicConfigSerializeWrapper;
|
||||
import com.alibaba.rocketmq.store.config.StorePathConfigHelper;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.io.IOException;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
* @author manhong.yqd
|
||||
*/
|
||||
public class SlaveSynchronize {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private final BrokerController brokerController;
|
||||
private volatile String masterAddr = null;
|
||||
|
||||
|
||||
public SlaveSynchronize(BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
}
|
||||
|
||||
|
||||
public String getMasterAddr() {
|
||||
return masterAddr;
|
||||
}
|
||||
|
||||
|
||||
public void setMasterAddr(String masterAddr) {
|
||||
this.masterAddr = masterAddr;
|
||||
}
|
||||
|
||||
|
||||
public void syncAll() {
|
||||
this.syncTopicConfig();
|
||||
this.syncConsumerOffset();
|
||||
this.syncDelayOffset();
|
||||
this.syncSubscriptionGroupConfig();
|
||||
}
|
||||
|
||||
|
||||
private void syncTopicConfig() {
|
||||
String masterAddrBak = this.masterAddr;
|
||||
if (masterAddrBak != null) {
|
||||
try {
|
||||
TopicConfigSerializeWrapper topicWrapper =
|
||||
this.brokerController.getBrokerOuterAPI().getAllTopicConfig(masterAddrBak);
|
||||
if (!this.brokerController.getTopicConfigManager().getDataVersion()
|
||||
.equals(topicWrapper.getDataVersion())) {
|
||||
|
||||
this.brokerController.getTopicConfigManager().getDataVersion()
|
||||
.assignNewOne(topicWrapper.getDataVersion());
|
||||
this.brokerController.getTopicConfigManager().getTopicConfigTable().clear();
|
||||
this.brokerController.getTopicConfigManager().getTopicConfigTable()
|
||||
.putAll(topicWrapper.getTopicConfigTable());
|
||||
this.brokerController.getTopicConfigManager().persist();
|
||||
|
||||
log.info("update slave topic config from master, {}", masterAddrBak);
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("syncTopicConfig Exception, " + masterAddrBak, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
private void syncConsumerOffset() {
|
||||
String masterAddrBak = this.masterAddr;
|
||||
if (masterAddrBak != null) {
|
||||
try {
|
||||
ConsumerOffsetSerializeWrapper offsetWrapper =
|
||||
this.brokerController.getBrokerOuterAPI().getAllConsumerOffset(masterAddrBak);
|
||||
this.brokerController.getConsumerOffsetManager().getOffsetTable()
|
||||
.putAll(offsetWrapper.getOffsetTable());
|
||||
this.brokerController.getConsumerOffsetManager().persist();
|
||||
log.info("update slave consumer offset from master, {}", masterAddrBak);
|
||||
} catch (Exception e) {
|
||||
log.error("syncConsumerOffset Exception, " + masterAddrBak, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
private void syncDelayOffset() {
|
||||
String masterAddrBak = this.masterAddr;
|
||||
if (masterAddrBak != null) {
|
||||
try {
|
||||
String delayOffset =
|
||||
this.brokerController.getBrokerOuterAPI().getAllDelayOffset(masterAddrBak);
|
||||
if (delayOffset != null) {
|
||||
|
||||
String fileName =
|
||||
StorePathConfigHelper.getDelayOffsetStorePath(this.brokerController
|
||||
.getMessageStoreConfig().getStorePathRootDir());
|
||||
try {
|
||||
MixAll.string2File(delayOffset, fileName);
|
||||
} catch (IOException e) {
|
||||
log.error("persist file Exception, " + fileName, e);
|
||||
}
|
||||
}
|
||||
log.info("update slave delay offset from master, {}", masterAddrBak);
|
||||
} catch (Exception e) {
|
||||
log.error("syncDelayOffset Exception, " + masterAddrBak, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
private void syncSubscriptionGroupConfig() {
|
||||
String masterAddrBak = this.masterAddr;
|
||||
if (masterAddrBak != null) {
|
||||
try {
|
||||
SubscriptionGroupWrapper subscriptionWrapper =
|
||||
this.brokerController.getBrokerOuterAPI()
|
||||
.getAllSubscriptionGroupConfig(masterAddrBak);
|
||||
|
||||
if (!this.brokerController.getSubscriptionGroupManager().getDataVersion()
|
||||
.equals(subscriptionWrapper.getDataVersion())) {
|
||||
SubscriptionGroupManager subscriptionGroupManager =
|
||||
this.brokerController.getSubscriptionGroupManager();
|
||||
subscriptionGroupManager.getDataVersion().assignNewOne(
|
||||
subscriptionWrapper.getDataVersion());
|
||||
subscriptionGroupManager.getSubscriptionGroupTable().clear();
|
||||
subscriptionGroupManager.getSubscriptionGroupTable().putAll(
|
||||
subscriptionWrapper.getSubscriptionGroupTable());
|
||||
subscriptionGroupManager.persist();
|
||||
log.info("update slave Subscription Group from master, {}", masterAddrBak);
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("syncSubscriptionGroup Exception, " + masterAddrBak, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+203
@@ -0,0 +1,203 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.subscription;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.BrokerPathConfigHelper;
|
||||
import com.alibaba.rocketmq.common.ConfigManager;
|
||||
import com.alibaba.rocketmq.common.DataVersion;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.subscription.SubscriptionGroupConfig;
|
||||
import com.alibaba.rocketmq.remoting.protocol.RemotingSerializable;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.io.File;
|
||||
import java.util.Iterator;
|
||||
import java.util.Map.Entry;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class SubscriptionGroupManager extends ConfigManager {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
|
||||
private final ConcurrentHashMap<String, SubscriptionGroupConfig> subscriptionGroupTable =
|
||||
new ConcurrentHashMap<String, SubscriptionGroupConfig>(1024);
|
||||
private final DataVersion dataVersion = new DataVersion();
|
||||
private transient BrokerController brokerController;
|
||||
|
||||
|
||||
public SubscriptionGroupManager() {
|
||||
this.init();
|
||||
}
|
||||
|
||||
private void init() {
|
||||
{
|
||||
SubscriptionGroupConfig subscriptionGroupConfig = new SubscriptionGroupConfig();
|
||||
subscriptionGroupConfig.setGroupName(MixAll.TOOLS_CONSUMER_GROUP);
|
||||
this.subscriptionGroupTable.put(MixAll.TOOLS_CONSUMER_GROUP, subscriptionGroupConfig);
|
||||
}
|
||||
|
||||
{
|
||||
SubscriptionGroupConfig subscriptionGroupConfig = new SubscriptionGroupConfig();
|
||||
subscriptionGroupConfig.setGroupName(MixAll.FILTERSRV_CONSUMER_GROUP);
|
||||
this.subscriptionGroupTable.put(MixAll.FILTERSRV_CONSUMER_GROUP, subscriptionGroupConfig);
|
||||
}
|
||||
|
||||
{
|
||||
SubscriptionGroupConfig subscriptionGroupConfig = new SubscriptionGroupConfig();
|
||||
subscriptionGroupConfig.setGroupName(MixAll.SELF_TEST_CONSUMER_GROUP);
|
||||
this.subscriptionGroupTable.put(MixAll.SELF_TEST_CONSUMER_GROUP, subscriptionGroupConfig);
|
||||
}
|
||||
|
||||
{
|
||||
SubscriptionGroupConfig subscriptionGroupConfig = new SubscriptionGroupConfig();
|
||||
subscriptionGroupConfig.setGroupName(MixAll.ONS_HTTP_PROXY_GROUP);
|
||||
subscriptionGroupConfig.setConsumeBroadcastEnable(true);
|
||||
this.subscriptionGroupTable.put(MixAll.ONS_HTTP_PROXY_GROUP, subscriptionGroupConfig);
|
||||
}
|
||||
|
||||
{
|
||||
SubscriptionGroupConfig subscriptionGroupConfig = new SubscriptionGroupConfig();
|
||||
subscriptionGroupConfig.setGroupName(MixAll.CID_ONSAPI_PULL_GROUP);
|
||||
subscriptionGroupConfig.setConsumeBroadcastEnable(true);
|
||||
this.subscriptionGroupTable.put(MixAll.CID_ONSAPI_PULL_GROUP, subscriptionGroupConfig);
|
||||
}
|
||||
|
||||
{
|
||||
SubscriptionGroupConfig subscriptionGroupConfig = new SubscriptionGroupConfig();
|
||||
subscriptionGroupConfig.setGroupName(MixAll.CID_ONSAPI_PERMISSION_GROUP);
|
||||
subscriptionGroupConfig.setConsumeBroadcastEnable(true);
|
||||
this.subscriptionGroupTable.put(MixAll.CID_ONSAPI_PERMISSION_GROUP, subscriptionGroupConfig);
|
||||
}
|
||||
|
||||
{
|
||||
SubscriptionGroupConfig subscriptionGroupConfig = new SubscriptionGroupConfig();
|
||||
subscriptionGroupConfig.setGroupName(MixAll.CID_ONSAPI_OWNER_GROUP);
|
||||
subscriptionGroupConfig.setConsumeBroadcastEnable(true);
|
||||
this.subscriptionGroupTable.put(MixAll.CID_ONSAPI_OWNER_GROUP, subscriptionGroupConfig);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public SubscriptionGroupManager(BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
this.init();
|
||||
}
|
||||
|
||||
|
||||
public void updateSubscriptionGroupConfig(final SubscriptionGroupConfig config) {
|
||||
SubscriptionGroupConfig old = this.subscriptionGroupTable.put(config.getGroupName(), config);
|
||||
if (old != null) {
|
||||
log.info("update subscription group config, old: " + old + " new: " + config);
|
||||
} else {
|
||||
log.info("create new subscription group, " + config);
|
||||
}
|
||||
|
||||
this.dataVersion.nextVersion();
|
||||
|
||||
this.persist();
|
||||
}
|
||||
|
||||
public void disableConsume(final String groupName) {
|
||||
SubscriptionGroupConfig old = this.subscriptionGroupTable.get(groupName);
|
||||
if (old != null) {
|
||||
old.setConsumeEnable(false);
|
||||
this.dataVersion.nextVersion();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public SubscriptionGroupConfig findSubscriptionGroupConfig(final String group) {
|
||||
SubscriptionGroupConfig subscriptionGroupConfig = this.subscriptionGroupTable.get(group);
|
||||
if (null == subscriptionGroupConfig) {
|
||||
if (brokerController.getBrokerConfig().isAutoCreateSubscriptionGroup() || MixAll.isSysConsumerGroup(group)) {
|
||||
subscriptionGroupConfig = new SubscriptionGroupConfig();
|
||||
subscriptionGroupConfig.setGroupName(group);
|
||||
SubscriptionGroupConfig preConfig = this.subscriptionGroupTable.putIfAbsent(group, subscriptionGroupConfig);
|
||||
if (null == preConfig) {
|
||||
log.info("auto create a subscription group, {}", subscriptionGroupConfig.toString());
|
||||
}
|
||||
this.dataVersion.nextVersion();
|
||||
this.persist();
|
||||
}
|
||||
}
|
||||
|
||||
return subscriptionGroupConfig;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public String encode() {
|
||||
return this.encode(false);
|
||||
}
|
||||
|
||||
@Override
|
||||
public String configFilePath() {
|
||||
//return BrokerPathConfigHelper.getSubscriptionGroupPath(this.brokerController.getMessageStoreConfig().getStorePathRootDir());
|
||||
return BrokerPathConfigHelper.getSubscriptionGroupPath(System.getProperty("user.home") + File.separator + "store");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void decode(String jsonString) {
|
||||
if (jsonString != null) {
|
||||
SubscriptionGroupManager obj = RemotingSerializable.fromJson(jsonString, SubscriptionGroupManager.class);
|
||||
if (obj != null) {
|
||||
this.subscriptionGroupTable.putAll(obj.subscriptionGroupTable);
|
||||
this.dataVersion.assignNewOne(obj.dataVersion);
|
||||
this.printLoadDataWhenFirstBoot(obj);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public String encode(final boolean prettyFormat) {
|
||||
return RemotingSerializable.toJson(this, prettyFormat);
|
||||
}
|
||||
|
||||
private void printLoadDataWhenFirstBoot(final SubscriptionGroupManager sgm) {
|
||||
Iterator<Entry<String, SubscriptionGroupConfig>> it = sgm.getSubscriptionGroupTable().entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<String, SubscriptionGroupConfig> next = it.next();
|
||||
log.info("load exist subscription group, {}", next.getValue().toString());
|
||||
}
|
||||
}
|
||||
|
||||
public ConcurrentHashMap<String, SubscriptionGroupConfig> getSubscriptionGroupTable() {
|
||||
return subscriptionGroupTable;
|
||||
}
|
||||
|
||||
|
||||
public DataVersion getDataVersion() {
|
||||
return dataVersion;
|
||||
}
|
||||
|
||||
|
||||
public void deleteSubscriptionGroupConfig(final String groupName) {
|
||||
SubscriptionGroupConfig old = this.subscriptionGroupTable.remove(groupName);
|
||||
if (old != null) {
|
||||
log.info("delete subscription group OK, subscription group: " + old);
|
||||
this.dataVersion.nextVersion();
|
||||
this.persist();
|
||||
} else {
|
||||
log.warn("delete subscription group failed, subscription group: " + old + " not exist");
|
||||
}
|
||||
}
|
||||
}
|
||||
+440
@@ -0,0 +1,440 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.topic;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.broker.BrokerPathConfigHelper;
|
||||
import com.alibaba.rocketmq.common.ConfigManager;
|
||||
import com.alibaba.rocketmq.common.DataVersion;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.TopicConfig;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import com.alibaba.rocketmq.common.constant.PermName;
|
||||
import com.alibaba.rocketmq.common.protocol.body.KVTable;
|
||||
import com.alibaba.rocketmq.common.protocol.body.TopicConfigSerializeWrapper;
|
||||
import com.alibaba.rocketmq.common.sysflag.TopicSysFlag;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.io.File;
|
||||
import java.util.HashSet;
|
||||
import java.util.Iterator;
|
||||
import java.util.Map;
|
||||
import java.util.Map.Entry;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.locks.Lock;
|
||||
import java.util.concurrent.locks.ReentrantLock;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class TopicConfigManager extends ConfigManager {
|
||||
private static final Logger LOG = LoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
|
||||
private static final long LOCK_TIMEOUT_MILLIS = 3000;
|
||||
private transient final Lock lockTopicConfigTable = new ReentrantLock();
|
||||
|
||||
private final ConcurrentHashMap<String, TopicConfig> topicConfigTable =
|
||||
new ConcurrentHashMap<String, TopicConfig>(1024);
|
||||
private final DataVersion dataVersion = new DataVersion();
|
||||
private final Set<String> systemTopicList = new HashSet<String>();
|
||||
private transient BrokerController brokerController;
|
||||
|
||||
|
||||
public TopicConfigManager() {
|
||||
}
|
||||
|
||||
|
||||
public TopicConfigManager(BrokerController brokerController) {
|
||||
this.brokerController = brokerController;
|
||||
{
|
||||
// MixAll.SELF_TEST_TOPIC
|
||||
String topic = MixAll.SELF_TEST_TOPIC;
|
||||
TopicConfig topicConfig = new TopicConfig(topic);
|
||||
this.systemTopicList.add(topic);
|
||||
topicConfig.setReadQueueNums(1);
|
||||
topicConfig.setWriteQueueNums(1);
|
||||
this.topicConfigTable.put(topicConfig.getTopicName(), topicConfig);
|
||||
}
|
||||
{
|
||||
// MixAll.DEFAULT_TOPIC
|
||||
if (this.brokerController.getBrokerConfig().isAutoCreateTopicEnable()) {
|
||||
String topic = MixAll.DEFAULT_TOPIC;
|
||||
TopicConfig topicConfig = new TopicConfig(topic);
|
||||
this.systemTopicList.add(topic);
|
||||
topicConfig.setReadQueueNums(this.brokerController.getBrokerConfig()
|
||||
.getDefaultTopicQueueNums());
|
||||
topicConfig.setWriteQueueNums(this.brokerController.getBrokerConfig()
|
||||
.getDefaultTopicQueueNums());
|
||||
int perm = PermName.PERM_INHERIT | PermName.PERM_READ | PermName.PERM_WRITE;
|
||||
topicConfig.setPerm(perm);
|
||||
this.topicConfigTable.put(topicConfig.getTopicName(), topicConfig);
|
||||
}
|
||||
}
|
||||
{
|
||||
// MixAll.BENCHMARK_TOPIC
|
||||
String topic = MixAll.BENCHMARK_TOPIC;
|
||||
TopicConfig topicConfig = new TopicConfig(topic);
|
||||
this.systemTopicList.add(topic);
|
||||
topicConfig.setReadQueueNums(1024);
|
||||
topicConfig.setWriteQueueNums(1024);
|
||||
this.topicConfigTable.put(topicConfig.getTopicName(), topicConfig);
|
||||
}
|
||||
{
|
||||
|
||||
String topic = this.brokerController.getBrokerConfig().getBrokerClusterName();
|
||||
TopicConfig topicConfig = new TopicConfig(topic);
|
||||
this.systemTopicList.add(topic);
|
||||
int perm = PermName.PERM_INHERIT;
|
||||
if (this.brokerController.getBrokerConfig().isClusterTopicEnable()) {
|
||||
perm |= PermName.PERM_READ | PermName.PERM_WRITE;
|
||||
}
|
||||
topicConfig.setPerm(perm);
|
||||
this.topicConfigTable.put(topicConfig.getTopicName(), topicConfig);
|
||||
}
|
||||
{
|
||||
|
||||
String topic = this.brokerController.getBrokerConfig().getBrokerName();
|
||||
TopicConfig topicConfig = new TopicConfig(topic);
|
||||
this.systemTopicList.add(topic);
|
||||
int perm = PermName.PERM_INHERIT;
|
||||
if (this.brokerController.getBrokerConfig().isBrokerTopicEnable()) {
|
||||
perm |= PermName.PERM_READ | PermName.PERM_WRITE;
|
||||
}
|
||||
topicConfig.setReadQueueNums(1);
|
||||
topicConfig.setWriteQueueNums(1);
|
||||
topicConfig.setPerm(perm);
|
||||
this.topicConfigTable.put(topicConfig.getTopicName(), topicConfig);
|
||||
}
|
||||
{
|
||||
// MixAll.OFFSET_MOVED_EVENT
|
||||
String topic = MixAll.OFFSET_MOVED_EVENT;
|
||||
TopicConfig topicConfig = new TopicConfig(topic);
|
||||
this.systemTopicList.add(topic);
|
||||
topicConfig.setReadQueueNums(1);
|
||||
topicConfig.setWriteQueueNums(1);
|
||||
this.topicConfigTable.put(topicConfig.getTopicName(), topicConfig);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public boolean isSystemTopic(final String topic) {
|
||||
return this.systemTopicList.contains(topic);
|
||||
}
|
||||
|
||||
|
||||
public Set<String> getSystemTopic() {
|
||||
return this.systemTopicList;
|
||||
}
|
||||
|
||||
|
||||
public boolean isTopicCanSendMessage(final String topic) {
|
||||
return !topic.equals(MixAll.DEFAULT_TOPIC);
|
||||
}
|
||||
|
||||
|
||||
public TopicConfig selectTopicConfig(final String topic) {
|
||||
return this.topicConfigTable.get(topic);
|
||||
}
|
||||
|
||||
|
||||
public TopicConfig createTopicInSendMessageMethod(final String topic, final String defaultTopic,
|
||||
final String remoteAddress, final int clientDefaultTopicQueueNums, final int topicSysFlag) {
|
||||
TopicConfig topicConfig = null;
|
||||
boolean createNew = false;
|
||||
|
||||
try {
|
||||
if (this.lockTopicConfigTable.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
|
||||
try {
|
||||
topicConfig = this.topicConfigTable.get(topic);
|
||||
if (topicConfig != null)
|
||||
return topicConfig;
|
||||
|
||||
TopicConfig defaultTopicConfig = this.topicConfigTable.get(defaultTopic);
|
||||
if (defaultTopicConfig != null) {
|
||||
if (defaultTopic.equals(MixAll.DEFAULT_TOPIC)) {
|
||||
if (!this.brokerController.getBrokerConfig().isAutoCreateTopicEnable()) {
|
||||
defaultTopicConfig.setPerm(PermName.PERM_READ | PermName.PERM_WRITE);
|
||||
}
|
||||
}
|
||||
|
||||
if (PermName.isInherited(defaultTopicConfig.getPerm())) {
|
||||
topicConfig = new TopicConfig(topic);
|
||||
|
||||
int queueNums =
|
||||
clientDefaultTopicQueueNums > defaultTopicConfig.getWriteQueueNums() ? defaultTopicConfig
|
||||
.getWriteQueueNums() : clientDefaultTopicQueueNums;
|
||||
|
||||
if (queueNums < 0) {
|
||||
queueNums = 0;
|
||||
}
|
||||
|
||||
topicConfig.setReadQueueNums(queueNums);
|
||||
topicConfig.setWriteQueueNums(queueNums);
|
||||
int perm = defaultTopicConfig.getPerm();
|
||||
perm &= ~PermName.PERM_INHERIT;
|
||||
topicConfig.setPerm(perm);
|
||||
topicConfig.setTopicSysFlag(topicSysFlag);
|
||||
topicConfig.setTopicFilterType(defaultTopicConfig.getTopicFilterType());
|
||||
} else {
|
||||
LOG.warn("create new topic failed, because the default topic[" + defaultTopic
|
||||
+ "] no perm, " + defaultTopicConfig.getPerm() + " producer: "
|
||||
+ remoteAddress);
|
||||
}
|
||||
} else {
|
||||
LOG.warn("create new topic failed, because the default topic[" + defaultTopic
|
||||
+ "] not exist." + " producer: " + remoteAddress);
|
||||
}
|
||||
|
||||
if (topicConfig != null) {
|
||||
LOG.info("create new topic by default topic[" + defaultTopic + "], " + topicConfig
|
||||
+ " producer: " + remoteAddress);
|
||||
|
||||
this.topicConfigTable.put(topic, topicConfig);
|
||||
|
||||
this.dataVersion.nextVersion();
|
||||
|
||||
createNew = true;
|
||||
|
||||
this.persist();
|
||||
}
|
||||
} finally {
|
||||
this.lockTopicConfigTable.unlock();
|
||||
}
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
LOG.error("createTopicInSendMessageMethod exception", e);
|
||||
}
|
||||
|
||||
if (createNew) {
|
||||
this.brokerController.registerBrokerAll(false, true);
|
||||
}
|
||||
|
||||
return topicConfig;
|
||||
}
|
||||
|
||||
public TopicConfig createTopicInSendMessageBackMethod(
|
||||
final String topic,
|
||||
final int clientDefaultTopicQueueNums,
|
||||
final int perm,
|
||||
final int topicSysFlag) {
|
||||
TopicConfig topicConfig = this.topicConfigTable.get(topic);
|
||||
if (topicConfig != null)
|
||||
return topicConfig;
|
||||
|
||||
boolean createNew = false;
|
||||
|
||||
try {
|
||||
if (this.lockTopicConfigTable.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
|
||||
try {
|
||||
topicConfig = this.topicConfigTable.get(topic);
|
||||
if (topicConfig != null)
|
||||
return topicConfig;
|
||||
|
||||
topicConfig = new TopicConfig(topic);
|
||||
topicConfig.setReadQueueNums(clientDefaultTopicQueueNums);
|
||||
topicConfig.setWriteQueueNums(clientDefaultTopicQueueNums);
|
||||
topicConfig.setPerm(perm);
|
||||
topicConfig.setTopicSysFlag(topicSysFlag);
|
||||
|
||||
LOG.info("create new topic {}", topicConfig);
|
||||
this.topicConfigTable.put(topic, topicConfig);
|
||||
createNew = true;
|
||||
this.dataVersion.nextVersion();
|
||||
this.persist();
|
||||
} finally {
|
||||
this.lockTopicConfigTable.unlock();
|
||||
}
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
LOG.error("createTopicInSendMessageBackMethod exception", e);
|
||||
}
|
||||
|
||||
if (createNew) {
|
||||
this.brokerController.registerBrokerAll(false, true);
|
||||
}
|
||||
|
||||
return topicConfig;
|
||||
}
|
||||
|
||||
public void updateTopicUnitFlag(final String topic, final boolean unit) {
|
||||
|
||||
TopicConfig topicConfig = this.topicConfigTable.get(topic);
|
||||
if (topicConfig != null) {
|
||||
int oldTopicSysFlag = topicConfig.getTopicSysFlag();
|
||||
if (unit) {
|
||||
topicConfig.setTopicSysFlag(TopicSysFlag.setUnitFlag(oldTopicSysFlag));
|
||||
} else {
|
||||
topicConfig.setTopicSysFlag(TopicSysFlag.clearUnitFlag(oldTopicSysFlag));
|
||||
}
|
||||
|
||||
LOG.info("update topic sys flag. oldTopicSysFlag={}, newTopicSysFlag", oldTopicSysFlag,
|
||||
topicConfig.getTopicSysFlag());
|
||||
|
||||
this.topicConfigTable.put(topic, topicConfig);
|
||||
|
||||
this.dataVersion.nextVersion();
|
||||
|
||||
this.persist();
|
||||
this.brokerController.registerBrokerAll(false, true);
|
||||
}
|
||||
}
|
||||
|
||||
public void updateTopicUnitSubFlag(final String topic, final boolean hasUnitSub) {
|
||||
TopicConfig topicConfig = this.topicConfigTable.get(topic);
|
||||
if (topicConfig != null) {
|
||||
int oldTopicSysFlag = topicConfig.getTopicSysFlag();
|
||||
if (hasUnitSub) {
|
||||
topicConfig.setTopicSysFlag(TopicSysFlag.setUnitSubFlag(oldTopicSysFlag));
|
||||
}
|
||||
|
||||
LOG.info("update topic sys flag. oldTopicSysFlag={}, newTopicSysFlag", oldTopicSysFlag,
|
||||
topicConfig.getTopicSysFlag());
|
||||
|
||||
this.topicConfigTable.put(topic, topicConfig);
|
||||
|
||||
this.dataVersion.nextVersion();
|
||||
|
||||
this.persist();
|
||||
this.brokerController.registerBrokerAll(false, true);
|
||||
}
|
||||
}
|
||||
|
||||
public void updateTopicConfig(final TopicConfig topicConfig) {
|
||||
TopicConfig old = this.topicConfigTable.put(topicConfig.getTopicName(), topicConfig);
|
||||
if (old != null) {
|
||||
LOG.info("update topic config, old: " + old + " new: " + topicConfig);
|
||||
} else {
|
||||
LOG.info("create new topic, " + topicConfig);
|
||||
}
|
||||
|
||||
this.dataVersion.nextVersion();
|
||||
|
||||
this.persist();
|
||||
}
|
||||
|
||||
|
||||
public void updateOrderTopicConfig(final KVTable orderKVTableFromNs) {
|
||||
|
||||
if (orderKVTableFromNs != null && orderKVTableFromNs.getTable() != null) {
|
||||
boolean isChange = false;
|
||||
Set<String> orderTopics = orderKVTableFromNs.getTable().keySet();
|
||||
for (String topic : orderTopics) {
|
||||
TopicConfig topicConfig = this.topicConfigTable.get(topic);
|
||||
if (topicConfig != null && !topicConfig.isOrder()) {
|
||||
topicConfig.setOrder(true);
|
||||
isChange = true;
|
||||
LOG.info("update order topic config, topic={}, order={}", topic, true);
|
||||
}
|
||||
}
|
||||
|
||||
for (Map.Entry<String, TopicConfig> entry : this.topicConfigTable.entrySet()) {
|
||||
String topic = entry.getKey();
|
||||
if (!orderTopics.contains(topic)) {
|
||||
TopicConfig topicConfig = entry.getValue();
|
||||
if (topicConfig.isOrder()) {
|
||||
topicConfig.setOrder(false);
|
||||
isChange = true;
|
||||
LOG.info("update order topic config, topic={}, order={}", topic, false);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (isChange) {
|
||||
this.dataVersion.nextVersion();
|
||||
this.persist();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public boolean isOrderTopic(final String topic) {
|
||||
TopicConfig topicConfig = this.topicConfigTable.get(topic);
|
||||
if (topicConfig == null) {
|
||||
return false;
|
||||
} else {
|
||||
return topicConfig.isOrder();
|
||||
}
|
||||
}
|
||||
|
||||
public void deleteTopicConfig(final String topic) {
|
||||
TopicConfig old = this.topicConfigTable.remove(topic);
|
||||
if (old != null) {
|
||||
LOG.info("delete topic config OK, topic: " + old);
|
||||
this.dataVersion.nextVersion();
|
||||
this.persist();
|
||||
} else {
|
||||
LOG.warn("delete topic config failed, topic: " + topic + " not exist");
|
||||
}
|
||||
}
|
||||
|
||||
public TopicConfigSerializeWrapper buildTopicConfigSerializeWrapper() {
|
||||
TopicConfigSerializeWrapper topicConfigSerializeWrapper = new TopicConfigSerializeWrapper();
|
||||
topicConfigSerializeWrapper.setTopicConfigTable(this.topicConfigTable);
|
||||
topicConfigSerializeWrapper.setDataVersion(this.dataVersion);
|
||||
return topicConfigSerializeWrapper;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String encode() {
|
||||
return encode(false);
|
||||
}
|
||||
|
||||
@Override
|
||||
public String configFilePath() {
|
||||
// return BrokerPathConfigHelper.getTopicConfigPath(this.brokerController.getMessageStoreConfig()
|
||||
// .getStorePathRootDir());
|
||||
return BrokerPathConfigHelper.getTopicConfigPath(System.getProperty("user.home") + File.separator + "store");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void decode(String jsonString) {
|
||||
if (jsonString != null) {
|
||||
TopicConfigSerializeWrapper topicConfigSerializeWrapper =
|
||||
TopicConfigSerializeWrapper.fromJson(jsonString, TopicConfigSerializeWrapper.class);
|
||||
if (topicConfigSerializeWrapper != null) {
|
||||
this.topicConfigTable.putAll(topicConfigSerializeWrapper.getTopicConfigTable());
|
||||
this.dataVersion.assignNewOne(topicConfigSerializeWrapper.getDataVersion());
|
||||
this.printLoadDataWhenFirstBoot(topicConfigSerializeWrapper);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public String encode(final boolean prettyFormat) {
|
||||
TopicConfigSerializeWrapper topicConfigSerializeWrapper = new TopicConfigSerializeWrapper();
|
||||
topicConfigSerializeWrapper.setTopicConfigTable(this.topicConfigTable);
|
||||
topicConfigSerializeWrapper.setDataVersion(this.dataVersion);
|
||||
return topicConfigSerializeWrapper.toJson(prettyFormat);
|
||||
}
|
||||
|
||||
private void printLoadDataWhenFirstBoot(final TopicConfigSerializeWrapper tcs) {
|
||||
Iterator<Entry<String, TopicConfig>> it = tcs.getTopicConfigTable().entrySet().iterator();
|
||||
while (it.hasNext()) {
|
||||
Entry<String, TopicConfig> next = it.next();
|
||||
LOG.info("load exist local topic, {}", next.getValue().toString());
|
||||
}
|
||||
}
|
||||
|
||||
public DataVersion getDataVersion() {
|
||||
return dataVersion;
|
||||
}
|
||||
|
||||
public ConcurrentHashMap<String, TopicConfig> getTopicConfigTable() {
|
||||
return topicConfigTable;
|
||||
}
|
||||
}
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker.transaction;
|
||||
|
||||
public class TransactionRecord {
|
||||
// Commit Log Offset
|
||||
private long offset;
|
||||
private String producerGroup;
|
||||
|
||||
|
||||
public long getOffset() {
|
||||
return offset;
|
||||
}
|
||||
|
||||
|
||||
public void setOffset(long offset) {
|
||||
this.offset = offset;
|
||||
}
|
||||
|
||||
|
||||
public String getProducerGroup() {
|
||||
return producerGroup;
|
||||
}
|
||||
|
||||
|
||||
public void setProducerGroup(String producerGroup) {
|
||||
this.producerGroup = producerGroup;
|
||||
}
|
||||
}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker.transaction;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
|
||||
public interface TransactionStore {
|
||||
public boolean open();
|
||||
|
||||
|
||||
public void close();
|
||||
|
||||
|
||||
public boolean put(final List<TransactionRecord> trs);
|
||||
|
||||
|
||||
public void remove(final List<Long> pks);
|
||||
|
||||
|
||||
public List<TransactionRecord> traverse(final long pk, final int nums);
|
||||
|
||||
|
||||
public long totalRecords();
|
||||
|
||||
|
||||
public long minPK();
|
||||
|
||||
|
||||
public long maxPK();
|
||||
}
|
||||
+240
@@ -0,0 +1,240 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker.transaction.jdbc;
|
||||
|
||||
import com.alibaba.rocketmq.broker.transaction.TransactionRecord;
|
||||
import com.alibaba.rocketmq.broker.transaction.TransactionStore;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.constant.LoggerName;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.net.URL;
|
||||
import java.sql.*;
|
||||
import java.util.List;
|
||||
import java.util.Properties;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
|
||||
|
||||
public class JDBCTransactionStore implements TransactionStore {
|
||||
private static final Logger log = LoggerFactory.getLogger(LoggerName.TRANSACTION_LOGGER_NAME);
|
||||
private final JDBCTransactionStoreConfig jdbcTransactionStoreConfig;
|
||||
private Connection connection;
|
||||
private AtomicLong totalRecordsValue = new AtomicLong(0);
|
||||
|
||||
public JDBCTransactionStore(JDBCTransactionStoreConfig jdbcTransactionStoreConfig) {
|
||||
this.jdbcTransactionStoreConfig = jdbcTransactionStoreConfig;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean open() {
|
||||
if (this.loadDriver()) {
|
||||
Properties props = new Properties();
|
||||
props.put("user", jdbcTransactionStoreConfig.getJdbcUser());
|
||||
props.put("password", jdbcTransactionStoreConfig.getJdbcPassword());
|
||||
|
||||
try {
|
||||
this.connection =
|
||||
DriverManager.getConnection(this.jdbcTransactionStoreConfig.getJdbcURL(), props);
|
||||
|
||||
this.connection.setAutoCommit(false);
|
||||
|
||||
|
||||
if (!this.computeTotalRecords()) {
|
||||
return this.createDB();
|
||||
}
|
||||
|
||||
return true;
|
||||
} catch (SQLException e) {
|
||||
log.info("Create JDBC Connection Exeption", e);
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
private boolean loadDriver() {
|
||||
try {
|
||||
Class.forName(this.jdbcTransactionStoreConfig.getJdbcDriverClass()).newInstance();
|
||||
log.info("Loaded the appropriate driver, {}",
|
||||
this.jdbcTransactionStoreConfig.getJdbcDriverClass());
|
||||
return true;
|
||||
} catch (Exception e) {
|
||||
log.info("Loaded the appropriate driver Exception", e);
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
private boolean computeTotalRecords() {
|
||||
Statement statement = null;
|
||||
ResultSet resultSet = null;
|
||||
try {
|
||||
statement = this.connection.createStatement();
|
||||
|
||||
resultSet = statement.executeQuery("select count(offset) as total from t_transaction");
|
||||
if (!resultSet.next()) {
|
||||
log.warn("computeTotalRecords ResultSet is empty");
|
||||
return false;
|
||||
}
|
||||
|
||||
this.totalRecordsValue.set(resultSet.getLong(1));
|
||||
} catch (Exception e) {
|
||||
log.warn("computeTotalRecords Exception", e);
|
||||
return false;
|
||||
} finally {
|
||||
if (null != statement) {
|
||||
try {
|
||||
statement.close();
|
||||
} catch (SQLException e) {
|
||||
}
|
||||
}
|
||||
|
||||
if (null != resultSet) {
|
||||
try {
|
||||
resultSet.close();
|
||||
} catch (SQLException e) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private boolean createDB() {
|
||||
Statement statement = null;
|
||||
try {
|
||||
statement = this.connection.createStatement();
|
||||
|
||||
String sql = this.createTableSql();
|
||||
log.info("createDB SQL:\n {}", sql);
|
||||
statement.execute(sql);
|
||||
this.connection.commit();
|
||||
return true;
|
||||
} catch (Exception e) {
|
||||
log.warn("createDB Exception", e);
|
||||
return false;
|
||||
} finally {
|
||||
if (null != statement) {
|
||||
try {
|
||||
statement.close();
|
||||
} catch (SQLException e) {
|
||||
log.warn("Close statement exception", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private String createTableSql() {
|
||||
URL resource = JDBCTransactionStore.class.getClassLoader().getResource("transaction.sql");
|
||||
String fileContent = MixAll.file2String(resource);
|
||||
return fileContent;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
try {
|
||||
if (this.connection != null) {
|
||||
this.connection.close();
|
||||
}
|
||||
} catch (SQLException e) {
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean put(List<TransactionRecord> trs) {
|
||||
PreparedStatement statement = null;
|
||||
try {
|
||||
this.connection.setAutoCommit(false);
|
||||
statement = this.connection.prepareStatement("insert into t_transaction values (?, ?)");
|
||||
for (TransactionRecord tr : trs) {
|
||||
statement.setLong(1, tr.getOffset());
|
||||
statement.setString(2, tr.getProducerGroup());
|
||||
statement.addBatch();
|
||||
}
|
||||
int[] executeBatch = statement.executeBatch();
|
||||
this.connection.commit();
|
||||
this.totalRecordsValue.addAndGet(updatedRows(executeBatch));
|
||||
return true;
|
||||
} catch (Exception e) {
|
||||
log.warn("createDB Exception", e);
|
||||
return false;
|
||||
} finally {
|
||||
if (null != statement) {
|
||||
try {
|
||||
statement.close();
|
||||
} catch (SQLException e) {
|
||||
log.warn("Close statement exception", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private long updatedRows(int[] rows) {
|
||||
long res = 0;
|
||||
for (int i : rows) {
|
||||
res += i;
|
||||
}
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void remove(List<Long> pks) {
|
||||
PreparedStatement statement = null;
|
||||
try {
|
||||
this.connection.setAutoCommit(false);
|
||||
statement = this.connection.prepareStatement("DELETE FROM t_transaction WHERE offset = ?");
|
||||
for (long pk : pks) {
|
||||
statement.setLong(1, pk);
|
||||
statement.addBatch();
|
||||
}
|
||||
int[] executeBatch = statement.executeBatch();
|
||||
this.connection.commit();
|
||||
} catch (Exception e) {
|
||||
log.warn("createDB Exception", e);
|
||||
} finally {
|
||||
if (null != statement) {
|
||||
try {
|
||||
statement.close();
|
||||
} catch (SQLException e) {
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public List<TransactionRecord> traverse(long pk, int nums) {
|
||||
return null;
|
||||
}
|
||||
|
||||
@Override
|
||||
public long totalRecords() {
|
||||
return this.totalRecordsValue.get();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long minPK() {
|
||||
return 0;
|
||||
}
|
||||
|
||||
@Override
|
||||
public long maxPK() {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.broker.transaction.jdbc;
|
||||
|
||||
public class JDBCTransactionStoreConfig {
|
||||
private String jdbcDriverClass = "com.mysql.jdbc.Driver";
|
||||
private String jdbcURL = "jdbc:mysql://xxx.xxx.xxx.xxx:1000/xxx?useUnicode=true&characterEncoding=UTF-8";
|
||||
private String jdbcUser = "xxx";
|
||||
private String jdbcPassword = "xxx";
|
||||
|
||||
|
||||
public String getJdbcDriverClass() {
|
||||
return jdbcDriverClass;
|
||||
}
|
||||
|
||||
|
||||
public void setJdbcDriverClass(String jdbcDriverClass) {
|
||||
this.jdbcDriverClass = jdbcDriverClass;
|
||||
}
|
||||
|
||||
|
||||
public String getJdbcURL() {
|
||||
return jdbcURL;
|
||||
}
|
||||
|
||||
|
||||
public void setJdbcURL(String jdbcURL) {
|
||||
this.jdbcURL = jdbcURL;
|
||||
}
|
||||
|
||||
|
||||
public String getJdbcUser() {
|
||||
return jdbcUser;
|
||||
}
|
||||
|
||||
|
||||
public void setJdbcUser(String jdbcUser) {
|
||||
this.jdbcUser = jdbcUser;
|
||||
}
|
||||
|
||||
|
||||
public String getJdbcPassword() {
|
||||
return jdbcPassword;
|
||||
}
|
||||
|
||||
|
||||
public void setJdbcPassword(String jdbcPassword) {
|
||||
this.jdbcPassword = jdbcPassword;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
CREATE TABLE t_transaction(
|
||||
offset NUMERIC(20) PRIMARY KEY,
|
||||
producerGroup VARCHAR(64)
|
||||
)
|
||||
@@ -0,0 +1,87 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
/**
|
||||
* $Id: SendMessageTest.java 1831 2013-05-16 01:39:51Z shijia.wxr $
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.api;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.client.hook.SendMessageContext;
|
||||
import com.alibaba.rocketmq.client.impl.CommunicationMode;
|
||||
import com.alibaba.rocketmq.client.impl.MQClientAPIImpl;
|
||||
import com.alibaba.rocketmq.client.producer.SendResult;
|
||||
import com.alibaba.rocketmq.common.BrokerConfig;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.message.Message;
|
||||
import com.alibaba.rocketmq.common.message.MessageDecoder;
|
||||
import com.alibaba.rocketmq.common.protocol.header.SendMessageRequestHeader;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyClientConfig;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyServerConfig;
|
||||
import com.alibaba.rocketmq.store.config.MessageStoreConfig;
|
||||
import org.junit.Test;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class SendMessageTest {
|
||||
@Test
|
||||
public void test_sendMessage() throws Exception {
|
||||
BrokerController brokerController = new BrokerController(//
|
||||
new BrokerConfig(), //
|
||||
new NettyServerConfig(), //
|
||||
new NettyClientConfig(), //
|
||||
new MessageStoreConfig());
|
||||
boolean initResult = brokerController.initialize();
|
||||
System.out.println("initialize " + initResult);
|
||||
|
||||
brokerController.start();
|
||||
|
||||
MQClientAPIImpl client = new MQClientAPIImpl(new NettyClientConfig(), null, null, null);
|
||||
client.start();
|
||||
|
||||
for (int i = 0; i < 100; i++) {
|
||||
String topic = "UnitTestTopic_" + i % 3;
|
||||
Message msg = new Message(topic, "TAG1 TAG2", "100200300", ("Hello, Nice world\t" + i).getBytes());
|
||||
msg.setDelayTimeLevel(i % 3 + 1);
|
||||
|
||||
try {
|
||||
SendMessageRequestHeader requestHeader = new SendMessageRequestHeader();
|
||||
requestHeader.setProducerGroup("abc");
|
||||
requestHeader.setTopic(msg.getTopic());
|
||||
requestHeader.setDefaultTopic(MixAll.DEFAULT_TOPIC);
|
||||
requestHeader.setDefaultTopicQueueNums(4);
|
||||
requestHeader.setQueueId(i % 4);
|
||||
requestHeader.setSysFlag(0);
|
||||
requestHeader.setBornTimestamp(System.currentTimeMillis());
|
||||
requestHeader.setFlag(msg.getFlag());
|
||||
requestHeader.setProperties(MessageDecoder.messageProperties2String(msg.getProperties()));
|
||||
|
||||
SendResult result = client.sendMessage("127.0.0.1:10911", "brokerName", msg, requestHeader, 1000 * 5,
|
||||
CommunicationMode.SYNC, new SendMessageContext(), null);
|
||||
System.out.println(i + "\t" + result);
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
client.shutdown();
|
||||
|
||||
brokerController.shutdown();
|
||||
}
|
||||
}
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
/**
|
||||
* $Id: ConsumerOffsetManagerTest.java 1831 2013-05-16 01:39:51Z shijia.wxr $
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.offset;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.common.BrokerConfig;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyClientConfig;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyServerConfig;
|
||||
import com.alibaba.rocketmq.store.config.MessageStoreConfig;
|
||||
import org.junit.Test;
|
||||
|
||||
import java.util.Random;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class ConsumerOffsetManagerTest {
|
||||
@Test
|
||||
public void test_flushConsumerOffset() throws Exception {
|
||||
BrokerController brokerController = new BrokerController(//
|
||||
new BrokerConfig(), //
|
||||
new NettyServerConfig(), //
|
||||
new NettyClientConfig(), //
|
||||
new MessageStoreConfig());
|
||||
boolean initResult = brokerController.initialize();
|
||||
System.out.println("initialize " + initResult);
|
||||
brokerController.start();
|
||||
|
||||
ConsumerOffsetManager consumerOffsetManager = new ConsumerOffsetManager(brokerController);
|
||||
|
||||
Random random = new Random();
|
||||
|
||||
for (int i = 0; i < 100; i++) {
|
||||
String group = "DIANPU_GROUP_" + i;
|
||||
for (int id = 0; id < 16; id++) {
|
||||
consumerOffsetManager.commitOffset(null, group, "TOPIC_A", id,
|
||||
random.nextLong() % 1024 * 1024 * 1024);
|
||||
consumerOffsetManager.commitOffset(null, group, "TOPIC_B", id,
|
||||
random.nextLong() % 1024 * 1024 * 1024);
|
||||
consumerOffsetManager.commitOffset(null, group, "TOPIC_C", id,
|
||||
random.nextLong() % 1024 * 1024 * 1024);
|
||||
}
|
||||
}
|
||||
|
||||
consumerOffsetManager.persist();
|
||||
|
||||
brokerController.shutdown();
|
||||
}
|
||||
}
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
/**
|
||||
* $Id: TopicConfigManagerTest.java 1831 2013-05-16 01:39:51Z shijia.wxr $
|
||||
*/
|
||||
package com.alibaba.rocketmq.broker.topic;
|
||||
|
||||
import com.alibaba.rocketmq.broker.BrokerController;
|
||||
import com.alibaba.rocketmq.common.BrokerConfig;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.TopicConfig;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyClientConfig;
|
||||
import com.alibaba.rocketmq.remoting.netty.NettyServerConfig;
|
||||
import com.alibaba.rocketmq.store.config.MessageStoreConfig;
|
||||
import org.junit.Test;
|
||||
|
||||
import static org.junit.Assert.assertTrue;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class TopicConfigManagerTest {
|
||||
@Test
|
||||
public void test_flushTopicConfig() throws Exception {
|
||||
BrokerController brokerController = new BrokerController(//
|
||||
new BrokerConfig(), //
|
||||
new NettyServerConfig(), //
|
||||
new NettyClientConfig(), //
|
||||
new MessageStoreConfig());
|
||||
boolean initResult = brokerController.initialize();
|
||||
System.out.println("initialize " + initResult);
|
||||
brokerController.start();
|
||||
|
||||
TopicConfigManager topicConfigManager = new TopicConfigManager(brokerController);
|
||||
|
||||
TopicConfig topicConfig =
|
||||
topicConfigManager.createTopicInSendMessageMethod("TestTopic_SEND", MixAll.DEFAULT_TOPIC,
|
||||
null, 4, 0);
|
||||
assertTrue(topicConfig != null);
|
||||
|
||||
System.out.println(topicConfig);
|
||||
|
||||
for (int i = 0; i < 10; i++) {
|
||||
String topic = "UNITTEST-" + i;
|
||||
topicConfig =
|
||||
topicConfigManager
|
||||
.createTopicInSendMessageMethod(topic, MixAll.DEFAULT_TOPIC, null, 4, 0);
|
||||
assertTrue(topicConfig != null);
|
||||
}
|
||||
|
||||
topicConfigManager.persist();
|
||||
|
||||
brokerController.shutdown();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
mvn -Dmaven.test.skip=true deploy -Pclient-shade
|
||||
@@ -0,0 +1,2 @@
|
||||
mvn -Dmaven.test.skip=true clean package install -Pclient-shade -U
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain producerGroup copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<parent>
|
||||
<groupId>com.alibaba.rocketmq</groupId>
|
||||
<artifactId>rocketmq-all</artifactId>
|
||||
<version>4.0.0-SNAPSHOT</version>
|
||||
</parent>
|
||||
|
||||
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<packaging>jar</packaging>
|
||||
<artifactId>rocketmq-client</artifactId>
|
||||
<name>rocketmq-client ${project.version}</name>
|
||||
|
||||
<profiles>
|
||||
<profile>
|
||||
<id>client-shade</id>
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-shade-plugin</artifactId>
|
||||
<version>2.4.3</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<phase>package</phase>
|
||||
<goals>
|
||||
<goal>shade</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
<configuration>
|
||||
<keepDependenciesWithProvidedScope>false</keepDependenciesWithProvidedScope>
|
||||
<promoteTransitiveDependencies>false</promoteTransitiveDependencies>
|
||||
<createDependencyReducedPom>true</createDependencyReducedPom>
|
||||
<minimizeJar>false</minimizeJar>
|
||||
<createSourcesJar>true</createSourcesJar>
|
||||
<artifactSet>
|
||||
<includes>
|
||||
<include>com.alibaba:fastjson</include>
|
||||
<include>io.netty:netty-all</include>
|
||||
<include>com.alibaba.rocketmq:rocketmq-client</include>
|
||||
<include>com.alibaba.rocketmq:rocketmq-common</include>
|
||||
<include>com.alibaba.rocketmq:rocketmq-remoting</include>
|
||||
</includes>
|
||||
</artifactSet>
|
||||
<relocations>
|
||||
<relocation>
|
||||
<pattern>io.netty</pattern>
|
||||
<shadedPattern>com.alibaba.rocketmq.shade.io.netty</shadedPattern>
|
||||
</relocation>
|
||||
<relocation>
|
||||
<pattern>com.alibaba.fastjson</pattern>
|
||||
<shadedPattern>com.alibaba.rocketmq.shade.com.alibaba.fastjson</shadedPattern>
|
||||
</relocation>
|
||||
</relocations>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</profile>
|
||||
</profiles>
|
||||
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>junit</groupId>
|
||||
<artifactId>junit</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>rocketmq-common</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.slf4j</groupId>
|
||||
<artifactId>slf4j-api</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</project>
|
||||
@@ -0,0 +1,202 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.client;
|
||||
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.UtilAll;
|
||||
import com.alibaba.rocketmq.remoting.common.RemotingUtil;
|
||||
|
||||
|
||||
/**
|
||||
* Client Common configuration
|
||||
*
|
||||
* @author shijia.wxr
|
||||
* @author vongosling
|
||||
*/
|
||||
public class ClientConfig {
|
||||
public static final String SEND_MESSAGE_WITH_VIP_CHANNEL_PROPERTY = "com.rocketmq.sendMessageWithVIPChannel";
|
||||
private String namesrvAddr = System.getProperty(MixAll.NAMESRV_ADDR_PROPERTY, System.getenv(MixAll.NAMESRV_ADDR_ENV));
|
||||
private String clientIP = RemotingUtil.getLocalAddress();
|
||||
private String instanceName = System.getProperty("rocketmq.client.name", "DEFAULT");
|
||||
private int clientCallbackExecutorThreads = Runtime.getRuntime().availableProcessors();
|
||||
/**
|
||||
* Pulling topic information interval from the named server
|
||||
*/
|
||||
private int pollNameServerInteval = 1000 * 30;
|
||||
/**
|
||||
* Heartbeat interval in microseconds with message broker
|
||||
*/
|
||||
private int heartbeatBrokerInterval = 1000 * 30;
|
||||
/**
|
||||
* Offset persistent interval for consumer
|
||||
*/
|
||||
private int persistConsumerOffsetInterval = 1000 * 5;
|
||||
private boolean unitMode = false;
|
||||
private String unitName;
|
||||
private boolean vipChannelEnabled = Boolean.parseBoolean(System.getProperty(SEND_MESSAGE_WITH_VIP_CHANNEL_PROPERTY, "true"));
|
||||
|
||||
public String buildMQClientId() {
|
||||
StringBuilder sb = new StringBuilder();
|
||||
sb.append(this.getClientIP());
|
||||
|
||||
sb.append("@");
|
||||
sb.append(this.getInstanceName());
|
||||
if (!UtilAll.isBlank(this.unitName)) {
|
||||
sb.append("@");
|
||||
sb.append(this.unitName);
|
||||
}
|
||||
|
||||
return sb.toString();
|
||||
}
|
||||
|
||||
public String getClientIP() {
|
||||
return clientIP;
|
||||
}
|
||||
|
||||
public void setClientIP(String clientIP) {
|
||||
this.clientIP = clientIP;
|
||||
}
|
||||
|
||||
public String getInstanceName() {
|
||||
return instanceName;
|
||||
}
|
||||
|
||||
public void setInstanceName(String instanceName) {
|
||||
this.instanceName = instanceName;
|
||||
}
|
||||
|
||||
public void changeInstanceNameToPID() {
|
||||
if (this.instanceName.equals("DEFAULT")) {
|
||||
this.instanceName = String.valueOf(UtilAll.getPid());
|
||||
}
|
||||
}
|
||||
|
||||
public void resetClientConfig(final ClientConfig cc) {
|
||||
this.namesrvAddr = cc.namesrvAddr;
|
||||
this.clientIP = cc.clientIP;
|
||||
this.instanceName = cc.instanceName;
|
||||
this.clientCallbackExecutorThreads = cc.clientCallbackExecutorThreads;
|
||||
this.pollNameServerInteval = cc.pollNameServerInteval;
|
||||
this.heartbeatBrokerInterval = cc.heartbeatBrokerInterval;
|
||||
this.persistConsumerOffsetInterval = cc.persistConsumerOffsetInterval;
|
||||
this.unitMode = cc.unitMode;
|
||||
this.unitName = cc.unitName;
|
||||
this.vipChannelEnabled = cc.vipChannelEnabled;
|
||||
}
|
||||
|
||||
public ClientConfig cloneClientConfig() {
|
||||
ClientConfig cc = new ClientConfig();
|
||||
cc.namesrvAddr = namesrvAddr;
|
||||
cc.clientIP = clientIP;
|
||||
cc.instanceName = instanceName;
|
||||
cc.clientCallbackExecutorThreads = clientCallbackExecutorThreads;
|
||||
cc.pollNameServerInteval = pollNameServerInteval;
|
||||
cc.heartbeatBrokerInterval = heartbeatBrokerInterval;
|
||||
cc.persistConsumerOffsetInterval = persistConsumerOffsetInterval;
|
||||
cc.unitMode = unitMode;
|
||||
cc.unitName = unitName;
|
||||
cc.vipChannelEnabled = vipChannelEnabled;
|
||||
return cc;
|
||||
}
|
||||
|
||||
public String getNamesrvAddr() {
|
||||
return namesrvAddr;
|
||||
}
|
||||
|
||||
public void setNamesrvAddr(String namesrvAddr) {
|
||||
this.namesrvAddr = namesrvAddr;
|
||||
}
|
||||
|
||||
public int getClientCallbackExecutorThreads() {
|
||||
return clientCallbackExecutorThreads;
|
||||
}
|
||||
|
||||
|
||||
public void setClientCallbackExecutorThreads(int clientCallbackExecutorThreads) {
|
||||
this.clientCallbackExecutorThreads = clientCallbackExecutorThreads;
|
||||
}
|
||||
|
||||
|
||||
public int getPollNameServerInteval() {
|
||||
return pollNameServerInteval;
|
||||
}
|
||||
|
||||
|
||||
public void setPollNameServerInteval(int pollNameServerInteval) {
|
||||
this.pollNameServerInteval = pollNameServerInteval;
|
||||
}
|
||||
|
||||
|
||||
public int getHeartbeatBrokerInterval() {
|
||||
return heartbeatBrokerInterval;
|
||||
}
|
||||
|
||||
|
||||
public void setHeartbeatBrokerInterval(int heartbeatBrokerInterval) {
|
||||
this.heartbeatBrokerInterval = heartbeatBrokerInterval;
|
||||
}
|
||||
|
||||
|
||||
public int getPersistConsumerOffsetInterval() {
|
||||
return persistConsumerOffsetInterval;
|
||||
}
|
||||
|
||||
|
||||
public void setPersistConsumerOffsetInterval(int persistConsumerOffsetInterval) {
|
||||
this.persistConsumerOffsetInterval = persistConsumerOffsetInterval;
|
||||
}
|
||||
|
||||
|
||||
public String getUnitName() {
|
||||
return unitName;
|
||||
}
|
||||
|
||||
|
||||
public void setUnitName(String unitName) {
|
||||
this.unitName = unitName;
|
||||
}
|
||||
|
||||
|
||||
public boolean isUnitMode() {
|
||||
return unitMode;
|
||||
}
|
||||
|
||||
|
||||
public void setUnitMode(boolean unitMode) {
|
||||
this.unitMode = unitMode;
|
||||
}
|
||||
|
||||
|
||||
public boolean isVipChannelEnabled() {
|
||||
return vipChannelEnabled;
|
||||
}
|
||||
|
||||
|
||||
public void setVipChannelEnabled(final boolean vipChannelEnabled) {
|
||||
this.vipChannelEnabled = vipChannelEnabled;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return "ClientConfig [namesrvAddr=" + namesrvAddr + ", clientIP=" + clientIP + ", instanceName=" + instanceName
|
||||
+ ", clientCallbackExecutorThreads=" + clientCallbackExecutorThreads + ", pollNameServerInteval=" + pollNameServerInteval
|
||||
+ ", heartbeatBrokerInterval=" + heartbeatBrokerInterval + ", persistConsumerOffsetInterval="
|
||||
+ persistConsumerOffsetInterval + ", unitMode=" + unitMode + ", unitName=" + unitName + ", vipChannelEnabled="
|
||||
+ vipChannelEnabled + "]";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,173 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.client;
|
||||
|
||||
import com.alibaba.rocketmq.client.exception.MQBrokerException;
|
||||
import com.alibaba.rocketmq.client.exception.MQClientException;
|
||||
import com.alibaba.rocketmq.common.message.MessageExt;
|
||||
import com.alibaba.rocketmq.common.message.MessageQueue;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingException;
|
||||
|
||||
|
||||
/**
|
||||
* Base interface for MQ management
|
||||
*
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public interface MQAdmin {
|
||||
/**
|
||||
* Creates an topic
|
||||
*
|
||||
* @param key
|
||||
* accesskey
|
||||
* @param newTopic
|
||||
* topic name
|
||||
* @param queueNum
|
||||
* topic's queue number
|
||||
*
|
||||
* @throws MQClientException
|
||||
*/
|
||||
void createTopic(final String key, final String newTopic, final int queueNum)
|
||||
throws MQClientException;
|
||||
|
||||
|
||||
/**
|
||||
* Creates an topic
|
||||
*
|
||||
* @param key
|
||||
* accesskey
|
||||
* @param newTopic
|
||||
* topic name
|
||||
* @param queueNum
|
||||
* topic's queue number
|
||||
* @param topicSysFlag
|
||||
* topic system flag
|
||||
*
|
||||
* @throws MQClientException
|
||||
*/
|
||||
void createTopic(String key, String newTopic, int queueNum, int topicSysFlag)
|
||||
throws MQClientException;
|
||||
|
||||
|
||||
/**
|
||||
* Gets the message queue offset according to some time in milliseconds<br>
|
||||
* be cautious to call because of more IO overhead
|
||||
*
|
||||
* @param mq
|
||||
* Instance of MessageQueue
|
||||
* @param timestamp
|
||||
* from when in milliseconds.
|
||||
*
|
||||
* @return offset
|
||||
*
|
||||
* @throws MQClientException
|
||||
*/
|
||||
long searchOffset(final MessageQueue mq, final long timestamp) throws MQClientException;
|
||||
|
||||
|
||||
/**
|
||||
* Gets the max offset
|
||||
*
|
||||
* @param mq
|
||||
* Instance of MessageQueue
|
||||
*
|
||||
* @return the max offset
|
||||
*
|
||||
* @throws MQClientException
|
||||
*/
|
||||
long maxOffset(final MessageQueue mq) throws MQClientException;
|
||||
|
||||
|
||||
/**
|
||||
* Gets the minimum offset
|
||||
*
|
||||
* @param mq
|
||||
* Instance of MessageQueue
|
||||
*
|
||||
* @return the minimum offset
|
||||
*
|
||||
* @throws MQClientException
|
||||
*/
|
||||
long minOffset(final MessageQueue mq) throws MQClientException;
|
||||
|
||||
|
||||
/**
|
||||
* Gets the earliest stored message time
|
||||
*
|
||||
* @param mq
|
||||
* Instance of MessageQueue
|
||||
*
|
||||
* @return the time in microseconds
|
||||
*
|
||||
* @throws MQClientException
|
||||
*/
|
||||
long earliestMsgStoreTime(final MessageQueue mq) throws MQClientException;
|
||||
|
||||
|
||||
/**
|
||||
* Query message according tto message id
|
||||
*
|
||||
* @param offsetMsgId
|
||||
* message id
|
||||
*
|
||||
* @return message
|
||||
*
|
||||
* @throws InterruptedException
|
||||
* @throws MQBrokerException
|
||||
* @throws RemotingException
|
||||
* @throws MQClientException
|
||||
*/
|
||||
MessageExt viewMessage(final String offsetMsgId) throws RemotingException, MQBrokerException,
|
||||
InterruptedException, MQClientException;
|
||||
|
||||
|
||||
/**
|
||||
* Query messages
|
||||
*
|
||||
* @param topic
|
||||
* message topic
|
||||
* @param key
|
||||
* message key index word
|
||||
* @param maxNum
|
||||
* max message number
|
||||
* @param begin
|
||||
* from when
|
||||
* @param end
|
||||
* to when
|
||||
*
|
||||
* @return Instance of QueryResult
|
||||
*
|
||||
* @throws MQClientException
|
||||
* @throws InterruptedException
|
||||
*/
|
||||
QueryResult queryMessage(final String topic, final String key, final int maxNum, final long begin,
|
||||
final long end) throws MQClientException, InterruptedException;
|
||||
|
||||
/**
|
||||
|
||||
* @param topic
|
||||
* @param msgId
|
||||
* @return The {@code MessageExt} of given msgId
|
||||
* @throws RemotingException
|
||||
* @throws MQBrokerException
|
||||
* @throws InterruptedException
|
||||
* @throws MQClientException
|
||||
*/
|
||||
MessageExt viewMessage(String topic, String msgId) throws RemotingException, MQBrokerException, InterruptedException, MQClientException;
|
||||
|
||||
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.client;
|
||||
|
||||
import com.alibaba.rocketmq.client.consumer.DefaultMQPullConsumer;
|
||||
import com.alibaba.rocketmq.client.log.ClientLogger;
|
||||
import com.alibaba.rocketmq.common.message.MessageQueue;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.MessageModel;
|
||||
import org.slf4j.Logger;
|
||||
|
||||
import java.util.Set;
|
||||
import java.util.TreeSet;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class MQHelper {
|
||||
public static void resetOffsetByTimestamp(
|
||||
final MessageModel messageModel,
|
||||
final String consumerGroup,
|
||||
final String topic,
|
||||
final long timestamp) throws Exception {
|
||||
resetOffsetByTimestamp(messageModel, "DEFAULT", consumerGroup, topic, timestamp);
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset consumer topic offset according to time
|
||||
*
|
||||
* @param messageModel
|
||||
* which model
|
||||
* @param instanceName
|
||||
* which instance
|
||||
* @param consumerGroup
|
||||
* consumer group
|
||||
* @param topic
|
||||
* topic
|
||||
* @param timestamp
|
||||
* time
|
||||
*
|
||||
* @throws Exception
|
||||
*/
|
||||
public static void resetOffsetByTimestamp(
|
||||
final MessageModel messageModel,
|
||||
final String instanceName,
|
||||
final String consumerGroup,
|
||||
final String topic,
|
||||
final long timestamp) throws Exception {
|
||||
final Logger log = ClientLogger.getLog();
|
||||
|
||||
DefaultMQPullConsumer consumer = new DefaultMQPullConsumer(consumerGroup);
|
||||
consumer.setInstanceName(instanceName);
|
||||
consumer.setMessageModel(messageModel);
|
||||
consumer.start();
|
||||
|
||||
Set<MessageQueue> mqs = null;
|
||||
try {
|
||||
mqs = consumer.fetchSubscribeMessageQueues(topic);
|
||||
if (mqs != null && !mqs.isEmpty()) {
|
||||
TreeSet<MessageQueue> mqsNew = new TreeSet<MessageQueue>(mqs);
|
||||
for (MessageQueue mq : mqsNew) {
|
||||
long offset = consumer.searchOffset(mq, timestamp);
|
||||
if (offset >= 0) {
|
||||
consumer.updateConsumeOffset(mq, offset);
|
||||
log.info("resetOffsetByTimestamp updateConsumeOffset success, {} {} {}",
|
||||
consumerGroup, offset, mq);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("resetOffsetByTimestamp Exception", e);
|
||||
throw e;
|
||||
} finally {
|
||||
if (mqs != null) {
|
||||
consumer.getDefaultMQPullConsumerImpl().getOffsetStore().persistAll(mqs);
|
||||
}
|
||||
consumer.shutdown();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.client;
|
||||
|
||||
import com.alibaba.rocketmq.common.message.MessageExt;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class QueryResult {
|
||||
private final long indexLastUpdateTimestamp;
|
||||
private final List<MessageExt> messageList;
|
||||
|
||||
|
||||
public QueryResult(long indexLastUpdateTimestamp, List<MessageExt> messageList) {
|
||||
this.indexLastUpdateTimestamp = indexLastUpdateTimestamp;
|
||||
this.messageList = messageList;
|
||||
}
|
||||
|
||||
|
||||
public long getIndexLastUpdateTimestamp() {
|
||||
return indexLastUpdateTimestamp;
|
||||
}
|
||||
|
||||
|
||||
public List<MessageExt> getMessageList() {
|
||||
return messageList;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return "QueryResult [indexLastUpdateTimestamp=" + indexLastUpdateTimestamp + ", messageList="
|
||||
+ messageList + "]";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.client;
|
||||
|
||||
import com.alibaba.rocketmq.client.exception.MQClientException;
|
||||
import com.alibaba.rocketmq.client.producer.DefaultMQProducer;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.UtilAll;
|
||||
import com.alibaba.rocketmq.common.message.Message;
|
||||
import com.alibaba.rocketmq.common.protocol.ResponseCode;
|
||||
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
|
||||
/**
|
||||
* Common Validator
|
||||
*
|
||||
* @author manhong.yqd
|
||||
*/
|
||||
public class Validators {
|
||||
public static final String VALID_PATTERN_STR = "^[%|a-zA-Z0-9_-]+$";
|
||||
public static final Pattern PATTERN = Pattern.compile(VALID_PATTERN_STR);
|
||||
public static final int CHARACTER_MAX_LENGTH = 255;
|
||||
|
||||
/**
|
||||
* @param origin
|
||||
* @param patternStr
|
||||
*
|
||||
* @return The resulting {@code String}
|
||||
*/
|
||||
public static String getGroupWithRegularExpression(String origin, String patternStr) {
|
||||
Pattern pattern = Pattern.compile(patternStr);
|
||||
Matcher matcher = pattern.matcher(origin);
|
||||
while (matcher.find()) {
|
||||
return matcher.group(0);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate group
|
||||
*
|
||||
* @param group
|
||||
*
|
||||
* @throws com.alibaba.rocketmq.client.exception.MQClientException
|
||||
*/
|
||||
public static void checkGroup(String group) throws MQClientException {
|
||||
if (UtilAll.isBlank(group)) {
|
||||
throw new MQClientException("the specified group is blank", null);
|
||||
}
|
||||
if (!regularExpressionMatcher(group, PATTERN)) {
|
||||
throw new MQClientException(String.format(
|
||||
"the specified group[%s] contains illegal characters, allowing only %s", group,
|
||||
VALID_PATTERN_STR), null);
|
||||
}
|
||||
if (group.length() > CHARACTER_MAX_LENGTH) {
|
||||
throw new MQClientException("the specified group is longer than group max length 255.", null);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param origin
|
||||
* @param pattern
|
||||
*
|
||||
* @return <tt>true</tt> if, and only if, the entire origin sequence
|
||||
* matches this matcher's pattern
|
||||
*/
|
||||
public static boolean regularExpressionMatcher(String origin, Pattern pattern) {
|
||||
if (pattern == null) {
|
||||
return true;
|
||||
}
|
||||
Matcher matcher = pattern.matcher(origin);
|
||||
return matcher.matches();
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate message
|
||||
*
|
||||
* @param msg
|
||||
* @param defaultMQProducer
|
||||
*
|
||||
* @throws com.alibaba.rocketmq.client.exception.MQClientException
|
||||
*/
|
||||
public static void checkMessage(Message msg, DefaultMQProducer defaultMQProducer)
|
||||
throws MQClientException {
|
||||
if (null == msg) {
|
||||
throw new MQClientException(ResponseCode.MESSAGE_ILLEGAL, "the message is null");
|
||||
}
|
||||
// topic
|
||||
Validators.checkTopic(msg.getTopic());
|
||||
// body
|
||||
if (null == msg.getBody()) {
|
||||
throw new MQClientException(ResponseCode.MESSAGE_ILLEGAL, "the message body is null");
|
||||
}
|
||||
|
||||
if (0 == msg.getBody().length) {
|
||||
throw new MQClientException(ResponseCode.MESSAGE_ILLEGAL, "the message body length is zero");
|
||||
}
|
||||
|
||||
if (msg.getBody().length > defaultMQProducer.getMaxMessageSize()) {
|
||||
throw new MQClientException(ResponseCode.MESSAGE_ILLEGAL,
|
||||
"the message body size over max value, MAX: " + defaultMQProducer.getMaxMessageSize());
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate topic
|
||||
*
|
||||
* @param topic
|
||||
*
|
||||
* @throws com.alibaba.rocketmq.client.exception.MQClientException
|
||||
*/
|
||||
public static void checkTopic(String topic) throws MQClientException {
|
||||
if (UtilAll.isBlank(topic)) {
|
||||
throw new MQClientException("the specified topic is blank", null);
|
||||
}
|
||||
|
||||
if (!regularExpressionMatcher(topic, PATTERN)) {
|
||||
throw new MQClientException(String.format(
|
||||
"the specified topic[%s] contains illegal characters, allowing only %s", topic,
|
||||
VALID_PATTERN_STR), null);
|
||||
}
|
||||
|
||||
if (topic.length() > CHARACTER_MAX_LENGTH) {
|
||||
throw new MQClientException("the specified topic is longer than topic max length 255.", null);
|
||||
}
|
||||
|
||||
//whether the same with system reserved keyword
|
||||
if (topic.equals(MixAll.DEFAULT_TOPIC)) {
|
||||
throw new MQClientException(
|
||||
String.format("the topic[%s] is conflict with default topic.", topic), null);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.client.admin;
|
||||
|
||||
/**
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public interface MQAdminExtInner {
|
||||
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.client.common;
|
||||
|
||||
public class ClientErrorCode {
|
||||
public static final int CONNECT_BROKER_EXCEPTION = 10001;
|
||||
public static final int ACCESS_BROKER_TIMEOUT = 10002;
|
||||
public static final int BROKER_NOT_EXIST_EXCEPTION = 10003;
|
||||
public static final int NO_NAME_SERVER_EXCEPTION = 10004;
|
||||
public static final int NOT_FOUND_TOPIC_EXCEPTION = 10005;
|
||||
}
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.rocketmq.client.common;
|
||||
|
||||
import java.util.Random;
|
||||
|
||||
public class ThreadLocalIndex {
|
||||
private final ThreadLocal<Integer> threadLocalIndex = new ThreadLocal<Integer>();
|
||||
private final Random random = new Random();
|
||||
public ThreadLocalIndex(int value) {
|
||||
|
||||
}
|
||||
|
||||
public int getAndIncrement() {
|
||||
Integer index = this.threadLocalIndex.get();
|
||||
if (null == index) {
|
||||
index = Math.abs(random.nextInt());
|
||||
if (index < 0) index = 0;
|
||||
this.threadLocalIndex.set(index);
|
||||
}
|
||||
|
||||
index = Math.abs(index + 1);
|
||||
if (index < 0)
|
||||
index = 0;
|
||||
|
||||
this.threadLocalIndex.set(index);
|
||||
return index;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return "ThreadLocalIndex{" +
|
||||
"threadLocalIndex=" + threadLocalIndex.get() +
|
||||
'}';
|
||||
}
|
||||
}
|
||||
+60
@@ -0,0 +1,60 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.client.consumer;
|
||||
|
||||
import com.alibaba.rocketmq.common.message.MessageQueue;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/**
|
||||
* Strategy Algorithm for message allocating between consumers
|
||||
*
|
||||
* @author shijia.wxr
|
||||
* @author vongosling
|
||||
*/
|
||||
public interface AllocateMessageQueueStrategy {
|
||||
|
||||
/**
|
||||
* Allocating by consumer id
|
||||
*
|
||||
* @param consumerGroup
|
||||
* current consumer group
|
||||
* @param currentCID
|
||||
* current consumer id
|
||||
* @param mqAll
|
||||
* message queue set in current topic
|
||||
* @param cidAll
|
||||
* consumer set in current consumer group
|
||||
*
|
||||
* @return The allocate result of given strategy
|
||||
*/
|
||||
List<MessageQueue> allocate(
|
||||
final String consumerGroup,
|
||||
final String currentCID,
|
||||
final List<MessageQueue> mqAll,
|
||||
final List<String> cidAll
|
||||
);
|
||||
|
||||
|
||||
/**
|
||||
* Algorithm name
|
||||
*
|
||||
* @return The strategy name
|
||||
*/
|
||||
String getName();
|
||||
}
|
||||
+381
@@ -0,0 +1,381 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.rocketmq.client.consumer;
|
||||
|
||||
import com.alibaba.rocketmq.client.ClientConfig;
|
||||
import com.alibaba.rocketmq.client.QueryResult;
|
||||
import com.alibaba.rocketmq.client.consumer.rebalance.AllocateMessageQueueAveragely;
|
||||
import com.alibaba.rocketmq.client.consumer.store.OffsetStore;
|
||||
import com.alibaba.rocketmq.client.exception.MQBrokerException;
|
||||
import com.alibaba.rocketmq.client.exception.MQClientException;
|
||||
import com.alibaba.rocketmq.client.impl.consumer.DefaultMQPullConsumerImpl;
|
||||
import com.alibaba.rocketmq.common.MixAll;
|
||||
import com.alibaba.rocketmq.common.message.MessageDecoder;
|
||||
import com.alibaba.rocketmq.common.message.MessageExt;
|
||||
import com.alibaba.rocketmq.common.message.MessageQueue;
|
||||
import com.alibaba.rocketmq.common.protocol.heartbeat.MessageModel;
|
||||
import com.alibaba.rocketmq.remoting.RPCHook;
|
||||
import com.alibaba.rocketmq.remoting.exception.RemotingException;
|
||||
|
||||
import java.util.HashSet;
|
||||
import java.util.Set;
|
||||
|
||||
|
||||
/**
|
||||
* Default pulling consumer
|
||||
*
|
||||
* @author shijia.wxr
|
||||
*/
|
||||
public class DefaultMQPullConsumer extends ClientConfig implements MQPullConsumer {
|
||||
protected final transient DefaultMQPullConsumerImpl defaultMQPullConsumerImpl;
|
||||
|
||||
/**
|
||||
* Do the same thing for the same Group, the application must be set,and
|
||||
* guarantee Globally unique
|
||||
*/
|
||||
private String consumerGroup;
|
||||
/**
|
||||
* Long polling mode, the Consumer connection max suspend time, it is not
|
||||
* recommended to modify
|
||||
*/
|
||||
private long brokerSuspendMaxTimeMillis = 1000 * 20;
|
||||
/**
|
||||
* Long polling mode, the Consumer connection timeout(must greater than
|
||||
* brokerSuspendMaxTimeMillis), it is not recommended to modify
|
||||
*/
|
||||
private long consumerTimeoutMillisWhenSuspend = 1000 * 30;
|
||||
/**
|
||||
* The socket timeout in milliseconds
|
||||
*/
|
||||
private long consumerPullTimeoutMillis = 1000 * 10;
|
||||
/**
|
||||
* Consumption pattern,default is clustering
|
||||
*/
|
||||
private MessageModel messageModel = MessageModel.CLUSTERING;
|
||||
/**
|
||||
* Message queue listener
|
||||
*/
|
||||
private MessageQueueListener messageQueueListener;
|
||||
/**
|
||||
* Offset Storage
|
||||
*/
|
||||
private OffsetStore offsetStore;
|
||||
/**
|
||||
* Topic set you want to register
|
||||
*/
|
||||
private Set<String> registerTopics = new HashSet<String>();
|
||||
/**
|
||||
* Queue allocation algorithm
|
||||
*/
|
||||
private AllocateMessageQueueStrategy allocateMessageQueueStrategy = new AllocateMessageQueueAveragely();
|
||||
/**
|
||||
* Whether the unit of subscription group
|
||||
*/
|
||||
private boolean unitMode = false;
|
||||
|
||||
private int maxReconsumeTimes = 16;
|
||||
|
||||
|
||||
public DefaultMQPullConsumer() {
|
||||
this(MixAll.DEFAULT_CONSUMER_GROUP, null);
|
||||
}
|
||||
|
||||
|
||||
public DefaultMQPullConsumer(final String consumerGroup, RPCHook rpcHook) {
|
||||
this.consumerGroup = consumerGroup;
|
||||
defaultMQPullConsumerImpl = new DefaultMQPullConsumerImpl(this, rpcHook);
|
||||
}
|
||||
|
||||
|
||||
public DefaultMQPullConsumer(final String consumerGroup) {
|
||||
this(consumerGroup, null);
|
||||
}
|
||||
|
||||
|
||||
public DefaultMQPullConsumer(RPCHook rpcHook) {
|
||||
this(MixAll.DEFAULT_CONSUMER_GROUP, rpcHook);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void createTopic(String key, String newTopic, int queueNum) throws MQClientException {
|
||||
createTopic(key, newTopic, queueNum, 0);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void createTopic(String key, String newTopic, int queueNum, int topicSysFlag) throws MQClientException {
|
||||
this.defaultMQPullConsumerImpl.createTopic(key, newTopic, queueNum, topicSysFlag);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public long searchOffset(MessageQueue mq, long timestamp) throws MQClientException {
|
||||
return this.defaultMQPullConsumerImpl.searchOffset(mq, timestamp);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public long maxOffset(MessageQueue mq) throws MQClientException {
|
||||
return this.defaultMQPullConsumerImpl.maxOffset(mq);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public long minOffset(MessageQueue mq) throws MQClientException {
|
||||
return this.defaultMQPullConsumerImpl.minOffset(mq);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public long earliestMsgStoreTime(MessageQueue mq) throws MQClientException {
|
||||
return this.defaultMQPullConsumerImpl.earliestMsgStoreTime(mq);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public MessageExt viewMessage(String offsetMsgId) throws RemotingException, MQBrokerException,
|
||||
InterruptedException, MQClientException {
|
||||
return this.defaultMQPullConsumerImpl.viewMessage(offsetMsgId);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public QueryResult queryMessage(String topic, String key, int maxNum, long begin, long end)
|
||||
throws MQClientException, InterruptedException {
|
||||
return this.defaultMQPullConsumerImpl.queryMessage(topic, key, maxNum, begin, end);
|
||||
}
|
||||
|
||||
|
||||
public AllocateMessageQueueStrategy getAllocateMessageQueueStrategy() {
|
||||
return allocateMessageQueueStrategy;
|
||||
}
|
||||
|
||||
|
||||
public void setAllocateMessageQueueStrategy(AllocateMessageQueueStrategy allocateMessageQueueStrategy) {
|
||||
this.allocateMessageQueueStrategy = allocateMessageQueueStrategy;
|
||||
}
|
||||
|
||||
|
||||
public long getBrokerSuspendMaxTimeMillis() {
|
||||
return brokerSuspendMaxTimeMillis;
|
||||
}
|
||||
|
||||
|
||||
public void setBrokerSuspendMaxTimeMillis(long brokerSuspendMaxTimeMillis) {
|
||||
this.brokerSuspendMaxTimeMillis = brokerSuspendMaxTimeMillis;
|
||||
}
|
||||
|
||||
|
||||
public String getConsumerGroup() {
|
||||
return consumerGroup;
|
||||
}
|
||||
|
||||
|
||||
public void setConsumerGroup(String consumerGroup) {
|
||||
this.consumerGroup = consumerGroup;
|
||||
}
|
||||
|
||||
|
||||
public long getConsumerPullTimeoutMillis() {
|
||||
return consumerPullTimeoutMillis;
|
||||
}
|
||||
|
||||
|
||||
public void setConsumerPullTimeoutMillis(long consumerPullTimeoutMillis) {
|
||||
this.consumerPullTimeoutMillis = consumerPullTimeoutMillis;
|
||||
}
|
||||
|
||||
|
||||
public long getConsumerTimeoutMillisWhenSuspend() {
|
||||
return consumerTimeoutMillisWhenSuspend;
|
||||
}
|
||||
|
||||
|
||||
public void setConsumerTimeoutMillisWhenSuspend(long consumerTimeoutMillisWhenSuspend) {
|
||||
this.consumerTimeoutMillisWhenSuspend = consumerTimeoutMillisWhenSuspend;
|
||||
}
|
||||
|
||||
|
||||
public MessageModel getMessageModel() {
|
||||
return messageModel;
|
||||
}
|
||||
|
||||
|
||||
public void setMessageModel(MessageModel messageModel) {
|
||||
this.messageModel = messageModel;
|
||||
}
|
||||
|
||||
|
||||
public MessageQueueListener getMessageQueueListener() {
|
||||
return messageQueueListener;
|
||||
}
|
||||
|
||||
|
||||
public void setMessageQueueListener(MessageQueueListener messageQueueListener) {
|
||||
this.messageQueueListener = messageQueueListener;
|
||||
}
|
||||
|
||||
|
||||
public Set<String> getRegisterTopics() {
|
||||
return registerTopics;
|
||||
}
|
||||
|
||||
|
||||
public void setRegisterTopics(Set<String> registerTopics) {
|
||||
this.registerTopics = registerTopics;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void sendMessageBack(MessageExt msg, int delayLevel)
|
||||
throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
|
||||
this.defaultMQPullConsumerImpl.sendMessageBack(msg, delayLevel, null);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void sendMessageBack(MessageExt msg, int delayLevel, String brokerName)
|
||||
throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
|
||||
this.defaultMQPullConsumerImpl.sendMessageBack(msg, delayLevel, brokerName);
|
||||
}
|
||||
|
||||
@Override
|
||||
public Set<MessageQueue> fetchSubscribeMessageQueues(String topic) throws MQClientException {
|
||||
return this.defaultMQPullConsumerImpl.fetchSubscribeMessageQueues(topic);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void start() throws MQClientException {
|
||||
this.defaultMQPullConsumerImpl.start();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void shutdown() {
|
||||
this.defaultMQPullConsumerImpl.shutdown();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void registerMessageQueueListener(String topic, MessageQueueListener listener) {
|
||||
synchronized (this.registerTopics) {
|
||||
this.registerTopics.add(topic);
|
||||
if (listener != null) {
|
||||
this.messageQueueListener = listener;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public PullResult pull(MessageQueue mq, String subExpression, long offset, int maxNums)
|
||||
throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
|
||||
return this.defaultMQPullConsumerImpl.pull(mq, subExpression, offset, maxNums);
|
||||
}
|
||||
|
||||
@Override
|
||||
public PullResult pull(MessageQueue mq, String subExpression, long offset, int maxNums, long timeout)
|
||||
throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
|
||||
return this.defaultMQPullConsumerImpl.pull(mq, subExpression, offset, maxNums, timeout);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void pull(MessageQueue mq, String subExpression, long offset, int maxNums, PullCallback pullCallback)
|
||||
throws MQClientException, RemotingException, InterruptedException {
|
||||
this.defaultMQPullConsumerImpl.pull(mq, subExpression, offset, maxNums, pullCallback);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void pull(MessageQueue mq, String subExpression, long offset, int maxNums, PullCallback pullCallback, long timeout)
|
||||
throws MQClientException, RemotingException, InterruptedException {
|
||||
this.defaultMQPullConsumerImpl.pull(mq, subExpression, offset, maxNums, pullCallback, timeout);
|
||||
}
|
||||
|
||||
@Override
|
||||
public PullResult pullBlockIfNotFound(MessageQueue mq, String subExpression, long offset, int maxNums)
|
||||
throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
|
||||
return this.defaultMQPullConsumerImpl.pullBlockIfNotFound(mq, subExpression, offset, maxNums);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void pullBlockIfNotFound(MessageQueue mq, String subExpression, long offset, int maxNums, PullCallback pullCallback)
|
||||
throws MQClientException, RemotingException, InterruptedException {
|
||||
this.defaultMQPullConsumerImpl.pullBlockIfNotFound(mq, subExpression, offset, maxNums, pullCallback);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void updateConsumeOffset(MessageQueue mq, long offset) throws MQClientException {
|
||||
this.defaultMQPullConsumerImpl.updateConsumeOffset(mq, offset);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long fetchConsumeOffset(MessageQueue mq, boolean fromStore) throws MQClientException {
|
||||
return this.defaultMQPullConsumerImpl.fetchConsumeOffset(mq, fromStore);
|
||||
}
|
||||
|
||||
@Override
|
||||
public Set<MessageQueue> fetchMessageQueuesInBalance(String topic) throws MQClientException {
|
||||
return this.defaultMQPullConsumerImpl.fetchMessageQueuesInBalance(topic);
|
||||
}
|
||||
|
||||
@Override
|
||||
public MessageExt viewMessage(String topic, String uniqKey) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
|
||||
try {
|
||||
MessageDecoder.decodeMessageId(uniqKey);
|
||||
return this.viewMessage(uniqKey);
|
||||
} catch (Exception e) {
|
||||
}
|
||||
return this.defaultMQPullConsumerImpl.queryMessageByUniqKey(topic, uniqKey);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void sendMessageBack(MessageExt msg, int delayLevel, String brokerName, String consumerGroup)
|
||||
throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
|
||||
this.defaultMQPullConsumerImpl.sendMessageBack(msg, delayLevel, brokerName, consumerGroup);
|
||||
}
|
||||
|
||||
public OffsetStore getOffsetStore() {
|
||||
return offsetStore;
|
||||
}
|
||||
|
||||
|
||||
public void setOffsetStore(OffsetStore offsetStore) {
|
||||
this.offsetStore = offsetStore;
|
||||
}
|
||||
|
||||
|
||||
public DefaultMQPullConsumerImpl getDefaultMQPullConsumerImpl() {
|
||||
return defaultMQPullConsumerImpl;
|
||||
}
|
||||
|
||||
|
||||
public boolean isUnitMode() {
|
||||
return unitMode;
|
||||
}
|
||||
|
||||
|
||||
public void setUnitMode(boolean isUnitMode) {
|
||||
this.unitMode = isUnitMode;
|
||||
}
|
||||
|
||||
|
||||
public int getMaxReconsumeTimes() {
|
||||
return maxReconsumeTimes;
|
||||
}
|
||||
|
||||
|
||||
public void setMaxReconsumeTimes(final int maxReconsumeTimes) {
|
||||
this.maxReconsumeTimes = maxReconsumeTimes;
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user