【原创】《从0开始学RocketMQ》—单机搭建

内容目录

1. RocketMQ是什么?

2. 下载并解压

3. 启动NameServer

4. 启动 Broker

5. 关闭消息队列

1. RocketMQ是什么?

RocketMQ是一种消息队列。何为消息队列?即数据结构中一种“先进先出”的数据结构。在微服务中,分布式消息队列可以解决什么问题?应用解耦、流量削峰、消息分发、保证最终一致性、方便动态扩容等。

RocketMQ中不可不知的四个角色:Producer(消息生产者)、Consumer(消息消费者)、Broker(消息暂存者)、NameServer(消息协调者)。由这四个角色的作用可知,启动 RocketMQ 时,应该先启动 NameServer,然后再启动 Broker,后续需要发送消息就用 Producer,需要接收消息就用 Consumer。

2. 下载并解压

cd /usr/local
wget -c http://mirrors.shu.edu.cn/apache/rocketmq/4.3.2/rocketmq-all-4.3.2-bin-release.zip
unzip rocketmq-all-4.3.2-bin-release.zip -d .
cd rocketmq-all-4.3.2-bin-release
ls
[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# ls
benchmark  bin  conf  lib  LICENSE  NOTICE  README.md

3. 启动NameServer

启动NameServer:nohup sh bin/mqnamesrv &,然而发现不能正常启动,如下:

[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqnamesrv &
[1] 32673
[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’

[1]+  Exit 1                  nohup sh bin/mqnamesrv

查看nohup.out:

[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# cat nohup.out
ERROR: Please set the JAVA_HOME variable in your environment, We need java(x64)! !!

可是,本机明明已经安装jdk并且已经配置了 JAVA_HOME 环境变量呀:

[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# java -version
java version "1.8.0_171"
Java(TM) SE Runtime Environment (build 1.8.0_171-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)

环境变量设置:

#####################java env###############
JAVA_HOME=/opt/jdk1.8.0_171
CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar
PATH=$JAVA_HOME/bin:$HOME/bin:$HOME/.local/bin:$PATH

查看 runserver.sh 脚本,发现有这么三行命令:

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

将其注释之,继续往下看,发现下面两行:

export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"

嗯,果断修改 JAVA_HOME 的值如下:

export JAVA_HOME=/opt/jdk1.8.0_171

再次启动 NameServer :

[[email protected]157-89 rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqnamesrv &
[1] 31098
[[email protected]157-89 rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’

[[email protected]157-89 rocketmq-all-4.3.2-bin-release]# ls
benchmark  bin  conf  lib  LICENSE  nohup.out  NOTICE  README.md
[[email protected]157-89 rocketmq-all-4.3.2-bin-release]#  cat nohup.out
Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON

此处可以看到 NameServer 已成功启动,查看 NameServer 日志:

[[email protected]157-89 rocketmq-all-4.3.2-bin-release]# tail -f ~/logs/rocketmqlogs/namesrv.log
2019-01-07 20:19:36 INFO main - tls.client.certPath = null
2019-01-07 20:19:36 INFO main - tls.client.authServer = false
2019-01-07 20:19:36 INFO main - tls.client.trustCertPath = null
2019-01-07 20:19:36 INFO main - Using OpenSSL provider
2019-01-07 20:19:37 INFO main - SSLContext created for server
2019-01-07 20:19:37 INFO NettyEventExecutor - NettyEventExecutor service started
2019-01-07 20:19:37 INFO FileWatchService - FileWatchService service started
2019-01-07 20:19:37 INFO main - The Name Server boot success. serializeType=JSON
2019-01-07 20:20:37 INFO NSScheduledThread1 - --------------------------------------------------------
2019-01-07 20:20:37 INFO NSScheduledThread1 - configTable SIZE: 0

NameServer 启动成功

4. 启动 Broker

启动Broker:nohup sh bin/mqbroker -n localhost:9876&,然而发现不能正常启动,如下:

[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqbroker -n localhost:9876&
[1] 28571
[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’

[1]+  Exit 1                  nohup sh bin/mqbroker -n localhost:9876

查看nohup.out,发现里面多了这么几行错误信息:

Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000005c0000000, 8589934592, 0) failed; error=‘Cannot allocate memory‘ (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 8589934592 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /usr/local/rocketmq-all-4.3.2-bin-release/hs_err_pid28580.log

遂,查看 hs_err_pid28580.log:

[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# tail -1000f /usr/local/rocketmq-all-4.3.2-bin-release/hs_err_pid28580.log
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 8589934592 bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_linux.cpp:2640), pid=28580, tid=0x00007fbc93a6c700
#
# JRE version:  (8.0_171-b11) (build )
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.171-b11 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
........
VM Arguments:
jvm_args: -Xms8g -Xmx8g -Xmn4g -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=15g -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/opt/jdk1.8.0_171/jre/lib/ext:/usr/local/rocketmq-all-4.3.2-bin-release/bin/../lib
java_command: org.apache.rocketmq.broker.BrokerStartup -n localhost:9876
java_class_path (initial): .:/usr/local/rocketmq-all-4.3.2-bin-release/bin/../conf:.:/opt/jdk1.8.0_171/lib/tools.jar:/opt/jdk1.8.0_171/lib/dt.jar
Launcher Type: SUN_STANDARD
......

大概意思呢,就是说,启动broker时内存不足,为什么内存不足呢?因为启动时候指定了JVM最大可用内存和最小可用内存均为8G,年轻代大小为2G。
查看Broker启动脚本,发现果然指定了这三个jvm启动参数,然后意外发现对于JAVA_HOME的设置,runbroker.sh与runserver.sh是一样一样的:

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=$(dirname $0)/..
export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}

#===========================================================================================
# JVM Configuration
#===========================================================================================
JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g -Xmn4g"

遂,修改启动脚本中JAVA_HOME的设置以及JVM启动脚本如下:

#[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
#[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
#[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

export JAVA_HOME=/opt/jdk1.8.0_171
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=$(dirname $0)/..
export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}

#===========================================================================================
# JVM Configuration
#===========================================================================================
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m"

再次启动Broker:

[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqbroker -n localhost:9876&
[1] 29092
[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’

[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# cat nohup.out
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000005c0000000, 8589934592, 0) failed; error=‘Cannot allocate memory‘ (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 8589934592 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /usr/local/rocketmq-all-4.3.2-bin-release/hs_err_pid28580.log
The broker[153-215, 39.107.153.215:10911] boot success. serializeType=JSON and name server is localhost:9876

此处可以看到Broker已启动成功,查看Broker日志:

[[email protected]157-89 rocketmq-all-4.3.2-bin-release]# tail -1000f ~/logs/rocketmqlogs/broker.log
2019-01-09 16:13:08 INFO main - rocketmqHome=/usr/local/rocketmq-all-4.3.2-bin-release
2019-01-09 16:13:08 INFO main - namesrvAddr=localhost:9876
......
2019-01-09 16:13:09 INFO FileWatchService - FileWatchService service started
2019-01-09 16:13:09 INFO PullRequestHoldService - PullRequestHoldService service started
2019-01-09 16:13:09 INFO brokerOutApi_thread_1 - register broker to name server localhost:9876 OK
2019-01-09 16:13:09 INFO main - Start transaction service!
2019-01-09 16:13:09 INFO main - The broker[157-89, 39.107.157.89:10911] boot success. serializeType=JSON and name server is localhost:9876

5. 关闭消息队列

关闭 Broker:

[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# sh bin/mqshutdown broker
The mqbroker(29099) is running...
Send shutdown request to mqbroker(29099) OK

关闭 NameServer:

[[email protected]153-215 rocketmq-all-4.3.2-bin-release]# sh bin/mqshutdown namesrv
The mqnamesrv(14283) is running...
Send shutdown request to mqnamesrv(14283) OK

原文地址:https://www.cnblogs.com/mengyi/p/10296620.html

时间: 2024-10-30 20:47:12

【原创】《从0开始学RocketMQ》—单机搭建的相关文章

【原创】《从0开始学RocketMQ》—集群搭建

用两台服务器,搭建出一个双master双slave.无单点故障的高可用 RocketMQ 集群.此处假设两台服务器的物理 IP 分别为:192.168.50.1.192.168.50.2. 内容目录 1. 启动 NameServer 集群 2. 启动 Broker 集群 3. RocketMQ 可视化管理控制台:rocketmq-console 4. 集群测试 1. 启动 NameServer 集群 在两台服务器上分别启动 NameServer,可以得到一个无单点故障的 NameServer 服

rocketmq单机搭建

RocketMQ 是alibaba开源的消息队列. 本文使用的是开源版本v3.18 系统: centos6.x最小化安装 需要用到的软件包: jdk-7u67-linux-x64.tar.gz alibaba-rocketmq-3.1.8.tar.gz 开始安装 #tar xvf jdk-7u67-linux-x64.tar.gz -C /opt/ #tar xvf alibaba-rocketmq-3.1.8.tar.gz -C /opt/ #ln -s /opt/jdk1.7.0_67 /o

从0开始学架构(二)

此系列文章为极客时间上从0开始学架构学习后感悟总结,虽然隔了一段时间了,那么就再看一遍并且进行感悟升华,排版格式上有问题,后期再复习时也会进行更新   一.    高性能数据库集群:读写分离 读写分离的基本原理是将数据库读写操作分散到不同的节点上. 数据库服务器搭建主从集群,一主一从.一主多从都可以 数据库主机负责读写操作,从机只负责读操作 数据库主机通过复制将数据同步到从机,每台数据库服务器都存储了所有的业务数据 业务服务器将写操作发给数据库主机,将读操作发给数据库从机 从代码层面与运维层面实

如何从 0 开始学 ruby on rails (漫步版)

如何从 0 开始学 ruby on rails (漫步版) ruby 是一门编程语言,ruby on rails 是 ruby 的一个 web 框架,简称 rails. 有很多人对  rails 感兴趣,但又不知道从何下手.学习路线是什么,因为在多个场合下回答过类似问题,所以决定整理成文章供大家参观. 有很多人选择直接学习 rails,在学习使用 rails 的过程中学习 ruby.但我觉得这有些本末倒置,我更推崇先学 ruby 再学 rails,在对 ruby 有了一定的了解后再学 rails

如何从 0 开始学 Ruby on Rails

如何从 0 开始学 Ruby on Rails (漫步版)Ruby 是一门编程语言,Ruby on Rails 是 Ruby 的一个 web 框架,简称 Rails. 有很多人对 Rails 感兴趣,但又不知道从何下手.学习路线是什么,因为在多个场合下回答过类似问题,所以决定整理成文章供大家参观. 有很多人选择直接学习 Rails,在学习使用 Rails 的过程中学习 Ruby.但我觉得这有些本末倒置,我更推崇先学 Ruby 再学 Rails,在对 Ruby 有了一定的了解后再学 Rails 有

Windows Server 2003 IIS6.0+PHP5(FastCGI)+MySQL5环境搭建教程

准备篇 一.环境说明: 操作系统:Windows Server 2003 SP2 32位 PHP版本:php 5.3.14 MySQL版本:MySQL5.5.25 二.相关软件下载: 1.PHP下载地址: http://windows.php.net/downloads/releases/php-5.3.14-nts-Win32-VC9-x86.zip 2.MySQL下载地址: http://gd.tuwien.ac.at/db/mysql/Downloads/MySQLInstaller/my

从0开始学大数据-Java运算符(3)

我们从零开始学习大数据技术,从java基础,到Linux技术涉猎,再深入到大数据技术的Hadoop.Spark.Storm技术,最后到大数据企业平台的搭建,层层递进,由点到面!希望技术大牛能过来指导学习. 上一节了解Java基础语法,本节我们开始学习Java基础-运算符,将会围绕以下几个知识点进行展开学习: 算术运算符 赋值运算符 比较运算符 逻辑运算符 位运算符 三元运算符 PS:开始之前我们先针对上节数据类型中补充的几个小问题 一.数据类型中补充的几个小问题 1.byte值的问题 byte

从0开始学大数据-Java基础-三元运算符/键盘录入(4)

我们从零开始学习大数据技术,从java基础,到Linux技术涉猎,再深入到大数据技术的Hadoop.Spark.Storm技术,最后到大数据企业平台的搭建,层层递进,由点到面!希望技术大牛能过来指导学习. 上一节了解Java运算符,其中三元运算符没有做讲解,本节我们开始学习Java基础-三元运算符/键盘录入,将会围绕以下几个知识点进行展开学习: 三元运算符 键盘录入数据 一.运算符 1.三元运算符 接着上一节的话题运算符,本节讲三元运算符,在讲三元运算符之前,可能会有很多朋友会问,是不是有一元运

对比学习用 Keras 搭建 CNN RNN 等常用神经网络

Keras 是一个兼容 Theano 和 Tensorflow 的神经网络高级包, 用他来组件一个神经网络更加快速, 几条语句就搞定了. 而且广泛的兼容性能使 Keras 在 Windows 和 MacOS 或者 Linux 上运行无阻碍. 今天来对比学习一下用 Keras 搭建下面几个常用神经网络: 回归 RNN回归 分类 CNN分类 RNN分类 自编码分类 它们的步骤差不多是一样的: [导入模块并创建数据] [建立模型] [定义优化器] [激活模型] [训练模型] [检验模型] [可视化结果