openstack 组件通讯是通过ZeroMQ+ceilometer发送组件调用信息，具体是通过TCP通讯，发送数据和接收数据是用同一个端口（在配置文件指定），下面通过代码稍作解析：
IceHouse/ceilometer/ceilometer/openstack/common/rpc/impl_zmq.py
def _multi_send(method, context, topic, msg, timeout=None,
envelope=False, _msg_id=None):
"""Wraps the sending of messages.
Dispatches to the matchmaker and sends message to all relevant hosts.
"""
conf = CONF
LOG.debug(_("%(msg)s") % {‘msg‘: ‘ ‘.join(map(pformat, (topic, msg)))})
queues = _get_matchmaker().queues(topic)
LOG.debug(_("Sending message(s) to: %s"), queues)
# Don‘t stack if we have no matchmaker results
if not queues:
LOG.warn(_("No matchmaker results. Not casting."))
# While not strictly a timeout, callers know how to handle
# this exception and a timeout isn‘t too big a lie.
raise rpc_common.Timeout(_("No match from matchmaker."))
# This supports brokerless fanout (addresses > 1)
for queue in queues:
(_topic, ip_addr) = queue
_addr = "tcp://%s:%s" % (ip_addr, conf.rpc_zmq_port)
if method.__name__ == ‘_cast‘:
eventlet.spawn_n(method, _addr, context,
_topic, msg, timeout, envelope,
_msg_id)
return
return method(_addr, context, _topic, msg, timeout,
envelope)
debug日志:/var/log/ceilometer/compute.log
compute.log:2015-05-18 11:36:58.972 2436 DEBUG ceilometer.openstack.common.rpc.common [-] Sending message(s) to: [(u‘metering.ceilometer‘, u‘ceilometer‘)] _multi_send /usr/lib/python2.6/site-packages/ceilometer/openstack/common/rpc/impl_zmq.py:750 compute.log:2015-05-18 11:36:58.982 2436 DEBUG ceilometer.openstack.common.rpc.common [-] Sending message(s) to: [(u‘metering.ceilometer‘, u‘ceilometer‘)] _multi_send /usr/lib/python2.6/site-packages/ceilometer/openstack/common/rpc/impl_zmq.py:750
1. conf.rpc_zmq_port是通过/etc/nova/nova.conf中指定的，也即是nova服务（/usr/bin/oslo-messaging-zmq-receiver）启动的监听端口：
[[email protected] ceilometer]# netstat -lntp | grep 9502 tcp 0 0 192.168.213.202:9502 0.0.0.0:* LISTEN 17860/haproxy tcp 0 0 192.168.213.88:9502 0.0.0.0:* LISTEN 7611/python [[email protected] ceilometer]# ps -ef | grep 7611 nova 7611 1 0 May13 ? 00:00:02 /usr/bin/python /usr/bin/oslo-messaging-zmq-receiver --config-file /etc/nova/nova.conf root 26349 20271 0 11:44 pts/2 00:00:00 grep 7611
2. 通过上面代码可以看到nova与ceilometer通讯的指定的目的端口也是rpc_zmq_port，由此可以得出，在all in one环境中，需要对cinder、nova、neutron和ceilometer做端口转发，才能在不修改源码的基础上做到组件之间通讯(组件与ceilometer建立tcp连接)，用haproxy：
/etc/haproxy/haproxy.cfg
#/etc/haproxy/haproxy.cfg
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the ‘-r‘ option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log /dev/log local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 32768
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the ‘listen‘ and ‘backend‘ sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode tcp
log global
#option httplog
option dontlognull
#option http-server-close
#option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 32768
listen haproxy_stats
bind *:8888
mode http
stats uri /
#stats refresh 5
# stats realm HAproxy\ stats
# stats auth admin:admin
listen zmq-cinder
bind 192.168.213.202:9501
mode tcp
#log 127.0.0.1 local2
balance roundrobin
server test1 192.168.213.88:9500
listen zmq-nova
bind 192.168.213.202:9502
mode tcp
#log 127.0.0.1 local2
balance roundrobin
server test1 192.168.213.88:9500
listen zmq-neutron
bind 192.168.213.202:9505
mode tcp
#log 127.0.0.1 local2
balance roundrobin
server test1 192.168.213.88:9500
listen zmq-ceilometer
bind 192.168.213.202:9500
mode tcp
#log 127.0.0.1 local2
balance roundrobin
server test1 192.168.213.88:9500
3. 建立TCP连接时指定目的IP是在nova配置文件的[matchmaker_ring]模块中指定：
[matchmaker_ring] ringfile = /etc/nova/matchmaker_ring.json
{
"conductor":["test1"],
"scheduler":["test1"],
"compute": ["test1"],
"cert": ["test1"],
"consoleauth":["test1"],
"backendtask":["test1"],
"metering": ["ceilometer"],
"notifications-info": ["ceilometer"],
"notifications-error": ["ceilometer"]
}
时间： 2024-10-25 04:22:26