转载请注明出处：http://blog.csdn.net/cywosp/article/details/40781569
swift在版本号2.1.0之前假设各个服务的配置文件里打开下面配置后，且系统没有配置正确将会出现上传对象出错的情况
log_statsd_host = localhost
log_statsd_port = 8125
log_statsd_default_sample_rate = 1.0
log_statsd_sample_rate_factor = 1.0
log_statsd_metric_prefix =
详细错误log信息大概例如以下：
object-server ERROR __call__ error with PUT /sdc/2468/AUTH_8f9dbbadd64a43a0abb5e832c6ea766a/000008/013781 : #012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/swift/obj/server.py", line 938, in __call__#012
res = method(req)#012 File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 1558, in wrapped#012 return func(*a, **kw)#012 File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 520, in _timing_stats#012 resp = func(ctrl,
*args, **kwargs)#012 File "/usr/lib/python2.6/site-packages/swift/obj/server.py", line 712, in PUT#012 file.put(fd, metadata)#012 File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__#012 self.gen.throw(type, value, traceback)#012 File "/usr/lib/python2.6/site-packages/swift/obj/server.py",
line 286, in mkstemp#012 yield fd#012 File "/usr/lib/python2.6/site-packages/swift/obj/server.py", line 680, in PUT#012 ‘PUT.‘ + device + ‘.timing‘, elapsed_time, upload_size)#012 File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line
654, in wrapped#012 return func(self.logger.statsd_client, *a, **kw)#012 File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 506, in transfer_rate#012 sample_rate)#012 File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line
496, in timing#012 return self._send(metric, timing_ms, ‘ms‘, sample_rate)#012 File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 481, in _send#012 return sock.sendto(‘|‘.join(parts), self._target)#012 File "/usr/lib/python2.6/site-packages/eventlet/greenio.py",
line 371, in sendto#012 return self.fd.sendto(*args)#012error: [Errno 1] Operation not permitted (txn: tx8d76698250304466817aa99061637421)
依据log信息查到是在swift/common/utils.py文件的StatsdClient._send函数抛出了异常没有被捕捉导致的。该函数代码例如以下：
def _send(self, m_name, m_value, m_type, sample_rate):
if sample_rate is None:
sample_rate = self._default_sample_rate
sample_rate = sample_rate * self._sample_rate_factor
parts = [‘%s%s:%s‘ % (self._prefix, m_name, m_value), m_type]
if sample_rate < 1:
if self.random() < sample_rate:
parts.append(‘@%s‘ % (sample_rate,))
else:
return
# Ideally, we‘d cache a sending socket in self, but that
# results in a socket getting shared by multiple green threads.
with closing(self._open_socket()) as sock:
return sock.sendto(‘|‘.join(parts), self._target) #该函数调用抛出了异常
解决的方法：
在return sock.sendto(‘|‘.join(parts), self._target)中增加异常处理就可以。详细代码可參考官方最新代码
同一时候系统的/var/log/messages日志中出现大量的例如以下信息：
proxy-access Error sending UDP message to (‘127.0.0.1‘, 8125): [Errno 1] Operation not permitted
proxy-access Error sending UDP message to (‘127.0.0.1‘, 8125): [Errno 1] Operation not permitted
kernel: __ratelimit: 89 callbacks suppressed
kernel: nf_conntrack: table full, dropping packet.
kernel: nf_conntrack: table full, dropping packet.
kernel: nf_conntrack: table full, dropping packet.
解决的方法：
依据上面的信息。得知8125port是StatsD服务port，因此是StatsD的client出了问题。
同一时候内核报出了丢包错误。主要是因为server防火墙开启了过滤机制导致的（net.ipv4.netfilter.ip_conntrack_max太小）。在此将防火墙关闭就可以(service iptables stop)
參考：
[1] https://bugs.launchpad.net/swift/+bug/1183152
[2] http://www.cyberciti.biz/faq/ip_conntrack-table-ful-dropping-packet-error/
[3] http://stackoverflow.com/questions/6240951/sendto-operation-not-permitted-netsnmp
版权声明：本文博主原创文章。博客，未经同意不得转载。