redis celery too many connection

用django 框架，异步任务用celery，队列用redis

出现了这个问题，too many connection

Couldn‘t ack ‘5f41afc62d-a112-bef34d5de1cc‘, reason:ConnectionError(‘Too many connections‘,)

Traceback (most recent call last):

File "/srv/www/wom/env/lib/python2.6/site-packages/kombu/message.py", line 93, in ack_log_error

self.ack()

File "/srv/www/wom/env/lib/python2.6/site-packages/kombu/message.py", line 88, in ack

self.channel.basic_ack(self.delivery_tag)

File "/srv/www/wom/env/lib/python2.6/site-packages/kombu/transport/virtual/__init__.py", line 566, in basic_ack

self.qos.ack(delivery_tag)

File "/srv/www/wom/env/lib/python2.6/site-packages/kombu/transport/redis.py", line 152, in ack

self._remove_from_indices(delivery_tag).execute()

File "/srv/www/wom/env/lib/python2.6/site-packages/redis/client.py", line 1914, in execute

self.shard_hint)

File "/srv/www/wom/env/lib/python2.6/site-packages/redis/connection.py", line 395, in get_connection

connection = self.make_connection()

File "/srv/www/wom/env/lib/python2.6/site-packages/redis/connection.py", line 402, in make_connection

raise ConnectionError("Too many connections")

ConnectionError: Too many connections

看了文档之后，要设个变量

BROKER_POOL_LIMIT

New in version 2.3.

The maximum number of connections that can be open in the connection pool.

The pool is enabled by default since version 2.5, with a default limit of ten connections. This number can be tweaked depending on the number of threads/greenthreads (eventlet/gevent) using a connection. For example running eventlet with 1000 greenlets that use a connection to the broker, contention can arise and you should consider increasing the limit.

If set to None or 0 the connection pool will be disabled and connections will be established and closed for every use.

Default (since 2.5) is to use a pool of 10 connections.

CELERY_REDIS_MAX_CONNECTIONS

Maximum number of connections available in the Redis connection pool used for sending and retrieving results.

BROKER_TRANSPORT_OPTIONS

New in version 2.2.

A dict of additional options passed to the underlying transport.

See your transport user manual for supported options (if any).

Example setting the visibility timeout (supported by Redis and SQS transports):

BROKER_TRANSPORT_OPTIONS = {‘visibility_timeout‘: 18000}  # 5 hours



要设置 max_connections 才能解决下面的问题
				


				
时间： 2024-10-02 22:29:22 

		


		


	

	
	

		


		
redis celery too many connection的相关文章


								

		

			

                Redis in python, how do you close the connection?
			

		

		

									

				down voteaccepted Just use redis.Redis. It uses a connection pool under the hood, so you don't have to worry about managing at that level. If you absolutely have to use a low level connection, you need to do the response handling that is normally don			

		

	

				

		

			

                多个celery如何使用同一个redis做为broker？
			

		

		

									

				曾经有个哥们提了一个这方面的requests,但是最终没有合入的celery中去,所以目前celery没有这个功能,祥见: https://github.com/celery/kombu/pull/912 https://github.com/celery/kombu/issues/853 主要思路是给key加前缀.公司java语言的redis client已经实现了加前缀的功能,这个各个应用就不会出现使用相同key的问题 这个方案可以参考Run multiple Celeries on a s			

		

	

				

		

			

                虚拟环境virtualenv，安装MariaDB，redis简介，
			

		

		

									

				2. pyhton虚拟环境virtualenv 2.1 虚拟环境介绍 问题: 我要想在一台机器上面安装两个版本的django,该怎么做? 解决方案: 1 再买一台机器,在这台机器上安装python3解释器,再安装django2 还可以在原来这台机器上再安装一个版本的django(这个方法不可行)3 再重新编译安装一个python解释器(特别麻烦,很费时间) 原来的python3解释器, 以它为本体, 创建多个分身 第一个分身django2.2.0?第二个分身django3.0?第三个分身djan			

		

	

				

		

			

                redis基础入门
			

		

		

									

				:感觉介绍的浅显易懂,所以推荐 What is Redis? Redis是 Salvatore Sanfilippo 开发的,开源的,高效的基于内存的键值缓存服务,所谓的NoSQL数据库 .它可以存储 strings, hashes, lists, sets, sorted sets等结构的数据. 键值存储的实质是能够存储数据,而且在之后调用具体的键名,可以检索出之前插入的数据. Usage of Redis Salvatore Sanfilippo (creator of Redis) 说Re			

		

	

				

		

			

                Django使用celery异步发邮件
			

		

		

									

				Celery是Python开发的分布式任务调度模块,包含以下组件: Celery Beat: 任务调度器,自带的 Celery Worker: 执行任务的消费者,通常设置多个 Broker: 消息代理,就是任务队列,我们使用redis Producer: 任务生产者,要执行的函数加上@app.task Result Backend: 结果保存,还是redis Celery安装 1 pip install celery[redis] 还要安装redis 1 wget http://download			

		

	

				

		

			

                SaltStack源码分析之使用Redis模块
			

		

		

									

				Redis模块路径 /usr/lib/python2.6/site-packages/salt/modules/redismod.py Redis模块会首先检查是否安装有Redis的Python驱动 # -*- coding: utf-8 -*- ''' Module to provide redis functionality to Salt .. versionadded:: 2014.7.0 :configuration: This module requires the redis py			

		

	

				

		

			

                Redis Clients Handling
			

		

		

									

				This document provides information about how Redis handles clients from the point of view of the network layer: connections, timeouts, buffers, and other similar topics are covered here. The information contained in this document is only applicable t			

		

	

				

		

			

                Python连接Redis连接配置
			

		

		

									

				1. 测试连接: Python 2.7.8 (default, Oct 20 2014, 15:05:19) [GCC 4.9.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import redis >>> redisClient = redis.StrictRe			

		

	

				

		

			

                Redis详解（二）
			

		

		

									

				一.redis认证二.redis事务三.Connection&server相关命令四.Redis的发布与订阅//充当mq的功能五.Redis持久化六.redis复制七.master-slave实现八.sentinal哨兵九.redis限制十.redis性能测试        一.redis认证redis认证:    vim /etc/redis.conf         requirepass redispass //密码    [[email protected] ~]# systemctl