DBAPI部署 / 憋错料

1、添加源

sudo rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
sudo rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
sudo rpm -Uvh http://nginx.org/packages/centos/6/noarch/RPMS/nginx-release-centos-6-0.el6.ngx.noarch.rpm

2、安装nginx、mysql、uwsgi

sudo yum install nginx mysql mysql-server mysql-devel python-devel
wget -q http://peak.telecommunity.com/dist/ez_setup.py
sudo python ez_setup.py
sudo easy_install pip
sudo pip install uwsgi

sudo chkconfig --levels 235 mysqld on
sudo chkconfig --levels 235 nginx on
sudo service nginx start
sudo service mysqld start
sudo /usr/bin/mysql_secure_installation

3、配置

sudo useradd -M -s /sbin/nologin carder
sudo usermod -G dialout,nginx carder
sudo usermod -G carder nginx
sudo usermod -G carder hubery
sudo usermod -G hubery carder
sudo mkdir -p /var/log/carder/nginx/iksdb
sudo mkdir /var/run/carder
sudo mkdir /var/lock/carder
sudo chown -R carder:carder /var/log/carder /var/run/carder /var/lock/carder
sudo chmod -R g+rw /var/log/carder /var/run/carder /var/lock/carder /var/log/nginx

4、安装dbapi

cd /opt
sudo mkdir iksdb
chown -R carder：carder iksdb
git clone [email protected]:dicovi/iksdb.git ./iksdb

sudo pip install virtualenv
virtualenv venv

source ./venv/bin.active
pip install flask
pip install flask-restful
pip install peewee
pip install hsahids
pip install simplejson
pip install mysql-python
pip install httplib2
pip install restclientdeactivate
copy ~/shortcuts.py ./venv/lib/python2.6/site-packages/playhouse/

sudo ln -sf /opt/iksdb/etc/uwsgi_iksdb.ini /etc/uwsgi_iksdb.ini
sudo ln -sf /opt/iksdb/etc/nginx/uwsgi_params /etc/nginx/uwsgi_params
sudo ln -sf /opt/iksdb/etc/nginx/nginx.conf /etc/nginx/nginx.conf
sudo ln -sf /opt/iksdb/etc/nginx/conf.d/iksdb.conf /etc/nginx/conf.d/iksdb.conf
sudo ln -sf /opt/iksdb/etc/init.d/nginx /etc/init.d/nginx
sudo ln -sf /opt/iksdb/etc/init.d/uwsgi_iksdb /etc/init.d/uwsgi_iksdb

sudo chkconfig --levels 235 uwsgi_iksdb on
sudo echo 2201 > /var/run/carder/nginx.pid
sudo echo 2205 > /var/run/carder/uwsgi_iksdb.pid
sudo service uwsgi_iksdb start
sudo service nginx restart

5、防火墙配置

除了iptables外，如果selinux打开，nginx会无法绑定端口：

要么关闭selinux，临时：

sudo setenforce 0
sudo getenforce

永久：
修改 /etc/selinux/config 文件，将SELINUX=enforcing改为SELINUX=disabled，重启机器

要么配置selinux:

sudo yum provides /usr/sbin/semanage
sudo yum -y install policycoreutils-python
sudo semanage port -l|grep http
sudo semanage port -a -t http_port_t -p tcp 16000

sudo service nginx restart

6、错误处理

①uswgi_iksdb.log ->

问题：

192.168.0.187:16000 [pid: 2986|app: 0|req: 31842/64002] 192.168.0.156 () {32 vars in 404 bytes} [Thu Nov 12 18:27:57 2015] GET /api/v1.0 => generated 14 bytes in 1028 msecs (HTTP/1.0 200) 2 headers in 71 bytes (2 switches on core 0)

Thu Nov 12 18:27:54 2015 - *** uWSGI listen queue of socket "127.0.0.1:6000" (fd: 3) full !!! (101/100) ***

分析：

以这个错误为基础，查询了下相关资料，应该是系统级别参数的问题，具体可以参考 linux man page listen(2).

lzz注：简单的理解就是每个监听的socket，在没有accept之前，等待处理的socket队列长度，linux(至少在centos6.6中)默认是128，在我这个编译的uwsgi中默认是100，也就是说没有调整系统参数之前，最高也就是128。

那么怎样才能把队列的长度调整变长呢？

* 必须调整系统参数，使其生效 * 必须调整uwsgi配置，然后重启应用

解决：

修改系统参数，编辑 /etc/sysctl.conf，添加，

#对于一个经常处理新连接的高负载 web服务环境来说，默认的 128 太小了
net.core.somaxconn = 262144
?#表示SYN队列的长度，默认为1024，加大队列长度为8192，可以容纳更多等待连接的网络连接数
net.ipv4.tcp_max_syn_backlog = 8192
#网卡设备将请求放入队列的长度
net.core.netdev_max_backlog = 65536

使用 sysctl -p 重新加载参数。

uwsgi调整，编辑 /etc/uwsgi_iksdb.ini，添加，

listen=1024

重启应用，加载配置。

②/var/log/carder/nginx/error.log ->

问题：

2015/11/11 10:39:01 [emerg] 2260#0: bind() to 0.0.0.0:20000 failed (13: Permission denied)

解决：

关掉或配置selinux。

问题：

2015/11/12 18:46:03 [alert] 30272#0: setrlimit(RLIMIT_NOFILE, 32768) failed (1: Operation not permitted)

分析：

nginx.conf 中配置的 worker_rlimit_nofile 32768; 不能生效，需要调整系统参数。

解决：

In /etc/sysctl.conf or a conf file in /etc/sysconfig.d
fs.file-max = 1000000

and run /etc/sysctl -p /etc/sysctl.conf
in /etc/security/limits.conf
www-data soft nofile 500000
www-data hard nofile 800000
( assuming nginx is running as www-data ).

问题：

2015/11/12 19:11:38 [alert] 2271#0: setrlimit(RLIMIT_NOFILE, 32768) failed (13: Permission denied)

分析：

遇到这种 Permission denied 的看来一般是selinux的问题。

解决：

关掉或配置selinux。

问题：

2015/11/13 11:50:26 [alert] 2243#0: 1024 worker_connections are not enough

~$ http_load -parallel 1000 -fetches 10000 urls

...

http://192.168.0.187:16000/api/v1.0/snode: byte count wrong
10000 fetches, 1000 max parallel, 5.16988e+06 bytes, in 16.1539 seconds
516.988 mean bytes/connection
619.046 fetches/sec, 320039 bytes/sec
msecs/connect: 24.4066 mean, 1009.83 max, 0.016 min
msecs/first-response: 1510.02 mean, 2920.64 max, 0.142 min
790 bad byte counts
HTTP response codes:
code 200 -- 9210
code 500 -- 206

解决：编辑修改/etc/nginx/ngxin.conf,

events {
    worker_connections  2048;
}

③/var/log/carder/nginx/iksdb/error.log ->

问题：

2015/11/11 11:35:16 [crit] 2258#0: *3 connect() to 127.0.0.1:6000 failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: iksdbservices.vvshare.info, request: "GET /api/v1.0/customer HTTP/1.1", upstream: "uwsgi://127.0.0.1:6000", host: "127.0.0.1:16000"

解决：

关闭或配置selinux

问题：

2015/11/12 12:07:34 [alert] 2265#0: *2012 socket() failed (24: Too many open files) while connecting to upstream, client: 192.168.0.156, server: iksdbservices.vvshare.info, request: "GET /api/v1.0 HTTP/1.0", upstream: "uwsgi://127.0.0.1:6000", host: "192.168.0.187:16000"

解决：

调整系统参数

问题：

2015/11/12 12:07:42 [crit] 2265#0: accept4() failed (24: Too many open files)

解决：

调整系统参数

问题：

2015/11/12 12:08:00 [error] 2265#0: *15937 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 192.168.0.156, server: iksdbservices.vvshare.info, request: "GET /api/v1.0 HTTP/1.0", upstream: "uwsgi://127.0.0.1:6000", host: "192.168.0.187:16000"

解决：

调整系统参数

问题：

2015/11/12 12:14:47 [error] 2265#0: *36669 upstream prematurely closed connection while reading response header from upstream, client: 192.168.0.156, server: iksdbservices.vvshare.info, request: "GET /api/v1.0 HTTP/1.0", upstream: "uwsgi://127.0.0.1:6000", host: "192.168.0.187:16000"

解决：

调整系统参数

问题：

2015/11/12 12:20:10 [error] 2265#0: *50893 upstream timed out (110: Connection timed out) while connecting to upstream, client: 192.168.0.156, server: iksdbservices.vvshare.info, request: "GET /api/v1.0 HTTP/1.0", upstream: "uwsgi://127.0.0.1:6000", host: "192.168.0.187:16000"

解决：

调整系统参数

问题：

2015/11/13 10:47:29 [alert] 2235#0: *959 1024 worker_connections are not enough while connecting to upstream, client: 192.168.0.156, server: iksdbservices.vvshare.info, request: "GET /api/v1.0 HTTP/1.0", upstream: "uwsgi://127.0.0.1:6000", host: "192.168.0.187:16000"

解决：

编辑配置nginx.conf

参考：

CentOS 6.0 semanage: http://blog.csdn.net/pengwupeng2008/article/details/14451113

Nginx配置性能优化: http://blog.csdn.net/xifeijian/article/details/20956605

【uwsgi】 listen queue of socket (fd: 3) 错误分析: http://blog.csdn.net/orangleliu/article/details/48531759

Increase “Open Files Limit”: https://rtcamp.com/tutorials/linux/increase-open-files-limit/

Re: increasing open files limit： https://forum.nginx.org/read.php?2,248426,248427

socket: Too many open files (24) apache bench lighttpd: http://stackoverflow.com/questions/1504577/socket-too-many-open-files-24-apache-bench-lighttpd

Re: worker_connections is not enough while connecting to upstream： https://forum.nginx.org/read.php?2,2896,2900

时间： 2024-10-21 01:40:33