监听日志listener.log里面出现了 TNS-12518: TNS:listener could not hand off client connection与TNS-12540: TNS:internal limit restriction exceeded错误,如下所示,用户连接不上ORACLE数据库:
27-JAN-2015 10:10:19 * (CONNECT_DATA=(SERVICE_NAME=scm2)(CID=(PROGRAM=c:\windows\system32\inetsrv\w3wp.exe)(HOST=POS)(USER=SYSTEM)))
* (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.xxx.xxx)(PORT=1667)) * establish * scm2 * 12518
TNS-12518: TNS:listener could not hand off client connection
27-JAN-2015 10:10:19 * (CONNECT_DATA=(SERVICE_NAME=scm2)(CID=(PROGRAM=c:\windows\system32\inetsrv\w3wp.exe)(HOST=CEGWEB1)(USER=NETWO
RK?SERVICE))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.xxx.xxx)(PORT=1284)) * establish * scm2 * 12518
TNS-12518: TNS:listener could not hand off client connection
另外告警日志里面并没有发现任何错误信息。检查内存,发现物理内存只剩下261M大小,但是Swap还有13556M剩余。
查看processes参数值为870, session值为10000,当时数据库最大会话数为419,历史最大值为720,应该可以排除processes与session参数值设置过小
在官方文档查看TNS-12540: TNS:internal limit restriction exceeded错误的信息,如下所示:
TNS-12540: TNS:internal limit restriction exceeded
Cause: Too many TNS connections open simultaneously.
Action: Wait for connections to close and re-try.
TNS-12540/ORA-12540: TNS:internal limit restriction exceeded and TNS-00510: Internal limit restriction exceeded
Cause: An internal limit has been exceeded. Possible limits include:
Number of open connection that Oracle Net can process simultaneously
Number of memory buffers which can be used simultaneously
Number of processes a particular database instance is allowed
The first two are examples of hard limits. The third is an example of a limit which can be increased by setting PROCESSES parameter in the database initialization file to a larger value. In this case, a TNS-12500/ORA-12500 error is also returned. In some cases, these errors can be caused by the same conditions which cause TNS-12549/ORA-12549 and TNS-00519 errors.
Action: Perform these steps:
Wait for the open connections to close and retry. If the error persists, then check the sqlnet.log or listener.log file for detailed error stack information.
ORA-12540: TNS:internal limit restriction exceeded
Cause: Too many TNS connections open simultaneously.
Action: Wait for connections to close and re-try.
This error is not an indication of a network problem unless it occurs on all connections (in other words you have never been able to connect). If so it is a .ora configuration problem.
If you can make connections up to a certain point but then fails it is an indication of a resource limitation at the os level (this sounds like your scenario)
Typical problems are:
Out of system memory / swap
Out of process slots in the process table
Streams resources depleted
Physical connections allows by the kernal.
Out of File Handles
查看当前系统所有的资源限制.
[[email protected] ~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 193217
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65536
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 4194302
cpu time (seconds, -t) unlimited
max user processes (-u) 16384
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
为了解决问题,立马重启监听服务,问题解决了,但是是什么原因导致了这个错误呢?搜索了大量的资料,没有看到类似的案例,当然在检查过程也发现一些应用程序是使用DEDICATED连接到数据库,让相关人员修改为共享服务器连接模式,最后依然不能找到一个确切的导致TSN-12540错误的原因。纠结、研究了很久,依然不能解决问题,留待以后解决。
参考资料:
http://www.dba-oracle.com/t_ora_12540_tns_internal_limit_restriction_exceeded.htm
http://database.ccidnet.com/art/1105/20060601/569461_1.html
http://blog.itpub.net/81018/viewspace-812609/