Thursday, October 7, 2010

Resolving Too many files open error in Linux

Situation:
Users couldn't execute *ANY* commands. It gives "Too many open files" error as shown below:

orabi@miaash02-t1$ top
ksh: top: /usr/bin/top: cannot execute [Too many open files in system]
dwilliams@miaash02-t1$ ls -l
ksh: ls: /bin/ls: cannot execute [Too many open files in system]


Reason:
Everything in Linux are files; Linux forks most things including devices, sockets and pipes as files.  There is a kernel parameter called “file-max” which controls the maximum number of files that can be opened in a system. The default value is 65K (approx), can be find using the following command:
“sysctl -a | grep file-max”.
To check the count of number of files open, we can use the following command: “lsof | wc –l”. However this will not give you the exact number, because it is possible for a single file to be opened multiple times for readind and each additional concurrent open will increase the count for file-max value.  And in addition even connections to network ports can eat up the ‘file-max’ value.

Resolution:
As a temporary solution, we can increase the ‘file-max’ value by issuing the following command:
# echo “value” > /proc/sys/fs/file-max and then we need to identify the problem by analyzing either the System logs or by using lsof command itself with appropriate options. 

1 comment:

  1. Does the max number is 65535?? All of my centos5.5 systems are set to be that value, but today, one of them that handled nfs service is in bad status, I can't connect to it.

    ReplyDelete