Increase the maximum number of file descriptors

dockerlinux

What is file descriptors?

File desecriptors is an identifier for interacting with an outside of a process. In POSIX, it is an int type, 0 is stdin, 1 is stdout, and 2 is stderr. It is assigned by a system call such as open() to open files and devices, and socket() to create sockets to communicate with other processes.

Maximum number of file descriptors

There is a limit on the number of file descriptors a process can use so that one process does not consumpt all resources. You can get current process’ limit by ulimit -n.

$ ulimit -n
1024

You can get each process’ limit and file descriptors which is being assigned by following commands.

$ cat /proc/<pid>/limits
...
Max open files            1024                 4096                 files     
...

$ ls -l /proc/<pid>/fd

A process which communicates with a lot of outsides at the same time, such as web server, can reach this limit and occurs Too many open files error so the limit is needed to be increased. There are following methods to change the limit.

Change with /etc/security/limits.conf

Since these settings are applied at the time of PAM authentication when login etc, daemons started when server starts are not available.

$ cat /etc/pam.d/sshd
...
session    required     pam_limits.so
...

Following settings increase the maximum number of file descriptors processes of all users (*) can use to 64000 both soft limit, users can cange, and hard limit, only root user can change.

$ echo "* hard nofile 64000" >> /etc/security/limits.conf
$ echo "* soft nofile 64000" >> /etc/security/limits.conf
$ ulimit -n
64000

Change with ulimit -n

This value is applied to only current shell and processes started from it.

$ ulimit -n 64000

(If docker) Run with –ulimit

$ docker run -itd --ulimit nofile=11111 ubuntu
$ docker exec -it <id> /bin/bash -c "ulimit -n"
11111

References

/etc/security/limits.confに関するメモ | OpenGroove

ファイルディスクリプタ数の上限変更とlimits.confの罠

Linuxのファイルディスクリプタ数を変更・確認する方法: 小粋空間

ファイル記述子 - Wikipedia

調べなきゃ寝れない!と調べたら余計に寝れなくなったソケットの話 - Qiita