Linux: Open files (systemd, ulimit, netstat, lsof)

ulimit

$ man limits.conf
NAME
       limits.conf - configuration file for the pam_limits module

DESCRIPTION
       The pam_limits.so module applies ulimit limits, nice priority and number of
       simultaneous login sessions limit to user login sessions. This description of
       the configuration file syntax applies to the /etc/security/limits.conf file
       and *.conf files in the /etc/security/limits.d directory.

Also see:

Systemd resource limit

 

Overriding the Default systemd Configuration Using system.conf

The default configuration of systemd is defined during the compilation and it can be found in systemd configuration file at /etc/systemd/system.conf. Use this file if you want to deviate from those defaults and override selected default values for systemd units globally.

For example, to override the default value of the open files limit, which is set to 4096, use  the DefaultLimitNOFILE in /etc/systemd/system.conf

$ systemctl show | grep NOFILE
DefaultLimitNOFILE=4096
DefaultLimitNOFILESoft=1024

$ sudo vi /etc/systemd/system.conf
DefaultLimitNOFILE=8192

$ sudo systemctl daemon-reload

$ systemctl show | grep NOFILE
DefaultLimitNOFILE=8192
DefaultLimitNOFILESoft=8192
$ sudo systemctl show  | grep DefaultLimit
DefaultLimitCPU=18446744073709551615
DefaultLimitCPUSoft=18446744073709551615
DefaultLimitFSIZE=18446744073709551615
DefaultLimitFSIZESoft=18446744073709551615
DefaultLimitDATA=18446744073709551615
DefaultLimitDATASoft=18446744073709551615
DefaultLimitSTACK=18446744073709551615
DefaultLimitSTACKSoft=8388608
DefaultLimitCORE=18446744073709551615
DefaultLimitCORESoft=0
DefaultLimitRSS=18446744073709551615
DefaultLimitRSSSoft=18446744073709551615
DefaultLimitNOFILE=4096
DefaultLimitNOFILESoft=1024
DefaultLimitAS=18446744073709551615
DefaultLimitASSoft=18446744073709551615
DefaultLimitNPROC=3901
DefaultLimitNPROCSoft=3901
DefaultLimitMEMLOCK=65536
DefaultLimitMEMLOCKSoft=65536
DefaultLimitLOCKS=18446744073709551615
DefaultLimitLOCKSSoft=18446744073709551615
DefaultLimitSIGPENDING=3901
DefaultLimitSIGPENDINGSoft=3901
DefaultLimitMSGQUEUE=819200
DefaultLimitMSGQUEUESoft=819200
DefaultLimitNICE=0
DefaultLimitNICESoft=0
DefaultLimitRTPRIO=0
DefaultLimitRTPRIOSoft=0
DefaultLimitRTTIME=18446744073709551615
DefaultLimitRTTIMESoft=18446744073709551615

Example systemd unit file that set LimitNOFILE and LimitNPROC

$ sudo cat /etc/systemd/system/bitnami.service
[Unit]
SourcePath=/etc/init.d/bitnami
Description=LSB: bitnami init script
Before=runlevel2.target runlevel3.target runlevel4.target runlevel5.target shutdown.target
After=serial-getty@ttyS0.service
Wants=network-online.target
Conflicts=shutdown.target

[Service]
Type=forking
Restart=no
TimeoutSec=30min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
SysVStartPriority=1
ExecStart=/etc/init.d/bitnami start
ExecStop=/etc/init.d/bitnami stop
LimitNOFILE=65536
LimitNPROC=32768

# Output needs to appear in instance console output
StandardOutput=journal+console

[Install]
WantedBy=multi-user.target

user limit

$ tldr ulimit

  ulimit

  Get and set user limits.
  More information: https://manned.org/ulimit.

  - Get the properties of all the user limits:
    ulimit -a

  - Get hard limit for the number of simultaneously opened files:
    ulimit -H -n

  - Get soft limit for the number of simultaneously opened files:
    ulimit -S -n

  - Set max per-user process limit:
    ulimit -u 30
$ sudo cat /etc/security/limits.conf
# /etc/security/limits.conf
#
#Each line describes a limit for a user in the form:
#
#            
#
#Where:
# can be:
#        - a user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#        - NOTE: group and wildcard limits are not applied to root.
#          To apply a limit to the root user,  must be
#          the literal username root.
#
# can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
# can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open files
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit (KB)
#        - maxlogins - max number of logins for this user
#        - maxsyslogins - max number of logins on the system
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#        - sigpending - max number of pending signals
#        - msgqueue - max memory used by POSIX message queues (bytes)
#        - nice - max nice priority allowed to raise to values: [-20, 19]
#        - rtprio - max realtime priority
#        - chroot - change root to directory (Debian-specific)
#
#                 
#

#*               soft    core            0
#root            hard    core            100000
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#ftp             -       chroot          /ftp
#@student        -       maxlogins       4

# End of file

Check process limit with cat /proc/<PID>/limits

E.g.

$ sudo cat /proc/3406/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             32768                32768                processes
Max open files            1024                 1048576              files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       3901                 3901                 signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us
$ sudo cat /proc/3406/limits | grep 'open files'
Max open files            1024                 1048576              files

netstat

$ tldr netstat

  netstat

  Displays network-related information such as open connections, open socket ports, etc.
  More information: https://www.unix.com/man-page/osx/1/netstat.

  - List all ports:
    netstat -a

  - List all listening ports:
    netstat -l

  - List listening TCP ports:
    netstat -t

  - Display PID and program names for a specific protocol:
    netstat -p protocol

  - Print the routing table:
    netstat -nr
$ cheat netstat
# WARNING ! netstat is deprecated. Look below.

# To view which users/processes are listening to which ports:
sudo netstat -lnptu

# To view routing table (use -n flag to disable DNS lookups):
netstat -r

# Which process is listening to port 
netstat -pln | grep  | awk '{print $NF}'

# Example output: 1507/python

# Fast display of ipv4 tcp listening programs
sudo netstat -vtlnp --listening -4

# WARNING ! netstat is deprecated.
# Replace it by:
ss

# For netstat -r
ip route

# For netstat -i
ip -s link

# For netstat -g
ip maddr

lsof

$ man lsof

NAME
       lsof - list open files

SYNOPSIS
       lsof  [  -?abChKlnNOPRtUvVX  ]  [  -A  A  ] [ -c c ] [ +c c ] [ +|-d d ] [ +|-D D ] [ +|-e s ] [ +|-E ] [ +|-f
       [cfgGn] ] [ -F [f] ] [ -g [s] ] [ -i [i] ] [ -k k ] [ +|-L [l] ] [ +|-m m ] [ +|-M ] [ -o [o] ] [  -p  s  ]  [
       +|-r [t[m]] ] [ -s [p:s] ] [ -S [t] ] [ -T [t] ] [ -u s ] [ +|-w ] [ -x [fl] ] [ -z [z] ] [ -Z [Z] ] [ --
       ] [names]

...

EXAMPLES
       For a more extensive set of examples, documented more fully, see the 00QUICKSTART file of the  lsof  distribu‐
       tion.

       To list all open files, use:

              lsof

       To list all open Internet, x.25 (HP-UX), and UNIX domain files, use:

              lsof -i -U

       To list all open IPv4 network files in use by the process whose PID is 1234, use:

              lsof -i 4 -a -p 1234

       Presuming the UNIX dialect supports IPv6, to list only open IPv6 network files, use:

              lsof -i 6

       To list all files using any protocol on ports 513, 514, or 515 of host wonderland.cc.purdue.edu, use:

              lsof -i @wonderland.cc.purdue.edu:513-515

       To  list all files using any protocol on any port of mace.cc.purdue.edu (cc.purdue.edu is the default domain),
       use:

              lsof -i @mace

       To list all open files for login name ``abe'', or user ID 1234, or process 456, or  process  123,  or  process
       789, use:

              lsof -p 456,123,789 -u 1234,abe

       To list all open files on device /dev/hd4, use:

              lsof /dev/hd4

       To find the process that has /u/abe/foo open, use:

              lsof /u/abe/foo

       To send a SIGHUP to the processes that have /u/abe/bar open, use:

              kill -HUP `lsof -t /u/abe/bar`

       To find any open file, including an open UNIX domain socket file, with the name /dev/log, use:

              lsof /dev/log

       To  find processes with open files on the NFS file system named /nfs/mount/point whose server is inaccessible,
       and presuming your mount table supplies the device number for /nfs/mount/point, use:

              lsof -b /nfs/mount/point

       To do the preceding search with warning messages suppressed, use:

              lsof -bw /nfs/mount/point

       To ignore the device cache file, use:

              lsof -Di

       To obtain PID and command name field output for each process, file descriptor, file device  number,  and  file
       inode number for each file of each process, use:

              lsof -FpcfDi


       To  list the files at descriptors 1 and 3 of every process running the lsof command for login ID ``abe'' every
       10 seconds, use:

              lsof -c lsof -a -d 1 -d 3 -u abe -r10

       To list the current working directory of processes running a command that is exactly four characters long  and
       has an 'o' or 'O' in character three, use this regular expression form of the -c c option:

              lsof -c /^..o.$/i -a -d cwd

       To find an IP version 4 socket file by its associated numeric dot-form address, use:

              lsof -i@128.210.15.17

       To find an IP version 6 socket file (when the UNIX dialect supports IPv6) by its associated numeric colon-form
       address, use:

              lsof -i@[0:1:2:3:4:5:6:7]

       To find an IP version 6 socket file (when the UNIX dialect supports IPv6) by an associated numeric  colon-form
       address that has a run of zeroes in it - e.g., the loop-back address - use:

              lsof -i@[::1]

       To obtain a repeat mode marker line that contains the current time, use:

              lsof -rm====%T====

       To add spaces to the previous marker line, use:

              lsof -r "m==== %T ===="
$ cheat lsof

# To list all IPv4 network files:
sudo lsof -i4

# To list all IPv6 network files:
sudo lsof -i6

# To list all open sockets:
lsof -i

# To list all listening ports:
lsof -Pnl +M -i4

# To find which program is using the port 80:
lsof -i TCP:80

# To list all connections to a specific host:
lsof -i@192.168.1.5

# To list all processes accessing a particular file/directory:
lsof <path>

# To list all files open for a particular user:
lsof -u <username>

# To list all files/network connections a command is using:
lsof -c <command>

# To list all files a process has open:
lsof -p <pid>

# To list all files open mounted at /mount/point:
# (Particularly useful for finding which process(es) are using a mounted USB stick or CD/DVD.)
lsof +f -- <mount-point>
$ tldr lsof

lsof

Lists open files and the corresponding processes.
Note: Root privileges (or sudo) is required to list files opened by others.
More information: https://manned.org/lsof.

- Find the processes that have a given file open:
lsof path/to/file

- Find the process that opened a local internet port:
lsof -i :port

- Only output the process ID (PID):
lsof -t path/to/file

- List files opened by the given user:
lsof -u username

- List files opened by the given command or process:
lsof -c process_or_command_name

- List files opened by a specific process, given its PID:
lsof -p PID

- List open files in a directory:
lsof +D path/to/directory

- Find the process that is listening on a local IPv6 TCP port and don't convert network or port numbers:
lsof -i6TCP:port -sTCP:LISTEN -n -P

Leave a Comment

Your email address will not be published. Required fields are marked *