file-descriptors

File Descriptors (FD) in Linux & Unix Systems

A file descriptor (FD) is a non-negative integer that uniquely identifies an open file, socket, or other I/O resource in a Linux or Unix-like system. Every process has a table of file descriptors that track open files.

---

1. Standard File Descriptors

Every process starts with three standard file descriptors:

FD Number	Name	Description
0	stdin	Standard input (keyboard, pipe input)
1	stdout	Standard output (console, file, pipe output)
2	stderr	Standard error output (for error messages)

For example:

echo "Hello" > output.txt   # Redirects stdout (FD 1) to a file
echo "Error!" 2> error.txt  # Redirects stderr (FD 2) to a file

---

2. Viewing Open File Descriptors

Each process has an open file descriptor table stored in /proc/<PID>/fd/.

To list all open file descriptors for a process:

ls -l /proc/$(pgrep nginx)/fd/

Or for the current shell:

ls -l /proc/$$/fd/

255-file-descriptor

3️⃣ How Are File Descriptors and Inodes Connected?

When a process opens a file, the kernel:

1. Looks up the inode of the file.
2. Checks permissions.
3. Creates an open file table entry, which:
   • Stores the inode reference.
   • Maintains the file offset (read/write position).
4. Assigns a file descriptor (FD) to the process, pointing to the open file entry.

Example:

• If multiple processes open the same file, they have different file descriptors, but they point to the same inode.
• If you delete a file that a process has open, the file still exists in memory (until all file descriptors pointing to it are closed).

You can check inode-FD mappings using:

3. Checking System-Wide Open File Descriptors

To see the total number of open file descriptors:

cat /proc/sys/fs/file-nr

To list all open files across all processes:

lsof | less

For a specific process (e.g., MySQL):

lsof -p $(pgrep mysqld)

---

4. Setting File Descriptor Limits

Each process has a soft limit (can be changed by the user) and a hard limit (set by the system).

Check Current Limits

ulimit -n   # Show max open files per process
ulimit -Sn  # Show soft limit
ulimit -Hn  # Show hard limit

Important

Non-root users can only increase the soft limit up to the hard limit. Only the root user can change the hard limit.

Temporarily Increase Limits (Until Logout)

ulimit -n 100000

Permanently Increase Limits

Edit /etc/security/limits.conf:

your_username  hard  nofile  100000
your_username  soft  nofile  50000

Then, update PAM settings in /etc/pam.d/common-session:

session required pam_limits.so

For system-wide changes, edit /etc/systemd/system.conf Configuration Files:

DefaultLimitNOFILE=100000

Tip

Thilakarathne@fedora:/usr/lib/systemd$ cat system.conf | grep -i defaultlimitnofile
#DefaultLimitNOFILE=1024:524288

• 1024 is the soft limit.
• 524288 is the hard limit.

Then restart the system:

sudo systemctl daemon-reexec

---

5. Common Issues with File Descriptors

”Too Many Open Files” Error

Occurs when a process exceeds the ulimit -n value. Fix:

1. Increase file descriptor limits as explained above.

2. Check for file descriptor leaks:

   lsof -p <PID>

3. Close unused file descriptors in scripts:

   exec 3>&-   # Close FD 3

---

6. Practical Example: Managing File Descriptors in a Script

#!/bin/bash
exec 3>logfile.txt  # Open FD 3 for writing
echo "Logging info..." >&3  # Write to FD 3
exec 3>&-  # Close FD 3

---

7. File Descriptors in Networking

Sockets and pipes also use file descriptors. To monitor active network connections:

lsof -i -n -P

To check open network ports:

netstat -tulnp

---

Key Takeaways

✅ Every open file/socket is assigned a file descriptor.
✅ ulimit -n controls max open FDs per process.
✅ Use lsof, /proc/<PID>/fd/, and ulimit to monitor & manage FDs.
✅ Increase FD limits if needed to prevent errors in high-traffic applications.

Would you like a deep dive into debugging FD leaks or programming with FDs in C/Python? 🚀