Inode-busting

• to prevent inode busting git take first two letters of the commit and store the rest of the hashes in that folder. that way it limits the amount of files goes inside a particular folder
• CDN’s also follow this practice

What is inode busting?

Inode busting refers to a situation where the number of inodes on a filesystem is exhausted, even though there might still be plenty of free disk space available. This typically occurs when a large number of small files are created on the filesystem, and the inodes allocated to track file metadata run out.

What are Inodes?

An inode (index node) is a data structure used in many Unix-based file systems (like ext4) to store metadata about files and directories, such as:

• File size
• Ownership (user and group)
• Permissions
• Timestamps
• Pointers to data blocks

Each file or directory consumes one inode.

Causes of Inode Busting

1. Small Files: A filesystem can hold a huge number of small files (e.g., log files, cache files), exhausting the inodes while leaving most of the disk space unused.
2. Improper Filesystem Design: When the filesystem was formatted, the ratio of inodes to disk space might not have been optimized for the workload.
3. Misbehaving Applications: Applications that generate a large number of temporary files or logs without cleanup can cause inode exhaustion.

Effects of Inode Busting

• The system may not allow the creation of new files or directories.
• Applications that rely on writing temporary or output files may fail.
• Users may encounter errors like “No space left on device,” even though disk space is available.

How to Detect Inode Issues

1. Check inode usage:

   df -i

   This will show the total, used, and free inodes on each filesystem.

2. Identify directories with excessive files:

   find /path/to/check -xdev -printf '%h\n' | sort | uniq -c | sort -n

Solutions to Inode Busting

1. Cleanup Unused Files:

   • Identify and remove unnecessary or temporary files.
   • Use tools like tmpwatch or cron jobs for regular cleanup.

2. Reformat the Filesystem:

   • When reformatting a filesystem, use the -i option to set the inode ratio (e.g., more inodes for filesystems expected to handle many small files): bash mkfs.ext4 -i 1024 /dev/sdX storages

3. Move to a Different Filesystem:

   • Consider using filesystems like XFS or Btrfs, which handle large numbers of files more efficiently.

4. Application Adjustments:

   • Configure applications to aggregate small files into larger ones (e.g., logs rotation with compression).
   • Enable proper cleanup of temporary files.

Understanding inode usage patterns and designing the filesystem accordingly can prevent inode busting and ensure smooth system operations.