Overview
In this tutorial, learn to:
- Verify the integrity of filesystems
- Monitor free space and inodes
- Repair simple filesystem problems
This tutorial covers standard and journaling (also called journaled) filesystems with an emphasis on ext2 (standard filesystem) and ext3 (journaling filesystem), but tools for other filesystems are mentioned too.
Why check filesystems?
In cases when your system crashes or loses power, Linux may not be able to cleanly unmount your filesystems. Thus, your filesystems may be left in an inconsistent state, with some changes completed and some not. Operating with a damaged filesystem is not a good idea as you are likely to further compound any existing errors.
Other problems may arise for filesystems running out of space of having sufficient filesystem management space (inodes) to continue operating. We’ll introduce you to some tools to help you manage such problems.
This tutorial helps you prepare for Objective 104.2 in Topic 104 of the Linux Server Professional (LPIC-1) exam 101. The objective has a weight of 2.
Prerequisites
To get the most from the tutorials in this series, you should have a basic knowledge of Linux and a working Linux system on which you can practice the commands covered in this tutorial. Sometimes different versions of a program will format output differently, so your results may not always look exactly like the listings and figures shown here. Most examples in this tutorial use CentOS 6, with a 2.6.32 kernel. Your results on other systems may differ.
You should also be familiar with the material in our tutorial “Learn Linux 101: Create partitions and filesystems.”
Filesystem checking tools
The main tool for checking filesystems is fsck
, which, like mkfs
, is really a front end to filesystem-checking routines for the various filesystem types. Some of the underlying check routines are shown in Listing 1.
Listing 1. Some of the fsck programs
[ian@attic4‑cent ~]$ ls /sbin/fsck
/sbin/btrfsck /sbin/fsck /sbin/fsck.ext3 /sbin/fsck.msdos
/sbin/dosfsck /sbin/fsck.cramfs /sbin/fsck.ext4 /sbin/fsck.vfat
/sbin/e2fsck /sbin/fsck.ext2 /sbin/fsck.ext4dev /sbin/fsck.xfs
You may be surprised to learn that several of these files are hard links to just one file as shown in Listing 2. Remember that these programs may be used so early in the boot process that the filesystem may not be mounted and symbolic link support may not yet be available. See our tutorial Learn Linux, 101: Create and change hard and symbolic links for more information about hard and symbolic links.
Listing 2. One fsck program with many faces
[ian@attic4‑cent ~]$ find /sbin ‑samefile /sbin/e2fsck
/sbin/fsck.ext3
/sbin/fsck.ext4
/sbin/e2fsck
/sbin/fsck.ext4dev
/sbin/fsck.ext2
The system boot process uses fsck
with the -A
option to check the root filesystem and any other filesystems that are specified for checking in the /etc/fstab control file. If the filesystem was not cleanly unmounted, a consistency check is performed and repairs are made, if they can be done safely. This is controlled by the pass (or passno) field (the sixth field) of the /etc/fstab entry. Filesystems with pass set to 0 are not checked at boot time. The root filesystem has a pass value of 1 and is checked first. Other filesystems will usually have a pass value of 2 (or higher), indicating the order in which they should be checked.
Multiple fsck
operations can run in parallel if the system determines it is advantageous, so different filesystems are allowed to have the same pass value, as is the case for the /grubfile and /home/ian/data filesystems shown in Listing 3. Note that fsck
will avoid running multiple filesystem checks on the same physical disk. To learn more about the layout of /etc/fstab, check the man pages for fstab
.
Listing 3. Boot checking of filesystems with /etc/fstab entries
filesystem mount point type options dump pass
UUID=2f60a3b4‑ef6c‑4d4c‑9ef4‑50d7f75124a2 / ext3 defaults 1 1
UUID=3c3de27e‑779a‑44d5‑ad7a‑61c5fd03d9e7 /grubfile ext3 defaults 1 2
UUID=158d605e‑2591‑4749‑bf59‑5e92e1b1c01d swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
UUID=4c962b67‑c646‑467f‑96fb‑cbbd6de40140 /home/ian/data ext4 defaults 1 2
UUID=0998d33c‑3398‑463d‑b0e3‑7c13ca0c675f /home/ian/research ext3 defaults 1 2
UUID=e3be4658‑b79b‑470d‑82fe‑bb434bcdcc2f /home/ian/pictures ext4 defaults 1 2
Some journaling filesystems, such as ReiserFS and XFS, might have a pass value of 0 because the journaling code, rather than fsck
, does the filesystem consistency check and repair. On the other hand, some filesystems, such as /proc, are built at initialization time and therefore do not need to be checked.
You can check filesystems after the system is booted. You will need root authority, and the filesystem you want to check should be unmounted first. Listing 4 shows how to check two of our filesystems, using the device name, label, or UUID. You can use the blkid
command to find the device given a label or UUID, and the label and UUID, given the device.
Listing 4. Using fsck to check filesystems
[root@attic4‑cent ~]##Find label and UUID for /dev/sdc4
[root@attic4‑cent ~]#blkid /dev/sdc4
/dev/sdc4: LABEL="IAN‑GPT‑EXT4" UUID="f69e0b28‑beda‑4255‑ad5a‑4d73672ac9e4" TYPE="ext4"
[root@attic4‑cent ~]##Check /dev/sdc4
[root@attic4‑cent ~]#fsck /dev/sdc4
fsck from util‑linux‑ng 2.17.2
e2fsck 1.41.12 (17‑May‑2010)
IAN‑GPT‑EXT4: clean, 11/2424832 files, 197218/9681152 blocks
[root@attic4‑cent ~]##Check it by label using fsck.ext4
[root@attic4‑cent ~]#fsck.ext4 LABEL=IAN‑GPT‑EXT4
e2fsck 1.41.12 (17‑May‑2010)
IAN‑GPT‑EXT4: clean, 11/2424832 files, 197218/9681152 blocks
[root@attic4‑cent ~]##Check it by UUID using e2fsck
[root@attic4‑cent ~]#e2fsck UUID=f69e0b28‑beda‑4255‑ad5a‑4d73672ac9e4
e2fsck 1.41.12 (17‑May‑2010)
IAN‑GPT‑EXT4: clean, 11/2424832 files, 197218/9681152 blocks
[root@attic4‑cent ~]##Finally check the small vfat partition /dev/sda3
[root@attic4‑cent ~]#fsck /dev/sda3
fsck from util‑linux‑ng 2.17.2
dosfsck 3.0.9, 31 Jan 2010, FAT32, LFN
/dev/sda3: 0 files, 0/1265 clusters
If you attempt to check a mounted filesystem, you may see a warning. More recent versions of fsck
will abort the check as Listing 5 shows where we try to check our root filesystem. If the check does not abort, heed the warning and do not do it!
Listing 5. Do not attempt to check a mounted filesystem
[root@attic4‑cent ~]#df /
Filesystem 1K‑blocks Used Available Use% Mounted on
/dev/sda11 79040416 7444796 67580568 10% /
[root@attic4‑cent ~]#fsck /dev/sda11
fsck from util‑linux‑ng 2.17.2
e2fsck 1.41.12 (17‑May‑2010)
/dev/sda11 is mounted.
e2fsck: Cannot continue, aborting.
It is also a good idea to let fsck
figure out which check to run on a filesystem. Running the wrong check can corrupt the filesystem. If you want to see what fsck
would do for a given filesystem or set of filesystems, use the -N
option as shown in Listing 6.
Listing 6. Finding what fsck would do to check /dev/sda7, /dev/sda8, and /dev/sda9
[root@attic4‑cent ~]#fsck ‑N /dev/sda11 /dev/sdb*
fsck from util‑linux‑ng 2.17.2
[/sbin/fsck.ext3 (1) ‑‑ /] fsck.ext3 /dev/sda11
[/sbin/fsck.ext2 (2) ‑‑ /dev/sdb] fsck.ext2 /dev/sdb
[/sbin/fsck.ext3 (3) ‑‑ /dev/sdb1] fsck.ext3 /dev/sdb1
[/sbin/fsck.ext4 (4) ‑‑ /home/ian/data] fsck.ext4 /dev/sdb2
[/sbin/fsck.ext3 (5) ‑‑ /home/ian/research] fsck.ext3 /dev/sdb3
[/sbin/fsck.ext4 (6) ‑‑ /dev/sdb4] fsck.ext4 /dev/sdb4
So far, we have checked ext and vfat filesystems. Let’s now check the XFS filesystem on /dev/sdb3. As you can see in Listing 7, the fsck
command simply tells us that we should use the xfs_check
command. If there are no errors, then xfs_check
does not display any output. There is a -v
option for verbose output, but it is much too verbose for a simple check.
Listing 7. Using fsck with XFS
[root@attic4‑cent ~]#fsck ‑N /dev/sdc3
fsck from util‑linux‑ng 2.17.2
[/sbin/fsck.xfs (1) ‑‑ /dev/sdc3] fsck.xfs /dev/sdc3
[root@attic4‑cent ~]#fsck /dev/sdc3
fsck from util‑linux‑ng 2.17.2
If you wish to check the consistency of an XFS filesystem or
repair a damaged filesystem, see xfs_check(8) and xfs_repair(8).
[root@attic4‑cent ~]#xfs_check /dev/sdc3
Monitoring free space
On a storage device, a file or directory is contained in a collection of blocks. Information about a file is contained in an inode, which records information such as who the owner is, when the file was last accessed, how large it is, whether it is a directory, and who can read from or write to it. The inode number is also known as the file serial number and is unique within a particular filesystem. See our tutorial Learn Linux, 101: File and directory management for more information on files and directories.
Data blocks and inodes each take space on a filesystem, so you need to monitor the space usage to ensure that your filesystems have space for growth.
The df command
The df
command displays information about mounted filesystems. If you add the -T
option, the filesystem type is included in the display; otherwise, it is not. The output from df
for the CentOS 6 system that we used above is shown in Listing 8. For additional interest, we have created a mount point at /mnt/btrfs-test and mounted our btrfs filesystem (/dev/sdc5) over it. We have done the same for our small vfat partition on /dev/sda3.
Listing 8. Displaying filesystem usage
[root@attic4‑cent ~]#mkdir /mnt/btrfs‑test
[root@attic4‑cent ~]#mount /dev/sdc5 /mnt/btrfs‑test
[[root@attic4‑cent ~]#mkdir /mnt/vfat‑test
[root@attic4‑cent ~]#mount /dev/sda3 /mnt/vfat‑test
[root@attic4‑cent ~]#df ‑T
Filesystem Type 1K‑blocks Used Available Use% Mounted on
/dev/sda11 ext3 79040416 7444800 67580564 10% /
tmpfs tmpfs 1961548 232 1961316 1% /dev/shm
/dev/sda1 ext3 988420 116067 821349 13% /grubfile
/dev/sdb2 ext4 124077136 49155856 68611636 42% /home/ian/data
/dev/sdb3 ext3 60458064 30808664 26578276 54% /home/ian/research
/dev/sdc1 ext4 467126880 134497524 308894012 31% /home/ian/pictures
/dev/sda5 ext4 71168700 31178752 36368096 47% /mnt/sda5
/dev/sdc5 btrfs 39062528 56 36936704 1% /mnt/btrfs‑test
/dev/sda3 vfat 2530 0 2530 0% /mnt/vfat‑test
Notice that the output includes the total number of blocks as well as the number used and free (available). Also notice the filesystem, such as ext3 for our root filesystem on /dev/sda11, and its mount point: /. The tmpfs
entry is for a virtual memory filesystem. These exist only in RAM or swap space and are created when mounted without need for a mkfs
command.
For specific information on inode usage, use the -i
option on the df
command. You can exclude certain filesystem types using the -x
option, or restrict information to just certain filesystem types using the -t
option. Use these multiple times if necessary. See the examples in Listing 9.
Listing 9. Displaying inode usage
[root@attic4‑cent ~]#df ‑i ‑x tmpfs
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda11 5021696 236498 4785198 5% /
/dev/sda1 251000 537 250463 1% /grubfile
/dev/sdb2 7904304 125115 7779189 2% /home/ian/data
/dev/sdb3 3842048 40341 3801707 2% /home/ian/research
/dev/sdc1 29671424 48944 29622480 1% /home/ian/pictures
/dev/sda5 4530176 245603 4284573 6% /mnt/sda5
/dev/sdc5 0 0 0 ‑ /mnt/btrfs‑test
/dev/sda3 0 0 0 ‑ /mnt/vfat‑test
[root@attic4‑cent ~]#df ‑iT ‑t ext4 ‑t vfat
Filesystem Type Inodes IUsed IFree IUse% Mounted on
/dev/sdb2 ext4 7904304 125115 7779189 2% /home/ian/data
/dev/sdc1 ext4 29671424 48944 29622480 1% /home/ian/pictures
/dev/sda5 ext4 4530176 245603 4284573 6% /mnt/sda5
/dev/sda3 vfat 0 0 0 ‑ /mnt/vfat‑test
You may not be surprised to see that the FAT32 filesystem does not have inodes. You may be more surprised to see no inode information for the btrfs filesystem. Btrfs (and also ReiserFS) keep inode information in dynamically allocated structures so there are no special inode blocks as there are for ext2, ext3 or ext4.
There are several other options you may use with df
to limit the display to local filesystems or control the format of output. For example, use the -h
option to display human readable sizes, such as 1K for 1024, or use the -H
(or --si
) option to get sizes in powers of 10 (1K=1000).
If you aren’t sure which filesystem a particular part of your directory tree lives on, you can give the df
command a parameter of a directory name or even a filename as shown in Listing 10.
Listing 10. Human readable output for df
[root@attic4‑cent ~]#df ‑‑si ~ian/index.html
Filesystem Size Used Avail Use% Mounted on
/dev/sda11 81G 7.7G 70G 10% /
The tune2fs command
The ext family of filesystems also has a utility called tune2fs
, which can be used to inspect information about the block count as well as information about whether the filesystem is journaled (ext3 or ext4) or not (ext2). The command can also be used to set many parameters or convert an ext2 filesystem to ext3 by adding a journal. Listing 11 shows the output for a newly created ext4 filesystem using the -l
option to simply display the existing information.
Listing 11. Using tune2fs to display ext4 filesystem information
[root@attic4‑cent ~]#tune2fs ‑l /dev/sdc4
tune2fs 1.41.12 (17‑May‑2010)
Filesystem volume name: IAN‑GPT‑EXT4
Last mounted on: <not available>
Filesystem UUID: f69e0b28‑beda‑4255‑ad5a‑4d73672ac9e4
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent flex_bg
sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags: signed_directory_hash
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 2424832
Block count: 9681152
Reserved block count: 484057
Free blocks: 9483934
Free inodes: 2424821
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 1021
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8192
Inode blocks per group: 512
RAID stride: 1
Flex block group size: 16
Filesystem created: Wed Aug 5 10:58:58 2015
Last mount time: n/a
Last write time: Thu Aug 6 10:04:27 2015
Mount count: 0
Maximum mount count: 23
Last checked: Wed Aug 5 10:58:58 2015
Check interval: 15552000 (6 months)
Next check after: Mon Feb 1 09:58:58 2016
Lifetime writes: 725 MB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 28
Desired extra isize: 28
Journal inode: 8
Default directory hash: half_md4
Directory Hash Seed: cb683b6a‑2ef7‑4588‑954a‑d163b0219652
Journal backup: inode blocks
xfs_info
For XFS filesystems you can display the same information that mkfs.xfs
displayed when the filesystem was created using the xfs_info
as shown in Listing 12. You need to use xfs_info
on a mounted filesystem.
Listing 12. Using xfs_info to display XFS filesystem information
[root@attic4‑cent ~]#mkdir /mnt/xfs‑test
[root@attic4‑cent ~]#mount /dev/sdc3 /mnt/xfs‑test
[root@attic4‑cent ~]#xfs_info /mnt/xfs‑test
meta‑data=/dev/sdc3 isize=512 agcount=16, agsize=655360 blks
= sectsz=4096 attr=2, projid32bit=0
data = bsize=4096 blocks=10485760, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii‑ci=0
log =internal bsize=4096 blocks=5120, version=2
= sectsz=4096 sunit=1 blks, lazy‑count=1
The du command
The df
command gives information about a whole filesystem. Sometimes you might want to know how much space is used by your home directory, or how big a partition to use if you wanted to move /usr to its own filesystem. To answer this kind of question, use the du
command.
The du
command displays information about the filename (or filenames) given as parameters. If a directory name is given, then du
recurses and calculates sizes for every file and subdirectory of the given directory. The result can be a lot of output. Fortunately, you can use the -s
option to request just a summary for a directory. If you use du
to get information for multiple directories, then you can add the -c
option to get a grand total. You can also control output format using the same set of size options (-h
, -H
, --si
, and so on) that are used for df
. Listing 13 shows two views of the home directory of a newly created user who has logged in once and created an index.html file.
Listing 13. Using du
[testuser@attic4‑cent ~]$ du ‑hc *
4.0K Desktop
4.0K Documents
4.0K Downloads
16K index.html
4.0K Music
4.0K Pictures
4.0K Public
4.0K Templates
4.0K Videos
48K total
[testuser@attic4‑cent ~]$ du ‑hs .
980K .
The reason for the difference between the 48K total from du -c `du-cand the 980K summary from du -s
du-s` is that the latter includes the entries starting with a dot, such as .bashrc, while the former does not.
One other thing to note about du
is that you must be able to read the directories that you are running it against.
So now, let’s use du
to display the total space used by the /usr tree and each of its first-level subdirectories. The result is shown in Listing 14. Use root authority to make sure you have appropriate access permissions.
Listing 14. Using du on /usr
[root@attic4‑cent ~]#du ‑shc /usr/*
257M /usr/bin
4.0K /usr/etc
4.0K /usr/games
132M /usr/include
383M /usr/lib
1.5G /usr/lib64
62M /usr/libexec
136K /usr/local
67M /usr/sbin
2.4G /usr/share
98M /usr/src
0 /usr/tmp
4.8G total
Repairing filesystems
Occasionally, very occasionally we hope, the worst will happen and you will need to repair a filesystem because of a crash or other failure to unmount cleanly. The fsck
command that you saw above can repair filesystems as well as check them. Usually the automatic boot-time check will fix the problems and you can proceed.
If the automatic boot-time check of filesystems is unable to restore consistency, you are usually dumped into a single user shell with some instructions to run fsck
manually. For an ext2 filesystem, which is not journaled, you may be presented with a series of requests asking you to confirm proposed actions to fix particular blocks on the filesystem. You should generally allow fsck
to attempt to fix problems, by responding y
(for yes). When the system reboots, check for any missing data or files.
If you suspect corruption, or want to run a check manually, most of the checking programs require the filesystem to be unmounted, or at least mounted read-only. Because you can’t unmount the root filesystem on a running system, the best you can do is drop to single user mode (using telinit 1telinit1
) and then remount the root filesystem read-only, at which time you should be able to perform a consistency check. A better way to check a filesystem is to boot a recovery system, such as a live CD or a USB memory key, and perform the check of your unmounted filesystems from that.
If fsck
cannot fix the problem, you do have some other tools available, although you will generally need advanced knowledge of the filesystem layout to successfully fix it.
Why journal?
An fsck
scan of an ext2 disk can take quite a while to complete, because the internal data structure (or metadata) of the filesystem must be scanned completely. As filesystems get larger and larger, this takes longer and longer, even though disks also keep getting faster, so a full check may take one or more hours.
This problem was the impetus for journaled, or journaling, filesystems. Journaled filesystems keep a log of recent changes to the filesystem metadata. After a crash, the filesystem driver inspects the log in order to determine which recently changed parts of the filesystem may possibly have errors. With this design change, checking a journaled filesystem for consistency typically takes just a matter of seconds, regardless of filesystem size. Furthermore, the filesystem driver will usually check the filesystem on mounting, so an external fsck
check is generally not required. In fact, for the xfs filesystem, fsck
does nothing!
If you do run a manual check of a filesystem, check the man pages for the appropriate fsck
command (fsck.ext3
, e2fsck
, xfs_check
, and so on) to determine the appropriate parameters. The -p
option, when used with ext2, ext3, or ext4 filesystems will cause fsck
to automatically fix all problems that can be safely fixed. This is, in fact, what happens at boot time.
We’ll illustrate the use of e2fsck
and xfs_check
by first running e2fsck
on an empty XFS filesystem and then using xfs_check
to fix it. Remember we suggested that you use the fsck
front end to be sure you are using the right checker, and we warned you that failure to do so may result in filesystem corruption.
In Listing 15, we start running e2fsck
against /dev/sda8, which contains an XFS filesystem. After a few interactions we use ctrl-Break to break out, but it is too late. Warning: Do NOT do this unless you are willing to destroy your filesystem.
Listing 15. Deliberately running e2fsck manually on an XFS filesystem
[root@attic4‑cent ~]#xfs_check ‑s /dev/sdc3
[root@attic4‑cent ~]#e2fsck /dev/sdc3
e2fsck 1.41.12 (17‑May‑2010)
e2fsck: Group descriptors look bad... trying backup blocks...
/dev/sdc3 was not cleanly unmounted, check forced.
Resize inode not valid. Recreate<y>? yes
Pass 1: Checking inodes, blocks, and sizes
Deleted inode 163841 has zero dtime. Fix<y>? ctrl‑Break
/dev/sdc3: e2fsck canceled.
/dev/sdc3: * FILE SYSTEM WAS MODIFIED *
Even if you broke out at the first prompt, your XFS filesystem would still have been corrupted. Repeat after me. Do NOT do this unless you are willing to destroy your filesystem.
Now let’s use xfs_check
to check the XFS filesystem. The xfs_check
command is quite verbose, but it has a -s
option which reports only serious errors. The output is shown in Listing 16.
Listing 16. Repairing the XFS filesystem using xfs_check
[root@attic4‑cent ~]#xfs_check ‑s /dev/sdc3
xfs_check: cannot init perag data (117)
blocks 17039360/3..3 claimed by block 17039360/0
can't seek in filesystem at bb 89335319756824
can't read agfl block for ag 17039360
can't seek in filesystem at bb 89335588200448
can't seek in filesystem at bb 89335319756800
can't seek in filesystem at bb 89335319756800
You should use xfs_repair
to repair an XFS filesystem. Like xfs_check
, it is quite verbose, and it does not have an -s
option. If you’d like just to see what needs repair without actually repairing it, use xfs_repair -n
. Listing 17 shows part of the output from xfs_repair
. As you can see, running the wrong filesystem check can do a lot of damage very quickly.
Listing 17. Repairing a damaged XFS filesystem
[root@attic4‑cent ~]#xfs_repair /dev/sdc3
Phase 1 ‑ find and verify superblock...
Phase 2 ‑ using internal log
‑ zero log...
‑ scan filesystem freespace and inode maps...
bad magic #0x1060000 for agf 0
bad version #33947648 for agf 0
bad sequence #17039360 for agf 0
bad length ‑143002337 for agf 0, should be 655360
flfirst 25427968 in agf 0 too large (max = 1024)
fllast ‑42401760 in agf 0 too large (max = 1024)
bad magic #0x80024000 for agi 0
bad version #‑2130558976 for agi 0
bad sequence #16384 for agi 0
bad length #‑25362400 for agi 0, should be 655360
reset bad agf for ag 0
reset bad agi for ag 0
freeblk count 1 != flcount 1024 in ag 0
bad agbno 33555456 for btbno root, agno 0
bad agbno 0 for btbcnt root, agno 0
bad agbno 0 for inobt root, agno 0
agi_count 1024, counted 0 in ag 0
agi unlinked bucket 0 is 8404992 in ag 0 (inode=8404992)
agi unlinked bucket 1 is 4269604896 in ag 0 (inode=4269604896)
agi unlinked bucket 2 is 1024 in ag 0 (inode=1024)
...
agi unlinked bucket 62 is 2307015680 in ag 0 (inode=2307015680)
agi unlinked bucket 63 is 2323792896 in ag 0 (inode=2323792896)
sb_icount 64, counted 0
sb_ifree 61, counted 0
sb_fdblocks 10480520, counted 9825176
root inode chunk not found
Phase 3 ‑ for each AG...
‑ scan and clear agi unlinked lists...
‑ process known inodes and perform inode discovery...
‑ agno = 0
‑ agno = 1
‑ agno = 2
...
‑ agno = 15
‑ process newly discovered inodes...
Phase 4 ‑ check for duplicate blocks...
‑ setting up duplicate extent list...
‑ check for inodes claiming duplicate blocks...
‑ agno = 1
‑ agno = 0
‑ agno = 2
...
‑ agno = 15
Phase 5 ‑ rebuild AG headers and trees...
‑ reset superblock...
Phase 6 ‑ check inode connectivity...
‑ resetting contents of realtime bitmap and summary inodes
‑ traversing filesystem ...
‑ traversal finished ...
‑ moving disconnected inodes to lost+found ...
Phase 7 ‑ verify and correct link counts...
done
Superblocks
You may be wondering how all these checking and repairing tools know where to start. Linux and UNIX filesystems usually have a superblock, which describes the filesystem metadata, or data describing the filesystem itself. This is usually stored at a known location, frequently at or near the beginning of the filesystem, and replicated at other well-known locations. You can use the -n
option of mke2fs
to display the superblock locations for an existing filesystem. If you specified parameters such as the bytes per inode ratio, you should invoke mke2fs
with the same parameters when you use the -n
option. Listing 18 shows the location of the superblocks on /dev/sda5. Note that the filesystem must not be mounted when you do this.
Listing 18. Finding superblock locations
[root@attic4‑cent ~]#mke2fs ‑n /dev/sda5
mke2fs 1.41.12 (17‑May‑2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
4530176 inodes, 18109263 blocks
905463 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
553 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424
Advanced tools
There are several more advanced tools that you can use to examine or repair a filesystem. Check the man pages for the correct usage and the Linux Documentation Project (see resources on the right) for how-to information. Almost all of these commands require a filesystem to be unmounted, although some functions can be used on filesystems that are mounted read-only. A few of the commands are described below.
You should always back up your filesystem before attempting any repairs.
Tools for ext2 and ext3 filesystems
tune2fs
Adjusts parameters on ext2 and ext3 filesystems. Use this to add a journal to an ext2 system, making it an ext3 system, as well as display or set the maximum number of mounts before a check is forced. You can also assign a label and set or disable various optional features.
dumpe2fs
Prints the super block and block group descriptor information for an ext2 or ext3 filesystem.
debugfs
Is an interactive filesystem debugger. Use it to examine or change the state of an ext2 or ext3 filesystem.
Tools for Reiserfs filesystems
reiserfstune
Displays and adjusts parameters on ReiserFS filesystems.
debugreiserfs
Performs similar functions to dumpe2fs and debugfs for ReiserFS filesystems.
Tools for XFS filesystems
xfs_info
Displays XFS filesystem information.
xfs_growfs
Expands an XFS filesystem (assuming another partition is available).
xfs_admin
Changes the parameters of an XFS filesystem.
xfs_repair
Repairs an XFS filesystem when the mount checks are not sufficient to repair the system.
xfs_db
Examines or debugs an XFS filesystem.
Tools for btrfs filesystems
Note: you may have to search online for the btrfs man pages.
btrfs
Displays many aspects of btrfs filesystem information
btrfsck
Check btrfs filesystems
btrfs-find-root
Finds the block that is the root of the btrfs filesystem
btrfs-debug-tree
Displays btrfs internal metadata
btrfstune
Tune various btrfs filesystem parameters, and enables or disables some extended features
btrfs-restore
Attempt to restore files from a damaged btrfs filesystem
We will wrap up our tools review with an illustration of the debugfs
command, which allows you to explore the inner workings of an ext family filesystem. By default, it opens the filesystem in read-only mode. It does have many commands that allow you to attempt undeletion of files or directories, as well as other operations that require write access, so you will specifically have to enable write access with the -w
option. Use it with extreme care.
Listing 19 shows how to open the root filesystem on my system; navigate to my home directory; display information, including the inode number, about a file called index.html; and finally, map that inode number back to the pathname of the file. Use ?
to get a list of available commands while in debugfs. In this example, the filesystem is mounted, which is alright for inspecting. Do not attempt repair on a mounted filesystem.
Listing 19. Using debugfs
[root@attic4‑cent ~]#debugfs /dev/sda11
debugfs 1.41.12 (17‑May‑2010)
debugfs: cd home/ian
debugfs: pwd
[pwd] INODE: 3825697 PATH: /home/ian
[root] INODE: 2 PATH: /
debugfs: stat index.html
Inode: 3832472 Type: regular Mode: 0664 Flags: 0x0
Generation: 301826695 Version: 0x00000000
User: 1000 Group: 1000 Size: 13430
File ACL: 0 Directory ACL: 0
Links: 1 Blockcount: 32
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x55c37766 ‑‑ Thu Aug 6 11:04:06 2015
atime: 0x55c37c42 ‑‑ Thu Aug 6 11:24:50 2015
mtime: 0x55940f92 ‑‑ Wed Jul 1 12:04:34 2015
Size of extra inode fields: 4
Extended attributes stored in inode body:
selinux = "unconfined_u:object_r:user_home_t:s0\000" (37)
BLOCKS:
(0‑1):15323651‑15323652, (2):15323686, (3):15342497
TOTAL: 4
debugfs: ncheck 3832472
Inode Pathname
3832472 /home/ian/index.html
debugfs: q
Conclusion
We’ve covered many tools you can use for checking, modifying, and repairing your filesystems. Remember to always use extreme care when using the tools discussed in this tutorial or any other tools. Data loss may be only a keystroke away.