This page delivers up the results of some performance testing I have done on Linux filesystems. The tests were done with a very specific application in mind --- web caching. And will only apply to similar applications where there are a large number of small files and considerable activity.
The test server was an HP DL380G2 with two PIIIs running at 1.2Ghz, 1Gbyte of memory, a RAID controller with 2 disks configured as RAID1 and 6 further disks configured as RAID0. The operating system (and an optional partition for a journalling device) was installed on the mirrored device; the large stripped RAID was reserved as the cache storage.
The system was installed with RedHat9, and a 'custom' kernel installed (2.4.20) with patches to support XFS in addition to the standard list of ext2, ext3, ReiserFS, and JFS.
Because of stabilty issues and attrocious performance figures, JFS has been excluded from the results. I suspect that the problems are my fault somehow, and this page may contain updated results if I figure out the cause.
The first two benchmarks used were naive application specific benchmarks. A 9.4Gbyte tar archive containing a thin slice through another web cache server was copied onto the test system; the first benchmark was the time taken to extract all the files averaged over five runs. The second benchmark was the time taken to remove all the files averaged over 5 runs.
The third benchmark was PostMark which is allegedly good at benchmarking for mail server tasks. An average over 5 runs was taken.
| Filesystem | tar avg | rm avg | PostMark avg |
|---|---|---|---|
| ext2 | 412.6s | 172.7s | 500 IOPs |
| ext3 | 561.3s | 189.3s | 500 IOPs |
| reiserfs | 434.2s | 196.6s | 500 IOPs |
| reiserfs (notail) | 350.7s | 120.6s | 500 IOPs |
| XFS | 1447.0s | 736.9s | 500 IOPs |
| XFS with logdev | 1552.4s | 877.45s | N/A |
The first obvious conclusion is that the Postmark benchmark needs to be re-run with more strenuous parameters as the results show no differentiation. The fastest filesystem is ReiserFS with the 'notail' option, but ext2 and ext3 are not far behind. XFS is disappointingly slow; in normal use XFS does not seem to be anywhere near as slow. Perhaps the slant of this test towards small files, and deletion shows up XFS weaknesses.
The second (but less pronounced) surprise is how well ReiserFS performs in comparison to ext2 ... journalling is supposed to slow things down. But it would appear that ReiserFS's designed ability to cope with small files is certainly reflected in reality. The poor result of XFS is surprising and certinly isn't reflected when using XFS in more normal ways ... it is a bit unscientific, but I use XFS extensively and it does not feel slow in comparison to the other filesystems.
These results indicate that ReiserFS with the notail option is probably the best choice for a file system with many small files. More benchmarking needs to be done however.