The Key to Enhancing File System Performance in the Enterprise

 

 

It has long been recognized that file fragmentation is detrimental to Windows system performance.  Several studies have examined the impact of file fragmentation on overall performance of file systems and the negative impact fragmentation has on the total cost of ownership (TCO) for IT departments. Enterprises of virtually all sizes have implemented enterprise-wide defragmentation projects for a variety of reasons, including:

 

 

Microsoft and industry analysts concur on the problem. Indeed, Gartner Research has observed that "the Windows built-in defragmentation tool is a multi-pass defragmenter that must be run over and over to defragment the disk, especially when defragmenting very large disks with heavy fragmentation and limited free space. As such, multi-pass defragmenters characteristically fragment the remaining free space on the disk, which accelerates fragmentation later." 

 

The problem is exacerbated, particularly on critical servers, as disk sizes have evolved to several terabytes. If ignored, fragmentation can cause large drives in particular to deteriorate to a point where performance can be brought to a near standstill. This has a negative impact across the organization.

 

Defragmentation, seemingly a straightforward concept, has actually evolved considerably over the past few years.  Free space consolidation has received significantly increased interest over the last few years, and is now recognized as a critical element of the overall defragmentation concept. In order to understand why free space consolidation is so important to the overall concept of file fragmentation, this article will address some key points to consider.

 

File fragmentation is a function of how the file system allocates space to a file. To create a file, the NTFS file system looks to the $Bitmap file to determine where space is available. The $Bitmap file identifies which logical clusters are in use and which ones are free.  If the file system cannot allocate space for the entire file in a contiguous string of logical clusters, the file will be fragmented. When a read or write request is received for that file, the Master File Table is accessed and it provides the starting logical cluster number (LCN) and the run length for each fragment needed to satisfy the requested read range. The more fragments there are, the longer it takes to read the file, as each noncontiguous read incurs a disk seek10 (approximately 10 msec on today's disks, or about 30 million wasted processor cycles on a 3GHz processor). If a read request spans 10 fragments, the file system needs to report the 10 starting LCN's and run lengths to the disk controller.

 

When the file system receives a read/write request that spans a fragmented region of a file, it must split the single input/output request packet (IRP) into multiple associated request packets that read or write the scattered data. By measuring both the number of requests sent to the file system and the number of resulting requests sent to the disk, one  can determine the number of extraneous, or wasted, disk requests and thus wasted seeks.

 

When free space is not consolidated, the result is typically an increase in the number of free space fragments and overall fragmentation. And that results in increased seeks on the drive when reading and writing files. While historically fragmentation studies focused on read access times, modern studies focus on write access times as well. When the number of free space fragments increases, drive performance suffers. One often-overlooked fact is that write performance, not just read performance, suffers. In other words, defragmenting a drive without free space consolidation does not reliably increase drive performance, and can in fact, decrease it, because the number of wasted seeks increases. There are many real-world scenarios where file fragments are decreased and free space fragmentation increases, and the result is worse drive performance and an increased number of wasted seeks.

 

In conclusion, consolidating free space is singularly effective in postponing refragmentation of a file system.  When a file is copied to a volume or a new file created and its size set, Windows is successful in using contiguous free space when it is available.  If sufficient contiguous free space is not available, the file is created in a fragmented condition. This finding is consistent with the observation Gartner made in its assessment of the built-in Windows defragmenter and multi-pass defragmentation engines.

 

In summary: