Thursday 21 August 2008

Windows Defragmentation Explained!!!!

To understand defragmentation, we need to understand how Windows uses hard drive space. When we format a hard disk, hard disk is divided into sectors of 512bytes of data. To use disk I/O and space efficiently, Windows group sectors into Clusters. Cluster is a group of sectors. Cluster is the smallest unit of space available for allocation. NTFS determines the cluster size as follow (KB 314878):

Drive size

(logical volume)

Cluster size

Sectors

512 MB or less

512 bytes

1

513 MB - 1,024 MB (1 GB)

1,024 bytes (1 KB)

2

1,025 MB - 2,048 MB(2GB)

2,048 bytes (2 KB)

4

2,049 MB and larger

4,096 bytes (4 KB)

8

Over the time hard disk gets fragmented that means single file is not stored in continuous clusters. Problem is mechanical component of disk needs to do some overwork and hard disk cache as well as windows disk cache can not do read-ahead caching algorithms. Accessing cache is always faster than disk sought. This interns into performance hit and in general disk defragment is recommended. Apart from performance hit, from Rackspace Point of view, we recommend defrag in every case either sluggish server response, managed backup failure and what not (I have seen recommendation for defrag on Rackwatch tickets, don’t laugh you have seen this as well. don’t you?)!!!!

Fun apart, let’s see what defragmentation process does and its limitation as well J Defragmentation utility rearranges the files so that they are stored in physically contiguous clusters. Along with used sectors, defrag process will consolidate free space so that new files will not be defragmented when they created.

Let’s see the limitation:

1) Disk defragmentation can not defragment Recycle Bin. For efficient defrag, always empty the Recycle Bin.

2) Disk defragmenter can not touch page file unless it is zeroed out. Use PageDefrag from SysInternal when it is absolutely necessary. On a high performance server, do not leave page file to grow automatically. This makes page file to be fragmented and performance hit when initializing new page file space.

3) Disk defragmenter will not defragment files that are in use. For best results shutdown all running programs. There was a debate in past whether to shutdown SQL or not before defragment. Before recommending to customer for SQL shutdown, see point 4.

4) Disk defragment will not defrag files greater than 16,000 contiguous clusters (~64 MB on volume greater than 2 GB) because it had negligible performance improvement by default. It is possible to pin down those files and defrag. It is safe to assume that fragmented file greater than 64 MB fragments is not fragmented as far as disk caching and Windows caching goes.

Before suggesting disk defragmentation, answer following questions:

1) Is the process I/O bound? If so how many files in general it refers. Is it in 10s, 100s or 1000s. For better performance it is ideal that files are not defragmented but to make significant improvement in performance it has to be in 1000s.

2) Is process capable of High Speed I/O? (Separate post to come). In general Microsoft Office product is capable of High speed I/O. If Process uses High Speed I/O, it makes very little improvement after defragmentation.

3) Analyze the volume and check the following things?

No.

Fragments

File Size

Most Fragmented Files

a

2,586

614 MB

\SysBkUp\SystemState.bkf

b

21

1600 MB

\ProgramFiles\DebugDiag\Logs\PerfLogs\PerfLog_Date__05_06_2008__Time_09_05_16PM__161.blg

a. How Frequently Fragmented is used? E.g. in (a), fragmented file is system state backup and it is the first in Most Fragmented Files list. Are we troubleshooting the “long system state backup time” related problem?

b. Divide File Size with Fragments e.g. in (b) 1600 / 21 = 76 MB avg. fragmented unit size. Would defragmentation be able to defrag by default and if so would it make any performance improvement in terms of Disk I/O? This is exactly the case with SQL related files. If you see fragment unit is greater than 64 MB, there is very negligible performance hit and defrag with or without stopping SQL will not make noticeable improvement.

4) Do not trust Volume fragmentation report. Make reasonable guess from most fragmented files list.

In following circumstances, please proceed with defrag:

5) Is your MFT fragmented? If “Total MFT Fragments” are greater than 5, proceed with defrag. Check the analysis report and see the section MFT fragmentation:

Master File Table (MFT) fragmentation

Total MFT size = 208 MB
MFT record count = 124,340
Percent MFT in use = 58 %
Total MFT fragments = 2

6) If “Most Fragmented Files” List contains your website files and they are not cached files (e.g. .aspx and .asp are cached most of the time along with some static files like .jpg and .gif). There are very few circumstances when fragmentation is the root cause for IIS performance and most of them are related to new file creation via upload.

Learning Curve:

While troubleshooting performance problem, consider this option as last rather first. Think twice whether you really need to defrag the volume to solve the problem?

Suggestions and comments are welcome!!!

Uday Pandya

No comments: