How can I optimize the Windows 2000/XP/2003 virtual memory (Pagefile)?

How can I optimize the Windows 2000/XP/2003 virtual memory (Pagefile)?

Memory tweaking is an important part of the System Administrator's responsibilities. Having too much memory won't hurt you, but this is something we cannot expect to have all the time. This page will give you some working tips on memory management for Windows 2000, Windows XP and Windows Server 2003.

Warning: Before you start working with these tips you should read my Before you begin page.

Thanks to reader Perris Calderon for the heads-up on some needed fixes on this page (3/02/2003)

Parts of this page have information taken from MS MVP Alex Nichol's excellent article about XP Virtual Memory found HERE. Other pieces of information were collected from a blog written by Mike Lee, HERE.

What is Virtual Memory?

In a 32-bit computer, the memory addresses are 32 bits long and stored as binary (base 2) numbers. There are approximately 4 billion possible different 32-bit binary numbers (2^32=4,294,967,296). Because of this, there is a 4GB limit for addressable memory in a 32-bit computer.

A program instruction on an Intel 386 or later CPU can address up to 4GB of memory, using its full 32 bits. Each process is assigned an address space of 4GB of virtual memory, regardless of the amount of available physical memory. Each process is isolated from the rest and has its own 4GB address space. This means that the 4GB addressability limit applies on a per-application basis, not across all applications taken together.

This is normally far more than the RAM of the machine. The amount of physical memory on the computer is not related to the amount of memory address space. If a computer has 256MB of physical memory, there is still a 4GB memory address space, and if a computer has 8GB of physical memory, there is still a 4GB memory address space.

Applications are not allowed direct access to physical memory. When an application requests more memory, Windows maps some physical memory (as long as some is available) into the process's address space.

The hardware provides for programs to operate in terms of as much as they wish of this full 4GB space as Virtual Memory, those parts of the program and data which are currently active being loaded into Physical Random Access Memory (RAM). Windows maintains several tables that keep track of all of this, and the application knows only about the virtual memory address.

The processor itself then translates (‘maps’) the virtual addresses from an instruction into the correct physical equivalents, doing this on the fly as the instruction is executed. The processor manages the mapping in terms of pages of 4KB each - a size that has implications for managing virtual memory by the system.

Why do I need page file optimization?

Optimizing your page file when you're running low on RAM is always a good idea. When all physical RAM in a computer is in use, Windows starts using the hard disk as if it were additional RAM. This is why we have a Pagefile (also called the swap file). Because RAM memory is a lot more faster than the hard disk, whenever the computer begins to use the Pagefile to relieve memory pressure, we begin to experience a drastic performance degradation.

One of the most effective things you can do to improve performance is ensure that there is enough RAM available to avoid frequent paging (swapping) of memory contents between disk and RAM.

This means that the actual limit on the memory used by all applications is the amount of RAM installed plus the maximum size of the Pagefile.

How much swap space do you need? That depends the amount of RAM you have and the programs you use. The rule of the thumb is 1.5 times the amount of system memory, unless you have too much load on your system.

Can the Virtual Memory be turned off on a really large machine?

Strictly speaking Virtual Memory is always in operation and cannot be ‘turned off’. What is meant by such wording is ‘set the system to use no page file space at all’.

This would waste a lot of the RAM. The reason is that when programs ask for an allocation of Virtual memory space, they may ask for a great deal more than they ever actually bring into use - the total may easily run to hundreds of megabytes. These addresses have to be assigned to somewhere by the system. If there is a page file available, the system can assign them to it - if there is not, they have to be assigned to RAM, locking it out from any actual use.

Do you have any specific tweaking tips?

Here are some basic rules you should follow:

• First and most important - Add more RAM. No, you do NOT need to put in 4GB of RAM, but if your computer constantly utilizes X MB of memory, then make sure you have at least X MB of RAM available for it, and preferably more. There is no point in overloading a computer with graphic applications and limiting it to 256MB of RAM. While the applications will probably work, your performance will be lowered to the ground. Adding enough RAM will ensure that all the required memory used will indeed be used from the RAM, and not supplemented from the Pagefile.

• Move the Pagefile off the disk that holds your system and boot partitions to another fast and dedicated hard disk. If you do put the file elsewhere, you should leave a small amount on C: - an initial size of 2MB with a Maximum of 50 is suitable - so it can be used in emergency. Without this, the system is inclined to ignore the settings and either have no page file at all (and complain) or make a very large one indeed on the C: drive.

• Format the partition where the page file is placed with NTFS and a 4kb cluster size (which is in fact the default setting for an NTFS partition).

• Have the initial size be at least 1.5 times bigger than the amount of physical RAM. Do NOT make the Pagefile smaller than the amount of physical RAM you've got installed on your system.

• Make its initial size as big as the maximum size. Although this will cause the Pagefile to occupy more HD space, we do not want it to start off small, then having to constantly grow on the HD. Writing large files (and the Pagefile is indeed large) to the HD will cause a lot of disk activity that will cause performance degradation. Also, since the Pagefile only grows in increments, you will probably cause Pagefile fragmentation, adding more overhead to the already stressed HD.

• Do not place multiple paging files on different partitions on the same physical disk drive.

• If you have a RAID-0 (Stripe Set) array, use it to store the Pagefile.

• Avoid putting a paging file on a fault-tolerant drive, such as a mirrored volume (RAID-1) or a RAID-5 volume. Paging files do not need fault-tolerance, and some fault-tolerant systems suffer from slow data writes because they write data to multiple locations.

• If you use Windows XP and Fast User Switching, there are special considerations: When a user is not active, there will need to be space available in the page file to ‘roll out’ his or her work: therefore, the page file will need to be larger. Only experiment in a real situation will establish how big, but a start point might be an initial size equal to half the size of RAM for each user logged in.

If you can spare an extra Hard Disk (preferably a fast one) or if you have some free HD space on your system (again, preferably on a fast HD, and NOT on the HD that's holding the SYSTEM or DATA partitions) I'd suggest you use it for the sole purpose of the Pagefile. Remember that using another partition on the same SYSTEM or DATA HD will not improve your system's performance at all. To do that you'll need a separate fast HD, preferably connected to a different controller.

You can also improve performance by spreading your page file across multiple physical disks, don't worry about the relative speeds of the hard drives Windows 2000/XP/2003 automatically selects the fastest drive to page memory to.

When creating page files, don't confuse hard drives with partitions. You shouldn't create page files on multiple partitions on the same hard drive. This set-up degrades system performance significantly because when Windows NT writes to these page files, the disk arm of the hard drive is forced to swing back and forth across the disk rather than being able to stay in the general area of the single page file. For those with IDE hard drives, it is recommended to place the page file on separate IDE channels.

Let's say we have 512MB of RAM and we decided to create a Pagefile of 768MB.

In Windows 2000 go to My Computer, right click it and then choose Properties, go to the Advanced tab, click Performance Options, then click Change. Now you can view and set the parameters you need:

Notice how Windows 2000/XP/2003 place the Pagefile on the boot partition (I know it says "system" in the image, but still they reference it as the "boot" partition. In this case it's both), the one the OS is installed on (in this case C:\). Windows 2000/XP/2003 also make the Pagefile's initial size 1.5 times the size of the physical memory (RAM) installed. The maximum size is 3 times your RAM. Windows NT used a different calculation here.

You can write your settings in the initial and maximum boxes, and then click Set. You'll have to reboot in order for the changes to take place. Note that you have to specify the *same* amount for both values. This will keep your Pagefile from resizing, fragmenting and eventually slowing itself down.

You can also divide the Pagefile between your physical disks (not partitions!) to speed up its performance. Select the required disk from the Drive list, write the sizes in the initial and maximum boxes, and then click Set. Repeat this for all the required drives.

In Windows XP go to My Computer, right click it and then choose Properties, go to the Advanced tab, click Performance Options, then click Settings.

Again click the Advanced tab and then in the Virtual Memory section click Change.

Now you can view and set the parameters you need.

Under Drive [Volume Label], click the drive that contains the paging file you want to change.

Under Paging file size for selected drive, click Custom size, type a new paging file size in megabytes (MB) in the Initial size (MB) or Maximum size (MB) box, and then click Set.

If you decrease the size of either the initial or maximum paging file settings, you must restart your computer to see the effects of those changes. When you increase the paging file size, you typically do not need to restart your computer.

How big should my Pagefile be?

To have Windows choose the best paging file size, click System managed size. The recommended minimum size is equivalent to 1.5 times the amount of RAM on your system, and 3 times that figure for the maximum size. Example, if you have 256 MB of RAM, the minimum size would be 384, the maximum size would be 1152.

For best performance, do not set the initial size to less than the minimum recommended size under Total paging file size for all drives. The recommended size is equivalent to 1.5 times the amount of RAM on your system. Usually, you should leave the paging file at its recommended size, although you might increase its size if you routinely use programs that require a lot of memory.

To delete a paging file, set both initial size and maximum size to zero, or click No paging file. Microsoft strongly recommends that you do not disable or delete the paging file.

To determine your preferred Pagefile size you should monitor your system and see how much RAM and virtual memory it uses. Do a baseline monitoring for a week or so (you can skip the baseline or shorten its duration if you know exactly how much Pagefile you need) and see what was the largest amount of memory your system ever used. If you see it never gets close to the maximum amount of RAM you have installed – don't worry. Your Pagefile is virtually left untouched (not exactly correct, because the system uses the Pagefile for some internal operations and it should keep it at around 5%-10% usage on normal systems). If, on the other hand, you see your system uses more memory than it physically has, this means it uses some Pagefile.

How much Pagefile does your system use?

You should first find out how much memory you're using, and how much of it is on your disk.

Using System Monitor

One accurate way is by monitoring the %Usage Peak counter in the System Monitor tool (The peak usage of the Pagefile instance in percent):

Open Performance console from the Administrative Tools. Click System Monitor and then point at the PLUS (+) sign.

In the Add Counters dialog box open the Performance Objects drop-down list. Look for the Paging File object.

When you click that object select the %Usage counter and click Add.

Look at the graph. Notice if it's steady or increasing in size (you might want to capture this data over a longer period of time - a week or more - by using the System Logs in the Performance Logs and Alerts sub-menu of the Performance console). Notice the Average counter.

Using Task Manager

Another (faster) way to find out is by using Task Manager:

Open Task Manager and go to the Performance Tab.

Notice the Physical Memory section. Look at the Total figures: 785904 K (that's the amount of installed RAM).

How much RAM is available? 372924 K – more than half of the installed RAM. You're doing ok for now.

How much memory is your system committed to? Look at the Commit Charge section, at the Total figures: 429604 K.

What was the largest amount of memory your system has ever committed itself to since the last boot? Look at the Peak figures: 453168 K. This means that you're running close to your peak, and although your system has peaked to around 450000 K, it's still far from using up its RAM. You're ok for now.

Bill James MS MVP has a convenient tool, ‘WinXP-2K_Pagefile’, for monitoring the actual usage of the Page file, which can be downloaded here. A compiled Visual Basic version is available from Doug Knox's site which may be more convenient for some users. The value seen for ‘Peak Usage’ over several days makes a good guide for setting the Initial size economically.

Now, since our system automatically sets its initial Pagefile size to 1.5 times the amount of RAM, you're using 1152 MB or hard disk space on a Pagefile that's not even used (Remember? You're not even close to using ALL you RAM…) Seems like a lot of space to waste. You can safely lower the Pagefile size to around 700 MB, and you can safely assume you'll never even touch it. The only drawback it losing the ability to create a dump file in case your system BSODs on you…

I've taken another snapshot, this time of a Windows XP computer:

Here the Peak was 758064 K, which is ALMOST all the RAM you have. You cannot tell if this was a one-time surge, or something you should expect all the time (here you do need a baseline to tell). You can only say that right now you're safe because you're only using 192992 K or memory out of 785904 K you have. But if you do see that the peak is something that always happens to you, you should set your Pagefile to the minimum of 1.5 times the RAM (which is in fact the default setting).

By the way, you can also see the current amount of memory used by looking at the lower right corner of Task Manager, where it says Commit Charge: 188M / 1426M.

This next snapshot is of a Windows Server 2003 which is having problems:

Had this been YOUR computer, then you could have had problems on your system. Here you're using 530MB out of 934MB of memory you have (made of RAM + Pagefile). This means your system is relying on the Pagefile to operate. Here you must add more RAM ASAP, or at least configure the Pagefile for optimum use.

Are there any more tweaks that can be done to optimize the use and location of the Pagefile?

Yes. There are a few available options.

Registry Values:

Even when you set the values properly, some systems don't retain your settings. If this happens, perform the following steps:

1. Start regedt32.exe (not regedit.exe)

2. Go to

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management

3. Double-click PagingFiles. The entry will appear as

   D:\pagefile.sys 700 700

4. The first value is the location; the second is the minimum size; and the third is the maximum size.

5. Ensure the values are correct and click OK.

6. Reboot the machine.

Remotely changing Pagefile size and location:

You can also change the value of the Pagefile remotely by manipulating registry values.

Pagefile information is stored in the registry as a multi_string (so you HAVE to use regedt32.exe) and can be changed locally as follows:

1. Start the registry editor (regedt32.exe)

2. Move to

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management

3. Double click PagingFiles

4. There is one line for each pagefile, the format is:

   <location> <initial size> <maximum size>

   (e.g. C:\pagefile.sys 700 700)

5. Click OK

6. Close the registry editor

7. Reboot the machine

To change on another machine you should use the resource kit REG.EXE utility but the command below will replace the current Pagefile and will NOT check you have enough disk space so you may want to create a script that does check. Make sure the machine is rebooted after the change.

C:\> reg update "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PagingFiles"="<location> <initial size> <max size>" \\<remote machine>

For example:

C:\> reg update "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PagingFiles"="C:\pagefile.sys 700 700" \\london.nwtraders.msft

Make sure you test this before trying to use on live machines.

Pagefile Fragmentation

If the Pagefile gets fragmented your system's performance will decrease. System Internals has released PageDefrag, a free utility that shows fragmentation in the Pagefile and then offers the option of defragmentation at boot time.

The utility can be downloaded from HERE
(Thanks to Doug Bloch for the updated link 2/9/07)

Once you download just unzip the file and run pagedfrg.exe. Below is a sample output:

Group Policy Settings

There are a few Group Policy settings available for use.

Create a Pagefile:

Computer Configuration\Windows Settings\Security Settings\Local Policies\User Rights Assignment

This setting determines which users and groups can create and change the size of a Pagefile. You can create a Pagefile by specifying a paging file size for a given drive in System Properties Performance Options.

Default: Administrators

Shutdown: Clear virtual memory Pagefile:

Computer Configuration\Windows Settings\Security Settings\Local Policies\Security Options

This setting determines whether the virtual memory Pagefile is cleared when the system is shut down. On a running system, the Pagefile is opened exclusively by the operating system, and it is well protected. However, systems that are configured to allow booting to other operating systems might have to make sure that the system Pagefile is wiped clean when this system shuts down. This ensures that sensitive information from process memory that might go into the Pagefile is not available to an unauthorized user who manages to directly access the Pagefile.

When this policy is enabled, it causes the system Pagefile to be cleared upon clean shutdown. If you enable this security option, the hibernation file (hiberfil.sys) is also zeroed out when hibernation is disabled on a portable computer system.

Default: Disabled

Problems with Virtual Memory

It may sometimes happen that the system give ‘out of memory’ messages on trying to load a program, or give a message about Virtual memory space being low. Possible causes of this are:

• The setting for maximum size of the page file is too low, or there is not enough disk space free to expand it to that size.

• The page file has become corrupt, possibly at a bad shut-down. In the Virtual memory settings, set to have no page file, exit, shut down the machine and reboot. Delete PAGEFILE.SYS (on each drive if more than just C:), set the page file up again and reboot to bring it into use.

• The page file has been put on a different drive without leaving a minimal amount on C:.

• There is trouble with third party software. In particular, if the message happens at shutdown, suspect a problem with Symantec's Norton Live update, for which there is a fix posted HERE.
  If it happens at boot and the machine has an Intel chipset, the message may be caused by an early version (before version 2.1) of Intel's ‘Application Accelerator’. Uninstall this and then get an up-to-date version from Intel's site.

• Possibly there is trouble with the drivers for IDE hard disks; in Device Manager, remove the IDE ATA/ATAPI controllers (main controller) and reboot for Plug and Play to start over.

• With an NTFS file system, the permissions for the root directory of the drive where the page file is must give ‘Full Control’ to SYSTEM. If not, there is likely to be a message at boot that the system is ‘unable to create a page file’.

Links

Configuring Page Files for Optimization and Recovery - 197379

How to Configure Paging Files for Optimization and Recovery in Windows XP - 314482

HOW TO: Move the Paging File in Windows XP - 307886

HOW TO: Set Performance Options in Windows XP - 308417

HOW TO: Configure Recovery Techniques in Windows XP - 307973