Comprehensive Guide to the Windows Paging File 495
busfahrer writes "Adrian's Rojak Pot has a nice article about the internals of the Windows paging file. It explains what a paging file is and lists the differences between a swapfile and a paging file. But first and foremost, a large part of the article deals with the various methods of optimizing the Windows paging file, thus yielding a notable performance gain for people who are not overly blessed with RAM."
Defrag first, man. (Score:4, Informative)
I've come across workstations where the paging file is in thousands of fragments. This happens when someone messes with the settings. For instance, they might increase the size of the paging file thinking it'll help to have more. Normally, it's not a bad idea to increase it but if the drive is heavily fragmented, Windows dutifully uses the fragments for the new space.
The only way to fix it is to completely delete (deactivate) the page file, then do a defrag, then re-create the page file (several reboots involved).
That's probably the best way to tune the page file. There, I saved you from having to take the time to read the article.
Re:Defrag first, man. (Score:1, Informative)
Let Windows manage the pagefile. (Score:3, Informative)
But the best way to optimize the paging file in Win2000 (and later) is to leave it alone. Windows will manage the paging system just fine on its own.
FreeRAM (Score:5, Informative)
A free application that you can use to 'pre-page' out pages right before loading up your application. What it does is hog as much RAM as much as it can, forcing the OS to page out any unnessecary applications.
I've seen the standard Explorer + lsass + cwrss + all the svchosts memory footprint go from 80-ish megs to 20. Running this before your game will allow quick load-times and quicker performance.
Re:swap file vs. paging file (Score:5, Informative)
Here [iu.edu] is some more info about paging and swapping under unix
AFAIK a page is an group of memory addresses that are being changed/addressed at the same time.
But I could be mistaken
One of the first paragraphs made me stop reading. (Score:5, Informative)
Whenever the operating system has enough memory, it doesn't usually use virtual memory.
This flies in the face of real world experience. You can have 4 gigs of RAM and nothing but Windows 2000 running and your OS will still use the swap file frequently. Don't believe me? Run Performance Monitor and monitor Memory, Pages/Sec and just click on a few things and you'll see that I'm correct.
Another interesting one re: XP and page file (Score:3, Informative)
Virtual Memory in Windows XP [aumha.org] http://aumha.org/win5/a/xpvm.htm [aumha.org]
Windows 98 & WinME Memory Management [aumha.org] http://aumha.org/win4/a/memmgmt.htm [aumha.org]
and there is
How can I optimize the Windows 2000/XP/2003 virtual memory (Pagefile)? [petri.co.il] http://www.petri.co.il/pagefile_optimization.htm [petri.co.il]
CAUTION: Ignorant Article Writer detected (Score:5, Informative)
Take most, if not all, of what the article discusses with a large grain of salt. Everything, from his history (did Microsoft invent demand paging? Hardly) to his terminology is flawed.
Just reading the first 40 comments or so here reveals that VM remains one of those "black magic" areas, where reason is overcome by superstition and people will assert the most ridiculous, irrational and unsupportable things. Regrettably, the contents of this article do nothing to improve the situation.
= Mike
Re:Or (Score:5, Informative)
Re:Faster/easier method (Score:2, Informative)
Re:swap file vs. paging file (Score:5, Informative)
Linux only has a paging file (it's still called swap space though). This can either be a hard drive partition, or a regular file.
To make it as a regular file:
dd if=/dev/zero of=some_file bs=1M count=however_big_you_need_it_in_megs
Then:
mkswap some_file
Then:
swapon some_file
You don't need to reboot, and you can add/remove these files at will using swapon/swapoff and the normal filesystem tools.
The 'swappiness' of Linux can also be tuned: since kernel 2.6.0 there has been a proc file
echo 40 >/proc/sys/vm/swappiness
Most 2.6-based distros have some GUI tool that can tweak parameters like this (Fedora certainly does).
pagefile vs. swapfile (Score:4, Informative)
There is no difference.. He says that swapfiles would swap whole processes. I beg you pardon? Working on whole processes hasn't been the case since 'multiprogramming' on third generation computers (around 1965-1980).
btw, a good program to defrag your Windows page file is PageDefrag [sysinternals.com]
Together with Dirms & Buzzsaw [dirms.com], you can keep your disk defragmented for free. Especially Buzzsaw is nice since it will defragment recently accessed files in the background.
article still doesn't address misconceptions (Score:5, Informative)
I've just finished a project involving reconstructing process images from physical memory, including pages from swap, if available, so I've got a pretty good understanding of this stuff at a very low level.
Misconception 1: Swap-file usage = performance degradation
Yes, it is slower (usually by 3 orders of magnitude, not 1) to access a page (frequently 4K) from disk instead of memory. HOWEVER, effiency dictates that all available RAM be utilized as soon as possible.
For example, in addition to running processes, we also use volatile memory storage to cache file data. Clearly, we want to cache as much as possible. Page replacement policies then become important to determine how much swap space to use. But usually it is much greater than 0, because we've got process image pages that are less frequently used than a lot of file cache pages. So we've gotta remember that data, but we don't wanna waste fast RAM on it.
In other words, isn't it great that we can swap out pages from an unused process to make room for frequently accessed file data? Regardless of how much memory we have, that's a Good Thing.
Misconception 2: Virtual memory = disk space
Virtual memory is a mechanism for translating program-visible addresses to arbitrary storage locations transparently. This doesn't necessarily mean that disk space is used for swapping memory, it means, for example, that 5 processes can simultaneously be accessing address 0x5000, but the actual storage location is different for each. If the system (usually the page address translation facility on the CPU) determines that this address isn't at some location in volatile storage, it will bring in that memory from swap space and possible page out some other data. This is what the article is generally talking about.
I've seen some other questions about pages. There are a couple reasons for treating memory in page-size chunks, where a page is usually in the 1-8k range (4k for x86). First, the page address translation stuff needs to keep data on translations. It must do this in > 1 byte chunks, since keeping translation data on every byte would require way too much storage. Disk I/O and other I/O is frequently done on page-by-page basis for that reason as well as for the sake of performance.
Well, I rambled enough. Just wanted to clear a couple things up.
Another link to article (Score:5, Informative)
Re:swap file vs. paging file (Score:3, Informative)
Re:Rojak Pot? (Score:3, Informative)
The term Rojak Pot, I assume, is meant to portray that this website talks about a wide range of topics.
Very poor article (Score:2, Informative)
Moreover, all these performance claims are bandied about, and yet I see no benchmarking.
Try harder, windows folks.
Re:Defrag first, man. (Score:3, Informative)
Sysinternals does good work.
Re:linux swap (Score:2, Informative)
Linux knows about two styles of swap areas, old style and new style. The last 10 bytes of the first page of the swap area distinguishes them: old style has `SWAP_SPACE'new style has `SWAPSPACE2' as signature.
In the old style, the rest of this first page was a bit map, with a 1 bit for each usable page of the swap area. Since the first page holds this bit map, the first bit is 0. Also, the last 10 bytes hold the signature. So, if the page size is S, an old style swap area can describe at most 8*(S-10)-1 pages used for swapping. With S=4096 (as on i386), the useful area is at most 133890048 bytes (almost 128 MiB), and the rest is wasted. On an alpha and sparc64, with S=8192, the useful area is at most 535560992 bytes (almost 512 MiB).
The old setup wastes most of this bitmap page, because zero bits denote bad blocks or blocks past the end of the swap space, and a simple integer suffices to indicate the size of the swap space, while the bad blocks, if any, can simply be listed. Nobody wants to use a swap space with hundreds of bad blocks. (I would not even use a swap space with 1 bad block.) In the new style swap area this is precisely what is done. The maximum useful size of a swap area now depends on the architecture. It is roughly 2GiB on i386, PPC, m68k, ARM, 1GiB on sparc, 512MiB on mips, 128GiB on alpha and 3TiB on sparc64.
so.. this is it..
Re:Before the defrag (Score:5, Informative)
The first one will defragment your Windows pagefile on each boot. And Buzzsaw will defragment recently accessed files in the background, much less intrusive than Diskkeeper. Both are freeware tools.
Re:Does he really believe this? (Score:3, Informative)
Where does it say MS invented this?
It's implied in the phrase "they came up with" and in saying that the term Virtual Memory is one of "Microsoft's terms." They didn't come up with it. It was a concept widely used in computing since 1959 [gmu.edu]. Everything used virtual memory by the time MS included it in Windows 95 -- even the Macintosh. The concept and the word are ancient, but the article presents it like it's some sort of wonderful innovation invented and named by geniuses at Microsoft.
It's the kind of statements only an MS fanboy or someone else equally uninformed about the history of computing could make.
Re:corrupt (Score:5, Informative)
For example, even under Windows 3.1, the swap file only swapped in chunks of 4/8k. It's just that, on machines that were memory-poor (like we all were way back when), most users had their boxes configured with a 3/1 ratio of swap to real ram, and most of their free real ram then ended up being used to manage the swap file. 2/1 was the sweet spot.
The DOS Shell program, on the other hand, worked by swapping out whole processes, allowing DOS 5.0 and up to appear to multitask.
Pick up a copy of any decent assembler manual for the 2/386 from the early '90s, look for the instructions for swapping pages into ram. It's a hardware function. On a page fault, the cpu then goes through a look-aside buffer (8k worth of pointers each, IIRC, local and global), and then you can map pages of ram in/out as required. Just don't triple-fault, as the cpu would then go out of "enhanced" mode and/or reboot.
Must be a REALLY SLOW day for this to be "news".
Re:Or (Score:2, Informative)
I had run the Windows 2000 default defragger quite often before this, and most of the actual files were defragmented just fine, but there was tons of green slivers in the display there before I ran Diskeeper, and only a few chunks of solid green afterwards. I got rid of diskeeper because I didn't like it running in the background all the time, but its reboot defrag process was pretty good.
The Windows 2000/XP defragmentor is a limited version of the Diskeeper defragmentor that MS licensed. By definition the limited version does not defrag the MFT or Registry hives. Also, you don't have to run Diskeeper in the background. There is a configuration setting for this! Just run it when you want to defrag
Re:One of the first paragraphs made me stop readin (Score:4, Informative)
I don't know why screen savers would stop working, but I bet the developers never planned for the creation of shared memory failing.
Re:Does he really believe this? (Score:5, Informative)
Virtual memory is the mapping of physical memory pages to a "virtual" memory address using hardware translation tables. This is done so that every application lives in its own private memory space, and cannot interfere with other applications' memory (or the OS's). Basically, this technique of memory isolation keeps user apps from crashing the system or other applications. Virtual memory support has been added to x86 with the release of the 80386 processors and 32-bit flat memory addressing. Of course, virtual memory has been available for years before this on such OS's as DEC's VMS (the Virtual Memory System), IBM's MVS, UNIX, and a bunch of other systems I'm too young to know about.
The misnaming of demand paging was actually started by Apple (continuing their tradition of calling the box a "CPU") for their swap file management (long before MacOS's support for VM in OSX).
Re:Let Windows manage the pagefile. (Score:3, Informative)
Sorry, but that's not good advice. There are real issues with fragmentation on NTFS file systems. You can create an empty NTFS partition copy a few files to it, and you can be sure that if the files are larger than 4KB, those files will be fragmented. And if they are of substantial size, the files can be split into dozens of pieces. Moreoever, Windows provide no native ability to defragment metadata on any partition.
With respect to the swap file, Windows provides no native ability to defragment it. A default installation with default settings will have your swap file spread like diarrhea across the sytem partition. What is good advice is either to set the swap to a fixed min/fixed max, and/or invest in a defragment utitilty to replace the stripped down version of Diskkeeper that comes bundled as the default defragmenter. (Note that, when possible, replacing most most anything on any Windows box with better alternatives is *always* a good idea so leaving most anything to Windows to manage is generally a bad idea.)
Among the commerical alternatives available, Diskeeper and PerfectDisk are excellent choices, and provide the ability to defragment files, metadata and the swap file.
Also note that you will get noticeable performance boost if you mvoe the swap file is located on a separate drive (different IDE channel).
Re:FreeRAM (Score:4, Informative)
You can change the behavior by modifying the config.trim_on_minimize key (accessible via about:config). See http://windowssecrets.com/041202/ [windowssecrets.com]) for more info on this.
Makes a big difference - no more two second pauses when restoring firefox.
The author fundamentally doesn't understand... (Score:3, Informative)
...just to confuse things a bit more (Score:3, Informative)
Then you can put your windows pagefile into linux swap partitions. :-)
Comment removed (Score:4, Informative)
Re:Or (Score:2, Informative)
XP still has no buit-in method of defragging the pagefile or registry hives, however.
Re:swap file vs. paging file (Score:3, Informative)
You can then mount this file as a swap partition. You can make a fstab entry to mount this file at boot up as a swap partition.
Please see this URL:
http://enterprise.linux.com/article.pl?sid=05/03/
Re:Defrag first, man. (Score:1, Informative)
Have a nice day and thanks for playing.