Virtual Memory Tuning for Low-Memory (< 512M) Systems

note: This article is intended for a technical audience -- and while it seems to works for the two systems i've tried it on since, you should use caution when modifying a production system -- caveat emptor.

for anyone who doesn't care for the details: scroll down to the bottom for the 'ideal' defaults I came up with for our test rig.

Over the last few months, i've been playing with VMWare more and more on lower-end workstations -- primarily, these systems aren't mine, they're systems of friends or relatives who have not wanted to update to Microsoft Windows Vista because of the abhorrent memory requirements.

Australian PC World recently ran an article on how to use Virtualisation (via VMWare) to run Ubuntu Linux under Windows XP -- However, getting Windows to use VMWare while doing something else that it considers as being resource intensive (like Printing) turned a number of people off.

Switching things around, running Windows XP under Linux worked better, but occassionally the swap started grinding and top reported huge jumps in it's usage.

The test installation was Ubuntu 7.10 (Gutsy Gibbon), with VMWare Workstation 6.0.1 (build 55017) and Windows XP Home (having 160M RAM allocated to the VMWare image) and Word 2003.

First off, we tried turning off things that were superfluous to the 'average' home users desktop -- Tracker -- was the primary saver, freeing up around 30M of real RAM we could use for other things.

This made Windows start in a reasonable time, but resource intensive apps tended to grind the swap a second time, the 1G swap partition was pushing it's limits at 972M and there was only 4M of conventional RAM left.

Looking through the wonderfully helpful /usr/src/linux/Documentation/sysctl/vm.txt file, I figured the following might give some improvement:

overcommit_memory:

This value contains a flag that enables memory overcommitment.

When this flag is 0, the kernel attempts to estimate the amount
of free memory left when userspace requests more memory.

When this flag is 1, the kernel pretends there is always enough
memory until it actually runs out.

When this flag is 2, the kernel uses a "never overcommit"
policy that attempts to prevent any overcommit of memory.

This feature can be very useful because there are a lot of
programs that malloc() huge amounts of memory "just-in-case"
and don't use much of it.

The default value is 0.

Sure enough, if I echo this -- I get:

windows@bites:~$ cat /proc/sys/vm/overcommit_memory
0

So:

windows@bites:~$ echo 2 > /proc/sys/vm/overcommit_memory
windows@bites:~$ cat /proc/sys/vm/overcommit_memory
2

Restart VMWare, there's less grinding -- but there's also a little more usability, the CAPT-based Canon printer dialogue talked with the LBP-1120 that was hooked up to the machine in Word, but I figured I could get one better, as the 1G swap partition was still way-too-high at 920M and there was only now ~40M of conventional RAM left.

Then, I found the 'new' defaults for vm_dirty_ratio (10) and vm_dirty_background_ratio (5)

Which, as it turns out is one of the bigger bottlenecks for VMWare use on smaller systems -- after some googling, we tried 20 and 10 respectively, which caused us to have more free memory when windows was 'idle', but still grinding.

OK, what about swappiness -- i'd played with the idea of autotuning/auto-regulating this when Con Kolivas was doing kernel patches in the early days and I tend to need to tweak it on servers I look after anyway -- So, I wondered if the same 60/40 versus 40/60 principle I use on my servers would work on an overly loaded workstation?

First, what does Ubuntu do by default:

windows@bites:~$ cat /proc/sys/vm/swappiness
60

So, using the 60/40 principle I mentioned earlier:

windows@bites:~$ echo 40 > /proc/sys/vm/swappiness
windows@bites:~$ cat /proc/sys/vm/swappiness
40

Restart VMWare -- no grinding during the bootup process for XP, slight grinding when the Anti-Virus process ran -- but we were closer, now it was a case of finding the sweet spot for the dirty page ratio.

Shutdown VMWare and opened a terminal, luckily, there's a handy file in /proc we can look at to see where these pages are going, called vmstat, if we look for the word dirty, we see:

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 9988
nr_writeback 0

That's at 20, if we return the vm_dirty_ratio to the default (10), it was even worse:

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 11996
nr_writeback 322

After doing a bit of Googling on why nr_writeback would be higher than say 4-5, we found references to the Red Hat Linux Tuning Guide, which said if vm_dirty_ratio was set too low, the number of pages to be written-back would grow erratically because the ratio was constantly being hit.

OK, so 20 fixes that, but the amount of pages still looks too big, what if we made it 40?

windows@bites:~$ echo 40 > /proc/sys/vm/dirty_ratio
windows@bites:~$ cat /proc/sys/vm/dirty_ratio
40

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 7844
nr_writeback 0

Start up VMWare, load our image, fire up Word and print -- and this becomes:

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 8195
nr_writeback 1

... and (as I looked) a document came out of the printer and iTunes fired up.

A happy user, who could use Word in VMWare and Linux on the desktop -- on a Celeron 1.6G machine with 512M of RAM, without noise from the disc, the 1G swap partition evened out at about 800M and there was still 30M of conventional RAM left.

I was thanked, Evolution was set up, as was Epiphany and I put the settings we'd tweaked into /etc/rc.local file so they'd be used when the machine was next rebooted.

The sweetest spot I found, for those just looking to skip to the end -- was:

echo 10 > /proc/sys/vm/dirty_background_ratio
echo 40 > /proc/sys/vm/dirty_ratio
echo 40 > /proc/sys/vm/swappiness
echo 2 > /proc/sys/vm/overcommit_memory

Happy virtualisation country.