Posting Quotes In Blogger

If you use the Blogger website to post your items and you want to post backticks "'" or greater/less-than < or > symbols in the body of your text (for example, if you post code or HTML fragments) you may find you don't get the desired result when pulishing your post.

If you'd like to fix this, you can use the following HTML entities to get the same result, but Blogger won't consider these as HTML codes and will display them literally, meaning your posts will display as normal:

"'" (Backtick) = & # 39 ;
"<" (Left Bracket, Less Than) = & # 60 ;
">" (Right Bracket, Greater Than = & # 62 ;

Using mod_security 1.9.x with Zen-Cart (for users)

Since my last post, i've had a few people ask me what happens if they're with a hosting service that doesn't want to adopt the changes I mentioned to get Zen-Cart to work properly.

The answer, yes -- if you're stuck with a site that doesn't (or can't) use the changes above, you can filter them yourself, using the .htaccess files method that Zen-Cart uses.

Of course, this method has a few prerequisites:

1. Your hosting provider uses the standard GotRoot Rules for mod_security (ie. they haven't modified any of the ID's).


2. Your hosting provider allows the use of .htaccess files (if you can install Zen-Cart on your host at all, this is probably true).

To make this work, you need to edit two files, the main .htaccess file and the admin/.htaccess file.

important note: These changes are Case Sensitive, make sure you copy them exactly as written (or better than that, Copy-and-Paste them directly into your editor software).

Firstly, edit the main .htaccess file and add the following lines to the bottom:

<IfModule mod_security.c>
<LocationMatch "/ipn_main_handler.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>
</IfModule>

Save this file and upload it to your webhost, this should disable the required rules to make the PayPal IPN processor work correctly.

Next, edit your admin/.htaccess (if you have renamed your admin/ directory as suggested by the Zen-Cart instructions, put that directory name in instead) file and add the following lines to the bottom:

<IfModule mod_security.c>
<LocationMatch "/admin/attributes_controller.php">
SecFilterRemove 300015
</LocationMatch>

<LocationMatch "/admin/newsletters.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>

<LocationMatch "/admin/product.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>
</IfModule>

Save this file and upload it to your webhost, this should disable the required rules to make the Administration Pages work correctly.

Virtual Memory Tuning for Low-Memory (< 512M) Systems

note: This article is intended for a technical audience -- and while it seems to works for the two systems i've tried it on since, you should use caution when modifying a production system -- caveat emptor.

for anyone who doesn't care for the details: scroll down to the bottom for the 'ideal' defaults I came up with for our test rig.

Over the last few months, i've been playing with VMWare more and more on lower-end workstations -- primarily, these systems aren't mine, they're systems of friends or relatives who have not wanted to update to Microsoft Windows Vista because of the abhorrent memory requirements.

Australian PC World recently ran an article on how to use Virtualisation (via VMWare) to run Ubuntu Linux under Windows XP -- However, getting Windows to use VMWare while doing something else that it considers as being resource intensive (like Printing) turned a number of people off.

Switching things around, running Windows XP under Linux worked better, but occassionally the swap started grinding and top reported huge jumps in it's usage.

The test installation was Ubuntu 7.10 (Gutsy Gibbon), with VMWare Workstation 6.0.1 (build 55017) and Windows XP Home (having 160M RAM allocated to the VMWare image) and Word 2003.

First off, we tried turning off things that were superfluous to the 'average' home users desktop -- Tracker -- was the primary saver, freeing up around 30M of real RAM we could use for other things.

This made Windows start in a reasonable time, but resource intensive apps tended to grind the swap a second time, the 1G swap partition was pushing it's limits at 972M and there was only 4M of conventional RAM left.

Looking through the wonderfully helpful /usr/src/linux/Documentation/sysctl/vm.txt file, I figured the following might give some improvement:

overcommit_memory:

This value contains a flag that enables memory overcommitment.

When this flag is 0, the kernel attempts to estimate the amount
of free memory left when userspace requests more memory.

When this flag is 1, the kernel pretends there is always enough
memory until it actually runs out.

When this flag is 2, the kernel uses a "never overcommit"
policy that attempts to prevent any overcommit of memory.

This feature can be very useful because there are a lot of
programs that malloc() huge amounts of memory "just-in-case"
and don't use much of it.

The default value is 0.

Sure enough, if I echo this -- I get:

windows@bites:~$ cat /proc/sys/vm/overcommit_memory
0

So:

windows@bites:~$ echo 2 > /proc/sys/vm/overcommit_memory
windows@bites:~$ cat /proc/sys/vm/overcommit_memory
2

Restart VMWare, there's less grinding -- but there's also a little more usability, the CAPT-based Canon printer dialogue talked with the LBP-1120 that was hooked up to the machine in Word, but I figured I could get one better, as the 1G swap partition was still way-too-high at 920M and there was only now ~40M of conventional RAM left.

Then, I found the 'new' defaults for vm_dirty_ratio (10) and vm_dirty_background_ratio (5)

Which, as it turns out is one of the bigger bottlenecks for VMWare use on smaller systems -- after some googling, we tried 20 and 10 respectively, which caused us to have more free memory when windows was 'idle', but still grinding.

OK, what about swappiness -- i'd played with the idea of autotuning/auto-regulating this when Con Kolivas was doing kernel patches in the early days and I tend to need to tweak it on servers I look after anyway -- So, I wondered if the same 60/40 versus 40/60 principle I use on my servers would work on an overly loaded workstation?

First, what does Ubuntu do by default:

windows@bites:~$ cat /proc/sys/vm/swappiness
60

So, using the 60/40 principle I mentioned earlier:

windows@bites:~$ echo 40 > /proc/sys/vm/swappiness
windows@bites:~$ cat /proc/sys/vm/swappiness
40

Restart VMWare -- no grinding during the bootup process for XP, slight grinding when the Anti-Virus process ran -- but we were closer, now it was a case of finding the sweet spot for the dirty page ratio.

Shutdown VMWare and opened a terminal, luckily, there's a handy file in /proc we can look at to see where these pages are going, called vmstat, if we look for the word dirty, we see:

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 9988
nr_writeback 0

That's at 20, if we return the vm_dirty_ratio to the default (10), it was even worse:

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 11996
nr_writeback 322

After doing a bit of Googling on why nr_writeback would be higher than say 4-5, we found references to the Red Hat Linux Tuning Guide, which said if vm_dirty_ratio was set too low, the number of pages to be written-back would grow erratically because the ratio was constantly being hit.

OK, so 20 fixes that, but the amount of pages still looks too big, what if we made it 40?

windows@bites:~$ echo 40 > /proc/sys/vm/dirty_ratio
windows@bites:~$ cat /proc/sys/vm/dirty_ratio
40

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 7844
nr_writeback 0

Start up VMWare, load our image, fire up Word and print -- and this becomes:

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 8195
nr_writeback 1

... and (as I looked) a document came out of the printer and iTunes fired up.

A happy user, who could use Word in VMWare and Linux on the desktop -- on a Celeron 1.6G machine with 512M of RAM, without noise from the disc, the 1G swap partition evened out at about 800M and there was still 30M of conventional RAM left.

I was thanked, Evolution was set up, as was Epiphany and I put the settings we'd tweaked into /etc/rc.local file so they'd be used when the machine was next rebooted.

The sweetest spot I found, for those just looking to skip to the end -- was:

echo 10 > /proc/sys/vm/dirty_background_ratio
echo 40 > /proc/sys/vm/dirty_ratio
echo 40 > /proc/sys/vm/swappiness
echo 2 > /proc/sys/vm/overcommit_memory

Happy virtualisation country.

Using mod_security 1.9.x with Zen-Cart (for administrators)

note: This article is intended for a technical audience -- and while it seems to works for the systems i've tried it on since, you should use caution when modifying a production system -- caveat emptor.

I use a combination of the GotRoot mod_security rules, as well as some finer-grained custom rules on a number of Zen-Cart installations I look after, occassionally though -- there are problems with false positives that need to be worked around to allow the shop owners a seamless experience.

The GotRoot rules, for those unaware -- is a collection of rules that protect against over 6000 different web vulnerabilities, covering generic PHP attacks, PHPBB, e-Commerce software and more. It is fairly all-encompassing and I recommend it highly to everyone who asks.

Some of these generic attack signatures cause false positives when applied to Zen-Cart's admin/ directory, as well as the older-style IPN payment processor for PayPal.

To counter these, create a file called zencart-exclusions.conf in your rules directory (most installations use /etc/modsecurity/rules), open that file in a text editor and add:

<LocationMatch "/admin/attributes_controller.php">
SecFilterRemove 300015
</LocationMatch>

<LocationMatch "/admin/newsletters.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>

<LocationMatch "/admin/product.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>

<LocationMatch "/ipn_main_handler.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>

The SecFilterRemove tells mod_security to selectively remove the rules from the specific locations listed in LocationMatch, while the six digit numbers are the GotRoot ID's that will not be parsed.

Save the file and reload your webserver configuration for the changes to take affect.

SMTP Unblocking with Optus (Cable or DSL)

In the process of setting up a mail server for a collegue the other night (in order to deliver mail from 6+ domains to one IMAP account) I found an interesting quirk.

After you've set up and configured a working e-mail server, with all the UBL checks and correct delivery methods in place -- if you find other SMTP transactions work OK, but the local client can't connect and you're with Optus (In Australia), go here to turn off the automated port 25 block.

Thinking on though, this procedure unblocks port 25 globally, leaving one open to the prospects of exploit code spamming people by using port 25 on the local machine -- after all, the mail server wasn't on the persons local machine or even their LAN, but halfway across the world -- the Optus site makes reference to the unblocking process being for advanced users -- wouldn't advanced users be better off leaving the block in place for all sites not in a list?

(Yes, the server could use SSL, or even be on a different port -- both would be viable alternatives to opening SMTP from a customer premises to the world)

New Rhythmbox packages for Ubuntu 7.10

Playing with Rhythmbox over the weekend (the 'save art to ipod' patch was committed to SVN and several people I know feel this is a neat feature) -- so i've built a set of updated packages from GNOME SVN (based on revision 5535) for Gutsy Gibbon.

These packages have the code for using the new Totem Playlist Parser backed out (due to it not being available on Ubuntu 7.10, but does include #411634 -- which should stop duplicated tracks appearing on your device) as well as #345975 -- so MP3's with embedded cover art will be shown, just like iTunes)

Packages are in the usual location.