funk's corner of the interschnitzel

Trying Out Rygel

2009-09-18T13:32:00.002+10:00

Trying out the new, integrated into GNOME and uses GStreamer -- Media Sharing daemon, Rygel.

First thoughts, it's a _lot_ easier to configure than Mediatomb -- I am using the "media export" plugin because my Ubuntu Jaunty installation is rather bastardised, so Tracker is "unreliable at best" on my system.

I'm using the packages from James Henstridge's PPA Repository -- because they're available for Jaunty, as well as the Vala Compiler from their PPA.

Getting it going with the PS3 was easy though, first of all -- I disabled the LPCM transcoding (for some reason, none of my MP3's appeared until I did that), by unchecking the tickbox -- set the interface to wlan0, added my media directory -- and away we went.

Still have the odd issue where Wirelessly, my PS3 doesn't see anything exported from the Ubuntu box -- but i'm figuring that's more a router issue, more than an Ubuntu one -- having recently experimented with Fedora as well.

PackageKit, The Superior update-manager Alternative

2009-09-18T12:50:00.001+10:00

So, Dad decided to mess with his Ubuntu installation over the weekend -- he upgraded from Hardy to Jaunty as a one-step upgrade, which, somewhat unsurprisingly, broke heaps of stuff.

(personal opinion: It is for this reason, I hope the Ubuntu guys don't go head-long into GNOME 3.0 for "whatever-the-hell-the-L-release-is-called" if it's going to be a standard 6-month release cycle AND an LTS -- because they'll have people from 8.04 or 6.06 triggering ungrades and no-one wants a repeat of the PulseAudio debacle but on a far bigger scale. Either a) leave GNOME 2.28, continue polishing it and push GNOME 3.0 for the release after the LTS, so it gets three releases (or 2 years) of polish before it goes to the masses, or b) make the release cycle 6 months longer for an LTS, just like 6.06 was, but for the right reasons.)

Anyway, after re-installing Jaunty from scratch, fixing his RAID, re-installing his printer and playing with DKMS because his nVidia card doesn't like the newest driver in Jaunty, but will happily work with anything older -- we tripped over the 'incredible, vanishing update manager issue'.

(personal opinion II: big, big cock-up on the part of the Ubuntu guys -- Windows has an update icon, MacOSX has an update icon, every other distribution of UNIX has an update icon, but a 50+ year old man can have a taskbar full of update-manager windows, which makes the one browser window icon in the window list appear as 3-pixels wide because he doesn't understand he has to close them all individually? It would have been better to pop up a dialogue in the middle of the screen that says "Security Updates Are Available for Your Computer, would you like to install them now?" (or "Non-Essential Updates" in the case of non-security updates), or leave the icon where it was and hack in OSD notifications (like NetworkManager does) -- but don't just leave a glowing taskbar window in the bottom left hand corner, UI or Accessibility designers would turn in their graves.)

Anyway ... Given the track record that Ubuntu has for making seat-of-the-pant changes like that and taking forever to admit it was a mistake and reverting it to sensible behaviour, I decided to give PackageKit a try.

Short story -- impressive.

Installation was a simple: apt-get -f install packagekit packagekit-gnome gstreamer0.10-packagekit

The Ubuntu packages in Jaunty do add a duplicate "Update Manager" icon to the System ⇢ Administration menu, but that's purely cosmetic -- upon install it does make itself the default package update manager, which is a step in the right direction.

I have always been concerned about the way updates are presented for non-technical users, Microsoft nearly has it right -- too much information scares the non-technical user, but most UNIX variations give too much superfluous information to the user, great for technical types who actually fix issues -- and certainly something that shouldn't be removed, just 'shielded' from the user somehow).

PackageKit does this well, it gives the user notification that updates are pending in the standard way (which was our major gripe that prompted the change) but is also laid out in such a way that Dad found easier to drive than the traditional package manager (as he put it, "I already like the idea you can press a button to review changes before they happen.").

As I may of mentioned before in this blog, my dad is not a unintelligent man, he's been involved with computers, electronics, data communications, radio and just about any other form of technology you'd care to mention forever -- we've worked together on some massive projects, including hardware design, starting our own UNIX distribution and starting up smaller technology projects together -- and he has been a Linux user for nearly 5 years himself, but he's firmly a "a computer is a tool, just make the thing work" man, rather than a "let's experiment" one.

(He also builds RC Boats for a hobby, which is just cool.)

"Network Error 00000000", The PS3 and Mediatomb

2009-09-18T11:32:00.003+10:00

After requiring a fresh install of Ubuntu on my media-serving laptop due to disc failure -- I restored all the configuration settings from a backup, re-imported my library into Mediatomb, restarted the daemon and wandered into the lounge to watch TV via the PS3, a ritual with the two evening coffees before the cricket starts.

However, last night -- something strange kept happening, I could browse the videos, they even had thumbnails, but when I attempted to play them, a black screen would appear with "Network Error (000000000)".

Checked the configuration, checked the router settings, checked the wireless signal strength, all fine. (then figured, "paul, you'd not be able to see anything if these settings didn't work ;)")

Googled, no luck there either -- hence this post.

Enabled debugging mode by editing the /etc/default/mediatomb file and changing OPTIONS="" to OPTIONS="-D" and restarted the daemon and loaded up the /var/log/mediatomb.log file to find:

2009-09-17 23:34:00 DEBUG: [../src/io_handler_buffer_helper.cc:224] staticThreadProc(): starting buffer thread... thread: -1267590256
2009-09-17 23:34:02 DEBUG: [../src/process_io_handler.cc:234] read(): process exited with status 256
2009-09-17 23:34:02 DEBUG: [../src/io_handler_buffer_helper.cc:227] staticThreadProc(): buffer thread shut down. thread: -1267590256

Hmm, status 256 you say? "Could be something to do with the encoding then?" I thought.

Running ffmpeg -i testvideo.wmv -target dvd -y testvideo-ps3.avi from the command line, indeed gave me my answer:

Unknown encoder 'mpeg2video'

Turns out, the default ffmpeg installation for Ubuntu Jaunty and Karmic have mpeg2video support in ffmpeg disabled.

Two ways to fix this, the first is to implement a much, much, much longer command to encode your videos -- I use a generic script to encode videos based on their content type, but the PS3 can play (from Firmware 2.61, 3.00 and 3.01 anyway) anything encoded with the DVD profile (which encodes to MPEG2 based AVI's via the aforementioned mpeg2video codec), so instead of:

ffmpeg -i "$input" -target dvd -y "$output"

You'd use:

ffmpeg -i "$input" -vcodec mpeg1video -mbd rd -trellis 2 -cmp 2 -subcmp 2 -b 1500 -g 250 -y "$output"

(note: those commands are the only ones that are essential for making the video output work on the PS3, it'll produce a standards-compliant MPEG file with 64k MP2 audio -- but the quality may look suspect, as long as you leave those commands in, you can add things like "-flags qprd", "-bf 2" or "-flags +ilme+ildct" (good for interlaced video like animation) to improve the quality slighly)

Or, you can install the alternative versions of FFMPEG by enabling the multiverse repository (System ⇢ Administration ⇢ Software Sources) and then installing the -unstripped versions of FFMPEG with the command:

sudo apt-get -f install libavcodec-unstripped libavformat-unstripped

... and continue using the transcoding scripts you're using now :)

How can I tell if I have a broken FFMPEG installation?

The simplest way is to run: ffmpeg -formats | grep mpeg2video from the command line, if you receive:

D VSDT mpeg2video

Then you're missing the 'E'ncoding aspect of the codec, meaning you need to install the unstripped packages to make everything work correctly.

If it returns:

DEVSDT mpeg2video

Then you're all set to go :)

Getting iPod's Working With Banshee In Ubuntu 8.04.3

2009-08-11T18:33:00.000+10:00

Recently, I had need to install a newer Banshee from the Ubuntu PPA on a friend's Ubuntu LTS machine (I only ever recommend LTSes to friends converting from Windows and thus not used to the 6-month upgrade dance).

He plugged in his iPod, after it had been restored with iTunes from his wife's PC and -- no, nothing happens.

Unplugged, Replugged -- yep, appears on the desktop as an iPod and Banshee starts correctly, the iPod just doesn't appear in it.

Checked dmesg, yes, the device is a 8GB iPod, no it does not need fsck'ing and yes, it uses the correct mountpoint.

Checked Synaptic, all the Banshee code is there.

Checked the Bugtracker, found bug #139226.

So, we ran:

sudo apt-get install podsleuth

Unplugged, Re-plugged the iPod. Success.

Question is, if podsleuth is that important to getting DAP's working in Banshee, shouldn't it be at a dependency of the install?

Fixing Ubuntu PPA (Personal Package Archives) Signing Key Issues

2009-07-09T16:17:00.001+10:00

My default /etc/apt/sources.list.d/ has around 10-12 PPA listings in it all the time and with the release of 9.04, there doesn't seem to be an automatic way of ignoring authentication for packages on upgrades, so a number of upgrades choke because of unsatisfied authentication issues.

This thread was a lifesaver, download the script, install the dependencies, run it, then re-run sudo apt-get -f update ... problem solved!

Why No Linux Installation has an 'I Only Speak One Language' Option?

2009-06-10T15:43:00.006+10:00

Now that Fedora 11 has come out, I decided to give it a spin in a VMWare box -- i'm not entirely satisfied with the drive read speed of Ubuntu Jaunty, nor the compiled in IPv6 option making Transmission crawl into the floor in terms of connection speed with no true way to disable it, I am unsure if users will the new 'update notification' (it's almost like Windows XP in that regard, just without the forced rebooting when it's finished) and I can certainly take or leave the 'new' notification system, which makes VMWare go nuts with screen flicker while the notification is on-screen.

Needless to say, that while I thought Ubuntu Hardy was brilliant for both technical professionals and new users alike on the Desktop -- having used Jaunty for a month, I almost feel like i've installed a totally different system that feels totally foreign -- and one that has been dumbed-down to the least-common-demoninator level, an unfortunate side-effect of 'visionary progression' within parts of the Linux on the Desktop supporters.

Coupled with the fact i'd like to play with RPM building again and thought PackageKit sounded remarkably cool -- I thought this would be a good time to try Fedora (my last experience with it on the desktop was 4, other than building libgpod for 9 and 10) -- I only upgraded from Ubuntu 8.04 to 9.04 to try GNOME 2.26, which Fedora 11 has included.

Now, i've never, ever, ever accepted the defaults for a Linux installation -- so I selected the 'Customise Now' button to remove the packages I didn't need at install-time (F-Spot, Java, etc).

Fedora, like RHEL still has a very nice package selector, with an 'Optional' button that allows you to select with tick-boxes which packages the user would prefer.

So, after a while -- I found the Base section.

The Package Selection Dialogue

28 of 80 Fonts?

So I click the "Optional Packages" button, to find:

The 'Optional Packages - Fonts' Dialogue

Now, in my personal case -- as far as foreign languages go, I speak semi-minimal German. As a native English speaker, everything I do with a computer is in English, aside from seeing what the other fonts look like, all they really do for me is take up room and slow OpenOffice down when it loads.

I de-selected all the optional fonts with the exception of the DejaVu fonts and continued on with the install as normal.

I personally think that the internationalisation of Linux is incredible and a wonderful thing to watch, but installing over 100 megabytes of fonts that I will never use, never see and couldn't really understand anyway -- seems like 100 megabytes of space that could be better used elsewhere.

To be fair, On the Red Hat / Fedora side, this problem bugged me more back in the pre-Red Hat 9 days, where ISOs were shipped on a 650M CDROM and every piece of space had to be meticulously accounted for -- they've worked around the issue by using multiple CDs to do the install, as well as supplying an 'Everything on DVD' release.

Ubuntu, on the other hand -- has always had this problem, Epiphany (the GNOME based Web Browser Front-end) needs to be installed post-install, but South African English, several Thai and Indian dialects as well as Japanese fonts are installed from the core 700 megabyte ISO.

It just seems as though the installer needs a box that asks 'Is that the only language you speak?' ... and only if you select 'No' does it select the internationalisation components of whatever distribution you're running.

The one question it raises is how much of all the languages you should include by default -- since the advent of network-based installs, it might work if the various installer code provided the strings for 'Do you Speak ...' 'Is your Network Set Up (followed closely by 'How would you like your network set up' if you have a Wireless configuration) and then sent off to mother-goose to install the relevant packages based on the locale configuration to that point.

Trials with RecordMyDesktop

2009-05-17T19:01:00.001+10:00

I've been playing with recordmydesktop lately in order to put some suitable screencasts together for demonstrating various Linux-related activities to university students next week.

Unfortunately, both Fedora 11 and Ubuntu Jaunty have issues with RecordMyDesktop -- the current version (0.3.8.1) is much better than the last version I tried (0.3.5) in terms of both video and audio quality, but the more-than-occassional

Broken pipe: Overrun occurred.

Cause audio to be clipped rather badly.

The default settings make the video speed up and therefore finish well before the audio too -- so I set about trying to find the optimium settings to make the best encode I could.

First, if you are using a 3D Window Manager (such as Compiz), please use the --on-the-fly-encoding and --full-shots options, or everything from opening new windows to redrawing your background will end up with corruption.

If you do not know which Window Manager you are running, you can use the Visual Effects tab of the Appearence Preferences window.

If the button is in the first box (None) then you are not running a 3D Window Manager, if it is in either the Normal or Extra boxes, you are.

Now we've figured out if we need the options for 3D Window Management, it's time to move on to the audio -- in it's default configuration, the program will clip and/or drop audio because of the overrun mentioned above, but you can reduce that by a fair proportion by either:

Using pasuspender to launch the program. This particular solution was suggested by an Ubuntu developer on a reported bug about RecordMyDesktop that looks very similar to the particular issue we're covering here.

Configuring an .asoundrc file to use PulseAudio for all ALSA based audio by default.

The latter is done by creating a new blank file in your home directory and adding the following code:

pcm.!default {
type pulse
}

ctl.!default {
type pulse
}

Save the file, exit, log out of your session and login again.

This alone reduced the dropped audio from 7 occassions in one minute's worth of recorded footage to 3.

The former is done by adding: pasuspender -- to the beginning of the command line.

On my laptops, using pasuspender made no real difference to the problem, which is why we're preferring the .asoundrc solution for the remainder of this discussion.

Another thing that helps lower the frequency of audio dropouts is to increase the buffer size used by RecordMyDesktop -- the defaults are 4k, raising that to 16k helps dramatically, but any higher doesn't seem to offer that much of an improvement (I tried 32k (32769) and 64k (65538) when testing -- i'm only recording two channels on an internal microphone though, your milage may vary)

You can do this by adding the --buffer-size 16384 option to the command line.

update (2009-05-17): At the highest quality audio and video settings, 16384 can still cause dropouts, so 65538 is now preferred.

Next, if you're using onboard audio (like Altec Lansing laptop audio) you may see:

Playback frequency 22050Hz is not available...
Using 44100Hz instead.
Recording on device hw:0,0 is set to:
2 channels at 44100Hz

The Altec Lansing Audio, as well as several other Intel based sound cards that use the hda-intel-audio driver, will resample 44100Hz input to 48000Hz, hence the error.

To fix that, you pass --freq 48000, which allows recording of audio without any extra resampling.

Now, if you've used the pasuspender solution above, or your audio seems to be correctly synced, you can move on to the next section.

If, on the other hand -- you still see Buffer Overrun errors, or your audio is out of sync, or you use my .asoundrc solution above, you'll need to pass an additional audio-related option on the command line:

-device plughw:0,0

Using plughw, allows ALSA and PulseAudio's internals to handle all of the resampling work -- or endian-conversion work automatically, which reduces a painfully large ALSA configuration that will differ on every soundcard on the market, to a single configuration change for our purposes.

The 0,0 (aka. use the first detected virtual sound device) should be fine for almost everyone working on a standard configuration -- if you've got:

More than one Soundcard.

Have used the Systemwide PulseAudio On Ubuntu HOWTO to add a software equalizer to your configuration.

You may have more than one plughw device to choose from -- choosing which one is beyond the scope of this article though.

Now we can move onto improving the video quality.

Next, it helps to set the framerate -- i've seen numerous articles on the web about this, everyone suggesting everything from 10fps to 90fps to get a clear picture.

I tried 10, 12, 20, 25, 29, 50 and 60 (these seem to be the ones that are mentioned the most, in a four page Google search on the subject) -- 10 works if you want to keep the filesize low, but the playback video seems too jerky -- 60 doesn't increase the filesize, but does have an unwanted side-effect of the video finishing well before the audio when you play it back.

The optimum setting I found was --fps 20.

Finally, because I wanted to re-encode these for different purposes, I like to do the initial encoding with the highest quality possible.

However, the difference between -v_quality 1 and -v_quality 63 was minimal at best on my 1280x800 resolution.

Somewhat obviously though, the -v_bitrate option makes A LOT of difference, so I bumped that to the highest number available (2000000).

The recommended command line(s)

I've provided both a high and low quality setting for those people who are interested

The final command line, that works for me and provides the highest quality, lowest distortion with no dropouts -- was:

recordmydesktop --on-the-fly-encoding -v_quality 63 -v_bitrate 2000000 -s_quality 10 --full-shots --fps 20 --freq 48000 --buffer-size 65538 -device plughw:0,0

For a smaller filesize or for encoding on older or busier machines, with quality suitable for downloading or streaming, you can use:

recordmydesktop --quick-subsampling --on-the-fly-encoding -v_quality 10 -v_bitrate 50000 -s_quality 1 --full-shots --fps 10 --freq 48000 --buffer-size 16384 -device plughw:0,0

This option does the encoding after the video capture has stopped, which results in a smaller filesize at the expense of taking longer to encode your original footage, you can offset this somewhat by adding the --quick-subsampling option to your command line, which saves CPU time by discarding extra pixels when calculating the chroma values during the encoding process.

note: If you are using a 2D Window Manager (such as Metacity), you may omit the --full-shots option, which halves the filesize on my Ubuntu Jaunty Jackalope install.

Why No Examples?

Google doesn't support Theora based videos to be uploaded to Blogger, I had two videos recorded of 10 seconds each, the low quality one (using the settings above) clocked in at 220 kilobytes, the high quality one was 1.3 megabytes.

When El Goog does decide to allow Theora based videos on Blogger, i'll post them -- but converting them to .mp4's just to show you all what they looked like, seemed rather purpose defeating :)

Converting Videos to be Nokia N9x Compatible using FFMPEG

2009-05-12T13:45:00.004+10:00

This is being posted here so I have something easy to look up when I need to do it over again, but using FFMPEG 0.5.x, you can easily convert any playable movie to a Nokia N9x (N95, N96) compatible MP4 format file with the line:

(ffmpeg | ffmpeg.exe) -y -i [input file].[extension] -f mp4 -vcodec libx264 -level 21 -s 320x240 -b 768k -bt 768k -bufsize 2M -maxrate 2M -g 250 -trellis 1 -refs 3 -keyint_min 25 -acodec libfaac -b 128k [output filename].mp4

Which converts any playable video to a compatible (-level 21), correctly sized (-s 320x240) MP4 based video file with AAC audio (-acodec libfaac -b 128k) that doesn't crash the RealPlayer version on phones in Australia/New Zealand because the buffers and bitrate are cool (-b 768k -bt 768k -bufsize 2M -maxrate 2M).

The other options are entirely optional, but the -g 250 & -keyint_min 25 are recommended if you have a PAL based input stream and would like to be able to fast forward and rewind your video using the funkey buttons on the N96.

Automatic Vista Speedup with a D-Link DIR-625

2009-05-07T01:00:00.001+10:00

note: This article is intended for a technical audience -- you should use caution when modifying a production system -- caveat emptor.

This probably works with the other DIR-6xx models that D-Link sells too, but recently oldfeeb had an issue where his girlfriend's computer (Vista Home Premium) stopped talking wirelessly to the new DIR-625 he'd bought.

He uses Ubuntu, so he saw no difference, but the symptoms on Vista were related to Windows Mail and Internet Explorer refusing to browse, or overly long timeouts.

First up, update Vista for all the recent fixes and reboot -- no luck.

Noticing the cordless phone handset on the wall, I thought: change the wifi channel on the router to something that doesn't conflict with things like cordless 2.4Ghz phones, move channel 5 to channel 11, check.

Reboot Vista, slightly better signal, but the network is still slow -- Google works now, but only after 30 odd seconds, but her mail times out and heavier webpages die a quiet death.

So, using a tip that I found courtesy of Australian Personal Computer magazine some years ago and a bit of Googling, we switched off Vista's 'Autotuning of TCP parameters'.

To do this, you'll first need to open a command prompt as the Administrative User -- which involves:

Go to Start / Run

Type "cmd.exe" into the box provided.

Hold down the Control and Shift keys together -- then -- press [ENTER].

Now, type the following at the prompt:

netsh interface tcp set global autotuning=disabled

Restarted IE7, fired up Google, works flawlessly.

"OK, I thought -- put the Wireless Channel on the router back to 5 (the default)."

We did, rebooted Vista -- Stone, Cold, Silence.

Switched it back to 11, rebooted Vista -- everything's peachy again.

So, sometimes it's not just Vista's fault -- I guess if you're going to buy a Wireless router, you should check what channel ranges are valid for the country you're trying to use the hardware in, before blaming the workstations directly.

Youtube Costs Google.

2009-05-01T12:38:00.002+10:00

Stumbled across this article by David Silversmith this morning, which makes interesting reading in the year of the economic downturn.

I looked at the list of leaks that David lists -- and wondered twofold:

How many of those 375 vistors actually look at the content? -- You see sites like Facebook and LiveJournal have the ability to embed Youtube content, so people find a video they like, embed it and every subsequent pageview sends a request to YT to begin streaming the content (in order to show the thumbnail image).

Now, Facebook has the News Feed, as well as the Highlights, as well as Sponsored Clips -- more than once a day, these contain (for me, anyway -- with 100 or so friends) at least one YT link, 99% of which, i'll never actually play, but every one of them actually generate an HTTP request anyway.

Is Youtube simply Google's Loss Leader? -- Google does many other things which are less bandwidth and resource intensive than Youtube, such as Blogger, GMail, Google Scholar, Google Checkout and to a lesser extent, Google Earth to name a few, all of which have the same advertising strategy (Adwords) that Youtube has, at similar pricing scales, it strikes me that Google may take a bath on Youtube in order to draw people in to making a Google Account, thereby gaining access to their other resources and presenting Google with many, many more advertising opportunities they would not otherwise have had.

A thought along similar lines, was "Would Youtube have been this popular if Google hadn't acquired it? -- Reading David's piece further, he talks about Content Acquisition costing money, which is true -- but i'd suggest an ever growing amount of content that Google sends to viewers will be commercial over the course of the next 12-24 months, as the economic downturn bites over and over and causes people to cancel luxury items like subscription television, possibly turning to Youtube to pick up highlights of events as they happen instead.

Google has also done a lot for Youtube that i doubt they could have done on their own -- everything from striking deals for the content, to providing the bandwidth and storage space to sustain the "1 second attention span" internet generation, to adding features like the "High Definition" button to the site itself.

"With Popularity Comes Expense" is a phrase I came up with nearly 15 years ago to define the ability of the "modern" internet to actually exist -- at the time I was working in the pioneering phase of Internet Advertising, back before web based video and when only the technically elite and businesses with vast IT budgets had pages at all.

Nice to see it still rings true today.

How does Google stop the bleeding?

My guess is, it doesn't have to.

Installing VMWare Tools on Ubuntu Guest Servers

2009-04-14T17:24:00.004+10:00

After hunting around on the web for a while, I couldn't find anyone who answered this in a way that people could run step-by-step, so I hope this helps other people in the future.

I used Ubuntu 8.04.2 LTS for the purposes of this example, but a similar, if not identical set of commands should work for any version of Ubuntu Server.

First, you'll need to have installed VMWare on Windows or a UNIX server and have your Ubuntu Guest running, then you can go to "VM / Install VMWare Tools".

Your first problem, because you're not running a desktop system in your guest -- the CDROM is not automounted for you, so you'll need to do:

mount /media/cdrom

Next, you'll need to copy the .tar.gz file to a place on the installation that has write access, like /tmp:

cp /media/cdrom/VMwareTools-*.tar.gz /tmp/

Next, extract the file:

tar zvxf VMwareTools-*.tar.gz

Change to the directory:

cd /tmp/vmware-tools-distrib/

Now, before we actually run the installer, Ubuntu Server needs some packages installed so the new kernel modules can be built successfully.

apt-get -f install build-essential linux-headers-server linux-server

Once they've been successfully installed, you can run:

sudo perl vmware-install.pl

Most of the defaults are fine, when you are asked for "the location of the C header files for your running kernel" you'll need to answer with the include directory from the kernel you are currently running -- from the looks of things, this trips people up sometimes.

(Ubuntu doesn't ship with a /usr/src/linux directory, so if you press [ENTER] here, you'll get a "Directory not found" message and asked to re-enter the location)

As of the time of writing, the kernel is 2.6.24-23-server, so your location would be:

/usr/src/linux-headers-2.6.24-23-server/include

Once you've done that, the configuration routine will continue through to the end normally and you can reboot your guest OS in order for the changes to take effect.

"all zones must be in views", BIND 9 and You.

2009-04-11T16:43:00.000+10:00

note: This article is intended for a technical audience -- you should use caution when modifying a production system -- caveat emptor.

An interesting question raised by a client, who was experiencing this error in their daemon.log file and asked me to track it down.

A bit of background:

The client in question runs a "Split Zone" configuration, where a seperate set of DNS entries are published to public users (that are only queried by and cannot be altered in any way by external entities) than are published to their other master or slave servers (which are secured by DNSSEC and various other access controls and can be queried by internal LAN users.)

So:

Apr 10 05:41:17 vampire named[8933]: loading configuration from '/etc/bind/named.conf'
Apr 10 05:41:18 vampire named[8933]: /etc/bind/named.conf:12: when using 'view' statements, all zones must be in views

has been in their logs for about six weeks since the server was commissioned and the new administrator wonders if that's the reason their zone information appears out of date?

Well, no.

First off, that message is not actually an error, it's a warning -- but it's still unsightly.

Second, the file it is complaining about is not /etc/bind/zonefiles/m/[name_of_zone].zone but /etc/bind/named.conf which happens to be the default configuration file for most common Linux distributions.

In fact, if you look at the Debian / Ubuntu configuration file you'll see:

"If you are just adding zones, please do that in /etc/bind/named.conf.local"

Which, in this case -- we did, using internal and external views that look a bit like:

view "internal-in" in {
// This view includes our trusted machines, we perform
// recursion in this view and can cache requests for
// our client machines
match-clients { trusted; xfer; };
notify yes;
recursion yes;
additional-from-auth yes;
additional-from-cache yes;

zone "." in {
// Link in the root server hint file.
type hint;
file "/etc/bind/db.root";
};

zone "ministry-of-the-interior.local" in {
type master;
file "/etc/bind/zonefiles/m/forward.zone";
allow-query { any; };
allow-transfer { any; };
};

zone "33.168.192.in-addr.arpa" in {
type master;
file "/etc/bind/zonefiles/m/reverse.zone";
allow-query { any; };
allow-transfer { any; };
};
};

and:

view "external-in" in {
// This view includes our trusted machines, we perform
// recursion in this view and can cache requests for
// our client machines
match-clients { any; };
notify yes;
recursion no;
additional-from-auth no;
additional-from-cache no;

zone "." in {
// Link in the root server hint file.
type hint;
file "/etc/bind/db.root";
};

zone "ministry-of-the-interior.local" in {
type master;
file "/etc/bind/zonefiles/m/forward.zone";
allow-query { any; };
allow-transfer { none; };
};

zone "33.168.192.in-addr.arpa" in {
type master;
file "/etc/bind/zonefiles/m/reverse.zone";
allow-query { any; };
allow-transfer { none; };
};
};

As you can see, we defined the root server hint file for both views, BIND 9.x gets rather tetchy if you don't do that, but will happily continue as long as you do (and, of course -- all your zones verify correctly with named-checkzone)

What, of course -- we failed to do was to define localhost, 127.in-addr.arpa, 0.in-addr.arpa and 255.in-addr.arpa for the internal and external view too.

The error goes away, as expected if you:

Firstly, Comment out the lines for the required zones in the /etc/bind/named.conf file. (Use '//' for commenting in this file)

Then, ammend your zone configuration to look similar to:

view "internal-in" in {
// This view includes our trusted machines, we perform
// recursion in this view and can cache requests for
// our client machines
match-clients { trusted; xfer; };
notify yes;
recursion yes;
additional-from-auth yes;
additional-from-cache yes;

zone "." in {
// Link in the root server hint file.
type hint;
file "/etc/bind/db.root";
};

zone "localhost" {
type master;
file "/etc/bind/db.local";
};

zone "127.in-addr.arpa" {
type master;
file "/etc/bind/db.127";
};

zone "0.in-addr.arpa" {
type master;
file "/etc/bind/db.0";
};

zone "255.in-addr.arpa" {
type master;
file "/etc/bind/db.255";
};

zone "ministry-of-the-interior.local" in {
type master;
file "/etc/bind/zonefiles/m/forward.zone";
allow-query { any; };
allow-transfer { any; };
};

zone "33.168.192.in-addr.arpa" in {
type master;
file "/etc/bind/zonefiles/m/reverse.zone";
allow-query { any; };
allow-transfer { any; };
};
};

and:

view "external-in" in {
// This view includes our trusted machines, we perform
// recursion in this view and can cache requests for
// our client machines
match-clients { any; };
notify yes;
recursion no;
additional-from-auth no;
additional-from-cache no;

zone "." in {
// Link in the root server hint file.
type hint;
file "/etc/bind/db.root";
};

zone "localhost" {
type master;
file "/etc/bind/db.local";
};

zone "127.in-addr.arpa" {
type master;
file "/etc/bind/db.127";
};

zone "0.in-addr.arpa" {
type master;
file "/etc/bind/db.0";
};

zone "255.in-addr.arpa" {
type master;
file "/etc/bind/db.255";
};

zone "ministry-of-the-interior.local" in {
type master;
file "/etc/bind/zonefiles/m/forward.zone";
allow-query { any; };
allow-transfer { none; };
};

zone "33.168.192.in-addr.arpa" in {
type master;
file "/etc/bind/zonefiles/m/reverse.zone";
allow-query { any; };
allow-transfer { none; };
};
};

Check your new configuration validates by running named-checkconf, then restart BIND the warning should vanish and your administrators should all go away satisfied.

Web Design Meets Marketing = Fatality

2009-04-01T13:09:00.000+11:00

Viral Marketing at it's best.

I laughed ... and I don't think it was even meant as an April Fools Joke.

Could Not Modify Header Information - A Wordpress Experience.

2009-03-30T14:40:00.007+11:00

I've been playing with Wordpress as a CMS project recently and came across an interesting "semi-bug(let)" that took ages to resolve, but was rather simple in the end.

A white screen.

Attempting to login to my admin screen after being logged out after a connection dropout gave me a blank screen, nothing on view-source: and no errors in the Apache logfile.

Googling for the issue, I found several posts advising me to clean out my browser cache, clear cookies and check to see if my wp-config.php file ends with a blank line.

None of those fixed it, so I went hunting further and saw people suggesting that one should remove any plugin code -- having a Wordpress installation with heaps of plugins, removing them one-by-one was annoying at best, but some more searching found this neat plugin to resets Wordpress back to it's installation defaults with one click.

(Obviously, if you care about the contents of your blog, or the settings you have crafted for the individual plugins, this isn't for you -- but it helped me out immensely. Kudos to Matt for inventing it.)

Nope -- didn't fix it either.

Removed my theme, restored the default Kubrick theme ...

*shazam*

Fixed.

Put my theme back, logged out, logged back in:

A White Page.

OK, at least I have something to go on, it's a problem with the theme. So I started to poke around further.

It turns out, the functions.php file cannot have spaces in between the functions you define, compare the following code:

<?php
if ( function_exists('register_sidebars') )
register_sidebars(2, array(
'before_widget' => '<li class="widget %2$s">',
'after_widget' => '</li>',
'before_title' => '<h2 class="widgettitle">',
'after_title' => '</h2>',
));

function widget_search() { ?>
<li class="widget widget_search">
<h2 class="widgettitle">Search</h2>
<input type="text" id="searchfield" />
<img id="searchspinner" src="<?php bloginfo('template_url'); ?>/images/ajax-loader.gif" alt="∗" />
<script type="text/javascript">
var search = new Search('searchfield', 'searchspinner');
</script>
</li>
<?php }
if ( function_exists('register_sidebar_widget') )
register_sidebar_widget(__('Search'), 'widget_search');

?>

<?php

function theme_comments($comment, $args, $depth) {
$GLOBALS['comment'] = $comment;

(code edited)

Fails, but:

<?php
if ( function_exists('register_sidebars') )
register_sidebars(2, array(
'before_widget' => '<li class="widget %2$s">',
'after_widget' => '</li>',
'before_title' => '<h2 class="widgettitle">',
'after_title' => '</h2>',
));

function widget_search() { ?>
<li class="widget widget_search">
<h2 class="widgettitle">Search</h2>
<input type="text" id="searchfield" />
<img id="searchspinner" src="<?php bloginfo('template_url'); ?>/images/ajax-loader.gif" alt="∗" />
<script type="text/javascript">
var search = new Search('searchfield', 'searchspinner');
</script>
</li>
<?php }
if ( function_exists('register_sidebar_widget') )
register_sidebar_widget(__('Search'), 'widget_search');

?>
<?php

function theme_comments($comment, $args, $depth) {
$GLOBALS['comment'] = $comment;

(code edited)

Works perfectly.

So where was the problem?

the blank line between the closing ?> in the register_sidebar_widget call and the opening <?php in the theme_comments function.

Undoubtably, your error will occur between a different set of functions, but if you are using a functions.php file in your theme and it begins to fail unexplainedly on you, check the file for blank lines between functions as part of your diagnostics plan.

Microsoft's Support for XP Ending?

2009-03-28T02:12:00.002+11:00

Interesting read via ZDNet this morning regarding Microsoft beginning the end-of-life cycle for Windows XP (and, to a similar extent Office 2003).

I find it interesting -- still having people to this day asking if I can kindly remove Vista from their new machines and replace it with XP (and to a similar extent, Office 2007 for either 2003 or OpenOffice) that MS appears to have a longer support deadline for XP than either 2000 or Vista, which have a 5 year lifecycle compared with a 6.5 year one for XP.

I wonder if it'll just cause people to a) switch to Linux or b) use a completely unsupported operating system because Windows 7 is too new and 'innovative' for them and Vista is ... well, Vista.

One thing's for sure, it will have a ripple-down effect into finally removing Internet Explorer 6 woes from web developers minds.

Removing Dreamscape's SEO 3.0 Beta from Zen-Cart

2009-03-06T16:40:00.003+11:00

note: This article is intended for a technical audience -- you should use extreme caution when modifying a production system, as your data will be nearly impossible to recover if you use this command incorrectly -- caveat emptor.

One of the more painful things to do in Zen-Cart is removing references to broken module code from your database. You can remove all the PHP files you like, but you'll still see references in your Administration pages for the modules.

In a recent request, I had to switch multiple shops from Dreamscape's SEO 3.0.0beta1 to "something better" due to a change by several shared hosting providers to use SUPHP to protect their client accounts.

After choosing Simple SSU (which is, rather a well written module and is well supported by it's author) -- we were left with the problem of the Configuration Menu in Zen-Cart's admin having two SEO references, one to SSU, one to the older code.

For anyone in the same boat, you can use the following SQL to remove the older module's SQL code from your database:

From the Administration Panel, Go to "Tools -> Install SQL Patches" and enter the following code:

SET @configuration_group_id=0;
SELECT (@configuration_group_id:=configuration_group_id) FROM configuration_group WHERE configuration_group_title = 'Seo Urls 3.0.0-beta1' LIMIT 1;
DELETE FROM configuration WHERE configuration_group_id = @configuration_group_id AND configuration_group_id != 0;
DELETE FROM configuration_group WHERE configuration_group_id = @configuration_group_id AND configuration_group_id != 0;

note: That's four seperate lines with the ;'s ending each line.

I've also posted this as a solution to the thread on the Zen-Cart forums, you can follow the further discussion there -- if you'd like.

Useful 'mysqldump' one-liners.

2009-03-03T16:27:00.004+11:00

note: This article is intended for a technical audience -- you should use extreme caution when modifying a production system, as your data will be nearly impossible to recover if you use this command incorrectly -- caveat emptor.

I always seem to use these when doing remote administration from the command line -- but when I forget if I should be using -add-drop-table or -add-drop-databases, the usual suspects from Google render terribly on my mobile phone -- so I thought i'd put them here, for later.

To backup a single database:

mysqldump -u [username] -p [database_name] --add-drop-database --lock-all-tables > backupfile.sql

To backup multiple databases:

mysqldump -u [username] -p --all-databases --add-drop-database --lock-all-tables > backupfile.sql

To restore your backup:

mysql -u [username] -p < backupfile.sql

Lightboxes, Zen-Cart, Oh My!

2009-02-21T12:56:00.002+11:00

Recently, a customer asked if they could have image effects on their e-commerce store graphics -- having played around with Wordpress for a while, I knew about Lightbox, which uses either jQuery, Prototype or MooTools in order to show images in a seperate raised window with fancy effects (watermarking, shadowing, etc) while transparently showing the original page in the background.

What I didn't know -- was exactly how many variations of the Lightbox there are.

This site has to be one of the best for deciding which one suits your needs and application best, a must bookmark for anyone interested in knowing the compatibility and abilities of the interschnitzels new 'buzz app'

Creating Transparent .GIFs from .PNG's

2009-02-08T17:14:00.002+11:00

Easy one-liner using "convert" from the Imagemagick graphics suite to get decent .gif conversions of most^* transparent .png's.

convert [imagename].png -channel Alpha -threshold 80% [imagename].gif

This converts the Alpha channel (the one IE6 has the most problem with) to either completely transparent or opaque based on the 80% threshold.

I say most, because it doesn't work for things like backgrounds or overlays (which are often too dark -- greys are often black, for example.) -- you can change 80% to 60% if you have backgrounds of mostly one colour, but it's not foolproof and often leaves darker marks around the exterior of the image.

Ubuntu, Pulseaudio, Small Addendum.

2009-01-04T00:36:00.002+11:00

I've tended to use this howto to tweak audio configurations whenever I install a new Ubuntu box for someone -- it makes everything from laptop speakers to Skype work much better, shockwave flash videos work (Youtube, etc) and it makes audio a much better experience all-round.

Today, a user who was having audio stuttering issues in Skype mentioned that when running it in verbose mode (with -vv), the following error occurred:

I: main.c: Called SUID root and real-time/high-priority scheduling was requested in the configuration. However, we lack the necessary priviliges:
I: main.c: We are not in group 'pulse-rt' and PolicyKit refuse to grant us priviliges. Dropping SUID again.
I: main.c: For enabling real-time scheduling please acquire the appropriate PolicyKit priviliges, or become a member of 'pulse-rt', or increase the RLIMIT_NICE/RLIMIT_RTPRIO resource limits for this user.
I: main.c: Note that real-time/high-priority scheduling is NOT normally required. If you experience crackling or other sound anomalies, consider one or more of the above solutions.

The easiest way to fix that, is to open a terminal window and run:

sudo usermod -G pulse-rt -a [username] (where of course, [username] is your desktop username.

(note: it needs to be a capital -G and a little -a to add your user to the pulse-rt group and leave your existing groups untouched, one of those check twice, then check again, then press enter moments)

If you re-run Pulseaudio now, you'll get:

I: main.c: PolicyKit refuses acquire-high-priority privilige.
I: main.c: We're in the group 'pulse-rt', allowing real-time and high-priority scheduling.
I: core-util.c: Successfully gained nice level -11.

Which tends to make Skype and WINE (the two applications the user in question was having stuttering audio issues with) behave more normally.

Something Completely Different

2008-12-31T13:02:00.003+11:00

A completely non-technical post to end the year on -- highlighting two albums i'm almost sure nobody reading this has ever heard, but are definately worth the listen.

Lazyboy TV - Lazyboy

Personally, I wouldn't have thought i'd like an album that is more "spoken word" than Pop -- but this is a great album that deals with everything from homelessness to drug use to oddities you'll find in the news combined with an infectious beat that gets this played over and over again.

Buy This Album from Amazon

Angles - Dan le Sac versus Scroobius Pip

This one was completely out of left field, I overheard the track Tommy C in a bar and then hung around to listen to the rest of the album -- it's kind of like the Lazyboy album, but with an sound that makes me think of what would happen if you threw electro and dubstep in a blender and put themes like youth suicide, fads and the evangelism of pop music over the top.

Buy This Album from Amazon

Automounting SAMBA Shares

2008-12-26T22:01:00.005+11:00

note: This article is intended for a technical audience -- you should use caution when modifying a production system -- caveat emptor.

This is more of a collaboration of other people's posts, with some additions of my own for performance-related issues.

Basically, I manage a bunch of WD MyBook Network Drive (World Edition) boxes for various people -- typically, these are hooked up via SMB shares to various types of Linux install for redundant network backups over the LAN.

After various hacks, from mounting as part of a cronjob to modifying /etc/rc.local -- I decided to attempt automatic fstab mounting under the christmas break and figured i'd document my findings here.

First off, to automatically mount the filesystems on the MyBook -- you need to add lines similar to the following to your /etc/fstab file.

//[SMB SHARE]/[sharename] /media/[mountpoint] cifs credentials=/root/.smbcredentials,rw,iocharset=utf8,uid=[username],gid=groupname,file_mode=0664,dir_mode=0775 0 0

(note: Blogger has wordwrapped this post, but this should be one line when copied to your /etc/fstab file.)

Where:

SMB SHARE -- Is the NETBIOS name or IP address of the MyBook.

sharename -- Is the name of the share you need to mount (personally, I like to make at least shares based on the usernames using the box).

credentials=/root/.smbcredentials -- Is a plaintext file containing the username and password of the user you have created on the MyBook.

iocharset=utf8 -- Specifies that all files written or read from the device should be in the UTF-8 character set.

rw -- Specifies that access to the share should be read-write.

uid=username,gid=groupname -- Specifies the username and groupname on the local Linux machine.

file_mode=0664,dir_mode=0775 -- Specifies the octal permissions of the files written on the MyBook.

After you have edited your /etc/fstab file, you need to make your credentials file -- this file specifies the name and password of the user on the MyBook.

This file needs two lines, with a trailing blank line -- and should usually be placed in the /root or /etc/samba directory and have 0600 permissions.

An example of this file is:

username=winuser
password=winpassword

Save this file and change it's permissions, then alter your /etc/fstab file to point to it's location.

Once you've done this, you should be able to have your shares automatically mounted by your Linux box (after either rebooting or running mount -a as root).

One particular quirk you might find using this method -- is unmounting errors -- these occur because the shutdown routine (by default) shuts down the network devices (specifically, those machines running NetworkManager to control network resources) before unmounting any mounted network shares (ie. what we're trying to achieve here).

These errors usually halt the shutdown of your machine (usually meaning you have to power off using the power button, which can damage your filesystem).

To fix this, you can run the following as root:

ln -s /etc/init.d/umountnfs.sh /etc/rc0.d/K15umountnfs.sh
ln -s /etc/init.d/umountnfs.sh /etc/rc6.d/K15umountnfs.sh

Which will alter your system to unmount the network-attached shares before NetworkManager has a chance to shutdown the network devices.

Unattended Password Creation Failing?

2008-12-19T02:26:00.002+11:00

Earlier, I was asked if I had any good solutions for scripting a user account generator in bash -- asking the user what they had already, I received:

useradd -n -g users -p [password] -s /bin/false [username]

I asked what the problem seemed to be, and the response was the password didn't seem to work -- if they used 'passwd' interactively, it'd work -- but unattended, it failed.

Having a look at the useradd manual, we see:

-p -- The encrypted password, as returned by crypt(3).

After trying various combinations - I thought about OpenSSL. What if I gave it the password and got it to do the crypt work first, then fed the encrypted string to useradd?

Something like:

#!/bin/bash
clear="[password]"
crypt="openssl passwd -crypt $clear"

success="0"
failure="1"

useradd -n -g users -p $crypt -s /bin/false [username]

exit $?

Cut and Paste the above into a script and instantly you have a semi-autonomous way of adding generated (or defined) passwords to your machine, all with the help of OpenSSL.

Interesting Spin on Proposed Internet Filtering

2008-12-12T01:37:00.000+11:00

Oh, December seems to be rant month.

Some Swinburne University students asked me for a little more clarification on why the proposed Australian Labour Government's filtering idea is a bad one.

I'll be writing my concerns up at length and back-posting them here as they are finished. However, because tomorrow is protest day, i'll post an couple of interesting links you may wish to read:

The ACMA's Report on Closed-Environment Filtering for 2008 : Basically suggests 'The Filters were better than the last time we tested them in 2005, because they filter SSL based traffic now, but they could still degrade network performance between 2 and 87 percent and still have a 23 to 40 percent chance of false-positive filtering.

Telstra Says No To Filtering - The Australian : When Australia's Largest Carrier decides they can't participate because of 'customer management issues' (possibly due to them moving a large chunk of their support staff, who'd receive the brunt of the complaints offshore earlier in the week) -- it says something about the ways this particular idea will effect everyone in the country, in one way or another.

How To Easily Bypass Australia's Internet Filters (for free) - Sydney Morning Herald : Explains to the technical neophyte how to use VPN software and other proxy methods to bypass the filter, in a worst case scenerio.

Labor’s Mandatory ISP Internet Blocking Plan - Electronic Frontiers Australia : Analysis of the ACMA proposal, discussions on why this proposal effects everything from online commerce to the civil liberties of Australian citizens and a well reasoned argument on why parents should filter their children's use, followed by a locally installed filter on the computers in the home -- If you haven't read this, you certainly should.

Apple Sued Over iPhone Performance Issues

2008-12-10T23:54:00.004+11:00

I stumbled across this article on Wired a few days ago. US-centric as it is -- but the same drop-outs seem to happen here in Australia, whenever the phone switches from 3G to the older GSM network.

Two bars on 3G on my Nokia E66 or N73 versus 4 bars on the iPhone, less than 3 inches away from each from each other when the phones are idle, yet the Nokia's complete the calls and the iPhones drop out.

Back in August, Optus (SingTel) offered 'Goodwill Credits' to users who suffered woeful network performance following the launch of the iPhone, which they're not doing now -- yet the latest firmware update doesn't seem to make a scrap of difference.

Perhaps that's why I saw a paper-printed advert in my local Optus dealership that said 'iPhones available for pre-paid plans, $799 AUD for 8GB, $899 for 16GB - while stocks last.'

Does anyone else have similar issues with providers here, or is this just another case of 'never buy G1 hardware' coupled with 'if it isn't broken, don't replace it?'

Regarding 'The Free World', 'The Internet' and You.

2008-12-04T15:15:00.002+11:00

It's not often that politics gets my back up about something that I feel the need to post it here, but while watching Question Time in the Senate last night, the topic of the great firewall of Australia came up -- again.

For International Readers: This isn't exactly new, compulsory ISP-level filtering was tried in closed-quarters in 1999 and 2001 -- however, these were opt-out and focused on guarding against underaged illicit content.

For Local Readers: There's a protest going on in the state capitals for the weekend of the 13th and 14th of December, if you care -- you should be there.

For everyone: The EFA has a very well researched document into why said filtering of this type is a flawed exercise, which you should read.

This time around, there's at least two lists -- one banning underaged illicit content and the other banning 'undesirable content'.

Couple this with the fact both of these lists are privately built, without public consultation -- and the fact -- there's no ability to opt-out.

"Canberra, We may have a problem."

(oh, and the rest of the world is laughing at you, just by the by.)

Anyway...

While Conroy was discussing porn and blocked keyword sites that the ACMA list comprises of during the trial run (beginning on December 24th).

Senator Bernardi said:

I note that the minister failed miserably to answer that question, which was specifically about the number of people needed for a trial to be credible. I also note that in the expression of interest documents the second stream of the trial includes a filtering of other unwanted content. I ask the minister:
Has this unwanted content been identified, and by whom?

Senator Conroy came up with an interesting number in regards to the number of blocked sites, that i'd not heard before:

The list could contain 10,000 [potential sites].
When you look around the world at Interpol, the FBI, Europol and other law enforcement agencies and you look at the size of the lists that they are actually using at the moment, 1,300 would not be sufficient to cover the URLs that we would have supplied to us with the purpose of blocking.

(Quotes from Australian Government Senate Hansard - 03/12/2008) (any emphasis mine)

I wonder, Supplied by whom?

Given the ACMA already have a link for people to report prohibited content, one wonders if the government plans on listing these sites as well as the ones supplied by law enforcement verbatim, or at least vetting them to ensure rogue parties aren't submitting them for their own ends.

Repairing an Apache 2.2 Modules Installation.

2008-11-20T16:08:00.001+11:00

note: This article is intended for a technical audience -- you should use caution when modifying a production system -- caveat emptor.

Sometimes, Ubuntu (and before it, Debian) drives me up the wall.

Recently, an apache2.2-common upgrade saw fit to blow away my /etc/apache2/mods-enabled directory, but not recreate the defaults, so I was left with an empty directory and a server that wouldn't restart due to various errors that looked similar to:

"Invalid command 'Order', perhaps misspelled or defined by a module not included in the server configuration"?"

Looking around on the web, this doesn't appear to be a new issue, but it doesn't appear to be terribly well answered either -- all I knew, was that I had a box that needed SSL, PHP, Expiry Headers and well, that's it -- aside from the basic functionality.

My first test was to purge the package and re-install it, which did give me back the functionality I wanted, together with a bunch of other modules I didn't need -- I proceeded to leave it and go to bed, only to be greeted with a message from the hosting provider telling me i'd overblown my shared-hosting's RAM quota for the day and that my account was temporarily suspended.

So, I removed the directory and started working piece by piece to put things back together until both Wordpress and OSCommerce booted up and ran correctly.

note: You can also do this with Apache's handy a2enmod program, but i'm a purist -- so i'm going to do it here via 'ln -s'.

First, to obtain basic functionality, you'll need to log in as root (or sudo to root) and issue the following commands, depending on which error your Apache 2.2 installation gives you.

If you see: "Invalid command 'Order', perhaps misspelled or defined by a module not included in the server configuration?", run:

ln -s /etc/apache2/mods-available/authz_default.load /etc/apache2/mods-enabled/authz_default.load
ln -s /etc/apache2/mods-available/authz_host.load /etc/apache2/mods-enabled/authz_host.load

If you see: "Invalid command 'DirectoryIndex', perhaps misspelled or defined by a module not included in the server configuration?", run:

ln -s /etc/apache2/mods-available/dir.load /etc/apache2/mods-enabled/dir.load

If you see: "Invalid command 'Alias', perhaps misspelled or defined by a module not included in the server configuration?", run:

ln -s /etc/apache2/mods-available/alias.conf /etc/apache2/mods-enabled/alias.conf
ln -s /etc/apache2/mods-available/alias.load /etc/apache2/mods-enabled/alias.load

If you see: "Invalid command 'AddType', perhaps misspelled or defined by a module not included in the server configuration?", run:

ln -s /etc/apache2/mods-available/mime.conf /etc/apache2/mods-enabled/mime.conf
ln -s /etc/apache2/mods-available/mime.load /etc/apache2/mods-enabled/mime.load

If you don't see any of those, but Wordpress 2.6.x (or 2.7.x) will not let you login (ie. you can install it, and you see the admin screen, but you get 'permission denied', 'forbidden' or a directory index -- rather than your admin dashboard) then try running:

ln -s /etc/apache2/mods-available/asis.load /etc/apache2/mods-enabled/asis.load

and restart Apache.

update [12-12-2008]: Wordpress 2.7 will fail to install unless you have the 'env' module installed, so you may also need to run:

ln -s /etc/apache2/mods-available/env.load /etc/apache2/mods-enabled/env.load

Once you've restarted, your Apache installation should basically work -- to add advanced functionality, you should visit the Apache 2.2 Modules Documentation pages and adapt the lines above to suit the function you need.

Signs Of The Future.

2008-11-04T22:29:00.000+11:00

Since Luis Villa first mentioned it, i've been following the election blog at Princeton -- I don't read many political blogs anymore, but it does have to be one of the better ones I keep on my list, the posts are impartial and the graphs and charts are clear and concise.

I'm not an American, so I can't vote -- but having dealt with Americans on a near daily basis for the last eight years, i'd like to think there'll be enough people with enough power to enact some serious change in the political landscape, for the people, for the country and for the world.

It'll be interesting to watch the live graphs on the Princeton site over the next few hours.

update [05/11/2008 16:15 GMT+11]: Looks like Obama Wins -- the people have made their choice.

Smart-ISP Configuration for Postfix.

2008-10-13T00:17:00.002+11:00

note: This is another 'no-brainer, don't forget this again' type post, but I thought i'd put it here incase anyone else can make use of it.

Recently, I was told that a client needed to forward all their corporate mail via their local ISP and that I should set that up for them using their existing, internal-only mail handling Postfix server.

The key additions to their /etc/postfix/main.cf file was (aside from configuring SASL, which you should do anyway and setting the relayhost parameter correctly) was:

+++ mydestination =
+++ local_recipient_maps =
+++ local_transport = error: no local delivery service

and:

+++ myorigin = the outbound domain name you need.

Then, you need to comment out (with a #) the local transport method in your /etc/postfix/master.cf file.

Remember that, brain!

Clickjacking, 1999 Called...

2008-10-10T21:59:00.000+11:00

Betanews isn't something I read often anymore, but this article intrigued me.

It's amazing that after nearly 10 years of active development on the web, standards and the rest -- and the best idea people can come up with for preventing clickjacking is using security=restricted to break frames (aka. frame busting code).

Using mod_security, you can at least write filtering rules that eliminate iframes and other annoying content at the server level and for Firefox users, NoScript does an excellent job of handling it at the desktop level (after, of course enabling the "Forbid IFRAME" option.)

For Internet Explorer users, I don't know what to tell you -- something tells me that's why the Internet remains as it is.

I often wonder if the whole IFRAME tag will be removed from HTML 5.0 -- with all the forward-thinking ideas that CSS 2.1 and 3.0 bring to the table, the only people seeming to use frames on new pages now, seem to be those wishing to exploit it.

VSFTPd Configuration for Non-Interactive Users

2008-10-04T11:21:00.003+10:00

note: This article is intended for a technical audience -- you should use extreme caution when modifying a production system -- caveat emptor.

This is one of those 'it's not broken until you trip over it' things that I wish i'd known about earlier, in posting it here -- i'm hoping other people can find it without wasting the amount of time I did.

So, I had reason to set up a new VPS box with the LAMP stack, SSH/SCP and VSFTPd for FTP use -- my usual command says something like:

useradd -n --gid www-data -s /bin/false [username]

In a nutshell, this:

Does not create a new group specific to the user.

Makes the user's primary group www-data (useful for users being able to access / upload / modify their own web code).

Sets the default shell to /bin/false, which does not allow the account to login interactively.

However, when you use an FTP client, it bails out with a 530 error, telling the user either their username or password are wrong.

So, the next step is to reset their password and try again -- nope, no difference.

Try changing the shell from /bin/false to /bin/bash with chsh though:

chsh -s /bin/bash [username]

... and everything works correctly.

We don't want to use /bin/bash though, because (amongst other things) it allows interactive logins via SSH or the console, which poses a security risk for the other users of the box.

Ubuntu & Red Hat (and probably a whole bunch of others) include the nologin command, which does exactly what we want (provides a message that the user cannot login, and exits).

OK:

chsh -s /sbin/nologin [username]

Try FTP.

It still errors.

As it turns out, the reason it errors is due to it not being included in the /etc/shells file that VSFTPd and other system daemons use to determine if your shell is valid.

Turns out, it's a very easy thing to fix -- simply:

echo /sbin/nologin >> /etc/shells

note: Some distributions ship this as /usr/sbin/nologin -- you may wish to run whereis nologin first to determine where your copy is.

Try FTP again.

:)

Disabling Control-Alt-Delete on Linux Servers

2008-10-02T17:23:00.002+10:00

A colleague rebooted a server unintentionally by having the keyboard plugged into the wrong machine (or, more correctly, typing on a keyboard not connected to the box he was looking at)

After the users swore at him a little, he rang for advice -- luckily, in these days of event driven machines, on a recent Linux distribution (RHEL 5 / Ubuntu 8.04) disabling the C-A-D on critical machines is easy.

First, open /etc/event.d/control-alt-delete.

Now, change the line:

--- exec /sbin/shutdown -r now “Control-Alt-Delete pressed”

To:

+++ exec /bin/echo "Control-Alt-Disable has been disabled"

Save the file and restart the machine, the next time you press Control-Alt-Delete, you'll get a nice message saying things were disabled, rather than a bunch of angry users.

Bandwidth: Limits, Speeds, Standards?

2008-08-30T22:27:00.002+10:00

Christopher Blizzard recently wrote about Comcast (a major US ISP) imposing bandwidth limits on their customers -- unfortunately, something those of us in Australia know all too well.

My initial thoughts, included a comment:

I don’t think it’s ‘that’s a huge pile of angry non-americans’ any more that it is ‘the rest of us, would like to welcome you, the americans — to needing to put up with what the rest of the world deals with on a daily basis’ or, consumerism.

After pondering on it for a while longer, especially in relation to local users -- I have to wonder:

If the average cost of bandwidth in Australia is $50 AUD for 5/GB of bandwidth, which at 1.5M p/sec equates to 3.75hrs of sustained download time -- what does an average household actually do with their internet the rest of the time?

Telstra, ran a TV commercial recently where they demonstrated a 'BigPond Connected Home' with 2 Adults, 1 Adult Child, 2 Children and a Dog using the internet at the same time, one for streaming TV, one for viewing Facebook, one for browsing eBay and one for booking travel.

Presuming the family only paid the 'average' fee, $59.95 at 1.5M gives you 600M -- yes, that's Megabytes (not a typo), which excess usage charged at $0.15 a megabyte.

Looking at the list of Telstra unmetered sites, Facebook isn't in the list, YouTube isn't, neither is eBay, which begs the question -- how much is the average family's internet bill per month.

SingTel/Optus come out marginally better, providing a 15GB plan for $59.95 AUD a month, with no excess data charges -- but a throttled speed of 128kbps once you've hit your limit, as well as not apparent unmetered content.

Internode came out better still, providing a 25GB plan at ADSL2 speeds for $59.95 with no excess usage charge, albeit a slower throttled speed of 64kbps, but provided an extensive online games and free software unmetered mirror, as well as a variety of uncapped site access.

In the last week, i'd been using my Playstation 3 at home a fair amount -- and decided to do a bit of research into how much bandwidth the average game used by running a few demos, plugging my DSL model directly into the PS3, thus removing the rest of the traffic from my LAN from the equation.

Call Of Duty 4, 6-man multiplayer with a 15 minute time-limit used 23M of data the first time, and 24M of data the second time (the only difference was the map we used).

Grand Theft Auto 4, 12-man multiplayer with a 20 minute time limit used 40M of data each time.

The new demo for EA Sports NHL '09 came out on the Australian PS3 Network -- which I promptly grabbed, as a 'normal use case' test for the average household, at 1102M on it's own, that accounts for $169.33 AUD of excess usage charge at Telstra, or nearly a tenth of the available bandwidth at Optus using the above numbers on it's own.

After returning my LAN to the equation, I then stumbled over the following article, that talked about the USA lagging Japan in terms of aggregate bandwidth for the medium-to-long term and thought, sure -- Japan can reach the Japanese at 63M, possibly the Japanese eBay or Amazon too, but what's the speed like from there to Europe, or the USA.

I understand that is actually costs a lot of money for anything to get to Australia & the cost of cable, backchannel links and maintenance is prohibitive.

I also understand that Telstra is still in the position where they can play overlord to the communications network in this country and a mentality of 'we built the links, you paid for them, now we profit from them' is the standard, but that's the case with any 'Shareholder Concerned' business.

Maybe things would have been different (better?) if the government had forced Telstra to hand over control of the physical line infrastructure to AUSTEL 15-20 years ago, rather than dissolving AUSTEL, forming the ACA (and ACMA) instead and creating yet another mid-range government department while allowing the sole telecoms provider to become a complete juggernaut too.

They didn't -- and now we the consumers are paying and will continue to pay until either:

a) government regulates in consumers favour across the country (rather than concentrating on 'the bush')

or

b) the cows come home. (especially with the G9 facing resistance from the ACCC with their Fibre-To-The-Node proposal and Telstra pulling the telecoms version of an All-In-Over-The-Top-Raise, launching their own proposal for the rebuilding and revamping of Australia's telecommunications infrastructure.)

According to Chris's other comments, at least the pricing situation here isn't as bad as the one in South Africa were the equivalent of $140 AUD buys you 10GB, but that's for a userbase of around 10,000 users -- not 16.6 million.

The next few years will be interesting.

Playstation 3, Streaming, Formats ... Ick.

2008-08-23T13:06:00.002+10:00

I've been working away at a media streaming project for the last week or two -- mainly because running between the machine that holds most of my web-content and the TV becomes tiresome, especially when you've got most of the content in .mkv or .mp4 formats that don't fit more than an episode or two on a DVD-R.

I've tried TwonkyVision, which was widely recommended as the best solution for streaming with transcoding from external sources -- but is purchase-ware, as well as both FUPPES and MediaTomb from the world of OSS.

Quite frankly, I came away quite unimpressed.

Not because any of the software is bad, or particularly difficult to setup (albeit that you need a newer version of FFMPEG to get any reasonable quality from the HDTV transcodes, and that creates debian pain.) ...

More because the Playstation doesn't do half the things you'd expect of it.

Granted, it's behind the XBox360 juggernaut in terms of market time -- but if you're going to do format compatibility, please -- SONY, make a decent job of it.

First, I tried connecting the three media servers using the default instructions for each, enabled uPnP and a default multicast route for the boxes that held the data -- of course, while that meant my laptops could see the share -- the PS3 didn't.

Further investigation concluded it's nigh on impossible to get the PS3 (40GB, running firmware 2.42) to talk to a media server via the wireless interface.

15M of CAT5 later and a re-configuration of the PS3 to use a wired interface -- we had liftoff, well -- nearly, the PS3 had dropouts whenever it tried to look up directories on the share, it'd start searching, get through the first 20 or 30 entries and stop.

More poking ensued -- turns out the PS3 needed an explicit route to the box hosting the media, easily fixed -- but in no manual, it only triggered in my brain because the box doing the serving (running Red Hat Enterprise Linux 4.x) was triggering source route notifications in my logfiles.

Now the PS3 saw the media, as a variety of MPEG-2's, and Unknown Data.

Back to the interschnitzel, to find MediaTomb and FUPPES both have 'a workaround' to make the PS3 see DivX files, except they both don't actually work on PS3's with 2.4x firmware 'out of the box' (a phrase becoming my new favourite annoyance)

For MediaTomb this means adding:

<map from="avi" to="video/x-divx"/>
<map from="divx" to="video/x-divx"/>

For FUPPES this means:

<file ext="avi">
<type>VIDEO_ITEM</type>
<mime_type>video/x-divx</mime_type>
</file>

... to your configurations.

A 'Re-boot and Re-Import' later the PS3 saw a bunch of MPEG-2 and DivX files, Good.

Except half of them wouldn't play, Bad.

Infact, the PS3 seems more picky about which DivX/XviD files it'll play than the documentation suggests -- the exact same file with the exact same settings encoded in XviD 1.10 and 1.12 play differently on the PS3, the 1.10 file is called 'Corrupted Data' but the 1.12 copy plays normally, albeit with audio skew caused by lag.

Then there's MPEG-2's that won't play if they are in a TS container, but will if the stream is copied to a PS container first.

Matroska (MKV) based H.264 files won't play at all either, having a platform that says it supports 'new media' and not having a Matroska muxer/parser is ... er, strange.

My personal favourite though, is that transcoding anything high-definition fails using 'chunked' encoding because the 'buffer' size (I set for 512k) is too large.

Luckily, that's an easy fix for FUPPES. For MediaTomb it's not straightforward, you need to change the 'fill' size to workaround it ... but again, that's not documented anywhere either.

The working settings I came up with (which need to be added to each transcoding section where you're converting H.264 (.mkv or .mp4 mostly, unless you watch High-Def Pornography, in which case i'll leave it as an exercise to the reader) were:

MediaTomb:

<buffer size="20971520" chunk-size="524288" fill-size="10485760"/>

FUPPES:

<http_encoding>stream</http_encoding>

In the end, after recompiling both platforms -- I settled on MediaTomb, although I now have both RHEL 4 and Ubuntu 8.04 packages for both platforms from their respective RCSes from 20080818, so I can switch easily if I want to change.

There'll be a forthcoming post on how to configure the transcoder scripts and FFMPEG, sometime when i've got hours up my sleeve to document it with sane reasoning and screenshots, i've got it going now -- but if I wrote it up, it'd look ranty and like I was SONY-bashing.

I agree with this guy. The XMB is nice, the changing colour for the seasons is a nice touch too, but if the PS3 doesn't increase the titles available via the Playstation Network (In Australia, we don't have a Madden '09 demo yet, for example -- nor do we have Castlevania, but we do have a bunch of music videos and some streaming from various trade events .. for all the use they are.) then it'll be behind the X360 for a while to come.

However, if they don't fix the format capabilities -- probably by this holiday season, then they'll be behind the X360 for a long, long, long time to come.

Really, an Matroska muxer -- support for all three main H.264/AAC profiles, MPEG2-TS and -PS support and a differentiation between DivX (DX50/DX60) and XviD (XVID) would be nice, at least so I don't have to transcode the latter to view something you're natively capable of viewing in the first place.

If we got that, plus maybe an Dirac and Vorbis implementation (I mean, does anyone use ATRAC) -- the Playstation 3, to quote a great movie reference ... "would become more powerful than you could possibly imagine."

... but at the moment, it's a games platform, with a swanky front-end.

If SONY think it's anything more than that, they owe me my weekend back.

Totem Packages Available (see: Totem, GStreamer & nVidia Graphics Cards)

2008-08-19T14:44:00.001+10:00

Playing with a possible fix for these bugs in Totem.

There's packages for Ubuntu Hardy here -- that are the same as the release ones except one rather messy hack that shifts the Hue plus-or-minus 90 from whatever position Totem starts in.

It's basically the code from here cleaned up a tad and dropped in as a patch.

They seem to work for me, using the GStreamer pipelines I presented in the last post.

Not a clean solution, but I can play a full playlist of videos without the colour skewing once -- which is better than upstream can do at present.

Bluetooth's "Operation Not Supported By Backend" Message

2008-08-15T16:29:00.003+10:00

This morning, while trying to move some files between my mobile phone and my Ubuntu 8.04.1 machine -- I was greeted with a "Operation Not Supported By Backend" message with the address of my phone and the drag between Nautilus windows was terminated.

Turns out, it is because the GVFS backend for Bluetooth doesn't support the device -- and using the older, GNOME-VFS way isn't supported either.

The 'Send-To Bluetooth' option (right-clicking the files you want to send and selecting your mobile) works as per normal and successfully transferred 25M of files to my phone in under a minute.

Mozilla, SSL & the 'non-optimum' Security Warning

2008-08-11T17:39:00.000+10:00

A number of people have been blogging about the state of the SSL Certificate Security Warning since the release of Firefox 3.0.

I must admit, personally I don't mind the dialog that pops up -- it scares the everyday user into thinking twice before sending their data to Nigeria by accident.

It is actually far more awkward to import the various extra root certificates into the various operating environments, than it is to do certificate exemptions on a site by site basis.

I found the report that Federico linked to slightly disturbing -- if 58% of certificates are indeed invalid, expired or otherwise bad, that's a hell of a lot of users that are experiencing an all-too-confusing dialog box far too often.

(On that note: If you have an expired certificate, you should really get it renewed -- especially with a commercial signer, after all -- you've built a reputation with that certificate, you shouldn't have customers turning away because that little yellow bar they've been used to becomes a scary looking error message.)

I like CACert myself, I use it for things regularly and i've configured several e-commerce installations to use certificates for it, after going through the somewhat painful verification process to get a two-year certificate instead of a three-month one.

For commercial stuff though, CACert isn't really practical -- especially considering very few operating environments include their root certificates by default.

For semi-commercial stuff, there's no middle-ground, there's either commerical CA's, Homebrew, or nothing.

For personal use, there's GNU Privacy Guard -- a much better, but less Microsoft-supported way of confirming you really are who you say you are.

I've often thought about the issue in my business, where I see all sorts of certificates on a week-to-week basis -- and often need to handle the case of 'a user complained my certificate was invalid, I bought it and gave it to you, so you must have broken it.'

The thing I haven't been able to come up with yet, is the solution:

For big corporates, there's Verisign or Thwate, which is prohibitively expensive for a single-user in the home.

For SME's there's second-tier signers, like Comodo, GoDaddy or Network Solutions.

For Free Software Developers and other Personal Use there's basically CACert, or doing it yourself. Neither of which, are supported by anything remotely mainstream without doing a hell of a lot of legwork yourself.

Maybe Mozilla themselves, or Google could do something to help the situation by running a CA that works in parallel with the other services they provide -- but how would that be any less work that rubber-stamping CACert?

Well, even though the principle is the same, if Google did it -- it'd probably be supported everywhere -- but as of now, CACert are still running the gauntlet with Mozilla and will probably have a much more difficult task getting past Microsoft and Apple, accordingly.

Autodesk Backburner & VMWare Clones

2008-08-06T17:29:00.003+10:00

If you've installed Autodesk's Backburner product within VMWare (part of 3D Studio 7/8/9/2008) and have trouble getting the Backburner server to start because the "UDP interface is not valid in this context", the solution is two-fold:

First, Power off your VM
Then, edit the .vmx file for the failing VM and change:

ethernet0.generatedAddressOffset = "0"

To:

ethernet0.generatedAddressOffset = "1"

Power up the VM and log in to Windows -- using Windows Explorer, navigate to the C:\Program Files\Autodesk\Backburner\Network directory.

Delete the backburner.xml file.

Now when you restart the Backburner server application, a new configuration file will be created with the new GUID and SID of the VMWare instance and it should start up and run normally.

Encoding Videos for your PS3 using GStreamer

2008-08-05T16:44:00.002+10:00

Following up my previous post, i've been playing more with HDTV content on my machines -- and spending more time re-coding content for my PS3 so I can watch it from the comfort of my lounge.

Using Avidemux is nice, but occassionally -- it's nice to stick it in a terminal, with relatively low overheads and use the GStreamer framework to do the same job.

So, here's two examples of how to use the CLI to generate content playable on the PS3.

For Standard Definition (SDTV) content -- you can use:

gst-launch-0.10 filesrc location="input.avi" ! decodebin name="decode" decode. ! queue ! ffmpegcolorspace ! ffenc_mpeg4 bitrate=999999 gop-size=24 qmin=2 qmax=31 flags=0x00000010 ! avimux name=mux ! filesink location="output.avi" decode. ! queue ! audioconvert ! lame name=enc vbr=0 ! queue ! mux.

As of the 2.42 firmware, the PS3 has issues playing VBR audio -- so you have to explicitly turn it off (setting it to CBR audio) using the vbr=0 option to lame.

For High Definition (HDTV) content in .MKV format -- the x264 encoder works better, with a higher quality re-encode at the expense of a larger output file (obviously, if you've downloaded a HDTV file that's already been encoded in the .avi format, you'd want to use the SDTV code above, because the .avi file would already have significant quality loss over the original source material)

In HDTV, we can also use AAC audio to generate 4.1 channel audio -- as opposed to recoding to MP3.

gst-launch-0.10 filesrc location="input.avi" ! decodebin name="decode" decode. ! queue ! ffmpegcolorspace ! x264enc ! ffmux_mp4 name=mux ! filesink location="output.avi" decode. ! queue ! audioconvert ! faac name=enc ! queue ! mux.

Where input.avi is the path to your existing media source andoutput.avi is the path and file that you'd like to save.

important note: Yes, the full stop (after the mux.) in both cases is intentional -- if you take it out, GStreamer will complain about the pipeline.

Totem, GStreamer & nVidia Graphics Cards

2008-08-04T02:10:00.004+10:00

I've been bitten by these bugs fairly often on my HP DV6000 laptop -- and with nVidia claiming it's nothing to do with them, I decided to do a little investigation.

Turns out, Totem seems to reset the video settings after each video has been played.

If the quality sliders in Totem are dead center for all four settings (Saturation, Contrast, Hue and Brightness) ...

... the video displays with a bluish tinge unless you use the following GStreamer Video Output pipeline:

ffmpegcolorspace ! video/x-raw-yuv,format=(fourcc)YV12 ! videobalance contrast=1 brightness=0 hue=-1 saturation=1 ! autovideosink

If the colour settings slider for Hue is at the far left (as has been suggested as a solution by several people), the following pipeline works:

ffmpegcolorspace ! video/x-raw-yuv,format=(fourcc)YV12 ! videobalance contrast=1 brightness=0 hue=0 saturation=1 ! autovideosink

However, regardless of which pipeline one chooses, Totem seems to reset itself each time, seemingly trying to adapt to the optimum setting, which means the first video you play will display correctly, but following videos will be blue.

At this point, i'm not really sure how to fix it -- but nVidia suggest that it isn't their problem and Totem should fix it.

The interesting thing about that, is if I take a screenshot of a playing video -- the screenshot is the correct colour, all the time.

Tweaking XDG Settings on Ubuntu

2008-07-19T13:49:00.000+10:00

More of a remember this for later post, but the content took over an hour to figure out, even though it was remarkably simple.

XDG allows you to alter the default directories for a number of commonly used locations found on a users desktop.

This file is stored in $HOME/.config/user-dirs.dirs, you can change -- for example, Videos to be My Videos (making it more like Windows), or hide the Templates directory by changing it to .config/nautilus/Templates (which, you'll need to create first)

However, after editing this file and logging out/in, the GNOME panel and filechooser may show duplicates of these directories (once for the old directory, once for the new one).

The solution, is to remove the $HOME/.gtk-bookmarks file, then log out and re-log in, the file gets generated at login if it doesn't exist and will read the contents of your $HOME/.config/user-dirs.dirs file in order to get the correct locations.

Encoding Videos for your PS3 with Avidemux

2008-07-10T11:23:00.002+10:00

Sometimes, the interscnitzel is helpful -- sometimes, it leaves you without hair while you try every conceivable setting to make things work.

My better half wanted to watch some IPTV we'd downloaded on the big TV (Standard Def), which is hooked up to my Playstation 3. IPTV comes down as .AVI in SDTV and .MKV for HDTV stuff, but the PS3 decided all of that was 'Unsupported Data'

Checking the list of supported video files (please Sony, add .MKV to your list), we found that AVI files should be supported.

For those interested, there's a more detailed examination of what AVI files (DivX or XviD) the PS3 will play, over here.

After some searching, we fired up Avidemux (2.4.2, Linux -- but the Windows one would work too, if you didn't have anything else available) and played with Settings -- the best set the we came up with was:

Video: MPEG-4 ASP (lavc)
Audio: MP2 (lavc)
Format: AVI

To make the whole process a little easier, I also went into Edit/Preferences/Automation and switched on Automatically build VBR map, Automatically rebuild index and Automatically remove packed bitstream (as all the AVI files come as "AVI, pack VOP", which needs to be unpacked first before you can change the output format type).

Now, save your file -- put it on a USB stick or DVD and watch TV on your PS3 :)

note: if you're using DVD-RW's to watch movies from (my personal preference), the Sony DVD-RW (Gold, in Green cases) work flawlessly and seem much more reliable than the TDK or Imation ones (My 40GB PS3 has yet to successfully read an Imation DVD, for example).

important note about USB sticks: After some headscratching over why videos don't play from USB sticks, I discovered the USB device should be FAT32 formatted (default on SANDISK drives, if you've got access to a Linux box, mkfs.vfat will do the job. Then, on the device, you need to create a directory called "VIDEO" (all caps, without the quotes) and put your videos in that directory, then the PS3 will see them properly.

Fixing Printer Scaling on Ubuntu 8.04 (bug: 217151)

2008-07-02T15:03:00.002+10:00

During a recent rollout of "Ubuntu on the Desktop" I ran foul of this bug in Epiphany.

The symptoms, as the bug describes -- are the headers and footers of a document are printed, but no body text -- we tested this on various printers and came out with the same result.

To fix it, we applied a hack at login-time to manually set people's preferences to Scale=100, by editing the ~/.gnome2/epiphany/print-settings.ini file and changing:

--- Scale=1

to

+++ Scale=99

(note: I never use 100, because it makes badly-designed websites like Online Banking websites sometimes print outer-borders on a second page, 99 fixes that with no user-discernable difference to the print quality)

It fixed the problem, but hacks are never elegant, break easily and are not good to apply over a 400-workstation installation, so this morning -- my task was to track it down and fix it.

Turns out, it's an innoculous little bastard in the Epiphany gecko code that caused it.

In the embed/mozilla/GeckoPrintService.cpp file, there's a line (around 737-739) that reads:

" gtk_print_settings_set_scale (aGtkSettings, 1.0); "

But this states:

" scale : the scale in percent "

So, the solution turned out to be changing:

gtk_print_settings_set_scale (aGtkSettings, 1.0);

To:

gtk_print_settings_set_scale (aGtkSettings, 99.0);

The bug has now been noted, with a patch that fixes the issue here

Interesting Viewpoint

2008-06-27T12:18:00.004+10:00

I wandered across this article this morning and thought it deserved posting, especially in the light of similar Microsoft sentiments in recent months.

While Microsoft can't possibly 'buy out' Open Source as a whole, it'll be interesting to see if it means Microsoft start interacting with Open Source in a better light to ensure more interoperability between systems (possibly the only way to achieve continued market share), or if they'll make better quality end-products, or systems to deploy those products on, or if they'll continue to do exactly what they've done in the past.

One thing is nearly certain, within the next five years, Microsoft will need to do something a) special or b) underhanded to continue to operate in the market as they have done -- the amount of end-users that are coming to me and other ISV's since the advent of Microsoft's Vista product-line and asking about Linux on the desktop is increasing and that can only be a promising sign for the future.

The time for lipservicing the FOSS market, is over.

Using mod_security 2.5.x with Apache 2.x

2008-06-16T09:17:00.000+10:00

note: This article is intended for a technical audience -- you should use extreme caution when modifying a production system, as your data will be nearly impossible to recover if you use this command incorrectly -- caveat emptor.

There's a lot of posts about how to setup the mod_security module for Apache, but few on how to configure it -- hopefully people find this post useful in doing just that.

Before we start, i'm assuming you've actually installed mod_security 2.1.3 or 2.5.x here already (Red Hat/CentOS packages are here, Ubuntu / Debian ones are here, an OpenSuSE howto is here).

I'm also assuming you've made a copy of the core rules that come supplied with the package and put them in the /etc/modsecurity directory.

note: If your distribution of choice doesn't ship the core rules with the packages, you can download those from here.

Now, to make a decent configuration.

First, move (don't copy, or the default configuration may override any environment-specific changes you make) the /etc/modsecurity/modsecurity_crs_10_config.conf file to /etc/apache2/conf.d/mod_security.

Open the newly copied /etc/apache/conf.d/mod_securityfile and edit the following parameters:

SecResponseBodyLimit

262144

SecAuditLog

/var/log/apache2

/var/log/apache2/modsec_audit.log

SecDebugLog

/var/log/apache2/modsec_debug.log

At this point, you should save your file and restart your Apache 2.x server in order to ensure your configuration works. If you run:

cat /var/log/apache2/error.log | grep “ModSecurity”

You should see the string:

“[Fri Jun 13 23:23:23 2008] [notice] ModSecurity for Apache/2.5.5 (http://www.modsecurity.org/) configured.”

Which means we can proceed to add our rules to the configuration. To do this, open your configuration file again and add the following line to the bottom:

Include /etc/modsecurity/rules/*_crs_*.conf

This will add the core rules to your configuration. Once again, you can restart your server and the changes will take effect.

Transmission, MIME-types, Java?

2008-06-15T03:49:00.003+10:00

I'm posting this here for two reasons, a) the solution works but sounds like overkill and b) because I couldn't find any information on the interschnitzel directly -- it just happened to be a solution for another application that worked in this case too.

After updating Transmission (my favourite bittorrent client) on my Ubuntu 8.04 desktop machine, I found that torrent files I downloaded weren't appearing in my client, but they weren't appearing on my desktop either.

Sometimes they worked if I downloaded the .torrent file more than once, but more often than not, they didn't.

Looking through all the usual suspects (MIME-types in Nautilus, etc) proved unsatisfying, so I had a look in my .xsession-errors file (using: tail -f ~/.xsession-errors from a terminal while downloading a file, and found:

GCJ PLUGIN: thread 0x816c470: NP_GetMIMEDescription
GCJ PLUGIN: thread 0x816c470: NP_GetMIMEDescription return
GCJ PLUGIN: thread 0x816c470: NP_GetValue
GCJ PLUGIN: thread 0x816c470: NP_GetValue: returning plugin name.
GCJ PLUGIN: thread 0x816c470: NP_GetValue return
GCJ PLUGIN: thread 0x816c470: NP_GetValue
GCJ PLUGIN: thread 0x816c470: NP_GetValue: returning plugin description.
GCJ PLUGIN: thread 0x816c470: NP_GetValue return

GCJ?

Ah, the Open Java Variation of Sun's Java Runtime.

Turns out, if you have the Ubuntu Multiverse and Universe repositories configured & you install the ubuntu-restricted-extras package (the one that gives you multimedia encoding and playback capabilities, Java, Flashplayer amongst other things) it installs the Open Java (openjdk-6-jre) variations and not the Sun one.

To install the Sun Java Runtime from Multiverse, run the following from a terminal window:

sudo apt-get install sun-java6-bin

Then, to make Ubuntu use it, instead of the OpenJDK code, type:

sudo update-java-alternatives -s java-6-sun

Then restart Transmission & your chosen Web Browser and try downloading your .torrent files again.

Using GnuPG Agent on the Console

2008-06-13T14:14:00.004+10:00

After hunting around on the internet to find a definitive answer on how to use GnuPG's Agent with a remote console (it's easy if you have X installed, even easier than that if you use the awesome Seahorse application with GNOME -- but pretty awkward if you have neither of the above)

I think i've come up with a usable solution, that uses gpg-agent's --write-env-file variable & does some minimal extra checking to make sure it doesn't get killed accidently, as well as correctly exporting the GPG_TTY variable so applications like mutt and the package builder applications on the console get the key handling right.

In your .bash_profile file, you need to add the following code near the end:

# Invoke GnuPG-Agent the first time we login.
# If it exists, use this:
if test -f $HOME/.gpg-agent-info && \
kill -0 `cut -d: -f 2 $HOME/.gpg-agent-info` 2>/dev/null; then
GPG_AGENT_INFO=`cat $HOME/.gpg-agent-info | cut -c 16-`
GPG_TTY=`tty`
export GPG_TTY
export GPG_AGENT_INFO
else
# Otherwise, it either hasn't been started, or was killed:
eval `gpg-agent --daemon --no-grab --write-env-file $HOME/.gpg-agent-info`
GPG_TTY=`tty`
export GPG_TTY
export GPG_AGENT_INFO
fi

Save the file, then logout and re-login and you should find gpg-agent has been started correctly.

note: We use the "| cut -c 16-" in the first section in order to remove the duplicated GPG_AGENT_INFO= string from the output that causes errors like:

gpg-agent: can't connect to `/home/paul/.gnupg/S.gpg-agent': No such file or directory
gpg-agent: can't connect to the agent: invalid value

Fixing Font Resolution In Epiphany

2008-06-08T16:37:00.005+10:00

After recently scheduling a large client upgrade from Ubuntu 6.06 to Ubuntu 8.04, a number of people started complaining that fonts looked bad on the web browsers (Epiphany).

Although they all had video cards (nVidia) and monitors (Samsung SyncMaster's) in common -- a bit of Googling indicated an underlying software issue.

Fortunately, this is easy to fix.

If you open a web browser and type about:config into the location bar, which should take you to the configuration screen -- consisting of a filter box and a larger portion containing all of the relevant tweakable parameters.

In the filter textbox type: layout.css.dpi it should be a Default, Integer value (that is not bold) that looks like the screen below:

Now, if you right-click the bold text and select modify you should get a textbox appear in the center of the screen, simply use the keyboard or mouse to select and remove the -1 and enter either 70 (if you have a resolution lower than 1024x768) or 92 (resolutions of 1024x768 or higher) instead -- and press OK to return to the configuration menu.

At this point, the screen should look like the one below:

Now you can close the browser, the next time you restart it -- your fonts should look smoother and more readable.

edit: If you find that you can't edit the text-box on an OpenSUSE 11.0RC1 or Ubuntu Hardy 8.04 installation, try forcefully closing Epiphany by typing the following command in a terminal window:

killall -9 epiphany-browser && rm ~/.gnome2/epiphany/mozilla/epiphany/!lock

and then try to modify it again.

Updated Unofficial GStreamer FFMPEG Plugin Packages for Ubuntu 8.04

2008-06-04T19:58:00.002+10:00

As a follow-up to my recent post about the Fluendo Codecs, I decided to take a look at the GStreamer FFMPEG Plugin. -- and because a new release had just appeared with a lot of new fixes, built packages for it.

The packages are here, are based around the recently released 0.10.4 release and been built using the recommended upstream build of FFMPEG's libavcodec and libavformat code, rather than the (much) older code that ships by default in the Ubuntu 8.04 release.

These should be drop-in replacements for anyone using the current packages in the Ubuntu Multiverse repository.

Updated Unofficial Rhythmbox Packages for Ubuntu 8.04

2008-06-02T12:49:00.002+10:00

I've been playing about over the last week integrating a bunch of useful patches into my preferred music player, Rhythmbox.

The packages are here, are based around SVN revision 5710 and have a number of extra patches that haven't made it into the tree yet, including:

Bug 528814: RB should use podcast date and time when transferring to iPod (useful if you put your podcasts in playlists)
Bug 529873: The artdisplay plugin should be able to supply metadata (useful if you have coverart embedded in tracks and you'd like it transferred to the iPod)
Bug 345975: Show album covers embedded in files e.g. mp3 ID3 tags (very, very useful if you have art embedded in music from iTunes or some other tagging program)
Bug 140020: Song skips when position is moved maximum right (fixes a big bugbear i've had for some time, where clicking forward or back within the first second or two after fast-forwarding/rewinding with the slider causes Rhythmbox to skip the next track).

If you have been looking to try a newer Rhythmbox on your Ubuntu 8.04 installation, or have an iPod that you'd like to get more use out of, you might want to give these a go.

GStreamer Codec Install Gone Bad, Part Two?

2008-05-26T01:25:00.004+10:00

Tonight, while tracking down an issue with another video file that wouldn't play in Ubuntu 8.04, but would in anything else -- I discovered yet-another-quirk about the Fluendo codec megabundle, this time with the H.264 codec.

Attempting to play a movie on a fresh installation of Ubuntu 8.04, in Totem with the fluendo codecs installed brought up a black screen with a properties dialog that looked like:

By fresh -- I mean fresh, this box was a demo to show a new user how easy multimedia was (after successfully demonstrating how easy the printer was to set up using CUPS), it has only got the standard installation of Ubuntu, no multiverse, no universe -- infact, the only thing we'd done (aside from installing the foomatic printer drivers and configuring ufw to do basic firewalling) was install the binary codecs in the .gstreamer-0.10/plugins/ directory, as directed by the instructions.

By new user -- I mean someone who is planning on updating from Windows 98 to something for the first time, has enough computing power to run a non-Microsoft based operating environment, but not enough to run Vista and has heard enough people rabbit on about how good it could be to finally bite the bullet.

So, having hit the wall -- I fired up Totem from the command line using: totem --gst-debug-level=2 (which outputs minimal debug spew to the console) and found:

** Message: Error: File "/home/paul/.gstreamer-0.10/plugins/libmch264dec.so.7.4.0.20778" could not be used.
fluh264dec.c(414): gst_fluh264dec_setup (): /play/decodebin1/fluh264dec0:
Could not open module: libstdc++.so.5: cannot open shared object file: No such file or directory

After hunting around, the libstdc++5 package from the universe repository was installed, Totem was restarted and now happily displays the file correctly, albeit with a major visual issue when you use the arrow-keys to rewind the movie, but that's a story for another day.

Unfortunately, there's no documentation on the website, or in the tarball that tells you about needing to install older, relatively unsupported libraries to get something to play, hence this post.

I just hope the next version is compiled against something slightly more recent, so it works out of the box in future.

Customising the GNOME Screenshot Application

2008-05-23T16:49:00.003+10:00

Circa GNOME 2.20, the screenshot application became simplified -- it now just takes the screenshot of your entire desktop, with borders and asks you to save it somewhere.

What happens if you'd like to take a screenshot of a specific window, or remove the borders, or add a drop-shadow, or do any sort of resizing?

You can make a custom launcher for it. Luckily, none of the backend settings were removed, the user interface was just tided up.

First, right-click on your desktop and select 'Create Launcher' from the menu, which should take you to the configuration screen -- consisting of four boxes you will need to fill in.

In the Application drop-down box, make sure: Application is selected.

In the Name text-box, you enter the name of the launcher, Take Screenshot is a good choice, but you can call this anything you want.

In the Application text-box, you need to add the program with the options you need, you can select from:

--delay=X -- Delays the screenshot for X seconds after you double-click the link, very handy if you need to take shots of a menu bar or dialogue.
--window -- Takes a shot of the current window instead of the entire desktop.
--border-effect=X -- Takes a unmodified screenshot if you use 'none', adds a 2 pixel drop-shadow if you use 'shadow' or a 1 pixel black border if you use 'border'.
--window -- Takes a shot of the current window instead of the entire desktop.

The Comment text-box is optional, it offers you the ability to make a longer description of the launcher if you highlight it on the desktop, because of the options we've used above, we'll call it: Takes Screenshot of the current window after 10 seconds.

When you are finished, your launcher screen should look like the one below:

Click on the OK button when you're happy with your settings -- and the launcher should appear somewhere on your desktop.

OpenSSL Vulnerability for Ubuntu 6.06 LTS

2008-05-17T15:15:00.000+10:00

Had a phone-call earlier about this as well as a few e-mails since DSA 1571-1 appeared, so i thought i'd post this here in order to respond to multiple birds with one stone.

Servers that run Ubuntu 6.06 LTS in it's default configuration (or it's LAMP configuration) are not vulnerable to the OpenSSL problem, because they are running OpenSSL 0.9.7, not 0.9.8c-1, which is the first version to exhibit the bug.

Systems which are running any of the following releases are vulnerable to this bug:

Ubuntu 7.04 (Feisty)

Ubuntu 7.10 (Gutsy)

Ubuntu 8.04 LTS (Hardy)

Ubuntu “Intrepid Ibex” (development): libssl <= 0.9.8g-8

Debian 4.0 (etch) (see corresponding Debian security advisory)

This is not mean that you shouldn't check your users keys -- if you've got users who use affected versions of Debian or Ubuntu (above), you should use the dowkd.pl script available here (GPG key) with the user option to scan your servers for users who have potentially compromised keys.

You can scan the local server using:

perl dowkd.pl host localhost

... and local users with keys using:

perl dowkd.pl user

If you see something like:

/home/[username]/.ssh/id_dsa.pub:1: weak key

You should re-generate keys for that user, using:

ssh-keygen -t [rsa/dsa] -b [1024/2048/4096]

... depending on your individual security needs.

note: If you see:

/home/[username]/.ssh/id_dsa.pub:1: 2048 bits DSA key not recommended

You are not necessarily vulnerable, there's nothing wrong with using 2048-bit DSA keys, as longer key lengths provide better security at the cost of decreased performance.

note II: Using a blank passphrase for your public key is strongly discouraged -- if a would-be-intruder can just press the ENTER key to enter your machine, what security is a public key?

(The most-used analogy I have for passphrase-less SSH keys is: 'A public key with a passphrase is like a door with a lock -- without one, it's just a door.)

Blank Text Area in Zen-Cart?

2008-05-03T14:12:00.002+10:00

Have you ever been editing a product description, or a category title in Zen-Cart and had the field go blank?

Since updating a bunch of older 1.2.x stores to 1.3.x, I have -- and customers don't like data entry at the best of times, let alone when you have to tell them to edit it down to less than 200 characters and try inputting all again, rather than just editing out the excess characters.

So i've spent the day tracking down a solution, which happens to be via the javascript counters for the various products.

If we look at /includes/modules/pages/product_info/jscript_textarea_counter.js

Line 8-10 states:

if (excesschars > 0) {
field.value = field.value.substring(0, excesschars);
alert("Error:\n\n- You are only allowed to enter up to"+maxchars+" characters.");

If you change that to read:

if (excesschars > 0) {
field.value = field.value.substring(0, maxchars);
alert("Error:\n\n- You are only allowed to enter up to"+maxchars+" characters.");

You'll find you get the error, rather than the field going blank -- because it gives the user the ability to then edit what they've typed, it's a much better solution to the problem.

note: If you use "Free Shipping" for any of your products, you should also apply this fix to the /includes/modules/pages/product_free_shipping_info/jscript_textarea_counter.js file (the files are identical).

Transparent Proxying with Shorewall, SQUID and Privoxy on Ubuntu

2008-04-14T14:57:00.006+10:00

note: This article is intended for a technical audience -- and while it seems to works for the two systems i've tried it on since, you should use caution when modifying a production system -- caveat emptor.

A 'remember these for later brain' post, mainly because I always forget the order when I haven't done it for a while.

note:, you can do this with any distribution -- Ubuntu Server works and I use that, but you can substitute the names of your distributions packages instead and still get much the same result.

First, having installed Ubuntu Server, you'll need the universe repository, as privoxy isn't officially blessed by the Ubuntu maintainers.

Next, you'll need to install the various packages you'll need to make this fly -- as root, run:

apt-get install squid privoxy shorewall

After that's done, you'll want to make SQUID work first -- using Ubuntu 8.04, this entails opening the default SQUID configuration at /etc/squid/squid.conf and:

Adding transparent to the port configuration by changing:

--- http_port 3128
+++ http_port 8080 transparent

Adding your {W/L}AN block to the ACL list by adding:

+++ ACL local_network src 192.168.0.0/24

... and enabling it in the http_access list by adding:

+++ http_access allow local_network

Save the configuration and exit the editor.

Now restart SQUID with: /etc/init.d/squid restart to have your changes take effect.

At this point, configure a browser to use a manual proxy on the server's IP address on port 8080 and make sure it actually works, if you receive an error that talks about Access Control Lists, check that you added the right network mask to the SQUID local_network line you added above.

If that works, we can move on to the Privoxy part of things. Privoxy, for those who are unaware is one of the best 'web-crud(tm)' filters i've ever had the pleasure of using, it was originally built from the Internet Junkbuster (IJB) but has now got many more features, is stable on pretty much any platform for a wide-variety of users and protects your privacy too.

By far, the easiest way to configure Privoxy is via the web-interface, but the Ubuntu package disables that by default, so before we hook it up to SQUID, we should enable that.

Open the /etc/privoxy/config file and make the following changes:

Enable editing of actions via the web interface by changing:

--- enable-edit-actions 0
+++ enable-edit-actions 1

Save the configuration and exit the editor.

Now restart Privoxy with: /etc/init.d/privoxy restart to have your changes take effect.

Now, we need to hook Privoxy up to our proxy as the default parent cache -- you'll need to open the SQUID configuration file again and make the following adjustments:

note: If you haven't done so already, it's a good idea at this point to make a backup copy of your /etc/squid/squid.conf file before making these changes.

Adding Privoxy as a cache peer by changing:

+++ cache_peer 127.0.0.1 parent 8118 0 no-query no-delay no-digest no-netdb-exchange

note: because Privoxy cannot influence any of SQUID's cache settings, setting no-query no-delay no-digest no-netdb-exchange as options for the peer cache lessens the delay between Privoxy filtering the transaction and SQUID caching it of up to a second on slower hardware (for example, a Pentium 4 1.2Ghz machine with 1GB of memory).

Telling SQUID to always send traffic from the firewall directly to the internet by changing:

+++ always_direct allow localhost

Telling SQUID to never send traffic from the local LAN (thereby forcing users to use the Privoxy/SQUID cache) directly to the internet by changing:

+++ never_direct allow local_network

Save the configuration and exit the editor.

Now restart SQUID with: /etc/init.d/squid restart to have your changes take effect.

Finally we need to add the rules for transparently using our proxy to our Shorewall Firewall Configuration -- you'll need to open the Shorewall rules configuration file and make the following alteration:

note: For the purposes of simplicity, i'm assuming that your WAN interface is eth0, your LAN interface is eth1, your LAN IP range is 192.168.0.0/24 and your Shorewall configuration is already complete (for some reason I still don't fully understand, the Debian and Ubuntu packages don't ship with a default configuration file, so if you don't see a /etc/shorewall/rules file, you'll need to download one, or grab a prefabricated copy from the /usr/share/doc/shorewall directory and set it up first.

Adding a redirection to our transparent proxy by changing:

+++ REDIRECT loc 8080 tcp www -
+++ ACCEPT $FW net tcp www

note: the minus (-) with the trailing space at the end of the redirect line is important, it means it will ignore the source port when working out the request and force any already established request to continue to use the proxy.

Save the configuration and exit the editor.

Now restart the Shorewall Firewall with: /etc/init.d/shorewall restart -- fire up a browser, make sure it is not configured to use a proxy (ie. it uses a direct connection to the internet) and browse to your hearts content.

Australian'ising Epiphany's Keyword Search

2008-04-10T01:08:00.006+10:00

As part of the upgrading process to the new Ubuntu LTS release -- I found my default search engine had been reset to http://www.google.co.uk rather than http://www.google.com.au in Epiphany.

Everything else about Epiphany's default setup is Australian, the language defaults are en-au and the languages are set correctly, but the keyword search isn't.

Fortunately, this is easy to change.

First, fire up your browser and type about:config into the location bar, which should take you to the configuration screen -- consisting of a filter box and a larger portion containing all of the relevant tweakable parameters.

In the filter textbox type: keyword.URL it should be a User Set, String value (in bold) that looks like the screen below:

Now, if you right-click the bold text and select modify you should get a textbox appear in the center of the screen, simply use the keyboard or mouse to select and remove the .co.uk section of the highlighted URL and enter .com.au instead -- and press OK to return to the configuration menu.

At this point, the screen should look like the one below:

Now just restart Epiphany and do a search, the browser should now (correctly) take you to Google Australia's search.

OpenOffice 2.4's Compatibility with Microsoft Word 2000-2007

2008-04-08T21:15:00.016+10:00

After yet-another virus outbreak at my better half's university -- I set about installing Linux on her machine, overwriting the old Windows 2000 with Word 2003 setup that i've re-installed, patched and registered no less than four times in the last six months.

Two weeks in, she's mostly satisfied -- but there's sticking points with OpenOffice and the compatibility with Microsoft Word.

Thankfully, there's some cute things you can do to to improve the look and feel of OO.o under Linux.

note: This has been tested with Novell's version of OpenOffice (currently 2.3) and Ubuntu's Hardy version (currently 2.4.0) -- milage shouldn't, but may vary on other platforms.

Overall Document Compatibility

First of all, open OpenOffice Writer and find the Tools / Options menu.

Then select OpenOffice.org Writer from the left-hand menu -- your screen should look something like the screenshot below:

Next, choose Compatibility, your screen should look like:

Now, to obtain decent compatibility with Word 2000/2003/2007 make sure these options are checked:

Use printer metrics for document formatting
Add spacing between paragraphs and tables (in current document)
Add paragraph and table spacing at tops of pages (in current document)
Do not add leading (extra space) between lines of text
Add paragraph and table spacing at bottom of table cells
Consider wrapping style when positioning objects
Expand word space on lines with manual line breaks in justified paragraphs

When you've finished, your screen should look like:

Finally, to set these options as default -- simply click on the Use as Default button, then the OK button to return to the program.

Microsoft-Sized Margins

The next query had to do with the margin size in OpenOffice, which I personally like -- but university lecturers take umbrage with, so we'll alter those to be more compatible with their Microsoft brethren, which has the added side-effect of being WYSISYG to most extents when the document is printed via Word.

To do this, we need to make alterations to the default template. I'm presuming you're starting with a blank document on the screen, just by the way.

First of all, find the Format / Page menu -- your screen should look something like the screenshot below:

The default margin sizes in OpenOffice are 2cm (two centimetres -- or around 0.78 inches for US readers)

Whereas, the default margin sizes in Microsoft Word 2003 are 3.81cm (1.5 inches) for the top and bottom margin and 2.54 cm (1.0 inches) for the left and right ones, so you need to alter the Left, Right, Top and Bottom margins on this page to match the screenshot below:

Now that we've modified our blank page to be more Word Compatible at the expense of being less tree friendly, we can now save it for use as a template.

Find the File / Templates / Save... menu, your screen should look something like the one shown below:

There will be a text field you can edit called "New template". Enter "Microsoft Word Compatible" or a similar defining name here.

Below the edit box is a section titled Templates for organizing a lot of templates if you had many that you used. Given we are only modifying a single template, leaving it under the My Templates category is fine.

Finally, press OK to save it.

At this point, OpenOffice will continue to use the default margins, If you would prefer to set these margins as your default template for whenever you make a new document there are a few extra steps to follow:

First, go to the File / Templates / Organize menu and double click on the "My Templates" folder.

Under the directory, your "Microsoft Word Compatible" template that you created should appear. Right click on it and select "Set as Default Template" -- your screen should look similar to:

Now press "Close" to exit the dialogue and return to OpenOffice, from this point on -- all new documents will use Microsoft Word compatible margins and page-widths by default.

Speedier Startup

Not that it's really anything to do with compatibility, but if you don't use the Java components in OpenOffice on Linux (ie. you have no JDK installed) you can save a few seconds start-up time and have a snappier interface by turning the java components off, to do this, go into the Tools / Options menu.

Then select Java from the left-hand menu and uncheck the Use a Java Runtime Environment box.

Close OpenOffice and re-open it for a much speedier environment.

Streamlining the Ubuntu Boot Process

2008-04-02T20:52:00.000+11:00

note: This article is intended for a technical audience -- and while it seems to works for the two systems i've tried it on since, you should use caution when modifying a production system -- caveat emptor.

Not-so-simple one line fix that shaves a good 5-10 seconds off the boot up time of a 1.8ghz HP Notebook machine.

Open a terminal and use sudo with your favourite text editor to edit the /etc/init.d/rc file, eg.

sudo vim /etc/init.d/rc

Then, around line 24, change:

--- CONCURRENCY=none
+++ CONCURRENCY=shell

Save the file, then the next time you reboot -- Ubuntu's base bootup speed will be even more similar to a Microsoft operating environment than ever before.

Recovering your Password Database in Epiphany 2.20.x on Ubuntu

2008-03-25T21:31:00.004+11:00

Having just upgraded my laptop to the recently released beta version of Ubuntu's next Long Term Service release (Hardy Heron -- aka. 8.04), I started up my preferred browser (Epiphany) and discovered that my saved web password database was empty.

After a bit of web hunting, I discovered that the new Firefox 3.0 betas have a new version of the signons.txt file that Mozilla/Firefox/Epiphany use to store the actual password.

Looking in my .gnome2/epiphany/mozilla/epiphany directory, I noticed a signons2.txt file and a new, blank signons3.txt file.

I also noticed that the datestamp on the key3.db file had been updated to today's date.

First, I tried renaming the signons2.txt file to signons3.txt and restarting Epiphany -- as expected -- the datestamp of signon3.txt was now the same as key3.db.

Going into the Personal Data / Passwords menu in Epiphany now gave me access to some passwords, but not all -- plus there was some duplication between passwords.

So, I tried deleting both the key3.db and the signons3.txt file -- and copying a backup version of both files (with an older datestamp) to my epiphany profile directory and then renaming signons2.txt to signons3.txt.

Restarted Epiphany and from a terminal window, noticed both files had been updated to the current time.

This time, going back to the Personal Data / Passwords menu in Epiphany now gave me access to all my old password information.

So, in a nutshell -- to upgrade from Ubuntu Gutsy to Ubuntu Hardy's Epiphany seamlessly, you should use a three step process:

* Backup your .gnome2/epiphany directory before doing your upgrade.

* After doing your upgrade, copy the key3.db and signons2.txt files from your backup into your updated .gnome2/epiphany/mozilla/epiphany directory.

* Before using Epiphany for the first time, copy your signons2.txt file to a new file called signons3.txt.

note: this issue has also been reported to the Ubuntu Bug Tracker as #180205.

WD's 'My Book' Product & Linux

2008-03-23T03:11:00.002+11:00

note: This article is intended for a technical audience -- you should use extreme caution when modifying a production system, as your data will be nearly impossible to recover if you use this command incorrectly -- caveat emptor.

Today I picked up one of the new 'essential edition' 750 GiB USB External Hard Drives -- very nice piece of kit, it looks like the volume of a book and fits on the bookshelf (albeit if one runs the power supply and related cords down the back of said bookshelf first) as advertised on the box.

To make it work under Linux wasn't hard either.

It comes with a FAT32 (vfat) filesystem by default and includes Windows and MacOSX versions of Google Desktop, Skype and a few other things.

The procedure here was run on OpenSuSE 10.3, but any semi-recent Linux distribution will do -- it uses one command, mke2fs -- which is present on all Linux distributions back to the dark ages:

First, unmount the drive (either from the right-click desktop option, or from the command line)

Second, open a terminal and sudo to root.

Third, format the drive -- I use ext3 filesystems, so my command line was:

mke2fs -j -L "My Book" -m 1 -O dir_index,sparse_super -T largefile /dev/sdX1

Where sdX is the WD device, I bought two of them and they were sdd and sde respectively. (note: this is never usually /dev/hda or /dev/sda -- please make sure you're writing to the correct device before pressing ENTER, type once, look twice!).

As for options, I used those to get the most storage and speed from my new drives:

-j -- Adds an ext3 journal to the filesystem.
-L "My Book" -- Adds a Label to the new drive (by default, the drive mounts out of the box with a label of 'My Book', so I use that, but you could call yours anything as long as it is less than 16 characters long, just remember to put it in double quotes on the command line.
-m 1 -- Uses 1% of the total drive space (750MiB) as reserved blocks for the root user, which is handy to have when the drive dies and one needs to run recovery tools to get back their data.
-O dir_index,sparse_super -- Two options, seperated by commas -- one (dir_index) to speed up lookups in large directories (say, one with the contents of your digital camera or music collection) and the other (sparse_super)to gain a few extra megabytes of usable space by not creating as many superblock backups (on a 750GiB drive, this option creates 24, rather than 80-odd due to the sheer size)
-T largefile -- Creates a filesystem suitable for medium to large files (a few megabytes or more per file) rather than one suited to a standard home directory (lots of files worth a few kilobytes, intermixed with some larger ones).

Once you have pressed ENTER, the mke2fs will format your drive using the parameters specified on the command line, then return you to the prompt.

Fourth, you will need to give your user account permission to write to the drive (at the moment, only root can do this) -- as root, type:

chown -R username:users

Where username is your username, mine's paul, yours probably isn't.

Finally, remove the USB cable and plug it back in -- if you're running a recent Linux distribution (2006 onwards) it should re-appear on your desktop as 'My Book' with an unreadable directory called lost+found.

Now you can use your drive as normal, with a native filesystem, access rights and all the rest.

Repairing Zen-Cart's use of HTML Characters (umlauts, cyrillic, etc)

2008-03-20T12:01:00.001+11:00

One of the bigger alterations i've made to the Zen-Cart code recently has been a large update of it's country and state zone files (so instead of users typing their state and city into text fields, they can select them from a drop-down box that only shows references for their particular country).

Unfortunately, in 1.3.7 and 1.3.8 -- the code for handling HTML entities is slightly broken, so users from Nordic or Germanic countries see:

Baden-Württemberg rather than Baden-Württemberg

In places like the Shipping Information or the Account Creation code.

The solution i've been using, is to edit the files that reference the zen_js_zone_list code and correctly sanitise the output-- this is done in two places (one for admin, one for userland).

Firstly, edit the admin/includes/functions/html_output.php file and around line 188 change:

--- return $output_string;
+++ return html_entity_decode($output_string);

Secondly, edit the includes/functions/functions_general.php file and around line 1448 change:

--- return $output_string;
+++ return html_entity_decode($output_string);

You should now have accented characters in places they should be.

Improving Browser Speeds in Epiphany

2008-03-19T17:04:00.017+11:00

Mainly a 'remember these for later brain' post -- but people might still get a fairly good speed improvement out of them with Xulrunner or Gecko.

updated (10-04-2008): added screenshots -- so it matches the other HOWTO's here :)

Remove the initial rendering delay

Firstly, Open Epiphany (all of these work for Firefox as well, mind you.) and type about:config into your location bar.

You should see a screen that looks like the one below:

First, Gecko (the Mozilla Backend) uses a delay when rendering any new page on-screen -- you can remove this delay by right-clicking anywhere in the main window and creating a New Integer key.

In the textbox that appears, name the new key “nglayout.initialpaint.delay” and set its value to “0″.

Disabling IPv6

Next, if you don't use it -- IPv6 can add delays to any page you view -- you can turn it off in Epiphany by toggling the "network.dns.disableIPv6" key to true.

First, find the "network.dns.disableIPv6" key in the filter, your screen should look similar to:

Right click the key and press the Toggle button -- your screen should now look like:

Disabling Prefetching

Prefetching, otherwise known as the ability to cache the next page you might want to read can actually slow your browsing down if you're on a larger network, behind a proxy or just browsing a site on a slow server -- also, SSL pages (such as those on online banking sites) are much snappier if you turn this off, although i've yet to prove why this is true, or if it's just a side-effect something else.

Anyway, to turn prefetching off -- find the "network.prefetch-next" key in the filter, your screen should look similar to:

Right click the key and press the Toggle button -- your screen should now look like:

Tuning Pipelining

Pipelining refers to the ability to use one HTTP connection to obtain multiple pieces of data -- reducing the load on the servers you are browsing and maximising the amount of data your connection can receive.

There's a LOT of discussion about pipelines and their use -- i've seen pages that tell people to increase their pipeline count from 4 to 30, 60 or even 100 -- of course, telling the server you can fit more data into one connection than you can possibly receive is annoying for server administrators ...

Why?

Imagine you've got a server that hosts something reasonably popular and you've got a client that doesn't support pipelining, one that does using the default and and one that's tuned their client to receive 100 requests at once ...

The server, that basically knows nothing about what an end-user can support when it is asked to build a new connection, so in this case it would allocate enough resources to send 105 requests at once.

Even though the average broadband (512k ADSL) connection would use 5 in about the same fashion at the receivers end.

Considering the maximum number of actual connections to a particular server is in the range of 4 and 16 and that HTTP requests remain open between your client and the server for 300 seconds (both of which we'll tune later in this article) -- requesting an excessive amount of data per request is just poor considered poor etiquette.

So, let's tune our pipelining to a value that makes sense from both an available bandwidth point of view and doesn't annoy every network administrator on the planet.

To do this, we need to make sure pipelining is turned on, then we need to set two values -- one for connections created normally and one for those created by a proxy.

First, find the "network.http.pipelining" key in the filter, your screen should look similar to:

Right click the key and press the Toggle button.

Next, repeat the process for the network.http.proxy.pipelining key.

Finally, we need to define a value -- this basically depends on the type of site you're browsing, but it is considered to be poor form to make this value more than a double digit number -- I'm using 8 on all my boxes, 12 works quite well too on my 3 megabit connection, but I personally wouldn't recommend anything higher than that -- when you're done, your screen should now look like:

Tuning Per-Server Connection Limits

Now that we've tuned the pipelined connections, we need to increase the number of connections that your browser can open to a server before needing to queue new ones -- this speeds up sites that host their static content on other servers to the dynamic content, as well as making more efficent use of prefetching too.

To do this, find the "network.http.max-persistent-connections-per-server" key in the filter, your screen should look similar to:

The defaults in Firefox 2.x (xulrunner 1.8) are 2, they have been recently increased for Firefox 3.x (xulrunner 1.9) to 6.

Personally, i've found 4 to be a reasonable number on my lower-end machines (those less than a P4 1.5ghz), I use 8 everywhere else, because it matches my pipelining use.

Increasing this number to more than 10 makes the performance of Epiphany actually reduce considerably, it makes significant delays after the fourth or so page I access at once, possibly due to the fact the maximum total number of connections that Gecko opens at once is 24 (or 32, if you're using a newer Firefox 3.x build).

Right click the key and press the Modify button. Now enter the value in seconds that you'd like to change it to -- remembering the guidelines listed above.

Your screen should now look like:

Tuning Keepalive Timeouts

HTTP Keepalives play an important role in how long upstream servers keep connections open to the one client. It's incredibly useful being able to hold a connection open for a client, but having your client tell the server to hold it for 300 seconds (5 minutes) uses valuable resources on internet servers, as well as holds valuable RAM within your browser -- Basically, on the off chance you might go back to that site within the 5 minute window.

In my experience, 60 seconds (1 minute) works just as well, as is far friendler to the internet at large.

Before anyone asks (someone over my shoulder just did) -- this is not the same thing as a website timing your session out due to inactivity (like an online shop or a bank) -- this is the lower-level server infrastructure of the interschnitzel, not the website itself.

To change the keepalives to something more friendly:

Find the "network.http.keep-alive.timeout" key in the filter, your screen should look similar to:

Right click the key and press the Modify button. Now enter the value in seconds that you'd like to change it to -- personally, I wouldn't recommend going below 30 seconds (or the pipelining improvements we made above become mooted), but 30, 60, 90 and 120 are all reasonably acceptable values.

When you're done, your screen should now look like:

At this point, you might be inclined to say -- that's fast enough -- but there's one other rendering tweak which makes Epiphany much snappier ...

Tuning Page Re-rendering Delay

These tweaks control the number of times that Gecko/Xulrunner will refresh the page as data is being retrieved. The default is off, which will make the backend re-render the page periodically as each piece of data is being retrived.

If you're not behind a caching proxy, these particular tweaks will improve the amount of time it takes to render a page that contains content coming from a number of different sources (for example, Youtube or the Australian Stock Exchange) as well as remove the hideous delay that the non-free Flash Player has when it is being embedded into a page that contains .swf content.

In order to make this work, we have to add two new entries to our configuration -- right-click anywhere in the main window and creating a New String key.

In the textbox that appears, name the new key “content.notify.ontimer” and set its value to “true″.

Repeat the process, except this time -- make a New Integer key.

In the textbox that appears, name the new key “content.notify.backoffcount” and set its value to “5″.

After pressing OK, your screen should look similar to the one below:

Skip To The End...

Don't want to hear the explainations? Just want to cut-and-paste some recommendations? If so, make sure Epiphany is completely shutdown (or the preferences file will be overwritten with the defaults after you eventually close it).

Open the .gnome2/epiphany/mozilla/epiphany/prefs.js file with your favourite text editor and add:

user_pref("content.notify.backoffcount", 5);
user_pref("content.notify.ontimer", true);
user_pref("nglayout.initialpaint.delay", 0);
user_pref("network.dns.disableIPv6", true);
user_pref("network.http.keep-alive.timeout", 60);
user_pref("network.http.max-persistent-connections-per-server", 8);
user_pref("network.http.pipelining", true);
user_pref("network.http.proxy.pipelining", true);
user_pref("network.http.pipelining.maxrequests", 8);
user_pref("network.prefetch-next", false);

Save your file and re-open your browser for a much, much speedier experience.

Other Advanced Tweaks

note: These particular tweaks should be used by a technical audience only -- they may adversely affect your browsing speed on low-end machines, Epiphany's memory usage, or cause some machines to crash, you should use caution when modifying a production system -- caveat emptor.

These two tweaks control the amount of time (in milliseconds) that it takes the backend to return control to the rendering engine after the user has moved the mouse, used the keyboard or done some other "high-latency" task that pauses the rendering engine in favour of user interaction.

They both depend on the key “content.notify.ontimer” being available and enabled, which you should already have if you have followed the rest of this guide.

user_pref("content.notify.interval", 750000);
user_pref("content.switch.threshold", 750000);

This means the browser will wait less than 1 second (0.75 seconds) to return processing to the renderer, rather than the default 1 second.

On much slower machines, you may want to move them to slightly higher than 1 second values -- which makes the backend give more processing to the user interface at the expense of rendering time. A friend with an older Pentium III on a 256k ADSL connection has better performance from:

user_pref("content.notify.interval", 1250000);
user_pref("content.switch.threshold", 1250000);

Which causes the browser to wait an extra quarter of a second for the user to stop doing anything before giving everything back to the rendering engine -- it also means they can view an extra few pages before Epiphany starts becoming sluggish.

The End...

Context-Sensitive Menus in GNOME ...

2008-03-18T01:55:00.003+11:00

While wandering the interschnitzel today, I wandered pass planet.gnome and noticed an entry that i've given a bit of thought to over the years ...

Lucas wonders why GNOME uses context sensitive menus for all devices / locations?

While I agree, in theory -- if Nautilus (and other GNOME components) start removing these context-sensitive actions, consider the use-case for USB sticks (CF cards, etc) that are mounted read-only, been mounted by the system (thus you don't have rights) or have run out of room?

Personally, I find the greyed out icons for Cut/Copy/Paste actions a big timesaver -- if I do a copy and paste, I can instantly tell right now if I can 'do' that, because the icons are there, but context-disabled if I cannot.

If GNOME chooses to remove those because of 'clutter' -- the next notification one gets is a 'cannot write to XXX' *after* they've attempted once, twice, three times to do such an action.

Care should be taken not to be too overzealous with this sort of thing, in my opinion.

Anyone else agree, disagree, other -- voice your comment on the bug.

Adventures with GARNOME 2.22.0

2008-03-12T00:29:00.001+11:00

Just to see if it would work, i've just attempted to compile the platform and desktop directories from the upcoming GARNOME 2.22.0 release of GNOME on both OpenSuSE 10.3 and Slackware 12.

Needless to say, the experience was better than previous releases -- a few minor caveats to be aware of for those who are stuck:

OpenSuSE will not build the platform/gnome-vfs directory without the libtasn1-dev libraries from Factory being installed first, libtasn is a spinoff of the gnupg libraries -- but it doesn't seem to be included in the official repositories, you can use the OpenSuSE package search to find a version suitable for your machine.

Slackware gets much further (Slackware 10, in contrast -- never got really far) -- a common build mistake that the uninitiated may trip on is that some required includes (like limits.h) are included in the kernel-headers package.

A full Slackware 12 build toolchain -- as required to build GARNOME is:

binutils, gcc, gcc-c++, gcc-g++, diffutils, patch, flex, bison, gawk, m4, make & kernel-headers

edit i: A new dependency list (DEPS-LIST) detailing all the packages required, has been sent to the current GARNOME maintainers -- hopefully it'll arrive for GARNOME 2.20 (fingers are crossed)

Securely deleting the contents of a USB stick

2008-03-07T13:28:00.004+11:00

note: This article is intended for a technical audience -- you should use extreme caution when modifying a production system, as your data will be nearly impossible to recover if you use this command incorrectly -- caveat emptor.

When discarding, selling or lending a USB memory stick to someone -- you probably want to ensure there's no critical or sensitive data on it, so here's another handy one-liner to purge data from a USB memory stick when you don't have access to specific wiping programs such as shred or wipe on your platform.

This uses the 'dd' command, so it can be used on most systems with a random entropy generator and the coreutils/fileutils/base-utils package installed:

dd if=/dev/urandom of=/dev/[X] bs=512 count=1 conv=fsync oflag=direct

Where [X] is the device you wish to wipe (note: this is never usually /dev/hda or /dev/sda -- please make sure you're writing to the correct device before pressing ENTER, type once, look twice!).

This will write blocks using direct I/O, 512 bytes at a time to your USB stick from the entropy buffer -- additionally, the conv=fsync option makes sure that each block is completely written to disk before proceeding on to the next one.

When it is finished, you should see something along the lines of:

dd: writing to `/dev/sdd1': No space left on device
10490446+0 records in
10490445+0 records out
1971107432 bytes (1.9 GB) copied, 2097.36 seconds, 1.0 MB/s

Now you can go off and put a filesystem on your drive, as normal.

Converting FLAC to MP3 with GStreamer

2008-03-06T00:48:00.004+11:00

Handy one-line command to convert a FLAC file into an MP3 suitable for use with iTunes when you've only got access to the command-line:

gst-launch filesrc location = "song_name.flac" ! flacdec ! lame ! filesink location = "song_name.mp3"

I personally like CBR 320 bitrate MP3's, so my particular command-line looks like:

gst-launch filesrc location = "song_name.flac" ! flacdec ! lame vbr=0 bitrate=320 ! filesink location = "song_name.mp3"

You'll need gst-launch (or gst-launch-0.8 | gst-launch-0.10 if you're using Debian or Ubuntu) and the relevant codecs. (in this case, flac and lame)

Adventures with a D-Link DWL-G123

2008-03-04T13:28:00.004+11:00

I've spent the last few hours attempting to get a newly purchased D-Link USB Wireless Adapter working on one of my older desktop boxes that had a dead ISA NIC.

(Yep, not a PCI NIC, but an ISA one)

So, after purchasing a USB daughterboard with 4 ports on it and plugging in the USB adapter -- there's no lights.

A quick search on the web indicated that the card might be atheros based, but the Madwifi site didn't gleam any good answers.

Next, the box in question had a half-complete Ubuntu 7.04 install (upgraded from 6.10) on it and no working NIC.

Easier solution to that, having found the CDRW in the box doesn't like DVDRW discs -- putting the Ubuntu 7.10 installation on a USB key and mounting the ISO via loopback using:

mount -t iso9660 -o loop /media/disk/ubuntu-7.10-alternate-i386.iso /media/cdrom

Then adding:

deb file:///media/cdrom gutsy main restricted universe

To the /etc/apt/sources.list file, then doing a standard apt-get update && apt-get -f dist-upgrade command line got things underway.

After an hour or so, the upgrade had completed and prompted me to reboot -- after I did, it bombed to the command line and told me that /dev/sda1 had been used by another device or driver -- while the kernel spewed device-mapper lines across the screen talking about devices it couldn't locate.

Returning to the internet on another machine, I found this bug post that looked similar -- sure enough, removing EVMS with apt-get remove evms and rebooting did the trick.

At this point, I had a desktop -- but no Internet.

So I put the driver CD into the machine and run:

ndiswrapper -i /media/cdrom/Drivers/2KXP/NetA5AGU.inf

Which returns:

installing neta5agu ...
forcing parameter MapRegisters from 256 to 64
forcing parameter MapRegisters from 256 to 64
forcing parameter MapRegisters from 256 to 64
forcing parameter MapRegisters from 256 to 64
forcing parameter MapRegisters from 256 to 64
forcing parameter MapRegisters from 256 to 64
forcing parameter MapRegisters from 256 to 64
forcing parameter MapRegisters from 256 to 64

Then ndiswrapper -l to make sure the driver is correctly loaded, which returns:

neta5agu : driver installed
device (2001:3A03) present

Finally, I modprobe the ndiswrapper module with modprobe ndiswrapper

... and the lights come on -- and clicking on the network-manager icon finds my network and prompts me for my password.

But that's as far as it gets, the lower green dot stays on for a good 90 seconds, then goes back to showing disconnected.

OK, off to get the newest drivers -- which are located here

Extract the drivers (use the 2KXP drivers and not the Vista ones, which are incompatible with ndiswrapper) and remove and replace the drivers.

Use modprobe -r ndiswrapper && ndiswrapper -r neta5agu to remove the drivers.

Then use the instructions above to install the new drivers, replacing /media/cdrom with the patch you extracted your downloaded drivers to.

But ...

Network Manager still didn't want to connect, I selected my network, entered my password and waited -- but Network Manager still didn't connect to my WLAN.

In a fit of desperation, I decided to backport the version of ndiswrapper that is in the upcoming version of Ubuntu to 7.10.

You can get those from here.

Then I installed those using dpkg -i ndiswrapper*.deb

Rebooted.

Then re-inserted the ndiswrapper module using modprobe.

... both lights came on ...

Logged into my machine.

... crossed my fingers while the green dot sat with the grey one ...

... then the green one ...

... then the strength bar! ...

At this point, I had networking, could use Epiphany and all.

So to finish up, I built the configuration for ndiswrapper using ndiswrapper -m then added ndiswrapper to the /etc/modules file so that the ndiswrapper module would be automatically loaded on startup.

I rebooted again, just to make sure everything remained in a working fashion -- everything came up normally, complete with the strength bar in the NetworkManager applet.

It's all good.

In summary, if you use the latest driver from D-Link and Ubuntu Hardy (8.0x) it might "just work" -- but if you use any of the earlier Ubuntu's and are having trouble getting your card working, I hope the instructions above work for you too.

grepping for periods and at's

2008-02-13T16:27:00.005+11:00

A handy one-line hack:

I needed a way of detecting if a user had entered a domain name correctly within a shell script (or had entered an FQDN, rather than just the domain name) -- thankfully, the power of grep came to my rescue:

if [ ! -z "$(echo $dname | grep -oE '[[:alnum:]+\.\_\-]*\.[[:alnum:]+\.\_\-]*\.[[:alnum:]+\.\_\-]*\.')" ] ; then
echo "Please enter the *domain name* only" >&2
exit 0
fi

GStreamer Codec Install Gone Bad?

2008-02-08T00:51:00.000+11:00

Sometime before linux.conf.au I purchased the Fluendo binary codecs for GStreamer so I could transcode a bunch of WMA music I acquired into something more useful.

I've had the chance tonight to sit down and play with them -- I followed the instructions for a user based installation, namely, create a directory called .gstreamer-0.10/plugins in my home directory, untar the bundle into this directory and run /usr/bin/gst-inspect in order to update the registry to recognise them.

This part went smoothly enough, running gst-inspect-0.10 | grep flu I get:

flump3dec: flump3dec: Fluendo MP3 Decoder (C build)
flumpeg2vdec: flumpeg2vdec: Fluendo MPEG-2 Video Decoder
fluisodemux: fluisodemux: ISODemux Demuxer
flumms: flummssrc: Fluendo MMS source
fluasf: fluasfdemux: Fluendo ASF Demuxer
fluasf: fluasfcmdparse: Fluendo ASF Command Parser
fluh264dec: fluh264dec: Fluendo H264 Decoder
fluac3dec: fluac3dec: AC3 decoder
fluwmvdec: fluwmvdec: Fluendo WMV Decoder
flumpegdemux: flupsdemux: MPEG Program Demuxer
flumpegdemux: flutsdemux: MPEG Transport stream demuxer
flumcaacdec: flumcaacdec: Fluendo AAC Decoder
flumpeg4vdec: flumpeg4vdec: Fluendo MPEG-4 ASP Video Decoder
fluwma: fluwmadec: Fluendo WMA Decoder
fluwma: fluwmsdec: Fluendo WMS Decoder

However, the WMA codec doesn't load -- instead, the Ubuntu 'easy codec installer' tells me that I should go off and install the gstreamer0.10-ffmpeg package, which, while being nice in theory ... is not what I just paid 40-odd euros for.

Then I fired up Totem from the command line using: totem --gst-debug-level=2 (which outputs minimal debug spew to the console) and found:

0:00:04.868806000 19249 0x809b470 WARN GST_PLUGIN_LOADING gstplugin.c:459:gst_plugin_load_file: module_open failed: /home/paul/.gstreamer-0.10/plugins/libgstflumpegdemux.so: failed to map segment from shared object: Operation not permitted

(totem:19249): GStreamer-WARNING **: Failed to load plugin '/home/paul/.gstreamer-0.10/plugins/libgstflumpegdemux.so': /home/paul/.gstreamer-0.10/plugins/libgstflumpegdemux.so: failed to map segment from shared object: Operation not permitted
0:00:04.868982000 19249 0x809b470 WARN GST_PLUGIN_LOADING gstplugin.c:928:gst_plugin_load_by_name: load_plugin error: Opening module failed: /home/paul/.gstreamer-0.10/plugins/libgstflumpegdemux.so: failed to map segment from shared object: Operation not permitted
0:00:04.869028000 19249 0x809b470 WARN GST_PLUGIN_LOADING gstpluginfeature.c:132:gst_plugin_feature_load: Failed to load plugin containing feature 'flupsdemux'.
0:00:04.869073000 19249 0x809b470 WARN GST_ELEMENT_FACTORY gstelementfactory.c:406:gst_element_factory_create: loading plugin returned NULL!
0:00:04.869115000 19249 0x809b470 WARN decodebin gstdecodebin.c:909:try_to_link_1: could not create an element from flupsdemux
0:00:04.871628000 19249 0x80d5748 WARN GST_PLUGIN_LOADING gstplugin.c:459:gst_plugin_load_file: module_open failed: /home/paul/.gstreamer-0.10/plugins/libgstflumpeg2vdec.so: failed to map segment from shared object: Operation not permitted

(totem:19249): GStreamer-WARNING **: Failed to load plugin '/home/paul/.gstreamer-0.10/plugins/libgstflumpeg2vdec.so': /home/paul/.gstreamer-0.10/plugins/libgstflumpeg2vdec.so: failed to map segment from shared object: Operation not permitted
0:00:04.871677000 19249 0x80d5748 WARN GST_PLUGIN_LOADING gstplugin.c:928:gst_plugin_load_by_name: load_plugin error: Opening module failed: /home/paul/.gstreamer-0.10/plugins/libgstflumpeg2vdec.so: failed to map segment from shared object: Operation not permitted
0:00:04.871697000 19249 0x80d5748 WARN GST_PLUGIN_LOADING gstpluginfeature.c:132:gst_plugin_feature_load: Failed to load plugin containing feature 'flumpeg2vdec'.
0:00:04.871713000 19249 0x80d5748 WARN GST_ELEMENT_FACTORY gstelementfactory.c:406:gst_element_factory_create: loading plugin returned NULL!
0:00:04.871727000 19249 0x80d5748 WARN decodebin gstdecodebin.c:909:try_to_link_1: could not create an element from flumpeg2vdec
0:00:04.873951000 19249 0x80d5748 WARN mpegdemux gstmpegdemux.c:767:gst_mpeg_demux_parse_packet: unknown stream id 0xbe
0:00:04.874523000 19249 0x80d5748 WARN GST_PLUGIN_LOADING gstplugin.c:459:gst_plugin_load_file: module_open failed: /home/paul/.gstreamer-0.10/plugins/libgstflump3dec.so: failed to map segment from shared object: Operation not permitted

(totem:19249): GStreamer-WARNING **: Failed to load plugin '/home/paul/.gstreamer-0.10/plugins/libgstflump3dec.so': /home/paul/.gstreamer-0.10/plugins/libgstflump3dec.so: failed to map segment from shared object: Operation not permitted
0:00:04.874568000 19249 0x80d5748 WARN GST_PLUGIN_LOADING gstplugin.c:928:gst_plugin_load_by_name: load_plugin error: Opening module failed: /home/paul/.gstreamer-0.10/plugins/libgstflump3dec.so: failed to map segment from shared object: Operation not permitted
0:00:04.874586000 19249 0x80d5748 WARN GST_PLUGIN_LOADING gstpluginfeature.c:132:gst_plugin_feature_load: Failed to load plugin containing feature 'flump3dec'.
0:00:04.874609000 19249 0x80d5748 WARN GST_ELEMENT_FACTORY gstelementfactory.c:406:gst_element_factory_create: loading plugin returned NULL!

Operation not permitted?

Removing the .gstreamer-0.10/plugins directory fired up the MAD mp3 decoder, which was what it should have done, but that still didn't use the binary codecs.

Looking at /etc/fstab gave me the answer:

UUID=d85165d4-57a0-4f75-b5b1-a07fb8a5c444 /home ext3 nodev,noexec,nosuid 0 2

Needed to be:

UUID=d85165d4-57a0-4f75-b5b1-a07fb8a5c444 /home ext3 nodev,nosuid 0 2

In other words, you need to explicitly allow executables within your home directory by removing the noexec option.

(Of course, if you're not as paranoid about launching binaries from your home directory as I am, making nodev,nosuid just say defaults again will work for you too.)

Rebooted the machine & instantly files using the new codecs started playing correctly.

Probably something that won't happen to everyone, but it was a nice gotcha that kept me amused for an hour or two before :)

note: Your UUID will differ from mine too, just by the by -- there's no need to edit that part, just the mount options.

note ii: Fluendo seem to use OSCommerce for their shop, I wonder if they've after heard of Shop Of Shops?

Posting Quotes In Blogger

2008-01-27T12:39:00.002+11:00

If you use the Blogger website to post your items and you want to post backticks "'" or greater/less-than < or > symbols in the body of your text (for example, if you post code or HTML fragments) you may find you don't get the desired result when pulishing your post.

If you'd like to fix this, you can use the following HTML entities to get the same result, but Blogger won't consider these as HTML codes and will display them literally, meaning your posts will display as normal:

"'" (Backtick) = & # 39 ;
"<" (Left Bracket, Less Than) = & # 60 ;
">" (Right Bracket, Greater Than = & # 62 ;

Using mod_security 1.9.x with Zen-Cart (for users)

2008-01-23T23:23:00.002+11:00

Since my last post, i've had a few people ask me what happens if they're with a hosting service that doesn't want to adopt the changes I mentioned to get Zen-Cart to work properly.

The answer, yes -- if you're stuck with a site that doesn't (or can't) use the changes above, you can filter them yourself, using the .htaccess files method that Zen-Cart uses.

Of course, this method has a few prerequisites:

Your hosting provider uses the standard GotRoot Rules for mod_security (ie. they haven't modified any of the ID's).

Your hosting provider allows the use of .htaccess files (if you can install Zen-Cart on your host at all, this is probably true).

To make this work, you need to edit two files, the main .htaccess file and the admin/.htaccess file.

important note: These changes are Case Sensitive, make sure you copy them exactly as written (or better than that, Copy-and-Paste them directly into your editor software).

Firstly, edit the main .htaccess file and add the following lines to the bottom:

<IfModule mod_security.c>
<LocationMatch "/ipn_main_handler.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>
</IfModule>

Save this file and upload it to your webhost, this should disable the required rules to make the PayPal IPN processor work correctly.

Next, edit your admin/.htaccess (if you have renamed your admin/ directory as suggested by the Zen-Cart instructions, put that directory name in instead) file and add the following lines to the bottom:

<IfModule mod_security.c>
<LocationMatch "/admin/attributes_controller.php">
SecFilterRemove 300015
</LocationMatch>

<LocationMatch "/admin/newsletters.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>

<LocationMatch "/admin/product.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>
</IfModule>

Save this file and upload it to your webhost, this should disable the required rules to make the Administration Pages work correctly.

Virtual Memory Tuning for Low-Memory (< 512M) Systems

2008-01-23T18:33:00.000+11:00

note: This article is intended for a technical audience -- and while it seems to works for the two systems i've tried it on since, you should use caution when modifying a production system -- caveat emptor.

for anyone who doesn't care for the details: scroll down to the bottom for the 'ideal' defaults I came up with for our test rig.

Over the last few months, i've been playing with VMWare more and more on lower-end workstations -- primarily, these systems aren't mine, they're systems of friends or relatives who have not wanted to update to Microsoft Windows Vista because of the abhorrent memory requirements.

Australian PC World recently ran an article on how to use Virtualisation (via VMWare) to run Ubuntu Linux under Windows XP -- However, getting Windows to use VMWare while doing something else that it considers as being resource intensive (like Printing) turned a number of people off.

Switching things around, running Windows XP under Linux worked better, but occassionally the swap started grinding and top reported huge jumps in it's usage.

The test installation was Ubuntu 7.10 (Gutsy Gibbon), with VMWare Workstation 6.0.1 (build 55017) and Windows XP Home (having 160M RAM allocated to the VMWare image) and Word 2003.

First off, we tried turning off things that were superfluous to the 'average' home users desktop -- Tracker -- was the primary saver, freeing up around 30M of real RAM we could use for other things.

This made Windows start in a reasonable time, but resource intensive apps tended to grind the swap a second time, the 1G swap partition was pushing it's limits at 972M and there was only 4M of conventional RAM left.

Looking through the wonderfully helpful /usr/src/linux/Documentation/sysctl/vm.txt file, I figured the following might give some improvement:

overcommit_memory:

This value contains a flag that enables memory overcommitment.

When this flag is 0, the kernel attempts to estimate the amount
of free memory left when userspace requests more memory.

When this flag is 1, the kernel pretends there is always enough
memory until it actually runs out.

When this flag is 2, the kernel uses a "never overcommit"
policy that attempts to prevent any overcommit of memory.

This feature can be very useful because there are a lot of
programs that malloc() huge amounts of memory "just-in-case"
and don't use much of it.

The default value is 0.

Sure enough, if I echo this -- I get:

windows@bites:~$ cat /proc/sys/vm/overcommit_memory
0

So:

windows@bites:~$ echo 2 > /proc/sys/vm/overcommit_memory
windows@bites:~$ cat /proc/sys/vm/overcommit_memory
2

Restart VMWare, there's less grinding -- but there's also a little more usability, the CAPT-based Canon printer dialogue talked with the LBP-1120 that was hooked up to the machine in Word, but I figured I could get one better, as the 1G swap partition was still way-too-high at 920M and there was only now ~40M of conventional RAM left.

Then, I found the 'new' defaults for vm_dirty_ratio (10) and vm_dirty_background_ratio (5)

Which, as it turns out is one of the bigger bottlenecks for VMWare use on smaller systems -- after some googling, we tried 20 and 10 respectively, which caused us to have more free memory when windows was 'idle', but still grinding.

OK, what about swappiness -- i'd played with the idea of autotuning/auto-regulating this when Con Kolivas was doing kernel patches in the early days and I tend to need to tweak it on servers I look after anyway -- So, I wondered if the same 60/40 versus 40/60 principle I use on my servers would work on an overly loaded workstation?

First, what does Ubuntu do by default:

windows@bites:~$ cat /proc/sys/vm/swappiness
60

So, using the 60/40 principle I mentioned earlier:

windows@bites:~$ echo 40 > /proc/sys/vm/swappiness
windows@bites:~$ cat /proc/sys/vm/swappiness
40

Restart VMWare -- no grinding during the bootup process for XP, slight grinding when the Anti-Virus process ran -- but we were closer, now it was a case of finding the sweet spot for the dirty page ratio.

Shutdown VMWare and opened a terminal, luckily, there's a handy file in /proc we can look at to see where these pages are going, called vmstat, if we look for the word dirty, we see:

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 9988
nr_writeback 0

That's at 20, if we return the vm_dirty_ratio to the default (10), it was even worse:

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 11996
nr_writeback 322

After doing a bit of Googling on why nr_writeback would be higher than say 4-5, we found references to the Red Hat Linux Tuning Guide, which said if vm_dirty_ratio was set too low, the number of pages to be written-back would grow erratically because the ratio was constantly being hit.

OK, so 20 fixes that, but the amount of pages still looks too big, what if we made it 40?

windows@bites:~$ echo 40 > /proc/sys/vm/dirty_ratio
windows@bites:~$ cat /proc/sys/vm/dirty_ratio
40

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 7844
nr_writeback 0

Start up VMWare, load our image, fire up Word and print -- and this becomes:

windows@bites:~$ grep -A 1 dirty /proc/vmstat
nr_dirty 8195
nr_writeback 1

... and (as I looked) a document came out of the printer and iTunes fired up.

A happy user, who could use Word in VMWare and Linux on the desktop -- on a Celeron 1.6G machine with 512M of RAM, without noise from the disc, the 1G swap partition evened out at about 800M and there was still 30M of conventional RAM left.

I was thanked, Evolution was set up, as was Epiphany and I put the settings we'd tweaked into /etc/rc.local file so they'd be used when the machine was next rebooted.

The sweetest spot I found, for those just looking to skip to the end -- was:

echo 10 > /proc/sys/vm/dirty_background_ratio
echo 40 > /proc/sys/vm/dirty_ratio
echo 40 > /proc/sys/vm/swappiness
echo 2 > /proc/sys/vm/overcommit_memory

Happy virtualisation country.

Using mod_security 1.9.x with Zen-Cart (for administrators)

2008-01-18T14:59:00.006+11:00

note: This article is intended for a technical audience -- and while it seems to works for the systems i've tried it on since, you should use caution when modifying a production system -- caveat emptor.

I use a combination of the GotRoot mod_security rules, as well as some finer-grained custom rules on a number of Zen-Cart installations I look after, occassionally though -- there are problems with false positives that need to be worked around to allow the shop owners a seamless experience.

The GotRoot rules, for those unaware -- is a collection of rules that protect against over 6000 different web vulnerabilities, covering generic PHP attacks, PHPBB, e-Commerce software and more. It is fairly all-encompassing and I recommend it highly to everyone who asks.

Some of these generic attack signatures cause false positives when applied to Zen-Cart's admin/ directory, as well as the older-style IPN payment processor for PayPal.

To counter these, create a file called zencart-exclusions.conf in your rules directory (most installations use /etc/modsecurity/rules), open that file in a text editor and add:

<LocationMatch "/admin/attributes_controller.php">
SecFilterRemove 300015
</LocationMatch>

<LocationMatch "/admin/newsletters.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>

<LocationMatch "/admin/product.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>

<LocationMatch "/ipn_main_handler.php">
SecFilterRemove 300015
SecFilterRemove 300016
</LocationMatch>

The SecFilterRemove tells mod_security to selectively remove the rules from the specific locations listed in LocationMatch, while the six digit numbers are the GotRoot ID's that will not be parsed.

Save the file and reload your webserver configuration for the changes to take affect.

SMTP Unblocking with Optus (Cable or DSL)

2008-01-08T11:39:00.003+11:00

In the process of setting up a mail server for a collegue the other night (in order to deliver mail from 6+ domains to one IMAP account) I found an interesting quirk.

After you've set up and configured a working e-mail server, with all the UBL checks and correct delivery methods in place -- if you find other SMTP transactions work OK, but the local client can't connect and you're with Optus (In Australia), go here to turn off the automated port 25 block.

Thinking on though, this procedure unblocks port 25 globally, leaving one open to the prospects of exploit code spamming people by using port 25 on the local machine -- after all, the mail server wasn't on the persons local machine or even their LAN, but halfway across the world -- the Optus site makes reference to the unblocking process being for advanced users -- wouldn't advanced users be better off leaving the block in place for all sites not in a list?

(Yes, the server could use SSL, or even be on a different port -- both would be viable alternatives to opening SMTP from a customer premises to the world)

New Rhythmbox packages for Ubuntu 7.10

2008-01-07T13:18:00.000+11:00

Playing with Rhythmbox over the weekend (the 'save art to ipod' patch was committed to SVN and several people I know feel this is a neat feature) -- so i've built a set of updated packages from GNOME SVN (based on revision 5535) for Gutsy Gibbon.

These packages have the code for using the new Totem Playlist Parser backed out (due to it not being available on Ubuntu 7.10, but does include #411634 -- which should stop duplicated tracks appearing on your device) as well as #345975 -- so MP3's with embedded cover art will be shown, just like iTunes)

Packages are in the usual location.

Bluetooth & Ubuntu 7.10

2007-12-21T15:03:00.000+11:00

Quick-and-dirty HOWTO for people looking to get a bluetooth compatible phone to talk to Ubuntu 7.10 (Gutsy) -- any phone should work, i've used a Nokia N80 successfully.

First, make sure you've got the Bluetooth Icon in the taskbar, if you don't -- there's a possibility your bluetooth adapter doesn't work with (or wasn't detected by) Ubuntu.

Second. go to a terminal and type:

sudo apt-get -f install gnome-vfs-obexftp

Which should pull in a bunch of dependancies for libopenobex1 and company.

Thirdly, if you'd like to use GNOME's "Send To" function to push files to your Phone (like to can to Evolution currently) you can type:

sudo apt-get -f install gnome-bluetooth

note: currently, the gnome-bluetooth package doesn't authomatically start the transfer daemon, you'll need to go through System -> Preferences -> Sessions and add a new program to the startup list, my settings for this were as follows

Name: Bluetooth File Transfer
Command: gnome-obex-server
Comment (optional): Transfer files via Bluetooth from "Send To"

Now, rather than seeing something similar to:

The Browse action reports:
"obex://[nn:uu:mm:bb:ee:rr]" is not a valid location.

You should be able to transfer files between your phone and your Gutsy machine :)

Better TTF fonts for LCD devices

2007-12-15T11:30:00.000+11:00

A small follow-up to my last post -- because after you've installed your fonts, you'd probably like them to look good.

note: In some countries, the use of hinting is covered by patents -- which is why some distributions don't ship this functionality by default (prefering instead to use an auto-hinter included within the Freetype 2.x code specifically designed to completely ignore the TrueType bytecode instructions) -- still, you should use caution when modifying a production system -- caveat emptor.

First, you need to make sure you are actually using the native renderer, rather than the auto-hinter for your fonts -- in Ubuntu you can do this by opening a Terminal and typing:

sudo dpkg-reconfigure fontconfig-config

On the first screen, you need to select 'native' rather than 'autohinter' -- you can safely leave the other screens as their default values.

Second, you'll need to change your theme to use the better hinting -- you can do this on Ubuntu in the Font preferences from: System > Preferences > Appearance and select Fonts tab.

On the first screen, select the 'Details' button in the bottom right-hand-corner -- which will take you to a more detailed menu where you can customise your font and rendering preferences -- the key options to set here are:

Smoothing to Subpixel (LCDs),
Hinting to Slight
Subpixel Order to RGB. (Should be the default already)

At this point, instead of having something like:

You'll have:

Which, I think you'll agree -- looks much better (and is much easier to read, too)

Installing TTF fonts in Linux

2007-12-13T11:50:00.000+11:00

Recently, I was asked how to get downloaded fonts onto an Ubuntu workstation in order to make prettier documents in OpenOffice -- there's two methods to doing this, which I thought i'd document for anyone having difficulties doing it themselves.

This article works on a Fedora 8, an Ubuntu 7.10 and an Ubuntu 6.06 machine -- but mainly relies on an XFree86 4.2 (or later) install with the fontconfig package installed (most current distributions have this installed already).

Method One: Single-user Fonts

Individual Users can install fonts by opening a terminal (Accessories -> Terminal) and making a .fonts directory in their home directory, like:

mkdir -p .fonts

Then, find a TTF font you like (Google thinks here is a good place to start looking) -- download it, extract it (fonts are usually distributed in .zip archives, so you'll need to extract the .ttf font file from it first) and copy it to the .fonts directory you just made.

Then you can either logout and login again, or restart the application you want the fonts to appear in and they should appear for use.

Method Two: System-wide Fonts

Administrators can install fonts by opening a terminal (Accessories -> Terminal), becoming the root user using either:

su - (Fedora)

or:

sudo -s (Ubuntu)

Then making a specific directory in the font location on their machine for TTF fonts (usually /usr/share/fonts/truetype), like:

mkdir -p /usr/share/fonts/truetype/winfonts

Download and extract the fonts to the new winfonts directory you made above, then update the font cache information for the machine by typing:

fc-cache -f -v

note: Some older Linux distributions (Fedora Core 3 and KUbuntu 5.04 both need this) may need to run:

mkfontscale && mkfontdir as the root user before running the fc-cache command above, so that your machine correctly creates a fonts.dir file that can be found by fc-cache.

If you're interested in Free and Open Sourced fonts, you should look at Ed Trager's site on the subject.

SSHFS (the alternative to fish://) on Ubuntu 7.10

2007-12-09T22:52:00.000+11:00

Working on a file located elsewhere in the world, or in the workplace can sometimes become very tedious -- especially when using the SSH option in the GNOME file manager Nautilus, which relies on gnome-vfs module, which has some notable performance issues that still have to be addressed.

On the console, we have the fish:// transport in programs such as Midnight Commander and the excellent FTP/SCP program lftp, but fish:// still tends to be much slower than SSH itself.

Luckily, there's now FUSE & an equally useful drop-in module, SSHFS which allows a user to mount a remote SSH/SCP session as a virtual filesystem, having any modified data copied over the secure tunnel with little to no extra overhead than a straight SSH session instelf.

FUSE is a kernel module for mounting different filesystems by an unprivileged user, which was adopted into the mainline linux kernel a while back. SSHFS is an add-on created by the author of FUSE enabling users to mount remote folders/filesystems using SSH.

The idea is very simple - a remote SSH folder is mounted as a local folder in the filesystem. After that, almost all operations on this folder work exactly as if this was a normal local folder. The difference is that the files are silently transferred though SSH in the background.

Installing FUSE with sshfs in Ubuntu can be done via the Synaptic Package Manager (System->Administration->Synaptic Package Manager or by typing (as root):

apt-get install sshfs

Next, you need to give permission to use FUSE to the relevant users by adding the FUSE group to their account -- again, you can either use the Users and Groups GUI (System->Administration->Synaptic Package Manager) or via the command line using:

usermod -G -a fuse [username]

In Ubuntu Gutsy, the fuse module is already loaded at boot-time via the /etc/modules file -- If you are using Ubuntu Dapper, or another distribution -- you may need to load the module manually, you can do this via the command line by typing:

modprobe fuse

Now we can try to mount a remote folder using the sshfs transport. First, make a directory to use as the mount point for your remote folder (in this example, it's ssh_mount, but you could call it anything you liked):

mkdir ~/ssh_mount

Which would create the /home/username/ssh_mount folder.

Then, run the SSHFS command as you would a standard SCP command:

sshfs remote_username@remote_server:~/files ~/ssh_mount

The command above will cause the folder /home/remote_username/files on the remote server to be mounted as /home/username/ssh_mount on the local machine.

For now on, any file action (copying, editing, removing) to this folder will result in transparent copying over the network via SCP directly, which provides a massive performance improvement over the fish:// transport or the GNOME-VFS implementation.

Once you've finished working with the remote filesystem, unmount the remote folder by running:

fusermount -u ~/ssh_mount

OpenSSH and 'Matching' Rules

2007-11-23T13:25:00.000+11:00

Most of the time on the servers I maintain, there's a strict set of options in the /etc/ssh/sshd_config file -- after all, things like X11Forwarding yes or any of the Plain-text challenge options should really be tightly controlled on operational servers.

Occassionally though, there's a reason to relax these permissions, but rather than doing this globally, OpenSSH has the ability to do this on a per user, group or connecting IP address basis.

A box that I look after has MaxAuthTries set to 1 and LoginGraceTime set to 30 in order to slow down brute-force password guessing attacks (something I highly recommend for boxes in the field, by the way) -- but the owner of the box is a little heavy on the keys sometimes and ends up having to try two or three times to get their password right.

Let's say our users name is Tom and he connects from 100.50.0.1

Simple solution -- First, open your /etc/ssh/sshd_config (or possibly /etc/sshd/sshd_config if you're using a Fedora/RHEL/SuSE box) file and add:

Match User tom Address 100.50.0.1
MaxAuthTries 5
LoginGraceTime 120

So, while normal users get 30 seconds and 1 try to enter their password -- the owner of the box gets 5 tries and 2 minutes before he's disconnected.

note: When using Match rules, the User (and/or Address) keywords are case-sensitive, at least on Ubuntu 6.06 and RHEL 4.

note ii: When specifying multiple Match rules on the line above (such as User and Address) the expression is treated as a logical AND -- so all the parts must match before the block is invoked.

.torrent files from the command line

2007-11-20T19:02:00.000+11:00

Over the past few weeks, i've been playing with BitTorrent more for moving files around, but found no good way of being able to make .torrent files from the command line.

About a week ago, I found CreateTorrent and attempted to build packages for it, there were two major issues -- 1) there was no ability to make torrents private and 2) the program kept segfaulting whenever there was a '(' or ')' in the directory the files were stored in.

Today, when searching for createtorrent patches on Google, I found:

buildtorrent, which looked to do exactly what I wanted.

The code wasn't GNU Autotools ready, so I spent an hour or so fixing that (patches have gone upstream to the author) -- then I was able to build packages for it.

Ubuntu Gutsy users can grab them from here, packages for other distributions will be available as I have time to build them.

Fedora and The iPod Nano

2007-11-09T01:42:00.000+11:00

In semi-celebration of Fedora 8 being released today, I have spent the day wrestling with mock and rpmbuild to bring you up-to-date libgpod packages (built from SVN 1759), allowing Fedora users to try out the new iPod handling code just like Ubuntu users can.

Completely unofficial, just like their Ubuntu cousins -- but they build cleanly and correctly write the Firewire ID to a brand new 3rd Gen Nano (and thus, gtkpod actually syncs music and rhythmbox reads the music).

Packages available from here.

Enjoy.

edit i: It appears the packages weren't actually mentioned in the original post, thanks to James for pointing this out.

edit ii: Yes, these include the snazzy HAL callout method that means you can plug your iPod in and everything is done automatically, no manual firewire hacking required :)

Changing the Zen-Cart 'Sales Pitch'

2007-11-06T00:23:00.000+11:00

If you use Zen-Cart or OSCommerce, you may have found the need to change the 'Tagline Here' text on the main page at one point or another.

The usual solution is to copy includes/languages/english.php to your template directory and then change the tagline text there -- an adequate solution, if you only work with one template, but what if you run multiple stores, or you are dealing with a user who has no access to the filesystem, or an FTP program, or is simply uncomfortable editing something across a network.

The solultion I came up with, allows you to edit this field directly from the Administration Panel -- first edit the includes/languages/english/header.php file and change:

--- define('HEADER_SALES_TEXT', 'TagLine Here');
+++ define('HEADER_SALES_TEXT', STORE_TAGLINE);

Next, open your Administration Panel and navigate to Tools -> Install SQL Patches and in the cut-and-paste box, add the following:

--- cut here ---

INSERT INTO configuration (configuration_title, configuration_key,
configuration_value, configuration_description,
configuration_group_id, sort_order, date_added) VALUES ('Store
Advertising - Tagline', 'STORE_TAGLINE', 'Tagline Here', 'Set Shop Tagline /
Slogan
Default = Tagline Here', '1', '4', now());

--- cut here ---

Now you can go to Configuration -> My Store and configure the tagline
that is used for your shop, much easier than adding it by hand and much easier than editing your languages file to it.

dos2unix and recursion using bash

2007-11-03T12:05:00.000+11:00

More a 'don't ever forget this line again' command for me, but a handy one-liner for a box that has find and dos2unix -- but not a perl interpreter.

find . -type f $ -iname '*.css' -o -iname '*.html' $ \! -path "*images*" -exec dos2unix {} \;

Finds all files with a .css or .html extension, starting from the current working directory.
Ignores anything in a directory including the word images.
Then executes dos2unix on anything that matches.

Of course, you could execute anything (just change dos2unix to whatever you need to run).

Social Networking

2007-10-30T01:16:00.000+11:00

Some of you may have noticed the icon above appearing next to posts -- this is the 'socialiser' icon, which is like a meta-index of all the social networking sites that you could submit articles to.

It seems easier than trying to add icons for each site (furl, digg, stumbleupon, slashdot, etc) -- just click the one link to submit the particular post you are interested in to whatever sites you have accounts on.

So, if you're into that sort of thing, click away :)

VMWare 6.x with Ubuntu 7.10

2007-10-21T02:59:00.000+10:00

I've just spent a good hour wrestling with VMWare 6.0 in my new Ubuntu 7.10 installation -- there would seem to be an issue with the vmware-any-any-update113, that thinks it's building the vmmon module for VMWare 2.x, not 6.x.

After installing the update (see: http://platan.vc.cvut.cz/ftp/pub/vmware/vmware-any-any-update113.tar.gz) and running the script, everything seems to build successfully -- but attempting to run VMWare gives you:

and:

Eventually, I was able to work around this issue by:

Extracting the vmmon.tar file from the update.

Editing the getversion.pl script with a text editor and changing line 66:

--- print 'VME_V4';

+++ print 'VME_V6';

Save the file and tar up the directory.
Re-run the runme.pl file -- which should rebuild the kernel modules correctly and allow you to start VMWare correctly.

Apache2, SSL and Ubuntu 6.06

2007-10-08T12:42:00.000+10:00

Recently, I had to configure a dedicated SSL certificate with Apache2 on a Virtual Host -- and proceeded to use a combination of information from Verisign & the Ubuntu Documentation Project.

After tweaking the install, I ran foul of the infamous "error -12263" in my browser.

My configuration consisted of:

* cp'ing the original vhost configuration to 'secure' in /etc/apache2/sites-available/ and symlinking them to /etc/apache2/sites-enabled/

* adding "Listen 443" followed by a newline to /etc/apache2/ports.conf

* Changing /etc/apache2/sites-available/secure to read:

--- <virtualhost>*:80</virtualhost>
+++ <virtualhost>*:443</virtualhost>

* and beneath the 'ServerAdmin' line -- Adding:

+++ SSLCertificateFile /etc/ssl/certs/server.crt
+++ SSLCertificateKeyFile /etc/ssl/private/server.key
+++ SSLCACertificateFile /etc/ssl/certs/Verisign_Class_3_Public_Primary_Certification_Authority.pem

At this point, Apache starts normally & listens on both the HTTP and HTTPS ports, as referenced by the change in /var/log/apache2/error_log

from:

[notice] Apache/2.0.55 (Ubuntu) PHP/5.1.2 configured -- resuming normal operations

to:

[notice] Apache/2.0.55 (Ubuntu) PHP/5.1.2 mod_ssl/2.0.55 OpenSSL/0.9.8a configured -- resuming normal operations

However, when you try and https:// address, browsers error out.

Worse still, Apache doesn't actually log anything about this request by default -- but we know the site works because using a standard http:// request works fine.

Actually, that's not quite true -- it does log a cryptic error message that looks similar to:

127.0.0.1 - - [04/Aug/2007:23:38:58 +1000] "\x80\x8c\x01\x03\x01" 200 16564 "-" "-"

Some googling later, this seems to indicate that port 443 is serving non-SSL requests to clients.

Luckily, you can use OpenSSL to confirm this behaviour:

Running:

openssl s_client -connect localhost:443

Returns:

CONNECTED(00000003)
7431:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:567:

Instead of your key.

The trick, of course -- turns out to be remarkably simple, open /etc/apache2/sites-available/secure -- and add:

+++ SSLEngine on

So your configuration becomes:

---
SSLEngine on
SSLCertificateFile /etc/ssl/certs/server.crt
SSLCertificateKeyFile /etc/ssl/private/server.key
SSLCACertificateFile /etc/ssl/certs/Verisign_Class_3_Public_Primary_Certification_Authority.pem
---

Save the file and restart Apache, then run the OpenSSL command line again:

openssl s_client -connect localhost:443

It should now return your key information properly -- now fire up your browser and use https:// to your hearts content.

Ubuntu and The iPod Nano

2007-10-02T02:28:00.000+10:00

My iPod (Mini, 3rd-Generation, 4G Green) died last week -- used it every day for the four years i've owned it, so it's had a fairly good run.

So, I upgraded to a new, shiny iPod Nano.

Or so went the plan.

Long & Short -- I bought, under some sufferance (ie. in AUD -- it worked out cheaper to buy a full iPod than getting replacement parts for the old one) a 8G Nano.

After looking at Will's Backdot Page & GTKPod's subversion repository -- I set about putting packages together for anyone that's using Ubuntu's latest beta (Gutsy Gibbon) -- that should be nice drop-in replacements for your old ones.

Currently, these are hosted: here (and are based around revision 1710, for those keeping score at home) and they'll be updated for any new revisions that appear in the near future.

edit: There's also updated Rhythmbox packages from GNOME SVN now -- which fix a number of irks (cross-fading crashers, slow database read times on +1000 song libraries and they also include the patch for #411634 -- which should stop duplicated tracks appearing on your device)

edit ii: Updated to include the first pass coverart support as well as adding the ipod-read-sysinfo-extended code to automate the discovery of your iPod -- thanks teuf.

edit iii: As teuf points out in the comments, the libgpod2 package contains a small binary that allows the iPod to find and write the Extended Information file that one needs to make their iPod work -- getting this to work is a three step process:

You first need to enable (if you haven't already) the 'universe' repository via 'System -> Administration -> Software Sources'.
Plug in your iPod.
Open a terminal window.
sudo to root.
Run /usr/bin/ipod-read-sysinfo-extended /dev/sd[abcde] /media/IPOD (where sd[abcde] is the device name of your iPod -- use the df command if you're unsure)

Packages for other distributions might be considered too -- comment if you're interested.

As the technical world turns ...

2007-10-01T01:16:00.000+10:00

First came the incident. (1996-1999)
Then came cipherfunk. (2000-2006)

Now comes blogger.

A place for random technical bits, pointers to workarounds & general notifications of stuff i'm working on, for anyone that cares.

Any questions, that's what the comment function is for.