Trying Out Rygel

Trying out the new, integrated into GNOME and uses GStreamer -- Media Sharing daemon, Rygel.

First thoughts, it's a _lot_ easier to configure than Mediatomb -- I am using the "media export" plugin because my Ubuntu Jaunty installation is rather bastardised, so Tracker is "unreliable at best" on my system.

I'm using the packages from James Henstridge's PPA Repository -- because they're available for Jaunty, as well as the Vala Compiler from their PPA.

Getting it going with the PS3 was easy though, first of all -- I disabled the LPCM transcoding (for some reason, none of my MP3's appeared until I did that), by unchecking the tickbox -- set the interface to wlan0, added my media directory -- and away we went.

Still have the odd issue where Wirelessly, my PS3 doesn't see anything exported from the Ubuntu box -- but i'm figuring that's more a router issue, more than an Ubuntu one -- having recently experimented with Fedora as well.

PackageKit, The Superior update-manager Alternative

So, Dad decided to mess with his Ubuntu installation over the weekend -- he upgraded from Hardy to Jaunty as a one-step upgrade, which, somewhat unsurprisingly, broke heaps of stuff.

(personal opinion: It is for this reason, I hope the Ubuntu guys don't go head-long into GNOME 3.0 for "whatever-the-hell-the-L-release-is-called" if it's going to be a standard 6-month release cycle AND an LTS -- because they'll have people from 8.04 or 6.06 triggering ungrades and no-one wants a repeat of the PulseAudio debacle but on a far bigger scale. Either a) leave GNOME 2.28, continue polishing it and push GNOME 3.0 for the release after the LTS, so it gets three releases (or 2 years) of polish before it goes to the masses, or b) make the release cycle 6 months longer for an LTS, just like 6.06 was, but for the right reasons.)

Anyway, after re-installing Jaunty from scratch, fixing his RAID, re-installing his printer and playing with DKMS because his nVidia card doesn't like the newest driver in Jaunty, but will happily work with anything older -- we tripped over the 'incredible, vanishing update manager issue'.

(personal opinion II: big, big cock-up on the part of the Ubuntu guys -- Windows has an update icon, MacOSX has an update icon, every other distribution of UNIX has an update icon, but a 50+ year old man can have a taskbar full of update-manager windows, which makes the one browser window icon in the window list appear as 3-pixels wide because he doesn't understand he has to close them all individually? It would have been better to pop up a dialogue in the middle of the screen that says "Security Updates Are Available for Your Computer, would you like to install them now?" (or "Non-Essential Updates" in the case of non-security updates), or leave the icon where it was and hack in OSD notifications (like NetworkManager does) -- but don't just leave a glowing taskbar window in the bottom left hand corner, UI or Accessibility designers would turn in their graves.)

Anyway ... Given the track record that Ubuntu has for making seat-of-the-pant changes like that and taking forever to admit it was a mistake and reverting it to sensible behaviour, I decided to give PackageKit a try.

Short story -- impressive.

Installation was a simple: apt-get -f install packagekit packagekit-gnome gstreamer0.10-packagekit

The Ubuntu packages in Jaunty do add a duplicate "Update Manager" icon to the System ⇢ Administration menu, but that's purely cosmetic -- upon install it does make itself the default package update manager, which is a step in the right direction.

I have always been concerned about the way updates are presented for non-technical users, Microsoft nearly has it right -- too much information scares the non-technical user, but most UNIX variations give too much superfluous information to the user, great for technical types who actually fix issues -- and certainly something that shouldn't be removed, just 'shielded' from the user somehow).

PackageKit does this well, it gives the user notification that updates are pending in the standard way (which was our major gripe that prompted the change) but is also laid out in such a way that Dad found easier to drive than the traditional package manager (as he put it, "I already like the idea you can press a button to review changes before they happen.").

As I may of mentioned before in this blog, my dad is not a unintelligent man, he's been involved with computers, electronics, data communications, radio and just about any other form of technology you'd care to mention forever -- we've worked together on some massive projects, including hardware design, starting our own UNIX distribution and starting up smaller technology projects together -- and he has been a Linux user for nearly 5 years himself, but he's firmly a "a computer is a tool, just make the thing work" man, rather than a "let's experiment" one.

(He also builds RC Boats for a hobby, which is just cool.)

"Network Error 00000000", The PS3 and Mediatomb

After requiring a fresh install of Ubuntu on my media-serving laptop due to disc failure -- I restored all the configuration settings from a backup, re-imported my library into Mediatomb, restarted the daemon and wandered into the lounge to watch TV via the PS3, a ritual with the two evening coffees before the cricket starts.

However, last night -- something strange kept happening, I could browse the videos, they even had thumbnails, but when I attempted to play them, a black screen would appear with "Network Error (000000000)".

Checked the configuration, checked the router settings, checked the wireless signal strength, all fine. (then figured, "paul, you'd not be able to see anything if these settings didn't work ;)")

Googled, no luck there either -- hence this post.

Enabled debugging mode by editing the /etc/default/mediatomb file and changing OPTIONS="" to OPTIONS="-D" and restarted the daemon and loaded up the /var/log/mediatomb.log file to find:

2009-09-17 23:34:00 DEBUG: [../src/io_handler_buffer_helper.cc:224] staticThreadProc(): starting buffer thread... thread: -1267590256
2009-09-17 23:34:02 DEBUG: [../src/process_io_handler.cc:234] read(): process exited with status 256
2009-09-17 23:34:02 DEBUG: [../src/io_handler_buffer_helper.cc:227] staticThreadProc(): buffer thread shut down. thread: -1267590256

Hmm, status 256 you say? "Could be something to do with the encoding then?" I thought.

Running ffmpeg -i testvideo.wmv -target dvd -y testvideo-ps3.avi from the command line, indeed gave me my answer:

Unknown encoder 'mpeg2video'

Turns out, the default ffmpeg installation for Ubuntu Jaunty and Karmic have mpeg2video support in ffmpeg disabled.

Two ways to fix this, the first is to implement a much, much, much longer command to encode your videos -- I use a generic script to encode videos based on their content type, but the PS3 can play (from Firmware 2.61, 3.00 and 3.01 anyway) anything encoded with the DVD profile (which encodes to MPEG2 based AVI's via the aforementioned mpeg2video codec), so instead of:

ffmpeg -i "$input" -target dvd -y "$output"

You'd use:

ffmpeg -i "$input" -vcodec mpeg1video -mbd rd -trellis 2 -cmp 2 -subcmp 2 -b 1500 -g 250 -y "$output"

(note: those commands are the only ones that are essential for making the video output work on the PS3, it'll produce a standards-compliant MPEG file with 64k MP2 audio -- but the quality may look suspect, as long as you leave those commands in, you can add things like "-flags qprd", "-bf 2" or "-flags +ilme+ildct" (good for interlaced video like animation) to improve the quality slighly)

Or, you can install the alternative versions of FFMPEG by enabling the multiverse repository (System ⇢ Administration ⇢ Software Sources) and then installing the -unstripped versions of FFMPEG with the command:

sudo apt-get -f install libavcodec-unstripped libavformat-unstripped

... and continue using the transcoding scripts you're using now :)

How can I tell if I have a broken FFMPEG installation?

The simplest way is to run: ffmpeg -formats | grep mpeg2video from the command line, if you receive:

D VSDT mpeg2video

Then you're missing the 'E'ncoding aspect of the codec, meaning you need to install the unstripped packages to make everything work correctly.

If it returns:

DEVSDT mpeg2video

Then you're all set to go :)

Getting iPod's Working With Banshee In Ubuntu 8.04.3

Recently, I had need to install a newer Banshee from the Ubuntu PPA on a friend's Ubuntu LTS machine (I only ever recommend LTSes to friends converting from Windows and thus not used to the 6-month upgrade dance).

He plugged in his iPod, after it had been restored with iTunes from his wife's PC and -- no, nothing happens.

Unplugged, Replugged -- yep, appears on the desktop as an iPod and Banshee starts correctly, the iPod just doesn't appear in it.

Checked dmesg, yes, the device is a 8GB iPod, no it does not need fsck'ing and yes, it uses the correct mountpoint.

Checked Synaptic, all the Banshee code is there.

Checked the Bugtracker, found bug #139226.

So, we ran:

sudo apt-get install podsleuth

Unplugged, Re-plugged the iPod. Success.

Question is, if podsleuth is that important to getting DAP's working in Banshee, shouldn't it be at a dependency of the install?

Fixing Ubuntu PPA (Personal Package Archives) Signing Key Issues

My default /etc/apt/sources.list.d/ has around 10-12 PPA listings in it all the time and with the release of 9.04, there doesn't seem to be an automatic way of ignoring authentication for packages on upgrades, so a number of upgrades choke because of unsatisfied authentication issues.

This thread was a lifesaver, download the script, install the dependencies, run it, then re-run sudo apt-get -f update ... problem solved!

Why No Linux Installation has an 'I Only Speak One Language' Option?

Now that Fedora 11 has come out, I decided to give it a spin in a VMWare box -- i'm not entirely satisfied with the drive read speed of Ubuntu Jaunty, nor the compiled in IPv6 option making Transmission crawl into the floor in terms of connection speed with no true way to disable it, I am unsure if users will the new 'update notification' (it's almost like Windows XP in that regard, just without the forced rebooting when it's finished) and I can certainly take or leave the 'new' notification system, which makes VMWare go nuts with screen flicker while the notification is on-screen.

Needless to say, that while I thought Ubuntu Hardy was brilliant for both technical professionals and new users alike on the Desktop -- having used Jaunty for a month, I almost feel like i've installed a totally different system that feels totally foreign -- and one that has been dumbed-down to the least-common-demoninator level, an unfortunate side-effect of 'visionary progression' within parts of the Linux on the Desktop supporters.

Coupled with the fact i'd like to play with RPM building again and thought PackageKit sounded remarkably cool -- I thought this would be a good time to try Fedora (my last experience with it on the desktop was 4, other than building libgpod for 9 and 10) -- I only upgraded from Ubuntu 8.04 to 9.04 to try GNOME 2.26, which Fedora 11 has included.

Now, i've never, ever, ever accepted the defaults for a Linux installation -- so I selected the 'Customise Now' button to remove the packages I didn't need at install-time (F-Spot, Java, etc).

Fedora, like RHEL still has a very nice package selector, with an 'Optional' button that allows you to select with tick-boxes which packages the user would prefer.

So, after a while -- I found the Base section.

The Package Selection Dialogue

28 of 80 Fonts?

So I click the "Optional Packages" button, to find:

The 'Optional Packages - Fonts' Dialogue

Now, in my personal case -- as far as foreign languages go, I speak semi-minimal German. As a native English speaker, everything I do with a computer is in English, aside from seeing what the other fonts look like, all they really do for me is take up room and slow OpenOffice down when it loads.

I de-selected all the optional fonts with the exception of the DejaVu fonts and continued on with the install as normal.

I personally think that the internationalisation of Linux is incredible and a wonderful thing to watch, but installing over 100 megabytes of fonts that I will never use, never see and couldn't really understand anyway -- seems like 100 megabytes of space that could be better used elsewhere.

To be fair, On the Red Hat / Fedora side, this problem bugged me more back in the pre-Red Hat 9 days, where ISOs were shipped on a 650M CDROM and every piece of space had to be meticulously accounted for -- they've worked around the issue by using multiple CDs to do the install, as well as supplying an 'Everything on DVD' release.

Ubuntu, on the other hand -- has always had this problem, Epiphany (the GNOME based Web Browser Front-end) needs to be installed post-install, but South African English, several Thai and Indian dialects as well as Japanese fonts are installed from the core 700 megabyte ISO.

It just seems as though the installer needs a box that asks 'Is that the only language you speak?' ... and only if you select 'No' does it select the internationalisation components of whatever distribution you're running.

The one question it raises is how much of all the languages you should include by default -- since the advent of network-based installs, it might work if the various installer code provided the strings for 'Do you Speak ...' 'Is your Network Set Up (followed closely by 'How would you like your network set up' if you have a Wireless configuration) and then sent off to mother-goose to install the relevant packages based on the locale configuration to that point.

Trials with RecordMyDesktop

I've been playing with recordmydesktop lately in order to put some suitable screencasts together for demonstrating various Linux-related activities to university students next week.

Unfortunately, both Fedora 11 and Ubuntu Jaunty have issues with RecordMyDesktop -- the current version (0.3.8.1) is much better than the last version I tried (0.3.5) in terms of both video and audio quality, but the more-than-occassional

Broken pipe: Overrun occurred.

Cause audio to be clipped rather badly.

The default settings make the video speed up and therefore finish well before the audio too -- so I set about trying to find the optimium settings to make the best encode I could.

First, if you are using a 3D Window Manager (such as Compiz), please use the --on-the-fly-encoding and --full-shots options, or everything from opening new windows to redrawing your background will end up with corruption.

If you do not know which Window Manager you are running, you can use the Visual Effects tab of the Appearence Preferences window.



If the button is in the first box (None) then you are not running a 3D Window Manager, if it is in either the Normal or Extra boxes, you are.

Now we've figured out if we need the options for 3D Window Management, it's time to move on to the audio -- in it's default configuration, the program will clip and/or drop audio because of the overrun mentioned above, but you can reduce that by a fair proportion by either:

1. Using pasuspender to launch the program. This particular solution was suggested by an Ubuntu developer on a reported bug about RecordMyDesktop that looks very similar to the particular issue we're covering here.


2. Configuring an .asoundrc file to use PulseAudio for all ALSA based audio by default.

The latter is done by creating a new blank file in your home directory and adding the following code:

pcm.!default {
type pulse
}

ctl.!default {
type pulse
}

Save the file, exit, log out of your session and login again.

This alone reduced the dropped audio from 7 occassions in one minute's worth of recorded footage to 3.

The former is done by adding: pasuspender -- to the beginning of the command line.

On my laptops, using pasuspender made no real difference to the problem, which is why we're preferring the .asoundrc solution for the remainder of this discussion.

Another thing that helps lower the frequency of audio dropouts is to increase the buffer size used by RecordMyDesktop -- the defaults are 4k, raising that to 16k helps dramatically, but any higher doesn't seem to offer that much of an improvement (I tried 32k (32769) and 64k (65538) when testing -- i'm only recording two channels on an internal microphone though, your milage may vary)

You can do this by adding the --buffer-size 16384 option to the command line.

update (2009-05-17): At the highest quality audio and video settings, 16384 can still cause dropouts, so 65538 is now preferred.

Next, if you're using onboard audio (like Altec Lansing laptop audio) you may see:

Playback frequency 22050Hz is not available...
Using 44100Hz instead.
Recording on device hw:0,0 is set to:
2 channels at 44100Hz

The Altec Lansing Audio, as well as several other Intel based sound cards that use the hda-intel-audio driver, will resample 44100Hz input to 48000Hz, hence the error.

To fix that, you pass --freq 48000, which allows recording of audio without any extra resampling.

Now, if you've used the pasuspender solution above, or your audio seems to be correctly synced, you can move on to the next section.

If, on the other hand -- you still see Buffer Overrun errors, or your audio is out of sync, or you use my .asoundrc solution above, you'll need to pass an additional audio-related option on the command line:

-device plughw:0,0

Using plughw, allows ALSA and PulseAudio's internals to handle all of the resampling work -- or endian-conversion work automatically, which reduces a painfully large ALSA configuration that will differ on every soundcard on the market, to a single configuration change for our purposes.

The 0,0 (aka. use the first detected virtual sound device) should be fine for almost everyone working on a standard configuration -- if you've got:

• More than one Soundcard.


• Have used the Systemwide PulseAudio On Ubuntu HOWTO to add a software equalizer to your configuration.

You may have more than one plughw device to choose from -- choosing which one is beyond the scope of this article though.

Now we can move onto improving the video quality.

Next, it helps to set the framerate -- i've seen numerous articles on the web about this, everyone suggesting everything from 10fps to 90fps to get a clear picture.

I tried 10, 12, 20, 25, 29, 50 and 60 (these seem to be the ones that are mentioned the most, in a four page Google search on the subject) -- 10 works if you want to keep the filesize low, but the playback video seems too jerky -- 60 doesn't increase the filesize, but does have an unwanted side-effect of the video finishing well before the audio when you play it back.

The optimum setting I found was --fps 20.

Finally, because I wanted to re-encode these for different purposes, I like to do the initial encoding with the highest quality possible.

However, the difference between -v_quality 1 and -v_quality 63 was minimal at best on my 1280x800 resolution.

Somewhat obviously though, the -v_bitrate option makes A LOT of difference, so I bumped that to the highest number available (2000000).

The recommended command line(s)

I've provided both a high and low quality setting for those people who are interested

The final command line, that works for me and provides the highest quality, lowest distortion with no dropouts -- was:

recordmydesktop --on-the-fly-encoding -v_quality 63 -v_bitrate 2000000 -s_quality 10 --full-shots --fps 20 --freq 48000 --buffer-size 65538 -device plughw:0,0

For a smaller filesize or for encoding on older or busier machines, with quality suitable for downloading or streaming, you can use:

recordmydesktop --quick-subsampling --on-the-fly-encoding -v_quality 10 -v_bitrate 50000 -s_quality 1 --full-shots --fps 10 --freq 48000 --buffer-size 16384 -device plughw:0,0

This option does the encoding after the video capture has stopped, which results in a smaller filesize at the expense of taking longer to encode your original footage, you can offset this somewhat by adding the --quick-subsampling option to your command line, which saves CPU time by discarding extra pixels when calculating the chroma values during the encoding process.

note: If you are using a 2D Window Manager (such as Metacity), you may omit the --full-shots option, which halves the filesize on my Ubuntu Jaunty Jackalope install.

Why No Examples?

Google doesn't support Theora based videos to be uploaded to Blogger, I had two videos recorded of 10 seconds each, the low quality one (using the settings above) clocked in at 220 kilobytes, the high quality one was 1.3 megabytes.

When El Goog does decide to allow Theora based videos on Blogger, i'll post them -- but converting them to .mp4's just to show you all what they looked like, seemed rather purpose defeating :)