Category: Ubuntu

RAID Tuning

/ alex / 2 Comments

In order to get maximum performance with the newly setup RAID, I added some udev rules (by placing them in /etc/udev/rules.d/83-md-tune.rules) to increase caching. The file has one entry for each of the involved disks (sdX) to adjust the read-ahead:

ACTION=="add", KERNEL=="sdX", ATTR{bdi/read_ahead_kb}="6144"

And one for the mdX device to adjust the read-ahead as well as the size of the stripe cache:

ACTION=="add", KERNEL=="mdX", ATTR{bdi/read_ahead_kb}="24576", ATTR{md/stripe_cache_size}="8192"

With these settings dd yields the following results when copying a large file:

$ sync; echo 3 > /proc/sys/vm/drop_caches
$ dd if=largefile of=/dev/null bs=16M
20733648232 bytes (21 GB) copied, 60.4592 s, 343 MB/s

Which is nice – and rather pointless as the clients connect with 1G links so they see only one third of that performance at best… Note that the caches will cost extra kernel memory, so if you’re low on RAM you might want to opt for lower cache sizes instead.

Update: I forgot to mention that I also switched from the deadline (which is the default for current Ubuntu systems when installed as servers) to the cfq I/O scheduler as the test results from this article suggest that it is the optimal scheduler for RAID Level 5 no matter whether it is HW or SW controlled.

That time of the decade again – upgrading the RAID

/ alex / Leave a comment

With my Linux SW RAID screaming “grow me!” for quite a while now, I finally brought myself to replace the old 2TB disks with new 6TB ones (RAID 5 with 4 disks). While such a disk-upgrade has to be performed regularly, the frequency is so low that it is hard to remember the details when you finally get to do it again. Unfortunately the “official” method (replace & resync disk-by-disk and then grow the md and the filesystem) as suggested in the Linux RAID Wiki has a few drawbacks:

  • you have no backup in case of failures during the 4 RAID rebuilds
  • you continue to operate on the old filesystem, in my case where the RAID has been full for quite a while you will inherit quite a bit of unnecessary fragmentation – and you cannot switch nor re-tune the filesystem which could make sense for a significantly bigger RAID

Luckily Adrian reminded me of mdadm’s missing parameter, so I could perform this alternate RAID upgrade which I’ll detail below (should come in handy for my next upgrade). Continue reading →

New scratching device

/ alex / Leave a comment

Today I finally managed to release a new version of terminatorX, which had been broken for quite a while as some changes in either gtk+ or X.org broke the mouse grabbing code. Anyway, just recently I bought a new mouse as the left button of its predecessor was worn out. I decided to go for a high precision device, so I ended up with a Razer laser mouse (Lachesis).

At first I was stunned that the manufacturer actually labels the device as a “weapon of mass destruction”, but hey I use with a piece of software called “terminatorX” – so who am I to judge. Once you get used to the high resolution, the mouse really is a very precise input device – and it does help operating terminatorX quite a bit. The guys from Phoronix gave the mouse (or an earlier version of it) a test and liked it, too, however they were disappointed that Razer does not provide official Linux support. While this still holds true today, I have to say that my experience with the device under Linux has been excellent so far: you plug it in, it works perfectly an you can switch the resolution with the two buttons on top.

Some might be disappointed that there is no fancy user interface allowing you to tune and configure the device – I appreciate the fact that I don’t have to configure anything.

RAID away

/ alex / Leave a comment

During the last week I’ve replaced the disks of my software RAID with larger ones as the capacity was exceeded. While this is theoretically an easy task, I had to learn a few things along the way:

  • Trying to perform such an upgrade on a headless system without console will fail.
  • fdisk silently fails to parse integer values larger than 2147483647.
  • The md superblock is located at the end of the partition/disk that you add to the RAID.
  • If the kernel associates the complete drive to a specific md device instead of the last partition, blocking the use of other partitions for other md devices, resize the last partition to leave some (wasted) space at the end to ensure that the end of the last RAID partition differs from the end of the drive.
  • Some manufactures build ‘green’ disks that constantly unload/load their heads, causing the drive to run out of spec in a very short time. If the manufacturer provides a DOS tool to correct that behavior, a pretty easy solution is to put it onto a bootable CD.
  • This stride calculation script helps to optimize the performance of the filesystem running on a RAID5.
  • Cheap desktop drives might be a bad choice for a RAID, if they break during the first re-sync of the RAID you can try to recover your data by re-creating the RAID – Thanks, Adrian!

Floppies Revisted

/ alex / Leave a comment

Last weekend I upgraded most of my home systems to run Lucid Lynx. From the software point of view everything went pretty smoothly and I am really happy so far. I like the new look which is not surprising as I’ve been using the Dust theme prior to 10.04 and they are not very far apart. The new Ubuntu One integration is an interesting way of trying to make Ubuntu sustainable, I do hope however that it will stay out of my way if I don’t want to use it.

I was close to downloading an album through Ubuntu One until they requested me to register my computer. This is something I do not want to do just to buy an album, so I stopped right there and resorted to the wonderful clamz.

Anyway, during the setup I had to realize that CD-Rs have become the floppies of 2010 – not only capacity-wise but also regarding the reliability. I’ve been having this problem with Ubuntu as well as Fedora setups: When you burn the CD-R just before running the setup on another machine with a different optical drive you will often get read errors at some point in time – typically after being halfway through the setup process. This brings me to my request to the authors of Linux distribution setup procedures: If you cannot read a package from the CD please try downloading it from the Internet after asking the user whether it is OK to do so. I fixed one of the setups with a manual chroot onto the new root fs after modifying the sources.list, on another machine I simply used the mini iso which downloads eveything via the network.

Pulse Pounding

/ alex / 2 Comments

Yes, I knew this would happen. However, that does not diminish my frustration. Of all the desktop machines that I work with, I only use three to play audio frequently. These machines currently run Ubuntu Jaunty, Ubuntu Karmic and Fedora 11. For each setup I had the good intention of keeping PulseAudio after installation, but it failed on each installation for a separate reason:

  • On the machine running Karmic, vlc (the only player capable to properly play my AVCHD recordings) will drop frames like hell when running with PulseAudio.
  • The Jaunty machine is a rather powerful quad-core with a high-end sound card and just listening to music with totem I will actually get occasional buffer underruns (stuttering audio) when running a kernel compile.
  • On the Fedora machine I’d like to run mpd on start-up as a different user than the one logged in (who is forced to run PulseAudio) and this is not easily possible (or maybe not at all).

Even worse, it is becoming more and more painful to remove PulseAudio. You will loose ubuntu-desktop and gnome-bluetooth (also on Fedora). For Karmic I had to recompile gnome-session or else it will fully load one core trying to connect to PulseAudio. Gnome will no longer let you control the volume, neither from the panel nor via the keyboard.

So now we have shiny new features (that I never had a chance to use, because I always have to disable PulseAudio), but solid, reliable and easy sound output is history. Congratulations on breaking Linux Audio!

GNOME does

/ alex / Leave a comment

The level of maturity the GNOME desktop has achieved by now, seems to a have a negative effect on innovation. Even for experienced GNOME users it is becoming harder and harder to detect or name the changes that came with the recent GNOME releases. Whether this is a good thing (the learning curve for using a GNOME update is practically non-existent) or a bad thing (boooooring) is still the subject of numerous discussions on Planet GNOME and the GNOME mailing lists.

I am happy to see that GNOME innovation is not dead yet: I just discovered GNOME Do and I am impressed. Obviously I just started using it, so I cannot say whether it will stick, but this tool could severly influence the way I use my GNOME desktop in the future. What it does is actually hard to describe, basically it brings to GNOME what the new location bar brought to Firefox 3. I recommend trying the latest version, which is easy with Ubuntu, with Fedora however you are stuck with the 0.4.0.2 release as even in development the necessary dependencies are not available yet.

Update: Adrian let me know that there is a bug requesting Fedora to update GNOME Do to 0.5.0.1, though it doesn’t look like it’s going to be resolved quickly.

The Return Of The Daemons

/ alex / Leave a comment

There was a time when Linux distributions automatically started sound daemons for the user. Typically these were aRts for KDE and esd for Enlightenment, Gnome and others. The main goal was to allow simultaneous audio playback from multiple applications.

Unfortunately, using these daemons has side effects:

  • they introduce significant latency
  • they require applications to be ported and configured to use them
  • they block the audio device for either serious or rather dumb audio applications

Then, thanks to an tremendous development effort, a new audio layer for Linux was born that addressed the main goal of these daemons, allowing hardware mixing where available and enabling software mixing when necessary. The daemons became obsolete and were eventually removed from the default install of all major distributions. The only daemon that stayed relevant was JACK as it addressed audio production needs and most sound creation tools were extended to support that interface, too.

Now, guess what. There’s a new daemon in town. It’s better than ESD, but will still block your audio device. And from what I’ve read it is not designed to compete with JACK in the ultra-low-latency sector. A major motivation for its creation must have been a severe allergic reaction to ALSA. And Fedora and Ubuntu intend to force it upon us.

I got aware of that after upgrading my laptop to hardy, when mplayer failed to playback audio until I killed the pulseaudio daemon. Oh man, was I happy when the daemons where gone.

Yeah, I know, they promise some interesting features. I also read that PulseAudio is causing significant load under certain conditions. I’m just not sure that you really need an extra CPU hog just to play audio, unless you really dedicate your machine to audio production in which case I cannot see why we should not stick with JACK.

I really wonder whether investing all that energy into improving ALSA instead (and maybe gstreamer or JACK) might have helped to provide some of the same features without the hassle of yet another daemon with yet another API.

The Gibbon Can Bite

/ alex / Leave a comment

After having installed gusty from scratch on some boxes and via update-manager on others, I decided it’s about time to upgrade my desktop machine as that was still running feisty. Unlike my previous experiences with gutsy this upgrade didn’t turn out to be as easy as pie as it should have been. Ever since I started using debian I upgrade my sources.list and run an apt-get dist-upgrade afterwards to perform a distribution upgrade, so I chose that path for this upgrade, too. Well I shouldn’t have done so: upgrading this way does not remove the deprecated evms package, that renders a gutsy system unusable.

Additionally I had moved my installation partition prior to this upgrade, but I didn’t know that update-grub keeps track of that in commented lines in menu.lst, so it repeatedly rewrote that file with the wrong boot partition, breaking unattended booting. From previous experiments I still had the xserver-xgl package installed without actually using it. Unfortunately the upgrade activated Xgl as default X server, breaking my desktop setup.

Then I had to learn that my favorite audio player is broken in gutsy. At first I considered capitulation and gave other players a try, but they still don’t behave the way an audio player should. So I investigated the problem further and it turns out that re-building the package with the new upstream release (which is available for a month now) fixes the problem. Rather frustrated I added that information to the original bug report.

No, this not the end of the list, yet. After recompiling VDR (more on that later), I discovered that the system crashes after a few minutes of watching TV. Using the ‘generic‘ instead of the ‘rt‘ linux-image actually fixes this, but I had no problems with the previous low-latency images. So, I filed another bug.

Oh, and while writing this entry I went out for while, just to find my mouse cursor to be dysfunctional on my return. Hopefully the bird in approach will be a little more gentle.