This month’s come together will be held on 2009-04-01, 8pm at our regular pub (Trödler).

This month’s come together will be held on 2009-03-04, 8pm at our regular pub (Trödler).

$ date +%s
1234567890

This month’s come together will be held on 2009-02-04, 8pm at our regular pub (Trödler).

The PHP scripts behind our information service are now available through: git clone http://ftp-stud.hs-esslingen.de/info/git/info/

This month’s come together will be held on 2009-01-07, 8pm at our regular pub (Trödler).

Jan 1 00:59:59 rhlx01 kernel: Clock: inserting leap second 23:59:60 UTC

In the last four weeks the data has been moved to a new, even bigger RAID. The amount of RAM has also been increased. The system is now running with 40GB of RAM and over 10TB of diskspace.

Added new mirror: ELDK.

Started to mirror fedora-secondary.

Short unannounced downtime to boot a new kernel.

Fedora. Nobody expected anything else (of course).

The first one and a half days since the release of Fedora 9 we are maxing out our bandwidth again. Today we already pushed more than 5.5TB and it looks like we will get close to transmitting 7TB on one day. This is much more than during the last Ubuntu release.

With the help of munin I can again provide a nice bandwidth graph:

The small dent, just after the start of the release, is due to the fact that I had to restart apache because of our cache drive. We are using a fast hard disk to reduce the load on our main RAID as cache, but it seems that it somehow cannot handle over a thousand simultaneous accesses and that is why I disabled that cache drive (which should have improved the situation and not worsened it). I can also prove that the Fedora release is the reason for all the traffic:

Traffic Breakdown 2008-05-14

Traffic Breakdown 2008-05-13

We always thought our mirror server is connected with 2 GBit/s (two times an e1000 card using bonding mode=6), but the current Ubuntu release proved that somewhere along the way to the Internet there must still be something that limits us to 1 GBit/s. The following diagram shows this pretty clearly:

Now we only need to find out where and if it is something that we can fix or if we need help from our provider.

Maybe we can fix it before the release of Fedora 9 so that we finally can transmit more than 1 GBit/s.

Downtime for a few hours to install a fast (15K) 300GB SAS drive. This disk is used to act as a disk cache for apache. The goal is to reduce the number of simultaneous accesses on our main RAID. We also changed the bonding mode to: mode=6 (balance-alb)

Stopped the fedora and fedora.extras mirror scripts. These scripts were pointing to rsync modules which were removed on the master server because they were only necessary up to Fedora 6 which is already EOL since a few months.

Today was for a few hours one of the RAIDs offline to do the following:

tune2fs /dev/sdb1 -O dir_index

e2fsck -C 0 -f -D /dev/sdb1

This should, if correctly informed, speed up lookups in large directories.

Rebooted for a new kernel to fix the current available local root exploit.

New mirror of http://www.debian-multimedia.org/ at http://ftp-stud.hs-esslingen.de/pub/Mirrors/debian-multimedia/.

All volumes are mounted again. Back at normal operation.

We unmounted /ftp/pub/.2 again because of problems with the backend. This means that following mirrors are currently not available: ccux-linux.de, ftp.kde.org, ftp.rootlinux.org, ftp.xfree86.org, ftp.ximian.com, Mandrivalinux and wikipedia. Estimates are that it will be fixed sometime tomorrow.