Archive for the 'fedora' Category

Archaeology

Published on July 11, 2010 in adrian and fedora. 1 Comment Tags: .

If I remember it correctly my server at home (file-server, print-server, router, …) has been installed a long time ago using Red Hat Linux 8.0. Since the initial installation I have done live upgrades using rpm, apt-get or yum to its current version (Fedora 11). Now I just started doing a live upgrade using yum to Fedora 13 and I got an interesting dependency problem:

--> Finished Dependency Resolution
lilo-21.4.4-26.i386 from installed has depsolving problems
 --> Missing Dependency: mkinitrd >= 3.4.7 is needed by package lilo-21.4.4-26.i386 (installed)

It seems I still have an unused version of lilo installed on my system and now that mkinitrd has been replaced yum starts complaining. The lilo package is from 2004 and has also been installed in 2004 (according to the RPM database). It is the oldest package on my system but now it has to go.

Update To Fedora 12

Published on January 11, 2010 in fedora and mirror server. 0 Comments

Last week I have finally updated our mirror server to Fedora 12. It was still running Fedora 10 which has reached its end of life. The server was running Fedora 10 for a long time and it was always running with a CentOS kernel. The Fedora kernels were, at the beginning, not stable enough (crashing after three or four days) so that I quickly switched to a CentOS kernel. I know that I should have reported bugs, but in the case of the mirror server I am more concerned to keep it up and running than getting debug data from it. It also not easy for me to get physically to the machine so that I had a lot of good excuses to switch to a CentOS kernel.

Now the system is running using the Fedora 12 kernel and after a week it is still up without any problems.

RPM Fusion Mirrorlist Server

Published on January 8, 2010 in fedora and mirror server. 1 Comment Tags: , , .

On the last day of the last year (2009-12-31) both RPM Fusion’s mirrorlist server were most of the time not reachable. The problem started at 00:53 (UTC) and it was at least going on until 16:00 (UTC). Both mirrorlist servers have been on the same network and the router for that network  broke down. If it would have been the link to our provider the router had a backup route to stay on-line, but this time it actually hit the single point of failure – and everything was off-line. See: error report of the provider (german).

I was never happy that both mirrorlist server were running in the same network and I especially wanted to get the mirrorlist server off my mirror server. Thanks to Patrick I have now access to another VM at a different provider where I am running a new mirrorlist server instance. It does not require much in terms of resources and bandwidth, but having root access makes everything so much easier.

RPM Fusion’s mirrorlist server are now two dedicated VMs at two different providers and that should protect the functionality from failures like the one on 2009-12-31.

Leonidas Traffic (Part 2)

Published on June 24, 2009 in fedora and mirror server. 0 Comments Tags: .

In one of my last posts I wrote about how much Fedora related traffic we had on our mirror server during the Fedora 11 release. I got one huge comment from Jef with three questions which I am trying to answer now.

1) Assuming you could find an accurate count of EU mirrors on F10 release. Can you use the ratio of available mirrors then to available mirrors now to re-scale the activity…sort of like a mirror inflation correction to scale activity in terms of available bandwidth.

As the mirrorlist is very dynamic I do not think I can answer that.  But if somebody has some useful numbers concerning the number of EU mirrors during the F10 release as well as during the F11 release it can probably be done.

2) Can you trend the “shape” of the first week of F-11 compared to the first week “shape” of F-10 activity on your mirror?  Forget about absolute scales. Normalize each to the maximum associated with the first 24 hours of activity and see how the activity trends in time relative to that normalization. Does F-10 for example see the same second day uptick relative to the first day that you see in F-11?

That should be possible. After some gnuplot-ing I have the following diagrams:

Downloaded Data (Normalized)

This shows the data transferred for each release (normalized to the first day). It is important, however, to know that the first day of a Fedora release is not 24 hours, but only 8 hours on our mirror server as the release usually happens at 16:00 local time. Therefore I also made another diagram using the absolute numbers:

Downloaded Data (Absolute)

It can be seen that basically only the first day differs for some reason. The following days were pretty much the same, just a bit less traffic than during the Fedora 10 release. So maybe this difference is related to my assumption that there are more mirrors in Europe. Although the amount of a traffic does not differ so much the mirror server has been in a much better state during the whole release. The load used to be much higher and the http server had no free connection slots available. This time the load was not really high and after the first day it was always possible to make a http connection (although it took longer than usual).

3)I’d be really interested to know if you could identify any upticks related to F-10 downloads further away from F-10 release that correlate with ambassador activity at an EU event.

No idea, but I have the amount of data downloaded per day for each mirrored project for at least the last year available here. So if there are certain dates it can be looked up.

For my own reference these are the gnuplot commands used to create the two diagrams:

gnuplot> set terminal png size 400,300
gnuplot> set output "absolute.png"
gnuplot> set xlabel 'Days Since Release'
gnuplot> set ylabel 'Terrabytes'
gnuplot> plot 'F10' smooth csplines title 'Fedora 10', 'F11' smooth csplines title 'Fedora 11'
gnuplot> set output "normalize.png"
gnuplot> a=2.23
gnuplot> b=1.54
gnuplot> set ylabel 'Downloaded Data (Normalized)'
gnuplot> plot 'F10' using 1:($2/a) smooth csplines title 'Fedora 10', \
>'F11' using 1:($2/b) smooth csplines title 'Fedora 11'

Leonidas Traffic

Published on June 16, 2009 in fedora and mirror server. 1 Comment Tags: .

The Fedora 11 (Leonidas) release has been the Fedora release with the least pressure on our mirror server. This is probably due to the fact that there are more mirror servers in Europe than ever. In addition to the usual http/ftp/rsync traffic I had bittorrent running for the first time, but the bittorrent client was never using more than 50 MBit/s of the bandwidth (and that also dropped after the second day). Compared to the normal mirror traffic that is not really much. After running for about a week the bittorrent client has uploaded around 600 GB.

The Fedora traffic during the first few days after the release was of course higher than the average ~450 GB/day:

  • 2009-06-09: 1.54 TB
  • 2009-06-10: 2.88 TB
  • 2009-06-11: 1.62 TB
  • 2009-06-12: 1.01 TB

On the second day of the release almost 70% of the mirror traffic was Fedora related:

2009-06-10

And on the bandwidth graph it can also be seen when the bitflip was (around 15:00 local time) and when the release actually went live (16:00 local time):

2009-06-10

I do not know why the traffic dropped so significantly at around 00:00, but probably our mirror was dropped from the mirror list, because the crawler (from MirrorManager) was no longer able to connect and verify that our mirror was up to date.

Following Fedora 11

Published on June 10, 2009 in fedora and mirror server. 0 Comments

Syncing Fedora 11 to my mirror went pretty well. With hardlinks it took not much longer than 24 hours to transfer all the data.

On Monday rawhide was updated again and my mirror scripts started. This time it took more than 55 hours for the first run and it just finished a few minutes ago. I re-started the mirror script immediately after it finished because in the 55 hours it was running rawhide had already changed again. So it looks like to get my rawhide mirror in sync with the master server it will take more than three days. The slowness is probably not only caused by the large number of changes but also by the fact that it is now also necessary to transmit all the DeltaRPMS. Somehow it should be possible to use the DeltaRPMS for mirror syncing. Another reason might be that my mirror server has been pushing Fedora 11 data out at about 1Gbit/s, which probably does not help to speed things up.

Wireless Fun

Published on April 22, 2009 in Holidays and fedora. 1 Comment Tags: , .

Last week we were for 5 days on holidays on Sylt. In addition to the mandatory holiday pictures I had fun trying to get connected to the wireless network in the hotel.

The hotel had an unencrypted wireless network for its guests called Gastenetz. Unfortunately it only worked in the lobby and our room was a bit too far away. With iwlist wlan0 scan I was sometimes able to see the network, but I never managed to connect to it. There were three other unencrypted wireless networks available in our room (I do not know where they belonged to) but although I was able to connect to them, they seemed to have no connection to any other network.

At one time I was connected to one of the other networks (thanks to NetworkManager) and all of sudden my jabber client was online. After doing some research I found out that one of the other networks (Haus10) was using the same channel (11) as the hotel’s network Gastenetz and if I hardcoded the ARP entry of the gateway with arp -s 192.168.2.254 00:1x:4x:8x:6x:f4 to the hardware address of the hotel’s router it worked all time.  It seems that the access point close to me extended somehow the network to the access point of the hotel I wanted actually connect to.

Cambridge Mirror Fun

Published on November 26, 2008 in fedora and mirror server. 0 Comments

New Fedora releases are always great because they are one of the few times in the year at which our mirror server really has something to do. And I really like to watch all our little statistics. So instead of the usual 2-3TB we are pushing out each day we pushed almost 5TB on the release day and the release happened not before 1600. So I expect that that number will be even higher tomorrow.

The bandwidth graph generated by munin:

Cambridge Release

And the mirror traffic breakdown from the release day:

Mirror Traffic Breakdown 2008-11-25

During the eight hours at which Fedora 10 was available from our mirror server on the release day we transmitted 2.23TB Fedora related data.

MirrorManager In RPM Fusion

Published on November 20, 2008 in fedora. 2 Comments Tags: , .

RPM Fusion now also has a MirrorManager instance running. I have written some minimal documentation about RPM Fusion’s MirrorManager setup but it is pretty much the same as already documented for Fedora. It took me a few days to understand how all the different parts of the MirrorManager are working together, but I think I have now a pretty good overview how it all works. Matt has really written a nice piece of software and he accepted all my patches. My patches were mostly getting rid of hardcoded Fedora URLs and putting Fedora specific stuff in configuration files.

The public mirror list can be accessed here: http://mirrors.rpmfusion.org/mm/publiclist/

The mirrorlist which is generated for yum can be accessed here: http://mirrors.rpmfusion.org/mirrorlist?…

To provide at least some kind of “high” availability mirrors.rpmfusion.org consists of two hosts. One is run by me and the other is provided by Oliver.

The MirrorManager database web interface is running at https://lisas.de/mm/. The reason for not running it under the rpmfusion.org domain was that I have signed SSL certificate for lisas.de and because it transmits a user name and password I wanted it to run over an encrypted connection. But as only mirror admins have to use that interface it should cause not too much confusion for the users of RPM Fusion.

I am using the information from the mirrorlist accesses to generate some statistics about the countries where RPM Fusion is used as well as statistics about the usage of the repositories and architectures: http://mirrors.rpmfusion.org/statistics/

Internet Explorer Is Important

Published on August 16, 2008 in fedora. 14 Comments Tags: .

A few days ago I found out that Internet Explorer is important. If you stop using it (either because you cannot use it like me (no windows) or because you have installed a better browser) you easily forget how broken it is. I was writing my album script and checking it if was valid XHTML 1.1. There was still a warning that the mime type was not application/xhtml+xml. So I fixed it and was happy that it was looking like I wanted it to and it validated without any warnings.

A few days later I wanted to show my pictures to someone using Internet Explorer and found out that instead of displaying my valid XHTML 1.1 page it complained about an unknown file type and wanted to download it as a binary. After googling a bit around I found a few articles describing just this problem and it looks like it will not be fixed, even in the next version. Some workarounds are to rewrite the mime type with something like this:

RewriteCond %{HTTP_USER_AGENT} .*MSIE.*
RewriteCond %{REQUEST_URI} \.xhtml$
RewriteRule ^/~adrian/2008/.* - [T=text/html]

I really dislike this solution and I will probably go back to serve XHTML 1.1 with the wrong mime type.

So, it seems it is important to use Internet Explorer once in a while to not forget how lucky you are using Linux and some browser which actually works.