I've been meaning to scribble something about paludis for a while now. I was tempted to do that right after I started using it, but then I thought it would be better to get some perspective and that would also cover issues that may come up some months into actual use.

So the moment has come. I installed paludis sometime in mid-November 2007. Long before that it was announced portage compatible, so it should be safe enough. Migration is neither all that long nor that complicated, it's mostly a matter of getting used to paludis's different philosophy. One of the odd things is setting up repositories (overlays) in /etc/paludis/repositories, but it's easy enough.

From a portage user's perspective configuring paludis is not the most pleasant experience. The documentation is quite complete, but it really demands that you know exactly what you want. There aren't any texts to read, it's generally just a FAQ. As far as user guides go it's not the most friendly one:

Non-Problem: There's no PORTAGE_NICENESS equivalent.

Rationale: Learn how to use nice. There's no GCC_NICENESS or VIM_NICENESS either.

To me personally (although I'm sure I'm not alone in this), it is portage's "rounded corners" that made it such a great package manager to use from the beginning. It had all this built-in convenience, like PORTAGE_NICENESS, like color output, like output that is verbose enough to be informative, but not overly verbose, like make.conf where you could set a range of optional, useful settings, like having emerge --ask which I would use all the time etc. Contrast that with something like apt-get and there's absolutely no doubt what the nicer tool is. Perhaps this bling also undermines portage's conceptual integrity, eventually turning it into an unmanageable codebase. But it's also what made me choose gentoo: the fact that it had, as it aspires to, the best tool.

Now paludis is more puritanical about this. What that means in practice is that it pushes that burden onto you, the user. We don't want it in paludis, so it's now your problem. As evidenced by advice like this:

Non-Problem: Paludis doesn't restore the xterm title on exit.

Rationale: Neither does anything else. Some programs do set it to a guessed value based upon a default prompt for certain distributions, but they don't restore it. You should be using PROMPT_COMMAND to do that yourself -- see the bash documentation.

So since paludis won't do this for me, it's now my problem to set in place the proper infrastructure for this, and to maintain it. It ceases to be a configuration option, it becomes a user environment issue. And I have to maintain this environment across machines, because it's no longer part of the application. Paludis is a tool that is technically superior, but inferior on user friendliness.

Not having FEATURES also means that I have to set all these things on the command line:

$ type ipal
ipal is aliased to `paludis -i --dl-reinstall if-use-changed --debug-build none --log-level warning --continue-on-failure if-independent'

And I'm still not sure if I'm setting all the optimal options, because there's tons of them. (Yes, there is PALUDIS_OPTIONS, but I wonder if it's useful to have different options on install, query etc.)

One serious usability problem is that paludis is ridiculously verbose. I wonder what kind of giant monitors the paludis developers own, but for my part paludis = lots of scrolling. Even running paludis --help has to be piped to less to refresh my memory on the most useful switches. What's more, the most important output is always at the top, so I always have to scroll the longest distance. If you want to flood the screen, put the crucial bits at the bottom, that's common sense. Case in point, if I'm installing packages and I do a paludis --pretend, I have to scroll up through all the use flags, then I come to the list of packages, but each package entry is several lines long, so by the time I get to the top I'm quite annoyed. The verbosity of paludis has to be easily 3-4 times that of portage.

Furthermore, default settings matter a great deal, and as a developer I think you are entirely culpable for setting poor defaults. On my first day with paludis I reached a show stopping bug when openssl refused to compile. As it turns out, it was the test suite that was broken, and since paludis by default runs the tests (or did, anyway), it just wouldn't install. SKIP_FUNCTIONS="test" fixes this, but that was clearly a misguided default setting. But there are many examples of this. --debug-build is enabled by default, which I think is wrong because most gentoo users don't actively debug *every* package they install. Meanwhile, these files take up quite a bit of space. Furthermore, coming back to the verbosity problem, --log-level is set to qa. This means that I, as a user, have seen this message every time I invoked paludis for the last two months:

paludis@1200787359: [QA] In program inquisitio -s perl:
... When performing query action from command line:
... When handling query 'php':
... When fetching versions of 'dev-lang/php' in gentoo:
... When loading versions for 'dev-lang/php' in gentoo:
... When extracting version from '/usr/portage/dev-lang/php/php-5.2.4_pre200708051230-r2.ebuild':
... When parsing version spec '5.2.4_pre200708051230-r2':
... Number part '200708051230' exceeds 8 digit limit permitted by the Package Manager Specification (Paludis supports arbitrary lengths, but other package managers do not)

Not only is that a trifle of a bug, how exactly am I the user served by seeing this warning? I'm not a developer, so I couldn't fix it if I wanted to. Furthermore, the irony of it all is that apparently paludis, which gracefully handles this problem, is the one emitting the error, whereas portage, which may actually be affected by it, doesn't. A pure exercise in futility.

Installing packages also outputs chunks of lines that never seem to change with any package, output I have no need to see:

>>> Running ebuild phase prepare as root:root...
>>> Starting builtin_prepare
>>> Done builtin_prepare
>>> Completed ebuild phase prepare
>>> Running ebuild phases init saveenv as root:root...
>>> Starting builtin_init
>>> Done builtin_init
>>> Starting builtin_saveenv
>>> Done builtin_saveenv
>>> Completed ebuild phases init saveenv
>>> Running ebuild phases loadenv setup saveenv as root:root...
>>> Starting builtin_loadenv
>>> Done builtin_loadenv

Is this supposed to be useful information for the user? I don't even know what it means.

Clearly, there are a few glaring problems. And maybe that's not so shocking from a development team that seems dead focused on technical issues. No surprise then, perhaps, that from a technical standpoint paludis fires on all cylinders. It took me a few days to get used to paludis, but since then it hasn't done anything weird or unexpected, it hasn't crashed, it has been rock solid. And those "issues" that may come up in the fullness of time? They never came up.

And this you already knew: paludis is fast.

Gentoo is fabled to be the maverick of a distro that blows up right, left and center. While I find that to be a gross exaggeration, it does occasionally crash in scary ways. :) Although I don't think it's any worse than the others, upgrading Ubuntu from release to another is still broken (Edgy -> Feisty, check), and last time I tried the Fedora upgrade cycle it was busted as well. Obviously, I wouldn't be running Gentoo these last five years if it were as fragile as some people think. I actually find it the most sane distro in many ways.

But yesterday I must say I had quite a bit of fun with it. I keep to a fairly recent upgrade path, so I rarely fall behind more than a week. The expat1->2 upgrade had been in the works for some time, but the einfo flew by without me seeing it, as it often is. Interestingly, half the packages on the system depend on expat, and all of them were suddenly broken. The first thing I noticed was Firefox behaving odd.

Some things cannot readily be captured in a picture, an audio recording, or a video. Not even in a piece of text. A cool bug is one of those things. It has to be investigated, experimented upon, to understand the extent of it. I regret that I had no way to record this work of art for posterity. So Firefox was running happily along, as it had been doing for hours. Then at one point, I open a new tab and type fa in the location field to call up facebook from the drop down list. But I never got to a. Just as I hit f on the keyboard, Firefox crashed.

Of course, Firefox *does* love the crash, it crashes several times a day. But it almost always crashes over Adobe's frightfully stable flash plugin. This was not one of those times, I didn't have flash content in any of the open tabs. Well, never mind, it was probably a coincidence. So I start it up again, I get my tabs back, and again I open a new tab to load facebook. Again I never get to a. :D Hm, this is starting to get interesting.

It turns out that a single key press would crash Firefox, but it didn't have a problem with the mouse. And not just Firefox, any Gtk application. :cool: At this point I was quite amused.

Not surprisingly, bugs are more fun when you have an idea what to do about them. I did track down the expat problem after some digging, but it made me recompile a boat load of packages. All of them seem to work again, although revdep-rebuild hadn't been run for a while and exposed some other problems. One package that didn't recover, however, was digikam. I'm back on 0.8.2-r1 after running 0.9.2 happily for weeks. Odd.

The moral is this: if you're going to crash, find a fun way to crash. :party:

Eyecandy is somehow nicer when it serves some purpose aside from just looking pretty, wouldn't you say? Then it has the same kind of effect as a great car or fine architecture. "Wow, it's great. And it looks awesome." Otherwise the appearance on its own seems a bit shallow and pointless. Now for the demo, here's my newest wallpaper (click to see the fullsize hosted on deviantart):

If you've ever thought that having one particular image on your desktop gets a bit dull, then this may be something for you. xplanet generates images of the Earth at set intervals (for example every 10 minutes) that shows the Earth roughly at this point in time. In addition, what I have here is cloud cover updated every 3 hours, so it's like a weather map. xplanet is phenomenally flexible, it can render multiple bodies at the same time (for example the Earth and the moon), it renders stars, it renders all the planets in the Solar System (yes, Pluto too) and many more bodies. What I have is a pretty standard configuration. So where to pick up the goodies? First, install xplanet (it's in portage :) ). Then if you run KDE, right click on the desktop and go into the config. On the Background tab, click Advanced Options and xplanet should appear in the list there.

When you click Modify... xplanet will most likely have filled in the blanks for you, but otherwise something like this will do:

The Preview cmd isn't really important, but for Command you could use:

xplanet -config ~/.xplanet.rc -radius 60 -latitude 52 -longitude 5 --geometry %xx%y --num_times 1 --output %f.jpg && mv %f.jpg %f

This will center the view on Utrecht more or less, but you can pick your own coordinates. Since we've supplied a configuration file, we have to create one.

$ echo -e "[earth]\ncloud_map=/tmp/.xplanet/clouds_2048.jpg" > ~/.xplanet.rc

Now we want to rig up a system that will download updates of the cloud map when they are available. We've already declared that they should be written to /tmp/.xplanet/clouds_2048.jpg, so let's create that path now.

$ mkdir -p /tmp/.xplanet

We'll use Michal Pasternak's python script for this. First save the file in /usr/local/bin, make it executable, then open it and edit this line:

defaultOutputFile = "/tmp/.xplanet/clouds_2048.jpg"

And finally we're going to use our friend cron to execute the script every hour:

$ crontab -e

And add this line:

0 * * * * python /usr/local/bin/download_clouds.py &>/dev/null

And that's it. Now you have a totally kickass wallpaper. :cool:

References

1. Tomasz Karbownicki's original entry which explains how to do this in Gnome [pl]
2. Kamil Baćkowski's follow-up entry on using xplanet in KDE [pl]
3. xplanet website with tons of info and hacks

Sometimes, especially when disk space is low (or when system backups grow unreasonably large), it's nice to know exactly how much space the biggest packages occupy. Obviously, OpenOffice is never above suspicion, but certain others can take up way more space than you would think.

I wrote a little script to print the size of all packages installed on the system. It uses the CONTENTS file for every installed ebuild to check the size of the files which belong to a package and give a sorted listing of packages by size.

#!/usr/bin/env python
#
# Author: Martin Matusiak <numerodix@gmail.com>
# Licensed under the GNU Public License, version 2.
#
# revision 1 - bugfix for paludis symlink in pkgdb

pkgdb = "/var/db/pkg"


import os, string, stat
from operator import itemgetter

sizes = {}

cats = os.listdir(pkgdb)
for c in cats:
	cpath = os.path.join(pkgdb, c)
	if os.path.isdir(cpath):
		cat = os.listdir(cpath)
		for p in cat:
			size = 0
			
			cont = os.path.join(pkgdb, c, p, "CONTENTS")
			fd = open(cont, 'r')
			
			strings = fd.readlines()
			for s in strings:
				line = string.split(s, " ")
				if line[0] == "obj" and os.path.exists(line[1]):
					size += os.path.getsize(line[1])
			
			fd.close()
			
			sizes[os.path.join(c, p)] = size

pkglist = sorted(sizes.items(), key=itemgetter(1))

for i in pkglist:
	(size, pkg) = ( str(i[1]), i[0] )
	print string.rjust(size, 11), " ", pkg

The output looks like this:

          0   virtual/x11-7.0-r2
         66   kde-base/kde-env-3-r4
        393   kde-base/kdebase-pam-6
        889   sys-apps/coldplug-20040920-r1
...
      94217   net-ftp/ftp-0.17-r6
      94642   kde-base/kcminit-3.5.3
      95629   sys-process/psmisc-22.2
      95931   sys-apps/ivman-0.6.12
      97358   app-admin/gnomesu-0.3.1
...
  122593614   dev-java/sun-jdk-1.5.0.08
  132864794   dev-lang/ghc-6.4.2
  145477793   app-text/tetex-2.0.2-r8
  221943002   sys-kernel/gentoo-sources-2.6.17-r7
  340336824   app-office/openoffice-bin-2.0.3

Unsurprisingly, OpenOffice claims victory, but this is a small reminder about how big kernel sources are. Tetex and GHC aren't minimalistic either.

UPDATE: Paludis bug fixed.

The reason I wanted to try Linux in the first place was that I was doing some websites and needed to mess around with cgi scripts in perl. I thought since the servers run linux, it would make sense to emulate that environment at home. The first thing I tried was Slackware, I think v3. Well that was a little much, I had a book that I borrowed from a friend but it was quite involved and I didn't really know what I was doing. I tried a couple versions of Red Hat in the years following that but there was no easy way for me to find out what exactly I needed to do to get an up and running install of apache and perl. I didn't try looking on the net because I didn't want to spend that much time on it. I also got the impression that without a thick book on Red Hat, I was totally lost. Then I heard something about Gentoo. I didn't really want to try it because I knew there are so many distros out there that trying them all doesn't make a lot of sense. The reason I did was that someone had mentioned FreeBSD and how great the ports system is, and apparently Gentoo had something similar. I don't think I would have given it much time if it wasn't for that install document. Sure, it was tough to get everything to work the first time but having this forum as well to ask questions, I succeeded in the end. By this time it wasn't really about perl anymore, I wanted to get into Linux because Windows was quite dull. So I had a working install of Gentoo, indeed I found out that portage is delightfully convenient. Apart from the time lost in compilation, packages get installed with no effort, dependencies are handled nicely and there is no issue of failed dependencies. Another thing that I really liked was the colors in the console. It's a simple detail but it makes a difference, it makes the shell less intimidating. I also felt that the documentation was not merely trying to explain how to do it, but also briefly explain why you're doing it. This is brilliant for someone who doesn't know what is what.

Since then I've been running Gentoo on a server with various services like apache, samba, cvs, mail, mysql etc. Lately I've also started thinking about using it as a desktop OS. I still rely on certain things like Dreamweaver but

the list of missing software is getting shorter. I've stuck with Gentoo for a couple of reasons.

Number one is this forum. Whenever there's a problem, chances are the issue has already been noted and fixed by someone else. Now people will say that you can get support for any distro, there are websites, mailing lists, irc channels etc. I have to say I'm not a big fan of the latter two. I happen to like fora, they're convenient, they have a certain atmosphere to them and they're easily searchable. I have tried finding fora for Suse but I haven't found any, the links were either 404 or the fora were deserted.

Reason #2 is portage. In the past year or so I've tried a bunch of different distros. After all, if there is something better, why not use it? I've looked at Red Hat, Suse, Mandrake, Lindows, Xandros, Debian and Knoppix. I've come to realize that none of them work as well for me as Gentoo does. The main reason is always portage. I've heard people say rpm is easy, there's a simple set of commands to use and you can compile from source if you want to. That very well may be, but for some reason I have never succeeded. Perhaps I'm not fit to understand but I can never figure out where to get the packages and how to install them smoothly. Whenever I try, I always have a list of dependencies missing and chances are that along the way one of them is incompatible with something else or a previously installed version of itself. Even when I succeed at installing something, I feel locked down because I know that should I want to upgrade it or remove it, I may have to go through another long session. And while compiling does take a long time, it's effortless, everything is done for me. Even though I don't have to compile rpm, it always me over an hour, sometimes more, to find the right package, solve dependencies etc. I prefer to leave emerge running while I do something else. Then there's the option to compile, well I've tried that as well, a couple of weeks ago I was trying to install ntop on Red Hat 7.3. I have barely compiled anything in my life save watching portage and I tried for hours without succeeding. I thought everything was in order, but autoconf wouldn't stop complaining about a failed dependency.

So that's rpm, and unfortunately most of the biggest distros are rpm-based. They're not very good at keeping your system up-to-date, Red Hat had that RHN service but apparently it would stop working after a short while unless you register for it. Mandrake lets you define an ftp path as a source of updates but I could never get that working evenso. And Suse takes ages often, to just figure out what packages are new and the selection is limited. This is how portage really excels, the sheer volume of packages available is stunning and unmatched in any easy-to-use distribution I have seen. So then Debian. Well, Debian is not as nice as Gentoo. And I didn't know where to look for documentation. So as long as I have Gentoo and I don't mind compiling everything, there is little incentive to use Debian. Xandros and Lindows are Debian based but they don't make Debian that much easier, they just give you a nice desktop.

And some people say the builds always break, I can't compile anything because it won't finish. Perhaps I have been exceptionally fortunate but from all the packages I've installed, very few builds have failed to complete. Even when they don't, all you have to do is try the older or the newer version of the ebuild, and if you're stuck check the forum/bugzilla and that's it.

Thus I've found out that most distributions are fine if you're satisfied just using whatever they give you on the install disc. But if I want to use mplayer, then I'm gonna have to look elsewhere and rarely is it as easy to install as with Gentoo. And it's not that I have to use all the newest software. But as long as I know that I can, I'm thinking there's no reason not to be able to do it on any other distribution.

In the course of these two years, using the Gentoo documents and most importantly, reading this forum, I have learnt a lot about Linux. And I'm still a complete newbie but with a little guidance, I can compile a kernel, I can install a couple of services and sometimes I can even predict where to find something in this cryptic filesystem. As a Windows user for a decade, I'm not used to learning anything because there's not a lot to learn. But I've come to accept that some things take a little understanding and reading documentation before trying to install something can actually save you some time later on when you're trying to find out why this or that.

So thanks a lot Daniel Robbins and long live Gentoo!