findpkgs: Find packages for application

October 14th, 2009

Every distribution has a package manager and a whole lot of work goes into maintaining packages and correctly resolving their dependencies. This is a descriptive kind of dependency tracking.

The other day I had the idea of using a more "evidence based" method. Given a linked binary, you can find out what libraries it uses with ldd. (This, however, will not account for any dynamic linking that happens during runtime.) More interestingly, perhaps, given a running process, you can figure out which files it using to run. There is lsof, and if not, /proc/pid/maps has that information too.

Such a list of files can then be fed to the package manager to find the packages which own them.

For instance, which package owns init (on an Ubuntu system)?

$ findpkgs 1
upstart

What's needed to run ls (on a Gentoo system)?

$ findpkgs ls
sys-apps/acl
sys-apps/attr
sys-apps/coreutils
sys-libs/glibc

What about a Python application like iotop?

$ findpkgs `pgrep iotop`
dev-lang/python
dev-libs/openssl
sys-libs/glibc
sys-libs/ncurses
sys-libs/zlib

The query-package-manager-for-owner-of-file tries to figure out which package manager is used on the system in this order:

  1. paludis
  2. qfile
  3. equery
  4. dpkg
  5. rpm

To be honest I'm not really sure how useful this is, I just put it together since I figured out it could be done. It *can* answer the question: which packages are required to run this application? (Or to be more precise: to achieve this specific runtime state of the application.) So if you write an app, send it to a friend and he can't make it run, you could use findpkgs to get a list of them he needs to install (provided he's on the same distro and all that).

# Author: Martin Matusiak <numerodix@gmail.com>
# Licensed under the GNU Public License, version 3
#
# <desc> Find packages by binary or process pid </desc>
#
# <usage>
# source this file in bash, then run `findpkgs`
# </usage>


function _findpkgfor() {
	local file="$1";shift;

	if which paludis &>/dev/null; then
		paludis -o "$file" 2>/dev/null | grep '::installed' \
			| sed "s/::installed//g" | tr -d ' '
	elif which qfile &>/dev/null; then
		qfile "$file" 2>/dev/null | awk '{print $1}'
	elif which equery &>/dev/null; then
		equery belongs "$file" 2>/dev/null | awk '{print $1}'
	elif which dpkg &>/dev/null; then
		dpkg -S "$file" 2>/dev/null | awk '{print $1}' | tr -d ':'
	elif which rpm &>/dev/null; then
		rpm -qf "$file" 2>/dev/null | grep -v "not owned"
	else
		echo "No known package manager found"
	fi
}

function findpkgs() {
	local arg="$1";shift;

	if [ ! "$arg" ]; then
		echo "Usage:  findpkgs [ pid | /path/to/binary ]"
		return
	fi

	local pid=
	local arg_new=
	local bin=
	if echo "$arg" | grep "^[0-9]*$" &>/dev/null; then
		pid="$arg"
	else
		arg_new=$(which "$arg" 2>/dev/null)
		[ "$arg_new" ] && arg="$arg_new"
		if ! echo "$arg" | grep '^/' &>/dev/null; then
			echo "Can't find absolute path (or not a binary) for: $arg" >&2
			return
		fi
		arg=$(readlink -f "$arg")
		if ! file "$arg" | grep 'ELF' &>/dev/null; then
			echo "Not a binary: $arg" >&2
			return
		fi
		bin="$arg"
	fi


	local fst=
	local fst_new=
	local files=
	if [ "$pid" ]; then
		fst=$(ps aux \
					| sed "s/^[^ ]* *//g" \
					| grep "^$pid " \
					| awk '{print $10}' \
					| tr -d ':')
		fst_new=$(which "$fst" 2>/dev/null)
		[ "$fst_new" ] && fst="$fst_new"
		if ! echo "$fst" | grep '^/' &>/dev/null; then
			echo "Can't find absolute path for: $fst" >&2
			unset fst
		fi

		if $(which lsof &>/dev/null); then
			files=$(lsof \
						| sed "s/^[^ ]* *//g" \
						| grep "^$pid " \
						| awk '{print $8}' \
						| grep '^/' \
						| sort \
						| uniq)
		else
			files=$(cat "/proc/$pid/maps" \
						| awk '{print $6}' \
						| grep '^/' \
						| sort \
						| uniq)
		fi

		files="$fst $files"
		for file in `echo $files`; do
			_findpkgfor "$file"
		done | sort | uniq

	elif [ "$bin" ]; then
		files=$(ldd "$bin" \
					| awk '{print $3}' \
					| grep '^/' \
					| sort \
					| uniq)
		files="$bin $files"
		for file in `echo $files`; do
			_findpkgfor "$file"
		done | sort | uniq
	fi
}

un giorno lento

September 22nd, 2009

Oggi avevo mal di testa. Ecco perché non ho fatto molto. Ho lavorato un po', e poi ho fatto un giro sulla mia bicicletta. Quando sono tornato a casa avevo sempre il mal di testa, ma un po' meno. Poco dopo ho fatto alcuni esercizi di italiano del mio libro per imparare la lingua, e quindi mi sentivo meno male. Forse imparare l'italiano fa bene per il mal di testa?

Starbucks? Not all that it's cracked up to be

August 19th, 2009

So I went by Starbucks today. The line wasn't quite as long as I had seen it on previous occasions. I went in. What struck me right away was how noisy it was inside. Mind you they're located inside a busy train terminal, with nothing but a delimiting glass panel wall, so it's not going to be a quiet, cozy café. But they clearly had done nothing to insulate from the noise. Which is just as well, because most of it was coming from inside the Starbucks. There was music blasting, and five staffers yelling orders over the music. It was literally as noisy as MacDonalds on a busy day. So much for ambience, none of it. Those stories about people hanging out in a Starbucks - those places must be categorically different from this one.

As I was pondering this, the line advanced and soon I found myself making my order. And that's when I noticed how poorly organized the staffers were. First I made my order to the girl leaning over the cake counter. She proceeded to yell it down the line to the production division. Then I approached the cash register and had to repeat my order to another girl. "A large caffè latte." "Was it a venti?" "I have no idea, I said large." I refuse to speak their pretend language.  The girl who had taken the order chimed in to clarify. I forked over cash and, strangely enough, did not receive a receipt. Are they saving on paper?

That's when I got in line behind the six previous customers who were still waiting to get theirs. Clearly, they were backed up with orders.  My coffee was ready before all of theirs was. But the girl in production had no idea whom it was for, because there were no receipts to work from. The orders were filled, but how to match them with customers? So she asks every person in turn "what are you waiting for?" That was the third time I had to say my order.

All of this somehow distracted me from noticing the beating I took on the coffee. €3.95 for a large caffè latte? That's hilarious. At MacDonalds you can get a meal for a little over 5 bucks. Maybe they could justify the price if the café was a great place to hang out. Or if the coffee were especially good. Or if they were especially well organized and efficient.

koffie where coffee is arriving late

August 16th, 2009

Last week Starbucks opened a branch here in Utrecht, which didn't go unnoticed. I'm not one of those people who are completely nuts about coffee, but I am quite fond of it. :lick: I went by Central Station, but the line was much too long. I like coffee, but I'm not going to stand in line like it's Michael Jackson tickets.

Meanwhile, Greg debunks the idea that chains actually have a consistent product and asks why would people flock to Starbucks when they could be having better coffee at a local outlet? The obvious answer is: preconception. Much could probably be attributed to MacDonalds, who have forever maintained that they serve you the same food no matter where you are. Starbucks doesn't really have to do anything to push that idea, it's already been implanted.

But the deeper reason is that this country doesn't really have a coffee culture. Yet. When I moved here I was pleased to find out that koffie is part of the fabric of this society. It isn't a "new" thing like it is in, say, Poland, where tea has always been the dominant beverage. But the Dutch koffie predates the cawfee of today. In countries like Poland and Norway, without a strong existing coffee culture, the yuppie caffè latte gimmick has been a big hit in recent years. (Although in Norway, to "our" credit, most of the coffee places are local entrepreneurs, not foreign chains, and provide considerably better ambiance than sitting in a Starbucks.)  Not so here. With an existing "coffee infrastructure", Holland has not been quite the fertile ground, and the coffee revolution that has placed a coffee place on every corner hasn't happened. Coffee is still, by and large, the traditional sit down experience, and an unremarkable, humble one at that.

You won't find a lot of cafés that serve a caffè latte, or even a solid espresso. Around these parts coffee is still, much of the time, "just coffee". Not the fast food inspired, to-go in a paper cup with a selection of 30 different drinks.

is your town rainy?

August 13th, 2009

Complaining about rain seems to be some kind of ingrained human reflex. But if (almost) everyone is doing it then surely not everyone has a case? Before I moved to the Netherlands I heard people say it rained here all the time. That doesn't agree at all with my observations. But what's a lot? Compared to what?

I know, facts!

The World Weather Information Service has climate information on 1000+ cities around the world. I decided to do a little data mining. All conclusions to follow are based on that data. Just for kicks, I should mention there are very serious methodological problems with this exercise. First of all, they say countries don't necessarily use the same definitions for what's a rainy day or measure precipitation the same way. So you're not really supposed to compare cities. Secondly, when placing cities on a world scale, sampling is everything. And nothing is known about the criteria by which cities are included in the list. We're not gonna let petty details like that stop us, though.

I should also mention that "rain" as used here also includes snow where applicable.

Is it rainy here or what!

So is it rainy here? Let's compare it to the control point: Trondheim. And let's throw in some other data points too, for good measure.

  • Cities with a reputation for rain: Bergen, London, Seattle, some town on the Spanish north coast (found A Coruna on the list).
  • Cities known to be dry: Los Angeles, Zaragoza (turns out to be the driest city in Europe)
  • Other cities: Berlin, Dublin, Milan, Paris, Stockholm

The first number is the total annual precipitation. The second is the number of rainy days.

city            mm/y   days
Bergen          2250   235
A Coruna        1008   171
Seattle          945   122
Milan            943    86
Trondheim        892   216
Amsterdam        780   185
Dublin           733   129
Paris            650   112
London           611   145
Berlin           571   106
Stockholm        539   173
Zaragoza         317    79
Los Angeles      305    27

Bergen's reputation is well justified, it turns out to be the rainiest city in Europe. Rainy London, on the other hand, is a complete myth.

No, not like that, use *all* the facts!

But what do random cities tell us about the big picture? Not much. Let's see how countries place on the overall ranking. Here we see the span they occupy in the list, from the rainiest city on the left to the driest on the right. (Spain refers to mainland Spain.)

rainplot

The Netherlands place smack in the middle. So to say it's a rainy country is an obvious exaggeration, it's pretty much average. The very narrow span is probably due to the small and uniform geography. Norway, on the other hand, is definitely rainy, occupying the space in the rainy portion of the list. Bergen is most rainy, Oslo is least. As for London, it's actually the lower end of the UK span, further diffusing its rainy reputation.

If you like staying dry the very best place to be is Luxor in Egypt, where precipitation is a minuscule 3mm annually. At the other end is Quibdo in Colombia, with an incredible 8m of rainfall and 305 days of rain.

But you didn't...

I know, but this isn't supposed to be comprehensive. If you want to do further work on this data you can get the list I generated: