The swanky atmosphere Sunday morning was slightly impaled by a grammar blooper in the program. But never mind. Wikitravel proclaims:

Concertgebouw

Famous for its orchestra and its accoustics (among the top ten in the world), this is the world's most frequently visited concert hall.

I have to say I'm starting to suspect it's the staff authoring this glowing review. Personally I wasn't all that impressed with the acoustics. The large performance hall is a venerable interior with profuse decor, however the technological realization seems to have evaded modern technology. The chamber orchestra was not heard terribly well in the rear half of the hall, which seems a bit amateurish from a sound technology perspective in this day and age.

But for just 15 bucks it wasn't bad value for money, even though I expected more from the venue.

Every detective story or for that matter, spy novel, has the same overreaching goal: to give you an interesting puzzle to solve. A story is being told - you have to pay attention, and while you're taking notes you're also continuously assembling a repository of information and scouring it for relationships and patterns. And once you link two facts together, you've understood a part of the puzzle. A couple of factors determine why this is fun. First of all, the content of the puzzle should be fun, that is if you're putting together a puzzle of the Eiffel Tower, you will have fun if you like the Eiffel Tower. Then there are different ways of creating a puzzle, deciding what the pieces will be, how clear or faint the connections are going to be and so on. And finally there are different ways of giving you the pieces, at a certain pace, in a certain order. In as much as there are dependencies between the pieces, such that one will not help you without having a certain other piece, you can craft a complicated scheme. All these factors vary from story to story, some puzzles are fun, some are not.

This process of solving a puzzle is actually very intuitive. It's the same process by which we learn things. The difference is that when learning we're trying to understand an incidental truth, which isn't governed by any person's capricious decisions to make it so. And it could well be beyond your ability to solve. It may also be that no one has solved that puzzle. A man made plot, on the other hand, is a puzzle created specifically for our enjoyment, where the creator makes all the decisions, limits the scope appropriately, and deliberately sets the difficulty at a certain level.

A spy novel is always about deception upon deception, and as a reader you participate. Not only is the character in the story being deceived, you are as well. The author will be meticulous about telling you facts that point to a certain conclusion, but it's actually a decoy. Equally, he will mention facts that don't have any apparent use at the moment, but they will explain something further along, and if you're careful, keep you from walking right into a trap. It's really just a mind game.

The Ambler Warning was actually published in 2005, based on an unfinished manuscript Ludlum was working on when he passed away in 2001. In other stories of his that I've read he generally likes to run a chronological race, with very economical use of facts. This requires efficient reading, because rarely is anything said twice. The socio-political setting of the story also lets certain bits of information permeate from the surroundings. And when it comes to giving up information, Ludlum keeps you from seeing very far into the horizon. Most of the time he lets you guess and suspect things that won't be revealed for a long time yet. This puts you into a mode of forensics, trying to make what you know go as far as possible. It culminates in the economical resolution of some question, solving it with the least amount of facts possible. At other times he likes to keep you starved of new facts for a while, causing you to go back over what you already know and dig for new deductions in vain, before he finally drops something significant that has a big impact, but which changes so many existing variables that you get overwhelmed by the amount of information you have to process.

In The Ambler Warning we get a different kind of story telling. For one thing, only pieces of the timeline are accounted for, the rest isn't told. But it's enough to make sense of, again playing on the economical use of information. Secondly, the pieces are not told in chronological order. There is a leap from chapter one into what chronologically is the third chapter, but told in a different order.

This changes a lot of the assumptions of a continuous story, the fact book keeping takes on a different model. Since it's not chronological, you can't keep the chapters completely self sufficient, you have to make some modest references to chapter three in order to maintain a smooth reading experience. And as shown in the figure, this builds certain expectations about chapter three. But it does mean that you have to stack up certain facts from chapter two whose meaning you don't yet know, and won't, until you reach chapter three. This is a different mode of puzzle solving than I've seen in other Ludlum books, but it's pretty interesting.

The plot and the characters are actually a bit below par. I wonder if this is the influence of the editor who shaped an unfinished manuscript into a novel. And the conspiracy holds up nicely until the resolution. In fairness, this is very common. Authors build expectation with a meticulous use of detail, but then fail to make the conspiracy live up to those expectations. It's a difficult thing to do, and what they pull off is impressive enough.

You probably remember ethereal, right? Yeah, the gtk packet sniffer that has been around forever, but people like me who are pretty lame about networks tried it and didn't know what to do with it. In principle it's all very simple: you capture traffic and then you look at the data. Erm... kay. Well, in practice I had a tough time getting something out of that. Even a simple http request when split into an array of tcp packets doesn't really enlighten me that much when I have a little chunk of it here and the next there and so on. There are so many of them that I'm looking for the needle in the haystack.

This is a lesson in user interface design. ethereal is now called wireshark, and it's the same exact gui window that it always has been (with some minor incremental improvements). Here's the difference though. They figured out that data visualization goes so much further when you actually give a thought to representation.

The before picture, googled for an old screenshot of ethereal:

The window is split into three panes. The top one is the one you'll look at first, this is the list of packets we captured. Notice how small the scrollbar is and just imagine we have a large chunk of traffic in front of our eyes. Scroll up, scroll down, sort by protocol, whatever. In the other two panes you have info on the packet that is selected right now. In the middle one you have nicely structured info, in the bottom you have the raw data. This is the way to look at packet content. Btw tcp packets can be up to 1.5kb in size, so it's not exactly a sizable amount of content.

Gear up for the after picture:

Yeap, colors. Really, that's all I had to say. People underestimate what a difference it makes to have the same exact output in contextual coloring. The same data just jumps up at you if you know it's gonna be in red and you don't have to parse the whole chunk of text to get to it. Notice how the old version and this one have the protocol column just the same, but how much easier is it now to spot different types of traffic? It makes a world of difference.

wireshark has another brilliant addition that I don't remember from years ago. You right click on a packet, like the tcp packet selected in the screenshot above that belongs to an http stream, and go Follow TCP Stream. What you now get is a new window with aaaaall the packets that belong to this conversation assembled and you can observe the whole thing in one place. Fabulous. You can also filter the data (on message level, not packet level mind you) on host if you like. It's fantastic.

If you thought *that* was bling, this will really blow you away. wireshark does not only assemble http conversations, it assembles a lot more protocols. For instance, I saw a talk the other day where the guy was demonstrating how he sniffed a voip conversation and then reassembled the whole thing to a simple audio file with a few clicks!

These two features, packets colored by protocol and packet assembly, alone make wireshark about 200% more useful than it used to be. And think about what has changed here. It's still the same data. The only difference is how you represent it.

It's always fun to start coding something from scratch, because it has no faults yet, you get a clean slate.

Being there from the beginning of the software life cycle also means that you see the whole thing up close. Baby's first steps, first school play, it's an adventure. I've noticed something that happens to my perception, though.

In the very beginning I have a clear vision of what I want to do and how I'm going to realize it. But the code isn't there yet, and the code that has been written up to this point isn't ready either. So when I'm coding and testing I find a lot of bugs. Some are things I didn't realize at the outset, and some are just errors in bad code. Both get fixed, but in the throes of this process I'm very in tune with how raw this thing still is. It's unstable: push it two inches beyond where it's supposed to be and it self destructs.

At this point I'm excited about the project, because it's new and I'm in the middle of it, enthusiasm running high. But I'm also conscious of how unstable it is presently. So although I'd like to, I'm not eager to give it to people. I don't want to expose them to this that I very well know is unlikely not to crash while they use it. I want to protect them from that. I also don't want someone to come back with a desk and bash me over the head with it. It's great, it's just not ready to be used yet.

The perception also depends on the type of bugs I find. There are both incidental and structural bugs, although the latter feeds the former much of the time. If I've coded a similar kind of thing before, I probably did something really stupid the last time, and I may have learned from that. Unfortunately, the space of mistakes is huge, so by the time I've exhausted all the mistakes and I'm guaranteed that from now on I'm not making any more, I would have aged considerably. But at least I can get it right when I'm starting with a clean slate. That or I can hack it up and redo the structure if I'm not too far into the process yet.

So when I'm in the midst of development and the code is visibly unstable, it sometimes feels like this is not only the current station but also the final destination. It's always going to feel like this.

But over time something unexpected happens. I run the code 50 times, it crashes 3 times. When I start to close in on feature completeness and say 80% code completion the code actually starts to be dependable. Structurally the system is almost ready, and what remains is to fill in the blanks. If the structure is decent the incidental bugs start dropping off too. So it's turning into a piece of code I can actually use, just what I had in mind in fact. :cap:

Without quite knowing when or how I begin to gain confidence in the code. And when I bounce off a few pieces of bad input and a couple of unforeseen scenarios get handled gracefully, my mood almost points toward celebratory.

I know I promised you a curve. Well, I lied, I don't know what the curve looks like. :P But it's there, be sure of it!

When I was 15 I spent a few weeks at a camp in the summer. We were sailing. Up in the top right corner of Poland there's a region called Masuria, a region of lakes. It's pretty much the only place in the country you really wanna go sailing. It's vast and quite uncontaminated by urbanity and worse: industry. You'll find small picturesque towns scattered between the lakes and the way of life is to stay on the lake most of the day and set up shop at night in tents.

So where was I going with this? Oh yes, sailing is fun, but it's hard to fill 14 consecutive days to the brim with it. There was a lot of down time, and plenty of boredom as well. I didn't know any of the people and I felt quite alone. I was also the youngest. There formed cliques, but I wasn't in any of them. I shared a tent with a guy who was 18 and we didn't fraternize. And anyway he was busy with his girlfriend.

Toward the end of the camp it got a little better for me. I had nothing to do, but I noticed my tent partner started reading a book he'd brought with him. I wasn't big on reading back then, but when you're that bored, a book will do nicely. One time he was out I got curious. It was a nicely bounded volume and I started looking through it, reading a few words out of the middle. The characters appealed to me, I kept on reading. Before long I was caught up in the book.

I guess my tent partner was a little put off, but well he knew I had nothing to do and most of the time I was indeed not doing anything, so he just let me. Anyway, it was his book, he could read it anytime, and his girlfriend was more interesting to him. Somehow word got around that there's a good book in the camp and several people got interested. The camp was coming to a close and I started to worry that, especially with this new interest, I wouldn't finish it. But despite the hype I kept on reading it pretty much all the time, and most people aren't the type to rip an item you're holding out of your hands, so I was in control. On the trip back from camp I finished the book with about 2 hours to spare.

Unfortunately, I forgot the title and years later I remembered the incident and tried to figure out what it was. That turned out to be harder than I expected. All I could remember was that it was a story about prisoners of war held by the Japanese during World War 2. And I vaguely remembered the title, but not accurately. I also didn't know what it would be called in English.

At one point I thought it might be Lord of the flies, the title sounded close enough. But that turned out to be a children's book, and a bad one at that. Eventually I tracked down the story, called King Rat, which is a literal translation of the title in Polish.

I read it a decade ago, and when I read it today the story is still captivating. It's a tale of Allied troops being held in a POW camp in Singapore during the war, under strict Japanese rule. Their living conditions are difficult to imagine, and prisoners have regular afflictions of fever and disease, those that are surviving. Food is scarce, drugs are lacking. They know nothing of the outside world, nothing about how the war is going, no news of home.

These are the circumstances in which survival is a privilege. From a sociological point of view it's almost like.. a controlled experiment. Prisoners realize they cannot survive alone, so they form groups of 2 of 3, accomplices they can trust, and take care of each other. They share food, help the sick, and look out for trouble. Everyone is putting some food aside just in case it might get even worse.

And this is the world of the King. A lowly American corporal is the one man who knows how to survive, nay how to live, in these surroundings. The only one not malnourished, the only one not diseased, the only one not clad in rags. He is.. a business man. Those prisoners who have somehow managed to hide their last possessions are keen to sell them when they have no other choice. Watches, jewelry, things that can buy a wealth of food to last weeks. The King is the only man who can arrange such a trade. He trades with the outside, clandestinely. The Japanese overlords strictly forbid it.

And this is the gripping story of a not-so-distant past.