Archive for 2015

django girls 2015 summary

December 16th, 2015

I've never been too far removed from Django Girls. I was present at EuroPython 2014 where the very first Django Girls workshop was held. But at the time I had no idea what that even was. It wasn't until a few months later that the story of Django Girls started reverberating through the Python web.

I decided to take the plunge in September and attend my first Django Girls in Groningen. It was a great experience and I knew right away that I wanted to do more of these. I kept checking the event calendar every few weeks and eventually I settled on Bordeaux in November and Rome in December.

Stats

Groningen: 60 participants, 20 coaches

Bordeaux: 27 participants, 10 coaches

Rome: 35 participants, 11 coaches

Our combined group of 40 coaches has taught programming for a day to 120 women! And globally Django Girls is nearing 3000 participants with multiple events happening pretty much every single weekend.

What about 2016?

The big news is that we are organizing our own event in Den Haag in March!

Beyond that I'm sure I will be present at other Django Girls events too - I just don't know which ones yet.

djangogirls as a keyframe moment

December 3rd, 2015

In video compression there is a concept known as the keyframe. The simplest way to store a video recording is to store each frame (let's say it's 25 frames per second) as an image, all together in one big file. But people noticed that when progressing from frame to frame most of the image of a frame does not change between frames. That makes sense, otherwise the whole screen would flash constantly. The new model of storing video was to store frame 1 in its entirety, and then for frame 2, frame 3 and onwards you would only store what changed from the previous frame.

This gives enormous gains in terms of filesize and all common video formats use it. But we don't always view video from frame 1 right up until the end. Sometimes we want to seek backwards and forwards. If we only had one frame in its entirety and then a long sequence of frame changes it would be very expensive (and therefore slow) to calculate what frame 37500 at the 25 minute mark should look like.

That's why we also have keyframes. A keyframe is simply a frame that is stored in its entirety (like frame 1), which occurs at certain moments in the video. For instance, once every minute (every 1500 frames) we store the whole frame, and in between keyframes we only store changes or "deltas".

As it happens, keyframes also have a second purpose as an error correction mechanism. If you ever try to play back a video and all you see is the whole screen full of weird purple rectangles with something moving here and there, that's because the video player is applying changes frame by frame, to a keyframe which is damaged. So all you see is the changes, but not the original image from which the changes have been calculated. And thus it looks like the video is broken. But if you wait long enough, you might reach a keyframe. And once you have a keyframe you have the whole image again, and calculating changes from that image makes the video "work" again.1 So if some of the keyframes are missing or damaged it breaks part of the video, but not all of it. Keyframes are a mechanism to recover from a partially damaged video file.

We can even view a keyframe as a metaphor in life. A keyframe is a point where we have excellent clarity - we feel we know exactly what we need to keep doing, what we need to stop doing, what we need to start doing. But then the moment passes, we chug along, and after a while it's not all that easy to measure exactly how much progress we've made, or whether we're headed in the right direction. Not until the next moment of clarity. The next keyframe, if you will.

---

I'm heading home from Django Girls Bordeaux and as I'm sitting on the plane I realize I'm incredibly at peace with the world today. Nothing seems to bother me at all. As if I've just nourished myself on the exact right diet that my body needed. Even physiologically speaking - over the last two days I've lived on less food and less sleep than normal. Yet my body kept saying: that's all we need. Like an engine running on the perfect fuel, at peak performance.

I've coached at Django Girls once before, but this time was even better. As "meta coach" my mission was to supplement the coaches - one per team - by helping the participants whose coach was busy, or by helping with obscure problems. In practice I ended up more like a doctor doing rounds - I would quite systematically visit each team and help anyone who needed assistance. This was super awesome, since I had a chance to talk to all the participants and all the coaches.

At its core Django Girls is a really simple idea. We invite a group of women and teach them some basic programming and web development. Big deal, right? But think about the last time you learned some basic programming... What did that lead to? A career in software by any chance? A life long interest in programming perhaps? So you see, it *is* a big deal. It's actually one of the bigger deals out there.

Let me illustrate why.

On Friday night we had a drinks event for the coaches. It was basically like any group of programmers drinking beer and talking shop, with splinters of pip, pypi mirrors, wheels and devpi flying everywhere (mercifully, a critique on the state of python packaging was omitted). For me it was doubly interesting, since that's literally the first time I've ever talked about programming in French.2 But I've heard a version of that conversation any number of times, and basically it's a group of coders making fun of and complaining about stuff all night long, albeit in an entertaining manner.3 This is our culture, people. Oh and I forgot to mention that 9 of 10 coaches were male. (But that's irrelevant, right?) End of scene.

Saturday rolls around, it's the big day. The room fills with 27 participants and their coaches. Everyone's ready to roll. The participants are a little shell shocked at first, not surprisingly. How would you feel if you had to type cryptic stuff in a terminal window? But quite soon everyone is humming along nicely. The coaches aren't talking amongst themselves now, they're with their teams. And each one is doing a fantastic job. (I saw the whole thing, remember? I was walking around.) They're being supportive. They're enthusiastic. They explain things patiently. They help the participants relax and understand that they're on the right track.

Let me repeat that in case you got distracted for a second. The *participants* are making the *coaches* better. They're making *us* think and feel more positive and have more fun than usual.

And the participants? Well, where do I start? When is the last time you had the courage and the patience to spend 9 hours learning something new? Something that many people think "isn't for people like yourself"? Right, chew on that one for a minute. You see, Django Girls isn't like being in school where you mindlessly sit through one class after another. The people who participate are motivated to learn. They work through a tutorial that, despite having been written for beginners, is full of technical terms they've never heard before.4 They're expected to learn mostly through reading and performing exercises. They're expected to absorb a lot of knowledge throughout the day. They're expected to show a lot of patience and perseverance. And boy do they ever. Not only that, they are some of the nicest and most fun people you could ever spend a day with.

So you see, it's not that we're doing them a favor because one can have a nice career in IT these days. You don't have to work in IT to have an awesome career, there are many other options. "We" need "them" much more than "they" need "us".5

They are beginners. That sounds like a bad thing, but it's actually an awesome thing. They didn't come up through our computer science programs. They didn't cut their teeth on PHP back when it was just C macros.6 They come from a different background, with a different mindset, and a fresh perspective. They're not jaded like us, joking around about how broken software is. And we, the coaches, are seeing it through their eyes. Why is coaching at Django Girls so fulfilling? Because wow, that's what programming is *supposed* to be like!

Some highlights that stuck in my mind:

  • The participant who said "so a=6 and b=4, but how do you get 24?" Thirty minutes later not only is assignment under control, she says "oh so len and str are functions I can use just like in Excel"?
  • The participant who would explain back to me what she had learned, in her own words, and her explanation was dead on every time. Later on she got into a discussion with her coach on the merits of Django vs Wordpress and what you could do/couldn't do with either one.
  • The participant who early in the day was almost too focused to talk. By the afternoon she'd racked up enough wins that she was listening to music on her phone, reading the tutorial out loud and nodding along, totally getting it.
  • The participant who finished the whole tutorial with time to spare (trust me, it's really long) and was browsing sites on code bootcamps while telling me about a site she was planning to build to serve files for use at the library where she works.

And that, to make a long story short, is why someone who's been a programmer for 20 years is raving about how incredible it is to coach at Django Girls. A moment where everything comes together and everything is awesome. A keyframe moment that reminds me that *this* is what I want my life to be about.

What are the keyframe moments of your life?

---

  1. I say "work" because as far as the video player is concerned a frame is just a bunch of pixels with colors. It has no way of knowing what the frame is supposed to look like.
  2. The whole idea of coaching in French was a massive YOLO. It was tough, but it was super fun!
  3. I know it extremely well, because in my previous team that was literally my favorite way to pass the time with my programmer buddies.
  4. I worked through it myself. Last week I didn't know fichier from dossier. I made a stack of flashcards for all the technical terms you encounter in the tutorial and crammed them. Thank you for saving my skin, Anki!
  5. The "we" vs "they" rhetoric is pretty painful to read, isn't it?
  6. Or whatever your claim to fame is.

no metro de lisboa

August 4th, 2015

O metro de Lisboa tem um sistema de passagens eletrônico. Você tem que comprar o seu bilhete numa máquina da estação antes de viajar. O bilhete mesmo custa 50 cêntimos e depois você precisa carrega-lo com crédito suficiente para a sua viagem.

Então, em Lisboa tem uma companhia de metro, e múltiplas companhias de comboios.

A coisa louca é que o crédito que é valido para uma companhia não pode ser utilizado por outra. Assim você tem de carregar o seu bilhete com crédito para a primeira etapa, pois na estação seguinte tem de o carregar de novo pela viagem seguinte.

Se você pretende viajar primeiro de metro, tem de carregar para o metro, e só depois para o comboio. Não é possível carregar só uma volta para 20 euros e viajar muitas vezes com isso. Porque não? Não sei, isso não faz sentido.

Me lembro de que um dia um senhor aproximou-se de mim a perguntar "você conseguiu?". Eu não estava a entender. Consegui o que? Ah, comprar o bilhete? Claro que sim. Mas porque o senhor me faz esta pergunta? Ele era um homem de cinquenta-sessenta anos. Acho que era de Lisboa, então eu era turista. Então o que este cara quer? Quer dinheiro?

Logo percebi que ele não conseguia comprar o bilhete porque não é fácil compreender a máquina. Só queria que eu lhe ajudasse a comprar o bilhete. Que bizarro. Nunca antes vi um turista ajudar um residente porque este não compreendia o sistema de bilhetes.

a little help with bitwise operators

August 3rd, 2015

Binary numbers are easy, right? You just do stuff with bits.

But invariably whenever I code C I can never remember how to actually set a bit or test a bit, I keep getting and and or confused.

So I made a cheat sheet I can look up any time. These are the key ones:

All the others are available on the cheat sheet.

so do you know how your program executes?

August 2nd, 2015

The answer is no, and here's why you shouldn't feel peer pressured into saying yes.

I was reading about disassembly recently when I came across this nugget:

(...) how can a disassembler tell code from data?

The problem wouldn't be as difficult if data were limited to the .data section of an executable and if executable code were limited to the .code section of an executable, but this is often not the case. (...) A technique that is often used is to identify the entry point of an executable, and find all code reachable from there, recursively. This is known as "code crawling".

(...)

The general problem of separating code from data in arbitrary executable programs is equivalent to the halting problem.

Well, if we can't even do *that*, what can we do?

We start with a program you wrote, awesome. We know that part. We compile it. Your favorite language constructs get desugared into very ordinary looking code - all the art you put into your program is lost! Abstract syntax tree, data flow analysis, your constants are folded and propagated, your functions are inlined. By now you wouldn't even know that it's the same program, and we're still in "high level language" territory (probably some intermediate language in the compiler). Now we get basic blocks and we're gonna lay them out in a specific order. This is where the compiler tries to play nice with the branch prediction in your cpu. Your complicated program aka "my ode to control flow" now looks very flat - because it's assembly code. And at last we assemble into machine code - the last vestiges of intelligent life (function names, variable names, any kind of symbolic information) are lost and become just naked memory addresses.

Between your program and that machine code... so many levels. And at each level there are ways to optimize the code. And all of those optimizations have just happened while you stared out the window just now.

So I started thinking about how programs execute. The fact is that predicting the exact execution sequence of a program, even in C, even in C 101 (no pointers, no threading) is basically impossible. Okay, I'm sure it's possible, but I'd have to know the details of my exact cpu model to have a chance.

I need to know how big the pre-fetch cache is. I bet there are some constants that control exactly how the out of order execution engine works - I need to know those. And I need to know the algorithms that are used there (like branch prediction, remember?). I need to know... oh shoot, multicore! Haha, multicore is a huge problem.

Basically, I need to know exactly what else is running on my system at this very time, because that's gonna wreak havoc with my caches. If my L1 gets hosed by another process that's gonna influence a load from memory that I was just about do. Which means I can't execute this instruction I was going to. So I have to pick some other instructions I have lying around and execute those speculatively while we wait for that delivery truck to return all the way from main memory.

Speculatively, you say? Haha yes, since we have these instructions here we'll just go ahead and execute them in case it turns out we needed to do that. Granted, a lot of what a cpu does is stuff like adding numbers, which is pretty easy to undo. "Oops, I 'accidentally' overwrote eax." I guess that addition never happened after all.

And then hyper threading! Do you know how hyper threading works? It's basically a way of saying "this main memory is so damn slow that I can simulate the execution of *two* different programs on a single core and noone's really gonna notice".

This whole thing gives rise to a philosophical question: what is the truth about your program? Is it the effect you observe based on what you read from memory and what you see in the registers (ie. the public API of the cpu)? Or is it the actual *physical* execution sequence of your instructions (the "implementation details" you don't see)?

I remember when virtual machines were starting to get popular around 2000 and there was a lot of discussion about whether they were a good thing - "think about the performance implications". Hell, our so-called physical machines have been virtual for a very long time already!

It's just that the cpu abstraction doesn't seem to leak very much, so you think your instructions are being executed in order. Until you try to use threads. And then you have to ask yourself the deep existential question: what is the memory model of my machine anyway? Just before you start putting memory barriers everywhere.

So no, you don't. But none of your friends do either (unless they work for Intel).