Archive for 2016

two weeks of rust

January 10th, 2016

Disclaimer: I'm digging Rust. I lost my hunger for programming from doing too many sad commercial projects. And now it's back. You rock, Rust!

I spent about two weeks over the Christmas/New Year break hacking on emcache, a memcached clone in Rust. Why a memcached clone? Because it's a simple protocol that I understand and is not too much work to implement. It turns out I was in for a really fun time.


The build system and the package manager is one of the best parts of Rust. How often do you hear that about a language? In Python I try to avoid even having dependencies if I can, and only use the standard library. I don't want my users to have to deal with virtualenv and pip if they don't have to (especially if they're not pythonistas). In Rust you "cargo build". One step, all your dependencies are fetched, built, and your application with it. No special cases, no build scripts, no surprising behavior *whatsoever*. That's it. You "cargo test". And you "cargo build --release" which makes your program 2x faster (did I mention that llvm is pretty cool?)

Rust *feels* ergonomic. That's the best word I can think of. With every other statically compiled language I've ever used too much of my focus was being constantly diverted from what I was trying to accomplish to annoying little busy work the compiler kept bugging me about. For me Rust is the first statically typed language I enjoy using. Indeed, ergonomics is a feature in Rust - RFCs talk about it a lot. And that's important, since no matter how cool your ideas for language features are you want to make sure people can use them without having to jump through a lot of hoops.

Rust aims to be concise. Function is fn, public is pub, vector is vec, you can figure it out. You can never win a discussion about conciseness because something will always be too long for someone while being too short for someone else. Do you want u64 or do you want WholeNumberWithoutPlusOrMinusSignThatFitsIn64Bits? The point is Rust is concise and typeable, it doesn't require so much code that you need an IDE to help you type some of it.

Furthermore, it feels very composable. As in: the things you make seem to fit together well. That's a rare quality in languages, and almost never happens to me on a first project in a new language. The design of emcache is actually nicely decoupled, and it just got that way on the first try. All of the components are fully unit tested, even the transport that reads/writes bytes to/from a socket. All I had to do for that is implement a TestStream that implements the traits Read and Write (basically one method each) and swap it in for a TcpStream. How come? Because the components provided by the stdlib *do* compose that well.

But there is no object system! Well, structs and impls basically give you something close enough that you can do OO modeling anyway. It turns out you can even do a certain amount of dynamic dispatch with trait objects, but that's something I read up on after the fact. The one thing that is incredibly strict in Rust, though, is ownership, so when you design your objects (let's just call them them that, I don't know what else to call them) you need to decide right away whether an object that stores another object will own or borrow that object. If you borrow you need to use lifetimes and it gets a bit complicated.

Parallelism in emcache is achieved using threads and channels. Think one very fast storage and multiple slow transports. Channels are async, which is exactly what I want in this scenario. Like in Scala, when you send a value over a channel you don't actually "send" anything, it's one big shared memory space and you just transfer ownership of an immutable value in memory while invalidating the pointer on the "sending" side (which probably can be optimized away completely). In practice, channels require a little typedefing overhead so you can keep things clear, especially when you're sending channels over channels. Otherwise I tend to get lost in what goes where. (If you've done Erlang/OTP you know that whole dance of a tuple in a tuple in a tuple, like that Inception movie.) But this case stands out as atypical in a language where boilerplate is rarely needed.

Macros. I bet you expected these to be on the list. To be honest, I don't have strong feelings about Rust's macros. I don't think of them as a unit of design (Rust is not a lisp), that's what traits are for. Macros are more like an escape hatch for unpleasant situations. They are powerful and mostly nice, but they have some weird effects too in terms of module/crate visibility and how they make compiler error messages look (slightly more confusing I find).

The learning resources have become very good. The Rust book is very well written, but I found it a tough read at first. Start with Rust by example, it's great. Then do some hacking and come back to "the book", it makes total sense to me now.

No segfaults, no uninitialized memory, no coercion bugs, no data races, no null pointers, no header files, no makefiles, no autoconf, no cmake, no gdb. What if all the problems of c/c++ were fixed with one swing of a magic wand? The future is here, people.

Finally, Rust *feels* productive. In every statically compiled language I feel I would go way faster in Python. In Rust I'm not so sure. It's concise, it's typeable and it's composable. It doesn't force me to make irrelevant nit picky decisions that I will later have to spend tons of time refactoring to recover from. And productivity is a sure way to happiness.


The standard library is rather small, and you will need to go elsewhere even for certain pretty simple things like random numbers or a buffered stream. The good news is that Rust's crates ecosystem has already grown quite large and there seem to be crates for many of these things, some even being incubated to join the standard library later on.

While trying to be concise, Rust is still a bit wordy and syntax heavy with all the pointer types and explicit casts that you see in typical code. So it's not *that easy* to read, but I feel once you grasp the concepts it does begin to feel very logical. I sure wouldn't mind my tests looking a bit simpler - maybe it's just my lack of Rust foo still.

The borrow checker is tough, everyone's saying this. I keep running into cases where I need to load a value, do a check on it, and then make a decision to modify or not. Problem is the load requires a borrow, and then another borrow is used in the check, which is enough to break the rules. So far I haven't come across a case I absolutely couldn't work around with scopes and shuffling code around, but I wouldn't call it fun - nor is the resulting code very nice.

Closures are difficult. In your run-of-the-mill language I would say "put these lines in a closure, I'll run them later and don't worry your pretty little head about it". Not so in Rust because of move semantics and borrowing. I was trying to solve this problem: how do I wrap (in a minimally intrusive way) an arbitrary set of statements so that I can time their execution (in Python this would be a context manager)? This would be code that might mutate self, refers to local vars (which could be used again after the closure), returns a value and so on. It appears tricky to solve in the general case, still haven't cracked it.

*mut T is tricky. I was trying to build my own LRU map (before I knew there was a crate for it), and given Rust's lifetime rules you can't do circular references in normal safe Rust. One thing *has to* outlive another in Rust's lifetime model. So I started hacking together a linked list using *mut T (as you would) and I realized things weren't pointing to where I thought they were at all. I still don't know what happened.

The builder pattern. This is an ugly corner of Rust. Yeah, I get that things like varargs and keyword arguments have a runtime overhead. But the builder pattern, which is to say writing a completely separate struct just for the sake of constructing another struct, is pure boilerplate, it's so un-Rust. Maybe we can derive these someday?

Code coverage. There will probably be a native solution for this at some point. For now people use a workaround with kcov, which just didn't work at all on my code. Maybe it's because I'm on nightly? Fixed!


So there you have it. Rust is a fun language to use, and it feels like an incredibly well designed language. Language design is really hard, and sometimes you succeed.