general purpose video conversion has arrived!

September 18th, 2008

When I started undvd I set out to solve one very specific, yet sizeable, problem: dvd ripping&encoding. I did that not because I really felt like diving head first into the problem would be fun, but because there was nothing "out there" that I could use with my set of skills (none). Meanwhile, I needed a dvd ripper from time to time, and since I didn't need it often I would completely forget everything I had researched the last time I had used one. This was a big hassle, I felt like I had no control over the process, and I could never assure myself that the result would be good. Somehow, somewhere, there was a reason why all my outputs seemed distinctly mediocre. Visibly downgraded from the source material.

Writing undvd was a decent challenge in itself, because of all the complexity involved in the process. I had to find out all the stuff about video encoding that I didn't really care about, but I thought if I put it into undvd, and make sure it works, then I can safely forget all about it and just use my encoder from that point on. When you start a project you really have no idea of where it's going to end up. undvd has evolved far beyond anything I originally set out to build. That's just what happens when you add a little piece here and another piece there. It adds up.

It's been about 20 months. undvd is quite well tested and has been "stable" (meaning I don't find bugs in it myself anymore) for over a year. One of the by products is a tool called vidstat for checking properties of videos. I wrote that one just so I could easily check the video files undvd was producing. But it turns out to be useful and I use it all the time now (way more than undvd). In the beginning I was overwhelmed by the number of variables that go into video encoding, and I wanted to keep as many of them as I could under tight control. I have since backtracked on a number of features I initially thought would be a really bad idea for encoding stability. But that's just the way code matures, you start with something simple and when you've given it enough thought and enough tests, you can afford to build a little more complexity into the code.

Codec selection landed just recently. And once I was done scratching my head and trying to decide which ones to allow and/or suggest, I suddenly realized that with this last piece of the puzzle I was a stone's throw away from opening up undvd to general video conversion. Urgently needed? Not really. But since it's so easy to do at this point, why not empower?

The new tool is called encvid. It works just like undvd, stripped of everything dvd specific. It also doesn't scale the video by default (generally in conversion you don't want that). So if you've figured out how to use undvd, you already know how to use encvid, you dig? :cap:

Demo time

Suppose you want to watch a talk from this year's Fosdem (which incidentally, you can fetch with spiderfetch if you're so inclined). You get the video and play it. But what's this? Seeking doesn't work, mplayer seems to think the video stream is 21 hours long, that's obviously not correct (incidentally, I heard a rumor that ffmpeg svn finally fixed this venerable bug). It seems a little heavy handed, but if you want to fix a problem like this, one obvious option is to transcode. If the source video is good quality, at least from my observations so far, the conversion won't noticeably degrade it.

So there you go, a conversion with the default options. You can also set the codecs and container to your heart's content.

You can also use encvid (or undvd for that matter) to cut some segment of a video with the --start and --end options. :)

I'm sold, where can I buy it?

how to pick a codec

September 10th, 2008

The great thing about standards is there are so many to choose from.
- Someone

undvd 0.5.0 introduced a new option to choose the codec and container for the rip. The only problem is that you have to know which ones to choose. mencoder supports a staggering number of codecs and containers, most of which are now exposed also in undvd. The resulting rip can also be remuxed to a couple of other popular containers with additional tools.

But I wasn't content with solving a problem by introducing a new problem. Now, it's not so easy to say exactly which combinations are good and bad, but if at least you knew which ones definitely do not work, that would be a start, wouldn't it? Then at least you can rescue the user from phase one of the Monte Carlo method in getting something that actually works.

The methodology is like this:

  1. Rip 5 seconds of the dvd using undvd with a given container/video codec/audio codec combination.
  2. Attempt playback with mplayer.

This is what codectest does. The result is either a text file showing line by line whether or not the given combination successfully produced a rip, or a pretty matrix picture. This gives you an idea of what you can expect to use. If you run this on your system, it's also a tip off if you see something that should work but doesn't.

I must stress that if the given combination of codecs does produce a file, this is no guarantee that the file is to be considered a good rip. It may not play on other media players, it may not even play on mplayer (incidentally, this is something akin to a fuzzer, I've discovered that some combinations really aren't expected :D ). So if codectest says it works, verify that you get a working video file out of it!

The standard set looks something like this:

It's also possible to run it on the full combination of all codecs and containers that are now exposed in undvd. You'll need a few hours to do it:

of codecs and containers

September 8th, 2008

I have been very skeptical about adding options for other codecs in undvd, purely because of the test burden. With a single combination of container and pair of audio/video codecs I can be reasonably confident that I've done enough manual testing (and judging video quality doesn't trivially lend itself to automated testing, sadly) to account for most potential problems.

But at the end of the day it's a question of priorities, and having scratched all the important technical itches by now, if anything this is the right time for it. I got some user feedback recently that set me onto this path. The user was having trouble playing the files encoded in the classical avi+h264+mp3 format on other platforms, and that's when I asked myself how important is it really to have a single format? As long as the default still works well, what's the harm in offering a little customization?

Testing is a huge problem, which is why this new feature is considered to be experimental. The most common seems to be bad a/v sync. There is just no way to account for all the possible combinations of codecs and containers, and to maintain an up-to-date document for this as things evolve. So the burden of testing is squarely on the user here (which is quite unfortunate).

The new functionality is available in undvd 0.5 and up. Here's a shot of the new goodness. All these files were encoded from the same dvd title. A 22 minute title was ripped with different containers (represented with different filenames). The audio codec is mostly the same in all cases (mad = mp3), except for 1.mp4 (faad = aac). The video codec is also mostly the same (h264 = avc1), except for 1.flv. The only variation here is the container being set to different values, all the other settings are defaults. You can also witness that some containers are more wasteful than others (given the same a/v streams), but not by a huge amount. (The audio bitrates shown are actually misleading, mplayer seems to give the lowest bitrate in a vbr setting.)

This demo is by no means exhaustive of the full collection of codecs that can be used, for that see the user guide. There is also an option to use the copy codec, which just copies the audio/video stream as is.

coast to coast

September 7th, 2008

So you wanna drive coast to coast huh? That's what people say anyway, "oh how romantic, all those small towns, the landscapes".

Some people have a dream of driving in the US. I guess they relish an exciting drive across the Bible Belt and then desert country.

According to Google, you can do New York-LA in a day and 17 hours, provided you pick the shortest (and fastest?) roads and drive non-stop.

What about coast to coast somewhere closer to home? Every summer a bunch of tourists pack into their cars and drive up to the north or Norway for a relaxing road trip on our narrow and windy roads.

The scale here is 1:2.


I had to put in two extra pins on the map, or Google would send me into Sweden. But there's your Lindesnes-Nordkapp connection.

It turns out the East-West coast span of the US isn't even twice as wide as our North-South run, and you can do it in roughly the same amount of time.

So basically if you've done North-South in Norway you've done more than half the distance across the US. Doesn't sound that impressive at all anymore, does it? :D

Ps. In the Netherlands you can do Maastricht-Groningen in 3 hours, it's like a paper route. :howler:

a coder's bookshelf

August 30th, 2008

What is this obsession people have with books? They put them in their houses - like they're trophies. What do you need it for after you read it?
- Jerry Seinfeld

I think it's because reading a book takes a lot of effort, and we want to get credit for it. Reading a big book takes considerably longer than anything else you might do for "fun". And then you can point to it and say, "look, this is what I know".

I have a bunch of computer books, a lot of them from college, that I'll probably never toss out even though I'm unlikely to ever re-read them. Meanwhile, I can do what a lot of people are doing and put them on display. "Look, I must be really clever, I have all these books!"

Frankly that's all they're good for after I'm done with them.