One of the benefits of coding in a scripting language is the ability to easily move the code around and it runs everywhere. undvd is easy to install, you just extract the source and it's ready to go. Or, if you want to install it globally on the system, just run make install and you're set.

But undvd depends on a bunch of other tools, so what you really want is to let your package manager deal with satisfying the dependencies, and let you upgrade easily when a new version comes out.

Where can I find undvd?

I package undvd in three flavors myself: Gentoo, Fedora and Ubuntu. I run Gentoo and Ubuntu on my boxen, so those are the most well tested. In addition, ripdvd's author wedge does the honors for Slackware. :cool:

But there are some unofficial packages out there as well, so when you add it all up undvd is pretty well supported at this point.

• Arch :: aur
• Fedora :: sf.net
• Gentoo :: portage / sf.net
• Slackware :: sf.net
• Suse :: packman.links2linux.org
• Ubuntu :: ppa / sf.net

Making friends with posix

Recently, a user expressed an interest in running undvd on a Mac, which is a whole new angle. I guess it never occurred to me that someone would use undvd outside linux, but why not. The recent switch to perl has obsoleted the many external tools that undvd previously relied on, which is a big [if unplanned] step towards portability. I ran some tests that revealed there were still a couple linuxisms left, so I've gotten rid of them now.

undvd-0.7.1 should in principle run on a Posix system with perl-5.8.8. The big thing isn't undvd, it's getting mplayer and the other binaries, cause they have to be built specifically for that platform. And outside linux, there's sometimes no standard package manager, which doesn't help at all.

At the end of the day, there is no escaping the fact that undvd on Posix is a fish out of water (true not only of undvd but probably most free software) - the environment just isn't as well supported as it is in a linux distribution that ships all the tools. There are no packages for any of these, but you can just extract the tarball and run undvd from whatever location.

DesktopBSD 1.6 (FreeBSD Ubuntufied)

It runs (except mencoder can't find x264, so I used xvid). The dvd device is called /dev/cdrom or /dev/acd0. Cloning with dd fails, vobcopy works.

OpenSolaris 2008.05

It runs (except mencoder can't find x264, so I used xvid). I have no idea how the device naming works, but when you put in a cd/dvd it should automount and you can look up the dvd device in the mount table. Packages for mplayer are available from Blastwave. Playback with mplayer worked, encoding segfaulted mencoder.

Mac OS X Leopard

It runs. The dvd device is called something like /dev/rdisk1. Mac people seem to be obsessed with guis, so any builds I could find only had mplayer playback, no encoding. Your best bet is probably MacPorts, modeled loosely on FreeBSD's ports. You'll want to build mplayer with the codecs you want selected as variants (think Gentoo USE flags). Disc cloning doesn't seem to work at all.

To install globally you'll want something like DESTDIR=/opt/local sudo make install.

Cygwin

It runs, and you can get builds of mplayer from the official site. The dvd device is called d: (or some other drive letter). Set PATH=/cygdrive/c/mplayer:$PATH and make sure c:\mplayer doesn't contain both a directory mplayer/ and a binary mplayer.exe, this makes cygwin very confused. On the whole it seems to work quite well, the process even successfully sets the cpu priority (nice).

So it turns out you can do a whole lot with bash. More than I knew. But when you get to a point where you start hitting the limitations of your language*, it gets frustrating. The biggest problem with bash is that it doesn't have functions. You can wrap a bunch of code and call it with arguments, but it doesn't return a value. I've tried to come up with a hack to emulate functions returning arguments, but in the end there just aren't enough pieces in the box to build it from.

To date, undvd has been using various tricks to get around this. Let the function echo the value and capture this in the caller. But then what if you have a failing condition? Well, you can echo the value to stdout and echo the error to stderr, so it doesn't get captured as the result of the function. And then kill $$ to force an exit (you can't just exit cause that is equivalent to a return from the function).

That kind of works, but eventually you break down when you have to return more than one string that may contain whitespace. Sure, you could quote them, let the caller find both strings based on where the quotes are, then chop off the quotes and voila. But all this just for a function call? It's too much, and it's unacceptable from a maintenance standpoint.

Bash's overall weak support for other features of a typical programming language makes it a challenge to write structured programs. undvd-0.6 is therefore pretty much a dead end from a development standpoint. It works well enough, but it's hard to get anything more out of bash. In order to keep evolving, undvd needs a new language.

Another substantial problem with bash is that you're executing commands in the shell, in other words you build execution strings. There is a lot of potential for quoting bugs when you're dealing with filenames that have spaces and quotes in them. And not just when feeding them as arguments to executables, but on every "function call" just the same. I've spent a lot of time trying to safeguard against this, but all it takes is one instance where the strings aren't quoted correctly, and you have a fatal parsing error.

So it's time to think about porting to another language. A language that is close to the shell. A language that lets you run a subprocess by passing in the arguments as a list, not a string. A language that has basic programming constructs, like functions. That has good string handling. That can do simple floating point arithmetic. That is as widely available as possible. A language like... perl.

It sounds absurd, doesn't it? Porting to perl in the name of maintainability. But when you're in bash and most of what you're doing is string manipulation and calls to other executables, it's the right choice. And I bet you have perl on your box.

Not that it hasn't been fun. Bash was the right place to start, and I've learned a lot of things about bash on the way. I've also learned that you have to do obscene things like echo strings to bc to do simple floating point math.

The port

It's a straight port. undvd 0.7 runs on perl, but the way it was written was to reproduce 0.6 exactly. The code is completely new, obviously, but the functionality is the same.

As a result of running in perl, all the string/numerical processing logic has been internalized, and all the calls to awk, sed, bc and so on are gone. This makes it run faster, scandvd is especially noticeable. This isn't a big impact, since most of the work is done in mencoder, and that is still the same. Nevertheless, it's a welcome side effect. It also makes me happier, since it's less dependent on all these outside tools.

In terms of size the code is about the same. The perl code is actually 5% bigger.

What this means for you

• 0.6 and 0.7 are functionally equivalent.
• If you find a bug in 0.6, it's probably also in 0.7.
• If you find a bug in 0.7, try 0.6.
• Please report bugs.

* I'm using the term "language" loosely here. I'm talking about both the language, and the implementation, and the execution environment (ie. standard libraries, or in bash's case the gnu userland). Often we just pile all of this under "language", because it's easier to talk about it that way.

When I started undvd I set out to solve one very specific, yet sizeable, problem: dvd ripping&encoding. I did that not because I really felt like diving head first into the problem would be fun, but because there was nothing "out there" that I could use with my set of skills (none). Meanwhile, I needed a dvd ripper from time to time, and since I didn't need it often I would completely forget everything I had researched the last time I had used one. This was a big hassle, I felt like I had no control over the process, and I could never assure myself that the result would be good. Somehow, somewhere, there was a reason why all my outputs seemed distinctly mediocre. Visibly downgraded from the source material.

Writing undvd was a decent challenge in itself, because of all the complexity involved in the process. I had to find out all the stuff about video encoding that I didn't really care about, but I thought if I put it into undvd, and make sure it works, then I can safely forget all about it and just use my encoder from that point on. When you start a project you really have no idea of where it's going to end up. undvd has evolved far beyond anything I originally set out to build. That's just what happens when you add a little piece here and another piece there. It adds up.

It's been about 20 months. undvd is quite well tested and has been "stable" (meaning I don't find bugs in it myself anymore) for over a year. One of the by products is a tool called vidstat for checking properties of videos. I wrote that one just so I could easily check the video files undvd was producing. But it turns out to be useful and I use it all the time now (way more than undvd). In the beginning I was overwhelmed by the number of variables that go into video encoding, and I wanted to keep as many of them as I could under tight control. I have since backtracked on a number of features I initially thought would be a really bad idea for encoding stability. But that's just the way code matures, you start with something simple and when you've given it enough thought and enough tests, you can afford to build a little more complexity into the code.

Codec selection landed just recently. And once I was done scratching my head and trying to decide which ones to allow and/or suggest, I suddenly realized that with this last piece of the puzzle I was a stone's throw away from opening up undvd to general video conversion. Urgently needed? Not really. But since it's so easy to do at this point, why not empower?

The new tool is called encvid. It works just like undvd, stripped of everything dvd specific. It also doesn't scale the video by default (generally in conversion you don't want that). So if you've figured out how to use undvd, you already know how to use encvid, you dig? :cap:

Demo time

Suppose you want to watch a talk from this year's Fosdem (which incidentally, you can fetch with spiderfetch if you're so inclined). You get the video and play it. But what's this? Seeking doesn't work, mplayer seems to think the video stream is 21 hours long, that's obviously not correct (incidentally, I heard a rumor that ffmpeg svn finally fixed this venerable bug). It seems a little heavy handed, but if you want to fix a problem like this, one obvious option is to transcode. If the source video is good quality, at least from my observations so far, the conversion won't noticeably degrade it.

So there you go, a conversion with the default options. You can also set the codecs and container to your heart's content.

You can also use encvid (or undvd for that matter) to cut some segment of a video with the --start and --end options. :)

I'm sold, where can I buy it?

• Gentoo: undvd-0.6.0 is already in portage :cool:
• Ubuntu: use my personal package archive (small apt repository)
• Rpm/others: grab it from sourceforge

The great thing about standards is there are so many to choose from.
- Someone

undvd 0.5.0 introduced a new option to choose the codec and container for the rip. The only problem is that you have to know which ones to choose. mencoder supports a staggering number of codecs and containers, most of which are now exposed also in undvd. The resulting rip can also be remuxed to a couple of other popular containers with additional tools.

But I wasn't content with solving a problem by introducing a new problem. Now, it's not so easy to say exactly which combinations are good and bad, but if at least you knew which ones definitely do not work, that would be a start, wouldn't it? Then at least you can rescue the user from phase one of the Monte Carlo method in getting something that actually works.

The methodology is like this:

1. Rip 5 seconds of the dvd using undvd with a given container/video codec/audio codec combination.
2. Attempt playback with mplayer.

This is what codectest does. The result is either a text file showing line by line whether or not the given combination successfully produced a rip, or a pretty matrix picture. This gives you an idea of what you can expect to use. If you run this on your system, it's also a tip off if you see something that should work but doesn't.

I must stress that if the given combination of codecs does produce a file, this is no guarantee that the file is to be considered a good rip. It may not play on other media players, it may not even play on mplayer (incidentally, this is something akin to a fuzzer, I've discovered that some combinations really aren't expected :D ). So if codectest says it works, verify that you get a working video file out of it!

The standard set looks something like this:

It's also possible to run it on the full combination of all codecs and containers that are now exposed in undvd. You'll need a few hours to do it:

I have been very skeptical about adding options for other codecs in undvd, purely because of the test burden. With a single combination of container and pair of audio/video codecs I can be reasonably confident that I've done enough manual testing (and judging video quality doesn't trivially lend itself to automated testing, sadly) to account for most potential problems.

But at the end of the day it's a question of priorities, and having scratched all the important technical itches by now, if anything this is the right time for it. I got some user feedback recently that set me onto this path. The user was having trouble playing the files encoded in the classical avi+h264+mp3 format on other platforms, and that's when I asked myself how important is it really to have a single format? As long as the default still works well, what's the harm in offering a little customization?

Testing is a huge problem, which is why this new feature is considered to be experimental. The most common seems to be bad a/v sync. There is just no way to account for all the possible combinations of codecs and containers, and to maintain an up-to-date document for this as things evolve. So the burden of testing is squarely on the user here (which is quite unfortunate).

The new functionality is available in undvd 0.5 and up. Here's a shot of the new goodness. All these files were encoded from the same dvd title. A 22 minute title was ripped with different containers (represented with different filenames). The audio codec is mostly the same in all cases (mad = mp3), except for 1.mp4 (faad = aac). The video codec is also mostly the same (h264 = avc1), except for 1.flv. The only variation here is the container being set to different values, all the other settings are defaults. You can also witness that some containers are more wasteful than others (given the same a/v streams), but not by a huge amount. (The audio bitrates shown are actually misleading, mplayer seems to give the lowest bitrate in a vbr setting.)

This demo is by no means exhaustive of the full collection of codecs that can be used, for that see the user guide. There is also an option to use the copy codec, which just copies the audio/video stream as is.