beware of shell expansion

July 24th, 2008

This is one of those details that will bite you if you don't know about it and you might struggle to find the answer for something that looks like a bug.

The characters * and ? have special meaning in the shell, as you know. * means zero or more characters while ? means any character. Favorably, these glob patterns are still the same for those of us who go back to the murky days of Ms DOS.

Glob patterns are very useful in the shell, but when used with programs that themselves accept glob patters the results may be surprising. For example:

$ ls
file2.zip  file.zip
$ find -iname *.zip
find: paths must precede expression
Usage: find [-H] [-L] [-P] [path...] [expression]
$ find . -iname *.zip
find: paths must precede expression
Usage: find [-H] [-L] [-P] [path...] [expression]

Huh? To see what's happening here, run the shell in echo mode:

$ bash -x
$ find -iname *.zip
+ find -iname file2.zip file.zip
find: paths must precede expression
Usage: find [-H] [-L] [-P] [path...] [expression]

So now we see what's happening. *.zip was expanded because it matched two files in this directory. Those two files were then passed as arguments to find, not the *.zip pattern.

Bash will expand the glob pattern whether or not you use it with find, so you just have to tell it not to expand it:

$ find -iname '*.zip'
+ find -iname '*.zip'
./file2.zip
./file.zip

This is confusing, because if *.zip doesn't match any files, then it will be sent verbatim to the program. Therefore you should always quote your glob patterns if they are meant for a program, not the shell.

:: random entries in this category ::

4 Responses to "beware of shell expansion"

  1. Brian says:

    I never knew about echo mode for bash. That's awesome.

  2. james says:

    Brian: that would be the echo mode of korn, bourne and c shells, which bash has chosen to implement.

    also, your find examples rely on gnu find behaviour, viz. the path being optional, this is contrary to the single unix specification: http://www.opengroup.org/onlinepubs/000095399/utilities/find.html
    this also applies to -iname, though that is implemented outside of gnu find, but still not part of susv3

  3. numerodix says:

    Yeah, the gnu <> unix situation is really a pain. Nothing works the same outside of linux.

  4. Shawn Hood says:

    I'm a little late to the party, but here goes: You can also achieve echo mode by executing 'set -x', as opposed to forking another bash shell as shown above. 'set +x' will disable echo mode. Very handy in debugging shell scripts.