How do I know how to get quote right in bash?
I am constantly confused by the quoting and scoring rules when I write bash scripts. I know some of the basics, like the difference between "and" and "", but I still seem to get it wrong too often and boil down to experimenting with different kinds of ways to say the same thing.
Any individual problem that I can usually work out with brute force, but I think my conceptual model of how it works must be hopelessly violated in some unknown way.
I have no problem with the lisp quote, eval, read, print, syntax-quote system. I actually wrote a little kata to help people understand what's going on: http://www.learningclojure.com/2010/11/syntax-quote-kata-for-confused.html
I guess I am looking for something similar for bash (which seems a lot more complicated). A good model or set of exercises to help me shape a model that makes me look at a complex shell script where variables are transformed and evaluated, printed and read, and figure out what's going on so as not to try it.
Otherwise, good methods of debugging the process and observing what happens at each stage of the assessment will be really helpful.
source to share
Bruce Barnett's UNIX Shell Quote Tutorial is awesome, and the Bash FAQ / pitfalls / word splitting "> Articles contain tons of helpful hints. Brief Description:
Unquoted lines can contain most characters, but not all (such as newlines), and many (including space) must be escaped. Just don't use them. If you are tempted, you may find that someone who modified the script forgot to include quotes when they became necessary.
String with a single quote can contain most characters, including NUL and new lines, but not single quotes, so they are also useful only for simple values.
Backticks are for teams. They should only be used if your shell doesn't support $()
. Example:
current_dir=`pwd` # BAD! Don't do this!
This command is bad because when the right side of an assignment is not quoted, the shell performs word splitting on it. This often leads to hard-to-reproduce errors, since spaces are difficult to visually inspect. For command quotes, you need to use double quotes :
current_dir="$(pwd)" # OK, but loses newlines at EOF
Newlines in EOF are especially tricky. You can add one character and split it using for example
# Works for some commands, but not pwd
current_dirx="$(pwd; echo x)"
current_dir="${current_dirx%x}"
printf %s "$current_dir"
but there is an additional difficulty, because some commands (for example pwd
) will add a newline at the end of their output anyway, so you will have to remove that as well:
# Works for some commands, including pwd
current_dirx="$(pwd; echo x)"
current_dir="${current_dirx%$'\nx'}"
printf %s "$current_dir"
Double quotes can contain any character (Try echo -ne "\0" | wc -c
), but note that variables cannot contain a NUL character.
ANSI-C quotes can contain any characters other than NUL (Tryecho -ne $'\0' | wc -c
), and provides convenient output codes to make it easier to work with special characters:
printf %s $'--$`!*@\a\b\E\f\r\t\v\\\'"\360\240\202\211 \n'
printf %q $'--$`!*@\a\b\E\f\r\t\v\\\'"\360\240\202\211 \n'
touch -- $'--$`!*@\a\b\E\f\r\t\v\\\'"\360\240\202\211 \n'
rm -- $'--$`!*@\a\b\E\f\r\t\v\\\'"\360\240\202\211 \n'
source to share
Use single quotes ''
to quote the source text (even backslashes don't escape anything in single quotes):
> echo '\'
\
> echo '$PATH'
$PATH
Use double quotes ""
to quote text that contains things that the shell should evaluate as variables ( $bla
), subnetting ( $(ls)
) and evaluating ( $((5 + 3))
) calls .
> echo "$PATH"
/home/alfe/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
> echo "$(ls | tail -1)"
bla
> echo "$((5 + 3))"
8
Use backticks ``
if for some reason you can't use it $()
(for example, on the rare occasion that you need to sh
instead bash
). Typically use $()
to collect subshell output into the current command.
> echo "$(ls | tail -1) is the last file in the current dir."
bla is the last file in the current dir.
One of the main problems I run into with other people's bash code is that there are no duplicates around something that is often just a word, but in rare cases, there may be more than one word or may contain special characters. Therefore, use double quotes wherever possible.
> a="four spaces"
> echo $a
four spaces
> echo "$a"
four spaces
source to share
On the shell command line
set -x
set -v
and create the following python program 'args.py'
#!/usr/bin/env python
import sys
print sys.argv
for arg in sys.argv[1:]:
for c in arg:
print c,"|",
print
Then experiment with command calls, for example:
U=hello\ world ; V="-->$U<--"; W="1 $U 2 $V 3"; args.py $W
Until you realize that there is no logical way to think about what's going on. It's really all done by the moody magic pixie skins.
source to share