7 things I wish I knew about Shell Scripting

Some not-so-obvious stuff I wish I didn’t have to learn the hard way

Generated with Canva

1. Tooling

Before you do any scripting, install these must have tools:

• VS Code: Seriously, don’t use the default text editor to write shell script
• ShellCheck: A powerful static analysis tool. Available standalone or more preferably as a VS Code extension
• shell-format: Code formatter for VS Code
• Code Spell Checker: spell checker for VS code

Feel free to try these optional VS Code extensions out:

• Bash Debug
• shellman

2. Types

Shell variable types work different than what you might be used to with languages like C or even Python. Every variable is a STRING, until you use it in a context where the shell expects it to be a number.

e.g. comparing “02” with “2” gives different results depending on where you compare them as integers or strings:

# String comparison
> [ "02" = "2" ]; echo $?
1

# Integer comparison
> [ "02" -eq "2" ]; echo $?
0

3. Globing

In the following contrived example we can see a subtle difference in the results:

# When globbing kicks in
> find . -name *.jpg
./root.jpg

# When globbing is suppressed
> find . -name "*.jpg"
./root.jpg
./test/hidden.jpg

You need to be very careful with “filename expansion” (a.k.a globbing). Remember to use quotes to protect against the shell reinterpreting special characters and remember to NOT do so when you actually want globbing to kick in.

4. Variable assignment and error handling

> VAL=$(false) || echo "This runs if the left side fails"
This runs if the left side fails

If you know enough to be dangerous with shell scripts, you might be very confused as to how “This runs if the left side fails” gets printed. Surely the VAL assignment would never produce a non-zero exit code right? WRONG! assignment actually does not have ANY status, thus the left side of the ||takes the exit status of the subshell in this example.

Thus the following pattern can be used to capture both the stdout and return code of a command in one line.

ret_status=0
ret_value=$(COMMAND) || ret_status=$?

if [ $ret_status -ne 0 ]; then
   echo "Error running COMMAND"
   exit 1
fi

echo $ret_value

5. Return value of a pipe is the return value of the last command

# Command sequenc dosen't fail if the last command didn't fail
> false | false | true
> echo $?
0

# It would only fail if the last command in the pipe failed
> true | true | false
> echo $?
1

Use set -o pipefail to disable this behavior and force your piped command sequence to fail if any of the commands in a pipe fail.

6. EXIT traps

A.k.a don’t trust function names!

#!/bin/bash -e

ErrorHandler()
{
    # some clean up logic
    exit 0
}
trap ErrorHandler EXIT

# The actual script's logic

Don’t be fooled by the ErrorHandler name! That function will run for both a normal exit and if the script terminates early due to an error. Read more about traps here.

7. Output redirection

Order matters!

# Direct both stdout and stderror to a file
$ somecmd >my.file 2>&1

# Direct stdout to a file and stderror to screen
$ somecmd 2>&1 >my.file

Bonus tip: Don’t use Bash scripts

Sometimes Bash scripts are not the right tool for the job! Especially for complex tasks.

An alternative like Python will be less error prone thanks to better tooling and unit test support.