🧯 Bash Error Handling in Depth

Bash error handling in depth: set -e, pipefail, traps and exit codes
set -Eeuo pipefail; trap cleanup EXIT

Most shell scripts are written for the day everything works. The scripts that matter are the ones that run at 03:00 from cron, with nobody watching, on the day a disk is full or a server does not answer. Whether they stop, carry on and do damage, or leave a half-finished mess depends on a handful of rules that are easy to get wrong: what set -e really does, why a pipeline hides failures, what a trap can and cannot catch, and what the exit status means.
This page does not repeat the basics from the Bash Programming Tutorial (Lesson 10 introduces set -Eeuo pipefail and trap). It goes deeper, and it is built on experiments: every behaviour below was run for real with GNU bash 5.2.21 on Ubuntu 24.04, and the output shown is what came back. Several results are surprising, and that is the point.

Every command ends with an exit status from 0 to 255, available as $? right afterwards. Zero means success, anything else means failure, and the value says what kind. if, while, &&, || and set -e all act on this number, so it is the only error channel a script really has.
  0       success (also "found" for grep, "true" for test)
  1       general failure; for grep and diff also "no match" and "differences found"
  2       misuse of a command (bad option or argument); grep uses it for real errors such as an unreadable file
  124     timeout killed the command
  126     found but not executable
  127     command not found
  128+N   killed by signal N: 130 is Ctrl+C (INT, 2), 137 is kill -9 (KILL, 9), 143 is the default kill (TERM, 15), 141 is a broken pipe (PIPE, 13)
Here they are for real:
$ bash -c './noexec.sh'
bash: line 1: ./noexec.sh: Permission denied
[exit 126]
$ bash -c 'nosuchcommand'
bash: line 1: nosuchcommand: command not found
[exit 127]
$ bash -c 'sh -c "kill -9 \$\$"; echo "child killed, status $?"'
bash: line 1: 12078 Killed                  sh -c "kill -9 \$\$"
child killed, status 137
$ bash -c 'timeout 1 sleep 5'; echo "[exit $?]"
[exit 124]
$ bash -c 'grep -q zzz /etc/hostname'; echo "[exit $?]"
[exit 1]
$ bash -c 'ls /nonexistent_dir 2>/dev/null'; echo "[exit $?]"
[exit 2]
Two surprises when you choose your own numbers: only the low 8 bits count, so a value above 255 wraps around, and a negative value wraps the other way:
$ bash -c 'exit 256'; echo "[exit $?]"
[exit 0]
$ bash -c 'exit -1'; echo "[exit $?]"
[exit 255]
exit 256 is a success. A script that computes an exit code from a count (the number of failed files) must cap it: exit $(( n > 255 ? 255 : n )). Pick numbers below 126 for your own meanings, so they cannot be confused with "not executable", "not found" or a signal.

set -e tells the shell: if a simple command fails, exit at once, with the same status. The classic demonstration:
$ cat e1.sh
#!/bin/bash
set -e
echo start
false
echo "not reached"
$ bash e1.sh; echo "[exit $?]"
start
[exit 1]
That is the useful half. The other half is that set -e has a long list of exceptions, so it cannot be relied on as the only safety net. The next two sections show the exceptions for real, and the Bash FAQ's advice follows from them: use set -e as a seat belt, and still check the commands that matter yourself (cmd || die "cmd failed").

A failing command is ignored when the shell is already using its status for a decision: in the condition of if, while and until, as part of a && or || list (except the last command), or when negated with !. All of these ran with set -e on:
$ cat e2.sh
#!/bin/bash
set -e
if false; then :; fi;            echo "1 if condition: continues"
false || echo "2 || list: continues"
false && echo never;             echo "3 left side of &&: continues"
! true;                          echo "4 negated command: continues"
f() { false; echo "5 inside a function called from if: continues"; }
if f; then :; fi
g() { local v=$(false); echo "6 local v=\$(false) hides the failure: continues, \$?=$?"; }
g
h() { local v; v=$(false); echo "7 not reached"; }
h
echo "8 not reached"
$ bash e2.sh; echo "[exit $?]"
1 if condition: continues
2 || list: continues
3 left side of &&: continues
4 negated command: continues
5 inside a function called from if: continues
6 local v=$(false) hides the failure: continues, $?=0
[exit 1]
Read the list carefully:
  1-4 are the documented exceptions. Note that "false && echo never" does not stop the script, because the && list as a whole "succeeded" in the sense that the failing part was tested.
  5 is the nasty one. When a function is called as a condition (if f; then), set -e is switched off for the WHOLE function body, not only for the test: the failing "false" inside f was ignored, and the next line ran. The same applies to f && other and f || other. So a function called in an if cannot be protected with set -e at all: check each step inside it, or return explicitly.
  6 shows that "local v=$(command)" always succeeds: the status of the command substitution is replaced by the status of "local", which is 0. The failure is lost.
  7 is the fix: declare first (local v), assign on a separate line (v=$(command)), and set -e catches the failure. Declare-then-assign is the right habit for every local variable that takes its value from a command.
The script ended at 7 with status 1 and never printed the "8" line.

The arithmetic command (( expression )) returns status 1 when the expression equals 0, and 0 otherwise. i++ is a post-increment: it returns the OLD value, so the first time through, with i=0, the command "fails":
$ cat e3.sh
#!/bin/bash
set -e
i=0
((i++))
echo "after ((i++)) with i=0"
$ bash e3.sh; echo "[exit $?]"
[exit 1]
The script ended silently on the increment, with no message. It is a famous source of "my script dies for no reason" reports, usually inside a counting loop. The safe forms:
$ cat e4.sh
#!/bin/bash
set -e
i=0
i=$((i+1))
echo "i=$i (assignment form is safe)"
n=0; ((n++)) || true; echo "n=$n (|| true form)"
$ bash e4.sh
i=1 (assignment form is safe)
n=1 (|| true form)
Use i=$((i+1)) (an assignment is never a failure), or ++i, or ((i++)) || true. The same applies to any (( )) used as a statement whose value can be zero: ((count)) and ((total -= x)) can also return 1.

set -u makes the use of an unset variable an error instead of an empty string. It turns a typo ($filname) into a loud failure, and protects against the classic "rm -rf $DIR/" when DIR is empty:
$ cat u1.sh
#!/bin/bash
set -u
name=${1:-world}
echo "hello $name"
echo "value: $undefined_var"
$ bash u1.sh; echo "[exit $?]"
hello world
u1.sh: line 5: undefined_var: unbound variable
[exit 1]
The message names the variable and the line. Give optional variables a default with ${var:-default} (also covers a variable that is set but empty) or ${var-default} (only when unset); an empty default is legal: ${var:-}. To require an argument with a custom message, use ${1:?message}:
$ cat u2.sh
#!/bin/bash
: "${1:?usage: u2.sh FILE}"
echo ok
$ bash u2.sh; echo "[exit $?]"
u2.sh: line 2: 1: usage: u2.sh FILE
[exit 1]
In bash 4.4 and newer, an empty array is fine under set -u. In older bash (RHEL 7 has 4.2) "${arr[@]}" of an empty array is an "unbound variable" error; write ${arr[@]+"${arr[@]}"} there.
$ bash -c 'set -u; a=(); echo "empty array ok in bash $BASH_VERSINFO: [${a[@]}]"'
empty array ok in bash 5: []
Testing whether a variable is set, under set -u, needs the default form too: [[ -n ${var:-} ]] (never [[ -n $var ]]).

By default the status of a pipeline is the status of its LAST command, so a failure earlier in the pipe is invisible:
$ bash -c 'false | true; echo "no pipefail: status $?"'
no pipefail: status 0
$ bash -c 'set -o pipefail; false | true; echo "pipefail: status $?"'
pipefail: status 1
With pipefail the pipeline fails if ANY command failed (the status is that of the last one that failed). That is why set -o pipefail is in every script header: without it, "mysqldump db | gzip > backup.gz" reports success when mysqldump died halfway and gzip happily compressed the half-dump.
To see each command's status, read the array PIPESTATUS right after the pipeline (it is overwritten by the next command, so copy it at once: st=("${PIPESTATUS[@]}")):
$ bash -c 'false | true | cat; echo "PIPESTATUS: ${PIPESTATUS[*]}"'
PIPESTATUS: 1 0 0
pipefail has one famous side effect, SIGPIPE. When the reader of a pipe exits early (head -1 has all it wants), the writer is killed by the next write with signal 13, and its status is 141. Under pipefail that counts as a failure:
$ bash -c 'set -o pipefail; yes | head -1; echo "yes|head status: ${PIPESTATUS[*]} -> $?"'
y
yes|head status: 141 0 -> 141
$ bash -c 'set -o pipefail; set -e; yes | head -1; echo "not reached under set -e"'; echo "[exit $?]"
y
[exit 141]
Combined with set -e, a perfectly good "produce a lot, take the first line" pipeline killed the script. Typical victims: cmd | head -n1, cmd | grep -q PATTERN (grep -q exits at the first match and the producer gets SIGPIPE), and ls | head. Fixes: read the first line without a pipe (read -r first < <(cmd)), capture the output once and then test it (out=$(cmd); [[ $out == *PATTERN* ]]), or accept the status for that one pipeline: cmd | head -n1 || true.

Command substitution $(...) and subshells (...) run in a child shell. In bash, errexit is NOT inherited by a command substitution unless you ask for it with shopt -s inherit_errexit (bash 4.4 and newer):
$ cat i1.sh
#!/bin/bash
set -e
x=$(false; echo "command substitution kept going")
echo "1: $x"
shopt -s inherit_errexit
y=$(false; echo "kept going")
echo "2: not reached, y=[$y]"
$ bash i1.sh; echo "[exit $?]"
1: command substitution kept going
[exit 1]
Without inherit_errexit the "false" inside $( ) was ignored and the substitution carried on. With it, the substitution fails at the first error, the assignment fails, and set -e ends the script. Put shopt -s inherit_errexit in the header (guard for old bash: shopt -s inherit_errexit 2>/dev/null || true).
A plain subshell (...) does inherit set -e, and its failure is a failure of the whole ( ) command, so the parent stops too:
$ bash -c 'set -e; (false; echo "inside subshell: not reached"); echo "parent also exited: not reached"'; echo "[exit $?]"
[exit 1]
A variable set inside a subshell or inside the right side of a pipe (cmd | while read ...) does not exist outside it. That is covered on the Bash Gotchas page.

A trap runs a command when a signal arrives or the shell reaches a special event. The three that matter in scripts: EXIT (the shell is ending, for any reason), ERR (a command failed), and signals such as INT (Ctrl+C) and TERM (kill).
EXIT: the cleanup hook. It runs when the script ends normally, when it ends with exit, when set -e stops it, and when a trapped signal makes it exit. Always clean up here, not at the end of the script. Inside it, $? still holds the status the script was about to exit with, so save it first:
$ cat t1.sh
#!/bin/bash
set -e
tmp=$(mktemp -d)
cleanup() { rc=$?; rm -rf -- "$tmp"; echo "cleanup ran (exit status was $rc), tmp removed: $([ -d "$tmp" ] && echo no || echo yes)"; }
trap cleanup EXIT
echo "working in a temp dir"
false
echo "not reached"
$ bash t1.sh; echo "[exit $?]"
working in a temp dir
cleanup ran (exit status was 1), tmp removed: yes
[exit 1]
The directory is gone although the script died on an error, and the exit status of the script is still 1.
ERR: fires when a command fails (with the same exceptions as set -e: conditions and && || lists do not count). It is the right place for a diagnostic: which line, which command. But by default an ERR trap is NOT inherited by functions, command substitutions or subshells. Compare the same script without and with -E (errtrace):
$ cat t2.sh
#!/bin/bash
set -e
trap 'echo "ERR trap: line $LINENO: $BASH_COMMAND (status $?)" >&2' ERR
f() { echo "in f"; false; echo "after false in f"; }
f
$ bash t2.sh; echo "[exit $?]"
in f
[exit 1]
$ sed 's/^set -e$/set -eE/' t2.sh > t3.sh; bash t3.sh; echo "[exit $?]"
in f
ERR trap: line 4: false (status 1)
[exit 1]
Without -E the failure inside the function exited the script with no diagnostic at all. With -E the trap fired and named line 4 and the command "false". That is why the header is set -Eeuo pipefail (the E matters), and why $BASH_COMMAND and $LINENO belong in the ERR trap. Remember that $? inside the trap is the status of the command that failed; save it first if you do anything else.
Signals: a script that is stopped with Ctrl+C or kill should still clean up and should exit with the conventional 128+N status. Without a trap, bash is simply killed by the signal:
$ bash -c 'kill -TERM $$; echo not reached'; echo "[exit $?]"
Terminated
[exit 143]
With traps, the script cleans up and chooses its own exit status. In the first test the script was sent TERM while it waited on a child; in the second it sent itself INT:
$ cat t4.sh
#!/bin/bash
trap 'echo "got TERM, cleaning up"; exit 143' TERM
trap 'echo "EXIT trap"' EXIT
echo "sleeping"
sleep 30 &
wait $!
$ bash t4.sh & P=$!; sleep 0.5; kill -TERM $P; wait $P; echo "[exit $?]"
sleeping
got TERM, cleaning up
EXIT trap
[exit 143]
$ cat t5.sh
#!/bin/bash
trap 'echo "caught INT"; exit 130' INT
echo "waiting"
kill -INT $$
sleep 5
echo "not reached"
$ bash t5.sh; echo "[exit $?]"
waiting
caught INT
[exit 130]
Note that the TERM handler ended with exit 143, which ran the EXIT trap as well: both ran, in that order. A trap that does not call exit lets the script continue after the signal, which is occasionally what you want and usually a bug. Two details that bite: bash runs a trap for a signal only after the foreground command ends, so a long sleep delays the handler (run long waits in the background and use wait, as above, because wait returns at once when a trapped signal arrives); and KILL (kill -9) and STOP cannot be trapped, so nothing runs after them: leftover lock files and temp files from a kill -9 need a cleanup at the NEXT start (check whether the PID in the lock file still exists).
Other trap rules: a new trap for the same signal replaces the old one (add to it, do not overwrite it blindly); trap - EXIT removes it (do this after a successful finish if the trap should not run, for example when a temp file was meant to be kept); the traps are reset in subshells; and trap -p lists the current ones.

Error messages go to standard error (>&2), so they are not captured in the output a caller is piping or saving. Put the script name in the message, say what failed and why, and exit with a non-zero code. A small helper makes this one line at every call site. The first version of this helper, written as die "message" CODE, had a bug that the test found at once: "$*" includes the code and prints "...argument 2". The exit code must come FIRST:
$ cat d2.sh
#!/bin/bash
set -Eeuo pipefail
die() { local rc=$1; shift; printf '%s: error: %s\n' "${0##*/}" "$*" >&2; exit "$rc"; }
[[ $# -ge 1 ]] || die 2 "missing FILE argument"
[[ -r $1 ]]   || die 1 "cannot read '$1'"
echo "reading $1"
$ bash d2.sh; echo "[exit $?]"
d2.sh: error: missing FILE argument
[exit 2]
$ bash d2.sh /nonexistent; echo "[exit $?]"
d2.sh: error: cannot read '/nonexistent'
[exit 1]
$ bash d2.sh /etc/hostname; echo "[exit $?]"
reading /etc/hostname
[exit 0]
Conventions: 2 for wrong usage (like most tools), 1 for a general failure, and small distinct numbers (3, 4, 5...) for failures a caller may want to tell apart (not found, lock held, remote unreachable); document them in the usage text. Use printf instead of echo for messages that contain variables, so a value starting with - or containing backslashes cannot be taken as an option.
Do not leave the script without a message: a script that exits 1 with no output is a puzzle for the person on call. And do not print the error and then carry on as if nothing happened: return or exit.

Networks and APIs fail briefly. Retrying a few times with a growing delay turns many "incidents" into log lines. Keep the number of attempts small and bounded, and report the last failure:
$ cat r1.sh
#!/bin/bash
retry() {
  local max=$1 delay=$2 n=1; shift 2
  until "$@"; do
    if (( n >= max )); then echo "giving up after $n attempts: $*" >&2; return 1; fi
    echo "attempt $n failed, retrying in ${delay}s" >&2
    sleep "$delay"; n=$((n+1)); delay=$((delay*2))
  done
}
count=0
flaky() { count=$((count+1)); [ "$count" -ge 3 ]; }
retry 5 1 flaky && echo "succeeded on attempt $count"
retry 2 1 false || echo "retry returned $?"
$ bash r1.sh
attempt 1 failed, retrying in 1s
attempt 2 failed, retrying in 2s
succeeded on attempt 3
attempt 1 failed, retrying in 1s
giving up after 2 attempts: false
retry returned 1
The command is passed as separate words ("$@"), so quoting is preserved: retry 5 2 curl -fsS "$url" -o "$out". The delay doubles (1 s, 2 s, 4 s...). Because the command is the condition of until, set -e does not trigger inside it, which is exactly what you want here. Only retry operations that are safe to repeat (a download, a read, an idempotent API call), never "send the email" or "charge the card".

A job started with & does not stop the script when it fails, and set -e never sees it. Its status is available only through wait PID:
$ cat b1.sh
#!/bin/bash
set -u
(sleep 0.2; exit 3) & p1=$!
(sleep 0.1; exit 0) & p2=$!
wait "$p1"; echo "job 1 status: $?"
wait "$p2"; echo "job 2 status: $?"
$ bash b1.sh
job 1 status: 3
job 2 status: 0
To run several jobs in parallel and fail if any of them failed, wait for each one and count:
$ cat b2.sh
#!/bin/bash
fail=0
for n in 1 2 3; do ( sleep 0.$n; [ $n -ne 2 ] ) & done
for p in $(jobs -p); do wait "$p" || fail=$((fail+1)); done
echo "failed jobs: $fail"
$ bash b2.sh
failed jobs: 1
Job 2 was the one that failed, and the loop found it. A bare "wait" with no arguments waits for everything but always returns 0, which hides failures: always wait by PID when you care. Limit the parallelism (xargs -P N or a counter with wait -n) so the script does not start a thousand jobs at once, and trap EXIT/INT to kill the children (kill 0 or kill $(jobs -p)) so Ctrl+C does not leave orphans.

set -x prints every command before it runs, after expansion, so you see what the shell actually did. The prefix is the variable PS4; the default "+ " is not much help, so add the file and line:
$ cat x1.sh
#!/bin/bash
PS4='+ ${BASH_SOURCE##*/}:${LINENO}: '
set -x
name="a b"
echo "$name"
set +x
$ bash x1.sh
+ x1.sh:4: name='a b'
+ x1.sh:5: echo 'a b'
a b
+ x1.sh:6: set +x
The trace goes to standard error, shows the value after expansion (note 'a b' quoted), and set +x switches it off, so you can trace just the part you suspect. bash -x script.sh traces the whole script without editing it, and bash -n script.sh only parses it (it catches syntax errors but runs nothing). In an ERR trap, $BASH_COMMAND, $LINENO, ${FUNCNAME[@]} and ${BASH_LINENO[@]} give the failing command, line and call chain. Do not leave set -x in a script that handles secrets: the trace prints passwords and tokens in the clear.

This header gives you most of the safety with little cost:
  #!/usr/bin/env bash
  set -Eeuo pipefail
  shopt -s inherit_errexit 2>/dev/null || true
  IFS=$'\n\t'
  trap 'echo "error: line $LINENO: $BASH_COMMAND" >&2' ERR
  tmp=$(mktemp -d); trap 'rm -rf -- "$tmp"' EXIT
-E lets the ERR trap work in functions, -e stops on errors, -u catches unset variables, -o pipefail catches failures inside pipelines, inherit_errexit extends -e into $( ). Setting IFS to newline and tab (without the space) makes word splitting less dangerous for unquoted variables, but quote your variables anyway.
Be honest about what it does not do. set -e does not catch: a failing command in an if condition or && || list, a failing function called as a condition, local x=$(cmd), a failing background job, an error in the left side of a pipe without pipefail, and most (( )) edge cases. It also behaves differently in different shells and versions, which is why serious scripts also check the important steps by hand:
  cp -- "$src" "$dst" || die 1 "cannot copy $src to $dst"
  data=$(curl -fsS "$url") || die 3 "download failed: $url"
  if ! mv -- "$tmp" "$final"; then die 1 "cannot replace $final"; fi
Explicit checks give a precise message and a chosen exit code, and they work in every case above. Use set -e as the net under them, not instead of them.

The script stops with no message                     set -e fired on a command whose failure you did not expect (grep with no match returns 1; ((i++)) with i=0; a pipe closed early with pipefail). Run it with bash -x, or add the ERR trap above
The script does NOT stop although a command failed   the command was in an if or && || list, a function called as a condition, local v=$(cmd), a background job, or the left side of a pipe without pipefail
"unbound variable"                                   set -u found an unset variable: fix the typo, give a default ${var:-}, or check the arguments first
Status 141 after a pipeline with head or grep -q     SIGPIPE under pipefail: capture the output first, or use read -r line < <(cmd)
Temp files are left behind                           the cleanup is at the end of the script instead of in a trap EXIT, or the script was killed with kill -9 (clean up stale files at the next start)
The ERR trap does not fire inside functions          add -E (set -E or set -Eeuo pipefail)
$? is wrong in the trap                              another command ran first: assign rc=$? as the very first thing in the handler
Exit code 0 although something failed               exit 256 wraps to 0, or the last command in the script succeeded (the script returns the status of its last command: end with an explicit exit)
Ctrl+C leaves child processes running                the traps do not kill the children: trap 'kill 0' INT TERM (or kill $(jobs -p)) and wait for them
The retry loop never ends                            no maximum number of attempts or no timeout on the command: always bound both

Treat every exit status as information: 0 is success, 1 and 2 are errors, 126 and 127 are launch problems, 128+N is a signal. Pick your own codes below 126.
Use set -Eeuo pipefail in the header, shopt -s inherit_errexit, an ERR trap for diagnostics and an EXIT trap for cleanup.
Declare and assign separately: local v; v=$(cmd). Increment with i=$((i+1)).
Check the steps that matter yourself with || die; do not rely on set -e alone.
Send errors to stderr with the script name, exit non-zero, and never fail silently.
Wait for background jobs by PID, and bound every retry.
Test the failure paths on purpose: run the script with a missing file, an unreadable directory, a full disk and kill it with Ctrl+C. The error handling you did not test is the error handling you do not have.