Shell Scripts That Don't Embarrass You at 3 AM
set -euo pipefail, quoting like you mean it, and other bash habits that turn fragile one-liners into scripts you can trust when it matters.
Somewhere in this business there is a script named fix_final_v2_REAL.sh and it will run during your worst week. Every DevOps engineer inherits a few of these — written quickly, works on the author's machine, nobody's sure what line 14 does.
I've written my share. The fix isn't becoming a bash wizard; it's a small set of habits that catch 90% of the foot-guns before they fire.
The header that saves you
Every script I write now starts with:
#!/usr/bin/env bash
set -euo pipefail
Each word earns its place:
-e— exit on any failed command. Without it, your script happily continues after the critical step failed, which is how "backup" scripts report success with nothing in them.-u— treat unset variables as errors. Catches the$DIRtypo that would otherwise expand to empty and hand yourrma terrifyingly broad target.-o pipefail— a pipeline fails if any stage fails, not just the last one. Without it,curl ... | tar -xsucceeds even when curl died, because tar's exit code was fine.
Add trap 'echo "failed on line $LINENO"' ERR and you get the failing line for free. Twenty seconds of setup, endless debugging time saved.
Quote your variables. All of them.
Unquoted variables are the classic foot-gun:
DIR="$1"
rm -rf "$DIR"/* # correct — even if path has spaces
rm -rf $DIR/* # wait, is $DIR one path or four?
If the path is My Projects, the unquoted version operates on My and Projects as separate words — and whatever else those words happen to match. Quote everything that expands: "$var", "$@", "$(command)". Make it muscle memory.
Useful patterns I keep reusing
Fail fast with a usage message:
usage() { echo "Usage: $0 <env> <version>" >&2; exit 1; }
[[ $# -eq 2 ]] || usage
ENV="$1"
VERSION="$2"
[[ "$ENV" =~ ^(dev|staging|prod)$ ]] || usage
Validate inputs before touching anything, especially before touching prod.
Temp files that clean themselves up:
tmp="$(mktemp)"
trap 'rm -f "$tmp"' EXIT
Works whether the script succeeds, fails, or gets Ctrl-C'd.
Idempotency over cleverness. Scripts that can run twice without changing anything the second time are worth more than scripts that are one line shorter:
grep -q "export EDITOR=vim" "$HOME/.bashrc" \
|| echo 'export EDITOR=vim' >> "$HOME/.bashrc"
Re-runnable scripts mean retrying during an incident is safe. That property alone has saved me from writing rollback scripts I didn't need.
Prefer explicit commands over clever one-liners. Future-you reading this during an incident is not in a puzzle-solving mood. Eight readable lines beat three lines of sed wizardry with two flags nobody remembers.
Shellcheck before anyone else sees it
shellcheck myscript.sh
It's free, it's instant, and it catches quoting bugs, unreachable code, and deprecated usage you've gone nose-blind to. If you only adopt one thing from this post, make it set -euo pipefail plus shellcheck in CI.
Know when to stop writing bash
Bash is excellent for gluing commands together. It gets uncomfortable when you're writing: complex data structures, retry logic with backoff, parallelism, or anything with more than ~150 lines. That's Python, Go, or a proper CLI tool's territory — not a moral failing, just the right tool for the job.
Also: no bash in production daemons. Systemd units, containers, and orchestrators exist; a while true; sleep 5 loop with no restart policy is how you get a process that died quietly on Saturday.
The 3 AM test
Before I commit a script, I ask: if this runs during an incident and it's wrong, how wrong can it get? Does it fail loudly? Can it run twice? Does it touch prod only when I meant it to? Would I want a colleague to read it cold?
set -euo pipefail, quotes, validation, traps, shellcheck — five habits, zero frameworks. The scripts I trust most are still the most boring ones I've written.