Bash is not a second citizen programming language. Advices to create a robust bash script.

1. BASH
BASH
is not a second zone citizen programming language
is not a second zone citizen programming language
Agile Grenoble
Agile Grenoble
November 2019
November 2019

3. 2 words about me
2 words about me
$ cat ~/.bash_profile
NAME="René Ribaud"
UPTIME=44
source ./1998.sh
PROFESSION="Unices system and storage"
"administrator , since 2003"
HISTORY="Lots (too many) infrastructure"
"implementation projects"
"Discover Linux & FLOSS between"
"1995 / 2000"
"First step in the cloud around 2011"
", pre-sales solution architect"
"2014 (Cloud, DevOps)"
"I’m an Ops !"
COMPANY="CGI 20th November 2017"
JOB="Information system architect"
"specialized around DevOps technologies"

4. What is Bash ?
What is Bash ?
A command line
interpreter
A Turing complete
language
Default shell for most
Linux distributions
A 30 years old free
software project
https://en.wikipedia.org/wiki/Bash_(Unix_shell)
https://en.wikipedia.org/wiki/Bash_(Unix_shell)

5. Bash ⚠ 💥
Bash ⚠ 💥
Bash does not respect Posix completely and have
bashism
Bashism extends functionalities
Debian is aliasing sh to Dash !
Shell scripts must use Unix file format

6. Why should Bash usage not be underestimated ?
Why should Bash usage not be underestimated ?
Bash is everywhere !
In application startup scripts
In Docker images (entrypoint)
In CI/CD
...
Bash is running critical stuffs !
Managing the system
Monitoring / backups
Packaging artifacts
Checking file integrity and signature
...
So why are bash scripts not written with the same
discipline as other languages ?

7. Pitfall #1
Pitfall #1
#!/bin/bash
# Synchornize fses
# Use this script to migrate fs data to a remote server
# Parameters:
# $1: remote server
# $2: src fs name
# $3: dst fs name
#
# Example to copy all data fses to rmt_server:
# for i in $(mount | grep data | awk '{print $3}')
# do
# pitfall1.sh $i rmt_server:$i
# done
RSERVER=$1
SRC=$2
DST=$3
echo "Synchronize $SRC to $RSERVER..."
rsync -avx --progress --delete /"$SRC" "$RSERVER":/"$DST"

8. Pitfall #2
Pitfall #2
#!/bin/bash
# Backup mysqldb and gzip the content
# Parameters:
# $1: db name
usage() {
echo "Missing parameter DB" && exit 1
}
DBNAME=$1
if [ -z "$DBNAME" ]; then
usage
fi
echo "Backuping $DBNAME..."
mysqldump "$DBNAME" --add-drop-table | gzip -c >"$DBNAME.sql.gz"
if [ $? -ne 0 ]; then
echo "Backup failed"
exit 1
fi

9. How to prevent such pitfalls ?
How to prevent such pitfalls ?
How to make shell scripts more reliable ?
How to make shell scripts more reliable ?

10. Bash strict mode
Bash strict mode
Shebang to not hardcode bash path and use the one specified in PATH
-e immediately exit if any command has a non-zero exit
-u requires variables to be defined
-o pipefail if any command in a pipeline fails, that return code will be
used as the return code of the whole pipeline
IFS Internal Filed Separator set to tab and carriage return
#!/usr/bin/env bash
set -euo pipefail
IFS=$'nt'
http://redsymbol.net/articles/unofficial-bash-strict-mode/
http://redsymbol.net/articles/unofficial-bash-strict-mode/

11. Shell style guide from Google
Shell style guide from Google
https://google.github.io/styleguide/shell.xml
https://google.github.io/styleguide/shell.xml

12. When to not use a shell script
When to not use a shell script
Performance needs
Complex data structures

13. Declarations
Declarations
Immutability
Use declare -r or readonly
Use uppercase for constants
Variable scope
Use local to restrict visible scope to a function
local uses the same options as declare

14. Trap signals
Trap signals
for proper exit
for proper exit
#!/usr/bin/env bash
set -euo pipefail
IFS=$'nt'
_exit() {
echo "Exit time"
date -Iseconds
exit 2
}
trap _exit SIGINT
date -Iseconds
sleep 5m

15. Use [[ ]]
Use [[ ]]
Replace and enhanced test
Bash builtin (not a spawn process)
Allow regexp using =~
if [[ ! -f bin/bfg-1.13.0.jar ]]; then
[[ "$output" =~ "DiG" ]]

16. Function naming
Function naming
Use explicit naming to "auto document" code
Snake case is recommended
Do not use the function keyword

17. Always use functions and a main for complex scripts
Always use functions and a main for complex scripts
...
install_bfg_via_curl() {
if [[ ! -f bin/bfg-1.13.0.jar ]]; then
mkdir -p bin
...
}
main() {
if [[ "${PART}" == "bash" ]]; then
install_tools_via_packages
install_screenkey_via_git
install_bfg_via_curl
backup_regular_file_and_stow bash
source "${HOME}/.bashrc"
elif [[ "${PART}" == "tmux" ]]; then
install_tpm_via_git
backup_regular_file_and_stow tmux
fi
}
main

18. Productivity tools
Productivity tools

19. Linter: Shellcheck
Linter: Shellcheck
A shell script static analysis tool
A shell script static analysis tool
Really, really help writing shells ! 💎 🤩
Can be plugged into editors
https://github.com/koalaman/shellcheck
https://github.com/koalaman/shellcheck

20. Formatter: shfmt
Formatter: shfmt
Format code no headache ! 🔨
Can be configured to respect Google guide using '-i 2 -
ci'
Can be plugged into editors
https://github.com/mvdan/sh#shfmt
https://github.com/mvdan/sh#shfmt

21. Unit test: Bats
Unit test: Bats
Bash Automated Testing System
Bash Automated Testing System
TAP compliant testing
framework (Jenkins
plugin)
Run and skip helper
function to control test
execution
Setup, teardown helper
functions to manage pre
and post test scripts
https://github.com/sstephenson/bats#bats-bash-automated-testing-system
https://github.com/sstephenson/bats#bats-bash-automated-testing-system

22. Bats
Bats
Examples
Examples
#!/usr/bin/env bats
shopt -s expand_aliases
source ${HOME}/.aliases
@test "Test xclip is available for aliases" {
run xclip -h
[[ "$status" -eq 0 ]]
[[ "$output" =~ "Usage: xclip" ]]
}
@test "Test sudo is available for aliases" {
run sudo -h
[[ "$status" -eq 0 ]]
[[ "$output" =~ "usage: sudo -h" ]]
}

23. Test sandbox: Vagrant or Docker
Test sandbox: Vagrant or Docker
Bash can be tested
against a fresh minimal
installation
Help reproduce cases
CI/CD with Travis CI +
Docker for open source
projects

24. GNU parallel
GNU parallel
A tool to simplify executing job in parallel
A tool to simplify executing job in parallel

25. Ripgrep (rg)
Ripgrep (rg)
An application written in Rust
An application written in Rust
A grep enhanced
Compatible with standard grep
Search speed is really impressive
Look also at fd, exa, lsd...

26. The million dollar Guides !
The million dollar Guides !
Advanced Bash-Scripting Guide
The art of command line

27. THANK YOU
THANK YOU
René Ribaud <rene.ribaud@cgi.com>
René Ribaud <rene.ribaud@cgi.com>
🎬 Source: https://github.com/uggla/presentation/tree/agile2019/strictbash
🎬 Source: https://github.com/uggla/presentation/tree/agile2019/strictbash