"Tricks and Craft" in Bash Scripting

Process Input Line by Line

1#!/bin/bash
2input="path/to/file"
3while IFS= read -r line
4do
5    # process each read-in line
6    echo "$line"
7done < "$input"  # NORICE HERE

Pay attention to the < part following done on the last line. It redirects the $input file to stdin  of read process, making read reads the file line by line and then do the processing in the loop body. As for the preceding IFS=, there is a section explains it in this post.

References: Linux/UNIX: Bash Read a File Line By Line ¹ on nixCraft .

shift Built-in

 1if [ $# != 0 ]; then
 2   while true; do
 3       case "$1" in
 4           --first-option)
 5           $ARG="$1"
 6           shift
 7           ;;
 8           --second-option)
 9           $NEXT_ARG="$1"
10           shift
11           ;;
12           # --more-option)
13           # ...
14       esac
15   done
16fi

shift NUM shifts the positional parameters ($1, $2 etc) to the left by NUM. If unset, NUM is 1. Check the Bash manual for more details.

Parameter Substitution

• ${var#Pattern}, ${var##Pattern} From the front, # matches the shortest part, ## matches the longest part
• ${var%Pattern}, ${var%%Pattern} From the back, % matches the shortest part, %% matches the longest part

Conditional Expression

For strings:

• [ STRING == STRING ] True if two strings are equal. ​**"=" may be used instead of “==” for strict POSIX compliance.**
• [ -z STRING ] True if the string is empty
• [ -n STRING ] or [ STRING ] True if the string is non-empty For files:
• [ -e FILE ] Exists.
• [ -f FILE ] Regular file.

: Built-in

It’s a Bourne Shell’s built-in and works as true (with some minor differences stated here ). Example:

1:> file # simply create a file and nothing else

References:

• Section 4.1 of Bash Manual

${BASH_SOURCE[0]} Variable

BASH_SOURCE is a built-in array variable that stores the script path of functions on the call stack, and ${BASH_SOURCE[0]} is the script path of the top of the stack, i.e. the script of the current executing code. It’s similar to $0 but with the following difference. For a script called foo:

1#!/bin/bash
2echo "[$0] v.s. [$BASH_SOURCE]"

Run the following commands:

1$ bash ./foo
2[./foo] v.s. [./foo]
3$ ./foo
4[./foo] v.s. [./foo]
5$ . ./foo
6[bash] v.s. [./foo]

In other words, it works in both sourcing and executing scenarios. Last but not least, it’s a Bash variable and not POSIX-compliant.

References:

• This answer
• Section 5.2 of Bash Manual

Sourcing or Executing a script?

As this answer states:

Sourcing a script will run the commands in the current shell process.

Executing a script will run the commands in a new shell process.

1# Executing
2./my_script
3my_script
4# Sourcing
5source my_script
6. my_script

IFS Variable

$IFS, which stands for Internal Field Separator, is a special shell variable that specifies delimiters to split strings into words in loops²:

1IFS=","
2INPUT="a,b,c,d"
3for field in ${INPUT}
4do
5    # ...
6done

Or with the read built-in command:

 1cat > file << "EOF"
 2big_machine|202.54.1.1|/home/alice|Alice
 3small_machine|202.54.1.2|/home/bob|Bob
 4EOF
 5
 6IFS='|'
 7while read -r hostname ip homedir username
 8do
 9    # ...
10done < file

More information on:

• this example of RIP Tutorial
• this wiki page of Linux Shell Scripting Tutorial

set Built-in

As mentioned in the Bash Manual :

This built-in is so complicated that it deserves its own section.

set is a versatile Bash built-in command having multiple functions. Therefore we only list those that are encountered before.

• set -/+<opts> turns on/off shell options
  • -e make Bash stop executing the script if a command in the script returns an error.
• set -- [args] sets the positional parameters to args. More example here .

Copy All (including hidden) Files in a Directory

As this answer suggests:

1mkdir /home/<new_user>
2cp -r /etc/skel/. /home/<new_user>
3#               ^ the dot here makes things work

Change Password in Script

Use chpasswd. In its manual:

NAME chpasswd - update passwords in batch mode

……

DESCRIPTION The chpasswd command reads a list of user name and password pairs from standard input and uses this information to update a group of existing users. Each line is of the format:

1user_name:password

Example of using it:

1echo 'user_name:password' | chpasswd

List Files with Full Path

As recommended by this post :

1ls -d $PWD/*

And another option by this answer :

1find -type f -maxdepth 1 "$(PWD)"

I personally prefer the first one, which I think is simpler and more straightforward.

“find, and then -exec”

There is a option -exec of the find command, which specifies the action to take for those files that are found out by the command. Examples:

1find ./ -type f -exec chmod 644 {} \;
2
3find . -exec /bin/rm {} \; # useful when you can't do `rm *` because there are too many files and Bash can't manage their names

The semicolon ; following -exec COMMAND is to separate the command body with other arguments of find³. Note that Bash takes ; as a list operator , so in order to keep it as a literal and pass it to find, we need to escape it to remove its special meaning to Bash (example provided by this answer ):

1find . -exec echo {} \;
2find . -exec echo {} ';'
3find . -exec echo {} ";"
4find . -exec echo {} \+
5find . -exec echo {} +

Check this post for more examples.

Bash Print with Color

 1function __print() {
 2   #color
 3   local COLOR_NONE="\e[0m"
 4   local COLOR_RED="\e[0;31m"
 5   local COLOR_GREEN="\e[0;32m"
 6   local COLOR_YELLOW="\e[0;33m"
 7
 8   #debug level
 9   local INFO=1
10   local ERR=2
11   local STEP=3
12
13   color=$COLOR_NONE
14   if [ $1 == $INFO ];then
15   	color=$COLOR_GREEN
16   elif [ $1 == $ERR ];then
17   	color=$COLOR_RED
18   elif [ $1 == $STEP ];then
19   	color=$COLOR_YELLOW
20   fi
21
22   echo -e $color"$2"$COLOR_NONE
23}