Workshop Bash

by

Do you know the meaning of: exec 5<>/dev/tcp/pamoller.com/80? Knowing Bash well opens up the possibility to write helpful programs using the hundreds of utilities (commands) shipped with a Linux or UNIX operating system. This workshop steps through Bash studying its possibilities in detail like the one-liner from the beginning of the paragraph. If you know that this expression opens up a TCP connection you shouldn't read the workshop. If not you may learn a lot. At least you should know the essentials of a UNIX like system. Consult the official documentation for a complete picture of Bash.

What is Bash?

Starting and stopping programs is the most common task in using computers. You may have typed a hundred or thousand times the commands ls and cd. To do so a text based program called the shell is included in UNIX since its beginning in 1969. The shell used in the early years of UNIX was the Thompson Shell.

But the limitations of the Thompson Shell were explored soon. In 1977 the more advanced Bourne Shell replaced the Thompson Shell. Up today it is the standard shell of UNIX, called sh. Starting the GNU project raised the need of a free reimplementation of sh. This was done by Brian Fox. Bash is a free replacement of the Bourne Shell. The name is an acronym for Bourne-Again SHell. Bash was released first in 1989.

Bash is a command interpreter that allows the execution of commands as well as redirecting their in- and outputs and a programing language, which provides built-in commands, control structures, shell functions, shell expansion, shell redirection or recursive programing. Bash adopt the builtin functions from the Bourne Shell as well as features from the Korn Shell and C Shell. The rules for evaluation and quoting are taken from the POSIX specification for the standard UNIX shell.

Bash Basics

Bash operates in different modes: It executes programs and performs interactive sessions (both called shell). So, the login command starts an interactive session after successful authentication, as well as each opened terminal. The short key [STRG]+[ALT]+[T] starts a terminal in Ubuntu Linux.

Terminal with interactive Bash session
Figure 1: Terminal with interactive Bash session

The command bash starts an interactive Bash session explicit from command line. Each command has to be confirmed by the [ENTER] key:

$ bash
$

After invoking an interactive session, Bash prints the prompt "$" to the terminal and waits for user input from keyboard. After hitting the [ENTER] key Bash parses the input into a sequence of commands and executes them:

$ ls
bin  datas  notes.dat  texts

The input is parsed into one token: "ls". This token is interpreted as the command ls.

$ ls -a
.  ..  bin  datas  notes.dat  texts

The input is parsed into two tokens: "ls" and "-a". The token "-a" is interpreted as option of the command ls. It treats ls to list entries starting with "." also. Comments starts with "#" and ends on line end. They won't be parsed

$ # comment, not parsed

They are discarded from parsing. The following input:

$ echo "Hello $USER today is"; date

is parsed into four tokens:

"echo" "Hello $USER today is" ";" "date"

The four tokens are evaluated (expanded) before commands are performed and executed. The expression "Hello $USER today is " includes a reference to the variable USER. The value of the variable is expanded in place of the reference $USER. USER stores the login name of the user. After expansion this token becomes:

"Hello pa today is"

Bash does a lot of expansions before performing commands from user input. Expansions are one of the most powerful features of Bash and explained in detail below. After the expansions take place Bash performs two commands: echo "Hello pa today is" and date. The commands were separated by the ";" operator. The commands are executed sequentially within an own environment:

$ echo "Hello $USER today is"; date
Hello pa today is
Wed Oct  3 19:52:47 CEST 2012

Both commands print their output to the so called standard out. By default the standard out is connected with the terminal. But the output may also redirected to a file.

(echo "Hello $User today is"; date) > output

The commands echo and date are programs. They are kept as executable files somewhere inside the directory tree. Bash searches for the programs within all directories listed in the PATH parameter before execution :

/home/pa/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

":" acts as separator. echo is found at /bin/echo and date at /bin/date in the directory tree.

Quotes

Quoted characters are expanded by own rules. A character following the escape character "\" is taken as literal. "\$" is taken as literal "$" instead of the beginning the start of a variable expansion:

$ echo \$USER
$USER

Only the sequence "\newline" has a special meaning. \newline is interpreted as line continuation:

$ echo 12\
34
1234

Single quoted strings are taken always literal:

$ echo '$USER'
$USER

Double quoted strings allow parameter expansion and command substitution:

$ echo "$(echo $USER)"
pa

"\" disables the special meaning of characters within double quotes:

$ echo "\"Oranges and lemons\""
"Oranges and lemons"

To translate special characters, ANSI-C Quoting can be used. Escape sequences between "$'...'" operator:

echo $'\n\t\\$USER'

	\$USER

will be translated according to the ANSI-C standard: \n - newline, \t - tab, \\ backslash.

Shell Parameters

Bash allows the storage of values by parameters. They can be denoted by a name, a number or a special character. A variable is parameter denoted with a name:

$ declare film="A Clockwork Orange"

The value of a variable is obeyed by parameter expansion:

$ echo $film
A Clockwork Orange

The declare command is optional:

$ verse="Oranges and lemons"

A variable can store a null value:

$ value=

The value of the variable undergoes tilde expansion, parameter expansion, command substitution, arithmetic expansion (except declared as integer) and quote removal:

$ value='~/$(date)/$((1+2))'
$ echo $value
~/$(date)/$((1+2))

Expansion would give:

/home/pa/Thu Oct  4 19:24:22 CEST 2012/3

Variables may have options. They are set by the declare command:

$ declare -irx maxint=2**32-1

The option -i treats declare to evaluate the value of maxint as arithmetic expression:

$ echo $maxint
4294967295

-r marks the variable as read only:

$ maxint=orange
bash: maxint: readonly variable

-x marks the variable for export to child processes:

$ env | grep maxint
maxint=4294967295

Concatenation appends content to the value of a variable by the "+=" operator:

$ verse+=" says the bells of St. Clement's"
$ echo $verse
Oranges and lemons say the bells of St. Clement's

For variables marked as integers the "+=" operator does an incrementation:

$ declare -i int=4
$ int+=4-2**3
$ echo $int
0

To unset a variable, except a read only variable, the unset command is used:

$ unset int
$ echo $int

Arrays

Bash supports indexed and associative arrays. An indexed array is declared by the declare command:

$ declare -a vegetables

Or simply by the "()" operator without declare:

$ fruits=(orange lemon apple banana)

The array elements are accessed by their indexes within parameter expansion:

$ echo ${fruits[0]}
orange

and set by their indexes without parameter expansion:

$ fruits[2]=kiwi;

The first index is 0. Each value of an array element is expanded to a proper word using the subscripts "@" or "*":

$ echo ${fruits[@]}
orange lemon kiwi banana

The number of array elements can be expanded using the "#" operator:

$ echo ${#fruits[@]}
4

Within double quotes subscript "@" still expands each value to a word:

$ words=("${fruits[@]}");
$ echo ${#words[@]}
4

but "*" expands all array elements to a single word separated by the first character of the IFS parameter:

$ word=("${fruits[*]}");
$ echo ${#word[*]}
1

To expand the indexes of any array indirect, expansion by the "!" operator is used:

echo ${!fruits[@]}
0 1 2 3

To append an indexed array to another, the concatenation operator "+=" is used:

$ fruits+=(strawberry cranberry)

To unset an array element or the whole array, the unset command is used:

$ unset fruits[2]
$ echo ${fruits[@]}
orange lemon banana strawberry cranberry

An associative array has to be declared explicit by the declare command:

$ declare -A assoc

Everything described above to indexed arrays can be done to associative arrays, except the use of the "()" and "+=" operators. An approach to find the last added key-value-pair of an associative array is:

$ declare -A assoc;
$ assoc[mem]=128;
$ assoc[shared]=2;
$ assoc[swap]=64;
$ assoc[free]=4;
$ keys=(${!assoc[@]})
$ echo ${keys[$((${#keys[@]}-1))]};
free

Shell Expansions

Bash does eight different types of expansions. They are done for performing complex commands easily. The expansions are done in a defined order from left to right: brace expansion, tilde expansion, parameter expansion, arithmetic expansion, command substitution, word splitting, filename expansion, quote removal. Hence brace expansion is done before tilde expansion:

$ echo ~{pa,user19}
/home/pa /home/user19

and command substitution:

$ echo {$(echo 1,2,3)}{a,b}
{1,2,3}a {1,2,3}b

Filename expansion is done after word splitting:

$ words=(*)
$ echo ${words[@]}
0file Afile _file abcfile bfile f le fiiile fiile file fjjle fjle fle
$ echo ${#words[@]}
12

So each filename in the current directory is taken into one word by the "*" operator, even the filename "f le". The resulting array words holds 12 elements. But word splitting is done after command substitution:

$ words=($(ls))
$ echo ${words[@]}
0file Afile _file abcfile bfile f le fiiile fiile file fjjle fjle fle
$ echo ${#words[@]}
13

First all filenames in the current directory are listed by the ls command. The output of the ls command is taken into one word separated by a blanks " ". Afterwards word splitting takes place on the blank " ". "f le" is splitted into two words. Hence the resulting array holds 13 elements

As last quote removal takes place: It removes remaining unquoted escape characters "\", double """ and single quotes "'" from the expanded input. The lasting words are used to perform commands and execute them.

Brace Expansion

Brace expansion is done for characters within curly braces: {...}. Braced expressions generate lists of words. Ranges are declared by "start..end":

$ echo {01..10}
01 02 03 04 05 06 07 08 09 10

The step width can be increased by appending the step-width operator "..width" to the range:

$ echo {a..z..2}
a c e g i k m o q s u w y

An enumeration of words is separated by the "," operator:

$ echo {A,Clockwork,Orange}
A Clockwork Orange

A sequence of braced expressions is expanded like a Cartesian product. This means: Each expanded word from any braced expression is combined with any word expanded from any other braced expression:

echo {a..c}{1..3}
a1 a2 a3 b1 b2 b3 c1 c2 c3

A sub tree of hundreds of directories can be created by brace expansion easily:

$ mkdir -pv /home/pa/{foo,bar,baz}/{a..z..2}/{01..10}

Brace expansion is nestable and the evaluation starts with the innermost expression:

echo {a{1{i,j},2,3}}
a1i a1j a2 a3

Tilde Expansion

Tilde expansion shorten paths. It translates the "~" character to the first matching directory. Instead of typing /home/user19/datas tilde can be used:

$ ls -d ~user19/datas
/home/user19/datas

Any characters following the tilde up to the first path separator "/" are taken together as possible login name and the home directory of this login name is substituted with "". If the login doesn't exists, its taken literal:

$ ls ~moore
ls: cannot access ~moore: No such file or directory

If no login name is given, the value of the HOME variable is substituted for "~". So, addressing own home is always done by "~/":

$ ls -d ~/datas
/home/pa/datas

If HOME is unset, the home directory of the user, who is executing the shell, is taken:

$ unset HOME
$ ls -d ~/datas
/home/pa/datas

Shell Parameter Expansion

Parameter expansion allows the substitution of parameter names by their values. Parameter names put between the "${...}" operator will expanded:

$ word="Orange"
$ echo ${word}
Orange

as well as parameter names prefixed by the "$" operator:

$ echo $word 
Orange

Before the value is substituted it can be transformed by a large set of operations. By using the ":-" operator a undefined variable can be replace by a value (default):

$ unset integer
$ echo ${integer:-0}
0

or a null value of a declared variable:

$ integer=
$ echo ${integer:-0}
0

To initialize an undeclared variable before substitution the ":=" operator is used:

$ unset integer
$ echo ${integer:=1}
1

To write a message about a null value or an undeclared variable to stderr the "?" operator is used:

$ unset verse
$ echo ${verse:?"Oranges and lemons say the bells of St. Clement's"}
bash: verse: Oranges and lemons say the bells of St. Clement's

To expand a sub string from a value the ":offset:length" operator is used. :

$ string="/home/pa/bin"
$ echo ${string:6:2}
pa

Offset is the offset taken before sub string, length its length. Without length the offset is removed:

$ echo ${string:6}
pa/bin

Beware of the order of expansions:

$ ls "~${string:6:2}/bin"
ls: cannot access ~pa/bin: No such file or directory

Tilde expansion is done before parameter expansion. Hence "${string:6:2}" is taken literal as login name. Because of this user dosen't exists "~" is left unchanged by tilde expansion. But the literal path "~pa/bin" doesn't exist. Sub arrays can be expanded using subscripts "@" or "*" too:

$ fruits=(orange lemon apple melone banana)
$ echo ${fruits[@]:2,2}
apple melone

Offset is index of the first element within the expanded sub array, length its length. Each value of an array element is expanded to a proper word using the subscripts "@" or "*":

$ echo ${fruits[@]}
orange lemon kiwi banana

The number of array elements can be expanded using the "#" operator:

$ echo ${#fruits[@]}
4

Within double quotes subscript "@" still expands each value to a word:

$ words=("${fruits[@]}");
$ echo ${#words[@]}
4

but "*" expands all array elements to a single word separated by the first character of the IFS (Input Field Separators) variable:

$ word=("${fruits[*]}");
$ echo ${#word[*]}
1

To expand the indexes of any array indirect expansion by the "!" operator is used:

echo ${!fruits[@]}
0 1 2 3

The length of a value can be expanded by the "#" also:

$ verse="Oranges and lemons say the bells of St. Clement's"
$ echo ${#verse}
50

To expand the number of positional parameters:

$ echo ${#@}
0

To expand all variable names starting with a prefix into a single word, separated by the first character of the IFS variable, indirect expansion "!" in conjunction with subscript "@" is used:

$ declare vertext=2
$ echo ${!ve@}
verse vertex

"@" masks the remaining characters of the names. To remove the longest match of a pattern, as described in section filename expansion, from the beginning of a value, the "##pattern" operator is use:

$ path="archive.tar.gz"
$ echo ${path##*.}
gz

To remove the shortest match of a pattern from the beginning of a value, the "#pattern" operator is used:

$ echo ${path#*.}
tar.gz

To remove the longest match of a pattern from the end of a value the, "%pattern" operator is used:

$ echo ${path%%.*}
archive

To remove the shortest match of a pattern from the end of a value "%pattern" operator is used:

$ echo ${path%*.*}
archive.tar

To replace the first of the longest matches of a pattern in a value by a string, the "/pattern/string" operator is used:

$ echo ${path/./:}
archive:tar.gz

To replace all matches, the "//pattern/string" operator is used:

$ echo ${path//./:}
archive:tar:gz

To remove the match from the value, the "/pattern" operator is used:

$ echo ${path//.ar}
chive.t.gz

The transformations from above can be done even to all positional parameters:

$ echo ${@//./:}

as well as on all array elements:

$ echo ${fruits[@]//a*/A}
orA lemon A melone bA

A certain character in an variable, any array element or any positional parameters can be brought to upper case by the "^^character" operator:

$ echo ${fruits[@]^^n}
oraNge lemoN apple meloNe baNaNa

or to lower case by the ",,character" operator:

$ echo ${verse,,C}
Oranges and lemons say the bells of St. clements

Parameter expansion is not nestable, but indirect expansion can be used:

$ ref="verse"
$ echo ${!ref}
Oranges and lemons say the bells of St. Clement's.

The value of ref is treated as name of the variable to be expanded.

Command Substitution

Command substitution replaces the output of a command with its notation. Command substitution takes place between the "$(...)" operator:

echo "Welcome, today is $(date)"
Welcome, today is Fri Oct  5 17:52:30 CEST 2012

But command substitution takes also place between the "`...`" (backticks) operator:

ls /home/`whoami`/bin/*
logrotate.sh lstree.sh pstree.sh

Command substitution is a very common in Bash. It can be nested:

$ echo $(echo $(date))
Fri Oct 5 17:53:24 CEST 2012

Arithmetic Expansion

Bash expands characters between the "$((...))" operator as an arithmetic expression. Arithmetic expressions operate on integer values:

echo $(( (1 + 2) / 2 ))
1

Inside arithmetic expansion variables are expanded by name only:

$ integer=2
$ echo $(( integer - 3 ))
-1

There is no need of the parameter expansion notation. Values of undeclared variables, the null value or values being not an integer are evaluated to 0:

$ unset eight
$ echo $(( 2 ** eight + word))
1

Octal values can be written by a leading "0":

$ echo $(( 010 ))
8

while hexadecimal values are prefixed by "0x":

echo $(( 0x10 ))
16

The operators and their precedence, associativity, and values are the same as in the C language. Variables can be incremented and decremented in prefix and postfix notation:

$ int=0
$ echo $(( ++int + int++ + --int + int--))
4

Exponentiation (**), multiplication (*), division (/), remainder (%), addition (+) and subtraction (-) can be done like with any calculator:

$ echo $(( 255%16 - 2*2**3 + 100/10**2 ))
0

Integers can be shifted bitwise to higher (<<) or lower values (>>):

$ echo $(( 8 << 2))
32

The relations greater (>), greater equal (>=), lower (<) and lower equal (<=) compare integers by size. A True value will be given by 1, a False by 0:

$  echo $(( 1 <  2 <= 3))
1

Even for the equality (==) and inequality (!=) operators:

$ echo $(( (1 != 2 ) == 1 ))
1

Values can be compared bitwise by the AND (&), exclusive OR (^) or OR (|) operators:

$ echo $(( (8 ^ 7) & 16 ))
0

or logically by the AND (&&) or OR (||) operators:

$ echo $(( ( 0 || 1 ) && 1 ))
1

Even the eternity operator exists:

$ echo $(( 1>0?1:0 ))
1

Last but not least assignments can be done (=, *=, /=, +=, -=, <<=, >>=, &=, ^= and |=). The operators returns their assigned values:

$ echo $(( int=99 ))
99

Arithmetic expansion is also nestable:

$ echo $(( $(echo $(( 1 + 2 ))) + $int + $(echo 1 + 2) ))
105

Word Splitting

After all the expansion from above are done, word splitting takes place: The remaining unquoted characters are splitted into words using each character within the variable IFS as possible separator. "space", "tab" and "newline" are used as defaults:

$ unset IFS
$ words=($(echo -e "Oranges and lemons\nsay the bells of\tSt. Clement's"))
echo ${#words[@]}
9

The -e option treats the echo command to convert the escape sequences to special chars. After echoing the string includes eight separators from the IFS parameter: six blanks, one newline (\n) and one tab (\t). So the characters are splitted into nine words. Hence, the array words is initialized by nine words. Putting double quotes around the echo command treats Bash to suppress the word splitting:

$ words=("$(echo -e Oranges and lemons\nsay the bells of\tSt. Clement\'s)")
echo ${#words[@]}
1

Finally the array words is initialized by one word instead of nine. It's common to surround variables with double quotes. This prevents the variables value to be splitted into several words:

$ file="my file"
$  ls "$file" $file
ls: cannot access file: No such file or directory
ls: cannot access my: No such file or directory
my file

Words are not only performed by word splitting. They are performed by: brace expansion, filename expansion word splitting and the use of the subscripts "@" or "*":

$ words=(bin/* ${fruits[@]} A Clockwork Orange {a..c})
$ echo ${#words[@]}
14

The first three words orginates from filename expansion "bin/*":

$ ls ${words[@]:0:3}
bin/logrotate.sh bin/lstree.sh bin/pstree.sh

the next five from parameter expansion "${fruits[@]}":

$ echo ${words[@]:3:5}
orange lemon apple melone banana

the next three from word splitting:

$ echo  ${words[@]:8:3} 
A Clockwork Orange

and the last three from brace expansion "{a..c}":

$ echo ${words[@]:11:3}
a b c

Filename Expansion

Filename expansion allwos the selection of paths by simple patterns. Words containing one of the characters "*", "?", "[" are treated as pattern and replaced by file- or directory names matching this pattern. The "*" matches any characters, except a leading ".":

$ ls *
0file  Afile  _file  abcfile  bfile  f le  fiiile  fiile  file  fjjle  fjle  fle

The leading "." must match explicitly::

$ ls .*
.file

So the actual directory with alias "." and the parent directory with alias ".." are excluded from the matches. Otherwise, unwanted recursion and system damage may occur. If no filename can be matched the literal value string is taken:

$ echo g*rl
g*rl

The "?" matches any single character, except ".":

$ ls ?file
0file  Afile  _file  bfile

Sets of characters can be matched:

$ ls [Ab]file
Afile bfile

as well as character ranges:

$ ls [a-zA-Z]file
Afile bfile

The range depends on the collation locale, stored by the LC_COLLATE variable. Some character ranges are predefined. They are called character classes, e.g. alnum, alpha, ascii, blank, cntrl, digit, graph, lower, print, punct, space, upper, word and xdigit:

$ ls [:alpha:]file
0file Afile bfile

To exclude character from a pattern the "[^...]" operator is used. The following expression excludes all filenames which start by 0, 1, 2,.., 9, "_", or "-":

$ ls [^0-9._-]*
Afile  abcfile  bfile  f le  fiiile  fiile  file  fjjle  fjle  fle

When the option extglob is set, Bash allows even the quantification of patterns:

$ shopt -o extglob

To treat a pattern to be matched zero or one time the "?(...)" operator is used:

$ ls f?(i)le
fle file

or any time:

$ ls f*([ij])le
fiiile  fiile  file  fjjle  fjle  fle

or at least one time:

$ ls f+(i)le
fiiile  fiile  file

Alternative patterns are separated by the "|" operator between the "?(...)", "*(...)" or "+(...)" operators:

$ ls -d /home/+(pa|user19)
/home/pa /home/user19

To allow only one match from the alternative patterns prefix the list by the "@" operator:

$ ls -d /home/@(pa|user19)
/home/pa /home/user19

To allow any match without the alternatives prefix the list by the "!" operator:

$ ls -d /home/!(mark|baracuda)
/home/pa /home/user19

Redirections

The output of a command is usually printed to its standard out, stdout. By default the stdout is connected with the terminal. That's why the user sees it. But the output may also be saved by redirecting the stdout to a file using the ">" operator:

$ env > file

The output of the env command is written the file file. The content of an existing file will be truncated (deleted). To append content to the end of a file the ">>" operators is used:

$ date >> file
$ tail file
_=/usr/bin/env
Thu Oct 11 17:49:55 CEST 2012

Technically stdout is a file descriptor, that is connected to the terminal by default. By redirection, the descriptor is connected to another target. All file descriptors are listed within /dev/fd:

$ ls -l /dev/fd/
lrwx------ 1 pa pa 64 Oct  8 19:32 0 -> /dev/pts/1
lrwx------ 1 pa pa 64 Oct  8 19:32 1 -> /dev/pts/1
lrwx------ 1 pa pa 64 Oct  8 19:32 2 -> /dev/pts/1

0 is file descriptor for reading, called standard input (stdin), 1 for writing, called standard output (stdout) and 2 for writing errors, called standard error (stderr). Using file descriptors the redirection from above is written as:

$ env 1> file

To redirect errors from stderr to a file its file descriptor 2 is used explicit:

$ ls undefined 2> file

To redirect both, stdin and stderr, to the same file, two redirections can be used:

$ ls ~ undefined 1>file 2>&1

Redirections are being processed from left to right. First, stdout is connected to file, second the output of stderr is duplicated to the same target as stdout. Beware of the order, cause:

ls ~ undefined 2>&1 >file
ls: cannot access undefined: No such file or directory

doesn't redirect stderr to file. First stderr is duplicated to same output as stdout - the terminal. Second, stdout is connected to file. stdin and stderr can be redirected to the same target by using the "&>" or ">&" operators:

$ ls ~ undefined &>file

To discard all errors message which are printed to stderr, it is redirected to the null device:

$ ls ~ undefined 2>/dev/null
/home/pa:
bin datas fifo file poem

To discard every output, the "&>" or ">&" operators are redirects to the null device:

$ ls ~ undefined &>/dev/null

By default, stdin is connected with the keyboard:

$ grep pa
123
pappel
pappel

The command grep matches each line that is entered by the keyboard against the pattern pa. [CTRL]+[D] terminates the input. To redirect stdin from a file, the "<" operator is used:

$ grep pa < file
/home/pa:

The combination of redirecting the input and can be written as:

$ grep pa < file > output

But redirecting from stdin isn't needed by grep:

$ grep pa file > output 

Custom file descriptors can be created by the exec command. To create a file descriptor for writing:

$ exec 4>poem

The file descriptor can be used now for writing:

$ echo "Oranges and lemons" >&4

A second write dosn't truncate poem, it appends the input to the end of the file:

$ echo "say the bells of St. Clement's" >&4

Similar a file descriptor for reading is created:

$ exec 5<poem

The file descriptor can be used for reading:

$ cat <&5
Oranges and lemons
say the bells of St. Clement's

Reading twice is without effect, because the file pointer points to the end of the file:

$ cat <&5
$

But appended content is added after the file pointer. So it can be read:

$ echo "You owe me five farthings" >&4
$ cat <&5
You owe me five farthings

To prevent a file from being overwritten the noclobber option has to be set:

$ set -o noclobber
$ echo "Here comes a candle to light you to bed" > poem
bash: poem: cannot overwrite existing file

To overwrite a file even if noclobber is set the ">|" operator is used:

$ echo "Here comes a candle to light you to bed" >| poem

To unset the noclobber option the set command is used again with "+o" instead of "-o":

$ set +o noclobber

File descriptors can be used when reading input by the read command. The descriptor will be selected by the -u option:

$ exec 5<poem
$ read -u 5 verse
$ echo $verse
Here comes a candle to light you to bed

A file descriptor can be used bidirectional for writing and reading also:

$ exec 6<>poem

File descriptors can be moved by the ">&digit-" operator:

$ exec 7>&6-
$ ls -l /dev/fd/
lrwx------ 1 pa pa 64 Oct 12 16:36 0 -> /dev/pts/2
lrwx------ 1 pa pa 64 Oct 12 16:36 1 -> /dev/pts/2
lrwx------ 1 pa pa 64 Oct 12 16:36 2 -> /dev/pts/2
lr-x------ 1 pa pa 64 Oct 12 16:36 3 -> /proc/2639/fd
l-wx------ 1 pa pa 64 Oct 12 16:36 4 -> /host/Users/pa/Documents/daten/poem
lr-x------ 1 pa pa 64 Oct 12 16:36 5 -> /host/Users/pa/Documents/daten/poem
lrwx------ 1 pa pa 64 Oct 12 16:36 7 -> /host/Users/pa/Documents/daten/poem

Descriptor 6 will be closed. By default it is moved to 1 (stdin). Similar for stdout, the reverse operator "<&digit-" exists:

exec 6<&7-

To close a file descriptor the ">&-" operator is used:

$ exec 6>&-

The "<&-" operator can be used also:

$ exec 5<&-

File Descriptors can also be used to read from or write to named pipes (fifos). The fifos are usual used for interprocess communication. They can be accessed by paths but their content is kept in the memory. Fifos are created by the fifo command:

$ mkfifo fifo
exec 4<fifo

The fifo blocks until a second shell opens the fifo for writing:

$ exec 5>fifo
$ while (( 1 ))
do
  date >&5;
  sleep 1;
done

Now within the first shell the content can be read from the file descriptor 4:

$ cat <&4
Mi 10. Okt 18:06:00 CEST 2012
Mi 10. Okt 18:06:01 CEST 2012
Mi 10. Okt 18:06:02 CEST 2012
Mi 10. Okt 18:06:03 CEST 2012
Mi 10. Okt 18:06:04 CEST 2012

[CTRL]+[C] terminates the program. On some systems file descriptors can also access tcp or udp resources:

$ exec 4<>/dev/tcp/pamoller.com/80
$ echo -en "GET / HTTP/1.1\n\r\n\r" >&4
$ cat <&4

The First line opens a tcp connection to the web server at pamoller.com on port 80 for writing and reading. The echo command prints a HTTP get request to the TCP socket. The HTTP server response is read by the cat command from the same file descriptor. File descriptors can be stored in variables by using the "{...}<" operator:

$ exec {desc}<poem

Internaly a file descriptor greater 10 is assigned to the variable descriptor:

$ cat <&$desc
Here comes a candle to light you to bed

A special kind of redirections are here documents. Here documents are expressed by the "<<WORD....WORD" operator:

cat > poem <<EOF
"Oranges and lemons", say the bells of St. Clement's
"You owe me five farthings", say the bells of St. Martin's
"When will you pay me?" say the bells of Old Bailey
"When I grow rich", say the bells of Shoreditch
"When will that be?" say the bells of Stepney
"I do not know", says the great bell of Bow
Here comes a candle to light you to bed
And here comes a chopper to chop off your head!
Chip chop chip chop – The last man's dead.
EOF

Pipes

The stdout can also be connected to the stdin of a second command directly using the "|" (pipe) operator:

$ env | grep pa
OLDPWD=/home/pa
USER=pa
PWD=/home/pa/daten
HOME=/home/pa

The output of env is filtered by grep for lines containing the string "pa". A pipe may consist of many commands. So, a ranking of file types within a given sub tree can be done easily:

$ ls -R  | egrep -o "\..*$" | sort | uniq -c | sort -n -r  | head -5
    349 .png
    318 .txt
    289 .au
    208 .js
    111 .html

ls -R prints all directory entries of the current directory recursively to stdout. The grep command filters this output by lines which matches the regular expression: "\..*$". The pattern matches every file extension. "$" matches the end of a line. -o treats grep to print only the match to stdout. Uniq summarize the occurrence of the sorted list entries. sort -r -n sorts the lines again, but numerically in descending order. The head command prints the first 5 lines. The exit status of the pipe is the exit status of the outer right command:

$ (exit 0) | (exit 254) | (exit 1)
$ echo $?
1

In conjunction to the anonymous pipe described above a named pipe, called fifo, exists. Fifos are created by the fifo command:

$ mkfifo fifo

A fifo is accessed by a path, but its content is always kept in memory. Writing to a fifo is done by:

$ echo "Oranges and lemons" > fifo

If no process is connected for reading the fifo blocks the writing process. It seems to hang until the fifo will be read by a second opened shell:

$ cat <fifo
Oranges and lemons

Even reading is blocked:

$ exec 4<fifo

until something is written to the fifo in the first shell:

$ echo "says the bells of St. Clement's" >fifo

It can be read now by:

$ cat <4
says the bells of St. Clement's

Environments

After all expansions of the user input are done and quote removal has succeeded. Bash performs commands and executes them. Built-in commands like read, for or exit are executed all within the current Bash process. External commands, like find or commands running in a pipe as well as subshells are executed in a own child processes of the shell. Each child process has it's own execution environment derived from it's parent shell.

Program sequence: Built-ins are run in the shell process, all others in a separate child process.
Figure 2: Program sequence: Built-ins are run in the shell process, all others in a separate child process.

An environment is basically a set of variables and functions which are marked for copying into a child process. A mark can be set by the declare command:

$ declare -x verse="Oranges and lemons say the bells of St. Clement's"

or by using the export command:

$ export film="A Clockwork Orange"

Functions can be marked by applying the -f option to their name to the export command:

$ function parameters() {
  echo $@
}
$ export -f parameters

The environment can be listed by the env command:

$ env
!::=::\
_=/usr/bin/env
film=A Clockwork Orange
verse=Oranges and lemons say the bells of St. Clement's
parameters=() {  echo $@
}

The execution environment of a shell consists of all files opened by the shell, variables and functions both derived from it's parent environment or set after its creation, traps being set, options being set, aliases, various process ids, the file mask mode and the current working directory both derived from it's parent environment or set after its creation.

Commands running in a pipe, started asynchronous or orginating by command substitution, are run in a subshell. The subshell is invoked within a copy of the current execution environment, except traps. They are reset. A subshell can be formed by enclosing commands within round brackets also:

$ (exit 1)

The exit command is a Bash built-in. Without the brackets, it would close the terminal, because built-in commands are executed in the same process as the shell. Within brackets, exit terminates the process of the subshell.

External commands are invoked in a separate execution environment that includes the actual create mask and working directory, opened files, variables and functions being exported. Traps are reset to the values the shell inherited from its parent.

Job Control

Tasks are done within UNIX like systems by processes, sharing system resources in a concurrent manner. All commands except the builtin commands of Bash and commands told to do so are executed in a separate process also. The ps command shows the actual running processes. The following print is a subset of processes:

$ ps -eo pid,ppid,uid,comm
 PID  PPID   UID COMMAND
    1     0     0 init
 1023     1     0 lightdm
 1576  1023     0 lightdm
 1679  1576  1000 gnome-session
 1756  1679  1000 compiz
 2054  1756  1000 sh
 2055  2054  1000 gnome-terminal
 4025  2055  1000 bash
 4136  4025  1000 ps

The processid pid is a positive integer. It is used to manage the process. The ppid is the processid of the parent process. New processes (childs) are started by an existing process (parent). The ps command with the processid 4136 was started by the Bash process with the processid 4025. Bash itself was started by the terminal program gnome-terminal with processid 2055. The ancestor of each process is the first process, init with pid 1. The consumption of system resources by a process can be displayed by the time key word:

$ time sleep 2

real    0m2.168s
user    0m0.046s
sys     0m0.031s

The sleep command terminates after a given time without doing anything else. The command was run within 2.168 seconds. The values of system and user time are proportional to the cycles of the CPU (wall clock) which were done during running the command. User time was needed to process the command while the System time elapsed. The consumption of system resources per time can be lowered by the nice command:

$ nice -9 sleep 2

Values from 0..19 lower the consumption of system resources per time. nice is used for long running tasks with a low priority. Bash executes a list of commands synchronous - one after another. This guarantees that the next command will process the results of the previous command (see pipes). Commands are started asynchronous by the "&" operator:

$ sleep 2000 & 
[1] 6900
$

Asynchronous started commands are called jobs. 1 is the job number, 6900 the pid of the job. Asynchronous commands are started in an own process and Bash doesn't wait until the command succeeds. Bash proceeds with the execution of the next command immediately. All jobs can be listed by the jobs command:

$ jobs
[1]+  Running                 sleep 2000 &

Asynchronous execution does a more efficient use of system resources by it's concurrent behavior. But any kind of order will get lost. To ship back into synchronization, the wait command is used:

$ sleep 2 &; wait; echo done;
[1] 3783
[1]+  Done                    sleep 2
done

The termination of a process hangs up its childs. To proceed with a command even through termination of its parent, the nohup (no hang up) command is used:

$ nohup sleep 2 &
[2] 7200
$ exit

By use of nohup the stdin is redirected from /dev/null and the output is redirected to nohup.out by default:

$ sleep 2 </dev/null > nohup.out

Jobs are managed by signals. The signals are sent to the processes by inter process communication. But most signals can be ignored. Otherwise, they interrupt the program and invoke a signal handler. In most cases, the default handler can be overwritten by a custom handler. After completion of the handler, the program continues. The following list shows common signals:

$ kill -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGEMT       8) SIGFPE       9) SIGKILL     10) SIGBUS
11) SIGSEGV     12) SIGSYS      13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGURG      17) SIGSTOP     18) SIGTSTP     19) SIGCONT     20) SIGCHLD
21) SIGTTIN     22) SIGTTOU     23) SIGIO       24) SIGXCPU     25) SIGXFSZ
26) SIGVTALRM   27) SIGPROF     28) SIGWINCH    29) SIGPWR      30) SIGUSR1
31) SIGUSR2     32) SIGRTMAX

Signals are sent by the kill command. Either the pid or the jobspec, %jobnumber, are used to address the job. Either the signal name (SIGHUP) or its numerical representation (1) can be used. To stop the execution of a process the SIGSTOP signal is sent best, because it can't be caught or ignored:

$ sleep 2000 &
$ kill -SIGSTOP %1
$ jobs
[1]+  Stopped                 sleep 2000

To restart the job, the SIGCONT signal is sent:

$ kill -SIGCONT %1
$ jobs
[1]+  Running                 sleep 2000 &

The best way to terminate a job is sending the SIGKILL signal. SIGKILL can't be caught or ignored:

$ kill -SIGKILL %1
$ jobs
[1]+  Killed                  sleep 2000

Bash supports the built-in command trap to react on signals:

$ trap 'echo "receive signal"'  SIGINT
$ echo $$ 
4536

Now within a second terminal a signal will sent to the first terminal with pid 4536:

$ kill -SIGINT 4536

The somehow crypted message will be printed to the first terminal:

$ ^Creceive signal

Signals can be ignored by calling trap with the NULL argument and the signal name:

$ trap '' SIGHUP

If SIGHUP is sent from the second terminal:

$ kill -SIGHUP 4536

the first terminal won't be closed. To unset trap is called without argument, but with the signal name:

$ trap SIGHUP

Sending SIGHUP again from the second terminal will close the first terminal. Trap doesn't listen to signals only. It reacts on the call of the exit command as well as on the return command:

$ trap 'echo "receive builtin call"' EXIT RETURN

as well as on commands that exits with a status not equal 0:

trap 'echo $? >> log' ERR

To list currently all set traps, trap is called without any argument:

$ trap
trap -- 'echo "receive builtin call"' EXIT
trap -- 'echo "receive signal"' SIGINT
trap -- 'echo $? >> log' ERR
trap -- 'echo "receive builtin call"' RETURN

Some signals can be sent manually by the keyboard to the actual running foreground process. SIGSTOP suspends a process by pressing [CTRL]+[Z]:

 $ sleep 2000
^Z
[1]+  Stopped                 sleep 2000

SIGINT terminates the process almost. It can be sent by pressing [CTRL]+[C]. The fg and the bg commands can be used to put a process from background into the foreground, vice versa.

Exit Status

A process terminates by calling exit system call along with a status from 0 to 255. Within the UNIX world 0 means successful termination. Any number greater 0 points to an irregular termination:

$ (exit 254)
$ echo "Last command exits by: $?"
Last command exits by: 254

The parameter $? keeps the exit status of the most recent executed foreground command. The exit status can be used to determine further procession. The execution of a sequence of commands separated by the "&&" (AND) operator breaks after the first command which does not terminate successfully:

$ echo a && echo b && (exit 1) && echo c
a
b

The execution of a sequence of commands separated by || (OR operator) breaks after the first command which terminates successfully:

$ (exit 1) || echo a || echo b
a

Conditional Constructs

Bash is not only a command line interpreter, it is also a programing language. Therefore its include conditional constructs like the if command. The evaluation is done by examining the return status of the command lists following the if or elif keywords. On finding an exit status of 0 all consequent commands are executed:

$ if [ 0 -lt 1 ]
then
  echo "0 is lower than 1"
elif [ 1 -lt 2 ]
then
  echo "1 is lower than 2"
fi

The second condition is never proofed. If no command list exits by 0, Bash executes the alternate consequent commands within the else section:

if [ 1 -lt 0 ]
then
  echo "1 is lower than 0"
else
  echo "1 is never lower than 0"
fi

Bash even supports the case command. Bash matches the word between "case" and "in" against a list of patterns. If one list matches, Bash executes the corresponding commands:

$ case "orange" in
 orange) echo "almost sweet";;
 lemon) echo "never sweet";;
esac

Alternatives are separated by the "|" operator:

$ case "melone" in
 orange|banana|melone) echo "almost sweet";;
 lemon) echo "never sweet";;
esac

The "*" operator matches every word:

$ case "quince" in
   orange|banana|melone) echo "almost sweet";;
   lemon) eho "never sweet";;
   *) echo "never eaten this fruit";;
esac

Unless the nocasematch option is set, Bash regards the matching to the alphabetical order. If the matching command list ends by ;; no further procession takes place. If ending on ;& the following command list will executed also:

$ case "quince" in
   orange|banana|melone) echo "almost sweet";;
   quince) echo "not eatable";&
   lemon) echo "never sweet";;
   *) echo "never eaten this fruit";;
esac

If ends on ;;& the following pattern list will also be matched.

$ case "orange" in
  orange) echo "almost sweet";;&
  melone|orange) echo "smells great";;
esac

The word undergoes tilde expansion, parameter expansion, command substitution, arithmetic expansion and quote removal. Each pattern undergoes tilde expansion, parameter expansion, command substitution and arithmetic expansion:

$ case ~$USER in
  ~$USER) echo "found string"
esac

An arithmetic expression is also a conditional expression. It returns 0 if the expression is expanded to a value not 0. Otherwise 1 is returned:

$ (( 1 > 0 )) && echo "arithmetic expressions returns successfuly"

Conditional expressions are evaluated by the test command:

$ test ! -z $USER && echo "varaible is not null"

or in between the "[ ... ]" and "[[ ... ]]" operators. A true expression exits on 0, a false on 1:

$ [ 1 -gt 0 ] && [[ 1 -gt 0 ]] && echo "both expressions are true"

Words can be compared lexicographically according to the current locale by the "<" or ">" operator:

$ [ a > b ] && echo "TODO examine"

The "[[ ... ]]" does no word splitting nor filename expansion on conditional expressions. Tilde expansion, parameter expansion, arithmetic expansion, command substitution and quote removal takes place:

$ [[ $USER = "pa" ]] && [[ ~ = "/home/pa" ]] && [[ $((2*2)) -eq 4 ]] && echo "expansions allowed"

To match a word against a pattern the "==" (equal) or "!=" (not equal) operators are used. The right side is taken as pattern as described in the section about filename expansion:

$ [[ "Oranges and lemons" == *and* ]] && echo "pattern matches"

Double quoted patterns are compared by string comparison with the left side:

$ [[ "say the bells of St. Clement's." != " B.." ]] && echo "strings not equal"

To match a word against a regular expression the "=~" operator is used:

$ [[ "Oranges and lemons" =~ e. ]] && echo "match extended regular expression"

The matches and their sub matches can be referenced by round brackets. They are saved in the array BASH_REMATCH:

$ [[ "A Clockwork Orange." =~ ( Clock.+ (Or(.+)))e ]] && echo ${BASH_REMATCH[*]}
Clockwork Orange Clockwork Orang Orang angr

Within the "[[ ... ]]" operator expressions can be combined by Boolean operators. To check both expressions to be true, the "&&" (AND) operator is used:

$ [[ 1 -eq 1 && 0 -ne 1 ]] && echo "both expressions are true"

To check at least one expression is true, the "||" (OR) operator is used:

$ [[ 1 -eq 1 || 0 -eq 1 ]] && echo "at least one expressions is true"

To negate an expression, the "!" operator is used:

$ [[ ! 0 -eq 1 ]] && echo "the negation of a wrong expression is true"

The order of evaluation can be grouped with brackets. The expression within the innermost braket is evaluated first:

$ [[ 0 -eq 1 && (1 -eq 1 || 0 -eq 0) ]] || echo "expression is evaluated to false"

Surrounding first two expressions by brackets:

$ [[ (0 -eq 1 && 1 -eq 1) || 0 -eq 0 ]] && echo "expression is evaluated to true"

Conditional Expressions

Conditional expression are evaluated by the tests command or within the "[ ... ]" or "[[ ... ]]" operators. Bash supports a many tests. True tests exists on 0, false on 1. To test the existence of a file:

$ > file
$ [ -a file ] && echo "file exists"

To test the existence of a block special file:

$ [ -b /dev/sda ] && echo "block special file exists"

To test the existence of a character special file:

$ [ -c /dev/tty ]  && echo "character special file exists"

To test the existence of a directory:

$ [ -d ~ ] && echo "directory exists"

To test the existence of a regular file:

$ [ -f file ] && echo "regular file exists"

To test that the set-group-id bit is set:

$ chmod g+s file 
$ [ -g file ] && echo "set-group-id bit set"

To test the existence of a symbolic link:

$ ln -s file slink
$ [ -h slink ] && echo "symbolic link exists"

To test that the sticky bit is set:

$ chmod +t file
$ [ -k file ] && echo "sticky bit set"

To test the existence of a named pipe:

$ mkfifo fifo
$ [ -p fifo ] && echo "fifo exists"

To test the existence that the file is readable:

$ [ -r file ] && echo "file is readable"

To test that the file is greater than 0 byte:

$ echo 1 >> file
$ [ -s file ] && echo "file size greater 0"

To test that the file descriptor is open and refers to a terminal:

$ exec 3<> /dev/tty
$ [ -t 3 ] && echo "open file descriptor refers to terminal" 

To test the existence that the set-userid-bit is set:

$ chmod u+s file
$ [ -u file ] && echo "set-userid-bit is set" 

To test that the file or directory is writable:

$ chmod u+w file
$ [ -w file ] && echo "file is writeable"

To test that the file or directory is executable:

$ chmod u+x file
$ [ -x file ] && echo "file is executeable"

To test that the file is owned by the effective group id:

$ [ -G file ] && echo "file owned by effective group id"

To test that the file was modified since the last read:

$ cat file && touch file
$ [ -N file ] && echo "modified since last read"

To test that the file is owned by the effective user id

$ [ -O file ] && echo "file owned by effective user id"

To test the existence of a socket:

$ exec 4<> /dev/udp/127.0.0.1/80
$ [ -S 4 ] && echo "socket exists"

To test that both files refer to the same device and inode:

$ [ file -ef file ] && echo "the two files refers to same device and inode"

To test that the first file exists and was modified after second file or exists only:

$ touch file2 && sleep 1 && touch file
$ [ file -nt file2 ] && echo "first newer than second file"

To test that the second file exists and was modified after first file or exists only (vice versa):

$ [ file2 -ot file ] && echo "first file older than second file"

To test that a option is set:

$ [ -o i ] && echo "shell is interactive"

To test that a variable is set:

$ [ -v HOME ] && echo "HOME variable has been assigned a value"

To test that the length of string is 0:

$ [ -z "" ] && echo "length is 0"

To test that the length of a string is not 0 (negation):

$ [ "eins" ] && echo "string not 0"

To test that two strings are equal:

$ [ "1" == "1" ] && echo "strings are equal"

To test that two strings are not equal (negation):

$ [ "0" != "1" ] && echo "strings are not equal"

To test that the first string sorts before the second string lexicographically:

$ [ "abc" < "def" ] && echo "first string sorts before second"

To test that the second string sorts before the first string lexicographically (vice versa):

$ [ "def" > "abc"] && echo "second string sorts before first"

To test that two integers are equal:

$ [ 1 -eq 1 ] && echo "integers are equal"

To test that two integers are not equal (vice versa):

$ [ 1 -ne 0 ] && echo "integers are not equal"

To test that the first integer is lower or equal than the second:

$ [ 0 -le  0 ] && echo "first integer is lower equal"

To test that the first integer is lower than the second:

 $ [ 0 -lt 1 ] && echo "first integer is lower"

To test that the first integer is greater than the second:

$ [ 1 -gt 0 ] && echo "first integer is greater"

To test that first integer is greater or equal than the second:

$ [ 1 -ge 1 ] && echo "first integer is greater equal"

For clarity: The following expressions compare two integers:

$ [ 1 -gt 0 ] && (( 1 > 0 )) && echo "comparing two integers"

and the next expressions compare two strings:

$ [ a > 0 ] && [[ (( a > 0 )) ]] && echo "this is a comparison of two strings"

The difference is made by the operators, not from the values! The following expression is always evaluated to true, because the return status of the arithmetic expansion is always true, even if the expanded string is evaluated to false:

$ [ $(( 0 > 1 )) ] && echo "always true, even if arithimetic expression is false"

Loops

Bash supports three different types of loops. The until loop executes a list of commands until the test command exits not on success:

$ a=0
$ until (( a > 2 ))
do
  echo $a
  let a+=1;
done

The loop body is always encapsulated with the keywords do and done. The while loop executes a list of commands as long as the test command exits on success:

$ a=3
$ while (( a > 0 ))
do
  echo $a
  let a-=1
done

A defined number of iterations can be done by the for loop:

$ for file in *
do
  echo $file
done

The for loop iterates over the words which were expanded by the filename expansion "*". Before the commands of the loop body are executed, each word is saved once in the loop variable file. The for loop can also be written as in C:

$ files=(*)
$ for ((i = 0; i<${#files[@]}; i++))
do
  echo ${files[$i]};
done

The loop variable i is initialized by the first expression within the round brackets after the keyword for. As long as the second expression is expanded by arithmetic expansion to true the commands from the loop body are executed once. Afterwards the third command of the loop head is executed. An endless loop can be written by:

while [ 1 ]
do
  echo "Press [CTRL]+[C] to escape from the command";
done

To cancel the execution of a loop the break command is used:

while [ 1 ]
do
  echo "I will terminate";
  break;
done

To proceed with the next iteration before reaching the end of the function body, the continue command is used:

$ while [ 1 ]
do
  continue
  echo "never reached";
done

Loops can be nested:

$ for((i=0; i<3; i++))
do
  for((j=0; j<3; j++))
  do
    echo "($i,$j)";
  done
done

To break the execution of the nth loop, the break command with label n is used:

$ while [ 1 ]
do
  while [ 1 ]
  do
    echo "reached only once";
    break 2;
  done
done

The first surrounding loop of the break command is referenced by 1, the second by 2. To continue the nth loop is done similar by the continue command. The output of loops can also be redirected:

$ for((i=0; i<3; i++))
do
  echo -n "$i**2=";
  echo $((i**2));
done > results

Shell Functions

Bash allows reuse of a list of commands by functions. Therefore, a function has to be declared before it is first called. The declaration may start with the keyword function:

$ function greet {
  echo "Welcome ${1}, today is $(date)."
}

In absence of the keyword function brackets are needed after the function name:

$ greet() {
  echo "Welcome ${1}, today is $(date)."
}

The function body is always encapsulated by curly braces. The commands within the body are executed each time the function is called. Functions are invoked like commands:

$ greet $USER
Welcome pa, today is Tue Oct 16 17:02:08 CEST 2012.

The parameters of the function are given after the functions name. Only inside the function body the parameters are accessible through the positional parameters: $1, $2, $3, .... Beware of word splitting:

$ parameters() {
  echo '"' $1 $2 $3 '"' a film by $4
}
$ words="A Clockwork Orange"
$ parameters ${words} "Stanley Kubrick"
" A Clockwork Orange " a film by Stanley Kubrick

Its not possible to pass arrays to functions:

$ parameters() {
  echo $*;
}
$ fruits=(orange lemon apple melone banana)
$ parameters $fruits
orange

Functions declared last overwrite functions and commands with the same name:

$ function ls() {
  echo "Sorry I cover the ls command";
}
$ ls
Sorry I cover the ls command

Functions are unset like variables by the unset command:

$ unset ls

The return status (exit status) of the function declaration is 0 unless no syntax error occurs. The return status of the function is the return status of the last executed command. The return command returns from a function with a given status.

$ returns() {
  return $1;
  echo "never reached";
}
$ returns 72
$ echo $?
72

The return status can be interpreted as return value of the function. The return value is read from the special parameter $?. Local variables can be declared inside functions by the keyword local only. Local variables hide the actual value of a global variable:

$ int=0
$ function klocal() {
  local int=1
  echo $int
}
$ echo $int && klocal && echo $int
0
1
0

Functions are nestable:

$ wrapper() {
  local USER="insider"
  mylogin() {
    echo "Hello $USER"
  }
  mylogin
}
$ wrapper && mylogin
Hello insider
Hello pa

The parameter USER is covered inside wrapper but not if calling mylogin directly. So there are really no closures in Bash. Otherwise calling mylogin directly would output insider also. A function can call itself recursively:

$ recursive() {
  if (( ${1:-0} < 4 ))
  then
    recursive $((${1:-0} + 1))
  fi
  echo "${1:-0}. recursive call"  
}
$ recursive
4. recursive call
3. recursive call
2. recursive call
1. recursive call
0. recursive call

The output of a function can be redirected also:

redirect() {
 date
} > /dev/null
$ redirect

Bash Scripts

Bash can read and execute commands from a file with the extension ".sh". The file extension is the same under UNIX Shell:

$ echo "echo \$0, \$*" > bin/commands.sh

These files are called Bash scripts. To execute commands from a file apply its path to the bash command:

$ bash bin/commands.sh

To execute the commands from file like any command the executable flag of the file has to be set:

$ chmod u+x bin/commands.sh

If the file could not be found within a directory, that is listed in the PATH parameter, the relative:

$ ./bin/commands.sh

or the absolute path of the file has to be used:

$ /home/pa/bin/commands.sh

To find a command, which parent directory is not listed in the PATH variable:

$ echo $PATH
/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

the name of the directory is appended to the value of PATH by:

$ PATH=$PATH:~
$ commands.sh

To ensure, that the Bash interpreter is used a "#!/bin/bash" shebang is set into the first line of the script:

#!/bin/bash
echo $0, $*

The shebang treats referenced interpreter to execute the following lines. Even Perl, Python or Ruby start this usually. Script parameters are set after the command name:

$ commands.sh param1 param2

The parameter values are accessiable through the positional parameters: $1, $2,... All parameters can be selected by the $@ or $* parameters. $0 holds the script name. Options can be parsed using the getopts command:

#!/bin/bash
while getopts "ne:" optname
do
 case $optname in
  n) echo "-n - no value expected";;
  e) echo "-e - $OPTARG";;
 esac
done

The expected options are listed after getops. Options that require a value are marked by a following ":" operator:

$ options.sh -e
/home/pa/bin/options.sh: option requires an argument -- e

Each time getopts is called, the next option name is stored in the variable optname and it's value in the OPTARG parameter:

$ options.sh -n -e "A Clockwork Orange"
-n - no value expected
-e - A Clockwork Orange

Beware of word splitting: Without double quotes the value of -e expands "A" only:

$ options.sh -e A Clockwork Orange
-e - A

To process a Bash script in situ the source command or its alias the "." operator are used:

$ . bin/options.sh

In contrast to the script execution described above: The execution is done within the current shell. source or "." enables the possibility to reuse the code of a scripts like a library. But the commands are read and executed immediately.

Bash scripts are processed in a separate shell. Their execution environment is inherited from the context in which it was started. Depending on the invocation a shell will be read and executed from various locations. This files holds parameters like PATH. On startup, interactive login shells, started by:

$ exec -l bash

or shells started with --login:

$ bash --login

read and execute commands from /etc/profile and from first file found of either ~/.bash_profile or ~/.bash_login or ~/.profile. So if ~/.bash_profile is found, ~/.bash_login and ~/.profile not read. Note: The latter executed file overwrites variables, traps, and functions of the previous. An interactive non-login shell:

$ bash

reads and executes commands from ~/.bashrc. Usually, .profile contains a reference to .bashrc:

if [ -f "$HOME/.bashrc" ]
then
  . "$HOME/.bashrc"
fi

The none interactive shell, like:

$ bash bin/commands.sh

reads and executes commands from the file referenced by the variable BASH_ENV. Note: Invoking a Bash script by exec system call, like cron does, leads to an environment determined by the BASH_ENV variable. The environment that is formed for the script started in a login shell is determined by /etc/profile and either ~/.bash_profile or ~/.bash_login or ~/.profile as well as the history of the interactive session.

~/.bashrc or ~/.profile are approbative locations for customizing a shell. Aliases and environment variables can be recorded:

alias make="make -C ~/daten bashscripting"
export ORATAB=LMS

Special Parameters

Bash supports a lot of special read only parameters. The positional parameters 1, 2, 3, ... are set to the parameters which were given to a Bash script or Shell function:

$ ppars() {
  echo $1, $2, $3
}
$ ppars Oranges lemons
Oranges, lemons,

@ and * expands each positional parameter to a single word:

$ ppars() {
  echo $@
}
$ ppars Oranges lemons
Oranges lemons

Within double quotes * expands all postional parameters to a single word only, separated by the first character of the variable IFS:

$ ppars() {
  IFS="~|-"
  echo "$*"
}
$ ppars Oranges lemons
Oranges~lemons

"#" expands to the number of positional parameters:

$ pnum() {
  echo $#
}
$ pnum Oranges lemons  
2

"?" expands to exit status the most recently executed foreground pipe:

$ (exit 255) 
$ echo $?
255

"-" expands to the current option flags set by the set command or given when invoking bash:

$ set -b
$ echo $-
bhimBH

"$" expands to the pid of the shell:

echo $$
4936

"0" expands to the name of the bash script:

$ echo $0

"_" expands initially to the absolute path of the Bash script, otherwise to the last argument of last called command:

$ ls /root /home
$ echo $_
/home

Examples

Programcode 1: lstree
Programcode 2: Example - log rotate