14. Redirections

The output of a command is usually printed to its standard out, stdout. By default the stdout is connected with the terminal. That's why the user sees it. But the output may also be saved by redirecting the stdout to a file using the ">" operator:

$ env > file

The output of the env command is written the file file. The content of an existing file will be truncated (deleted). To append content to the end of a file the ">>" operators is used:

$ date >> file
$ tail file
_=/usr/bin/env
Thu Oct 11 17:49:55 CEST 2012

Technically stdout is a file descriptor, that is connected to the terminal by default. By redirection, the descriptor is connected to another target. All file descriptors are listed within /dev/fd:

$ ls -l /dev/fd/
lrwx------ 1 pa pa 64 Oct  8 19:32 0 -> /dev/pts/1
lrwx------ 1 pa pa 64 Oct  8 19:32 1 -> /dev/pts/1
lrwx------ 1 pa pa 64 Oct  8 19:32 2 -> /dev/pts/1

0 is file descriptor for reading, called standard input (stdin), 1 for writing, called standard output (stdout) and 2 for writing errors, called standard error (stderr). Using file descriptors the redirection from above is written as:

$ env 1> file

To redirect errors from stderr to a file its file descriptor 2 is used explicit:

$ ls undefined 2> file

To redirect both, stdin and stderr, to the same file, two redirections can be used:

$ ls ~ undefined 1>file 2>&1

Redirections are being processed from left to right. First, stdout is connected to file, second the output of stderr is duplicated to the same target as stdout. Beware of the order, cause:

ls ~ undefined 2>&1 >file
ls: cannot access undefined: No such file or directory

doesn't redirect stderr to file. First stderr is duplicated to same output as stdout - the terminal. Second, stdout is connected to file. stdin and stderr can be redirected to the same target by using the "&>" or ">&" operators:

$ ls ~ undefined &>file

To discard all errors message which are printed to stderr, it is redirected to the null device:

$ ls ~ undefined 2>/dev/null
/home/pa:
bin datas fifo file poem

To discard every output, the "&>" or ">&" operators are redirects to the null device:

$ ls ~ undefined &>/dev/null

By default, stdin is connected with the keyboard:

$ grep pa
123
pappel
pappel

The command grep matches each line that is entered by the keyboard against the pattern pa. [CTRL]+[D] terminates the input. To redirect stdin from a file, the "<" operator is used:

$ grep pa < file
/home/pa:

The combination of redirecting the input and can be written as:

$ grep pa < file > output

But redirecting from stdin isn't needed by grep:

$ grep pa file > output 

Custom file descriptors can be created by the exec command. To create a file descriptor for writing:

$ exec 4>poem

The file descriptor can be used now for writing:

$ echo "Oranges and lemons" >&4

A second write dosn't truncate poem, it appends the input to the end of the file:

$ echo "say the bells of St. Clement's" >&4

Similar a file descriptor for reading is created:

$ exec 5<poem

The file descriptor can be used for reading:

$ cat <&5
Oranges and lemons
say the bells of St. Clement's

Reading twice is without effect, because the file pointer points to the end of the file:

$ cat <&5
$

But appended content is added after the file pointer. So it can be read:

$ echo "You owe me five farthings" >&4
$ cat <&5
You owe me five farthings

To prevent a file from being overwritten the noclobber option has to be set:

$ set -o noclobber
$ echo "Here comes a candle to light you to bed" > poem
bash: poem: cannot overwrite existing file

To overwrite a file even if noclobber is set the ">|" operator is used:

$ echo "Here comes a candle to light you to bed" >| poem

To unset the noclobber option the set command is used again with "+o" instead of "-o":

$ set +o noclobber

File descriptors can be used when reading input by the read command. The descriptor will be selected by the -u option:

$ exec 5<poem
$ read -u 5 verse
$ echo $verse
Here comes a candle to light you to bed

A file descriptor can be used bidirectional for writing and reading also:

$ exec 6<>poem

File descriptors can be moved by the ">&digit-" operator:

$ exec 7>&6-
$ ls -l /dev/fd/
lrwx------ 1 pa pa 64 Oct 12 16:36 0 -> /dev/pts/2
lrwx------ 1 pa pa 64 Oct 12 16:36 1 -> /dev/pts/2
lrwx------ 1 pa pa 64 Oct 12 16:36 2 -> /dev/pts/2
lr-x------ 1 pa pa 64 Oct 12 16:36 3 -> /proc/2639/fd
l-wx------ 1 pa pa 64 Oct 12 16:36 4 -> /host/Users/pa/Documents/daten/poem
lr-x------ 1 pa pa 64 Oct 12 16:36 5 -> /host/Users/pa/Documents/daten/poem
lrwx------ 1 pa pa 64 Oct 12 16:36 7 -> /host/Users/pa/Documents/daten/poem

Descriptor 6 will be closed. By default it is moved to 1 (stdin). Similar for stdout, the reverse operator "<&digit-" exists:

exec 6<&7-

To close a file descriptor the ">&-" operator is used:

$ exec 6>&-

The "<&-" operator can be used also:

$ exec 5<&-

File Descriptors can also be used to read from or write to named pipes (fifos). The fifos are usual used for interprocess communication. They can be accessed by paths but their content is kept in the memory. Fifos are created by the fifo command:

$ mkfifo fifo
exec 4<fifo

The fifo blocks until a second shell opens the fifo for writing:

$ exec 5>fifo
$ while (( 1 ))
do
  date >&5;
  sleep 1;
done

Now within the first shell the content can be read from the file descriptor 4:

$ cat <&4
Mi 10. Okt 18:06:00 CEST 2012
Mi 10. Okt 18:06:01 CEST 2012
Mi 10. Okt 18:06:02 CEST 2012
Mi 10. Okt 18:06:03 CEST 2012
Mi 10. Okt 18:06:04 CEST 2012

[CTRL]+[C] terminates the program. On some systems file descriptors can also access tcp or udp resources:

$ exec 4<>/dev/tcp/pamoller.com/80
$ echo -en "GET / HTTP/1.1\n\r\n\r" >&4
$ cat <&4

The First line opens a tcp connection to the web server at pamoller.com on port 80 for writing and reading. The echo command prints a HTTP get request to the TCP socket. The HTTP server response is read by the cat command from the same file descriptor. File descriptors can be stored in variables by using the "{...}<" operator:

$ exec {desc}<poem

Internaly a file descriptor greater 10 is assigned to the variable descriptor:

$ cat <&$desc
Here comes a candle to light you to bed

A special kind of redirections are here documents. Here documents are expressed by the "<<WORD....WORD" operator:

cat > poem <<EOF
"Oranges and lemons", say the bells of St. Clement's
"You owe me five farthings", say the bells of St. Martin's
"When will you pay me?" say the bells of Old Bailey
"When I grow rich", say the bells of Shoreditch
"When will that be?" say the bells of Stepney
"I do not know", says the great bell of Bow
Here comes a candle to light you to bed
And here comes a chopper to chop off your head!
Chip chop chip chop – The last man's dead.
EOF