More Shell Scripting

If Statements

In the last lesson on shell scripting, we saw how a script could access its arguments. Sometimes, we want to ensure that a certain number of arguments are passed. For example, in one version of our backup script, we took the files to put in the archive as arguments:


#!/bin/bash

filename="backup-$(date +%Y-%m-%d).tar.gz"

echo "Backing up..."
tar -czvf $filename $*
echo "All done backing up!"

If we run this without arguments, we get this output:

ifinlay@cpsc:~$ backup.sh
Backing up...
tar: Cowardly refusing to create an empty archive
Try `tar --help' or `tar --usage' for more information.
All done backing up!

We would ideally test if the user passed no arguments and print our own error message instead, and not print the inaccurate "All done backing up!" message.

This can be accomplished with an if statement in our script which could be done with:


if [ $# -lt 1 ]
then
    echo "Please pass some file(s) to backup!"
    exit 1
fi

Recall that the $# variable is set to the number of arguments our script received. The "-lt" comparison is a less than test. Because the shell uses "<" for input redirection, that symbol cannot be used for less than. So this tests if the number of arguments we received is less than one. If so, we do the commands between the "then" line and the "fi" line. "fi" being "if" backwards.

Note the spacing around the brackets containing the condition. These spaces are mandatory and the shell is quite picky about syntax. The "then" also has to be on the next line after the "if".

We can also add an else clause which has the following form:


if [ $# -lt 1 ]
then
    echo "Please pass some file(s) to backup!"
    exit 1
else
    echo "At least one argument supplied.  Proceeding..."
fi

We can also add "elif" conditions as follows:


if [ $# -lt 1 ]
then
    echo "Please pass some file(s) to backup!"
    exit 1
elif [ $# -lt 2 ]
then
    echo "Exactly one argument supplied. Proceeding..."
else
    echo "More than one argument supplied.  Proceeding..."
fi

As is the case for most languages, the shell tests the conditions from top to bottom. As soon as one satisfies its condition, the body is executed. If none do, the else statements are executed.

Condition Tests

In addition to "-lt", the shell supports the following comparison tests:

Test	Meaning
`-lt`	Less than.
`-gt`	Greater than.
`-eq`	Equal to.
`-ne`	Not equal to.
`-le`	Less than or equal to.
`-ge`	Greater than or equal to.

In addition to the above numeric operators, we can use the following for general string comparisons:

Test	Meaning
`=`	Equal to.
`!=`	Not equal to.

You should use these for strings which should be compared exactly instead of numerically. For instance the condition 03 = 3 would be false because the strings are not the same. 03 -eq 3 would be true, however.

There are also a few unary string conditions we can use:

Test	Meaning
`-n`	The string has non-zero length.
`-z`	The string has zero length.

These are useful for testing if a variable is set, since the shell gives the empty strings for variables which have never been set. The following snippet of a script tests if the EDITOR environment variable is set:


if [ -z $EDITOR ]
then
    echo 'Please set your $EDITOR variable'
fi

We can also perform several tests on files:

Test	Meaning
`-e`	Tests if the file exists.
`-s`	Tests if the file exists and is non-empty.
`-f`	Tests if the file is a regular file (i.e. not a directory or special file).
`-d`	Tests if the file is a directory.
`-r`	Tests if the file is readable (for whoever is running the script).
`-w`	Tests if the file is writeable (for whoever is running the script).
`-x`	Tests if the file is executable (for whoever is running the script).

We can invert any condition by preceding it with the ! not operator. The following tests if a file is not readable:


if [ ! -r file ]
then
    echo "Cannot read 'file'"
fi

We could use these to improve our backup script which uses a file to store the current backup number. Before, we assumed the file existed and was readable when reading it, and we assumed that it was writeable when we wrote it.

These assumptions aren't made in the following version:


#!/bin/bash

# test if the backup-number does not exist or is empty
if [ ! -s .backup-number ]
then
    # if not, then create it with a 0 value
    echo 0 > .backup-number
fi

# test if we can read backup-number
if [ ! -r .backup-number ]
then
    echo "Error, insufficient permissions to read .backup-number"
    exit 1
fi

# now we can be sure that we can read it
current=$(cat .backup-number)

# increment the current backup number
next=$(($current + 1))

# test if we are allowed to write to a file called .backup-number
if [ ! -w .backup-number ]
then
    echo "Error, insufficient permissions to write to .backup-number"
    exit 1
fi

# overwrite the file with the new backup number
echo $next > .backup-number

# use current as a version
filename="backup-${current}.tar.gz"

echo "Backing up..."
tar -czvf $filename $*
echo "All done backing up file $filename."

Just as in other types of programs, checking and handling error cases can be a lot of work. Notice how our script calls exit 1 in the error cases. This gives it the return code 1 so that other programs will know if it fails. By default, all scripts exit with the error code of 0 meaning success.

Shifting Arguments

One common way of going through arguments is to use the shift command. This command essentially shifts all the arguments from right to left. This way you can always deal with $1 and shift the next arguments over, without needing to keep track of argument numbers. The following program demonstrates this by printing out the first three arguments it's given:


#!/bin/bash

echo $1
shift
echo $1
shift
echo $1

ifinlay@cpsc:~$ shift.sh this is a test
this
is
a

As a practical example, say we want to give our backup script an optional first argument, -q (for quiet), which will make it not print the messages to the screen, but just get on with the work.

We can start the script by checking if the first argument is "-q". If it is, then we set a variable and shift past it (so that the -q doesn't get treated as a file name):


quiet=0
if [ "$1" = "-q" ]
then
    quiet=1
    shift
fi

Then we can later check the variable:


if [ $quiet = 1 ]
then
    echo "Backing up..."
fi

Note that the quiet variable uses 0/1 values because the shell does not include a real boolean type.

While Loops

Like most languages, shell scripts allow while loops which continue to execute a block of code while some condition remains true. The script below demonstrates while loops:


#!/bin/bash

# read the value from the user
read -p "Enter a value: " n

# set factorial to 1
fact="1"

# loop until it's 1
while [ $n -gt "1" ]
do
    # multiply n into factorial
    fact=$(($fact * $n))

    # decrement n
    n=$(($n - 1))
done

# print the result
echo "$fact"

This script uses a while loop to compute factorials. You would probably want to use a more conventional programming language for a task like this, but the shell can do it.

For Loops

Shell scripts can also make use of for loops which allow us to loop through multiple values. This is often used to loop through script arguments. In the backup script above, we use "$*" to reference all of the arguments which we pass on to tar. Sometimes we may want to loop through them ourselves.

The following script shows how this can be done with a for loop:


#!/bin/bash

for file in $*
do
    echo "Processing $file"
done

The for loop works by taking a set of values ($* in this case), and assigning them one by one to a variable (file in this case). In the body of the loop, we can reference $file which will refer to each value in turn. Below is a sample run of this script:

ifinlay@cpsc:~$ loop.sh thing1.txt thing2.txt thing3.txt
Processing thing1.txt
Processing thing2.txt
Processing thing3.txt

The "set of values" for a for loop can be any set of values separated by spaces:


#!/bin/bash

days="Mon Tue Wed Thu Fri Sat Sun"

for day in $days
do
    echo "Today is $day"
done

Which produces the following output:

ifinlay@cpsc:~$ days.sh
Today is Mon
Today is Tue
Today is Wed
Today is Thu
Today is Fri
Today is Sat
Today is Sun

We can even do a counting type of for loop using the seq command. seq produces a list of numbers in a sequence. If you pass seq one argument, it gives you the numbers from 1 up to that number. If you pass it two arguments, it gives you the range of numbers between them (including both end points). If you pass it three, it counts from the first to the last by increments of the middle:

ifinlay@cpsc:~$ seq 4
1
2
3
4
ifinlay@cpsc:~$ seq 3 7
3
4
5
6
7
ifinlay@cpsc:~$ seq 1 2 9
1
3
5
7
9

We can use this handy program in a script to loop over a range of numbers:


#!/bin/bash

for i in $(seq 5 -1 0)
do
    echo $i...
done

echo "Blastoff!"

Which produces:

ifinlay@cpsc:~$ countdown.sh 
5...
4...
3...
2...
1...
0...
Blastoff!

Functions

Just as with regular programs, having a large script with no organizational structure can get messy. For this reason it's possible to create functions inside of shell scripts.

Shell functions basically serve as "mini-scripts" which can be called by name. The following example demonstrates how functions may be created and called:


#!/bin/bash

dostuff () {
    echo "Inside the dostuff function!"
}

# we can now call the function as if it were a command
dostuff

As you can see, once a function is defined, it can be called the same way as a regular command can.

We can also pass arguments to a function in the same way as they are passed to a command. A function also can read its arguments using the same special variables as a script:


#!/bin/bash

dostuff () {
    echo "dostuff called with:"
    for arg in $*
    do
        echo $arg
    done
}

# call dostuff with some arguments
dostuff a b c

# print the overall script arguments
for arg in $*
do
    echo $arg
done

Inside a function, only the function's arguments are read:

ifinlay@cpsc:~$ functions.sh thing1 thing2
dostuff called with:
a
b
c
thing1
thing2

As you can see, the function reads its own arguments independently of the overall script arguments. If you need a function to be able to see the script arguments, they would need to be passed in explicitly.

When to Use Shell Scripts

Shell scripts are great for the following cases:

Automation
Any time you are manually doing something repetitive, you should ask yourself whether the process can be automated with a script. Whether these are things that you only need to do a handful of times, or something more lasting, a script can be a great solution. I often write "one off" scripts to solve some specific task and then delete them.
System Management
Scripts are good for things like configuring, installing or removing software. By using a script, you can make a process like this easier and less error-prone. Installing third party software on Unix systems usually involves running an installation script provided by the developers.
Gluing Together Programs
Because shell scripts can run other programs so easily, and even connect their inputs and outputs together with pipes or redirections, they are great for making these programs work together. If there are existing commands that can do most of the work, a script can often finish the job.