A shell script is a set of Unix commands which are placed in a file and are executed from top to bottom, as if you typed them in manually.
Shell scripts allow us to write Unix "programs" which can automate tasks done in the shell. Thus far we have used the shell to execute single line commands but, as we will see, the shell allows for many programming constructs as well.
A shell script is just a text file containing commands. Sometimes scripts are given the ".sh" extension, and sometimes they are given no extension at all. I will use the .sh extension for clarity. We can create a simple script with Vim:
ifinlay@cpsc:~$ vim hello.sh
Then type in the following:
#!/bin/bash
echo "Hello World!"
The first line is called a shebang and tells the shell which interpreter should run the script. While not strictly necessary, it is a good idea to put in place. Here we specify that the script should be executed by bash.
Note: A shebang can be used for programs as well. For
instance, if you place "#!/usr/bin/python3" at the top of a
Python program, then you can run it by name, without needing the python3
command.
Any line in a shell script which begins with a pound sign (including the shebang line), is not executed by the shell. This allows us to put comments in scripts.
The second line is a command to execute, which just runs the echo
command to print a "Hello World!" message to he the screen.
To run the shell script, we can either pass it as an argument to bash:
ifinlay@cpsc:~$ bash hello.sh Hello World!
Or, as is more common, we can execute it directly. To do this, we must first give the script executable permissions:
ifinlay@cpsc:~$ chmod u+x hello.sh ifinlay@cpsc:~$ ls -l hello.sh -rwxrw-r-- 1 ifinlay faculty 22 Jul 15 10:38 hello.sh
We now can execute it directly:
ifinlay@cpsc:~$ ./hello.sh Hello World!
The ./ portion specifies that we should run the file called hello.sh which is in the current directory. Without this, the shell would search our PATH environment variable.
If we want to be able to run the script from any working directory, we need to put it in a location which is in our PATH. I create a directory for my scripts at "~/bin" and make sure that directory is in my PATH. You can refresh your memory of the PATH environment variable on this page.
If we do this, then we can simply run our script as:
ifinlay@cpsc:~$ hello.sh Hello World!
We have seen environment variables which are variables holding some string of text which are available to any command running in your shell. Scripts can also create their own variables for storing things.
The following example uses a variable called name:
#!/bin/bash
name="Ian"
echo Hello $name
echo Goodbye $name
When we run this script the output is:
ifinlay@cpsc:~$ hello.sh Hello Ian Goodbye Ian
When a command contains a variable reference, the shell
will substitute it before executing the command. So the
shell replaces "Hello $name" with "Hello Ian" and passes
that string to the echo command.
Notice that when a variable is created, there is no $ sign in its name, but referencing a variable does begin with the $ sign.
Also note that, unlike most programming languages, we cannot put spaces around the = sign in a variable assignment. This line:
name = "Ian"
produces this error:
ifinlay@cpsc:~$ hello.sh hello.sh: line 3: name: command not found
Shell scripts are somewhat more syntactically picky than most other languages.
Sometimes we may want a variable to have text immediately after it. Suppose we are writing a script which backs up a file, and the original filename is stored in a variable:
#!/bin/bash
filename="data.txt"
echo Backing up $filename
cp $filename $filenamebackup
This script attempts to copy the file name it is given to the same file name with "backup" appended to it. When we run this, however, we will get this error:
ifinlay@cpsc:~$ backup.sh Backing up data.txt cp: missing destination file operand after 'data.txt' Try 'cp --help' for more information.
The problem is that the shell sees the variable name
as "filenamebackup" which is not defined. Rather than
give an error if an undefined variable is used, the shell
simply uses an empty string for its value. This means that
it passes only the argument "data.txt" to the cp
command, hence the error about a missing argument.
To fix this, we can use another form of variable reference, demonstrated below:
#!/bin/bash
filename="data.txt"
echo Backing up $filename
cp $filename ${filename}backup
By wrapping the name of the variable in curly braces, it's clear to the shell that the variable name is "filename" which is expanded correctly.
The examples above use double-quotes for variable assignment. This is not really necessary in these cases, and the scripts will run the same without quotes.
However, if the values being assigned contain spaces, we cannot forgo the quotes:
#!/bin/bash
name=Ian Finlayson
echo Hello $name
echo Goodbye $name
ifinlay@cpsc:~$ hello.sh hello.sh: line 3: Finlayson: command not found Hello Goodbye
Here, we need quotes around the name:
#!/bin/bash
name="Ian Finlayson"
echo Hello $name
echo Goodbye $name
ifinlay@cpsc:~$ hello.sh Hello Ian Finlayson Goodbye Ian Finlayson
I generally use the quotes even when not necessary.
We could also use single quotes in the example above:
name='Ian Finlayson'
In this case, the behavior is exactly the same. However, there is an important difference between single and double quotes.
Single quoted strings are not expanded at all by the shell prior to use. This means they do not have variables replaced with their values.
Suppose we are writing a line in a script to tell the user to set their EDITOR environment variable. Here, we may want to output the actual text '$EDITOR' and not the value of the variable.
The following commands demonstrate this difference:
ifinlay@cpsc:~$ echo 'Please set your $EDITOR variable' Please set your $EDITOR variable ifinlay@cpsc:~$ echo "Please set your $EDITOR variable" Please set your vim variable
See that the single-quoted version did not get its included variable expanded, but the double-quoted version did!
This is also sometimes needed on the shell if you want to suppress a wild card expansion. For instance, if you want to pass a program the '*' character, you would need to wrap it in single quotes:
ifinlay@cpsc:~$ echo * backup backup.sh bin config downloads hello.sh projects ifinlay@cpsc:~$ echo '*' *
Here, the first command had the * expanded by the shell so that echo is passed a list of all the files. In the second command the single quotes cause an actual * character to be passed directly to echo, without expansion.
There is another kind of quote which you will sometimes
see in shell scripts which is the back-tick quotes ``.
These behave quite differently than the others. They
run whatever is between them as a command and evaluate to the
output of that command. This allows us to capture the result
of a command and save it in a variable or use in
some other command:
#!/bin/bash
now=`date`
echo The current time is ${now}.
When run this will produce:
ifinlay@cpsc:~$ ./now.sh The current time is Wed Jul 15 11:50:54 EDT 2018.
However, this method of capturing the result of a command
is deprecated (though still common). Instead, its better to
wrap the command as $(date):
#!/bin/bash
now=$(date)
echo The current time is ${now}.
ifinlay@cpsc:~$ ./now.sh The current time is Wed Jul 15 11:54:04 EDT 2018.
These are preferred because they are more easily nested.
Suppose we want to write a script which will backup our projects directory into a compressed tar file which has the current date as a part of the name.
The first thing we could start with is getting the
current date in a nice format. The output of date
contains spaces which are not nice to put in file
names.
Luckily date is a very flexible program
and supports passing a "format string" which describes
how the date should be formatted. The man page has
all the details, but we can get a simple "year-month-day"
output like:
ifinlay@cpsc:bin$ date +%Y-%m-%d 2018-07-15
We can go ahead and make a variable for the file name which contains this formatted date like so:
filename="backup-$(date +%Y-%m-%d).tar.gz"
Here we are making the filename variable equal to the text "backup" concatenated with the result of running the above date command, then concatenating the extension ".tar.gz"
Next we can make a variable which refers to the directories and/or files to backup:
files="projects"
This way we can add to the list of directories to back up simply by changing that line.
The whole script might look like this:
#!/bin/bash
# the files to backup
files="projects"
# the filename has the date in it
filename="backup-$(date +%Y-%m-%d).tar.gz"
# the actual work
echo "Backing up..."
tar -czvf $filename $files
echo "All done backing up!"
Note that this script has comments in it which begin with the pound character.
We can run it as:
ifinlay@cpsc:~$ backup.sh Backing up... projects/ projects/input.py projects/output.py projects/main.py All done backing up!
Notice that the lines beginning with "projects/" get printed by tar itself, because we passed it the "-v", verbose flag.
Instead of putting the list of files directly in the shell script above, we may want to pass them to our script as arguments.
When we run a script, the shell populates some special variables that allow us to see our arguments:
| Variable | Meaning |
| $# | The number of arguments passed to the script. |
| $0 | The name of the script itself as it was written on the command line. This is rarely useful. |
| $1, $2, ... | The first, second, etc. argument to the script. |
| $* | All of the arguments to the script written together. |
The following script shows how these are populated:
#!/bin/bash
echo '$#' is $#
echo '$0' is $0
echo '$1' is $1
echo '$2' is $2
echo '$*' is $*
Below shows an example run of this script:
ifinlay@cpsc:~$ args.sh have some arguments $# is 3 $0 is args.sh $1 is have $2 is some $* is have some arguments
If we do not supply arguments, they are simply blank:
ifinlay@cpsc:~$ args.sh $# is 0 $0 is args.sh $1 is $2 is $* is
We can now make our backup script take the files as parameters, and use $* to reference them:
#!/bin/bash
filename="backup-$(date +%Y-%m-%d).tar.gz"
echo "Backing up..."
tar -czvf $filename $*
echo "All done backing up!"
We can now pass any files we like:
ifinlay@cpsc:~$ backup.sh data.txt projects file.txt Backing up... data.txt projects/ projects/input.py projects/output.py projects/main.py file.txt All done backing up!
If we pass nothing, tar will complain. We'll see how to
detect errors in the next lesson.
In addition to having user interaction via passed arguments, we
can also have scripts get user input directly with the read
command.
read gets user input and stores it in a variable. It takes
an optional prompt which is passed after a "-p" flag.
The following script demonstrates this:
#!/bin/bash
read -p "Enter your name: " name
echo Hello $name!
Running this, it will ask for our name:
ifinlay@cpsc:~$ input.sh Enter your name: Ian Finlayson Hello Ian Finlayson!
read also accepts a "-s" silent option in which it will
not print back our input:
#!/bin/bash
read -p "Enter your user name: " name
read -s -p "Enter your password: " password
This script uses the -s flag to avoid your password showing up in the terminal:
ifinlay@cpsc:~$ input.sh Enter your user name: ifinlay Enter your password:
To perform mathematical calculations, in the shell, we can
place expressions between $(( and )) delimiters.
That will evaluate to the value of the mathematical expression:
ifinlay@cpsc:~$ echo $((3 + 4)) 7 ifinlay@cpsc:~$ echo $((3 + 4 * 5)) 23 ifinlay@cpsc:~$ echo $((10 / 5)) 2
This can allow us to do things like perform arithmetic in shell scripts. For instance, the following backup script uses an incrementing number to version its backups instead of the current date:
#!/bin/bash
# read the current backup number from a hidden file
current=$(cat ~/.backup-number)
# increment the current backup number
next=$(($current + 1))
# overwrite the file with the new backup number
echo $next > ~/.backup-number
# use current as a version
filename="backup-${current}.tar.gz"
echo "Backing up..."
tar -czvf $filename $*
echo "All done backing up file $filename."
This script uses a configuration file called "~/.backup-number" which
stores the number to use in the backup. Ideally the script should
create this file if it does not exist, but we'll see how to that next time.
Currently, it assumes the file is there and reads its contents into a variable
called "current". It then calculates "current + 1"
into the variable "next". Note that the variable $current is
expanded inside of the $(()) expression.
It then overwrites the configuration file with
the new value by redirecting an echo into the file.
Our script now automatically increments the backup number:
ifinlay@cpsc:~$ echo 0 > ~/.backup-number ifinlay@cpsc:~$ backup.sh projects Backing up... projects/ projects/input.py projects/output.py projects/main.py All done backing up file backup-1.tar.gz. ifinlay@cpsc:~$ backup.sh projects Backing up... projects/ projects/input.py projects/output.py projects/main.py All done backing up file backup-2.tar.gz. ifinlay@cpsc:~$ backup.sh projects Backing up... projects/ projects/input.py projects/output.py projects/main.py All done backing up file backup-3.tar.gz.
Note that shell scripts do not have any concept of variable
types. All variables are just text. If the variable text happens
to contain a number, it will be used in calculations contained in
$(()), but there is nothing like type checking or declaration.
Non-numerical variables just have the value 0 when used in expressions.
Shell scripts provide a powerful means of saving commands in a file where they can easily be run all at once. Writing shell scripts can be tricky, but once written, they can be run every time we need them.
In the next lesson, we will see the shell actually supports programming language features such as functions, loops and conditions.
Copyright © 2024 Ian Finlayson | Licensed under a Creative Commons BY-NC-SA 4.0 License.