/bin/sh. The Bourne Shell. The Bell Shell. The completely retarded pedantic bastard shell from hell28.1. Call it what you will, no matter what Unix system you're using, it's bound to be there. So here's a brief section on how you can program it in its own built-in scripting language28.2.
Believe it or not, sh scripts are really really useful things. The basic idea is that a shell script is a plain ASCII file, containing a sequence of commands you'd ordinarily type in on the command line. However, if the list of commands needed to perform some function is quite long or complex, it makes sense to type them once in a script and run the script each time after that (which is also a hell of a lot faster than typing 1000 lines every time28.3).
In addition to commands that you would type in at the command prompt, there are other features, such a flow control statements, usually only found in high level languages. This facility makes it easy to do perform New And Exciting And Powerful tasks.
Type "help" and press enter. If you're running bash you should get a long screed that begins by telling you you're running bash. If you get something else, start up bash by typing
$ bash $
You can also make bash your login shell, if it isn't already; see elsewhere in this Manual.
A shell program is a straight ASCII file containing a series of shell commands, and is input by any means that one would input text into the Unix system --- vi, cat, ed for normal people and emacs or aXe for masochists.
Theoretically, a shell script should start with the magic incantation #!/bin/sh on the first line and hard up against the left margin. In practice, this is generally not required as most modern systems assume a program is a shell script if it is ASCII text and executable28.4. One thing to note is that you shouldn't have a hash (#) by itself on the first position of the first line, as this is a incantation to invoke the C-shell (csh) rather than the Hell Shell (sh).28.5 Programming csh is not really discussed in this manual.
Why /bin/sh instead of /bin/bash? Well, on UNIX, /bin/sh is guaranteed to exist, and for Linux systems it's basically bash28.6. However, on other systems, such as FreeBSD, /bin/sh actually is a completely separate and different shell. This will confuse the hell out of you and waste your time. Consider yourself warned.
To demonstrate a shell script, we'll input a program to displays the date and time. (Use vi, the ProgSoc editor of choice, to enter the code below. Then use the UNIX command cat to check everything was input correctly.)
#!/bin/sh date
When a text editor creates a file in Unix, it has permissions directly related to your umask. The umask default permission is read and write, but not execute. To make your shell script executable, add execute permission to your script file using chmod.
foo$ ls -l datescript -rw-r--r-- 1 chris 15 Jan 4 15:20 datescript foo$ chmod +x datescript foo$ ls -l datescript -rwxr-xr-x 1 chris 15 Jan 4 15:20 datescript foo$
Once the script has been successfully entered and made executable, it can be run. There are two ways to execute a shell script. The first and easiest is to type in its name.
foo$ datescript Mon Jan 4 15:21:56 EST 1993 foo$
An alternative is to run the script through /bin/sh as a direct interpreter.
foo$ sh datescript Mon Jan 4 15:21:56 EST 1993
or
foo$ sh < datescript Mon Jan 4 15:21:56 EST 1993
As a special case, you can run the script as part of this shell, rather than starting up a separate one. This is useful if your script sets environment variables, and you want them to be available to the shell you're using. Type:
foo$ . datescript Mon Jan 4 15:21:56 EST 1993
Using sh has the same result, but datescript is treated as a simple ASCII file that sh reads, rather than a program in itself. This knowledge, combined with knowing that !/bin/sh thing - when it's reading in the file, sh just ignores the line, since it starts with a comment.
/bin/sh provides facilities to trace the execution of shell scripts. These are the -x and -v flags to the /bin/sh program. Using the earlier example of datescript, we can apply the -x option to examine the steps passed through by the executing program.
foo% cat datescript #!/bin/sh date foo% sh -x datescript + date Mon Jan 4 15:21:56 EST 1993 foo%
Before each command in the script is executed, it is printed with a preceding + symbol. The program code below (a program called datescript2) is executed as an example:
foo$ cat datescript2 #!/bin/sh date echo foobie doobie who | grep chris echo `hostname` foo$ datescript2 Mon Jan 4 15:22:30 EST 1993 foobie doobie chris console Jan 4 15:19 chris ttyp0 Jan 4 15:20 chris ttyp1 Jan 4 15:20 chris ttyp2 Jan 4 15:20 foo foo% sh -x datescript2 + date Mon Jan 4 15:22:30 EST 1993 + echo foobie doobie foobie doobie + who | grep chris chris console Jan 4 15:19 chris ttyp0 Jan 4 15:20 chris ttyp1 Jan 4 15:20 chris ttyp2 Jan 4 15:20 + echo foo foo
The last echo command comes out as ``+ echo foo'' rather than ``+ echo `hostname`'' because `hostname` is enclosed in backquotes, which have a special meaning to the shell and are explained a little further on.
If the -v option was used instead of the -x option, the debugging output from the execution would not be rewritten, that is, in the above example, ``+ echo `hostname`'' would have appeared instead of ``+ echo foo''.
Under Unix all files, whether ASCII or binary, are treated as simple streams of bytes. There is no structure imposed on the stream other than that imposed by application programs reading data. As a result, all files may generally be treated the same for file redirection, etc.
Because files are simple streams of characters, it becomes a trivial matter to read their contents into programs or for programs to write information to files. Overall, there are three facilities for information interchange. These are called ``standard in'' (stdin), ``standard out'' (stdout) and ``standard error'' (stderr). In addition, other files may be opened and closed by the shell script.
cmd < fileThe arrowhead-in (less than) symbol used in the above fashion caused the contents of file to be input to the command cmd as if they had been typed on the keyboard. Normal keyboard activity is ignored.
cmd << STRING text text STRING
The arrowhead up-'til (double less than) symbol is used to divert stdin in a similar fashion to arrowhead-in. Execution finishes when STRING is reached; this can be any arbitary string of characters with meaning to you. e.g.
foo% mail chris@foo << EOF Hi there chris! Foobie Doobie! EOF foo%
cmd > fileAll stdout output from the execution of cmd is written to a new file called file. If file exists, it is overwritten with the output of cmd.
cmd >> fileSimilar to ``arrowhead out'', arrowhead append causes all output from the execution of cmd to be written to a file called file. However, if this file already exists, the output is appended to the file rather than overwriting it.
2> (file arrowhead out) 2>&1 (arrowhead and)usage:
cmd 2> errfile cmd > file 2>&1
To understand this, you need to know that internally the /bin/sh has special file descriptors for stdin, stdout and stderr. These are 0, 1 and 2 respectively. The ``file arrowhead out'' symbol places any output from the given descriptor---in this case stderr, descriptor 2--- into the named file. Note that
cmd 2>> errfile
is also legal if you wish to append to a file.
The ``arrowhead ampersand'' symbol assigns to file descriptor 2 (stderr) the same output file as file descriptor 1 (stdout).
exec 3> file3 exec 4< file4and then used as described in the `stderr' section above. e.g:
cmd 1>&3 To output to file3 cmd 0<&4 To input from file4
Pipes are discussed in the UNIX section of this manual.
One concept that may seem strange is that devices under Unix are also treated as ordinary files at the shell level. As a result, you can use standard redirection techniques such as those described above to read from and write to devices. Since quite a lot of this type of behaviour is likely considered quite naughty, and will probably get you a good solid spanking, this topic won't be further expounded on. So don't blame me when you get caught sending banners to other logged on users. Whoops.
When writing shell scripts, its useful to know what's in your toolkit. This section gives a brief description of the utilities that get the biggest workout in shell scripts. Each of the commands mentioned here have manual page entries---read them. (man command-name.)
foo% bc 2 + 3 5 ^D foo%
for batch processing,
foo% echo '2 + 3' | bc 5 foo%
The result of the expression is written to stdout, and so can therefore be redirected, assigned to a variable or piped to another application.
foo% cat File1 Hi! I'm file1! foo% cat file2 Hi! I'm file2! foo% cat file1 file2 Hi! I'm file1! Hi! I'm file2! foo% cat < file1 Hi! I'm file1! foo%
foo$ echo hello hello foo$ cat > /tmp/qwerty echo -n "hello " echo there ^D foo$ chmod +x /tmp/qwerty foo$ /tmp/qwerty hello there foo$
foo$ grep tree /usr/dict/words Peachtree rooftree street streetcar tree treetop foo$
In the /bin/sh, there is a small subset of characters that are ``reserved'', that is, they have special meaning to the shell. For example, reserved characters may be >, <, | etc for file redirection and pipes and *, ?, [, -, ] for pattern matching. It would be difficult to use these reserved characters on the command line for some command, for example.
foo$ echo old -> new foo$ ls -l new -rw-r--r-- 1 chris 6 Jan 28 16:08 new foo% cat new old - foo$
The use of echo in the above fashion creates a new file and writes the
string ``old -'' into it---the ``
There are a number of ways to protect reserved characters so you
get what you really want.
So what if you want to print out a bash? Putting it
on the command line results in it protecting the following
character, rather than being printed itself. The answer is to
protect the bash with another bash.
Most Unix commands take command line arguments to modify their behaviour.
For example, supplying the -l argument to
the ls command results in a ``long'' listing of files in the
current directory.
In /bin/sh, command line arguments are passed to shell script as shell
variables. Shell variables are similar to variables in other higher-level
languages, but do not need to be declared before use.
From inside the shell script, the command line arguments may be referenced
by a number corresponding to their position on the command line. For example,
the first command line argument would be $1, the second $2 etc.
With positional parameters described above, it is possible to ``shift'' them
down so that parameters already recognised and processed may be discarded
and the next parameter takes its place, for example, $1 is
discarded, $2 becomes $1, $3 becomes $2 etc.
The shift command is particularly useful for looping commands to process the
entire command line painlessly.
Shell variables are labels for string storage in /bin/sh. There is no distinction between variables that contain a
single letter, multiple letters, or a numeric value.
In /bin/sh, all variables are treated as containing zero or more
characters.
Before a value is assigned to a variable, it exists but contains zero
characters. Variables may be introduced and used at any stage of the shell
script and there is no need (there is no facility) to declare them
before use. Traditionally, shell variable names are in all upper case,
but there is no technical reason except readability that this is the
case.
Wherever they appear in a shell script, an instance of a variable name
is replaced by that variable's contents. If a variable
contains an executable command, that command will be run (if the variable
name is in a position that makes this possible).
Out there in the great ether of things hanging around your account is the
``environment''. You can look at what's in your
environment with the env command. You can add things to your environment
using the export command from within your script.
Your environment is passed as a series of shell variables to all programs you
run, that is, to all the child processes of the current process. It is
impossible, however for a child process to change the environment of its parent
process. If you export a variable from within your shell script, it is passed
to all programs that you run from that shell script, but is not passed back to
the process that originally ran the script. This is why you need to use the
``dot-trick'' (. greet instead of sh greet) discussed above to run
your scripts as part of the current shell if you wish to change your
environment.
These shell variables are automagically set by /bin/sh when a
shell script is executed.
Because variables aren't declared, it's often difficult for /bin/sh to
understand what you're really talking about. In addition, it is possible to
provide alternate actions depending on whether a variable is set or not.
is horribly confusing for both humans and /bin/sh (which
would look for a variable called HOMEiswheretheheartis),
is much more intelligible.28.9
As discussed previously, backquotes run the
command they surround and rewrite the surrounded command with its output.
This may be assigned to variables, as well.
On occasion, it is necessary to read input from the keyboard or stdin
into a shell variable. This can be done with the use of the read
command, which is built into /bin/sh.
The read command will assign the string that is read from stdin to the
variable given on its command line. If this works, read returns
zero (success) status. If some fault occurs---the user types CTRLD, or the
end of file is reached if stdin is being redirected from a file---then read
returns non-zero status.
/bin/sh has some flow control constructors built in, similar to
high level languages, but slightly different to take account of the
string nature of /bin/sh variables and the return status of programs.
There are four available constructors: if, while, for and case.
The if construct executes the single command in list1. If
the command returns a zero value (i.e. success), the commands in list2
are executed in sequence. If list1 returned a non-zero value, the the
first element in list3 (if it exists) is executed; if this element
returns a
non-zero value, the remaining list3 commands are executed. If
neither list1 nor the first element of list3 return a zero status,
list4 is executed if it exists.
A very useful program to use as part of list1 is the test
or [ program. Test performs a large number of functions such
as string matching and file existence testing. man test gives you
the full run-down.
In the above example, the [ argument is actually a program that
lives in /usr/bin. It has the same functionality as the test
program, but looks a lot nicer. Because it is a program, it requires the
same spacing as would normally be expected.
The arguments to the [ program are "$NAME", =,
"Arthur" and ]. $NAME is surrounded by doubles
quotes in case the string that $NAME represents contains a
space character, which would severely stuff up the command line
arguments.
while will continue to execute both lists of commands until list1 fails. This is useful with the read
command. read returns a zero value and an
initialized shell variable upon success, and a non-zero value upon
CTRLD or end of file.
There is a rarely used construct similar to while---the until
construct. The difference is that both list1 and list2 are
executed until list1 succeeds.
Unlike most high level languages, the for loop in /bin/sh
does not increment a counter or whatever, but loops around placing
each element of word-list into the shell variable VAR.
Word-list may be normal shell pattern-match.
The case construct matches the supplied word against the list of
possible patterns. It proceeds in order though the list of patterns
supplied and executes the list associated with the first matching pattern.
The patterns/lists are separated by
a double semi-colon. The pattern may be a literal string which must
match the supplied word exactly, or it may be a wildcard that will
match a range of words.
args is a program that accepts the command line arguments -a, -b, -c and -d. If argument -b is
supplied, an additional argument ($BFILE) is expected.
/bin/sh has a number of built-in commands, some of which have been discussed
previously. Other interesting ones may be looked up in the manual
page for /bin/sh. (man sh.)
Functions are an extremely useful part of shell scripts. They are similar
to functions in any other language, in that they take arguments and can
return a value. But just to make things that much more exciting, they
do so with a complete set of sh subtleties and inconsistencies.
And is called like this:
So far, it's nice and easy! A function becomes something like another
command usable in an sh script. Remember that sh's idea of
local and global variables are not the same as C's. An sh variable
is accessible anywhere in the script. Even if it's declared outside a
function, it can still be read and changed within the function.
Function arguments are not very obvious in their use from the man
pages. Imagine each function is running within its own subshell (although
they don't on my FreeBSD system). Function arguments are handled in the
exact same way command line arguments are. $1 through to $9
all work, as does $*, but $0 still refers to the name of the
shell script, not the name of the function. Return values are really
simple, just return <number>. Note that you can only return numbers
greater than or equal to zero. An example will demonstrate better:
Recursion within sh functions is possible. The man page states
there is no limit to the depth of recursion bar the resources of the host
machine. The following test I did on a Pentium MMX 200 managed to allocate
80MB of memory (about 65 of it swap) in 15 seconds:
Some of the shell script example programs used in this chapter
are based on similar examples in
"The Unix Shell Programming Language" a book by Manis and Meyer.
foo$ echo old -\> new
old -> new
foo$
foo$ echo old =\= new
old == new
foo$ echo old =\\= new
old =\= new
foo$
foo$ echo "old -> new"
old -> new
foo$ ls "*"
* not found
foo$
foo$ echo "You are currently logged into host `hostname`"
You are currently logged into host foo
Shell Variables
Positional Parameters
foo$ cat >greet
#!/bin/sh
echo hello $1
foo$ . greet Chris
hello Chris
foo$
The Shift Command
foo$ cat greet
#!/bin/sh
echo hello $1
shift
echo hello $1
foo$ greet Chris Kylie
hello Chris
hello Kylie
foo$
Using shell variables
foo$ cat >greet
#!/bin/sh
HELLO="Hello $1"
echo $HELLO
shift
HELLO="$HELLO, $1"
echo $HELLO
foo$ . greet Chris Kylie
Hello Chris
Hello Chris, Kylie
foo$
The Export Command
Variables Set Automatically
foo$ cat >argv
#!/bin/sh
echo $*
foo$ argv 1 2 3 4
1 2 3 4
foo$ cat >argc
#!/bin/sh
echo $#
foo$ argc 1 2 3 4
4
foo$
foo$ cat mktmpf
#!/bin/sh
TMP=/tmp/temp$$
echo $TMP
foo$ mktmpf
/tmp/temp23106
foo$
More About Variables
foo$ echo $HOMEiswheretheheartis
foo$
foo$ echo ${HOME}iswheretheheartis
/system/usr/chrisiswheretheheartis
foo$
foo$ cat >thingy
#!/bin/sh
HOME=
echo ${HOME:?"No HOME directory set"}
^D
foo$ . thingy
sh: HOME: No HOME directory set
foo$
Variable Substitution Via Backquotes
foo$ cat >host
#!/bin/sh
HOSTNAME="You are currently logged into `hostname`"
echo $HOSTNAME
^D
foo$ . host
You are currently logged into foo
foo$
Variable Substitution from Keyboard
foo$ cat >prompt
#!/bin/sh
echo -n "What is your name? "
read NAME
echo Hello, $NAME
^D
foo$ . prompt
What is your name? Arthur, King of the Britons
Hello, Arthur, King of the Britons
foo$
Control Flow
foo$ cat >query
#!/bin/sh
echo -n "What is your name? "
read NAME
if [ "$NAME" = "Arthur" ]
then
echo "Hello!"
else
echo "Who the hell are you?"
fi
^D
foo$ . query
What is your name? Arthur
Hello!
foo$ query
What is your name? Sir Galahad
Who the hell are you?
foo$
foo$ cat >s
#!/bin/sh
echo -n "Hostname to ssh to? "
read HOST
if [ "`hostname`" = "$HOST" ]
then
echo "You are already logged into $HOST!"
echo "Swine!"
else
echo "ssh to $HOST"
ssh $HOST
fi
^D
foo$ . s
Hostname to ssh to? foo
You are already logged into foo
Swine!
foo$ . s
Hostname to ssh to? biggles
ssh to biggles
wzdd@biggles's password:
foo$ cat >foobie
#!/bin/sh
while read FOOBIE
do
echo $FOOBIE
done
^D
foo$ . foobie
Repeat after me
Repeat after me
hello
hello
Stop it!
Stop it!
^D
foo$
foo$ cat >thingy
#!/bin/sh
for THINGY in thingy foobie wookly dwang
do
echo "for $THINGY"
done
^D
foo$ . thingy
for thingy
for foobie
for wookly
for dwang
foo$
foo$ cat >doobie
#!/bin/sh
# Changes all files in the current directory
# ending with .out to .foo
TALLY=0
# Start the for loop, substitute .out on the end of files
# with nothing when it gets stuffed into $FILE
for FILE in `ls *.out | sed "s/.out//"`
do
mv ${FILE}.out ${FILE}.foo
echo moved ${FILE}.out to ${FILE}.foo
TALLY="`echo $TALLY + 1 | bc`"
done
echo $TALLY files renamed.
^D
foo$ . doobie
moved goo.out to goo.foo
moved swine.out to swine.foo
moved thing.out to thing.foo
3 files renamed.
foo$
foo$ cat >args
#!/bin/sh
while [ -n $1 ]
do
case $1 in
-a) OPTA=true
;;
-b) shift;
BFILE=$1
OPTB=true
;;
-c|-d) OPTCD=true
;;
*) echo Option $1 unknown 1>&2
exit 1
;;
esac
shift
done
^D
Built-in Commands
break, continue, cd, eval, exit, export, read, readonly,
set, shift, test, trap, wait
Functions
Declaration and Basic Use
A function is declared in an sh script like so:
function_name() {
commands
more commands
yet more commands
}
function_name
Arguments and Return Values
#!/bin/sh
#
# tst.sh
# Example program for demonstrating functions
exit_func()
{
echo "Leaving $0 with return value $1"
exit $1
}
print_cmd_line()
{
LINE=$*
echo "Command line is: $LINE"
}
while [ "$1" != "" ]; do
print_cmd_line $*
shift
done
exit_func 0
bash$ . tst2.sh fred joe smith
Command line is: fred joe smith
Command line is: joe smith
Command line is: smith
Leaving tst2.sh with return value 0
bash$
Recursion
#!/bin/sh
foo() {
foo
}
foo
Disclaimer