Aim
After successfully working through this exercise, You will: s write simple shell scripts using for, if, while, case, getopts statements;
s s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang

Shell Programming—an Introduction
Copyright Conditions: Open Publication License (see http://www.opencontent.org/openpub/)

q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments

write shell script functions, and be able to handle parameters; understand basic regular expressions, and be able to create your own regular expressions; understand how to execute and debug these scripts; understand some simple shell scripts written by others.

Nick Urbanik
nicku@vtc.edu.hk Department of Information and Communications Technology

q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example OSSI — ver. 1.9 Shell Programming – p. 1/86 q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable Shell Programming – p. 3/86 Shell Programming – p. 2/86

Why Shell Scripting?
s

Where to get more information
s

s s s

Basic startup, shutdown of Linux, Unix systems uses large number of shell scripts x understanding shell scripting important to understand and perhaps modify behaviour of system Very high level: powerful script can be very short Can build, test script incrementally Useful on the command line: “one liners”

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution

s

s

the Libarary has two copies of the book, Learning the Bash Shell, second edition, by Cameron Newham & Bill Rosenblatt, O’Reilly, 1998. There is a free on-line book about shell programming at: http://tldp.org/LDP/abs/html/index.html and http://tldp.org/LDP/abs/abs-guide.pdf. It has hundreds of pages, and is packed with examples. The handy reference to shell programming is: $ pinfo bash or $ man bash

q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 4/86

The Shell is an Interpreter
s s s

The Shebang
s s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

Some languages are compiled: C, C++, Java,. . . Some languages are interpreted: Java bytecode, Shell Shell is an interpreter: kernel does not run shell program directly: x kernel runs the shell program /bin/sh with script file name as a parameter x the kernel cannot execute the shell script directly, as it can a binary executable file that results from compiling a C program

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

s

Linux kernel starts shell script kernel reads first two characters of each executable x If first 2 chars are “#!” then x kernel executes the name that follows, with the file name of the script as a parameter Example: a file called find.sh has this as the first line: #! /bin/sh then kernel executes this: /bin/sh find.sh

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 5/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 6/86

Making the script executable
To easily execute a script, it should: s be on the PATH s have execute permission. How to do each of these? s Red Hat Linux by default, includes the directory ∼/bin on the PATH, so create this directory, and put your scripts there: $ mkdir ∼/bin s If your script is called script, then this command will make it executable: $ chmod +x script

Special Characters
s

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

Many characters are special to the shell, and have a particular meaning to the shell.
Character ∼
‘ # $ & * |

Temporary Files: mktemp q Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script Writing Shell Scripts q Variables—Unsetting Them Useful External Programs—1 q Command-line Parameters Useful Built-in Programs—2 q SpecialExternal Variables Regular Expressions q Variables: use Braces ${...} What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution

q getopts: Command-Line Options—4 Making the script executable getopts: Command-Line Options—5 q Special Characters Temporary Files: mktemp q Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting Useful External Programs—1 q Command-line Parameters Useful Built-in Programs—2 q SpecialExternal Variables q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Meaning Home directory Command substitution. Better: $(...) Comment Variable expression Background Job File name matching wildcard Pipe § 15 §7 § 24

See slide

2.10 on page 43 2.18 on page 51 2.9 on page 42

q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement sed—Backreferencees q for Statement sed—Backreferencees: Example q for Loops: Another Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

Shell Programming – p. 7/86

Shell Programming – p. 8/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Special Characters—continued: 2

Special Characters—continued: 3

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information

Character
( ) [ ] { ; \ ’ "

Meaning Start subshell End subshell Start character set file name matching End character set file name matching Start command block Command separator Quote next character Strong quote Weak quote

See slide § 45, 17, 39 § 45, 17, 39 2.9 on page 42 2.9 on page 42 § 39 § 40 § 23 § 23 § 23

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example Shell Programming – p. 9/86 q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement

Character
< > / ? !

Meaning Input redirect Output redirect Pathname directory separator Single-character match in filenames Pipline logical NOT

See slide 2.7 on page 40 2.6 on page 39 2.18 on page 51 § 28

s Note that references to pages in the tables above refer to the modules in the workshop notes

Shell Programming – p. 10/86

Quoting
s

Quoting—2
s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

s s

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 Functions q Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script Writing Shell Scripts q Variables—Unsetting Them Useful External Programs—1 q Command-line Parameters Useful Built-in Programs—2 q SpecialExternal Variables Regular Expressions q Variables: use Braces ${...} What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution

s

Sometimes you want to use a special character literally; i.e., without its special meaning. Called quoting Suppose you want to print the string: 2 * 3 > 5 is a valid inequality? If you did this: $ echo 2 * 3 > 5 is a valid inequality the new file ‘5’ is created, containing the character ‘2’, then the names of all the files in the current directory, then the string “3 is a valid inequality”.

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 Functions q Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments Debugging Shell to Script q Variables—Local Scripts—2 Writing Shell Scripts q Variables—Unsetting Them Useful External Programs—1 q Command-line Parameters Useful Built-in Programs—2 q SpecialExternal Variables Regular Expressions q Variables: use Braces ${...} What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

s

To make it work, you need to protect the special characters ‘*’ and ‘>’ from the shell by quoting them. There are three methods of quoting: x Using double quotes (“weak quotes”) x Using single quotes (“strong quotes”) x Using a backslash in front of each special character you want to quote This example shows all three: $ echo "2 * 3 > 5 is a valid inequality" $ echo ’2 * 3 > 5 is a valid inequality’ $ echo 2 \* 3 \> 5 is a valid inequality

q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement sed—Backreferencees q for Statement sed—Backreferencees: Example q for Loops: Another Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

Shell Programming – p. 11/86

Shell Programming – p. 12/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Quoting—When to use it?
s

True and False
s s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

s

Use quoting when you want to pass special characters to another program. Examples of programs that often use special characters: x find, locate, grep, expr, sed and echo Here are examples where quoting is required for the program to work properly: $ find . -name \*.jpg $ locate ’/usr/bin/c*’ $ grep ’main.*(’ *.c $ i=‘expr i \* 5‘

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s

s

Shell programs depend on executing external programs When any external program execution is successful, the exit status is zero, 0 An error results in a non-zero error code To match this, in shell programming: x The value 0 is true x any non-zero value is false This is opposite from other programming languages

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 13/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 14/86

Variables—1
s s

Variables—Assignments
s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False Example, Calling a Function q Variables—1 Debugging Shell Scripts—1 q Variables—Assignments Debugging Shell to Script q Variables—Local Scripts—2 Writing Shell Scripts q Variables—Unsetting Them Useful External Programs—1 q Command-line Parameters Useful Built-in Programs—2 q SpecialExternal Variables Regular Expressions q Variables: use Braces ${...} What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

s

Variables not declared; they just appear when assigned to Assignment: x no dollar sign x no space around equals sign x examples: $ x=10 # correct $ x = 10 # wrong: try to execute program called ‘‘x’’ Read value of variable: x put a ‘$’ in front of variable name x example: $ echo "The value of x is $x"

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 Debugging Shell Scripts—1 q Variables—Assignments Debugging Shell to Script q Variables—Local Scripts—2 Writing Shell Scripts q Variables—Unsetting Them Useful External Programs—1 q Command-line Parameters Useful Built-in Programs—2 q SpecialExternal Variables Regular Expressions q Variables: use Braces ${...} What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s

You can put multiple assignments on one line: i=0 j=10 k=100 You can set a variable temporarily while executing a program: $ echo $EDITOR emacsclient $ EDITOR=gedit crontab -e $ echo $EDITOR emacsclient

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 15/86

Shell Programming – p. 16/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Variables—Local to Script
s s

Variables—Unsetting Them
s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information

Variables disappear after a script finishes Variables created in a sub shell disappear x parent shell cannot read variables in a sub shell x example: $ cat variables #! /bin/sh echo $HOME HOME=happy echo $HOME $ ./variables /home/nicku happy $ echo $HOME /home/nicku

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

You can make a variable hold the null string by assigning it to nothing, but it does not disappear totally: $ VAR= $ set | grep ’ˆVAR’ VAR= You can make it disappear totaly using unset: $ unset VAR $ set | grep ’ˆVAR’

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example Shell Programming – p. 17/86 q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 18/86

Command-line Parameters
s s

Special Built-in Variables
s s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting Useful External Programs—1 q Command-line Parameters Useful Built-in Programs—2 q SpecialExternal Variables Regular Expressions q Variables: use Braces ${...} What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

Command-line parameters are called $0, $1, $2, . . . Example: when call a shell script called “shell-script” like this: $ shell-script param1 param2 param3 param4
variable
$0 $1 $2 $3 $4 $#

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 Useful Built-in Programs—2 q SpecialExternal Variables Regular Expressions q Variables: use Braces ${...} What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s s s

value
shell-script param1 param2 param3 param4

number of parameters to the program, e.g., 4

x

Note: these variables are read-only.

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Both $@ and $* are a list of all the parameters. The only difference between them is when they are quoted in quotes—see manual page for bash $? is exit status of last command $$ is the process ID of the current shell Example shell script: #! /bin/sh echo $0 is the full name of this shell script echo first parameter is $1 echo first parameter is $2 echo first parameter is $3 echo total number of parameters is $# echo process ID is $$

Shell Programming – p. 19/86

Shell Programming – p. 20/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Variables: use Braces ${...}
s

Braces and Parameters after $9
q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example Shell Programming – p. 21/86 q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 22/86

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

It’s good to put braces round a variable name when getting its value Then no problem to join its value with other text: $ test=123 $ echo ${test} 123 # No good, variable $test456 is undefined: $ echo $test456
$ echo ${test}456 123456

s

s

Need braces to access parameters after $9: $ cat paramten #! /bin/sh echo $10 echo ${10} $ ./paramten a b c d e f g h i j a0 j Notice that $10 is the same as ${1}0, i.e., the first parameter “a” then the literal character zero “0”

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information

More about Quoting
s

Command Substitution
s s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 Literal characters q More about Quoting q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

s

s

Double quotes: "..." stops the special behaviour of all special characters, except for: x variable interpretation ($) x backticks (‘) — see slide 24 x the backslash (\) Single quotes ’...’: x stop the special behaviour of all special characters Backslash: x preserves literal behaviour of character, except for newline; see slides §29, §31, §35 x Putting “\” at the end of the line lets you continue a long line on more than one physical line, but the shell will treat it as if it were all on one line.

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters Character Substitution q Command Classes: [...] q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s

Enclose command in $(...) or backticks:‘...‘ Means, “Execute the command called within these quotes and put the output back here.” Here is an example using expr: $ expr 3 + 2 5 $ i=expr 3 + 2 # error: try execute command ‘3’ $ i=$(expr 3 + 2) # correct $ i=‘expr 3 + 2‘ # also correct

Matching the Beginning or End of q Conditions—Integer Comparisons Line q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 23/86

Shell Programming – p. 24/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Command Substitution—Example
s

Conditions—String Comparisons
s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

$ hostname nickpc.tyict.vtc.edu.hk $ h=hostname $ echo $h hostname $ h=$(hostname) $ echo $h nickpc.tyict.vtc.edu.hk

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s

All programming languages depend on conditions for if statements and for while loops Shell programming uses a built-in command which is either test or [...] Examples of string comparisons:
[ [ [ [ [ "$USER" = root ] # "$USER" != root ] # -z "$USER" ] # string1 < string2 ] # string1 > string2 ] # true true true true true if if if if if the value of $USER is "root" the value of $USER is not "root" the string "$USER" has zero length string1 sorts less than string2 string1 sorts greater than string2

s

Note: the spaces after the “[“ and before the “]” are essential.

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 25/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 26/86

Conditions—Integer Comparisons
s

Conditions—File Tests, NOT Operator
s s s
$ $ $ $ $ $ $ $ $

Examples of numeric integer comparisons:
[ [ [ [ [ [ "$x" "$x" "$x" "$x" "$x" "$x" -eq -ne -lt -gt -le -ge 5 5 5 5 5 5 ] ] ] ] ] ] # # # # # # true true true true true true if if if if if if the value of $x is 5 integer $x is not 5 integer $x is < 5 integer $x is > 5 integer $x is ≤ 5 integer $x is ≥ 5

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} Matching Alternatives: “|” q Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

The shell provides many tests of information about files. Do man test to see the complete list. Some examples:
[ [ [ [ [ [ [ [ [ -f file ] # true if file is an ordinary file ! -f file ] # true if file is NOT an ordinary file -d file ] # true if file is a directory -u file ] # true if file has SUID permission -g file ] # true if file has SGID permission -x file ] # true if file exists and is executable -r file ] # true if file exists and is readable -w file ] # true if file exists and is writeable file1 -nt file2 ] # true if file1 is newer than file2

s

Note again that the spaces after the “[“ and before the “]” are essential.

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} Matching Alternatives: “|” q Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 27/86

Shell Programming – p. 28/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Conditions—Combining Comparisons
s

Arithmetic Assignments
s Can do with the external program expr x . . . but expr is not so easy to use, although it is very standard and

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

Examples of combining comparisons with AND: -a and OR: -o, and grouping with \(...\)
# true if the value of $x is 5 AND $USER is not equal to root: [ "$x" -eq 5 -a "$USER" != root ] # true if the value of $x is 5 OR $USER is not equal to root: [ "$x" -eq 5 -o "$USER" != root ] # true if ( the value of $x is 5 OR $USER is not equal to root ) AND # ( $y > 7 OR $HOME has the value happy ) [ \( "$x" -eq 5 -o "$USER" != root \) -a \ \( "$y" -gt 7 -o "$HOME" = happy \) ]

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

portable
x Easier is to use the built in let command x Examples:
$ $ $ $ $ $ $ let let let let let let let x=1+4 x=’1 + 4’ ’x = 1 + 4’ x="(2 + 3) * 5" "x = 2 + 3 * 5" "x += 5" "x = x + 5"

s

s

Note again that the spaces after the “[“ and before the “]” are essential. Do man test to see the information about all the operators.

# # # #

now now now now

x x x x

is is is is

25 17 22 27; note do not need $ on rhs

x Notice that you do not need to quote the special characters with let. x Quote if you want to use white space.

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters Shell Programming – p. 29/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 30/86

Arithmetic Expressions with $((...))
s

Arithmetic Conditions with ((...))
s

s

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

s

The shell interprets anything inside $((...)) as an arithmetic expression You could calculate the number of days left in the year like this: $ echo "There are \ $(( (365-$(date +%j)) / 7 )) weeks \ left till December 31" You don’t need to put a dollar sign in front of variables in arithmetic expressions, but it is okay to put them there.

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s

s

A (less portable) alternative to the arithmetic conditions in slide 27 is putting the expression in ((...)) So you can do (( (3>2) && (4<=1) )) instead of [ \( 3 -gt 2 \) -a \( 4 -le 1 \) ] Operators that work with let, $((...)) and ((...)) are: + - * / % << >> & | ˜ ! ˆ < > <= >= == != which have exactly the same effect as in the C programming language.

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 31/86

Shell Programming – p. 32/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

if Statement
Syntax:
if test-commands then statements-if-test-commands-1-true elif test-commands-2 then statements-if-test-commands-2-true else statements-if-all-test-commands-false fi

while Statement
Syntax:
while test-commands do loop-body-statements done

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

s

Example:
i=0 while [ "$i" -lt 10 ] do echo -n "$i " i=‘expr $i + 1‘ done # -n suppresses newline. # let "i = i + 1" also works.

s

Example:
if grep nick /etc/passwd > /dev/null 2>&1 then echo Nick has a local account here else echo Nick has no local account here fi

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 33/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 34/86

for Statement
Syntax:
for name in words do loop-body-statements done

for Loops: Another Example
s

s

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

Here the shell turns *.txt into a list of file names ending in “.txt”:
for i in *.txt do echo $i grep ’lost treasure’ $i done

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

s

Example:
for planet in Mercury Venus Earth Mars \ Jupiter Saturn Uranus Neptune Pluto do echo $planet done

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

s

You can leave the in words out; in that case, name is set to each parameter in turn:
i=0 for parameter do let ’i = i + 1’ echo "parameter $i is $parameter" done

x

The backslash “\” quotes the newline. It’s just a way of folding a long line in a shell script over two or more lines.

Shell Programming – p. 35/86

Shell Programming – p. 36/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

for Loops: second, C-like syntax
There is a second (less frequently used, and less portable) C-like for loop syntax:
for (( expr1 ; expr2 ; expr3 )) do loop-body-statements done
s s

break and continue
Use inside a loop Work like they do in C break terminates the innermost loop; execution goes on after the loop continue will skip the rest of the body of the loop, and resume execution on the next itteration of the loop.

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s s

s

Rules: same as for arithmetic conditions—see slide 32 Example:
for (( i = 0; i < 10; ++i )) do echo $i done

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 37/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 38/86

Blocks: {...}
s

Error Handling: ||, && and exit
s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

s

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s s

A subshell is one way of grouping commands together, but it starts a new process, and any variable changes are localised An alternative is to group commands into a block, enclosing a set of commands in braces: {...} Useful for grouping commands for file input or output See next slide for another application.

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s s s

Suppose we want the user to provide exactly two parameters, and exit otherwise A common method of handling this is something like:
[ $# -eq 2 ] || { echo "Need two parameters"; exit 1; }

s

s

Read this as “the number of parameters is two OR exit” Works because this logical OR uses short-circuit Boolean evaluation; the second statement is executed only if the first fails (is false) Logical AND “&&” can be used in the same way; the second statement will be executed only if the first is successful (true) A note about blocks: must have semicolon “;” or newline at end of last statement before closing brace

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 39/86

Shell Programming – p. 40/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Output: echo and printf
s

Input: the read Command
s s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s s

s

To perform output, use echo, or for more formatting, printf. Use echo -n to print no newline at end. Just echo by itself prints a newline printf works the same as in the C programming language, except no parentheses or commas: $ printf "%16s\t%8d\n" $my_string $my_number Do man printf (or look it up in the bash manual page) to read all about it.

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s

s

s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 41/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement

For input, use the built-in shell command read read reads standard input and puts the result into one or more variables If use one variable, variable holds the whole line Syntax: read var1 var2... Often used with a while loop like this: while read var1 var2 do # do something with var1 and var2 done Loop terminates when reach end of file

Shell Programming – p. 42/86

set: Splitting a Multi-Word Variable
Sometimes may want to split a multi-word variable into single-word variables read won’t work like this: MY_FILE_INFO=$(ls -lR | grep $file) # ... echo $MY_FILE_INFO | read perms links \ user group size month day time filename Use the builtin command set instead: MY_FILE_INFO=$(ls -lR | grep $file) # ... set $MY_FILE_INFO perms=$1 links=$2 user=$3 group=$4 size=$5 month=$6 day=$7 time=$8 filename=$9

More about set, and IFS
s s s

s

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

s

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s s s

s

set splits its arguments into pieces (usually) at whitespace It sets the first value as $1, the second as $2, and so on. Note that you can change how set and the shell splits things up by changing the value of a special variable called IFS IFS stands for Internal Field Separator Normally the value of IFS is the string “ space tab newline ” Next slide shows how changing IFS to a colon let us easily next slide split the PATH into separate directories:

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 43/86

Shell Programming – p. 44/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Example: Changing IFS
s

case Statement
Similar to the switch statement in C, but more useful and more general Uses pattern matching against a string to decide on an action to take Syntax: case expression in pattern1 ) statements ;; pattern2 ) statements ;; ... esac

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

Notice that here, I make the change to IFS in a subshell. I have simply typed the loop at the prompt. As I said in slide 17, changes in a subshell are local to the subshell:
$ echo $PATH /usr/bin:/bin:/usr/X11R6/bin:/home/nicku/bin $ (IFS=: > for dir in $PATH > do > echo $dir > done >) /usr/bin /bin /usr/X11R6/bin /home/nicku/bin

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

s

s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 45/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 46/86

case Statement: Example
This example code runs the appropriate program on a graphics file, depending on the file extension, to convert the file to another format: case $filename in *.tif) tifftopnm $filename > $ppmfile ;; *.jpg) tjpeg $filename > $ppmfile ;; *) echo -n "Sorry, cannot handle this " echo "graphics format" ;; esac
s

shift: Move all Parameters Up
Sometimes we want to process command-line parameters in a loop The shift statement is made for this Say that we have four parameters: $1 has the value one, $2 has the value two, $3 has the value three and $4 has the value four Then after executing the shift statement, the values are now: $1 has the value two, $2 has the value three and $3 has the value four, and $4 no longer exists.

q case Statement: Example shift: Move all Parameters Up q Aim shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information

s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

s s

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 47/86

Shell Programming – p. 48/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

shift: Many Places
You can give a number argument to shift: x If before, we have four parameters: $1 has the value one, $2 has the value two, $3 has the value three and $4 has the value four x After executing the statement: $ shift 2 we have two parameters left: $1 has the value three and $2 has the value four; $3 and $4 no longer exist.

Command-Line Options—1
s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

Sometimes we want to modify the behaviour of a shell script x For example, want an option to show more information on request x could use an option “-v” (for “verbose”) to tell the shell script that we want it to tell us more information about what it is doing x If script is called showme, then we could use our -v option like this: $ showme -v x the script then shows more information.

s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang Shell Programming – p. 49/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 50/86

Command-Line Options—2
s

Command-Line Options—3
s

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s s

For example, We might provide an option to give a starting point for a script to search for SUID programs Could make the option -d directory If script is called findsuid, could call it like this: $ findsuid -d /usr to tell the script to start searching in the directory /usr instead of the current directory

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

We could do this using shift, a while loop, and a case statement, like this: while [ -n "$(echo $1 | grep ’-’)" ] do case $1 in -v) VERBOSE=1 ;; -d) shift DIRECTORY=$1 ;; *) echo "usage: $0 [-v] [-d dir]" exit 1 ;; esac shift done

Shell Programming – p. 51/86

Shell Programming – p. 52/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

getopts: Command-Line Options—4
Problems with above solution: inflexibility: x Does not allow options to be “bundled” together like -abc instead of -a -b -c x Requires a space between option and its argument, i.e., doesn’t let you do -d/etc as well as -d /etc x Better method: use the built-in command getopts:
while getopts ":vd:" opt do case opt in v) VERBOSE=1 ;; d) DIRECTORY=$OPTARG ;; *) echo "usage: $0 [-v] [-d dir]" exit 1 ;; esac done shift $((OPTIND - 1))

getopts: Command-Line Options—5
s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 53/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments

s

getopts takes two arguments: x first comes the string that can contain letters and colons. s Each letter represents one option s A colon comes after a letter to indicate that option takes an arguement, like -d directory s A colon at the beginning makes getopts less noisy, so you can provide your own error message, as shown in the example. x The second is a variable that will hold the option (without the hyphen “-”) Shift out all processed options using the variable OPTIND, leaving any other arguments accessible Search for getopts in the bash man page

q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 54/86

Temporary Files: mktemp
s

Signals that may Terminate your Script
s s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

s s

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s

s

Sometimes it is convenient to store temporary data in a temporary file The mktemp program is designed for this We use it something like this: TMPFILE=$(mktemp /tmp/temp.XXXXXX) || exit 1 mktemp will create a new file, replacing the “XXXXXX” with a random string Do man mktemp for the complete manual.

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

Many key strokes will send a signal to a process Examples: ¨ 
x

x s

s

s

When you log out, all your processes are sent a SIGHUP (hangup) signal If your script is connected to another process that terminates unexpectedly, it will receive a SIGPIPE signal If anyone terminates the program with the kill program, the default signal is SIGTERM

Control-C © sends a SIGINT signal to the current process  running in the foreground  ¨ Control-\ © sends a SIGQUIT signal 

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 55/86

Shell Programming – p. 56/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Signals: trap
s

Signals: trap Example
s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

Sometimes you want your script to clean up after itself nicely, and remove temporary files Do this using trap

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

s

s

s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 57/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement

Supose your script creates some temporary files, and you want to remove them if your script recieves any of these signals You can “catch” the signal, and remove the files when the signals are received before the program terminates Suppose the temporary files have names stored in the variables TEMP1 and TEMP2 Then you would trap these signals like this: trap "rm $TEMP1 $TEMP2" HUP INT QUIT PIPE TERM Conveniently, (but not very portably), bash provides a “pretend” signal called EXIT; can add this to the list of signals you trap, so that the temporary files will be removed when the program exits normally.

Shell Programming – p. 58/86

Functions
s s

Parameters in Functions
s s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

The shell supports functions and function calls A function works like an external command, except that it does not start another process Syntax: function functname { shell commands } Or: functname () { shell commands }

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s s s

Work the same as parameters to entire shell script First parameter is $1, second is $2,. . . , the tenth parameter is ${10}, and so on. $# is the number of parameters passed to the function As with command line parameters, they are read-only Assign to meaningful names to make your program more understandable

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 59/86

Shell Programming – p. 60/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Example, Calling a Function
s

Debugging Shell Scripts—1
s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

This is a simple example program: #! /bin/sh
function cube { echo $(($1 * $1 * $1)) } j=$(cube 5) echo $j # Output is 125 Note the use of command substitution to get a return value. The function prints result to standard output.

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

If you run the script with: $ sh -v script then each statement will be printed as it is executed If you run the script with: $ sh -x script then an execution trace will show the value of all variables as the script executes.

s s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 61/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 62/86

Debugging Shell Scripts—2
s

Writing Shell Scripts
s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

s

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

s

Use echo to display the value of variables as the program executes You can turn the -x shell option on in any part of your script with the line: set -x and turn it off with: set +x The book Learning the bash Shell includes a bash shell debugger if you get desperate

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

s

Build your shell script incrementally : x Open the editor in one window, have a terminal window open in which to run your program as you write it x Test as you implement: this makes shell script development easy x Do not write a very complex script, and then begin testing it! Use the standard software engineering practice you know: x Use meaningful variable names, function names x Make your program self-documenting x Add comment blocks to explain obscure or difficult parts of your program

Shell Programming – p. 63/86

Shell Programming – p. 64/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Useful External Programs—1
Each of these has a manual page, and many have info manuals. Read their online documentation for more information. s awk — powerful tool for processing columns of data s basename — remove directory and (optionally) extension from file name s cat — copy to standard output s cut — process columns of data s du — show disk space used by directories and files s egrep, grep — find lines containing patterns in files s find — find files using many criteria

Useful External Programs—2
s s s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s s s

last — show the last time a user was logged in lastb — show last bad log in attempt by a user rpm — RPM package manager: manage software package database sed — stream editor: edit files automatically sort — sort lines of files by many different criteria tr — translate one set of characters to another set uniq — replace repeated lines with just one line, optionally with a count of the number of repeated lines

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces q What is and Parameters after $9 Braces in a Regular Expression? q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example Shell Programming – p. 65/86 q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 66/86

Regular Expressions
s

What is in a Regular Expression?
s

s

s

Many programs and programming languages use regular expressions, including Java 1.4, Perl and VB.NET (plus many others; even MS Word uses regular expressions under Find → Advanced) These programs use regular expressions: x grep, egrep, sed, awk Regular expressions provide a powerful language for manipulating data and extracting important information from masses of data

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

There are two types of character in a regular expression: x Metacharacters s These include: s *\.+?^()[{| x Ordinary, literal characters: s all the other characters

Shell Programming – p. 67/86

Shell Programming – p. 68/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Literal characters
s

Character Classes: [...]
s s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s s s

Find all lines containing "chan" in the password file: $ grep chan /etc/passwd The regular expressions is "chan" It is made entirely of literal characters It matches only lines that contain the exact string It will match lines containing the words chan, changed, merchant, mechanism,. . .

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

A character class represents one character Examples:
# $ # $ # $ # $ Find grep Find grep Find grep Find grep all words in the dictionary that contain a vowel: "[aeiou]" /usr/share/dict/words all lines that contain a digit: "[0123456789]" /usr/share/dict/words all lines that contain a digit: "[0-9]" /usr/share/dict/words all lines that contain a capital letter: "[A-Z]" /usr/share/dict/words

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 69/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 70/86

Negated Character Classes: [^...]
s

Match Any Character
s s

Examples of negated character classes:
# # $ # # $ $ # # $ Find all words in the dictionary that contain a character that is not a vowel: grep "[ˆaeiou]" /usr/share/dict/words Two ways of finding all lines that contain a character that is not a digit: grep "[ˆ0123456789]" /usr/share/dict/words grep "[ˆ0-9]" /usr/share/dict/words Find all lines that contain a character that is not a digit, or a letter grep "[ˆ0-9a-zA-Z]" /usr/share/dict/words

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

The dot “.” matches any single character, except a newline. The pattern ’.....’ matches all lines that contain at least five characters

s

Remember: each set of square brackets represents exactly one character.

Shell Programming – p. 71/86

Shell Programming – p. 72/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Matching the Beginning or End of Line
s

Match Repetitions: *, ?, +, {n}, {n,m}
s s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s

s s s

To match a line that contains exacly five characters: $ grep ’ˆ.....$’ /usr/share/dict/words The hat, ˆ represents the position right at the start of the line The dollar $ represents the position right at the end of the line. Niether ˆ nor $ represents a character They represent a position Sometimes called anchors, since they anchor the other characters to a specific part of the string

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s s s s

To match zero or more: a* represents zero or more of the lower case letter a, so the pattern will match "" (the empty string), “a”, “aa”, “aaaaaaaaaaaaaaa”, “qwewtrryu” or the “nothing” in front of any string! To match one or more: ‘a+’ matches one or more “a”s ‘a?’ matches zero or one “a” ‘a{10}’ matches exactly 10 “a”s ‘a{5,10}’ matches between 5 and 10 (inclusive) “a”s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 73/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 74/86

Matching Alternatives: “|”
s s

Putting it All Together: Examples
s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s s

the vertical bar represents alternatives: The regular expresssion ‘nick|albert|alex’ will match either the string “nick” or the string “albert” or the string “alex” Note that the vertical bar has very low precedence: the pattern ‘^fred|nurk’ matches “fred” only if it occurs at the start of the line, while it will match “nurk” at any position in the line

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

Find all words that contain at least two vowels:
$ grep ’[aeiou].*[aeiou]’ /usr/share/dict/words

s q getopts: Command-Line Options—4 Making the script executable

q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

s

Find all words that contain exactly two vowels: $ egrep \ ’[ˆaeiou]*[aeiou][ˆaeiou]*[aeiou][ˆaeiou]*’ \ /usr/share/dict/words Find all lines that are empty, or contain only white space: $ grep ’ˆ\s*$’ file

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement sed—Backreferencees q for Statement sed—Backreferencees: Example q for Loops: Another Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

Shell Programming – p. 75/86

Shell Programming – p. 76/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Basic awk
s s

What Does awk Do?
s s

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

awk is a complete programming language Mostly used for one-line solutions to problems of extracting columns of data from text, and processing it A complete book is available on awk at http://www.ssc.com/ssc/eap/.

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s s s

s

awk reads file(s) or standard input one line at a time, and automatically splits the line into fields, and calls them $1, $2,. . . , $NF NF is equal to the number of fields the line was split into $0 contains the whole line awk has an option -F that allows you to select another pattern as the field separator x Normally awk splits columns by white space To execute code after all lines are processed, create an END block.

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 77/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 78/86

awk Examples
Print the sizes of all files in current directory: ls -l | awk ’{print $5}’ Add the sizes of all files in current directory: ls -l | awk ’{sum += $5} END{print sum}’ Print only the permissions, user, group and file names of files in current directory: ls -l | awk ’{print $1, $3, $4, $NF}’

sed—the Stream Editor
sed provides many facilities for editing files The substitute command, s///, is the most important The syntax (using sed as an editor of stanard input), is: $ sed ’s/original/replacement/’ Example: replace the first instance of Windows with Linux on each line of the input: sed ’s/Windows/Linux/’ Example: replace all instances of Windows with Linux on each line of the input: sed ’s/Windows/Linux/g’

s

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

s s s

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

s

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s

s

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement q sed—Backreferencees: Example for Loops: Another Example find Examples q for Loops: second, C-like syntax OSSI — ver. 1.9 q Findingand continue break SUID Programs q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

Shell Programming – p. 79/86

Shell Programming – p. 80/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

sed—Backreferencees

sed—Backreferencees: Example
If you do find /etc | xargs file -b, you will get a lot of output like this: symbolic link to bg5ps.conf.zh_TW.Big5 symbolic link to rc.d/rc.local symbolic link to rc.d/rc symbolic link to rc.d/rc.sysinit symbolic link to ../../X11/xdm/Xservers If you want to edit each line to remove everything after “symbolic link”, then you could pipe the data through sed like this: $ find /etc | xargs file -b \ | sed ’s/\(symbolic link\).*/\1/’ See slide 83 for an application

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

s s

You can match part of the original in a sed substitute command, and put that part back into the replacement part. You enclose the part you want to refer to later in \(...\) You can get the first value in the replacement part by \1, the second opening parenthesis of \(...\) by \2, and so on.

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up Aim q shift: Many Places Why Shell Scripting? q Command-Line Options—1 Where to get more information Shell Programming – p. 81/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments

s

q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement Shell Programming – p. 82/86

find Examples
Count the number of unique manual pages on the computer: $ find /usr/share/man -type f | wc -l Print a table of types of file under the /etc directory, with the most common file type down at the bottom: $ find /etc | xargs file -b \ | sed ’s/\(symbolic link\).*/\1/’ \ | sort \ | uniq -c \ | sort -n

Finding SUID Programs
s

s

q case Statement: Example shift: Move all Parameters Up q Aim
shift: Many Places q Why Shell Scripting? Command-Line Options—1 q Where to get more information Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang

Command-Line Options—2 q The Shell is an Interpreter q Command-Line Options—3 The Shebang q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters

s

q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

q getopts: Command-Line Options—4 Making the script executable q getopts: Command-Line Options—5 Special Characters q Temporary Files: mktemp Special Characters—continued: 2 q Signals that may Terminate your 3 Special Characters—continued: Script q Signals: trap Quoting q Signals: trap Example Quoting—2 q Functions Quoting—When to use it? q Parameters in Functions True and False q Example, Calling a Function Variables—1 q Debugging Shell Scripts—1 Variables—Assignments q Debugging Shell Scripts—2 Variables—Local to Script q Writing Shell Scripts Them Variables—Unsetting q Useful ExternalParameters Command-line Programs—1 q Useful External Variables Special Built-in Programs—2 q Regular Expressions ${...} Variables: use Braces What is in a Regular Expression? q Braces and Parameters after $9 q Literalabout Quoting More characters q Character Classes: [...] Command Substitution q Negated Character Classes: [^...] Command Substitution—Example q Match Any Character Conditions—String Comparisons

s

Finding SUID or SGID files: $ sudo find / -type f \( perm -2000 -o -perm -4000 \) \ > files.secure Let’s compare with a list of SUID and SGID files to see if there are any changes, since SUID and SGID programs can be a security risk: $ sudo find / -type f \( perm -2000 -o -perm -4000 \) | diff - files.secure

q Matching the Beginning or End of Line Conditions—Integer Comparisons q Match Repetitions: *, ?,NOT Operator Conditions—File Tests, +, {n}, {n,m} q Matching Alternatives: “|” Conditions—Combining Comparisons q Putting it All Together: Examples Arithmetic Assignments q Basic awk Expressions with $((...)) Arithmetic q What Does awk Do? with ((...)) Arithmetic Conditions q awk Examples if Statement q sed—the Stream Editor while Statement q sed—Backreferencees for Statement for Loops: Another Example q sed—Backreferencees: Example Shell Programming – p. 83/86
for Loops: second, C-like syntax q find Examples OSSI — ver. 1.9 q Findingand continue break SUID Programs

Shell Programming – p. 84/86

q A find {...} with Many Options Blocks: Example q rpm Database Query Commands Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

q A find {...} with Many Options Blocks: Example rpm Handling: ||, && and exit q ErrorDatabase Query Commands q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable

A find Example with Many Options
s

rpm Database Query Commands
The rpm software package management system includes a database with very detailed information about every file of every software package that is installed on the computer. You can query this database using the rpm command. The manual page does not give the complete picture, but there is a book called Maximum RPM that comes on the Red Hat documentation CD This package is installed on ictlab You can see the appropriate section at this URL:
http://ictlab.tyict.vtc.edu.hk/doc/maximum- rpm- 1.0/html/s1- rpm- query- parts.html

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

Set all directories to have the access mode 771, set all backup files (*.BAK) to mode 600, all shell scripts (*.sh) to mode 755, and all text files (*.txt) to mode 644:
-type -name -name -name d "*.BAK" "*.sh" "*.txt" -a -a -a -a exec exec exec exec chmod chmod chmod chmod 771 600 755 644

q Aim q Why Shell Scripting? q Where to get more information q The Shell is an Interpreter q The Shebang q Making the script executable q Special Characters q Special Characters—continued: 2 q Special Characters—continued: 3 q Quoting q Quoting—2 q Quoting—When to use it? q True and False q Variables—1 q Variables—Assignments q Variables—Local to Script q Variables—Unsetting Them q Command-line Parameters q Special Built-in Variables q Variables: use Braces ${...} q Braces and Parameters after $9 q More about Quoting q Command Substitution q Command Substitution—Example q Conditions—String Comparisons

s

$ find . \( \( \( \(

{} {} {} {}

\; \; \; \;

\) -o \ \) -o \ \) -o \ \)

s s

s s

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up q shift: Many Places q Command-Line Options—1 q Command-Line Options—2 q Command-Line Options—3 q getopts: Command-Line Options—4 q getopts: Command-Line Options—5 q Temporary Files: mktemp q Signals that may Terminate your Script q Signals: trap q Signals: trap Example q Functions q Parameters in Functions q Example, Calling a Function q Debugging Shell Scripts—1 q Debugging Shell Scripts—2 q Writing Shell Scripts q Useful External Programs—1 q Useful External Programs—2 q Regular Expressions q What is in a Regular Expression? q Literal characters q Character Classes: [...] q Negated Character Classes: [^...] q Match Any Character q Matching the Beginning or End of Line q Match Repetitions: *, ?, +, {n}, {n,m} q Matching Alternatives: “|” q Putting it All Together: Examples q Basic awk q What Does awk Do? q awk Examples q sed—the Stream Editor q sed—Backreferencees q sed—Backreferencees: Example q find Examples q Finding SUID Programs q A find Example with Many Options q rpm Database Query Commands Shell Programming – p. 85/86

q Conditions—Integer Comparisons q Conditions—File Tests, NOT Operator q Conditions—Combining Comparisons q Arithmetic Assignments q Arithmetic Expressions with $((...)) q Arithmetic Conditions with ((...)) q if Statement q while Statement q for Statement q for Loops: Another Example q for Loops: second, C-like syntax OSSI — ver. 1.9 q break and continue q Blocks: {...} q Error Handling: ||, && and exit q Output: echo and printf q Input: the read Command q set: Splitting a Multi-Word Variable q More about set, and IFS q Example: Changing IFS q case Statement q case Statement: Example q shift: Move all Parameters Up q shift: Many Places q Command-Line Options—1 q Command-Line Options—2 q Command-Line Options—3 q getopts: Command-Line Options—4 q getopts: Command-Line Options—5 q Temporary Files: mktemp q Signals that may Terminate your Script q Signals: trap q Signals: trap Example q Functions q Parameters in Functions q Example, Calling a Function q Debugging Shell Scripts—1 q Debugging Shell Scripts—2 q Writing Shell Scripts q Useful External Programs—1 q Useful External Programs—2 q Regular Expressions q What is in a Regular Expression? q Literal characters q Character Classes: [...] q Negated Character Classes: [^...] q Match Any Character q Matching the Beginning or End of Line q Match Repetitions: *, ?, +, {n}, {n,m} q Matching Alternatives: “|” q Putting it All Together: Examples q Basic awk q What Does awk Do? q awk Examples q sed—the Stream Editor q sed—Backreferencees q sed—Backreferencees: Example q find Examples q Finding SUID Programs q A find Example with Many Options q rpm Database Query Commands Shell Programming – p. 86/86