CMT3321 (02/03) (Linux study guide 4 – Shell scripting) For Lab. 4, 5 Lecturer : Mr. Lee Bo-sing (Joe) Agenda ? Programming basics ? Shell script basics ? Shell script content ? Shell quoting ? Shell variable ? Relational condition ? Logical operation ? Arithmetic statement ? Flow control statements ? Std & file I/O statement ? Sed & awk introduction ?? Programming basics (1) ? What does a program/script do: – Input data (from keyboard or file) – Process input data based on specification – Output processed result (to monitor or file) ? 2 types of programs : – Compiled program ? Need to compile before running, eg. C program ? Executes faster – Interpreted program ? Don't need to compile. An interpreter (shell) is responsible for interpret/translate it during running. eg shell script ? Executes slower ? Shell script is just a kind of interpreted program Programming basics (2) ? 2 types of programming bugs : – Syntax bug ? mainly due to unfamiliar with the programming language syntax & linux environment. Lab 4, 5 helps to develop your ability of shell scripting language syntax. ? Read the error messages carefully. It usually provides useful hints. – Logical bug ? most difficult to discover ? Logical thinking error ? Careless human mistake Programming basics (3) ? General procedures for interpreted program : – Understand specification of your program from end user or your boss – Design and write down your programming specification using any pseudo- code or flow-chart – Edit source code of your program using any editor such as pico or vi or any editor you get used to – chmod your source code to executable in unix environment. If you just type the name of script file, make sure that current directory (".") is included in the variable $PATH. Or, type a relative or full pathname of your script file. – Run and check for any conflict with specification. – If conflict exists, modify your script file, debug and re-run. Repeat step 5, 6 until specification is fulfilled Shell script basics (1) ? How does UNIX know this is shell script? – Kernel check the first 2 byte of line of script file for #! – Kernel then check whole first line for #!/bin/ksh or #!/bin/sh or #!/bin/csh or #!/bin/bash to determine which shell interpreter to use. – Note that syntax of different shell varies. For consistency, I use K- shell for teaching. ?? ? How to make shell script executable – In unix prompt, chmod 700 script_filename – Current Directory (".") is included in $PATH Shell script basics (2) ? How to run shell script? – use shell specified in your script to interpret your script (better as portable) ? Just type pathname of script – Or use your current shell to interpret your script ? For ksh or sh script, . pathname_of_script (eg. . ~/.test.ksh) ? For csh script source Pathname_of_script (eg. source ~/.test.csh) ? Exit status : ? 0 => successful ? 1 => fail Shell script content (1) ? A script can include any of followings : – First line MUST be #!/bin/?? – Comments prefixed by # – Any shell built-in commands (mkdir, cd, echo, eval, etc.) – Any UNIX external utilities or executable complied programs (cp, tar, grep, awk, etc.) – Shell variable assignment statement (var_name="value"; variable is declared and assigned at the same time) – Read shell variable content by including $VAR_NAME in script statement. Shell script content (2) – Arithmetic or relational statement – Control flow statement : ? Conditional expression (if, then, else, elseif) ? Looping expression ? Case statement ? Test statement – Std I/O statement : – File I/O statement – Shell function definition & calling Shell quoting (1) ? Some characters have special meaning to shell and need to be quoted when that special meaning is not wanted, eg. You want print $. ? Special characters includes : – ; & ( ) | ^ < > ? * [ ] \ $ ` " ' { } newline tab space ? 3 types of quoting : – Quote single character by backslash (\) eg. Echo \$. Output is $ – Quote a string by a single quote (' '). All special meaning of quoted special characters is lost and these characters will be printed. – Quote a string by a double quote (" "). Similar to single quote but some special characters such as $, `, \ , still have their special meanings. Shell quoting (2) ? When to use quoting : ? Pass special meaning to the called program . eg. find . -name "*.rpm" ? A string with white space ? Eg. echo "There are space in the sentence" ? Intend to print out special character. ? Eg. echo "dollar sign is \$. Back quote is \`. Backslash is \\" shell variable (1) ? Declaration and assignment within one statement – In assignment statement, the variable is not prefixed with dollar sign. White space (space, tab) is not allowed on either side of the equal sign (=) – Eg. VAR = abc (wrong) – Eg. VAR=abc (correct) – However, to access a variable, the variable must be prefixed with dollar sign. – Eg. ${VAR-NAME} (better) or $VAR-NAME – Eg. echo "VAR_NAME is ${VAR_NAME} ? Variable or environment variable defined in a script is not available after the script's execution completes. ? Variables defined in sub-shell are localized and parent shell cannot access these variables ? Read and run file var1.ksh shell variable (2) ? Content of an variable is manipulated as string. Even when a variable contains a number, it is stored as a string of numeric character. ? Content can be – single, double quoted for string – backslash quoted for single character with special meaning , eg. $ ? & ' ' " " ( ) [ ] | < > ` ? Eg. VAR1=abc ? Eg. VAR2='this is a variable' ? Eg. VAR3="today is `date` and VAR1 is $VAR1" ? Eg. VAR4="\$VAR1" ? Read and run file var2.ksh shell variable (3) ? Special predefined variables : – $? => exit status of last executed command – $$ => process ID of current command – $0, $1, $2, $3, $4, ….. – e.g. Command-name par1 par2 par3 par4 – $0 equals to command-name – $1 equals to par1 – $2 equals to par2 – $3 equals to par3 – $4 equals to par4 – $# equals to number of parameters, i.e. 4 ? Read and run file var3.ksh Relational condition ? [ expr1 -eq expr2 ] – Is equal to ? [ expr1 -ne expr2 ] – Is not equal to ? [ expr1 -lt expr2 ] – Is less than ? [ expr1 -le expr2 ] – Is less than or equal to ? [ expr1 -gt expr2 ] – Is greater than ? [ expr1 -ge expr2 ] – Is greater than or equal to ? Refer to file if1.ksh, if2.ksh Logical operation of conditions ? AND of 2 or more conditions : – Condition-1 && condition-2 – [ expr1 -lt expr2 ] && [ expr2 -gt expr3 ] ? OR of 2 or more conditions : – Condition-1 || condition-2 – [ expr1 -lt expr2 ] || [ expr2 -gt expr3 ] ? Logical NOT of a condition : ? ! Condition ? ! [ expr1 -lt expr2 ] && [ expr2 -gt expr3 ] ? Refer to file if1.ksh, if2.ksh arithmetic ? Works for + - * / % and any combination ? Syntax : – Let var-name=3+4-3*2/2 ? Read and run file arith.ksh Shell flow control statement ? Conditional expression (if, then, else, elseif) ? Case statement ? While loop statement ? For loop statement Conditional statement (1) ? if condition1 then command_block1 fi ? if condition1 then command_block1 else command_block2 fi ? Read and run file if1.ksh Conditional statement (2) ? if condition1 then command_block1 elif condition2 command_block2 else command_block3 fi ? Read and run file if2.ksh Case statement ? case string in pattern1) command_block1 ;; pattern2) command_block2 ;; patterni) command_blocki ;; *) command_block ;; esac ? Read and run file case.ksh While & for loop statement ? While loop : – while condition do command_block done ? For loop : – for name in words do Command_block done ? Read and run file loop.ksh Std I/O ? Read data from stdin (keyboard) and assign this data to a variable by : – read VAR_NAME ? Print(or write) to stdout (monitor) by : – echo "anything you want to print, can include variable content, space, execution result of a command" file I/O (1) –write to a file ? 3 steps : ? Open a file and assign a file descriptor "n" to this file. This file is created if not existed. If already created, all original content will be erased. ? exec n>filename (n >= 3) – Redirect any std output of a command to this open file (file descriptor n) using > ? echo "message" 1>&n ? Close the file ? exec n>&- ? Above 3 steps can be simplified into : ? echo "message" > filename file I/O (2) - Append to an existed file ? 3 steps : ? Open the file and assign a file descriptor to this file. ? exec n>>filename (n >= 3) – Redirect any stdout of a command to this open file (file descriptor n) using >> ? echo "message" 1>>&n ? Close the file ? exec n>&- ? Above 3 steps can be simplified into : ? echo "message" >> filename ? Read & run file1.ksh file I/O (3) –read from an existed file ? 3 steps : – Open existed file for reading and assign a file descriptor to this file. ? exec n= 3) ? Redirect any std input of a command to read from this open file (file descriptor n) ? while read LINE 0<&n do …….. done – Close the file ? exec n<&- ? Read & run file2.ksh Sed ? Function : works on stdin or file. Very useful tool – to replace any src_string from stdin to dest_string and print to monitor by "s" ? eg. cat file | sed -e "s/src_str/dest_str/g" ? s stands for substitute, ? g stands for global, – if g is not given, only first src_str is replaced by dest_str. – If g is given, all src_src of each line are replaced by dest_str ? No limitation to length of src_str and dest_str – To delete particular target_str (just specify dest_str as nothing by "s" – Eg. cat file | sed -e "s/target_str//g" Sed – To delete any line containing any target_str by "d" – cat filename | sed -e "/target_str/d" – d stands for delete – To delete any empty line by "d" – cat filename | sed -e "/^$/d" – To delete particular line(s) by "d" – cat filename | sed -e "1,4d" (line 1 to 4) – cat filename | sed -e "1d" (line 1 only) ? cat filename | sed –e "$d" (last line) ? cat filename | sed -e "2,$d" (2nd to last line) ? To select particular line(s) by "p" & "-n" ? cat filename | sed -n "1,4p" (line 1 to 4) ? cat filename | sed -n "4p" (line 4 only) Sed ? More precisely to specify the src_str, target_str by : – Eg. cat file | sed -e "s/^src_str/dest_str/g" – ^src_str stands for only src_str at the beginning of each line – Eg. cat file | sed -e "s/src_str\$/dest_str/g" – src_str\$ stands for any src_str at the end of each line (\ is used as quote special meaning of $ to shell) ? More on sed from http://www.ptug.org/sed/sedfaq.htm awk ? Actually a complete programming language. You can make use its column selection feature to write your script – Similar to cut – Eg. To select field 1,2,3 (field starts counting at 1) , separated by ":", from a file – cat filename | awk -F: '{print $1,$2,$7}' – -F: specify : as field separator, output field 1, 2, 7 – More on awk from http://www.ssc.com/ssc/eap/ Open Discussions Exercise 1 ? Run & understand each of demo. Files : ? var1.ksh, var2.ksh, var3.ksh ? if1.ksh, if2.ksh ? arith.ksh ? loop.ksh ? case.ksh ? file1.ksh file2.ksh Exercise 2 ? Write a script which display "Good morning", "good afternoon" or "good evening" , depending on the time of running the script, to monitor when it runs. Exercise 3 ? Write a script which – a. prompt to read your name & age ? Enter your name: Joe ? Enter your age: 25 – b. write the read name to first line of file, named my_info. – c. change your age by "age*2-3" and write to second line of my_info. ? Content of my_info : – my name is Joe – my modified age is 72 Exercise 4 ? Write a script which read a number in units of seconds and convert it hour:minute:second and print to monitor ? Your script must prompt for re-input if negative value is input – Enter number of seconds : 12345 – Result : – 12345 seconds is 3:25:45 Exercise 5 ? Write a script calculate, which accepts 4 arguments a, b, c, d and print out "a*20-b*2+c/d" to monitor – $ calculate 2 12 5 2 – result of "2*20 – 12*2 + 5/2" is 18