Linux Training Materials Project GBdirect Limited 27 Park Drive Bradford, BD9 4DS West Yorkshire tel: +44 (0)1274 772277 linux@gbdirect.co.uk with contributions from Nick Urbanik nicku@vtc.edu.hk 30 June 2004 Contents 1 Overview 1.1 Generic Features of Unix . . . . . . . . . . . 1.2 Linux — The Kernel of a System . . . . . . . 1.3 Fundamental Characteristics of Linux . . . . . 1.4 Multiuser Multitasking and Time-sharing . . . 1.5 Protected memory mode . . . . . . . . . . . . 1.6 Multiple Filesystem Types . . . . . . . . . . . 1.7 The Many Faces of a GNU/Linux System . . 1.8 The Filesystem . . . . . . . . . . . . . . . . . 1.9 Filenames . . . . . . . . . . . . . . . . . . . . 1.10 Filename Extensions and File Types . . . . . 1.11 Hidden Filenames . . . . . . . . . . . . . . . 1.12 The Shell (bash) . . . . . . . . . . . . . . . . 1.13 Key Features of the Bash Shell . . . . . . . . 1.14 Interacting with a Linux ‘Terminal’ . . . . . . . 1.15 Software Tools: The UNIX Philosophy . . . . 1.16 Tasks/Processes . . . . . . . . . . . . . . . . 1.17 Process Communication . . . . . . . . . . . . 1.18 Re-directing I/O to and from Files . . . . . . . 1.19 Re-directing I/O to and from Files (continued) 1.20 Pipes & Tools . . . . . . . . . . . . . . . . . . 1.21 Linux as a Programming Environment . . . . 1.22 Networking . . . . . . . . . . . . . . . . . . . 1.23 TCP/IP . . . . . . . . . . . . . . . . . . . . . . 1.24 Documentation . . . . . . . . . . . . . . . . . 1.25 Using the man pages (On-Line Manual) . . . 1.26 Overview Exercises . . . . . . . . . . . . . . 1.27 Overview Solutions . . . . . . . . . . . . . . . 2 Basic Shell 2.1 Introduction . . . . . . . . . . . . . 2.2 Getting around the command line . 2.3 History . . . . . . . . . . . . . . . . 2.4 Plumbing . . . . . . . . . . . . . . 2.5 Plumbing (continued) . . . . . . . . 2.6 Output Redirection . . . . . . . . . 2.7 Input Redirection . . . . . . . . . . 2.8 Combining Redirection . . . . . . . 2.9 Pipelines . . . . . . . . . . . . . . . 2.10 Background Processes . . . . . . . 2.11 Background Processes (continued) 2.12 Background Processes and nohup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 30 33 34 35 36 37 38 39 40 41 42 43 44 45 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Contents iv 2.13 2.14 2.15 2.16 2.17 2.18 2.19 2.20 2.21 2.22 Command Grouping and Sub-shells Process Management . . . . . . . . Signals . . . . . . . . . . . . . . . . . Signals (continued) . . . . . . . . . . Background Processes: top . . . . . Filename Generation . . . . . . . . . Quoting Mechanisms . . . . . . . . . Shell built-in commands . . . . . . . Basic Shell Exercises . . . . . . . . Basic Shell Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 47 48 49 50 51 52 53 54 56 59 60 61 62 63 64 66 67 68 69 70 71 72 73 75 76 77 78 79 80 81 82 83 85 86 88 89 90 91 92 93 95 96 97 98 99 100 101 102 104 3 Basic Tools 3.1 Introduction . . . . . . . . . . . . . . . . . 3.2 Using Tools . . . . . . . . . . . . . . . . . 3.3 The On-Line Manual (man) . . . . . . . . . 3.4 Finding Files the Long Way (find) . . . . 3.5 Find examples . . . . . . . . . . . . . . . 3.6 Locate Files (locate) . . . . . . . . . . . . 3.7 View and Concatenate Files (cat) . . . . . 3.8 View Large Files & Output (less) . . . . . 3.9 Viewing Parts of Files (head and tail) . . 3.10 Listing File Information (ls) . . . . . . . . 3.11 File Classification (file) . . . . . . . . . . 3.12 Count Words, Lines, Characters (wc) . . . 3.13 Differences Between Files (diff) . . . . . 3.14 Compare Binary Files (cmp) . . . . . . . . 3.15 Regular Expression Searches (grep) . . . 3.16 grep examples . . . . . . . . . . . . . . . 3.17 Sort and Merge Files (sort) . . . . . . . . 3.18 sort Examples . . . . . . . . . . . . . . . 3.19 Display Unique Lines (uniq) . . . . . . . . 3.20 Split Files (split) . . . . . . . . . . . . . . 3.21 Splitting Files by Context (csplit) . . . . 3.22 Dividing files into columns: cut . . . . . . 3.23 Compression Utilities: bzip2 and gzip . . 3.24 Store and Retrieve Archives (tar) . . . . . 3.25 Translating Characters (tr) . . . . . . . . 3.26 Examples of tr Usage . . . . . . . . . . . 3.27 Execute programs at specified times (at) . 3.28 Options and commands related to at . . . 3.29 Running commands regularly (crontab) . 3.30 Evaluate expressions (expr) . . . . . . . . 3.31 Linux Printing . . . . . . . . . . . . . . . . 3.32 LPRng and CUPS . . . . . . . . . . . . . 3.33 Main Printing Tools . . . . . . . . . . . . . 3.34 Using lpr . . . . . . . . . . . . . . . . . . 3.35 Using lpq . . . . . . . . . . . . . . . . . . 3.36 Using lprm . . . . . . . . . . . . . . . . . 3.37 Printing Information . . . . . . . . . . . . . 3.38 Basic Tools Exercises . . . . . . . . . . . 3.39 Basic Tools Solutions . . . . . . . . . . . . www.linuxtraining.co.uk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . c GBdirect Ltd and parts c Nick Urbanik, 2004 v Contents 4 More Tools 4.1 Introduction . . . . . . . . . . . . . . . . . . 4.2 Displaying System Processes (top) . . . . . 4.3 Options and Interactive Commands for top 4.4 Reporting process status (ps) . . . . . . . . 4.5 Options for Reporting process status (ps) . 4.6 Sorting output of ps . . . . . . . . . . . . . . 4.7 Flavours of ps Options . . . . . . . . . . . . 4.8 Examples using ps . . . . . . . . . . . . . . 4.9 Finding Files using specified criteria (find) 4.10 Criteria used in find expressions . . . . . . 4.11 Examples of using (find) . . . . . . . . . . 4.12 Reporting virtual memory statistics (vmstat) 4.13 Output from vmstat . . . . . . . . . . . . . . 4.14 free . . . . . . . . . . . . . . . . . . . . . . . 4.15 ldd . . . . . . . . . . . . . . . . . . . . . . . 4.16 uptime . . . . . . . . . . . . . . . . . . . . . 4.17 xargs — Filters . . . . . . . . . . . . . . . . 4.18 xargs — an Adapter . . . . . . . . . . . . . 4.19 xargs . . . . . . . . . . . . . . . . . . . . . 4.20 Options to xargs . . . . . . . . . . . . . . . 4.21 Positioning filenames with xargs . . . . . . 4.22 cpio . . . . . . . . . . . . . . . . . . . . . . 4.23 gzip . . . . . . . . . . . . . . . . . . . . . . 4.24 Unzipping . . . . . . . . . . . . . . . . . . . 4.25 tar . . . . . . . . . . . . . . . . . . . . . . . 4.26 tar Examples . . . . . . . . . . . . . . . . . 4.27 Raw devices and tar . . . . . . . . . . . . . 4.28 Exercises . . . . . . . . . . . . . . . . . . . 4.29 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 5 Basic Filesystem 5.1 Filesystem Overview . . . . . . . . . . . . . . . . . 5.2 Files . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Directories . . . . . . . . . . . . . . . . . . . . . . . 5.4 Directory Hierarchy . . . . . . . . . . . . . . . . . . 5.5 Pathnames . . . . . . . . . . . . . . . . . . . . . . 5.6 Current Directory, Home Directory . . . . . . . . . 5.7 Dot (.) and DotDot(..) . . . . . . . . . . . . . . . . . 5.8 Moving and Copying Files . . . . . . . . . . . . . . 5.9 Removing Files . . . . . . . . . . . . . . . . . . . . 5.10 Operations on Directories . . . . . . . . . . . . . . 5.11 Inodes . . . . . . . . . . . . . . . . . . . . . . . . . 5.12 Inodes: ls -i and stat . . . . . . . . . . . . . . . 5.13 Links . . . . . . . . . . . . . . . . . . . . . . . . . . 5.14 Hard links . . . . . . . . . . . . . . . . . . . . . . . 5.15 Symbolic Links (Soft Links) . . . . . . . . . . . . . 5.16 Symbolic (or Soft) Links (continued) . . . . . . . . 5.17 File Ownership, Users and Groups . . . . . . . . . 5.18 Access Control, Users and Groups . . . . . . . . . 5.19 Categories of Access Control . . . . . . . . . . . . 5.20 Access Control — Example . . . . . . . . . . . . . 5.21 Examples of minimum file permission requirements 5.22 Changing Access Permission: chmod . . . . . . . . c GBdirect Ltd and parts c Nick Urbanik, 2004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . www.linuxtraining.co.uk Contents vi 5.23 5.24 5.25 5.26 5.27 5.28 5.29 5.30 5.31 5.32 5.33 5.34 5.35 5.36 5.37 5.38 5.39 5.40 chmod symbolically . . . . . . . . . . . . . . . . . . . chmod numerically . . . . . . . . . . . . . . . . . . . Special Permissions: SUID, SGID . . . . . . . . . . . chmod: Symbolic Permissions . . . . . . . . . . . . . chmod: SUID, SGID . . . . . . . . . . . . . . . . . . . Set Group ID Directory . . . . . . . . . . . . . . . . . Set Group ID Directory — Example . . . . . . . . . . Restricted Deletion Flag (“Sticky Bit”) on Directories umask . . . . . . . . . . . . . . . . . . . . . . . . . . Special Files — /dev . . . . . . . . . . . . . . . . . . Special Files — /proc . . . . . . . . . . . . . . . . . Filesystem Structure and /etc/fstab . . . . . . . . /etc/fstab — Example . . . . . . . . . . . . . . . . Mounting Additional Volumes . . . . . . . . . . . . . Mounting shared filesystems . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . Filesystem Exercises . . . . . . . . . . . . . . . . . . Filesystem Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 180 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 199 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 6 Finding Documentation 6.1 Documentation everywhere? . . . . . . . . . . . . . . . . . . . 6.2 Where is the documentation on my computer? . . . . . . . . . 6.3 Some main sources of information from the Internet . . . . . . 6.4 Mailing Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Asking Questions on a Mailing List . . . . . . . . . . . . . . . . 6.6 Online Magazines . . . . . . . . . . . . . . . . . . . . . . . . . 6.7 Info . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.8 Using the info command . . . . . . . . . . . . . . . . . . . . . 6.9 Using emacs to read info pages . . . . . . . . . . . . . . . . . 6.10 Using rpm to identify all documentation for a software package 6.11 A quick guide to rpm . . . . . . . . . . . . . . . . . . . . . . . . 6.12 A quick guide to dpkg (on Debian Linux) . . . . . . . . . . . . . 6.13 Browsing Documentation Via Your Web Server . . . . . . . . . 6.14 The exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.15 Documentation: Solutions . . . . . . . . . . . . . . . . . . . . . 7 Administering User Accounts and Permissions with sudo 7.1 System Administration without always being SuperUser 7.2 Setting your PATH . . . . . . . . . . . . . . . . . . . . . 7.3 Linux is a Multiuser System . . . . . . . . . . . . . . . . 7.4 User account overview . . . . . . . . . . . . . . . . . . . 7.5 password file . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Example passwd file . . . . . . . . . . . . . . . . . . . . 7.7 group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8 shadow file . . . . . . . . . . . . . . . . . . . . . . . . . . 7.9 logging in . . . . . . . . . . . . . . . . . . . . . . . . . . 7.10 logging in—Pluggable Authentication Modules (PAM) . . 7.11 Adding User Accounts with useradd . . . . . . . . . . . 7.12 What happens when you create a user account? . . . . 7.13 Local accounts and LDAP accounts . . . . . . . . . . . 7.14 Configuring useradd to create local accounts . . . . . . 7.15 Creating a group . . . . . . . . . . . . . . . . . . . . . . 7.16 Adding a user to a secondary group . . . . . . . . . . . 7.17 What groups does this user belong to? . . . . . . . . . . www.linuxtraining.co.uk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . c GBdirect Ltd and parts c Nick Urbanik, 2004 vii Contents 7.18 7.19 7.20 7.21 7.22 7.23 7.24 7.25 Effective group ID and newgrp . . . . . . . . . . . . Directory for a Group Project . . . . . . . . . . . . File permissions for directories . . . . . . . . . . . Examples of minimum file permission requirements Set Group ID Directory . . . . . . . . . . . . . . . . Set Group ID Directory — Example . . . . . . . . . User Management Exercises . . . . . . . . . . . . User Management Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 220 221 222 223 224 225 227 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 270 8 Managing Users–quotas 8.1 Checking /etc/passwd and /etc/shadow with pwck . . . . . . . . . . . 8.2 Checking /etc/group with grpck . . . . . . . . . . . . . . . . . . . . . 8.3 Managing User Connections: login, /etc/securetty, /etc/usertty 8.4 Limiting User Resources with ulimit . . . . . . . . . . . . . . . . . . . 8.5 Managing Disk Use with Quotas . . . . . . . . . . . . . . . . . . . . . 8.6 Setting up Quotas on a Filesystem . . . . . . . . . . . . . . . . . . . . 8.7 Specifying Quotas for Users and Groups . . . . . . . . . . . . . . . . . 8.8 Checking and Reporting on Quotas . . . . . . . . . . . . . . . . . . . . 8.9 Managing Users–quotas: Exercises . . . . . . . . . . . . . . . . . . . 9 Introduction to Editing With vi 9.1 Text editors under Linux . . . . . . 9.2 vi and your terminal . . . . . . . . 9.3 vi screen layout . . . . . . . . . . 9.4 Opening files with vi . . . . . . . . 9.5 vi Modes . . . . . . . . . . . . . . 9.6 Saving, changing file and quitting . 9.7 Moving around in command mode 9.8 Numeric Prefixes . . . . . . . . . . 9.9 Further Movement . . . . . . . . . 9.10 Further Movement — Example . . 9.11 Movement by lines . . . . . . . . . 9.12 Movement by lines — Examples . 9.13 Inserting text . . . . . . . . . . . . 9.14 i command . . . . . . . . . . . . . 9.15 Multiple Insertion . . . . . . . . . . 9.16 Deleting Text . . . . . . . . . . . . 9.17 Changing Text . . . . . . . . . . . . 9.18 Copy and Paste . . . . . . . . . . . 9.19 Finding your place . . . . . . . . . 9.20 Miscellaneous Commands . . . . . 9.21 Search and replace . . . . . . . . . 9.22 Regular Expressions . . . . . . . . 9.23 Regular Expression Conventions . 9.24 Regular Expression Examples . . . 9.25 Regular Expression Replacement . 9.26 Help . . . . . . . . . . . . . . . . . 9.27 vi Exercises . . . . . . . . . . . . 9.28 vi Solutions . . . . . . . . . . . . . c GBdirect Ltd and parts c Nick Urbanik, 2004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . www.linuxtraining.co.uk Contents viii 10 Basic X-Windows 10.1 What X-Windows Is . . . . . . . . . . . . . . . . 10.2 X Needs Window Managers . . . . . . . . . . . 10.3 Window Managers Are Applications . . . . . . 10.4 Desktop Environments . . . . . . . . . . . . . . 10.5 Starting X . . . . . . . . . . . . . . . . . . . . . 10.6 Stopping X . . . . . . . . . . . . . . . . . . . . 10.7 Running Shells (Xterms) Under X . . . . . . . . 10.8 Running Applications from an xterm . . . . . . 10.9 Running Applications from a window manager . 10.10 Configuring X . . . . . . . . . . . . . . . . . . . 10.11 Basic X Hardware Configuration . . . . . . . . 10.12 Basic X Software Configuration . . . . . . . . . 10.13 Networked X — The Client-Server Relationship 10.14 Principles of Running Remote X Apps . . . . . 10.15 How to Run Remote X Apps . . . . . . . . . . . 10.16 Authentication . . . . . . . . . . . . . . . . . . . 10.17 Better Authentication . . . . . . . . . . . . . . . 10.18 Basic X Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 317 318 319 320 321 322 323 324 325 326 327 11 Fundamentals of TCP/IP 11.1 Fundamentals of TCP/IP Networking . . . . . . . . 11.2 History . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Recap of basic IP Concepts — Components . . . . 11.4 IP versions . . . . . . . . . . . . . . . . . . . . . . 11.5 Packets . . . . . . . . . . . . . . . . . . . . . . . . 11.6 Encapsulation . . . . . . . . . . . . . . . . . . . . . 11.7 Internet Protocol Datagram . . . . . . . . . . . . . 11.8 TCP Header . . . . . . . . . . . . . . . . . . . . . . 11.9 UDP Header . . . . . . . . . . . . . . . . . . . . . 11.10 Addresses . . . . . . . . . . . . . . . . . . . . . . . 11.11 Addresses (continued) . . . . . . . . . . . . . . . . 11.12 Netmasks and subnetting . . . . . . . . . . . . . . 11.13 CIDR: Classless Inter-Domain Routing . . . . . . . 11.14 CIDR: Classless Inter-Domain Routing—examples 11.15 Transferring Data . . . . . . . . . . . . . . . . . . . 11.16 Hosts & Interfaces . . . . . . . . . . . . . . . . . . 11.17 Routing . . . . . . . . . . . . . . . . . . . . . . . . 11.18 Ports . . . . . . . . . . . . . . . . . . . . . . . . . . 11.19 Ports cont.. . . . . . . . . . . . . . . . . . . . . . . 11.20 Exercises . . . . . . . . . . . . . . . . . . . . . . . 11.21 Solutions . . . . . . . . . . . . . . . . . . . . . . . 12 Practical TCP/IP 12.1 Ping Protocols . . . . . . . . . . . . . . 12.2 Network Statistics (netstat) in Practice 12.3 netstat (continued) . . . . . . . . . . . 12.4 netstat — Further Examples . . . . . . 12.5 Network Traffic (tcpdump) in Practice . . 12.6 tcpdump Options . . . . . . . . . . . . . 12.7 tcpdump Examples . . . . . . . . . . . . 12.8 Firewalling . . . . . . . . . . . . . . . . . 12.9 Basic Theory . . . . . . . . . . . . . . . 12.10 Basic Theory (continued) . . . . . . . . www.linuxtraining.co.uk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . c GBdirect Ltd and parts c Nick Urbanik, 2004 ix Contents 12.11 12.12 12.13 12.14 12.15 12.16 12.17 12.18 12.19 12.20 12.21 12.22 12.23 12.24 12.25 12.26 12.27 12.28 12.29 12.30 12.31 12.32 12.33 ipchains . . . . . . . . . . . . . . . . . . ipchains Details . . . . . . . . . . . . . . ipchains Options . . . . . . . . . . . . . . Options For Rules . . . . . . . . . . . . . ipchains — Examples . . . . . . . . . . . Removing Rules . . . . . . . . . . . . . . Implementing ipchains . . . . . . . . . . . Save and restore . . . . . . . . . . . . . . ipchains setup script . . . . . . . . . . . Real World ipchains . . . . . . . . . . . . Interface Configuration and Management Point-and-Click Interface Administration . /etc/sysconfig/network-scripts . . . ifcfg-ethx . . . . . . . . . . . . . . . . . Altering An Interface . . . . . . . . . . . . Adding an Interface . . . . . . . . . . . . . The ‘Proper’ Way . . . . . . . . . . . . . . Drivers . . . . . . . . . . . . . . . . . . . . The Secure Shell in Practice (ssh) . . . . Secure Copying in Practice (scp) . . . . . Summary . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 378 379 380 381 13 SSH — The Secure Shell 13.1 What is the Secure Shell? . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2 But what’s wrong with telnet? . . . . . . . . . . . . . . . . . . . . . . . 13.3 Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4 OpenSSH and its history . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5 Okay, I like the blowfish—what else does OpenSSH provide? . . . . . . 13.6 So okay, how do I use this Secure Shell? . . . . . . . . . . . . . . . . . . 13.7 Using scp to copy files over the network . . . . . . . . . . . . . . . . . . 13.8 Useful options with scp . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.9 SSH uses public and private keys . . . . . . . . . . . . . . . . . . . . . . 13.10 SSH Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.11 Overview of SSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.12 Steps of establishing a connection . . . . . . . . . . . . . . . . . . . . . 13.13 Using ssh-keygen to create a personal pair of private and public keys . 13.14 The host keys in /etc/ssh/ssh known hosts and ∼/.ssh/known hosts 13.15 The file ∼/.ssh/authorized keys . . . . . . . . . . . . . . . . . . . . . 13.16 The User’s Public and Private Keys . . . . . . . . . . . . . . . . . . . . . 13.17 SSH1 and SSH2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.18 The public and private key pairs: a summary . . . . . . . . . . . . . . . 13.19 Files and Permissions I Recommend . . . . . . . . . . . . . . . . . . . . 13.20 Using ssh-agent to log in without typing passwords . . . . . . . . . . . 13.21 Setting up ssh-agent: logging in without typing passwords . . . . . . . . 13.22 Using ssh-add: logging in without typing passwords . . . . . . . . . . . 13.23 An easier way: using keychain . . . . . . . . . . . . . . . . . . . . . . . 13.24 What keychain Does . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.25 Setting your hostname . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.26 Configuring your own account to use keychain . . . . . . . . . . . . . . 13.27 Running X applications remotely . . . . . . . . . . . . . . . . . . . . . . 13.28 Configuring SSH for X . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.29 Security options for the client in /etc/ssh/ssh config . . . . . . . . . . c GBdirect Ltd and parts c Nick Urbanik, 2004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . www.linuxtraining.co.uk Contents x 13.30 13.31 13.32 13.33 13.34 13.35 13.36 13.37 rsync: using it with SSH to mirror data Examples of using rsync . . . . . . . Using ssh from Windows, with Cygwin What else can SSH do? . . . . . . . . Summary . . . . . . . . . . . . . . . . SSH References . . . . . . . . . . . . Secure Shell Exercises . . . . . . . . . Secure Shell Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 383 384 385 386 387 388 390 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 14 Shared File Systems 14.1 NFS (Network File System) . . . 14.2 NFS Basics . . . continued . . . . 14.3 Exporting File Systems . . . . . . 14.4 Viewing exports . . . . . . . . . . 14.5 Importing File Systems . . . . . . 14.6 Samba . . . . . . . . . . . . . . . 14.7 Samba — Availability . . . . . . . 14.8 Samba Documentation . . . . . . 14.9 Samba Installation . . . . . . . . 14.10 Samba Basics . . . . . . . . . . . 14.11 Access to Files and Printers . . . 14.12 Testing Samba . . . . . . . . . . 14.13 Smbclient . . . . . . . . . . . . . 14.14 Samba configuration File . . . . . 14.15 Samba Configuration Example . 14.16 Directories for Samba as a PDC 14.17 Testing Samba . . . . . . . . . . 14.18 Exercises . . . . . . . . . . . . . 14.19 Solutions . . . . . . . . . . . . . 15 Apache Basics 15.1 What is Apache? . . . . . . 15.2 Installation . . . . . . . . . . 15.3 How Apache Listens . . . . 15.4 Configuration File(s) . . . . 15.5 Key Configuration Directives 15.6 ServerRoot, DocumentRoot 15.7 Is Apache running? . . . . . 15.8 ServerAdmin . . . . . . . . 15.9 BindAddress, and Port . . . 15.10 Listen . . . . . . . . . . . . 15.11 User and Group . . . . . . . 15.12 Apache Processes . . . . . 15.13 Logging . . . . . . . . . . . 15.14 Customizable Logging . . . 15.15 CustomLog examples . . . 15.16 Example Configuration . . . 15.17 Basic Exercises . . . . . . . 15.18 Solutions . . . . . . . . . . www.linuxtraining.co.uk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . c GBdirect Ltd and parts c Nick Urbanik, 2004 xi Contents 16 Apache 16.1 Two sites and more . . . . . . . . . . . . . . . . . . . . . . 16.2 Two sites and more . . . continued . . . . . . . . . . . . . 16.3 Virtual Hosting Options . . . . . . . . . . . . . . . . . . . 16.4 Name-based hosting . . . . . . . . . . . . . . . . . . . . 16.5 Name-based hosting (continued) . . . . . . . . . . . . . 16.6 IP-based hosting . . . . . . . . . . . . . . . . . . . . . . 16.7 Block Directives . . . . . . . . . . . . . . . . . . . . . . . 16.8 Block Directives (continued) . . . . . . . . . . . . . . . . 16.9 DirectoryMatch, et al. . . . . . . . . . . . . . . . . . . . . 16.10 Access Control using .htaccess files . . . . . . . . . . . 16.11 Access Control (continued) . . . . . . . . . . . . . . . . 16.12 Authorisation Files . . . . . . . . . . . . . . . . . . . . . 16.13 Authorisation Files (continued) . . . . . . . . . . . . . . 16.14 Access Control using httpd.conf . . . . . . . . . . . . . 16.15 Pros and Cons of using Access Files for Authentication 16.16 How Can Users Change Their Password? . . . . . . . . 16.17 WebDAV: a protocol for web collaboration . . . . . . . . 16.18 WebDAV and Apache . . . . . . . . . . . . . . . . . . . 16.19 WebDAV Configuration . . . . . . . . . . . . . . . . . . . 16.20 Apache WebDAV configuration example . . . . . . . . . 16.21 Configuring WebDAV: directories and files . . . . . . . . 16.22 What is WebDAV useful for? . . . . . . . . . . . . . . . . 16.23 What is the future of WebDAV? . . . . . . . . . . . . . . 16.24 Information about WebDAV . . . . . . . . . . . . . . . . 16.25 Other useful directives . . . . . . . . . . . . . . . . . . . 16.26 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 16.27 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . 16.28 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Key Configuration Files 17.1 /etc/passwd . . . . . . . . . . . . 17.2 /etc/passwd (continued) . . . . . . 17.3 Editing /etc/passwd . . . . . . . . 17.4 Other Changes To /etc/passwd . . 17.5 /etc/group . . . . . . . . . . . . . 17.6 Editing /etc/group . . . . . . . . . 17.7 Important Note . . . . . . . . . . . 17.8 Shadow Passwords . . . . . . . . . 17.9 /etc/shadow . . . . . . . . . . . . 17.10 Scheduling Jobs (Cron) . . . . . . 17.11 /etc/crontab . . . . . . . . . . . . 17.12 run-parts . . . . . . . . . . . . . . 17.13 logrotate . . . . . . . . . . . . . . 17.14 Module Configuration . . . . . . . . 17.15 Modules Configuration — ‘Options’ 17.16 Mounting Filesystems . . . . . . . 17.17 Runlevels . . . . . . . . . . . . . . 17.18 Single User Mode . . . . . . . . . . 17.19 Multi User Mode . . . . . . . . . . 17.20 Starting up and Shutting down . . 17.21 Changing runlevel . . . . . . . . . 17.22 Initscripts . . . . . . . . . . . . . . 17.23 rcn.d . . . . . . . . . . . . . . . . c GBdirect Ltd and parts c Nick Urbanik, 2004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . www.linuxtraining.co.uk Contents xii 17.24 17.25 17.26 17.27 Initscripts — An example . Restarting Services . . . Exercises . . . . . . . . . Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489 490 491 492 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 Module 1 Overview Objectives Having completed this module, you will have an overview of a Linux system, including its: • Underlying philosophy • System layering — kernel vs. applications • Core services • Multiuser and timesharing facilities • File System • Network Services • Desktop and X windowing system 1 1.1 Generic Features of Unix 2 1.1 Generic Features of Unix • Component-based systems • Very popular with technically skilled • Not ‘solution’ oriented • Building blocks not the building • Highly network-aware • Robust, powerful, reliable www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 3 Overview 1.2 Linux — The Kernel of a System Text Editors Graphics Apps Compilers DBMS Mailers Shells & Utilities Kernel Hardware X Windowing System Figure 1.1: kernel-layering • What is called Linux is actually a collection of components from many sources ◦ freely copiable, under ‘open source’ licences • Linux is, strictly, just the kernel which provides: ◦ A common interface between user process and hardware ◦ Minimal functions to user applications, i.e. system calls ◦ Scheduling c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.3 Fundamental Characteristics of Linux 4 1.3 Fundamental Characteristics of Linux • Multi-tasking • Multi-user access • Multi-processor • Architecture independence • POSIX 1003.1 plus basic System V and BSD • Protected memory mode • Multiple filesystem types • Comprehensive networking (TCP/IP and others) • Multiple executable formats (MS-DOS, iBCS UNIX, SCO, etc) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 5 Overview 1.4 Multiuser Multitasking and Time-sharing • Designed as a multi-user system ◦ Each user’s shells, apps and commands are separate processes ◦ Number of simultaneous users limited only by: – CPU speed and available memory – Min. response times required by users/apps • Multi-tasking: ◦ Many jobs can be under way at the same time ◦ Jobs truly simultaneous on multi-cpu • Time-sharing: A single cpu is shared by all processes ◦ Processes exec briefly, passing cpu to others ◦ Process switches occur in miliseconds or less ◦ Kernel gives process a sense of total control c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.5 Protected memory mode 6 1.5 Protected memory mode • Uses the processor’s protection mechanisms • Prevent access to memory already allocated to kernel or other processes • Bad programs can’t crash the system ◦ Theoretically www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 7 Overview 1.6 Multiple Filesystem Types • Native FS is ext3 (Third Extended File System) ◦ File names up to 255 chars ◦ More secure than conventional UNIX • Others include: ◦ MS-DOS (FAT16), VFAT, FAT32 ◦ ISO9660 (CD-ROM) ◦ HPFS (OS/2) ◦ NTFS (Windows NT) ◦ reiserfs, XFS, other journalling file systems for Linux, ◦ UPS, SysV and other proprietory UNIX ◦ NFS (Unix network file system) ◦ SMB / CIFS (MS Windows file sharing) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.7 The Many Faces of a GNU/Linux System 8 1.7 The Many Faces of a GNU/Linux System • The user may see up to five aspects of Linux: ◦ the filesystem ◦ processes ◦ the shell ◦ the X windowing system ◦ Inter-Process Communication (IPC) • The system is very highly configurable • Different users may experience totally different views of the same system • Multiple simultaneous users are normal ◦ Linux is designed from the ground up as a multi-user system, NOT a ‘personal’ system www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 9 Overview 1.8 The Filesystem • The filesystem contains all data in the system • A name in the filesystem can refer to: ◦ a data file, which can be: – a plain file – a directory ◦ a device (disk, tape etc.) ◦ internal memory ◦ OS information (the proc system) • Directories are groups of files ◦ Grouped in hierarchical trees • Files are fully specified with their pathname • An original Unix structure; copied by most OSs c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.9 Filenames 10 1.9 Filenames • Maximum length depends on filesystem type ◦ Most allow up to 255 characters • Can use almost any character in a filename, but avoid ambiguity by sticking to: ◦ (A-Z) Uppercase letters ◦ (a-z) Lowercase letters ◦ (0-9) Numbers ◦ (.) Full-stop ◦ (,) Comma ◦ ( ) Underscore ◦ (-) Hyphen • Should convey meaningful info about contents • Type longer filenames using completion for: ◦ Filenames ◦ Pathnames ◦ Commands www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 11 Overview 1.10 Filename Extensions and File Types • Filenames don’t determine other attributes of file, i.e. do not, automatically, cause command interpreters to treat them in a particular way • However: ◦ Extensions can enable meaningful naming and automatic file manipulation ◦ C compilers and some other programs do depend on specific file extensions to carry out particular tasks • Common conventions for extensions: Filename program.c program.o program.sh letter.txt letter.ps letter.ps.gz letter.tar.bz2 letter.tgz letter.tar.gz letter.Z Meaning of Extension C programming source file Object code Shell executable Text file of a letter Postscript version of same letter file gzip compressed version of same tar archive of same compressed by bzip2 tar archive of same compressed by gzip Another, more common, way of naming *.tgz Same file compressed with outdated compress utility Table 1.1: Common conventions for filename extensions c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.11 Hidden Filenames 12 1.11 Hidden Filenames • Filenames beginning with a full-stop are hidden • Typically used: ◦ To hide personal configuration files ◦ To avoid cluttering dirs with rarely used files • Every dir contains 2 special hidden files: . The current directory file .. The parent directory file www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 13 Overview 1.12 The Shell (bash) • A shell is a program that you interact with User Shell Kernel • Can be any program, but is normally a command interpreter • A command interpreter is usually started when you log in (but this is just one way) • The ‘standard’ Linux command interpreter is a Bourne shell look-alike called bash ∗ • The command line syntax provided by bash enables manipulation of files & processes • The command-line frightens beginners but is the preferred home of the skilled Bash has more functions than true Bourne shells; incorporating most of the innovations added by the C and Korn shells. Bash functions and flags differ between implementations of UNIX and Linux. The version of bash in current Linux releases tends to be the most fully functional Bourne shell around. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk ∗ 1.13 Key Features of the Bash Shell 14 1.13 Key Features of the Bash Shell • Command history • Command aliasing • Shell scripting • Filename completion • Command completion • Command line editing (emacs and vi styles) • Job control • Key Bindings • Directory stacking • Tilde directory notation • Help function, e.g. $ help history history: history [n] [ [-awrn] [filename]] Display the history list with line numbers. Lines listed with with a ‘*’ have been modified. Argument of N says to list only the last N lines. Argument ‘-w’ means to write out the current history file; www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 15 Overview 1.14 Interacting with a Linux ‘Terminal’ • Linux can support any number of ‘terminal’ types ◦ nowadays, monitor/keyboard combinations ◦ previously, dumb terminals ◦ occasionally, printers (debugging servers) • Most will use the console or a windowed terminal, but if not: ◦ Linux usually keeps a database of terminal capabilities in /etc/termcap ∗ ◦ If your terminal type is not recorded in /etc/termcap, you’ll have problems running certain programs e.g. – cursor driven apps (top, linuxconf, vi etc) ◦ The environmental variable TERM tells programs what terminal type you are using AT&T flavours of UNIX use /usr/lib/terminfo to store the same information and Linux can, if necessary. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk ∗ 1.15 Software Tools: The UNIX Philosophy 16 1.15 Software Tools: The UNIX Philosophy • True UNIX-like systems treat programs as tools ◦ Each tool should: – Do just one thing well – Be generic (untied to specific applications) ◦ For new jobs, build new tools ◦ (Re-)combine, don’t complicate old tools • Linux can do this because it has: ◦ two simple objects: – the file – the process ◦ simple methods of connecting: – processes to files – processes to processes FILE 1 PROCESS PROCESS FILE 2 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 17 Overview 1.16 Tasks/Processes • A program is an executable object, stored in a file • A process is an executing object, i.e. ∗ ◦ an instance of a program currently being run • Existing processes can ‘fork’ to create other processes ◦ the only way to make new processes • A user may run multiple copies of same program • Multiple users may run single/multiple copies • System tracks ownership and permission ∗ Processes are often called tasks, as in ‘multi-tasking’ www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 1.17 Process Communication 18 1.17 Process Communication • Processes may need to co-operate by ◦ sharing files ◦ signalling events ◦ direct transfer of data ◦ pipelines (data streams) ◦ synchronising with each other • Linux provides facilities for: ◦ signals ◦ shared memory ◦ pipes, both named and unnamed ◦ semaphores ◦ and others • Processes may use network connections for communication, permitting client-server model ◦ Common for shared services like printing www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 19 Overview 1.18 Re-directing I/O to and from Files • Most processes will take input from the keyboard and output to the screen • Both input and output streams can be re-directed to/from files • Output to a file (creating or overwriting): $ ls > my-system.txt • Appending output to a file: $ who >> my-system.txt my−system.txt ls > kenny khc kkc montague my−system.txt who >> kenny khc kkc montague root root root root pts/1 pts/2 pts/3 pts/4 Sep Sep Sep Sep 5 2 5 4 10:50 14:47 10:56 17:27 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.19 Re-directing I/O to and from Files (continued) 20 1.19 Re-directing I/O to and from Files (continued) • Take input from one file, output to another: $ sort < /etc/passwd > pwd.sorted passwd < sort > pwd.sorted www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 21 Overview 1.20 Pipes & Tools • Linux tools act as filters: ◦ taking data from input streams, modifying it, sending it elsewhere ◦ expecting data to come from other tools ◦ producing output which any other tool can process, e.g. ASCII text • One tool’s output is connected to another’s input: ◦ Indirectly, via a file created by the first tool ◦ Directly, via a pipe or pipeline • For example, to page through a reverse-sorted version of your password file on screen: $ sort -r < /etc/passwd | less passwd < sort −r | less c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.21 Linux as a Programming Environment 22 1.21 Linux as a Programming Environment • Hierarchical Filestore • Extensive set of powerful tools ◦ for software production, admin and support • A common system interface ◦ only one set of procedures to learn • Processes interface with anonymous files ◦ programs output to files or devices identically • Modular architecture provides for a completely customised OS, e.g. ◦ An OS dedicated solely to graphics rendering ◦ A general-purpose system on one floppy • Flexible user interface allows for uniquely customised programming environments www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 23 Overview 1.22 Networking • Linux is a network operating system. • The Internet network protocols (TCP/IP) are implemented in the kernel • Although other media are supported (e.g. radio, infra-red), links are usually across: ◦ Ethernet ◦ Serial Line (Point-to-point) • Proprietory file/print serving protocols supported: ◦ Appletalk ◦ DECNET ◦ IPX / Novell Netware ◦ SMB / CIFS (MS Windows/NT) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.23 TCP/IP 24 1.23 TCP/IP • A suite of Internet-standard protocols and apps for managing data transfers • Depicted as a ‘stack’ ◦ hardware and transport control protocols at the bottom ◦ user applications (e.g. browsers) at the top • Client-server apps provide facilities for: ◦ Remote login ◦ File transfer ◦ Resource sharing (e.g. expensive peripherals) ◦ Remote command execution ◦ Email (internet/intranet/extranet) ◦ Web browsing www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 25 Overview 1.24 Documentation • Copious, but fragmented and/or duplicated Programmer’s Manual /usr/man info pages The classic ‘man pages’, first stop for skilled users, worth learning hypertext browsable texts, often identical or updated versions of man pages /usr/share/doc/program-name ascii/html docs installed with the named program Howtos Tutorials on Linux-related topics, available on-line if installed (usually in /usr/share/doc) www Recently-released programs are usually documented on authorised web sites, many (including older tools) are documented by third-party sites Table 1.2: Sources of Linux Documentation • Linux man pages divided into sections: 1. 2. 3. 4. 5. 6. 7. 8. User Commands System calls Subroutines (inc library routines) Devices (inc network interfaces) File Formats Games Miscellaneous System Administration • The apropos command word searches the description line in man pages. Thus: $ apropos printer will find man pages relating to printers, e.g. lp (4) - line printer devices lpd (8) - line printer spooler daemon lprm (1) - remove jobs from the line printer spooling queue c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.25 Using the man pages (On-Line Manual) 26 1.25 Using the man pages (On-Line Manual) • Use man to see man pages on a named command, e.g $ man date • The result should be something like: DATE(1) NAME date - print or set the system date and time SYNOPSIS date [OPTION]... [+FORMAT] date [-u|--utc|--universal] [MMDDhhmm[[CC]YY][.ss]] FSF DATE(1) • DATE(1) Shows page is in manual section 1 • To view a page from a certain section use: $ man -S section-number command-name • Square brackets surround optional arguments ls [-abcdfgiklmnpqrstuxABCFGLNQRSUX1] www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 27 Overview 1.26 Overview Exercises What should I focus on? At the end of this practical session, as a minimum, you should be able to: • within a man page, be able to: ◦ go to the beginning of the man page with one keystroke ◦ similarly, go to the end of the man page ◦ search for a word within the man page ◦ search again for the same word using one keystroke. • Find the man page that you need: ◦ quickly find a man page with a description that matches a keyword ◦ exhaustively search all man pages to find all that contain a keyword. • You should know how to use the df command to tell how much space is left in your root (/) partition • Have a basic understanding of what a pipe (|) is useful for. • You should know how to use the wc command, and clearly understand that it has nothing much to do with the wash room. The exercises 1. Changing password (a) Set yourself a new password using the passwd command. Run the command by typing passwd, followed by a . 2. Navigating Man Pages (a) Type man man to open the man page which details how to use the man command (b) Press the h (help) key, which opens a “Summary of Less Commands”, including all the keystrokes you need to navigate a man page (c) Make sure you can quit this page (by typing q) and quit the man page (by typing q again). When you get back to the shell prompt, repeat the first 2 steps to open the “Summary of Less Commands” from the man man page. (d) Use the “Summary of Less Commands” to make sure you know how to do the following bits of navigation inside a man page: i. ii. iii. iv. v. Move to the top and bottom of the man page Move up and down one screen of text Move up and down one line of text Search forward for a pattern (e.g. a word) Search backwards for a pattern www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 1.26 Overview Exercises 28 vi. Repeat a forward pattern search using one key vii. Repeat a backward pattern search using one key viii. Move to a specific line number. For this, read the text at the top of “Summary of Less Commands”, particularly: Commands marked with * may be preceded by a number, N . Notes in parentheses indicate the behavior if N is given. The phrase preceded by a number N means that the number N may come first, before the command. Next, have a look at what is under the heading “Jumping”. Note that you can check where you are by pressing “=”, which shows the line numbers at the first and last lines in the terminal window. Try this on the man page for ethereal, which has more than 40,000 lines. See if you can go to line 10,000. (e) With a partner, test each other on how well you can navigate the man man page, e.g. set each other target locations or words to go to. 3. Invoking the Right Man Pages Note that some time after you start your lab session, a program automatically builds a file /var/cache/man/whatis which contains the one summary line from each of the thousands of man pages on your computer. There are two programs, apropos, mentioned in section 1.24 on page 25. There is another program, whatis, that matches a program name in the same “whatis” database. The man command has options that call these programs. Each has their own man page. (a) Using the man man page, find the command string you need to use to get the following: i. A list of man pages whose description lines contain details about the ‘whatis’ database ii. A list of man pages containing the string ‘cdrom’ ∗ iii. A list of man pages from a specific section (e.g. 1) of the manual, whose description lines contain ‘print’ (b) Practice using these flags to find and view man pages which deal with computer keywords your partner sets for you (and vice versa), e.g. i. bitmap formats like jpg, gif, xpm, bmp ii. communications concepts like modem, serial, telnet, pcmcia, ppp iii. filesystems like NFS, ext3, ext2, FAT, vfat, msdos, samba 4. Finding Out About Your System and Users (a) Type the following commands. Identify what each of them tells you about your system. i. ii. iii. iv. v. vi. vii. viii. ix. ∗ $ $ $ $ $ $ $ $ $ whoami who am i users who w date cal 10 2003 cal 9 1752 † df Actually running this sort of search can take a long time, given that many systems contain over 10,000 man pages, some of which are very long. † You should notice something very strange about the output from this string. The cal utility is perfectly functional, so what’s wrong? www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 29 Overview x. xi. xii. xiii. xiv. $ $ $ $ $ which man type man whereis less help cd time sleep 2 (b) Use the appropriate man page, to check that you have interpreted the screen output correctly 5. Creating New Files (a) Try creating a new empty file in your home directory using the touch command, e.g. $ touch filename (b) Get the file details on filename using this command: $ ls -l filename (c) Wait 1 minute, then repeat the previous two steps, i.e. $ touch filename $ ls -l filename i. Which of the file details have changed? ii. What does this tell you about the purpose of touch? Check the man page if you are unsure. (d) Creating new files using re-direction i. Create a new file containing the output from the df command, using re-direction, e.g. $ df > diskspace.txt ii. Ask a partner to create new files, with appropriate filenames, containing output from the commands used in the questions on “Finding Out About Your System and Users”. 6. Appending information to files (a) With a partner, choose several of the system information commands whose outputs may have changed since you completed the previous question. Practice appending the updated information to the file which contains the earlier output. (b) Create a file containing output from w, then append the output from date to it, i.e. time-stamp the output data. 7. Using Simple Pipes (a) Pipe the output from who through the sort command to reverse its order. (b) Sort your /etc/passwd file alphabetically and send the output to a new file (passwd.sorted). (c) Find out what wc does from its man page, then use it at the end of a pipe to analyse the output from other utlities. (d) Repeat the last step, limiting wc to counting words only c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.27 Overview Solutions 30 1.27 Overview Solutions 1. Changing password (a) The system will tell you it is changing your LDAP password. It also tests the quality of your password using a library called cracklib, which rejects passwords that are too short or are dictionary words, or that are too closely related to your user name. 2. Navigating Man Pages (b) It is possible that some Linux distributions won’t use less to display man pages. If that is the case, try to find out how you navigate under that setup and answer the same questions about it. (d) Keystrokes for basic man page navigation: Instruction Top of man page Bottom of man page Forward one screen Backward one screen Up one line Down one line pattern Search forward pattern Search backward Repeat pattern Search forward Repeat pattern Search backward Move to nth line Keystroke(s) g < ESC-< G > ESC-> f ^F ^V SPACE b ^B ESC-v y ^Y k ^K ^P e ^E j ^N RETURN /pattern ?pattern n N ng Table 1.3: Keystrokes for basic man page navigation N.B. Several different keystrokes can be used for the same movement. This is common in UNIX tools designed to operate from any keyboard. less always has a single key method. Multi-key methods are shown without spaces between them. 3. Invoking the Right Man Pages (a) i. $ man -k whatis or, slightly differently: $ man -f whatis ii. $ man -K cdrom will print each man page name and ask you if you want to view the page or quit: [ynq]. If you just want the list of man pages and not view the man pages, you could answer “n” to each question using the yes n command, which just prints “n” to each prompt. You could do it like this: $ yes n | man -K cdrom . . . and if you wanted the names each on a line, with no prompt, then you could filter the output still further with the stream editor sed: $ yes n | man -K cdrom | sed ’s/\? \[ynq\] /\n/g’ but you would need to wait a while for the output. Suppose you wanted the output with just the name of the command without the directory name or the “.gz” at the end? Well, you could do this: $ yes n | man -K cdrom | sed ’s/\? \[ynq\] /\n/g’ | xargs --replace basename {} .gz And suppose. . . Oh well, never mind. Nick really gets carried away sometimes. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 31 Overview iii. There is no easy way to do this yet. Later on you will learn about grep which will allow you to filter the output of man -k print to see only the information you require. You could do it like this: $ man -k print | fgrep ’(1)’ (b) Practice using these flags to find and view man pages which deal with computer keywords your partner sets for you (and vice versa), e.g. i. e.g. $ man -K jpg ii. e.g. $ man -K modem iii. e.g. $ man -K NFS 4. Finding Out About Your System and Users (b) See Table 1.4. Command string $ whoami $ who am i $ users $ who $w $ date $ cal 10 2003 $ cal 9 1752 Output Your username Your username plus machine(s) and terminal you are on Usernames of currently logged on users Who is logged on, when and where Who’s logged on, when, where, what process and what system resources they are using Current date and time, can set date/time Calendar for October 2003 Calendar for September 1752. Strange because 12 days were ‘lost’ in the transition from Gregorian to Julian calendars. Simply search for “September 1752” in Google Disk free, i.e. summarises disk usage Full file and path name for the man executable file Much the same as which man Locates the less executable and its man page Very brief help notes on the cd command. N.B. help only works on very few built-in commands The sleep command puts itself to sleep for 2 seconds. The time command then times the whole process and provides other data on the operation of the sleep command $ $ $ $ $ df which man type man whereis less help cd $ time sleep 2 Table 1.4: Output from basic system information commands 5. Creating new files (a) Your output should be something like: $ touch filename.txt (b) Your output should be something like: $ ls -l filename.txt -rw-rw-r-1 davef davef 0 Jul 21 17:59 filename.txt (c) Your output should now be something like: c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 1.27 Overview Solutions 32 $ touch filename.txt $ ls -l filename.txt -rw-rw-r-1 davef davef 0 Jul 21 17:59 filename.txt $ touch filename.txt $ ls -l filename.txt -rw-rw-r-1 davef davef 0 Jul 21 18:01 filename.txt i. The time stamp has changed ii. The real purpose of touch is to change time stamps, but it is handy for creating new empty files (d) i. Reading diskspace.txt should produce something like this: $ cat diskpace.txt Filesystem Used Available Capacity Mounted on /dev/hda1 65571 406394 14% / /dev/hdc1 5030416 650563 89% /backup /dev/hda5 2000097 857401 70% /home /dev/hda7 14289 457676 3% /tmp /dev/hda6 1136861 741727 61% /usr /dev/hdb 653004 0 100% /mnt/cdrom 6. Appending information to files (b) Your screen should look something like this: $ w > test.txt $ date >> test.txt $ cat test.txt 6:36pm up 16 days, 23:07, 4 users, load average: 1.03, 1.08, 1.02 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT davef ttyp7 oakleigh:0.0 9:39am 0.00s 2.58s 0.02s w mikeb ttyp4 kebab Tue 3pm 23:07 0.20s 0.13s -bash davef ttyp2 oakleigh 9:04am 15:34 7:24 0.07s -bash davef ttypb oakleigh 3:02pm 3:15m 3:00m 0.08s -bash Wed Jul 21 18:36:39 BST 1999 7. Using Simple Pipes (a) $ who | sort -r (b) $ sort /etc/passwd > passwd or $ cat /etc/passwd | sort > passwd.sorted (c) wc prints the number of lines, words, and bytes in files. To get these details for your /etc/mime.types file, you could do the following: $ cat /etc/mime.types | wc 480 726 12682 i.e. 480 lines, 726 words, and 12682 bytes ∗ Another way of doing the same thing without a pipe is $ wc /etc/mime.types 480 726 12682 /etc/mime.types (d) E.g. $ cat /etc/mime.types | wc -w 726 We can get a similar result by typing: $ wc -w /etc/mime.types ∗ N.B. wc only counts whitespace-separated words www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 Module 2 Basic Shell Objectives On completion of this module, you should be able to understand and use the Linux shell to create and combine tools. Topics covered include: • An overview of the command line • The software tools model • File names and types • Shell programming • Command scripts • Job control • I/O — pipes and redirection 33 2.1 Introduction 34 2.1 Introduction • The standard command line interpreter under Linux is bash (/bin/bash or /bin/sh) • An enhanced version of the classic Bourne shell • Shares most features of other shells (C, Korn, etc) and has some more advanced features ◦ ‘Plumbing’ — transparent redirection and pipes ◦ Background processes ◦ Process suspension, resumption, termination ◦ Filename completion and wildcard generation ◦ History www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 35 Basic Shell 2.2 Getting around the command line • You can use the cursor keys to move around and edit the current line∗ • By default, bash uses emacs-like keystrokes for navigation and editing. Here are 4 examples: Keystroke ^a ^e ^k ^w Action Move to the beginning of the line Move to the end of the line Delete to the end of the line Delete the previous ’word’ • To choose emacs or vi-like keystrokes: $ set -o emacs $ set -o vi • bash man page gives details of all keystrokes ∗ This may not work on badly-configured systems www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 2.3 History 36 2.3 History • Bash remembers used commands (in a ‘history’) • Old commands are retrievable in different ways • Repeat the previous command by typing !! • Execute the nth previous command by typing !-n • Typing !string repeats the last command beginning with string • To view your history command by command, use the up and down cursor keys • View your history at any time by typing history • History is a very useful feature, if used well ◦ Incrementally searchable using CTRL-R www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 37 Basic Shell 2.4 Plumbing • Processes typically start with three files open: Name Standard input Standard output Standard error Descriptor 0 1 2 • Later we see how to refer to their file descriptors • These are normally connected to the keyboard and your command-line terminal c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.5 Plumbing (continued) 38 2.5 Plumbing (continued) • Data can be redirected by the shell ◦ Transparently to the process concerned ◦ Any or all streams can be redirected ◦ You can redirect to/from a file or to/from another process • Redirection to a process is known as ‘piping’ www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 39 Basic Shell 2.6 Output Redirection • Redirection of output is done using ’>’ • For example: $ command > output Creates the file output (or overwrites it if it already exists) and places the standard output from command into it • We can append to a file rather than overwriting it by using >> • > and >> are actually shorthands for 1> and 1>> • Error output can be redirected using 2> or 2>> $ command 2> error.out 1> command 2> error.out 2>> c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.7 Input Redirection 40 2.7 Input Redirection • < redirects standard input from a file, e.g. $ command < input • command will now take the contents of the file input as its input • This could also be written as 0< • Consistent with > and 1> input (file) 0< d st in command stdo ut www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 41 Basic Shell 2.8 Combining Redirection • Redirect more than one descriptor by giving more than one redirection, e.g. $ command 1> output 2> error • Group redirections using the >& operator, e.g. $ command 1> output 2>&1 ◦ Output to the file called output ( > output ) ◦ Send errors to the same place as the standard output ( 2>&1 ) • The order of these is very important • The redirections are evaluated left-to-right, e.g. the following differs from the previous example $ command 2>&1 > output ◦ It sends error to the normal output and normal output to the file called output c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.9 Pipelines 42 2.9 Pipelines • You can output to another process with ‘|’ ◦ Known as the pipe symbol • A pipe connects the output of one process to the input of another • The data waiting to be transferred is buffered • The processes run concurrently • Linux ensures that the processes keep in step • For example: $ sort document | uniq | mail lee document a b d d c document sort a b c d d uniq document mail a b c d Lee’s mailbox www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 43 Basic Shell 2.10 Background Processes • Most commands run to completion before you get your shell prompt back • A ‘background’ process continues while you get your prompt back immediately • To launch a process in the background place & at the end of the line, e.g. $ sort /var/log/maillog > output & • Unless you use redirection (plumbing), output and error continue to appear on your terminal • Input is disconnected, so typing goes to the shell, not to the background process • If a process needs user input, and can’t take it from a file, it is ’stopped’ ◦ It won’t resume until brought to the foreground to receive input • You should normally start background processes with their output and error redirected to a file, e.g. $ sort big_file > output 2> error_output & c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.11 Background Processes (continued) 44 2.11 Background Processes (continued) • Running processes can be put in the background ◦ Suspend the process by typing ^Z in the terminal that the process is running in ◦ Put the process in the background using bg • Bring a process back to the foreground using fg • fg and bg operate on the most recent process by default ◦ Change to a process, by job number or name • jobs displays current shell processes: $ jobs [1]+ Stopped (tty output) $ fg %1 top www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 45 Basic Shell 2.12 Background Processes and nohup • Sometimes it is necessary to start a process and leave it running when you log out • If your shell is killed, any background processes will also be lost • nohup gets round this by detaching the process from the terminal • Always redirect output and error with nohup, e.g. $ nohup sort bigfile > out 2>&1 err.out • If you don’t redirect them then they will end up in ./nohup.out and ./nohup.err c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.13 Command Grouping and Sub-shells 46 2.13 Command Grouping and Sub-shells • bash can execute multiple commands on a line • Sequential commands are separated by ‘;’ $ sort data ; mail lee < sorted_data • It’s possible to launch a sub-shell to execute a command or group of commands ◦ Put commands in parentheses, e.g. $ (command1 ; command2) • Can also put a subshell in background, e.g. $ (command1 ; command2) & www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 47 Basic Shell 2.14 Process Management • ps (process status) prints info about a users’ processes: PID 22074 22075 22081 22590 TTY p0 p0 p0 p5 STAT S S S R TIME 0:02 2:13 0:00 0:00 COMMAND Eterm -t trans emacs -bg black asclock ps • jobs only prints info about processes belonging to the current shell • wait postpones shell until process is finished ◦ Usually given a process id as an argument ◦ If no argument is given it waits until all the shell’s processes have terminated • kill is used to send signals to processes ◦ Can terminate background processes • Some processes use signals to trigger tasks, e.g. log rotation, re-reading config files, etc c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.15 Signals 48 2.15 Signals • kill can be given a signal name or number • There are a variety of signals: SIGHUP 1 Hangup detected on controlling terminal or death of controlling process Interrupt from keyboard Quit from keyboard Kill signal Termination signal User-defined signal 1 SIGINT SIGQUIT SIGKILL SIGTERM SIGUSR1 SIGUSR2 2 3 9 15 30 10 16 31 12 17 User-defined signal 2 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 49 Basic Shell 2.16 Signals (continued) • Unless specified, kill sends a SIGTERM which causes most processes to terminate • If a process is unresponsive, it can be forcibly killed by sending it SIGKILL $ kill -9 1512 1512: Terminated or $ kill -KILL 1512 1512: Terminated • Can only signal your own processes ◦ Superuser can signal all c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.17 Background Processes: top 50 2.17 Background Processes: top • top displays the processes running on a machine • Results can be sorted in various ways • Options: ◦ See man top for full details, including command-line options ◦ Inside top use h for help on interactive options • Typical output: PID 22594 1 2 3 486 869 838 48 230 239 250 261 272 283 295 306 317 USER PRI NI SIZE RSS SHARE STAT LIB %CPU user 10 0 736 736 556 R 0 8.2 root 0 0 144 96 76 S 0 0.0 root 0 0 0 0 0 SW 0 0.0 root -12 -12 0 0 0 SW< 0 0.0 root 5 5 3160 2356 804 S N 0 0.0 root 0 0 68 12 12 S 0 0.0 www 0 0 11468 6280 488 S 0 0.0 root 0 0 100 80 48 S 0 0.0 root 0 0 384 372 272 S 0 0.0 root 0 0 164 120 72 S 0 0.0 daemon 0 0 164 132 88 S 0 0.0 root 0 0 192 160 112 S 0 0.0 bin 0 0 244 224 168 S 0 0.0 root 0 0 572 296 248 S 0 0.0 root 1 0 136 88 60 S 0 0.0 root 0 0 516 488 224 S 0 0.0 root 0 0 124 56 48 S 0 0.0 %MEM 0.5 0.0 0.0 0.0 1.8 0.0 4.9 0.0 0.2 0.0 0.1 0.1 0.1 0.2 0.0 0.3 0.0 TIME 0:00 0:03 0:19 2:42 0:03 0:00 4:14 0:01 0:12 0:00 0:00 0:01 0:00 0:17 0:00 0:06 0:00 COMMAND top init kflushd kswapd mysqld mingetty squid kerneld syslogd klogd atd crond portmap snmpd inetd named lpd • N.B. top is not available on all unices www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 51 Basic Shell 2.18 Filename Generation • Some characters are ‘special’ to the shell Chars * ? [...] Meaning Matches any string, including the null string Matches any single character Matches any one of the enclosed characters. A pair of characters separated by a minus sign denotes a range. Any character lexically between those two characters, inclusive, is matched. If the first character following the ‘[’ is a ‘!’ or a ‘^’ then any character not enclosed is matched. A ‘-’ or ‘]’ may be matched by including it as the first or last character in the set. Table 2.1: Special characters under bash • Special characters can be used to match filenames, e.g. to show files beginning with f $ echo f* • To show files starting with ‘f’, followed by a vowel: $ echo f[aeiou]* c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.19 Quoting Mechanisms 52 2.19 Quoting Mechanisms • Sometimes it’s necessary to ignore a character’s special meaning • Use a backslash (\) to quote a special character, e.g. to list a file called f* $ ls f • To quote a longer string, enclose it in quotes: ’ " disable all interpretation disable filename generation and blank space interpretation www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 53 Basic Shell 2.20 Shell built-in commands • Some commands must be built in to the shell, because they can’t be executed independently ◦ cd, if executed independently would change its own directory, not that of your shell ◦ umask, would change its umask, not the shell’s ◦ logout ◦ history • Other commands are built in for speed e.g. ◦ pwd ◦ echo c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.21 Basic Shell Exercises 54 2.21 Basic Shell Exercises What should I focus on? You should: • clearly understand what redirection is, and what a pipe is • understand that it is the shell that expands the filename generation symbols (*, ? and [...]), and not echo, ls,. . . • understand the differences between these three ways of quoting: ’...’, "...", and quoting a single character with ‘\’ • understand how to put a process into the background, bring it to the foreground, and how to interrupt it (not with Control-z!) • understand the difference between using Control-c and Control-z • understand that subshells have their own working directories. The exercises 1. Redirection (a) Try typing the following commands exactly as they appear here. Note that the cat command copies input files (or standard input) to standard output. If standard input comes from the  ¨  ¨ Control-d © means “end of file” (like  Control-z © Windows). Refer in keyboard, the keystroke  to sections 2.6 on page 39 to § 2.9 on page 42 first. Also refer to section 2.13 on page 46 before doing the last exercise. $ # $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ cat § ¤ Then type some words, followed by ¦ Control-d ¥ cat > newfile 2> newfile.error car > newfile 2> newfile.error car Space Shuttle echo echo foo ech ech foo cat > newfile 2>&1 car > newfile 2>&1 cat < newfile echo foo | cat > newfile 2>&1 ech foo | cat > newfile 2>&1 echo foo | car > newfile 2>&1 (ech foo | cat) > newfile 2>&1 After each step, examine the contents of newfile and newfile.error by typing: $ cat newfile $ cat newfile.error www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 55 Basic Shell Make sure you understand what happens in each case, ask the tutor if you are not sure. 2. Filename expansion and Quoting (a) Do the following in the /bin directory; refer to section 2.18 on page 51 first. i. ii. iii. iv. List all filenames with exactly three characters. List all filenames with exactly three characters in which the second character is a vowel. List all filenames with a, b, c, or d as the last character. Construct a command to print the number of filenames consisting of exactly three characters. (You may combine commands together in a pipeline. You may find the wc utility useful here; check man wc for more information.) v. Construct a command to print the total number of files with exactly two, three or four characters in their name. (Again, you may find the wc utility useful.) (b) Compare the effect of the following commands. Refer to section 2.19 on page 52 first. Then search for “quoting” in the bash man page. $ $ $ $ $ $ $ $ $ echo echo echo echo echo echo echo echo echo $HOME "$HOME" ’$HOME’ * "*" ’*’ $HOME/bs* "$HOME/bs*" ’$HOME/bs*’ (c) Change back to your home directory and try to create a file with the name *. Was this a sensible thing to do? How would you delete it? (Be very careful!) (d) Create a file called --file. Try to remove this. Use the rm man page to help you. 3. Background processes and nohup Refer to section § 2.10 on page 43 to § 2.12 on page 45 first. (a) Start the command sort /dev/random in the background in your current shell  ¨ (b) Bring it back to the foreground and terminate it by typing  Control-c © (c) Start it again, and once more so that you have two copies running in the background (d) Bring them to the foreground and terminate them in the order you started them (e) Start the same command in the background, and terminate it using kill 4. Grouped commands Compare the following command sequences, and make sure you understand the differences. Refer to section § 2.13 on page 46 first. (a) $ cd /tmp $ cd /usr; ls $ pwd (b) $ cd /tmp $ (cd /usr; ls) $ pwd (c) $ sleep 5 & sleep 5 (d) $ (sleep 5; sleep 5) & Check you can use your history to get at and repeat any of the commands you have typed. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 2.22 Basic Shell Solutions 56 2.22 Basic Shell Solutions 2. Filename expansion and Quoting (a) These are solutions which will do the job, there may be other ways of achieving the same thing i. ii. iii. iv. v. ls ??? ls ?[aeiou]? ls *[abcd] ls ??? | wc -l (ls ??; ls ???; ls ????) | wc -l or more simply, ls ?? ??? ???? | wc -l (b) When not quoted $HOME gives the name of your home directory. This is variable substitution. We can see that this substition still happens inside " quotes, but not inside ’ quotes. echo * expands to all the filenames in the current directory. This is filename generation and doesn’t happen in either type of quote. (c) It is not a wise choice to name anything with a filename containing special characters. You can delete the file safely by quoting the * using any one of these ways: • rm "*" • rm ’*’ • rm \* (d) You can delete the file by using rm -- --file or with rm ./--file 3. Background processes and nohup (a) $ sort /dev/random & (b) $ fg ^C (c) $ sort /dev/random & [1] 26409 $ sort /dev/random & [2] 26410 (d) $ jobs [1]- Running [2]+ Running $ fg %1 sort /dev/random ^C $ fg %2 sort /dev/random ^C (e) $ sort /dev/random & [1] 26462 $ kill 26462 # Or: $ kill %1 4. The semicolon ‘;’ is a separator, like the newline, to the shell, so in: www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 sort /dev/random & sort /dev/random & 57 Basic Shell $ cd /usr; ls first the cd /usr command is executed, then the ls command. In part 4b, these commands are executed in their own subshell, so the cd does not change the current directory of the parent process (the process from which the subshell was fork()ed), so the pwd command shows that we are still in the /tmp directgory from which we started. In part 4d, the whole subshell is put into the background and the subshell terminates after 10 seconds, whereas in part 4c, only the first sleep 5 command is put into the background, so both sleep 5 commands run in parallel and both terminate after 5 seconds. You can see this for yourself: $ sleep 5 && echo ’done 1’ & sleep 5 && echo ’done 2’ [3] 30269 done 1 done 2 [3]+ Done sleep 5 && echo done 1 You can also measure the time it takes for any command to complete with the time command: $ time sleep 5 && echo ’done 1’ & time sleep 5 && echo ’done 2’ [4] 30258 real 0m5.008s user 0m0.000s [3] Done sys 0m0.000s done 1 real 0m5.009s user 0m0.000s sys 0m0.000s done 2 [4]+ Done time sleep 5 && echo ’done 2’ time sleep 5 && echo ’done 1’ And just for the fun of it, you can try: $ time { sleep 5; time sleep 5; } real user sys 0m5.010s 0m0.000s 0m0.000s real 0m10.013s user 0m0.000s sys 0m0.000s $ ( time { sleep 5; time sleep 5; } ) & [3] 30455 $ real 0m5.010s user 0m0.000s sys 0m0.000s real user sys [3]+ 0m10.013s 0m0.000s 0m0.000s Done ( time { sleep 5; time sleep 5; } ) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk Module 3 Basic Tools Objectives At the end of this section, you will be able to: • Use the most frequently used Linux tools to: ◦ Find files ◦ Get information about commands ◦ View file contents ◦ Get information about files ◦ Operate on file contents ◦ Do simple text manipulation ◦ Schedule jobs • Combine tools to solve problems • Understand and use the Linux printing subsystem 59 3.1 Introduction 60 3.1 Introduction The basic Linux command-line utilities dealt with here, are: Finding files: Getting info about commands: Viewing file contents: find locate man cat less head/tail Getting information about a file: ls file wc diff cmp Operating on file contents: grep sort uniq split/csplit cut paste tar gzip/bzip2 tr expr at crontab Simple text manipulation: Scheduling jobs: Table 3.1: Basic Linux utilities www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 61 Basic Tools 3.2 Using Tools • Typical Linux systems contain over 1000 command-line tools • Tools are combined (via pipes and redirection) to solve specific problems • Most tools have a standard syntax: $ command [options] [files ... • Some arguments must be quoted • Standard input often read if no filename given • Most tools can take several filename arguments • Desktop/windowing environments may provide graphical wrappers to some tools • Serious Linux administrators and users know the key command-lines well • The terms ‘command’ and ‘tool’ are used interchangeably here ] c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.3 The On-Line Manual (man) 62 3.3 The On-Line Manual (man) • Most commands have an associated man page • Accessed by typing: $ man command[s] • Brings up a page of information usually detailing: ◦ command name, section number, description ◦ syntax ◦ options ◦ version information ◦ location of configuration files ◦ other related commands ◦ examples of usage ◦ known bugs (if any . . . ) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 63 Basic Tools 3.4 Finding Files the Long Way (find) • find searches the filesystem in real time; ◦ Makes disks work hard • Can find files by name, type, size, dates, e.g ◦ To find all files ending with .jpg under the current directory: find . -name "*.jpg" ◦ To find all filenames ending in .jpg and modified in the last 8 days below /etc find /etc -name "*.jpg" -mtime -8 • Tests can be combined with -o and negated with !, for example: ◦ To find all filenames not ending in .jpg or files which were modified in the last 8 days under /etc find /etc \! -name "*.jpg" -o -mtime -8 • Can execute commands on the files it finds. The name of the file found is placed in {} ∗ find . -name "*.gif" -exec ls -l {} \; ∗ This is not a resource friendly way of doing things. xargs may be better www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 3.5 Find examples 64 3.5 Find examples • First, an example which is wrong: $ find /usr/bin/c* # WRONG! This is wrong because the shell expands the c* to all the files before find even begins. • Here is the right way to find files beginning with “c” under the /usr/bin directory: $ find /usr/bin -name c\* Note that we need to quote the “*”, because we want find to have the star, we don’t want the shell to turn it into a list of files starting with “c” in the current directory. • Find man pages that are bigger than 10 k: $ find /usr/share/man -size +10k • Find man pages that are bigger than 10 k but smaller than 15 k: $ find /usr/share/man -size +10k -size -15k • Find man pages that are smaller than 10 k OR bigger than 15 k: $ find /usr/share/man -size -10k -o -size +15k www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 65 Basic Tools We use the logical OR operator “-o” • Find man pages that are smaller than 10 k but bigger than 15 k and which are ordinary files (not directories or symbolic links): $ find /usr/share/man -type f \( -size -10k -o -size +15k \) We need to use parentheses to group the OR operation first, otherwise it will test for files that are smaller than 10 k and which are normal files, OR for any files that are bigger than 15 k. The parentheses must be quoted or the shell will try to start a shubshell. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.6 Locate Files (locate) 66 3.6 Locate Files (locate) • locate searches a periodically-updated database of the filesystem(s) ◦ Not available on all systems ◦ Very fast, but DB needs regular updating, e.g. $ updatedb ◦ Usually updated nightly automatically • Won’t be there on a fresh install • Won’t show files created since last database update • Given the command ‘locate string’, locate will show all files containing string in their full pathname, e.g. $ locate logrotate /usr/man/man8/logrotate.8 /usr/sbin/logrotate /etc/logrotate.d /etc/logrotate.d/cron /etc/logrotate.d/syslog /etc/logrotate.d/linuxconf /etc/logrotate.d/ftpd /etc/logrotate.d/samba /etc/cron.daily/logrotate /etc/logrotate.conf www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 67 Basic Tools 3.7 View and Concatenate Files (cat) • Displays and/or joins (con cat enates) files • Sends the content of named file(s) to standard output • If no filename is given, it reads from standard input and writes to standard output • Given more than one filename, it displays each file’s contents sequentially, i.e, joins them • Example: $ cat file1 file2 file3 > all_files c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.8 View Large Files & Output (less) 68 3.8 View Large Files & Output (less) • less displays the contents of file(s) in a controlled way on stdout ◦ Usually, one page at a time ◦ Like UNIX/DOS command more, on steroids • You can search for patterns in the file • It allows you to move quickly to any point (backwards or forwards) • Similar usage to more, vi, and lynx: Action Top of page Bottom of page Forward one screen Backward one screen Up one line Down one line pattern Search forward pattern Search backward Repeat pattern Search forward Repeat pattern Search backward Move to nth line !command —Xcommand v Keystokes g < ESC-< G > ESC-> f ^F ^V SPACE b ^B ESC-v y ^Y k ^K ^P e ^E j ^N RETURN /pattern ?pattern n N ng Execute the shell command with $SHELL Pipe file between current pos & mark X to shell command Edit the current file with $VISUAL or $EDITOR Table 3.2: Commands within less www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 69 Basic Tools 3.9 Viewing Parts of Files (head and tail) • head displays the first few lines of a file • tail displays the last few lines of a file • You can specify how many lines are displayed ◦ To display only the first 4 lines: $ head -4 filename • tail -f often used to monitor growing files, e.g., $ sudo tail -f /var/log/messages c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.10 Listing File Information (ls) 70 3.10 Listing File Information (ls) • Without any options, ls lists files in the current directory • By default all files starting with . (dot) aren’t shown • The most common options to ls include: Flag Option -l Long (detailed) listing of file info, including: size, ownership, permissions and type -a Show all files, including hidden ones -F Highlight directories and executables with / and @ respectively -R Recursively list subdirectories -t Sort list by last modification time -u Sort list by last access time (with -t) -X Sort list by file eXtension -r Reverse order of listing -d Show directory information not directory contents Table 3.3: Common options to ls • For example: $ ls -lrt show files in reverse order based on their modification time www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 71 Basic Tools 3.11 File Classification (file) • file displays the type of data contained in named file(s) • It works by: 1. opening each named file 2. reading the beginning of the file 3. comparing what it finds there with the patterns in the “magic” file /usr/share/magic 4. reports the matching file type found in the magic file. • Results not always correct • Classifications include: executable, archive, C program, ASCII text, JPEG image . . . • You can use it like this: $ file /usr/bin/* c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.12 Count Words, Lines, Characters (wc) 72 3.12 Count Words, Lines, Characters (wc) • wc displays the number of lines, words,∗ and characters in a file Flag -l -w -c Option Only displays the number of lines Only displays the number of ‘words’ Only displays the number of characters Table 3.4: Options to the wc command A ‘word’, in this context, is a character string surrounded by SPACEs, TABs, NEWLINEs, or a combination of them. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 73 Basic Tools 3.13 Differences Between Files (diff) • diff displays the difference between two text files, line-by-line • Output from diff can be confusing • For example, given the files text1 and text2: text1: This is a temprary test to check the diff utility text2: This is a temporary test to check the diff utility. 1. A simple line-by-line comparison: $ diff text1 text2 1c1 < This is a temprary test --> This is a temporary test 3c3 < utility --> utility. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.13 Differences Between Files (diff) 74 2. Using the context output format (-c): $ diff -c text1 text2 *** text1 Mon Apr 19 14:46:25 1999 --- text2 Mon Apr 19 14:46:05 1999 *************** *** 1,3 **** ! This is a temprary test to check the diff ! utility --- 1,3 ---! This is a temporary test to check the diff ! utility. 3. Using the unified output format (-u): (Most common) $ diff -u text1 text2 --- text1 Mon Apr 19 14:46:25 1999 +++ text2 Mon Apr 19 14:46:05 1999 @@ -1,3 +1,3 @@ -This is a temprary test +This is a temporary test to check the diff -utility +utility. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 75 Basic Tools 3.14 Compare Binary Files (cmp) • Displays differences between 2 binary files • Locates the byte and line number of the first difference • Can show all differences if required, e.g. cmp -l file1 file2 • cmp -s suppresses output and returns exit status ◦ 0 if the files are identical ◦ 1 if the files differ ◦ 2 if an error has occurred • Often used in shell scripts c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.15 Regular Expression Searches (grep) 76 3.15 Regular Expression Searches (grep) • Search for regular expressions in file(s) i.e “ g lobally find r egular e xpressions and p rint” • Usage: grep [options] search-pattern files • Reads standard input if no filenames are given • Matching lines are printed to standard output • Popular options: Flag -i -l -v -w -E -F Option Ignore case List only filenames containing the expression Reverse sense of test, i.e. find non-matching lines Word search, i.e. match whole word Extended regular expression search (more complex patterns), egrep similar Fixed string pattern search, same as fgrep Table 3.5: Popular grep options www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 77 Basic Tools 3.16 grep examples • Search for a user in the password file $ grep lee /etc/passwd lee:x:500:500:Lee Willis:/home/lee:/bin/bash • Search for events in a log∗ # grep Mar 8 Mar 13 Mar 13 Mar 20 connect /var/log/secure 14:42:53 rafters in.telnetd[579]: connect from 192.168.0.129 10:27:40 rafters in.telnetd[724]: connect from 192.168.0.139 16:54:24 rafters in.telnetd[2135]: connect from 192.168.0.139 10:54:38 rafters in.telnetd[816]: connect from 127.0.0.1 • Search for a function in source code $ grep "int main" *.cpp checkrefint.cpp:int mainbit(); checkrefint.cpp:int mainbit(){ do_maint.cpp:int maint(Form &Myform){ expire.cpp:int main(void){ maint_login.cpp:int maint_login(Form &) { ∗ You need to be logged in as root to see inside /var/log/secure. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 3.17 Sort and Merge Files (sort) 78 3.17 Sort and Merge Files (sort) • sort and/or merge files • Acts as a filter without file arguments • Sorts entire lines lexically, by default • Alternative sort orders: Flag Option -n Numerical order -r Reverse order Table 3.6: Alternative sort orders • Other popular options: Flag -b -f -i -m -tx -u -k POS1[,POS2] Option Blanks (TAB, SPACE) ignored Fold lowercase to upper before sorting Ignore non-printable characters Merge files, without checking if sorted Set field delimiter in file as x Unique, outputs repeat lines once only Specify a field in each line as a sorting key, starting at POS1 and ending at POS2 (or NEWLINE). Fields and character positions are numbered starting at 1 Table 3.7: Popular sort options www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 79 Basic Tools 3.18 sort Examples Consider /etc/passwd which typically contains lines in the following format: username:password:UID:GID:Realname:Homedirectory:shell • To sort by username: $ sort -t: -f -k1,1 /etc/passwd • To sort numerically by user ID: $ sort -t: -n -k3,3 /etc/passwd • To sort by real name within group ID: $ sort -t: -k4,4n -k5,5f /etc/passwd c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.19 Display Unique Lines (uniq) 80 3.19 Display Unique Lines (uniq) • Removes all but one of successively repeated lines • Acts on standard input, often piped from sort • Most popular options: Flag Option -c Put a count of how many lines were duplicate in front of each line -d Duplicated lines only are displayed -u Unique lines only are displayed -n Ignore the first n fields +n Ignore the first n characters -w Specify the number of chars to compare Table 3.8: Popular uniq options • Example: document b a b y z z document sort −r z z y b b a document uniq z y b a www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 81 Basic Tools 3.20 Split Files (split) • Split a file into pieces • Outputs sections to new files or standard output • Creates files named prefixaa, prefixab, prefixac . . . • Main split options: Flag -ln -bn -Cn Option Put n lines of the input file into each output file Put n bytes of the input file into each output file Put as many complete lines of the input file as is possible into the output file, up to n bytes Table 3.9: Main split options c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.21 Splitting Files by Context (csplit) 82 3.21 Splitting Files by Context (csplit) • Splits file into sections determined by context (patterns or regular extressions) • Syntax: csplit [-f prefix] [-b suffix] [-n digits] filename pattern... • Main csplit arguments: Argument Instruction /regexp/[offset] Split the file at occurrence of regexp. The line after the optional offset (‘+’ or ‘-’ followed by a number) begins next bit of input {repeat-count} Repeat the previous pattern split n times. Substitute an asterisk for n to repeat until the input is exausted -fstring Use string as prefix of output filename -bstring Use string as suffix of output filename -nn Use output filenames n digits long Table 3.10: Main csplit arguments www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 83 Basic Tools 3.22 Dividing files into columns: cut • Often need to separate one or more columns from input. ◦ cut provides a simple way; ◦ awk provides another way (more flexible; see later). ◦ Use paste to join columns together • Syntax: cut options [ files ] • Cuts out the selected columns or fields from one or more files . • Can specify a range of values with a hyphen, and use a comma to separate values, e.g., 1-10,15,20, or 50• Options: -b, --bytes list Specify list of positions; only bytes in those positions will be printed. -c, --characters list Cut the column positions given in list -d, --delimiter c Use -f to specify field delimiter as character c (default is tab); special characters must be quoted. -f, --fields list Cut the fields given in list • Examples: ◦ Extract usernames and real names from /etc/passwd: cut -d: -f1,5 /etc/passwd ◦ Find out who is logged in, but only login names: c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.22 Dividing files into columns: cut 84 who | cut -d" " -f1 ◦ Cut characters from the fourth column of file and paste them back as the first column in the same file: cut -c4 file | paste - file www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 85 Basic Tools 3.23 Compression Utilities: bzip2 and gzip • Linux has many compression utilities; bzip2 and gzip dominate thanks to integration with tar, and excellent compression • Takes files or standard input and compress them to file(s) or standard output • Uses lossless compression, so safe on any file • Key bzip2 and gzip options: Flag -r -d -[1-9] Option Recursive compression of subdirectories Decompress (same as gunzip) Fast or best compression, where 1 is fastest and 9 is most intense compression Table 3.11: Key bzip2 and gzip options • bzip2 has significantly better compression ratios and is now widely used for distributing software. • A set of files put into an archive with tar and bzip2 gives much better compression than can be achieved in zip files or rar archives. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.24 Store and Retrieve Archives (tar) 86 3.24 Store and Retrieve Archives (tar) • Originally designed to make tape archives • Takes a group of files and creates one big file containing their contents and details • Widely used for: ◦ Compression with gzip or bzip2 ∗ ◦ Maintaining Linux file details (permissions, dates, ownership etc) on other filesystems ◦ Bundling file trees for distribution (called a tarball) • Key tar options: Flag -t -x -c -r -v -j -z -f -p Option List files in an archive Extract the contents from an archive Create a new archive Append files to an existing archive Verbose: list file names, tell more of what’s happening Create/Open bzip2 compressed tar file(s) Create/Open gzip compressed tar file(s) Filename of the file or device to hold the archive preserve permissions when extracting files Table 3.12: Key tar options • To create a gzip compressed archive of /etc: $ tar -cvzf /home/username/filename.tar.gz /etc The gzip or bzip2 compression options are not available on all UNIX versions of tar. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 87 Basic Tools • To extract the files into the current directory: $ tar -xvzf filename.tar.bz2 • To create a bzip2 compressed archive of /etc: $ tar -cvjf /home/username/filename.tar.gz /etc • To extract the files into the current directory: $ tar -xvjf filename.tar.bz2 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.25 Translating Characters (tr) 88 3.25 Translating Characters (tr) • Translate characters in standard input into different characters in output • Syntax: tr [options] [string1 [string2]] • Characters in string1 are replaced by the corresponding character in string2 $ tr ‘abc‘ ‘123‘ abcdad 123d1d • Character position in both strings matters • Both strings should be the same length ∗ If string1 is shorter than string2, the extra characters at the end of string2 are ignored. If string1 is longer, GNU tr follows BSD in padding string2 to the length of string1, by repeating the last character. With the -t option it follows AT&T by truncating string1 to the length of string2. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 89 Basic Tools 3.26 Examples of tr Usage • To replace all vowels in the input with spaces $ tr ’aeiou’ ’ ’ • Using character ranges to translate all lower case letters into their upper case equivalents $ tr ’a-z’ ’A-Z’ • Using the asterisk * to replace all 10 digits with the same letter ‘n’. In other words, [n*10] is the same as ’nnnnnnnnnn’ $ tr ’0-9’ ’[n*10]’ • Use the -c option (complement) to replace all characters in string1 which don’t belong in a range $ tr -cs ’a-zA-Z0-9’ ’[\n*]’ N.B. This puts every word on a line by itself, by converting all non-alphanumeric characters to newlines, then ‘squeezing’ repeated newlines (with -s) into a single newline. The pattern [\n*] in the second set means repeat the newline \n as many times as there are characters in the first set. See man tr. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.27 Execute programs at specified times (at) 90 3.27 Execute programs at specified times (at) • at executes a shell script at a specified time • Syntax: $ at [options] time [date] +increment • stdin is scanned for commands to execute, e.g. at 2300 at> fetchmail +D • Some more examples of how to specify time and date: at at at at at at at at at at at 11pm 1am 5am tomorrow 08:00 Sep 12 now now + 4 hours now + 1 day now + 2 months now + 3 years 5:30pm Thursday teatime • Commands are executed in the current environment at the given time • stdout and stderr are sent as mail www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 91 Basic Tools 3.28 Options and commands related to at • at belongs to a family of utilities for managing time-specified commands Command atq atrm batch Purpose Display list of queued at commands Remove queued at commands Schedule jobs at low CPU loading • All of these can be run as at options: Option -l -d -b -ffile -m Purpose Display list of queued at commands Remove queued at commands Schedule jobs at low CPU loading Specify script file in command-line Send mail after running at, whatever the stdout or stderr • The use of at is controlled by /etc/at.allow and /etc/at.deny ∗ ∗ UNIX NOTE: On System V these live in /usr/lib/cron/ www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 3.29 Running commands regularly (crontab) 92 3.29 Running commands regularly (crontab) • crontab lets you submit job lists at regular times using the crond daemon • 2 syntax formats: $ crontab filename $ crontab [-u username] [options] • Options, etc: Command crontab myfile crontab -r crontab -l crontab -e Purpose Install contents of myfile (stdin if no file specified) in appropriate directory Remove the crontab for the current user List (on stdout) current user’s crontab. (might be useful for editing a cron table) Run a text editor on your crontab file Table 3.13: crontab usage • Examples of Crontab Entries 01 * * * * root run-parts /etc/cron.hourly 02 4 * * * root run-parts /etc/cron.daily 22 4 * * 0 root run-parts /etc/cron.weekly 42 4 1 * * root run-parts /etc/cron.monthly # LW poll client1 for mail 0 10,12,16,18 * * 1-5 root /www/bin/client1_poll LW 16/09/1998 - Hylafax cron stuff ... 0 0 * * * root /usr/local/sbin/faxqclean 10 0 * * * fax /usr/local/sbin/faxcron -info 7 [0-59/10] 9-18 * * 1-5 root /usr/local/bin/domail www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 93 Basic Tools 3.30 Evaluate expressions (expr) • expr is used to evaluate expressions, like bash arithmetic expressions, e.g., echo $((1 + 2)) • Takes arguments and operators on the command line • Prints the result • Also returns true or false depending on the result; can be tested with shell—see example with the while loop below. • Watch out for special meaning to shell of characters like * and < or > $ expr 1 + 2 3 $ expr 6 \* 7 #must escape the ’*’ 42 $ expr 5 + 10 / 2 10 $ expr \( 5 + 10 \) / 2 7 $ i=10 $ i=‘expr $i + 1‘ $ echo $i 11 expr prints 1 for true comparisons, 0 for false. • Has some string manipulation facilities too: $ expr index abcdefg d 4 $ expr substr abcdefghijk 3 3 cde c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.30 Evaluate expressions (expr) 94 • Sometimes used in shell scripts for looping; expr returns false if the expression is null or 0, and true otherwise: i=0 while expr $i \< 20 >/dev/null do echo $i i=$(expr $i + 1) done • See info expr for details. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 95 Basic Tools 3.31 Linux Printing • Two main printing sytems: ◦ Common Unix Printing System (CUPS) http://www.cups.org/ ◦ LPRng http://www.lprng.com/ • Both currently provided on Red Hat Linux 8.0 and 9 • Only CUPS provides on Fedora Core 1+. ◦ Can select either system with redhat-switch-printer command • Expect CUPS to be more prevalent in future c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.32 LPRng and CUPS 96 3.32 LPRng and CUPS • Completely network-oriented • Any printer can be made available to any client (machine and application) • All print jobs are sent to a queue • Queues can be viewed, edited, maintained from anywhere ◦ Subject to permission • Formatted files can be sent straight to queues • Both CUPS and LPRng both share common printing commands www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 97 Basic Tools 3.33 Main Printing Tools • lpr sends job to the queue for a named printer • lpq returns info about jobs in a queue • lprm removes unwanted jobs from a queue • lpc enables system administrator to control the operation of the printing system ◦ see man lpc for details • Desktop environments may offer “drag ‘n’ drop”, visual facilities, etc c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.34 Using lpr 98 3.34 Using lpr • Syntax: lpr [options] file ... • Main Options: Flag -P -n -m Options Name of the printer to send the job to Print n copies of the document Send mail on completion Table 3.14: Main lpr options • Example: $ lpr -Pdjrmt -2 filetypes.txt www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 99 Basic Tools 3.35 Using lpq • Syntax: lpq [options] • Options: Flag Options -P Name of the printer/queue to interrogate -l Get info on each file within a job Table 3.15: lpq options • Example: $ lpq -Pdjrmt -l c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.36 Using lprm 100 3.36 Using lprm • Syntax: lprm [options] • Options: Flag -P user n Options Remove jobs from named printer/queue Remove all jobs belonging to yourself Remove all jobs belonging to user Remove job number n Table 3.16: lprm options • Example: $ lprm -Pdjrmt -davef www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 101 Basic Tools 3.37 Printing Information • This is a bare introduction • See chapter 8 of the Red Hat Getting Started Guide (for Red hat 9), and • chapter 27 of the Red Hat Customization Guide • All these are available on the documentation CDROM, or can be downloaded from http://www.redhat.com/docs/manuals/linux/. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.38 Basic Tools Exercises 102 3.38 Basic Tools Exercises In these exercises, you will combine simple tools together using pipes. Do this one step at a time, and check the results as you go. 1. Find and Locate Files Use first find then locate to do each of the following. First read § 3.4 on page 63 to § 3.6 on page 66, especially § 3.5 on page 64. Also, please read § 3.25 on page 88 and § 3.26 on page 89 before doing part 1c. (a) Display all the filenames under /usr/sbin. (b) Display all the filenames under /usr/sbin begining with a lowercase ‘c’. (c) Repeat the previous question, but translate the output to uppercase. (d) Display all the files under /usr/sbin which are over 5k in size in uppercase. 2. Display Parts of Files Note that the file mime.types lists the standard names used for different file types in both http and email. Before sending any data, a web server sends a header which includes the mime type. That is how the web browser knows what to do with the data (i.e., how to display it or play it). Similarly, email containing attachments puts a mime type header before each attachment. That is how the email client knows what to do with the attachment (such as to automatically execute it and infect the machine with a worm). (a) Display the first 10 lines of the file /etc/mime.types (b) Display the last 10 lines of /etc/mime.types (c) Display the first 25 lines of /etc/mime.types (d) Display /etc/mime.types one screen at a time (e) While viewing /etc/mime.types page-by-page, search for ‘html’ 3. Classify, Count and Compare Files (a) Find out what file types you have in the following directories and below. Please read § 3.11 on page 71. Please also see the example on the bottom of § 3.4 on page 63. i. /etc ii. /usr/bin (b) Repeat the previous question, but this time: i. Re-direct /etc listing to new file filetypes.txt ii. Append the listing for /usr/bin to filetypes.txt. Please refer to § 2.6 on page 39. (c) Build a tool (i.e. write a command) to find out how many files are in the /usr/bin directory. Hint: consider using wc. (d) Create two new files from listings of 2 user’s home directories, then find the differences between them. To do this, you could: • Find a neighbour you can work with, and open a terminal window on their computer • In that terminal window, type: $ su - 012345678 (assuming your user ID is 012345678). Of course, you need to enter your password. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 103 Basic Tools • Make a list of files from your home directory into /tmp, making sure that the file names do not have /home/012345678 in front of them, or otherwise your files will all seem to have different names! • Compare your file list with that generated by your neighbour, who should also put their file list into /tmp. (e) Use diff and patch to make the two files created in the last question identical. (Check man patch) 4. grep Use the filetypes.txt file that you created before, and do the following. Please read § 3.15 on page 76 and § 3.16 on page 77 first. (a) List all the lines that contain ‘directory’ (b) List all the lines that don’t contain ‘directory’. (c) Find out how many files are directories, then find out how many aren’t. (d) Why does the following give an error message (try redirecting the output to /dev/null so you can see the error). $ grep ASCII text filetypes.txt (e) Find out how many English text files are listed in the filetypes.txt file. 5. Sorting (a) Sort the filetypes.txt file into reverse alphabetical order on the first field. Please see § 3.17 on page 78 and § 3.18 on page 79 first. You may notice that capital and lowercase letters are sorted independently, e.g. ‘A’ comes before ‘a’. (b) Repeat the first sorting exercise but ignoring case differences (c) Sort the filetypes.txt files into alphabetical order on the second field (the file type). c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.39 Basic Tools Solutions 104 3.39 Basic Tools Solutions 1. Find and Locate Files (a) Either $ find /usr/sbin or $ locate ’/usr/sbin/*’ Note that the quotes are very important. If you leave them out, the program will run much more slowly, and the output will be wrong. Why? (b) Either $ find /usr/sbin -name "c*" Why is find /usr/sbin/c* wrong? Why are the quotes important around the c*? or $ locate ’/usr/sbin/c*’ The quotes are really important for speed and correctness. Why? If you do $ locate /usr/bin/c\* | wc -w 23 Very fast because only lookup one pattern in the database. $ locate /usr/sbin/c* | wc -w 30 Very slow because need look up 23 patterns in the database. Many file names are shown twice; for example, if you do: % $ locate /usr/bin/cc /usr/bin/ccexample /usr/bin/cc /usr/bin/ccexample /usr/bin/ccexample We get three results, not two, since /usr/bin/cc matches /usr/bin/ccexample as well as itself. (c) Either $ find /usr/sbin -name c\* | tr ’a-z’ ’A-Z’ or $ locate ’/usr/sbin/c*’ | tr ’a-z’ ’A-Z’ (d) $ find /usr/sbin -size +5k | tr ’a-z’ ’A-Z’ You should check that the size is correct. One way to do that is: $ find /usr/sbin -size +5k -ls | sort -k5nr What are the arguments to sort for? Try looking at the output of find /usr/sbin -size +5k -ls first. 2. Display Parts of Files www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 105 Basic Tools (a) $ head /etc/mime.types (b) $ tail /etc/mime.types (c) $ head -n 25 /etc/mime.types (d) Use one of the following $ less /etc/mime.types or $ more /etc/mime.types (e) While in one of the above, type /html 3. Classify, Count and Compare Files (a) Use these commands: i. $ find /etc -exec file {} \; Note that the following will not execute file on files in subdirectories of /etc: $ file /etc/* ii. $ find /usr/bin -exec file {} \; (b) Use these commands i. $ find /etc -exec file {} \; > filetypes.txt ii. $ find /usr/bin -exec file {} \; >> filetypes.txt (c) The easiest solution is $ echo /usr/bin/* | wc -w But that will count any file with a space in its name as two files. This will count all files only once: $ ls -d /usr/bin | wc -l Note that when ls sends its output to a pipe, it always puts it in one column, so that there is one file per line. See man ls. (d) Something like this: $ cd $ find . > /tmp/test1.txt $ cp /tmp/test1.txt ∼ Now another student logs into another window: $ su - 012345678 Password: $ find . > /tmp/test2.txt $ cp /tmp/test1.txt /tmp/test2.txt ∼ $ diff -u test2.txt test1.txt > file-diff.patch $ less file-diff.patch $ exit And you can copy the second students file list into your own directory: $ cp /tmp/test2.txt ∼ $ diff -u test1.txt test2.txt > my-file-diff.patch $ less file-diff.patch Make sure you understand the format of diff -u, and given output from diff and the original file, can determine what the second file contains. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 3.39 Basic Tools Solutions 106 (e) This technique is very important in software development and software packaging: $ diff -u test1.txt test2.txt > test.patch $ patch < test.patch patching file test1.txt 4. grep (a) $ grep directory filetypes.txt (b) $ grep -v directory filetypes.txt (c) Use either $ grep -c directory filetypes.txt or $ grep directory filetypes.txt | wc -l then either $ grep -cv directory filetypes.txt or $ grep -v directory filetypes.txt | wc -l (d) Without escaping the space between ASCII and text the shell assumes the pattern has ended and takes text as a filename to look for; hence the error about a non-existing file. grep continues, however, showing lines that match ”ASCII” in the files it can find. What you probably wanted was: $ grep "ASCII text" filetypes.txt (e) $ grep -c "English text" filetypes.txt 5. Sorting (a) $ sort -r filetypes.txt (b) $ sort -fr filetypes.txt (c) $ sort -k2 filetypes.txt www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 Module 4 More Tools Objectives Having completed this module you should be able use the following tools appropriately: • top • ps • find • vmstat • free • ldd • uptime • xargs • cpio • tar • gzip 107 4.1 Introduction 108 4.1 Introduction Tools covered in this module have these functions: Command Function top ps find vmstat free ldd uptime xargs cpio tar gzip display top CPU processes display process status find files in a directory hierarchy display virtual memory statistics display free and used memory display shared library dependencies display system uptime build and exec commands from stdin copy files to and from archives create and extract *.tar archive files create and extract *.gz archive files Table 4.1: More tools and their functions www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 109 More Tools 4.2 Displaying System Processes (top) • Displays ongoing processor activity in real time • Shows processes for all users, unlike ps • Has several modes: ◦ Secure Mode, disables potentially dangerous interactive commands ◦ Cummulative Mode, shows time for a process and its dead children ◦ No-idle Mode, ignores idle or zombie processes • Typical output may be: 12:37:53 up 11 days, 2:48, 13 users, load average: 1.71, 1.35, 0.76 143 processes: 137 sleeping, 3 running, 3 zombie, 0 stopped CPU states: cpu user nice system irq softirq iowait idle total 12.1% 0.0% 4.6% 0.0% 0.0% 0.0% 83.1% Mem: 255616k av, 251200k used, 4416k free, 0k shrd, 10148k buff 17452k active, 206676k inactive Swap: 1024120k av, 397956k used, 626164k free 81720k cached PID USER 4621 nicku 5519 nicku 1466 root 1 root 2 root 3 root 4 root 6 root 5 root 7 root 8 root 13 root 63 root 786 root 1097 root 1101 root 1110 rpc 1129 rpcuser 1182 root 1234 root 1271 root PRI NI SIZE 18 0 10164 17 0 1248 15 0 19552 16 0 492 15 0 0 15 0 0 34 19 0 25 0 0 15 0 0 15 0 0 21 0 0 15 0 0 15 0 0 15 0 0 15 0 584 16 0 440 16 0 604 18 0 644 15 0 496 16 0 580 15 0 832 RSS SHARE STAT %CPU %MEM 9.9M 688 R 8.3 3.9 1248 848 R 3.7 0.4 3172 2636 S 2.7 1.2 464 440 S 0.0 0.1 0 0 SW 0.0 0.0 0 0 SW 0.0 0.0 0 0 SWN 0.0 0.0 0 0 SW 0.0 0.0 0 0 SW 0.0 0.0 0 0 SW 0.0 0.0 0 0 SW 0.0 0.0 0 0 SW 0.0 0.0 0 0 SW 0.0 0.0 0 0 SW 0.0 0.0 564 524 S 0.0 0.2 380 380 S 0.0 0.1 580 580 S 0.0 0.2 568 568 S 0.0 0.2 444 444 S 0.0 0.1 516 496 S 0.0 0.2 612 612 S 0.0 0.2 TIME CPU COMMAND 1:22 0 rsync 0:00 0 top 1:03 0 spamd 0:14 0 init 0:01 0 keventd 0:00 0 kapmd 0:00 0 ksoftirqd/0 0:00 0 bdflush 3:35 0 kswapd 1:06 0 kupdated 0:00 0 mdrecoveryd 1:13 0 kjournald 0:00 0 khubd 0:02 0 kreiserfsd 0:33 0 syslogd 0:00 0 klogd 0:00 0 portmap 0:00 0 rpc.statd 0:00 0 apmd 0:00 0 automount 0:05 0 sshd c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.3 Options and Interactive Commands for top 110 4.3 Options and Interactive Commands for top Significant command-line options for top include: Option -d -q -S -s -i -c Function delay between screen updates (seconds) Refresh without any delay. Specifies cumulative mode Secure mode Non-idle mode Show full command line instead of command name Table 4.2: Command line options for top The key interactive commands are: Command fF oO h or ? S i c l m t k r P M T u n or # W q Function Update display add and remove fields Change order of displayed fields Help on interactive commands Toggle cumulative mode Toggle display of idle proceses Toggle display of command name/line Toggle display of load average Toggle display of memory information Toggle display of summary information Kill a task (with any signal) Renice a task Sort by CPU usage Sort by resident memory usage Sort by time / cumulative time Show only a specific user Set the number of process to show Write configuration file /.toprc Quit Table 4.3: Interctive commands for top www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 111 More Tools 4.4 Reporting process status (ps) • ps prints info about a user’s processes: PID 22074 22075 22081 22590 TTY p0 p0 p0 p5 STAT S S S R TIME 0:02 2:13 0:00 0:00 COMMAND Eterm -t trans emacs -bg black asclock ps • Unlike jobs which only prints info about processes belonging to the current shell • Various display formats: ◦ Long format (l), e.g. $ ps l FLAGS 0 100000 UID PID PPID PRI 512 19665 19653 0 512 23656 23652 0 NI 0 0 SIZE RSS 7000 5616 1076 548 WCHAN 12d63d 1897da STA TTY TIME COMMAND S p2 0:19 /usr/bin/emacs S p5 0:00 /usr/bin/less -is ◦ User format (u), e.g. $ ps u $ ps u USER nicku nicku PID %CPU %MEM 15317 0.0 0.4 4897 0.0 0.3 VSZ RSS TTY 5940 1224 pts/26 4260 828 pts/26 STAT START S Nov20 R 09:02 TIME COMMAND 0:00 bash 0:00 ps u ◦ Jobs format (j), e.g. $ ps j PPID PID PGID SID TTY 2374 15317 15317 15317 pts/26 15317 4980 4980 15317 pts/26 TPGID STAT 4980 S 4980 R UID 1000 1000 TIME COMMAND 0:00 bash 0:00 ps j ◦ Virtual Memory format (v), e.g. $ ps v PID TTY 15317 pts/26 4987 pts/26 STAT S R TIME 0:00 0:00 MAJFL 968 173 TRS 562 62 DRS RSS %MEM COMMAND 5377 1312 0.5 bash 4033 828 0.3 ps v c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.5 Options for Reporting process status (ps) 112 4.5 Options for Reporting process status (ps) More options to ps: • Show ‘family tree’ (f) for processes, e.g. $ ps f PID TTY 19652 p2 19653 p2 19664 p2 19665 p2 19668 p2 19681 p2 STAT S S S S S S TIME COMMAND 0:00 /bin/login -h oakleigh gbdirect.co.uk -p -bash 0:00 /home/lee/bin/emacs -f gnus-no-server 3:02 0:23 /usr/bin/emacs 3:41 /usr/lib/netscape/netscape-communicator 0:00 (dns helper) Try pstree also. • Other important options: Option a w www wwww Effect Show all processes Show a wider display (normally just 80 characters wide) Show an even wider display Show an even wider display • Show environment after command line: $ ps e PID TTY STAT TIME COMMAND 19653 p2 S 0:00 -bash DISPLAY=oakleigh:0.0 TERM=xterm HOME=/home/davef PATH= www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 113 More Tools 4.6 Sorting output of ps • Sort ps results by specific fields: Option Ou OU OP Oo OK OJ OT Ov OC Function user uid parent PID session ID system time cumulative system stime start time VM size full command line Option Function Oc Op Ot Ok Oj Oy Or Os OS simple executable name PID tty user time cumulative user time priority resident set size size share Table 4.4: CLI options for sorting ps results • In man ps, search for SORT KEYS. • See http://mail.gnome.org/archives/gnome-list/ 1999-April/msg02337.html, where Miguel de Icaza explains the memory usage options of ps Summary: size is the total size of a process, including what is in swap. Not so useful. resident set size (rss) is the total amount of RAM memory actually used by a process, i.e., not in swap, but includes memory shared with other processes share the amount of memory used by a process that is shared with other processes. • So actual RAM used by a process is rss − share c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.7 Flavours of ps Options 114 4.7 Flavours of ps Options • There are three flavours of options for ps: ◦ BSD options (that we use here), with no dash ◦ Unix98 options that must be preceded with a dash ◦ GNU long options that must be preceded with two dashes • It is important to understand that there are BSD options and Unix98 options that have the same letter but which do completely different things, e.g., the BSD option ef $ ps ef PID TTY STAT TIME COMMAND 6744 pts/3 S 0:00 bash SSH_AGENT_PID=6649 HOSTNAME=nicksbox.tyict.vtc.e27436 pts/3 R SSH_AGENT_PID=6455 HOSTNAME=nicksbox.tyict.vtc. 0:00 ps ef is completely different from the Unix98 option: $ ps -ef UID PID PPID root 1 0 root 2 1 root 3 1 root 4 1 root 6 1 ... nicku 27369 8764 nicku 27375 27369 nicku 27438 6744 C 0 0 0 0 0 STIME Dec15 Dec15 Dec15 Dec15 Dec15 TTY ? ? ? ? ? TIME 00:00:04 00:00:00 00:00:00 00:00:00 00:00:00 CMD init [5] [keventd] [kapmd] [ksoftirqd/0] [bdflush] 0 07:03 pts/4 0 07:03 pts/4 0 07:12 pts/3 00:00:00 /bin/sh /usr/bin/xdvi masterfile 00:00:00 xdvi.bin -name xdvi masterfile 00:00:00 ps -ef • You can mix these options together, e.g., to get a wider display of ps -ef: $ ps -ef www UID PID root 1 root 2 ... PPID 0 1 C STIME TTY 0 Dec15 ? 0 Dec15 ? STAT S SW TIME CMD 0:04 init [5] 0:00 [keventd] • See man ps www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 115 More Tools 4.8 Examples using ps • To determine information about the yum process: $ ps auxwww | grep yum root 6440 2.2 2.8 14832 7284 pts/29 nicku 6542 0.0 0.1 968 332 pts/11 S S 09:19 09:20 0:00 /usr/bin/python /usr/bin/yum 0:00 grep yum • Hmm, how do we get rid of the line showing the “grep yum” process? One way is to exclude it using grep -v: $ ps auxwww | grep yum | grep -v grep root 6440 4.7 5.0 20684 12960 pts/29 S 09:19 0:05 /usr/bin/python /usr/bin/yum • But we can do that more simply with: $ ps auxwww | grep [y]um root 6440 2.6 5.0 20684 12960 pts/29 S 09:19 0:05 /usr/bin/python /usr/bin/yum c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.9 Finding Files using specified criteria (find) 116 4.9 Finding Files using specified criteria (find) • find searches your filesystem for files matching certain criteria, as we saw in section $ 3.4 on page 63. • Can match on name, owner, size, modification/access time, name ◦ and many others • Can execute commands on files it finds • Commonly used to archive sets of files, or clear out old files www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 117 More Tools 4.10 Criteria used in find expressions • Basic syntax: find base directory ... search criteria Test -name string -mtime -/+ days -user UID/username -size -/+ size -perm -/+ mode -type t What It Does Filename matches string (Shell metacharacters included) Modification time matches days Owner matches UID or username Size of the file matches size Permissions of the file match mode File is of type t (f — normal file, x — -executable file, d —directory,. . . . See man page for full details) • The values to match are very flexible, e.g. to find all files below /home/lee that were last modified less than 24 hours ago: $ find /home/lee -mtime -1 • Find all files below current directory greater than 1000k in size and with permissions 664 (-rw-rw-r---) $ find . -size +1000k -perm 664 -exec ls -l {} \; c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.11 Examples of using (find) 118 4.11 Examples of using (find) • Find all files ending in .jpg under current dir: $ find . -name "*.jpg" • Find filenames ending in .jpg and modified in the last 8 days below /etc: $ find /etc -name "*.jpg" -mtime -8 • Combine tests with -o and negate with !, e.g: ◦ To find all filenames not ending in .jpg or modified in the last 8 days: $ find . \! -name "*.jpg" -o -mtime -8 • Execute commands on files found. For example, to find then gzip-compress tar files: $ find . -name "*.tar" -exec gzip {} \; i.e. found filenames substitute for {} above ∗ • N.B. find searches the filesystem in real time; making disks work hard ∗ Piping the find results to xargs (section 4.19 ) is a more efficent approach www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 119 More Tools 4.12 Reporting virtual memory statistics (vmstat) • vmstat is used to identify system bottlenecks • Reports on processes, memory, paging, block IO, interrupts (traps), and cpu activity • SYNTAX: vmstat [-n] [-V] [delay [count]] • If no delay specified gives averages since last reboot ◦ Otherwise updates every delay seconds ◦ Shows averages since last report • count is the number of updates to give ◦ If no count is specified, just keeps on running • Option -n causes header display only once c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.13 Output from vmstat 120 4.13 Output from vmstat • The headings in the output have very short names: Section Head Procs Field r b w swpd free buff si so bi bo in cs us sy id Description no. of runnable processes no. of processes sleeping no. of processes swapped out but otherwise runnable virtual memory used (kb) idle memory (kb) memory used as buffers (kb) memory swapped in from disk (kb/s) memory swapped out to disk (kb/s) Blocks sent to a block device (blocks/s) Blocks received from a block device (blocks/s) interrupts per second, inc the clock context switches per second user time (as % of total CPU time) system time (as % of total CPU time) idle time (as % of total CPU time) Memory Swap IO System CPU Table 4.5: Field descriptions for vmstat output www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 121 More Tools 4.14 free • Another tool to examine memory status • Displays in kilobytes by default -k Output in kilobytes -m Output in megabytes -b Output in bytes -s x Poll every x seconds -t Display a ‘total’ line • Simpler alternative to vmstat • May be useful if vmstat not available • Quick check on overall memory usage c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.15 ldd 122 4.15 ldd • Modern operating systems support shared libraries ◦ In Windows, they are called DLLs — dynamic link libraries • Link the program with dynamically loaded shared libraries ◦ instead of including a copy of the same executable library code in all applications ◦ Makes the application much smaller ◦ But the application will not run correctly unless the correct version of each shared library is installed – In Windows, this is sometimes called DLL hell (Do a google search for “dll hell” or see http: //www.desaware.com/Articles/DllHellL3.htm – Linux shared libraries have the version number in their file name, solving the problem in a simple way. • ldd prints out dependencies on shared libraries for a given executable $ ldd /bin/bash libtermcap.so.2 => /lib/libtermcap.so.2 (0x40003000) libc.so.6 => /lib/libc.so.6 (0x40006000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00000000) • /bin/bash requires three shared libraries to run correctly • ldd is a very useful debugging tool • ldd is also needed for setting up restricted (chroot’d) environments www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 123 More Tools 4.16 uptime • uptime prints out basic information about performace of system $ uptime 4:26pm up 165 days, 23:23, 9 users, load average: 0.23, 0.13, 0.10 • Shows: ◦ Current Time ◦ Time since last reboot (Days, Hours:Minutes) ◦ Number of logged-in connections ◦ Load average past minute, past 5 minutes and past fifteen minutes – Load average is the average number of processes that are ready to run, but no CPU is available. – A load average of 0.5 is light loading; a load average of 3 is a fairly heavy work load; a load average of 10 will make people complain about the system being “slow”; a load average of 100 may result in a five minute wait between pressing a key on the keyboard and seeing the result. • Same information given in top — see section 4.2 on page 109. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.17 xargs — Filters 124 4.17 xargs — Filters • Most tools we examine are filters; they read standard input and, when no filename is specified, write to standard output: standard input filter e.g., sort,tr, grep, wc,... standard output • However, some, such as ls, file, echo, find, rm, cp, mv read the command line or file system, and do not read standard input: Reads cmd line, not stdin standard output • Sometimes we still want to combine these tools into a pipeline: data filter e.g., sort,tr, grep, wc,... pipe filter pipe filter transformed data • For example, how can we pipe data from the find command into the file command? find pipe file files names with types • This doesn’t work: $ find /etc | file Usage: file [-bcikLnNsvz] [-f namefile] [-F separator] [-m magicfiles] file... file -C -m magicfiles Try ‘file --help’ for more information. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 125 More Tools 4.18 xargs — an Adapter • The xargs command works as an adapter to read standard input, execute another command by passing the intput data to the command line of the second command: command line Reads cmd line, not stdin standard input xargs standard output • We can use xargs as an adapter between find and file to allow them to be part of en efficient pipeline: file names on file’s command line file names find xargs file file names with types • We can write it like this: $ find /etc | xargs file c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.19 xargs 126 4.19 xargs • Constructs and executes command-lines from information given on standard input • Commonly used in conjunction with find • Syntax: | xargs command • N.B. Data is normally piped to xargs • Example: Delete all files named core not modified in the past 8 days $ find / -name "core" -mtime +8 | xargs rm -f • This is more effient than $ find / -name "core" -mtime +8 -exec rm -f {} \; as it spawns fewer processes, therefore easing the load on the machine www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 127 More Tools 4.20 Options to xargs Interactive mode Verbose mode Limit arguments used to command -p -t -n num • Verbose mode will print out the commands it executes • Interactive mode prints out the command-line and awaits confirmation before executing • -n tells xargs to use at most num arguments to the command you are running with xargs • Example: $ find / -name "core" -mtime +8 | xargs -p -n 2 rm -f rm -f /home/lee/core /home/davef/core ?...y rm -f /usr/local/bin/core /root/core ?...y $ c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.21 Positioning filenames with xargs 128 4.21 Positioning filenames with xargs • By default xargs places the filenames at the end of the command you give • If they need to be somewhere else you can use --replace • Put {} at the point that you want the filenames inserted • Example: $ find /var/log -mtime -0.5 | xargs --replace cp {} /tmp • Disadvantage: execute new command process for each file, e.g., ◦ Here, when xargs executes echo just once, i.e., there is one echo process $ ls file1 file2 file3 $ ls | xargs echo file1 file2 file3 ◦ But when we use --replace, xargs executes echo three times, once for each file, i.e., we have three separate echo processes: $ ls | xargs --replace echo {} file1 file2 file3 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 129 More Tools 4.22 cpio • Similar to tar • Creates archives of files • Operates in copy-in or copy-out mode ◦ Copy-out mode writes archives ◦ Copy-in extracts from them • Takes filelist on standard input ◦ Not given on the command line ◦ Can use find to feed it filenames -i -o -A -F file -H format “Copy-in mode” — extract from archive “Copy-out mode” — create an archive Append to an archive Use file instead of standard input/output Use archive format format • Example: make a cpio archive of /etc: $ find /etc | cpio -o -F /tmp/etc.cpio • There are two main reasons for sometimes using cpio instead of tar: ◦ cpio will not include the contents of a directory if you list the directory name as a file. This is good when you want to include some but not all files from that directory. ◦ The RPM Package Manager (rpm) provides the rpm2cpio command that allows you to extract individual files from an RPM package. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.23 gzip 130 4.23 gzip • Compresses data taken from stdin or a file • ‘Lossless’ compression ◦ Safe on any file • Various compression levels, 1-9 1 — Fast, less compression 9 — Slower, more compression • By default takes filename and turns it into filename.gz $ ls foo* foobar $ gzip -9 foobar $ ls foo* foobar.gz ◦ Replaces original file • Can write compressed data to stdout $ ls foo* foobar $ gzip -9 -c foobar > foobar.gz $ ls foo* foobar foobar.gz ◦ Leaves original file intact www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 131 More Tools 4.24 Unzipping • To unzip a file there are two methods ◦ gunzip ◦ gzip -d • Really one file linked to two names • Uncompress all files in the current directory $ gzip -d *.gz • gzip uses Lempel-Ziv coding • Can also unpack files created with zip, compress and pack c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.25 tar 132 4.25 tar • tar creates archives of files • Used for transferring files between machines ◦ Or from place to place • Options: -x Extract from an archive -c Create archive -t List files in an archive Be verbose Compress/decompress archive with bzip2 Compress/decompress archive with gzip Find differences between archive and the filesystem -f Specifies a file name for the archive. Can be a device, such as tape device /dev/nst0 -v -j -z -d • Can use with or without the dash www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 133 More Tools 4.26 tar Examples • tar all files ending in .tex into a gzipped ∗ archive called alltex.tar.gz $ tar cvzf alltex.tar.gz *.tex • Check which files were included $ tar tvzf alltex.tar.gz • Extract them again $ tar xvzf alltex.tar.gz Same again, but with bzip2 compression: $ tar cvjf alltex.tar.bz2 *.tex • Check which files were included $ tar tvjf alltex.tar.bz2 • Extract them again $ tar xvjf alltex.tar.bz2 Integrated gzipping is available with GNU tar, but may not be available on some proprietary versions of tar. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk ∗ 4.27 Raw devices and tar 134 4.27 Raw devices and tar • Originally designed to talk to magnetic tapes ◦ My SCSI tape device is /dev/nst0 ◦ See kernel documentation in /usr/share/doc/ kernel-doc-2.4.22/devices.txt • Can also write to other raw devices • Useful to maximize use of the space on the floppy ◦ Fit 1.44Mb of data on a floppy ◦ Don’t need any space for filesystem information $ tar cvj filelist > /dev/fd0 or simply $ tar cvjf /dev/fd0 filelist or simply • Extract it again $ tar xvj < /dev/fd0 or $ tar xvjf /dev/fd0 • A useful option with floppy disks or writable CDROMs is -M proMpt for disk change when full www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 135 More Tools 4.28 Exercises 1. Use top to show the processes running on your machine. See section 4.2 on page 109. ¨ 2. Make top sort the list by memory usage. Press  © also see section 4.3 on page 110 h , and 3. Try killing a process, a good example would be your top process itself! Note that you can only kill your own processes unless you are root. 4. Use top to re-nice a process so that it gets more, or less CPU time. You can only lower the priority of your own processes. Only root can raise the priority of a process. Nice value of −20 is highest priority. Nice value of 19 gives lowest priority. See man nice. 5. Find a full list of every process on your machine and their full command name using ps. See the table in section 4.5 on page 112 and the examples in section 4.8 on page 115 6. Get the same view but tell ps to display the output in ‘family tree’ mode. Use the options axf to see the family relationships. See section 4.5 on page 112. 7. Request that ps sort it’s output by system time used. See section 4.6 on page 113. 8. Set vmstat running in a spare terminal updating every 5 seconds. See section 4.12 on page 119. 9. Set up free in another window doing the same thing. See section 4.14 on page 121. 10. Use ldd (see section 4.15 on page 122) to find out what shared libraries some common applications use: (a) /sbin/halt (b) /usr/bin/gnome-session (c) /usr/bin/display (d) /bin/bash (e) /sbin/nash 11. See sections 4.17 on page 124 to 4.21 for more about xargs. See section 4.19 on page 126 for examples of using xargs with find. Practice using xargs by finding sets of files and performing simple (Non-destructive!) operations on them, e.g., (a) Find all files in the file system modified in the last 24 hours (see §117) and make copies of them in a directory called backup in your home directory. See section 4.21 on page 128. (You may want to redirect standard error to the null device, /dev/null, both for find and for cp.) Do this two ways: i. Using the --replace option with xargs, described in section 4.21 on page 128, and ii. Using the --target-directory option to cp. See man cp. iii. Which of the two methods is most efficient? See section 4.21 on page 128. (b) Find all files over 5000k and make copies of them in the backup directory (c) Find all files ending in .txt in your home directory and below, and compress them using bzip2 12. Use cpio and tar to create an archive of the files you’ve copied in to the backup directory. See section 4.22 on page 129 and section 4.25 on page 132 to section 4.27. 13. Write an archive of /etc to a floppy as a raw archive. See section 4.27 on the facing page. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 4.29 Solutions 136 4.29 Solutions 1. Simply starting top from the command line will show all processes 2. Hitting M will sort the list by the resident set size, which includes all RAM used by each process, excludes the part that is paged out to the hard disk, but does include memory use shared with other programs. 3. Find your top process in the list (It will probably be near the head of a CPU-sorted list. Press k. You will be prompted for a process number, give the number from the PID column adjacent to your top process. You are then prompted for a signal to send the process. The default (15) should kill top and return you to your shell prompt. Signal 15 is the TERM signal. You can list all possible signals with the command kill -l. 4. Again find your top process, then press r. You are prompted for the PID and a nice value. Unless you are super-user you can only lower the process priority (Give it a higher nice value). 5. $ ps auxw Shows a full list of processes and doesn’t truncate the command-line to fit on one output line 6. $ ps axf Shows the family-tree view. Processes can create child processes using the fork() system call. The family-tree view shows this parent-child relationship. The parent of all processes is the init process, which has process ID 1. 7. Adding OK to the first command line should sort the output by system time; to see the system time, you may need to tell ps tp show it, for eaxmple, llke this: $ ps O user,%cpu,%mem,stime axOK The option “O” has two purposes: it can be used to select what data ps shows, and it can tell ps how to sort the data. When the option “0” is the first option to ps, it tells ps what coluns to display. When it appears anywhere else, it tells ps how to sort the data. In the above exammple, the options ax mean to show data on all the processes, even those that are not attached to a terminal. $ ps auxw OK 8. $ vmstat 5 9. $ free -s 5 10. (a) $ ldd /sbin/halt libc.so.6 => /lib/tls/libc.so.6 (0x00bd3000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00d4d000) (b) $ ldd /usr/bin/gnome-session libXrandr.so.2 => /usr/X11R6/lib/libXrandr.so.2 (0x0034f000) libXrender.so.1 => /usr/X11R6/lib/libXrender.so.1 (0x00611000) libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x008ee000) ... librt.so.1 => /lib/tls/librt.so.1 (0x0063b000) libnsl.so.1 => /lib/libnsl.so.1 (0x0064f000) libexpat.so.0 => /usr/lib/libexpat.so.0 (0x0075d000) (c) $ ldd /usr/bin/display libMagick-5.5.6-Q16.so.0 => /usr/lib/libMagick-5.5.6-Q16.so.0 (0x00701000) libtiff.so.3 => /usr/lib/libtiff.so.3 (0x00d22000) libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x00111000) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 137 More Tools ... libc.so.6 => /lib/tls/libc.so.6 (0x00371000) libdl.so.2 => /lib/libdl.so.2 (0x00c44000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00e5e000) 11. (a) i. $ find / -mtime -1 2>/dev/null | xargs --replace cp {} ~/backup 2>/dev/null ii. $ find / -mtime -1 2>/dev/null | xargs cp --target-directory=~/backup 2>/dev/null iii. Using cp --target-directory as shown in question 11(a)ii is much more efficient than xargs --replace. xargs --replace creates a separate cp process for each file, whereas using cp --target-directory allows xargs to call cp only a very small number of times, but each time copying a large number of files. I have explained this in section 4.21 on page 128. (b) Note that it is better to create the backup directory in /tmp, otherwise you will probably cause your network home directory to exceed your allowed storage quota! $ mkdir /tmp/backup Then either: $ find / -size +5000k 2>/dev/null | xargs --replace cp {} /tmp/backup2>/dev/null Or better, $ find / -size +5000k 2>/dev/null | xargs cp --target-directory=/tmp/backup 2>/dev/null (c) $ find ∼ -name "*.txt" | xargs gzip 12. $ find /tmp/backup -type f | cpio -o > cpio_archive.cpio and $ tar czjf tar_archive.tar.bz2 /tmp/backup or $ tar czvf tar_archive.tar.gz /tmp/backup 13. $ tar cvz /etc > /dev/fd0 or $ tar cvzf /dev/fd0 /etc c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk Module 5 Basic Filesystem Objectives • After completing this section, you will be able to: ◦ Understand a typical Linux filesystem ◦ Navigate the file hierarchy ◦ Manipulate files and directories ◦ Handle access control ◦ Deal with ‘special files’ and links 139 5.1 Filesystem Overview 140 5.1 Filesystem Overview • Linux uses ext3, reiserfs and ext2 as its native filesystems ◦ Also supports many other types • All data stored on a Linux system is a file • Ext2/3 file names can be 1 to 255 characters long ◦ Only / and nul are disallowed • Non-native filesystems have different features • Ext2/3 sees only two basic types of files: ◦ directories ◦ files • Other specialised types exist (FIFOs, and ‘special files’), these are covered later www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 141 Basic Filesystem 5.2 Files • Linux imposes no structure on files • All files are accessible at the byte level • Individual files used to have a maximum size of around 2GB (in an ext2 filesystem with an older kernel, 2.2.x) • Modern kernels (2.4.x) have a maximum file size of about 8TB • They have a minimum size of 0 bytes • Files can be extended after creation • Filename extensions such as .exe and .bat are unnecessary • Executable files are simply marked as such using file permissions (see later) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.3 Directories 142 5.3 Directories • Directories are files that list other files ◦ Can be normal files or directories ◦ Enables a hierarchy to be built • Each directory entry consists of two parts: a file name and an inode number (An inode is roughly a pointer to a file, see below) Filename . .. bin basic linux.tex Inode number 512 500 17324 24567 • The topmost directory is always called / ◦ Called the fileystem ‘root’ • Directory information can only be changed by Linux itself ◦ Ensures a proper structure is maintained www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 143 Basic Filesystem 5.4 Directory Hierarchy • By the Linux Filesystem Hierarchy Standard http://www.pathname.com/fhs/, many directories have ˆ specialised roles essential executable commands, available when no other filesystems are mounted /sbin essential system administrator commands, needed for repairing filesystems, when no other filesystems are mounted /boot static files of the boot loader /etc system configuration files /lib essential shared libraries and kernel modules /dev device files which control peripheral devices /tmp temporary files /mnt to mount external devices /var variable data files, e.g. logs, status and lock files, spooling files—let /usr be mounted read only /proc system information, dynamically generated by kernel /usr shareable, read-only data /usr/bin most user commands /usr/sbin further system administration commands /usr/lib further libraries for programming and shared libraries used by software packages /usr/share architecture independent data, including man pages and documentation /bin • User-installed programs typically go under the /usr/local hierachy • /mnt is a convenience to place all ‘mounted’ devices under one place c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.5 Pathnames 144 5.5 Pathnames '$ / &%     '$   home &%   d   '$ d '$   d lee &%   d   d d   File1 File2 mike &% • Files can be referred to by relative or absolute pathnames • Absolute pathnames begin with / /usr/sbin/httpd /usr/local/bin/safe-mysqld • The absolute pathname refers to one file only • A relative pathname does not begin with / and describes the path from the current directory to find a file, e.g. sbin/httpd bin/safe-mysqld www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 145 Basic Filesystem 5.6 Current Directory, Home Directory • Every process has a current directory • When you log in your shell’s current directory is your home directory ◦ Typically /home/username ◦ Superuser usually has /root for a home directory • pwd tells you the current directory ∼ the shell expands this to name of your home directory • The “∼” character is called “tilde” ∼ user the shell expands this to the name of the home directory of the user with user name user The cd Command • cd changes your current directory ◦ Typing cd path changes your current directory to path • path can be absolute or relative • Without arguments cd changes to your home directory c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.7 Dot (.) and DotDot(..) 146 5.7 Dot (.) and DotDot(..) • Directories always contain two entries “.” and “..” . .. Current directory Parent directory • Used for relative pathnames and navigation • Example: $ cd .. Change to the parent directory $ mv file .. Move file to the parent directory $ du . Display space used by files in current directory and below $ du .. Display space used by files in parent directory and below $ ./a.out Execute the file a.out in the current directory • This last row above shows execution of the particular file that is in the current directory ◦ If we had simply typed a.out then our PATH environment variable would be used to search for the file ◦ It may execute another a.out instead of the one we actually want www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 147 Basic Filesystem 5.8 Moving and Copying Files • The mv command is used to move files: mv [ options ] source file dest file mv [ options ] source file ... target directory e.g. $ mv oldname newname $ mv somefile .. • The cp command is used to copy files: cp [ options ] source file dest file cp [ options ] source file ... target directory e.g. $ cp -a thisfile newfile $ cp file1 file2 file3 /tmp ◦ A very important option is -a, which preserves permissions and symbolic links (see slide § 5.15 on page 154 for more about symbolic links). ◦ Can even use cp -ax to clone a hard disk partition. • See man cp and man mv c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.9 Removing Files 148 5.9 Removing Files • Files are removed using the rm command: rm [ options ] file ... e.g. $ rm -i thisfile thatfile rm: remove ‘thisfile’? y rm: remove ‘thatfile’? y $ • Most useful options are: rm -f Force removal and say nothing if cannot remove rm -r Recursively delete files e.g. $ rm -rf somedir will delete all files and subdirectories in directory somedir and below • See man rm • Removing a file is not considered an operation on the file ◦ It is an operation on the directory ◦ Filenames are merely links (Explained below) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 149 Basic Filesystem 5.10 Operations on Directories mkdir Create a new directory rmdir Remove a directory ls List the contents of a directory • These commands can take many arguments • mkdir can be told to create the whole pathname of directories if they don’t exist, e.g. $ mkdir -p ∼/new1/test/directory Will create the directories ∼/new1 and ∼/new1/test as well as ∼/new1/test/directory if they don’t already exist • ls arguments control what information is shown and how it’s sorted ◦ Some are explained later, consult man ls for full details. Also see figure 5.1 on page 156 for meaning of output of ls -l. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.11 Inodes 150 5.11 Inodes • Each file is represented by an inode∗ • An inode contains information about: ◦ File type (ordinary, directory, FIFO, block device etc.) ◦ Owner ID (user the file belongs to) ◦ Size (in bytes) ◦ Access, creation, and modification times ◦ Group ID (group the file belongs to) ◦ File permissions ◦ Mapping of the file contents (data sectors) • Inode layout and location varies with filesystem type The term inode was invented by Dennis Ritchie of AT&T. He admits to forgetting why he chose that name. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 151 Basic Filesystem 5.12 Inodes: ls -i and stat • ls -i displays inode numbers of entries, e.g. $ ls -i 200808 Ext2fs-0.1-14.html 200795 Ext2fs-0.1-2.html 200797 Ext2fs-0.1-4.html 200802 Ext2fs-0.1-8.html 200803 Ext2fs-0.1-9.html 200793 Ext2fs-0.1.html 204802 systemprogramming 542726 188447 333831 329729 278533 428042 include info ldap man misc nsmail • stat prints the inode contents for files inc. permissions, size, links, access times etc. $ stat /home/lee File: "/home/lee" Size: 4096 Filetype: Directory Mode: (0755/drwxr-xr-x) Uid:(504/lee) Gid:(502/lee) Device: 3,0 Inode: 200705 Links: 30 Access: Thu May 27 10:54:55 1999(00000.00:01:43) Modify: Thu May 27 10:44:41 1999(00000.00:11:57) Change: Thu May 27 10:44:41 1999(00000.00:11:57) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.13 Links 152 5.13 Links • More than one filename can refer to an inode ◦ These file names are links to the file • ln creates links to files ◦ Creates hard links by default ◦ ln -s creates symbolic or soft links • Erasing a file just removes its directory entry ◦ The file is only lost when all entries for it have been removed • Important: A filename is not the file ◦ The inode is the file ◦ All names are simply links (references) to the inode ◦ A bit like a Windows’ ‘shortcut’ www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 153 Basic Filesystem 5.14 Hard links • A hard link is merely a directory entry with the relevant inode number • Consider the following ◦ Start with: $ ls -li 428175 -rw-rw-r-- ... 4 May 26 13:18 test ◦ We create a hard link: $ ln test hl $ ls -li 428175 -rw-rw-r-- ... 4 May 26 13:18 hl 428175 -rw-rw-r-- ... 4 May 26 13:18 test • N.B. Hard links cannot cross filesystems • Inode numbers are filesystem specific directory test hl 428175 428175 inode 428175 file contents 1 hard disk c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.15 Symbolic Links (Soft Links) 154 5.15 Symbolic Links (Soft Links) • Symbolic (or Soft) links store the pathname of the linked file • This means they can cross filesystems. • Adding a symbolic link: $ ln -s test sl $ ls -li 428175 -rw-rw-r-- ... 4 May 26 13:18 hl 428176 lrwxrwxrwx ... 4 May 26 13:19 sl -> test 428175 -rw-rw-r-- ... 4 May 26 13:18 test directory test hl sl 428175 428175 428176 428176 test inodes 428175 file contents 1 hard disk www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 155 Basic Filesystem 5.16 Symbolic (or Soft) Links (continued) • If we replace the test file with another then the symbolic link still works, but the hard one still points to the old file! $ mv ../test2 test $ ls -li 428175 -rw-rw-r-- ... 5 May 26 13:45 hl 428176 lrwxrwxrwx ... 4 May 26 13:19 sl -> test 428178 -rw-rw-r-- ... 15 May 26 13:47 test inodes 428178 hard disk file contents 2 428175 file contents 1 428176 test directory test hl sl 428178 428175 428176 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.17 File Ownership, Users and Groups 156 5.17 File Ownership, Users and Groups • Every user has a unique user ID and a unique primary group • When a user starts a process, the process is owned by the user ID and primary group of the user • Every file has a user that owns the file, and a group that owns the file. • When a process creates a file, the file is owned by the same user and group as the process • E.g., if user nicku with primary group staff creates a file, then the file will be owned by user nicku and group staff. • Figure 5.1 shows how ls -l shows file ownership. $ ls −l suid−sgid.tex −rw−r−−r−− 1 nicku user others group file type number of hard links to this file (see later) staff 9985 Dec 19 18:16 suid−sgid.tex time and date the file was last modified user who owns the file group that owns the file size in bytes Figure 5.1: The output of ls -l: what each field is. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 157 Basic Filesystem 5.18 Access Control, Users and Groups • File access can be limited to specific users • Super user(s) can override access control • Access control is set by user and group ID • Each user has a user-id (UID) and one or more group-ids GIDs) • Processes have an associated UID and GID ◦ Inherited from the user who created the process • They can however can be changed: ◦ Processes are known as set-user ID (SUID) if they set their own user ID equal to the user that owns the executable file, ◦ or set-group ID (SGID) if they set their own group ID equal to the group owner of the file • There are three special modes that we will examine in the next module: “set user ID”, “set group ID” and the “restriction deletion flag”. You can read about them in info: $ info ’(coreutils)Mode Structure’ $ info ’(coreutils)Numeric Modes’ c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.19 Categories of Access Control 158 5.19 Categories of Access Control • There are three categories of access, and three categories of user: Category of Access read write execute Permission For Files read write permission to open the file for reading permission to open the file for writing r w x Set of Users user group others For Directories u g o execute permission to list the file names permission to create and delete files in the directory permission to execute permission to change the file; scripts also into the directory need read permission. • There are three sets of access control, one for each of the three sets of users: user: the user that owns the file group: members of the group that owns the file, except for the user that owns the file others: all users who are not the owner of the file, and who are not members of the group that owns the file. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 159 Basic Filesystem 5.20 Access Control — Example • ls -l shows the access permissions, e.g. $ ls -l -rw-rw-r-- 1 www lrwxrwxrwx 1 lee -rw-rw-r-- 1 lee www lee lee x_windows.tex img -> ../linux/img/ test.log • There are three sets of permissions, each of which applies to one set of users • To find permissions that apply to a user, determine what set of users the user belongs to, and then those permissions apply to that user. drwx rwx rwx Apply to other users who are not the user that owns the file nor members of the group that owns the file Apply to members of the group owner of the file, but excluding the user that owns the file Apply to the user that owns the file File type c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.21 Examples of minimum file permission requirements 160 5.21 Examples of minimum file permission requirements • This table shows examples of minimum file permission requirements • It is based on table 2-2 on page 37 of the book Essential System Administration by Æleen Frisch, O’Reilly 2002. • I recommend this excellent book. Command cd /var/project ls /var/project ls -l /var/project cat /var/project/user1.txt echo "hello" >> /var/project/user1.txt /var/project/binary-program /var/project/script-program rm /var/project/user1.txt minimum access required on the file on the directory no file ----r--w--x r-x ----x r-r-x --x --x --x --x -wx www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 161 Basic Filesystem 5.22 Changing Access Permission: chmod • Only the owner of a file (or the super-user) may alter its access permissions • chmod (change mode) changes access permissions ◦ Works in two ways, symbolically or numerically ◦ Symbolically is easier to remember (for most) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.23 chmod symbolically 162 5.23 chmod symbolically • Select who you want to change permissions for (u=user, g=group, o=others, a=all) • Decide whether you want to grant a permission(+), remove(-), or set(=) it • Take the permission that you want to change (r=read, w=write, x=execute) • Example: $ chmod gu+w filename Adds write permission for user and group • You can make several changes by separating the settings with commas, e.g. $ chmod a-w,gu=rw filename Removes write permission for all, then grants it for the user and group www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 163 Basic Filesystem 5.24 chmod numerically • Once you know this it is often quicker • An octal digit represents each permission type 4 2 1 read permission write permission execute permission • Add up the permission numbers you want for each user group (owner, group, all) and supply these to chmod • Example: $ chmod 755 filename grants all permissions to the owner (4+2+1), and read and execute (4+1) to group and all others ◦ chmod 755 filename is equivalent to chmod u=rwx,go=rx filename c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.25 Special Permissions: SUID, SGID 164 5.25 Special Permissions: SUID, SGID • When the SUID permission applies to an executable file, then, when it is executed, the file will execute with the UID of the owner of the file, instead of the UID of the person executing the program. • When the SGID permission applies to an executable file, then, when it is executed, the file will execute with the GID of the group owner of the file, instead of the GID of the person executing the program. • When the SGID permission applies to a directory, then all files created there have the same group owner as the directory, instead of the group owner of the process that created the file, as would be the case otherwise. • When the restricted deletion flag (“sticky bit”) is enabled on a directory, then only the owner of a file (and root) may delete or modify the file. Normally, if other permissions allow it, anyone may delete or modify a file that they do not own. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 165 Basic Filesystem 5.26 chmod: Symbolic Permissions • To apply the SUID permission to file: $ chmod u+s file • To apply the SGID permission to file: $ chmod g+s file • The restricted deletion flag (“sticky bit”) permission applies only to the “others” part of the file permissions, but it behaves as I described above. To apply the “restricted deletion flag” permission to directory: $ chmod o+t directory c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.27 chmod: SUID, SGID 166 5.27 chmod: SUID, SGID • The permissions for SUID and SGID are specified numerically like this: numeric mode changes: 2000 4000 1000 • Example: # $ # $ # $ make the program executable file /tmp/ash suid: sudo chmod 4755 /tmp/ash make the program executable file /tmp/ash both suid and sgid: sudo chmod 6755 /tmp/ash remove the suid and sgid permissions from /tmp/ash: sudo chmod 755 /tmp/ash set group ID set user ID restricted deletion flag (“Sticky bit”) • The permission for SUID and SGID is specified symbolically with ‘s’, and can be added or removed from user or group permissions. • Example: # $ # $ # $ make the program executable file /tmp/ash suid: sudo chmod u+s /tmp/ash make the program executable file /tmp/ash both suid and sgid: sudo chmod ug+s /tmp/ash remove the suid and sgid permissions from /tmp/ash: sudo chmod ug-s /tmp/ash www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 167 Basic Filesystem 5.28 Set Group ID Directory • If the “set group id” (SGID) permission is set on a directory, then: ◦ if a user makes a change to a file or creates a file in that directory, the file will have group owner the same as the directory. ◦ If a user creates a directory in that directory, it too will have the Set Group ID bit set. As with the file, it will have a group owner the same as the group owner of its parent SGID directory. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.29 Set Group ID Directory — Example 168 5.29 Set Group ID Directory — Example • Let’s see the result of creating a file in /var/project both before and after adding this permission: $ ls -ld /var/project drwxrwx--2 root admin $ touch /var/project/test1 $ ls -l /var/project -rw-rw-r-1 nicku nicku 4096 Jan 2 13:48 project 0 Jan 2 13:53 test1 Now we add the set group ID bit to the directory permissions, and see the effect: $ sudo chmod g+s /var/project $ ls -ld /var/project drwxrws--2 root admin $ touch /var/project/test2 $ ls -l /var/project -rw-rw-r-1 nicku nicku -rw-rw-r-1 nicku admin 4096 Jan 2 13:48 project 0 Jan 0 Jan 2 13:53 test1 2 13:54 test2 ◦ Note that the user nicku has primary group nicku, and is also a member of the admin group. • when the directory is not SGID, files I create have my primary group ID. • when the directory is SGID, files I create have the group ID of the directory. This allows others in the group to read, write and change the files created by any group member. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 169 Basic Filesystem 5.30 Restricted Deletion Flag (“Sticky Bit”) on Directories • When a directory has the restricted deletion flag (“sticky bit”) set: ◦ Files cannot be overwritten or deleted by any user except the user that created the file ◦ Normally, if other permissions on the directory allow, anyone may delete a file they do not own. • Use in /tmp to protect users’ files from being altered by other users $ ls -ld /tmp drwxrwxrwt ↑ 397 root root 12288 Feb 5 13:07 /tmp • Notice that instead of “x” for others, there is a “t” • Anyone can create a new file in /tmp, but only the root user can overwrite or delete files they do not own. • Useful for many applications besides protecting files in the /tmp directory. • Add to a directory with this command: $ sudo chmod o+t directory c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.31 umask 170 5.31 umask • Files begin with a default access setting; files with -rw-rw-rw-, or 0666, and directories with drwxrwxrwx (0777) ◦ Specified by a user’s umask setting • This only works numerically • Unlike chmod, specified permissions are turned off ◦ umask specifies permissions which are absent • With a umask setting of 000 files are created with permissions rw-rw-rw- (666) • Default umask (on Red Hat systems) is 002 which means files are typically created rw-rw-r-- (664) ∗ e.g., $ umask 002 $ touch foo $ ls -l foo -rw-rw-r-- 1 lee lee 0 Feb 10 17:17 foo $ umask 222 $ touch foo2 $ ls -l foo2 -r--r--r-- 1 lee lee 0 Feb 10 17:17 foo2 • The default access setting is bitwise ANDed with the complement of the umask. For example, with a umask of 027, and a default setting of 666, we have: default umask ∗ octal 0666 ∼0027 → binary 110 110 110 ∼000 010 111 AND 110 110 110 & → 111 101 000 110 100 000 → octal 0640 → rw-r----- This is the case on Redhat systems where users typically belong to a group of their own; other distributions will probably use a default umask of 022. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 171 Basic Filesystem 5.32 Special Files — /dev • Files under /dev typically represent devices attached to your computer • Programs can open and close them and read from and write to them — as with regular files • Kernel code handles exactly how these work • Two types ◦ Block — Disk drives, RAID devices, SCSI devices, CDROMs ◦ Character — Printers, modems, tape drives, mice, sound devices, USB devices, etc. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.33 Special Files — /proc 172 5.33 Special Files — /proc • The section of the filesystem called /proc doesn’t contain real files • It contains system status information • It is built dynamically by the Linux kernel • For example: Location /proc/ number Information On specific running processes. See man proc for details /proc/meminfo How much memory is in your system and how much is being used /proc/cpuinfo What CPU(s) you are currently using /proc/filesystems Filesystems your kernel supports /proc/mounts The definitive list of what is actually mounted on your computer /proc/uptime The uptime of the system /proc/kcore An image of your physical memory /proc/net Network status of your machine /proc/pci PCI devices found at initialization /proc/sys Details on kernel variables, e.g. ◦ Maximum number of files we can open (fs/file-max) ◦ Number of files currently open (fs/file-nr) ◦ Whether IP is forwarded between network interfaces (net/ipv4/ip forward) Set the values “permanently” in /etc/sysctl.conf. See man sysctl.conf and man sysctl Table 5.1: System Information from /proc www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 173 Basic Filesystem 5.34 Filesystem Structure and /etc/fstab Multi-Volume Filesystems • The filesystem can be held on several devices • Large disks can be divided into partitions ◦ This creates several logical devices • A basic Linux system must be present on / • Other parts of the fs may be mounted at any time • The main ones are mounted at boot time • This is controlled by the important /etc/fstab file which says which file system is mounted where: ◦ while the computer starts up, and ◦ when you type mount mount point or mount /dev/ device name c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.35 /etc/fstab — Example 174 5.35 /etc/fstab — Example Mount Point / /home /tmp /usr swap /mnt/floppy /mnt/cdrom /mnt/kashmir /var/admin /home/lee/LANDLORD /mnt/winas FS type ext3 ext3 ext3 ext3 swap ext3 iso9660 smbfs nfs nfs ntfs Options defaults defaults defaults defaults defaults noauto noauto,ro guest defaults defaults default,ro,umask=0000 Dump 1 1 1 1 0 0 0 0 0 0 0 Check order 1 2 2 2 0 0 0 0 0 0 0 # Logical Volume # /dev/hda1 /dev/hda5 /dev/hda7 /dev/hda6 /dev/hda8 /dev/fd0 /dev/cdrom \\kashmir\c landlord:/var/admin landlord:/home/lee /dev/hda5 • See man fstab • The first field is a device • The second field is a directory, the mount point, that must exist; • The third field is the type of the file system (see man mount and also cat /proc/filesystems) • The fourth field is options to mount. See man mount • The second last field says whether the program dump should backup this filesystem • The last field is: 0 if the program fsck (file system check) should check the file system on boot, 1 if it should be checked first (should be just the root filesystem “/”), 2 if it should be checked later in boot sequence. • Currently (2004) Red Hat are not putting the NTFS module into the 2.4.x kernel—get the module RPM from http://linux-ntfs.sourceforge.net/rpm/fedora1.html www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 175 Basic Filesystem 5.36 Mounting Additional Volumes • To mount a filesystem use mount, e.g. $ mount /dev/cdrom /mnt/cdrom • Mounts the filesystem /dev/cdrom in the directory /mnt/cdrom • cd /mnt/cdrom changes directory to the root of the CDROM’s filesystem • To unmount use umount name where name is either the filesystem name or the mount point: $ umount /dev/cdrom $ umount /mnt/cdrom • Note — A filesystem can only be unmounted when it is no longer in use. ‘In use’ includes: ◦ Having any file on that filesystem open ◦ Having a shell in a directory on that filesystem • Use the lsof (“list open files”) program to identify and kill processes that prevent unmounting a file. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.37 Mounting shared filesystems 176 5.37 Mounting shared filesystems • NFS filesystems can be mounted with $ mount -t nfs hostname : path mount-point mount-point is a directory which must already exist. path is a directory on the NFS server that it exports to the client. • Example: $ mount -t nfs landlord:/backup /mnt/backup • The mount program is smart enough to determine that the type of the filesystem is nfs, so you do not need to specify it: $ mount ictlab:/var/ftp/pub /mnt/nfs • Share files from MS-Windows machines using Samba • This is a free implementation of the Windows file-sharing protocols, e.g. $ mount -t smbfs ’\\ntbox\c’ /mnt/ntbox • N.B. Linux does not use the ‘drive letter’ concept at all ◦ Drives and shares integrate seamlessly into the filename tree www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 177 Basic Filesystem 5.38 Summary • The primary Linux filesystems are Ext3, Reiserfs and Ext2 • It has a tree-like hierarchy of directories • Directories merely contain pointers to files (inodes) • inodes contain all the information about a file • Can have multiple links to the same file • Read/Write access is controlled per file • Creation/Deletion of files is controlled by permissions of the directory • Several filesystems can be mounted to create the directory hierarchy c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.39 Filesystem Exercises 178 5.39 Filesystem Exercises 1. Basic navigation (a) Log in and use pwd to discover what the full path of your home directory is. (b) Change directory to /bin and then /tmp. Use pwd to check you got there each time. (c) When in /tmp type cd .. and use pwd to find out where you end up. See section § 5.7 on page 146 (d) What is the parent directory of the root of the filesystem? Why is this so? (e) Move back to your home directory. Think of three ways you can do this. See section § 5.6 on page 145. 2. Directories (a) Start in your home directory and create a directory called new (b) Change to the new directory and create a directory called newer (c) Go to your home directory. Now create a directory under newer called newerstill There are two ways to do this what are they? (Hint: You don’t have to change directories to solve this.) (d) Remove all the directories that you’ve just created, there are several ways to do this. See section § 5.10 on page 149. (e) Create the same directory structure with one command. See section § 5.10 on page 149. 3. Links You may wish to see the handout: http://ictlab.tyict.vtc.edu.hk/ossi/lab/sym-link/sym-link.pdf for more details about links. (a) Create a file called test in your home directory (Typing echo abc > test should do this). Now create a hard link to test called h test and a symbolic link to test called s test (b) Find out the inode number of the files. Check you understand why they are what they are. (c) Remove the original file called test. Can you still get at the contents of the original file? (d) What happens if you try cat s test? Make sure you understand the distinction between h test, and s test (e) Try to make a hard link to your home directory. Why does this fail? (f) From your home directory, try to make a hard link from a file in the /boot directory to your home directory. In other words, the link should be in your home directory, and when you examine it, you should see the contents of the file in the /boot directory. Does it work? (g) From your home directory, try to make a symbolic link from a file in the /boot directory to your home directory. The link should be in your home directory. Does it work? How do you know whether it works properly or not? (Hint: look at the -L option to ls). (h) Change to the /boot directory. Make a symbolic link from a file in that directory to your home directory. The link should be in your home directory. Make sure that your result is correct. How to check? (i) Make a symbolic link from a file in the /boot directory to your home directory that is a relative symbolic link. A relative symbolic link is one where the target does not start with a /. Here is an example, in my home directory: $ ls -l kernel.h lrwxrwxrwx 1 nicku www.linuxtraining.co.uk nicku 19 Feb 4 08:22 kernel.h -> ../../boot/kernel.h c GBdirect Ltd and parts c Nick Urbanik, 2004 179 Basic Filesystem The target file is the one shown to the right of the arrow “->”. The link is absolute if it starts with a slash “/”; it is relative if it does not start with a slash. 4. /proc — See section 5.33 on page 172 (a) Use the files in /proc to find out how much memory your system has and what processor it is running on. (b) Find out what PCI devices are attached to your machine. (c) Find out what environment variables are set for your currently running shell using the information in /proc. Hint you can get the process-id of your shell using $$ (d) A router does IP forwarding; it allows IP traffic entering one network interface to travel though the machine to another network interface, even if the destination IP address is not on the router itself. Whether or not your machine is doing IP forwarding is stored in the file /proc/sys/net/ipv4/ip forward. You can cat this file: a value of 1 means that IP forwarding is turned on. Find out whether or not your machine will forward IP. Note: the file /etc/sysctl.conf will change the values of files under /proc/sys while the computer is starting up. See man sysctl.conf and man sysctl. (e) Find out how many files are currently open on your system. Hint: do $ cat /proc/sys/fs/file-nr 2812 70 52416 $ man proc and then search the manual page for the string “file-nr”. 5. /etc/fstab (a) Add an entry to your /etc/fstab file to automatically mount the NFS filesystem /var/ftp/pub from ictlab on a directory /mnt/nfs whenever the computer starts up. See section 5.35 on page 174. See also section 5.37 on page 176. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 5.40 Filesystem Solutions 180 5.40 Filesystem Solutions 1. Basic Navigation (a) You will probably see $ pwd /home/ username where username is the name you log in with. (b) Check that the output of pwd is /bin and /tmp (c) You should end up in / This is the root of the filesystem. (d) The parent of / is itself. You can use ls -lia to show that the inode numbers for . and .. are the same when you are in /: $ cd / $ ls -id / .. 2/ 2 .. You can see here that ‘/’ and ‘/..’ both have an inode number of 2, so both are the same file, since both are in the same file system. (e) You can move back to your home directory by using any of the following: i. ii. iii. iv. v. $ cd $ cd ∼ $ cd ∼ username $ cd $HOME $ cd /home/ username if your home directory is under /home. 2. Directories (a) $ cd $ mkdir new (b) $ cd new $ mkdir newer (c) $ cd then either: i. $ cd new/newer $ mkdir newerstill ii. or $ mkdir new/newer/newerstill (d) Any of the following will work: i. $ $ $ $ $ $ $ ii. $ $ cd cd new/newer rmdir newerstill cd .. rmdir newer cd .. rmdir new cd rmdir -p new/newer/newerstill c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 181 Basic Filesystem iii. $ cd $ rm -rf new/ (e) $ cd $ mkdir -p new/newer/newerstill 3. Links (a) This is achieved as follows: $ ln test h_test $ ln -s test s_test (b) ls -li should show that the inode number of the original file and h test are identical h test is another name for the original file. The inode number for s test will be different. It is a separate file that contains information about the location of the file it is a link to. (c) After you created the hard link the original file had two names test and h test. You have removed test but until all names for a file have been removed it is still accessible. In this case you can do cat h test to see the contents of the file. (d) This should fail with a No such file or directory message. s test contained a pointer to the file test not the inode number. There is no longer a file named test so this cannot work. Hard links reference a file by its inode number, symbolic links reference it by its name (e) This is not allowed, as it could stop the filesystem being strictly hierarchical. It would cause problems with all software that searches the filesystem, since loops could be created. If this were the case, all software would need extra code to check for loops, making the software bigger, slower, more complex and hence more likely to have bugs. (f) $ cd $ ln /boot/kernel.h . ln: creating hard link ‘./kernel.h’ to ‘/boot/kernel.h’: Invalid cross-device link The reason that it is not possible to create a hard link from one partition to another is that a hard link is specified by an inode number. An inode number is only unique within one partition or filesystem. (g) $ cd $ ln -s /boot/kernel.h . $ ls -l kernel.h lrwxrwxrwx 1 nicku nicku 14 Feb 28 05:50 kernel.h -> /boot/kernel.h $ ls -Ll kernel.h -rw-r--r-1 root root 473 May 8 2002 kernel.h So it is clear that this symbolic link works fine. We can examine the file through the link with $ less kernel.h and verify that we can see the same content as when we view /boot/kernel.h. (h) $ rm ∼/kernel.h $ cd /boot $ ln -s /boot/kernel.h ∼ $ ls -l kernel.h lrwxrwxrwx 1 nicku nicku $ ls -Ll kernel.h -rw-r--r-1 root root (i) $ cd $ ln -s ../../boot/kernel.h . $ ls -l kernel.h lrwxrwxrwx 1 nicku nicku Note that the command c GBdirect Ltd and parts c Nick Urbanik, 2004 14 Feb 28 05:50 kernel.h -> /boot/kernel.h 473 May 8 2002 kernel.h 19 Feb 28 05:55 kernel.h -> ../../boot/kernel.h www.linuxtraining.co.uk 5.40 Filesystem Solutions 182 $ ln -s ../../boot/kernel.h ∼ will work no matter where your current directory is. 4. /proc (a) cat /proc/meminfo should show you memory usage. (b) cat /proc/pci gives a list of all PCI devices. (c) cat /proc/$$/environ will give a list of the environment of your current shell. Each variable is delimited with the nul character (decimal 0). The following will show the output with one variable per line: $ cat /proc/$$/environ | tr ’\000’ ’\n’ (d) In this example machine (a router), ip forwarding is on: $ cat /proc/sys/net/ipv4/ip_forward 1 However, on a workstation, ip forwarding is normally off. It is controlled conveniently by an entry in the file /etc/sysctl.conf; for example, in the example router, the file contains these lines: net.ipv4.ip_forward = 1 fs.file-max = 16384 which ensure that /proc/sys/net/ipv4/ip forward will contain ‘1’ and that /proc/sys/fs/file-max will contain ‘16384’. See man sysctl.conf and man sysctl. (e) $cat /proc/sys/fs/file-nr 16384 13018 16384 $ man proc And in this document, we see: The (read-only) file file-nr gives the number of files presently opened. It contains three numbers: The number of allocated file handles, the number of free file handles and the maximum number of file handles. The kernel allocates file handles dynamically, but it doesn’t free them again. If the number of allocated files is close to the maximum, you should consider increasing the maximum. When the number of free file handles is large, you’ve encountered a peak in your usage of file handles and you probably don’t need to increase the maximum. we see that nalloc = 16384, nfree = 13018, nmax = 16384. So to answer the question, nopen = nalloc − nfree = 16384 − 13018 = 3366. However, at one stage since it was booted, this system ran out of open file handles, so it may be a good idea to increase the value in this line of /etc/sysctl.conf: fs.file-max = 16384 5. Add a line like this to /etc/fstab: ictlab:/var/ftp/pub /mnt/nfs nfs defaults 0 0 The file system table /etc/fstab is critical for successful startup of a Linux system; it is important to understand the basics of how filesystems are mounted at system startup. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 Module 6 Finding Documentation Objectives After completing this section, you will be able to: • Identify sources of information to answer your particular questions • Be familiar with the main sources of documentation on your machine: ◦ man pages ◦ info ◦ documentation in the /usr/share/doc directory. ◦ Ultimate documentation: the source code • Make some of that documentation accessable through a web browser • Identify all the documentation that is part of any software package • Be able to locate documentation from the Linux Documentation Project, including HOWTOs and Guides • Know how to find information from other people through the Internet, news groups and mailing lists 183 6.1 Documentation everywhere? 184 6.1 Documentation everywhere? • Linux has a huge amount of documentation • Much of it is installed on your hard disk • For large distributions, there may be an additional few many megabytes of documentation available (e.g., the documentation CDROM with Red Hat Linux) • The only problem is: “how do I find the documentation that I need to answer my questions?” www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 185 Finding Documentation 6.2 Where is the documentation on my computer? • manual pages (read using man command) • info (read using info command or emacs) • documentation in the /usr/share/doc directory. • documentation in the /usr/src/Linux/Documentation directory. • The ultimate documentation: the source code c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 6.3 Some main sources of information from the Internet 186 6.3 Some main sources of information from the Internet • The Linux Documentation Project: http://tldp.org/ provides these documents in many formats: ◦ HOWTOs about a large number of topics ◦ Guides: free online books • Search using Google: http://www.google.com/ • Groups in http://www.google.com/, and usenet • Red Hat (and other distributions) provide plenty of online information. These URLs are current 30 Oct 2003: http://www.redhat.com/docs/, http://www.redhat.com/docs/manuals/linux/. After you have read the Getting Started Guide, I particularly recommend the Red Hat Reference Guide, and the Red Hat Customisation Guide. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 187 Finding Documentation 6.4 Mailing Lists • A mailing list is a discussion group over email ◦ Mostly have a narrow technical focus ◦ a great way to get help from an expert on the subject. • You subscribe to a mailing list by: ◦ Send email to the list server, or fill in a simple web form ◦ The list server sends you an email ◦ You reply to that email • Any subscribed user can send an email to the list ◦ All subscribed users will receive that email ◦ Set one mail folder for each list ◦ Set your email software to filter all list email into the right folder, to keep list email separate from your personal email. • Examples: ◦ Red Hat and Fedora lists: http://www.redhat.com/mailman/listinfo ◦ the Apache lists: http://httpd.apache.org/lists.html ◦ the Samba lists: http://us1.samba.org/samba/archives.html, plus many others (I subscribe to about 40!) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 6.5 Asking Questions on a Mailing List 188 6.5 Asking Questions on a Mailing List • Before sending questions to a mailing list, read Eric Raymond’s How To Ask Questions The Smart Way : http: //www.catb.org/∼esr/faqs/smart-questions.html www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 189 Finding Documentation 6.6 Online Magazines • Some online magazines include: ◦ Alan Cox’s Portaloo: http://www.linux.org.uk/cgi-bin/portaloo This site is surprisingly useful; it collects the headlines from about 25 web sites with technical news, so that you can review the headlines from all the sites, and click on any headline that looks interesting. This takes you directly to the story. It saves me a lot of time. ◦ Apache Week: http://www.apacheweek.com/ ◦ LWN.net (used to be Linux Weekly News): http://lwn.net/ ◦ NewsForge: http://www.newsforge.com/ ◦ Linux Magazine: http://www.linux-mag.com/ ◦ Linux Today: http://linuxtoday.com/ c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 6.7 Info 190 6.7 Info • Many GNU tools are documented properly using the Texinfo system. • Texinfo provides two forms of documentation from one source: ◦ online hyperlinked info pages ◦ Nicely formated printed documentation • Can read info pages using the info or pinfo commands, or (my choice): emacs www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 191 Finding Documentation 6.8 Using the info command • To get information about command, type: $ info command • For example, to get information about chmod, type: $ info chmod • The page has “cross references” (like hyperlinks) and menu items. • You can press the tab key to select the next cross reference, and Enter to go there • Go to the next page with the n key • Go to the previous page with the p key • You can go up, and visit the last page you just visited (like the back button on your browser) • Get detailed help with the h key c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 6.9 Using emacs to read info pages 192 6.9 Using emacs to read info pages • While running emacs, you can enter info by pressing C-h i (That’s Control-h, then press the key i). • You will see a large number of links. You can search for the right topic by: ◦ pressing C-s and typing the name you are looking for, or  m then ◦ go straight to the menu item by pressing   the name of the menu item. • press the middle mouse button while the mouse is over a cross reference or menu item ◦ If your mouse only has two buttons, press both buttons at once. ◦ If that doesn’t work, enable “Emulate 3 buttons” in the mouseconfig program, and restart X. • There are navigation buttons on the top of emacs www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 193 Finding Documentation 6.10 Using rpm to identify all documentation for a software package • Suppose: ◦ you want to find the information for the Apache web server, httpd ◦ You type man httpd; there is no result. ◦ What should you do next? • Use the RPM package manager. • The RPM package manager (RPM) is a set of programs to manage installed software. • RPM maintains a database containing very detailed information about all installed software • RPM can: ◦ list all files in a software package, ◦ list all documentation files in the package, ◦ list all configuration files in the package, ◦ determine if a file has changed since the original installation, ◦ verify that a software package is correctly installed, ◦ tell you what RPM package any file comes from, ◦ . . . and countless other things. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 6.11 A quick guide to rpm 194 6.11 A quick guide to rpm • Here is a brief list of rpm query commands. I have used the httpd package as an example. command rpm -qa | less rpm -q httpd effect list all installed software packages show the version of the httpd package, if it is installed rpm -qa | grep httpd show all installed packages that have httpd in their name rpm -ql httpd list all files in the httpd package rpm -qd httpd list all documentation files in the httpd package rpm -qc httpd list all configuration files in the httpd package rpm -qi httpd display information about the package rpm -V httpd v erify that the httpd package is correctly installed rpm -qf /etc/passwd determine which package the /etc/passwd f ile belongs to • Further information about rpm: ◦ man rpm ◦ The chapter on rpm in the Red Hat Linux Customization Guide ◦ The book Maximum RPM, available in html on the Red Hat documentation CDROM. ◦ It is worth learning how to build RPM packages. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 195 Finding Documentation 6.12 A quick guide to dpkg (on Debian Linux) • The Debian Linux distribution is the easiest to upgrade (but not so easy to install), thanks to the wonderful apt-get command. For you reference, here is a table comparing some rpm and dpkg commands: command dpkg --list | less dpkg -l httpd dpkg dpkg dpkg dpkg effect list all installed software packages show the version of the httpd package, if it is installed --list | grep httpd show all installed packages that have httpd in their name --listfiles httpd list all files in the httpd package --print-avail apache display information about the package -S /etc/passwd determine which package the /etc/passwd file belongs to c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 6.13 Browsing Documentation Via Your Web Server 196 6.13 Browsing Documentation Via Your Web Server • You may configure the web server, apache, to serve the documentation online • Two steps may be necessary to achieve this: ◦ create a file /etc/httpd/conf.d/doc.conf containing: Alias /doc/ /usr/share/doc/ Options Indexes FollowSymLinks ◦ enable your web server service: $ sudo /sbin/chkconfig httpd on $ sudo /sbin/service httpd start • Now you should be able to browse the documentation in /usr/share/doc from the URL: http:// ip-address /doc/ where ip-address is the IP address or DNS name of your computer. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 197 Finding Documentation 6.14 The exercises The aim of these exercises is for you to learn how to find helpful information, not to just sit here reading it now! The information you need to do each of these exercises is provided for you in this chapter. 1. The Linux Documentation Project provides a number of guides; these are full length, online books. Locate: (a) a book about network administration (b) A book about shell programming using bash 2. Locate the documentation a copy of it. CDROM image on our server for Red Hat 9, and determine how to burn 3. Go to the location for Red Hat mailing lists, select the shrike list, and browse the most recent threaded archive of discussion about Red Hat 9. Subscribe to this list, and set up filtering on your mail client, so that the mailing list information is kept in a separate mailbox from your normal email. See section § 6.4 on page 187. 4. Install a copy of the Red Hat documentation onto your computer from the network filesytem containing the documentation RPMs, and install them. The last step is to read them! Here is how: (a) At a prompt, type: $ cd /home/nfs/redhat-9/doc/RedHat/RPMS $ sudo rpm -Uhv *.rpm (b) Now view this documentation in your web browser at the location: file:///usr/share/doc/ and look for the Red Hat “Getting Started Guide” under rhl-gsg-en-9. Click on the file index.html. I particularly recommend this book for you to get started with Linux. (c) The book Maximum RPM under maximum-rpm-1.0 explains much about how 5. Visit Alan Cox’s “Portaloo” (see section 6.6 on page 189) (a) add all the channels by clicking on “Add Everything” (b) place a bookmark to this site in your browser (c) Note that each rectangle contains current links to headlines on a technical news web site. 6. Visit the slashdot web site at http://slashdot.org/ (a) Click on a “Read more . . . ” link (b) Change your “Threshold” to 3. This means that you only see postings that have a score of 3, 4 or 5. (c) See whether the discussion is sensible or not! (d) Slashdot uses a moderation system, a voting system, where silly posts are “modded down”, (given a lower score), while useful posts are “modded up” (are given a higher score). This is essential, because anyone can post, and there are enough silly people in the world to make Slashdot unusable without such a moderation system. 7. View the on-disk documentation for the bash shell: (a) In the man page (b) using the info bash command (c) using the pinfo bash command c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk RPM works. 6.14 The exercises 198 (d) using info within emacs 8. Determine the location of the documentation for the bash software package using rpm. 9. Determine what software package the file /usr/lib/libc.so belongs to, and find its documentation. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 199 Finding Documentation 6.15 Documentation: Solutions 7. (a) $ man bash (b) $ info bash (c) $ pinfo bash bash: pinfo: command not found $ sudo rpm -Uhv /home/nfs/rh-9-updated/RedHat/RPMS/pinfo-0.6.6-4.i386.rpm Preparing... ########################################### [100%] 1:pinfo ########################################### [100%] $ pinfo bash (d) $ emacs & Then the following keystrokes:  ¨ • Control-h i © start the help system) (to  ¨ ¨ ¨¤ ¨ §  ash •  ©b © © ¥ © select the bash menu item) m   ¦  (to 8. $ rpm -qd bash /usr/share/doc/bash-2.05b/CHANGES /usr/share/doc/bash-2.05b/COMPAT /usr/share/doc/bash-2.05b/FAQ /usr/share/doc/bash-2.05b/INTRO ... /usr/share/man/man1/unset.1.gz /usr/share/man/man1/wait.1.gz 9. $ rpm -qf /usr/lib/libc.so glibc-devel-2.3.2-27.9 $ rpm -qd glibc /usr/share/info/libc.info-1.gz /usr/share/info/libc.info-10.gz /usr/share/info/libc.info-11.gz ... /usr/share/man/man3/sigwait.3thr.gz This software package contains the main libraries you link with when you write programs that interact with the operating system. You can read the info documentation with, well, info: $ info libc c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk Module 7 Administering User Accounts and Permissions with sudo Objectives • After completing this section, you will be able to: ◦ Create user accounts that include the user’s full name, using the industry standard program useradd ◦ Delete user accounts with userdel ◦ Create and delete groups ◦ Add users to, and remove users from groups ◦ Set permissions on files and directories that enable and restrict access to users and groups of users. ◦ Use sudo to run administration programs without becoming root 201 7.1 System Administration without always being SuperUser 202 7.1 System Administration without always being SuperUser • Working as the SuperUser (root) all the time is a disaster waiting to happen. • Example: if you run netscape as root, and download a malicious Java or JavaScript program, it can execute with SuperUser privileges and destroy the server you are working on, and affect your career badly! • Trust me: be root as little as possible. Never run X as root. If you are the system administrator, step one = make your own non-root account. Always log in as yourself, not root. • So how to get work done? • Solution: sudo • Refer to the notes about configuring and using sudo www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 203 Administering User Accounts and Permissions with sudo 7.2 Setting your PATH • When you type a program name, the shell searches a list of directories for it. • The list of directories is stored in an environment variable PATH. • Programs for system administration are kept in directories /sbin and /usr/sbin. (sbin = system binary). • Normally these are not put on your path. • As a system administrator, it is good for you to put these directories onto your path, by editing your login script (not root’s!) ∼/.bash profile and adding the line: PATH=$PATH:/sbin:/usr/sbin • After saving your log in script, source your log in script: $ source ∼/.bash_profile • The alternative is to type the full path name for each system administrator command: $ sudo /usr/sbin/useradd -c "Noris Lurka" nlurka • . . . otherwise you will see something like this: $ sudo useradd user1 sudo: useradd: command not found c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.3 Linux is a Multiuser System 204 7.3 Linux is a Multiuser System • Linux is a multiuser system • This is different from what you have experienced with Microsoft operating systems∗. • Many users can log into the computer at the same time (usually over the network using secure shell—see later) • All can run programs (graphical and in text mode) interactively on the one computer • Each user is protected by the operating system from accidental (and some malicious) damage from other users ∗ Although the terminal server is changing NT into a multiuser system. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 205 Administering User Accounts and Permissions with sudo 7.4 User account overview • Each user belongs to at least one group • Each user has a user id and one group id for each group they belong to • These ids are integer numbers • All account information is stored in the following files: ◦ /etc/passwd maps user name to user id, main group id, full name, home directory, and default shell (usually bash for normal users) ◦ /etc/shadow maps user name to password, holds user password aging policies ◦ /etc/group maps group names to group ids ◦ /etc/gshadow which holds group passwords c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.5 password file 206 7.5 password file • It is a text file • You looked at this file in sections 1.19 and 1.20, and in question 7b on page 29. It is also discussed in some detail in section 17.1 on page 466. • If deleted, no one can log into the system! • It is used when you type ls -l to show user names rather than user ids. • The 7 fields are separated by colons ‘:’ • The manual page is in section 5 (section 5 is about file formats); read it with $ man 5 passwd • In the old days, second field was the password, but not now. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 207 Administering User Accounts and Permissions with sudo 7.6 Example passwd file Here is part of a passwd file: root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin: daemon:x:2:2:daemon:/sbin: adm:x:3:4:adm:/var/adm: lp:x:4:7:lp:/var/spool/lpd: sync:x:5:0:sync:/sbin:/bin/sync shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown halt:x:7:0:halt:/sbin:/sbin/halt mail:x:8:12:mail:/var/spool/mail: news:x:9:13:news:/var/spool/news: uucp:x:10:14:uucp:/var/spool/uucp: operator:x:11:0:operator:/root: games:x:12:100:games:/usr/games: gopher:x:13:30:gopher:/usr/lib/gopher-data: ftp:x:14:50:FTP User:/home/ftp: nobody:x:99:99:Nobody:/: xfs:x:100:102:X Font Server:/etc/X11/fs:/bin/false gdm:x:42:42::/home/gdm:/bin/bash postgres:x:101:233:PostgreSQL Server:/var/lib/pgsql:/bin/bash squid:x:102:234::/var/spool/squid:/dev/null nicku:x:500:500:Nick Urbanik,C440,24368576:/home/nicku:/bin/bash c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.7 group 208 7.7 group • Maps group names to group ids • Maps users to additional secondary or supplementary groups (besides their primary group) • The primary group is the group specified by the fourth field in the /etc/passwd file. • Read the manual page with $ man 5 group • Here is part of a group file: root:x:0:root bin:x:1:root,bin,daemon daemon:x:2:root,bin,daemon sys:x:3:root,bin,adm adm:x:4:root,adm,daemon tty:x:5: disk:x:6:root lp:x:7:daemon,lp mem:x:8: utmp:x:101: xfs:x:102: floppy:x:19: console:x:103: gdm:x:42: pppusers:x:230: popusers:x:231: postgres:x:233: slocate:x:21: squid:x:234: nicku:x:500:pam www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 209 Administering User Accounts and Permissions with sudo 7.8 shadow file • Current Linux distributions default to using the shadow file to hold passwords • If it doesn’t, consider changing to a better distribution! • Holds the following information: ◦ Login name ◦ Encrypted password ◦ Days since Jan 1, 1970 that password was last changed ◦ Days before password may be changed ◦ Days after which password must be changed ◦ Days before password is to expire that user is warned ◦ Days after password expires that account is disabled ◦ Days since Jan 1, 1970 that account is disabled • Must have permissions like this: $ ls -l /etc/shadow -rw------1 root root 1114 Oct 30 12:50 /etc/shadow • Here is a line from an /etc/shadow file: root:$1$CBBmBWbw$OOH0LNS.gH1KXsMX6ACC2.:11175:0:99999:7:::134539268 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.9 logging in 210 7.9 logging in • Many programs for a user to login; examples shown on next slide • A group may also have a password (stored in the /etc/gshadow file) • If a group has a password, and you know that password, you can become a member of that group with the command newgrp • You can change or add a group password with the gpasswd command. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 211 Administering User Accounts and Permissions with sudo 7.10 logging in—Pluggable Authentication Modules (PAM) • Many programs ask for a password. Examples: ◦ login ◦ slogin ◦ telnet ◦ ftp ◦ samba ◦ su ◦ gdm (the Gnome login to X) ◦ xscreensaver ◦ sudo ◦ . . . and many others (see /etc/pam.d/*) • The login programs are very important for security; one small mistake, and your system can be cracked. • To avoid replicating the code in every application, one library handles login for all these programs. Called Pluggable Authentication Modules library (PAM) • Allows total customisation of login, including replacement or supplementing of password file by other systems, such as: ◦ one-time passwords ◦ smart cards ◦ LDAP servers ◦ biometric systems ◦ ... c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.11 Adding User Accounts with useradd 212 7.11 Adding User Accounts with useradd • Many programs exist for adding users • GUI programs: ◦ linuxconf and ◦ userconf • one program is standard across many other Unix systems: useradd • To add the user Chan Hei-man with the user name heiman, and unrestricted password age: $ sudo useradd -c "Chan Hei-man" heiman • To set Chan Hei-man’s initial password: $ sudo passwd heiman Changing password for user heiman New UNIX password: Retype new UNIX password: passwd: all authentication tokens updated successfully • Password is not shown as you type it, of course. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 213 Administering User Accounts and Permissions with sudo 7.12 What happens when you create a user account? • When you create an account with useradd, the program does a number of things: ◦ The program identifies the highest used user ID number, and adds one to this value as the new user ID number ◦ It does the same with the group numbers ◦ It creates a home directory ◦ It copies the default login scripts and other setup files from /etc/skel ◦ It changes the ownership of this home directory to the new user and group ID numbers ◦ It locks the password and group files, updates them with the new user and group entries c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.13 Local accounts and LDAP accounts 214 7.13 Local accounts and LDAP accounts • You have configured your computer to use LDAP authentication ◦ All the user account information is stored on an LDAP server – LDAP = Lightweight Directory Access Protocol ◦ All the information normally stored in the passwd and group files is stored in the LDAP server ◦ LDAP also provides information used by the automounter, a program running on the client that automatically mounts your home directory by NFS when you log in • The automounter manages the directory /home all access to /home is controlled by the automounter • You cannot create local directories in /home • . . . so you need to tell useradd to use another directory when creating local accounts www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 215 Administering User Accounts and Permissions with sudo 7.14 Configuring useradd to create local accounts • First create a new directory where your new local users will have their home directories: $ sudo mkdir /home2 • Next tell useradd where new home directories should be based: $ sudo useradd -D -b /home2 ◦ Note that -D sets the new defaults for future use of useradd ◦ The option -b /home2 selects /home2 as the base of all future home directories created using useradd c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.15 Creating a group 216 7.15 Creating a group • The industry standard for creating a new group is groupadd. • To create the group admin, type: $ sudo groupadd admin www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 217 Administering User Accounts and Permissions with sudo 7.16 Adding a user to a secondary group • The command gpasswd can be used to add a user to a secondary or supplementary group, and also to remove a user from a group. • A secondary group is a group that a user belongs to, which is not their primary group. • You can add heiman to the admin group more simply with: $ sudo gpasswd -a heiman admin • Note that this will not affect any other group memberships heiman already has. • Type: $ man gpasswd for more information c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.17 What groups does this user belong to? 218 7.17 What groups does this user belong to? • To find out what groups you belong to, type: $ groups • To find out what groups a user user belongs to, type: $ groups user • The id command shows information about your groups and user id: $ id uid=2270(nicku) gid=2270(nicku) groups=2270(nicku),14171(staff) • The id command is very useful, and has a number of options. See man id www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 219 Administering User Accounts and Permissions with sudo 7.18 Effective group ID and newgrp • When a person logs in, any files they create are normally owned by their primary group. • A user can change their effective GID to another group that they already belong to, using the newgrp command • The effective GID is shown on the left when you type groups. • When you create a file, the group owner is the same as your effective GID. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.19 Directory for a Group Project 220 7.19 Directory for a Group Project • As system administrator, you may need to provide a directory that a group of people can use for a project, so that all can read and write that directory, but access by others is restricted. • Specifications: ◦ directory name is /var/project ◦ members of the group admin are allowed read and write access to /var/project ◦ other users cannot read or write or change into the directory. • Change the ownership of the directory so that it is owned by the user root and has group owner admin: $ sudo chown root:admin /var/project $ ls -ld /var/project drwxr-xr-x 2 root admin 4096 Dec 21 12:27 /var/project • Change the permissions of the directory so that the user root and the group owners have read, write and execute permission, and other users have no access: $ sudo chmod ug=rwx,o= /var/project # or sudo chmod 770 /var/project $ ls -ld /var/project drwxrwx--2 root admin 4096 Dec 21 12:27 /var/project • See section 5.18 on page 157 for more about file permissions and chmod. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 221 Administering User Accounts and Permissions with sudo 7.20 File permissions for directories • It is easy enough to understand the read, write and execute permission when applied to an ordinary file; • What about for directories? • The execute permission is the permission to use the cd command to change into the directory. • The read permission is the right to list the contents of the directory with ls • The write permission is the write to change entries in the directory • What is a directory entry ? For each file in the directory, there are only two items: ◦ The file name ◦ The inode number So the write permission is the right to change either of these. • This write permission includes: ◦ The right to create, delete or rename a file ◦ The right to change the inode number c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.21 Examples of minimum file permission requirements 222 7.21 Examples of minimum file permission requirements • This table shows examples of minimum file permission requirements • It is based on table 2-2 on page 37 of the book Essential System Administration by Æleen Frisch, O’Reilly 2002. • Buy this excellent book! SysAdmins can’t work well without it! Command cd /var/project ls /var/project ls -l /var/project cat /var/project/user1.txt echo "hello" >> /var/project/user1.txt /var/project/binary-program /var/project/script-program rm /var/project/user1.txt minimum access required on the file on the directory no file ----r--w--x r-x ----x r-r-x --x --x --x --x -wx www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 223 Administering User Accounts and Permissions with sudo 7.22 Set Group ID Directory • There are three “special permissions” that can apply to a file. You can read about them if you type: $ info chmod $ info "(coreutils)File permissions" See section 6.7 on page 190 for more about info. • Here we look at the “set group id” (SGID) permission for directories. • If this permission is set on a directory, then: ◦ if a user makes a change to a file or creates a file in that directory, the file will have group owner the same as the directory. ◦ If a user creates a directory in that directory, it too will have the Set Group ID bit set. As with the file, it will have a group owner the same as the group owner of its parent SGID directory. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.23 Set Group ID Directory — Example 224 7.23 Set Group ID Directory — Example • Let’s see the result of creating a file in /var/project both before and after adding this permission: $ ls -ld /var/project drwxrwx--2 root admin $ touch /var/project/test1 $ ls -l /var/project -rw-rw-r-1 nicku nicku 4096 Jan 2 13:48 project 0 Jan 2 13:53 test1 Now we add the set group ID bit to the directory permissions, and see the effect: $ sudo chmod g+s /var/project $ ls -ld /var/project drwxrws--2 root admin $ touch /var/project/test2 $ ls -l /var/project -rw-rw-r-1 nicku nicku -rw-rw-r-1 nicku admin 4096 Jan 2 13:48 project 0 Jan 0 Jan 2 13:53 test1 2 13:54 test2 • when the directory is not SGID, files I create have my primary group ID. • when the directory is SGID, files I create have the group ID of the directory. This allows others in the group to read, write and change the files created by any group member. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 225 Administering User Accounts and Permissions with sudo 7.24 User Management Exercises 1. Configure your PATH as described in section 7.2 on page 203. You should already have done that. 2. Configure sudo as discussed in the handout on sudo. Again, you should already have done that. 3. Configure the new default base for local home directories, as discussed in section 7.14 on page 215 4. Create four local user accounts, and create two additional local groups, using sudo useradd and sudo groupadd. Set their passwords using sudo passwd username. 5. Add two users to one of the groups (make this their secondary group). Similarly, add the remaining 2 users to the other group (make it their secondary group). Do not change the users’ effective group ID using the newgrp command in these exercises: simply leave their effective group ID as their primary group. 6. Create a directory /var/project, using sudo of course. 7. Set the permissions and ownership as shown in 7.19 on page 220: $ ls -ld /var/project drwxrwx--2 root admin 4096 Jan 2 13:48 project Note: instead of admin, type the name of one of the groups you made in step 4. 8. Log in as one of the users who belongs to the same group as the one who owns the directory /var/project by typing: $ su - username 9. As the user that you logged in as in step 8, Create a file in the directory /var/project. (a) What are the permissions on that file? (b) Which user owns the file? (c) Which group owns the file? 10. In another window, log in as the other user who is also in the same group that owns the directory /var/project. Edit the same file as you created in step 9 and save it. Are you successful? Why or why not? 11. In another window, log in as the other user who is not in the group that owns the directory /var/project. Edit the same file as you created in step 9 and save it. Are you successful? Why or why not? 12. Now set the permissions on the directory to be SGID as shown in 7.22 on page 223. Create a second file as the first user, in the group that owns the directory. What are the permissions and ownership of the second file? 13. Can you edit and save the second file as the second user, who is also in the same group that owns the directory? Why or why not? 14. Can you edit and save the second file as a user who is also not in the same group that owns the directory? Why or why not? What access rights do you have as a user from a group that does not own the directory? 15. Additional exercise: marked by your supervisor on a scale of 0 to 4. You have thirty minutes to complete this exercise: Requirements: c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 7.24 User Management Exercises 226 • you are to create three user accounts and one directory. • two users should have read and write access to the directory, and if one user saves a file, the other must be able to edit and save the file. So if these two users are user1 and user2, then if user1 creates a file in the directory, then user1 and user2 should have read and write access to that file, but not other users should have write access. • No other user (including the third you created) should have write access to the directory; they should, however, be able to change into the directory and list the content of the directory, and should be able to read any files created in that directory by the first two users. • No manual intervention is required by any of the users. It should all just work. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 227 Administering User Accounts and Permissions with sudo 7.25 User Management Solutions 1. $ echo $PATH /usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/nicku/bin Oh dear, the directories containing system administrator commands are not on my PATH. Better add them. $ echo PATH=$PATH:/usr/sbin:/sbin >> ∼/.bash_profile $ . ∼/.bash_profile $ echo $PATH /usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/nicku/bin:/usr/sbin:/sbin 2. Follow the procedures given in the handout on sudo. This is how a competent senior system administrator will configure sudo for you when delegating a system administration task to you. If you are the senior system adminstrator, then you can configure sudo simply by adding a line like this: nicku ALL=(ALL) ALL (substituting your own username instead of nicku, of course). This means that user nicku can execute all of the commands as any user (the user is specified in the parentheses), and can execute any command using sudo (the program list is given by the last ALL). 3. $ sudo mkdir /home2 $ sudo useradd -D -b /home2 Then when we add user accounts from now on, the home directories will go under /home2 instead of under /home. We do this because the automounter controls access to the directory /home, and only the home directories for network accounts can be there. 4. There are a number of ways of doing this and the next step; I will show a few different ways: $ $ $ $ $ $ $ $ $ $ sudo sudo sudo sudo sudo sudo sudo sudo sudo sudo groupadd g1 groupadd g2 useradd u1 -G useradd u2 -G useradd u3 -G useradd u4 -G passwd u1 passwd u2 passwd u3 passwd u4 g1 g1 g2 g2 Or you can do it in two steps: $ $ $ $ $ $ $ $ $ sudo sudo sudo sudo sudo sudo sudo sudo sudo groupadd g1 groupadd g2 useradd u1 useradd u2 useradd u3 useradd u4 passwd u1 passwd u2 passwd u3 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 7.25 User Management Solutions 228 $ $ $ $ $ sudo sudo sudo sudo sudo passwd u4 usermod u1 usermod u2 usermod u3 usermod u4 -G -G -G -G g1 g1 g2 g2 Or, if you have read the bash man page, and want to try some shell for loops, you could do it like this: $ $ $ $ for for for for g u u u in in in in g1 g2; do sudo u1 u2; do sudo u3 u4; do sudo u{1,2,3,4}; do groupadd $g; done useradd -G g1 $u; done useradd -G g2 $u; done sudo passwd $u; done bash will expand u{1,2,3,4} to u1 u2 u3 u4: try $ echo u{1,2,3,4} 6. $ sudo mkdir /var/project 7. $ sudo chown root:g1 /var/project Note that the chown command has the user, then group owner of the file separated by a colon (or a dot). We can specify the permissions of the file numerically or symbolically: $ sudo chmod 770 /var/project or $ sudo chmod ug=rwx /var/project 8. Open another window (or a new tab in your terminal window), then type: $ su - u1 Note the minus sign - here, which means: “execute the login scripts of the user.” Otherwise, many of the environment variables will remain as they were for your own account. 9. $ cd /var/project $ echo "I am user 1" > user1.txt $ ls -l user1.txt -rw-rw-r-1 u1 u1 12 Jan 16 14:45 user1.txt (a) The permissions are read and write for the user that owns the file (for u1), and for the group owner (the group u1), and read for all other users. (b) u1, the user that created it. (c) u1, the effective group ID of the user that created it. 10. Open a new window (or a new tab in your terminal window), and type: $ su - u2 $ cd /var/project $ vi user1.txt then insert some more text into the file, and try to save with the vi command :wq As u2, I can change into the directory and open the file in an editor, but I cannot save it, unless, in vi, I type :w! www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 229 Administering User Accounts and Permissions with sudo This is because u2 is not the user u1 that owns the file, is not a member of the group u2 that owns the file, so the permissions that apply to u2 are those of “other”, i.e., only the read permission. Note that user u2 has permission to read, write and change into the directory /var/project, since u2 is a member of group g1, the group owner of the directory, hence the directory permissions that apply to user u2 are those of the group, i.e., read, write and execute. 11. (Open another window for u3 first, to save time later.) As user u3, I do not have permission to change into or list the directory /var/project. Even though it seems I have read permission on /var/project/user1.txt, I cannot do this as u3: $ cat /var/project/user1.txt and get anything more than a “permission denied” message Since u3 is not a member of group g1, the group owner of the directory, and is not the user root that owns the directory, hence the directory permissions that apply to user u3 are those of “other”, i.e., no permissions. The user u3 needs to have at least execute permissions on /var/project before $ cat /var/project/user1.txt will display the contents of the file. 12. Click on the window where you, the administrator, is logged in, and do: $ sudo g+s /var/project and click back on the window for u1, and do: $ echo "I am user 1 back again" > user1-second-try.txt The permissions are the same as before. The user that owns the file is u1, the same as for user1.txt. Only the group owner of this new file is different: it is g1, the same group that owns the directory. 13. I have no trouble to edit and save the file user1-second-try.txt as user u2. When I save it, the permissions are unchanged, the group owner is still the same as the group owner of the directory, but the user that owns the file is now u2. Note that user u1 can continue to edit the same file user1-second-try.txt. 14. As u3, I still have no access rights to the directory or the files in it. The reasons are the same as for question 11. 15. This is the same set up as before, execpt that the permission on the directory is 775 instead of 770. Note that it is possible for the users u1 and u2 to type: $ newgrp g1 to change their effective group ID to g1 before they create the files in the shared directory. They can then easily edit each other’s work. However, it is better as the system administrator to demand less of the users (that takes less of your effort!) and to make things easier for them to manage. These directory sharing systems all depend on each user having their own private group to which noone else belongs, and the umask being 002 instead of 022 as on other non Red Hat type systems. This is sometimes called the Red Hat private user group scheme. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk Module 8 Managing Users–quotas Objectives On completion of this module you should be able to: • Understand key Unix user management issues • Use key Unix user management tools ∗ Many user management issues and tools are dealt with in other GBdirect modules, e.g. admin of users, passwords, groups, permissions, etc is covered in the Key Configuration Files module and the Filesystem modules. This module only covers points not dealt with there. 231 ∗ 8.1 Checking /etc/passwd and /etc/shadow with pwck 232 8.1 Checking /etc/passwd and /etc/shadow with pwck • pwck verifies the integrity of password files • It checks: ◦ Number of fields ◦ Uniqueness of user names ◦ Validity of user and group identifiers ◦ Validity of primary groups ◦ Validity of home directories ◦ Validity of login shells • Typically, offers to delete error-ridden or duplicate lines • Checks both /etc/passwd and /etc/shadow by default • Can be run in read-only mode (-r) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 233 Managing Users–quotas 8.2 Checking /etc/group with grpck • grpck checks /etc/group for: ◦ Correct number of fields ◦ Uniqueness of group names ◦ To ensure all group members are defined in /etc/passwd • Typically prompts to delete error-ridden or duplicate lines • Can be run in read-only mode (-r) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 8.3 Managing User Connections: login, /etc/securetty, /etc/usertty 234 8.3 Managing User Connections: login, /etc/securetty, /etc/usertty • login configures user connections ◦ Most notably by setting what’s available at login time ◦ Part of the Shadow software suite ◦ Works closely with /etc/login.defs ◦ See man login • /etc/securetty restricts the terminals that system administrators can use ◦ One device name per line (no /dev prefix) ◦ Consulted by login • /etc/usertty restricts general user access ◦ According to user ID and/or time of login ◦ One rule per line, 3 colon separated fields 1. Terminal (no dev prefix), * means all 2. Users, * means all 3. Lists of allowed times – Times in HHMM format, e.g. 0800-1732 – Days Su Mo Tu We Th Fr Sa Wk (mon-fri) Al (all days, default) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 235 Managing Users–quotas 8.4 Limiting User Resources with ulimit • ulimit prevents users exhausting system resources ◦ By limiting access to those resources • Built-in to bash and Ksh shells • Two types of restriction: ◦ Soft limits, set default resource usage when a process is created, variable ◦ Hard limits, set upper threshold soft limits can’t exceed • ulimit -a displays current soft limits • ulimit -Ha displays hard limits • There are about a dozen more options ◦ See man ulimit for details c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 8.5 Managing Disk Use with Quotas 236 8.5 Managing Disk Use with Quotas • Limit the amount of fs storage a user can consume • Available on most Unix systems ◦ As optional patch to Linux kernel ∗ ◦ Not on SCO UNIX • Two distinct types: ◦ Hard limits can never be exceeded ◦ Soft Limits allow temporary excesses, e.g. – For a period of time – For a number logins • Typically applied to /home filesystems • NOT /tmp or / • Main Commands: Command quotaon, quotaoff quota repquota quotacheck edquota quotactl Description turn file system quotas on and off display disk usage and limits summarize quotas for a file system scan a file system for disk usages edit user quotas manipulate disk quotas Must be made available either by choosing to enable quotas at installation time, or by compiling the option into a new kernel. Kernel compilation is the subject of another GBdirect training module entirely. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 237 Managing Users–quotas 8.6 Setting up Quotas on a Filesystem • Set quotas on a filesystem in the 4th field of /etc/fstab ∗ # device /dev/hda1 /dev/hda2 /dev/hda3 /dev/hdb1 /dev/hdb2 none directory / none /usr /usr/users /usr/src /proc type ext2 swap ext2 ext2 ext2 proc options defaults sw defaults defaults,usrquota,grpquota defaults,usrquota defaults • Create quota records quota.user and quota.group at the root of relevant filestems: † $ cd filesystem $ su Password: $ touch /partition/quota.user $ touch /partition/quota.group $ chmod 600 /partition/quota.user $ chmod 600 /partition/quota.group $ halt -r now BSD introduced quotas and configured them in this way. See man fstab for details. Most actively developing Unixes e.g Linux, FreeBSD, NetBSD, etc follow the same pattern. System V format fs config files simply change rw to rq. AIX puts “quota = userquota,groupquota” in /etc/filesystems. † Record files quota.user and quota.group, should be owned by root, and read-writeable by root alone. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk ∗ 8.7 Specifying Quotas for Users and Groups 238 8.7 Specifying Quotas for Users and Groups • Set Users’ quotas using edquota • Invoking it with a user or group name creates a tmp file containing hard and soft limits for them $ edquota username(s) • Then opens the file with the editor specified in $EDITOR environment variable • Each line describes one filesystem ∗ Quotas for group www: /dev/hda4: blocks in use: 5349, limits (soft = 8000, hard = 10000) inodes in use: 1745, limits (soft = 3000, hard = 4000) • If saved before exit, the editor auto-writes details to the quota records • Can be used just on the command line, e.g. $ edquota -p davef lee julie Sets Julie and Lee’s quotas to match Dave’s • Options: Command edquota -u edquota -g edquota -t edquota -p username Meaning Set individual user quotas Set group quotas Set grace period in days, hours, minutes or seconds Set others’ quotas equal to match username’s ∗ Formats vary between Unixes, the example above is from Linux. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 239 Managing Users–quotas 8.8 Checking and Reporting on Quotas • Use quotaon to activate quota system and enable quota checking quotaon filesystem quotaon -a Enable quota system on specified fs Enable on all filesystems • quotaoff does the obvious • quotacheck looks for consistency ◦ Within quota records ◦ Between records and current disk usage quotacheck filesystem quotacheck -a Check consistency on specified fs Check on all filesystems • quotaon -a and quotacheck -a should run at boot time, i.e. be in system init scripts • repquota reports the current quotas for specified filesystems ◦ Can take multiple filesystems as arguments ◦ Can report on all filesystems (-a) • quota gives ordinary users basic info on current quota status c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 8.9 Managing Users–quotas: Exercises 240 8.9 Managing Users–quotas: Exercises 1. Checking Password and Group Files (a) Use pwck and grpck to check their respective user detail files, twice. i. Begin in read-only mode. ii. Switch to write-mode if you find errors, (b) If you found no errors, assume super-user status and introduce some. Then try these commands in write-mode. N.B. Don’t mess with the root user’s details 2. User Connections (a) Create a new fictitious user on your system (b) Edit /etc/usertty to prevent their access at particular times and terminals (c) Attempt to login as the fictitious user at a “banned” terminal/time. 3. Quotas (a) Set quotas for a fictitious new user on their /etc/home directory (b) Use the quota checking tools to test your set up (c) Try to figure out and implement a practical test to see that it is really working on a live system. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 Module 9 Introduction to Editing With vi Objectives In this section, you will learn how to: • Use the vi editor to view, create and edit files ◦ the vi screen layout ◦ move round in files ◦ replace, insert and change text ◦ search files 241 9.1 Text editors under Linux 242 9.1 Text editors under Linux • There are a number of text-editors available • vi is on virtually every Linux distribution • Also comes with 99% of Unix systems • Everyone should have a basic understanding • vi is like Linux ◦ Has some very complex and powerful functions that can make your life easier ◦ However, you don’t have to know everything; you get by knowing the basics ◦ Shares key bindings with many utilities • We’ll just cover the basics here, vi is too big to cover everything! www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 243 Introduction to Editing With vi 9.2 vi and your terminal • vi is fundamentally text-based ◦ Graphical adaptations are available (gvim) • Needs to know your terminal’s capabilities ◦ May not function if your terminal is misconfigured ◦ Check your TERM environment variable ◦ Terminal capabilities are listed in /etc/termcap ◦ Generally not an issue except with Windows 98 telnet client. When use windows telnet, always type export TERM=vt100 before starting vi and other such programs, or they will not work properly. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.3 vi screen layout 244 9.3 vi screen layout • Lines containing simply a ~ show that you are past the end of the file and there is nothing here. • The terminal’s bottom line is the status line ◦ Shows status messages ◦ Where you type some commands (The ‘ed’/‘ex’ command set, explained later) This is a test document Some lines of text here One, two, three four, five, six ~ ~ ~ ~ ~ ~ "test" 5 lines, 77 characters written www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 245 Introduction to Editing With vi 9.4 Opening files with vi • Launch vi by typing its name on command line • With no arguments vi starts with an un-named and empty buffer • vi filename opens a specific file • If you don’t have write permission on a file the status line will tell you : "/etc/aliases" [readonly] 152 lines, 3215 characters • If there is no such file status line will say something like : "some_filename" [New File] c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.5 vi Modes 246 9.5 vi Modes • Unlike many editors vi does not always insert what you type into the file • Has several ‘modes’ ◦ Only one is responsible for inserting text into the current file • vi has 3 modes: ∗ command mode Moving the cursor, searching and manipulating existing text insert mode Entering new text ‘:’ (‘ed’) mode File manipulation, advanced searching and substitution • vi starts in command mode • Return to command mode at any time by hitting ∗ Some people refer to ”ex” instead of ”ed”. They are the same thing www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 247 Introduction to Editing With vi 9.6 Saving, changing file and quitting • When you open a file, a copy of it is opened into memory • Any changes you make apply to this copy only • File on disk only changes if you explicitly say so • To save (or write) a file you must be in command-mode, then type :w • Can save your file under a new name, e.g. :w newfilename • To quit vi type :q • vi normally prompts you if you have unsaved work • To quit without saving your work type :q! • ZZ will save your work and then quit c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.7 Moving around in command mode 248 9.7 Moving around in command mode • Many ways to move around a document • You must be in command mode for the following : ◦ On ‘friendly’ terminals you can use arrow keys ◦ Arrow keys are sometimes unavailable on some terminals so vi has some alternatives k h l j • Although ‘awkward’ at first, these make your life easier ◦ Always work, regardless of system type ◦ Fingers stay on the ‘home’ keys www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 249 Introduction to Editing With vi 9.8 Numeric Prefixes • Key concept: ‘numeric prefixes’ or ‘multipliers’ ◦ Vastly improves the usefulness of many commands • To supply a prefix simply type the number before the command ◦ vi will then perform the command the specified number of times. • Note: In subsequent examples a small box indicates the position of the cursor • Starting with The quick brown fox jumped over the lazy dog and pressing 2l will result in The quick brown fox jumped over the lazy dog c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.9 Further Movement 250 9.9 Further Movement • vi also allows movements by units other than characters. • Moving by pages : Key ^f ^b ^u ^d Result Forward one screenful Back one screenful Forward half a screenful Back half a screenful • Moving by ‘words’ : Key w e b Result Go to beginning of next word Go to end of next word Go to start of previous word • For these commands punctuation is not counted as part of a word • The commands W, E and B act the same but do include punctuation in words • NOTE: Case is important to vi commands, b and B are different commands! ◦ The upper and lower case versions of commands are usually related www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 251 Introduction to Editing With vi 9.10 Further Movement — Example • From This, he said, is an example Key w W e E b B Result This, he This, he This, he This, he This, he This, he said, said, said, said, said, said, is is is is is is an an an an an an example example example example example example • As with virtually all commands these may be given a numeric prefix • From the original start-point : Key 2w 2W 2e 2E 2b 2B Result This, he This, he This, he This, he This, he This, he said, said, said, said, said, said, is is is is is is an an an an an an example example example example example example • It’s not necessary to know these, but they make life a lot easier when you get used to them! c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.11 Movement by lines 252 9.11 Movement by lines • What if we want to get to the beginning of the next line∗? • Commands to move to line start/end: Key $ 0 ^ Result Move to the end of the current line Move to start of current line Move to first character of line • Moving to start of a previous or subsequent line Key + Result Move to beginning of the next line Move to beginning of the previous line A ‘line’ is the set of characters contained between newline characters, not necessarily what appears on one line in your terminal www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 253 Introduction to Editing With vi 9.12 Movement by lines — Examples • In each case here, we start from: This, he said, is a most interesting example Key Result + This, he said, is a most interesting example This, he said, is a most interesting example 0 or ^ This, he said, is a most interesting example $ This, he said, is a most interesting example c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.13 Inserting text 254 9.13 Inserting text • You probably want more from a text editor than the ability to move a cursor! • At the bare minimum you need to be able to insert text into a file • Don’t worry, vi does this with ease • As with everything else, though, there’s more than one way • Again, while this may seem confusing, you only need to know the bare minimum • But, the more you know, the easier your life becomes! www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 255 Introduction to Editing With vi 9.14 i command • The i command inserts text before the cursor • This places vi into ‘insert’ mode • Anything you type now is treated as text to insert into the file rather than as a command • You leave insert mode by typing • This is insertion at its simplest! • To insert text after the cursor we use the a (append) command • Also : Key A I o O Result Append at the end of the line Insert at the beginning of the line Create blank line below cursor for insertion Create blank line above cursor for insertion • If your cursor keys work then you may move around the line while in insert mode • You can delete characters from the current insertion using backspace c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.15 Multiple Insertion 256 9.15 Multiple Insertion • Insertion commands can take numeric prefixes • The result may be surprising! • Consider the following sequence of keypresses (from command mode) in an empty document 5i  • The result will be :       www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 257 Introduction to Editing With vi 9.16 Deleting Text • vi has a vast array of commands for deleting text • The ‘odd-one-out’ is x which deletes the character under cursor • The rest of the deletion commands are based-around the easy to remember d command • d on its own does nothing • You have to tell it how much to delete • The amount to delete is given by the keys you used when studying movement Example: Key dw 3dw de db d$ d0 Result Delete to the beginning of the next ‘word’ Delete 3 ‘words’ Delete to the end of the ‘word’ Delete everything before cursor to the beginning of the ‘word’ Delete to the end of the line Delete to the beginning of the line • Two more special cases : Key dd D Result Delete the entire line Delete to the end of the line c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.17 Changing Text 258 9.17 Changing Text • Now we know everything we need to know to delete text, insert new text and save changes • vi however likes to give us choices! • If we find a word that is wrong, we can delete it and insert the replacement • We’re actually ‘changing’ the word • vi has a family of commands for just this, all starting with c • Similar to deletion, i.e. you can use cw to change a word, c$ to change to the end of the line, or 3cw to change three words • What actually happens is that the designated amount is deleted and you are placed in insert mode Key cw 3cw c$ c0 Result Change a word Change 3 words Change to the end of the line Change to the beginning of the line www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 259 Introduction to Editing With vi 9.18 Copy and Paste • We’re still missing the ability to copy a piece of text and paste it somewhere else • vi does support this, but it calls it ‘yanking’ and putting • All ‘yanking’ commands are prefixed with a y and follow the same rules as before, i.e. yw, y$, 3yw • yy and Y yank a whole line and the rest of a line, respectively • Paste text using p or P ◦ p pastes text after the cursor ◦ Uppercase P pastes it before • Deleted text is also considered to be yanked ◦ xp will transpose two characters c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.19 Finding your place 260 9.19 Finding your place • You can search through a file using / • You will get a / as a prompt on your status line • To search for the string exam type /exam and press • If vi found your search string it will move the screen to a relevant place and highlight it • n will skip to the next occurence; N to the previous • Search backwards by using ? instead of / www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 261 Introduction to Editing With vi 9.20 Miscellaneous Commands • vi has a number of commands that don’t really fit anywhere else • ~ toggles the case of character under cursor • . repeats the last action • u undoes the last action ◦ Linux vi supports multilevel undo ◦ Standard vi does not • J Join the current and following line c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.21 Search and replace 262 9.21 Search and replace • vi can also replace the words it finds • Basic form is: s/searchfor/replacewith/modifier • By default it only changes one occurence per line, and only checks the current line ◦ If we tag the g modifier on the end it will replace all matches on the current line • If we use a range∗ we can search and replace a specified part of a document, e.g. ◦ To search and replace from lines 10 to 15 inclusive: :10,15 s/foo/bar/g ◦ To search and replace on the whole document :1,$ s/foo/bar/g ∗ Not explained here, this is an advanced topic www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 263 Introduction to Editing With vi 9.22 Regular Expressions • Sometimes it’s desireable to search for a word ‘fuzzily’ • You may know the start of a word, or the end ◦ Or both, but not the bit in the middle! • Regular expressions can come in useful here • Can be used in normal searches or ‘search and replace’ commands c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.23 Regular Expression Conventions 264 9.23 Regular Expression Conventions • Lots of things in Linux use regular expressions ◦ Not all ‘exactly’ the same ◦ 95% similar though • Defines certain special characters Character . [a-z] * ^ $ \< \> Result Match any character Match any character in the range a to z Match the preceeding character zero or more times Match the beginning of a line Match the end of a line Match the beginning of a word Match the end of a word • Strictly speaking * can apply to more than one character ◦ We won’t cover that here www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 265 Introduction to Editing With vi 9.24 Regular Expression Examples • Suppose we want to find all words ending in ent • We could read the entire document to check for ent by hand ◦ Takes far too long ◦ We’d probably still miss some ◦ Easier to get the computer to do it • We could do a search using /ent ◦ Unfortunately that would also match words beginning with ent or with ent in the middle • /ent\> will jump to the next word that ends with ent c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 9.25 Regular Expression Replacement 266 9.25 Regular Expression Replacement • We can also use regular expressions in the search section of search and replace commands, e.g. s/\ first. vi /etc/passwd will start vi with /etc/passwd opened (c) vi should tell you that this file is read only. This is because you don’t have sufficient permissions to change the file. vi should also tell you how many lines and characters are in the file. 2. Getting used to vi (a) vi /etc/passwd (b) You should be fairly comfortable with the various navigation methods such as moving left, right, up and down, to the end or beginning of the line and moving up and down by intervals of pages and half pages. 3. Creating with vi (a) vi vi test. The status line should tell you that it is a new file and each line on the main screen should begin with a ~ indicating lack of content (b) To insert my name simply type: imy name There are several ways to save and exit: i. :w followed by :q ii. :wq iii. ZZ (c) vi vi test and check that the text you entered is there. If not try again. (d) There are several ways to do this: i. When inserting using i you may type RETURN to insert a newline character. it is possible therefore to start with the cursor at the beginning of the file and type: iname1name2name3 and so on ii. Typing o or O will open a new line for insertion (e) You should check that you understand which of o and O inserts above, and which below the current line (f) Appending to the end of a line can be done using either: i. $atext to append ii. Atext to append Inserting at the beginning can be done using any of: i. ^itext to insert ii. 0itext to insert iii. Itext to insert 4. Movement and Multipliers (a) You should practice moving around using the movement characters with the numerical prefixes i. 3j, 3+, or 3 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 271 Introduction to Editing With vi ii. 3e, or 2w iii. 2+, or 2 5. Deleting with vi (a) You should make sure that the various deleting methods work as you expected. If they surprise you, try to work out how they do work. (b) Again check you understand the various possibilities. (c) The following represent only possible solutions: i. ii. iii. iv. v. vi. dw d$ d0 dd 2dd 2dw 6. Changing with vi (a) The answers for this are the same as for delete except substituting c for d in each case. 7. Yanking and Pasting (a) Move to the first line of the file and type yy, then move to the end of the file and type p (b) Move back to the top line of the file and type P which will paste it above the current line. (c) Check you can tell the difference between the two commands. (d) The ‘Yank buffer’ only holds the contents of one yank operation. Both sets of keypresses yank the line we start on and the line below. However the first does this as two seperate operations and the ‘yank buffer’ only remembers the most recent. The second example yanks two lines at once, therefore placing both in the yank buffer. (e) You should try ddp and check that the text appears after being deleted. 8. Miscellaneous (a) . repeats the last action. In this case it is a paste operation. It could equally well have been an insert, change word or delete operation. (b) The xp command pair is useful for transposing letters. (c) Starting at the top of your file pressing J will join the following line to the current line. Repeat this until the entire file is on one line. Pressing one of the down a line keys (Such as j, + or should have no effect despite the illusion that there is more than one line. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk Module 10 Basic X-Windows Objectives • On completion, you should be able to: ◦ Understand the basic concepts behind networked X windowing ◦ start and stop X ◦ run shells and user applications under X ◦ set preferences for X ◦ change window managers and desktops ◦ use X over a network 273 10.1 What X-Windows Is 274 10.1 What X-Windows Is X is a windowing system • Provides the basic graphic functions for Linux • Designed to provide windowing to any workstation across a network, regardless of OS • Operates on a client-server model • Is an application, i.e. not a part of the OS • The standard Linux X server is Xfree86, commercial alternatives include: ◦ Metro-X ◦ Accelerated-X www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 275 Basic X-Windows 10.2 X Needs Window Managers • Window managers provide the controls which allow you manipulate all graphic apps, e.g. ◦ move, size and stick ◦ open and close ◦ maximize, minimize, iconize ◦ title bars • Determine the ‘look and feel’ of X, e.g. ◦ Win95 ◦ Motif ◦ Next Step • Can provide ‘virtual desktops’ c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 10.3 Window Managers Are Applications 276 10.3 Window Managers Are Applications • Linux distributions contain many window managers, e.g. Manager sawfish fvwm2 fvwm2 twm olwm olvwm WindowMaker enlightenment wm2 Description Used with Gnome, lightweight, programmable Motif-like look Win95-like configuration for fvwm2, Red Hat default Bare-bones Tab WM Open Look (Sun) Virtual Screen Open Look Next Step look, fast and lean Gnome-compatible WM, powerful, rich, heavy weight Ultra-lean • Window managers are X applications, thus: ◦ change manager without re-starting X ◦ change X behaviour without re-start www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 277 Basic X-Windows 10.4 Desktop Environments • X + WM alone don’t provide everything expected of modern desktops, e.g. ◦ completely integrated drag and drop ◦ universal access to a clip board • Desktop Environments bring these facilities to Linux, bundling: ◦ desktop-capable window manager ◦ URL-based file manager ◦ facilities to share clipboard and other data between optimized apps (inc. object linking) • Linux currently has 3 main desktop environments: ◦ CDE . . . the original commercial UNIX standard ◦ KDE ◦ GNOME — now adopted by Sun Microsystems as the window desktop for Solaris. Helix Gnome is excelent and is available from http://www.helixcode.com. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 10.5 Starting X 278 10.5 Starting X • Many possibilities ◦ You may be using the graphical xdm tool which does it for you ◦ or login to command prompt then type $ startx if .xinitrc is not setup: $ xinit $ window manager filename www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 279 Basic X-Windows 10.6 Stopping X • Stopping: ++ • Use the window manager’s menus • If you started via xinit, type the following in the startup xterm: $ exit • If all these fail, switch to another virtual terminal using the following keys, then kill X from the command prompt: ++ ... c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 10.7 Running Shells (Xterms) Under X 280 10.7 Running Shells (Xterms) Under X • Even under X, the most productive way to work is often via the command line (i.e. a shell) • The standard way to access a shell prompt under X is via a terminal emulator called an xterm∗ • An xterm shell behaves like a non-X shell, except that you can cut and paste between it and X applications • Any number of xterms can be open at the same time • Using ssh (or if you are desperate, telnet) the xterms can provide shells to any number of other hosts • To start an xterm: ◦ From an already open xterm: $ xterm & ◦ From a window manager menu (invariably top-level) Linux provides other terminal emulators for specialised hosts, but they are rarely necessary. There is also another category of emulators that provide advanced features such as transparent terminals etc. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 281 Basic X-Windows 10.8 Running Applications from an xterm • Character-based apps: ◦ Run exactly as they would outside X, unless the xterm itself has been misconfigured • X applications: ◦ Type the program’s file name at the prompt: ∗ $ filename & ∗ The ampersand allows the application to run independently of the shell www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 10.9 Running Applications from a window manager 282 10.9 Running Applications from a window manager • Every window manager provides simple menu- based access to applications • Application Menus are usually accessible by clicking Mouse Button 1 on: ◦ Buttons set into a task bar ◦ The desktop background (root window) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 283 Basic X-Windows 10.10 Configuring X • Default installations of Linux provide a fully functional setup for using graphic X apps • 2 different types of X configuration that system administrators or users may need to change: • Basic configuration of screen, mouse, keyboard behaviour, fonts ◦ Could be a course in itself (classic O’Reilly manual fills a bookshelf) ◦ Configuration files best edited via config tools (see next Section 10.11) • Behaviour of desktop objects (windows, icons, taskbars, xterms) ◦ Window manager dependent ◦ Best configured via window manager preferences c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 10.11 Basic X Hardware Configuration 284 10.11 Basic X Hardware Configuration • Basic configuration for hardware is defined in the XF86Config file, located in /etc/X11 ∗ • XF86Config is easier to edit using the following tools: ◦ Xconfigurator . . . Red Hat tool sets monitor, card, screen mode, colour depth and resolution with probing ◦ XF86Setup . . . an X application which edits most basic hardware preferences (Mouse, Keyboard, Card, Monitor, Graphic Modes) ◦ xf86config a character-based application which prompts for the same settings ◦ mouseconfig . . . Red Hat tool sets the mouse type with probing. Useful for setting 2-button mice to emulate 3-button types by simultaneous clicking on both buttons On some older systems you may find the /usr/lib/X11,/usr/X11R6/lib/X11 or /var/X11R6/lib/ www.linuxtraining.co.uk ∗ X configuration in c GBdirect Ltd and parts c Nick Urbanik, 2004 285 Basic X-Windows 10.12 Basic X Software Configuration • Under X, the user can configure every conceivable aspect of graphic display • Users may need to change: ◦ Screen font sizes, styles, familes ◦ Pointer behaviour ◦ Screen colours ◦ Window manager • All desktop environments and many window managers provide graphic tools for changing these configurations • They can be set, on a system-wide or per-user basis, in the following two files: • .xinitrc ◦ to set the default window manager and style to be used by the startx command • ∼/.Xresources or ∼/.Xdefaults ◦ for fonts, pointers, colours, etc ◦ Read this file after changing it with xrdb ∼/.Xresources c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 10.13 Networked X — The Client-Server Relationship 286 10.13 Networked X — The Client-Server Relationship • X works in a client-server relationship • The client is a user application (e.g. netscape) which needs X services to display itself on a given screen • The server is the application which provides these services, e.g. Xfree86 • On a single-user Linux system, both apps reside on the same system • On a networked Linux system the user can run an X application which is installed on a remote system but see it displayed on the local monitor, i.e. ◦ The client application (e.g. netscape) is remote and the X server (e.g. Xfree86) is local www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 287 Basic X-Windows 10.14 Principles of Running Remote X Apps • The most common use for networked X is to run client apps which are installed on remote hosts • Reasons for running X apps on remote hosts: ◦ Using graphical tools to administer a remote machine ◦ No local installation of the app ◦ Local processing or memory are insufficient ◦ No local access to data c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 10.15 How to Run Remote X Apps 288 10.15 How to Run Remote X Apps The right way (and simplest way) to do this is to use openssh • Simply type: $ slogin remotehost • . . . then type any commands to start a graphical epplication in the terminal window; it just works. The wrong way: • Start the local x server: $ startx • Enable (dangerous) lack of authentication $ xhost +remote-hostname • Open a telnet connection to the remote host: $ telnet remote-hostname • Set the your $DISPLAY environment variable on the remote host so that applications re-direct their graphic output to your local monitor: $ export DISPLAY=oakleigh:0 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 289 Basic X-Windows 10.16 Authentication • Xservers only allow authenticated hosts to connect • The right way to connect to a remote host is to use openssh. All the other methods described here are difficult, insecure, troublesome. . . So don’t use them. • On a trusted LAN you might use xhost in an xterm $ xhost +beehive • Or edit /etc/X0.hosts (0 refers to display 0): $ cat /etc/X0.hosts landlord kebab samosa • This is dangerous ◦ Allows hosts to grab your mouse and keyboard • Only use in a trusted environment c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 10.17 Better Authentication 290 10.17 Better Authentication • Can use cookie-based authentication • Done for you if using xdm or ssh • Clients look in ∼/.Xauthority for cookies to feed to server • Server must be started with appropriate argument ◦ Reads its ∼/.Xauthority file • Server only looks when started ◦ Too late to change once running • Both server and clients must use the same cookies ◦ Involves merging ∼/.Xauthority files using xauth • Hard to manage — most resort to xdm for local access or ssh for remote access. • Documentation is not very penetrable; ssh is much simpler to use, and is secure. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 291 Basic X-Windows 10.18 Basic X Exercises 1. Figure out how to get an X session running 2. In an xterm window type ’xterm’ — what happens? 3. In an xterm window type export DISPLAY=xyz xterm what happens? 4. Start up another xterm. (a) Type: echo hello You should get ‘hello’ echoed. (b) Select the echo hello text so that it highlights — do this by clicking the first mouse button and dragging. (c) Move the mouse to another xterm window; click into it to make it active if necessary. (d) You should be able to paste the selected text by clicking the middle mouse button (3 button mouse) or simultaneously clicking both buttons on a 2 button mouse. Try it and see. 5. Find another machine on the same network. Use xhost to tell it to accept connections from your machine. Start an xterm on your machine but tell it (using the DISPLAY variable) to display on the remote machine. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 10.18 Basic X Exercises 292 6. Go to the /usr/X11R6/bin directory. Try these commands: xeyes xsnow xfishtank xbill xdemineur xclock and any others that you like the idea of. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 Module 11 Fundamentals of TCP/IP Objectives This module is intended as an introduction to the the basic concepts of IP networking. By the end of it you should understand: • The history and uses of various protocols • How subnetting and netmasks work • About interfaces • The use of ports 293 11.1 Fundamentals of TCP/IP Networking 294 11.1 Fundamentals of TCP/IP Networking Key concepts: • Packets • TCP vs UDP • Services • Subnetting inc /xx form • Routing www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 295 Fundamentals of TCP/IP 11.2 History • Developed by ARPA for university & military research • Robust, reliable, wide area network protocol, system-independent • Will route traffic around network outages (if routing protocols used) • Came into widespread use in mid-late 1970s • Popularity hugely helped by free availability of the BSD Unix implementation i.e. the pre-Linux reference platform • Now the standard protocol — the Internet based totally upon it c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.3 Recap of basic IP Concepts — Components 296 11.3 Recap of basic IP Concepts — Components • Properly, The Internet Protocol Suite (IP Suite) • Usually erroneously referred to as TCP/IP • Consists of numerous protocols • IP is used to encapsulate: ◦ TCP (Transmission Control Protocol) ◦ UDP (User Datagram Protocol) ◦ ICMP (Internet Control Message Protocol) ◦ other routing & management protocols www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 297 Fundamentals of TCP/IP 11.4 IP versions • Currently at Version 4 (IPV4) ◦ Entire Internet based on IPV4 ◦ Quickly running out of spare numbers • IPV6 well standardised ◦ Important improvements ◦ Currently in miniscule use ◦ Migration will occur eventually ◦ Support already in Linux c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.5 Packets 298 11.5 Packets • All data transferred in packets (datagrams) • Each packet contains various flags & admin information ◦ Source address (32 bits) ◦ Destination address (32 bits) • Addresses identify hosts ◦ Usually an interface on a host • Addresses are the basis of packet routing • Packets can be split reassembled, differentially routed, arrive out-of-order or just get lost • Higher-level protocols (e.g. TCP) add sequencing reliability, flow control etc. IP TCP HTTP www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 299 Fundamentals of TCP/IP 11.6 Encapsulation • As data sent by application to network passes through layers of TCP/IP protocol stack, headers are appended to it. • As data received by application from network passes through layers of TCP/IP protocol stack, headers are stripped from it. Application Layer (HTTP, FTP, DNS, SMTP, DHCP, etc.) Data Transport Layer (TCP, UDP, ICMP) Header Data Internet Layer (IP) Header Header Data Network Access Layer (Ethernet, FDDI, ATM, PPP, etc.) Header Header Header Data c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.7 Internet Protocol Datagram 300 11.7 Internet Protocol Datagram • IP uses datagram packet switching • each IP datagram has a max. length 65,535 bytes including a 60-byte header (20-byte + optional part). • 64 bits in an IP datagram is used to represent the address of data terminals, 32 bits for the source and 32 bits for the destination. • version now 4; soon 6 will be common • IHL = Internet Header length = number of 32-bit words in header • TOS = 3 bits for precedence, 4 TOS bits: delay, throughput, reliability, cost. Bits 12 16 20 24 28 Padding 31 0 4 8 1 2 Words 3 4 5 Version IHL Type of Service DF MF Total Length Fragmentation Offset Header Checksum Identification Time to Live Protocol Source Address Destination Address Options (0 to 40 bytes) 5−16 Your data starts here www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 301 Fundamentals of TCP/IP 11.8 TCP Header Bits 12 16 20 24 28 Padding 31 0 4 8 Source Port Sequence Number Acknowledgement Number URG ACK PSH RST SYN 1 2 Words 3 header 4 length 5 Destination Port Reserved Checksum FIN Window Urgent Pointer Options (0 to 40 bytes) 5−15 Your data starts here c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.9 UDP Header 302 11.9 UDP Header • UDP is relatively simple: no ARQ • So no need to keep track of sequence numbers, no acknowlegdment number Bits 16 31 Destination Port Checksum Your data starts here c GBdirect Ltd and parts c Nick Urbanik, 2004 0 Source Port Length www.linuxtraining.co.uk 303 Fundamentals of TCP/IP 11.10 Addresses • Addresses shown in ‘dotted decimal’ — break into 4 bytes ◦ 192.168.0.129 • Four address families ◦ Class A 0.x.x.x–127.x.x.x ◦ Class B 128.x.x.x–191.x.x.x ◦ Class C 192.x.x.x–223.x.x.x ◦ ’reserved’ 224.x.x.x • Class A network 127 is special ◦ Refers to the current network (any network) ◦ Current host is always 127.0.0.1 ◦ ‘loopback’ address c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.11 Addresses (continued) 304 11.11 Addresses (continued) • Addresses identify: ◦ Network (used for routing between networks) ◦ Hosts on a particular network – Class A 8 network bits, 24 host bits – Class B 16 network bits, 16 host bits – Class C 24 network bits, 8 host bits Network Host Class A Network Host Class B Network Host Class C • In all networks, host-parts of all zeros (0) and all ones (255) are reserved ◦ Host-part zero refers to the network itself ◦ Host-part all ones is ’broadcast’ address (all hosts) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 305 Fundamentals of TCP/IP 11.12 Netmasks and subnetting • Netmasks split host and network part of address • Says which machines can be reached directly • Example: Netmask 255 .255 .255 .0 Binary 11111111.11111111.11111111.00000000 IP Binary 192 .168 .0 .129 11000000.10101000.00000000.10000001 • To work out the network part Netmask 11111111.11111111.11111111.00000000 IP 11000000.10101000.00000000.10000001 -------------------------------------------Result 11000000.10101000.00000000.00000000 -------------------------------------------192 168 0 0 • To work out the host part Netmask 11111111.11111111.11111111.00000000 IP 11000000.10101000.00000000.10000001 -------------------------------------------Result 00000000.00000000.00000000.10000001 -------------------------------------------0 0 0 129 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.13 CIDR: Classless Inter-Domain Routing 306 11.13 CIDR: Classless Inter-Domain Routing • Classes A, B, C are now history. WHY? • Class C too small, running out of class B! • Solution: CIDR • Specify network with two sets of numbers, e.g., ◦ 15/8 is the old class A network that starts with the bits 0001111. ◦ 128.32/16 is the old class B network 128.32.0.0 ◦ 192.168.0.128/25 is the 128 IP addresses from 192.168.0.128 to 192.168.0.255 ◦ 172.19.64/18 is the 214 = 16384 addresses from 172.19.64.0 to 172.19.127.255 14 = 32 − 18 172.19.64.0 ⇒ 1010 1100.0001 0011.0100 0000.0000 0000 The netmask has 18 bits that are ‘1’ (255.255.192.0) This is the range of addresses allocated to the CM labs from now on. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 307 Fundamentals of TCP/IP 11.14 CIDR: Classless Inter-Domain Routing—examples Class Network no. Host no. Address 10.2.1.1 128.63.2.100 201.222.5.64 192.6.141.2 130.113.64.16 192.168.129.49/23 202.4.192.60/24 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.15 Transferring Data 308 11.15 Transferring Data • IP allows datagrams to be sent and routed between hosts • Contains no application-level data • Data part will be one of UDP, TCP, ICMP etc. • TCP is ‘session’ oriented data, used for long-lived connections • UDP used for fire-and-forget messages • ICMP used for control & testing, not seen by most applications or users • Examples: ◦ Email transferred using SMTP over TCP, (maybe many bytes, order important) ◦ Web pages use HTTP over TCP ◦ UDP more obscure, used for NFS ◦ ICMP: ‘ping’ utility, used to test visibility www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 309 Fundamentals of TCP/IP 11.16 Hosts & Interfaces • Hosts are individual computers/systems • Each host has one or more interfaces ◦ Each interface is a point of connection to a network (often a NIC or modem) • Many hosts have a single interface, so the address is the host • May have more than one interface ◦ Interfaces could be on different networks ◦ Can act as routers, forwarding packets • Each ‘interface’ will have a single address c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.17 Routing 310 11.17 Routing • Hosts receive packets on one or more interfaces • Check to see if packet is for current host ◦ If so, deliver to the UDP/TCP etc mechanisms • Otherwise ◦ If routing enabled ∗ – Forward packet to appropriate host ◦ Routing based on internal routing table ◦ Manipulated by route command – Superuser only Non-Routing Host OS NIC 10.0.0.1 192.168.0.10 Routing Host NIC 10.0.0.1 NIC 192.168.0.1 192.168.0.10 ∗ Often referred to as ‘IP forwarding’ www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 311 Fundamentals of TCP/IP 11.18 Ports • Not enough just to deliver packets to hosts • Deliver to correct applications on the host ◦ Hosts presumed to be multitasking • UDP & TCP both include port numbers ◦ 16 bit numbers (0–65535) ◦ Each UDP/TCP packet contains source & destination port ◦ sourceport/sourceaddress & destinationport/destinationaddress uniquely identify a conversation telnetd 23 sendmail/ exim 25 httpd 80 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.19 Ports cont.. 312 11.19 Ports cont.. • Many ‘well known’ ports published for client-server applications • See /etc/services under Linux ◦ TCP/25 — SMTP mail ◦ TCP/23 — telnet (remote terminal access) ◦ TCP/80 — HTTP (web protocol) • Unix-like systems reserve ports below 1024 for super-user • Ordinary users cannot run ‘special’ services without authorisation • This cannot be trusted in other environments, such as Windows www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 313 Fundamentals of TCP/IP 11.20 Exercises 1. Using ifconfig, explore the interfaces available on your current system 2. Discover the IP addresses of some other machines on your network and check that you can ping them all. What class (A, B or C) of network are they on? 3. From the man page for ping, discover how to set a regular ping running every five seconds. Then investigate how you can send extra-long ping packets (try sending a ping longer than 2K bytes). 4. What ports and protocols are used to run the following services? • Telnet • SMTP • Printer • Talk 5. What happens if you telnet to various ports? (Try 25 or 110) 6. Use this fact to discover what mail system your machine runs, and see if it runs a webserver (Port 80) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 11.21 Solutions 314 11.21 Solutions 1. ifconfig by default shows a list of the currently configured interfaces including the IP addresses and netmasks. 2. ping 10.0.0.2 will send pings to the 10.0.0.2 interface provided routing is set up correctly. You should be able to find out what class of network you are on from the IP address. See section 11.10 for details. 3. To send a regular ping every 5 seconds use $ ping -i 5 To alter the packet size you use the -s option to give a size in bytes $ ping -s 3000 4. The following values are taken from /etc/services Port 23 25 515 517 Service Telnet SMTP Printer Talk 5. You should be able to talk directly to the daemon at the other end, e.g. $ telnet localhost 110 Trying 127.0.0.1... Connected to localhost. Escape character is ’^]’. +OK POP3 localhost v4.47 server ready USER lee +OK User name accepted, password please PASS ******** +OK Mailbox open, 2 messages RETR 1 +OK 332 octets Return-Path: Received: (from lee@localhost) by gbdirect.co.uk (8.8.7/8.8.7) id LAA12997 for lee@localhost; Mon, 14 Feb 2000 11:39:19 GMT Date: Mon, 14 Feb 2000 11:39:19 GMT From: Lee Message-Id: <200002141139.LAA12997@gbdirect.co.uk> To: lee@gbdirect.co.uk Subject: Test Status: RO This is a test. . QUIT +OK Sayonara Connection closed by foreign host. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 315 Fundamentals of TCP/IP 6. You can sometimes find out what webserver a site is using by telnetting to port 80 and requesting the headers of the main page, e.g. $ telnet www.bbc.co.uk 80 Trying 212.58.224.31... Connected to www.bbc.net.uk. Escape character is ’^]’. HEAD / HTTP/1.0 HTTP/1.1 302 Moved Temporarily Date: Mon, 14 Feb 2000 11:43:06 GMT Server: Apache/1.3.1 (Unix) Location: http://www.bbc.co.uk/home/today/ Connection: close Content-Type: text/html Connection closed by foreign host. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk Module 12 Practical TCP/IP Objectives After completing this module you should be able to understand and utilise: • Firewalling principles • Basic firewalling with ipchains • Network/routing debugging procedures • Interface configuration under Linux • The secure shell (sshd, ssh, and scp) 317 12.1 Ping Protocols 318 12.1 Ping Protocols • ping used to test network/host availability • A little about its implementation ◦ Uses ICMP protocol ◦ Send requests of type echo-request ◦ Receives answer echo-reply www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 319 Practical TCP/IP 12.2 Network Statistics (netstat) in Practice • Show network status; many options • Most useful: -r and -n flags (show routes, numeric addresses only) $ netstat -rn Kernel IP routing table Destination Gateway 192.168.0.0 0.0.0.0 202.80.80.0 192.168.0.1 192.100.100.0 192.168.0.1 194.217.100.0 192.168.0.143 192.168.1.0 192.168.0.1 192.168.3.0 192.168.0.1 127.0.0.0 0.0.0.0 Genmask 255.255.255.0 255.255.255.0 255.255.255.0 255.255.255.0 255.255.255.0 255.255.255.0 255.0.0.0 Flags U UG UG UG UG UG U MSS 1500 1500 1500 1500 1500 1500 3584 Window irtt Iface 0 0 eth0 0 0 eth0 0 0 eth0 0 0 eth0 0 0 eth0 0 0 eth0 0 0 lo • Note ‘gateway‘ above ◦ route to networks 202.80.80.0, 192.100.100.0, 192.168.1.0 and 192.168.3.0 use gateway 192.168.0.1 ◦ 192.168.0.1 is a gateway (router) which knows how to access those networks ◦ route to network 194.217.100.0 is via gateway at 192.168.0.143 • Often see destination of 0.0.0.0 ◦ ‘default‘ route ◦ send all otherwise unrouteable packets to designated gateway • Iface column shows which interface will be used • Note interface for 127.0.0.0 — ‘loopback‘ interface; the host itself c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.3 netstat (continued) 320 12.3 netstat (continued) • Also show information about connected sockets • netstat -a shows no. of active connections (useful for seeing system load) • Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address tcp 0 0 *:6000 *:* tcp 0 0 linux.gazcl:domain *:* tcp 0 0 localhost:domain *:* tcp 0 0 *:linuxconf *:* tcp 0 0 *:auth *:* tcp 0 0 *:finger *:* raw 0 0 *:icmp *:* raw 0 0 *:tcp *:* Active UNIX domain sockets (servers and established) Proto RefCnt Flags Type State I-Node unix 1 [] STREAM CONNECTED 8937 unix 1 [] STREAM CONNECTED 8906 unix 1 [] STREAM CONNECTED 8933 unix 0 [ ACC ] STREAM LISTENING 327 unix 1 [] STREAM CONNECTED 8949 unix 1 [] STREAM CONNECTED 8926 unix 1 [] STREAM CONNECTED 8644 State LISTEN LISTEN LISTEN LISTEN LISTEN LISTEN 7 7 Path @00000082 @0000007b @00000081 /dev/log @00000086 @0000007f @00000059 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 321 Practical TCP/IP 12.4 netstat — Further Examples • Configured interfaces • netstat -i Kernel Interface table Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg eth0 1500 0 0 0 0 0 3107 0 0 0 BRU lo 3924 0 1035 0 0 0 1035 0 0 0 LRU • netstat -p shows processes listening on each socket ∗ ◦ Includes PID ◦ Useful to kill processes hogging key ports $ netstat -pn Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 10.0.0.1:1025 10.0.0.2:telnet ESTABLISHED tcp 0 0 10.0.0.1:1024 10.0.0.3:telnet ESTABLISHED tcp 0 0 10.0.0.1:1023 10.0.0.4:ssh ESTABLISHED Active UNIX domain sockets (w/o servers) Proto RefCnt Flags Type State I-Node PID/Program name unix 1 [] STREAM CONNECTED 662 432/ssh unix 1 [] STREAM CONNECTED 591 388/login -- lee PID/name 443/telnet 442/telnet 432/ssh Path @00000037 @0000002f ∗ Only supported in more recent versions www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 12.5 Network Traffic (tcpdump) in Practice 322 12.5 Network Traffic (tcpdump) in Practice • Used to monitor network traffic ◦ Need sufficient privilege to monitor devices • Can show only particular information ◦ Traffic to/from a particular host ◦ Traffic on a certain port ◦ Certain types of traffic, e.g. TCP, ARP, UDP • Very configurable ◦ Decide what you want to do ◦ Then look at manual page www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 323 Practical TCP/IP 12.6 tcpdump Options • Some options -i -n -N -t -q -v Says which network interface to show Print IP addresses not names Don’t print domain name of address Don’t print timestamp Show only minimal output (quiet) Verbose info (time-to-live etc.) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.7 tcpdump Examples 324 12.7 tcpdump Examples $ tcpdump dst host 192.168.0.143 -i eth0 -n -t tcpdump: listening on eth0 192.168.0.131 > 192.168.0.143: icmp: echo request 192.168.0.131 > 192.168.0.143: icmp: echo request arp who-has 192.168.0.143 tell 192.168.0.131 192.168.0.131 > 192.168.0.143: icmp: echo request 192.168.0.131.1026 > 192.168.0.143.telnet: S 73945916:73945916(0) win 32120 (DF) 192.168.0.131.1026 > 192.168.0.143.telnet: . ack 3134108710 win 32120 (DF) 192.168.0.131.1026 > 192.168.0.143.telnet: P 0:27(27) ack 1 win 32120 (DF) 192.168.0.131.1026 > 192.168.0.143.telnet: . ack 13 win 32120 (DF) 192.168.0.131.1026 > 192.168.0.143.telnet: P 27:35(108) ack 5 win 32120 (DF) 192.168.0.131.1026 > 192.168.0.143.telnet: P 35:38(3) ack 55 win 32120 (DF) 192.168.0.131.1026 > 192.168.0.143.telnet: P 38:41(3) ack 125 win 32120 (DF) 192.168.0.131.1026 > 192.168.0.143.telnet: . ack 132 win 32120 (DF) 192.168.0.131.1026 > 192.168.0.143.telnet: P 41:42(1) ack 132 win 32120 (DF) $ tcpdump dst host 192.168.0.143 tcpdump: listening on eth0 09:56:32.947997 landlord.mysql > 09:56:32.955822 landlord.mysql > 09:56:32.963597 landlord.mysql > 09:56:32.970917 landlord.mysql > 09:56:32.979341 landlord.mysql > 09:56:32.987218 landlord.mysql > 09:56:32.995902 landlord.mysql > -i eth0 -N -q samosa.5660: samosa.5660: samosa.5660: samosa.5660: samosa.5660: samosa.5660: samosa.5660: tcp tcp tcp tcp tcp tcp tcp 166 166 182 166 166 166 555 (DF) (DF) (DF) (DF) (DF) (DF) (DF) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 325 Practical TCP/IP 12.8 Firewalling • Allows you to protect your machine ◦ As well as machines behind them • Checks packet headers before acting on them ◦ Can ignore, reject or accept packets ◦ Makes decision based on source, destination, or packet type – Or a combination • Set up using ipchains under kernel 2.2 ◦ Older kernels used ipfwadm c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.9 Basic Theory 326 12.9 Basic Theory • Two main considerations ◦ Port Filtering ◦ Host Filtering • Block services you don’t need • Limit services you do need to specific machines/networks www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 327 Practical TCP/IP 12.10 Basic Theory (continued) • Firewalling can be done with inetd ◦ /etc/hosts.allow ◦ /etc/hosts.deny ◦ /etc/inetd.conf • Flaw in inetd would still let things through • Best to drop the packets as soon as possible ◦ Kernel-level filtering c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.11 ipchains 328 12.11 ipchains • Packet filtering set up using ipchains • All the filtering is done in the kernel ◦ Not by ipchains ◦ ipchains just sets up/modifies the rules • All packets entering and leaving are examined ∗ ∗ Including loopback traffic which conceptually leaves the machine www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 329 Practical TCP/IP 12.12 ipchains Details • Every packet goes through one or more ‘chains’ ◦ A ‘chain’ is a set of rules ◦ Rules can accept, reject, or deny a packet – Can also send it to another chain • Three default chains, input, output, forward ◦ If a packet passes through a default chain without matching: – Fate is determined by the chains policy – Can be Accept, deny, or reject ◦ If it reaches the end of a user defined chain – Carries on where it left off • forward is for IP masquerading systems ◦ Not covered here c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.13 ipchains Options 330 12.13 ipchains Options • Dealing with chains: -N -X -P -L -F Create a new chain Delete an empty chain Change the policy for a chain List the rules in a chain Flush (delete) all rules from a chain • Dealing with rules: -A Append a rule to a chain -D Delete a single rule from a chain -I Insert a rule at some point in a chain www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 331 Practical TCP/IP 12.14 Options For Rules • Use the following to specify packets to match -s -d -p -j chain --sport --dport Source address Destination address Protocol (TCP, UDP, ICMP) Jump to chain/action Source Port Destination Port c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.15 ipchains — Examples 332 12.15 ipchains — Examples • In most cases default chains will be sufficient • To block all ping requests to our machine: $ ipchains -A input -p icmp -s 0.0.0.0/0 \ > --icmp-type echo-request -j DENY $ ipchains -L input Chain input (policy ACCEPT): target prot opt source destination DENY icmp ------ anywhere anywhere ports echo-request • To block outgoing ping packets: $ ipchains -A output -p icmp -d 0.0.0.0/0 \ > --icmp-type echo-request -j DENY $ ipchains -L output Chain output (policy ACCEPT): target prot opt source destination DENY icmp ------ anywhere anywhere ports echo-request $ ping -c1 landlord PING landlord.gbdirect.co.uk (192.168.0.129): 56 data bytes ping: sendto: Operation not permitted ping: wrote landlord.gbdirect.co.uk 64 chars, ret=-1 --- landlord.gbdirect.co.uk ping statistics --1 packets transmitted, 0 packets received, 100% packet loss • Very simple examples but they show the theory www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 333 Practical TCP/IP 12.16 Removing Rules • Rules can be removed by number, e.g. to delete the first rule in the input chain: $ ipchains -D input 1 • or definition, e.g. delete the first matching rule: $ ipchains -D output -p icmp -d 0.0.0.0/0 --icmp-type echo-request -j DENY • To clear an entire chain use: $ ipchains -F chainname • If no chainname is given, it clears all chains c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.17 Implementing ipchains 334 12.17 Implementing ipchains • The rules are normally set up in the machines ‘init scripts’ • Typically by creating a script in init.d that is run just before networking starts ◦ Example in section 12.19 • Ensure you flush existing rules first (just in case): $ ipchains -F • Generally start with the DENY rules then add what you want • Maximum security www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 335 Practical TCP/IP 12.18 Save and restore • Often useful to create a firewalling ‘config file’ • ipchains-save outputs a text file you can store [Setup firewall rules as you want] $ ipchains-save > /etc/ip.rules Saving ‘input’. Saving ‘forward’. Saving ‘output’. $ • Can reinitialise your firewalling with ipchains-restore and your ‘config file’, e.g. $ ipchains-restore < /etc/ip.rules $ • Usually done in a startup script c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.19 ipchains setup script 336 12.19 ipchains setup script • A sample script may look like: #! /bin/sh # Script to control packet filtering. # If no rules, do nothing. # Altered from the ipchains HOWTO [ -f /etc/ipchains.rules ] || exit 0 case "$1" in start) echo -n "Turning on packet filtering:" /sbin/ipchains-restore < /etc/ipchains.rules || exit 1 echo "." ;; stop) echo -n "Turning off packet filtering:" /sbin/ipchains -X /sbin/ipchains -F /sbin/ipchains -P input ACCEPT /sbin/ipchains -P output ACCEPT /sbin/ipchains -P forward ACCEPT echo "." ;; restart) $0 stop $0 start ;; *) echo "Usage: $0 {start|stop|restart}" exit 1 ;; esac exit 0 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 337 Practical TCP/IP 12.20 Real World ipchains • Connect out to a host but not in $ ipchains -A input -d 192.168.0.131/32 -p TCP -y -j DENY • -y limits matching to packets with the SYN bit set ◦ Used when establishing connections • No-one can open a connection from 192.168.0.131 ◦ Can still connect to it from here . . . c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.21 Interface Configuration and Management 338 12.21 Interface Configuration and Management • An interface is a point of connection to a network • Usually a single device ◦ Network card ◦ PPP link • A device can have more than one interface ◦ Referred to as ‘aliases’ ◦ Commonly used for virtual web sites www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 339 Practical TCP/IP 12.22 Point-and-Click Interface Administration • Number of ways to add/edit interface details ◦ Linuxconf ◦ Redhat control-panel ◦ ‘By hand!’ • For most cases you can probably use one of the two graphical methods • Useful to understand the configuration files behind it all • See the Linux Network Administrator’s Guide at http://www.linuxdoc.org/LDP/nag2/ for much there is to know about this. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.23 /etc/sysconfig/network-scripts 340 12.23 /etc/sysconfig/network-scripts • Directory containing scripts and config files ∗ • ifup & ifdown activate/deactivate an interface • Argument specifies interface to act on, e.g. $ ifdown eth0 • ifcfg-eth* are config files for each interface ◦ Should be numbered sequentially from 0 – ifcfg-eth0 is the first interface ◦ Files ending in :n (where n is a number) are aliases – ifcfg-eth0:0 is the first alias for the first interface This applies to RedHat only, you should see section 12.27 for information on other distributions www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 341 Practical TCP/IP 12.24 ifcfg-ethx • Describes characteristics of a given interface ◦ What device it should be known as (DEVICE) ◦ IP address, network, and netmask (IPADDR, NETWORK, NETMASK) ◦ Whether it is activated at boot time (ONBOOT) ◦ Whether it can be controlled by normal users (USERCTL) • Example: DEVICE=eth0 IPADDR=192.168.0.129 NETMASK=255.255.255.0 NETWORK=192.168.0.0 BROADCAST=192.168.0.255 ONBOOT=yes BOOTPROTO=none USERCTL=no c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.25 Altering An Interface 342 12.25 Altering An Interface • It is perfectly allowable to alter interfaces while the system is running • Requires only minimal disruption to network connectivity ◦ Not a reboot • Two simple steps 1. Make alterations (by hand or through GUI) 2. Restart networking • Networking is just another service ◦ /etc/rc.d/init.d/network restart ◦ /etc/init.d/network restart www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 343 Practical TCP/IP 12.26 Adding an Interface • Adding an alias is even easier! 1. Add the alias 2. Activate it • Example: Add the following to /etc/sysconfig/network-scripts/ifcfg-eth0:0 DEVICE=eth0:0 USERCTL=no ONBOOT=yes BOOTPROTO=none BROADCAST=192.168.0.255 NETWORK=192.168.0.0 NETMASK=255.255.255.0 IPADDR=192.168.0.141 • Then execute $ ifup eth0:0 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.27 The ‘Proper’ Way 344 12.27 The ‘Proper’ Way • Previous examples use scripts (not always provided) • You can do everything manually • Add an alias: /sbin/ifconfig eth0:0 192.168.0.128 • Check with ifconfig that it succeeded • Setup routing to that interface: /sbin/route add -host 192.168.0.128 dev eth0:0 • Removing an alias: ◦ /sbin/ifconfig eth0:0 down ◦ /sbin/route del 192.168.0.128 • Adding an interface is similar . . . • Probably want to add a route to the entire network not just the host /sbin/route add -net 192.168.0.0 netmask 255.255.255.0 dev eth1 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 345 Practical TCP/IP 12.28 Drivers • Network drivers invariably handled by kernel modules ◦ PCI NE2000 card handled by ne2k-pci.o • Kernel cannot tell which module should be used by which interface ◦ module loader uses lines from /etc/modules.conf (older: /etc/conf.modules), e.g., alias eth0 ne alias eth1 ne options ne io=0x320,0x340 irq=2,12 ◦ Above says that interfaces eth0 and eth1 handled by ne module (NE2000 ISA) ◦ Options line is module-specific; permits port/IRQ specification if not autodetected: this is common for old ISA cards. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.29 The Secure Shell in Practice (ssh) 346 12.29 The Secure Shell in Practice (ssh) • How you use ssh varies across systems • Some require stricter authentication than others • For example, within a secure environment it may not require a password ◦ Works on ‘trusted host’ concept ◦ Much better than rsh due to encryption and server key authentication • Can often be used as a drop-in replacement for rsh or telnet • Has numerous advantages . . . ◦ Sets up forwarding of X connections ◦ Can compress the data sent ◦ No passwords sent in plain text (and hence trivially read by others) ◦ Not trivially hijacked using hunt (from http://lin.fsid.cvut.cz/∼kra/) and other such tools www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 347 Practical TCP/IP 12.30 Secure Copying in Practice (scp) • Replacement for rcp • Much more secure ◦ Encrypts all traffic ◦ Uses same authentication as ssh • Can copy local to remote, remote to local or remote to remote • Example: $ scp localfilename user@remotehost:remotefilename c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.31 Summary 348 12.31 Summary • Wide range of network utilities available ◦ Both maintenance and user-orientated • Very flexible system ◦ Can be hard to setup/maintain • Pros outweigh cons • Common jobs become second nature www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 349 Practical TCP/IP 12.32 Exercises 1. Network tools (a) Use netstat -rn to investigate the routes on your network. Explain each line of entry to a colleague. (b) Read the man page for tcpdump. Use it to monitor traffic on your host’s network interface whilst other hosts are pinging each other. 2. ipchains (a) Use ipchains to set up the following configurations. In each case you should first set up the system by hand, check it. Then set it up so that the firewall rules are in place when the machine reboots. i. ii. iii. iv. v. Block all incoming ICMP packets Block only incoming ICMP ‘echo-request’ packets Block all incoming telnet connections Block all telnet connections Block all outgoing web requests (Port 80) 3. Network configuration (a) Using one of the admin tools (linuxconf or control-panel etc.) add an alias on your network interface so that your host can masquerade as some other host. DO NOT DO THIS IF YOU ARE NOT SURE YOU ARE USING A SPARE IP ADDRESS. Investigate what ifconfig and netstat -rn now report. Check that you can ping the alias from another host on the network. (b) If possible, fit an extra network card to one of the hosts (host b) and configure it to be on a different network. Check it can be pinged from its own host. Go to another host (host a) on the original network and add a route to host b’s new interface, using as a gateway host b’s original network interface. Check that you can ping it and then use traceroute to see the path taken by packets. Host b will have to have IPV4 forwarding enabled for this to work. Ask the tutor about which machine will be set up for this. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 12.33 Solutions 350 12.33 Solutions 1. (a) If you don’t understand the output check section 12.2 or the netstat manpage (b) tcpdump -i eth0 should monitor all network traffic. If you want to see the traffic to a particular host use tcpdump dst host 10.0.0.3 2. (a) The following are the list of rules needed to satisfy each situation. You should flush the chains before each one (ipchains -F). i. ipchains -A input -p icmp -j DENY ii. ipchains -A input -p icmp --icmp-type echo-request -j DENY iii. ipchains -A input -p tcp -d 127.0.0.1 --dport telnet -j DENY ipchains -A input -p tcp -d 192.168.0.131 --dport telnet -j DENY iv. ipchains -A output -p tcp -d 0/0 --dport telnet -j DENY ipchains -A input -p tcp -s 0/0 --dport telnet -j DENY v. ipchains -A output -p tcp -d 0/0 --dport www -j DENY 3. (a) (b) Ask the tutor for details. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 Module 13 SSH — The Secure Shell Objectives • After completing this section, you will be able to: ◦ Use secure shell as a replacement for telnet ◦ Understand the weaknesses of the secure shell, and in particular, how to protect your connections from being hijacked ◦ Understand some of the dangers of telnet ◦ Use scp to copy files ◦ Understand how to create private and public keys for the secure shell using ssh-keygen ◦ Understand how to configure the files ∼/.ssh/authorized keys, /etc/ssh/ssh known hosts with public keys ◦ Know how to configure ssh-agent and ssh-add to allow login without a password. ◦ Be able to configure some important options of the client and server, including X11 forwarding. 351 13.1 What is the Secure Shell? 352 13.1 What is the Secure Shell? • Allows users to log into remote computers and use them interactively. • Provides secure network access for system administrators, replacing obsolete protocols such as telnet and many others • Allows tunneling of other protocols (such as X) through the encrypted connection • Allows secure, encrypted copying of data over the Internet with scp • Very useful for remote backup of servers and firewalls. • After it is set up, it is very easy to use www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 353 SSH — The Secure Shell 13.2 But what’s wrong with telnet? • We still use telnet for some of our other classes. What could be wrong with it? • The program hunt, freely available from http://lin.fsid.cvut.cz/∼kra/, lets you very easily: ◦ read passwords ◦ hijack telnet sessions ◦ monitor what the person is typing • Pavel Krauz wrote the program just to show how insecure telnet is—try it out! • Other tools can do this too (but just with less convenience) • Spread the news! telnet is dead! Demand the Secure Shell! Use the secure shell at your work; persuade people to stop using telnet, except where passwords are not required, and the information is not useful (in which case, you should be doing something else!) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.3 Cryptography 354 13.3 Cryptography • Two main categories of encryption: ◦ public key or asymmetric cryptography – Different key used to encrypt and decrypt the data – The public key used to encrypt – private key used to decrypt ◦ secret key or symmetric cryptography – Same key used to encrypt and decrypt the data – In Secure Shell, the session key performs the encryption and decryption at each end. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 355 SSH — The Secure Shell 13.4 OpenSSH and its history ¨ • Originally, SSH was free, developed by Tatu Ylonen, • later, increasingly restrictive licensing. • Two versions of SSH: SSH1 and SSH2, available from SSH Communications Security, and Datafellows, at high prices ($3120.00 US for a 25 user client only license when I checked!) • International group of programmers chose to develop a completely free SSH from Tatu’s original SSH ◦ Development moving very rapidly ◦ well implemented ◦ used by many system administrators round the world ◦ Completely free (both in the sense of “freedom” and in the sense of “free beer”), open protocol ◦ now IETF is standardising SSH • Provides a very nice blowfish logo, seen here burying the old insecure protocols: telnet, rlogin, rsh. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.5 Okay, I like the blowfish—what else does OpenSSH provide? 356 13.5 Okay, I like the blowfish—what else does OpenSSH provide? • OpenSSH 2.3 and onward is compatible with both SSH1 and SSH2, and interoperates with the commercial versions • OpenSSH provides powerful encryption to the entire session • OpenSSH 2.3 and above provides the following main programs: ◦ A server sshd which provides services for all the clients below, as well as sftp, an ftp replacement, that works with CuteFTP among many other clients ◦ The clients ssh, scp: – ssh is for executing programs on other remote computers, and replaces telnet – scp is a network copy program; works like cp, but over the network. ◦ The utilities: – ssh-keygen for generating the private and public keys used by SSH – ssh-agent to conveniently hold the personal private key to allow you to login without typing passwords – ssh-add for adding the personal private key to ssh-agent www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 357 SSH — The Secure Shell 13.6 So okay, how do I use this Secure Shell? • To use it like telnet is easy • Just type ssh computer name or IP • The first time you connect, you will receive a warning something like this: The authenticity of host ’nms.tyict.vtc.edu.hk (172.19.64.56)’ can’t be established. RSA key fingerprint is 48:2a:cb:1f:ac:7d:21:89:39:de:47:20:cf:06:d3:44 Are you sure you want to continue connecting (yes/no)? For now, enter yes (but see section 13.14 on page 365). The program will reply something like this: Warning: Permanently added ’nms.tyict.vtc.edu.hk’(RSA) to the list of known hosts. • Enter your password when prompted • You now have a high security, military-strength encrypted link to the other computer.∗ ∗ Provided your connection was not hijacked! See section 13.14 on page 365. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 13.7 Using scp to copy files over the network 358 13.7 Using scp to copy files over the network • Use like the cp command • Put the computer name and a colon in front of a remote file • Put a username then @ in front of this if you want to connect as a different user from yourself • Enough! An example of copying a file file from your home directory on machine neighbour to the current directory: $ scp neighbour’s IP :file ∼ • Here is an another example: copying a file from my account nickl on the remote computer called server to my home directory on my local workstation: $ scp nick@server:/usr/share/emacs/site-lisp/site-start.el ∼ • To copy a file called file from your current directory to the existing directory /tmp/directory on your neighbour’s computer (that you have an account on), do: $ scp file neighbour’s IP :/tmp/directory www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 359 SSH — The Secure Shell 13.8 Useful options with scp • The most useful options to scp: option purpose preserve the permissions (and owner if you are root) -r recursively copy all the subdirectories and their contents -C enable compression. Good for modem speed links. -P 23 use port 23 instead of port 22. Good for transferring files to or from home through a firewall that has port 23 open, but which blocks port 22. -p c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.9 SSH uses public and private keys 360 13.9 SSH uses public and private keys • The Secure Shell uses a public key and a private key for each computer (public key cryptography) • Each user can also have their own private and public keys • Public key cryptography allows a public key (a very long number) to be freely distributed and made public • The private key must be kept secret. Your personal private key is usually stored encrypted on the hard disk, protected by a long pass phrase • The host keys are used to: ◦ make sure the computer you are connecting to is really the one you want ◦ to establish secret session keys used to encrypt all communication in the session, including any passwords • The user keys are used to: ◦ To establish the identity of the person who has initiated the session (authentication) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 361 SSH — The Secure Shell 13.10 SSH Architecture Client Server Host key known_hosts public H SSH connection H private session key H session key User Account public User key Target account U U private U user public key ~/.ssh/authorized_keys c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.11 Overview of SSH 362 13.11 Overview of SSH • The client computer is usually the system administrator’s computer (or the person who wants to establish the connection) • The server computer is often exactly that — a server the system administrator needs to administer. • The host keys are used to: ◦ authenticate the server (make sure it really is the company server, and does not belong to the competitor who wants to steal your company’s secrets) ◦ Establish the session key at each end of the connection. • Note: the actual encryption of the session is done using the session keys, with symmetric encryption, since it is much faster than public key encryption. • The user keys are optional; they allow strong authentication without passwords. • The user private key is encrypted with a strong passphrase. • The agent is a program used to hold the decrypted private key on the client, to make the private key available to SSH applications. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 363 SSH — The Secure Shell 13.12 Steps of establishing a connection The following steps are not complete; SSH2 is actually three separate protocols, which can be used in various ways. Here I give an approximate view of SSH establishing a connection. • The client makes a TCP connection to the port the server is listening on (normally 22) • The client and server announce the protocols they support, and decide on what they will use • If the client does not have the server’s public host key, then the client will be prompted as to whether they want to accept the host key. This is the warning given in section 13.6 on page 357. If you type “yes”, then the host key will be transferred in clear text over the network, and appended to your ∼/.ssh/known hosts file. At this point, your session is vulnerable to hijacking. To avoid this risk, transfer the public host key yourself (manually) and append it to /etc/ssk/ssh known hosts on the client as described in section 13.14 on page 365. • Now the server and client generate and share a session key using an algorithm called the Diffie-Hellman Key Exchange. They use the host key pair to do this. • The session is now encrypted, and authentication of the user now takes place, using either the user key pair, or a password. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.13 Using ssh-keygen to create a personal pair of private and public keys 364 13.13 Using ssh-keygen to create a personal pair of private and public keys • When you first booted Red Hat Linux, the startup script for sshd generated the public and private host keys automatically. • You can create your own pair of user keys: just type $ ssh-keygen -t rsa ◦ accept the default file name ◦ when asked for a pass phrase, type a long sentence that you can easily remember, but which will be impossible for other people to guess. $ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/nicku/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/nicku/.ssh/id_rsa. Your public key has been saved in /home/nicku/.ssh/id_rsa.pub. The key fingerprint is: 13:6f:6b:1f:ac:6a:21:89:f0:de:47:20:d4:06:5c:2b • This will generate the files ∼/.ssh/id rsa.pub and ∼/.ssh/id rsa • The private key ∼/.ssh/id rsa must have access permissions only for you, the owner, i.e., “-rw-------” or “-r--------” www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 365 SSH — The Secure Shell 13.14 The host keys in /etc/ssh/ssh known hosts and ∼/.ssh/known hosts • An important step in using SSH is having the public host key of the computers you want to connect to. Unfortunately, the only really secure way is to copy it on a floppy disk from the remote machine, and take it to your own computer. • Copy the file /etc/ssh/ssh host rsa key.pub from the remote machine to a floppy. • On the local machine, append the file as one line to the file /etc/ssh/ssh known hosts • An alternative is just to enter yes when asked, as in section 13.6 on page 357 ◦ The public host key will be copied over the network from /etc/ssh/ssh host rsa key.pub on the server and appended to ∼/.ssh/known hosts on the client. ◦ But: this is a serious security hazard, and cracking tools exist (sshmitm) to easily hijack SSH sessions before the host key is transferred, • There are tools to automate this insecure procedure (of building the /etc/ssh/ssh known hosts file). c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.15 The file ∼/.ssh/authorized keys 366 13.15 The file ∼/.ssh/authorized keys • This file contains the personal public keys of user accounts that you trust (your own) to connect to your computer without a password. • To set it up on a remote computer: ◦ Copy your local personal public key to the remote computer (do not copy to ∼/.ssh, or you will overwrite your own local key!) Here put the host name of the remote computer instead of “ remote IP ”. Note: “local $” and “remote $” are shell prompts; don’t type them! local $ scp -p ∼/.ssh/id_rsa.pub remote IP : ◦ Log into the remote machine using ssh: local $ ssh remote IP ◦ Make the directory ∼/.ssh if it does not exist, and change to mode 700: remote $ mkdir ∼/.ssh remote $ chmod 700 ∼/.ssh ◦ Append ∼/id rsa.pub to ∼/.ssh/authorized keys on the remote machine, in the ssh session: remote remote remote remote $ $ $ $ cat ∼/id_rsa.pub >> ∼/.ssh/authorized_keys rm ∼/id_rsa.pub chmod 600 ∼/.ssh/authorized_keys exit • You now have installed your personal public key from your local computer account to the remote computer, allowing you to log in to the remote computer using public key cryptography instead of passwords. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 367 SSH — The Secure Shell 13.16 The User’s Public and Private Keys • There are two computers here: the local client machine, and the remote server machine. • When configuring SSH, most of the work is done on your local client computer. • You generate a public and private key pair using ssh-keygen -t rsa on your client computer—see section 13.13 on page 364. You do not need to generate public and private user keys in your account on the remote server. • Your private key is on the client. Your public key must be on the server, in the file ∼/.ssh/authorized keys. The client does not need a file ∼/.ssh/authorized keys. • You install the secure shell agent on your client machine—see section 13.20 on page 371. You do not need to install it on the server. • You add your private key ∼/.ssh/id rsa on the client computer to the agent, which is also on the client computer, using the ssh-add command. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.17 SSH1 and SSH2 368 13.17 SSH1 and SSH2 • SSH1 and SSH2 use totally different, incompatible protocols. • The commercial programs have separate programs for each protocol (pay for each separately) • OpenSSH supports both SSH1 and SSH2. • SSH2 is more secure than SSH1. It is best to avoid SSH1 clients and servers wherever possible. • SSH1 uses only RSA authentication, while SSH2 uses both RSA and DSA • See http: //www.snailbook.com/faq/ssh-1-vs-2.auto.html www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 369 SSH — The Secure Shell 13.18 The public and private key pairs: a summary • SSH1 and SSH2 use different files to hold the public and private keys. I’ve made a table to summarise their purposes: SSH1 file ∼/.ssh/known hosts SSH2 file ∼/.ssh/known hosts Purpose other host public keys added automatically when you type yes as in section 13.6 other host public keys added manually, as in section 13.14 host private key (chmod 400) host public key User’s personal private key User’s personal public key (just used for copying into the authorized keys* files of other servers) Holds personal public keys of clients authorised to log into this account on this SSH server /etc/ssh/ssh known hosts /etc/ssh/ssh host key /etc/ssh/ssh host key.pub ∼/.ssh/identity ∼/.ssh/identity.pub /etc/ssh/ssh known hosts /etc/ssh/ssh host rsa key /etc/ssh/ssh host rsa key.pub ∼/.ssh/id rsa ∼/.ssh/id rsa.pub ∼/.ssh/authorized keys ∼/.ssh/authorized keys c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.19 Files and Permissions I Recommend 370 13.19 Files and Permissions I Recommend • If permissions are wrong, SSH will just not work. The manual pages recommend permissions, but here I have made a list from my own experience: File ∼/.ssh ∼/.ssh/known hosts ∼/.ssh/identity ∼/.ssh/id rsa ∼/.ssh/identity.pub ∼/.ssh/id rsa.pub ∼/.ssh/authorized keys Min Permission Max Permission drwx------rw-------r--------r--------r--------r--------rw------drwxr-xr-x -rw-r--r--rw-------rw-------rw-rw-r--rw-rw-r--rw-r--r-- • I use and recommend the minimum permissions. Note that SSH will refuse to work if: ◦ the private keys are readable, writeable or executable by group/others ◦ the authorized keys* are writable by group/others ◦ the ∼/.ssh directory is not readable or executable by group/others ◦ the ∼/.ssh directory is writable by group/others www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 371 SSH — The Secure Shell 13.20 Using ssh-agent to log in without typing passwords • Now you can use public key cryptography to login to the remote machine. But you need to type that long pass phrase each time! Oh, that’s not convenient. • So let’s use the utilities ssh-agent and ssh-add to store the private key on your client computer. • Your private key is stored encrypted on the hard disk of your client computer in your account, protected by your pass phrase. • The agent provides a container to hold your decrypted private key in the memory of your client machine. • You use the program ssh-add to add the private key to the agent once after you boot your client machine. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.21 Setting up ssh-agent: logging in without typing passwords 372 13.21 Setting up ssh-agent: logging in without typing passwords • We do this on the client machine only. Here’s what we do, once only: ◦ edit your login script ∼/.bash profile in your account on the client machine, and add the line: eval $(ssh-agent) Note this is command substitution. ◦ Log out of X and log back in again. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 373 SSH — The Secure Shell 13.22 Using ssh-add: logging in without typing passwords • We do the following every time you boot your client computer: ◦ Open a window, and type: $ ssh-add then type your pass phrase when prompted. • Your client user’s secret key is now held decrypted by the agent in the memory of the client machine, letting you conveniently log in to your account on a remote server. If you do this: local $ ssh remote IP you will connect to your account on the remote computer without having to type a password. And that’s not all! Note: “local $” is the shell prompt; you do not type that. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.23 An easier way: using keychain 374 13.23 An easier way: using keychain • keychain is a very useful shell script that automates the management of personal private keys and the agent. • It is also very useful in automating the use of secure shell using cron; see section § 3.29 on page 92. • Get it from http://www.gentoo.org/projects/keychain. • Installation is simple: 1. get the tarball (click on the link to http://gentoo.oregonstate.edu/distfiles/ keychain-2.2.0.tar.bz2, save it to your home directory). 2. Unpack it into any directory $ tar xvjf ∼/keychain-2.2.0.tar.bz2 3. change into the directory keychain-x.x 4. install it into the directory /usr/bin with: $ sudo install -m 755 keychain /usr/bin www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 375 SSH — The Secure Shell 13.24 What keychain Does • The agent holds your user private key, decrypted • SSH clients talk to the agent through a socket ◦ A socket is a method of inter-process communication (IPC) ◦ It is a special file that allows two processes to talk to each other in both directions at the same time • When the agent starts, it creates a socket, and writes commands to standard output that create environment variables that let SSH clients know how to find the agent; for example: $ ssh-agent SSH_AUTH_SOCK=/tmp/ssh-sif17405/agent.17405; export SSH_AUTH_SOCK; SSH_AGENT_PID=17406; export SSH_AGENT_PID; • You can see this socket: $ ls -l /tmp/ssh-sif17405/agent.17405 srwxrwxr-x 1 nicku nicku 0 May 2 11:19 /tmp/ssh-sif17405/agent.17405 • The keychain shell script makes sure that a file ∼/.keychain/${HOSTNAME}-sh contains these two values, and that they are correct. The algorithm is: if ∼/.keychain/$HOSTNAME-sh holds info about a running agent if that agent does not hold all required keys add those keys else exit else terminate all running agents start new agent store socket, pid info in ∼/.keychain/$HOSTNAME-sh add all required private user keys c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.25 Setting your hostname 376 13.25 Setting your hostname • keychain will save the information about your SSH agent in the file ∼/.keychain/${HOSTNAME}-sh. ◦ Previously, I set up a name server so that each of our computers would have its own name, but technical staff have disabled it. ◦ The problem is that because all our machines are called localhost.localdomain, keychain will save your ssh-agent information in the same file name, overwriting the information about the agent in your own desktop. ◦ To avoid this problem, we can give our own computer a hostname. Here I explain how you can do that. 1. Edit the file /etc/sysconfig/network: $ xhost +localhost $ sudo -v $ sudo emacs /etc/sysconfig/network & and edit the line HOSTNAME=localhost.localdomain and change it into HOSTNAME= your name .tyict.vtc.edu.hk where your name is your own name; please try to make this unique, so that it is not the same as anyone elses. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 377 SSH — The Secure Shell • Do not add spaces, or any characters except letters and digits and hyphen. Note the first character of a host name should be a letter. I suggest just use lower case letters. 2. Edit the host file, /etc/hosts: $ sudo -v $ sudo emacs /etc/hosts & This file is used by the operating system to map host names to IP addresses and back, before the operating sytem checks the name server. Note that on Windows, the file is in %SystemRoot%/System32/drivers/etc/hosts Edit the line 127.0.0.1 localhost.localdomain localhost and change it to: 127.0.0.1 your name .tyict.vtc.edu.hk your name localhost.localdomain localh 3. Then restart networking with: $ sudo service network restart 4. Log out and log in again. Your hostname is now changed to your name .tyict.vtc.edu.hk. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.26 Configuring your own account to use keychain 378 13.26 Configuring your own account to use keychain • Next add these lines to the end of your login script ∼/.bash profile: [ -x /usr/bin/keychain ] && keychain ~/.ssh/id_rsa [ -r ~/.keychain/${HOSTNAME}-sh ] && source ~/.keychain/${HOSTNAME}-sh • Remove the line: eval $(ssh-agent) from your log in script ∼/.bash profile. • Add this to the end of ∼/.bashrc: [ -r ~/.keychain/${HOSTNAME}-sh ] && source ~/.keychain/${HOSTNAME}-sh Note that when you log in, bash will source your log in script, ∼/.bash profile. Every time you start a new bash process, then bash will source ∼/.bashrc. • Log out and log back into the computer. You will be prompted to enter your passphrase. You will never need to enter it again until you reboot the computer. Automating network transfers using keychain and SSH • You can run shell scripts from your cron table. See section § 3.29 on page 92. Add the line: [ -r ~/.keychain/${HOSTNAME}-sh ] && source ~/.keychain/${HOSTNAME}-sh near the beginning of your script. It will be able to use the secure shell commands to connect to other computers. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 379 SSH — The Secure Shell 13.27 Running X applications remotely • You can run X applications remotely. • Simply log into the remote machine using ssh, • type the name of the graphical program, and it just works, as if it were running locally! • The program executes on the remote computer, but the X protocol sends the graphics commands through the encrypted session, • the application is displayed on your local machine, though the processing is done remotely. • Great for running remote graphical system administration applications (remote administration) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.28 Configuring SSH for X 380 13.28 Configuring SSH for X Note that the settings of the RPMs from Red Hat are already configured to support X forwarding. • The configuration file for the client is /etc/ssh/ssh config • The configuration file for the server is /etc/ssh/sshd config • Letting X work over SSH: ◦ An important configuration option in /etc/ssh/ssh config is: ForwardX11 yes ◦ An important configuration option in /etc/ssh/sshd config is: X11Forwarding yes ◦ These enable X to work over SSH as described in section 13.27 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 381 SSH — The Secure Shell 13.29 Security options for the client in /etc/ssh/ssh config • RhostsAuthentication no disable fallback to insecure remote hosts file “authentication.” The default is “yes”. This option applies to protocol version 1 only. • RhostsRSAAuthentication no disable fallback to insecure remote hosts file and host key authentication. The default is “yes”. This option applies to protocol version 1 only. • FallBackToRsh no Totally avoid rsh (buried on page 355!) The default is “no”. • PubkeyAuthentication yes Usually turned on by default, but if you have trouble with logging in without passwords, add to /etc/ssh/ssh config. • UsePrivilegedPort no This makes SSH work better with some firewalls. The default is “no” Also remove the SUID bit from /usr/bin/ssh: $ sudo chmod u-s /usr/bin/ssh c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.30 rsync: using it with SSH to mirror data 382 13.30 rsync: using it with SSH to mirror data More often than scp I use rsync to transfer data. • Advantages: ◦ rsync can preserve the time stamps, ownership, and can transfer device files and symbolic links accurately ◦ rsync only transfers the differences between files, not the whole file. – For example, if a single 50 megabyte email file has one email added at the end, and one deleted from the middle of the file, only a very small fraction of the total file will be transferred. ◦ The tool of choice for mirroring web sites, ftp sites, and for downloading ISO images. • To use it with Secure Shell, either: ◦ use the option -e ssh with rsync, or ◦ Add the following line to your login script: export RSYNC_RSH=ssh • Most useful options (besides -e ssh): -a Preserve all properties (permissions, ownership, use recursion to copy subdirectories, keep synmbolic links,. . . -v verbose; show the files as they are transferred; -z use compression. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 383 SSH — The Secure Shell 13.31 Examples of using rsync • Make a copy of ∼/tmp on ictlab in the current directory. $ rsync -avz -e ssh ictlab:tmp . ◦ If tmp is a directory containing files, – all of the files will be copied, and – their ownership and permissions will be preserved as far as possible – If you are root, they will be preserved exactly – Without a trailing ‘/’, the directory tmp will be created on the local computer in the current directory. • Make a copy of ∼/tmp on ictlab in the current directory. $ rsync -avz -e ssh ictlab:tmp/ . ◦ If tmp is a directory containing files, same as previous example except that: – With a trailing ‘/’, the directory tmp will not be created on the local computer in the current directory; the files and subdirectories of ∼/tmp on ictlab will be copied to the current directory. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.32 Using ssh from Windows, with Cygwin 384 13.32 Using ssh from Windows, with Cygwin • Install Cygwin by: ◦ Go to http://cygwin.com/, click on the “Install Cygwin now” icon ◦ Install at least XFree86, openssh and cygrunsrv, but I prefer to install the lot (it makes Windows much less painful to me). Note that Henry has already installed cygwin on all the computers in our A204 laboratories. • From the Cygwin icon on your desktop, or the Cygwin menu item at Start → Programs → Cygwin, start a bash shell. • At the bash prompt, type: $ startx & On older Cygwin installations in our laboratories, this starts a twm window manager session. You can type $ man twm on the Windows machine (if you installed man), or on any Linux machine for the manual that explains how it works. • In a terminal window in the resulting X session, type: $ ssh -X your user name @ IP address • When you connect to the remote machine, you can run any graphical application on that machine, and display it locally, so you could run emacs remotely on this Windows machine in a graphical mode: $ emacs assignment & www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 385 SSH — The Secure Shell 13.33 What else can SSH do? • Can execute programs remotely (i.e., for doing a backup of a remote computer) • Can do “port forwarding”, allowing amazing flexibility in sending any protocol over the encrypted tunnel, so that the application appears to be executing locally. Useful for VNC (http://www.uk.research.att.com/vnc/sshvnc.html), many other applications. • Can provide a basic type of encrypted tunnel, a simple VPN (virtual private network) • Many other things. There is a great book published in 2001 about it: Daniel Barrett and Richard Silverman, SSH, The Secure Shell: The Definitive Guide see details at http://www.oreilly.com/catalog/sshtdg/ (the Snail Book) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.34 Summary 386 13.34 Summary • SSH allows remote login using strong encryption • The host public key must be transferred using some other secure means: if you transfer it automatically, it is subject to a trivial man-in-the-middle attack. • The host key pair performs two importand functions: ◦ authenticate the server (only possible if you transfer the public host key from the server to the client in some secure way, such as by floppy disk) ◦ begin encryption • The user key pair has only one role: to provide strong authentication of the user. • The agent is used to hold the decrypted private user key on the client computer. Without the agent, you would need to type your pass phrase with every SSH connection. • The public user key is appended to the file ∼/.ssh/authorized keys in your user account on the servers you will connect to. You can do this over the network once the public host key is securely copied. • The actual encryption is done using the session key. The session key is never stored on the hard disk, and a new pair is generated regularly during a session. • The session is encypted using symmetric encryption, because it is much faster than public key (asymmetric) encryption. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 387 SSH — The Secure Shell 13.35 SSH References • A useful guide to OpenSSH: http: //perso.club-internet.fr/ffaure/openssh.html • A concise guide to OpenSSH: http://www.samag.com/archive/0910/feature.shtml, the SysAdmin Magazine • The OpenSSH press page lists many useful resources: http://www.openssh.com/press.html • The Red Hat Reference Guide has a chapter on SSH: http://www.redhat.com/docs/manuals/linux/ RHL-9-Manual/ref-guide/ch-ssh.html, or locally, http://ictlab.tyict.vtc.edu.hk/doc/rhl-rg-en-9/ ch-ssh.html • The Snail Book FAQ: http://www.snailbook.com/faq/ (and, of course, the Snail Book itself in the library) • SSH FAQ: http://www.onsight.com/faq/ssh/ssh-faq.html • OpenSSH FAQ: http://www.openssh.com/faq.html • USENET SSH FAQ: http: //www.faqs.org/faqs/computer-security/ssh-faq/ • ssh mailing list: http: //www.onsight.com/faq/ssh/ssh-faq-8.html#ss8.3 • OpenSSH developer’s mailing list: http://www.openssh.com/list.html • Don’t forget the OpenSSH man pages! c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.36 Secure Shell Exercises 388 13.36 Secure Shell Exercises ALL=(ALL) ALL 1. Set up sudo with a line like this: your-user-name as described in the handout about sudo. 2. Check that the directories /sbin and /usr/sbin are on your PATH: $ echo $PATH These directories contain system administrator’s commands. If you do not see them there, • add this command to your login script. Be careful: do not overwrite your login script! $ echo ’PATH=$PATH:/sbin:/usr/sbin’ >> ∼/.bash_profile • source this file into your shell process: $ source ∼/.bash_profile • Now check your PATH contains those two directories: $ echo $PATH 3. Identify your IP address, and tell your neighbour: $ ip addr 1: lo: mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 brd 127.255.255.255 scope host lo 2: eth0: mtu 1500 qdisc pfifo_fast qlen 100 link/ether 00:01:03:45:99:12 brd ff:ff:ff:ff:ff:ff inet 172.19.32.30/22 brd 172.19.35.255 scope global eth0 You can see my address is 172.19.32.30. 4. Connect to your neighbour’s computer, then log out again: $ ssh neighbour’s IP address $ exit 5. After your neighbour connected to you, compare the content of your ∼/.ssh/known hosts file with the public host key on your partner’s computer, in /etc/ssh/ssh host rsa key.pub. Are they different? Why or why not? 6. Use the scp program (see section 13.7 on page 358) to copy a file to the /tmp directory on your neighbour’s computer. 7. Log into your account on your neighbour’s computer (as described in section 13.6 on page 357) and verify that you copied the file to their machine. 8. Set up your own private and public keys as described in section 13.13. You should use a pass phrase for this exercise. Do this on the client machine only. 9. Set up your ∼/.ssh/authorized keys on the server using the methods described in section 13.15. Note: your network drive is the same for your account, both on the local client and on the remote server, so you can simply append your public key to your ∼/.ssh/authorized keys file without using scp. Think about it! www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 389 SSH — The Secure Shell 10. Setup ssh-agent on your client computer and use ssh-add to make your personal private key available to ssh, as described in section 13.20. 11. Configure keychain as explained in section 13.23 on page 374. 12. Demonstrate to your tutor that you can log in to your account on your neighbour’s computer without typing any passwords. Show that you can execute a graphical program, such as emacs or xclock. 13. Transfer the entire content of your home directory (complete with the directory name) to the /tmp directory of your neighbour, using rsync, preserving all timestamps and permissions. Use the --stats option to rsync. Verify the the permissions and time stamps are preserved. 14. Modify one or two files in your home directory a little. Repeat the data transfer of your home directory to the same location in your same neighbour’s /tmp directory, again using the --stats option. Do you notice any difference? 15. To avoid being hihacked, it is wise to transfer a host public key on a floppy disk and append it to the file /etc/ssh/ssh known hosts. (a) Should you take the host key from your client machine to the /etc/ssh/ssh known hosts on the server, or should you take the public key from the server and install it on your client? (b) What two important functions are performed using the host public and private key pair? (c) What function is performed by the user public and private key pair? 16. Install Cygwin on your Windows 2000 machine, and then set up the secure shell server, as described in http://tech.erdelynet.com/cygwin-sshd.html. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 13.37 Secure Shell Solutions 390 13.37 Secure Shell Solutions 5. The host public key (in /etc/ssh/ssh host rsa key.pub) is copied automatically over the network and appended to your ∼/.ssh/known hosts file if a copy is not stored in any of your known hosts files. So they must be the same. 11. Because the name service that I set up has been disabled by the technical staff, you will need to set the hostname of your computer as I described in section 13.25 on page 376. You will need to install the keychain shell script as I described in section 13.23 on page 374, and edit your login scripts ∼/.bash profile and ∼/.bashrc as described in section 13.26 on page 378. And no, copying this text into a text file and submitting it will not answer question 8 of your assignment! 13. Verify that your terminal window is open on your own computer by checking its $ ip a then just transfer it: $ rsync -avz -e ssh ∼ neighbour’s-IP-address :/tmp 15. (a) Take the public host key from the server to your client machine. Use a floppy disk or some secure means of transfer (that does not use SSH to the server) to avoid a man-in-the-middle attack. Specifically you could do it like this. On the server: $ mount /mnt/floppy $ cp -a /etc/ssh/ssh_host_dsa_key.pub /mnt/floppy $ umount /mnt/floppy Then take this floppy to your workstation, and then: $ mount /mnt/floppy $ sudo sh -c "cat /mnt/floppy/ssh_host_dsa_key.pub >> /etc/ssh/ssh_known_hosts" $ umount /mnt/floppy A simpler way to avoid hijacking: Note that a possibly more convenient alternative to transferring the key on a floppy disk is to first print out the fingerprint of your public host key, and put this in your wallet. Then, when you log into your machine from a new remote host, check that the fingerprint matches that on the paper in your wallet, and you are safe from hijacking if they match. Here is how you can determine the fingerprint of your host key: $ ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key 1024 c6:23:d6:5b:e9:7c:d2:04:0e:69:63:94:c4:3b:b6:95 /etc/ssh/ssh_host_rsa_key.pub (b) The host public and private key pair are essential to: i. authenticate the server to your client machine (to ensure that no one is performing a man-in-the middle attack on you) ii. initiate the creation of the session keys, used to encrypt all subsequent communication. (c) The client user account is authenticated to the server by the fact that it has the user’s private key, that matches the user public key held in the user’s account on the server. 16. Try it; it’s really worth while, especially if the server has the Microsoft resource kit installed (or most of cygwin, and Perl and some useful Windows system administration modules). You could then adminster the Windows machine over a slow network connection, even it were on the other side of the world. All you need to do is: www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 IP address: 391 SSH — The Secure Shell (a) Go to http://cygwin.com/, click on the “Install Cygwin now” icon (b) Install an absolute minimum of cygrunsrv, openssh, but I prefer to install the lot (it makes Windows much less painful to me). Note that Henry has already installed cygwin on all the computers in our A204 laboratories. (c) Open a bash shell, and type: $ ssh-host-config -y  ¨ Enter © accept the default when asked about what value to set CYGWIN to to Just press  (d) Then type: $ cygrunsrv -S sshd and you can then go and log into the machine remotely. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk Module 14 Shared File Systems Objectives After completing this chapter you should be able to: • Understand basic remote file and print sharing • Appreciate the pros and cons of Samba and NFS • Install Samba and NFS servers • Configure basic Samba and NFS services • Access remote resources using SMB and NFS 393 14.1 NFS (Network File System) 394 14.1 NFS (Network File System) • NFS developed by Sun Microsystems (early 80’s) • Native method for file sharing between Unix/Linux systems • Stateless protocol ◦ Means server keeps no state ◦ Renders server crashes ‘easily recoverable’ • Should be compatible with all Unix-like systems • Best in trusted environment, not highly secure • Best where all user/group IDs are same • Often used with Network Information Services (NIS) to synchronise user/group IDs www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 395 Shared File Systems 14.2 NFS Basics . . . continued • Systems are clients, servers or both • Clients import shared filesystems • Servers export shared filesystems • Servers easy to implement via network daemons • Clients require kernel modifications • Linux systems normally work as both already • NFS is NOT Unix/Linux specific (e.g. PC-NFS) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 14.3 Exporting File Systems 396 14.3 Exporting File Systems • Exporting handled by daemons rpc.nfsd and mountd • Must be running for NFS export to work • Exported file systems listed in /etc/exports, format is: fsname hostname(flags) [hostname(flags)] • Example: /tmp *.blah.co.uk(ro) Exports /tmp to all systems belonging to domain read-only ∗ • Important flags: ◦ ro (read only) ◦ rw (read/write) ◦ all squash (map all uid/gid to something) ◦ anonuid (specify user ID to map to) ◦ anongid (specify group ID to map to) • After changing /etc/exports, restart NFS exportfs -av or killall -HUP rpc.nfsd killall -HUP mount or /etc/rc.d/init.d/nfs restart ∗ For full detail on flags use man exports www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 397 Shared File Systems 14.4 Viewing exports • Use showmount: $ showmount -e $ showmount -e hostname Export list for landlord.gbdirect.co.uk: /usr/local/gbdirect/cvsroot roti.gbdirect.co.uk /home/adamg roti.gbdirect.co.uk /home/andylong along2.gbdirect.co.uk /home/mikeb kebab.gbdirect.co.uk /mnt/cdrom • NFS uses a portmapper to handle requests • This must be running (and you must have access to it) to use NFS • Check that hosts.allow contains an entry to permit you access, e.g. ◦ portmap: or ◦ portmap: ALL my.ip.network. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 14.5 Importing File Systems 398 14.5 Importing File Systems • Mount a remotely exported directory • Usually have to be superuser $ mount hostname:/sharename /local/directory • If successful, the export named /sharename on host hostname is mounted on your mountpoint /local/directory • Files accessed just as if local • Remote host must be exporting the directory • You must have access permission • Your local mountpoint must exist • Exactly like mounting a device www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 399 Shared File Systems 14.6 Samba • Implementation of Server Message Block protocol (SMB) ◦ Core of Microsoft’s file and print sharing ◦ Now ‘re-invented’ as CIFS • Developed in Australia by Andrew Tridgell et al • Info, sources, distributions at www.samba.org • High performance – competitive with NT • Server is purely application code ◦ Not part of the OS • Provides some clients ◦ smbfs requires OS support ◦ Client module smbfs not part of Samba c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 14.7 Samba — Availability 400 14.7 Samba — Availability • Samba is provided packaged with all large Linux distributions • PDC support for Windows 2000 clients is new, currently only with version 2.2 and is available from CVS • Samba 2.2 is currently alpha quality software ◦ will have reached production quality by your graduation • Nick Urbanik has packaged the current Samba 2.2 into an RPM, ready for installation into Red Hat 7. • Currently available from CSAlinux:/var/ftp/pub/samba, http://CSAlinux.tycm.vtc.edu.hk/ftp/samba/, ftp://CSAlinux.tycm.vtc.edu.hk/ftp/pub/samba/, \\CSALINUX\pub\samba • Expect further updates. I will improve the RPM to work as a PDC with minimum cutomisation required. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 401 Shared File Systems 14.8 Samba Documentation • The book Using Samba is distributed free with Samba (or buy for HK$315) • Documentation about using Samba as a PDC is currently available from http: //us1.samba.org/samba/docs/samba-pdc-faq.html and http: //us1.samba.org/samba/docs/samba-pdc-howto.html. • Latest docs are available from http://us1.samba.org/samba/docs/ • The documentation for the Samba configuration file is important: $ man smb.conf c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 14.9 Samba Installation 402 14.9 Samba Installation • Will vary — may come preinstalled, may come as RPMs or similar • Key components are nmbd and smbd ◦ nmbd is the name services daemon; mostly fit-and-forget ◦ smbd is the samba server; listens for connections and then forks one copy per client • Other tools & utilities exist, e.g. smbclient • Configuration file is /etc/samba/smb.conf • Comes with the Samba Web Administration Tool (swat); listens on port 901 • To install Nick’s RPMs, do: $ sudo mount CSAlinux:/var/ftp/pub /mnt $ cd /mnt/samba $ sudo rpm -Uhv samba*.i386.rpm www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 403 Shared File Systems 14.10 Samba Basics • Most likely started as daemons in init scripts • Can be run-on-demand via inetd, but unlikely ◦ Gives poor performance • Exclusively uses TCP/IP. Microsoft clients need to be configured for it — they may use NETBEUI • Permits: ◦ full file sharing, browsing and domain controller services ◦ full access to printers ◦ extensive customising c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 14.11 Access to Files and Printers 404 14.11 Access to Files and Printers • Linux and Win/NT access controls don’t match • Various options can be set • Attempts to match logged-on Windows Username to Linux user names and passwords • Modern versions use encrypted passwords — takes some setting up (see documentation) • Has concept of ‘guest’ users — may map to ‘nobody’ on Linux • Take a look in your smb.conf file and read man smb.conf www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 405 Shared File Systems 14.12 Testing Samba • Use smbclient (see screen dump below) • May need to provide a password • Check DIAGNOSIS.TXT from distribution (usually installed at /usr/share/doc/samba-2.x.x) if you have problems $ smbclient -L localhost Added interface ip=192.168.0.129 bcast=192.168.0.255 nmask=255.255.255.0 Password: Domain=[GBDIRECT] OS=[Unix] Server=[Samba 2.0.3] Sharename -------www software tmp admin printers IPC$ okirmt txtdj djrmt fax Server -------LANDLORD Workgroup -------GBDIRECT WORKGROUP Type --Disk Disk Disk Disk Printer IPC Printer Printer Printer Printer Comment ------WWW files Installable Software Temporary file space GBdirect admin files All Printers IPC Service (Samba Server) Comment ------Samba Server Master ------LANDLORD KEBAB c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 14.13 Smbclient 406 14.13 Smbclient • Numerous options: smbclient servicename [password] [-s smb.conf] [-B IP addr] [-O socket options][R name resolve order] [-M Net-BIOS name] [-i scope] [-N] [-n NetBIOS name] [-d debuglevel] [-P] [-p port] [-l log basename] [-h] [-I dest IP][-E] [-U username] [-L NetBIOS name] [-t terminal code][-m max protocol] [-W workgroup] [-TIXFqgbNan] [-Ddirectory] [-c command string] • Example: $ smbclient //landlord/admin Added interface ip=192.168.0.129 bcast=192.168.0.255 nmask=255.255.255.0 Password: xxxxx Domain=[GBDIRECT] OS=[Unix] Server=[Samba 2.0.3] smb: \> ls q3.dir actwin2 courses cvs domreg finance informat julie ............. etc 85 Tue D 0 Sun D 0 Wed D 0 Mon D 0 Tue D 0 Thu D 0 Wed D 0 Fri ........... Jun Mar May Mar Sep Jul Jun Jul 29 7 12 22 1 1 23 2 13:01:44 22:01:28 10:02:20 12:36:13 10:14:12 12:33:49 09:56:34 10:06:43 1999 1999 1999 1999 1998 1999 1999 1999 www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 407 Shared File Systems 14.14 Samba configuration File • Three sections to smb.conf ◦ global ◦ directories ◦ printers, if enabled, will export the printers known in /etc/printcap • Lots of help in the book Using Samba, on line with installation. ◦ With Red Hat 7, and Nick’s RPM, it is available under /usr/share/swat/using samba/ • Lots of other documentation comes with Samba: ◦ Usually under /usr/share/doc/samba-versionnumber, e.g. /usr/share/doc/samba-2.2.2 • Read the man pages • Via the web • and others c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 14.15 Samba Configuration Example 408 14.15 Samba Configuration Example • This is an example /etc/samba/smb.conf, suitable for use with Nick’s Samba RPM: [global] security = user status = yes workgroup = { Your domain name here } wins server = { ip of a wins server if you have one } encrypt passwords = yes domain logons =yes logon script = scripts\%U.bat domain admin group = @smbadm add user script = /usr/sbin/useradd -n -g machines -c Machine -d /dev/null -s /bin/false %m$ share modes=no os level=65 [homes] guest ok = no read only = no create mask = 0700 directory mask = 0700 oplocks = false locking = no [netlogon] path = /var/samba/netlogon writeable = no guest ok = no www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 409 Shared File Systems 14.16 Directories for Samba as a PDC • Need some directories to hold user profiles and login scripts ◦ match the above configuration $ sudo mkdir -p /var/samba/netlogon/scripts $ sudo chown -R root.root /var/samba/netlogon $ sudo chmod -R 755 /var/samba/netlogon c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 14.17 Testing Samba 410 14.17 Testing Samba • Use testparm and smbstatus ◦ testparm is used before starting Samba to check that smb.conf is ok ◦ smbstatus reports status of Samba, all connected clients and file share modes Notes on Testing Samba • Note that Samba is a server implementation • Has ftp-like smbclient, but file share access is provided the kernel. • Cannot be used by Linux to import shared files, only export them • Some Linuxes have import facilities too — but requires kernel support (smbfs module) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 411 Shared File Systems 14.18 1. NFS Exercises (a) Set up your local host so you can use showmount to show exported directories. (b) Find other hosts on your network which list exports. (c) Set up your host to export /tmp (d) Go to some other system and mount the exported /tmp (e) Play with file access on the mountpoint!, e.g. Try accessing files you normally wouldn’t have access to, creating files and seeing what the ownership and permissions are on the local copy. 2. Samba (a) Locate the file DIAGNOSIS.txt (b) Read through it, then carefully work through all of its instructions to check your Samba installation. (c) Run testparm on your current smb.conf, pipe the output through less to see the results. (d) Run smbstatus and explain to your neighbour what the results mean. (e) Set up a share so that your /etc directory is exported read-only and test it with smbclient. (f) Figure out how to export users’ home directories and get a colleague to test your work. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 14.19 Solutions 412 14.19 1. NFS Solutions (a) You should ensure that the portmap and nfs services are running before using showmount (b) You can give showmount a hostname to query, e.g. $ /usr/sbin/showmount -e somehost Export list for somehost: /home/adamg roti.gbdirect.co.uk /home/lee rafters.gbdirect.co.uk /backup /home/james oakleigh.gbdirect.co.uk /mnt/cdrom (c) You should add the following to /etc/exports: /tmp (d) (e) *(ro) 2. Samba (a) The file should be in /usr/share/doc/samba-x.xx/docs/textdocs (b) You should carry out all the test given to reach a working samba system (c) Check that your smb.conf is correct (d) Check the various smb manpages (smb.conf, smbd, smbstatus to see what the output means (e) You should add the following to your smb.conf, and restart samba with /etc/rc.d/init.d/smb restart: [etcshare] path = /etc comment = Shared etc directory writeable = no browseable = yes www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 413 Shared File Systems (f) You should ensure that the homes share is uncommented in smb.conf, and restart samba if necessary. You can test this by using: $ smbclient ’\\localhost\username’ added interface ip=192.168.0.135 bcast=192.168.0.255 nmask=255.255.255.0 Password: Domain=[MYGROUP] OS=[Unix] Server=[Samba 2.0.6] smb: \> dir .muttrc H 1387 Thu Jan 27 11:45:21 2000 .addressbook.lu H 2285 Mon Jan 24 14:37:29 2000 .procmailrc H 38 Mon Jan 24 18:11:04 2000 .newsrc.eld H 1151 Tue Jan 25 13:24:44 2000 .mail_aliases H 56 Thu Jan 27 16:52:13 2000 Desktop D 0 Tue Feb 8 15:48:40 2000 .opera DH 0 Thu Jan 27 12:53:23 2000 .balsarc H 1391 Wed Feb 9 14:27:36 2000 .mozilla DH 0 Wed Feb 9 10:27:15 2000 ltculogo.gif 8202 Wed Feb 9 11:49:06 2000 LANDLORD D 0 Wed Feb 16 14:49:16 2000 nltculogo.xcf 53886 Wed Feb 9 12:54:53 2000 nltculogo.gif 8398 Wed Feb 9 13:09:16 2000 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk Module 15 Apache Basics Objectives On completion of this module you should be able to: • Install the Apache webserver • Perform basic configuration 415 15.1 What is Apache? 416 15.1 What is Apache? • Apache is the most widely-used web-server∗ • Listens for requests and hands something back • Normally the contents of a file ◦ Possibly the result of a program • Designed to be stable and configurable ◦ Fast at serving synamic content ◦ Use kernel based http server tux or khttpd for static content for maximum speed 61.88% of all servers as of February 2001 (Netcraft — http://www.netcraft.com/ Survey/Reports/200102/platform.html) www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 417 Apache Basics 15.2 Installation • Basic installation is easy • You may be able to install from your distribution ◦ Most come with Apache • Otherwise just follow the download instructions from the official site ◦ http://www.apache.org/ • Then follow the instructions in the INSTALL file ◦ Normally just $ ./configure $ make $ make install ◦ If you have problems check the docs ◦ Available at http://www.apache.org/docs c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 15.3 How Apache Listens 418 15.3 How Apache Listens • Apache runs several processes at any one time ◦ Parent and several children • Parent ‘watches over’ the children ◦ Tracks how many are answering requests ◦ Spawns more if free processes drop below a certain point ◦ Kills spare processes if there are lots free • Configure child numbers using MinSpareServers and MaxSpareServers directives ◦ Default is reasonable for a small business ◦ Tune it for busier sites www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 419 Apache Basics 15.4 Configuration File(s) • If compiled from source, Apache installs in /usr/local/apache ◦ Earlier versions installed under /usr/local/etc/httpd ◦ Your distribution may differ again . . . • Configuration file is called httpd.conf ◦ Older versions use – httpd.conf – srm.conf – access.conf • Controls what requests Apache answers ◦ and how . . . ∗ Redhat installs config files under /etc/httpd and the sample web pages and logs directories under /home/httpd c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk ∗ 15.5 Key Configuration Directives 420 15.5 Key Configuration Directives • Wide range of configuration directives • For a very basic server you need at least the following: ◦ ServerRoot ◦ DocumentRoot ◦ ServerAdmin ◦ BindAddress ◦ Port ◦ Listen ◦ User ◦ Group www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 421 Apache Basics 15.6 ServerRoot, DocumentRoot • Tells Apache where its files live • ServerRoot tells Apache where its conf and logs directories live ◦ Not always necessary ◦ Good practice to have it • DocumentRoot tells Apache where to look for documents to serve up • Requested filenames are appended to this • If you have DocumentRoot /var/www/html then a request to http://www.domain.co.uk/foo.html points to the file /var/www/html/foo.html c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 15.7 Is Apache running? 422 15.7 Is Apache running? • Sometimes it is useful to check the server using the telnet program: $ telnet csalinux 80 Trying 192.168.128.53... Connected to CSAlinux.tycm.vtc.edu.hk (192.168.128.53). Escape character is ’^]’. GET / HTTP/1.0 HTTP/1.1 200 OK Date: Mon, 05 Mar 2001 01:51:59 GMT Server: Apache/1.3.14 (Unix) (Red-Hat/Linux) mod_ssl/2.7.1 OpenSSL/0.9.5a DAV/1.0.2 PHP/4.0.4pl1 mod_perl/1.24 Last-Modified: Wed, 28 Feb 2001 05:23:07 GMT ETag: "28363-129e-3a9c8b3b" Accept-Ranges: bytes Content-Length: 4766 Connection: close Content-Type: text/html CSA Linux ... Connection closed by foreign host. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 423 Apache Basics 15.8 ServerAdmin • Apache sometimes can’t complete requests • In these cases it serves up an error page • ServerAdmin is given as a contact address • Usually set to something like webmaster@tycm.vtc.edu.hk ◦ You should of course ensure that it is a valid email address • Possible to specify a different error page ◦ Doesn’t have to use ServerAdmin c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 15.9 BindAddress, and Port 424 15.9 BindAddress, and Port • Tells Apache which requests to answer • By default Apache listens to every IP address on your machine ◦ But only to the port given by the Port directive • BindAddress 192.168.0.1 tells Apache to ignore anything that doesn’t come in on 192.168.0.1 • Port 8080 ignores all but the specified port • You can use more than one Port directive, e.g. Port 80 Port 8080 • If you don’t specify a port then a default is used∗ • You can only use one BindAddress! This is usually 80, but if you are using a binary package then bear in mind whoever compiled your package may have chosen a different value www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 ∗ 425 Apache Basics 15.10 Listen • Listen is a replacement for BindAddress and Port • Given IP:port or just port, e.g. Listen 192.168.0.1:8080 will answer requests on the IP address 192.168.0.1 and port 8080 and no others • To answer requests to all valid IP addresses, but only a certain port (e.g. 80) use: Listen 80 • Can use more than one Listen directive • Should be used instead of BindAddress and Port in new servers c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 15.11 User and Group 426 15.11 User and Group • Apache should normally be started as root ◦ So it can change the user ID of the children ◦ These should not run as root • User and Group directives say what user/group the children should run as ◦ Important security feature • Should be set to something that has no real power on your system ◦ Most people use user and group nobody • Web documents should be readable by this user • Nothing should be writeable except log files www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 427 Apache Basics 15.12 Apache Processes • Looking at a process list∗ you can see ◦ The parent root S Jul 4 0:37 /usr/local/apache/bin/httpd -d /www ◦ The children nobody nobody nobody nobody nobody nobody S S S S S S 10:11 10:20 10:58 10:58 11:06 11:13 0:00 0:00 0:00 0:00 0:00 0:00 /usr/local/apache/bin/httpd /usr/local/apache/bin/httpd /usr/local/apache/bin/httpd /usr/local/apache/bin/httpd /usr/local/apache/bin/httpd /usr/local/apache/bin/httpd -d -d -d -d -d -d /www /www /www /www /www /www • Spare processes don’t use processor time ◦ They are ‘sleeping’ • They do use memory, however ◦ Negligible for a default Apache ◦ Watch carefully the more modules you add! ◦ Particularly, mod perl adds a heavy memory requirement. ∗ Some fields from the ps output have been left out to aid clarity www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 15.13 Logging 428 15.13 Logging • Apache can log information about accesses • Use the TransferLog and ErrorLog directives • TransferLog logs/access log will log all requests in the file ServerRoot/logs/access log • If the filename starts with a / then it is treated as a proper pathname, not appended to ServerRoot • ErrorLog is similar but controls where error messages go ◦ Useful for debugging CGI scripts and misconfigurations ◦ Check here first if Apache won’t start www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 429 Apache Basics 15.14 Customizable Logging • Customizable logs available with CustomLog CustomLog filename format-string • format-string consists of ‘% directives’ and/or text • % directives include: %b %f %{headername}i %P %r %t %T %u %U %v Bytes sent, excluding HTTP headers Filename The contents of headername: header in the request The process ID of the child that serviced the request First line of request Time, in common log format time format The time taken to serve the request, in seconds Remote username (may be bogus if return status (%s) is 401) The URL path requested The ServerName of the server answering the request c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 15.15 CustomLog examples 430 15.15 CustomLog examples • To log the referer information in the file ServerRoot/logs/referer CustomLog logs/referer "%r Refered by: %{Referer}i" • % directives can be conditional on reply status CustomLog logs/referer "%r Refered by: %200,304,302{Referer}i" ◦ Logs the refering page only on status 200,304,302 ∗ • For full details consult the Apache documentation ◦ Gives list of all possible % directives ∗ For full details consult the Apache documentation www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 431 Apache Basics 15.16 Example Configuration • A sample configuration file could look like this: ServerRoot /usr/local/apache DocumentRoot /usr/local/apache/htdocs ServerAdmin webmaster@domain.co.uk Listen 192.168.0.131:80 User nobody Group nobody ErrorLog /usr/local/apache/logs/error_log • We recommend starting with the default httpd.conf rather than from scratch ◦ Correctly configures many things for you • The default is well annotated ◦ Everything after a # character is a comment ◦ Ignored by Apache • Apache can check the syntax of its configuration ◦ httpd -t ◦ apachectl configtest . . . if you installed apachectl on your system. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 15.17 Basic Exercises 432 15.17 Basic Exercises 1. Apache Installation (a) Find out if Apache is installed on your machine . . . if not, install it. (b) Check Apache is running on your system. i. You should be able to point your web browser at http://127.0.0.1/ to check this ii. You might have to try http://127.0.0.1:8080/ (c) If Apache is not running, start it with /etc/rc.d/init.d/httpd start (d) If Apache still doesn’t appear to be running, find its configuration and log files and try to fix the error. 2. Basic configuration (a) Familiarise yourself with the httpd.conf file. (b) How would you change the directory where the log files are kept?. (c) How would you change the ’root’ for documents? (d) How would you enable symbolic links to be followed on the cgi-bin directory? Warning: this is a really bad idea! (e) Make your site only accessible on Port 8080 (f) Now make it only accessible on the IP address 127.0.0.1, and port 80 (g) Make the changes and check them. (h) Place the following line in your /etc/hosts file: IP_ADDRESS www.test.com www where IP ADDRESS is the IP address of your machine. You should now be able to browse http://www.test.com/ 3. Logging (a) Take a look at the access logs and familiarise yourself with the information they contain. (b) Set up a custom log to give the time of the request, the request, referer, and number of bytes sent, as well as the time taken to serve the request. (c) Alter your custom log to show the time taken and bytes sent only if a 200 status response occured. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 433 Apache Basics 15.18 Solutions 1. Apache Installation (a) If Apache is not installed you should be able to install it off a RedHat CD by mounting the CD and typing rpm -ivh /mnt/cdrom/RedHat/RPMS/apache.rpm (b) There are several ways to check this. One is to telnet to port 80 of your machine and see if you get a response. i. This should work for a default RedHat install, though the port number that Apache first listens on changes in various different packaging so you should try both 80 and 8080. (c) You can start Apache one of two ways (Which may be the same on some machines!) • /etc/rc.d/init.d/httpd start • somepath/apachectl start ∗ (d) If you can’t work out why Apache isn’t running ask the tutor for assistance. 2. Basic configuration (a) You should make sure that you understand everything in the httpd.conf including those sections that are commented out. (b) Alter the CustomLog and ErrorLog directives to change where the log files are kept, e.g. ErrorLog /var/log/myerrorlog CustomLog /var/log/myaccesslog common (c) The ’root’ for documents is specified by the DocumentRoot directive, e.g. DocumentRoot /path/to/my/web/documents (d) You can enable symbolic links by adding Options +ExecCGI FollowSymLinks to the section for your cgi-bin, e.g. Options +ExecCGI FollowSymLinks (e) Add/Change the Port directive in your httpd.conf file to read Port 8080 (f) Add the following to your httpd.conf: Listen 127.0.0.1:80 (g) Restart the server and try to access it on both port 80 and 8080. Check that it only works as you expect and fetches documents from the correct place. (h) Check that you can browse http://www.test.com ∗ You may have to dig a little to find where this script is www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 15.18 Solutions 434 3. Logging (a) Make sure you understand what each of the columns in the access logs is for. Try tailing the logs as you browse your webserver (b) The following should create a file newlogformat which holds the desired log format. LogFormat "%t %U %{Referer}i %b %T" newlog CustomLog logs/newlogformat newlog (c) Change your LogFormat line to LogFormat "$t %U %{Referer}i %200b %200T" newlog CustomLog logs/newlogformat newlog www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 Module 16 Apache Objectives On completion of this module you should be able to: • Set up virtual hosts on the Apache webserver • Use access controls • Set up basic authentication • Configure WebDAV web publishing 435 16.1 Two sites and more . . . 436 16.1 Two sites and more . . . • Many companies specialise in web hosting • One company may manage thousands of web sites • One solution is to buy thousands of servers and set up each web site on each server. ◦ Problem: cost ◦ Must be a cheaper way • There is a simpler way ◦ Called Virtual Hosting • Virtual hosting allows one server to provide many independent web sites ◦ each web site has its own name ◦ each web site is independent of the other ◦ Apache is better at this than any other web server www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 437 Apache 16.2 Two sites and more . . . continued • Apache can serve multiple sites easily • Known as ‘Virtual Hosting’, e.g. DocumentRoot /www/web.test2/docs ServerName www.test2.co.uk ServerAdmin www@test2.co.uk ErrorLog /www/web.test2/logs/error_log TransferLog /www/web.test2/logs/access_log to make apache answer requests to address 192.168.0.2 from /www/web.test2/docs • Your machine must answer to this address∗ • Apache must be listening on the address ◦ Listen 80 will make Apache answer to all available addresses on port 80 ◦ Note that each virtual web site may have its own logging • This is known as virtual hosting ∗ If you don’t know how to set up IP aliases ask the instructor www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 16.3 Virtual Hosting Options 438 16.3 Virtual Hosting Options • IP-based ◦ Each site must have a unique, IP address ◦ Uses up valuable IP addresses ◦ Site accessible by all browsers • Name-based ◦ Sites share an IP address ◦ Useful if short of available addresses ◦ Some browsers may have problems • Most use IP-based hosting where possible • Ensures maximum accessibility • However, Internic are now requiring all sites to use name based hosting as much as possible to reduce drain on IP addresses. • Name based hosting doesn’t work with SSL encryption. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 439 Apache 16.4 Name-based hosting • Name-based hosting looks like: NameVirtualHost 10.1.1.108 DocumentRoot /var/www/foo/docs ServerName foo.domain.com.hk ServerAdmin foomaster@domain.com.hk DocumentRoot /var/www/bar/docs ServerName bar.domain.com.hk ServerAdmin barmaster@domain.com.hk c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.5 Name-based hosting (continued) 440 16.5 Name-based hosting (continued) • NameVirtualHost tells Apache that an IP address can serve multiple hosts • VirtualHost sections describe how documents for each site are served ◦ Apache must be able to resolve the names in the directives to the IP address • Apache looks at the Host: header to decide which documents to serve ◦ Not sent by all browsers • Requests on other IP addresses will be processed as normal • Can use both IP-based and name-based hosting www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 441 Apache 16.6 IP-based hosting • IP-based hosting looks like: DocumentRoot /var/www/foo/docs ServerName foo.domain.com.hk ServerAdmin foomaster@domain.com.hk DocumentRoot /var/www/bar/docs ServerName bar.domain.com.hk ServerAdmin barmaster@domain.com.hk The differences from name-based virtual hosting include: ◦ The IP addresses must be different (and Apache must be listening to them!) ◦ There is no NameVirtualHost directive ◦ We may be wasting precious IP addresses! • The Apache manual contains a very useful and complete guide to implementing virtual hosts ◦ It is installed when you installed Apache ◦ On Red Hat, it is available as http://localhost/manual/ c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.7 Block Directives 442 16.7 Block Directives • Apache has several block directives ◦ Limit enclosed directives to apply to a certain set of ‘things’ • is a block directive ◦ Enclosed directives apply only to that virtual host • Others are: ... ... ... ... ... ... www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 443 Apache 16.8 Block Directives (continued) • Limits the enclosed directives to apply to everything below the directory name ◦ name can be anywhere on the filesystem ◦ Independent of DocumentRoot • is similar but is a URL path rather than a filesystem path • limits directives to files called name ◦ Path of the file is irrelevant ◦ Only checks the file name, not its location c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.9 DirectoryMatch, et al. 444 16.9 DirectoryMatch, et al. • DirectoryMatch, FilesMatch and LocationMatch are similar ◦ Accept regular expressions as arguments, e.g. ... • More flexible • Need more thought to match only intended files www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 445 Apache 16.10 Access Control using .htaccess files • Create a file in the directory to be protected ◦ Usually .htaccess or .acl ◦ Can be anything • Example: AuthType AuthName AuthUserFile AuthGroupFile require group require user Basic "Members Only" /etc/httpd/conf/auth.user /etc/httpd/conf/auth.group testgroup testuser • Only the user testuser, or a user in the group testgroup, may access files in this directory • Validation is done on the files /etc/httpd/conf/auth.user and /etc/httpd/conf/auth.group c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.11 Access Control (continued) 446 16.11 Access Control (continued) • Access control is off by default ◦ Unnecessary for many sites • Switched on by: AccessFileName .htaccess AllowOverride AuthConfig • AccessFileName identifies which filename(s) constitute an Access Control File • Every directory in the request path is checked for a relevant file • AllowOverride says that Access Control files can override authorisation directives only ◦ Can have other values ◦ Change behaviours through your .htaccess file ◦ See Apache docs for further details • Note that there is quite an overhead if turn on AllowOverride, since web server has to search for this file through each directory in the entire path of each document it fetches. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 447 Apache 16.12 Authorisation Files • Authorisation files are very straightforward • Group file is groupname: • For example: firstgroup: user1 user2 user3 secondgroup: user2 user3 user4 othergroup: user4 user5 user6 userlist • Listed users belong to that group • Create this file by hand c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.13 Authorisation Files (continued) 448 16.13 Authorisation Files (continued) • User file is a little more complicated • Format is username:encryptedpassword • For example: testuser:6SlrYaxUFml • Create/edit this with htpasswd ◦ Part of the Apache distribution ◦ Give it an authorisation file and a username $ htpasswd -m /etc/httpd/conf/auth.user newuser New password: Re-type new password: Adding password for user newuser www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 449 Apache 16.14 Access Control using httpd.conf • As alternative to using .htaccess files, can use main httpd.conf configuration file for apache ◦ centralised • Exactly the same as using the .htaccess files, but put into a block directive in httpd.conf • Refer to the worksheet How to create a password protected directory on a web server for more details. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.15 Pros and Cons of using Access Files for Authentication 450 16.15 Pros and Cons of using Access Files for Authentication • You have a choice to put authentication configuration into .htaccess files or into the main server configuration. • Each has advantages and disadvantages. httpd.conf: advantages .htaccess: advantages • The server does not waste time • Convenient to modify the configreading all directories looking uration; no need to reload the for .htaccess files server, just edit the .htaccess file. • The administrator can control all access to the server • Easy to delegate access control to other people httpd.conf: disadvantages .htaccess: disadvantages • Harder to delegate authentica- • The server needs to check evtion control to others ery single directory, starting with the root directory on the local • Need to reload the server to hard disk of the server, all the read a new configuration way down to the last directory. This slows the server down considerably. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 451 Apache 16.16 How Can Users Change Their Password? • Apache provides no solution to this directly • Many solutions to this problem are available • One of the best is user manage by Lincoln Stein, available at http://stein.cshl.org/∼lstein/user manage/ c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.17 WebDAV: a protocol for web collaboration 452 16.17 WebDAV: a protocol for web collaboration • WebDAV is a standard, open protocol for collaboration on the Web • Allows authors to write to a web server • WebDAV enabled software (such as Microsoft Office 2000) can edit documents directly on the web server, as if working with a local file • Currently provides three main facilities: 1. Locking: WebDAV prevents two authors writing to the same file at once 2. Properties: information is stored about each file 3. Namespace manipulation: you can copy and rename files, create collections (which are basically directories on the web server) • Clients include: ◦ Microsoft Internet Explorer 5 and later (“Web Folders”) ◦ Microsoft Office 2000 ◦ DreamWeaver 4.0 and later ◦ A number of other commercial and Open Source products. . . ◦ . . . but not FrontPage 2000! • See http://www.webdav.org/ www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 453 Apache 16.18 WebDAV and Apache • Apache has had support for WevDAV for some time. • Consists of an Apache module called mod dav • Provided with Red Hat 7.0, enabled by default • Very stable. • See http://www.webdav.org/mod dav/ and http://www.webdav.org/mod dav/install.html c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.19 WebDAV Configuration 454 16.19 WebDAV Configuration • mod dav requires a directory to store lock and property information • You need to provide authentication • Configuration options include: Directive value DAV On DAVLockDB lock file name DAVMinTimeout minimum lifetime of a lock in seconds www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 455 Apache 16.20 Apache WebDAV configuration example • Here is an example section from the /etc/httpd/conf/httpd.conf configuration file: DAVLockDB /var/lock/WebDAV/DAVLock DAV On Options Indexes AllowOverride None AuthType Basic AuthName "CM Web site management and upload" AuthUserFile /etc/httpd/conf/passwd Require valid-user • Could instead of ... use: Require valid-user (all on one line) c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.21 Configuring WebDAV: directories and files 456 16.21 Configuring WebDAV: directories and files • A web site controlled by WebDAV must be owned and writable by the process running Apache. • You need to create the lock file directory, and make this owned and writable by the same user. ◦ In the example above, you would do: $ sudo mkdir /var/lock/WebDAV $ sudo chown apache.apache /var/lock/WebDAV www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 457 Apache 16.22 What is WebDAV useful for? • Useful for a department to collaborate: ◦ provide a browsable repository of information ◦ members can directly edit these resources ◦ A democratised web • Useful for Home user web publishing. Quote from WebDAV in 2 Minutes: A home user can simplify his or her interface to a web server by interfacing with it through DAV (assuming support on the server side). First the user sets up their site on their home computer. Then, using Internet Explorer 5, they set up a Web Folder through the Add Web Folders icon. After providing information on their web server and user id, they can save to and access their directory on the server transparently by using the Web Folder on their desktop, which appears and behaves as a typical local folder. c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.23 What is the future of WebDAV? 458 16.23 What is the future of WebDAV? • WebDAV is a standard Internet protocol ◦ In other words, it is specified by the Internet Engineering Task Force (IETF) • IETF is working on specifying the following features: ◦ Advanced Collections: support for ordered collections, referential resources ◦ Versioning and Configuration Mangement: support for maintaining a complete history of all versions of a resource. ◦ Access Control: the ability to set and clear access control lists. • This will allow WebDAV to replace many current Internet protocols, such as POP3, IMAP and CVS. • Will have potential to help democratise the Web. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 459 Apache 16.24 Information about WebDAV • The best sources of information about WebDAV include: ◦ The home page: http://www.webdav.org/ ◦ The WebDAV FAQ: http://www.webdav.org/other/faq.html ◦ WebDAV in 2 Minutes: http://www.fileangel.org/docs/DAV 2min.html c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.25 Other useful directives 460 16.25 Other useful directives • There are around 200 Apache directives ◦ More if you add modules e.g. mod ssl • The previous ones are the essentials • Some other useful directives are given below: Directive Redirect url-path new-url RewriteRule pattern new-pattern AddEncoding type ext ForceType type HostNameLookups on—off—double ExpiresDefault Action Redirect Requests to url-path to new-url Rewrite requests, replace pattern with new-pattern Serve up documents with extension ext with encoding type type Force all documents to be served up with MIME type type Whether to do DNS lookups for logging purposes Set the default expiry time of documents www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 461 Apache 16.26 Examples Redirect permanent /ents/theatre/fab-gere http://www.fabgere.com Redirect /gbdirect/logo.gif http://www.gbdirect.co.uk/logo.gif Redirect permanent /gbdirect http://www.gbdirect.co.uk/ RewriteEngine on RewriteRule ^/linuxtraining.*\.htm /ltcu_moved.htm AddEncoding x-gzip gz ForceType text/plain HostNameLookups off ExpiresDefault "access plus 1 month" ExpiresByType text/html "access plus 1 week" c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.27 Exercises 462 16.27 Exercises 1. IP based hosting (a) Start with the default installation file and add an IP based virtual host: i. Add an IP alias for your machine (Ensure it doesn’t clash with any others on your network!) Here are two methods: • The simplest is to use the ifconfig program directly: $ sudo ifconfig eth0:0 ipaddress where ipaddress is the second IP address. If you want to add another alias, use eth0:1; use eth0:2 for the next alias,. . . • The other method uses the netcfg program: A. Use sudo netcfg, then click on the Interfaces tab, select the ethernet device, and click on the Alias button. B. Save your changes, activate the alias, and check that your address works, first by typing ifconfig, then see if you can ping the address. C. If the interface was not started, then do so with sudo ifup eth0, or sudo ifup eth0:0 ii. Create a dummy index page so you will be able to tell the difference between your two sites. Call the page index.html. Copy them to the document root for each site. iii. Set up Apache to serve this site and check from a browser that everything works (for both sites) as you expected. 2. Name based hosting (a) Set up your apache so that it will serve the same sites but on a single IP address (Name-based virtual hosting). i. First, edit your hosts table using sudo emacs /etc/hosts. ii. Add one line for each web site: put your main IP address first, then the name for the site. Example: 10.1.1.39 www.nice.com 10.1.1.39 www.acme.com 10.1.1.125 sales.acme.com iii. set up name based virtual hosting for the sites with the same IP address. Verify that you can read them. 3. Access control (a) Create two directories on one of your sites and set up access controls so that anyone can see the main index page, testuser can see the first directory and anyone in group testgroup can see the second. 4. WebDAV (a) Configure your main directory with WebDAV, then demonstrate that you can access this directory using the Web Folders option from Internet Explorer (File → Open) on one of the Windows 2000 clients. Make sure that the directory is protected using Basic Authentication. (b) Demonstrate that you can edit and save a file on the WebDAV-enabled server directly using Microsoft Word. www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 463 Apache 16.28 Solutions 1. IP based hosting (a) The first thing that you will have to do is set up an IP alias for your machine so that it has two distinct IP addresses. You might find it easiest to use the Red Hat program netcfg for this. If you aren’t sure how to achieve this ask the instructor. A list of spare IP addresses will be made available. An example from a working multi-hosted Apache is given below Listen 192.168.0.3:80 Listen 192.168.0.2:80 ServerAdmin webmaster@gbdirect.co.uk DocumentRoot /home/www/web.llord/docs ServerName llord.gbdirect.co.uk ErrorLog /home/www/web.llord/logs/error-log TransferLog /home/www/web.llord/logs/access-log ServerAdmin webmaster@gbdirect.co.uk DocumentRoot /home/www/web.trainingpages/docs ServerName trainingpages.gbdirect.co.uk ErrorLog /home/www/web.trainingpages/logs/error-log TransferLog /home/www/web.trainingpages/logs/access-log c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 16.28 Solutions 464 2. Name based hosting (a) An equivalent example using name-based hosting would be: NameVirtualHost 192.168.0.2 ServerAdmin webmaster@gbdirect.co.uk DocumentRoot /home/www/web.llord/docs ServerName llord.gbdirect.co.uk ErrorLog /home/www/web.llord/logs/error-log TransferLog /home/www/web.llord/logs/access-log ServerAdmin webmaster@gbdirect.co.uk DocumentRoot /home/www/web.trainingpages/docs ServerName trainingpages.gbdirect.co.uk ErrorLog /home/www/web.trainingpages/logs/error-log TransferLog /home/www/web.trainingpages/logs/access-log Note that the two names given llord.gbdirect.co.uk and trainingpages.gbdirect.co.uk should both resolve to 192.168.0.2 3. Access Control (a) You should create a file called .htaccess in both directories, the first should be: AuthType AuthName AuthUserFile AuthGroupFile require user Basic "First Directory" /etc/httpd/conf/auth.user /etc/httpd/conf/auth.group testuser and the second should be: AuthType AuthName AuthUserFile AuthGroupFile require group Basic "Second Directory" /etc/httpd/conf/auth.user /etc/httpd/conf/auth.group testgroup www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 Module 17 Key Configuration Files Objectives After completing this module, you should be able to configure the following: • The password files /etc/passwd, /etc/shadow • The group file /etc/group • cron management /etc/crontab • Kernel modules (/etc/modules.conf) • Filesystem mounting (/etc/fstab and /etc/exports) • System startup and shutdown scripts 465 17.1 /etc/passwd 466 17.1 /etc/passwd • Stores information about users ◦ Password (on some systems) ◦ Id, and primary group ◦ Finger information ◦ Home directory ◦ Default shell www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 467 Key Configuration Files 17.2 /etc/passwd (continued) • Colon-separated fields, e.g. lee:Df18jed/nienysd:501:501:Lee Willis,Rm 1,013 567,013 765:/home/lee:/bin/bash • First field is the username • Second is the encrypted password ∗ • Third and fourth fields give the user ID and the primary group ID respectively • Finger information is a comma separated list of information about a user ◦ Typically stores full name, office room, office phone number and home phone number • The sixth field is the user’s home directory • The user’s default shell is given by the last field On systems which support shadow passwords this will just be an x, see 17.8 for an explanation c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk ∗ 17.3 Editing /etc/passwd 468 17.3 Editing /etc/passwd • You should never edit /etc/passwd directly ◦ Can lose information on multi-user systems • Use the passwd command • Normal users simply type passwd ◦ Prompted for old password ◦ Type new password twice (to avoid typos) • Superuser can change anyone’s password passwd username ◦ Enters only the new password ◦ Don’t have to know the old password • Superuser may also disable/enable accounts ◦ passwd -l username disables or locks an account ◦ passwd -u username unlocks the account www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 469 Key Configuration Files 17.4 Other Changes To /etc/passwd • chfn allows you to change the finger information for a user e.g. $ chfn -f "Lee Willis" -o "Room 1" -p "01234 5678" -h "0123 45678" • chsh -s shell lets you change your default shell ◦ Must be listed in /etc/shells ◦ chsh --list-shells will give a list of valid values • Example: $ chsh --list-shells /bin/bash /bin/sh /bin/ash /bin/bsh /bin/tcsh /bin/csh $ chsh -s /bin/tcsh lee Changing shell for lee. Password: Shell changed. • Note: Both chfn and chsh require you to give your password c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.5 /etc/group 470 17.5 /etc/group • Effective control of file access is one of the strengths of Linux/Unix • One aspect of this is the concept of groups • Users belong to one or more of these groups • Access to files can be granted or denied on the basis of group priveleges • Group membership is controlled by the file /etc/group www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 471 Key Configuration Files 17.6 Editing /etc/group • Like /etc/passwd shouldn’t be edited directly • Tools can change it and ensure locking • To create a group with ID gid and name gname: $ groupadd -g gid gname • To change name of group gname to newname: $ groupmod -n newname gname • usermod changes the groups a user belongs to, e.g. to add the user lee to groups www, project, and tempgroup: $ usermod -G www,project,tempgroup lee • N.B. It also removes him from any groups not listed (excluding his primary group) • usermod can also change the information in /etc/passwd ◦ Can only be run by the superuser c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.7 Important Note 472 17.7 Important Note • Changing user information shouldn’t be undertaken lightly • There are a number of restraints on changing usernames, IDs, and group IDs • You can’t change name while a user is logged in • You can’t change ID while user has processes running • See man usermod and man groupmod for others • Mostly common sense www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 473 Key Configuration Files 17.8 Shadow Passwords • shadow passwords are a security feature ◦ Normal users could get others’ passwords if encrypted versions were readable ◦ Some information in /etc/passwd needs to be readable, but Passwords don’t! • Solution: ◦ Keep everything except passwords in /etc/passwd ◦ Password field contains just a single ’x’ • Encrypted passwords are stored in /etc/shadow ◦ Only readable by superuser c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.9 /etc/shadow 474 17.9 /etc/shadow • /etc/shadow also stores other information • Mainly password expiry information • Can force users to change their password • Most important benefit is increased security • All modern systems should use shadow passwords www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 475 Key Configuration Files 17.10 Scheduling Jobs (Cron) • cron schedules jobs to run at times; specified in the file /etc/crontab SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root HOME=/ # run-parts 01 * * * * root run-parts /etc/cron.hourly 02 4 * * * root run-parts /etc/cron.daily 5,35 * * * 1-5 root /usr/local/bin/domail • The first section sets environment variables • Cron jobs run when the current time/date matches a crontab entry • The first 5 fields in /etc/crontab are minute hour day_of_month month day_of_week • * Matches all possible values • Commas separate sets of values within a field • Ranges can also be specified, e.g. [1-5] • Can also specifiy steps, e.g. [0-59/5] c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.11 /etc/crontab 476 17.11 /etc/crontab • /etc/crontab also specifies what user the job runs as, e.g. 01 * * * * root run-parts /etc/cron.hourly 02 4 * * * root run-parts /etc/cron.daily • Second line runs the command run-parts /etc/cron.daily as root at 4:02am every day • To run the command /usr/local/bin/domail as root at 10 minutes past and 40 minutes past the hour, between 9am and 6pm on weekdays: 10,40 9-18 * * 0-5 mail /usr/local/bin/domail www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 477 Key Configuration Files 17.12 run-parts • run-parts is a script designed for use with cron • Runs all the programs in the specified directory • Allows administrators to easily add jobs ◦ Simply place an executable script/program in the correct directory • N.B. Not a standard cron feature c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.13 logrotate 478 17.13 logrotate • Log rotation is normally handled by logrotate • Run by cron, which reads /etc/logrotate.conf for configuration • Example: $ cat /etc/logrotate.conf #Rotate the logs weekly weekly # keep 4 weeks worth of backlogs rotate 4 # send errors to root errors root # create new (empty) log files after rotating old ones create # uncomment this if you want your log files compressed #compress # RPM packages drop log rotation information into this directory include /etc/logrotate.d # no packages own lastlog or wtmp --- we’ll rotate them here /var/log/wtmp { monthly rotate 1 } /var/log/lastlog { monthly rotate 1 } www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 479 Key Configuration Files 17.14 Module Configuration • The Linux kernel can be modular in nature • Needs to know which devices use which drivers • /etc/modules.conf contains this information ∗ • Typical file may look like: alias eth0 ne2k-pci alias eth1 3c509 • States that the device eth0 requires the module ne2k-pci, and eth1 requires 3c509 ∗ Warning, on some older systems this is /etc/conf.modules www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 17.15 Modules Configuration — ‘Options’ 480 17.15 Modules Configuration — ‘Options’ • Some modules allow you to specify options • Mainly used for ISA peripherals, e.g. to provide I/O and IRQ information: alias eth0 ne options ne irq=10 Specifies that eth0 requires the module ne which should be passed the argument irq=10 • Can also specify actions to be executed when loading unloading modules, e.g. pre-install pcmcia_core /etc/rc.d/init.d/pcmcia start Run /etc/rc.d/init.d/pcmcia start before loading the pcmcia core module www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 481 Key Configuration Files 17.16 Mounting Filesystems • Linux can store its files on multiple disks • It decides what part of the filesystem each of these lives on using /etc/fstab Logical Volume /dev/hda1 /dev/hda5 /dev/hda7 /dev/hda6 /dev/hda8 /dev/fd0 /dev/cdrom \\kashmir\c landlord:/var/admin landlord:/home/lee Mount Point / /home /tmp /usr swap /mnt/floppy /mnt/cdrom /mnt/kashmir /var/admin /home/lee/LANDLORD FS type ext2 ext2 ext2 ext2 swap ext2 iso9660 smbfs nfs nfs Options defaults defaults defaults defaults defaults noauto noauto,ro guest defaults defaults Dump Check order 1 1 1 2 1 2 1 2 0 0 0 0 0 0 0 0 0 0 0 0 c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.17 Runlevels 482 17.17 Runlevels • Linux has several modes of operation • Referred to as runlevels • The Linux Standards Base (http://www.linuxbase. org/spec/refspecs/LSB 1.1.0/gLSB/runlevels.html) defines the following standard runlevels that all distributions must follow to be compliant: 0 halt 1 single user mode 2 multiuser with no network services exported 3 normal/full multiuser 4 reserved for local use, default is normal/full multiuser 5 multiuser with xdm or equivalent 6 reboot www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 483 Key Configuration Files 17.18 Single User Mode • Mainly used for diagnostic purposes • Starts only a subset of the possible services, e.g. ◦ No networking ◦ No mail services ◦ No name lookup services – Except /etc/hosts ◦ No file-sharing services etc c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.19 Multi User Mode 484 17.19 Multi User Mode • The ‘normal’ operating state • All configured services are running • Multiple users can log in • /sbin/runlevel shows the previous and current runlevel of your machine www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 485 Key Configuration Files 17.20 Starting up and Shutting down • Only the superuser can shutdown or reboot • halt will shut down the machine totally ◦ For safety you should type /sbin/halt • Makes sure all processes are stopped • Stops services cleanly • Writes unsaved data to the disk ◦ ‘Syncing’ • /sbin/reboot will shut down cleanly and reboot c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.21 Changing runlevel 486 17.21 Changing runlevel • It is sometimes necessary to change runlevel • Rare, but useful to know • You can instruct a system to change runlevel using the telinit command • Example: $ telinit 5 • Changes to runlevel 5 • telinit 1 takes the system down to single user mode www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 487 Key Configuration Files 17.22 Initscripts • The precise behaviour of each of the runlevels is controlled by initscripts • Control which services run in each runlevel • Live in init.d ◦ On Debian it’s in /etc/init.d ◦ On Redhat it’s /etc/rc.d/init.d • Each file here is a script that can be called with an argument, start, stop, or restart c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.23 rcn.d 488 17.23 rcn.d • The contents of the directories rcn.d control which services start and stop in runlevel n • The directories hold symbolic links to the files in init.d • The links are named informatively • To start service abc you would create a link typically named Sxxabc, to init.d/abc • The xx specifies the order to run the scripts, e.g. S00foo will be run before S90foo • Links that stop a service are of the form Kxxabc www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 489 Key Configuration Files 17.24 Initscripts — An example • Consider the following: ∗ lee @ 12:22:08 /etc/rc.d/rc3.d ls -l S* lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx lrwxrwxrwx ... ... ... ... ... ... ... ... ... ... ... ... S01kerneld -> ../init.d/kerneld S10network -> ../init.d/network S15nfsfs -> ../init.d/nfsfs S20random -> ../init.d/random S30syslog -> ../init.d/syslog S40atd -> ../init.d/atd S40crond -> ../init.d/crond S40portmap -> ../init.d/portmap S40snmpd -> ../init.d/snmpd S45pcmcia -> ../init.d/pcmcia S50inet -> ../init.d/inee S55named -> ../init.d/named • We can see that the first thing started is kerneld, followed by network services, nfs services, etc • There are also a series of Kxxyyy scripts which shut down the services in a sensible order Unimportant information has been removed from the screen dump so do not be alarmed if this doesn’t look like you’d expect! c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk ∗ 17.25 Restarting Services 490 17.25 Restarting Services • Can be necessary to restart a particular service, e.g. so it can re-read a modified configuration file • This can be done without a complete reboot • It must, however, be done by the superuser • To restart samba (smb) we can do the following: $ cd /etc/rc.d/init.d $ ./smb restart www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 491 Key Configuration Files 17.26 Exercises 1. Passwords (a) Find out whether your machine is using standard or shadow passwords? 2. Users (a) Add a new user (useradd) and set them up with the correct Full Name, password, home directory. Set their default shell to csh 3. Groups (a) Create a new group and add your user to this group (b) Now remove both the user and the group. How would you ensure that all files belonging to that user have been removed? 4. Scheduling (a) Add a cron job to eject your CDROM drive at 5 minutes past every hour and put it back in at ten minutes past the hour 5. Mounting (a) Set up your fstab so that $ mount /dev/cdrom will automatically mount your CD drive under /mnt/cdrom 6. Runlevels (a) Switch your machine between runlevels 3 and 5. What is happening? What happens if you change to runlevel 6? (b) Make sure your machine runs the same set of services in both runlevels 7. Stop, Start and Restart Services (a) Check you can stop, start, or restart services (b) Can you do this as a normal (ie non-root) user? c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk 17.27 Solutions 492 17.27 Solutions 1. Passwords (a) Your machine will have an /etc/shadow file if it is using shadow passwords. The password field will be set to an ’x’ in /etc/passwd. 2. Users (a) The following would set the details for the user Lee Willis $ useradd leewillis $ passwd leewillis Changing password for user leewillis New UNIX password: Retype new UNIX password: passwd: all authentication tokens updated successfully $ chfn -f "Lee Willis" leewillis Changing finger information for leewillis. Finger information changed. $ chsh -s /bin/csh leewillis Changing shell for leewillis. Shell changed. The home directory should be set up properly (/home/leewillis), if not you can change it with $ usermod -d /home/leewillis leewillis 3. Groups (a) $ groupadd newgroup $ usermod -G newgroup leewillis (b) To remove the group, the user and the user’s home directory $ groupdel newgroup $ userdel -r leewillis There are a few important points here! Firstly there may still be files in the filesystem belonging to that user. To locate them all you should have done find / -user leewillis -exec rm -f {} \; prior to removing the user. You should also have located all files belonging to the group and re-parented and/or removed them before removing the group 4. Scheduling (a) The following lines should acheive the desired effect 05 * * * * root eject /dev/cdrom 10 * * * * root eject -t /dev/cdrom 5. Mounting (a) The entry should look like /dev/cdrom 6. Runlevels www.linuxtraining.co.uk c GBdirect Ltd and parts c Nick Urbanik, 2004 /mnt/cdrom iso9660 noauto,ro 00 493 Key Configuration Files (a) You can change runlevels by using telinit 5 and telinit 3. All non-relevant services are stopped and the new ones started each time you change runlevel. Runlevel 6 reboots the machine! (b) You should ensure that the directory listings for /etc/rc.d/rc3.5 and /etc/rc.d/rc5.d are the same. This should ensure that the same services are started/stopped when entering either runlevel. 7. Start, Stop and Restart services (a) (b) - c GBdirect Ltd and parts c Nick Urbanik, 2004 www.linuxtraining.co.uk