SlideShare ist ein Scribd-Unternehmen logo
1 von 71
UNIX and Linux
 an Introduction
  2-day intensive training
      developed by
     Mathias Homann
Goals
•   provide insights on why UNIX matters
•   learn about what makes UNIX special
•   practice executing basic concepts
•   spark interest for deeper self-study




UNIX and Linux an Introduction, developed   2 of 71
by Mathias Homann
Chapter Index
•   A short history of UNIX
•   UNIX today
•   Linux is not UNIX
•   How to access a UNIX system
•   Secure shell – SSH
•   SSH key management
•   Building tunnels with SSH

UNIX and Linux an Introduction, developed   3 of 71
by Mathias Homann
Chapter Index (2)
•   Learning details about UNIX commands
•   Shells and the environment
•   Internal and external commands
•   Interactive shells and startup scripts
•   Wildcards
•   Command aliases


UNIX and Linux an Introduction, developed   4 of 71
by Mathias Homann
Chapter Index (3)
•   Paths
•   Finding files – locate vs find
•   Filesystems
•   Mounting
•   Links
•   Devices and drivers
•   Permissions and ownership

UNIX and Linux an Introduction, developed   5 of 71
by Mathias Homann
Chapter Index (4)
•   Background processes
•   Scheduled processes
•   Processes and threads
•   Output and input redirection
•   Pipes
•   Shell history


UNIX and Linux an Introduction, developed   6 of 71
by Mathias Homann
Chapter Index (5)
•   The power of grep
•   Less is more
•   The vi editor
•   Regular expressions
•   Sed – the stream editor
•   Awk – pattern scanner for text files
•   Writing your own scripts
•   Syslog and log files
UNIX and Linux an Introduction, developed   7 of 71
by Mathias Homann
Chapter Index (6)
•   Finding facts about a system
•   Gathering statistics
•   Writing advanced scripts
•   More exercises
•   Useful commands
•   Higher wizardry – perl
•   Discussion and closing

UNIX and Linux an Introduction, developed   8 of 71
by Mathias Homann
A short history of UNIX
                       Graphic credit: http://en.wikipedia.org/wiki/File:Unix.svg




UNIX and Linux an Introduction, developed                                           9 of 71
by Mathias Homann
A short history of UNIX (2)
• 1969: Thompson, Ritchie and others start to “play with
  that unused PDP-7 in the corner” at Bell Labs, develop a
  system called “Unicks” based on Multics
• 1971: UNIX was developed; first released version on
  March 11, 1971
• 1975: Bourne shell introduced
• 1982: SGI introduces IRIX, their commercial flavour of
  UNIX for their workstation line
• 1983: SCO delivers their first packaged UNIX for Intel
  architecture PCs; Richard Stallman introduces the GNU
  operating system on September 27



UNIX and Linux an Introduction, developed      10 of 71
by Mathias Homann
A short history of UNIX (3)
• 1986: HP releases HP-UX 1.0
• 1987: SUN and AT&T form an alliance to work on what
  will be UNIX System V Release 4.0
• 1990: IBM releases AIX
• 1991: SUN releases Solaris OS, specially designed for
  symmetric multiprocessing
• 1991: Linux Thorvalds sends his famous email to the
  minix developers mailing list, it starts with the words “I'm
  doing a (free) operating system ...”



UNIX and Linux an Introduction, developed          11 of 71
by Mathias Homann
UNIX today
• today, UNIX is the operating system of choice for both
  large server farms and small embedded devices
• Amazon, Google, your android phone and your media
  center that is hooked up to your TV all run on Linux, a
  free UNIX-like operating system




UNIX and Linux an Introduction, developed        12 of 71
by Mathias Homann
Linux is not UNIX
• the open source community takes great joy in using
  “recursive acronyms”:
    – “GNU is not UNIX”
    – “Linux is not UNIX”
• Linux is “UNIX-Like”
• a lot in common, “UNIX-Compatible”, but also:
    – Not a serious, commercial development history / process
    – In a constant flow
• UNIX is:
    – “carved in stone” before first deployment
    – commercially developed
    – Not free to use, unlike Linux, UNIX costs money



UNIX and Linux an Introduction, developed               13 of 71
by Mathias Homann
How to access a UNIX system
• a “traditional” UNIX host usually has one local console,
  and several “dumb” terminals attached
• today, such “dumb terminals” are usually PCs running
  terminal emulation software, instead of the old fashioned
  “true” serial consoles
• a Linux system has one local physical console (keyboard
  and screen), with several virtual consoles available
  (usually six, to switch between them use the keyboard
  shortcuts ctrl-alt-F1 to ctrl-alt-F6)
• virtual console #7 is usually occupied by the graphical
  desktop environment if such is used



UNIX and Linux an Introduction, developed       14 of 71
by Mathias Homann
Secure shell – SSH
• a better way to access a UNIX system is through SSH
    – The most common SSH client for Windows is called putty,
      available from http://www.chiark.greenend.org.uk/~sgtatham/putty/
• SSH uses strong encryption
    – communication between the user and the system can not be
      eavesdropped on
    – the administrator password can not be intercepted
    – on the contrary, Telnet sends plaintext, unencrypted data over
      the network     do not use telnet … ever
• to access a UNIX system you need
    – a user account, and either have the password for that account
    – or have your SSH public key installed in the allowed keys list for
      that account

UNIX and Linux an Introduction, developed                       15 of 71
by Mathias Homann
Secure shell – SSH (2)
• to login to a UNIX system through SSH, you use a
  command like this:
   ssh joe@example.host.com
• useful commandline options for SSH:
    -1 and -2 to enforce SSH protocol version 1 or 2
    -4 and -6 to enforce IPv4 or IPv6
    -A enabled SSH agent forwarding (-a to disable)
    -X enable X11 forwarding (-x to disable)
    -C compresses data (useful on slower connections)




UNIX and Linux an Introduction, developed               16 of 71
by Mathias Homann
SSH key management
• SSH uses an asymmetric key pair
• to generate a key pair, use ssh-keygen on UNIX, or
  puttygen on Windows
• the public key is what you install on servers that you
  need to access
• the private key needs a passphrase to be used, and is
  the part that you need to keep safe




UNIX and Linux an Introduction, developed       17 of 71
by Mathias Homann
SSH key management (2)
• the pass phrase for unlocking your private key can be
  significantly longer and more complex than a system
  password
• to register your key for access to a system under one
  user account, append the content of your public key to
  ~/.ssh/authorized_keys
• verify host keys by comparing the fingerprint of the host
  key with the fingerprint of the host key in your local
  known_hosts file
• host keys do not change randomly



UNIX and Linux an Introduction, developed        18 of 71
by Mathias Homann
Building tunnels with SSH
• a tunnel is an encrypted “hole” that data can flow through
• tunnels are used to create secure connections through
  firewalls
• tunnels can be used to access services securely
• a tunnel can:
    – forward a local port to the remote machine
    – a port on the remote machine to the local machine
    – a port on a machine in the same network as the remote machine
      to a port on the local machine




UNIX and Linux an Introduction, developed              19 of 71
by Mathias Homann
Building tunnels with SSH (2)
• parameters for tunnels:
    -L x:host:y – forward localhost:x to host:y
    -R x:host:y – forward host:x to localhost:y
• example:
    – a tunnel could be used to map the web server on a remote
      system to the local port 8080
• this way, entering http://localhost:8080/ in your browser
  would access the remote server




UNIX and Linux an Introduction, developed              20 of 71
by Mathias Homann
Building tunnels with SSH (3)
• exercise
    – Generate your key pair with putty
    – Connect to (hostname) as (username) ← will have to be adapted
      for actual training environment
    – Forward the webserver running on the loopback interface on
      (hostname) to port 8080 on your local computer
    – Verify with your web browser that the tunnel works




UNIX and Linux an Introduction, developed             21 of 71
by Mathias Homann
Learning about UNIX commands
• UNIX has two built-in documentation systems:
    – info pages
    – man pages
• to read the man page (manual page) of a given
  command, use man command
    – Try this out with man man
• to look if there is a command that does something, try
  the apropos command
• the apropos command lists the summary lines of all man
  pages that contain the term you specified


UNIX and Linux an Introduction, developed        22 of 71
by Mathias Homann
Learning about UNIX commands                             (2)



• to look at what a certain command does without reading
  the whole man page, use whatis program
• this prints out the summary only of the command's man
  page
• to find where in the filesystem a command is, use the
  which command




UNIX and Linux an Introduction, developed     23 of 71
by Mathias Homann
Shells and the environment
• the shell is the interface to the system
• the shell executes commands and shows the results
• the “environment” is the total of stored settings inside a
  shell
• important shell variables on UNIX systems:
        $PATH
        $PS1
        $CWD




UNIX and Linux an Introduction, developed         24 of 71
by Mathias Homann
Shells and the environment (2)
•   UNIX knows several different shells:
        sh
        bash
        ksh
        zsh
        csh
•   the default shell on most Linux systems is bash
•   the set command shows all environment variables
•   change environment variables by assigning and
    exporting them
•   all exercises will use bash if not stated otherwise


UNIX and Linux an Introduction, developed        25 of 71
by Mathias Homann
Internal and external commands
• shells have built-in versions of most “system commands”
• to “force” the use of a system command instead of the
  builtin, use the absolute path to the binary: /bin/ls
• external commands are standardized
• builtins differ from shell to shell




UNIX and Linux an Introduction, developed     26 of 71
by Mathias Homann
Interactive shells + startup scripts
• an “interactive” shell is one that you type commands into
• “non-interactive” shells are the ones that execute scripts
• a “login” shell is the first interactive shell that gets started
  when you log in
• the “profile” or “rc file” is a script that gets executed by a
  shell on startup
• most shells execute more than one startup script, and
  execute a different set of scripts when started non-
  interactive, and yet another for the login shell



UNIX and Linux an Introduction, developed             27 of 71
by Mathias Homann
Interactive shells + startup scripts                           (2)



• examples - the bash shell executes on startup as a login
  shell (in this order):
    /etc/profile
    ~/.bash_profile
    ~/.bash_login
    ~/.profile
• if the bash shell is started interactively but not as a login
  shell it only executes ~/.bashrc
• a non-interactive bash shell executes the file listed in
  $BASH_ENV if the variable is set and the file exists
• note: a login bash executes ~/.bash_logout on exit



UNIX and Linux an Introduction, developed           28 of 71
by Mathias Homann
Wildcards
• a wildcard is a pattern that describes a path or filename
• wildcard expansion in shell commands happens on the
  shell level, before the actual command gets executed
• different shells handle wildcards in different ways
• exercises:
    – Which wildcard would mean “all jpeg files in this folder”?
    – Which pattern means “all jpeg files in this folder that start with
      the letter a”?




UNIX and Linux an Introduction, developed                    29 of 71
by Mathias Homann
Command aliases
• a command alias is a “placeholder” for a command and
  optionally one or more commonly used options
• for example, on openSUSE Linux there is an alias “dir”
  which calls ls -l
• a list of all aliases is shown by entering the alias
  command with no parameters
• to see the aliases and wildcards expansion before
  commands get executed, enter set -x




UNIX and Linux an Introduction, developed      30 of 71
by Mathias Homann
Paths
• paths on UNIX systems are all inside a “tree” of folders
  that starts at the “root” folder or /
    – the user's home folder is usually held under /home and named
      after the username, e.g. /home/fred
    – the “search path” describes which folders to look in when a shell
      tries to find a command
    – the search path is stored in the environment variable $PATH
    – the minimal search path should contain /bin and /usr/bin
    – directories in the search path are separated by the colon:
      PATH=/bin:/usr/bin




UNIX and Linux an Introduction, developed                 31 of 71
by Mathias Homann
Everything is a file ...
• ... not just a file
• /proc/12345/status is a “virtual file” containing status
  information about process 12345
• /dev/mem is a “device file” that is actually an interface to
  the system's memory
• /dev/sg0 is a “device file” for the first “generic” SCSI
  device in a system
• /dev/sdb5 is a “device file” that references the first logical
  partition on the second external, SCSI or SATA disk




UNIX and Linux an Introduction, developed           32 of 71
by Mathias Homann
Finding files – locate vs find
• there are two ways to find a file, locate and find
• the locate command uses a database of existing files
  which needs to be updated regularly, which usually
  happens once a day
• the find command actually searches through the
  filesystem – this can take a while, but does not rely on a
  database that can be outdated
    locate can only search by shell wildcards
    find can search for anything, not just names
• exercise:
    – find all files that changed since yesterday

UNIX and Linux an Introduction, developed           33 of 71
by Mathias Homann
Filesystems
• UNIX knows several different filesystems
• one of the main differences is whether a filesystem is
  “journaling” or not
• A journaling filesystem keeps track of the changes that
  will be made in a journal, before committing them to the
  main filesystem. In the event of a system outage, such
  filesystems are quicker to bring back online and less
  likely to be corrupted.
• commonly used filesystems on Linux are:
    – ext2, ext3, ext4, jfs, xfs, reiserfs
    – Ext2 is the only of these that is not journaling

UNIX and Linux an Introduction, developed                34 of 71
by Mathias Homann
Filesystems (2)
• filesystem limits:
                    names      paths       filesize      volume size


         ext2       255        unlimited   16GB - 2TB    2TB-32TB

         ext3       255        unlimited   16GB - 2TB    2TB-32TB

         ext4       256        unlimited   16GB - 16TB   1EB

         jfs        255        unlimited   8TB           512TB-4PB

         xfs        255        unlimited   8EB           8EB

         reiserfs   4032/226   unlimited   8EB           16TB




UNIX and Linux an Introduction, developed                           35 of 71
by Mathias Homann
Mounting
• all disk partitions need to be mounted to be accessible
• a disk partition gets mounted into a folder, called the
  “mount point”
• a UNIX system has a file with a list of all mount points
  that need to be mounted when the system starts:
  /etc/fstab
    – to mount a filesystem that is not listed in /etc/fstab you need
      superuser privileges
    – to access a partition you need to know its device node
    – device nodes are special files in /dev



UNIX and Linux an Introduction, developed                   36 of 71
by Mathias Homann
Mounting (2)
• for example, the device node for a partition on a hard
  disk would be /dev/hda2 - meaning “the second primary
  partition on the first hard disk”
• exercise:
    – explore what is mounted, and what is mounted on boot




UNIX and Linux an Introduction, developed              37 of 71
by Mathias Homann
Links
• a link is a “pointer” to a file
    – can be a “soft” or a “hard” link
    – a softlink is a pointer at a file, and can link across filesystems
    – a hardlink is another directory entry that points at the same
      starting sector of the filesystem
• hardlinks are only possible within one filesystem, and
  cannot point at directories
• links are created with the ln command
    ln creates hardlinks by default
    – for softlinks use the -s option
• exercise:
    – create a link to /usr/bin/bash in ~/bin/

UNIX and Linux an Introduction, developed                     38 of 71
by Mathias Homann
Devices and drivers
• every piece of hardware is accessed through a device
  file under /dev
• examples:
  /dev/hda - the first IDE harddisk
  /dev/sdb - the second SCSI or SATA disk
  /dev/nvidia – an NVIDIA graphics card
  /dev/dsp0 – the sound card




UNIX and Linux an Introduction, developed     39 of 71
by Mathias Homann
Devices and drivers (2)
• access to hardware is governed by the file permissions
  on those special files
• for example, the permissions on a graphics card could
  be set to allow only root and anyone in the group “video”
  access to write to it, therefore only users in the group
  “video” would be able to run 3d accelerated applications
• exercise:
    – find out which group has write access to the cd/rw drive, e.g.
      would have sufficient permissions to use a CDRW writer on the
      system




UNIX and Linux an Introduction, developed               40 of 71
by Mathias Homann
Permissions and ownership
• there are 3 levels of ownership of any file:
    – “owner”
    – “group”
    – “others”
• permissions on a file:
    – read, write, execute, separately for all three owner types
• example:
    – a file usually is read/write for the owner, and read-only for
      anyone else: -rw-r--r--
• change the owner: chown username file
• change the group: chgrp groupname file
• change permissions: chmod <permissions>              file

UNIX and Linux an Introduction, developed                     41 of 71
by Mathias Homann
Background processes
• UNIX can run commands in the background so that you
  can do something else
    – To execute a program in the background, end the commandline
      with &
    – To send a running program to the background, press ctrl-z and
      enter bg
    – To view the list of running background processes, enter jobs
• when you logout while you still have background
  processes running, they will be killed unless you started
  them with nohup:
    nohup sleep 10000 will run sleep 10000 in the background
    – the output of the command will be written to nohup.out


UNIX and Linux an Introduction, developed              42 of 71
by Mathias Homann
Scheduled processes
•   to run a program at regular intervals, use cron
•   cron keeps lists of scheduled jobs
•   edit the list with crontab -e
•   to run a program at a certain time once, use at
        at 23:00 asks for a sequence of commands
• the job gets deleted once it runs




UNIX and Linux an Introduction, developed          43 of 71
by Mathias Homann
Processes and threads
• a process is one running executable
    – A single process can have more than one running thread
    – To see all processes, use top, htop, or the ps command
    – To send signals to processes, use the kill command
• several “levels” of “kill signal”, ordered by “lethality”:
    kill   -HUP tells a process to shut down nicely
    kill   -QUIT – the same but a bit more urgent
    kill   -KILL kills a process
    kill   -l lists all possible signals with name and numeric value
    kill   -9 is equivalent to kill -KILL
• combined exercise:
    – start a process in the background that waits for 10 minutes
    – kill it before the time is over


UNIX and Linux an Introduction, developed                  44 of 71
by Mathias Homann
Output and input redirection
•   in- and Output can be redirected with the < and > symbol
•   redirect input: cat < /tmp/file.txt
•   redirect output: cat < /tmp/file.txt > /tmp/foo.bar
•   redirect the error output: sort < foo 2>/dev/null




UNIX and Linux an Introduction, developed        45 of 71
by Mathias Homann
Pipes
• pipes are used to direct the output of one command to
  the input of another
• pipes use the pipe symbol: |
• example: ls -la | less pipes the output of the ls
  command into the less pager
• exercise: sort the content of the passwd file by alphabet




UNIX and Linux an Introduction, developed        46 of 71
by Mathias Homann
Shell history
• UNIX shells keep track of your commands
• to browse through this list use the cursor up and cursor
  down keys
• to re-execute a previous command, find it in your history
  and press enter
• to find a previous command more quickly, press ctrl-r
  and start typing; the history will “jump” to the newest
  match of what you're typing




UNIX and Linux an Introduction, developed       47 of 71
by Mathias Homann
Shell history (2)
• to view the whole history, use the “history” command
• to re-execute a certain command, find its number in the
  history, then enter !number, e.g. !514 to re-run number
  514 from your history
• to search in your history: press ctrl-r, then start typing
• exercise:
    – use a few commands (ls, man, ls -l, ls -la) then view the
      history, then re-run some of them, then search in your history




UNIX and Linux an Introduction, developed               48 of 71
by Mathias Homann
The power of grep
•   grep searches for text in files
•   grep can use regular expressions
•   grep has many options - read the man page
•   most useful options:
    – -i for “search case insensitive”, and -v for “inverse” or “search for
      anything that does not match my search string”
• example:
    grep myusername /var/log/messages
    – find all lines in logfile that contain myusername
• exercise:
    – filter the main system logfile for all lines from last Monday

UNIX and Linux an Introduction, developed                    49 of 71
by Mathias Homann
Less is more
• less is a “pager” to view large text files on screen
• less can search for text:
    – press : then enter a slash followed by your text
• less can switch to “continuous refresh” to refresh a file
  that is still being written; press F to switch
• less can read from “stdin”; pipe commands into it to
  paginate the output
• g and g jump to the end and the start of the file
• to search in less, type /searchterm
• to jump to a line number, type :number
• example: less /var/log/messages
• exercise:
    – display the content of /usr/bin in less

UNIX and Linux an Introduction, developed                50 of 71
by Mathias Homann
The vi editor
• the editor vi exists on every UNIX system
• vi has two modes: insertion mode and command mode
    – The editor begins in command mode, where the cursor
      movement and text deletion and pasting occur
    – Insertion mode begins upon entering an insertion or change
      command
    – [ESC] returns the editor to command mode (where you can quit,
      for example by typing :q!)
• most commands execute as soon as you type them
  except for "colon" commands which execute when you
  press the return key
• nifty: vi does syntax highlighting; activate with :syntax

UNIX and Linux an Introduction, developed              51 of 71
by Mathias Homann
The vi editor (2)
• quitting vi:
    :x Exit, saving changes
    :q Exit as long as there have been no changes
    ZZ Exit and save changes if any have been made
    :q!Exit and ignore any changes
• switching to edit mode:
    i – insert
    r – replace
    a – append




UNIX and Linux an Introduction, developed            52 of 71
by Mathias Homann
The vi editor (3)
• cursor movement
• cursor keys, and:
    ^ or 0 – move to start of line
    $ - move to end of line
• block and line operations:
    dd – delete line
    yy – copy line
    p – paste line




UNIX and Linux an Introduction, developed   53 of 71
by Mathias Homann
Regular expressions
• regular expressions are powerful string matching
  patterns
• examples:
    . - any one character
    * - any repetitions of the pattern left of the star
    .* - any number of any characters
    [:upper:] - all UPPER CASE characters
    n – newline
    [a-q] – any letter between a and q




UNIX and Linux an Introduction, developed                 54 of 71
by Mathias Homann
Regular expressions (2)
• more info:
• the “Regular expressions cheat sheet v2”
    – http://www.addedbytes.com/cheat-sheets/regular-expressions-
      cheat-sheet/
• example:
    – a regular expression cat.*dog matches any string that has “cat”
      and “dog” in it with any other characters in between them, but
      only in that order!




UNIX and Linux an Introduction, developed                55 of 71
by Mathias Homann
Sed – the stream editor
• Sed reads text from stdin, does edit operations on it, and
  prints the result on stdout
• examples:
    sed -e 's/red/green/g' <colors.txt >colors1.txt
    – replaces all occurrences of “red” with “green”
• exercise:
    – read the man page, then write a text file, then replace all “the”
      with “THE”
• exercise:
    – print out each directory in the path variable in a new line




UNIX and Linux an Introduction, developed                   56 of 71
by Mathias Homann
Awk – pattern scanner for text files
• Awk scans text files for patterns and operates on them
• examples:
    awk '{print $2“n“}' < file.txt
    – print the second word of every line of file.txt
• exercise:
    – read the man page, then use vi to write a text file with a few
      lines of the format: Username:Firstname:Lastname:Telephone
      number
    – then, use awk to convert this into ldap syntax:

    dn: uid=Username, dc=example, dc=com
    cn: Firstname Lastname
    sn: Lastname
    telephoneNumber: Telephone number

UNIX and Linux an Introduction, developed                57 of 71
by Mathias Homann
Writing your own scripts
•   scripts can be written with any text editor
•   scripts don't need a special filename suffix
•   scripts need to be executable (hint: chmod)
•   scripts can explicitly state which shell should be used to
    execute it by putting the “hashbang” for the shell in the
    first line:
        #!/bin/bash – a bash script
        #!/bin/csh – a c shell script
        #!/bin/sh – a “sh” shell script
• “sh” is the most basic shell; exists on any UNIX
• exercise:
    – write a script that prints “hello world” ten times


UNIX and Linux an Introduction, developed                  58 of 71
by Mathias Homann
Syslog and log files
• syslog is a UNIX daemon that collects and handles log
  file messages
• syslog handles messages according to severity and
  facility
• details: see man page
• syslog can write messages to one or more files
  depending on severity and facility




UNIX and Linux an Introduction, developed     59 of 71
by Mathias Homann
Syslog and log files (2)
• logfiles usually exist in /var/log
• the syslog configuration file is in /etc and states which
  facility.severity goes in which file
• exercise:
    – check the syslog configuration to see which files hold messages
      of failed logins
    – then check if there were any recent failed login attempts




UNIX and Linux an Introduction, developed                60 of 71
by Mathias Homann
Finding facts about a system
• most Linux systems have a file in /etc/ that contains
  information about which distribution is running
    – for example, /etc/redhat-Release
    uname -a prints out information about the system
• use mount and df to find out about disk sizes and mount
  points
• a lot of information is in /proc
• almost all configuration files are in /etc and its
  subdirectories




UNIX and Linux an Introduction, developed              61 of 71
by Mathias Homann
Finding facts about a system (2)
• most UNIX systems use some form of software
  package management
    – the list of installed packages can tell you a lot about the purpose
      of a system
    – on Linux there are two widely used package management
      systems, rpm and apt/dpkg
    – rpm is used on openSUSE, redhat, and their derivatives and
      commercial versions
    – on debian, ubuntu and derivatives, apt and dpkg is used
    – commands such as rpm -qa and dpkg –get-selections show
      the list of installed packages
    – these lists are long, less is your friend

UNIX and Linux an Introduction, developed                   62 of 71
by Mathias Homann
Finding facts about a system (3)
• to list all loaded kernel modules lsmod
• use insmod or modprobe to load additional kernel modules into a
  running system
• use modprobe -r to remove a module and all dependencies from a
  running system
• most of them are hardware drivers or drivers for “features”




UNIX and Linux an Introduction, developed            63 of 71
by Mathias Homann
Gathering statistics
• commands to explore the status of the system
    vmstat and free show memory usage
    iostat shows drive accesses
    netstat shows network statistics
    netstat can also show active network connections and listening
      processes; read the man page
• exercise:
    – check how much memory the machine has, and how much is
      used; then check how many active network connections exist
      that are not SSH connections




UNIX and Linux an Introduction, developed               64 of 71
by Mathias Homann
Writing advanced scripts
• scripts can do math: a=$(($b+$c))
• scripts can have flow control
    – if (condition) then {command; command;}; else { command;
      command;}; endif
    – do { command; command; } while (condition);
    – for (iterator); do { command; command; } done;
• scripts can have functions
• scripts can return values to the calling process




UNIX and Linux an Introduction, developed              65 of 71
by Mathias Homann
Writing advanced scripts (2)
• exercise:
    –   write a script that:
    –   finds all log files under /var log
    –   takes all files older than 2 days
    –   prints their name, size and age
    –   (size in 'human readable' format, 1024k = 1 M etc)




UNIX and Linux an Introduction, developed                    66 of 71
by Mathias Homann
More exercises
• create a file “a”, write “hello world” into it, then hardlink it
  as “b”
• check the inodes of both files
    – (hint: there's an option for ls that does that)
• create a softlink “c” to “a”, check inodes
• run tail -f /var/log/messages, send it to the background,
  edit a different file in vi, exit vi, bring the tail command
  back to the front
• compare the differences between the various ps output
  formats:
    ps –ax
    ps -ef
    ps -axf
    – sort /etc/passwd alphabetically
    – sort it by uid
UNIX and Linux an Introduction, developed               67 of 71
by Mathias Homann
Useful commands
•   cd – change directory
•   pushd – change directory but it stores the previous
    directory on a fifo stack
•   popd – returns to previous directory from the stack
•   cat – read from file or stdin, write to file or stdout
    (example: cat /etc/motd to print that file to screen)
•   cp – copy files
•   mv – move / rename files
•   head & tail – print the first or last lines of a file
•   find – search for files




UNIX and Linux an Introduction, developed             68 of 71
by Mathias Homann
Useful commands (2)
•   less – display a file in a pager
•   grep – search for text in files
•   sed – command-line editing of files
•   wc – count lines/words/characters in files
•   df – disk free (shows free space in partitions)
•   du – disk usage (adds up file sizes, more accurate than
    df, takes some time to finish)
•   gzip – compress files, use gunzip to decompress
•   diff – compare files




UNIX and Linux an Introduction, developed         69 of 71
by Mathias Homann
Higher wizardry – perl
• perl is disputedly the most powerful, simple, complicated,
  and obscurely easy to understand scripting language
• perl is enough for its own workshop day :)




UNIX and Linux an Introduction, developed        70 of 71
by Mathias Homann
Discussion and closing
• questions
• comments
• Instructor Evaluation Form




UNIX and Linux an Introduction, developed   71 of 71
by Mathias Homann

Weitere ähnliche Inhalte

Was ist angesagt?

Linux programming lecture_notes
Linux programming lecture_notesLinux programming lecture_notes
Linux programming lecture_notes
IMRAN KHAN
 
Operating systems linux
Operating systems linuxOperating systems linux
Operating systems linux
william_morg
 

Was ist angesagt? (20)

Linux introduction
Linux introductionLinux introduction
Linux introduction
 
History of Linux
History of LinuxHistory of Linux
History of Linux
 
Linux introduction, class 1
Linux introduction, class 1Linux introduction, class 1
Linux introduction, class 1
 
Linux programming lecture_notes
Linux programming lecture_notesLinux programming lecture_notes
Linux programming lecture_notes
 
Linux basics
Linux basicsLinux basics
Linux basics
 
Linux
LinuxLinux
Linux
 
Introduction to linux
Introduction to linuxIntroduction to linux
Introduction to linux
 
what is LINUX ? presentation.
what is LINUX ? presentation.what is LINUX ? presentation.
what is LINUX ? presentation.
 
An Introduction to Linux
An Introduction to LinuxAn Introduction to Linux
An Introduction to Linux
 
Linux Simple Introduction
Linux Simple IntroductionLinux Simple Introduction
Linux Simple Introduction
 
comparing windows and linux ppt
comparing windows and linux pptcomparing windows and linux ppt
comparing windows and linux ppt
 
ZendCon - Linux 101
ZendCon - Linux 101ZendCon - Linux 101
ZendCon - Linux 101
 
UNIX Operating System
UNIX Operating SystemUNIX Operating System
UNIX Operating System
 
Unit 1-a-brief-history-of-unix-ppt
Unit 1-a-brief-history-of-unix-pptUnit 1-a-brief-history-of-unix-ppt
Unit 1-a-brief-history-of-unix-ppt
 
A History of Linux
A History of LinuxA History of Linux
A History of Linux
 
Linux introduction Class 02
Linux introduction Class 02Linux introduction Class 02
Linux introduction Class 02
 
Operating systems linux
Operating systems linuxOperating systems linux
Operating systems linux
 
Operating Systems: Linux in Detail
Operating Systems: Linux in DetailOperating Systems: Linux in Detail
Operating Systems: Linux in Detail
 
Linux Introduction
Linux IntroductionLinux Introduction
Linux Introduction
 
Linux OS presentation
Linux OS presentationLinux OS presentation
Linux OS presentation
 

Andere mochten auch

Unix(introduction)
Unix(introduction)Unix(introduction)
Unix(introduction)
meashi
 
Unix memory management
Unix memory managementUnix memory management
Unix memory management
Tech_MX
 
Unix command-line tools
Unix command-line toolsUnix command-line tools
Unix command-line tools
Eric Wilson
 
Red hat enterprise_linux-5.5-release_notes-en-us
Red hat enterprise_linux-5.5-release_notes-en-usRed hat enterprise_linux-5.5-release_notes-en-us
Red hat enterprise_linux-5.5-release_notes-en-us
Duong Hieu
 

Andere mochten auch (20)

UNIX/Linux training
UNIX/Linux trainingUNIX/Linux training
UNIX/Linux training
 
Unix Operating System
Unix Operating SystemUnix Operating System
Unix Operating System
 
intro unix/linux 02
intro unix/linux 02intro unix/linux 02
intro unix/linux 02
 
Unix - An Introduction
Unix - An IntroductionUnix - An Introduction
Unix - An Introduction
 
Unix/Linux Basic Commands and Shell Script
Unix/Linux Basic Commands and Shell ScriptUnix/Linux Basic Commands and Shell Script
Unix/Linux Basic Commands and Shell Script
 
Shell Scripting in Linux
Shell Scripting in LinuxShell Scripting in Linux
Shell Scripting in Linux
 
Unix
UnixUnix
Unix
 
QSpiders - Unix Operating Systems and Commands
QSpiders - Unix Operating Systems  and CommandsQSpiders - Unix Operating Systems  and Commands
QSpiders - Unix Operating Systems and Commands
 
Unix(introduction)
Unix(introduction)Unix(introduction)
Unix(introduction)
 
Unix Command Line Productivity Tips
Unix Command Line Productivity TipsUnix Command Line Productivity Tips
Unix Command Line Productivity Tips
 
Unix OS & Commands
Unix OS & CommandsUnix OS & Commands
Unix OS & Commands
 
Unix memory management
Unix memory managementUnix memory management
Unix memory management
 
Basic Unix
Basic UnixBasic Unix
Basic Unix
 
Unix
UnixUnix
Unix
 
Unix command-line tools
Unix command-line toolsUnix command-line tools
Unix command-line tools
 
Unix operating system
Unix operating systemUnix operating system
Unix operating system
 
Unix Training - 1
Unix Training - 1Unix Training - 1
Unix Training - 1
 
Linux+03
Linux+03Linux+03
Linux+03
 
Red hat enterprise_linux-5.5-release_notes-en-us
Red hat enterprise_linux-5.5-release_notes-en-usRed hat enterprise_linux-5.5-release_notes-en-us
Red hat enterprise_linux-5.5-release_notes-en-us
 
Tu hoc su dung linux 2
Tu hoc su dung linux 2Tu hoc su dung linux 2
Tu hoc su dung linux 2
 

Ähnlich wie UNIX and Linux - an introduction by Mathias Homann

1 Intro, Linux system Architecture, Features, File system Structure and appli...
1 Intro, Linux system Architecture, Features, File system Structure and appli...1 Intro, Linux system Architecture, Features, File system Structure and appli...
1 Intro, Linux system Architecture, Features, File system Structure and appli...
PAVAN KUMAR ILLA
 
Linux [2005]
Linux [2005]Linux [2005]
Linux [2005]
Raul Soto
 
unixoperatingsystem-130327073532-phpapp01.pdf
unixoperatingsystem-130327073532-phpapp01.pdfunixoperatingsystem-130327073532-phpapp01.pdf
unixoperatingsystem-130327073532-phpapp01.pdf
IxtiyorTeshaboyev
 

Ähnlich wie UNIX and Linux - an introduction by Mathias Homann (20)

Course 101: Lecture 5: Linux & GNU
Course 101: Lecture 5: Linux & GNU Course 101: Lecture 5: Linux & GNU
Course 101: Lecture 5: Linux & GNU
 
Unix _linux_fundamentals_for_hpc-_b
Unix  _linux_fundamentals_for_hpc-_bUnix  _linux_fundamentals_for_hpc-_b
Unix _linux_fundamentals_for_hpc-_b
 
Linux Operating System.pptx
Linux Operating System.pptxLinux Operating System.pptx
Linux Operating System.pptx
 
Embedded Systems: Lecture 6: Linux & GNU
Embedded Systems: Lecture 6: Linux & GNUEmbedded Systems: Lecture 6: Linux & GNU
Embedded Systems: Lecture 6: Linux & GNU
 
Linux basics
Linux basicsLinux basics
Linux basics
 
Linux forensics
Linux forensicsLinux forensics
Linux forensics
 
Linux Operating System. UOG MARGHAZAR Campus
 Linux Operating System. UOG MARGHAZAR Campus Linux Operating System. UOG MARGHAZAR Campus
Linux Operating System. UOG MARGHAZAR Campus
 
1 Intro, Linux system Architecture, Features, File system Structure and appli...
1 Intro, Linux system Architecture, Features, File system Structure and appli...1 Intro, Linux system Architecture, Features, File system Structure and appli...
1 Intro, Linux system Architecture, Features, File system Structure and appli...
 
Linux
LinuxLinux
Linux
 
Basics you should know about UNIX and LINUX
Basics you should know about UNIX and LINUXBasics you should know about UNIX and LINUX
Basics you should know about UNIX and LINUX
 
Unix Operaring System
Unix Operaring SystemUnix Operaring System
Unix Operaring System
 
Linux [2005]
Linux [2005]Linux [2005]
Linux [2005]
 
Unix Security
Unix SecurityUnix Security
Unix Security
 
Unix lecture1
Unix lecture1Unix lecture1
Unix lecture1
 
Ch1-Unix.pptx
Ch1-Unix.pptxCh1-Unix.pptx
Ch1-Unix.pptx
 
Introduction to unix (1).pptx
Introduction to unix (1).pptxIntroduction to unix (1).pptx
Introduction to unix (1).pptx
 
Linux Knowledge Transfer
Linux Knowledge TransferLinux Knowledge Transfer
Linux Knowledge Transfer
 
Linux for beginners
Linux for beginnersLinux for beginners
Linux for beginners
 
Bn1025 demo basic unix
Bn1025 demo  basic unixBn1025 demo  basic unix
Bn1025 demo basic unix
 
unixoperatingsystem-130327073532-phpapp01.pdf
unixoperatingsystem-130327073532-phpapp01.pdfunixoperatingsystem-130327073532-phpapp01.pdf
unixoperatingsystem-130327073532-phpapp01.pdf
 

Kürzlich hochgeladen

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Kürzlich hochgeladen (20)

Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 

UNIX and Linux - an introduction by Mathias Homann

  • 1. UNIX and Linux an Introduction 2-day intensive training developed by Mathias Homann
  • 2. Goals • provide insights on why UNIX matters • learn about what makes UNIX special • practice executing basic concepts • spark interest for deeper self-study UNIX and Linux an Introduction, developed 2 of 71 by Mathias Homann
  • 3. Chapter Index • A short history of UNIX • UNIX today • Linux is not UNIX • How to access a UNIX system • Secure shell – SSH • SSH key management • Building tunnels with SSH UNIX and Linux an Introduction, developed 3 of 71 by Mathias Homann
  • 4. Chapter Index (2) • Learning details about UNIX commands • Shells and the environment • Internal and external commands • Interactive shells and startup scripts • Wildcards • Command aliases UNIX and Linux an Introduction, developed 4 of 71 by Mathias Homann
  • 5. Chapter Index (3) • Paths • Finding files – locate vs find • Filesystems • Mounting • Links • Devices and drivers • Permissions and ownership UNIX and Linux an Introduction, developed 5 of 71 by Mathias Homann
  • 6. Chapter Index (4) • Background processes • Scheduled processes • Processes and threads • Output and input redirection • Pipes • Shell history UNIX and Linux an Introduction, developed 6 of 71 by Mathias Homann
  • 7. Chapter Index (5) • The power of grep • Less is more • The vi editor • Regular expressions • Sed – the stream editor • Awk – pattern scanner for text files • Writing your own scripts • Syslog and log files UNIX and Linux an Introduction, developed 7 of 71 by Mathias Homann
  • 8. Chapter Index (6) • Finding facts about a system • Gathering statistics • Writing advanced scripts • More exercises • Useful commands • Higher wizardry – perl • Discussion and closing UNIX and Linux an Introduction, developed 8 of 71 by Mathias Homann
  • 9. A short history of UNIX Graphic credit: http://en.wikipedia.org/wiki/File:Unix.svg UNIX and Linux an Introduction, developed 9 of 71 by Mathias Homann
  • 10. A short history of UNIX (2) • 1969: Thompson, Ritchie and others start to “play with that unused PDP-7 in the corner” at Bell Labs, develop a system called “Unicks” based on Multics • 1971: UNIX was developed; first released version on March 11, 1971 • 1975: Bourne shell introduced • 1982: SGI introduces IRIX, their commercial flavour of UNIX for their workstation line • 1983: SCO delivers their first packaged UNIX for Intel architecture PCs; Richard Stallman introduces the GNU operating system on September 27 UNIX and Linux an Introduction, developed 10 of 71 by Mathias Homann
  • 11. A short history of UNIX (3) • 1986: HP releases HP-UX 1.0 • 1987: SUN and AT&T form an alliance to work on what will be UNIX System V Release 4.0 • 1990: IBM releases AIX • 1991: SUN releases Solaris OS, specially designed for symmetric multiprocessing • 1991: Linux Thorvalds sends his famous email to the minix developers mailing list, it starts with the words “I'm doing a (free) operating system ...” UNIX and Linux an Introduction, developed 11 of 71 by Mathias Homann
  • 12. UNIX today • today, UNIX is the operating system of choice for both large server farms and small embedded devices • Amazon, Google, your android phone and your media center that is hooked up to your TV all run on Linux, a free UNIX-like operating system UNIX and Linux an Introduction, developed 12 of 71 by Mathias Homann
  • 13. Linux is not UNIX • the open source community takes great joy in using “recursive acronyms”: – “GNU is not UNIX” – “Linux is not UNIX” • Linux is “UNIX-Like” • a lot in common, “UNIX-Compatible”, but also: – Not a serious, commercial development history / process – In a constant flow • UNIX is: – “carved in stone” before first deployment – commercially developed – Not free to use, unlike Linux, UNIX costs money UNIX and Linux an Introduction, developed 13 of 71 by Mathias Homann
  • 14. How to access a UNIX system • a “traditional” UNIX host usually has one local console, and several “dumb” terminals attached • today, such “dumb terminals” are usually PCs running terminal emulation software, instead of the old fashioned “true” serial consoles • a Linux system has one local physical console (keyboard and screen), with several virtual consoles available (usually six, to switch between them use the keyboard shortcuts ctrl-alt-F1 to ctrl-alt-F6) • virtual console #7 is usually occupied by the graphical desktop environment if such is used UNIX and Linux an Introduction, developed 14 of 71 by Mathias Homann
  • 15. Secure shell – SSH • a better way to access a UNIX system is through SSH – The most common SSH client for Windows is called putty, available from http://www.chiark.greenend.org.uk/~sgtatham/putty/ • SSH uses strong encryption – communication between the user and the system can not be eavesdropped on – the administrator password can not be intercepted – on the contrary, Telnet sends plaintext, unencrypted data over the network do not use telnet … ever • to access a UNIX system you need – a user account, and either have the password for that account – or have your SSH public key installed in the allowed keys list for that account UNIX and Linux an Introduction, developed 15 of 71 by Mathias Homann
  • 16. Secure shell – SSH (2) • to login to a UNIX system through SSH, you use a command like this: ssh joe@example.host.com • useful commandline options for SSH: -1 and -2 to enforce SSH protocol version 1 or 2 -4 and -6 to enforce IPv4 or IPv6 -A enabled SSH agent forwarding (-a to disable) -X enable X11 forwarding (-x to disable) -C compresses data (useful on slower connections) UNIX and Linux an Introduction, developed 16 of 71 by Mathias Homann
  • 17. SSH key management • SSH uses an asymmetric key pair • to generate a key pair, use ssh-keygen on UNIX, or puttygen on Windows • the public key is what you install on servers that you need to access • the private key needs a passphrase to be used, and is the part that you need to keep safe UNIX and Linux an Introduction, developed 17 of 71 by Mathias Homann
  • 18. SSH key management (2) • the pass phrase for unlocking your private key can be significantly longer and more complex than a system password • to register your key for access to a system under one user account, append the content of your public key to ~/.ssh/authorized_keys • verify host keys by comparing the fingerprint of the host key with the fingerprint of the host key in your local known_hosts file • host keys do not change randomly UNIX and Linux an Introduction, developed 18 of 71 by Mathias Homann
  • 19. Building tunnels with SSH • a tunnel is an encrypted “hole” that data can flow through • tunnels are used to create secure connections through firewalls • tunnels can be used to access services securely • a tunnel can: – forward a local port to the remote machine – a port on the remote machine to the local machine – a port on a machine in the same network as the remote machine to a port on the local machine UNIX and Linux an Introduction, developed 19 of 71 by Mathias Homann
  • 20. Building tunnels with SSH (2) • parameters for tunnels: -L x:host:y – forward localhost:x to host:y -R x:host:y – forward host:x to localhost:y • example: – a tunnel could be used to map the web server on a remote system to the local port 8080 • this way, entering http://localhost:8080/ in your browser would access the remote server UNIX and Linux an Introduction, developed 20 of 71 by Mathias Homann
  • 21. Building tunnels with SSH (3) • exercise – Generate your key pair with putty – Connect to (hostname) as (username) ← will have to be adapted for actual training environment – Forward the webserver running on the loopback interface on (hostname) to port 8080 on your local computer – Verify with your web browser that the tunnel works UNIX and Linux an Introduction, developed 21 of 71 by Mathias Homann
  • 22. Learning about UNIX commands • UNIX has two built-in documentation systems: – info pages – man pages • to read the man page (manual page) of a given command, use man command – Try this out with man man • to look if there is a command that does something, try the apropos command • the apropos command lists the summary lines of all man pages that contain the term you specified UNIX and Linux an Introduction, developed 22 of 71 by Mathias Homann
  • 23. Learning about UNIX commands (2) • to look at what a certain command does without reading the whole man page, use whatis program • this prints out the summary only of the command's man page • to find where in the filesystem a command is, use the which command UNIX and Linux an Introduction, developed 23 of 71 by Mathias Homann
  • 24. Shells and the environment • the shell is the interface to the system • the shell executes commands and shows the results • the “environment” is the total of stored settings inside a shell • important shell variables on UNIX systems: $PATH $PS1 $CWD UNIX and Linux an Introduction, developed 24 of 71 by Mathias Homann
  • 25. Shells and the environment (2) • UNIX knows several different shells: sh bash ksh zsh csh • the default shell on most Linux systems is bash • the set command shows all environment variables • change environment variables by assigning and exporting them • all exercises will use bash if not stated otherwise UNIX and Linux an Introduction, developed 25 of 71 by Mathias Homann
  • 26. Internal and external commands • shells have built-in versions of most “system commands” • to “force” the use of a system command instead of the builtin, use the absolute path to the binary: /bin/ls • external commands are standardized • builtins differ from shell to shell UNIX and Linux an Introduction, developed 26 of 71 by Mathias Homann
  • 27. Interactive shells + startup scripts • an “interactive” shell is one that you type commands into • “non-interactive” shells are the ones that execute scripts • a “login” shell is the first interactive shell that gets started when you log in • the “profile” or “rc file” is a script that gets executed by a shell on startup • most shells execute more than one startup script, and execute a different set of scripts when started non- interactive, and yet another for the login shell UNIX and Linux an Introduction, developed 27 of 71 by Mathias Homann
  • 28. Interactive shells + startup scripts (2) • examples - the bash shell executes on startup as a login shell (in this order): /etc/profile ~/.bash_profile ~/.bash_login ~/.profile • if the bash shell is started interactively but not as a login shell it only executes ~/.bashrc • a non-interactive bash shell executes the file listed in $BASH_ENV if the variable is set and the file exists • note: a login bash executes ~/.bash_logout on exit UNIX and Linux an Introduction, developed 28 of 71 by Mathias Homann
  • 29. Wildcards • a wildcard is a pattern that describes a path or filename • wildcard expansion in shell commands happens on the shell level, before the actual command gets executed • different shells handle wildcards in different ways • exercises: – Which wildcard would mean “all jpeg files in this folder”? – Which pattern means “all jpeg files in this folder that start with the letter a”? UNIX and Linux an Introduction, developed 29 of 71 by Mathias Homann
  • 30. Command aliases • a command alias is a “placeholder” for a command and optionally one or more commonly used options • for example, on openSUSE Linux there is an alias “dir” which calls ls -l • a list of all aliases is shown by entering the alias command with no parameters • to see the aliases and wildcards expansion before commands get executed, enter set -x UNIX and Linux an Introduction, developed 30 of 71 by Mathias Homann
  • 31. Paths • paths on UNIX systems are all inside a “tree” of folders that starts at the “root” folder or / – the user's home folder is usually held under /home and named after the username, e.g. /home/fred – the “search path” describes which folders to look in when a shell tries to find a command – the search path is stored in the environment variable $PATH – the minimal search path should contain /bin and /usr/bin – directories in the search path are separated by the colon: PATH=/bin:/usr/bin UNIX and Linux an Introduction, developed 31 of 71 by Mathias Homann
  • 32. Everything is a file ... • ... not just a file • /proc/12345/status is a “virtual file” containing status information about process 12345 • /dev/mem is a “device file” that is actually an interface to the system's memory • /dev/sg0 is a “device file” for the first “generic” SCSI device in a system • /dev/sdb5 is a “device file” that references the first logical partition on the second external, SCSI or SATA disk UNIX and Linux an Introduction, developed 32 of 71 by Mathias Homann
  • 33. Finding files – locate vs find • there are two ways to find a file, locate and find • the locate command uses a database of existing files which needs to be updated regularly, which usually happens once a day • the find command actually searches through the filesystem – this can take a while, but does not rely on a database that can be outdated locate can only search by shell wildcards find can search for anything, not just names • exercise: – find all files that changed since yesterday UNIX and Linux an Introduction, developed 33 of 71 by Mathias Homann
  • 34. Filesystems • UNIX knows several different filesystems • one of the main differences is whether a filesystem is “journaling” or not • A journaling filesystem keeps track of the changes that will be made in a journal, before committing them to the main filesystem. In the event of a system outage, such filesystems are quicker to bring back online and less likely to be corrupted. • commonly used filesystems on Linux are: – ext2, ext3, ext4, jfs, xfs, reiserfs – Ext2 is the only of these that is not journaling UNIX and Linux an Introduction, developed 34 of 71 by Mathias Homann
  • 35. Filesystems (2) • filesystem limits: names paths filesize volume size ext2 255 unlimited 16GB - 2TB 2TB-32TB ext3 255 unlimited 16GB - 2TB 2TB-32TB ext4 256 unlimited 16GB - 16TB 1EB jfs 255 unlimited 8TB 512TB-4PB xfs 255 unlimited 8EB 8EB reiserfs 4032/226 unlimited 8EB 16TB UNIX and Linux an Introduction, developed 35 of 71 by Mathias Homann
  • 36. Mounting • all disk partitions need to be mounted to be accessible • a disk partition gets mounted into a folder, called the “mount point” • a UNIX system has a file with a list of all mount points that need to be mounted when the system starts: /etc/fstab – to mount a filesystem that is not listed in /etc/fstab you need superuser privileges – to access a partition you need to know its device node – device nodes are special files in /dev UNIX and Linux an Introduction, developed 36 of 71 by Mathias Homann
  • 37. Mounting (2) • for example, the device node for a partition on a hard disk would be /dev/hda2 - meaning “the second primary partition on the first hard disk” • exercise: – explore what is mounted, and what is mounted on boot UNIX and Linux an Introduction, developed 37 of 71 by Mathias Homann
  • 38. Links • a link is a “pointer” to a file – can be a “soft” or a “hard” link – a softlink is a pointer at a file, and can link across filesystems – a hardlink is another directory entry that points at the same starting sector of the filesystem • hardlinks are only possible within one filesystem, and cannot point at directories • links are created with the ln command ln creates hardlinks by default – for softlinks use the -s option • exercise: – create a link to /usr/bin/bash in ~/bin/ UNIX and Linux an Introduction, developed 38 of 71 by Mathias Homann
  • 39. Devices and drivers • every piece of hardware is accessed through a device file under /dev • examples: /dev/hda - the first IDE harddisk /dev/sdb - the second SCSI or SATA disk /dev/nvidia – an NVIDIA graphics card /dev/dsp0 – the sound card UNIX and Linux an Introduction, developed 39 of 71 by Mathias Homann
  • 40. Devices and drivers (2) • access to hardware is governed by the file permissions on those special files • for example, the permissions on a graphics card could be set to allow only root and anyone in the group “video” access to write to it, therefore only users in the group “video” would be able to run 3d accelerated applications • exercise: – find out which group has write access to the cd/rw drive, e.g. would have sufficient permissions to use a CDRW writer on the system UNIX and Linux an Introduction, developed 40 of 71 by Mathias Homann
  • 41. Permissions and ownership • there are 3 levels of ownership of any file: – “owner” – “group” – “others” • permissions on a file: – read, write, execute, separately for all three owner types • example: – a file usually is read/write for the owner, and read-only for anyone else: -rw-r--r-- • change the owner: chown username file • change the group: chgrp groupname file • change permissions: chmod <permissions> file UNIX and Linux an Introduction, developed 41 of 71 by Mathias Homann
  • 42. Background processes • UNIX can run commands in the background so that you can do something else – To execute a program in the background, end the commandline with & – To send a running program to the background, press ctrl-z and enter bg – To view the list of running background processes, enter jobs • when you logout while you still have background processes running, they will be killed unless you started them with nohup: nohup sleep 10000 will run sleep 10000 in the background – the output of the command will be written to nohup.out UNIX and Linux an Introduction, developed 42 of 71 by Mathias Homann
  • 43. Scheduled processes • to run a program at regular intervals, use cron • cron keeps lists of scheduled jobs • edit the list with crontab -e • to run a program at a certain time once, use at at 23:00 asks for a sequence of commands • the job gets deleted once it runs UNIX and Linux an Introduction, developed 43 of 71 by Mathias Homann
  • 44. Processes and threads • a process is one running executable – A single process can have more than one running thread – To see all processes, use top, htop, or the ps command – To send signals to processes, use the kill command • several “levels” of “kill signal”, ordered by “lethality”: kill -HUP tells a process to shut down nicely kill -QUIT – the same but a bit more urgent kill -KILL kills a process kill -l lists all possible signals with name and numeric value kill -9 is equivalent to kill -KILL • combined exercise: – start a process in the background that waits for 10 minutes – kill it before the time is over UNIX and Linux an Introduction, developed 44 of 71 by Mathias Homann
  • 45. Output and input redirection • in- and Output can be redirected with the < and > symbol • redirect input: cat < /tmp/file.txt • redirect output: cat < /tmp/file.txt > /tmp/foo.bar • redirect the error output: sort < foo 2>/dev/null UNIX and Linux an Introduction, developed 45 of 71 by Mathias Homann
  • 46. Pipes • pipes are used to direct the output of one command to the input of another • pipes use the pipe symbol: | • example: ls -la | less pipes the output of the ls command into the less pager • exercise: sort the content of the passwd file by alphabet UNIX and Linux an Introduction, developed 46 of 71 by Mathias Homann
  • 47. Shell history • UNIX shells keep track of your commands • to browse through this list use the cursor up and cursor down keys • to re-execute a previous command, find it in your history and press enter • to find a previous command more quickly, press ctrl-r and start typing; the history will “jump” to the newest match of what you're typing UNIX and Linux an Introduction, developed 47 of 71 by Mathias Homann
  • 48. Shell history (2) • to view the whole history, use the “history” command • to re-execute a certain command, find its number in the history, then enter !number, e.g. !514 to re-run number 514 from your history • to search in your history: press ctrl-r, then start typing • exercise: – use a few commands (ls, man, ls -l, ls -la) then view the history, then re-run some of them, then search in your history UNIX and Linux an Introduction, developed 48 of 71 by Mathias Homann
  • 49. The power of grep • grep searches for text in files • grep can use regular expressions • grep has many options - read the man page • most useful options: – -i for “search case insensitive”, and -v for “inverse” or “search for anything that does not match my search string” • example: grep myusername /var/log/messages – find all lines in logfile that contain myusername • exercise: – filter the main system logfile for all lines from last Monday UNIX and Linux an Introduction, developed 49 of 71 by Mathias Homann
  • 50. Less is more • less is a “pager” to view large text files on screen • less can search for text: – press : then enter a slash followed by your text • less can switch to “continuous refresh” to refresh a file that is still being written; press F to switch • less can read from “stdin”; pipe commands into it to paginate the output • g and g jump to the end and the start of the file • to search in less, type /searchterm • to jump to a line number, type :number • example: less /var/log/messages • exercise: – display the content of /usr/bin in less UNIX and Linux an Introduction, developed 50 of 71 by Mathias Homann
  • 51. The vi editor • the editor vi exists on every UNIX system • vi has two modes: insertion mode and command mode – The editor begins in command mode, where the cursor movement and text deletion and pasting occur – Insertion mode begins upon entering an insertion or change command – [ESC] returns the editor to command mode (where you can quit, for example by typing :q!) • most commands execute as soon as you type them except for "colon" commands which execute when you press the return key • nifty: vi does syntax highlighting; activate with :syntax UNIX and Linux an Introduction, developed 51 of 71 by Mathias Homann
  • 52. The vi editor (2) • quitting vi: :x Exit, saving changes :q Exit as long as there have been no changes ZZ Exit and save changes if any have been made :q!Exit and ignore any changes • switching to edit mode: i – insert r – replace a – append UNIX and Linux an Introduction, developed 52 of 71 by Mathias Homann
  • 53. The vi editor (3) • cursor movement • cursor keys, and: ^ or 0 – move to start of line $ - move to end of line • block and line operations: dd – delete line yy – copy line p – paste line UNIX and Linux an Introduction, developed 53 of 71 by Mathias Homann
  • 54. Regular expressions • regular expressions are powerful string matching patterns • examples: . - any one character * - any repetitions of the pattern left of the star .* - any number of any characters [:upper:] - all UPPER CASE characters n – newline [a-q] – any letter between a and q UNIX and Linux an Introduction, developed 54 of 71 by Mathias Homann
  • 55. Regular expressions (2) • more info: • the “Regular expressions cheat sheet v2” – http://www.addedbytes.com/cheat-sheets/regular-expressions- cheat-sheet/ • example: – a regular expression cat.*dog matches any string that has “cat” and “dog” in it with any other characters in between them, but only in that order! UNIX and Linux an Introduction, developed 55 of 71 by Mathias Homann
  • 56. Sed – the stream editor • Sed reads text from stdin, does edit operations on it, and prints the result on stdout • examples: sed -e 's/red/green/g' <colors.txt >colors1.txt – replaces all occurrences of “red” with “green” • exercise: – read the man page, then write a text file, then replace all “the” with “THE” • exercise: – print out each directory in the path variable in a new line UNIX and Linux an Introduction, developed 56 of 71 by Mathias Homann
  • 57. Awk – pattern scanner for text files • Awk scans text files for patterns and operates on them • examples: awk '{print $2“n“}' < file.txt – print the second word of every line of file.txt • exercise: – read the man page, then use vi to write a text file with a few lines of the format: Username:Firstname:Lastname:Telephone number – then, use awk to convert this into ldap syntax: dn: uid=Username, dc=example, dc=com cn: Firstname Lastname sn: Lastname telephoneNumber: Telephone number UNIX and Linux an Introduction, developed 57 of 71 by Mathias Homann
  • 58. Writing your own scripts • scripts can be written with any text editor • scripts don't need a special filename suffix • scripts need to be executable (hint: chmod) • scripts can explicitly state which shell should be used to execute it by putting the “hashbang” for the shell in the first line: #!/bin/bash – a bash script #!/bin/csh – a c shell script #!/bin/sh – a “sh” shell script • “sh” is the most basic shell; exists on any UNIX • exercise: – write a script that prints “hello world” ten times UNIX and Linux an Introduction, developed 58 of 71 by Mathias Homann
  • 59. Syslog and log files • syslog is a UNIX daemon that collects and handles log file messages • syslog handles messages according to severity and facility • details: see man page • syslog can write messages to one or more files depending on severity and facility UNIX and Linux an Introduction, developed 59 of 71 by Mathias Homann
  • 60. Syslog and log files (2) • logfiles usually exist in /var/log • the syslog configuration file is in /etc and states which facility.severity goes in which file • exercise: – check the syslog configuration to see which files hold messages of failed logins – then check if there were any recent failed login attempts UNIX and Linux an Introduction, developed 60 of 71 by Mathias Homann
  • 61. Finding facts about a system • most Linux systems have a file in /etc/ that contains information about which distribution is running – for example, /etc/redhat-Release uname -a prints out information about the system • use mount and df to find out about disk sizes and mount points • a lot of information is in /proc • almost all configuration files are in /etc and its subdirectories UNIX and Linux an Introduction, developed 61 of 71 by Mathias Homann
  • 62. Finding facts about a system (2) • most UNIX systems use some form of software package management – the list of installed packages can tell you a lot about the purpose of a system – on Linux there are two widely used package management systems, rpm and apt/dpkg – rpm is used on openSUSE, redhat, and their derivatives and commercial versions – on debian, ubuntu and derivatives, apt and dpkg is used – commands such as rpm -qa and dpkg –get-selections show the list of installed packages – these lists are long, less is your friend UNIX and Linux an Introduction, developed 62 of 71 by Mathias Homann
  • 63. Finding facts about a system (3) • to list all loaded kernel modules lsmod • use insmod or modprobe to load additional kernel modules into a running system • use modprobe -r to remove a module and all dependencies from a running system • most of them are hardware drivers or drivers for “features” UNIX and Linux an Introduction, developed 63 of 71 by Mathias Homann
  • 64. Gathering statistics • commands to explore the status of the system vmstat and free show memory usage iostat shows drive accesses netstat shows network statistics netstat can also show active network connections and listening processes; read the man page • exercise: – check how much memory the machine has, and how much is used; then check how many active network connections exist that are not SSH connections UNIX and Linux an Introduction, developed 64 of 71 by Mathias Homann
  • 65. Writing advanced scripts • scripts can do math: a=$(($b+$c)) • scripts can have flow control – if (condition) then {command; command;}; else { command; command;}; endif – do { command; command; } while (condition); – for (iterator); do { command; command; } done; • scripts can have functions • scripts can return values to the calling process UNIX and Linux an Introduction, developed 65 of 71 by Mathias Homann
  • 66. Writing advanced scripts (2) • exercise: – write a script that: – finds all log files under /var log – takes all files older than 2 days – prints their name, size and age – (size in 'human readable' format, 1024k = 1 M etc) UNIX and Linux an Introduction, developed 66 of 71 by Mathias Homann
  • 67. More exercises • create a file “a”, write “hello world” into it, then hardlink it as “b” • check the inodes of both files – (hint: there's an option for ls that does that) • create a softlink “c” to “a”, check inodes • run tail -f /var/log/messages, send it to the background, edit a different file in vi, exit vi, bring the tail command back to the front • compare the differences between the various ps output formats: ps –ax ps -ef ps -axf – sort /etc/passwd alphabetically – sort it by uid UNIX and Linux an Introduction, developed 67 of 71 by Mathias Homann
  • 68. Useful commands • cd – change directory • pushd – change directory but it stores the previous directory on a fifo stack • popd – returns to previous directory from the stack • cat – read from file or stdin, write to file or stdout (example: cat /etc/motd to print that file to screen) • cp – copy files • mv – move / rename files • head & tail – print the first or last lines of a file • find – search for files UNIX and Linux an Introduction, developed 68 of 71 by Mathias Homann
  • 69. Useful commands (2) • less – display a file in a pager • grep – search for text in files • sed – command-line editing of files • wc – count lines/words/characters in files • df – disk free (shows free space in partitions) • du – disk usage (adds up file sizes, more accurate than df, takes some time to finish) • gzip – compress files, use gunzip to decompress • diff – compare files UNIX and Linux an Introduction, developed 69 of 71 by Mathias Homann
  • 70. Higher wizardry – perl • perl is disputedly the most powerful, simple, complicated, and obscurely easy to understand scripting language • perl is enough for its own workshop day :) UNIX and Linux an Introduction, developed 70 of 71 by Mathias Homann
  • 71. Discussion and closing • questions • comments • Instructor Evaluation Form UNIX and Linux an Introduction, developed 71 of 71 by Mathias Homann