Perl for System Automation - 01 Advanced File Processing

1. Perl for System Automation
1
Danairat T.
Line ID: Danairat
FB: Danairat Thanabodithammachari
+668-1559-1446

2. Danairat T.
Contents
Module-1: Advanced File Processing with Perl
• Types of open Filehandles and File Locking
• Read/Write File line by line
– Read Line
– Write Line
– Append Line
• Lock File
• Read entire files into Array
• Read entire files into Scalar using file record separator
• Read file paragraph by paragraph using file record separator
• Read files in the Directory
• Read SubDir in the Directory
• Read files recursively in Directory

3. Danairat T.
Contents
Module-2: System Interaction
• Enumerating OS user identity
• Enumerating OS group identity
• Activating System Command
– Using exec()
– Using system()
• Grabbing a program's output
– Using qx()
• Display current running processes
– Using ProcessTable module
• Terminate the Process
– Built in kill() function

4. Danairat T.
Contents
Module-3: Managing Networks with Perl
• Host file
• IP Lookup
• FTP
• E-mail
• Web
• LDAP

5. Danairat T.
Contents
Module-4: Perl Coding Guidelines
• Perl Coding Style Guides
– The 12 general guidelines
– Don’t do in perl codes
– To avoid in perl codes
– Performance coding guidelines
• Perl Coding Guides for Security
– Command line inputs
– The exec(), system(), qw() and Perl modules.

6. Danairat T.
What is Perl?
(Practical Extraction and Report Language )
• Perl, release in 1987, is a high-level
programming language written by Larry Wall.
Perl's process, file, and text manipulation
facilities make it particularly well-suited for
tasks system utilities, system management
tasks, database access, networking. These
strengths make it especially popular with
system administrators.
6

7. Danairat T.
Which version of Perl should I use?
• No one is actively supporting Perl 4. Five years ago it
was a dead camel carcass
• There is no Perl 6 release scheduled.
• The immediate releases are Perl 5.8 (i.e. Perl5.8.x )
7

8. Danairat T.
OS Platforms Support
• UNIX: More or less every UNIX or UNIX-like operating system ever created, notably
Linux , Solaris, AIX, IRIX, HP/UX, BSD, and Tru64
• MS Windows: DOS, Windows 3.1, 95, 98, NT and 2000
• Other Desktop OSs: Apple Macintosh (68k and PPC, both pre and post MacOS X),
Acorn Risc OS, Amiga, BeOS, OS/2, and many others
• Mainframes: AS/400, OS390, VMS and OpenVMS, Stratus (VOS), and Tandem
• PDAs: EPOC (Psion/Symbian), but not PalmOS or Windows CE at time of writing,
although porting efforts are being made for both platforms
8

9. Danairat T.
Identify the Perl version
• perl –v
• perl -V
9

10. Danairat T.
www.CPAN.org
• CPAN stands for
Comprehensive Perl
Archive Network,
containing libraries,
documentation, and many
Perl third-party modules
and extensions
10

11. Danairat T.
Install Perl
• On Unix System
– pkgadd -d /cdrom/sol_10_1009_x86/Solaris_10/Product SUNWperl584core
– pkgadd –d /cdrom/sol_10_1009_x86/Solaris_10/Product SUNWperl584usr
– pkgadd –d /cdrom/sol_10_1009_x86/Solaris_10/Product SUNWperl584man
• On Windows System
– run ActivePerl-5.8.9.826-MSWin32-x86-290470.msi
Available for download at www.activestate.com
11

12. Danairat T.
Perl file structure and location
• Perl executable: /usr/perl5/bin/perl or /usr/bin/perl
• Perl Pain Old Documentation: /usr/perl5/pod
• Perl Manual: /usr/perl5/man
12

13. Danairat T.
Perl Libraries (Perl Modules)
• The Perl Libraries directories
– /usr/perl5/5.8.4/lib/i86pc-solaris-64int
– /usr/perl5/5.8.4/lib
– /usr/perl5/site_perl/5.8.4/i86pc-solaris-64int
– /usr/perl5/site_perl/5.8.4
– /usr/perl5/site_perl
– /usr/perl5/vendor_perl/5.8.4/i86pc-solaris-64int
– /usr/perl5/vendor_perl/5.8.4
– /usr/perl5/vendor_perl
13

14. Danairat T.
Perl Library (The modules for Unix)
http://www.cpan.org/modules/01modules.index.html
14

15. Danairat T.
Perl Library (PPM modules for windows)
http://ppm.activestate.com/PPMPackages/zips/8xx-builds-only/Windows/
15

16. Danairat T.
Install Perl Modules on Unix
gzip -dc yourmodule.tar.gz | tar -xof -
cd /your/new/lib/to/install/
perl Makefile.PL
make
make test
make install
Or, if you have internet access
perl -MCPAN -e shell
cpan> i /Time/
cpan> install Time::CTime
16

17. Danairat T.
Install Perl Modules on Unix
bash-3.00# cd /tmp/Class-Simple-0.19
bash-3.00# perl Makefile.PL
Checking if your kit is complete...
Looks good
Writing Makefile for Class::Simple
bash-3.00# make
cp lib/Class/Simple.pm blib/lib/Class/Simple.pm
Manifying blib/man3/Class::Simple.3
bash-3.00# make test
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0,
'blib/lib', 'blib/arch')" t/*.t
t/anon.............ok
t/attributes.......ok
t/Class-Simple.....ok
t/inherit..........ok
t/isa..............ok
t/nonew............ok
t/override.........ok
All tests successful.
Files=13, Tests=95, 3 wallclock secs ( 0.99 cusr + 0.52 csys = 1.51 CPU)
bash-3.00# make install
Installing /usr/perl5/site_perl/5.8.4/Class/Simple.pm
Installing /usr/perl5/5.8.4/man/man3/Class::Simple.3
Writing /usr/perl5/site_perl/5.8.4/i86pc-solaris-64int/auto/Class/Simple/.packli
st
Appending installation info to /usr/perl5/5.8.4/lib/i86pc-solaris-64int/perlloca
l.pod
bash-3.00#
Example:-
17

18. Danairat T.
Installing Perl modules on Sun Solaris 10
1. Login as root
2. Add path /usr/sfw/bin to /.profile
3. Download and unzip the Perl module then cd
into its directory
4. Follow the below command steps:-
– /usr/perl5/bin/perlgcc Makefile.PL
– gmake
– gmake test
– gmake install
18

19. Danairat T.
Install Perl Module on Windows
1. Unzip the lib file
2. Cd to the downloaded lib
3. ppm install yourdownloadedlib.ppd
19

20. Module-1: Advanced File
Processing with Perl
20

21. Danairat T.
Topics
• Types of open Filehandles and File Locking
• Read/Write File line by line
– Read Line
– Write Line
– Append Line
• Lock File
• Read entire files into Array
• Read entire files into Scalar using file record separator
• Read file paragraph by paragraph using file record separator
• Read many files in a Directory
• Read SubDir in the Directory
• Read files recursively from Directories
21

22. Danairat T.
File Handle Options
22
mode operand create
delete and recreate
file if file exists
read <
write > ✓ ✓
append >> ✓
read/write +<
read/write +> ✓ ✓
read/append +>> ✓

23. Danairat T.
Open File Options
23
• Using < for file reading.
#!/usr/bin/perl
use strict;
use warnings;
my $myFile = "fileread.txt"; # the file “filetest.txt” must be exist
my $myLine;
if (open (MYFILEHANDLE, '<' , $myFile)) { # using ‘<‘ and . for file read
while ($myLine = <MYFILEHANDLE>) { # read line
chomp($myLine); # trim whitespace at end of line
print "$myLine n";
}
close (MYFILEHANDLE);
} else {
print "File could not be opened. n";
}
exit(0);
OpenFileReadEx01.pl
Results:-
<print the file content>

24. Danairat T.
Open File Options
24
• Using > for file writing to new file.
#!/usr/bin/perl
use strict;
use warnings;
my $myFile = "filewrite.txt";
my @myData = ("line1", "line2", "line3");
if (open (MYFILEHANDLE, '>' , $myFile)) {
foreach my $myLine (@myData) {
print MYFILEHANDLE "$myLine n"; # print to filehandle
}
close (MYFILEHANDLE);
} else {
print "File could not be opened. n";
}
exit(0);
OpenFileWriteEx01.pl
Results:-
<see from the output file>

25. Danairat T.
Open File Options
25
• Using >> to append data to file. If the file does not exist then it is create a
new file.
#!/usr/bin/perl
use strict;
use warnings;
my $myFile = "filewrite.txt";
my @myData = ("line4", "line5", "line6");
if (open (MYFILEHANDLE, ‘>>' , $myFile)) {
foreach my $myLine (@myData) {
print MYFILEHANDLE "$myLine n"; # print to filehandle
}
close (MYFILEHANDLE);
} else {
print "File could not be opened. n";
}
exit(0);
OpenFileAppendEx01.pl
Results:-
<see from the output file>

26. Danairat T.
File Locking
• Lock File for Reading (shared lock): Allow other to
open the file but no one can modify the file
• Lock File for Writing (exclusive lock): NOT allow
anyone to open the file either for reading or for
writing
• Unlock file is activated when close the file
26
Shared lock: 1
Exclusive lock: 2
Unlock: 8

27. Danairat T.
File Locking – Exclusive Locking
27
#!/usr/bin/perl
use strict;
use warnings;
use Fcntl;
my $file = 'testfile.txt';
# open the file
open (FILE, ">>", "$file") || die "problem opening $filen";
# immediately lock the file
flock (FILE, 2);
# test keeping the lock on the file for ~20 seconds
my $count = 0;
while ($count++ < 30)
{
print "count = $countn";
print FILE "count = $countn";
sleep 1;
}
# close the file, which also removes the lock
close (FILE);
exit(0);
FileExLockEx01.pl
Please run this concurrence
with FileExLockEx02.pl, see
next page.
Results:-
<see from the output file>

28. Danairat T.
File Locking – Exclusive Locking
28
#!/usr/bin/perl
use strict;
use warnings;
use Fcntl;
my $file = 'testfile.txt';
# open the file
open (FILE, ">>", "$file") || die "problem opening $filen";
# immediately lock the file
flock (FILE, 2);
# test keeping the lock on the file for ~20 seconds
my $count = 0;
while ($count++ < 30)
{
print "count : $countn";
print FILE "count : $countn";
sleep 1;
}
# close the file, which also removes the lock
close (FILE);
exit(0);
FileExLockEx02.pl
Please run this concurrency
with FileExLockEx01.pl
Results:-
<see from the output file>

29. Danairat T.
File Locking – Shared Locking
29
#!/usr/bin/perl
use strict;
use warnings;
use Fcntl;
my $file = 'testfile.txt';
# open the file
open (FILE, "<", "$file") || die "problem opening $filen";
# immediately lock the file
flock (FILE, 1);
# test keeping the lock on the file for ~20 seconds
my $count = 0;
while ($count++ < 30)
{
print "Shared Lockingn";
sleep 1;
}
# close the file, which also removes the lock
close (FILE);
exit(0);
FileShLockEx01.pl
Please run this concurrency
with FileExLockEx02.pl
Results:-
<see from the output file>

30. Danairat T.
Read Files from Command Line
• The command line argument will be stored in
@ARGV as default or you may straight away using <>
for file read.
• The below codes is to read file from command line
argument and print their content to the screen.
30
#!/usr/bin/perl -w
while (<>) { print $_; }

31. Danairat T.
Read Files from Command Line
31
#!/usr/bin/perl
use strict;
use warnings;
my $lineNo = 1;
# read values from @ARGV
foreach my $myFile (@ARGV) {
# read each file
if (open (FILEREAD, "<", $myFile)) {
flock (FILEREAD,1);
print "====Start $myFile====nn";
while (my $myLine = <FILEREAD>) {
chomp($myLine);
print "$lineNo: $myLine";
print "n";
$lineNo++;
}
close (FILEREAD);
} else {
print "Failed to open $myFilenn";
}
}
exit(0);
01_ReadFilesFromARGVEx01.pl
perl 01_ReadFilesFromARGVEx01.pl data01.dat data02.dat
Please see the result screen.

32. Danairat T.
Read File as a whole into Array
• Perl allows you to read file into array in a single
statement using = (assign) operator.
32
open (FILEREAD, $myFile);
@myLines = <FILEREAD>;
close (FILEREAD);

33. Danairat T.
Read File as a whole into Array
33
#!/usr/bin/perl
use strict;
use warnings;
# read values from @ARGV
foreach my $myFile (@ARGV) {
# read each file
die "Failed to open $myFilenn" unless (open (FILEREAD, "<", $myFile));
flock (FILEREAD,1);
print "====Start $myFile====nn";
my @myLines = <FILEREAD>; # read file into single array
close(FILEREAD);
my $lineNo = 1;
foreach my $myLine (@myLines) {
chomp($myLine);
print $lineNo++ . ": ". $myLine . "n";
}
}
exit(0);
02_ReadFilesIntoArrayEx01.pl
perl 02_ReadFilesIntoArrayEx01.pl data01.dat data02.dat
Please see the result screen.

34. Danairat T.
Read File as a whole into Scalar
• Perl provides you the quick way to read file into a
single scalar variable and process in your program
structure. Eg. You may need to read many input files
and append to one output file.
• $/ is a file record separator. Default is n. You need to
turn it off otherwise you can only read just one line.
34
undef $/; # undefine the file record separator
my $myLines = <FILEREAD>;

35. Danairat T.
Read File as a whole into Scalar
35
#!/usr/bin/perl
use strict;
use warnings;
# read values from @ARGV
foreach my $myFile (@ARGV) {
# read each file
die "Failed to open $myFilenn" unless (open (FILEREAD, "<", $myFile));
flock (FILEREAD,1);
print "====Start $myFile====nn";
undef $/; # undefine the file record separator
my $myLines = <FILEREAD>;
close(FILEREAD);
print "$myLinesn";
}
exit(0);
03_ReadFilesIntoScalarEx01.pl
perl 03_ReadFilesIntoScalarEx01.pl data01.dat data02.dat
Please see the result screen.

36. Danairat T.
Read File as Paragraph
• To read file paragraph by paragraph, you need to
assign “” the empty string to the file record separator
$/ for perl to detect the blank line as a break of
elements until match with the next line content
36
$/ = ""; # assign empty string to the file record separator
my @myParagraphs = <FILEREAD>;

37. Danairat T.
Read File as Paragraph
37
#!/usr/bin/perl
use strict;
use warnings;
# read values from @ARGV
foreach my $myFile (@ARGV) {
# read each file
die "Failed to open $myFilenn" unless (open (FILEREAD, "<", $myFile));
flock (FILEREAD,1);
print "====Start $myFile====nn";
$/ = ""; # assign empty string to the file record separator
my @myParagraphs = <FILEREAD>;
close(FILEREAD);
my $paragraphNo = 1;
foreach my $myParagraph (@myParagraphs ) {
chomp($myParagraph);
print "====Paragraph " . $paragraphNo++ . "====n" . $myParagraph . "n";
}
}
exit(0);
04_ReadFilesParagraphEx01.pl
perl 04_ReadFilesParagraphEx01.pl data01.dat data02.dat
Please see the result screen.

38. Danairat T.
Read many files in the Directory
• Perl allow you to open the directory using:-
– opendir (MYDIR, “/mydirectory”);
• To read the item from directory, using:-
– readdir (MYDIR);
• To close the directory, using:-
– closedir (MYDIR, “/mydirectory”);
38
#!/usr/bin/perl –w
opendir(DIR, '.') or die "Couldn't open directory, $!";
foreach (sort grep(/^.*.txt$/,readdir(DIR))) {
print "$_n";
}
closedir (DIR);

39. Danairat T.
Read Files in the Directory
39
#!/usr/bin/perl -w
use strict;
my $dir = '.';
opendir(DIR, $dir) or die $!;
while (my $file = readdir(DIR)) {
# We only want files
next unless (-f "$dir/$file");
# Use a regular expression to find files ending with .txt
next unless ($file =~ /.txt$/);
print "$filen"; # you can open the file here
}
closedir(DIR);
exit(0);
05_ReadFilesFromDirectory.pl
Please see the result screen.

40. Danairat T.
Read SubDir in the Directory
40
#!/usr/bin/perl -w
use strict;
my $dir = '../';
opendir(DIR, $dir) or die $!;
while (my $subDirName = readdir(DIR)) {
next unless (-d "$dir/$subDirName");
print "$subDirName";
}
closedir(DIR);
exit 0;
06_ReadSubDirFromDirectory.pl
Please see the result screen.

41. Danairat T.
Read File Recursively in Directory
1. The Cwd module for you to find the current
working directory.
2. The chdir() function is to use when you want
to change “perl program working directory”
to the another directory.
3. The recursive programming provides the
repeat of the same action with keeping your
history variables in the program stack and
automatically pop it up to complete the
program
41

42. Danairat T.
Read File Recursively in Directory
42
#!/usr/bin/perl -w
use strict;
use Cwd; # module for finding the current working directory
my $myDirectory = '../../';
ScanDirectory($myDirectory);
exit 0;
sub ScanDirectory {
my $workdir = shift; # same as shift(@_)
my $startdir = cwd; # keep track of where we began
chdir $workdir or die "Unable to enter dir $workdir: $!n";
opendir (DIR, '.') or die "Unable to open $workdir: $!n";
my @names = readdir (DIR) or die "Unable to read $workdir: $!n";
closedir (DIR);
foreach my $name (@names) {
next if ( $name eq '.' );
next if ( $name eq '..' );
if ( -d $name ) { # is this a directory?
ScanDirectory($name);
next;
}
if ( $name =~ /.txt$/i ) { # is this a file named "core"?
print "$startdir/$workdir/$name n";
}
}
chdir $startdir or die "Unable to change to dir $startdir: $!n";
}
07_ReadFilesFromDirectoryRecursive_Ex01.pl
Please see the result screen.

43. Danairat T.
Line ID: Danairat
FB: Danairat Thanabodithammachari
+668-1559-1446
Thank you