Vector Databases 101 - An introduction to the world of Vector Databases
Schwartz ez proxy-logs
1. Application of EZProxy logs, Voyager’s
Patron Database, MySQL, and
ColdFusion for Comparative Usage
Analysis of Library Resources
Ray Schwartz,
William Paterson University of NJ,
schwartzr2@wpunj.edu
ENUG Conference, University of Bridgeport, CT
Thursday, October 27, 2011
5. Example of EZProxy log entry
• Ip address nj.dhcp.embarqhsd.net
• (Not used) ‐
• user id theuser
• date/time 1/1/2008 4:25:15 AM
• Method GET
• page http://ezproxy.wpunj.edu:2048/connect?session=sGHMbeSss121YxZa&url=
retrieved http://www.wpunj.edu/scripts/webscript.exe?fs.scr
• Version HTTP/1.1
• response 302
code
• no. of bytes 537
• Referring http://ezproxy.wpunj.edu:2048/login?url=http://www.wpunj.edu/scripts/w
URL ebscript.exe?fs.scr
• User agent Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)
5
6. MySQL operations
• EZProxy transactions would be stored in one
table with patron statistical categories, but
without the user ID.
• User ID s would be stored in another table
with counts for each service divided by
academic year.
• Logs are collected monthly and loaded and
deleted monthly.
6
7. Perl Script for loading ezproxy log
use strict;
into MySQL
my
%month=(Jan=>'01',Feb=>'02',Mar=>'03',Apr=>'04',May=>'05',Jun=>'06',Jul=>'07',Aug=>'08'
,Sep=>'09',Oct=>'10',Nov=>'11',Dec=>'12');
while (<>){
my $pattern =
'^(S*) (S*) (S*) (S*) '.
'[(..)/(...)/(....):(..):(..):(..) .....]'.
' "(S*) (S*) (S*)" '.
'(d*) (‐|d*) "([^"]*)" "([^"]*)"';
if (m/$pattern/){
my ($tgt,$ref,$agt) = (esc($12),esc($16),esc($17));
my $byt = $15 eq '_'?'NULL':$15;
print "INSERT INTO ezproxylogs VALUES ('$1','$2','$3',".
" TIMESTAMP '$7/$month{$6}/$5 $8:$9:$10','$11','$tgt',".
"'$13',$14,$byt,'$ref','$agt');r.";
}else{
print "‐‐Skipped line $.n";
}
}
sub esc{
my ($p) = @_;
$p =~ s/'/''/g;
return $p; 7
}
