This document discusses the benefits of using Catalyst, DBIC, and Template Toolkit for building web applications. It notes that these tools help address issues like repetitive code, SQL injection vulnerabilities, and separating concerns that arise when using plain Perl CGI scripts and DBI. The document provides an overview of these frameworks and some lessons learned, such as spending time learning Template Toolkit syntax, using models to encapsulate database access, and writing tests to validate models.
2. My Background
• Lazy (Programmer)
• Often involved in sys admin (repetitive)
work
• Leverage CPAN so I don’t have to figure
things out others already have (Lazy)
3. Scripts to:
In the beginning...
• automate maintenance
• systems checks, report by email.
• check status of various processes/disk/
memory
• automate user tasks
4. Scripts to:
In the beginning...
• automate maintenance
• Job failures difficult to detect
5. In the beginning...
Scripts to:
• systems checks, report by email
• Inbox flooded.
• Still have to read and process them
• Follow ups often require system access
6. Scripts to:
In the beginning...
• check status of various processes/disk/
memory ($> appname_health.pl)
• often requires privileged access
• output in fixed font
7. Scripts to:
In the beginning...
• automate user tasks (mkuser, unlock
account, etc)
• user at least has to have a login
9. CGI - Benefits
• Accessible to anyone with a web browser
• Programs easy to deploy
• No need to login to host
10. CGI - Bad habits
• Accessible to anyone with a web browser
• Perl compile cost every page hit
• Long running processes not easily possible
• HTML Code mixed in with perl code
(HERE or qw{}) print q{<HTML>
hello world
</HTML>
};
11. DBI - Benefits
• Standardized access to back end data
• Database not perl process does hard query
work
12. DBI - Bad habits
• Vulnerable to SQL injection
• Perl code is hard coded to a specific DB type
(MySQL, Postgres, Oracle, Sybase, MSSQL)
• Redundant prep/execute/extract code usually
splashed all over the code
• Detection of errors usually not standardized
• SQL code (often many lines long) intermixed with
perl code. (Hard to read)
• Badly formed SQL can reveal sensitive information
to end user
13. DBI - Bad habits
• Vulnerable to SQL injection
• select * foo where bar = “$test”
• $test = q{5; insert into admin_table...};
• $test = q{“blah}
• Difficult to handle BLOB data
• Placeholders limited
• where of select, set of update, values for insert)
• From clause inaccessible via placeholders
16. The Learning Curve
• Moose!
• DBIx::Class
• TT
• Catalyst
• Model/View/Controller paradigm a difficult balance to maintain
• Perl Testing techniques in Catalyst
• Catalyst unit testing (how do I exclude the models in the
test?)
• Model testing (build a test DB complicated enough to test
the model)
• Selenium (minor changes in TT easily break tests)
17. Moose
• Easy to learn for if you’ve done OO
programming before
• http://search.cpan.org/perldoc?
Moose::Cookbook
• No more new!?!?
• Not critical to learn but useful when
thinking about how Catalyst works
18. DBIx::Class
• Best way to call SQL but read like perl
• Standard interface to returning query from sub
my $rs = $schema->resultset(‘CD’)
->search({ year => 2000 },
{ prefetch => 'artist' });
while ($rs->next) {
print $rs->artist . “n”;
}
my $dbi = DBI->connect(...) or die DBI->error;
my $query = q{select artist from CD where year = 2000};
my $sth = $dbi->prepare($query) or die;
$sth->execute or die;
while(my ($artist) = $sth->fetchrow_array) {
print “$artistn”;
}
19. DBIx::Class table setup
• Schema Loader Magic! (unless it’s not a well
built Schema)
• Make sure fkeys,pkeys setup in db first if you
can.
• Not all Pg features pull from schema loader
• DBIx::Class::Schema::Loader to load schema
• Alternatively: cd ~catapp; script/create.pl model
CatalystModelName...
20. DBIx::Class - Caveats
• wrap insert/updates in eval {} or x; blocks
• txn_do($code_ref) (still inside eval block)
• left outer joins by default. (null rows removed)
• Understand when a query is a row or a result set.
They behave differently.
• inflate/deflate only works on rows, not $rs
• relation columns can have same name as variables.
causes you to have to do $row->get_column
21. Learning TT
• Think Cool PHP (if it was perl)
• http://template-toolkit.org/docs/manual/
• Syntax differences can be subtle and TIMTOWTDI
• Catalyst will do all the heavy lifting here.
• Mailing list is very friendly.
22. TT: Advice from the
trenches
• “Spend a solid day or more dedicated to learning Template::Toolkit. Andy Wardley has
done an amazing job at creating and maintaining this system. By spending a little time
up front, it will save you a TON of time in the future and get you familiar with its
capabilities. Learn how to make it sort arrays, sort hash keys, loop through keys of
hashes, etc - things that are vital to the template development for web applications.”
• TT manual can be confusing to the beginner.
• “some basic concepts were buried in the middle of examples of various features,
so I had to read about almost everything before I could do more than the trivial.”
• “I felt the TT language had too many syntactic variations for accomplishing the
same thing, and again the examples randomly chose which version they used in
order to demonstrate this breadth, but until you get to the part which explains
the equivalences, you are left scratching your head trying to figure out what the
semantic differences are.”
• “Aside from that, TT seems to be a powerful, stable templating system.”
23. TT - Trenches II
• Case matters: [% FOR foo IN chairs.keys.sort %]
• Prevent line wrap: [% author -%]
• Altering the stash is local (sometimes): [% foo = ‘HELP’ %]
24. Learning Catalyst
(are we there yet?)
• CPAN Tutorial
• Catalyst Examples from SVN
• Catalyst Advent Calendar
• Johnathan Rockway’s Book “Catalyst”
• Initial setup is 90% of the battle
• catalyst.pl AppName
25. Cost Justification for
the learning Curve
• Multi-developer approach easier when code is broken into
multiple modules
• Automated testing setup for you
• Stupid DBI mistakes no longer a risk (Now it’s stupid DBIC
mistakes but they fail more consistently)
• DB Structure separated from code in one place.
• Annoying HTML page setup for each CGI script centralized
• Model structure allows for easy re-use outside of Catalyst
• Application portable to multiple developers with self
contained web server.
26. MVC
• Spend time thinking about and talking about
the demarcation for MVC
• http://en.wikipedia.org/wiki/Model-view-controller
27. View
• What presents the data to the user
• email
• HTML
• JSON
• file download
• csv
28. Controllers
• Air traffic control of the application
• Receives user input
• Authenticates and authorizes access with help of
models
• Requests information from models based on user input
• Data cleansing of user input
• Formats data (input and output) to present back to
view for display
29. Models
• Library of calls for controllers and outside programs to:
Query/update/insert information by:
• Invoking programs
• Accessing databases
• Messaging (email, IM)
• Maintains Data integrity where pkey/fkeys stop.
• Enforces authorization based on controllers provided
credentials
• Centralizes auditing of events
30. Lessons learned from
Controller Development
• Do all authentication and authorization in
root.pm via auto (Private)
• Take care not to do view or model work in
the controllers. This is the easiest place to
do too much
• Common Controller routines or repetitive
code sections indicates you have a model
routine
31. Lessons learned from
Model Development
• Pod EVERYTHING. It takes a very short time to forget
what it does or why it’s there or how to use it.
• Inheritance is probably not the right solution - Use Moose
Roles instead to separate the sections of your models for
better management
• Models provided us a way to standardize back end
interface to on-boarding.
• Create a common library to validate hash passed to model
• my $params = validate_hash(shift, qw/date_inserted id
owner on_hold/) or return toss(“reason...”);
32. Is it a model or
controller job?
• Whether you agree with all of this
doesn’t matter. What matters is you
become aware of the coding issues and
decide which way you’re going to go.
33. Model test suite
• Can be difficult to create test databases
complicated enough to validate functionality
or replicate errors.
• Clear /re-build database between tests
• input/output can be complex. Build helper to
make life easier (test_model_func({model=>..,
func => ..., args => ...., toss =>...,}))
• Controller bugs require a test to fix in model
34. Apache + FastCGI
• Allows app to be scalable.
• Multiple perl engines can run in the back end
• Engines can be on external hosts
• Scales down connections on idle
• Remember [% c->uri_for(‘/’) %] in all your
templates
• Catalyst::View::TT->CATALYST_VAR
35. In the end
• Splitting up areas of responsibility between
controllers and models helps enforce:
• documentation
• interface standards
• testing
