This document summarizes Neil Bowers' process for reviewing CPAN modules. He begins reviews by searching for existing modules on a topic and writing sections comparing the modules. He aims to submit fixes and improvements, get involved in maintenance, and publish reviews iteratively. The reviews have led Neil to make contributions to CPAN and improve his own practices, like benchmarking and reducing dependencies. He encourages others to search CPAN before writing new modules and to improve documentation.

1. Reviewing CPAN modules
Neil Bowers
1

2. • Apparently there are 1,900+ Torch/Flashlight apps…
2

3. 3

4. What is CPAN?
• Something like a university library?
• Lots of useful items, nicely catalogued, easily found
Atoolchest full of precision instruments that good programmers
can wield to make great things
- chromatic
• I think more like Rube Goldberg's workshop:
• A bunch of useful things, some half-finished ideas, practical jokes, stuff
that no longer works, items that shouldn't be there, forgotten things in
the corner
Holy crap, I forgot about [that module that I wrote].
- Andy Lester
4

5. Review #1
Generating passwords
Modules that can be used to automatically generate passwords,
either random sequences of characters or pronounceable pseudo words
5

6. Modules for generating passwords
6

7. Generating random char strings
• 100,000 passwords
7

8. Structure of reviews
8

9. After doing the review
• I'm now using Crypt::YAPassGen
• v0.02 released 2004-05-30
• App::Genpass is good
• I've submitted changes to add features I liked in other modules
• I've ended up using Data::Random elsewhere
• Because I was aware of it having done this review
• Hoping to finally release fix for Crypt::RandPasswd
• I was granted co-maint yesterday (23rd Nov 2012)
9

10. Review #2
Looking up location of an IP address
Modules that can map an IP address to ISO country code,
and possibly return additional info like language, county/state, etc
10

11. Modules for getting IP location
co-maint to release fixes and doc updates
11

12. Notes
• A number of these are 'commercial'
• Pay for full data set, or to get unthrottled access to backend service
• Performance & coverage tests took the most time
• In particular it's hard to test for correctness
• A number of good options
• Competition keeps them on their toes?
• Which one you use might depend on what other info you want for an IP
• Ended up not having the need, so not using any
• But have used them occasionally, now I know about them
12

13. Review #3
Spelling out a number in English
Modules for converting 327 to "three hundred and twenty-seven",
-3.25 to "minus three point two five", etc
13

14. Modules for spelling out numbers
14

15. Notes on review #3
• Fewer modules meant more time to 'play'
• I had fun writing a script to automatically compare results between
modules for 1 .. N
• First time I hit a module that just doesn't work
• Lingua::EN::Numbers
• The first module where I remember looking at the code and thinking
"hey, nice"
• I took over maintenance from Sean Burke to release fixes
• And have a todo-list, adding ideas that came up during the review
15

16. My rules of engagement
• Treat authors and their code with respect
• This might be someone's first tentative steps at code sharing
• Report all bugs I find
• If I can see how to fix them, then submit a fix
• Fix and improve documentation
• Especially if I had to look at the code to work out how to use it
• If appropriate, try and get co-maint to release fixes
• Because CPAN is forever
• This can be to just mark a module as deprecated & update SEE ALSO
• If there are four or more modules for something
• It goes onto my backlog of potential reviews
16

17. Observations based on the rules
• What encourages me to submit fixes?
• Github
• Makefile.PL (or more accurately: not dist.ini)
• Getting co-maint is relatively costly
• People disagreed when I said this at LPW 2011 ("You're doing it wrong")
• I still think this (getting co-maint to fix doc is relatively costly)
• I'm toying with writing a tool to manage this
• I might submit bugs or send email on 10 different modules in a review
17

18. Review #4
Parsing User-Agent strings
Modules for extracting and inferring information from HTTP User-Agent
strings.
Eg to tell how many of your site's users are using IE vs Chrome vs Firefox
18

19. Modules for parsing User-Agent
19

20. Review structure: module section
20

21. Comparison
Coverage (test corpus)
Speed
Review-driven development!
21

22. Curation
• Requirements for these modules
• Continuing good coverage of (versions of) browsers, bots, libraries, etc
• Good enough performance (processing large log files)
• Identifying other things (is it a mobile? what's the screen size?)
• Working with Olaf Alders on "complete solution"
• Currently at the R&D stage 
22

23. Review #5
Defining constants
Modules for defining constants and immutable variables
23

24. Modules for defining constants
24

25. Some firsts with this review
• I was prompted to look at the Perl source
• To make sure I understood SvREADONLY
• Emailed an author for help understanding his code
• Enough modules to prompt me to map them out
25

26. Observations
• I had been using Readonly
• But I clearly hadn't read all the documentation before
• Now I use Const::Fast
• Latency: all those modules using Readonly …
• Surprising variety of designs / approaches
• I learned a lot looking at the code for all these modules
• But quite a few experiments
• "I wonder if you can …"
26

27. Review #6
Getting module dependency info
Finding out what modules (and dists) are used by a module or modules
27

28. Tried a new approach
• Previously I only published after a lot of work
• But I still missed plenty of modules
• Decided to experiment with a more lean approach
• Found 6 modules, none did what I wanted, so wrote my own
• Wrote a quick sketch review and published it
• More modules came out of the woodwork every time I updated
• I was looking at dependencies for my next review
• But ended up pushing this one out sooner
28

29. Modules for getting dependencies
29

30. Module Author
• I had a "bug" reported – author is listed as:
• But META.json says:
• End result was PAUSE::Permissions
30

31. Notes
• The 23rd module nearly matched mine!
• Very much still in progress
• Curation needed:
• There are 5 or so distinct module types (out of 26 so far)
• But lots of different variations on each theme
31

32. Why so many modules?
• Whipupitude
• Low barrier to entry for CPAN
• Hard to find (all) appropriate modules
• Easier to upload than contribute or take over
• Playing with Perl is fun (eg constants, dependencies)
32

33. Review #7
Making HTTP requests
Modules for making HTTP requests (GET, POST, PUT, DELETE, etc)
Excluding higher-level modules like WWW::Mechanize
33

34. Modules for making HTTP requests
The only module I've given up on 34

35. Performance on GET requests
35

36. Runtime dependencies
Require a C library
Requires a C library and Moose!
36

37. What module to use?
37

38. Review #8
LUHN check
Modules for checking credit card numbers
38

39. Modules for checking CC numbers
39

40. Performance of LUHN modules
Moose?
40

41. Review #9
Getting a module's path
Modules for finding where a module has been installed locally,
preferably without having to load the module
41

42. Modules for getting a module's path
42

43. Comparison
Nearly three orders of magnitude
43

44. Feature matrix
44

45. Features vs Dependencies
45

46. Appropriate # of dependencies
46

47. Observations
And some mild venting
47

48. What do people use CPAN for?
• Sharing code with others!
• Configuration management
• A cloud-based store for configuration (Task::BeLike::*)
• A transport mechanism
• A todo list (egBusiness::BankCard)
• A blog / dumping ground for ideas
• None of the above (ACME::*)
48

49. Version numbers
• Version numbers should be designed for humans
• PAUSE just wants them to be monotonically increasing
• All modules in a dist should have 0.01
• A version number 0.004
• The same version number 1.121660
0.0.20
• Don't be afraid of 1.x 0.6.1
5
• 73% of dists on CPAN are version 0.* 0.000
0.001005
• our $VERSION = '0.01'; v0.0.3
2.4
6632
2011121001
49

50. How can you tell a good module?
• Version number
• But see Net::HTTP::Tiny v0.01
• Recently updated
• Some modules are 'finished'
• Volume and/or quality of documentation
• See Crypt::RandPasswd
• Number of bugs
• Some big hitters have a long list though…
• Number of dependent modules
• from different authors
• Ratings / +1's
• Perl::Critic
• Author
• All of the above, and more
51

51. How have I changed as a result?
• I've learned new things, & new ways to do things, & …
• I do developer releases & pay a lot more attention to CPAN Testers
• I'm more paranoid
• I benchmark and profile
• I use IRC, and have had only positive experiences
• How I export functions (thank-you Zefram)
• How I think about dependencies
• Moose / Mouse / Moo
• I'm a lot more circumspect about releasing code to CPAN
• Beginner's mind
52

52. My review process
• Quickly skim and find initial set of modules
• Start writing sections for each module
• Write generic SYNOPSIS style example, and evolve it as I go along
• Report bugs and documentation issues as I go
• First sketch at benchmarking & comparing
• More thorough search for modules to include
• Email the authors, give them a chance to comment
• Publish first version
• Iterate as people point out modules that I've missed
54

53. It's easy – why not give it a go?
% mkdir json
% cd json
% cat > module-list.txt
JSON
JSON::DWIW
JSON::JOM
JSON::Parse
JSON::Syck
JSON::Tiny
JSON::Util
JSON::XS
JSON::YAJL
JSON::Streaming::Reader
JSON::Streaming::Writer
Mojo::JSON
Pegex::JSON
Test::JSON
^D
% cpan-review
55

54. What do I want you to do?
• Stop and search CPAN before writing code
• Ok, I accept it's hard to find (all of the) relevant modules for a task
• Just because you wrote it, you don't have to upload it
• Only release if it's needed for another module, or another user
• Put it on github
• Write an article about it on blogs.perl.org
• Add links to similar modules in SEE ALSO
• MetaCPAN will include those in search results
• It's helpful for users
• You'll make my life easier
• Write a helpful ABSTRACT in your NAME section
• "a perl interface to foolib"
• "the hardest working two letters in Perl"
56

55. Review-related todo list
• Related to existing reviews
• Fix Crypt::RandPasswd
• Configurability for Lingua::EN::Numbers
• User-Agent module with Olaf
• New constant module: Exporter::Constants
• Dependencies: +5 modules to review, my module, curation
• HTTP requests: Mojo::UserAgent, async request modules
• Changes to Business::CCCheck
• New reviews in progress
• Hash objects
• JSON
• Module loading
57

56. Finding modules is hard
• Thinking about trying automatic clustering
• Help identify groups of modules for review
• Possible contribution to MetaCPAN
• Automatic tagging of modules
• Use keywords field in metadata
58