Yet Another grep for the CPAN ! 5 min presentation of grep.metacpan.org at TPC2017 Lightning talk at The Perl Conference 2017 in Washington D.C.

1. or Yet Another grep for the CPAN !
TPC 2017 Nicolas atoomic@cpan.org @atoomic
grep.metacpan.org
or ack.metacpan.org ?

2. The Original One
from David Leadbeater

3. Other alternatives ?
• ☞ CPAN::Visitor
• quick & dirty solution ?
download all the tarballs
extract them
use “grep -r” or ack ?
And YES this works !

4. Could we improve it ?

5. Yes we can !
> git init .
> git add .
> git commit -m init
> git tag root
> git grep My::Package
# git alias start

6. you said, git grep ?
-E, --extended-regexp, -G, --basic-regexp
Use POSIX extended/basic regexp for
patterns. Default is to use basic regexp.
-P, --perl-regexp
Use Perl-compatible regexp for patterns.
Requires libpcre to be compiled in.

7. Pushing git to the limits
1,042,384 files
( 1,008,366 for the extracted CPAN )
~17 Gb git repo
~1.5 Gb .git index

8. GitHub…

10. Build a frontend
very basic Dancer App
daily cron to update the git repo

11. Let’s use the website

14. Filter

15. How Fast ?
worst case: no match - timeout ~15 sec
easy search: pretty fast ~5 sec
Optimizations
• return when we have enough results (for first page)
• keep running the query in background
• queries are cached
• only do “grep -P” when required

16. Where is it ?
grep.metacpan.org

17. Known Bugs
• pretty young project
• GitHub tickets
PCRE search not working in production
Links on module name broken in code extract
patches welcome !

18. Toolchain Summit 2017 Lyon
Thanks to everyone and all event sponsors