Introducing PunkyBrowster, a programmatic web browser that uses a real web browser.

1. How My Comic Book
Obsession Birthed a New
Functional Testing Tool
Feihong Hsu
Testing in Python Birds of a Feather
March 12, 2011

2. I like comics

3. I like studying Chinese
我喜欢学中文
我喜歡學中文

4. These are two tastes that go
great together!

5. Especially when
you find a real
gem like this:

6. Problem: Chinese comics portals load
slowly and have poor usability
Solution: Write a web scraping tool to
batch download images, so I can
be a proper Apple fanboy and read
comics on my iPad

7. First approach: urllib2 + lxml.html
Verdict: Total fail, couldn't handle
JavaScript and cookies

8. Second approach: XULRunner
Verdict: Asynchronous logic was
painful, Python integration would not
have been trivial

9. Third approach: spynner
Verdict: Not bad! But... some sites
don't load. Why?

10. Fourth approach: Make spynner
dumber
Verdict: Frickin' awesome

11. Coincidence: At Leapfrog, we needed
a better way to test JavaScript
behavior on our sites
Result: Leapfrog subsidizes my
comic book addiction

12. Over time, we added some nice stuff
to our spynner fork
- Ability to ignore SSL errors
- Form manipulation methods
- Screen capture (consistently-sized
images)
- Other stuff that I can't remember

13. Question: Hey Feihong, where can I get
this sexy library?
Answer: Nowhere, I'm too busy
reading comics to open source it

14. Real answer: We're working on
open sourcing it, but we ran into
some blocks
(Cast sidelong glance at Terry)

15. However, we are NOT
soliciting suggestions for a
name. We have the
PERFECT name already.

16. PunkyBrowster

17. Some basic usage

18. from punky import Browster
browser = Browster(auto_load_images=True)
browser.create_webview(show=True)
browser.load('http://www.duckduckgo.com/')
browser.fill('input#hfih',
'How do I de-pube-ify waterless urinals?')
browser.submit('form#hfh', wait_load=True)
for element in browser.all('#r12 > div'):
print unicode(element.toPlainText())

19. Random note: I
never want to have
the need to see a
urologist. But if I
do, I hope he's
wearing a badge
like this:

20. I am done