In-house web automation?
•Download as KEY, PDF•
1 like•861 views
My brain dump experience on building CI systems.
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Similar to In-house web automation?
Similar to In-house web automation? (20)
Recently uploaded
Recently uploaded (20)
In-house web automation?
- 1. In-house web automation? Open Source Bridge 2010 Adam Christian
- 4. Brain Dump
- 5. Talk Format
- 6. Question
- 7. Tools
- 9. Infrastructure • Boxes
- 11. Infrastructure • Boxes • VM’s • The Cloud
- 13. Continuous Integration • Hudson
- 14. Continuous Integration • Hudson • BuildBot
- 15. Continuous Integration • Hudson • BuildBot • Cruise Control
- 16. Continuous Integration • Hudson • BuildBot • Cruise Control • Home brew, etc
- 17. Frameworks!
- 18. Frameworks! • Windmill
- 19. Frameworks! • Windmill • JS Tests!
- 20. Frameworks! • Windmill • JS Tests! • WatiR
- 21. Frameworks! • Windmill • JS Tests! • WatiR • WatiN (Iron)
- 22. Frameworks! • Windmill • JS Tests! • WatiR • WatiN (Iron) • Selenium
- 23. Frameworks! • Windmill • JS Tests! • WatiR • WatiN (Iron) • Selenium • Grid
- 24. Frameworks! • Windmill • JS Tests! • Twill • WatiR • WatiN (Iron) • Selenium • Grid
- 25. Frameworks! • Windmill • JS Tests! • Twill • WatiR • FunkLoad • WatiN (Iron) • Selenium • Grid
- 26. Frameworks! • Windmill • JS Tests! • Twill • WatiR • FunkLoad • WatiN (Iron) • HTML Unit • Selenium • Grid
- 27. Frameworks! • Windmill • JS Tests! • Twill • WatiR • FunkLoad • WatiN (Iron) • HTML Unit • Selenium • Grid • Web Driver Story
- 28. Flash?
- 29. Flash? • FlexPilot (my story)
- 30. Flash? • FlexPilot (my story) • FlexMonkey
- 31. Flash? • FlexPilot (my story) • FlexMonkey • Flash-Selenium
- 32. Flash? • FlexPilot (my story) • FlexMonkey • Flash-Selenium • Selenium Flex
- 33. Flash? • FlexPilot (my story) • FlexMonkey • Flash-Selenium • Selenium Flex • Commercial (QTP)
- 35. DEMO
- 36. Workload
- 37. Maintenance
- 38. Maintenance • Software Updates
- 39. Maintenance • Software Updates • WPKG - Windows
- 40. Maintenance • Software Updates • WPKG - Windows • Purity (Dirty Machines)
- 41. Maintenance • Software Updates • WPKG - Windows • Purity (Dirty Machines) • Image Management
- 42. Expansion
- 43. Expansion • Hardware budget
- 44. Expansion • Hardware budget • Define your OS/Version matrix
- 45. Expansion • Hardware budget • Define your OS/Version matrix • Cloud advantages
- 46. Languages
- 48. Languages • JavaScript! • Platform of the web
- 49. Languages • JavaScript! • Platform of the web • Integration
- 50. Languages • JavaScript! • Platform of the web • Integration • DB, Email, XDomain Services
- 51. Languages • JavaScript! • Platform of the web • Integration • DB, Email, XDomain Services • Focus on productivity
- 52. Coverage
- 53. Coverage • User base
- 54. Coverage • User base • Browser matrix for speed
- 55. Coverage • User base • Browser matrix for speed • Smoke fringes
- 57. DEMO
- 58. Methodology
- 59. Tests
- 60. Tests • Functional
- 62. Tests •Functional • Unit • Separate efforts
- 63. Reporting
- 64. Reporting • True / False
- 65. Reporting • True / False • JUnit / XUnit
- 66. Reporting •True / False • JUnit / XUnit • Notifications!
- 67. Reporting •True / False • JUnit / XUnit • Notifications! • Email, SMS, Tweets,Yams
- 68. Automation
- 69. Automation • Take the leap
- 70. Automation •Take the leap • QA
- 71. Automation •Take the leap • QA • Development
- 72. Automation •Take the leap • QA • Development • Block Releases
- 73. Money Path
- 74. Money Path • Start with what counts
- 75. Money Path • Start with what counts • Robust and reliable
- 77. Alternatives
- 78. Pay Someone :)
- 79. Pay Someone :) • Sauce Labs
- 80. Pay Someone :) •Sauce Labs • SOASTA
- 81. Pay Someone :) •Sauce Labs • SOASTA • BrowserMob
- 82. DEMO
- 83. Manually Test
- 84. Conclusion
- 85. More Info • http://adamchristian.com • http://saucelabs.com • #windmill, #selenium, #watir, #saucelabs on FreeNode
Editor's Notes
- Introduce self, what it takes to build and maintain a web automation system. My story!
- I work at Sauce Labs, we are trying to make your life less painful.
- This is my cat, he’s a nice guy, and yeah - I am a real person.
- How many people are doing any sort of web automation? Of you how many have to deal with flash?
- Awesome red wrench. When designing your automation system, be very careful about the tools you choose.
- General Boxes - Great starting place, don’t scale that well VM’s - Expensive hardware and licensing, no recurring costs ++ Cloud - hard to test internal apps without tunnels, recurring costs, slow
- You need it, evaluate, pick one and go from there.
- I’m not going to go into the “best one”, they all have strengths and weaknesses and you need to decide for yourself which fits your needs. If you can get away with not having a JS enabled browser, run your tests without one.
- Demo FlexPilot in Sel IDE
- Eventually you will need to expand your setup Upgrading VM servers gets expensive You should decide what configurations you really care about, because the matrix gets immense The cloud stays pay per machine per minute
- JavaScript is a great platform for testing the web However it has some shortfalls when it comes to integration with various services You need to focus on what makes your QA/Dev teams the most productive
- What are your users using? That plays a huge role in your technology choices If you can test just safari, IE, chrome and FF on windows, you should focus on that Smoke test macosx and linux as they are probably a tiny portion of your userbase
- Hudson running a Selenium test
- There are a few schools of thought behind how testing should be done, lets explore those
- Functional test your user experience Unit test your backend code, integrate as much as you need and run those tests often I would advise you to separate those efforts functional testing and unit testing are important for different reasons, and different points in your process
- True/False can tell you if 100% of your functional tests are passing (which is important), but its also incredibly important to know exactly which aren't passing Know all the time the status of your system, and be dedicated to it.
- Automation is at a point where it has proven itself All elements of your company should be involved Take it seriously, if your testing suites aren’t stabilized, releases shouldn't go out
- Start with what matters, your money path Improve that small set of tests to the point where failures really mean something serious
- Shamless
- Running a test in the Cloud
- This does not scale, do I really have to talk about this?
- I have told you all the different kinds of solutions that you can chose and how to tailor each of them to your specific needs, please go make it happen.