1. 1
Subline
HEADLINE
• whoami?
• How we use Selenium
• Selenium as a Service (Saucelabs / BrowserStack / …)
• Building your own Selenium GRID on AWS EC2
Selenium Meetup
What are we gonna talk about?
8. FROM LOCALHOST TO CLOUD
First steps - Localhost
8
• „chief cook and bottle washer“ server (eierlegende Wollmilchsau Server)
• Jenkins for Test Execution
• Tomcat (webapp) for collecting and presenting test results
• Firefox instances as many as Jenkins Executors
• Selenium Standalone
• -> Left-over Firefox Instances
• -> Full Hard Drives
• -> Other Services not being able to run
• -> Resource Conflicts
• Either everything works or nothing
9. 9
HEADLINE
Subline
FROM LOCALHOST TO CLOUD
First steps - Saucelabs
• Account at Saucelabs
• - Firewall Exclusions for Tests on Staging Systems
• wide ip range
• with changes every couple of months
• - Sometimes long response times -> failing tests
• + Throw-away Servers == no Re-Usage of VMs
• + Video Recordings of Sessions
• + No Stuck Browsers
• >1000$ / month
10. HEADLINE
Subline
10
FROM LOCALHOST TO CLOUD
First steps - BrowserStack
• Evaluation of BrowserStack
• - Hosting on AWS
• -> Wide AWS IP Range had to be whitelisted in Firewall for Stages
• -> Security / Privacy Issue
• - Re-Usage of VMs
• - A lot of False Positives
• - NO Video Recordings of Sessions
• - In the end more expensive than Saucelabs
18. HEADLINE
Subline
18
CloudWatch - Custom Metrics
FROM LOCALHOST TO CLOUD
Upscaling
• Pushing Pending Requests with cron job from HUB
-> increases ‚Desired‘ Size at AutoScalingGroup
Downscaling
• AWS EC2 pay at least 1h
• Nodes check from 50 to 60min if they are idle
if idle ->
a) detach from Grid
b) self destroy
c) decrease Desired Size at ASGroup
• if they are not idle in that timespan they got to live for another 50-60min
20. 20
LIMITATIONS OF AWS
Auto Scaling + AMIs
• Analytics showed that >50% of our users are on Windows
• -> right now 100% Windows Nodes
• unix nodes planed for DesiredCapability Platform=ANY
• Unix nodes are even easier, because they can be completely provisioned
• Challenges of Windows Nodes
-> No easy provisioning with Chef
• so far manually configured Windows AMI, that evolved over time
• (not yet time invested in provisioning naked Windows,
e.g. Windows Templates for Packer)
• AutoScalingGroups work with AMIs
• ASGroups are based on LaunchConfigurations
• in LC’s you have to configure an AMI to launch
• -> no downside of Windows Nodes for our setup
21. 21
LEARNINGS
Limitations of Webdriver and Selenium
• Keep requests to the DOM at a minimum
• e.g.: Don’t iterate through an XML file with WD requests
• No matter what, always stop the Browser at the end of your test
• Clean temporary files often
• Have a TimeOut in your code for interrupting page loads being stuck
22. FUTURE CHALLENGES
Testing Microservices
22
• WELTN24 is going to make a big switch from a monolith to micro services:
• Advantage: Teams can deploy independently and more often
• Requirement: Fully automated deployment
• Limitations of GUI-Tests (End2End Tests):
• The higher you get in the testing pyramid the more flaky your tests get
• -> Flaky tests don’t suit a fully automated deployment pipeline!?
• Challenge: Can GUI Tests tests be integrated in automated deployments?
• Proposal: Stable and smart Test Journeys through GUI for business critical paths
• combined with Contract Based Testing for ensuring functional behavior?
24. 24
THANK YOU
STAY IN TOUCH
• Slides will be on:
• http://de.slideshare.net/urbanharry
• STAY IN TOUCH
• @HarrysIdeAS
• „Harry Urban“ ->
• LinkedIn
• XING
Questions?