Pre-Con Lab: Test Automation Across Nine Open Source Tools—Taurus

World®
’16
Test Automation across 9 Open
Source Tools: Taurus
Andrey Pokhilko - Chief Scientist, BlazeMeter - CA Technologies
DO5X52L
DEVOPS

2 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
© 2016 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.
The content provided in this CA World 2016 presentation is intended for informational purposes only and does not form any type of
warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.
For Informational Purposes Only
Terms of this Presentation

Abstract
In the not so distant past, load testing tools had one clear purpose: to generate load. All the tools created at
that time answered this requirement perfectly. The rationale has evolved as we discovered, fine-tuned, and
extended aspects of the process. Now, the test preparation stage is so complex and crucial that we spend
more time preparing tests than running the load. We want to see live test results alongside the hardware
monitoring data. Finally, the post-test phase requires checking SLA conditions, sending automated
notifications, and storing results for future analytics. The 'good old' tools were not designed well for such a
broad use case. But it isn't just the scope of work that has changed. Even the habits of the people building the
services have also changed. They primarily use two types of tools: command-line tools and Web 2.0 services.
Think how often you encounter Linux, Git and Ansible, Wget and cURL. Or how many of these tabs - Gmail and
AWS, GitHub and Splunk, Salesforce and Hubspot, NewRelic and Jenkins - you currently have open? The
"cherry on top" is the "continuous" aspect of modern development processes. Teams want to automate all
possible parts of the flow, and, in such cases, the testing tool should naturally fit into the "pipeline chain" of
the CI machine. We're left with two possible solutions: modernizing old tools or creating new tools from
scratch. Neither solution is perfect, both require spending a lot of resources on redoing something that has
already been done. In this session, Andrey Pokhilko will explore a third approach - an open source hybrid tool
called "Taurus". This tool meets the broad use cases mentioned above, while enabling you to utilize 'tried and
tested' tools for generating load. To suit the user's habits, Taurus is a simple-to-learn command-line tool that
also connects to Web 2.0 services. This session will have a hand-on section that will provide you with an
opportunity to create, configure and execute Selenium scripts for performance tests in the Cloud using Taurus.
Andrey
Pokhilko
CA Technologies
Chief Scientist,
BlazeMeter

http://gettaurus.org

Agenda
WHY LEARN TAURUS?
TAURUS OVERVIEW
9 TOOLS SUPPORTED
Q & A
1
2
3
4

Why Learn Taurus?
§ DevOps-style usage of load testing tools (CLI + SaaS)
§ Freedom to choose tool that fits use-case
§ Enable “shift left” for load testing
§ Enable automation and CI for load testing

Taurus is:
§ Text-based configuration file format
§ Executed by the command-line tool
§ That wraps existing load generating tools
§ And connects to SaaS for reporting and scale

YAML Syntax Intro
§ Whitespace matters <= main source of mistakes
§ Dictionaries, lists and scalars
§ No need to quote strings unless special chars used
§ YAML-aware editor helps (Notepad++)
§ If YAML does not work for you, you can use JSON with Taurus

Taurus Config Structure
§ Scenarios - collection of scripts
§ Execution - which scenarios execute and how
§ Reporting - how to analyze data
§ Services - extra actions on different stages
§ Modules - change settings for all above
§ General settings - some top-level options

Good Defaults
§ You can omit almost every option
§ Almost everything has meaningful defaults

Writing First Config
run> bzt first.yml
---
execution:
- scenario:
requests:
- http://blazedemo.com/

Understanding what happens inside
1. Load configs: base defaults, personal preferences, config itself
2. Generate JMX from YAML scenario
3. Modify JMX to integrate with Taurus
4. Run JMeter as parallel process
5. Analyze results and overwatch JMeter
6. Report stats into command line at the end

Applying Some Pressure
---
execution:
- scenario: blazedemo
concurrency: 10
ramp-up: 1m
hold-for: 2m
scenarios:
blazedemo:
requests:
- http://blazedemo.com/

Learning Quick Overrides
run> bzt first.yml ⇒
-o execution.0.concurrency=20
-o modules.console.disable=true

Using Online Reports
run> bzt first.yml -report

Important Files and Directories
§ Artifacts location, configurable, shown twice a run:
– merged.yml, effective.yml, logs
... ... ...
04:03:26 INFO: Artifacts dir: /tmp/bzt-artifacts/1107-035917
... ... ...

Scaling With Cloud Provider
run> bzt first.yml –cloud
You will need the API key in ~/.bzt-rc for this
---
execution:
concurrency: 50
hold-for: 5m
locations:
us-west-1: 2
eu-west-1: 3

Scenario Building: DSL in YAML
§ Scenario building is a lot specific for each tool
§ JMeter support is the most rich:
– extractors,
– assertions,
– logic,
– config elements
– JSR223
– thread actions

Scenario Building: DSL in YAML
---
scenarios:
from-script:
requests:
- url: http://blazedemo.com
assert:
- Welcome

Scenario Building: Source Files
---
scenarios:
from-script:
script: my-predefined.jmx
modifications:
disable:
- Constant Timer

9 Tools Supported
§ Some are cross-platform: JMeter, Gatling, Selenium, Grinder,
Locust, ab
§ Some are Linux/Unix-only: Siege, Pbench, Tsung
§ Selenium is very special: Java, Python, JavaScript, Ruby
supported
§ You can write your own and plug it into Taurus

Switching Tools
Available: JMeter, Gatling, Selenium, Grinder, Locust, ab, Siege, Pbench, Tsung
---
execution:
hold-for: 1m
executor: gatling

Taurus
• Open Source initiative sponsored by CA BlazeMeter
• Automation-friendly framework for Continuous Testing
• Define scenarios in Taurus YAML or execute existing tests
• Wide variety of test executors, including JMeter, Selenium, Gatling, …
• Ability to generate load locally or in the cloud
• Consistent reporting through Taurus UI or BlazeMeter
• Easy integration with Continuous Delivery systems
Summary
scenarios:
home-page:
requests:
- label: Home
url: /
method: GET
think-time: 500ms
assert:
- contains:
- Welcome
subject: body
regexp: false

Free Play Time!

Questions?

Recommended Sessions
SESSION # TITLE DATE/TIME
DO5T03P Financial Industry Panel – Theater 5 11/16/2016 at 04:30 pm
DO5X39S
GM's Removing Testing Constraints with Respect to Test
Modeling and TDM – Lagoon L
11/16/2016 at 04:30 pm
DO5X55S Walmart Brasil—Seasonal Peak Traffic Readiness 11/16/2016 at 04:30 pm

Must See Demos
Modernize
App Delivery
Integrated CD
Theater 5 - DOV501P
Deliver Test
Data Faster
Test Data Manager
Theater 5 - DOV511P
Deliver
Better Apps
Service Virtualization
Theater 5 - DOV507P
Orchestrate
Your Release
Release Automation
Theater 5 - DOV513P

Thank you.
Stay connected at communities.ca.com

Pre-Con Lab: Test Automation Across Nine Open Source Tools—Taurus

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Pre-Con Lab: Test Automation Across Nine Open Source Tools—Taurus

Ähnlich wie Pre-Con Lab: Test Automation Across Nine Open Source Tools—Taurus (20)

Mehr von CA Technologies

Mehr von CA Technologies (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Pre-Con Lab: Test Automation Across Nine Open Source Tools—Taurus