The document discusses how to deliver software fast while maintaining high quality through modern QA practices. It describes setting up a DevTest team that combines agile methodologies, automated testing, and testing software-as-a-service products on tight 1.5 month release cycles. The DevTest team builds reliable automation frameworks, performs continuous testing on every code push, and has the goal of deploying fixes within one day through a continuous deployment pipeline.

1. Deliver Fast, On Time & With High Quality
Description
Karim Fanadka QA Manager HPE Software

2. About me: Karim Fanadka
Job: QA Manager at HPE Software, 7+
Product: StormRunner Load
Favorite activates: Traveling, Ski, Blogging
Ski – France 2013
Road Trip – Iceland 2015

3. We are going to
talk about:
Agile
Modern QA
SaaS
Automation
Environments
Hotfixes Continuous Deployment

4. SaaS – Software as a Service

5. Before SaaS – On Premise
Software
Users

6. SaaS – Software as a Service
Data Base
Servers
File System
Firewall
Users SaaS

7. journal note
I have been asked to build the first
DevTest team that combines:
• Modern technologies and techniques
• Agile Methodologies
• SaaS product testing
• Deliver fast

8. StormRunner Load - SaaS product
Which product my team is testing :
Performance testing solution that makes it easy to plan, run, and
scale web and mobile testing over the cloud.

10. 10
1 Create and run a
load test using
your favorite
scripting tool
2 Load Generators spin
up in AWS EC2 or
MS Azure
3 Results and app
insights are
available in real
time
The product my team is testing: StormRunner Load
New
Soon
Performance testing solution that makes it easy to plan, run, and
scale web and mobile testing over the cloud.

12. StormRunner Load as a Service
Data Base
Servers
File System
Firewall
Users SaaS Cloud

13. Agile – Do It Fast

14. Before Agile - Delivery Timeline
1 or 2 Major releases to production in 1 year

15. Agile StormRunner Load Delivery Timeline
March April AugustJune Nov Dec
Whats the CodeName of the next release?
If you know the answer, I will invite you to a
AlleyCat BubbleBobble Elevator Action Frogger Golden Axe Hang-On
FebruaryDec
CommanderKeen Digger
6 Major releases to production in 1 year
Year: 2015

16. Agile – The Challenges
• Test new features
• Test Regression
• Automation
• Environments
• Continues Integration
/ Continues Delivery
• Hotfixes
Do it in 1.5
months
• Data Migration (When update SaaS with
the newer version)
• Performance Testing
• Security Testing
• Globalization
• Documentation
• And more …

17. journal note
As the first DevTest engineer at
HPE I have been asked to build the
first DevTest Team to handle:
• SaaS product testing
• Deliver fast

18. Modern QA Team (DevTest)

19. What is DevTest?
Using top end QA/DEV practices combined with DevOps techniques
QA & DEV Working very close, on the same IDE, code
base, and in united process
Automation is a MUST
Manual is for new features / exception flows that are
difficult to automate
NO MORE “Deliveries for QA”
YES for “Push to Production”

20. Training
“we are testing,
and breaking code
constantly but we
help rebuild it
better…….”

21. Continues Testing
Push new code
Release branch (1.5 months)
Test new code
Short Regression
Push to production
Automation
Developer Engineer
Test Engineer DevTest Team
Builds in 1 Release (1.5 month): 1,400
• Full: 236
• Commit: 1,164

22. Continuous Testing
Push new code
Automation – Full package
Builds in 1 Release (1.5 month): 1,400
Automation – Commit package
Feature branch
Release branch
Full:
236
Commit: 1,164
Developer Engineer

23. Modern QA
DevTest
DevOpsSaaS
Dev
Product
Customer

24. Automation

25. journal note
I was aware that automation must find
real defects and not just false
alarm, DevTest team has a new
challenge, and it’s to build:
• Reliable frameworks
• Fast results (not more than 7 min)

28. Frameworks:
Frameworks Tests
Client Testing Framework
API Testing Framework
Cloud Framework
Results

29. Example for Automation challenge
17 Regions in the Cloud 6 different images for each region
15 different metrics
102 Tests
(~1500 metrics)

30. Automation – Cloud Framework - Report

31. Grunt
Protractor
Test Results
Web
Driver
Angular
Selenium API’s
UI Controls
(Storm)
Test
DB Layer
Protractor
Wrapper
Jasmine
Style
JS
Automation Library
Executer
Client automation framework

32. Grunt
Mocha Test Framework
Tests
DB
Wrapper
API
Wrapper
Automation Library
Strom Runner APIs
Reports
Tests
Results
Backend automation framework

33. API discovery tool

34. Cost cloud tool

35. 0
30
80
100
120
150
170
200
0 0
75
120
147
220
390
600
0 0 0 0
100
200
270 270
0
100
200
300
400
500
600
700
1.1 - AlleyCat 1.2 - Bubble 1.3 - CK 1.4 - Digger 1.5 - Elevator 1.6 - Frogger 1.7 - Golden
Axe
1.8 - Hang-On
Automation Active Tests - Trend
Client (UI) Backend Cloud
85% Coverage
100% Coverage
60% Coverage

36. Next Generation: Application Lifecycle Management

37. Test Environments

38. Test Environments
Local
Lab – Simulate SaaS Env
• Each tester havehis own localmachinefor fast testing
• Lab Env = 7 Servers
• We have6 Lab Env:
• QA1
• QA2
• Security
• Performance
• Integration
Staging
• SaaS responsiblefor staging:
• Maintenance
• Deployment
• Sanity

39. Test Environments Farm
QA Security Performance IntegrationLocal (Linux)
Staging

40. Continuous Deployment

42. Continuous Deployment
Push new code
Automation – Full package
Builds in 1 Release (1.5 month): 1,400
Automation – Commit package
Feature branch
Release branch
Full:
236
Commit: 1,164
Developer Engineer

43. Continuous Deployment
Test Engineer
Developer Engineer
Hey Tester, How do you manage
to handle this amount of builds
and deployments on daily basis
I’m a Modern QA

44. Continuous Delivery
Lab
Nexus
Tel
Aviv
PIT
Jenkins
QA2
Security
PCoE
QA2 (Portal)
PIT
PIT
PIT

45. Hotfixes in 1 day

46. Hotfixes Lifecycle
07:00 AM
Production bug
08:00 AM
Research by QA and Dev
10:00 AM
Fixing bug
16:00 AM
Testing (Staging)
14:00 AM
Testing (Local)
18:00 Push to production

47. Articles
• 10 best practices for QA teams to deliver quality software, fast
• Blurring the lines: How the development tester role changes on agile teams
• 3 steps to building test databases for the real world

48. Tip
“Changing our mindset takes
precedence over technology.
When we work together, we will
succeed.”
Thank you

49. @kim0z85
Question?
https://www.facebook.com/karim.f.as
https://il.linkedin.com/in/kareemfakareem9k@gmail.com
Karim Fanadka QA Manager HPE Software