My presentation is a case study about implementing System testing using AGILE methodologies in a corporate environment, together with its challenges and lessons learned; most of the time , this sounds like Mission impossible , with all the movie versions, but without Tom Cruise. What we did: We took piece by piece the products included in the customer solution and aligned them into a delivery train. The catch was in convincing the team to support and “buy into” this way of working and to bring visibility in each step of the process (train station). Creating the train, revealed a lot of process misunderstandings , different tools which were not compatible and that were bringing overhead to teams backlog. We did fail several times in: finding the right sponsors, showing essential relevant things to the teams or products owners implementing testing techniques using Agile synergizing a multinational working team ( France-Bulgaria-Romania) Due to these, at some point we were on the verge of closing the Delivery train or at least to stop investing in it. The ice breaker happened when we had 2 scenarios implemented that took automatically the latest release and found a blocker that passed product borders. Today, the system has a sexy dashboard, easy to be understood by everyone, it is integrated in Jenkins & Jira and has runs at every product release with 8 big scenarios put in place. That’s what we call success story, after all the struggle.

1. Delivery train
System testing in AGILE

2. Agenda
• A.D (anno Domini)
• The Magic Solution
• The plan
• Implementation
• Failures
• What is today’s status

3. • Context: The customer wanted a solution
composed by Axway products.
• Problem: Multiple products not made from start
to work together, there was no testing between
them, not 1:1, not 1-n or n-n-n
• Consequences:
• Buggy solution,
• No overview on customer needs,
• Axway was loosing credibility
A.D (anno Domini)

4. Main Pain Points
❏Some of the pain points discovered when we
needed to integrate the products into a solution
were:
❏Each and every team had a different roadmap
❏Distributed teams: France, Romania, Bulgaria
❏Each team had their own structure ( Product
owner, program manager, developers, QA)
❏Every single team was with their own
understanding of Agile
❏Each team used their own tools, or in some
cases, using the same tools but differently

5. The Magic Solution

6. The solution - broken into pieces
• Managing continuously the impact of
any change/alteration
• Clear chain of validation and avoid
duplication of effort
• Solution validation
• Manage risk

7. The plan – main issues
Mentality people which proved resistance to change or to try new things
Infrastructure no available servers to support the installation & testing of the solution
Hiring (sourcing people) No team had any members to dispose for this particular project.
Deadlines There are many deliverables and short terms to meet, no time for experiments.
Fear of failure

8. Main Issues

9. Mentality

10. The plan – solutions
Mentality Bugs vs features
Infrastructure New servers from the market
Hiring (sourcing people) Did publicity within the teams worked?
Deadlines The disappearances of deadlines
Fear of failure Replace fair with persevere

11. Milestone 1
• Download from artifactory every Monday a new
build of each product and complete the version into
an excel - https://axway.jiveon.com/docs/DOC-
50622
• Install it manually
• Run manually the subset of tests
• If the tests passed we moved to 1:1 manual testing
• If the tests passed we moved to 1-n manual testing
get the train moving starting for one
use case.

12. Milestone 2
• Use Jenkins to extract from Artifactory
the latest product version, every Monday
• Extract from repository the subset of
tests for each product & run it
• If the tests passed, a new Jenkins job was
triggered for 1:1 tests
• If the tests passed a new Jenkins job was
triggered for 1-n tests
• If any step failed, the process was
stopped and investigated
automate the moving train
& increase content

13. Milestone 3
• Use Jenkins to extract from Artifactory
the latest product version, every night
• Develop more use cases to cover critical
features
• Link Jenkins to Jira, so if any job is failing,
a ticket is raised
• Improve the technology used: migrate
from SVN to GIT and from VMs to Dockergrow content &
update technologies

14. Today’s
status

15. Today’s
status

16. Instead of conclusions
We needed to get support from
managers, we wanted especially
product owners to sponsor us in
implementing the solution. Surprisingly,
the R&D was the most reluctant to the
idea.
We needed to gain/internalize quick
expertise on : Docker, Jenkins, TestNG ,
Selenium, JavaScript, ReactJs ,
Bootstrap.
It was a long and hard road, but now we
are starting a new journey where we are
in the process of adding three more
products on a similar chain.
One of the greatest accomplishments is
that by implementing a full automated
testing process: at product level, 1-1
integration, 1-n integration – the
number of incidents decreased by 60%.