Learn how BISAM, the leading software editor into Performance, Attribution & Composites Analytics, decided to refocus on quality after more than 10 years of disciplined Agile practices. A fascinating Lean IT story presented by Renaud Wilsiud, CTO of BISAM at the Lean IT Summit 2017. Discover more Lean IT REX on www.lean-it-summit.com

1. HOW LEAN HELPED US PUT QUALITY BACK AT
THE HEART OF OUR AGILE PROCESS?
Renaud Wilsius
bisam.com

2. 2
I have been a Quant, then a developer for 10 year. Strong believer in Agile and
now R&D director at BISAM
BISAM IS LEADER SOFTWARE EDITOR
INTO PERFORMANCE, ATTRIBUTION
& COMPOSITES ANALYTICS

3. WHO ARE WE?
3
BISAM has 10+ years of disciplined Agile practices
- eXtreme Programming in 2005
- Scrum in 2010
- Lean in 2014

4. AGILE MANIFESTO REMINDER
4
We are uncovering better ways of developping software by
doing it and helping others do it.
Through this work we have come to value :
Individuals and Interactions over processes and tools
Working Software over comprehensive documentation
Customer Collaboration over contract negociation
Responding to change over following a plan

5. SCRUM REMINDER
5

6. BUT, WHAT ABOUT QUALITY?
6

7. WELL ….
7
Client :
• I will not encourage any client to take the 3.9
release.
• They don’t believe us anymore.
Installations
• A lot of packaging problems on upgrades and
patches. We have not tested the right
combinations.
Support
• Our tickets went into the big black hole.
5.7/10

• Customers say we have a very good
product. Very comprehensive.
• Customers like the upgrade in 20 minutes.
• They really like the new functionality.

With a dangerous trend on Issues

8. APPROACH
8

9. TWO OPTIONS
9
Empower Outsource

10. LEAN
10
Lean approach
• Specialized consulting firm
• Put our client in the centre
• Focus on the product
• Bring departments together
Define visual performance
Indicators
Deliver one good version first
Obeya

11. WHAT HIDES BEHIND “QUALITY”?
11

12. ACTIONS
12

13. OUR TYPICAL LEAN JOURNEY
13
We had a crisis.
We made the problem very specific.
We demanded immediate results (3 Months).

14. OUR TYPICAL LEAN JOURNEY
14
Stream Mapping

15. OUR TYPICAL LEAN JOURNEY
15
Stand Up
Meetings
and Visual
Management

16. OUR TYPICAL LEAN JOURNEY
16
Quality Wall

17. OUR TYPICAL LEAN JOURNEY
17
Red Bins

18. OUR TYPICAL LEAN JOURNEY
18
PDCA

19. OUR TYPICAL LEAN JOURNEY
19
PDCA
Again

20. OUR TYPICAL LEAN JOURNEY
20
Delivery
Platform

21. ZERO INSTALL AND UPGRADE ISSUES
21
Test as a client
• Take client’s database
• Take client’s O/S
• Follow Migration Guide steps
to the letter as a client
What
• 4 different clients upgraded
• 3 initial versions: 3.8.0-n,
3.8.5-f, 3.6.0-aa
• 2 Operating systems (2 Linux,
2 Windows)
Definition of a successful upgrade
• Key Processes run smoothly
• Acceptable upgrade time
• No issue browsing the application
Found
• No issue
• Upgrade from3.8.0-l to 3.9 on
client production data (2 billion
EQA results) took 15 min
APPROACHOUTCOME

22. ZERO REGRESSION ISSUES
22
Test as a client
• User Interface Tests
• Non Regression Tests
• Challenge and review our
own test pack
Challenge and review our own
test pack
• 15,316 unit tests & 1,874
functional tests
• Adapted functional tests to
run not only on HSQLDB but
also Oracle & SqlServer
• => Found one major issue
It is all about Data
• On client data
• On client site pre-test
User Interface Tests
• Opening and testing 1,050+ screens on 4
different client databases
=>Found and fixed 79 issues/
improvements with one critical KPI issues
Non Regression Tests
• Run 31 client specific test procedures on
client upgraded environment
• Take client major blocking and critical
issues that might have been impacted by
3.9.0 and retest them
• =>Found and fixed 2 issues
APPROACHOUTCOME

23. ZERO PERFORMANCE ISSUES
23
Performance Lab
• Isolate Lab from any other activities
• Invest 15,000 euros on SAN bay to
harbor Client Databases
Performance Campaign
• Run performance campaign
throughout the version lifecycle
• Check performance non-regression
again at the end of the version.
Web Services load testing
• Run Total Return Simulation
web service via SoapUI
• Iterations: 1,000 // 8
concurrent calls (threads)
Excel Formula load testing
• Open 6 large excel reports.
One of them containing
12,000+ formulas
APPROACHOUTCOME
Action Result
Performance Campaign Found and fixed 1 issue (BON-4003)
Web Service load testing On average 15% improvement
Excel load testing No Issue found

24. 3 MONTHS LATER
24
Efficient
Quality Wall

25. 3 MONTHS LATER
25
Daily
Obeya

26. 3 MONTHS LATER
26
Team
Performance

27. 3 MONTHS LATER
27
Success!

28. LESSON LEARNED
28
Keep doing
• Systematic packaging testing on
Client configuration (Major &
Minor Release)
• Extend our unit and functional
tests coverage
• Run Performance campaign
during development cycle
• Enhance Product, Migration &
Performance Documentation
Start doing
• Have Client satisfaction
measures/feedback before, during and
after upgrade
• Develop and formalize Beta Partnership
Program (BPP)
• Acquiring Client database and test plans
• Monitor Support tickets stock, Number
of Regression, Number of
successful/failed upgrade
• Run Client database migration during
development cycle
• Test feature from GUI on Client
upgraded version during development
cycle
• Foster cross-department
communication and collaboration

29. 3 YEARS LATER…
29

30. BETA PARTNERSHIP
30
Version after version, the number of Beta
Partner per version keeps growing.
3 weeks to releasing any general
availability version, we started enrolling
active Beta Partners who:
- Are heavily testing new features
- Ensuring non regression in a real client
environment
- Have direct access to R&D SME 0 0
1 1 1
2
3
0
1
2
3
4
3.7
Nov 13
3.8
Apr 14
3.9
Dec 14
3.10
May 15
3.11
Nov 15
3.12
May 16
16.2
Nov 16
# Beta Partner

31. RELENTLESSLY INCREASE OUR AUTOMATED TESTS
31
Over the last three years ,
we have exceeded 18,600
Unit tests in B-One v 16.2
(+21%), while increasing
the number of End to End
Tests by 74%.
0
500
1 000
1 500
2 000
2 500
3 000
3.7
Nov 13
3.8
Apr 14
3.9
Dec 14
3.10
May 15
3.11
Nov 15
3.12
May 16
16.2
Nov 16
# of Functional Tests per version
1374
2389

32. WHILE REDUCING OUR NUMBER OF OPEN ISSUES
32
Every year the R&D has KPI
objectives to reduce the
number of open issues
(Client / BISAM identified)
Last two years, the number
of open issues in B-One
dropped by 77%.
0
20
40
60
80
100
120
140
160
3.7
Nov 13
3.8
Apr 14
3.9
Dec 14
3.10
May 15
3.11
Nov 15
3.12
May 16
16.2
Nov 16
# Open Bugs
# Open Bugs
Reported by Clients
# Open Bugs
Reported by BISAM

33. RESULTING IN FASTER PROMOTION OF RELEASES
33
Client are going live with early B-One
version (-a, -c) .
We kept track of the first minor
release which went to production on
a given release, and measure how
many days passed between the
official release and this production
ready minor release. It takes today 3
times less than 3 years ago.
106
74
55
1
34 32
0
20
40
60
80
100
120
3.7
Nov 13
3.8
Apr 14
3.9
Dec 14
3.10
May 15
3.11
Nov 15
3.12
May 16
# Open days to receive go live version

34. CONCLUSION
34

35. CONCLUSION
35
Trust your people.
Empower your people.
Teach lean thinking and skills to everyone: “Fall seven times. Stand up eight.”
Support Lean practices at each level of the hierarchy.
Develop sustainable and sticky pattern for long term customer benefit.

36. bisam.com
THANK YOU
Follow us on