1) Quality assurance is changing in the digital world as services become digitalized and connected through the Internet of Things. This brings new challenges around security, usability, and performance. 2) To keep up with faster software development, companies must balance speed and quality. This can be done through practices like DevOps, agile methodologies, increased test automation, and ensuring team members have versatile competencies. 3) Prototyping and testing early helps build quality into products from the start and allows for faster feedback to catch issues relating to security, usability, and performance.

1. Quality Challenges of the Digitalized
World – How Does Quality Assurance
Change?
Kari Kakkonen, Director, Quality and Competences, Knowit Oy, Finland
WCSQ, Lima, Peru 21.3.2017

2. Kari Kakkonen
ROLES
• Knowit Oy, Director/Quality and Competences, Lead Consultant, Trainer and Coach
• Treasurer of ISTQB Executive Committee
• Chairman of Finnish Software Testing Board (FiSTB)
• Chairman of research project STX, Lappeenranta University of Technology.
ACHIEVEMENTS
• Influencing testing since 1996
• Ranked in 100 most influential IT persons in Finland (Tietoviikko magazine)
• Number of presentations in local and global seminars
• TestausOSY/FAST founding member.
EDUCATION
• ISTQB Expert Level Test Management Full & Advanced Full & Agile Tester certified
• DASA DevOps Fundamentals, Scrum Master and SAFe certified
• SPICE provisionary assessor certified
• M.Sc.(Eng), Helsinki University of Technology (present Aalto University), Otaniemi, Espoo
• Marketing studies, University of Wisconsin-Madison, the USA.
PAGE 2 [CONFIDENTIAL] QUALITY ASSURANCE PROPOSAL
BUSINESS DOMAINS
• Wide spread of business domain knowledge
• Embedded, Industry, Public,
• Training, Telecom, Commerce,
• Insurance, Banking, Pension
SERVICES
• ISTQB Advanced, Foundation and Agile Testing +
Knowit Quality Professional
• DASA DevOps Fundamentals
• Quality & Test process and organization
development, Metrics
• Agile testing, Scrum, Kanban, Lean
• Leadership
• Test automation, mobile, cloud, DevOps
• Quality, Cost, Benefits.
Twitter: @kkakkonen
LinkedIn: fi.linkedin.com/in/karikakkonen/

4. Revolution of User Experience

5. Mobile is the new reality – research tells us
520.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0
www.shopify.com/blog
“Mobile Now Accounts for 50.3% of All Ecommerce Traffic”

6. Change of Business
1. Services are digitalized
2. Services processes are
digitalized
3. Internet of Things

7. 7
Central
Quality Challenges
Security
Usability
Performance
Speed of software development

8. 8
Security Testing
Everything in focus

9. Security Testing
920.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0
Preventive
Security
• Expert in the team
Application level
security
• Every level must be
secure
Interfaces
• Internet
• Machine-2-Machine
Testing
• Vulnerability scans
• Audits
Up-to-date
• OWASP etc.

10. 10
Usability testing
Think about the UX

11. 20.3.2017 © Copyright Knowit Oy 2013 | Confidential | Version 1.0 11
Surface
Skeleton
Structure
Scope
Strategy
Applied from source:
The Elements of
User Experience
By Jesse James Garrett
• Visual design
• Micro-interactions
• Service structure
• Page structure
• Content structure
• Content structure
• Information architecture
• Functionality
• Content needs
• Business needs
• User needs
Bottom-Up
Abstract
Concret
e

12. Usability testing can be…
Usability
assessments
Comparing
service to User
Experience
Levels
Assessing
usability
problems
criticality
Reporting
usability
problems
Suggesting
solutions
1220.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

13. User device coverage –
check usability with test automation
• Different devices, same functionality, same user experience
• Responsiveness
• One test library, many test objects
• Little to maintain
• Increasing number of objects
• Start early
• Repeat daily / by build
• Tests differ just a little
• Reporting
• Also other interfaces than UI
1320.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0
Apps to be tested (Web)
Test automation from cloud or devices

14. Performance testing
Test and Measure, don’t get surprised

16. Performance testing
Testing tools
• Commercial
• Open source
Continuous
monitoring
• Common goal with
performance testing
Identify
bottlenecks
• Fix
• Tune
Test coverage
• Devices
• Cloud
• Scenarios
Repeatability
• Include in
regression tests
1620.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

17. Speed of software development
Balance between speed and quality

18. Fast software development process
DevOps
• As a culture
• Technically
Agility
• In organization
• In projects
Test automation
• Supporting the Team
• Speeding the Process
• Expand the Coverage
• Help from AI
Exploratory Testing
• Defect finding
effectiveness
• Efficiency
Competences
• Versatile: testing,
development,
operations, business
1820.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

19. DevOps principles
• DevOps is a method which emphasizes
communication, cooperation and integration
between software development teams (Dev) and IT-
support / operations teams (Ops)
• Method requires agile software development
process
• It connects operations organization to software
development by using virtualized environments
• Production environments are part of software
development and as agile as the software
development
1920.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0
Software
Development
Quality
Assurance
Operations /
Production
DevOps

20. Development services
Reporting
End user
monitoring
Virtualised backend services
Test data management
Version
control
Continuous
Integration (CI)
Business
Code Commit
TestingDevelopment
Automated tests, build & install
CI build
CI install
Deliver
Integration UAT
Performance
Access controlQuality metrics Virtualisation
Deploy
203/20/2017 KNOWIT CONFIDENTIAL
Production
Deliver
Deploy
DevOps and Continuous Deployment

21. Agile goes forward, what fits together?
2120.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0
Scrum
Kanban
Waterfall, V-
Model
Scaled Agile
Framework SAFe
Lean
Lean Six Sigma
DevOps
Continuous
Deployment
Hybrid
XP
Nexus
LESS
Continuous
Delivery

22. Knowit Research 2014:
Agile test planning on company level
Strategic level
No plans Plan in high level Plan in detail
• Almost a quarter doesn’t plan agile
testing at all (is that agile anymore…)
• Mainly testing is planned in high level
(this is maybe the definition of agile
testing…)
• 85 % of company level testing is
documented (to some extent)

23. Company Level
• Not just
• Scrum
• Kanban
• …
• Organization level
• UVF
• M3.0
• SAFe
• Nexus
• Spotify Way
• Capo
• Agility Path
• Lean
• …
2320.3.2017 © Copyright Knowit Oy 2013 | Confidential | Version 1.0

24. SAFe 3.0
24

25. Cloud (MS, Amazon, IBM...)
Example of test automation and continuous integration
20-03-2017 © Copyright Knowit Oy 2015 | Confidential
Robot Framework
Jira +Quality Center
Jmeter
Python libraries
Selenium
Jenkins
GIT
Quick Test Pro
SoapUI
Sharepoint
Confluence

26. Agile Testing Quadrants
26© Copyright Knowit Oy 2015 | Confidential
Source: Lisa
Crispin and
Janet Gregory

27. Use Exploratory testing
Source: Sinkkonen, Elisabeth Hendrickson, Cem Kaner, James Bach, Lisa Crispin
“In exploratory testing you learn from previous tests, and put that
learning into your next tests”
Plan
• Test charter
Test
session
• Notes
• Bugs
Debriefing
• Dashboard
Testing
Opinion-forming
Reporting
Designing
actions
Observations

28. Learn versatilely about business, development, testing,
operations
Courses, certificates Coaching
Pair work Workshops
Increase
competences
2820.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

29. Prototype and test

30. Quality assurance in
Digitalized World
Build Quality, then also Test
• Security
• Usability
• Performance
Get faster feedback
• Team velocity
• Support of testing
• Speed to Operations

31. Questions?
kari.kakkonen@knowit.fi
@kkakkonen

32. 3/20/2017 Knowit 32
Creating digital
opportunities