My keynote talk at Samsung Bangalore's Quality Day on 29 Jul 2016. In this talk, I consider what is the context of quality in future where infrastructure, data and learning will be integral components, and will enable an organization to respond to threats or opportunities on a real-time basis.

1. Real-Time Quality
(RTQ)
Tathagat Varma
http://thoughtleadership.in

2. The complexity is increasing…

3. Devices continue to grow…

4. And so does the cloud…

5. …and will only get “10X’ed”!!!

6. Welcome to the future…!

7. Big questions…
What is the meaning of “quality” in this
world?
How does the scale and pace impact
quality?
What are the directions ahead?

8. From Traditional to Agile…
Known
Requirements
Stable
Technology
Static Business
Models
Dynamic
Requirements
Ever-changing
Technology
Evolving
Business
Models
Old World New World
Quality = conformance Quality = capabilities

9. The big shift in Quality…
1960s: Waterfall Model
1970s: Software Engineering
1980s: Spiral Model, RAD, SSAD, IID
1990s: Big Process Movement (CMM, ISO, 6 Sigma), V-model
2000s: Agile, XP, Scrum, Lean, Kanban, CI, TDD, ATDD, BDD
2010s: CD, Web Performance, Lean Startups, Business Model, Design
Thinking, Customer Development, A/B Testing
Now: App Performance, Data Quality, Prediction Quality, DevOps,
DevSecOps, NoOps…
Near future: ML/AI Quality, IOT, …

10. Business is The New Quality!
Software
(test the
code)
Product
(test against
specs)
Service
(test for
SLAs)
Business
(just get the
$$$)

11. Infrastructure Quality

12. Multiple Daily Builds: 3X up!
https://blog.newrelic.com/2016/02/04/data-culture-survey-results-faster-deployment/

13. High performers are killing it!

14. …and the metrics are showing!

15. Apollo - Amazon’s Deployment Engine
Autonomous teams own their service, and
directly interact with customers
Apollo was created to reliably deploy code
at Amazon scale
(2014) In past 12months, Apollo was used
for 50M deployments. That one per
second!

16. Data Quality
Complete, Valid, Consistent, Timely, Accurate
(2002) Cost of poor data quality (US): $600B
(2009) Average company loses $8m due to poor
data quality, as per Gartner
(2014) Average company loses 12% revenue due
to bad data
The challenge is to assure high data quality…

17. Learning Quality
An estimated 30% of Amazon’s revenue
comes from recommendation engines!
Worldwide average is 12%
60% of clicks on YouTube homepage due to
recommended videos!
With big data, how do you ensure high
learning and prediction quality…in real
time?

18. Examples…
Is this picture offensive or not?
Is this a spam mail or not?
Is “Big Ben” a monument, a movie or a book?
Whom to target for the next fashion product?
Should you offer discount to this shopper?
Is this the right recommendation for you?
Remember TAY :)
…

19. So, what’s Real Time Quality?
The ability of an org to respond in real-
time to a crisis or an opportunity
by offering high-quality product,
feature, fix, service, recommendation,
etc…
using high-quality data and machine
learning and prediction models

20. Real-Time Quality
Infra Data
Learn

21. Recap
IT Deployment is getting (near) real time, and
increasingly available off-the-shelf!
Business decisions are getting (near) real time, and
expect IT to support it.
The notion of quality needs to support this scenario with
increased capabilities
Quality will increasingly mean quality of data, quality
of learning, quality of recommendation, quality of
decisions, etc.…all made in real time!
This is Real Time Quality!

22. References
Google runs on 5000 times more code than the original space shuttle, https://www.weforum.org/agenda/
2016/07/google-runs-on-5000-times-more-code-than-the-original-space-shuttle
Codebases - Millions of lines of code, http://www.informationisbeautiful.net/visualizations/million-
lines-of-code/
The Internet of Things Explained, http://www.i-scoop.eu/internet-of-things/
Deploying at GitHub, https://github.com/blog/1241-deploying-at-github
https://www.quora.com/How-do-big-companies-like-Facebook-Google-manage-software-releases-
without-causing-system-outages-and-poor-user-experience
http://www.allthingsdistributed.com/2014/11/apollo-amazon-deployment-engine.html
https://www.quora.com/What-is-Amazon-Apollo
DevOps 2016 Report
https://astheqaworldturns.wordpress.com/2011/03/03/the-evolution-of-software-qa/
http://www.hugeinc.com/ideas/perspective/evolving-qa
http://www.testingexcellence.com/evolution-software-quality-assurance-test-engineering/
https://econsultancy.com/blog/64612-the-cost-of-bad-data-stats/