DevOps Deep Dive Roma 2017 Alla fiera dell'ovvio - Aggiungiamo i test di performance e scalabilità nella nostra pipeline

1. Alla fiera dell’ovvio
Aggiungiamo i test di performance e scalabilità nella nostra pipeline
Giulio Vian
http://blog.casavian.eu/
giulio@nolbej.com
@giulio_vian
http://www.nolbej.com/

2. About me
2

3. Di che si chiacchiera?
Perché ovvio?
Fondamenta de Perf / load/ stress testing
Integrare nella CI
Chiusura
3

4. Inquadramento
della Sessione
Livello 200
Le demo sono alla fine
4

5. Tocca a voi
Testing
Continuous Integration
Developers
Testers
Architects
Infra Engineers / IT Ops
Managers
5

6. I. Perché ovvio?
6

7. Il test di
performance di
quest’anno…
2M users
40,000 RPS
2Gbps
7
© 2016 IMG Universe, LLC.
All Rights Reserved

8. Performance is a
Feature
«there are two kinds of
websites: the quick and
the dead»
https://blog.codinghorror.com/performance-is-a-feature/

9. Response time
0.1 second is about the limit for having the user
feel that the system is reacting instantaneously,
meaning that no special feedback is necessary
except to display the result.
1.0 second is about the limit for the user's flow of
thought to stay uninterrupted, even though the
user will notice the delay. Normally, no special
feedback is necessary during delays of more than
0.1 but less than 1.0 second, but the user does lose
the feeling of operating directly on the data.
10 seconds is about the limit for keeping the user's
attention focused on the dialogue. For longer
delays, users will want to perform other tasks
while waiting for the computer to finish, so they
should be given feedback indicating when the
computer expects to be done. Feedback during
the delay is especially important if the response
time is likely to be highly variable, since users will
then not know what to expect.
Miller, R. B. (1968).
Response time in man-
computer
conversational
transactions. Proc. AFIPS
Fall Joint Computer
Conference Vol. 33, 267-
277.

10. Hard facts
Google
Half a second delay
caused a 20% drop in
traffic.
Amazon.com
Even very small delays
(100 ms) would result in
substantial and costly
drops in revenue.

11. In che fase diventa critico?
Gartner Hype Cycle, Source: Wikipedia

12. Perché l’olio?
Non grippare

13. II. Fondamenta de
Perf / load/ stress testing
13

14. Obiettivi (tecnici)
Benchmarking
Far emergere difetti
codice, configurazione, architettura
Capacity planning
Punto di rottura
Security leakage

15. Problemi emergenti
Esaurimento delle connessioni
File locks
Database locks
(Thread) deadlocks
Memoria esaurita
Effetto domino
Mancanza di scalabilità

16. Esaurimento
pool connessioni
Connessioni verso
database
Connessioni verso servizi
esterni
Librerie che gestiscono
la connessione
Di norma un problema
nel codice

17. File locks
Log
Eseguibili
File dati

18. Database locks
Table
Index

19. Deadlock
Threads
Ma anche no

20. Memoria
esaurita
Senza dimenticare il
thrashing in caso di
memoria virtuale

21. Effetto domino

22. Indicatori
Latenza
Throughput
Carico
Tassi di errore
Errori sul tool

23. Risultante
+Tassi di errore
+Errori sul tool
Carico relativo
Latenza
Required threshold
Max N seconds
100%60%
Throughput
Utilizzo

24. Cosa significa?
Carico relativo
Latenza
Required threshold
Max N seconds
100%60%
Throughput
Utilizzo

25. Elementi di analisi
Curve di risposta
Punti di flessione
Colli di bottiglia

26. Scenari tipici
Desktop / mobile / libreria Server / web
Database
server
Web serverClient
N1
N4
A1
A3 A2N3
N2

27. Cosa monitorare
CPU
RAM
Disk I/O
Network I/O
Threads / processes

28. III. Integrare nella CI
28

29. Fattore tempo
Test di breve durata
Test di lunga durata
Copertura dei test
Frequenza di esecuzione
Valore dei test

30. Strumenti: Script & Runners
JMeter
Visual Studio (.webtest / .loadtest)
Selenium
×Unit
Gatling
Custom* & More…

31. Strumenti: Load
VSTS Cloud-based load testing
CA BlazeMeter
SOASTA (Akamai) CloudTest
HPE LoadRunner
Custom*
& More…

32. Strumenti:
Monitoring
Loading tool
Basic
Performance Counters*
Application logs
Cloud
& More…

33. Web Demo
34

34. Considerazioni su
CI/CD
Pipeline lineare o grafo?
Risorse
Indicatori

35. Gremlins
Riscaldare
l’infrastruttura
Cloud
Configurazioni di default
nginx worker processes
CDN

36. IV. Conclusioni
37

37. Risorse
Fatica di Sisifo
Decisione di
investimento
38
(Photo: Elya)

38. Bibliografia & Riferimenti
http://www.slideshare.net/giuliov/presentations
AWS Well-Architected Framework - Performance
Efficiency Pillar
https://www.amazon.com/dp/B01MSSLHBX
Performance Testing Guidance for Web Applications
https://msdn.microsoft.com/en-
us/library/bb924375.aspx
http://www.brendangregg.com/linuxperf.html
39

39. Bibliografia (2)
Writing High-Performance .NET Code —
Ben Watson (Ben Watson)
https://www.amazon.it/Writing-High-
Performance-NET-Code-
Watson/dp/0990583430/
Time Is Money: The Business Value of
Web Performance — Tammy Everts
(O'Reilly Media)
https://www.amazon.com/Time-Money-
Business-Value-Performance/dp/1491928743
40

40. Bibliografia (3)
Software Performance and Scalability: A
Quantitative Approach — Henry H. Liu
(Wiley)
https://www.amazon.com/Software-
Performance-Scalability-Quantitative-
Approach/dp/0470462531
Continuous Delivery with Windows and .NET
— Matthew Skelton and Chris O'Dell
(O'Reilly)
http://www.oreilly.com/webops-
perf/free/continuous-delivery-with-windows-and-
net.csp
41

41. Bibliografia (4)
Continuous Delivery: Reliable Software
Releases through Build, Test, and
Deployment Automation — J.Humble,
D.Farley (Addison-Wesley)
https://www.amazon.com/Continuous-
Delivery/dp/0321601912/
The Phoenix Project — G.Kim, K.Behr,
G.Spafford (IT Revolution Press)
https://www.amazon.com/Phoenix-Project-
DevOps-Helping-Business/dp/0988262509/
42

42. Bibliografia (5)
The DevOps Handbook — G.Kim,
P.Debois, J.Willis, J.Humble (IT
Revolution Press)
https://www.amazon.com/DevOps-
Handbook-World-Class-Reliability-
Organizations/dp/1942788002/
Continuous Delivery with Visual Studio
ALM 2015 — M.Olausson, J.Ehn (Apress)
http://www.amazon.it/Continuous-Delivery-
Visual-Studio-2015/dp/1484212738/
43

43. Call to action
44
(Photo: Francesco Canu)

44. Contatti
giulio@nolbej.com
@giulio_vian
http://blog.casavian.eu/
@NolbeyLtd
http://www.nolbej.com/

45. Fine delle trasmissioni
46