devConf LK 2019 Letterkenny, 23 February 2019 http://bit.ly/devConfLK2019 How do compare Visual Studio Web & Load Test with JMeter? Can I replace one with the other? How hard is this open-source tool? Do I need to install and/or learn Java? We will answer these questions and more with a practical introduction, exploring: - Basics of JMeter - Recording - Collecting and analyzing results - Tokens and parametrization - Scenarios and distributions - Setting up a test rig

1. www.lk-mug.org #devConfLK2019#LKMUG
Letterkenny,Ireland
How’s relevant JMeter to me?
Giulio Vian – DevOps Lead – Glass, Lewis & Co.

2. www.lk-mug.org #devConfLK2019#LKMUG
Letterkenny,Ireland
devConf LK 2019
Saturday, Feb 23rd
2019
9:00 AM – 5:00 PM LYIT, Port Road,
Letterkenny
LK
MUG
http://bit.ly/devConf
LK2019
Event Partners

3. JMeter who?
JMeter!!!
What happened to VS?
I once hear this load testing thing

4. Agenda
Load testing basics
JMeter Authoring
Distributed Load testing
Compare with Visual Studio & CLT

5. Load testing basics

6. Performance is
a Feature
«there are two
kinds of websites:
the quick and the
dead»
https://blog.codinghorror.com/performance-is-a-feature/

7. Response time
0.1 second is about the limit for having the user
feel that the system is reacting instantaneously,
meaning that no special feedback is necessary
except to display the result.
1.0 second is about the limit for the user's flow of
thought to stay uninterrupted, even though the
user will notice the delay. Normally, no special
feedback is necessary during delays of more than
0.1 but less than 1.0 second, but the user does lose
the feeling of operating directly on the data.
10 seconds is about the limit for keeping the user's
attention focused on the dialogue. For longer
delays, users will want to perform other tasks
while waiting for the computer to finish, so they
should be given feedback indicating when the
computer expects to be done. Feedback during
the delay is especially important if the response
time is likely to be highly variable, since users will
then not know what to expect.
Miller, R. B. (1968).
Response time in man-
computer
conversational
transactions. Proc. AFIPS
Fall Joint Computer
Conference Vol. 33, 267-
277.

8. Queueing and Resource depletion
Client
SUT
Dep1
Dep2Client

9. Test Rig
Testing implies monitoring
Client
SUT
Dep1
Dep2Client
Monitor

10. Healthy
behaviour
+Error rates
+Tools errors
Relative Load
Latency
Required threshold
Max N seconds
100%60%
Throughput
Usage

11. JMeter Authoring

12. What is JMeter?
Buy one get three
Script editor Visual Studio Web Test
Runner Visual Studio Load Test
Distributed runner Azure Cloud load testing
Does not manage the test rig for you
Free & widely used
Apache JMeter may be used to test performance both on
static and dynamic resources, Web dynamic applications.
It can be used to simulate a heavy load on a server, group of
servers, network or object to test its strength or to analyze
overall performance under different load types.

13. JMeter basics
https://jmeter.apache.org/
JDK 8 / 9
I prefer XCopy deploy
GUI, -n, server
yeah, docker too

14. Recording

15. Collecting and analysing results

16. No Metrics

17. Tokens and parametrization

18. Scenarios and distributions

19. Demo
JMeter Templates
Recording
Elements of a script
XML script (just like VS)

20. In summary
Record using browser
Rinse and refactor
Throw garbage away
Parametrize and extract data
External data feeds
esp. Users

21. Distributed Load Testing

22. Setting up a test rig (1/2)
Create the VMs
Set environment variables for Java and JMeter; e.g. using a script like jcmd:
Title Java Environment
set JAVA_HOME=%~dp0jdk
set PATH=%PATH%;%JAVA_HOME%bin;%~dp0bin
cmd /k cls
Client (controller) jmeter.properties
remote_hosts=TEST-RIG1,TEST-RIG2,TEST-RIG3
server_port=4000
client.rmi.localport=4001
server.rmi.port=4000
server.rmi.localport=4001
server.rmi.ssl.disable=true
The last one is necessary to work around this bug
https://bz.apache.org/bugzilla/show_bug.cgi?id=62463

23. Setting up a test rig (2/2)
The Server (rig) jmeter.properties should have
server_port=4000
client.rmi.localport=4001
server.rmi.port=4000
server.rmi.localport=4001
server.rmi.ssl.disable=true
Open firewall ports
Invoke-Command -ComputerName "TEST-RIG0","TEST-RIG1","TEST-RIG2","TEST-RIG3" -
ScriptBlock {
New-NetFirewallRule -DisplayName 'JMeter Server' -LocalPort "4000-4010" -
Profile Any -Direction Inbound -Protocol 'TCP' -Action 'Allow' -Enabled 'True'
New-NetFirewallRule -DisplayName 'JMeter Server' -LocalPort "4000-4010" -
Profile Any -Direction Outbound -Protocol 'TCP' -Action 'Allow' -Enabled
'True’
}

24. Run test using rig
Server (rig)
jmeter-server -Djava.rmi.server.hostname=%COMPUTERNAME%
Use Task Scheduler
https://github.com/zloeber/Powershell/blob/master/Supplemental/Add-
UserToLoginAsBatch.ps1
Client (controller)
jmeter -n -t …Test.jmx -l …Test.log -r
Tune Java memory

25. Integrate in a pipeline
Cloud-based Apache JMeter Load Test task
Convert JMeter XML to JUnit

26. In summary
DIY
Choose between
SSL and firewall
Allocate resources
Get from a service

27. Visual Studio

28. Why JMeter is interesting to a Visual Studio developer?
Visual Studio 2019 will be the last version with load test
features.
https://docs.microsoft.com/en-us/visualstudio/releases/2019/release-notes-
preview#test-tools
https://devblogs.Microsoft.com/devops/cloud-based-load-testing-eol

29. Comparison
Visual Studio + CLT
CA BlazeMeter for an automated cloud rig
Gatling (also Java-based)

30. In summary
Tools are similar
Process always the same
Create / Edit
Run
Analyse
Repeat

31. Conclusion

32. Bio in pictures
32
giulio.dev@casavian.eu
@giulio_vian
https://www.slideshare.net/giuliov
https://tfsaggregator.github.io
http://blog.casavian.eu/
Hardware spec:
1KB RAM (upg. 16KB)
4KB ROM
First computer Companies Communities

33. Donegal

34. Bibliography & References
https://jmeter.apache.org/
http://www.slideshare.net/giuliov/presentations
http://www.testautomationguru.com/jmeter/
https://docs.microsoft.com/en-us/vsts/build-release/tasks/test/cloud-
based-load-test
Performance Testing Guidance for Web Applications
https://msdn.microsoft.com/en-us/library/bb924375.aspx
AWS Well-Architected Framework - Performance Efficiency Pillar
https://www.amazon.com/dp/B01MSSLHBX
http://www.brendangregg.com/linuxperf.html
https://github.com/giuliov/Stating-the-obvious/
34

35. Bibliography (2)
Writing High-Performance .NET Code —
Ben Watson (Ben Watson)
https://www.amazon.it/Writing-High-
Performance-NET-Code-
Watson/dp/0990583430/
Time Is Money: The Business Value of
Web Performance — Tammy Everts
(O'Reilly Media)
https://www.amazon.com/Time-Money-
Business-Value-Performance/dp/1491928743
35

36. Bibliography (3)
Software Performance and Scalability: A
Quantitative Approach — Henry H. Liu
(Wiley)
https://www.amazon.com/Software-
Performance-Scalability-Quantitative-
Approach/dp/0470462531
Continuous Delivery with Windows and .NET
— Matthew Skelton and Chris O'Dell
(O'Reilly)
http://www.oreilly.com/webops-
perf/free/continuous-delivery-with-windows-and-
net.csp
36

37. Bibliography (4)
Continuous Delivery: Reliable Software
Releases through Build, Test, and
Deployment Automation — J.Humble,
D.Farley (Addison-Wesley)
https://www.amazon.com/Continuous-
Delivery/dp/0321601912/
The DevOps Handbook — G.Kim, P.Debois,
J.Willis, J.Humble (IT Revolution Press)
https://www.amazon.com/DevOps-Handbook-
World-Class-Reliability-
Organizations/dp/1942788002/
37

38. The End