1.
Load Testing for Startups with
artillery.io
Hassy Veldstra <h@veldstra.org>
@hveldstra

2.
Story

3.
• Simulating a lot of users to see how your
application behaves
Load testing

4.
• Simulating a lot of users to see how your
application behaves
• Examples:
• Blog: a story on the front page of Reddit
• Dice.fm: Beyoncé’s gig announced
• Deliveroo: 6-9pm every day
• ASOS: Black Friday
Load testing

5.
Goals
• Analyze performance
• How fast is it / Is it fast enough?
• Soak testing – finding memory leaks and other resource leaks

6.
Goals
• Analyze performance
• How fast is it / Is it fast enough?
• Soak testing – finding memory leaks and other resource leaks
• Analyze reliability
• Does it degrade gracefully under stress?
• Monitoring and alerting working as intended?

7.
Goals
• Analyze performance
• How fast is it / Is it fast enough?
• Soak testing – finding memory leaks and other resource leaks
• Analyze reliability
• Does it degrade gracefully under stress?
• Monitoring and alerting working as intended?
• Capacity testing
• Can we handle the projected load with acceptable performance with the
architecture / hardware we have?

8.
Goals
• Analyze performance
• How fast is it / Is it fast enough?
• Soak testing – finding memory leaks and other resource leaks
• Analyze reliability
• Does it degrade gracefully under stress?
• Monitoring and alerting working as intended?
• Capacity testing
• Can we handle the projected load with acceptable performance with the
architecture / hardware we have?
• Analyze scalability
• How does the system scale if we add more servers / upgrade the
hardware? Is it linear? Where are the bottlenecks?

9.
1. Best case latency
2. 99th percentile latency under load
3. Throughput under load
Performance Metrics

10.
Backend Performance

11.
Backend Performance
1. Network performance
• How long it takes for HTML/CSS/JS to download

12.
Backend Performance
1. Network performance
• How long it takes for HTML/CSS/JS to download
2. Front-end performance
• How long it takes for it to render and display
• How it’s rendered and displayed (UI)

13.
Backend Performance
1. Network performance
• How long it takes for HTML/CSS/JS to download
2. Front-end performance
• How long it takes for it to render and display
• How it’s rendered and displayed (UI)
3. Backend performance
• How long it takes for the server to process a
request

14.
• npm install –g artillery
• artilery run my_test_script.json
How Artillery Works

15.
• npm install –g artillery
• artilery run my_test_script.json
• A test script is:
1. One or more scenarios
2. One or more load phases
How Artillery Works

16.
• ASOS:
• Load the homepage
• Run a search with some keyword
• Repeat three times:
• Pick a result at random and load the details for that product
• Add the product to basket
• Go to checkout
Scenarios

17.
1. Create 10 virtual users every second for 3 minutes
(warm up)
2. Ramp up to 50 virtual users a second over another 3
minutes
3. Run at 50 virtual users a second for 25 minutes
Load Phases

18.
2 hour plan

19.
2 hour plan
1. Get an APM tool (e.g. New Relic)

20.
2 hour plan
1. Get an APM tool (e.g. New Relic)
2. Install an error/crash-reporting (e.g. GetSentry)

21.
2 hour plan
1. Get an APM tool (e.g. New Relic)
2. Install an error/crash-reporting (e.g. GetSentry)
3. Write a simple Artillery script (just one happy-path
from the app)

22.
2 hour plan
1. Get an APM tool (e.g. New Relic)
2. Install an error/crash-reporting (e.g. GetSentry)
3. Write a simple Artillery script (just one happy-path
from the app)
4. Run that against dev/testing environment at a low
load

23.
2 hour plan
1. Get an APM tool (e.g. New Relic)
2. Install an error/crash-reporting (e.g. GetSentry)
3. Write a simple Artillery script (just one happy-path
from the app)
4. Run that against dev/testing environment at a low
load
5. Success!

24.
2 hour plan
1. Get an APM tool (e.g. New Relic)
2. Install an error/crash-reporting (e.g. GetSentry)
3. Write a simple Artillery script (just one happy-path
from the app)
4. Run that against dev/testing environment at a low
load
5. Success!
6. Ramp up gradually until the traffic is at the level you
need it to be

25.
2 hour plan
1. Any problems? Great. Now you know and can fix
them.
2. No problems? Great. Now you know.

26.
2 hour plan
1. Bonus: add a job to your CI to run the test every day

27.
Q&A