Node.js: scalability tips - Azure Dev Community Vijayawada

NODE.JS SCALABILITY TIPS
NODE.JS SCALABILITY TIPS
Luciano Mammino ( )
@loige
loige.link/node-adc
Vijayawada - 2021-12-13
1

@loige
Get the slides! 👇
loige.link/node-adc
2

HELLO!
👋I'm Luciano ( 🍕🍝 )
3

HELLO!
Senior Architect @ fourTheorem (Dublin )
3

HELLO!
nodejsdp.link
📔Co-Author of Node.js Design Patterns 👉
3

HELLO!
nodejsdp.link
📔Co-Author of Node.js Design Patterns 👉
Let's connect!
(blog)
(twitter)
(twitch)
(github)
loige.co
@loige
loige
lmammino
3

WE ARE BUSINESS FOCUSED TECHNOLOGISTS THAT DELIVER.
| |
Accelerated Serverless AI as a Service Platform Modernisation
We are hiring: do you want to ?
work with us
loige 4

"A service is said to be scalable if when we
increase the resources in a system, it
results in increased performance in a
manner proportional to resources added"
— Werner Vogels
@loige
5

🛫
TIP 1.
ESTABLISH A BASELINE
@loige
6

/?data=ciao👋
@loige
Site: - Code:
qrgen.lmammino.repl.co replit.com/@lmammino/QRGen 7

import { createServer } from 'http'
import QRCode from 'qrcode'
createServer(function handler (req, res) {
const url = new URL(req.url, 'http://localhost:8080')
const data = url.searchParams.get('data')
if (!data) {
res.writeHead(200, {'Content-Type': 'text/html'})
return res.end(`<html>
<body>
<p>To use this app you need to set the <code>data</code> querystrin
<p>Try for example <a href="/?data=hello">/?data=hello</a></p>
</body>
</html>`)
}
res.writeHead(200, { 'Content-Type': 'image/png' })
QRCode.toFileStream(res, data, { width: 300 })
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
@loige
8

if (!data) {
<body>
</body>
</html>`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
@loige
8

if (!data) {
<body>
</body>
</html>`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
18
@loige
8

if (!data) {
<body>
</body>
</html>`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
@loige
8

if (!data) {
<body>
</body>
</html>`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
if (!data) {
<body>
</body>
</html>`)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
}).listen(8080)
18
@loige
8

if (!data) {
<body>
</body>
</html>`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
if (!data) {
<body>
</body>
</html>`)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
}).listen(8080)
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
@loige
8

if (!data) {
<body>
</body>
</html>`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
if (!data) {
<body>
</body>
</html>`)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
}).listen(8080)
18
1
2
3
4
5
6
if (!data) {
7
8
9
<body>
10
11
12
</body>
13
</html>`)
14
}
15
16
17
}).listen(8080)
18
if (!data) {
<body>
</body>
</html>`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
@loige
8

autocannon -c 200 --on-port / -- node server.js
WRK
node server.js&
wrk -t8 -c200 -d10s http://localhost:8080/
@loige
9

autocannon -c 200 --on-port /?data=ciao👋 -- node server.js
Running 10s test @ http://localhost:8080/?data=ciao👋
200 connections
┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤
│ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │
└─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘
┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │
└───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘
Req/Bytes counts sampled once per second.
995 requests in 10.08s, 1.68 MB read
@loige
10

⛅
TIP 1-BIS
ALSO, FIND OUT YOUR CEILING
@loige
11

createServer((req, res) => {
if (req.method === 'GET' && req.url === '/') {
res.writeHead(200, { 'Content-Type': 'text/plain' })
res.end('Hello Worldn')
} else {
res.statusCode = 404
res.end()
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
@loige
12

} else {
res.end()
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
1
2
3
4
5
6
} else {
7
8
res.end()
9
}
10
}).listen(8080)
11
@loige
12

autocannon -c 200 --on-port / -- node server.js
Running 10s test @ http://localhost:8080/
200 connections
┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤
│ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │
└─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │
└───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘
Req/Bytes counts sampled once per second.
333k requests in 10.1s, 51.9 MB read
@loige
13

🍾
TIP 2.
FIND YOUR BOTTLENECK
@loige
14

Clinic.js
clinicjs.org
@loige
15

clinic doctor --autocannon [ -c 200 '/?data=ciao👋' ] -- node server.js
@loige
16

clinic flame --autocannon [ -c 200 '/?data=ciao👋' ] -- node server.js
@loige
17

clinic bubble --autocannon [ -c 200 '/?data=ciao👋' ] -- node server.js
@loige
18

🎳
TIP 3.
UNDERSTAND YOUR GOALS
@loige
19

WHAT DO WE OPTIMIZE FOR?
THROUGHPUT?
MEMORY?
LATENCY?
@loige
20

👁
TIP 4.
ALWAYS "OBSERVE"
@loige
21

I MEAN, IN PRODUCTION!
Logs - Metrics - Traces
@loige
22

🚀
TIP 5.
SCALE YOUR ARCHITECTURE
@loige
23

PERFORMANCE != SCALABILITY
@loige
24

HOW CAN WE SCALE A SYSTEM
BY ADDING RESOURCES?
@loige
25

THE " "
SCALE CUBE
x-axis
cloning
z-axis
partitioning
y-axis
functional
decomposition
@loige
26

27
Reverse proxy
CLONING
Inside the same server
Load Balancer
Using multiple server
@loige

THE MODULE
CLUSTER
Master process
Worker
process
Worker
process
Worker
process
28
@loige

import cluster from 'cluster'
import { cpus } from 'os'
const numCPUs = cpus().length
if (cluster.isMaster) {
// Fork workers
for (let i = 0; i < numCPUs; i++) {
cluster.fork()
}
} else {
// Worker code here...
}
1
2
3
4
5
6
7
8
9
10
11
12
13
@loige
29

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
3-4x req/sec
(8 core)
@loige
29

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
3-4x req/sec
(8 core)
@loige
29

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
} else {
}
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
11
12
13
3-4x req/sec
(8 core)
@loige
29

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
} else {
}
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
11
12
13
// Fork workers
cluster.fork()
}
1
2
3
4
5
6
7
8
9
10
} else {
11
12
}
13
3-4x req/sec
(8 core)
@loige
29

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
} else {
}
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
11
12
13
// Fork workers
cluster.fork()
}
1
2
3
4
5
6
7
8
9
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
3-4x req/sec
(8 core)
@loige
29

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
} else {
}
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
11
12
13
// Fork workers
cluster.fork()
}
1
2
3
4
5
6
7
8
9
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
3-4x req/sec
(8 core)
@loige
29

YOU COULD ALSO USE
CHECK OUT !
WORKER
THREADS
PISCINA
@loige
30

CLONING IS THE EASIEST STRATEGY TO SCALE A SERVICE...

... AS LONG AS YOUR APPLICATION IS "STATELESS"
@loige
31

API Gateway
FUNCTIONAL DECOMPOSITION
a.k.a. "Micro-services"
32
/products
/cart
cart DB
products DB
@loige

API Gateway
FUNCTIONAL DECOMPOSITION
a.k.a. "Micro-services"
33
/products
/cart
Functional decomposition can
also be combined with cloning!
cart DB
products DB
@loige

NODE.JS IS GREAT FOR MICROSERVICES
@loige
34

MICROSERVICES CAN ALSO HELP WITH
SCALING THE ORGANISATION!
@loige
35

MICROSERVICES ADD COMPLEXITY
Observability
Deployments
Versioning
Integration
@loige
36

PARTITIONING
Service and Data Partitioning along Customer Boundaries
Shard partitioning
/products/[A-L]/
/products/[M-Z]/
DB 2
37
DB 1
@loige

PARTITIONING IS GENERALLY USED
TO SCALE DATABASES
AND
SAAS SOFTWARE GEOGRAPHICALLY
@loige
38

SUMMARY
🛫Establish a baseline
@loige
39

SUMMARY
🍾Find your bottleneck
@loige
39

SUMMARY
🎳Understand your goals
@loige
39

SUMMARY
👁Always "observe"
@loige
39

SUMMARY
👁Always "observe"
🚀Scale your architecture
(cloning, decomposition & partitioning)
@loige
39

☝nodejsdp.link
THANK YOU!
Special thanks to , ,
, , ,
, , ,
, , , ,
, , , ,
, , , ,
,
Icons and SVGs by
Photo by on
@StefanoAbalsamo @matteocollina
@dagonzago @NullishCoalesce @DublinSvelte
@KViglucci @gjohnson391 @lucamaraschi
@laurekamalandua @giltayar @mrm8488 @adrirai
@haraﬁse @EugeneWare @Jauny @tlivings
@michaelcﬁne @leojino @shahidontech @Lordoomer
@zsadigov @dottorblaster
freepik.com
Jonas Kaiser Unsplash
@loige
loige.link/node-adc
40

Node.js: scalability tips - Azure Dev Community Vijayawada

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Node.js: scalability tips - Azure Dev Community Vijayawada

Ähnlich wie Node.js: scalability tips - Azure Dev Community Vijayawada (20)

Mehr von Luciano Mammino

Mehr von Luciano Mammino (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Node.js: scalability tips - Azure Dev Community Vijayawada