1. How the Network Protocols You Choose
Ultimately Affect Your Applications
Andrey Pokhilko
Open Source Dev Leader @ Komodor

2. Communication Efficiency at World War 2

3. Communication Efficiency at World War 2
Average word length in language:
English ~ 4.7
German ~ 6.4
“We need fire support!”
vs
“Wir brauchen Feuerunterstützung!”
30% longer!

4. The effects of communication protocol
build up over time and scale,
resulting sensible outcomes.

5. HTTP/1.1
HTTP/2
HTTP/3
gRPC
protobuf
JSON
Avro
Thrift
Kafka
TCP
UDP
Going to mention

6. How did I learn
this?
3
1
2
Performance Testing
Traffic capture in Kubernetes
Troubleshooting K8s on all
levels

7. The Sweetest Thing 😉

8. Low-hanging fruits
from changing the defaults

9. Why we fear to change the defaults?
Junior DevOps is changing the Defaults Senior DevOps is changing the Defaults

10. Simple Method:
4 things to remember

11. 1 Goal: Optimization
Costs, Quality, MTTR

12. Security and service quality
Public
Internet
Traffic volume and infra costs
Wi-Fi
GPRS / LTE
2 Sides

13. 3 Layers
● How the app uses protocol
● To choose and tune
● You rarely can change it
(cloud, NOC, OS admin)
Transport Layer
Application Protocol
Business Logic

14. 4 Things to Tune
Connection opening Encryption
Compression Serialization

15. Popular Protocols

16. HTTP/1.1
● From 1997, still the main workhorse
● Simple and plain-text on wire
● “Open TCP transport connection, write into it, then read from it”

17. HTTP/1.1 Anatomy: Open TCP, read and write
POST /restapi/alerts/add HTTP/1.1
Accept: application/json
Authorization: Basic dGVzdHVzZXIwMTpuZXRjb29s
Content-Type: application/json
Host: localhost
Connection: keep-alive
Content-Length: 984
{
"FirstOccurrence": 1341412087,
"Node": "localhost",
"AlertKey": "JUnitEventInstance",
"Summary": "This is a test event",
"LastOccurrence": 1341412087,
"Identifier": "JunitEventTestInstance####1",
"OwnerGID": 0,
"Severity": 4,
}
HTTP/1.1 201 Created
Location: http://localhost/restapi/alerts/status/kf/12481
Cache-Control: no-cache
Server: libnhttpd
Date: Wed Jul 4 15:31:53 2012
Connection: Keep-Alive
Content-Type: application/json;charset=UTF-8
Content-Length: 304
{
"entry": {
"affectedRows": 1,
"keyField": "12481%3ANCOMS",
"uri": "http://localhost/restapi/kf/12481"
}
}
Request Response
HEADERS
BODY
REQUEST STATUS

18. Optimizing HTTP/1.1: Keep-Alive connections
x2 slower
+23% CPU

19. Optimizing HTTP/1.1: TLS opening costs
x2 slower
+23% CPU
x111
slower
x20
CPU

20. Reuse TCP connections!

21. Optimizing HTTP: Compression
x2 slower
+23% CPU
x111
slower
x20
CPU
+38%
CPU
-18%
bytes

22. HTTP/1.1 Standard Compression
POST /restapi/alerts/add HTTP/1.1
Accept: application/json
Authorization: Basic dGVzdHVzZXIwMTpuZXRjb29s
Content-Type: application/json
Host: localhost
Connection: keep-alive
Content-Length: 984
{
"FirstOccurrence": 1341412087,
"Node": "localhost",
"AlertKey": "JUnitEventInstance",
"Summary": "This is a test event",
"LastOccurrence": 1341412087,
"Identifier": "JunitEventTestInstance####1",
"OwnerGID": 0,
"Severity": 4,
}
HTTP/1.1 201 Created
Location: http://localhost/restapi/alerts/status/kf/12481
Cache-Control: no-cache
Server: libnhttpd
Date: Wed Jul 4 15:31:53 2012
Connection: Keep-Alive
Content-Type: application/json;charset=UTF-8
Content-Length: 304
{
"entry": {
"affectedRows": 1,
"keyField": "12481%3ANCOMS",
"uri": "http://localhost/restapi/kf/12481"
}
}
Request Response

23. The biggest problem with HTTP/1.1

24. The biggest problem with HTTP/1.1

25. The biggest problem with HTTP/1.1

26. The biggest problem with HTTP/1.1

27. HTTP/2
Header Compression
Connection Multiplexing
Built-in WebSocket
HTTP/1.1
+
HTTP/2 matters mostly outside the cluster.

28. HTTP/2 Connection Multiplexing
Single TCP Socket

29. HTTP/2 is Still Not Enough
TCP “sloth slow start” TCP “head of line blocking”

30. HTTP/3
● TCP => UDP
● QUIC + TLS
● Browsers be like

31. HTTP Evolution Summary
Great progress for browsers!
Dev focus is in microservices today, do we care?
Comms edge is owned by cloud and CDN…

32. Like a virgin HTTP/1.1!
The future looks like…

33. More Protocols

34. GraphQL
Request
{
orders {
id
productsList {
product {
Name
Price
}
Quantity
}
totalAmount
}
}
Shrink Me!!!
Response
{
"data": {
"orders": [
{
"id": 1,
"productsList": [
{
"product": {
"name": "orange",
"price": 1.5
},
"quantity": 100
}
],
"totalAmount": 150
}
]
}
}

35. gRPC
Protobuf Serialization Generated Code
HTTP/2 Transport
Alternatives are Apache Thrift and Avro.
It’s mostly about coding preferences.

36. Message Queue Protocols
Message broker riding a car

37. Message Queue Protocols
● Most are binary, TCP-based
● Some HTTP in the cloud (Amazon SQS)
● Compression is on you

38. AMQP & MQTT
● 50% of cases or more
● Not much to tune, mind the connection costs

39. Kafka
● Not much to tune in protocol layer
● Acknowledgements and commits costs time
● Message batching to reduce overhead
● Data serialization: JSON, Protobuf, Avro

40. Data Serialization

41. JSON Encode and Decode speed is not the same

42. JSON in different tech is not the same

43. JSON vs Protobuf

44. JSON vs Protobuf: Faster or Not?
!!!

45. JSON vs Protobuf: Binary Data Stands Out
x30 times faster!

46. Much more graphs are
there, but the time is
running out!

47. Summary

48. Quick
Checklist
2 sides to optimize
● Inside cluster
● Outside cluster
3 layers to understand
● Transport layer
● Application protocol
● Business logic
4 things to tune
● Connection reuse
● Compression
● Encryption
● Serialization

49. “
Premature optimization
is the root of all evil.
by D. Knuth
“

50. THANK YOU!
andrei@komodor.io