Wix has a huge scale of traffic. more than 500 billion HTTP requests and more than 1.5 billion Kafka business events per day. This talk goes through 3 Caching Patterns that are used by Wix's 2000 microservices in order to provide the best experience for Wix users along with saving costs and increasing availability. A cache will reduce latency, by avoiding the need of a costly query to a DB, a HTTP request to a Wix servicer, or a 3rd-party service. It will reduce the needed scale to service these costly requests. It will also improve reliability, by making sure some data can be returned even if aforementioned DB or 3rd-party service are currently unavailable. The patterns include: * Configuration Data Cache - persisted locally or to S3 * HTTP Reverse Proxy Caching - using Varnish Cache * (Dynamo)DB+CDC based Cache and more - for unlimited capacity with continuously updating LRU cache on top each pattern is optimal for other use cases, but all allow to reduce costs and gain performance and resilience.

1. Advanced Microservices Caching
Patterns
Natan Silnitsky - Backend Infra TL, Wix.com
November 30, December 1-2, 2021

2. @NSilnitsky
@NSilnitsky
Caching Patterns
Advanced Microservices
Caching Patterns
Natan Silnitsky
Backend Infra TL,
Wix.com
natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil

3. @NSilnitsky
Caching Patterns
>200 Million registered users from 190 countries

4. @NSilnitsky
@NSilnitsky
Caching Patterns
Rebecca, Wix Stores
Stores
Checkout
Service
* issues

5. @NSilnitsky
@NSilnitsky
Caching Patterns
1. Get app
identification
2. Start
WIX APP
MARKET
Rebecca, Wix Stores
HTTP
Stores
Checkout
Service

6. @NSilnitsky
@NSilnitsky
Caching Patterns
WIX APP
MARKET
Rebecca, Wix Stores 1. Get app
identification
2. Start
Stores
Checkout
Service
Unreliable Requests
* still available

7. @NSilnitsky
@NSilnitsky
Caching Patterns
Alex, Wix Stores
Store
Catalog
Stores
Catalog
Service
retrieve

8. @NSilnitsky
@NSilnitsky
Caching Patterns
Alex, Wix Stores
Store
Catalog
Stores
Catalog
Service
Request Overload
DB latency and
failure increase

9. @NSilnitsky
@NSilnitsky
No Cache - Increase in
- loading and response
- Network costs
- DB failure
Caching Patterns
* traffic grows

10. @NSilnitsky
@NSilnitsky
With Cache -
- reduce latency
- reduce the needed scale
- improve reliability
Caching Patterns
* DB is unavailable

11. @NSilnitsky
When & how to Cache
Examples from 2000 microservices of various use cases
Caching Patterns
#1
high risk/cost
of network
failure
Like caching
external critical
configuration
data.
* S3 backed

12. @NSilnitsky
When & how to Cache
Examples from 2000 microservices of various use cases
Caching Patterns
#1
high risk/cost
of network
failure
Like caching
external critical
configuration
data.
#2
To improve
average latency
for data layer
access
DynamoDB +
CDC +
LRU cache.

13. @NSilnitsky
When & how to Cache
Examples from 2000 microservices of various use cases
Caching Patterns
#1
high risk/cost
of network
failure
Like caching
external critical
configuration
data.
#2
To improve
average latency
for data layer
access
DynamoDB +
CDC +
LRU cache.
#3
Some very high
external traffic
cases
Caches are very
important -
cache before
the service.

14. @NSilnitsky
When & how to Cache
Examples from 2000 microservices of various use cases
Caching Patterns
#1
high risk/cost
of network
failure
Like caching
external critical
configuration
data.
#2
To improve
average latency
for data layer
access
DynamoDB +
CDC +
LRU cache.
#3
Some very high
external traffic
cases
Caches are very
important -
cache before
the service.
#4
When NOT to -
young products
Don’t cache
prematurely

15. @NSilnitsky
Agenda 3 Caching Patterns:
1. S3 backed static cache
2. DynamoDB+CDC based cache
3. HTTP Reverse Proxy cache
Caching Patterns

16. @NSilnitsky
Agenda 3 Caching Patterns:
1. S3 backed static cache
2. DynamoDB+CDC based cache
3. HTTP Reverse Proxy cache
Caching Patterns

17. @NSilnitsky
Service
WIX APP
MARKET
Caching Patterns
for Unreliable Requests

18. @NSilnitsky
Service
WIX APP
MARKET
Caching Patterns
On Init - Fetch app defs
Koboshi /
Remote Data Fetcher

19. @NSilnitsky
Service
WIX APP
MARKET
Caching Patterns
Koboshi /
Remote Data Fetcher
Local
cache
or
AWS S3

20. @NSilnitsky
Service
WIX APP
MARKET
Caching Patterns
Koboshi /
Remote Data Fetcher
Local
cache AWS S3
or
+ Retry

21. @NSilnitsky
@NSilnitsky
Caching Patterns
Cache Type Cache Size Best For Not For
Read-through cache
Before the 3rd party service
Data has to
fit into memory
External source
of static configuration
Dynamically
updating data

22. @NSilnitsky
Next 3 Caching Patterns:
1. S3 backed static cache
2. DynamoDB+CDC based cache
3. HTTP Reverse Proxy cache
Caching Patterns

23. @NSilnitsky
Caching Patterns
Store
Catalog
Stores
Catalog
Service
for Request Overload
Read + Writes
Browser
HTTP
* slower interaction

24. @NSilnitsky
Caching Patterns
Stores
Catalog
Service
Cached KV Store
2. put
Store
Catalog
1. Write
V1
K1
V1
K2 K3 K8 K9
V1 V2 V1
head End

25. @NSilnitsky
Caching Patterns
Stores
Catalog
Service
Cached KV Store
1. get
Store
Catalog
2. Read
on miss
V1
K1
V1
K2 K3 K8 K9
V1 V2 V1
head End

26. @NSilnitsky
Caching Patterns
Stores
Catalog
Service
Cached KV Store
LRU Cache
put
Amazon
DynamoDB
put

27. @NSilnitsky
Caching Patterns
Stores
Catalog
Service
Cross-DC Data
replication & conflict
resolution
Amazon
DynamoDB
Cache

28. @NSilnitsky
Caching Patterns
Stores
Catalog
Service
Amazon
DynamoDB
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4 5
0 1 2 3 4 5
0 1 2 3 4 5
0 1 2 3 4 5
Event
K1- V2 is updated
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
0 1 2 3 4 5
0 1 2 3 4 5
0 1 2 3 4 5
0 1 2 3 4 5
K1
----
V1
K2
----
V1
K1
-----
V2
K3
----
V1
K4
----
V1
K5
----
V1
CDC Topic
Keeps a SEQUENCE OF EVENTS
Kafka Broker
Cache

29. @NSilnitsky
Caching Patterns
Stores
Catalog
Service
Amazon
DynamoDB
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4 5
0 1 2 3 4 5
0 1 2 3 4 5
0 1 2 3 4 5
Event
K1- V2 is updated
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
5
0 1 2 3 4
0 1 2 3 4 5
0 1 2 3 4 5
0 1 2 3 4 5
0 1 2 3 4 5
K1
----
V1
K2
----
V1
K1
-----
V2
K3
----
V1
K4
----
V1
K5
----
V1
CDC Topic
Keeps a SEQUENCE OF EVENTS
Kafka Broker
Consumer
(Consumes
from)
Cached KV Store
V1
K1
V1
K2 K3 K8 K9
V1 V2 V1
head End

30. @NSilnitsky
Cache
Caching Patterns
Warm up
Take most
recent events
Amazon
DynamoDB
CDC Topic
Kafka Broker
Stores
Catalog
Service

31. @NSilnitsky
Cache
Caching Patterns
Warm up (compact)
Take ALL
Up to date events
Amazon
DynamoDB
CDC Topic
Kafka Broker
Stores
Catalog
Service

32. @NSilnitsky
@NSilnitsky
Caching Patterns
Cache Type Cache Size Best For Not For
Cache-Aside
Before the DB
No limitation on DB data size
LRU cache size is configurable
Make DB read access faster
for “popular” values
Highly critical application
startup information

33. @NSilnitsky
Next 3 Caching Patterns:
S3 backed static cache
DynamoDB+CDC based cache
HTTP Reverse Proxy cache
Caching Patterns

34. @NSilnitsky
Caching Patterns

35. @NSilnitsky
Caching Patterns
Site
Renderer
Service
DNS
resolution
Load
Balancers
* simplified flow
Wix DC
Varnish
cache
* open source

36. @NSilnitsky
Caching Patterns
DNS
resolution
Load
Balancers
* simplified flow
Varnish
cache
Kafka Broker
Site Published
Purge request
(SiteId)
Site
Editor
Service
Site
Renderer
Service
Wix DC

37. @NSilnitsky
Caching Patterns
DNS
resolution
Load
Balancers
* simplified flow
Varnish
cache
Kafka Broker
Site Published
Purge request
(SiteId)
Purge
ETag: SiteId
Site
Editor
Service
Varnish
Purger
Service
Site
Renderer
Service
Wix DC

38. @NSilnitsky
@NSilnitsky
Varnish
cache
Size limit depends on the configured
Storage backends:
1. malloc is memory based — so the size is limited by
(more) expensive memory cost
2. file is memory-backed-by-disk based — which is
limited by less expensive disk cost, but can cause
performance penalty due to fragmentation…
Caching Patterns

39. @NSilnitsky
@NSilnitsky
Varnish
cache
Size limit depends on the configured
Storage backends:
1. malloc is memory based — so the size is limited by
(more) expensive memory cost
2. file is memory-backed-by-disk based — which is
limited by less expensive disk cost, but can cause
performance penalty due to fragmentation…
3. Massive Storage Engine (MSE) — also backed by file,
but has a “fragmentation proof algorithm.” part of
Varnish Cache Plus (paying customers only).
Caching Patterns

40. @NSilnitsky
@NSilnitsky
Caching Patterns
Cache Type Cache Size Best For Not For
Read-through
before the app
Depends on storage
backend
Improving latency for “stable”
HTTP responses
Hard to invalidate
aggregated data

41. @NSilnitsky
@NSilnitsky
Caching Patterns
Choose
Cache Pattern
Is your data crucial
for startup?
Is your data highly
dynamic?
No
Yes
S3 Persisted data cache

42. @NSilnitsky
@NSilnitsky
Caching Patterns
Choose
Cache Pattern
Is your data crucial
for startup?
Is your data highly
dynamic?
S3 Persisted data cache
No
Yes
Is your data highly
dynamic?
Yes
No
DynamoDB backed
cache / Redis
Yes

43. @NSilnitsky
@NSilnitsky
Caching Patterns
Choose
Cache Pattern
Is your data crucial
for startup?
Is your data highly
dynamic?
S3 Persisted data cache
No
Yes
Is your data highly
dynamic?
Yes
No
External clients &
easy invalidation?
No
DynamoDB backed
cache / Redis
Varnish cache /
CDN
Yes
No
Yes

44. @NSilnitsky
https://medium.com/wix-engineering/4-microservices-caching-pattern
s-at-wix-b4dfee1ae22f
The blog post
Caching Patterns

45. @NSilnitsky
A Java/Scala high-level SDK for Apache Kafka.
github.com/wix/greyhound
Caching Patterns
0.2 is out!

46. @NSilnitsky
@NSilnitsky
Thank
You!
👉 slideshare.net/NatanSilnitsky
Any questions?
natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
Caching Patterns

47. @NSilnitsky
@NSilnitsky
Q&A
natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
👉 slideshare.net/NatanSilnitsky
Any questions?