How Epic Games is using ScyllaDB for distribution of large game assets used by Unreal Engine across the world —enabling game developers to more quickly build great games.

1. Using ScyllaDB for
Distribution of Game
Assets in Unreal Engine
Joakim Lindqvist, Senior Tools Programmer

2. Joakim Lindqvist
■ Senior Tools Developer within the Foundation team of
Unreal Engine
■ 15 years experience working in games
■ Based out of Stockholm, Sweden

3. ■ The complexity of building games
■ Overview of Unreal Cloud DDC
■ Future work
Agenda

4. The Complexity of
Building Games

5. Games are Complicated to Make
■ 2 major types of content, source code and game assets
■ Large assets
■ Large teams work on them
■ Specialized skill - spread out across the world

7. The Process of Cooking Data
■ Cooking is data transformation
■ Historically cached on network file systems within a office
■ Geo distribution
■ Work From Home
■ Headache to scale storage
■ Can take hours to start up the editor without caching

8. Unreal Cloud DDC

9. Cached Objects
■ Custom self describing binary
format called Compact Binary
■ Structured object with
attachments that are content
addressed
■ Content addressing
■ Deduplication
■ Avoids replicating the same
content
{
“name”: “largeFile”,
“size” : 2480,
“attachment”: 9fffabc5e0a…1f084f8c5e
}

10. ■ Kubernetes
■ 5 regions
■ 7 TB local nvme cache
■ ~50 TB S3 storage
■ 10-500 users per region
■ Last access tracking
Unreal Cloud DDC

11. Architecture
ScyllaDB
(Metadata)
Web api
Local NVME
AWS S3
(Blob
Storage)
ScyllaDB
(Metadata) Web api
AWS S3
(Blob
Storage)

12. Why ScyllaDB
■ Performance sensitive workload
■ Geo replicated database
■ Eventual consistency works well with our workload
■ Small deployment (2-3 nodes of i4i.xlarge/i4i.2xlarge) in 5 DCs
■ ~3000 req/s per node

13. Future Work

14. Future Work
■ Small blob storage in ScyllaDB
■ ScyllaDB on Azure
■ Performance improvements on the API
■ Extending to more workloads

15. ■ Source Code: https://github.com/EpicGames
Unreal Engine

16. Thank You
Stay in Touch
Joakim Lindqvist
joakim.lindqvist@epicgames.com
twitter.com/J0CC0
github.com/jocco
www.linkedin.com/in/jocco/