ScyllaDB co-founder and CTO Avi Kivity will cover 2022 accomplishments and deliveries. Avi will focus on ScyllaDB core database in this talk. This year we enabled full repair-based-node-operations, brought distributed aggregation into Scylla as default, focused on goodput, at the face of overload and many other changes.

1. ScyllaDB 5.2 and
Beyond
Fresh from the ScyllaDB Oven
Avi Kivity, CTO and Co-Founder

2. Agenda
■ Increasing Streaming Robustness
■ Autoparallel Queries
■ WebAssembly User Defined Functions
■ Per-partition Throttling
■ Alternator Updates
■ Consistent Schema and Topology
■ New SSD Disk Modeling
■ Taming Corner Cases
■ What’s Cooking Now

3. Repair-Based Node Operations
■ Resumable bootstrap/decommission
■ Stream from primary replica
■ Or a quorum if primary is missing
■ Increases resilience and improves correctness

4. Autoparallel Queries
■ Aggregations previously done via Spark or custom code
■ Instead, recognize certain CQL patterns
■ Dispatch to all nodes, all vcpus within nodes
Node 5
Node 1
Node 2
Node 4 Node 3
SELECT COUNT(*)
FROM t

5. WebAssembly UDF/UDA
■ Push compute into database
■ Use any language*
■ Computations run in a WASM sandbox
■ Use case: analytics
*as long as it’s Rust

6. Per Partition Rate Limit
■ New CQL table attribute to limit access rate to partition
■ Works for reads and writes
■ Prevent bot accounts from spamming database
■ “Hot partition”

7. Alternator Updates
■ Time-to-live Expiration
■ Improved performance

8. ■ Eliminate classes of operator errors
■ Concurrent schema changes
■ Concurrent topology operations
■ Lay groundwork for more advanced features
■ Concurrent node bootstrap/decommission
■ Tablets
■ Strong consistency
Consistent Schema and Topology

9. ScyllaDB knows more
about the disk operating
envelope
New SSD Disk Modeling

10. Taming Corner Cases

11. Reverse Queries
■ 4.5 and older slow for large partitions
■ 4.6 fast, but skipped cache
■ 5.0+ fast, supports cache
■ Works well with paging SELECT *
FROM tab
WHERE …
ORDER BY clustering_key DESC

12. ■ Queries that encounter large consecutive tombstone runs are now well
supported
■ Partitions with many range tombstones work well
Better Handling of Tombstones

13. ■ Escalating countermeasures as memory usage increases
■ Prevent new queries from starting
■ Allow only one query to make progress
■ Kill all but one query
Improved Out-of-Memory Handling

14. Repair-Based Tombstone Garbage Collection
■ Eliminate gc_grace_seconds
■ Tie tombstone garbage collection to last repair
■ Improves performance for clusters that have frequent repair
■ Improves correctness for clusters that missed repair

15. Cooking Now

16. Nudging the CQL Grammar Towards SQL
■ Relaxing constraints
■ Reconciling semantic oddities
■ Increasing the scope of autoparallel queries

17. ■ A spectrum of cost/performance tradeoffs
■ RAM: Extremely fast (100ns), very expensive
■ NVMe: Very fast, (100µs), expensive
■ HDD: Slow (10ms), cheap
■ Cloud Object storage (S3 and similar)
■ Slow (40ms), cheap
■ Infinitely expandable
■ Easy to manipulate
■ Shared access
Object Storage

18. ■ Very dense databases
■ Where latency is not critical
■ Tiered storage
■ Mix service levels and cost
■ Optimize both cost and latency
Use Cases for Object Storage

19. Thank You
Stay in Touch
Avi Kivity
avi@scylladb.com
@AviKivity
@avikivity