Optimizing Performance in Rust for Low-Latency Database Drivers

Optimizing Performance
in Rust for Low-Latency
Database Drivers
Piotr Grabowski, Software Team Leader, ScyllaDB

Poll
Where are you in your NoSQL adoption?

Presenter
Piotr Grabowski, Software Team Leader, ScyllaDB
+ Software Team Leader at ScyllaDB
+ Responsible for all ScyllaDB drivers, ScyllaDB Kafka
Connectors (ScyllaDB Sink Connector and ScyllaDB CDC
Source Connector)
+ Joined ScyllaDB 2.5 years ago

+ For data-intensive applications that require high
throughput and predictable low latencies
+ Close-to-the-metal design takes full advantage of
modern infrastructure
+ >5x higher throughput
+ >20x lower latency
+ >75% TCO savings
+ Compatible with Apache Cassandra and Amazon
DynamoDB
+ DBaaS/Cloud, Enterprise and Open Source
solutions
The Database for Gamechangers
4
“ScyllaDB stands apart...It’s the rare product
that exceeds my expectations.”
– Martin Heller, InfoWorld contributing editor and reviewer
“For 99.9% of applications, ScyllaDB delivers all the
power a customer will ever need, on workloads that other
databases can’t touch – and at a fraction of the cost of
an in-memory solution.”
– Adrian Bridgewater, Forbes senior contributor

+ ScyllaDB runs only on Linux
+ We take advantage of many Linux-only APIs:
+ io_uring
+ (previously) epoll/aio
+ Avi Kivity, CTO and cofounder of ScyllaDB, began
the development of KVM in Linux kernel
+ Great performance and low latencies are our
focus, frequently looking into how ScyllaDB can
work more efficiently with Linux kernel
The Linux-native Database
5
“ScyllaDB stands apart...It’s the rare product
that exceeds my expectations.”
– Martin Heller, InfoWorld contributing editor and reviewer
“For 99.9% of applications, ScyllaDB delivers all the
power a customer will ever need, on workloads that other
databases can’t touch – and at a fraction of the cost of
an in-memory solution.”
– Adrian Bridgewater, Forbes senior contributor

6
+400 Gamechangers Leverage ScyllaDB
Seamless experiences
across content + devices
Digital experiences at
massive scale
Corporate fleet
management
Real-time analytics 2,000,000 SKU -commerce
management
Video recommendation
management
Threat intelligence service
using JanusGraph
Real time fraud detection
across 6M
transactions/day
Uber scale, mission critical
chat & messaging app
Network security threat
detection
Power ~50M X1 DVRs with
billions of reqs/day
Precision healthcare via
Edison AI
Inventory hub for retail
operations
Property listings and
updates
Unified ML feature store
across the business
Cryptocurrency exchange
app
Geography-based
recommendations
Global operations- Avon,
Body Shop + more
Predictable performance
for on sale surges
GPS-based exercise
tracking
Serving dynamic live
streams at scale
Powering India's top
social media platform
Personalized
advertising to players
Distribution of game
assets in Unreal Engine

Agenda
+ Introduction
+ ScyllaDB Rust Driver
+ Bindings to ScyllaDB Rust Driver

Drivers 101
+ Drivers (in this presentation) - libraries that allow sending queries to
ScyllaDB
+ Primary protocol: CQL (Cassandra Query Language) protocol
+ TCP
+ ScyllaDB supports CQL v4
+ Frame-based protocol, supporting multiple streams
+ Supports LZ4 and Snappy compression
+ ScyllaDB drivers support shard awareness:
+ Driver can connect to a specific shard of ScyllaDB

Drivers 101 - Role of Drivers
+ The role of drivers:
+ Serialization/deserialization of CQL frames
+ Serialization/deserialization of ScyllaDB types
+ Querying and maintaining metadata about tables/nodes
+ Routing requests to correct nodes (and shards)
+ Sending request across network
+ Conveniently constructing and executing queries in your language of choice:
+ gocqlx
+ Java Driver’s Mapper interface

Drivers 101 - Performance
+ How can the driver improve performance?
+ Shard awareness: sending the query to a correct shard
+ Partitioners: ScyllaDB’s CDC (Change Data Capture) implements a custom
partitioner which determines a node to send the query to
+ LWT Optimization: consistently prefer a single replica when executing a
LWT query to avoid Paxos conflicts
+ Optimizing hot paths in the driver:
+ Serialization/deserialization
+ Routing code
+ Avoiding copies, allocations and locks

ScyllaDB Rust Driver
+ The idea was born during a hackathon in 2020
+ Over the last 3 years we continued the development

+ The idea was born during a hackathon in 2020
+ Over the last 3 years we continued the development
+ Uses Tokio framework
+ The driver is now feature complete, supporting many advanced features:
+ Shard awareness
+ Asynchronous interface with support for large concurrency
+ Compression
+ All CQL types
+ Speculative execution
+ TLS support

ScyllaDB Rust Driver - Runtime
+ Async Rust is based on a quite unique future/promise model:
+ Running a function which returns a future does not automatically spawn an
asynchronous task, as in many other languages
+ Instead, async functions need a runtime to execute them
+ Which runtime to choose?
+ Tokio (http://tokio.rs) is a de-facto standard runtime for async Rust
projects.
We chose it due to its rich set of APIs, popularity and very active
community of developers and contributors.

ScyllaDB Rust Driver - API Design
+ A central component of our driver is a session, established once and then
used to communicate with Scylla. It has many customizable parameters,
but most of them have sensible defaults.
let uri = "127.0.0.1:9042";
let session: Session = SessionBuilder::new().known_node(uri).build().await?;
if let Some(rows) = session.query("SELECT a, b, c FROM ks.t", &[]).await?.rows {
for row in rows.into_typed::<(i32, i32, String)>() {
let (a, b, c) = row?;
println!("a, b, c: {}, {}, {}", a, b, c);
}
}

ScyllaDB Rust Driver - O(N²) in Tokio?
+ Issue raised by the author of latte - a benchmark tool for ScyllaDB and Cassandra
+ The driver had problems scaling with high concurrency of requests
+ We managed to identify a root cause in the implementation of FuturesUnordered, a
utility to gather many futures and wait for them
+ Due to cooperative scheduling in Tokio, it was possible for
FuturesUnordered to iterate over all futures each time
it was polled
+ A fix was merged to Tokio to limit the number of
Futures iterated over in each poll

ScyllaDB Rust Driver - Connection Management
Ability to customize the number of connections is critical for performance. Our driver
uses a default of 1 connection per shard, but can be customized to instead establish a
fixed number of connections, be it per node or per shard.

ScyllaDB Rust Driver - Shard Awareness
Scylla takes it even further - drivers can try to connect directly to a core which
owns a particular partition, which implies better latency. Shard awareness is built in
into Scylla Rust Driver from the start.

ScyllaDB Rust Driver - Load Balancing

SELECT * FROM table
WHERE partition_key = “R1250GS”
hash(“R1250GS”) = replica nodes

+ Main goal: reduce number of allocations and atomic operations while
building the query plan, especially on the happy path:
+ Plan function was split to pick() and fallback() methods. This allowed to
better optimize the most common case, where only one node from the load
balancing plan is needed
+ Precomputation of replica sets:
+ A struct introduced that precomputes replica lists of a given strategies, and
provides O(1) access to desired replica slices
Refactor

Refactor
Inserts:
----------
allocs/req: 15.00
reallocs/req: 8.00
frees/req: 15.00
bytes allocated/req: 2458.05
bytes reallocated/req: 269.06
bytes freed/req: 2456.80
(allocated - freed)/req: 1.25
Inserts:
----------
allocs/req: 6.01
reallocs/req: 6.00
frees/req: 6.00
Before After

Refactor
Inserts:
----------
allocs/req: 15.00
reallocs/req: 8.00
frees/req: 15.00
Inserts:
----------
allocs/req: 6.01
reallocs/req: 6.00
frees/req: 6.00
Before After
9 fewer allocations (-
60%)

Refactor
Inserts:
----------
allocs/req: 15.00
reallocs/req: 8.00
frees/req: 15.00
Inserts:
----------
allocs/req: 6.01
reallocs/req: 6.00
frees/req: 6.00
Before After
84% fewer
bytes
allocated

Refactor
Selects:
----------
allocs/req: 48.00
reallocs/req: 8.00
frees/req: 48.00
Selects:
----------
allocs/req: 39.00
reallocs/req: 6.00
frees/req: 39.00
Before After

Refactor
Selects:
----------
allocs/req: 48.00
reallocs/req: 8.00
frees/req: 48.00
Selects:
----------
allocs/req: 39.00
reallocs/req: 6.00
frees/req: 39.00
Before After
Less
difference
compared to
inserts

ScyllaDB Rust Driver - Other Efforts
+ Rack-aware load balancing
+ Reduce the cost of querying ScyllaDB nodes in other racks (corresponding
for example to AWS Availability Zones)
+ Reduce the latency by querying the nearest rack
+ Iterator-based deserialization
+ The current implementation deserializes row data into equivalent of
Vec<Vec<Option<CqlValue>>
+ Skip materializing all rows into vector, deserialize on-the-fly
+ Make great use of Rust lifetimes to guarantee memory safety

ScyllaDB Rust Driver - Iterator-based
Deserialization
+ Reworked Deserialization API
+ Solves performance issues and improves type safety
+ Old API marked as "Legacy" for backward compatibility
+ Problems with Current API
+ Inefficient representation with rows and vecs
+ Incomplete information for FromCqlVal and FromRow
+ New API with DeserializeCql and DeserializeRow
+ Allows on-demand deserialization, reducing allocations
+ More comprehensive type checking and improved deserialization
+ Migration from Legacy API
+ Mechanical changes for most use cases
+ Legacy and new API can be used simultaneously

ScyllaDB Rust Driver - Removing All
Allocations?
+ A community-started project, led by Joseph Perez (@wyfo) written from
scratch to have zero-copy deserialization, zero (or one) allocations per
request
+ Core ideas:
+ Query plan caching
+ Zero/one allocation per request
+ We are looking into incorporating the ideas shown in this project into

ScyllaDB Rust Driver - Profiling tools
Rust ecosystem makes it easy to look for performance issues in your
project. One of such tools is cargo flamegraph, a utility for creating
flamegraphs, which can be examined to see if any function calls take up too
much CPU time.

For projects based on Tokio, tokio-console can be used to inspect
running asynchronous tasks in real time, browse the used resources, and so
on.
Ref: https://tokio.rs/blog/2021-12-announcing-tokio-console

Bindings to ScyllaDB
Rust Driver

Bindings to ScyllaDB Rust Driver
+ When benchmarking ScyllaDB Rust Driver against other drivers, we
measured it was the most performant driver, beating the C++ driver
+ Why not develop a way to use ScyllaDB Rust Driver from C++ code?
+ Benefits of a unified core:
+ Higher performance
+ Easier maintenance
+ Fewer bugs

Bindings to ScyllaDB Rust Driver - C/C++
+ We started development for the C/C++ language
+ C++ bindings to the Rust driver; the same API as the original
C++ driver
+ Drop-in replacement (just replacing .so file)
+ The resulting project has an order-of-magnitude fewer LoC
+ Better stability, fewer problems compared to the original C++
driver

Bindings to ScyllaDB Rust Driver - C/C++
#[no_mangle]
pub unsafe extern "C" fn cass_future_ready(future_raw: *const
CassFuture) -> cass_bool_t {
let state_guard = ptr_to_ref(future_raw).state.lock().unwrap();
match state_guard.value {
None => cass_false,
Some(_) => cass_true,
}
}
Rust
__attribute__ ((visibility("default")))
cass_bool_t
cass_future_ready(CassFuture* future);
C

Q&A
ScyllaDB University
Free online learning
scylladb.com/university
scylladb.com/events
Build Low-Latency
Rust Applications
on ScyllaDB
June 21 2023
October 18 + 19, 2023
p99conf.io

Thank you
for joining us today.
@scylladb scylladb/
slack.scylladb.com
@scylladb company/scylladb/
scylladb/

Optimizing Performance in Rust for Low-Latency Database Drivers

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Optimizing Performance in Rust for Low-Latency Database Drivers

Ähnlich wie Optimizing Performance in Rust for Low-Latency Database Drivers (20)

Mehr von ScyllaDB

Mehr von ScyllaDB (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Optimizing Performance in Rust for Low-Latency Database Drivers

Hinweis der Redaktion