1.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
the fastest RPC fmwrk. in the west
Principal Engineer, Platform Engineering - Akamai Technologies
Alexander Gallego

2.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Alexander Gallego
2
Principal Engineer @ Akamai - Platform Group
Ex CTO / founder of Concord.io - A distributed
stream processor written in C++ atop Apache
Mesos (Now part of Akamai)
First employee, engineer for Yieldmo.com (ad-tech)
startup in NYC
maintainer of smf: github.com/senior7515/smf

3.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
background

4.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Can we do transactional streaming?
▪ At Concord.io, I worked on a streaming platform
o Can we do transactional writes (3x replication - even if in memory)
• Can we do it with low latency and high throughput?
– double digit ms *tail* latency at 1024 batches?
▪ Fastest open source queue did 150ms p90 and 2secs p9999
o Unpredictable JVM spikes -
• Spark once stalled for 4 seconds reading from Kafka - couldn’t replicate.
o Concord was in C++ - we wanted predictability
4

5.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Can we do better?
5

6.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
How about this! - per node overhead
6
7us p90 latency
26us p100
latency
8us p99 latency

7.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
How about this! - per node overhead
7
Read Socket RPC Parsing
Method
Execution
Flush Socket
60 byte payload + 20 bytes of TCP - with full type serialization!
p90 = 7 microseconds, p99 = 8 microseconds, p100 = 26 microseconds
p99=8us

8.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf

9.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf RPC
▪ Built for microsecond tail latencies
▪ Atop seastar::future<>s
▪ IDL Compiler/CodeGen - using Google Flatbuffers’ IDL
▪ Multi-language compatibility
▪ Small - 16 byte overhead (with rich types, headers, compression,etc)
… it’s like gRPC / Thrift / Cap n’ Proto - for microsecond latencies.
9

10.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
IDL

11.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
11
namespace smf_gen.demo;
table Request {
name: string;
}
table Response {
name: string;
}
rpc_service SmfStorage {
Get(Request):Response;
}

12.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
12
namespace smf_gen.demo;
table Request {
name: string;
}
table Response {
name: string;
}
rpc_service SmfStorage {
Get(Request):Response;
}
input

13.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
13
namespace smf_gen.demo;
table Request {
name: string;
}
table Response {
name: string;
}
rpc_service SmfStorage {
Get(Request):Response;
}
output
input

14.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
14
namespace smf_gen.demo;
table Request {
name: string;
}
table Response {
name: string;
}
rpc_service SmfStorage {
Get(Request):Response;
}
Service Definition
output
input

15.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
15
namespace smf_gen.demo;
table Request {
name: string;
}
table Response {
name: string;
}
rpc_service SmfStorage {
Get(Request):Response;
}
smf_gen --filename demo_service.fbs
Service Definition
output
input

16.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Demo

17.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf::rpc_client

18.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
18
smf::rpc_typed_envelope<Request> req;
req.data->name = "Hello, smf-world!";
auto client = SmfStorageClient::make_shared("127.0.0.1",2121);
client->Get(req.serialize_data()).then([ ](auto reply) {
std::cout << reply->name() << std::endl;
});

19.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
19
smf::rpc_typed_envelope<Request> req;
req.data->name = "Hello, smf-world!";
auto client = SmfStorageClient::make_shared("127.0.0.1",2121);
client->Get(req.serialize_data()).then([ ](auto reply) {
std::cout << reply->name() << std::endl;
});
data to send

20.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
20
smf::rpc_typed_envelope<Request> req;
req.data->name = "Hello, smf-world!";
auto client = SmfStorageClient::make_shared("127.0.0.1",2121);
client->Get(req.serialize_data()).then([ ](auto reply) {
std::cout << reply->name() << std::endl;
});
data to send
actual socket
seastar::shared_ptr<T>
non-thread safe

21.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
21
smf::rpc_typed_envelope<Request> req;
req.data->name = "Hello, smf-world!";
auto client = SmfStorageClient::make_shared("127.0.0.1",2121);
client->Get(req.serialize_data()).then([ ](auto reply) {
std::cout << reply->name() << std::endl;
});
data to send
actual socket
seastar::shared_ptr<T>
non-thread safe
Method to call

22.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf::rpc_server

23.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
23
class storage_service : public SmfStorage {
virtual seastar::future<rpc_typed_envelope<Response>>
Get(rpc_recv_typed_context<Request> rec) final {
rpc_typed_envelope<Response> data;
data.data->name = "Hello, cruel world!";
data.envelope.set_status(200);
return make_ready_future<decltype(data)>(std::move(data));
}
};

24.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
24
class storage_service : public SmfStorage {
virtual seastar::future<rpc_typed_envelope<Response>>
Get(rpc_recv_typed_context<Request> rec) final {
rpc_typed_envelope<Response> data;
data.data->name = "Hello, cruel world!";
data.envelope.set_status(200);
return make_ready_future<decltype(data)>(std::move(data));
}
};
code-gen’ed service

25.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
25
class storage_service : public SmfStorage {
virtual seastar::future<rpc_typed_envelope<Response>>
Get(rpc_recv_typed_context<Request> rec) final {
rpc_typed_envelope<Response> data;
data.data->name = "Hello, cruel world!";
data.envelope.set_status(200);
return make_ready_future<decltype(data)>(std::move(data));
}
};
code-gen’ed service
Method

26.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
26
class storage_service : public SmfStorage {
virtual seastar::future<rpc_typed_envelope<Response>>
Get(rpc_recv_typed_context<Request> rec) final {
rpc_typed_envelope<Response> data;
data.data->name = "Hello, cruel world!";
data.envelope.set_status(200);
return make_ready_future<decltype(data)>(std::move(data));
}
};
code-gen’ed service
Method
return data

27.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf::rpc_filter

28.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
28

29.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
29
template <typename T>
struct rpc_filter {
seastar::future<T> operator()(T t);
};

30.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
30
struct zstd_compression_filter : rpc_filter<rpc_envelope> {
explicit zstd_compression_filter(uint32_t min_size)
: min_compression_size(min_size) {}
seastar::future<rpc_envelope> operator()(rpc_envelope &&e);
const uint32_t min_compression_size;
};

31.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
31
// add it to your clients
client->outgoing_filters().push_back(
smf::zstd_compression_filter(1000));
// add it to your servers
using zstd_t = smf::zstd_compression_filter;
return rpc.invoke_on_all(
&smf::rpc_server::register_outgoming_filter<zstd_t>,1000);

32.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf
32
static thread_local auto incoming_stage =
seastar::make_execution_stage("smf::incoming",
&rpc_client::apply_incoming_filters);
static thread_local auto outgoing_stage =
seastar::make_execution_stage("smf::outgoing",
&rpc_client::apply_outgoing_filters);
SEDA-pipelined

33.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
request anatomy

34.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf request anatomy
34
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);

35.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf request anatomy
35
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
- Turn off padding by compiler.
- Enforce layout.
- Store everything in little endian
- X-lang, X-platform compat
- noop on most platforms

36.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf request anatomy
36
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4

37.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf request anatomy
37
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4
headers?

38.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf request anatomy
38
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4
headers?
max # of concurrent requests per client

39.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf request anatomy
39
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4
headers?
max # of concurrent requests per client
xxhash32 - very fast! 5.4GB/s

40.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf request anatomy
40
/// total = 128bits == 16bytes
MANUALLY_ALIGNED_STRUCT(4) header FLATBUFFERS_FINAL_CLASS {
int8_t compression_;
int8_t bitflags_;
uint16_t session_;
uint32_t size_;
uint32_t checksum_;
uint32_t meta_;
};
STRUCT_END(header, 16);
zstd, lz4
headers?
max # of concurrent requests per client
xxhash32 - very fast! 5.4GB/s
request_id or status (response) code

41.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf Code Gen’d XOR id
41
auto fqn = fully_qualified_name;
service_id = hash( fqn(service_name) )
method_id = hash( ∀ fqn(x) input_args_types,
∀ fqn(x) output_args_types,
fqn(method_name),
separator = “:”)
rpc_dispatch_id = service_id ^ method_id;

42.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf Code Gen’d XOR id
42
/// RequestID: 212494116 ^ 1719559449
/// ServiceID: 212494116
/// MethodID: 1719559449
future<smf::rpc_recv_typed_context<Response>>
Get(smf::rpc_envelope e) {
e.set_request_id(212494116, 1719559449);
return send<smf_gen::demo::Response>(std::move(e));
}
Method ID

43.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf Code Gen’d XOR id
43
handles.emplace_back(
"Get", 1719559449,
[this](smf::rpc_recv_context c) {
using req_t = smf::rpc_recv_typed_context<Request>;
auto session_id = c.session();
return Get(req_t(std::move(c))).then(
[session_id](auto typed_env){
typed_env....mutate_session(session_id);
return make_ready_future<rpc_envelope>(
typed_env.serialize_data());
});
Method ID

44.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf Code Gen’d XOR id
44
struct rpc_service {
virtual const char *service_name() const = 0;
virtual uint32_t service_id() const = 0;
virtual std::vector<rpc_service_method_handle> methods() = 0;
virtual ~rpc_service() {}
};

45.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
telemetry

46.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf built in telemetry
46
High Dynamic Range Histogram (HDR) … Expensive 185 KB
::hdr_init(1, // 1 microsec - minimum value
INT64_C(3600000000), // 1 hour in microsecs - max value
3, // Number of significant figures
&hist); // Pointer to initialize
// clients
client = ClientService::make_shared(std::move(opts));
client->enable_histogram_metrics();
// servers enabled by default

47.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf built in telemetry (prometheus)
47

48.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
performance?

49.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf DPDK client - DPDK server*
49
7us p90 latency
26us p100
latency
8us p99 latency

50.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf end-to-end latency (DPDK)
50
p100=500us
p90=51us
p99=56us
2 Threads. Includes connection open
time - cold cache

51.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf end-to-end latency (DPDK)
51
Same graph, minus the **first** request of each of
the 2 threads
p100=151us
p50=51us p99=56us

52.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf future work
52
● Currently could only do 1.5MM qps on the server setup
○ size = 60 byte payload + 20 TCP frame bytes
○ Hit TCP.hh bug in seastar with httpd/seawreck and my own impl
■ `(_snd.window > 0) || ((_snd.window == 0) && (len == 1))'
failed.
■ Could be my lab setup
■ Because of this - couldn't fill the wire fast enough
● Add JVM, Python, Go, codegen
● Improve Docs: https://senior7515.github.io/smf/

53.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
THANK YOU
gallego.alexx@gmail.com | alexgallego.org
@emaxerrno
Please stay in touch
Any questions? https://senior7515.github.io/smf/

54.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
extra

55.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
smf Write Ahead Log (latency)
55
percentile Apache Kafka smf WAL speedup
p50 878ms 21ms 41X
p95 1340ms 36ms 37x
p99 1814ms 49ms 37x
p999 1896ms 54ms 35x
p100 1930ms 54ms 35x

56.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
56

57.
PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company