This is an introduction to building our services in a different way, where state is moved out of the database and into the services (as opposed to mainstream stateless servers). It also describes one particular proof-of-concept tool that Cabify built during its annual offsite.

2. Cachopo.
Scalable Stateful
Services

3. Abel Muiño
Lead Software
Engineer at
@cabifyDev
@amuino

4. The old world. Stateless services.

5. 📱 databaseservice
📱
📱

6. 📱 databaseservice
📱
📱
📱
📱
📱

7. 📱 databaseservice
📱
📱
📱
📱
📱
service

8. 📱 databaseservice
📱
📱
📱
📱
📱
service
📱📱
📱📱

9. 📱 databaseservice
📱
📱
📱
📱
📱
service
📱📱
📱📱
service

10. 📱 databaseservice
📱
📱
📱
📱
📱
service
📱📱
📱📱
service
📱
📱
📱
📱
📱
📱
📱📱
📱📱
service
service
service
service

11. 2011 2012 2013 2014 2015 2016
Completed Journeys
(Axis has no legend because NDA and stuff)

12. Data shipping.

13. 📱 databaseservice
📱
📱
📱
📱
📱
service
📱📱
📱📱
service

14. The new world. Stateful services.

15. 📱 databaseservice
📱
📱
📱
📱
📱
service
📱📱
📱📱
service

16. More on Stateful Services:
"Building Scalable Stateful Services"
by Caitie McCaffrey on Youtube

17. Cooking Cachopo

18. Cachopo. The core ideas:
- move data to where it is used
- protect your database
- operate in memory

19. Cachopo. Use cases:
- full dataset in memory
- allows full in-memory queries
- core dataset
- hot dataset in memory
- allows only “by id” queries
- authorisations / online users / …

20. Constraints. Build what we can
during offsite (1 week).

21. Cachopo. Use cases:
- hot dataset in memory
- allows only “by id” queries
- authorisations / online users / …
- full dataset in memory
- allows full in-memory queries
- core dataset

22. Essentially we built a read-
through cache. ¯_(ツ)_/¯

23. Simplest API
{:ok, pid} = Cachopo.start_link(
adapter: MyAdapter,
adapter_config: [foo: “bar"])
{:ok, value} = Cachopo.get(pid, "my_key")

24. config = [adapter: MyAdapter,
adapter_config: [foo: “bar”]]
children = [
worker(Cachopo, [config, [name: :my_cache]]),
]
opts = [strategy: :one_for_one]
Supervisor.start_link(children, opts)
{:ok, value} = Cachopo.get(:my_cache, "my_key")
Supervision Tree 👍

25. In process storage.
Using erlang’s ETS

26. Erlang’s :ets module
Access to C code to store erlang terms, so there
is minimum overhead (no serialization to/
deserialization from, e.g. JSON)
At the top level, stores tuples. One of the
positions of the tuple is the primary key.
Out of heap
ETS data is stored outside erlang processes, so it
does not impact GC
Key-Value store
Fast access to “rows” by key.
Also allows scans and some matching syntax
not yet used for Cachopo.
No mailbox
ETS is designed for highly concurrent access.
Unlike a process mailbox, there is no serialisation
of the messages.
No network
No rountrips, no errors, no format conversions…
ETS. The S is for Speed.

27. ETSCache
Adapter
Cachopo
Database

28. Adapter. You keep using that
word…

29. defmodule Cachopo.Adapter do
@type id :: Cachopo.Cache.id
@type config :: Spec.config
@type key :: term
@type value :: term
@type on_get :: {:ok, value} | :not_found | {:error, term}
@callback setup(config) :: config
@callback start_link(id, config) :: Supervisor.on_start
@callback get(id, config, key) :: on_get
end
The Adapter

30. ⚠ cache invalidation!

31. defmodule Cachopo.Adapter do
...
@callback changes(id, config) :: Enumerable.t
end
Return of the Adapter

32. Push, don't pull. Updates to the
storage are published to the service.

33. ETSCache
Adapter
Cachopo
Database
Invalidator

34. Couchdb adapter.
Cabify <3 Couch.
Live changes feed
Clients can receive new versions as they are
written. Globally for a database or filtered.
Cachopo uses this on start.
Replay changes
Clients can “rewind” the feed to a given
change.
Cachopo uses this to recover from
disconnections so that no change is lost.

35. Couchdb Adapter. Three
implementation highlights.
HTTPoison streaming mode
The changes feed may be infinite, Cachopo
parses it line by line.
Stream API for changes feed
Store changes in the Changes Consumer, read
them using familiar iteration.
Connection behaviour
Simplifies connect/disonnect management,
backoffs on the Changes Consumer. Build on top
of GenServer.

36. defmodule Cachopo.Adapter.Couchdb.Client do
def follow(%{url: url,
http_client: http_client,
http_client_options: opts},
last_seq "now",
handler self()) do
url = changes_url_from(url, last_seq)
opts = opts ++ [stream_to: handler,
recv_timeout: :infinity]
http_client.get(url, [], opts)
end
end
1. HTTPoison streaming mode

37. def handle_async_message(%HTTPoison.AsyncChunk{chunk: chunk})
when chunk in ["n", ""],
do: :ignore
def handle_async_message(%HTTPoison.AsyncChunk{chunk: chunk}),
do: Parser.change(chunk)
def handle_async_message(%HTTPoison.AsyncStatus{code: 200}),
do: :ignore
def handle_async_message(%HTTPoison.AsyncStatus{} = message),
do: {:error, {:unexpected_status, message}}
def handle_async_message(%HTTPoison.AsyncEnd{}),
do: :closed
def handle_async_message(_anything),
do: :ignore
1. HTTPoison streaming mode

38. defmodule Cachopo.Adapter.Couchdb.Changes.Consumer do
def handle_info(message, %{client: client} = state) do
case client.handle_async_message(message) do
...
# parses and buffers or delivers changes
...
end
end
end
1. HTTPoison streaming mode

39. Stream API for changes
feed. Stream.resource/3
Stream.resource(
start_fun,
next_fun,
after_fun
)
start_fun
Starts a new consumer process that reads &
buffers changes.
next_fun
Retrieves a batch of buffered changes from the
consumer process.
If the buffer is empty, the requester is left waiting
and the next change will be delivered
immediately.
after_fun
No op.

40. defmodule Cachopo.Adapter.Couchdb.Changes do
defp start_fun(sup_mod, sup, config) do
fn ->
{:ok, pid} = sup_mod.start_changes_consumer(sup, config)
pid
end
end
defp next_changes_fun(consumer_mod) do
fn pid ->
changes = consumer_mod.pop_many(pid)
{changes, pid}
end
end
end
2. Stream API for changes feed

41. defmodule Cachopo.Adapter.Couchdb.Changes.Consumer do
def handle_call(:pop_many, from, state) do
case State.pop_all_changes(state) do
{:ok, changes, new_state} ->
{:reply, changes, new_state}
:empty ->
{:noreply, State.set_waiting(state, from)}
end
end
end
2. Stream API for changes feed
Keep caller
waiting
Store caller pid for fast
response when data arrives

42. defmodule Cachopo.Adapter.Couchdb.Changes.Consumer do
defp process(change, seq_num, state) do
state = State.set_seq_num(state, seq_num)
case State.pop_waiting(state) do
{:ok, from, new_state} ->
GenServer.reply(from, [change])
new_state
:empty ->
State.push_change(state, change)
end
end
end
2. Stream API for changes feed
Store sequence
number to allow
recovery
If there was a client
waiting, reply.
Otherwise, just
buffer

43. defmodule Cachopo.Adapter.Couchdb.Changes.Consumer do
use Connection
def init({config, client}),
do: {:connect, :init, State.new(config, client)}
def connect(_source, %{backoff: backoff} = state) do
case follow(state) do
{:ok, _info} -> {:ok, state}
{:error, _reason} -> {:backoff, backoff, state}
end
end
end
3. Connection behaviour

44. Benchmarking.
The unprofessional kind.

45. Request times. Couchdb in localhost, no load,
small documents.
Couchdb endpoint
[info] GET /couch/test
[info] Sent 200 in 25ms
[info] GET /couch/test
[info] Sent 200 in 1ms
[info] GET /couch/test
[info] Sent 200 in 1ms
[info] GET /couch/test
[info] Sent 200 in 1ms
Cachopo endpoint
[info] GET /document/test
[info] Sent 200 in 11ms
[info] GET /document/test
[info] Sent 200 in 67µs
[info] GET /document/test
[info] Sent 200 in 65µs
[info] GET /document/test
[info] Sent 200 in 66µs

46. Phoenix
Cachopo
Phoenix
Cachopo
Artillery CouchdbPhoenix
Cachopo

47. Mean request time (s). Synthetic concurrent served
requests.
0
1500
3000
4500
6000
1000 5000 10000 25000
Cachopo Couch

48. Questions? Show me the code?

49. Thanks!
@amuino
@cabifyDev