Sample application design with and without Elixir/OTP. GenServer with naming option and pooling option. Dynamic metrics with metrex.

1. Re-design with Elixir/OTP
2016 - 2017
ImgOut / On the fly thumbnail generator microservice using Elixir/OTP.
by Mustafa Turan
https://github.com/mustafaturan/imgout

2. Summary
On the fly thumbnail generator microservice, a web microservice:
- Basic Approach without Elixir/OTP
- Anatomy of an Elixir Process (optional)
- Fault Tolerance Systems with Supervision Tree Strategies (optional)
- Approach 1 / with Supervision Tree + GenServer
- Approach 2 / with Supervision Tree + GenServer + Partial Pooling
- Approach 3 / with Supervision Tree + GenServer + Pooling
- Metrics
- Source Codes
- Questions

3. Basic Approach without
Elixir/OTP *
* Web Based Approach without OTP Benefits

4. Building Blocks
Thumbnail Microservice For All Languages:
Inputs Outputs
Image identifier (ID)
Dimensions (width*, height*)
Thumbnail (Binary)
Content type (String)
Tools and Requirements Sample Elixir Specific Tools
Web Server
Image conversion library
Cache
Storage or Image Source(remote/local)
Metrics
Cowboy
Exmagick(Graphmagick NIF package)
Cachex, MemcacheClient
HttpPoison (Web Client)
Metrex (Metrics)

5. General Internal Process Flow
server
cache
(r)
storage
thumb
cache
(w)

6. Defining Modular & Functional Needs
defmodule ImgOut.CacheInterface do
@callback read(charlist) :: {:ok, binary} | charlist
@callback write(charlist, binary) :: {:ok, binary}
end
defmodule ImgOut.StorageInterface do
@callback read({:ok, binary})
@callback read({:error, integer, map})
@callback read(charlist)
end
defmodule ImgOut.ImageInterface do
@callback thumb({:error, integer, map}, any)
@callback thumb({:ok, binary}, map)
end
defmodule MicroserviceController do
def generate_thumb(id, %{} = dimensions) do
id
|> Cache.read
|> Storage.read
|> Image.thumb(dimensions)
end
end
defmodule AlternativeApproachController do
def generate_thumb(id, %{} = dimensions) do
id
|> Storage.read
|> Image.thumb(dimensions)
end
end

7. Implement CacheService using CacheInterface
defmodule ImgOut.CacheInterface do
@callback read(charlist) :: {:ok, binary} | charlist
@callback write(charlist, binary) :: {:ok, binary}
end
defmodule ImgOut.CacheService do
@behaviour ImgOut.CacheInterface
def read(key) do
response = Memcache.Client.get(key)
case response.status do
:ok -> {:ok, response.value}
_ -> key
end
end
def write(key, val) do
Memcache.Client.set(key, val)
{:ok, val}
end
end

8. Sample Directory Structure
app
- lib
- interfaces
- cache_interface.ex
- ...
- services
- cache_service.ex
- …
- imgout.ex

9. General Internal Process Flow
server
cache
(r)
storage
thumb
cache
(w)

10. How to Make Cache.write Async?
server
cache
(r)
storage
thumb
cache
(w)
Task.start(..)
defmodule ImgOut.CacheService do
...
# way 1: to make async write to cache
def write(key, val) do
Task.start(fn -> Memcache.Client.set(key, val) end)
{:ok, val}
end
end

11. Anatomy of an Elixir Process
What is an Erlang/Elixir Process?

12. An actor (Elixir/Erlang Process)
STATE
Mailbox
CALCULATION FUNCTIONS
(MSG LISTENERS)
@mustafaturan

13. Erlang Virtual Machine
@mustafaturan
OS
Process (1)
Process (2)
Process (3)
….
Process (n)
Erlang VM
-- pid 109
-- pid 206
-- pid 3114
-- ...
STATE
Mailbox
CALCULATION FUNCTIONS
(MSG LISTENERS)
(pid 109)

14. Fault Tolerance Systems
Erlang/Elixir Supervision Tree Strategies

15. Supervision Tree Strategies
:one_for_one
S
WW W
S
WW W
S
WW W
if one worker dies respawn it
down signal restart child
@mustafaturan

16. Supervision Tree Strategies
:one_for_all
@mustafaturan
S
WW W
S
WW W
S
WW W
if one worker dies
supervisor
respawns all
down signal restart all
S
WW
supervisor
kills rest of the workers

17. Supervision Tree Strategies
:rest_for_one
@mustafaturan
S
W
2
W
3
W
1
S
W
2
W
3
W
1
S
W
2
W
3
W
1
if one worker dies
supervisor
respawns all killed
down signal
restart all killed
workers
supervisor
kills rest of the workers
with start order (not W1)
Note: Assumed start order of workers are like W1, W2, W3 and W4
W
4
W
4
W
4
S
W
3
W
1
W
4
W
4

18. Supervision Tree Strategies
:simple_one_for_one
Same as :one_for_one
- Needs to implement Supervision.Spec
- You need to specify only one entry for a child
- Every child spawned by this strategy is same kind of process, can not be mixed.

19. With Named GenServer &
Supervision Tree
* Creating Elixir Processes with GenServer** and Supervising
** Process discovery with ‘name’ option

20. Building Our Supervision Tree
A
(S)
Srv
(S)
Im
(S)
Str
(S)
Ch
(S)
Str
(w)
Im
(w)
Srv
(w)
Ch
(w)
defmodule ImgOut do
use Application
def start(_type, _args) do
import Supervisor.Spec, warn: false
children = [
supervisor(ImgOut.WebServerSupervisor, []),
supervisor(ImgOut.ImageSupervisor, []),
supervisor(ImgOut.StorageSupervisor, [])
]
opts = [strategy: :one_for_one, name:
ImgOut.Supervisor]
Supervisor.start_link(children, opts)
end
end

21. Sample Code For CacheSupervisor
...
@doc false
def init([]) do
children = [
worker(ImgOut.CacheWorker, [])
]
opts = [strategy: :one_for_one, name: __MODULE__]
supervise(children, opts)
end
end
defmodule ImgOut.CacheSupervisor do
use Supervisor
@doc false
def start_link,
do: Supervisor.start_link(__MODULE__, [], name:
__MODULE__)
...

22. Sample Code For CacheWorker
...
def init(_opts),
do: {:ok, []}
## Private API
@doc false
def handle_call({:read, key}, _from, state),
do: {:reply, ImgOut.CacheService.read(key), state}
def handle_cast({:write, key, val}, state) do
ImgOut.CacheService.write(key, val)
{:noreply, state}
end
end
defmodule ImgOut.CacheWorker do
use GenServer
## Public API
def read(key),
do: GenServer.call(__MODULE__, {:read, key})
def write(key, val) do
GenServer.cast(__MODULE__, {:write, key, val})
{:ok, val}
end
def start_link,
do: GenServer.start_link(__MODULE__, [], name:
__MODULE__)
...

23. Sample Directory Structure
app
- lib
- interfaces
- cache_interface.ex
- ...
- services
- cache_service.ex
- ...
- supervisors
- cache_supervisor.ex
- ...
- workers
- cache_worker.ex
- ...

24. How is the Flow
server
cache
(r)
storage
thumb
cache
(w)

25. Problem: Long Running Processes
server
storage
+
cache
thumb
timeouts
Thumbnail generation
- A calculation
- Takes too much time to process

26. Solution: Spawning More Workers On Demand
…
def start_link,
do: GenServer.start_link(__MODULE__, [], name: __MODULE__)
…
def thumb({:ok, img}, dimensions),
do: GenServer.call(__MODULE__, {:thumb, {:ok, img}, dimensions})
…
def handle_call({:thumb, {:ok, img}, dimensions}, _from, state) do
data = ImgOut.ImageService.thumb({:ok, img}, dimensions)
{:reply, data, state}
end
…
Cons:
Spawn 1 Process Per GenServer
Pros:
No need to store pid to call process funcs

27. Solution: Spawning More Workers On Demand
Possible Solution to spawn more workers on GenServer:
- Process Registry
- We can save pid on creation and deregister on exit
- Too manual work
- Reinventing wheels(special for this service)
- Gproc Package
- Supports pooling
- Has advanced techniques to register, deregister, monitor etc...
- Poolboy Package
- Supports pooling
- Easy

28. With Partial Pooled GenServer &
Supervision Tree
* Creating Elixir Processes with GenServer** and Supervising
** Process discovery with worker pool option (poolboy)

29. Poolboy Configuration For ImageWorker
# prod.exs
config :imgout,
gm_pool_size: (System.get_env("GM_POOL_SIZE") || "25")
|> String.to_integer,
gm_pool_max_overflow:
(System.get_env("GM_POOL_MAX_OVERFLOW") || "0") |>
String.to_integer,
gm_timeout: (System.get_env("GM_TIMEOUT") || "5000") |>
String.to_integer
# dev.exs, test.exs
config :imgout,
gm_pool_size: 25,
gm_pool_max_overflow: 0,
gm_timeout: 5000

30. Sample Code For ImageSupervisor with Poolboy
def init([]) do
worker_pool_options = [
name: {:local, :gm_worker_pool},
worker_module: ImgOut.ImageWorker,
size: @pool_size,
max_overflow: @pool_max_overflow
]
children = [
:poolboy.child_spec(:gm_worker_pool,
worker_pool_options, [])
]
opts = [strategy: :one_for_one, name: __MODULE__]
supervise(children, opts)
end
end
defmodule ImgOut.ImageSupervisor do
use Supervisor
@pool_size Application.get_env(:imgout, :gm_pool_size)
@pool_max_overflow Application.get_env(:imgout,
:gm_pool_max_overflow)
def start_link,
do: Supervisor.start_link(__MODULE__, [], name:
__MODULE__)
…

31. Sample Code For ImageWorker with Poolboy
def start_link(_opts),
do: GenServer.start_link(__MODULE__, :ok, [])
@doc false
def init(_opts) do
{:ok, []}
end
def handle_call({:thumb, {:ok, img}, dimensions},
_from, state) do
data = ImgOut.ImageService.thumb({:ok, img},
dimensions)
{:reply, data, state}
end
end
defmodule ImgOut.ImageWorker do
use GenServer
@behaviour ImgOut.ImageInterface
@timeout Application.get_env(:imgout, :gm_timeout)
def thumb({:ok, img}, dimensions) do
:poolboy.transaction(:gm_worker_pool, fn(worker) ->
GenServer.call(worker, {:thumb, {:ok, img},
dimensions}, @timeout)
end)
end
def thumb({:error, status, reason}, _),
do: {:error, status, reason}
…

32. Solution: Spawned (n)Thumb Process
server
storage
+
cache
thumb
no timeouts
Spawned Multiple ImageWorker(s) with Poolboy
- Process Registry handled by poolboy
- We can change max, min spawned processes
thumb
thumb
thumb

33. Problem: Too many request to storage and cache worker
server
storage
+
cache
thumb
no timeouts
Since we solved the timeout problem for Thumbnail processor
- Now storage and cache worker getting too many request
- but not processing that fast with 1 instance!
thumb
thumb
thumb
timeouts

34. Solution: Spawning More Workers On Demand
Possible Solution to spawn more workers on GenServer:
- Process Registry
- We can save pid on creation and deregister on exit
- Too manual work
- Reinventing wheels(special for this service)
- Gproc Package
- Supports pooling
- Has advanced techniques to register, deregister, monitor etc...
- Poolboy Package
- Supports pooling
- Easy

35. With Fully Pooled GenServer &
Supervision Tree
* Creating Elixir Processes with GenServer** and Supervising
** Process discovery with worker pool option (poolboy)

36. Why Prefer Pooling Against Free Spawning?
Memcache
- Connection limit
- If you hit connection limit, you can’t get positive response
Storage
- Remote storage servers might have some limitation on requests
- Req/per second
- If you hit more, you will get error from remote

37. Metrics
A very useful for microservices and system health checks,
determining bottlenecks

38. Tracking Heartbeats of Your Elixir Processes
Metrex Package
- Creating new metric dynamically
- Incrementing metric
- TTL
- Dumping metric data
- Init and exit hooks
Results:
- 3260 req/min on Heroku Free Dyno
- http://bit.ly/2bYRnpp
config :metrex,
counter: ["all"],
meters: ["cache", "storage", "thumb"],
ttl: 900
# Dynamically create a metric
Metrex.start_meter("pageviews")
# Increment a meter by 1
Metrex.Meter.increment("pageviews")
# Increment a meter by x(number)
Metrex.Meter.increment("pageviews", 5)

39. Tips & Source Code

40. Libraries
Metrex
- Has implementation for counter and meter patterns
Exmagick
- NIF package which means you can spawn graphicsmagick as Elixir Process
Poolboy
Cowboy
Plug

41. Source
Github:
https://github.com/mustafaturan/imgout (Heroku Deploy Button available / Has 3
branches for all 3 approaches)
https://github.com/mustafaturan/metrex

42. QUESTIONS

43. THANK YOU