Smartling meetup - Developing distributed applications with Akka and Akka Cluster

1. Developing distributed applications
with Akka and Akka Cluster

2. What is Akka?

3. • A Scala and Java framework for scalability, fault-tolerance,
concurrency and remoting through actors.
• Inspired by Erlang OTP.
• Developed by Typesafe: https://www.typesafe.com/.

4. Concurrency paradigms

5. • Shared state and locks
• Software Transactional Memory (STM)
• Message-Passing Concurrency (Actors)
• Dataflow Concurrency
• and more…

6. STM

7. Dataflow Concurrency

8. Actors

9. • Originate in a 1973 paper by Carl Hewitt
• Implemented in Erlang
• Encapsulate state and behavior
• Closer to the definition of OO than classes

10. • Implements Message-Passing Concurrency
• Share nothing
• Isolated lightweight processes
• Communicates through messages
• Asynchronous and non-blocking

11. Concurrency model
• No shared state – no synchronization
• Each actor has a mailbox (message queue)
• Non-blocking send
• Blocking receive
• Messages are immutable

12. userActor ! User(“John Doe”)
class UserActor extends Actor {
def receive = {
case User(name) => sender ! “Hi $name”
}
}

13. Dispatchers
sample-dispatcher {
type = Dispatcher
executor = "fork-join-executor"
fork-join-executor {
parallelism-min = 2
parallelism-factor = 2.0
parallelism-max = 10
}
throughput = 100
}

14. Supervision and hierarchy

15. Routers
• Local
• Remote
• Various routing algorithms (round robin, random, consistent hashing
etc)

16. Let’s build a web crawler!

17. 1.Fetch a page
2.Parse the page to get links
3.Check if max crawl depth has been reached and if yes, finish
4.Go to 1 for all parsed links

20. Demo time

21. Going remote
• Everything works using asynchronous message passing which is
good for remoting
• Akka-remoting allows working with remote actors just as if they were
in the same JVM
• Still need to handle additional issues like serialization and handling
potential networking problems

22. Akka Cluster
• Based on DynamoDB clustering
• Completely decentralized, uses gossip protocol for membership and
failure detection
• Cluster aware routers can be used for balancing tasks across the
cluster

23. Cluster aware routers
actor {
deployment {
/crawlerService/crawlWorkers {
router = consistent-hashing-group
nr-of-instances = 100
routees.paths = ["/user/crawlWorker"]
cluster {
enabled = on
allow-local-routees = on
use-role = backend
}
}
}
provider = "akka.cluster.ClusterActorRefProvider"
}

25. Demo time

26. Detecting cycles
• Need to detect link cycles to avoid needless downloads
• Distributed shared state?
• Akka CRDTs can help!

27. CRDTs
• Good performance and scalability, the cost is eventual consistency
• Two main classes: operation based and state based

28. CmRDTs
• Based on messages
• Requires messages to be delivered and processed exactly once
(complex!)
• No need to transfer the whole state

29. CvRDTs
• Based on the object’s state
• Need merge function that must be commutative, associative, and
idempotent

30. Demo time

31. References
• Akka documentation: http://akka.io/
• Good presentation on CRDTs: https://vimeo.com/43903960
• On DynamoDB clustering: http://www.allthingsdistributed.com/files/amazon-dynamo-
sosp2007.pdf

32. Questions?