Concurrent and Distributed Applications with Akka, Java and Scala

1. Concurrent and Distributed Applications
with
Akka, Java and Scala
!
Buenos Aires, Argentina, Oct 2012
!
@frodriguez

2. Moore’s law
Moore's law says that every 18 months,
the number of transistors that can fit within a
given area on a chip doubles.

3. Moore’s law
Moore's law says that every 18 months,
the number of transistors that can fit within a
given area on a chip doubles.
Page's law says that every 18 months
software becomes twice as slow

4. Modern CPUs
intel ®
Xeon
®
processor

5. Modern CPUs
intel ®
Xeon
®
processor
Small improvements
in
execution speed

6. intel ®
Xeon
®
processor
Small improvements
in
execution speed
but...
Modern CPUs

7. Modern CPUs
Core Core Core Core
Cor
Core
Core
Core
Core
Core
Core
Core
Core Core
Core Core
Small improvements
in
execution speed
but...
Improved
parallelism support
Multicore processors

8. Alternative Moore’s & Page’s laws
Every 18 months,
the number of cores per chip doubles.

9. Alternative Moore’s & Page’s laws
Every 18 months,
the number of cores per chip doubles.
Every 18 months the number of IDLE
cores doubles

10. Modern Applications Demands...

11. Modern Applications Demands...
Scalability

12. Modern Applications Demands...
Scalability
Vertical (scaling up) - Concurrency
Horizontal (scaling out) - Distribution

13. Modern Applications Demands...
Scalability
Vertical (scaling up) - Concurrency
Horizontal (scaling out) - Distribution
Size (Resources, Users, ...)
Geographically
Admin / Management

14. Modern Applications Demands...

15. Modern Applications Demands...
High Availability
Fault Tolerance (local & distributed)

16. Modern Applications Demands...
High Availability
Fault Tolerance (local & distributed)
Performance & Low Latency
Efficient Use of Resources (CPU & Memory)
Asynchronous (reduced latency)
Distributed (cheap machines, better performance / $)

17. How to Write
Concurrent Applications
for
Multicore Processors ?

18. Traditional Approach
Threads (pools processing tasks)
Shared (and mutable) State
Synchronization and Locking
Concurrent Collections

19. Traditional Threads ?

20. Traditional Threads ?
process(...){
}

21. Traditional Threads ?
process(...){
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
}

22. Traditional Threads ?
process(...){
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
}

23. Traditional Threads ?
process(...){
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Blocked
Updating State in the Heap
Returning Results
}
Thread
Suspended

24. Traditional Threads ?
process(...){
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
}

25. Traditional Threads ?
process(...){
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
}

26. Traditional Threads ?
process(...){
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
}
Add concurrency...

27. Traditional Threads ?
process(...){
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
}

28. Traditional Threads ?
process(...){
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
Requires
Synchronization
can be blocked
Requires
Synchronization
can be blocked
}

29. Traditional Threads ?
process(...){
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
Requires
Synchronization
can be blocked
Requires
Synchronization
can be blocked
}
Bad for
CPU caches

30. Traditional Threads ?
How many threads ?
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results

31. Traditional Threads ?
How many threads ?
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results

32. Traditional Threads ?
How many threads ?
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
Improves with threads
(Assuming blocking,
non-async I/O is used...)

33. Traditional Threads ?
How many threads ?
Computing
Reading State from Heap
I/O (e.g: Disk, Network, DBs)
Processing Results
Updating State in the Heap
Returning Results
Degrades with more
threads than cores
Context Switching,
Contention,
L1 & L2 Caches

34. What About Latency ?
Client
Biz
DB
Fetching
and
Mapping N Items
Latency (time to first item)
Thread by task (instead of by layer). Sync Results

35. What About Latency ?
Client
Biz
DB
Fetching
and
Mapping N Items
Latency (time to first item)
Parallelism by Layer - Asynchronous and Partial Results
From Request/Response to Request Stream/Response Stream

36. How to Write
Distributed Applications ?

37. Traditional Approach
RPC (WS, RMI, ...)
Queues (JMS, AMQP, STOMP, etc),
Raw Sockets

38. Traditional Approach
RPC (WS, RMI, ...)
Queues (JMS, AMQP, STOMP, etc),
Raw Sockets
Local != Remote
Local should be an optimization,
not a forced early decision...

39. Akka
“Akka is a toolkit and runtime for
building highly concurrent,
distributed, and fault tolerant event-driven
applications on the JVM. ”
Based on the actor model

40. What is an Actor ?
Actors are objects which
encapsulate state and behavior
Communicate exclusively by
exchanging messages
Conceptually have their own
light-weight thread
No Need for Synchronization

41. Actors
Actor
State
Behavior

42. Actors
Actor
State
Behavior
Mailbox

43. Actors
Actor
State
Behavior
C B A
Mailbox

44. Actors
Actor
State
Behavior
C B A
Mailbox

45. Actors
Actor
State
Behavior
C B A
Mailbox

46. Actors
Actor
State
BehaAvior
C B
Mailbox

47. Actors
Actor
State
Behavior
C B
Mailbox

48. Actors
Actor
State
Behavior
C B
Mailbox

49. Actors
Actor
State
Behavior
C B
Millions of
Actors
Mailbox
One per Actor

50. Actors
Actor
State
Behavior
Number of
Threads
Proportional to
number of
Cores
C B
Millions of
Actors
Mailbox
One per Actor

51. Actors: Processing Messages
/myactor
State
Behavior
A

52. Actors: Processing Messages
/myactor
State
BehaAvior

53. Actors: Processing Messages
/myactor
New State
State
Behavior
Change State

54. Actors: Processing Messages
/myactor
State
NBeewh Baevhaioviorr
Change State
Change Behavior

55. Actors: Processing Messages
/myactor
State
Behavior
/someactor
State
B Behavior
Change State
Change Behavior
Send a Message

56. Actors: Processing Messages
/myactor
State
Behavior
/someactor
State
Behavior
B
Change State
Change Behavior
Send a Message

57. Actors: Processing Messages
/myactor
State
Behavior
/someactor
State
BehaBvior
Change State
Change Behavior
Send a Message

58. Actors: Processing Messages
/myactor
State
Behavior
/someactor
State
Behavior
/myactor/child
State
Behavior
Change State
Change Behavior
Send a Message
Create Actors

59. Hello World Actor
Define
class HelloWorld extends Actor {
def receive = {
case msg =>
printf(“Received %sn”, msg)
}
}
Create
val system = ActorSystem(“MySystem”)
val hello = system.actorOf(Props[HelloWorld], “hello”)
Send Message
hello ! “World”

60. Counter Actor
Define
class Counter extends Actor {
var total = 0
!
def receive = {
case Count(value) =>
total += value
case GetStats =>
sender ! Stats(total)
}
}
Protocol
case class Count(n: Int)
case class Stats(total: Int)
case object GetStats

61. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior

62. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior
actorB ! A

63. Sending a Message
/actorB
State
Behavior
/actorA
State
BehAavior
actorB ! A

64. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior
A
actorB ! A

65. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior A
actorB ! A

66. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior A
actorB ! A sender ! B

67. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior
B
actorB ! A sender ! B

68. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior
actorB ! A sender ! B
B

69. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior
B
actorB ! A sender ! B

70. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior
actorB ! A sender ! B

71. Sending a Message
/actorB
State
Behavior
/actorA
State
Behavior
actorB ! A
actorB tell A
sender ! B
sender tell B

72. Sending a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior

73. Sending a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB tell (A, actorC)

74. Sending a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
BehAavior
actorB tell (A, actorC)

75. Sending a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
A
actorB tell (A, actorC)

76. Sending a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior A
actorB tell (A, actorC)

77. Sending a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior A
actorB tell (A, actorC) sender ! B

78. Sending a Message
/actorB
State
Behavior
B
/actorC
State
Behavior
/actorA
State
Behavior
actorB tell (A, actorC) sender ! B

79. Sending a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB tell (A, actorC) sender ! B
B

80. Sending a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB tell (A, actorC) sender ! B
B

81. Sending a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB tell (A, actorC) sender ! B

82. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior

83. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB ! A

84. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
BehAavior
actorB ! A

85. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
A
actorB ! A

86. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior A
actorB ! A

87. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior A
actorB ! A actorC forward B

88. Forward a Message
/actorB
State
Behavior
B
/actorC
State
Behavior
/actorA
State
Behavior
actorB ! A actorC forward B

89. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB ! A actorC forward B
B

90. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB ! A actorC forward B
B

91. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB ! A actorC forward B
sender ! C B

92. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB ! A actorC forward B
sender ! C C

93. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB ! A actorC forward B
sender ! C
C

94. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
C
actorB ! A actorC forward B
sender ! C

95. Forward a Message
/actorB
State
Behavior
/actorC
State
Behavior
/actorA
State
Behavior
actorB ! A actorC forward B
sender ! C

96. Ask & Pipe Patterns
Ask
val response = actor ? Message
!
response onSuccess {
case Response(a) =>
printf(“Response %s”, a)
}
Pipe
val response = actor ? Message
!
response pipeTo actor2

97. Mailbox
UnboundedMailbox (default)
UnboundedPriorityMailbox
BoundedMailbox (*)
BoundedPriorityMailbox (*)
* May produce deadlocks if used unproperly

98. Routing
Round Robin Router
val actor = system.actorOf(
Props[MyActor].withRouter(RoundRobinRouter(4)),
name = “myrouter”
)
Using actor with routers (no changes)
!
actor ! Message

99. Routing
RoundRobinRouter
RandomRouter
SmallestMailboxRouter
BroadcastRouter
ScatterGatherFirstCompletedRouter

100. Routing Configuration
Configuration overrides code
akka.actor.deployment {
/myrouter {
router = round-robin
nr-of-instances = 8
}
}
Routers from Config
val actor = system.actorOf(
Props[MyActor].withRouter(FromConfig()),
name = “myrouter”
)

101. Remoting
Accessing remote actor
val actor = system.actorFor(
“akka://sys@server:2552/user/actor”
)
Using remote actor (no changes)
!
actor ! Message
!
// Replies also work ok
sender ! Response

102. Remote Deployment
Code without changes
val actor = system.actorOf(
Props[MyActor],
name = “myactor”
)
Configuration
!
akka.actor.deployment {
/myactor {
remote = “akka://sys@server:2553”
}
}

103. Remote Deployment (routers)
akka.actor.deployment {
/myrouter {
router = round-robin
nr-of-instances = 8
!
target {
nodes = [“akka://sys@server1:2552”
“akka://sys@server2:2552”]
}
}
}
Routers from Config
val actor = system.actorOf(
Props[MyActor].withRouter(FromConfig()),
name = “myrouter”
)

104. Fault Tolerance
override val supervisorStrategy = OneForOneStrategy(...)
{
case _: ArithmeticException => Resume
case _: NullPointerException => Restart
case _: IllegalArgumentException => Stop
case _: Exception => Escalate
}
Supervision Hierarchies across machines

105. Thanks
@frodriguez
frodriguez <at> gmail.com