Effect monads like IO are the way functional programmers interact with the real world. Yet, monadic effects in programming languages like Scala often perform poorly compared to their Haskell counterparts—as much as 10x slower in some benchmarks. In this presentation, John A. De Goes, author of the Scalaz 8 effect system, dredges up an old paper to cast new light on the question of how to model effects, and comes to the surprising conclusion that in Scala, monads may not be the fastest way to model purely functional effects. Join John as he shows a new model of effects that offers performance improvements without sacrificing the wonderful purity that functional programmers rely on to reason about their software.

1. 1
Blazing Fast, Pure
Effects without
Monads
LambdaConf 2018
By John A. De Goes — @jdegoes

2. 2
1991 1998 2018 - April
Eugenio Moggi John Hughes Flavio Brasil
F R O M M O N A D S T O A R R O W S
A brief history of effects leading to KleisliIO.
KleisliIO
2018 - June
Notions of
computation
and monads
Generalizing
monads to
arrows
TraneIO:
Arrows & tasks
in Scala

3. 3
THE TRINITY OF FP
Total
Deterministic
Free of Side-effects

4. 4
THE TRINITY OF FUNCTIONAL PROGRAMMING
Total
Deterministic
Free of Side-effects
f : A => B
a ∈ A ⇒ f(a) ∈ B

5. 5
THE TRINITY OF FUNCTIONAL PROGRAMMING
Total
Deterministic
Free of Side-effects
def notTotal1(a: Int): String = null
def notTotal2(a: Int): String =
throw new Error(" ")
def notTotal3(a: Int): String = notTotal3(a)

6. 6
THE TRINITY OF FUNCTIONAL PROGRAMMING
Total
Deterministic
Free of Side-effects
f : A => B
b1 = f(a) ∧ b2 = f(a) ⇒ b1 = b2

7. 7
THE TRINITY OF FUNCTIONAL PROGRAMMING
Total
Deterministic
Free of Side-effects
def notDeterministic1(a: Int): String =
if (Math.random() > 0.5) "Hello"
else "Goodbye"
def notDeterministic2(a: Int): String =
scala.io.StdIn.readLine()
def notDeterministic3(a: Int): String =
(System.nanoTime() + a).toString

8. 8
THE TRINITY OF FUNCTIONAL PROGRAMMING
Total
Deterministic
Free of Side-effects
f : A => B
f’s only computational effect is computing B

9. 9
THE TRINITY OF FUNCTIONAL PROGRAMMING
Total
Deterministic
Free of Side-effects
def notEffectFree1(a: Int): String =
{ println("Hello"); a.toString }
def notEffectFree2(a: Int): String = {
val _ = scala.io.StdIn.readLine()
a.toString
}
def notEffectFree3(a: Int): String =
{ Logger.log(a); (a * a).toString }

10. 10
THE RICHES OF PURITY
Type Reasoning
Equality Reasoning

11. 11
Type Reasoning
Equality Reasoning
def foo1[A](a: A): A
def foo2[A](a: A, f: A => A): A
def foo3[A, B](f: (A => A) => B): B
THE RICHES OF PURITY

12. 12
Type Reasoning
Equality Reasoning
def println(line: String): Unit
def readLine(): String
THE RICHES OF PURITY

13. 13
Type Reasoning
Equality Reasoning
def f = (x: Int) => x * x
val b = f(a)
val c = b + b
// c = b + b
// c = f(a) + f(a)
// c = (a * a) + (a * a)
// c = 2 * a * a
THE RICHES OF PURITY

14. 14
Type Reasoning
Equality Reasoning
def readLine = scala.io.StdIn.readLine()
val a = readLine
val b = a + a
// b = a + a
// b = readLine + readLine
// !?!?! readLine + readLine !=
// { val a = readLine; a + a }
THE RICHES OF PURITY

15. 15
MONADIC EFFECTS
Effects Into Values
Values Into Effects
The Cost of Value Effects

16. 16

17. 17
EFFECTS INTO VALUES
def println(line: String): Unit
println
IO[Unit]
String
def println(line: String): IO[Unit]

18. 18
EFFECTS INTO VALUES
IO[A]
Immutable value produced by the effect
Immutable value describing the effect

19. 19
EFFECTS INTO VALUES
sealed trait IO[A] {
def map[B](f: A => B): IO[B] =
flatMap((a: A) => IO.point(f(a)))
def flatMap[B](f: A => IO[B]): IO[B] =
FlatMap(this, f)
}
object IO {
def point[A](a: A): IO[A] = Point(a)
}
final case class PrintLn(line: String) extends IO[Unit]
final case class ReadLine() extends IO[String]
final case class FlatMap[A, B](fa: IO[A], f: A => IO[B]) extends IO[B]
final case class Point[A](a: A) extends IO[A]

20. 20
def readLine(): String
def println(line: String): Unit
val name = readLine()
println("Hello " + name + ", how are you?")
EFFECTS INTO VALUES
println
IO[Unit]
def readLine: IO[String]
def println(line: String): IO[Unit]
val program =
for {
name <- readLine
_ <- println("Hello " + name + ", how are you?")
} yield ()
readLine
IO[String]
String
Unit

21. 21
VALUES INTO EFFECTS
def unsafePerformIO(io: IO[A]): A = io match {
case PrintLn(line) => println(line)
case ReadLine() => readLine()
case FlatMap(fa, f) =>
unsafePerformIO(f(unsafePerformIO(fa)))
case Point(a) => a
}

22. 22
THE COST OF VALUE EFFECTS
println
F[Unit]
class SS {
for {
name <- readLine
_ <- println("Hello " + name +
", how are you?")
} yield ()
readLine
F[String]
String
Unit

23. 23
THE COST OF VALUE EFFECTS
readLine.flatMap(name =>
printLn("Hello, " + name +
", how are you?"))
Allocations
Allocation
Megamorphic
Dispatch

24. 24
THE COST OF VALUE EFFECTS
1 Statement in Procedural
Programming
4 Extra Allocations, 1 Extra
Megamorphic Dispatch in
Functional Programming
=

25. 25
IO.sync {
App.main()
}
THE COST OF VALUE EFFECTS
Monolith Composed
Pure Values

26. 26
THE COST OF VALUE EFFECTS
Future

27. 27
KLEISLI EFFECTS
Effects Into Values
Values Into Effects
The Cost of Value Effects

28. 28

29. 29
EFFECTS INTO VALUES
def println(line: String): Unit
println
FunctionIO[String, Unit]
val println: FunctionIO[String, Unit]

30. 30
EFFECTS INTO VALUES
FunctionIO[A, B]
Immutable input value to the function
Immutable value describing the effectful function
Immutable output value from the function

31. 31
EFFECTS INTO VALUES
final case class FunctionIO[A, B](apply0: A => IO[B])
extends (A => IO[B]) {
def apply(a: A): IO[B] = apply0(a)
def andThen[C](f: FunctionIO[B, C]): FunctionIO[A, C] =
FunctionIO(a => apply(a).flatMap(f.apply))
}
object FunctionIO {
def lift[A, B](f: A => B): FunctionIO[A, B] =
FunctionIO(IO.point.compose(f))
}

32. 32
def readLine(): String
def println(line: String): Unit
val name = readLine()
println("Hello " + name + ", how are you?")
EFFECTS INTO VALUES
val readLine: FunctionIO[Unit, String]
val println: FunctionIO[String, Unit]
val program =
readLine
.andThen(lift((name: String) => "Hello, " + name +
", how are you?"))
.andThen(println)
readLine
FunctionIO[Unit, String]
<anonymous>
FunctionIO[String,String]
println
FunctionIO[String, Unit]

33. 33
VALUES INTO EFFECTS
val program: FunctionIO[Unit, Unit]
unsafePerformIO(program())

34. 34
THE COST OF EFFECT VALUES
readLine.andThen(lift((name: String) =>
"Hello, " + name + ", how are you?"))
.andThen(println)
readLine
FunctionIO[Unit, String]
<anonymous>
FunctionIO[String,String]
println
FunctionIO[String, Unit]

35. 35
Megamorphic
Dispatches
final case class FunctionIO[A, B](apply: A => IO[B])
THE COST OF EFFECT VALUES
readLine.andThen(println)
Allocations Allocations

36. 36
THE COST OF EFFECT VALUES
1 Statement in Procedural
Programming
6 Extra Allocations, 3 Extra
Megamorphic Dispatches in
Functional Programming
=

37. 37
THE PROMISE OF KLEISLIIO
sealed trait FunctionIO[A, B] extends (A => IO[B]) { ... }
final class Pure[A, B](val apply0: A => IO[B])
extends FunctionIO[A, B] {
def apply(a: A): IO[B] = apply0(a)
}
final class Impure[A, B](val apply0: A => B)
extends FunctionIO[A, B] {
def apply(a: A): IO[B] = IO.point(apply0(a))
}
val readLine: FunctionIO[Unit, String] =
new Impure(_ => scala.io.StdIn.readLine())
val printLn: FunctionIO[String, Unit] = new Impure(println)

38. 38
THE PROMISE OF KLEISLIIO
class Impure[A, B](val apply0: A => B)
1 Invocation in Procedural
Programming
0 Extra Allocations,
1 Extra Dispatch in
Functional Programming
=

39. 39
KLEISLIIO
Running KleisliIO
Constructing KleisliIO
Composing KleisliIO
Introducing KleisliIO
Optimizing KleisliIO
Testing KleisliIO

40. 40
KLEISLIIO
sealed trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... }
object KleisliIO {
class KleisliIOError[E](error: E) extends Throwable {
def unsafeCoerce[E2] = error.asInstanceOf[E2]
}
class Pure[E, A, B](apply0: A => IO[E, B]) extends KleisliIO[E, A, B] {
override final def apply(a: A): IO[E, B] = apply0(a)
}
class Impure[E, A, B](val apply0: A => B) extends KleisliIO[E, A, B] {
override final def apply(a: A): IO[E, B] =
IO.suspend {
try IO.now[E, B](apply0(a))
catch {
case e: KleisliIOError[_] => IO.fail[E, B](e.unsafeCoerce[E])
}
}
}
...
}

41. 41
RUNNING KLEISLIIO
val printLn: KleisliIO[IOException, String, Unit]
printLn("Hello, world!") // IO[IOException, Unit]

42. 42
CONSTRUCTING PURE KLEISLIIO
object KleisliIO {
...
def pure[E, A, B](f: A => IO[E, B]):
KleisliIO[E, A, B] = ???
...
}
val printLn: KleisliIO[Void, String, Unit] =
KleisliIO.pure((a: String) => IO.sync(println(a)))

43. 43
CONSTRUCTING IMPURE KLEISLIIO
object KleisliIO {
...
def impure[E, A, B](catcher: PartialFunction[Throwable, E])(f: A => B)
...
}
val IOExceptions: PartialFunction[Throwable, IOException] = {
case io : IOException => io
}
val printLn: KleisliIO[Void, String, Unit] =
KleisliIO.impure(IOExceptions)(println)

44. 44
CONSTRUCTING IMPURE KLEISLIIO
object KleisliIO {
...
def impureVoid[A, B](f: A => B): KleisliIO[Void, A, B] = ???
...
}
val printLn: KleisliIO[Void, String, Unit] = KleisliIO.impureVoid(println)

45. 45
CONSTRUCTING KLEISLIIO
object KleisliIO {
...
def identity[E, A]: KleisliIO[E, A, A] = ???
...
}
A A
KleisliIO.identity[Void, Int](1) // IO.point(1)

46. 46
COMPOSING KLEISLIIO
trait KleisliIO[E, A, B] extends (A => IO[E, B])
{
...
def >>>[C](that: KleisliIO[E, B, C]):
KleisliIO[E, A, C] = ???
...
}
...
A B C
A C
readLine >>> printLn

47. 47
COMPOSING KLEISLIIO
A
B
C
(B, C) => D
A D
trait KleisliIO[E, A, B] extends (A => IO[E, B]) {
...
def zipWith[C, D](that: KleisliIO[E, A, C])(f: (B, C) => D):
KleisliIO[E, A, D] = ???
...
}
readLine.zipWith(readLine)((l1, l2) => l1 + l2)

48. 48
COMPOSING KLEISLIIO
trait KleisliIO[E, A, B] extends (A => IO[E, B]) {
...
def &&&[C]( that: KleisliIO[E, A, C]):
KleisliIO[E, A, (B, C)] = ???
...
}
A
B
C
A (B,C)
readLine &&& readLine

49. 49
COMPOSING KLEISLIIO
trait KleisliIO[E, A, B] extends (A => IO[E, B]) {
...
def |||[C](that: KleisliIO[E, C, B]):
KleisliIO[E, Either[A, C], B]
...
}
A
B
C
Either[A, C] D
(fancyPrintLn |||
standardPrintLn)(Left("Fancied!"))

50. 50
COMPOSING KLEISLIIO
trait KleisliIO[E, A, B] extends (A => IO[E, B]) {
...
def first: KleisliIO[E, A, (B, A)] =
this &&& KleisliIO.identity[E, A]
...
}
A B
A (B, A)
readLine >>> printLn.first // (Unit, String)

51. 51
COMPOSING KLEISLIIO
trait KleisliIO[E, A, B] extends (A => IO[E, B]) {
...
def second: KleisliIO[E, A, (A, B)] =
KleisliIO.identity[E, A] &&& this
...
}
A B
A (A, B)
readLine >>> printLn.second // (String, Unit)

52. 52
COMPOSING KLEISLIIO
trait KleisliIO[E, A, B] extends (A => IO[E, B]) {
...
def left[C]:
KleisliIO[E, Either[A, C], Either[B, C]] = ???
...
}
Either[A, C] Either[B, C]
A B
printLn.left[String]( Left("Hello")) //
Left[Unit]

53. 53
COMPOSING KLEISLIIO
trait KleisliIO[E, A, B] extends (A => IO[E, B]) {
...
def right[C]:
KleisliIO[E, Either[A, C], Either[B, C]] = ???
...
}
Either[A, C] Either[B, C]
A B
printLn.right[String]( Right("Hello")) // Right[Unit]

54. 54
COMPOSING KLEISLIIO
trait KleisliIO[E, A, B] extends (A => IO[E, B])
{
...
def asEffect: KleisliIO[E, A, A] =
self.first >>> KleisliIO._2
...
}
A A
B
printLn.asEffect // String

55. 55
COMPOSING KLEISLIIO
object KleisliIO {
...
def test[E, A](cond: KleisliIO[E, A, Boolean]):
KleisliIO[E, A, Either[A, A]] = ???
...
}
test(KleisliIO.lift(_ > 2))(4) // IO[Void, Either[Int,
Int]]
cond
A Bool
A Either[A, A]
Right(a)Left(a)

56. 56
COMPOSING KLEISLIIO
object KleisliIO {
...
def ifThenElse[E, A, B](cond : KleisliIO[E, A, Boolean])
(then0: KleisliIO[E, A, B])(else0: KleisliIO[E, A, B]):
KleisliIO[E, A, B] =
test[E, A](cond) >>> (then0 ||| else0)
...
}
ifThenElse(KleisliIO.lift(_ == "John"))(printLn)(const(()))
A B
else0 then0
A BA B
cond
A Bool

57. 57
COMPOSING KLEISLIIO
object KleisliIO {
...
def whileDo[E, A](check: KleisliIO[E, A, Boolean])
(body : KleisliIO[E, A, A]): KleisliIO[E, A, A] = ???
...
}
A A
A Boolean
A Abody
check
readLine >>> whileDo(lift(_ != “John”)) {
KleisliIO.point(“Wrong name”) >>> printLn >>> readLine
}

58. 58
OPTIMIZING KLEISLIIO
final def compose[E, A, B, C](second: KleisliIO[E, B, C], first: KleisliIO[E, A, B]):
KleisliIO[E, A, C] =
(second, first) match {
case (second: Impure[_, _, _], first: Impure[_, _, _]) =>
new Impure(second.apply0.compose(first.apply0))
case _ =>
new Pure((a: A) => first(a).flatMap(second))
}

59. 59
OPTIMIZING KLEISLIIO
final def ifThenElse[E, A, B](cond: KleisliIO[E, A, Boolean])
(then0: KleisliIO[E, A, B])(else0: KleisliIO[E, A, B]): KleisliIO[E, A, B] =
(cond, then0, else0) match {
case (cond: Impure[_, _, _], then0: Impure[_, _, _], else0: Impure[_, _, _]) =>
new Impure[E, A, B](a => if (cond.apply0(a)) then0.apply0(a) else else0.apply0(a))
case _ => test[E, A](cond) >>> (then0 ||| else0)
}

60. 60
OPTIMIZING KLEISLIIO
final def whileDo[E, A](check: KleisliIO[E, A, Boolean])(body: KleisliIO[E, A, A]): KleisliIO[E, A, A] =
(check, body) match {
case (check: Impure[_, _, _], body: Impure[_, _, _]) =>
new Impure[E, A, A]({ (a0: A) =>
val cond = check.apply0
val update = body.apply0
var a = a0
while (cond(a)) { a = update(a) }
a
})
case _ =>
lazy val loop: KleisliIO[E, A, A] =
KleisliIO.pure((a: A) =>
check(a).flatMap((b: Boolean) =>
if (b) body(a).flatMap(loop) else IO.now(a)))
loop
}

61. 61
TESTING KLEISLIIO - ARRAY FILL
Output: An array filled with 10,000 elements in increasing order.

62. 62
TESTING KLEISLIIO - ARRAY FILL
def arrayFill(array: Array[Int]): KleisliIO[Void, Int] = {
val condition = KleisliIO.lift(i => i < array.length)
val update = KleisliIO.impureVoid{ i =>
array.update(i, i); i + 1 }
KleisliIO.whileDo(condition)(update)
}
def arrayFill(array: Array[Int])(i: Int): IO[Unit] =
if (i >= array.length) IO.unit
else IO(array.update(i, i)).flatMap(_ => arrayFill(array)(i + 1))
KleisliIO Array Fill Monadic Array Fill

63. 63
TESTING KLEISLIIO - ARRAY FILL
7958.946 ops/s
3622.744 ops/s
3689.406 ops/s
Array Fill
2.18x Faster!

64. 64
TESTING KLEISLIIO - BUBBLE SORT
Input: Array of 10000 element with reversed order
Output: bubbleSort(array)
bubbleSort(array):
i <- 1 to 10000
j <- i + 1 to 9999
lessThanEqual = array(i) <= array(j)
if (!lessThanEqual) swap(array, i, j)

65. 65
TESTING KLEISLIIO - BUBBLE SORT
val sort: KleisliIO[Void, Int, Unit] =
KleisliIO
.whileDo(outerLoopCheck)(
innerLoopStart >>>
KleisliIO.whileDo(innerLoopCheck)(
extractIJIndexValue >>>
KleisliIO.ifNotThen(lessThanEqual)(swapIJ) >>>
extractIJAndIncrementJ
) >>>
extractIAndIncrementI
)
sort(0)
def outerLoop(i: Int): IO[Unit] =
if (i >= array.length - 1) IO.unit
else innerLoop(i, i + 1).flatMap(_ => outerLoop(i + 1))
def innerLoop(i: Int, j: Int): IO[Unit] =
if (j >= array.length) IO.unit
else IO((array(i), array(j))).flatMap {
case (ia, ja) =>
val maybeSwap = if (lessThanEqual0(ia, ja)) IO.unit
else swapIJ(i, ia, j, ja)
maybeSwap.flatMap(_ => innerLoop(i, j + 1))
}
outerLoop(0)
KleisliIO Bubble sort Monadic Bubble sort

66. 66
TESTING KLEISLIIO - BUBBLE SORT
58.811 ops/s
41.545 ops/s
33.229 ops/s
1.57x Faster!
Bubble Sort

67. 67
W H E R E F R O M H E R E
A brief roadmap for KleisliIO.
?
Infinite
Composition
Recursion
Combinator
Low-Cost
Propagation

68. 68
THANK YOU!
Thanks to the staff, volunteers, and speakers of
LambdaConf, and to Wiem Zine El Abidine for
help with the development and
implementation of KleisliIO.
Follow me @jdegoes
Follow Wiem @wiemzin
Join Scalaz at gitter.im/scalaz/scalaz