These are the slides that we used to ignite the conversation with the audience at Hadoop Summit EU. Come over to the Mahout dev list to be part of the ongoing conversation.

1. © 2014 MapR Technologies 1

2. © 2014 MapR Technologies 2

3. © 2014 MapR Technologies 3

4. © 2014 MapR Technologies 4
A typical
encounter with a
potential Mahout
user

5. © 2014 MapR Technologies 5
Which leads us to
the Mahout 1.0
vision

6. © 2014 MapR Technologies 6

7. © 2014 MapR Technologies 7

8. © 2014 MapR Technologies 8

9. © 2014 MapR Technologies 9
Example: Cooccurrence Analysis

10. © 2014 MapR Technologies 10
How often do items co-occur?
// load distributed matrix
val A = drmFromHDFS(...)
// compute co-occurrences
val C = A.t %*% A

11. © 2014 MapR Technologies 11
How often do items co-occur?
// load distributed matrix
val A = drmFromHDFS(...)
// compute co-occurrences
val C = A.t %*% A
Under the covers:
Optimizer rewrites the matrix multiplication and
transpose operations to a TransposeSelf operator
Optimizer chooses from two physical operators for
TransposeSelf

12. © 2014 MapR Technologies 12
Which items co-occur anomalously?
// compute & broadcast number
// of interactions per item
val numInteractions =
drmBroadcast(A.colSums)
// create indicator matrix
val I = C.mapBlock() {
case (keys, block) =>
// allocate sparse block of indicator matrix
val indicatorBlock = sparse(block.nrow, block.ncol)
// compute indicators with loglikelihood ratio test
for (row <- block)
indicatorBlock(row.index,::) = computeLLR(row,numInteractions)
keys -> indicatorBlock
}

13. © 2014 MapR Technologies 13
Runtime
• prototype on Apache Spark
– fast and expressive cluster
computing system
– general computation graphs, in-memory primitives, rich API, interactive
shell
• future: add Stratosphere
– project proposed to
Apache Incubator recently
– similar to Apache Spark, adds data flow optimization and efficient out-
of-core execution

14. © 2014 MapR Technologies 14

15. © 2014 MapR Technologies 15

16. © 2014 MapR Technologies 16
How Does This Apply?

17. © 2014 MapR Technologies 17
How Can I Start?

18. © 2014 MapR Technologies 18
Q&A
@ted_dunning @mapr maprtech
tdunning@mapr.com
Engage with us!
MapR
maprtech
mapr-technologies

19. © 2014 MapR Technologies 20