The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...
Using MapReduce in Hazelcast for Fast Big Data Processing
1. BIG DATA - FAST DATA
USING MAPREDUCE IN HAZELCAST
Source:http://www.newscientist.com/gallery/dn17805-computer-museums-of-the-world/11
www.hazelcast.com
2. WHO AM I
Christoph Engelbert(@noctarius2k)
8+ years of JavaWeirdoness
Performance, GC, traffic topics
Apache DirectMemoryPMC
Previous companies incl. Ubisoftand HRS
CastMapRMapReduce for Hazelcast3
www.hazelcast.com
22. Dataare mapped /transformed in asetof key-value pairs
SOME PSEUDO CODE (1/3)
MAPPING
map( key:String, document:String ):Void ->
for each w:word in document:
emit( w, 1 )
www.hazelcast.com
23. Multiple values are combined to an
intermediate resultto preserve traffic
SOME PSEUDO CODE (2/3)
COMBINING
combine( word:String, counts:List[Int] ):Void ->
emit( word, sum( counts ) )
www.hazelcast.com
24. Values are reduced /aggregated to the requested result
SOME PSEUDO CODE (3/3)
REDUCING
reduce( word:String, counts:List[Int] ):Int ->
return sum( counts )
www.hazelcast.com
25. FOR MATHEMATICIANS
Process: (K x V)*→ (L x W)* ⇒ [(l1, w1), …, (lm, wm)]
Mapping: (K x V) → (L x W)* ⇒ (k, v) → [(l1, w1), …, (ln, wn)]
Reducing: L x W*→ X* ⇒ (l, [w1, …, wn]) → [x1, …,xn]
www.hazelcast.com
26. MAPREDUCE PROGRAMS IN
GOOGLE SOURCE TREE
Source:http://research.google.com/archive/mapreduce-osdi04-slides/index-auto-0005.html
www.hazelcast.com