Mr bi

An Insight into Map Reduce and related technology Renjith Peediackal 09BM8040

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Importance of understanding MR

Recommendation System ,[object Object],[object Object],[object Object],[object Object]

A lot more questions ,[object Object],What pages are my customer’s reading

A lot more questions contd.. ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

And the typical problems with recommendation systems

Problems with popularity ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Mixing expert opinion ,[object Object],[object Object],[object Object]

But internet data is unfriendly ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The case for a new technique ,[object Object],[object Object]

What is Data in flight? ,[object Object],[object Object],[object Object],[object Object],[object Object]

Map and reduce ,[object Object],[object Object]

Terminology explained.. ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

How does a map reduce programme work Programmer has to specify two methods: Map and Reduce

map (k, v) -> <k', v'>* ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Shuffle or sort ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

– reduce (k', <v'>*) -> <k', v'>* ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

How this new way helpful for our recommendation system? ,[object Object],[object Object],[object Object],[object Object]

Maps into a denser smaller table

Fault tolerance two different types- Database school of thought

Fault tolerance two different types- MR school of thought

Hierarchy of Parallelism: Cycle of brute force fault tolerance

Criticisms ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Why it is valuable still? ,[object Object],[object Object],[object Object]

Why can’t parallel DB deliver the same? ,[object Object],[object Object],[object Object],[object Object],[object Object]

An example: Invite you to the complexity-sequential web access-based recommendation system

sequential web access-based recommendation system ,[object Object]

Recommendation ,[object Object],[object Object],[object Object],[object Object]

Some details ,[object Object],[object Object],[object Object],Session ID Web access sequence 1 abdac 2 eaebcac 3 babfae 4 abbacfc

Details Access events can be classified into frequent and infrequent based on frequency crossing a threshold level And a tree consisting of frequent access events can be created. Length of sequence Sequential web access pattern with support 1 a:4. b:4, c:3 2 aa:4. ab:4. oc3. ba:4. bc:3 3 aac:3, aba;4, obc:3, bac:3 4 Abac:3

The Map and reduce ,[object Object],[object Object],[object Object],[object Object],[object Object]

DBMS for the same case? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Parallel DB vs MapReduce ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Summary of advantages of MR ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

What is hadoop ,[object Object],[object Object]

Pig ,[object Object],[object Object],[object Object],[object Object]

What does hive do? ,[object Object],[object Object],[object Object],[object Object]

Mr bi

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Mr bi

Ähnlich wie Mr bi (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Mr bi