Hadoop Map Reduce

Hadoop Map Reduce Apurva Jadhav Senior Software Engineer TheFind Inc. (diagrams borrowed from various sources)

Introduction ,[object Object],[object Object],[object Object],[object Object],[object Object]

Hadoop Core ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Typical cluster ,[object Object],[object Object]

Hadoop Distributed File System ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HDFS Architecture Namenode (Master) Datanode Client Datanode Datanode Read Write Replication Meta data ops Stores FS metadata – namespace, block locations Stores the data blocks as linux files

Hadoop Map Reduce ,[object Object],[object Object],[object Object],[object Object],[object Object]

Word Count Problem ,[object Object],[object Object],[object Object],[object Object],[object Object]

Why Map Reduce ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map Reduce Architecture ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Jobtracker tasktracker tasktracker tasktracker Input Job (mapper, reducer, input) Data transfer Assign tasks

Map Reduce Programming Model ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map Reduce Programming Model ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Word Count Map Reduce Job ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map Reduce Architecture - 2 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map Reduce Architecture - 2 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Jobtracker tasktracker tasktracker tasktracker Input Job (mapper, reducer, input) Data transfer Assign tasks

Map Reduce Advantages ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Conclusion ,[object Object],[object Object],[object Object],[object Object]

References ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Hadoop Map Reduce

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Hadoop Map Reduce

Ähnlich wie Hadoop Map Reduce (20)

Mehr von VNIT-ACM Student Chapter

Mehr von VNIT-ACM Student Chapter (12)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Hadoop Map Reduce