This document summarizes an agenda for the SARA Hadoop Hackathon on December 7, 2010. It provides background on Hadoop and how it relates to earlier technologies like Nutch and MapReduce. It then outlines the agenda for the day which includes introductions, presentations on MapReduce at University of Twente and a kickoff for the hackathon project building period. An optional tour of the SARA facilities is also included. The day will conclude with presentations of hackathon results.

1. SARA Hadoop Hackathon
Evert.Lammerts@sara.nl
December 7, 2010

2. DJOERD HIEMSTRA
(UTwente)
EDGAR MEIJ
(UvA)
SARA Hadoop Hackathon, December 7, 2010

3. 2002 2004 2006
Nutch* MR/GFS** Hadoop
* http://nutch.apache.org/
** http://labs.google.com/papers/mapreduce.html
http://labs.google.com/papers/gfs.html
SARA Hadoop Hackathon, December 7, 2010

4. 2010: A Hype in Production
http://wiki.apache.org/hadoop/PoweredBy
SARA Hadoop Hackathon, December 7, 2010

5. Super computing
Cloud computing Grid computing
Cluster computing GPU computing
http://www.sara.nl/
SARA Hadoop Hackathon, December 7, 2010

6. :-(
Data Expensive!
Computation
:-)
Data Cheaper!
Computation
Ref: Luiz André Barroso and Urs Hölzle, Google Inc.
The Datacenter as a Computer: An Introduction to the Design of Warehouse­Scale Machines
SARA Hadoop Hackathon, December 7, 2010

7. NameNode JobTracker
DN TT DN TT DN TT DN TT
DN TT DN TT DN TT DN TT
DN DataNode
TT TaskTracker
SARA Hadoop Hackathon, December 7, 2010

8. File Map Shuffle Reduce Output
$ echo “${email#*@}, ${name}” $ sort $ wc ­l
ewi.utwente.nl, 1
gmail.com, 2
nbic.nl, 1
nikhef.nl, 3
sara.nl, 1
SARA Hadoop Hackathon, December 7, 2010

9. From: Hadoop, The Definitive Guide (2nd Edition), Tom White
SARA Hadoop Hackathon, December 7, 2010

10. Today
09.30 - 09.50 Welcome & Introduction
09.50 - 10.15 Map/Reduce @ University of Twente
10.15 - 10.30 Kick-off hackathon
14.00 - 15.00 Optional: SARA tour
10.30 - 17.00 Hackathon
17.00 - 17.30 Results and closing
SARA Hadoop Hackathon, December 7, 2010