1. De praktijk van
Big Data
Friso van Vollenhoven
fvanvollenhoven@xebia.com
En waarom de huidige
technologie niet (altijd)
voldoet

2. Big Data

3. Big Data

4. Big Data

5. Big Data
Requirement:
Full table scan, 200GB table

6. Big Data

7. Big Data
Egypte, 27 januari 2011

8. Big Data
Requirement:
40.000 updates per seconde, 24/7.

9. Databases
= + +

10. Databases
= + +
network
SAN
storage

11. HDFS en MapReduce
bottleneck
SELECT SESSION, COUNT(*) FROM
WEB_CLICKS GROUP BY SESSION;
CLIENT
storage
network

12. HDFS en MapReduce
SELECT SESSION, COUNT(*) FROM
WEB_CLICKS GROUP BY SESSION;
CLIENT
storage
network
bottleneck

13. HDFS en MapReduce

14. HDFS en MapReduce
SELECT * FROM
WEB_CLICKS;
SELECT * FROM SELECT * FROM
WEB_CLICKS; WEB_CLICKS;

15. HDFS en MapReduce
GROUP BY SESSION

16. HDFS en MapReduce
COUNT(*)
COUNT(*) COUNT(*)

17. HDFS en MapReduce
MAP REDUCE
SELECT * FROM
COUNT(*)
WEB_CLICKS;
SORT/SHUFFLE
GROUP BY SESSION
MAP REDUCE MAP REDUCE
SELECT * FROM SELECT * FROM
COUNT(*) COUNT(*)
WEB_CLICKS; WEB_CLICKS;

18. NoSQL
index A B C D E F G H I J K L M N O P Q R S T U V W X Y Z