1. 800 ” ”
Hadoop

2. • id:sasata299 ( )
• Ruby Perl
•
• http://blog.livedoor.jp/sasata299/

3. 1. Hadoop
2. Hadoop
3.
4.
5.

4. Hadoop

5. 816
30 3 1

6. ( )

7. ( )

8. •
• GROUP BY (
( Д`)
• 7000 (
)

9. !!

10. Hadoop

11. Hadoop

12. • Google MapReduce
•
•
• HDFS

13. ( ) ( )
Mapper Reducer
( ) ( )

15. ‣ Hadoop Streaming
‣ Ruby
‣ EC2 Hadoop (
50 )
‣ HDFS S3 (s3fs)

17. ( )
( )
Mapper ( )
( )

18. HDFS
Mapper, Reducer

19. Hadoop cat
`hadoop dfs -cat
s3://xxx/user/root/in/hoge`
※

20. require ‘csv’
path = ‘s3://xxx/user/root/in/user_info’ #
user_info = `hadoop dfs -cat #{path}`
ARGF.each_line do |line| #
line.chomp!
csv = CSV.parse(line)
# user_info
end

22. 7000 ( )→

23. 7000 ( )→
30

24. Hadoop !!

25. • Mapper, Reducer HDFS
(Hadoop cat)
•
• DB