code.talks 2015 hosted by ABOUT YOU

1. Big Data zum Anfassen
Kai Voigt, Cloudera Inc.

4. AIS
https://en.wikipedia.org/wiki/Automatic_Identification_System

7. Sailing Yacht A ("White Pearl")
Länge: 140+ m, Höhe: 100+ m, Kosten: 400+ Mio EUR

8. Dynamische Daten
{"msgid":1,"rot":0,"sub_message":806,"aisclient_id":
2051,"slot_timeout":6,"spare":2,"userid":211704630,"pos_acc":
1,"sentences":["!AIVDM,1,1,,A,139qK=R009PffhLO9MrU9@4v8H<V,
0*38"],"nav_status":2,"sync_state":0,"time_received":
1443099141551,"repeat":0,"pos":[10.20845,54.43495],"utc_sec":
31,"sog":9,"time_captured":1443099141000,"cog":1317,"raim":
0,"true_heading":2,"smi":0}

9. Statische Daten
{"imo":1012141,"dim_port":15,"aisclient_id":2216,"pos_type":
1,"userid":211704630,"draught":76,"sentences":["!AIVDM,2,1,2,A,
539qK=T0?LJl@l4H001LPU@F10D58h000000000T:0w?:
6J`hC4iA@PDP000,0*13","!AIVDM,2,2,2,A,
00000000000,2*26"],"dim_starboard":
10,"callsign":"DMAF@@@","repeat":0,"time_received":
1443098166711,"version":1,"name":"WHITE PEARL@@@@@@@@@","dte":
0,"eta_minute":48,"dim_bow":80,"msgid":5,"eta_hour":8,"ship_type":
36,"spare":0,"eta_month":9,"dim_stern":
63,"dest":"SEEBAR@@@@@@@@@@@@@@","eta_day":21,"time_captured":
1443098166000}

11. http://www.vesseltracker.com/

12. Satellite AIS

13. Big Data?
• Nord- und Ostsee, 500 Records/Sekunde
• 250 KByte/Sekunde
• Weltweit: 10.000 Records/Sekunde
• 5 MByte/Sekunde = ~400 GByte/Tag

14. λ
Apache Hadoop
AIS Data Stream
Flume/Kafka
Spark StreamingHBase/HDFS
Impala/Spark Near Real Time

15. Kollisionserkennung
• 2 Schiffe zur gleichen Zeit am gleichen Ort

16. Illegaler Fischfang

17. Illegaler Fischfang

18. Illegaler Fischfang

19. Weitere Anwendungsfälle
• Tracking
• Protokollierung von Unfällen
• Piraterie
• Routen-Optimierung
• Hafen-Logistik
• Flotten-Logistik
• Umweltschutz
• Search & Rescue
• Navigationshilfe

20. Danke!
Kai Voigt, kai@cloudera.com