Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

Apache FLINK.pptx

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Wird geladen in …3
×

Hier ansehen

1 von 22 Anzeige
Anzeige

Weitere Verwandte Inhalte

Aktuellste (20)

Anzeige

Apache FLINK.pptx

  1. 1. Apache FLINK
  2. 2. Agenda
  3. 3. Introduction
  4. 4. Processing Types
  5. 5. Batch Processing
  6. 6. Stream Processing
  7. 7. Live Stock Feed (Stream processing example)
  8. 8. Differences between Batch and Real-Time Processing Batch Processing Real-Time Processing Data Static Files Event Streams Speed Processed Periodically in minute, hour, day etc. Processed immediately nanoseconds Storage Past data on disk storage In Memory Storage Example Bill Generation ATM Transaction Alert
  9. 9. Deeper into FLINK
  10. 10. Eco-system Apache FLINK
  11. 11. FLINK program Data source Source is responsible for reading data from data sources such as HDFS, KAFKA … Transformation Responsible for data transformation operations Reduce(), sum(), max(), min() … Data Sink Responsible for final data outputs ()
  12. 12. Architecture
  13. 13. Job Running Process
  14. 14. FLINK time & window EVENT TIME CLASSIFICATION TYPES Event Time: Time when an event occurs Ingestion time: Time when an event arrives at the stream processing system Processing Time: Time when an event is processed by the stream.
  15. 15. Different Between Three Time
  16. 16. FLINK time & window DEFINITION Window is a method for splitting infinite data sets into finites blocks for processing. Windows split the stream into buckets of infinite size, which we can apply computation. TYPES
  17. 17. Time Windows based on Processing Time TUMBLING WINDOWS SLIDING WINDOWS
  18. 18. FLINK Watermark OUT-OF-ORDER PROBLEM WATERMARK SOLUTION
  19. 19. Tips and useful resources
  20. 20. Flink vs Spark vs Hadoop Apache Hadoop Apache Spark Apache Flink Data Processing Engine Batch Batch Stream Processing Speed Slower than Spark and Flink 100x Faster than Hadoop Faster than spark Throughput Medium High High Optimization Manual Manual Automatic Streaming Support NA Spark Streaming Flink Streaming Graph Support NA GraphX Gelly Machine Learning Support NA SparkML FlinkML SQL Support Hive, Impala SparkSQL Table API and SQL Data Transfer Batch Batch Pipelined and Batch
  21. 21. Features of Apache Flink 1) Has a streaming processor, which can run both batch and stream programs. 2) Can process data at lightning-fast speed. 3) APIs available in Java, Scala and Python. 4) Processes data in low latency (nanoseconds) and high throughput. 5) Its fault tolerant. If a node, application or a hardware fails, it does not affect the cluster. 6) In-memory management can be customized for better computation. 7) Windowing is very flexible in Apache Flink.
  22. 22. Thank You

×