Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Assaf Araki – Real Time Analytics at Scale

7.266 Aufrufe

Veröffentlicht am

Flink Forward 2015

Veröffentlicht in: Technologie
  • Have u ever tried external professional writing services like ⇒ www.WritePaper.info ⇐ ? I did and I am more than satisfied.
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Assaf Araki – Real Time Analytics at Scale

  1. 1. REAL time Analytics AT SCALE SMART DATA PIPES For THE INTERNET OF THINGS Assaf Araki, Big Data Analytics Architect Big Data Analytics, Intel
  2. 2. Intro to Big Data Analytics @ Intel People (+100) Data Scientists Management Big Data Developers Analytics PMs 13% 41% 9% 37% CONTRIBUTION TO Data Center Group CONTRIBUTION TO INTEL Operations MISSIO N #1 Operational excellence #2 Help Intel win area of Intelligent machines VISION Analytics is a competitive advantage for Intel Industry / Academy Technical due-diligence assessment for Intel Capital Benchmark with startups Academy Collaborations Assist Intel Sales & Marketing DESIGN Cut validations time-to-market MANUFACTURI NGReduce test cost SALES & MARKETINGIncrease sales through analytics Stream Analytics Cloud Parkinson Research Machine Learning Strategy
  3. 3. The IOT challenge CloudIngestionThings Cloud Infrastructure Data Platform Analytics Platform UI Services
  4. 4. Use case : The Parkinson Disease research 44 CLINICAL TRIALS Create and Validate Algorithms & Measures POPULATION STUDY Generate insights Using Big data analytics
  5. 5. 10 Medication reporting Medication reminder Report PATIENT REPORTING OTHER Configurable data collections Contribution score Integrated Login and registration Pebble notifications OBJECTIVE MEASURES Gait Sleep Tremor Activity Level Controlled Tests
  6. 6. So, Why is it Big-Data Problem? 30 subjects 5 DaysperSubject 0.15TB Weekly 500 subjects 30 DaysperSubject 1GB PerSubjectperDay 15TB Monthly 1000 subjects 365 DaysperSubject 365TB Yearly 1GB PerSubjectperDay 1GB PerSubjectperDay
  8. 8. Smart Ingestion characteristics Personalized Easy to use Smart Data Pipe • Per single device or user • Maintain state and required data for ML • Easily subscribe to any Stream • Use familiar development Languages (Java, Scala) • Developers focus on logic development • Apply analytics on the Stream • Trigger actions (close the feedback loop) in timely manner Scalability • Linear scalability (scale Out) • Extremely High concurrencies • High Throughput Fault Tolerance• No Single point of failure • Seamless recovery • Persistent
  9. 9. Smart Data Ingestion – High level overview 9 Device Device Device Device Scalable, Persistent Broker Processing, Stream Analytics
  10. 10. What is Akka? • Micro-service(Actor) oriented. • Message Driven • Lock-free • Location-transparent • High performance • Fault Tolerant • Scales linearly
  11. 11. Stream Processing - the Akka way… 11 Each actor is a small peace of Java or Scala code performing its role A set of actors creates a topology which is responsible for device’s data stream processing A single Akka node may have millions of concurrent actors handling different streams and operations Change detection Automatic change detection time rules matcher Detect & raise alert for matched rules Sleep quality calculating users’ sleep quality Tremor detection Tremor detection based on devices’ Aggregator Aggregation (50hz to minutes / hours) Sample Parkinson Disease re Subscriber Parser Aggregator HBase Writer Analytics Manager Change Detection UnZip Real Time Rules Sleep Quality
  12. 12. STREAM Processing MANAGEMENT Layer (“Pigeon”)
  13. 13. • Core OS & Docker containers enable portability and ease of deployment anywhere • Enables the flexibility of choosing a set of desired containers based on a given use case requirements Easy Portability With Docker & Core OS Preconfigured containers ready to be loaded
  14. 14. • IoT data Ingestion goes beyond moving the data into the cloud • We have deployed a scalable and fault tolerance, multi-protocol pipeline that enables stream Analytics • Stream Analytics platform is leveraged for Other IoT projects Summary
  15. 15. Thank You!