Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Pinot: Realtime Distributed OLAP datastore

580.139 Aufrufe

Veröffentlicht am

Pinot is a realtime distributed OLAP datastore, which is used at LinkedIn to deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka). Pinot is designed to scale horizontally.

Veröffentlicht in: Technologie
  • My Ex-boyfriend left me and my daughter since 6 weeks ago after, I tried ALMOST EVERYTHING to restore my relationship and I was disappointed. None was working out, A week later I saw Dr. Sam website and after I contact Dr. Sam for help, everything automatically change, my sadness became joy, smile was all over my face, everything happened very FAST and the result was effective. Dr Sam is the ONLY spell caster who i had success with, my boyfriend called me and said she wanted us to come back and also want to marry me. he came back to me begging me to accept him back and this happened Due to the help of Dr. Sam. I Am so happy that I have my boyfriend back. Contact Dr. Sam now for an urgent and fast love spell to win your relationship back. Supremespellcast@gmail.com Or Supremespellcast@yahoo.com WhatsApp or call him +2347087462033 His blog> https://24hourslovespell.blogspot.com Name: Mary Counts Country: Canada
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Contact Dr Raypower at urgentspellcast@gmail.com if you want to get your Husband, Wife, Boyfriend and Girlfriend back now and regain your happiness. Hi i am Laura Christ from USA I have just experience the wonders of Dr. Raypower love spell, that have been spread on the internet and worldwide, How he marvelously helped people all over the world to restored back their marriage life and get back lost lovers, and also help to win lottery. I contacted him after going through so many testimonies from different people how he help to bring back ex lover back, i told him about my husband that abandoned me about 8 months ago, and left home with all i had.. Dr Raypower only told me to smile and have a rest of mind he will handle all in just 24 hours, After the second day my husband called me, i was just so shocked, i pick the call and couldn't believe my ears, he was really begging me to forgive him and making promises on phone.. He came back home and also got me a new car just for him to proof his love for me. i was so happy and called Dr Raypower and thanked him, he only told me to share the good news all over the world .. Well if you need an effective and real spell caster contact Dr Raypower Via email: urgentspellcast@gmail.com or urgentspellcast@yahoo.com or Call or WhatsApp: +1 (424) 330-8109 and also see more reviews about him on his website: http://urgentspellcast.website2.me/ view his blog page: https://urgentpowerfullovespell.blogspot.com/
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • My Ex-boyfriend left me and my daughter since 6 weeks ago after, I tried ALMOST EVERYTHING to restore my relationship and I was disappointed. None was working out, A week later I saw Dr. Sam website and after I contact Dr. Sam for help, everything automatically change, my sadness became joy, smile was all over my face, everything happened very FAST and the result was effective. Dr Sam is the ONLY spell caster who i had success with, my boyfriend called me and said she wanted us to come back and also want to marry me. he came back to me begging me to accept him back and this happened Due to the help of Dr. Sam. I Am so happy that I have my boyfriend back. Contact Dr. Sam now for an urgent and fast love spell to win your relationship back. Supremespellcast@gmail.com Or Supremespellcast@yahoo.com WhatsApp or call him +2347087462033 His blog> https://24hourslovespell.blogspot.com Name: Mary Counts Country: Canada
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • My real time work with facebook Im making over $2000 a month operating low maint­enance. I continued h­earing distinctive people divulge to me how an lousy lot cash they can make on line so I selected to research it. All topics considered, it become all legitimate and has without a doubt changed my life. For more statistics visit below site HERE☛ ­­ www.profitclip.com
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Get Your Wife Back, Perfect Urgent Love Spell. Am Alexis scorpio from FL Usa All Thanks To Dr happy Who Brought Back The Love Of My Life. my girlfriend cheated on me and asked for a breakup. I don't believe at first i try to get back with her but all she told me was she’s with someone else. that she is no longer interested in marrying me at that point i was heart broken coz i love my girlfriend so much that i could not let go off her all of a sudden she left me, i really love her and never can imagine my life without her. not until i came across a powerful real spell caster Dr happy who promise me 24hours urgent love spell to get back with my girlfriend, good twenty-four {24} hours. hmm-mm, it was a good night time at 11:pm within the days that Dr happy told me that my girlfriend will be back, at first i heard the bell rings getting close to my door i heard someone saying honey!!!, it sound familiar i opened the door and i saw my girlfriend standing and weeping in front of me. i was not surprised because its all i have been praying for her to come back home. Guess what 2 weeks after she noticed her system and her body temperature was changed and i took her to clinic for check up and the doctor told me that there is life in her which means she was pregnant i really wants to use this opportunity to thanks Dr happy so much and the love page that directed me to Dr happy if you have any problem getting your ex back, or predicament that is worse or exactly like this you have been into, contact Dr happy on happylovespell2@gmail.com Whatsapp/cal +2348133873774 visit his website...happylovespell2.webnode.com/ about his love spell Blogs site.http://happylovespel.blogspot.com.ng/
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Pinot: Realtime Distributed OLAP datastore

  1. Pinot Kishore Gopalakrishna Tuesday, August 18, 15
  2. Agenda • Pinot @ LinkedIn - Current • Pinot - Architecture • Pinot Operations • Pinot @ LinkedIn - Future Tuesday, August 18, 15
  3. WVMP Tuesday, August 18, 15
  4. Slice and Dice Metrics Tuesday, August 18, 15
  5. Pinot @ LinkedIn Customers Members Internal tools Tuesday, August 18, 15
  6. • 100B documents • 1B documents ingested per day • 100M queries per day • 10’s of ms latency • 30 tables in prod, 250 * 3 std app nodes Pinot @ LinkedIn Tuesday, August 18, 15
  7. Key features SQL-like interface Columnar storage and indexing Real-time data load Tuesday, August 18, 15
  8. (S)QL: Filters and Aggs SELECT count(*) FROM companyFollowHistoricalEvents WHERE entityId = 121011 AND 'day' >= 15949 AND 'day' <= 15963 AND paid = 'y’ AND action = 'stop' Tuesday, August 18, 15
  9. (S)QL: Group By SELECT count(*) FROM companyFollowHistoricalEvents WHERE entityId = 121011 AND 'day' >= 15949 AND 'day' <= 15963 AND paid = 'y’ GROUP BY action Tuesday, August 18, 15
  10. (S)QL: ORDER BY and LIMIT SELECT * FROM companyFollowHistoricalEvents WHERE entityId = 121011 AND entityId = 1000 AND action = 'start' ORDER BY creationTime DESC LIMIT 1 Tuesday, August 18, 15
  11. Whats not supported • JOIN: unpredictable performance • NOT A SOURCE OF TRUTH • Mutation Tuesday, August 18, 15
  12. Pinot • Data flow • Query Execution • How to use/operate • Pinot @ LinkedIn - Future Tuesday, August 18, 15
  13. Broker Helix Real time Historical Kafka Hadoop Pinot Architecture Queries Raw Data Tuesday, August 18, 15
  14. Pinot • Pinot segments Tuesday, August 18, 15
  15. Pinot Segment layout: Columnar storage Tuesday, August 18, 15
  16. Pinot Segment layout: Sorted Forward Index Tuesday, August 18, 15
  17. Pinot Segment layout: Other techniques • Indexes: Inverted index, Bitmap, RoaringBitmap • Compression: Dictionary Encoding, P4Delta • Multi Valued columns, skip lists, • Hyperloglog for unique • T-digest for Percentile, Quantile Tuesday, August 18, 15
  18. Data aware pre-computation Star tree Index Tuesday, August 18, 15
  19. Pinot • Query Execution Tuesday, August 18, 15
  20. Pinot Query Execution: Distributed Servers S1 S3 S2 S1 S3 S2 Helix Brokers Tuesday, August 18, 15
  21. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix Brokers Tuesday, August 18, 15
  22. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers Tuesday, August 18, 15
  23. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers 3. Scatter Request Tuesday, August 18, 15
  24. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers 3. Scatter Request 4. Process Request & send response Tuesday, August 18, 15
  25. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers 3. Scatter Request 4. Process Request & send response 5. Gather Response Tuesday, August 18, 15
  26. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers 3. Scatter Request 4. Process Request & send response 5. Gather Response 6. Return Response Tuesday, August 18, 15
  27. Pinot Query Execution: Single Node Architecture EXECUTION ENGINE INVERTED INDEX BITMAP INDEX COLUMN FORMAT PLANNER Tuesday, August 18, 15
  28. Pinot Query Execution: Single Node Architecture SELECT campaignId, sum(clicks) FROM Table A WHERE accountId = 121011 AND 'day' >= 15949 GROUP BY campaignId account Id daycampaign Id click Filter Operator Projection Operator Aggregation Group by Operator Combine Operator Pinot Segments Data sources Matching doc ids campaignId,Click tuple Tuesday, August 18, 15
  29. Pinot • Operations Tuesday, August 18, 15
  30. Cluster Management: Deployment Helix Brokers Servers • Brokers and Servers register themselves in Helix • All servers start with no use case specific configuration Controller Tuesday, August 18, 15
  31. On boarding new use case Helix Brokers Servers XLNT XLNT XLNT Create Table command Controller XLNT XLNTTag Servers TableName Brokers 3 XLNT_T1 1 Tuesday, August 18, 15
  32. Segment Assignment Servers S3 S2 S1 Upload Segment S2 S1 S3 S2 S1 S3 Helix Brokers Copies TableName 2 XLNT_T1 Controller Tuesday, August 18, 15
  33. • AUTO recovery mode: Automatically redistribute segments on failure/addition of new nodes • Custom mode: Run in degraded mode until node is restarted/replaced. Pinot - Fault tolerance/Elasticity Tuesday, August 18, 15
  34. Pinot vs Druid Druid Pinot Architecture Realtime + Offline, Realtime only Realtime + Offline Realtime only -> consistency is hard and schema evolution/Bootstrap is hard Inverted Index Always On all columns, Fixed Configurable on per column basis Allows trade off between scanning v/s inverted index + scanning. More data can be fit in given memory size Data organization N/A Sorts data Organizing data provides speed/better compression and removes the need for inverted index Smart pre- materialization N/A star-tree Allows trade off between latency and space Query Execution Layer Fixed Plan Split into Planning and execution Smart choices can be made at runtime based on metadata/query. Tuesday, August 18, 15
  35. • Documentation & tooling • In progress - consistency among real time replicas. • Improve cost to serve - leverage SSD, partial pre materialization • ThirdEye - Business Metrics Monitoring Pinot - Future Tuesday, August 18, 15
  36. Thank You 30 Tuesday, August 18, 15

×