High Availability in YARN - Project Preview

•

1 gefällt mir•1,205 views

Arinto Murdopo

Project preview presentation for ID2219.

Bildung

Highly Available
NG Hadoop MapReduce
Arinto Murdopo
Mário Almeida
Umit Cavus
Strahinja Lazetic

Supervisor: Jim Dowling

Outcome
● Highly Available implementation of YARN
○ with Zookeeper before the hadoop team
○ with NDB MySql-Cluster before Jim kicks us out

● (Secondary goal) make RM stateless so we
can have more than one running at each
moment.
○ So we can have failover when there is failure in a
RM

● (Eventual goal) Get a job!

Empfohlen

IEEE SRDS'12: From Backup to Hot Standby: High Availability for HDFSAndré Oriani

Queens Parh Rangers AD410 น.ส.ฐิติมา ประเสริฐชัย เลขที่8yaying-yingg

Dan-leiri 2012Marko Havu

AB Consulting

ｈｏｗ　ｔｏ　ｓａｙ　ｆｏｏｄｓ　ａｎｄ　ｄｒｉｎｋｓ　ｉｎ　ｊａｐａｎｅｓｅCheyanneStotlar

An Integer Programming Representation for Data Center Power-Aware Management ...Arinto Murdopo

Intelligent Placement of Datacenter for Internet Services Arinto Murdopo

FacebookRaquel Palau Capell

Empfohlen

IEEE SRDS'12: From Backup to Hot Standby: High Availability for HDFSAndré Oriani

Queens Parh Rangers AD410 น.ส.ฐิติมา ประเสริฐชัย เลขที่8yaying-yingg

Dan-leiri 2012Marko Havu

AB Consulting

ｈｏｗ　ｔｏ　ｓａｙ　ｆｏｏｄｓ　ａｎｄ　ｄｒｉｎｋｓ　ｉｎ　ｊａｐａｎｅｓｅCheyanneStotlar

An Integer Programming Representation for Data Center Power-Aware Management ...Arinto Murdopo

Intelligent Placement of Datacenter for Internet Services Arinto Murdopo

FacebookRaquel Palau Capell

Practica 2 luis ivan cruz val.persi-10

Sam houston chess teamSam Houston Lmc

Maailmassa on parempia pankkejaPankki2

Uso correto de epi´s abafadoresPaulo Carvalho

Why File Sharing is Dangerous?Arinto Murdopo

Arviointi ja palaute 2011Marko Havu

Architecting a Cloud-Scale Identity FabricArinto Murdopo

The counting system for small animals in japaneseCheyanneStotlar

Moodboards edaedaozdemir

Pechakuchacmcgrupo8

Distributed Computing - What, why, how..Arinto Murdopo

153 test plan< <

UX homework4mylovelymac

Parts of SpeechJen Lawson

Quantum Cryptography and Possible AttacksArinto Murdopo

Pankki 2.0-hankkeen esittelyPankki2

Cultura mitesComalat1D

Netcare csi kelvin's talk aug 2015Kelvin Glen

Parallelization of Smith-Waterman Algorithm using MPIArinto Murdopo

Distributed Decision Tree Learning for Mining Big Data StreamsArinto Murdopo

Weitere ähnliche Inhalte

Andere mochten auch

Practica 2 luis ivan cruz val.persi-10

Sam houston chess teamSam Houston Lmc

Maailmassa on parempia pankkejaPankki2

Uso correto de epi´s abafadoresPaulo Carvalho

Why File Sharing is Dangerous?Arinto Murdopo

Arviointi ja palaute 2011Marko Havu

Architecting a Cloud-Scale Identity FabricArinto Murdopo

The counting system for small animals in japaneseCheyanneStotlar

Moodboards edaedaozdemir

Pechakuchacmcgrupo8

Distributed Computing - What, why, how..Arinto Murdopo

153 test plan< <

UX homework4mylovelymac

Parts of SpeechJen Lawson

Quantum Cryptography and Possible AttacksArinto Murdopo

Pankki 2.0-hankkeen esittelyPankki2

Cultura mitesComalat1D

Netcare csi kelvin's talk aug 2015Kelvin Glen

Parallelization of Smith-Waterman Algorithm using MPIArinto Murdopo

Andere mochten auch (20)

Practica 2 luis ivan cruz val.

Sam houston chess team

Maailmassa on parempia pankkeja

Uso correto de epi´s abafadores

Why File Sharing is Dangerous?

Arviointi ja palaute 2011

Architecting a Cloud-Scale Identity Fabric

The counting system for small animals in japanese

Moodboards eda

Pechakucha

Distributed Computing - What, why, how..

153 test plan

UX homework4

Parts of Speech

Quantum Cryptography and Possible Attacks

Pankki 2.0-hankkeen esittely

Cultura mites

Netcare csi kelvin's talk aug 2015

Parallelization of Smith-Waterman Algorithm using MPI

Mehr von Arinto Murdopo

Distributed Decision Tree Learning for Mining Big Data StreamsArinto Murdopo

Next Generation Hadoop: High Availability for YARN Arinto Murdopo

High Availability in YARNArinto Murdopo

An Integer Programming Representation for Data Center Power-Aware Management ...Arinto Murdopo

Quantum Cryptography and Possible Attacks-slideArinto Murdopo

Dremel Paper ReviewArinto Murdopo

Megastore - ID2220 PresentationArinto Murdopo

Flume Event ScalabilityArinto Murdopo

Large Scale Distributed Storage Systems in Volunteer Computing - SlideArinto Murdopo

Large-Scale Decentralized Storage Systems for Volunter Computing SystemsArinto Murdopo

Rise of Network VirtualizationArinto Murdopo

Consistency Tradeoffs in Modern Distributed Database System DesignArinto Murdopo

Distributed Storage System for Volunteer ComputingArinto Murdopo

Apache FlumeArinto Murdopo

Why Use “REST” Architecture for Web Services?Arinto Murdopo

Distributed SystemsArinto Murdopo

Mehr von Arinto Murdopo (17)

Distributed Decision Tree Learning for Mining Big Data Streams

Next Generation Hadoop: High Availability for YARN

High Availability in YARN

An Integer Programming Representation for Data Center Power-Aware Management ...

Quantum Cryptography and Possible Attacks-slide

Dremel Paper Review

Megastore - ID2220 Presentation

Flume Event Scalability

Large Scale Distributed Storage Systems in Volunteer Computing - Slide

Large-Scale Decentralized Storage Systems for Volunter Computing Systems

Rise of Network Virtualization

Consistency Tradeoffs in Modern Distributed Database System Design

Distributed Storage System for Volunteer Computing

Apache Flume

Why Use “REST” Architecture for Web Services?

Distributed Systems

Kürzlich hochgeladen

Grade Three -ELLNA-REVIEWER-ENGLISH.pptxkarenfajardo43

MS4 level being good citizen -imperative- (1) (1).pdfMr Bounab Samir

Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptxlancelewisportillo

How to Fix XML SyntaxError in Odoo the 17Celine George

4.16.24 Poverty and Precarity--Desmond.pptxmary850239

4.11.24 Poverty and Inequality in America.pptxmary850239

Reading and Writing Skills 11 quarter 4 melc 1GloryAnnCastre1

Mental Health Awareness - a toolkit for supporting young mindsPooky Knightsmith

Scientific Writing :Research DiscourseAnita GoswamiGiri

Faculty Profile prashantha K EEE dept Sri Sairam college of EngineeringSri Sairam College Of Engineering Bengaluru

Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptxDhatriParmar

Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Association for Project Management

Mattingly "AI & Prompt Design: Large Language Models"National Information Standards Organization (NISO)

Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfJemuel Francisco

Q-Factor General Quiz-7th April 2024, Quiz Club NITWQuiz Club NITW

Multi Domain Alias In the Odoo 17 ERP ModuleCeline George

Expanded definition: technical and operationalssuser3e220a

prashanth updated resume 2024 for Teaching ProfessionSri Sairam College Of Engineering Bengaluru

How to Make a Duplicate of Your Odoo 17 DatabaseCeline George

Mythology Quiz-4th April 2024, Quiz Club NITWQuiz Club NITW

Kürzlich hochgeladen (20)

Grade Three -ELLNA-REVIEWER-ENGLISH.pptx

MS4 level being good citizen -imperative- (1) (1).pdf

Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptx

How to Fix XML SyntaxError in Odoo the 17

4.16.24 Poverty and Precarity--Desmond.pptx

4.11.24 Poverty and Inequality in America.pptx

Reading and Writing Skills 11 quarter 4 melc 1

Mental Health Awareness - a toolkit for supporting young minds

Scientific Writing :Research Discourse

Faculty Profile prashantha K EEE dept Sri Sairam college of Engineering

Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx

Team Lead Succeed – Helping you and your team achieve high-performance teamwo...

Mattingly "AI & Prompt Design: Large Language Models"

Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf

Q-Factor General Quiz-7th April 2024, Quiz Club NITW

Multi Domain Alias In the Odoo 17 ERP Module

Expanded definition: technical and operational

prashanth updated resume 2024 for Teaching Profession

How to Make a Duplicate of Your Odoo 17 Database

Mythology Quiz-4th April 2024, Quiz Club NITW

High Availability in YARN - Project Preview

1. Highly Available NG Hadoop MapReduce Arinto Murdopo Mário Almeida Umit Cavus Strahinja Lazetic Supervisor: Jim Dowling

2. Aim High Availability in YARN

3. Aim HA with ZooKeeper

4. Aim HA with NDB

5. Outcome ● Highly Available implementation of YARN ○ with Zookeeper before the hadoop team ○ with NDB MySql-Cluster before Jim kicks us out ● (Secondary goal) make RM stateless so we can have more than one running at each moment. ○ So we can have failover when there is failure in a RM ● (Eventual goal) Get a job!

6. Expected graphs (fictional values)

7. Expected graphs (fictional values)