SlideShare ist ein Scribd-Unternehmen logo
1 von 23
ApachieOOzie
Mrs.G.chandraprabha.,M.sc.,m.phil.,
Assistant professor,
Department of information technology,
v.v.vanniaperumal college for women
Apache Oozie
Apache ooize is a java web application used to
schedule Apache Hadoop jobs.
Oozie combines multiple jobs sequentially into
one logical unit of work.
It’s integrated with the Hadoop stack.
Is an server based work flow scheduling system
to Manage Hadoop jobs, It Supports,
Three types of workflows
‱ Oozie workflow jobs
‱ Oozie Bundle
‱ Coordinator jobs
Oozie workflow jobs
 Sequence of action to be executed.
Oozie Bundle
 Package of multiple coordinator And
workflow jobs.
Coordinator jobs
 workflow jobs triggered by time and date
availability.
‱ Users are permitted to create Directed Acyclic
Graphs of workflow which can be run in parallel
and sequentially in Hadoop.
‱ It consist of two parts:
 workflow engine
 coordinator engine
workflow engine
 Responsibility of a workflow engine is to store
and run workflow composed of Hadoop jobs.
coordinator engine
 It runs workflow jobs based on predefined
schedules and a availability of data.
‱ Ooize is scalable and can manage the timely
execution of thousands of workflow in a Hadoop
cluster.
‱ Ooize is very much flexible as well one can easily
start ,stop, suspend and rerun jobs.
‱ Ooize makes it very easy to return failed workflow.
How it is work
‱ Ooize workflow consists of Action Nodes and
Control Nodes.
An Action node represents a workflow jobs .
‱ Moving files into HDFS,running a map reduce, pig
or Hive jobs, importing data using sqoop or
running a Shell Script of a program written java.
Control node
‱ Controls the workflow execution between actions
by allowing contracts like conditional logic where
in different branches dependent action node
Types of node
Start Node
 Designates the start of the workflow jobs.
End Node
Signals end of the job.
Error node
Designates the occurrences of an error and
corresponding error message to be printed.
Features of ooize
Using it’s web service APIs one control jobs
from anywhere.
Ooize has to send email notification upon
computation jobs
Oozie has provision to execute jobs which are
scheduled to run periodically
Using its Web Service APIs one can control
jobs from anywhere.
1. To add a group and users
2. After setting up Hadoop install the packages required for
setting up ooize
3. Next ,download and build ooize using the following
commands
4.Download EXt2J3 to the ‘libext’ directory, this required for the
ooize web console
5. Prepare the ooize WAR file
6. Next, create sharelib on HDFS
7. Next,creat the ooize DB
8. Finally use the command to start ooize server
9. The status of ooize can be checked from command line or the
web console
10. To setup the ooize client, copy the client tar file to the
“ooize client” and the path in bashrc file .
Ooize workflow for IOT data analysis
Assuming that the data received from a machine has the following
structure.
The goal of the analysis is to find the counts of each
status/error code and produce an output with a structure
The ooize workflow comprising of hadoop streaming map
reduce job action and email action that notify the
success or failure of the job.
The map program parses the status/error code from each
line in the input and emits key-value pairs.
Where key is the status/error code and value is 1.
The reduce program receives the key-pairs emitted by
the map program aggregated by the same key.
Each key ,the reduce program calculates the count and
emitskey,value pairs where key is the status/error code
and the value is the count
Thank you

Weitere Àhnliche Inhalte

Ähnlich wie Apache Oozie.pptx

Working with Hive Analytics
Working with Hive AnalyticsWorking with Hive Analytics
Working with Hive AnalyticsManish Chopra
 
Everything you wanted to know, but were afraid to ask about Oozie
Everything you wanted to know, but were afraid to ask about OozieEverything you wanted to know, but were afraid to ask about Oozie
Everything you wanted to know, but were afraid to ask about OozieChicago Hadoop Users Group
 
1. Apache HIVE
1. Apache HIVE1. Apache HIVE
1. Apache HIVEAnuja Gunale
 
Introduction to Oozie | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to Oozie | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to Oozie | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to Oozie | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab
 
Oozie & sqoop by pradeep
Oozie & sqoop by pradeepOozie & sqoop by pradeep
Oozie & sqoop by pradeepPradeep Pandey
 
oozieee.pdf
oozieee.pdfoozieee.pdf
oozieee.pdfwwww63
 
Oracle jet
Oracle jetOracle jet
Oracle jetamithabhat1
 
Breathing New Life into Apache Oozie with Apache Ambari Workflow Manager
Breathing New Life into Apache Oozie with Apache Ambari Workflow ManagerBreathing New Life into Apache Oozie with Apache Ambari Workflow Manager
Breathing New Life into Apache Oozie with Apache Ambari Workflow ManagerDataWorks Summit
 
Kalp Corporate Node JS Perfect Guide
Kalp Corporate Node JS Perfect GuideKalp Corporate Node JS Perfect Guide
Kalp Corporate Node JS Perfect GuideKalp Corporate
 
Overview of PaaS: Java experience
Overview of PaaS: Java experienceOverview of PaaS: Java experience
Overview of PaaS: Java experienceIgor Anishchenko
 
Overview of PaaS: Java experience
Overview of PaaS: Java experienceOverview of PaaS: Java experience
Overview of PaaS: Java experienceAlex Tumanoff
 
Data Analytics and IoT, how to analyze data from IoT
Data Analytics and IoT, how to analyze data from IoTData Analytics and IoT, how to analyze data from IoT
Data Analytics and IoT, how to analyze data from IoTAmmarHassan80
 
AWS Batch: Simplifying Batch Computing in the Cloud
AWS Batch: Simplifying Batch Computing in the CloudAWS Batch: Simplifying Batch Computing in the Cloud
AWS Batch: Simplifying Batch Computing in the CloudAmazon Web Services
 
Breathing new life into Apache Oozie with Apache Ambari Workflow Manager
Breathing new life into Apache Oozie with Apache Ambari Workflow ManagerBreathing new life into Apache Oozie with Apache Ambari Workflow Manager
Breathing new life into Apache Oozie with Apache Ambari Workflow ManagerArtem Ervits
 
198970820 p-oooooooooo
198970820 p-oooooooooo198970820 p-oooooooooo
198970820 p-oooooooooohomeworkping4
 
My Saminar On Php
My Saminar On PhpMy Saminar On Php
My Saminar On PhpArjun Kumawat
 
App_Engine_PPT.ppt
App_Engine_PPT.pptApp_Engine_PPT.ppt
App_Engine_PPT.pptArunPrakash330
 

Ähnlich wie Apache Oozie.pptx (20)

Working with Hive Analytics
Working with Hive AnalyticsWorking with Hive Analytics
Working with Hive Analytics
 
Everything you wanted to know, but were afraid to ask about Oozie
Everything you wanted to know, but were afraid to ask about OozieEverything you wanted to know, but were afraid to ask about Oozie
Everything you wanted to know, but were afraid to ask about Oozie
 
1. Apache HIVE
1. Apache HIVE1. Apache HIVE
1. Apache HIVE
 
Hadoop Oozie
Hadoop OozieHadoop Oozie
Hadoop Oozie
 
Introduction to Oozie | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to Oozie | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to Oozie | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to Oozie | Big Data Hadoop Spark Tutorial | CloudxLab
 
Oozie & sqoop by pradeep
Oozie & sqoop by pradeepOozie & sqoop by pradeep
Oozie & sqoop by pradeep
 
oozieee.pdf
oozieee.pdfoozieee.pdf
oozieee.pdf
 
Chapter 10
Chapter 10Chapter 10
Chapter 10
 
Oracle jet
Oracle jetOracle jet
Oracle jet
 
Breathing New Life into Apache Oozie with Apache Ambari Workflow Manager
Breathing New Life into Apache Oozie with Apache Ambari Workflow ManagerBreathing New Life into Apache Oozie with Apache Ambari Workflow Manager
Breathing New Life into Apache Oozie with Apache Ambari Workflow Manager
 
Kalp Corporate Node JS Perfect Guide
Kalp Corporate Node JS Perfect GuideKalp Corporate Node JS Perfect Guide
Kalp Corporate Node JS Perfect Guide
 
Overview of PaaS: Java experience
Overview of PaaS: Java experienceOverview of PaaS: Java experience
Overview of PaaS: Java experience
 
Overview of PaaS: Java experience
Overview of PaaS: Java experienceOverview of PaaS: Java experience
Overview of PaaS: Java experience
 
Data Analytics and IoT, how to analyze data from IoT
Data Analytics and IoT, how to analyze data from IoTData Analytics and IoT, how to analyze data from IoT
Data Analytics and IoT, how to analyze data from IoT
 
AWS Batch: Simplifying Batch Computing in the Cloud
AWS Batch: Simplifying Batch Computing in the CloudAWS Batch: Simplifying Batch Computing in the Cloud
AWS Batch: Simplifying Batch Computing in the Cloud
 
Breathing new life into Apache Oozie with Apache Ambari Workflow Manager
Breathing new life into Apache Oozie with Apache Ambari Workflow ManagerBreathing new life into Apache Oozie with Apache Ambari Workflow Manager
Breathing new life into Apache Oozie with Apache Ambari Workflow Manager
 
198970820 p-oooooooooo
198970820 p-oooooooooo198970820 p-oooooooooo
198970820 p-oooooooooo
 
My Saminar On Php
My Saminar On PhpMy Saminar On Php
My Saminar On Php
 
App_Engine_PPT.ppt
App_Engine_PPT.pptApp_Engine_PPT.ppt
App_Engine_PPT.ppt
 
App_Engine_PPT.ppt
App_Engine_PPT.pptApp_Engine_PPT.ppt
App_Engine_PPT.ppt
 

Mehr von V.V.Vanniaperumal College for Women

Mehr von V.V.Vanniaperumal College for Women (20)

Control Memory.pptx
Control Memory.pptxControl Memory.pptx
Control Memory.pptx
 
ADDRESSING MODES.pptx
ADDRESSING MODES.pptxADDRESSING MODES.pptx
ADDRESSING MODES.pptx
 
Data_Transfer&Manupulation Instructions.pptx
Data_Transfer&Manupulation Instructions.pptxData_Transfer&Manupulation Instructions.pptx
Data_Transfer&Manupulation Instructions.pptx
 
Timing & Control.pptx
Timing & Control.pptxTiming & Control.pptx
Timing & Control.pptx
 
Human Rights - 1.pptx
Human Rights - 1.pptxHuman Rights - 1.pptx
Human Rights - 1.pptx
 
Registers.pptx
Registers.pptxRegisters.pptx
Registers.pptx
 
Instruction Codes.pptx
Instruction Codes.pptxInstruction Codes.pptx
Instruction Codes.pptx
 
Features of Java.pptx
Features of Java.pptxFeatures of Java.pptx
Features of Java.pptx
 
JVM.pptx
JVM.pptxJVM.pptx
JVM.pptx
 
Constructors in JAva.pptx
Constructors in JAva.pptxConstructors in JAva.pptx
Constructors in JAva.pptx
 
IS-Crypttools.pptx
IS-Crypttools.pptxIS-Crypttools.pptx
IS-Crypttools.pptx
 
IS-Delibrate software attacks.pptx
IS-Delibrate software attacks.pptxIS-Delibrate software attacks.pptx
IS-Delibrate software attacks.pptx
 
IS-Nature of forces.ppt
IS-Nature of forces.pptIS-Nature of forces.ppt
IS-Nature of forces.ppt
 
IS-cryptograpy algorithms.pptx
IS-cryptograpy algorithms.pptxIS-cryptograpy algorithms.pptx
IS-cryptograpy algorithms.pptx
 
IS-Types of IDPSs.pptx
IS-Types of IDPSs.pptxIS-Types of IDPSs.pptx
IS-Types of IDPSs.pptx
 
IS-honeypot.pptx
IS-honeypot.pptxIS-honeypot.pptx
IS-honeypot.pptx
 
Sum of subset problem.pptx
Sum of subset problem.pptxSum of subset problem.pptx
Sum of subset problem.pptx
 
M-coloring.pptx
M-coloring.pptxM-coloring.pptx
M-coloring.pptx
 
storm.ppt
storm.pptstorm.ppt
storm.ppt
 
storm for RTA.pptx
storm for RTA.pptxstorm for RTA.pptx
storm for RTA.pptx
 

KĂŒrzlich hochgeladen

Solving Puzzles Benefits Everyone (English).pptx
Solving Puzzles Benefits Everyone (English).pptxSolving Puzzles Benefits Everyone (English).pptx
Solving Puzzles Benefits Everyone (English).pptxOH TEIK BIN
 
How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17Celine George
 
Crayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon ACrayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon AUnboundStockton
 
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxPOINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxSayali Powar
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityGeoBlogs
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13Steve Thomason
 
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdfEnzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdfSumit Tiwari
 
Presiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha electionsPresiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha electionsanshu789521
 
18-04-UA_REPORT_MEDIALITERAĐĄY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAĐĄY_INDEX-DM_23-1-final-eng.pdf18-04-UA_REPORT_MEDIALITERAĐĄY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAĐĄY_INDEX-DM_23-1-final-eng.pdfssuser54595a
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docxPoojaSen20
 
Class 11 Legal Studies Ch-1 Concept of State .pdf
Class 11 Legal Studies Ch-1 Concept of State .pdfClass 11 Legal Studies Ch-1 Concept of State .pdf
Class 11 Legal Studies Ch-1 Concept of State .pdfakmcokerachita
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Sapana Sha
 
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptxContemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptxRoyAbrique
 
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPTECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPTiammrhaywood
 
Separation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesSeparation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesFatimaKhan178732
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxheathfieldcps1
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformChameera Dedduwage
 
call girls in Kamla Market (DELHI) 🔝 >àŒ’9953330565🔝 genuine Escort Service đŸ”âœ”ïžâœ”ïž
call girls in Kamla Market (DELHI) 🔝 >àŒ’9953330565🔝 genuine Escort Service đŸ”âœ”ïžâœ”ïžcall girls in Kamla Market (DELHI) 🔝 >àŒ’9953330565🔝 genuine Escort Service đŸ”âœ”ïžâœ”ïž
call girls in Kamla Market (DELHI) 🔝 >àŒ’9953330565🔝 genuine Escort Service đŸ”âœ”ïžâœ”ïž9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxpboyjonauth
 

KĂŒrzlich hochgeladen (20)

Solving Puzzles Benefits Everyone (English).pptx
Solving Puzzles Benefits Everyone (English).pptxSolving Puzzles Benefits Everyone (English).pptx
Solving Puzzles Benefits Everyone (English).pptx
 
How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17
 
Crayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon ACrayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon A
 
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxPOINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activity
 
Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13
 
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdfEnzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
 
Presiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha electionsPresiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha elections
 
18-04-UA_REPORT_MEDIALITERAĐĄY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAĐĄY_INDEX-DM_23-1-final-eng.pdf18-04-UA_REPORT_MEDIALITERAĐĄY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAĐĄY_INDEX-DM_23-1-final-eng.pdf
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docx
 
Class 11 Legal Studies Ch-1 Concept of State .pdf
Class 11 Legal Studies Ch-1 Concept of State .pdfClass 11 Legal Studies Ch-1 Concept of State .pdf
Class 11 Legal Studies Ch-1 Concept of State .pdf
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
 
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptxContemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
 
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPTECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
 
Separation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesSeparation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and Actinides
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptx
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy Reform
 
call girls in Kamla Market (DELHI) 🔝 >àŒ’9953330565🔝 genuine Escort Service đŸ”âœ”ïžâœ”ïž
call girls in Kamla Market (DELHI) 🔝 >àŒ’9953330565🔝 genuine Escort Service đŸ”âœ”ïžâœ”ïžcall girls in Kamla Market (DELHI) 🔝 >àŒ’9953330565🔝 genuine Escort Service đŸ”âœ”ïžâœ”ïž
call girls in Kamla Market (DELHI) 🔝 >àŒ’9953330565🔝 genuine Escort Service đŸ”âœ”ïžâœ”ïž
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptx
 

Apache Oozie.pptx

  • 1. ApachieOOzie Mrs.G.chandraprabha.,M.sc.,m.phil., Assistant professor, Department of information technology, v.v.vanniaperumal college for women
  • 2. Apache Oozie Apache ooize is a java web application used to schedule Apache Hadoop jobs. Oozie combines multiple jobs sequentially into one logical unit of work. It’s integrated with the Hadoop stack. Is an server based work flow scheduling system to Manage Hadoop jobs, It Supports,
  • 3.
  • 4. Three types of workflows ‱ Oozie workflow jobs ‱ Oozie Bundle ‱ Coordinator jobs Oozie workflow jobs  Sequence of action to be executed. Oozie Bundle  Package of multiple coordinator And workflow jobs. Coordinator jobs  workflow jobs triggered by time and date availability.
  • 5. ‱ Users are permitted to create Directed Acyclic Graphs of workflow which can be run in parallel and sequentially in Hadoop. ‱ It consist of two parts:  workflow engine  coordinator engine workflow engine  Responsibility of a workflow engine is to store and run workflow composed of Hadoop jobs. coordinator engine  It runs workflow jobs based on predefined schedules and a availability of data.
  • 6. ‱ Ooize is scalable and can manage the timely execution of thousands of workflow in a Hadoop cluster. ‱ Ooize is very much flexible as well one can easily start ,stop, suspend and rerun jobs. ‱ Ooize makes it very easy to return failed workflow.
  • 7. How it is work ‱ Ooize workflow consists of Action Nodes and Control Nodes. An Action node represents a workflow jobs . ‱ Moving files into HDFS,running a map reduce, pig or Hive jobs, importing data using sqoop or running a Shell Script of a program written java. Control node ‱ Controls the workflow execution between actions by allowing contracts like conditional logic where in different branches dependent action node
  • 8. Types of node Start Node  Designates the start of the workflow jobs. End Node Signals end of the job. Error node Designates the occurrences of an error and corresponding error message to be printed.
  • 9. Features of ooize Using it’s web service APIs one control jobs from anywhere. Ooize has to send email notification upon computation jobs Oozie has provision to execute jobs which are scheduled to run periodically Using its Web Service APIs one can control jobs from anywhere.
  • 10. 1. To add a group and users
  • 11. 2. After setting up Hadoop install the packages required for setting up ooize
  • 12. 3. Next ,download and build ooize using the following commands
  • 13. 4.Download EXt2J3 to the ‘libext’ directory, this required for the ooize web console
  • 14. 5. Prepare the ooize WAR file
  • 15. 6. Next, create sharelib on HDFS
  • 16. 7. Next,creat the ooize DB
  • 17. 8. Finally use the command to start ooize server
  • 18. 9. The status of ooize can be checked from command line or the web console
  • 19. 10. To setup the ooize client, copy the client tar file to the “ooize client” and the path in bashrc file .
  • 20. Ooize workflow for IOT data analysis Assuming that the data received from a machine has the following structure.
  • 21. The goal of the analysis is to find the counts of each status/error code and produce an output with a structure
  • 22. The ooize workflow comprising of hadoop streaming map reduce job action and email action that notify the success or failure of the job. The map program parses the status/error code from each line in the input and emits key-value pairs. Where key is the status/error code and value is 1. The reduce program receives the key-pairs emitted by the map program aggregated by the same key. Each key ,the reduce program calculates the count and emitskey,value pairs where key is the status/error code and the value is the count