Suche senden
Hochladen
Ambari Meetup: APIs and SPIs of Ambari
•
7 gefällt mir
•
3,690 views
Hortonworks
Folgen
Melden
Teilen
Melden
Teilen
1 von 21
Jetzt herunterladen
Downloaden Sie, um offline zu lesen
Empfohlen
Ambari Meetup: Architecture and Demo
Ambari Meetup: Architecture and Demo
Hortonworks
Managing your Hadoop Clusters with Apache Ambari
Managing your Hadoop Clusters with Apache Ambari
DataWorks Summit
Ambari Meetup: Ambari Futures
Ambari Meetup: Ambari Futures
Hortonworks
Apache Ambari BOF - Overview - Hadoop Summit 2013
Apache Ambari BOF - Overview - Hadoop Summit 2013
Hortonworks
Managing your Hadoop Clusters with Ambari
Managing your Hadoop Clusters with Ambari
DataWorks Summit
Apache Ambari BOF - APIs - Hadoop Summit 2013
Apache Ambari BOF - APIs - Hadoop Summit 2013
Hortonworks
Discover.hdp2.2.ambari.final[1]
Discover.hdp2.2.ambari.final[1]
Hortonworks
Managing 2000 Node Cluster with Ambari
Managing 2000 Node Cluster with Ambari
DataWorks Summit
Weitere ähnliche Inhalte
Was ist angesagt?
Hortonworks SmartSense
Hortonworks SmartSense
Artem Ervits
Manage Hadoop Cluster with Ambari
Manage Hadoop Cluster with Ambari
TeK Charnsilp Chinprasert
Ambari Views - Overview
Ambari Views - Overview
Hortonworks
An Overview of Ambari
An Overview of Ambari
Chicago Hadoop Users Group
Apache Ambari - What's New in 1.4.3
Apache Ambari - What's New in 1.4.3
Hortonworks
zData BI & Advanced Analytics Platform + 8 Week Pilot Programs
zData BI & Advanced Analytics Platform + 8 Week Pilot Programs
zData Inc.
Apache Ambari - What's New in 1.7.0
Apache Ambari - What's New in 1.7.0
Hortonworks
Apache Ambari - What's New in 1.4.2
Apache Ambari - What's New in 1.4.2
Hortonworks
Apache Ambari - What's New in 1.5.0
Apache Ambari - What's New in 1.5.0
Hortonworks
Deploying and Managing Hadoop Clusters with AMBARI
Deploying and Managing Hadoop Clusters with AMBARI
DataWorks Summit
Apache Ambari: Past, Present, Future
Apache Ambari: Past, Present, Future
Hortonworks
Apache Ambari Stack Extensibility
Apache Ambari Stack Extensibility
Jayush Luniya
Hortonworks Technical Workshop: Apache Ambari
Hortonworks Technical Workshop: Apache Ambari
Hortonworks
2015 zData Inc. - Apache Ambari Overview
2015 zData Inc. - Apache Ambari Overview
zData Inc.
Hortonworks Technical Workshop: Interactive Query with Apache Hive
Hortonworks Technical Workshop: Interactive Query with Apache Hive
Hortonworks
Apache Ambari: Simplified Hadoop Cluster Operation & Troubleshooting
Apache Ambari: Simplified Hadoop Cluster Operation & Troubleshooting
Jayush Luniya
Ambari: Agent Registration Flow
Ambari: Agent Registration Flow
jsposetti
Past, Present and Future of Apache Ambari
Past, Present and Future of Apache Ambari
Artem Ervits
Managing Enterprise Hadoop Clusters with Apache Ambari
Managing Enterprise Hadoop Clusters with Apache Ambari
Jayush Luniya
Implementing SharePoint on Azure, Lessons Learnt from a Real World Project
Implementing SharePoint on Azure, Lessons Learnt from a Real World Project
K.Mohamed Faizal
Was ist angesagt?
(20)
Hortonworks SmartSense
Hortonworks SmartSense
Manage Hadoop Cluster with Ambari
Manage Hadoop Cluster with Ambari
Ambari Views - Overview
Ambari Views - Overview
An Overview of Ambari
An Overview of Ambari
Apache Ambari - What's New in 1.4.3
Apache Ambari - What's New in 1.4.3
zData BI & Advanced Analytics Platform + 8 Week Pilot Programs
zData BI & Advanced Analytics Platform + 8 Week Pilot Programs
Apache Ambari - What's New in 1.7.0
Apache Ambari - What's New in 1.7.0
Apache Ambari - What's New in 1.4.2
Apache Ambari - What's New in 1.4.2
Apache Ambari - What's New in 1.5.0
Apache Ambari - What's New in 1.5.0
Deploying and Managing Hadoop Clusters with AMBARI
Deploying and Managing Hadoop Clusters with AMBARI
Apache Ambari: Past, Present, Future
Apache Ambari: Past, Present, Future
Apache Ambari Stack Extensibility
Apache Ambari Stack Extensibility
Hortonworks Technical Workshop: Apache Ambari
Hortonworks Technical Workshop: Apache Ambari
2015 zData Inc. - Apache Ambari Overview
2015 zData Inc. - Apache Ambari Overview
Hortonworks Technical Workshop: Interactive Query with Apache Hive
Hortonworks Technical Workshop: Interactive Query with Apache Hive
Apache Ambari: Simplified Hadoop Cluster Operation & Troubleshooting
Apache Ambari: Simplified Hadoop Cluster Operation & Troubleshooting
Ambari: Agent Registration Flow
Ambari: Agent Registration Flow
Past, Present and Future of Apache Ambari
Past, Present and Future of Apache Ambari
Managing Enterprise Hadoop Clusters with Apache Ambari
Managing Enterprise Hadoop Clusters with Apache Ambari
Implementing SharePoint on Azure, Lessons Learnt from a Real World Project
Implementing SharePoint on Azure, Lessons Learnt from a Real World Project
Ähnlich wie Ambari Meetup: APIs and SPIs of Ambari
Apache Ambari - What's New in 1.6.0
Apache Ambari - What's New in 1.6.0
Hortonworks
Build a Node.js Client for Your REST+JSON API
Build a Node.js Client for Your REST+JSON API
Stormpath
REST APIs
REST APIs
Arthur De Magalhaes
REST API Best Practices & Implementing in Codeigniter
REST API Best Practices & Implementing in Codeigniter
Sachin G Kulkarni
Meetup 12-12-2017 - Application Isolation on Kubernetes
Meetup 12-12-2017 - Application Isolation on Kubernetes
dtoledo67
Simple REST with Dropwizard
Simple REST with Dropwizard
Andrei Savu
Slider: Applications on YARN
Slider: Applications on YARN
Steve Loughran
Azure cosmosdb
Azure cosmosdb
Udaiappa Ramachandran
Andrei shakirin rest_cxf
Andrei shakirin rest_cxf
Aravindharamanan S
Innovate2014 Better Integrations Through Open Interfaces
Innovate2014 Better Integrations Through Open Interfaces
Steve Speicher
Getting started with Apollo Client and GraphQL
Getting started with Apollo Client and GraphQL
Morgan Dedmon
Build AWS CloudFormation Custom Resources (DEV417-R2) - AWS re:Invent 2018
Build AWS CloudFormation Custom Resources (DEV417-R2) - AWS re:Invent 2018
Amazon Web Services
From System Engineer to Gopher
From System Engineer to Gopher
I-Fan Wang
Introduction to CloudStack API
Introduction to CloudStack API
Krunal Jain
Spring 3 - Der dritte Frühling
Spring 3 - Der dritte Frühling
Thorsten Kamann
RESTful API-centric Universe
RESTful API-centric Universe
Tihomir Opačić
AWS CloudFormation macros: Coding best practices - MAD201 - New York AWS Summit
AWS CloudFormation macros: Coding best practices - MAD201 - New York AWS Summit
Amazon Web Services
AMIS Oracle ADF 12c Launch event 06 Steven Davelaar future REST features
AMIS Oracle ADF 12c Launch event 06 Steven Davelaar future REST features
Getting value from IoT, Integration and Data Analytics
Kubernetes Architecture - beyond a black box - Part 1
Kubernetes Architecture - beyond a black box - Part 1
Hao H. Zhang
Building APIs in an easy way using API Platform
Building APIs in an easy way using API Platform
Antonio Peric-Mazar
Ähnlich wie Ambari Meetup: APIs and SPIs of Ambari
(20)
Apache Ambari - What's New in 1.6.0
Apache Ambari - What's New in 1.6.0
Build a Node.js Client for Your REST+JSON API
Build a Node.js Client for Your REST+JSON API
REST APIs
REST APIs
REST API Best Practices & Implementing in Codeigniter
REST API Best Practices & Implementing in Codeigniter
Meetup 12-12-2017 - Application Isolation on Kubernetes
Meetup 12-12-2017 - Application Isolation on Kubernetes
Simple REST with Dropwizard
Simple REST with Dropwizard
Slider: Applications on YARN
Slider: Applications on YARN
Azure cosmosdb
Azure cosmosdb
Andrei shakirin rest_cxf
Andrei shakirin rest_cxf
Innovate2014 Better Integrations Through Open Interfaces
Innovate2014 Better Integrations Through Open Interfaces
Getting started with Apollo Client and GraphQL
Getting started with Apollo Client and GraphQL
Build AWS CloudFormation Custom Resources (DEV417-R2) - AWS re:Invent 2018
Build AWS CloudFormation Custom Resources (DEV417-R2) - AWS re:Invent 2018
From System Engineer to Gopher
From System Engineer to Gopher
Introduction to CloudStack API
Introduction to CloudStack API
Spring 3 - Der dritte Frühling
Spring 3 - Der dritte Frühling
RESTful API-centric Universe
RESTful API-centric Universe
AWS CloudFormation macros: Coding best practices - MAD201 - New York AWS Summit
AWS CloudFormation macros: Coding best practices - MAD201 - New York AWS Summit
AMIS Oracle ADF 12c Launch event 06 Steven Davelaar future REST features
AMIS Oracle ADF 12c Launch event 06 Steven Davelaar future REST features
Kubernetes Architecture - beyond a black box - Part 1
Kubernetes Architecture - beyond a black box - Part 1
Building APIs in an easy way using API Platform
Building APIs in an easy way using API Platform
Mehr von Hortonworks
Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level
Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level
Hortonworks
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy
Hortonworks
Getting the Most Out of Your Data in the Cloud with Cloudbreak
Getting the Most Out of Your Data in the Cloud with Cloudbreak
Hortonworks
Johns Hopkins - Using Hadoop to Secure Access Log Events
Johns Hopkins - Using Hadoop to Secure Access Log Events
Hortonworks
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys
Hortonworks
HDF 3.2 - What's New
HDF 3.2 - What's New
Hortonworks
Curing Kafka Blindness with Hortonworks Streams Messaging Manager
Curing Kafka Blindness with Hortonworks Streams Messaging Manager
Hortonworks
Interpretation Tool for Genomic Sequencing Data in Clinical Environments
Interpretation Tool for Genomic Sequencing Data in Clinical Environments
Hortonworks
IBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data Landscape
Hortonworks
Premier Inside-Out: Apache Druid
Premier Inside-Out: Apache Druid
Hortonworks
Accelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at Scale
Hortonworks
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
Hortonworks
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Hortonworks
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Hortonworks
Making Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with Ease
Hortonworks
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
Hortonworks
Driving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data Management
Hortonworks
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
Hortonworks
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Hortonworks
Mehr von Hortonworks
(20)
Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level
Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy
Getting the Most Out of Your Data in the Cloud with Cloudbreak
Getting the Most Out of Your Data in the Cloud with Cloudbreak
Johns Hopkins - Using Hadoop to Secure Access Log Events
Johns Hopkins - Using Hadoop to Secure Access Log Events
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys
HDF 3.2 - What's New
HDF 3.2 - What's New
Curing Kafka Blindness with Hortonworks Streams Messaging Manager
Curing Kafka Blindness with Hortonworks Streams Messaging Manager
Interpretation Tool for Genomic Sequencing Data in Clinical Environments
Interpretation Tool for Genomic Sequencing Data in Clinical Environments
IBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data Landscape
Premier Inside-Out: Apache Druid
Premier Inside-Out: Apache Druid
Accelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at Scale
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Making Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with Ease
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
Driving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data Management
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Ambari Meetup: APIs and SPIs of Ambari
1.
Ambari API's and SPI's
of Ambari Tom Beerbower @ Hortonworks April 02, 2013 © Hortonworks Inc. 2013 Page 1
2.
Agenda • API Overview •
Monitoring • API Constructs • Management • Error Handling • SPI Overview • SPI Usage © Hortonworks Inc. 2013 Page 2
3.
API Overview -
Features • REST • Monitoring and Management of Hadoop Cluster • Partial Response • Query Predicates © Hortonworks Inc. 2013 Page 3
4.
Monitoring • Read state
of Hadoop resources. Example: api/v1/clusters/cluster1/services/HDFS/components/NAMENODE { "href" : "http://ec2…1/services/HDFS/components/NAMENODE", "metrics" : { "boottime" : 1.364912931E9, "process" : { "proc_total" : 752.333333333, "proc_run" : 1.64444444444 }, "rpc" : { "rpcAuthorizationSuccesses" : 141, "SentBytes" : 251758, "rpcAuthorizationFailures" : 0,… © Hortonworks Inc. 2013 Page 4
5.
API Overview –
Resource Types • Collection Resource: This resource type doesn’t refer to any specific resource; rather it refers to a collection of resources. For example: api/v1/clusters/cluster1/services Returns a collection of services • Instance Resource: This resource type refers to a single specific resource. For example: api/v1/clusters/cluster1/services/HDFS Refers to the service resource identified by the id “HDFS”. © Hortonworks Inc. 2013 Page 5
6.
API Overview -
Response • JSON • Collection resource • Always includes request href. • Inlcludes “items” of the collection. • Each item includes href and primary id fields. • Instance resource • Always includes request href. • Always includes primary id fields. • May include metrics and properties for instance. • May include href and primary id fields for sub-resources. © Hortonworks Inc. 2013 Page 6
7.
API Overview -
Response Collection Resource Response Example: api/v1/clusters/cluster1/services/ { "href" : "http://ec2…api/v1/clusters/cluster1/services/", "items" : [ { "href" : "http://ec2…/clusters/cluster1/services/HDFS", "ServiceInfo" : { "cluster_name" : "cluster1", "service_name" : "HDFS" } }, { "href" : "http://ec2…/cluster1/services/MAPREDUCE",… © Hortonworks Inc. 2013 Page 7
8.
API Overview -
Response Instance Resource Response Example: api/v1/clusters/cluster1 { "href" : "http://ec2...com:8080/api/v1/clusters/cluster1", "Clusters" : { "cluster_name" : "cluster1", "cluster_id" : 1, "version" : "HDP-1.2.1" }, "services" : [ { "href" : "http://ec2...com:8080/api/v1/clusters/cluster1/services/MAPR EDUCE", "ServiceInfo" : { "cluster_name" : "cluster1", "service_name" : "MAPREDUCE" } }, … © Hortonworks Inc. 2013 Page 8
9.
API Constructs –
Partial Response • Used to control which fields are returned by a query. – restrict which fields are returned. – reach down and return data from sub-resources. • Properties, categories and sub-resources can be specified. • The wildcard ‘*’ can be used to show all… – categories, fields and sub-resources. – provides „expand‟ functionality for sub-components. • Primary id fields of a resource are always shown regardless of the specifies partial response. © Hortonworks Inc. 2013 Page 9
10.
API Constructs –
Partial Response Query Specific Field Example: api/v1/clusters/cluster1/services/MAPREDUCE/components/JOBTRACKE R?fields=metrics/rpc/SentBytes { "href" : "http://…/api/v1/clusters/cluster1/services/MAPREDUCE/components/ JOBTRACKER?fields=metrics/rpc/SentBytes", "metrics" : { "rpc" : { "SentBytes" : 2763921 } }, "ServiceComponentInfo" : { "cluster_name" : "cluster1", "component_name" : "JOBTRACKER", "service_name" : "MAPREDUCE" } } © Hortonworks Inc. 2013 Page 10
11.
API Constructs –
Partial Response Expand Sub-Resource Example: api/v1/clusters/cluster1/services/MAPREDUCE?fields=components/met rics/jvm/gcCount { "href" : "http://…/services/MAPREDUCE?fields=components/metrics/jvm/gc Count", … }, "components" : [ { "href" : "http://…/MAPREDUCE/components/JOBTRACKER", "metrics" : { "jvm" : { "gcCount" : 47 } },… © Hortonworks Inc. 2013 Page 11
12.
API Constructs –
Query Predicates • Limits the set of resources returned by a query. • Consists of at least one relational expression. • Can only be applied to collection resources. • Relational operators ( =, !=, <, >, <=, >= ) • Logical operators ( |, &, !) • Functions (in(), isEmpty()) • Brackets can be used to provide explicit grouping of expressions. Expressions within brackets have the highest precedence. © Hortonworks Inc. 2013 Page 12
13.
API Constructs –
Query Predicates Query For Started Services Example: api/v1/clusters/cluster1/services?ServiceInfo/state=STARTED { "href" : "http://…/clusters/cluster1/services?ServiceInfo/state=STARTE D", "items" : [ { "href" : "http://…/clusters/cluster1/services/NAGIOS", "ServiceInfo" : { "cluster_name" : "cluster1", "state" : "STARTED", "service_name" : "NAGIOS" } },… © Hortonworks Inc. 2013 Page 13
14.
Management - Create •
Create a resource. Example: Create a cluster named „c1‟ with the property „Clusters/version‟ = „HDP-1.2.0‟ POST http://…:8080/api/v1/clusters/c1 { "Clusters": { "version" : "HDP-1.2.0” } } © Hortonworks Inc. 2013 Page 14
15.
Management - Update •
Update a resource. Example: Update the state of all „INSTALLED‟ services to be „STARTED‟ PUT http://…:8080/api/v1/clusters/c1/services?ServiceInf o/state=INSTALLED { "ServiceInfo": { "state" : "STARTED” } } © Hortonworks Inc. 2013 Page 15
16.
Management - Delete •
Delete a resource. Example: Delete the cluster named „c1‟ DELETE http://…:8080/api/v1/clusters/c1 © Hortonworks Inc. 2013 Page 16
17.
Error Handling HTTP
Code Description 200 OK 400 Bad Request 401 Unauthorized 403 Forbidden 404 Not Found 500 Internal Server Error © Hortonworks Inc. 2013 Page 17
18.
Error Handling -
Examples api/v1/clusters/BAD_CLUSTER_NAME 404 Not Found { "status" : 404, "message" : "The requested resource doesn't exist: Cluster not found,clusterName=BAD_CLUSTER_NAME" } api/v1/clusters/cluster1?fields=BAD_FIELD 400 Bad Request { "status" : 400, "message" : "The properties [BAD_FIELD] specified in the request or predicate are not supported for the resource type Cluster." } © Hortonworks Inc. 2013 Page 18
19.
SPI Overview • Service
framework for handling requests for the various resources that it manages and monitors. /api/v1/clusters/c1/hosts/host1/host_components Handler getResources(Resource.Type.HostComponent, request, predicate) Cluster ResourceProvider Service ResourceProvider Host ClusterController ResourceProvider Component ResourceProvider getResources(request, predicate) HostComponent ResourceProvider © Hortonworks Inc. 2013 Page 19
20.
SPI Use Case •
Pluggable resource providers allow API usage with a GSInstaller installed cluster. API SPI GSInstaller Default ResourceProviders ResourceProviders Ambari DB GSInstaller Artifacts © Hortonworks Inc. 2013 Page 20
21.
Q&A © Hortonworks Inc.
2013 Page 21
Jetzt herunterladen