Suche senden
Hochladen
Admission Control in Impala
•
15 gefällt mir
•
5,642 views
Cloudera, Inc.
Folgen
Software
Diashow-Anzeige
Melden
Teilen
Diashow-Anzeige
Melden
Teilen
1 von 26
Empfohlen
Dynamic filtering for presto join optimisation
Dynamic filtering for presto join optimisation
Ori Reshef
Strata London 2019 Scaling Impala
Strata London 2019 Scaling Impala
Manish Maheshwari
Hive Does ACID
Hive Does ACID
DataWorks Summit
Performance Optimizations in Apache Impala
Performance Optimizations in Apache Impala
Cloudera, Inc.
Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
Timothy Spann
Apache Spark-Bench: Simulate, Test, Compare, Exercise, and Yes, Benchmark wit...
Apache Spark-Bench: Simulate, Test, Compare, Exercise, and Yes, Benchmark wit...
Spark Summit
Hybrid Apache Spark Architecture with YARN and Kubernetes
Hybrid Apache Spark Architecture with YARN and Kubernetes
Databricks
A Closer Look at Apache Kudu
A Closer Look at Apache Kudu
Andriy Zabavskyy
Empfohlen
Dynamic filtering for presto join optimisation
Dynamic filtering for presto join optimisation
Ori Reshef
Strata London 2019 Scaling Impala
Strata London 2019 Scaling Impala
Manish Maheshwari
Hive Does ACID
Hive Does ACID
DataWorks Summit
Performance Optimizations in Apache Impala
Performance Optimizations in Apache Impala
Cloudera, Inc.
Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
Timothy Spann
Apache Spark-Bench: Simulate, Test, Compare, Exercise, and Yes, Benchmark wit...
Apache Spark-Bench: Simulate, Test, Compare, Exercise, and Yes, Benchmark wit...
Spark Summit
Hybrid Apache Spark Architecture with YARN and Kubernetes
Hybrid Apache Spark Architecture with YARN and Kubernetes
Databricks
A Closer Look at Apache Kudu
A Closer Look at Apache Kudu
Andriy Zabavskyy
The Impala Cookbook
The Impala Cookbook
Cloudera, Inc.
Evening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in Flink
Flink Forward
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
DataStax
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
Databricks
Tame the small files problem and optimize data layout for streaming ingestion...
Tame the small files problem and optimize data layout for streaming ingestion...
Flink Forward
Apache Kudu: Technical Deep Dive
Apache Kudu: Technical Deep Dive
Cloudera, Inc.
Cloudera Impala Source Code Explanation and Analysis
Cloudera Impala Source Code Explanation and Analysis
Yue Chen
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Databricks
Spark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka Streams
Guido Schmutz
TFA Collector - what can one do with it
TFA Collector - what can one do with it
Sandesh Rao
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Anya Bida
2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs
Marcus Vinicius Miguel Pedro
Cosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle Service
Databricks
Tuning Apache Kafka Connectors for Flink.pptx
Tuning Apache Kafka Connectors for Flink.pptx
Flink Forward
Cloudera Impala Internals
Cloudera Impala Internals
David Groozman
Airflow Clustering and High Availability
Airflow Clustering and High Availability
Robert Sanders
Introduction to Storm
Introduction to Storm
Chandler Huang
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
Alluxio, Inc.
Using Apache Hive with High Performance
Using Apache Hive with High Performance
Inderaj (Raj) Bains
Presto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performance
DataWorks Summit
Cloudera Impala technical deep dive
Cloudera Impala technical deep dive
huguk
Apache Impala (incubating) 2.5 Performance Update
Apache Impala (incubating) 2.5 Performance Update
Cloudera, Inc.
Weitere ähnliche Inhalte
Was ist angesagt?
The Impala Cookbook
The Impala Cookbook
Cloudera, Inc.
Evening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in Flink
Flink Forward
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
DataStax
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
Databricks
Tame the small files problem and optimize data layout for streaming ingestion...
Tame the small files problem and optimize data layout for streaming ingestion...
Flink Forward
Apache Kudu: Technical Deep Dive
Apache Kudu: Technical Deep Dive
Cloudera, Inc.
Cloudera Impala Source Code Explanation and Analysis
Cloudera Impala Source Code Explanation and Analysis
Yue Chen
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Databricks
Spark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka Streams
Guido Schmutz
TFA Collector - what can one do with it
TFA Collector - what can one do with it
Sandesh Rao
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Anya Bida
2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs
Marcus Vinicius Miguel Pedro
Cosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle Service
Databricks
Tuning Apache Kafka Connectors for Flink.pptx
Tuning Apache Kafka Connectors for Flink.pptx
Flink Forward
Cloudera Impala Internals
Cloudera Impala Internals
David Groozman
Airflow Clustering and High Availability
Airflow Clustering and High Availability
Robert Sanders
Introduction to Storm
Introduction to Storm
Chandler Huang
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
Alluxio, Inc.
Using Apache Hive with High Performance
Using Apache Hive with High Performance
Inderaj (Raj) Bains
Presto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performance
DataWorks Summit
Was ist angesagt?
(20)
The Impala Cookbook
The Impala Cookbook
Evening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in Flink
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
Tame the small files problem and optimize data layout for streaming ingestion...
Tame the small files problem and optimize data layout for streaming ingestion...
Apache Kudu: Technical Deep Dive
Apache Kudu: Technical Deep Dive
Cloudera Impala Source Code Explanation and Analysis
Cloudera Impala Source Code Explanation and Analysis
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Spark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka Streams
TFA Collector - what can one do with it
TFA Collector - what can one do with it
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs
Cosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle Service
Tuning Apache Kafka Connectors for Flink.pptx
Tuning Apache Kafka Connectors for Flink.pptx
Cloudera Impala Internals
Cloudera Impala Internals
Airflow Clustering and High Availability
Airflow Clustering and High Availability
Introduction to Storm
Introduction to Storm
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
Using Apache Hive with High Performance
Using Apache Hive with High Performance
Presto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performance
Andere mochten auch
Cloudera Impala technical deep dive
Cloudera Impala technical deep dive
huguk
Apache Impala (incubating) 2.5 Performance Update
Apache Impala (incubating) 2.5 Performance Update
Cloudera, Inc.
Nested Types in Impala
Nested Types in Impala
Cloudera, Inc.
Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5
Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5
Cloudera, Inc.
How Impala Works
How Impala Works
Yue Chen
Data Infused Product Design and Insights at LinkedIn
Data Infused Product Design and Insights at LinkedIn
Yael Garten
White paper hadoop performancetuning
White paper hadoop performancetuning
Anil Reddy
A Perspective from the intersection Data Science, Mobility, and Mobile Devices
A Perspective from the intersection Data Science, Mobility, and Mobile Devices
Yael Garten
Remix: On-demand Live Randomization (Fine-grained live ASLR during runtime)
Remix: On-demand Live Randomization (Fine-grained live ASLR during runtime)
Yue Chen
Impala SQL Support
Impala SQL Support
Yue Chen
Hadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorial
hadooparchbook
How to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organization
Yael Garten
SecPod: A Framework for Virtualization-based Security Systems
SecPod: A Framework for Virtualization-based Security Systems
Yue Chen
Data Modeling for Data Science: Simplify Your Workload with Complex Types in ...
Data Modeling for Data Science: Simplify Your Workload with Complex Types in ...
Cloudera, Inc.
Impala use case @ Zoosk
Impala use case @ Zoosk
Cloudera, Inc.
Architecting next generation big data platform
Architecting next generation big data platform
hadooparchbook
Faster Batch Processing with Cloudera 5.7: Hive-on-Spark is ready for production
Faster Batch Processing with Cloudera 5.7: Hive-on-Spark is ready for production
Cloudera, Inc.
Query Compilation in Impala
Query Compilation in Impala
Cloudera, Inc.
What no one tells you about writing a streaming app
What no one tells you about writing a streaming app
hadooparchbook
Hoodie: Incremental processing on hadoop
Hoodie: Incremental processing on hadoop
Prasanna Rajaperumal
Andere mochten auch
(20)
Cloudera Impala technical deep dive
Cloudera Impala technical deep dive
Apache Impala (incubating) 2.5 Performance Update
Apache Impala (incubating) 2.5 Performance Update
Nested Types in Impala
Nested Types in Impala
Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5
Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5
How Impala Works
How Impala Works
Data Infused Product Design and Insights at LinkedIn
Data Infused Product Design and Insights at LinkedIn
White paper hadoop performancetuning
White paper hadoop performancetuning
A Perspective from the intersection Data Science, Mobility, and Mobile Devices
A Perspective from the intersection Data Science, Mobility, and Mobile Devices
Remix: On-demand Live Randomization (Fine-grained live ASLR during runtime)
Remix: On-demand Live Randomization (Fine-grained live ASLR during runtime)
Impala SQL Support
Impala SQL Support
Hadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorial
How to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organization
SecPod: A Framework for Virtualization-based Security Systems
SecPod: A Framework for Virtualization-based Security Systems
Data Modeling for Data Science: Simplify Your Workload with Complex Types in ...
Data Modeling for Data Science: Simplify Your Workload with Complex Types in ...
Impala use case @ Zoosk
Impala use case @ Zoosk
Architecting next generation big data platform
Architecting next generation big data platform
Faster Batch Processing with Cloudera 5.7: Hive-on-Spark is ready for production
Faster Batch Processing with Cloudera 5.7: Hive-on-Spark is ready for production
Query Compilation in Impala
Query Compilation in Impala
What no one tells you about writing a streaming app
What no one tells you about writing a streaming app
Hoodie: Incremental processing on hadoop
Hoodie: Incremental processing on hadoop
Ähnlich wie Admission Control in Impala
Impala Resource Management - OUTDATED
Impala Resource Management - OUTDATED
Matthew Jacobs
Strata London 2019 Scaling Impala.pptx
Strata London 2019 Scaling Impala.pptx
Manish Maheshwari
YARN
YARN
Alex Moundalexis
Troubleshooting Hadoop: Distributed Debugging
Troubleshooting Hadoop: Distributed Debugging
Great Wide Open
Rev Up Your HPC Engine
Rev Up Your HPC Engine
inside-BigData.com
Kudu: Resolving Transactional and Analytic Trade-offs in Hadoop
Kudu: Resolving Transactional and Analytic Trade-offs in Hadoop
jdcryans
NGENSTOR_ODA_P2V_V5
NGENSTOR_ODA_P2V_V5
UniFabric
Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu
Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu
Jeremy Beard
London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server
London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server
JBUG London
Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise (dated - Dec -2014)
Govind Kanshi
Mtc learnings from isv & enterprise interaction
Mtc learnings from isv & enterprise interaction
Govind Kanshi
IBM MQ - High Availability and Disaster Recovery
IBM MQ - High Availability and Disaster Recovery
MarkTaylorIBM
MySQL Enterprise Backup apr 2016
MySQL Enterprise Backup apr 2016
Ted Wennmark
Performance tuning Grails applications SpringOne 2GX 2014
Performance tuning Grails applications SpringOne 2GX 2014
Lari Hotari
Postgresql in Education
Postgresql in Education
dostatni
IBM MQ High Availabillity and Disaster Recovery (2017 version)
IBM MQ High Availabillity and Disaster Recovery (2017 version)
MarkTaylorIBM
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
In-Memory Computing Summit
Updated Power of the AWR Warehouse, Dallas, HQ, etc.
Updated Power of the AWR Warehouse, Dallas, HQ, etc.
Kellyn Pot'Vin-Gorman
Eventual Consistency @WalmartLabs with Kafka, Avro, SolrCloud and Hadoop
Eventual Consistency @WalmartLabs with Kafka, Avro, SolrCloud and Hadoop
Ayon Sinha
Performance Tuning
Performance Tuning
Jannet Peetz
Ähnlich wie Admission Control in Impala
(20)
Impala Resource Management - OUTDATED
Impala Resource Management - OUTDATED
Strata London 2019 Scaling Impala.pptx
Strata London 2019 Scaling Impala.pptx
YARN
YARN
Troubleshooting Hadoop: Distributed Debugging
Troubleshooting Hadoop: Distributed Debugging
Rev Up Your HPC Engine
Rev Up Your HPC Engine
Kudu: Resolving Transactional and Analytic Trade-offs in Hadoop
Kudu: Resolving Transactional and Analytic Trade-offs in Hadoop
NGENSTOR_ODA_P2V_V5
NGENSTOR_ODA_P2V_V5
Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu
Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu
London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server
London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server
Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise interaction
Mtc learnings from isv & enterprise interaction
IBM MQ - High Availability and Disaster Recovery
IBM MQ - High Availability and Disaster Recovery
MySQL Enterprise Backup apr 2016
MySQL Enterprise Backup apr 2016
Performance tuning Grails applications SpringOne 2GX 2014
Performance tuning Grails applications SpringOne 2GX 2014
Postgresql in Education
Postgresql in Education
IBM MQ High Availabillity and Disaster Recovery (2017 version)
IBM MQ High Availabillity and Disaster Recovery (2017 version)
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
Updated Power of the AWR Warehouse, Dallas, HQ, etc.
Updated Power of the AWR Warehouse, Dallas, HQ, etc.
Eventual Consistency @WalmartLabs with Kafka, Avro, SolrCloud and Hadoop
Eventual Consistency @WalmartLabs with Kafka, Avro, SolrCloud and Hadoop
Performance Tuning
Performance Tuning
Mehr von Cloudera, Inc.
Partner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptx
Cloudera, Inc.
Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists
Cloudera, Inc.
2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists
Cloudera, Inc.
Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019
Cloudera, Inc.
Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19
Cloudera, Inc.
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Cloudera, Inc.
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19
Cloudera, Inc.
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Cloudera, Inc.
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Cloudera, Inc.
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19
Cloudera, Inc.
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Cloudera, Inc.
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18
Cloudera, Inc.
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3
Cloudera, Inc.
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2
Cloudera, Inc.
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1
Cloudera, Inc.
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the Platform
Cloudera, Inc.
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18
Cloudera, Inc.
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360
Cloudera, Inc.
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18
Cloudera, Inc.
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18
Cloudera, Inc.
Mehr von Cloudera, Inc.
(20)
Partner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptx
Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists
2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists
Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019
Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the Platform
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18
Kürzlich hochgeladen
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Bert Jan Schrijver
%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Harare
masabamasaba
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
masabamasaba
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
masabamasaba
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
masabamasaba
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
masabamasaba
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
Jittipong Loespradit
tonesoftg
tonesoftg
lanshi9
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
masabamasaba
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
SelfMade bd
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK Software
Jim McKeeth
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
Delhi Call girls
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
masabamasaba
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
WSO2Con2024 - Enabling Transactional System's Exponential Growth With Simplicity
WSO2Con2024 - Enabling Transactional System's Exponential Growth With Simplicity
WSO2
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdf
Willy Marroquin (WillyDevNET)
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
masabamasaba
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
panagenda
Kürzlich hochgeladen
(20)
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Harare
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
tonesoftg
tonesoftg
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK Software
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
WSO2Con2024 - Enabling Transactional System's Exponential Growth With Simplicity
WSO2Con2024 - Enabling Transactional System's Exponential Growth With Simplicity
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdf
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
Admission Control in Impala
1.
1 Admission Control in
Impala Matthew Jacobs | @mattjacobs | mj@cloudera.com
2.
2 ©2014 Cloudera,
Inc. All rights reserved. • Too many concurrent queries -> oversubscription • All queries take more time • Application layer can throttle queries? • Not much you can do before Impala 1.3 What’s the Problem
3.
3 ©2014 Cloudera,
Inc. All rights reserved. • Add an admission control mechanism to Impala! • Throttle incoming requests • Queues requests when workload increases • Queued requests executed when resources available So what do we do?
4.
4 ©2014 Cloudera,
Inc. All rights reserved. • Yarn is a resource manager for Hadoop • Assumes jobs are composed of tasks, tasks can be restarted • Need to ask for all resources up front, resources “trickle in” • Non-trivial overhead: job creates “application master” (AM) • But cost is small compared to long batch jobs • Great for MR, things like MR • Not good for • Low-latency, high volume workloads • Gang scheduling, “parts of jobs” can’t be restarted What about Yarn?
5.
5 ©2014 Cloudera,
Inc. All rights reserved. • “Long Lived Application Master” • Long running AMs • Create fake requests to acquire necessary resources • Provides a “gang scheduling” abstraction, waits for all resources • Offers a resource expansion mechanism -> don’t need to ask for all up front • Offers a throttling mechanism • Caches Yarn containers -> lower latency • Looks like a square peg in a round hole… • To be fair, multi-level scheduling is a hard problem! Llama Bridges the Gap
6.
6 ©2014 Cloudera,
Inc. All rights reserved. • Good for Impala sharing resources with other frameworks • Good general purpose resource mgmt solution However: • Not everyone wants/needs to run Yarn and Llama • Still requires round-trips to a central server • Increases query latency • Unlikely to scale for highest latency/throughput requirements • Impala should have a fast, built in throttling mechanism Impala + Llama + Yarn?
7.
7 ©2014 Cloudera,
Inc. All rights reserved. • Throttle number of concurrent requests or memory • Fast • Decentralized • Works without Yarn/Llama • Works with CDH4/CDH5 Impala Admission Control
8.
8 ©2014 Cloudera,
Inc. All rights reserved. • Configure one or more resource “pools” • Max # concurrent queries, max memory, max queue size • Each Impalad capable of making admission decisions • No new single bottleneck/single point of failure • Incoming queries are executed, queued, or rejected • Queue if too many queries OR not enough memory • Reject if queue is full Design Overview
9.
9 ©2014 Cloudera,
Inc. All rights reserved. • Requests admitted or queued locally • Each Impalad keeps track of local state • # queries, pool memory, local queue size • Disseminates local stats via statestore -> global state • Uses cached global state in admission decisions • Decisions are fast; negligible impact on query latency • No single point of failure Localized Admission Decisions
10.
10 ©2014 Cloudera,
Inc. All rights reserved. • Using cached global state -> may “over-admit” • E.g. multiple impalads think 1 request can be admitted and admit before receiving updated state • Configured pool limits are “soft” limits • Fn(Submission rate, distribution across impalads) • Not a big problem in practice • May occasionally admit a few extra queries • Can increase statestore heartbeat frequency • Can add some buffer to configured pool limits Localized Admission Decisions (II)
11.
11 ©2014 Cloudera,
Inc. All rights reserved. • Max memory • Many workloads are limited by memory • Impalads kill queries when running out of memory, anyway • Max number of concurrent queries • Generic mechanism, not resource specific (e.g. memory) • Not as good if workload is heterogeneous • Queries may still be killed if impalads run out of memory Pool Limits
12.
12 Memory Limits • Impalads
track memory hierarchically • Per-process memory • Queries killed when limit is hit • Per-pool memory • For admission control • Per-query memory Process Pool1 Query1 Query2 Pool2 ©2014 Cloudera, Inc. All rights reserved.
13.
13 ©2014 Cloudera,
Inc. All rights reserved. • Admission decisions need more than memory usage • Incoming queries use no memory yet • Queries recently admitted haven’t ramped up yet • Use memory estimates from planning • Estimate pool memory usage with actual usage & estimates • Accounts for future memory usage of recently started queries Admit if: Pool mem estimate + query mem estimate < pool limit Memory Limits (II)
14.
14 ©2014 Cloudera,
Inc. All rights reserved. • Not perfect, query mem estimates are wrong • Hard problem; never have perfect estimates from planning • Usually overly conservative • Leads to underutilization • But at least queries won’t be killed • Less likely to hit process mem limit • Workarounds • Increase pool mem limit • Override with “MEM_LIMIT” query option • Future improvement: Update estimates as query executes • Query mem usage will approach the updated estimate Memory Limits (III)
15.
15 ©2014 Cloudera,
Inc. All rights reserved. • Modeled after Yarn resource queues • Same configuration as Yarn queues • Have a single configuration for Yarn & Impala • Usually want to have the same resource allocations mapped to an organization • E.g. HR gets 10%, Finance gets 30%, Eng gets 60% Request Pools
16.
16 ©2014 Cloudera,
Inc. All rights reserved. • Users are mapped to pools using the placement policy • Users are authorized using the specified ACLs • Pools are defined hierarchically • ACLs are inherited • Currently only enforces limits on leaf pools (IMPALA-905) Request Pools (II)
17.
17 ©2014 Cloudera,
Inc. All rights reserved. • Uses Yarn + Llama configs • Yarn fair scheduler allocation configuration (fair- scheduler.xml) • Llama configuration (llama-site.xml) • Only some of the configuration properties are used • See the documentation for sample config files • Cloudera Manager has a nice UI to configure • No need to touch the xml files Request Pool General Configuration
18.
18 ©2014 Cloudera,
Inc. All rights reserved.
19.
19 ©2014 Cloudera,
Inc. All rights reserved. Placement Rule Configuration Please change the default values
20.
20 ©2014 Cloudera,
Inc. All rights reserved. • If only 1 pool is needed, a separate (easy) configuration path exists • Uses a single “default” pool • No Yarn/Llama configs involved (not even accepted) • Configure the pool limits with impalad flags: • default_pool_max_queued • default_pool_max_requests • default_pool_mem_limit • Doesn’t work with CM5.0, fixed in CM5.0.1 Easy Config Path (Singleton Pool Only)
21.
21 ©2014 Cloudera,
Inc. All rights reserved. Submitting to a Pool
22.
22 ©2014 Cloudera,
Inc. All rights reserved. • Rejections and timeouts return error messages • Metrics • Exposed in impalad web UI: /metrics • Will be available in CM5.1 • Query profile has admission result • Impalad logs have lots of useful information “Debugging” Admission Control Decisions admission-controller.cc:259] Schedule for id=c541aae43af74ed1:afdec812127f8097 in pool_name=root.test/admin PoolConfig(max_requests=20 max_queued=50 mem_limit=-1.00 B) query cluster_mem_estimate=42.00 MB admission-controller.cc:265] Stats: pool=root.test/admin Total(num_running=20, num_queued=7, mem_usage=239.07 MB, mem_estimate=800.00 MB) Local(num_running=20, num_queued=7, mem_usage=239.07 MB, mem_estimate=800.00 MB) admission-controller.cc:303] Queuing, query id=c541aae43af74ed1:afdec812127f8097
23.
23 ©2014 Cloudera,
Inc. All rights reserved. Metrics
24.
24 ©2014 Cloudera,
Inc. All rights reserved. Query Profile Information
25.
25 ©2014 Cloudera,
Inc. All rights reserved. • Queue timeout • Defaults to 60sec, change with --queue_wait_timeout_ms • Running with Yarn/Llama • Same configs: “hard limits” enforced by Yarn+Llama • Disabled by default for CDH4 • Hue (<CDH4.6) doesn’t close queries • Enable with impalad flag (see --disable_admission_control) Some Notes
26.
26 ©2014 Cloudera,
Inc. All rights reserved. Matthew Jacobs @mattjacobs mj@cloudera.com