High Volume Updates in Apache Hive

•Als PPTX, PDF herunterladen•

10 gefällt mir•5,282 views

Apache Hive provides a convenient SQL-based query language to data stored in HDFS. HDFS provides highly scaleable bandwidth to the data, but does not support arbitrary writes. One of Hortonworks` customers needs to store a high volume of customer data (> 1 TB/day) and that data contains a high percentage (15%) of record updates distributed across years. In many high-update use-cases, HBase would suffice, but the current lack of push down filters from Hive into HBase and HBase`s single level keys make it too expensive. Our solution is to use a custom record reader that stores the edit records as separate HDFS files and synthesizes the current set of records dynamically as the table is read. This provides an economical solution to their need that works within the framework provided by Hive. We believe this use case applies to many Hive users and plan to develop and open source a reusable solution.

Technologie Business

High Volume Updates in Hive
Owen O’Malley
owen@hortonworks.com
@owen_omalley
June 2012

© Hortonworks Inc. 2012 Page 1

Who Am I?

Page 2
© Hortonworks Inc. 2012

A Data Flood

Page 3
© Hortonworks Inc. 2012

The Dataflow

Page 4
© Hortonworks Inc. 2012

The Approach

Page 5
© Hortonworks Inc. 2012

Why not Hbase?

Page 6
© Hortonworks Inc. 2012

Limitations of a Single Key

Page 7
© Hortonworks Inc. 2012

Hive Table Layout

Page 8
© Hortonworks Inc. 2012

Design

Page 9
© Hortonworks Inc. 2012

Repeatable Reads

Page 10
© Hortonworks Inc. 2012

Stitching Buckets Together

Page 11
© Hortonworks Inc. 2012

Limitations

Page 12
© Hortonworks Inc. 2012

Additional Challenges from Hive

Page 13
© Hortonworks Inc. 2012

Hive’s Output Committer

Page 14
© Hortonworks Inc. 2012

Dynamic Partitions

Page 15
© Hortonworks Inc. 2012

Conclusion

Page 16
© Hortonworks Inc. 2012

Thank You!
Questions & Answers

Page 17
© Hortonworks Inc. 2012

Weitere ähnliche Inhalte

Andere mochten auch

Next Generation Hadoop Operations

Next Generation Hadoop Operations

Next Generation Hadoop Operations

Optimizing Hive Queries

Optimizing Hive Queries

Optimizing Hive Queries

Hive’s RCFile has been the standard format for storing Hive data for the last 3 years. However, RCFile has limitations because it treats each column as a binary blob without semantics. The upcoming Hive 0.11 will add a new file format named Optimized Row Columnar (ORC) file that uses and retains the type information from the table definition. ORC uses type specific readers and writers that provide light weight compression techniques such as dictionary encoding, bit packing, delta encoding, and run length encoding -- resulting in dramatically smaller files. Additionally, ORC can apply generic compression using zlib, LZO, or Snappy on top of the lightweight compression for even smaller files. However, storage savings are only part of the gain. ORC supports projection, which selects subsets of the columns for reading, so that queries reading only one column read only the required bytes. Furthermore, ORC files include light weight indexes that include the minimum and maximum values for each column in each set of 10,000 rows and the entire file. Using pushdown filters from Hive, the file reader can skip entire sets of rows that aren’t important for this query. Finally, ORC works together with the upcoming query vectorization work providing a high bandwidth reader/writer interface.

ORC Files

Hadoop Summit June 2016 The landscape for storing your big data is quite complex, with several competing formats and different implementations of each format. Understanding your use of the data is critical for picking the format. Depending on your use case, the different formats perform very differently. Although you can use a hammer to drive a screw, it isn’t fast or easy to do so. The use cases that we’ve examined are: * reading all of the columns * reading a few of the columns * filtering using a filter predicate * writing the data Furthermore, it is important to benchmark on real data rather than synthetic data. We used the Github logs data available freely from http://githubarchive.org We will make all of the benchmark code open source so that our experiments can be replicated.

File Format Benchmarks - Avro, JSON, ORC, & Parquet

File Format Benchmarks - Avro, JSON, ORC, & Parquet

File Format Benchmarks - Avro, JSON, ORC, & Parquet

Hive Quick Start Tutorial

Hive Quick Start Tutorial

Hive Quick Start Tutorial

Eric Hanson and I gave this presentation at Hadoop Summit 2013: Hive’s RCFile has been the standard format for storing Hive data for the last 3 years. However, RCFile has limitations because it treats each column as a binary blob without semantics. Hive 0.11 added a new file format named Optimized Row Columnar (ORC) file that uses and retains the type information from the table definition. ORC uses type specific readers and writers that provide light weight compression techniques such as dictionary encoding, bit packing, delta encoding, and run length encoding — resulting in dramatically smaller files. Additionally, ORC can apply generic compression using zlib, LZO, or Snappy on top of the lightweight compression for even smaller files. However, storage savings are only part of the gain. ORC supports projection, which selects subsets of the columns for reading, so that queries reading only one column read only the required bytes. Furthermore, ORC files include light weight indexes that include the minimum and maximum values for each column in each set of 10,000 rows and the entire file. Using pushdown filters from Hive, the file reader can skip entire sets of rows that aren’t important for this query. Columnar storage formats like ORC reduce I/O and storage use, but it’s just as important to reduce CPU usage. A technical breakthrough called vectorized query execution works nicely with column store formats to do this. Vectorized query execution has proven to give dramatic performance speedups, on the order of 10X to 100X, for structured data processing. We describe how we’re adding vectorized query execution to Hive, coupling it with ORC with a vectorized iterator.

ORC File and Vectorization - Hadoop Summit 2013

ORC File and Vectorization - Hadoop Summit 2013

ORC File and Vectorization - Hadoop Summit 2013

Hadoop Security Architecture

Hadoop Security Architecture

Hadoop Security Architecture

Apache Spark Use case for Education Industry

Apache Spark Use case for Education Industry

Apache Spark Use case for Education Industry

Vinayak Agrawal

Cancer Outlier Profile Analysis using Apache Spark

Mahmoud Parsian

How Totango uses Apache Spark

How Totango uses Apache Spark

How Totango uses Apache Spark

Getting Apache Spark Customers to Production

Getting Apache Spark Customers to Production

Getting Apache Spark Customers to Production

Hive User Meeting March 2010 - Hive Team

Hive User Meeting March 2010 - Hive Team

Hive User Meeting March 2010 - Hive Team

Kodu Game Lab e Project Spark

Kodu Game Lab e Project Spark

Kodu Game Lab e Project Spark

Fabrício Catae

Apache Hive is a data warehousing system for large volumes of data stored in Hadoop. However, the data is useless unless you can use it to add value to your company. Hive provides a SQL-based query language that dramatically simplifies the process of querying your large data sets. That is especially important while your data scientists are developing and refining their queries to improve their understanding of the data. In many companies, such as Facebook, Hive accounts for a large percentage of the total MapReduce queries that are run on the system. Although Hive makes writing large data queries easier for the user, there are many performance traps for the unwary. Many of them are artifacts of the way Hive has evolved over the years and the requirement that the default behavior must be safe for all users. This talk will present examples of how Hive users have made mistakes that made their queries run much much longer than necessary. It will also present guidelines for how to get better performance for your queries and how to look at the query plan to understand what Hive is doing.

Optimizing Hive Queries

Optimizing Hive Queries

Optimizing Hive Queries

DataWorks Summit

ORC 2015

Fighting Fraud with Apache Spark

Fighting Fraud with Apache Spark

Fighting Fraud with Apache Spark

Miklos Christine

Record Linkage, un cas d’utilisation en Spark ML par Alexis Seigneurin Le Record Linkage est le process qui consiste à trouver, dans un data set, les enregistrements qui représentent la même entité. Cette opération est particulièrement compliquée quand, comme nous, vous travaillez avec des données anonymisées. C’est là que le Machine Learning vient en renfort ! Nous avons implémenté un algorithme de Record Linkage en Spark SQL (DataFrames) et Spark ML plutôt que d’utiliser des règles statiques. Nous verrons le process de Feature Engineering, pourquoi nous avons dû étendre Spark DataFrames pour préserver des méta-données au travers du pipeline de traitement, et comment nous avons utilisé le Machine Learning pour réconcilier les enregistrements. Nous verrons enfin comment nous avons industrialisé cette application. Alexis Seigneurin : Développeur depuis 15 ans, j'attache beaucoup d'importance aux problématiques de traitement, d'analyse et de stockage de la donnée.Chez Ippon, j'interviens principalement sur des missions de conseil et d'architecture autour de technologies big data. Par ailleurs, j'anime la formation Spark chez Ippon.

Record linkage, a real use case with spark ml - Paris Spark meetup Dec 2015

Record linkage, a real use case with spark ml - Paris Spark meetup Dec 2015

Record linkage, a real use case with spark ml - Paris Spark meetup Dec 2015

Modern Data Stack France

This session will cover a series of problems that are adequately solved with Apache Spark, as well as those that are require additional technologies to implement correctly. Here’s an example outline of some of the topics that will be covered in the talk: Problems that are perfectly solved with Apache Spark: 1) Analyzing a large set of data files. 2) Doing ETL of a large amount of data. 3) Applying Machine Learning & Data Science to a large dataset. 4) Connecting BI/Visualization tools to Apache Spark to analyze large datasets internally. By Vida Ha at Spark Summit East 2016.

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Traditionally, data warehouse platforms have been perceived as cost prohibitive, challenging to maintain and complex to scale. The combination of Apache Spark and Spark SQL – running on AWS – provides a fast, simple, and scalable way to build a new generation of data warehouses that revolutionizes how data scientists and engineers analyze their data sets. In this webinar you will learn how Databricks - a fully managed Spark platform hosted on AWS - integrates with variety of different AWS services, Amazon S3, Kinesis, and VPC. We’ll also show you how to build your own data warehousing platform in very short amount of time and how to integrate it with other tools such as Spark’s machine learning library and Spark streaming for real-time processing of your data.

Building a Turbo-fast Data Warehousing Platform with Databricks

Building a Turbo-fast Data Warehousing Platform with Databricks

Building a Turbo-fast Data Warehousing Platform with Databricks

Andere mochten auch (19)

Next Generation Hadoop Operations

Next Generation Hadoop Operations

Next Generation Hadoop Operations

Optimizing Hive Queries

Optimizing Hive Queries

Optimizing Hive Queries

ORC Files

File Format Benchmarks - Avro, JSON, ORC, & Parquet

File Format Benchmarks - Avro, JSON, ORC, & Parquet

File Format Benchmarks - Avro, JSON, ORC, & Parquet

Hive Quick Start Tutorial

Hive Quick Start Tutorial

Hive Quick Start Tutorial

ORC File and Vectorization - Hadoop Summit 2013

ORC File and Vectorization - Hadoop Summit 2013

ORC File and Vectorization - Hadoop Summit 2013

Hadoop Security Architecture

Hadoop Security Architecture

Hadoop Security Architecture

Apache Spark Use case for Education Industry

Apache Spark Use case for Education Industry

Apache Spark Use case for Education Industry

Cancer Outlier Profile Analysis using Apache Spark

How Totango uses Apache Spark

How Totango uses Apache Spark

How Totango uses Apache Spark

Getting Apache Spark Customers to Production

Getting Apache Spark Customers to Production

Getting Apache Spark Customers to Production

Hive User Meeting March 2010 - Hive Team

Hive User Meeting March 2010 - Hive Team

Hive User Meeting March 2010 - Hive Team

Kodu Game Lab e Project Spark

Kodu Game Lab e Project Spark

Kodu Game Lab e Project Spark

Optimizing Hive Queries

Optimizing Hive Queries

Optimizing Hive Queries

ORC 2015

Fighting Fraud with Apache Spark

Fighting Fraud with Apache Spark

Fighting Fraud with Apache Spark

Record linkage, a real use case with spark ml - Paris Spark meetup Dec 2015

Record linkage, a real use case with spark ml - Paris Spark meetup Dec 2015

Record linkage, a real use case with spark ml - Paris Spark meetup Dec 2015

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Building a Turbo-fast Data Warehousing Platform with Databricks

Building a Turbo-fast Data Warehousing Platform with Databricks

Building a Turbo-fast Data Warehousing Platform with Databricks

Mehr von Owen O'Malley

Running An Apache Project: 10 Traps and How to Avoid Them

Running An Apache Project: 10 Traps and How to Avoid Them

Running An Apache Project: 10 Traps and How to Avoid Them

Big Data's Journey to ACID

Big Data's Journey to ACID

Big Data's Journey to ACID

ORC Deep Dive 2020

ORC Deep Dive 2020

ORC Deep Dive 2020

Fine-grained data protection at a column level in data lake environments has become a mandatory requirement to demonstrate compliance with multiple local and international regulations across many industries today. ORC is a self-describing type-aware columnar file format designed for Hadoop workloads that provides optimized streaming reads but with integrated support for finding required rows quickly. Owen O’Malley dives into the progress the Apache community made for adding fine-grained column-level encryption natively into ORC format, which also provides capabilities to mask or redact data on write while protecting sensitive column metadata such as statistics to avoid information leakage. The column encryption capabilities will be fully compatible with Hadoop Key Management Server (KMS) and use the KMS to manage master keys, providing the additional flexibility to use and manage keys per column centrally.

Protect your private data with ORC column encryption

Protect your private data with ORC column encryption

Protect your private data with ORC column encryption

Fine-grained data protection at a column level in data lake environments has become a mandatory requirement to demonstrate compliance with multiple local and international regulations across many industries today. ORC is a self-describing type-aware columnar file format designed for Hadoop workloads that provides optimized streaming reads, but with integrated support for finding required rows quickly. In this talk, we will outline the progress made in Apache community for adding fine-grained column level encryption natively into ORC format that will also provide capabilities to mask or redact data on write while protecting sensitive column metadata such as statistics to avoid information leakage. The column encryption capabilities will be fully compatible with Hadoop Key Management Server (KMS) and use the KMS to manage master keys providing the additional flexibility to use and manage keys per column centrally.

Fine Grain Access Control for Big Data: ORC Column Encryption

Fine Grain Access Control for Big Data: ORC Column Encryption

Fine Grain Access Control for Big Data: ORC Column Encryption

The landscape for storing your big data is quite complex, with several competing formats and different implementations of each format. Understanding your use of the data is critical for picking the format. Depending on your use case, the different formats perform very differently. Although you can use a hammer to drive a screw, it isn’t fast or easy to do so. The use cases that we’ve examined are: * reading all of the columns * reading a few of the columns * filtering using a filter predicate While previous work has compared the size and speed from Hive, this presentation will present benchmarks from Spark including the new work that radically improves the performance of Spark on ORC. This presentation will also include tips and suggestions to optimize the performance of your application while reading and writing the data. Finally, the value of having open source benchmarks that are available to all interested parties is hugely important and all of the code is available from Apache.

Fast Access to Your Data - Avro, JSON, ORC, and Parquet

Fast Access to Your Data - Avro, JSON, ORC, and Parquet

Fast Access to Your Data - Avro, JSON, ORC, and Parquet

Hive tables are an integral part of the big data ecosystem, but the simple directory-based design that made them ubiquitous is increasingly problematic. Netflix uses tables backed by S3 that, like other object stores, don’t fit this directory-based model: listings are much slower, renames are not atomic, and results are eventually consistent. Even tables in HDFS are problematic at scale, and reliable query behavior requires readers to acquire locks and wait. Owen O’Malley and Ryan Blue offer an overview of Iceberg, a new open source project that defines a new table layout addresses the challenges of current Hive tables, with properties specifically designed for cloud object stores, such as S3. Iceberg is an Apache-licensed open source project. It specifies the portable table format and standardizes many important features, including: * All reads use snapshot isolation without locking. * No directory listings are required for query planning. * Files can be added, removed, or replaced atomically. * Full schema evolution supports changes in the table over time. * Partitioning evolution enables changes to the physical layout without breaking existing queries. * Data files are stored as Avro, ORC, or Parquet. * Support for Spark, Hive, and Presto.

Strata NYC 2018 Iceberg

Strata NYC 2018 Iceberg

Strata NYC 2018 Iceberg

The landscape for storing your big data is quite complex, with several competing formats and different implementations of each format. Understanding your use of the data is critical for picking the format. Depending on your use case, the different formats perform very differently. Although you can use a hammer to drive a screw, it isn’t fast or easy to do so. The use cases that we’ve examined are: reading all of the columns reading a few of the columns filtering using a filter predicate While previous work has compared the size and speed from Hive, this presentation will present benchmarks from Spark including the new work that radically improves the performance of Spark on ORC. This presentation will also include tips and suggestions to optimize the performance of your application while reading and writing the data.

Fast Spark Access To Your Complex Data - Avro, JSON, ORC, and Parquet

Fast Spark Access To Your Complex Data - Avro, JSON, ORC, and Parquet

Fast Spark Access To Your Complex Data - Avro, JSON, ORC, and Parquet

ORC Column Encryption

ORC Column Encryption

ORC Column Encryption

Mehr von Owen O'Malley (9)

Running An Apache Project: 10 Traps and How to Avoid Them

Running An Apache Project: 10 Traps and How to Avoid Them

Running An Apache Project: 10 Traps and How to Avoid Them

Big Data's Journey to ACID

Big Data's Journey to ACID

Big Data's Journey to ACID

ORC Deep Dive 2020

ORC Deep Dive 2020

ORC Deep Dive 2020

Protect your private data with ORC column encryption

Protect your private data with ORC column encryption

Protect your private data with ORC column encryption

Fine Grain Access Control for Big Data: ORC Column Encryption

Fine Grain Access Control for Big Data: ORC Column Encryption

Fine Grain Access Control for Big Data: ORC Column Encryption

Fast Access to Your Data - Avro, JSON, ORC, and Parquet

Fast Access to Your Data - Avro, JSON, ORC, and Parquet

Fast Access to Your Data - Avro, JSON, ORC, and Parquet

Strata NYC 2018 Iceberg

Strata NYC 2018 Iceberg

Strata NYC 2018 Iceberg

Fast Spark Access To Your Complex Data - Avro, JSON, ORC, and Parquet

Fast Spark Access To Your Complex Data - Avro, JSON, ORC, and Parquet

Fast Spark Access To Your Complex Data - Avro, JSON, ORC, and Parquet

ORC Column Encryption

ORC Column Encryption

ORC Column Encryption

Kürzlich hochgeladen

Boost Fertility New Invention Ups Success Rates.pdf

Boost Fertility New Invention Ups Success Rates.pdf

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

Why Teams call analytics are critical to your entire business

Why Teams call analytics are critical to your entire business

ICT role in 21st century education and its challenges

ICT role in 21st century education and its challenges

ICT role in 21st century education and its challenges

rafiqahmad00786416

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the deployment of external web forms using Jotform for Bonterra Impact Management. This solution can be customized to your organization’s needs and deployed to support the common use cases below: - Intake and consent - Assessments - Surveys - Applications - Program registration Interested in deploying web form automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Jeffrey Haguewood

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

Powerful Google developer tools for immediate impact! (2023-24 C)

Powerful Google developer tools for immediate impact! (2023-24 C)

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Real Time Object Detection Using Open CV

Real Time Object Detection Using Open CV

Real Time Object Detection Using Open CV

FWD Group - Insurer Innovation Award 2024

FWD Group - Insurer Innovation Award 2024

FWD Group - Insurer Innovation Award 2024

The Digital Insurer

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

A Year of the Servo Reboot: Where Are We Now?

A Year of the Servo Reboot: Where Are We Now?

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Kürzlich hochgeladen (20)

Boost Fertility New Invention Ups Success Rates.pdf

Boost Fertility New Invention Ups Success Rates.pdf

Boost Fertility New Invention Ups Success Rates.pdf

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Why Teams call analytics are critical to your entire business

Why Teams call analytics are critical to your entire business

Why Teams call analytics are critical to your entire business

ICT role in 21st century education and its challenges

ICT role in 21st century education and its challenges

ICT role in 21st century education and its challenges

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Powerful Google developer tools for immediate impact! (2023-24 C)

Powerful Google developer tools for immediate impact! (2023-24 C)

Powerful Google developer tools for immediate impact! (2023-24 C)

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Real Time Object Detection Using Open CV

Real Time Object Detection Using Open CV

Real Time Object Detection Using Open CV

FWD Group - Insurer Innovation Award 2024

FWD Group - Insurer Innovation Award 2024

FWD Group - Insurer Innovation Award 2024

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

A Year of the Servo Reboot: Where Are We Now?

A Year of the Servo Reboot: Where Are We Now?

A Year of the Servo Reboot: Where Are We Now?

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

High Volume Updates in Apache Hive

1. High Volume Updates in Hive Owen O’Malley owen@hortonworks.com @owen_omalley June 2012 © Hortonworks Inc. 2012 Page 1

2. Who Am I? Page 2 © Hortonworks Inc. 2012

3. A Data Flood Page 3 © Hortonworks Inc. 2012

4. The Dataflow Page 4 © Hortonworks Inc. 2012

5. The Approach Page 5 © Hortonworks Inc. 2012

6. Why not Hbase? Page 6 © Hortonworks Inc. 2012

7. Limitations of a Single Key Page 7 © Hortonworks Inc. 2012

8. Hive Table Layout Page 8 © Hortonworks Inc. 2012

9. Design Page 9 © Hortonworks Inc. 2012

10. Repeatable Reads Page 10 © Hortonworks Inc. 2012

11. Stitching Buckets Together Page 11 © Hortonworks Inc. 2012

12. Limitations Page 12 © Hortonworks Inc. 2012

13. Additional Challenges from Hive Page 13 © Hortonworks Inc. 2012

14. Hive’s Output Committer Page 14 © Hortonworks Inc. 2012

15. Dynamic Partitions Page 15 © Hortonworks Inc. 2012

16. Conclusion Page 16 © Hortonworks Inc. 2012

17. Thank You! Questions & Answers Page 17 © Hortonworks Inc. 2012