Spark Summit EU 2015: Revolutionizing Big Data in the Enterprise with Spark

•

17 gefällt mir•7,432 views

CEO of Databricks, Ion Stoica, talks about the impact of Apache Spark in the enterprise.

Revolutionizing Big Data
in the Enterprise with
Spark
Ion Stoica
October 28,2015

We Have Seen a Lot
Worked with 100s companies to run Spark in production over five years
Collaboratewith all major Hadoop and Big Data vendors
2

How Does Spark Change Enterprise Big Data?
• Unifying data sources
• Unifying data processing
3

4
Unifying Data Sources

Need to process data from
• Multiple sources
• Different data stores and locations
• Different formats
Traditional solutions: ETL data into
data warehouse, …
Traditional Data Warehouses
ETL
Slow to access and combine data
Data Warehouse

6
Just-In-Time (JIT)
Data Warehouse

Process data in place or stream it
• No need to wait for data to be
ETLed
7
JIT Data Warehouse
ETL
Data Warehouse

Process data in place or stream it
• No need to wait for data to be
ETLed
Cachedata in memory or SSDs
8
JIT Data Warehouse
Low latency and easy to combine data: value!

Analogy
9
Stream/cache &
Play
Download &
Play

Analogy
10
ETL & Query
Data
Source A
ETL
Data Warehouse
Data
Source B
Data
Source B
Data
Source A
Data
Source B
Data
Source B
Stream/Cache + Query

12
Unifying Data Processing

Unified supportfor
• Batch
• Streaming
• ML/Graphs
• …
13
Spark: Unified Engine
GraphXMLlib
Core
Spark
Streaming
SparkSQL SparkR
Easy to manage, learn, and combine functionality

Analogy
First cellular
phones
Unified device
(smartphone)
Specialized
devices
Better Games Better GPSBetter Phone

Analogy
Batch processing Unified systemSpecialized systems
Real-time
analytics
Instant fraud
detection
Better Apps

Large On-line Service Company
Leverages
• Interactive query processing
• ML
and combines data from S3, Redshift, and HBase to provide
• data analyticsfor productmanagementteam
• advanced predictive analyticsto delivernew services(e.g.,
customized inventory displaystailored to each user)
16

17
Demo

Demo Setting
18
MLlib
Core
Spark
Streaming
SparkSQL
HDFS RedShift

Weitere ähnliche Inhalte

Was ist angesagt?

Insights into Customer Behavior from Clickstream Data by Ronald Nowling

Insights into Customer Behavior from Clickstream Data by Ronald Nowling

Insights into Customer Behavior from Clickstream Data by Ronald Nowling

PyCon Colombia 2020 Python for Data Analysis: Past, Present, and Future

PyCon Colombia 2020 Python for Data Analysis: Past, Present, and Future

PyCon Colombia 2020 Python for Data Analysis: Past, Present, and Future

Spark's Role in the Big Data Ecosystem (Spark Summit 2014)

Spark's Role in the Big Data Ecosystem (Spark Summit 2014)

Spark's Role in the Big Data Ecosystem (Spark Summit 2014)

Big Telco - Yousun Jeong

Big Telco - Yousun Jeong

Big Telco - Yousun Jeong

This session will cover a series of problems that are adequately solved with Apache Spark, as well as those that are require additional technologies to implement correctly. Here’s an example outline of some of the topics that will be covered in the talk: Problems that are perfectly solved with Apache Spark: 1) Analyzing a large set of data files. 2) Doing ETL of a large amount of data. 3) Applying Machine Learning & Data Science to a large dataset. 4) Connecting BI/Visualization tools to Apache Spark to analyze large datasets internally. By Vida Ha at Spark Summit East 2016.

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

What's new in pandas and the SciPy stack for financial users

What's new in pandas and the SciPy stack for financial users

What's new in pandas and the SciPy stack for financial users

Apache Arrow at DataEngConf Barcelona 2018

Apache Arrow at DataEngConf Barcelona 2018

Apache Arrow at DataEngConf Barcelona 2018

Apache Arrow: Open Source Standard Becomes an Enterprise Necessity

Apache Arrow: Open Source Standard Becomes an Enterprise Necessity

Apache Arrow: Open Source Standard Becomes an Enterprise Necessity

Clinical Suspecting at Scale Using PySpark

Clinical Suspecting at Scale Using PySpark

Clinical Suspecting at Scale Using PySpark

Building a Virtual Data Lake with Apache Arrow

Building a Virtual Data Lake with Apache Arrow

Building a Virtual Data Lake with Apache Arrow

Dremio Corporation

Improving data interoperability in Python and R

Improving data interoperability in Python and R

Improving data interoperability in Python and R

This talk discusses integrating common data science tools like Python pandas, scikit-learn, and R with MLlib, Spark’s distributed Machine Learning (ML) library. Integration is simple; migration to distributed ML can be done lazily; and scaling to big data can significantly improve accuracy. We demonstrate integration with a simple data science workflow. Data scientists often encounter scaling bottlenecks with single-machine ML tools. Yet the overhead in migrating to a distributed workflow can seem daunting. In this talk, we demonstrate such a migration, taking advantage of Spark and MLlib’s integration with common ML libraries. We begin with a small dataset which runs on a single machine. Increasing the size, we hit bottlenecks in various parts of the workflow: hyperparameter tuning, then ETL, and eventually the core learning algorithm. As we hit each bottleneck, we parallelize that part of the workflow using Spark and MLlib. As we increase the dataset and model size, we can see significant gains in accuracy. We end with results demonstrating the impressive scalability of MLlib algorithms. With accuracy comparable to traditional ML libraries, combined with state-of-the-art distributed scalability, MLlib is a valuable new tool for the modern data scientist.

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Data scientists spend too much of their time collecting, cleaning and wrangling data as well as curating and enriching it. Some of this work is inevitable due to the variety of data sources, but there are tools and frameworks that help automate many of these non-creative tasks. A unifying feature of these tools is support for rich metadata for data sets, jobs, and data policies. In this talk, I will introduce state-of-the-art tools for automating data science and I will show how you can use metadata to help automate common tasks in Data Science. I will also introduce a new architecture for extensible, distributed metadata in Hadoop, called Hops (Hadoop Open Platform-as-a-Service), and show how tinker-friendly metadata (for jobs, files, users, and projects) opens up new ways to build smarter applications.

Data Science with the Help of Metadata

Data Science with the Help of Metadata

Data Science with the Help of Metadata

Practical Medium Data Analytics with Python (10 Things I Hate About pandas, P...

Practical Medium Data Analytics with Python (10 Things I Hate About pandas, P...

Practical Medium Data Analytics with Python (10 Things I Hate About pandas, P...

Big Data Processing with Spark and .NET - Microsoft Ignite 2019

Big Data Processing with Spark and .NET - Microsoft Ignite 2019

Big Data Processing with Spark and .NET - Microsoft Ignite 2019

The Future of Real-Time in Spark

The Future of Real-Time in Spark

The Future of Real-Time in Spark

PyCon.DE / PyData Karlsruhe keynote: "Looking backward, looking forward"

PyCon.DE / PyData Karlsruhe keynote: "Looking backward, looking forward"

PyCon.DE / PyData Karlsruhe keynote: "Looking backward, looking forward"

Spark Meetup at Uber

Spark Meetup at Uber

Spark Meetup at Uber

How we, at eXelate, built an ETL pipeline for Elasticsearch using Spark, including : * Processing the data using Spark. * Indexing the processed data directly into Elasticsearch using elasticsearch-hadoop plugin-in for Spark. * Managing the flow using some of the services provided by AWS (EMR, Data Pipeline, etc.). The presentation includes some tips and discusses some of the pitfalls we encountered while setting-up this process.

Building an ETL pipeline for Elasticsearch using Spark

Building an ETL pipeline for Elasticsearch using Spark

Building an ETL pipeline for Elasticsearch using Spark

Was ist angesagt? (19)

Insights into Customer Behavior from Clickstream Data by Ronald Nowling

Insights into Customer Behavior from Clickstream Data by Ronald Nowling

Insights into Customer Behavior from Clickstream Data by Ronald Nowling

PyCon Colombia 2020 Python for Data Analysis: Past, Present, and Future

PyCon Colombia 2020 Python for Data Analysis: Past, Present, and Future

PyCon Colombia 2020 Python for Data Analysis: Past, Present, and Future

Spark's Role in the Big Data Ecosystem (Spark Summit 2014)

Spark's Role in the Big Data Ecosystem (Spark Summit 2014)

Spark's Role in the Big Data Ecosystem (Spark Summit 2014)

Big Telco - Yousun Jeong

Big Telco - Yousun Jeong

Big Telco - Yousun Jeong

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

Not Your Father's Database: How to Use Apache Spark Properly in Your Big Data...

What's new in pandas and the SciPy stack for financial users

What's new in pandas and the SciPy stack for financial users

What's new in pandas and the SciPy stack for financial users

Apache Arrow at DataEngConf Barcelona 2018

Apache Arrow at DataEngConf Barcelona 2018

Apache Arrow at DataEngConf Barcelona 2018

Apache Arrow: Open Source Standard Becomes an Enterprise Necessity

Apache Arrow: Open Source Standard Becomes an Enterprise Necessity

Apache Arrow: Open Source Standard Becomes an Enterprise Necessity

Clinical Suspecting at Scale Using PySpark

Clinical Suspecting at Scale Using PySpark

Clinical Suspecting at Scale Using PySpark

Building a Virtual Data Lake with Apache Arrow

Building a Virtual Data Lake with Apache Arrow

Building a Virtual Data Lake with Apache Arrow

Improving data interoperability in Python and R

Improving data interoperability in Python and R

Improving data interoperability in Python and R

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Data Science with the Help of Metadata

Data Science with the Help of Metadata

Data Science with the Help of Metadata

Practical Medium Data Analytics with Python (10 Things I Hate About pandas, P...

Practical Medium Data Analytics with Python (10 Things I Hate About pandas, P...

Practical Medium Data Analytics with Python (10 Things I Hate About pandas, P...

Big Data Processing with Spark and .NET - Microsoft Ignite 2019

Big Data Processing with Spark and .NET - Microsoft Ignite 2019

Big Data Processing with Spark and .NET - Microsoft Ignite 2019

The Future of Real-Time in Spark

The Future of Real-Time in Spark

The Future of Real-Time in Spark

PyCon.DE / PyData Karlsruhe keynote: "Looking backward, looking forward"

PyCon.DE / PyData Karlsruhe keynote: "Looking backward, looking forward"

PyCon.DE / PyData Karlsruhe keynote: "Looking backward, looking forward"

Spark Meetup at Uber

Spark Meetup at Uber

Spark Meetup at Uber

Building an ETL pipeline for Elasticsearch using Spark

Building an ETL pipeline for Elasticsearch using Spark

Building an ETL pipeline for Elasticsearch using Spark

Ähnlich wie Spark Summit EU 2015: Revolutionizing Big Data in the Enterprise with Spark

The term "Data Lake" has become almost as overused and undescriptive as "Big Data". Many believe that centralizing datasets in HDFS makes a data lake, but then they struggle to realize any tangible value. This talk will redefine the "Data Lake" by describing four specific, key characteristics that we at Koverse have learned are crucial to successful enterprise data lake deployments. These characteristics are 1) indexing and search across all data sets, 2) interactive access for all users in the enterprise, 3) multi-level access control, and 4) integration with data science tools. These characteristics define a system that lets people realize value from their data versus getting lost in the hype. The talk will go on to provide a technical description of how we have integrated several projects, namely Apache Accumulo, Hadoop, and Spark, to implement an enterprise data lake with these key features.

One Large Data Lake, Hold the Hype

One Large Data Lake, Hold the Hype

One Large Data Lake, Hold the Hype

The term "Data Lake" has become almost as overused and undescriptive as "Big Data". Many believe that centralizing datasets in HDFS makes a data lake, but then they struggle to realize any tangible value. This talk will redefine the "Data Lake" by describing four specific, key characteristics that we at Koverse have learned are crucial to successful enterprise data lake deployments. These characteristics are 1) indexing and search across all data sets, 2) interactive access for all users in the enterprise, 3) multi-level access control, and 4) integration with data science tools. These characteristics define a system that lets people realize value from their data versus getting lost in the hype. The talk will go on to provide a technical description of how we have integrated several projects, namely Apache Accumulo, Hadoop, and Spark, to implement an enterprise data lake with these key features.

One Large Data Lake, Hold the Hype

One Large Data Lake, Hold the Hype

One Large Data Lake, Hold the Hype

Mark Lewis, Senior MArketing Director EMEA, Cloudera. Hadoop and the Future of Data Management. As Hadoop takes the data management market by storm, organisations are evolving the role it plays in the modern data centre. Explore how this disruptive technology is quickly transforming an industry and how you can leverage it today, in combination with MongoDB, to drive meaningful change in your business.

MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...

MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...

MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...

Nov 2014 talk to SW Data Meetup by Mike Olson, co-founder and chairman of Cloudera. In business, we often deal with hype around trends in society, politics, economy and technology. We know we need to take claims of the next big thing with a grain of salt and that we should be careful not to set expectations too high. However, with Big Data analytics, the opposite is true. The hype that accompanies it actually conceals the enormity of its impact on the way we do business. In this talk I’ll discuss how new 'Data Driven' economies are emerging through relentless innovation across the public and private sectors. Mike (co-founded Cloudera in 2008 and served as its CEO until 2013 when he took on his current role of chief strategy officer (CSO.) As CSO, Mike is responsible for Cloudera’s product strategy, open source leadership, engineering alignment and direct engagement with customers. Prior to Cloudera Mike was CEO of Sleepycat Software, makers of Berkeley DB, the open source embedded database engine. Mike spent two years at Oracle Corporation as vice president for Embedded Technologies after Oracle’s acquisition of Sleepycat in 2006. Prior to joining Sleepycat, Mike held technical and business positions at database vendors Britton Lee, Illustra Information Technologies and Informix Software. Mike has a Bachelor’s and a Master’s Degree in Computer Science from the University of California, Berkeley.

Ask bigger questions

Ask bigger questions

Ask bigger questions

South West Data Meetup

Data Lakehouse, Data Mesh, and Data Fabric (r1)

Data Lakehouse, Data Mesh, and Data Fabric (r1)

Data Lakehouse, Data Mesh, and Data Fabric (r1)

Join Cloudian, Hortonworks and 451 Research for a panel-style Q&A discussion about the latest trends and technology innovations in Big Data and Analytics. Matt Aslett, Data Platforms and Analytics Research Director at 451 Research, John Kreisa, Vice President of Strategic Marketing at Hortonworks, and Paul Turner, Chief Marketing Officer at Cloudian, will answer your toughest questions about data storage, data analytics, log data, sensor data and the Internet of Things. Bring your questions or just come and listen!

Cloudian 451-hortonworks - webinar

Cloudian 451-hortonworks - webinar

Cloudian 451-hortonworks - webinar

Building a Modern Data Architecture by Ben Sharma at Strata + Hadoop World Sa...

Building a Modern Data Architecture by Ben Sharma at Strata + Hadoop World Sa...

Building a Modern Data Architecture by Ben Sharma at Strata + Hadoop World Sa...

Hadoop as a Data Hub

Hadoop as a Data Hub

Hadoop as a Data Hub

Webinar -Data Warehouse Augmentation: Cut Costs, Increase Power

Webinar -Data Warehouse Augmentation: Cut Costs, Increase Power

Webinar -Data Warehouse Augmentation: Cut Costs, Increase Power

This topic describes the use of Spark and SequoiaDB in the Operational Data Lake of China’s financial industry, including how to use SequoiaDB to provide online high concurrent services and how to use Spark for data processing and machine learning. China has the world’s largest population, and also the world’s second largest economy. Many of the best technologies used in the United States and Europe are difficult to play effectively in China. This topic will show you how Spark and SequoiaDB are able to provide online financial services to billions of population.

Building Operational Data Lake using Spark and SequoiaDB with Yang Peng

Building Operational Data Lake using Spark and SequoiaDB with Yang Peng

Building Operational Data Lake using Spark and SequoiaDB with Yang Peng

The Briefing Room with Rick van der Lans and Think Big, a Teradata Company Live Webcast on June 16, 2015 Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=197f8106531874cc5c14081ca214eaff Hadoop is arguably one of the most disruptive technologies of the last decade. Once lauded solely for its ability to transform the speed of batch processing, it has marched steadily forward and promulgated an array of performance-enhancing accessories, notably Spark and YARN. Hadoop has evolved into much more than a file system and batch processor, and it now promises to stand as the data management and analytics backbone for enterprises. Register for this episode of The Briefing Room to learn from veteran Analyst Rick van der Lans, as he discusses the emerging roles of Hadoop within the analytics ecosystem. He’ll be briefed by Ron Bodkin of Think Big, a Teradata Company, who will explore Hadoop’s maturity spectrum, from typical entry use cases all the way up the value chain. He’ll show how enterprises that already use Hadoop in production are finding new ways to exploit its power and build creative, dynamic analytics environments. Visit InsideAnalysis.com for more information.

The Maturity Model: Taking the Growing Pains Out of Hadoop

The Maturity Model: Taking the Growing Pains Out of Hadoop

The Maturity Model: Taking the Growing Pains Out of Hadoop

Inside Analysis

The Briefing Room with Dr. Robin Bloor and Think Big, a Teradata Company Live Webcast April 7, 2015 Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=4114b87441ab7b2b4c52f6b24776e5a1 The more things change in Big Data, the more they stay the same. Indeed, there are many similarities between a Hadoop-based Data Lake and today’s modern Data Warehouse. Regardless of platform, information workers must still be able to turn their assets into action quickly, without taking a hit on governance or downstream performance. Register for this episode of The Briefing Room to hear veteran Analyst Dr. Robin Bloor as he explains the challenges facing organizations who endeavor on Big Data projects. He’ll be briefed by Rick Stellwagen of Think Big, a Teradata Company, who will outline his company’s approach to handling Big Data implementations. Rick will discuss the role of the data lake, and how timely response of queries is critical for reporting and analysis. Visit InsideAnalysis.com for more information.

The Great Lakes: How to Approach a Big Data Implementation

The Great Lakes: How to Approach a Big Data Implementation

The Great Lakes: How to Approach a Big Data Implementation

Inside Analysis

Learn about the promise of data lakes: - Store all types of data in its raw format - Create refined, standardized, trusted datasets for various use cases - Store data for longer periods of time to enable historical analysis - Query and Access the data using a variety of methods - Manage streaming and batch data in a converged platform - Provide shorter time-to-insight with proper data management and governance

Strata San Jose 2017 - Ben Sharma Presentation

Strata San Jose 2017 - Ben Sharma Presentation

Strata San Jose 2017 - Ben Sharma Presentation

Splice machine-bloor-webinar-data-lakes

Splice machine-bloor-webinar-data-lakes

Splice machine-bloor-webinar-data-lakes

Edgar Alejandro Villegas

Information and data relevance to business

Information and data relevance to business

Information and data relevance to business

Warehousing Your Hits - The Why and How of Owning Your Data

Warehousing Your Hits - The Why and How of Owning Your Data

Warehousing Your Hits - The Why and How of Owning Your Data

Scott Arbeitman

Presentation at Data Summit 2015 in NYC. Elliott Cordo shared real-world insights across a range of topics, including the evolving best practices for building a data warehouse on Hadoop that also coexists with multiple processing frameworks and additional non-Hadoop storage platforms, the place for massively parallel-processing and relational databases in analytic architectures, and the ways in which the cloud offers the ability to quickly and cost-effectively establish a scalable platform for your Big Data warehouse. For more information, visit www.casertaconcepts.com

Hadoop and Your Data Warehouse

Hadoop and Your Data Warehouse

Hadoop and Your Data Warehouse

Dw Concepts

Big data 101

Paresh Motiwala, PMP®

Modern data warehouse presentation

Modern data warehouse presentation

Modern data warehouse presentation

Ähnlich wie Spark Summit EU 2015: Revolutionizing Big Data in the Enterprise with Spark (20)

One Large Data Lake, Hold the Hype

One Large Data Lake, Hold the Hype

One Large Data Lake, Hold the Hype

One Large Data Lake, Hold the Hype

One Large Data Lake, Hold the Hype

One Large Data Lake, Hold the Hype

MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...

MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...

MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...

Ask bigger questions

Ask bigger questions

Ask bigger questions

Data Lakehouse, Data Mesh, and Data Fabric (r1)

Data Lakehouse, Data Mesh, and Data Fabric (r1)

Data Lakehouse, Data Mesh, and Data Fabric (r1)

Cloudian 451-hortonworks - webinar

Cloudian 451-hortonworks - webinar

Cloudian 451-hortonworks - webinar

Building a Modern Data Architecture by Ben Sharma at Strata + Hadoop World Sa...

Building a Modern Data Architecture by Ben Sharma at Strata + Hadoop World Sa...

Building a Modern Data Architecture by Ben Sharma at Strata + Hadoop World Sa...

Hadoop as a Data Hub

Hadoop as a Data Hub

Hadoop as a Data Hub

Webinar -Data Warehouse Augmentation: Cut Costs, Increase Power

Webinar -Data Warehouse Augmentation: Cut Costs, Increase Power

Webinar -Data Warehouse Augmentation: Cut Costs, Increase Power

Building Operational Data Lake using Spark and SequoiaDB with Yang Peng

Building Operational Data Lake using Spark and SequoiaDB with Yang Peng

Building Operational Data Lake using Spark and SequoiaDB with Yang Peng

The Maturity Model: Taking the Growing Pains Out of Hadoop

The Maturity Model: Taking the Growing Pains Out of Hadoop

The Maturity Model: Taking the Growing Pains Out of Hadoop

The Great Lakes: How to Approach a Big Data Implementation

The Great Lakes: How to Approach a Big Data Implementation

The Great Lakes: How to Approach a Big Data Implementation

Strata San Jose 2017 - Ben Sharma Presentation

Strata San Jose 2017 - Ben Sharma Presentation

Strata San Jose 2017 - Ben Sharma Presentation

Splice machine-bloor-webinar-data-lakes

Splice machine-bloor-webinar-data-lakes

Splice machine-bloor-webinar-data-lakes

Information and data relevance to business

Information and data relevance to business

Information and data relevance to business

Warehousing Your Hits - The Why and How of Owning Your Data

Warehousing Your Hits - The Why and How of Owning Your Data

Warehousing Your Hits - The Why and How of Owning Your Data

Hadoop and Your Data Warehouse

Hadoop and Your Data Warehouse

Hadoop and Your Data Warehouse

Dw Concepts

Big data 101

Modern data warehouse presentation

Modern data warehouse presentation

Modern data warehouse presentation

Mehr von Databricks

DW Migration Webinar-March 2022.pptx

DW Migration Webinar-March 2022.pptx

DW Migration Webinar-March 2022.pptx

The world of data architecture began with applications. Next came data warehouses. Then text was organized into a data warehouse. Then one day the world discovered a whole new kind of data that was being generated by organizations. The world found that machines generated data that could be transformed into valuable insights. This was the origin of what is today called the data lakehouse. The evolution of data architecture continues today. Come listen to industry experts describe this transformation of ordinary data into a data architecture that is invaluable to business. Simply put, organizations that take data architecture seriously are going to be at the forefront of business tomorrow. This is an educational event. Several of the authors of the book Building the Data Lakehouse will be presenting at this symposium.

Data Lakehouse Symposium | Day 1 | Part 1

Data Lakehouse Symposium | Day 1 | Part 1

Data Lakehouse Symposium | Day 1 | Part 1

The world of data architecture began with applications. Next came data warehouses. Then text was organized into a data warehouse. Then one day the world discovered a whole new kind of data that was being generated by organizations. The world found that machines generated data that could be transformed into valuable insights. This was the origin of what is today called the data lakehouse. The evolution of data architecture continues today. Come listen to industry experts describe this transformation of ordinary data into a data architecture that is invaluable to business. Simply put, organizations that take data architecture seriously are going to be at the forefront of business tomorrow. This is an educational event. Several of the authors of the book Building the Data Lakehouse will be presenting at this symposium.

Data Lakehouse Symposium | Day 1 | Part 2

Data Lakehouse Symposium | Day 1 | Part 2

Data Lakehouse Symposium | Day 1 | Part 2

The world of data architecture began with applications. Next came data warehouses. Then text was organized into a data warehouse. Then one day the world discovered a whole new kind of data that was being generated by organizations. The world found that machines generated data that could be transformed into valuable insights. This was the origin of what is today called the data lakehouse. The evolution of data architecture continues today. Come listen to industry experts describe this transformation of ordinary data into a data architecture that is invaluable to business. Simply put, organizations that take data architecture seriously are going to be at the forefront of business tomorrow. This is an educational event. Several of the authors of the book Building the Data Lakehouse will be presenting at this symposium.

Data Lakehouse Symposium | Day 2

Data Lakehouse Symposium | Day 2

Data Lakehouse Symposium | Day 2

The world of data architecture began with applications. Next came data warehouses. Then text was organized into a data warehouse. Then one day the world discovered a whole new kind of data that was being generated by organizations. The world found that machines generated data that could be transformed into valuable insights. This was the origin of what is today called the data lakehouse. The evolution of data architecture continues today. Come listen to industry experts describe this transformation of ordinary data into a data architecture that is invaluable to business. Simply put, organizations that take data architecture seriously are going to be at the forefront of business tomorrow. This is an educational event. Several of the authors of the book Building the Data Lakehouse will be presenting at this symposium.

Data Lakehouse Symposium | Day 4

Data Lakehouse Symposium | Day 4

Data Lakehouse Symposium | Day 4

In this session, learn how to quickly supplement your on-premises Hadoop environment with a simple, open, and collaborative cloud architecture that enables you to generate greater value with scaled application of analytics and AI on all your data. You will also learn five critical steps for a successful migration to the Databricks Lakehouse Platform along with the resources available to help you begin to re-skill your data teams.

5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop

5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop

5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop

Bad data leads to bad decisions and broken customer experiences. Organizations depend on complete and accurate data to power their business, maintain efficiency, and uphold customer trust. With thousands of datasets and pipelines running, how do we ensure that all data meets quality standards, and that expectations are clear between producers and consumers? Investing in shared, flexible components and practices for monitoring data health is crucial for a complex data organization to rapidly and effectively scale. At Zillow, we built a centralized platform to meet our data quality needs across stakeholders. The platform is accessible to engineers, scientists, and analysts, and seamlessly integrates with existing data pipelines and data discovery tools. In this presentation, we will provide an overview of our platform’s capabilities, including: Giving producers and consumers the ability to define and view data quality expectations using a self-service onboarding portal Performing data quality validations using libraries built to work with spark Dynamically generating pipelines that can be abstracted away from users Flagging data that doesn’t meet quality standards at the earliest stage and giving producers the opportunity to resolve issues before use by downstream consumers Exposing data quality metrics alongside each dataset to provide producers and consumers with a comprehensive picture of health over time

Democratizing Data Quality Through a Centralized Platform

Democratizing Data Quality Through a Centralized Platform

Democratizing Data Quality Through a Centralized Platform

Data scientists face numerous challenges throughout the data science workflow that hinder productivity. As organizations continue to become more data-driven, a collaborative environment is more critical than ever — one that provides easier access and visibility into the data, reports and dashboards built against the data, reproducibility, and insights uncovered within the data.. Join us to hear how Databricks’ open and collaborative platform simplifies data science by enabling you to run all types of analytics workloads, from data preparation to exploratory analysis and predictive analytics, at scale — all on one unified platform.

Learn to Use Databricks for Data Science

Learn to Use Databricks for Data Science

Learn to Use Databricks for Data Science

Application performance monitoring (APM) has become the cornerstone of software engineering allowing engineering teams to quickly identify and remedy production issues. However, as the world moves to intelligent software applications that are built using machine learning, traditional APM quickly becomes insufficient to identify and remedy production issues encountered in these modern software applications. As a lead software engineer at NewRelic, my team built high-performance monitoring systems including Insights, Mobile, and SixthSense. As I transitioned to building ML Monitoring software, I found the architectural principles and design choices underlying APM to not be a good fit for this brand new world. In fact, blindly following APM designs led us down paths that would have been better left unexplored. In this talk, I draw upon my (and my team’s) experience building an ML Monitoring system from the ground up and deploying it on customer workloads running large-scale ML training with Spark as well as real-time inference systems. I will highlight how the key principles and architectural choices of APM don’t apply to ML monitoring. You’ll learn why, understand what ML Monitoring can successfully borrow from APM, and hear what is required to build a scalable, robust ML Monitoring architecture.

Why APM Is Not the Same As ML Monitoring

Why APM Is Not the Same As ML Monitoring

Why APM Is Not the Same As ML Monitoring

Autonomy and ownership are core to working at Stitch Fix, particularly on the Algorithms team. We enable data scientists to deploy and operate their models independently, with minimal need for handoffs or gatekeeping. By writing a simple function and calling out to an intuitive API, data scientists can harness a suite of platform-provided tooling meant to make ML operations easy. In this talk, we will dive into the abstractions the Data Platform team has built to enable this. We will go over the interface data scientists use to specify a model and what that hooks into, including online deployment, batch execution on Spark, and metrics tracking and visualization.

The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix

The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix

The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix

In this talk, I will dive into the stage level scheduling feature added to Apache Spark 3.1. Stage level scheduling extends upon Project Hydrogen by improving big data ETL and AI integration and also enables multiple other use cases. It is beneficial any time the user wants to change container resources between stages in a single Apache Spark application, whether those resources are CPU, Memory or GPUs. One of the most popular use cases is enabling end-to-end scalable Deep Learning and AI to efficiently use GPU resources. In this type of use case, users read from a distributed file system, do data manipulation and filtering to get the data into a format that the Deep Learning algorithm needs for training or inference and then sends the data into a Deep Learning algorithm. Using stage level scheduling combined with accelerator aware scheduling enables users to seamlessly go from ETL to Deep Learning running on the GPU by adjusting the container requirements for different stages in Spark within the same application. This makes writing these applications easier and can help with hardware utilization and costs. There are other ETL use cases where users want to change CPU and memory resources between stages, for instance there is data skew or perhaps the data size is much larger in certain stages of the application. In this talk, I will go over the feature details, cluster requirements, the API and use cases. I will demo how the stage level scheduling API can be used by Horovod to seamlessly go from data preparation to training using the Tensorflow Keras API using GPUs. The talk will also touch on other new Apache Spark 3.1 functionality, such as pluggable caching, which can be used to enable faster dataframe access when operating from GPUs.

Stage Level Scheduling Improving Big Data and AI Integration

Stage Level Scheduling Improving Big Data and AI Integration

Stage Level Scheduling Improving Big Data and AI Integration

In this talk, I would like to introduce an open-source tool built by our team that simplifies the data conversion from Apache Spark to deep learning frameworks. Imagine you have a large dataset, say 20 GBs, and you want to use it to train a TensorFlow model. Before feeding the data to the model, you need to clean and preprocess your data using Spark. Now you have your dataset in a Spark DataFrame. When it comes to the training part, you may have the problem: How can I convert my Spark DataFrame to some format recognized by my TensorFlow model? The existing data conversion process can be tedious. For example, to convert an Apache Spark DataFrame to a TensorFlow Dataset file format, you need to either save the Apache Spark DataFrame on a distributed filesystem in parquet format and load the converted data with third-party tools such as Petastorm, or save it directly in TFRecord files with spark-tensorflow-connector and load it back using TFRecordDataset. Both approaches take more than 20 lines of code to manage the intermediate data files, rely on different parsing syntax, and require extra attention for handling vector columns in the Spark DataFrames. In short, all these engineering frictions greatly reduced the data scientists’ productivity. The Databricks Machine Learning team contributed a new Spark Dataset Converter API to Petastorm to simplify these tedious data conversion process steps. With the new API, it takes a few lines of code to convert a Spark DataFrame to a TensorFlow Dataset or a PyTorch DataLoader with default parameters. In the talk, I will use an example to show how to use the Spark Dataset Converter to train a Tensorflow model and how simple it is to go from single-node training to distributed training on Databricks.

Simplify Data Conversion from Spark to TensorFlow and PyTorch

Simplify Data Conversion from Spark to TensorFlow and PyTorch

Simplify Data Conversion from Spark to TensorFlow and PyTorch

There is no doubt Kubernetes has emerged as the next generation of cloud native infrastructure to support a wide variety of distributed workloads. Apache Spark has evolved to run both Machine Learning and large scale analytics workloads. There is growing interest in running Apache Spark natively on Kubernetes. By combining the flexibility of Kubernetes and scalable data processing with Apache Spark, you can run any data and machine pipelines on this infrastructure while effectively utilizing resources at disposal. In this talk, Rajesh Thallam and Sougata Biswas will share how to effectively run your Apache Spark applications on Google Kubernetes Engine (GKE) and Google Cloud Dataproc, orchestrate the data and machine learning pipelines with managed Apache Airflow on GKE (Google Cloud Composer). Following topics will be covered: – Understanding key traits of Apache Spark on Kubernetes- Things to know when running Apache Spark on Kubernetes such as autoscaling- Demonstrate running analytics pipelines on Apache Spark orchestrated with Apache Airflow on Kubernetes cluster.

Scaling your Data Pipelines with Apache Spark on Kubernetes

Scaling your Data Pipelines with Apache Spark on Kubernetes

Scaling your Data Pipelines with Apache Spark on Kubernetes

Pipelines have become ubiquitous, as the need for stringing multiple functions to compose applications has gained adoption and popularity. Common pipeline abstractions such as “fit” and “transform” are even shared across divergent platforms such as Python Scikit-Learn and Apache Spark. Scaling pipelines at the level of simple functions is desirable for many AI applications, however is not directly supported by Ray’s parallelism primitives. In this talk, Raghu will describe a pipeline abstraction that takes advantage of Ray’s compute model to efficiently scale arbitrarily complex pipeline workflows. He will demonstrate how this abstraction cleanly unifies pipeline workflows across multiple platforms such as Scikit-Learn and Spark, and achieves nearly optimal scale-out parallelism on pipelined computations. Attendees will learn how pipelined workflows can be mapped to Ray’s compute model and how they can both unify and accelerate their pipelines with Ray.

Scaling and Unifying SciKit Learn and Apache Spark Pipelines

Scaling and Unifying SciKit Learn and Apache Spark Pipelines

Scaling and Unifying SciKit Learn and Apache Spark Pipelines

In this talk about zipline, we will introduce a new type of windowing construct called a sawtooth window. We will describe various properties about sawtooth windows that we utilize to achieve online-offline consistency, while still maintaining high-throughput, low-read latency and tunable write latency for serving machine learning features.We will also talk about a simple deployment strategy for correcting feature drift – due operations that are not “abelian groups”, that operate over change data.

Sawtooth Windows for Feature Aggregations

Sawtooth Windows for Feature Aggregations

Sawtooth Windows for Feature Aggregations

We want to present multiple anti patterns utilizing Redis in unconventional ways to get the maximum out of Apache Spark.All examples presented are tried and tested in production at Scale at Adobe. The most common integration is spark-redis which interfaces with Redis as a Dataframe backing Store or as an upstream for Structured Streaming. We deviate from the common use cases to explore where Redis can plug gaps while scaling out high throughput applications in Spark. Niche 1 : Long Running Spark Batch Job – Dispatch New Jobs by polling a Redis Queue · Why? o Custom queries on top a table; We load the data once and query N times · Why not Structured Streaming · Working Solution using Redis Niche 2 : Distributed Counters · Problems with Spark Accumulators · Utilize Redis Hashes as distributed counters · Precautions for retries and speculative execution · Pipelining to improve performance

Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink

Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink

Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink

In the era of microservices, decentralized ML architectures and complex data pipelines, data quality has become a bigger challenge than ever. When data is involved in complex business processes and decisions, bad data can, and will, affect the bottom line. As a result, ensuring data quality across the entire ML pipeline is both costly, and cumbersome while data monitoring is often fragmented and performed ad hoc. To address these challenges, we built whylogs, an open source standard for data logging. It is a lightweight data profiling library that enables end-to-end data profiling across the entire software stack. The library implements a language and platform agnostic approach to data quality and data monitoring. It can work with different modes of data operations, including streaming, batch and IoT data. In this talk, we will provide an overview of the whylogs architecture, including its lightweight statistical data collection approach and various integrations. We will demonstrate how the whylogs integration with Apache Spark achieves large scale data profiling, and we will show how users can apply this integration into existing data and ML pipelines.

Re-imagine Data Monitoring with whylogs and Spark

Re-imagine Data Monitoring with whylogs and Spark

Re-imagine Data Monitoring with whylogs and Spark

Machine learning (ML) models are typically part of prediction queries that consist of a data processing part (e.g., for joining, filtering, cleaning, featurization) and an ML part invoking one or more trained models. In this presentation, we identify significant and unexplored opportunities for optimization. To the best of our knowledge, this is the first effort to look at prediction queries holistically, optimizing across both the ML and SQL components. We will present Raven, an end-to-end optimizer for prediction queries. Raven relies on a unified intermediate representation that captures both data processing and ML operators in a single graph structure. This allows us to introduce optimization rules that (i) reduce unnecessary computations by passing information between the data processing and ML operators (ii) leverage operator transformations (e.g., turning a decision tree to a SQL expression or an equivalent neural network) to map operators to the right execution engine, and (iii) integrate compiler techniques to take advantage of the most efficient hardware backend (e.g., CPU, GPU) for each operator. We have implemented Raven as an extension to Spark’s Catalyst optimizer to enable the optimization of SparkSQL prediction queries. Our implementation also allows the optimization of prediction queries in SQL Server. As we will show, Raven is capable of improving prediction query performance on Apache Spark and SQL Server by up to 13.1x and 330x, respectively. For complex models, where GPU acceleration is beneficial, Raven provides up to 8x speedup compared to state-of-the-art systems. As part of the presentation, we will also give a demo showcasing Raven in action.

Raven: End-to-end Optimization of ML Prediction Queries

Raven: End-to-end Optimization of ML Prediction Queries

Raven: End-to-end Optimization of ML Prediction Queries

Semantic segmentation is the classification of every pixel in an image/video. The segmentation partitions a digital image into multiple objects to simplify/change the representation of the image into something that is more meaningful and easier to analyze [1][2]. The technique has a wide variety of applications ranging from perception in autonomous driving scenarios to cancer cell segmentation for medical diagnosis. Exponential growth in the datasets that require such segmentation is driven by improvements in the accuracy and quality of the sensors generating the data extending to 3D point cloud data. This growth is further compounded by exponential advances in cloud technologies enabling the storage and compute available for such applications. The need for semantically segmented datasets is a key requirement to improve the accuracy of inference engines that are built upon them. Streamlining the accuracy and efficiency of these systems directly affects the value of the business outcome for organizations that are developing such functionalities as a part of their AI strategy. This presentation details workflows for labeling, preprocessing, modeling, and evaluating performance/accuracy. Scientists and engineers leverage domain-specific features/tools that support the entire workflow from labeling the ground truth, handling data from a wide variety of sources/formats, developing models and finally deploying these models. Users can scale their deployments optimally on GPU-based cloud infrastructure to build accelerated training and inference pipelines while working with big datasets. These environments are optimized for engineers to develop such functionality with ease and then scale against large datasets with Spark-based clusters on the cloud.

Processing Large Datasets for ADAS Applications using Apache Spark

Processing Large Datasets for ADAS Applications using Apache Spark

Processing Large Datasets for ADAS Applications using Apache Spark

At Adobe Experience Platform, we ingest TBs of data every day and manage PBs of data for our customers as part of the Unified Profile Offering. At the heart of this is a bunch of complex ingestion of a mix of normalized and denormalized data with various linkage scenarios power by a central Identity Linking Graph. This helps power various marketing scenarios that are activated in multiple platforms and channels like email, advertisements etc. We will go over how we built a cost effective and scalable data pipeline using Apache Spark and Delta Lake and share our experiences. What are we storing? Multi Source – Multi Channel Problem Data Representation and Nested Schema Evolution Performance Trade Offs with Various formats Go over anti-patterns used (String FTW) Data Manipulation using UDFs Writer Worries and How to Wipe them Away Staging Tables FTW Datalake Replication Lag Tracking Performance Time!

Massive Data Processing in Adobe Using Delta Lake

Massive Data Processing in Adobe Using Delta Lake

Massive Data Processing in Adobe Using Delta Lake

Mehr von Databricks (20)

DW Migration Webinar-March 2022.pptx

DW Migration Webinar-March 2022.pptx

DW Migration Webinar-March 2022.pptx

Data Lakehouse Symposium | Day 1 | Part 1

Data Lakehouse Symposium | Day 1 | Part 1

Data Lakehouse Symposium | Day 1 | Part 1

Data Lakehouse Symposium | Day 1 | Part 2

Data Lakehouse Symposium | Day 1 | Part 2

Data Lakehouse Symposium | Day 1 | Part 2

Data Lakehouse Symposium | Day 2

Data Lakehouse Symposium | Day 2

Data Lakehouse Symposium | Day 2

Data Lakehouse Symposium | Day 4

Data Lakehouse Symposium | Day 4

Data Lakehouse Symposium | Day 4

5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop

5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop

5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop

Democratizing Data Quality Through a Centralized Platform

Democratizing Data Quality Through a Centralized Platform

Democratizing Data Quality Through a Centralized Platform

Learn to Use Databricks for Data Science

Learn to Use Databricks for Data Science

Learn to Use Databricks for Data Science

Why APM Is Not the Same As ML Monitoring

Why APM Is Not the Same As ML Monitoring

Why APM Is Not the Same As ML Monitoring

The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix

The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix

The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix

Stage Level Scheduling Improving Big Data and AI Integration

Stage Level Scheduling Improving Big Data and AI Integration

Stage Level Scheduling Improving Big Data and AI Integration

Simplify Data Conversion from Spark to TensorFlow and PyTorch

Simplify Data Conversion from Spark to TensorFlow and PyTorch

Simplify Data Conversion from Spark to TensorFlow and PyTorch

Scaling your Data Pipelines with Apache Spark on Kubernetes

Scaling your Data Pipelines with Apache Spark on Kubernetes

Scaling your Data Pipelines with Apache Spark on Kubernetes

Scaling and Unifying SciKit Learn and Apache Spark Pipelines

Scaling and Unifying SciKit Learn and Apache Spark Pipelines

Scaling and Unifying SciKit Learn and Apache Spark Pipelines

Sawtooth Windows for Feature Aggregations

Sawtooth Windows for Feature Aggregations

Sawtooth Windows for Feature Aggregations

Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink

Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink

Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink

Re-imagine Data Monitoring with whylogs and Spark

Re-imagine Data Monitoring with whylogs and Spark

Re-imagine Data Monitoring with whylogs and Spark

Raven: End-to-end Optimization of ML Prediction Queries

Raven: End-to-end Optimization of ML Prediction Queries

Raven: End-to-end Optimization of ML Prediction Queries

Processing Large Datasets for ADAS Applications using Apache Spark

Processing Large Datasets for ADAS Applications using Apache Spark

Processing Large Datasets for ADAS Applications using Apache Spark

Massive Data Processing in Adobe Using Delta Lake

Massive Data Processing in Adobe Using Delta Lake

Massive Data Processing in Adobe Using Delta Lake

Kürzlich hochgeladen

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein

%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain

%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain

%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain

8257 interfacing 2 in microprocessor for btech students

8257 interfacing 2 in microprocessor for btech students

8257 interfacing 2 in microprocessor for btech students

Define the academic and professional writing..pdf

Define the academic and professional writing..pdf

Define the academic and professional writing..pdf

PearlKirahMaeRagusta1

At the recent Microsoft Ignite 2023 conference, Microsoft unveiled a groundbreaking strategy that will redefine enterprise work management. The plan involves integrating Microsoft’s key planning tools, Microsoft To Do, Microsoft Planner, and Microsoft Project for the web into a unified experience called “Microsoft Planner.” What does this new strategy from Microsoft mean for current users? Join us and learn how best to take advantage of this announcement while gaining a clear path on how to elevate the current state of Microsoft Planner from a basic task manager to a comprehensive tool for Enterprise Work Management using OnePlan. Learn how OnePlan’s integration with Microsoft Planner allows for strategic alignment with business goals through advanced features like strategic planning, portfolio management, resource management, financial management, and more!

Introducing Microsoft’s new Enterprise Work Management (EWM) Solution

Introducing Microsoft’s new Enterprise Work Management (EWM) Solution

Introducing Microsoft’s new Enterprise Work Management (EWM) Solution

OnePlan Solutions

HR Software Buyers Guide in 2024 - HRSoftware.com

HR Software Buyers Guide in 2024 - HRSoftware.com

HR Software Buyers Guide in 2024 - HRSoftware.com

Fatema Valibhai

Technology has taken up space all over the world. From generating content with a single command on ChatGPT to getting your food served by Robots at your favorite restaurant, artificial advancements have ruled every space. Every industry is set to develop top-notch technology in every sector; finance, IT, healthcare, gaming, and banking, with competitive market standards. One of these rapidly growing industries is Mobile App Development. According to the Straits Research report, it is expected to reach USD 583.03 billion at a CAGR OF 12.8% between (2022 and 2030). It clearly shows how mobile app development has become an integral part of the digital landscape and revolutionized technology.

The Top App Development Trends Shaping the Industry in 2024-25 .pdf

The Top App Development Trends Shaping the Industry in 2024-25 .pdf

The Top App Development Trends Shaping the Industry in 2024-25 .pdf

Pharm-D Biostatistics and Research methodology

Pharm-D Biostatistics and Research methodology

Pharm-D Biostatistics and Research methodology

(Vivek)Call Us, 8448380779,Call girls in Delhi NCr – We Offer best in class call girls. escort Service At Affordable Price At low Rate with Space Night 8000 We Are One Of The Oldest Escort and Call girls Agencies in Delhi. You Will Find That Our Female Escorts Are Full Of Fun, Sexy And They Would Love Enjoy Your Company. We Have A Fantastic Selection Of Escort Ladies Available For In-Calls As Well As Out-Calls. Our Escorts Are Not Only Beautiful But All Have Great Personalities Making Them The Perfect Companion For Any Occasion. In-Call:- You Can Come At Our Place in Delhi Our place Which Is Very Clean Hygienic 100% safe Accommodation. Out-Call:- You have To Come Pick The Girl From My Place We Are Also Provide Door Step Services (Delhi Ncr, Noida, Gurgaon, Faridabad, Ghaziabad Note:- Pic Collectors Time Passers Bargainers Stay Away As We Respect The Value For Your Money Time And Expect The Same From You Hygienic:- Full Ac room And Clean Rooms Available In Hotel 24 * 7 Hourly In Delhi NCR More Details, With WhatsApp Number, +91-8448380779

Sector 18, Noida Call girls :8448380779 Model Escorts | 100% verified

Sector 18, Noida Call girls :8448380779 Model Escorts | 100% verified

Sector 18, Noida Call girls :8448380779 Model Escorts | 100% verified

Delhi Call girls

introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf

introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf

introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf

VishalKumarJha10

A great deal of attention in medical devices has shifted towards cybersecurity with the ratification of section 524B of the FD&C act. This new law enables the FDA to enforce cybersecurity controls in any medical device that is capable of networked communications or that has software. In this webinar we will recap the process for managing vulnerabilities, identify categories of vulnerabilities and solutions and more.

The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...

The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...

The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...

At TECUNIQUE, we're a stable and steadily growing Indian software services company with over 14 years of industry experience. Specializing in offshore software development and quality assurance services, we've built a reputation for delivering unique and effective solutions to start-ups, software development companies, enterprises, and digital agencies. We pride ourselves on our commitment to excellence and innovation. By blending insightful business domain knowledge with exceptional technical prowess, we craft tailor-made solutions that meet the unique needs of our clients. Our dedicated teams are adept in specific technologies, ensuring seamless integration of skills and delivering reliable, scalable, and high-quality software solutions aligned with our clients' preferences. Bespoke Dedicated Teams: Crafted to meet your specific needs and technology preferences, our dedicated teams are committed to delivering top-notch software solutions. Offshore Software Development: Accelerate your software development and scale up quickly with our 12+ years of expertise in offshore development. Quality Assurance Services: Ensure the quality of your software products with our dedicated teams of experienced QA professionals. IT Staff Augmentation: Overcome skill gaps with our client-centric software team, offering staff augmentation services. Expert Software Services: Unlock our capabilities in custom software development, product development, and quality assurance. Mission and Vision: Our mission at TECUNIQUE is to be the catalyst for our clients' success in the dynamic domain of software development. Rooted in our core values of respect, authenticity, and responsibility, we strive to ease the software outsourcing experience, reducing both time and cost to market for our clients. We envision ourselves as the leading Indian software services company, renowned for our unwavering commitment to excellence and innovation. www.tecunique.com

TECUNIQUE: Success Stories: IT Service provider

TECUNIQUE: Success Stories: IT Service provider

TECUNIQUE: Success Stories: IT Service provider

Direct Style Effect Systems -The Print[A] Example- A Comprehension Aid

Direct Style Effect Systems -The Print[A] Example- A Comprehension Aid

Direct Style Effect Systems -The Print[A] Example- A Comprehension Aid

LEVEL 5 - SESSION 1 2023 (1).pptx - PDF 123456

LEVEL 5 - SESSION 1 2023 (1).pptx - PDF 123456

LEVEL 5 - SESSION 1 2023 (1).pptx - PDF 123456

KiaraTiradoMicha

%in kempton park+277-882-255-28 abortion pills for sale in kempton park

%in kempton park+277-882-255-28 abortion pills for sale in kempton park

%in kempton park+277-882-255-28 abortion pills for sale in kempton park

%in Midrand+277-882-255-28 abortion pills for sale in midrand

%in Midrand+277-882-255-28 abortion pills for sale in midrand

%in Midrand+277-882-255-28 abortion pills for sale in midrand

Data spaces in distributed environments should be allowed to evolve in agile ways providing data space owners with large flexibility about which data they store. Agility and heterogeneity, however, jeopardize data exchanges because representations may build on varying ontologies and data consumers may not rely on the semantic correctness of their queries in the context of semantically heterogeneous, evolving data spaces. Graph data spaces are one example of a powerful model for representing and querying data whose semantics may change over time. To assert and enforce conditions on individual graph data spaces, shape languages (e.g SHACL) have been developed. We investigate the question of how querying and programming can be guarded by reasoning over SHACL constraints in a distributed setting and we sketch a picture of how a future landscape based on semantically heterogeneous data spaces might look like.

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Chinsurah Escorts ☎️8617697112 Starting From 5K to 15K High Profile Escorts In Chinsurah ❤Personal Whatsapp Number Chinsurah Call Girls 8617697112 💦✅. Chinsurah escorts we are avaliable for our all types budget customers with offer great deals in Chinsurah.Call Now our Chinsurah escort service & call girls ... Independent call girls in Chinsurah escorts available 24 hours a day for discreet incall and outcall bookings from trusted call girls - Elis.in. Nitya salvi Chinsurah escorts service agency # Are you looking for sexy call girls ? Call our agency to get you dream independent call girl, ... One shot: ₹2000/in-call, ₹5000/out-call Two shots with one girl: ₹3500/in-call, ₹6000/out-call Body to body massage with sex: ₹3000/in-call Full night for one person: ₹7000/in-call, ₹10000/out-call Flexibility Choices and options Lists of many beauty fantasies Turn your dream into reality Perfect companionship Cheap and convenient In-call and Out-call services And many more. WhatsApp Chat: 📞 8617697112 Visit The Website : https://www.nityasalvi.com/

Chinsurah Escorts ☎️8617697112 Starting From 5K to 15K High Profile Escorts ...

(Vivek)Call Us, 8448380779,Call girls in Delhi NCr – We Offer best in class call girls. escort Service At Affordable Price At low Rate with Space Night 8000 We Are One Of The Oldest Escort and Call girls Agencies in Delhi. You Will Find That Our Female Escorts Are Full Of Fun, Sexy And They Would Love Enjoy Your Company. We Have A Fantastic Selection Of Escort Ladies Available For In-Calls As Well As Out-Calls. Our Escorts Are Not Only Beautiful But All Have Great Personalities Making Them The Perfect Companion For Any Occasion. In-Call:- You Can Come At Our Place in Delhi Our place Which Is Very Clean Hygienic 100% safe Accommodation. Out-Call:- You have To Come Pick The Girl From My Place We Are Also Provide Door Step Services (Delhi Ncr, Noida, Gurgaon, Faridabad, Ghaziabad Note:- Pic Collectors Time Passers Bargainers Stay Away As We Respect The Value For Your Money Time And Expect The Same From You Hygienic:- Full Ac room And Clean Rooms Available In Hotel 24 * 7 Hourly In Delhi NCR More Details, With WhatsApp Number, +91-8448380779

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

Delhi Call girls

Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...

Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...

Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...

Kürzlich hochgeladen (20)

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein

%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain

%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain

%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain

8257 interfacing 2 in microprocessor for btech students

8257 interfacing 2 in microprocessor for btech students

8257 interfacing 2 in microprocessor for btech students

Define the academic and professional writing..pdf

Define the academic and professional writing..pdf

Define the academic and professional writing..pdf

Introducing Microsoft’s new Enterprise Work Management (EWM) Solution

Introducing Microsoft’s new Enterprise Work Management (EWM) Solution

Introducing Microsoft’s new Enterprise Work Management (EWM) Solution

HR Software Buyers Guide in 2024 - HRSoftware.com

HR Software Buyers Guide in 2024 - HRSoftware.com

HR Software Buyers Guide in 2024 - HRSoftware.com

The Top App Development Trends Shaping the Industry in 2024-25 .pdf

The Top App Development Trends Shaping the Industry in 2024-25 .pdf

The Top App Development Trends Shaping the Industry in 2024-25 .pdf

Pharm-D Biostatistics and Research methodology

Pharm-D Biostatistics and Research methodology

Pharm-D Biostatistics and Research methodology

Sector 18, Noida Call girls :8448380779 Model Escorts | 100% verified

Sector 18, Noida Call girls :8448380779 Model Escorts | 100% verified

Sector 18, Noida Call girls :8448380779 Model Escorts | 100% verified

introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf

introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf

introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf

The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...

The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...

The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...

TECUNIQUE: Success Stories: IT Service provider

TECUNIQUE: Success Stories: IT Service provider

TECUNIQUE: Success Stories: IT Service provider

Direct Style Effect Systems -The Print[A] Example- A Comprehension Aid

Direct Style Effect Systems -The Print[A] Example- A Comprehension Aid

Direct Style Effect Systems -The Print[A] Example- A Comprehension Aid

LEVEL 5 - SESSION 1 2023 (1).pptx - PDF 123456

LEVEL 5 - SESSION 1 2023 (1).pptx - PDF 123456

LEVEL 5 - SESSION 1 2023 (1).pptx - PDF 123456

%in kempton park+277-882-255-28 abortion pills for sale in kempton park

%in kempton park+277-882-255-28 abortion pills for sale in kempton park

%in kempton park+277-882-255-28 abortion pills for sale in kempton park

%in Midrand+277-882-255-28 abortion pills for sale in midrand

%in Midrand+277-882-255-28 abortion pills for sale in midrand

%in Midrand+277-882-255-28 abortion pills for sale in midrand

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Chinsurah Escorts ☎️8617697112 Starting From 5K to 15K High Profile Escorts ...

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...

Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...

Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...

Spark Summit EU 2015: Revolutionizing Big Data in the Enterprise with Spark

1. Revolutionizing Big Data in the Enterprise with Spark Ion Stoica October 28,2015

2. We Have Seen a Lot Worked with 100s companies to run Spark in production over five years Collaboratewith all major Hadoop and Big Data vendors 2

3. How Does Spark Change Enterprise Big Data? • Unifying data sources • Unifying data processing 3

4. 4 Unifying Data Sources

5. Need to process data from • Multiple sources • Different data stores and locations • Different formats Traditional solutions: ETL data into data warehouse, … Traditional Data Warehouses ETL Slow to access and combine data Data Warehouse

6. 6 Just-In-Time (JIT) Data Warehouse

7. Process data in place or stream it • No need to wait for data to be ETLed 7 JIT Data Warehouse ETL Data Warehouse

8. Process data in place or stream it • No need to wait for data to be ETLed Cachedata in memory or SSDs 8 JIT Data Warehouse Low latency and easy to combine data: value!

9. Analogy 9 Stream/cache & Play Download & Play

10. Analogy 10 ETL & Query Data Source A ETL Data Warehouse Data Source B Data Source B Data Source A Data Source B Data Source B Stream/Cache + Query

11. Top-3 Media Company Data sources • Traditional data warehouse:Customer transaction and profile data • S3: Clickstream and historical logs • Elasticsearch: User-submitted reviewsand comments • Kafka: Streaming online eventdata Build Spark-basedJIT Data Warehouseto perform real-time analytics 11

12. 12 Unifying Data Processing

13. Unified supportfor • Batch • Streaming • ML/Graphs • … 13 Spark: Unified Engine GraphXMLlib Core Spark Streaming SparkSQL SparkR Easy to manage, learn, and combine functionality

14. Analogy First cellular phones Unified device (smartphone) Specialized devices Better Games Better GPSBetter Phone

15. Analogy Batch processing Unified systemSpecialized systems Real-time analytics Instant fraud detection Better Apps

16. Large On-line Service Company Leverages • Interactive query processing • ML and combines data from S3, Redshift, and HBase to provide • data analyticsfor productmanagementteam • advanced predictive analyticsto delivernew services(e.g., customized inventory displaystailored to each user) 16

18. Demo Setting 18 MLlib Core Spark Streaming SparkSQL HDFS RedShift