Scheduling big data workloads on serverless infrastructure

•

0 gefällt mir•72 views

Ilai Malka from Nielsen at AWS Community Day TLV, December 2019 (https://awscommunitydaytelaviv2019.splashthat.com/): Scheduling big data workloads is challenging. It's extra challenging when running on Serverless infrastructure. At Nielsen Marketing Cloud, we've built a system that uploads 250 billion events per day to partner ad platforms, running on Serverless infrastructure (AWS Lambda and OpenFaaS). Creating a 'scheduler' for this system required: 1. Rate-limiting to prevent flooding partner platforms. 2. High utilization to keep costs low 3. Careful bottleneck management to keep the system humming https://www.linkedin.com/in/ilai-malka-93b06172/ https://twitter.com/IlaiMalka #Nielsen #NielsenMarketingCloud #AWSCommunityDay #Serverless

Daten & Analysen

Scheduling big data workloads
on Serverless infrastructure

What You’ll Hear About
• Our data pipeline
• Why Serverless ?
• Problems we had to solve
- Cost
- Rate Limiting

About me
My Post about Serverless:
https://medium.com/nmc-techblog/going-serverless-c334ae242ca6
NMC Tech Blog:
https://medium.com/nmc-techblog
Ilai Malka
Big Data Developer

Segmentation Upload To Networks Run Campaigns
About Nielsen Marketing Cloud (NMC)
10 Billion Proﬁles 140 Ad Networks 999+ Campaigns

Segmentation Upload To Networks Run Campaigns
About Nielsen Marketing Cloud (NMC)
10 Billion
Proﬁles
140 Ad
Networks 9999
Campaigns
250 Billion
Events

250 Billion
Events/Day
Scale up/down
Cost
Eﬀectiveness
The Challenges

250 Billion
Events/Day
0.3 - 1 TB/hour
Quickly Scales
Up and Down
Top Day Ever
750 Billion
Events/Day

~ $1000
per Day
~ $4
Per Billion Events
Total cost: $350k /year
Lambda 74%
S3 13%
DB 11%
Other 2%

Warning: Cost can skyrocket
Incentive to optimize
Cost is linear to computation power

$$$$$ == memory * duration
memory
duration
How does Lambda pricing work?

Ad Networks
Ad networks are not totally scalable

Problems:
1. Can’t support sub-minute windows
2. Manager is a bottleneck

11:00:00 - 11:00:59 11:01:00 - 11:01:59
1 minutes window
50MB
250Mb
250Mb

43%
utilization
28%
utilization
11:00:00 - 11:00:59 11:01:00 - 11:01:59

Key takeaways
• Serverless is the next revolution
• Serverless has a built in scalability feature + shorter time to market
• Cost is linear to computation power
• Incentive to optimize. optimize=costs saving
• Cost formula is not straightforward -> Find right memory setting with tool
• Costs can get out of control -> add alerts
• Hybrid solution = scalability + low costs
• Rest of the world don’t use serverless so we need to avoid flooding them
• Find the right bottleneck and solve it

Weitere ähnliche Inhalte

Was ist angesagt?

AWS Enterprise Summit London | National Rail Enquiries Darwin Migration

Amazon Web Services

Lessons from the SWIM Masterclass, XMAN and recent launch of the IHS Jane’s 2016 award winning Laminar Data platform in March 2016, bringing operational SWIM to a wider market. What is the Aviation API Economy and what is the role of SWIM alongside other ubiquitous web standards and ways of working? When a slow moving industry meets internet speed ideas - what are the challenges and opportunities? How can the latent value within operational data be applied to new use cases? This talk relays some lessons on real world APIs, customer feedback, Agile vs Waterfall, SWIM vs non-SWIM, and Cloud vs Edge.

The Aviation API Economy: More than SWIM and Waterfall

Alex Brooker

Reply Webinar Online - Mastering AWS - AI as a Service

Andrea Mercanti

Automating Multi-Cloud Policies for AWS, Azure, Google, and More

RightScale

AWS users group presentation optimizing your aws account with Cloudmgr & Clou...

Craig Deveson

So you're in the Cloud — but what does that mean? With choices like multi-region, multi-Cloud, hybrid and mixed environments, there's no "one size fits all" when it comes to Cloud. But whatever your choice, Akamai can help you with your Cloud infrastructure before, during and after your Cloud transformation. In this session, Akamai and Rackspace will explore and showcase common Cloud transformations: (1) typical ordering and fulfillment apps transformation from an on-premise, dedicated DC to the Cloud using Amazon AWS (2) similar transformation using Microsoft Azure (3) multi-Cloud environments across AWS, Azure, GCP (Google Cloud Platform), Rackspace and dedicated hosting. For each use case, we'll illustrate how Akamai can help you achieve your business needs of availability, scalability, security, cost and user-experience while you can focus on your core business goals.

Leveraging Akamai to Make Your Cloud Transformation Initiatives Successful

Akamai Developers & Admins

For many, Serverless is yet another technical solution for their IT and business challenges. But by correlating Serverless with digital innovation and other business evolutions, we can gain greater insights. During this presentation we will run through several reference cases in which Serverless application design and development on Amazon Web Services has positively influenced the business of our customers. This will be a non-technical session.

Openbar Kontich // Serverless - A view from the business side by Stef Ceyssen...

Openbar

As cloud use explodes, it’s important to develop a strategy for orchestrating and automating across all your cloud and non-cloud infrastructure environments. However, the use of the term orchestration has become overloaded and can refer to many different types of tools. Learn about the different types of orchestration automation tools and the role a cloud management platform (CMP) can play as an “orchestrator of orchestrators.”

Successful Cloud Orchestration with RightScale CMP

RightScale

Got a Multi-Cloud Strategy? How RightScale CMP Helps

RightScale

AWS Summit Berlin 2018 - Love is in the Cloud - MassMigration to AWS

Andrea Mercanti

Spotinst: Cut AWS Costs: Using Spot Instances for More Than Batch

RightScale

How to migrate Exact Online to AWS

Vincent Lesierse

The Benefits of a Public Cloud: Why You Really Can't Build a Better One

Chef

"As key players in the OTT market, BAMTech Media distributes over 50,000 live events annually. As BAM continues to add more partners and deliver larger events, Akamai technology and team have become an integral part of BAM’s delivery process. Leveraging the latest Akamai APIs, a template-driven provisioning process and iterative configuration enhancements, BAM is able to deliver an optimized experience to the end user on a consistent basis by employing the concept of a """"Golden Configuration."""" The goal of this session is to explore, the critical factors which enable BAM to deliver direct-to-consumer live streaming services at scale for a growing roster of sports, news, and entertainment partners. Our focus will revolve around two key success factors instrumental in scaling of BAM live events and evolution of their cloud strategy for streaming. Those factors are: * Standardization (through development of BAM “Golden” Config) * API Provisioning (Akamai configurations & workflows) We’ll take you through the actual Akamai delivery rules & variables used; even walk you through the custom tools we built as part of a live demo."

How BAMTECH Media Delivers 50,000 Live Events via APIs

Akamai Developers & Admins

C sharp annual conference 15-apr-18 - delhi

Albert Anthony

Session01 introduction to_cloud_by_kalyan_mahalingam

Ashish Pandey

Monolith to Micro-Services

Andrzej Czerwinski

Cloud Management for MSPs

RightScale

Building Scalable Data Ingestion

Michael Pearce

AWS Cloud Cost Optimization

Yogesh Sharma

Was ist angesagt? (20)

AWS Enterprise Summit London | National Rail Enquiries Darwin Migration

The Aviation API Economy: More than SWIM and Waterfall

Reply Webinar Online - Mastering AWS - AI as a Service

Automating Multi-Cloud Policies for AWS, Azure, Google, and More

AWS users group presentation optimizing your aws account with Cloudmgr & Clou...

Leveraging Akamai to Make Your Cloud Transformation Initiatives Successful

Openbar Kontich // Serverless - A view from the business side by Stef Ceyssen...

Successful Cloud Orchestration with RightScale CMP

Got a Multi-Cloud Strategy? How RightScale CMP Helps

AWS Summit Berlin 2018 - Love is in the Cloud - MassMigration to AWS

Spotinst: Cut AWS Costs: Using Spot Instances for More Than Batch

How to migrate Exact Online to AWS

The Benefits of a Public Cloud: Why You Really Can't Build a Better One

How BAMTECH Media Delivers 50,000 Live Events via APIs

C sharp annual conference 15-apr-18 - delhi

Session01 introduction to_cloud_by_kalyan_mahalingam

Monolith to Micro-Services

Cloud Management for MSPs

Building Scalable Data Ingestion

AWS Cloud Cost Optimization

Ähnlich wie Scheduling big data workloads on serverless infrastructure

RightScale Webinar: July 23, 2009 – Listen to a recorded live panel discussion moderated by Dave Nielsen, founder of CloudCamp and Principal Consultant at Platform D. Hear from leading experts in cloud computing to gain real-world insight into how their customers are using cloud computing and achieving new levels of backup, recovery and reliability. Learn how you can migrate your apps and data to the cloud in a way that's affordable and reliable. Watch video at http://vimeo.com/rightscale/transitioning-to-the-cloud.

Transitioning to the Cloud: Implications for Reliability, Redundancy & Recove...

RightScale

Cloud Computing

Ed Byrne

Biznet Gio Presentation - Cloud Computing

Yusuf Hadiwinata Sutandar

Financial impact of Cloud Computing

krisbliesner

Scaling highly available database infrastructure to 100x, 1000x, and beyond has historically been one of the hardest technical challenges that any successful web business must face. This is quickly changing with fully-managed database services such as Amazon DynamoDB and Amazon Redshift, as the scaling efforts which previously required herculean effort are now as simple as an API call. Over the last few years, Twilio has evolved their database infrastructure to a pipeline consisting of Amazon SQS, Sharded MySQL, Amazon DynamoDB, Amazon S3, Amazon EMR and Amazon Redshift. In this session, Twilio cover show they achieved success, specifically: - How they replaced their data pipeline deployed to Amazon EC2 to meet their scaling needs with zero downtime. - How they adopted Amazon DynamoDB and Amazon Redshift at the same scale as their MySQL infrastructure, at 1/5th the cost and operational overhead. - Why they believe adopting managed database services like Amazon DynamoDB is key to accelerating delivery of value to their customers. Sponsored by Twilio.

(BDT312) Using the Cloud to Scale from a Database to a Data Platform | AWS re...

Amazon Web Services

Learn how to accelerate and de-risk your cloud migration project Despite the surge in enterprises migrating applications to the public cloud, more than half of all projects are delayed or over budget and an even greater number are more difficult than expected.1 Cloud Migrations don’t begin when you start moving applications into the cloud. They begin with your application landscape discovery and assessment. The second phase comprises the actual migration where applications are moved to the public cloud. Working with purpose-built enterprise-grade cloud migration platforms, especially those that partner to integrate both phases greatly simplifies and accelerates projects. RISC Networks and Velostrata have teamed up to deliver this webinar where we’ll share real-world examples, tips, and tricks on crafting a seamless cloud migration from start to completion.

Risc and velostrata 2 28 2018 lessons_in_cloud_migration

RISC Networks

Keeping Technology Current: A Driver for Change

Cisco Canada

Cloud: The Commercial Silver Lining for Partners

Amazon Web Services

We’ve all seen the trend everywhere around us: customers want self-service. It offers them the agility they need and gives businesses the ability to scale and lower their costs. With cloud deployments, enterprises can experience similar benefits through the use of a self-service portal where internal customers can provision their own resources while Central IT maintains control and visibility. This saves both time and money. In this webinar, learn how to empower your internal customers to provision the necessary cloud resources when they need them but also ensure that what get receive is well within IT approved guidelines. Beyond simple convenience, this methodology permits you to operationalize your AWS cloud usage to easily roll cloud into an overall IT strategy. Architects from Amazon Web Services (AWS) and RightScale, an Advanced Technology Partner, will provide an overview of the key business and technical considerations for operationalizing your AWS cloud usage. In the second half of the webinar, our technical experts will answer your questions. Priority will be given to pre-submitted questions. To help illustrate the effectiveness of this approach, our architects will walk you through real-world examples and the overall impact on their organizations. Key Topics: 1. Create an IT Vending machine with consistent and reproducible processes. 2. Enable your end users while maintaining visibility and control. 3. Use cost planning and forecasting to fine-tune and understand cloud spend. 4. Discover reporting and auditing tools to ensure compliance. 5. Avoid downtime through proven HA/DR architectures.

RightScale Webinar: Operationalize Your Enterprise AWS Usage Through an IT Ve...

RightScale

Enterprises are rapidly adopting stream computing backbones, in-memory data stores, change data capture, and other low-latency approaches for end-to-end applications. As businesses modernize their data architectures over the next several years, they will begin to evolve toward all-streaming architectures. In this webcast, Wikibon, Attunity, and MemSQL will discuss how enterprise data professionals should migrate their legacy architectures in this direction. They will provide guidance for migrating data lakes, data warehouses, data governance, and transactional databases to support all-streaming architectures for complex cloud and edge applications. They will discuss how this new architecture will drive enterprise strategies for operationalizing artificial intelligence, mobile computing, the Internet of Things, and cloud-native microservices. Link to the Wikibon report - wikibon.com/wikibons-2018-big-data-analytics-trends-forecast Link to Attunity Streaming CDC Book Download - http://www.bit.ly/cdcbook Link to MemSQL's Free Data Pipeline Book - http://go.memsql.com/oreilly-data-pipelines

Digital Business Transformation in the Streaming Era

Attunity

Was asked by In-Q-Tel leadership to present on how Open Source competes with larger technology companies (e.g. VMWare) and how household cloud providers (SalesForce, Amazon...) are embedding open source for competitive advantages. Step through Navy CANES program as example of DoD usage. Covers IBM and DreamWorks commercial references. Illustrates DISA Anti-Drug Network (ADNET) and U.S. Intelligence Community Persons of Interest programs. Talk about Open Source initiatives at NGA lead by Bert Beaulieu (Director, Innovision, NGA) and Michael Cline (R&D Technologist, NGA).

2011-08-10 In-Q-Tel Technology Focus Day, Trends & Observations in Open Sourc...

Shawn Wells

As organizations make the move to cloud platforms to support modern analytics and applications, they often need to move more than their on-premises data warehouses and lakes. Organizations often have legacy mainframe systems that run their mission-critical and revenue-generating applications, and that should be included in modern cloud projects. Yet it can be difficult to replicate mainframe data onto cloud platforms to make it a part of your data-driven initiatives, due to the data’s complex format. When moving data to the cloud it needs to maintain accuracy, consistency, and context to be considered trusted data. At the same time, the data needs to be available in real time to cloud-based data management and analytics applications. Join this session for a discussion with experts from Precisely and AWS about how to bring your data to applications and services on the cloud. Topics include: The importance of mainframe data for organizationsHow to migrate/replicate mainframe data to cloud platformsHow to ensure trusted data on cloud platforms

The Future of Mainframe Data is in the Cloud

Precisely

Weidian is a social-based e-commerce platform that helps people with dreams to start a business easier. It is also committed to providing consumers with a useful, fun and attitude-oriented shopping platform. Since started from 2014, it has reached 170 millions app downloads in total, and millions of average DAU. This slides introduces how Weidian's architecture is migrating from monolithic application to Dubbo's based micro-service architecture.

Dubbo and Weidian's practice on micro-service architecture

Huxing Zhang

Lessons from Building Large-Scale, Multi-Cloud, SaaS Software at Databricks

Databricks

AWS Webcast - Informatica - Big Data Solutions Showcase

Amazon Web Services

Introduction to Cloud Computing with Amazon Web Services

Amazon Web Services

NoOps in a Serverless World

Gary Arora

With all the hype around Cloud and SDN, business decision makers are finding themselves trying to navigate through many new concepts and consequently needing to change the way they have traditionally selected their IT infrastructure. Technologies are now becoming more integrated and it is more important than ever to help your business be agile enough to keep up with the demands of your users and your customers. Come hear from Lisa Guess to learn how organizations can embrace Cloud technologies such as automation, SDN and Orchestration platforms to help you build next-generation networks.

Lisa Guess - Embracing the Cloud

centralohioissa

Every software team has its moments of truth. How does this impact the way agile developers, site reliability engineers, and IT operations teams work together? We'll break down the intricacies of modern monitoring and show you what to look for, particularly when monitoring microservices and dynamic clouds. With examples from New Relic customers, you'll learn what to look out for when preparing to conquer your digital moments of truth, master microservices, using cloud services for autoscaling, and getting your teams to work together. I also added a quick bit on quickly evaluating the security of a cloud service provider before you engage your infosec team.

11 Ways Microservices & Dynamic Clouds Break Your Monitoring

Abner Germanow

Fed up with stop and go in your data center? Why not shift into overdrive and pull into the fast lane? Learn how AutoScout24 are building their Autobahn in the cloud to become the market leader in Europe's vehicle classified business. Reinventing themselves by making a radical transition from monoliths to microservices, from .NET on Windows to Scala on Linux, from data center to AWS and from built by devs and run by ops to a devops mindset. While the current stack keeps running, ever more microservices will go live as you listen to stories from the trenches. Key takeaways from this talk includes: How to... … become cloud native … evolve the architecture … create “you build it you run it” teams … involve business people in the transformation Created and presented together with Wolf Schleger (ThoughtWorks)

Highway to heaven - XConf Manchester 2015

Christian Deger

Ähnlich wie Scheduling big data workloads on serverless infrastructure (20)

Transitioning to the Cloud: Implications for Reliability, Redundancy & Recove...

Cloud Computing

Biznet Gio Presentation - Cloud Computing

Financial impact of Cloud Computing

(BDT312) Using the Cloud to Scale from a Database to a Data Platform | AWS re...

Risc and velostrata 2 28 2018 lessons_in_cloud_migration

Keeping Technology Current: A Driver for Change

Cloud: The Commercial Silver Lining for Partners

RightScale Webinar: Operationalize Your Enterprise AWS Usage Through an IT Ve...

Digital Business Transformation in the Streaming Era

2011-08-10 In-Q-Tel Technology Focus Day, Trends & Observations in Open Sourc...

The Future of Mainframe Data is in the Cloud

Dubbo and Weidian's practice on micro-service architecture

Lessons from Building Large-Scale, Multi-Cloud, SaaS Software at Databricks

AWS Webcast - Informatica - Big Data Solutions Showcase

Introduction to Cloud Computing with Amazon Web Services

NoOps in a Serverless World

Lisa Guess - Embracing the Cloud

11 Ways Microservices & Dynamic Clouds Break Your Monitoring

Highway to heaven - XConf Manchester 2015

Mehr von Itai Yaffe

Yulia Antonovsky (Senior Software Engineer II) @ Akamai: Our cloud-based ingest pipeline processes over 10 Gb of security events data per second, which demands high-performance processing and analysis. To achieve this, we've implemented efficient partitioning using Java and Spark applications running on AKS and leveraging Kafka. This allows us to provide real-time analytics within two minutes and heavy batch processing for deeper analysis hourly. During this talk, we will cover how we use Kafka to scale our Spark application on K8s, partitioning strategies for high-volume data processing, and how partitioning helps avoid storage throttling issues.

Mastering Partitioning for High-Volume Data Processing

Itai Yaffe

Maayan Gad (Senior Big Data Engineer) @ Wix: Who likes to maintain long SQL files? No one. Column renames, additions, deletions, changes in the KPIs your tables are based on, or the date your table should start from - all are annoying, time consuming tasks your data engineers waste their time on. In this talk we’ll uncover how we solved this problem at Wix, allowing our data engineers to easily create Data Warehouse grade tables using configuration files, and even transfer some of the business-related-only parts to the data analysts.

Solving Data Engineers Velocity - Wix's Data Warehouse Automation

Itai Yaffe

Ada Sharoni (Software Engineering Architect) @ Hunters: Imagine you had to manage thousands of Spark applications that are automatically spinning up on-demand upon every customer interaction. Our unique constraints in Hunters have led us to adopt an architecture and concepts that we believe many other companies will find useful. In this lecture we will share our solutions and insights in running many lightweight, cheap Spark applications on Kubernetes, that can easily survive frequent restarts and smartly share resources on Spot EC2 instances.

Lessons Learnt from Running Thousands of On-demand Spark Applications

Itai Yaffe

María de la Fuente (Solutions Architect Manager for IMEA) @ Databricks While most companies understand the value creation of leveraging data and are taking on board an AI strategy, only 13% of the data science projects make it to production successfully. Besides the well-known skills gap in the market, we need to level up our end-to-end approach and cover all aspects involved when working with AI. In this session, we will discuss the main obstacles to overcome and how we can avoid the major pitfalls to ensure our data science journey becomes successful.

Why do the majority of Data Science projects never make it to production?

Itai Yaffe

Eynav Mass (VP R&D) @ Oribi: When it comes to data solutions, one-size doesn't fit all. Choosing the right best-matching database, or data tools, can be a game-changer for your system. How can you take such a decision effectively? The system, the company, the product, and probably your team - all are evolving, and the best solution for today may not fit tomorrow's needs. In order to pick a data solution for longer term, you should evaluate the optional data tools according to several factors. These factors will reflect the requirements looking forward. At the session, we will discuss these factors, along with sharing some real-life stories and lessons learned, to help you properly plan & prepare your data solutions.

Planning a data solution - "By Failing to prepare, you are preparing to fail"

Itai Yaffe

Evaluating Big Data & ML Solutions - Opening Notes

Itai Yaffe

Jon Bratseth (VP Architect) @ Verizon Media: The big data world has mature technologies for offline analysis and learning from data, but have lacked options for making data-driven decisions in real time. When it is sufficient to consider a single data point model servers such as TensorFlow serving can be used but in many cases you want to consider many data points to make decisions. This is a difficult engineering problem combining state, distributed algorithms and low latency, but solving it often makes it possible to create far superior solutions when applying machine learning. This talk will explain why this is a hard problem, show the advantages of solving it, and introduce the open source Vespa.ai platform which is used to implement such solutions in some of the largest scale problems in the world including the world's third largest ad serving system.

Big data serving: Processing and inference at scale in real time

Itai Yaffe

Sharon Dashet (Sr. Data Analytics Solution Lead) @ Google Cloud: The worlds of traditional RDBMS and Data Lake Hadoop systems are converging and moving to public cloud and SaaS offerings. In this session, Sharon will share her personal journey as a data professional since the 90s weaved into the history of data management systems. The session will also cover the differences between on-premise and cloud Data Lakes.

Data Lakes on Public Cloud: Breaking Data Management Monoliths

Itai Yaffe

Orit Alul (Sr. Solutions Architect) @ AWS: As data is growing at an exponential rate, we are interested not only in being able to analyze the past or present but also in predicting the future! In this session, Orit will talk about the power of data combined with machine learning. Building a highly scalable and flexible data architecture in the cloud to collect, process, and analyze data, in order to get timely insights and react quickly to new information. In addition, Orit will present best practices, performance and optimization tips for building a Data Lake in the cloud.

Unleashing the Power of your Data

Itai Yaffe

Data Lake on Public Cloud - Opening Notes

Itai Yaffe

Roi Teveth (Data Engineer) and Itai Yaffe (Tech Lead, Big Data group) @ Nielsen: At Nielsen Identity Engine, we use Spark to process 10’s of TBs of data. Our ETLs, orchestrated by Airflow, spin-up AWS EMR clusters with thousands of nodes per day. In this talk, we’ll guide you through migrating Spark workloads to Kubernetes with minimal changes to Airflow DAGs, using the open-sourced GCP Spark-on-K8s operator and the native integration we recently contributed to the Airflow project.

Airflow Summit 2020 - Migrating airflow based spark jobs to kubernetes - the ...

Itai Yaffe

Itai Yaffe (Tech Lead, Big Data group) @ Nielsen: Every day, millions of advertising campaigns are happening around the world. As campaign owners, measuring the ongoing campaign effectiveness (e.g "how many distinct users saw my online ad VS how many distinct users saw my online ad, clicked it and purchased my product?") is super important. However, this task (often referred to as "funnel analysis") is not an easy task, especially if the chronological order of events matters. So, while the combination of Druid and ThetaSketch aggregators can answer some of these questions, it still can’t answer the question "how many distinct users viewed the brand’s homepage FIRST and THEN viewed product X page?" In this talk, we will discuss how we combine Spark, Druid and ThetaSketch aggregators to answer such questions at scale.

DevTalks Reimagined 2020 - Funnel Analysis with Spark and Druid

Itai Yaffe

Virtual Apache Druid Meetup: AIADA (Ask Itai and David Anything)

Itai Yaffe

Introducing Kafka Connect and Implementing Custom Connectors

Itai Yaffe

Benjamin Hopp (Solutions Architect) @ Imply: Druid is an emerging standard in the data infrastructure world, designed for high-performance slice-and-dice analytics (“OLAP”-style) on large data sets. This talk is for you if you’re interested in learning more about pushing Druid’s analytical performance to the limit. Perhaps you’re already running Druid and are looking to speed up your deployment, or perhaps you aren’t familiar with Druid and are interested in learning the basics. Some of the tips in this talk are Druid-specific, but many of them will apply to any operational analytics technology stack. The most important contributor to a fast analytical setup is getting the data model right. The talk will center around various choices you can make to prepare your data to get best possible query performance. We’ll look at some general best practices to model your data before ingestion such as OLAP dimensional modeling (called “roll-up” in Druid), data partitioning, and tips for choosing column types and indexes. We’ll also look at how more can be less: often, storing copies of your data partitioned, sorted, or aggregated in different ways can speed up queries by reducing the amount of computation needed. We’ll also look at Druid-specific optimizations that take advantage of approximations; where you can trade accuracy for performance and reduced storage. You’ll get introduced to Druid’s features for approximate counting, set operations, ranking, quantiles, and more. And we will finish with the latest and greatest Druid news, including details about the latest roadmap and releases.

A Day in the Life of a Druid Implementor and Druid's Roadmap

Itai Yaffe

Dr. Edward (Eddie) Bortnikov (Senior Director of Research) @ Verizon Media: Ingestion and queries of real-time data in Druid are performed by a core software component named Incremental Index (I^2). I^2’s scalability is paramount to the speed of the ingested data becoming queryable as well as to the operational efficiency of the Druid cluster. The current I^2 Implementation is based on the traditional ordered JDK key-value (KV-)map. We present an experimental I^2 implementation that is based on a novel data structure named OakMap - a scalable thread-safe off-heap KV-map for Big Data applications in Java. With OakMap, I^2 can ingest data at almost 2x speed while using 30% less RAM. The project is expected to become GA in 2020.

Scalable Incremental Index for Druid

Itai Yaffe

Itai Yaffe (Tech Lead, Big Data group) @ Nielsen: Every day, millions of advertising campaigns are happening around the world. As campaign owners, measuring the ongoing campaign effectiveness (e.g “how many distinct users saw my online ad VS how many distinct users saw my online ad, clicked it and purchased my product?”) is super important. However, this task (often referred to as “funnel analysis”) is not an easy task, especially if the chronological order of events matters. So, while the combination of Druid and ThetaSketch aggregators can answer some of these questions, it still can’t answer the question "how many distinct users viewed the brand’s homepage FIRST and THEN viewed product X page?" In this talk, we will discuss how we combine Spark, Druid and ThetaSketch aggregators to answer such questions at scale.

Funnel Analysis with Spark and Druid

Itai Yaffe

Shir Bromberg (Big Data team leader) @ Yotpo: Nowadays, many of an organization’s main applications rely on Spark pipelines. As these applications become more significant to businesses, so does the need to quickly deploy, test and monitor them. The standard way of running spark jobs is to deploy it on a dedicated managed cluster. However, this solution is relatively expensive with potentially high setup time. Therefore, we developed a way to run Spark on any container orchestration platform. This allows us to run Spark in a simple, custom and testable way. In this talk, we will present our open-source dockers for running Spark on Nomad servers. We will cover: * The issues we had running spark on managed clusters and the solution we developed. * How to build a spark docker. * And finally, what you may achieve by using Spark on Nomad.

The benefits of running Spark on your own Docker

Itai Yaffe

Etti Gur (Senior Big Data developer) and Itai Yaffe (Tech Lead, Big Data group) @ Nielsen: At Nielsen Marketing Cloud, we provide our customers (marketers and publishers) real-time analytics tools to measure their ongoing campaigns' efficiency. To achieve that, we need to ingest billions of events per day into our big data stores and we need to do it in a scalable yet cost-efficient manner. In this talk, we will discuss how we significantly optimized our Spark-based in-flight analytics daily pipeline, reducing its total execution time from over 20 hours down to 2 hours, resulting in a huge cost reduction. Topics include: * Ways to identify optimization opportunities * Optimizing Spark resource allocation * Parallelizing Spark output phase with dynamic partition inserts * Running multiple Spark "jobs" in parallel within a single Spark application

Optimizing Spark-based data pipelines - are you up for it?

Itai Yaffe

GraphQL API on a Serverless Environment

Itai Yaffe

Mehr von Itai Yaffe (20)

Mastering Partitioning for High-Volume Data Processing

Solving Data Engineers Velocity - Wix's Data Warehouse Automation

Lessons Learnt from Running Thousands of On-demand Spark Applications

Why do the majority of Data Science projects never make it to production?

Planning a data solution - "By Failing to prepare, you are preparing to fail"

Evaluating Big Data & ML Solutions - Opening Notes

Big data serving: Processing and inference at scale in real time

Data Lakes on Public Cloud: Breaking Data Management Monoliths

Unleashing the Power of your Data

Data Lake on Public Cloud - Opening Notes

Airflow Summit 2020 - Migrating airflow based spark jobs to kubernetes - the ...

DevTalks Reimagined 2020 - Funnel Analysis with Spark and Druid

Virtual Apache Druid Meetup: AIADA (Ask Itai and David Anything)

Introducing Kafka Connect and Implementing Custom Connectors

A Day in the Life of a Druid Implementor and Druid's Roadmap

Scalable Incremental Index for Druid

Funnel Analysis with Spark and Druid

The benefits of running Spark on your own Docker

Optimizing Spark-based data pipelines - are you up for it?

GraphQL API on a Serverless Environment

Kürzlich hochgeladen

Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage with sex Booking Contact Details :- WhatsApp Chat :- +91-9920725232 4-May-2024(SMW) Call Girls In Model Towh +91-9920725232 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-9920725232 (Any Time. Any Where) Call Girls in , Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —9920725232 We are available 24*7 all days of the year. Call us — 9920725232 Thank you for Visiting.

Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...

amitlee9823

Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore Booking Contact Details :- WhatsApp Chat :- +91-7737669865 2-May-2024(SMW) Call Girls In Model Towh Bangalore +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in Bangalore NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in Bangalore, Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service Bangalore NCRWelcome To Bangalore Escorts Service – An All Over New Bangalore Very Sexy Hot Call Girls Agency Service Escorts In South BangaloreNCRBangalore’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Bangalore Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...

amitlee9823

Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage with sex Booking Contact Details :- WhatsApp Chat :- +91-9920725232 4-May-2024(SMW) Call Girls In Model Towh +91-9920725232 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-9920725232 (Any Time. Any Where) Call Girls in , Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —9920725232 We are available 24*7 all days of the year. Call us — 9920725232 Thank you for Visiting.

Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...

amitlee9823

Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore Booking Contact Details :- WhatsApp Chat :- +91-7737669865 2-May-2024(SMW) Call Girls In Model Towh Bangalore +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in Bangalore NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in Bangalore, Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service Bangalore NCRWelcome To Bangalore Escorts Service – An All Over New Bangalore Very Sexy Hot Call Girls Agency Service Escorts In South BangaloreNCRBangalore’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Bangalore Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...

amitlee9823

👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amritsar No💰Advance Cash On Delivery Service Escorts Service Available Whatsapp Chaya ☎️ : [+91-6367187148] Escorts Service Amritsar are always ready to make their clients happy. Their exotic looks and sexy personalities are sure to turn heads. You can enjoy with them, including massages and erotic encounters.#P12Our area Escorts are young and sexy, so you can expect to have an exotic time with them. They are trained to satiate your naughty nerves and they can handle anything that you want. They are also intelligent, so they know how to make you feel comfortable and relaxed SERVICE ✅ ❣️ ⭐➡️HOT & SEXY MODELS // COLLEGE GIRLS HOUSE WIFE RUSSIAN , AIR HOSTES ,VIP MODELS . AVAILABLE FOR COMPLETE ENJOYMENT WITH HIGH PROFILE INDIAN MODEL AVAILABLE HOTEL & HOME ★ SAFE AND SECURE HIGH CLASS SERVICE AFFORDABLE RATE ★ SATISFACTION,UNLIMITED ENJOYMENT. ★ All Meetings are confidential and no information is provided to any one at any cost. ★ EXCLUSIVE PROFILes Are Safe and Consensual with Most Limits Respected ★ Service Available In: - HOME & HOTEL Star Hotel Service .In Call & Out call SeRvIcEs : ★ A-Level (star escort) ★ Strip-tease ★ BBBJ (Bareback Blowjob)Receive advanced sexual techniques in different mode make their life more pleasurable. ★ Spending time in hotel rooms ★ BJ (Blowjob Without a Condom) ★ Completion (Oral to completion) ★ Covered (Covered blowjob Without condom ★ANAL SERVICES.

👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...

karishmasinghjnh

Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (Bangalore) Booking Contact Details :- WhatsApp Chat :- +91-7737669865 2-May-2024(SMW) Call Girls In Model Towh Bangalore +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in Bangalore NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in Bangalore, Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service Bangalore NCRWelcome To Bangalore Escorts Service – An All Over New Bangalore Very Sexy Hot Call Girls Agency Service Escorts In South BangaloreNCRBangalore’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Bangalore Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...

amitlee9823

Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore Booking Contact Details :- WhatsApp Chat :- +91-7737669865 2-May-2024(SMW) Call Girls In Model Towh Bangalore +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in Bangalore NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in Bangalore, Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service Bangalore NCRWelcome To Bangalore Escorts Service – An All Over New Bangalore Very Sexy Hot Call Girls Agency Service Escorts In South BangaloreNCRBangalore’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Bangalore Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore

amitlee9823

Saudi Arabia [ Abortion pills) Jeddah/riaydh/dammam/+966572737505☎️] cytotec tablets uses abortion pills 💊💊 How effective is the abortion pill? 💊💊 +966572737505) "Abortion pills in Jeddah" how to get cytotec tablets in Riyadh " Abortion pills in dammam*💊💊 The abortion pill is very effective. If you’re taking mifepristone and misoprostol, it depends on how far along the pregnancy is, and how many doses of medicine you take:💊💊 +966572737505) how to buy cytotec pills At 8 weeks pregnant or less, it works about 94-98% of the time. +966572737505[ 💊💊💊 At 8-9 weeks pregnant, it works about 94-96% of the time. +966572737505) At 9-10 weeks pregnant, it works about 91-93% of the time. +966572737505)💊💊 If you take an extra dose of misoprostol, it works about 99% of the time. At 10-11 weeks pregnant, it works about 87% of the time. +966572737505) If you take an extra dose of misoprostol, it works about 98% of the time. In general, taking both mifepristone and+966572737505 misoprostol works a bit better than taking misoprostol only. +966572737505 Taking misoprostol alone works to end the+966572737505 pregnancy about 85-95% of the time — depending on how far along the+966572737505 pregnancy is and how you take the medicine. +966572737505 The abortion pill usually works, but if it doesn’t, you can take more medicine or have an in-clinic abortion. +966572737505 When can I take the abortion pill?+966572737505 In general, you can have a medication abortion up to 77 days (11 weeks)+966572737505 after the first day of your last period. If it’s been 78 days or more since the first day of your last+966572737505 period, you can have an in-clinic abortion to end your pregnancy.+966572737505 Why do people choose the abortion pill? Which kind of abortion you choose all depends on your personal+966572737505 preference and situation. With+966572737505 medication+966572737505 abortion, some people like that you don’t need to have a procedure in a doctor’s office. You can have your medication abortion on your own+966572737505 schedule, at home or in another comfortable place that you choose.+966572737505 You get to decide who you want to be with during your abortion, or you can go it alone. Because+966572737505 medication abortion is similar to a miscarriage, many people feel like it’s more “natural” and less invasive. And some+966572737505 people may not have an in-clinic abortion provider close by, so abortion pills are more available to+966572737505 them. +966572737505 Your doctor, nurse, or health center staff can help you decide which kind of abortion is best for you. +966572737505 More questions from patients: Saudi Arabia+966572737505 CYTOTEC Misoprostol Tablets. Misoprostol is a medication that can prevent stomach ulcers if you also take NSAID medications. It reduces the amount of acid in your stomach, which protects your stomach lining. The brand name of this medication is Cytotec®.+966573737505) Unwanted Kit is a combination of two medici

Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec

Abortion pills in Riyadh +966572737505 get cytotec

Midocean dropshipping via API with DroFx

olyaivanovalion

Call Girls In Hsr Layout ☎ 7737669865 🥵 Book Your One night Stand Booking Contact Details :- WhatsApp Chat :- +91-7737669865 Call Girls In Model Towh +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in , Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At #K09 Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Call Girls In Hsr Layout ☎ 7737669865 🥵 Book Your One night Stand

amitlee9823

(NEHA) Call Girls Katra Call Now: 8617697112 Katra Escorts Booking Contact Details WhatsApp Chat: +91-8617697112 Katra Escort Service includes providing maximum physical satisfaction to their clients as well as engaging conversation that keeps your time enjoyable and entertaining. Plus, they look fabulously elegant, making an impression. Independent Escorts Katra understands the value of confidentiality and discretion; they will go the extra mile to meet your needs. Simply contact them via text messaging or through their online profiles; they'd be more than delighted to accommodate any request or arrange a romantic date or fun-filled night together. We provide:

(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7

Call Girls in Nagpur High Profile Call Girls

Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore Booking Contact Details :- WhatsApp Chat :- +91-7737669865 2-May-2024(SMW) Call Girls In Model Towh Bangalore +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in Bangalore NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in Bangalore, Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service Bangalore NCRWelcome To Bangalore Escorts Service – An All Over New Bangalore Very Sexy Hot Call Girls Agency Service Escorts In South BangaloreNCRBangalore’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Bangalore Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...

amitlee9823

This project demonstrates a machine learning approach to detecting credit card fraud using advanced algorithms and techniques. The project utilizes a dataset containing various features such as transaction amount, merchant location, time of transaction, and others to build a predictive model. The presentation covers data preprocessing steps, feature engineering techniques, and the selection of machine learning algorithms such as logistic regression or random forest. It also discusses model evaluation metrics and the importance of fraud detection in financial institutions for safeguarding against fraudulent activities. Visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/

Detecting Credit Card Fraud: A Machine Learning Approach

Boston Institute of Analytics

Model Call Girl Services in Delhi reach out to us at 🔝 9953056974 🔝✔️✔️ Our agency presents a selection of young, charming call girls available for bookings at Oyo Hotels. Experience high-class escort services at pocket-friendly rates, with our female escorts exuding both beauty and a delightful personality, ready to meet your desires. Whether it's Housewives, College girls, Russian girls, Muslim girls, or any other preference, we offer a diverse range of options to cater to your tastes. We provide both in-call and out-call services for your convenience. Our in-call location in Delhi ensures cleanliness, hygiene, and 100% safety, while our out-call services offer doorstep delivery for added ease. We value your time and money, hence we kindly request pic collectors, time-passers, and bargain hunters to refrain from contacting us. Our services feature various packages at competitive rates: One shot: ₹2000/in-call, ₹5000/out-call Two shots with one girl: ₹3500/in-call, ₹6000/out-call Body to body massage with sex: ₹3000/in-call Full night for one person: ₹7000/in-call, ₹10000/out-call Full night for more than 1 person: Contact us at 🔝 9953056974 🔝. for details Operating 24/7, we serve various locations in Delhi, including Green Park, Lajpat Nagar, Saket, and Hauz Khas near metro stations. For premium call girl services in Delhi 🔝 9953056974 🔝. Thank you for considering us!

CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE

9953056974 Low Rate Call Girls In Saket, Delhi NCR

➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Escorts Service Booking Contact Details :- WhatsApp Chat :- +91-7737669865 Call Girls In Model Towh +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in , Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At #K09 Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...

amitlee9823

Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand Booking Contact Details :- WhatsApp Chat :- +91-7737669865 Call Girls In Model Towh +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in , Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At #K09 Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand

amitlee9823

➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Service Booking Contact Details :- WhatsApp Chat :- +91-7737669865 Call Girls In Model Towh +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in , India,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At #K09 Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...

amitlee9823

Call Girl In Dwarka ☎92055#41914 ¶¶ Indian,Russian Best Quality full Educated And Full Cooperative Independent Call Girls Escort Services In New Delhi- I Have Extremely Beautiful Broad Minded Cute Sexy & Hot Call Girls and Escorts, We Are Located in 3* 4* 5* Hotels in Delhi. Safe & Secure High Class Services Affordable Rate 100% Satisfaction, Unlimited Enjoyment. Any Time for Model/Teens Escort in Delhi High class luxury and premium escorts agency Indian Russian Call Girls In Delhi Booking Good High Profile Escorts (Call Girls) In Delhi 5 Star Hotel ,Incall Service,OutCall Service, We provide services by Call Girls,College Girls,Modals Get High Profile queens,Well Educated,Good Looking,Full Cooperative Model, Russian Models,Punjabi Girls Kashmeri Girls Services etc… We Provide Hottest Female With Safe And Consensual With Most Limits Respected Complete Satisfaction Guaranteed…Service. Call Me Spacial For Including Incall//outcall Service In New Delhi Indian Russian Escorts Service,

Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night

Delhi Call girls

Anomaly detection and data imputation within time series

Paris Women in Machine Learning and Data Science

Probability Grade 10 Third Quarter Lessons

JoseMangaJr1

Kürzlich hochgeladen (20)

Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...

Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...

Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...

Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...

👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...

Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...

Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore

Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec

Midocean dropshipping via API with DroFx

Call Girls In Hsr Layout ☎ 7737669865 🥵 Book Your One night Stand

(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7

Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...

Detecting Credit Card Fraud: A Machine Learning Approach

CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE

➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...

Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand

➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...

Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night

Anomaly detection and data imputation within time series

Probability Grade 10 Third Quarter Lessons

Scheduling big data workloads on serverless infrastructure

1. Scheduling big data workloads on Serverless infrastructure

2. What You’ll Hear About • Our data pipeline • Why Serverless ? • Problems we had to solve - Cost - Rate Limiting

3. About me My Post about Serverless: https://medium.com/nmc-techblog/going-serverless-c334ae242ca6 NMC Tech Blog: https://medium.com/nmc-techblog Ilai Malka Big Data Developer

4. Segmentation Upload To Networks Run Campaigns About Nielsen Marketing Cloud (NMC) 10 Billion Proﬁles 140 Ad Networks 999+ Campaigns

5. Segmentation Upload To Networks Run Campaigns About Nielsen Marketing Cloud (NMC) 10 Billion Proﬁles 140 Ad Networks 9999 Campaigns 250 Billion Events

6. 250 Billion Events/Day Scale up/down Cost Eﬀectiveness The Challenges

7. Serveless (AWS Lambda )

8. 250 Billion Events/Day 0.3 - 1 TB/hour Quickly Scales Up and Down Top Day Ever 750 Billion Events/Day

9. ~ $1000 per Day ~ $4 Per Billion Events Total cost: $350k /year Lambda 74% S3 13% DB 11% Other 2%

10. Cost Tips

11. Warning: Cost can skyrocket Incentive to optimize Cost is linear to computation power

12. $$$$$ == memory * duration memory duration How does Lambda pricing work?

13. Our next plan: a Hybrid solution

14. Rate Limiting Tips

15. Ad Networks Ad networks are not totally scalable

16. Bottleneck Management

17. Problems: 1. Can’t support sub-minute windows 2. Manager is a bottleneck

18.

19. 11:00:00 - 11:00:59 250Mb 50MB 100Mb

20. 11:00:00 - 11:00:59 11:01:00 - 11:01:59 1 minutes window 50MB 250Mb 250Mb

21. 43% utilization 28% utilization 11:00:00 - 11:00:59 11:01:00 - 11:01:59

22.

23.

24. Key takeaways • Serverless is the next revolution • Serverless has a built in scalability feature + shorter time to market • Cost is linear to computation power • Incentive to optimize. optimize=costs saving • Cost formula is not straightforward -> Find right memory setting with tool • Costs can get out of control -> add alerts • Hybrid solution = scalability + low costs • Rest of the world don’t use serverless so we need to avoid flooding them • Find the right bottleneck and solve it

25. Questions?

Scheduling big data workloads on serverless infrastructure

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Scheduling big data workloads on serverless infrastructure

Ähnlich wie Scheduling big data workloads on serverless infrastructure (20)

Mehr von Itai Yaffe

Mehr von Itai Yaffe (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Scheduling big data workloads on serverless infrastructure