AWS User Group Padova event.
Which AWS services there are for forecasting scenarious ?
https://www.meetup.com/it-IT/aws-user-group-padova/events/298480058/
Saurabh_Shanbhag - Building_SaaS_on_AWS.pptxAWS Chicago
This document discusses building enterprise SaaS applications on AWS. It notes that the SaaS market size is projected to grow significantly by 2029. When building SaaS applications on AWS, developers should consider the different models for tenant isolation and leveraging shared services. The document also provides recommendations around security, compliance, connectivity options, and best practices to accelerate time to market and growth when building SaaS applications on AWS.
This document discusses forecasting in AWS and summarizes key points:
1. It discusses the importance of forecasting for retail demand planning, supply chain planning, resource planning, and operational planning.
2. It outlines the different types of data that can be used for forecasting, including target time series data, related time series data, categorical data, and geolocation data.
3. It reviews AWS services for forecasting, including Amazon SageMaker Canvas, Amazon Forecast, Amazon SageMaker JumpStart, and Amazon SageMaker built-in algorithms.
The document discusses a proposed solution for Octank Water Solutions to move their systems to AWS. It recaps Octank's business challenges around equipment failures reducing costs and improving maintenance. It then outlines a proposed architecture using AWS IoT, SageMaker, and other services to build a failure prediction system. Finally, it discusses costs, savings, and next steps to implement a proof of concept and further digital transformation initiatives.
Supply Chain Data Lake - Kartik Medha AWS Chicago.pptxAWS Chicago
This document discusses using AWS services to create a supply chain data hub. It proposes using AWS S3 to store supply chain planning, execution, and visibility data from various sources. It then suggests using AWS Athena, Glue, and Lake Formation to analyze the data. AWS Step Functions and SageMaker would be used to build models to predict shipment ETAs. This data hub would provide a single source of federated supply chain data to gain insights and improve planning, tracking, and visibility across an organization's supply chain operations.
This document discusses the AWS Analytics Automation Toolkit (AAA), which allows users to quickly and easily provision, migrate, and test Amazon Redshift and other AWS analytics services. The AAA uses infrastructure as code and integrates services like Redshift, DMS, and JMeter automatically based on best practices. It provides a flexible way to deploy only needed components via a configuration file. A demo of the AAA's abilities to perform customized load testing and test Redshift features is also mentioned.
Pandas on AWS - Let me count the ways.pdfChris Fregly
Chris Fregly (Principal Solution Architect, AI and machine learning at AWS) will give a brief presentation on the various ways to perform scalable Pandas, Modin, and Ray on AWS. He will then answer questions from the audience and moderator, Alejandro Herrera (whatever he is) at Ponder.
Chris Fregly is a Principal Solution Architect for AI and Machine Learning at Amazon Web Services (AWS) based in San Francisco, California. He is the organizer of the Global Data Science on AWS meetup. He is co-author of the O'Reilly Book, "Data Science on AWS."
Related Links
O'Reilly Book: https://www.amazon.com/dp/1492079391/
Website: https://datascienceonaws.com
Meetup: https://meetup.datascienceonaws.com
GitHub Repo: https://github.com/data-science-on-aws/
YouTube: https://youtube.datascienceonaws.com
Slideshare: https://slideshare.datascienceonaws.com
APAC Principal Solutions Architect, Johnathon Meichtry will run through the highlights of 2015 showcasing the biggest announcements and how customers are using these new features. This session will cover the entire breadth of the AWS platform, and is a chance to get a high level overview of all of the announcements, feature updates and new services that AWS has launched in 2015.
AWS RDS Data API and CloudTrail. Who drop the table_.pdfVladimir Samoylov
- Utilize AWS RDS Data API for secure database access and operations
- CloudTrail for auditing and activity monitoring
- Investigating incidents and preventing unauthorized access
- PostgreSQL Auditing (pgAudit) extension
Saurabh_Shanbhag - Building_SaaS_on_AWS.pptxAWS Chicago
This document discusses building enterprise SaaS applications on AWS. It notes that the SaaS market size is projected to grow significantly by 2029. When building SaaS applications on AWS, developers should consider the different models for tenant isolation and leveraging shared services. The document also provides recommendations around security, compliance, connectivity options, and best practices to accelerate time to market and growth when building SaaS applications on AWS.
This document discusses forecasting in AWS and summarizes key points:
1. It discusses the importance of forecasting for retail demand planning, supply chain planning, resource planning, and operational planning.
2. It outlines the different types of data that can be used for forecasting, including target time series data, related time series data, categorical data, and geolocation data.
3. It reviews AWS services for forecasting, including Amazon SageMaker Canvas, Amazon Forecast, Amazon SageMaker JumpStart, and Amazon SageMaker built-in algorithms.
The document discusses a proposed solution for Octank Water Solutions to move their systems to AWS. It recaps Octank's business challenges around equipment failures reducing costs and improving maintenance. It then outlines a proposed architecture using AWS IoT, SageMaker, and other services to build a failure prediction system. Finally, it discusses costs, savings, and next steps to implement a proof of concept and further digital transformation initiatives.
Supply Chain Data Lake - Kartik Medha AWS Chicago.pptxAWS Chicago
This document discusses using AWS services to create a supply chain data hub. It proposes using AWS S3 to store supply chain planning, execution, and visibility data from various sources. It then suggests using AWS Athena, Glue, and Lake Formation to analyze the data. AWS Step Functions and SageMaker would be used to build models to predict shipment ETAs. This data hub would provide a single source of federated supply chain data to gain insights and improve planning, tracking, and visibility across an organization's supply chain operations.
This document discusses the AWS Analytics Automation Toolkit (AAA), which allows users to quickly and easily provision, migrate, and test Amazon Redshift and other AWS analytics services. The AAA uses infrastructure as code and integrates services like Redshift, DMS, and JMeter automatically based on best practices. It provides a flexible way to deploy only needed components via a configuration file. A demo of the AAA's abilities to perform customized load testing and test Redshift features is also mentioned.
Pandas on AWS - Let me count the ways.pdfChris Fregly
Chris Fregly (Principal Solution Architect, AI and machine learning at AWS) will give a brief presentation on the various ways to perform scalable Pandas, Modin, and Ray on AWS. He will then answer questions from the audience and moderator, Alejandro Herrera (whatever he is) at Ponder.
Chris Fregly is a Principal Solution Architect for AI and Machine Learning at Amazon Web Services (AWS) based in San Francisco, California. He is the organizer of the Global Data Science on AWS meetup. He is co-author of the O'Reilly Book, "Data Science on AWS."
Related Links
O'Reilly Book: https://www.amazon.com/dp/1492079391/
Website: https://datascienceonaws.com
Meetup: https://meetup.datascienceonaws.com
GitHub Repo: https://github.com/data-science-on-aws/
YouTube: https://youtube.datascienceonaws.com
Slideshare: https://slideshare.datascienceonaws.com
APAC Principal Solutions Architect, Johnathon Meichtry will run through the highlights of 2015 showcasing the biggest announcements and how customers are using these new features. This session will cover the entire breadth of the AWS platform, and is a chance to get a high level overview of all of the announcements, feature updates and new services that AWS has launched in 2015.
AWS RDS Data API and CloudTrail. Who drop the table_.pdfVladimir Samoylov
- Utilize AWS RDS Data API for secure database access and operations
- CloudTrail for auditing and activity monitoring
- Investigating incidents and preventing unauthorized access
- PostgreSQL Auditing (pgAudit) extension
The document outlines an agenda for a Getting Started on AWS event, including sessions on AWS history, infrastructure, security, databases, elasticity and management tools, with breaks for asking AWS experts questions, and thanking the sponsor of the event. The agenda runs from 8:00 am to 4:30 pm and covers topics like AWS services, architecture, and best practices through presentations and hands-on learning opportunities.
AWS Data Pipeline Tutorial | AWS Tutorial For Beginners | AWS Certification T...Edureka!
This document discusses AWS Data Pipeline, which is a service that helps reliably process and move data between AWS services and on-premises sources at specified intervals. It describes how AWS Data Pipeline can be used to build a data pipeline that collects data from different sources, performs analytics using EMR, and generates weekly reports. The key components of AWS Data Pipeline include the pipeline definition, pipeline, task runner, schedules, and pipeline components like instances and attempts. It also provides an example of using AWS Data Pipeline to import and export data from DynamoDB.
A review of the highlights of AWS re:Invent 2017. There was simply too many announcement to include them all. This is my cheat sheet to the top ones for me (@marknca)
- AWS was asked to attend technical due diligence engagements for two companies to evaluate cost optimization and migration opportunities.
- For the first company, AWS projected $100K per month in cost savings from optimization and 11% lower costs from migrating applications to containers on AWS.
- For the second company, AWS estimated a 39% cost savings over 3 years from migrating applications to AWS, with average annual savings of $1.6M.
- After both deals closed, AWS collaborated on plans to realize identified savings and growth opportunities within 100 days.
In this presentation from the recent AWS Oil & Gas event in Aberdeen we introduce the AWS cloud, its benefits and some of the organisations that are using AWS today.
We also cover some specific use-case and case-studies in the oil and gas sector.
The document outlines an agenda for a Getting Started on AWS workshop. It includes:
- A registration period from 8:00-9:00am followed by a welcome session.
- Technical sessions on AWS history, infrastructure, security, databases, and tools from 9:15am-4:00pm with breaks scheduled throughout for asking experts questions.
- Closing remarks from 3:55-4:00pm and a final question period from 4:00-4:30pm. Experts will be available during all breaks and the lunch period to answer questions.
The document provides details on the agenda timing, topics to be covered, and opportunities to engage with AWS experts throughout the event
This document outlines an agenda for a webinar on building secure, event-driven microservices with Confluent Cloud on AWS. The agenda includes presentations on building modern streaming analytics with Confluent on AWS, event streaming made easy with Confluent, and a lab on building end-to-end streaming data pipelines with Confluent Cloud. The hosts for the webinar are Ahmed Zamzam from Confluent and Nuno Barreto from AWS.
Build real-time streaming data pipelines to AWS with Confluentconfluent
Traditional data pipelines often face scalability issues and challenges related to cost, their monolithic design, and reliance on batch data processing. They also typically operate under the premise that all data needs to be stored in a single centralized data source before it's put to practical use. Confluent Cloud on Amazon Web Services (AWS) provides a fully managed cloud-native platform that helps you simplify the way you build real-time data flows using streaming data pipelines and Apache Kafka.
Customers who run SAP on AWS have lowered costs, improved performance, resilience, security, and agility. Application modernization can start with SAP at the core – but it can also start with machine learning, internet of things, big data and analytics. In this session, AWS is presenting and demonstrating use cases for modernizing IT systems that incorporates SAP. Customer Larsen & Toubro Infotech (LTI) shares their innovation agenda and journey to the cloud with AWS.
Harpreet Singh, SAP Solution Architect, Amazon Web Services
Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora Datavail
Learn and leverage database migration best practices from moving off commercial Oracle databases to Amazon RDS or Aurora. We’ll cover common pitfalls, issues, the biggest differences between the engines, migration best practices, and how some of our customers have completed these migrations.
Now that you have assembled the delivery team, it's time to gain insights from the methodology and the various tools that AWS uses to help customers migrate their Data Centres to AWS. This session highlights some of the key native AWS tools and services that organisations are using to migrate their DCs into the Cloud.
Speaker: Shane Baldacchino, Solutions Architect, Amazon Web Services
The document discusses Amazon Web Services (AWS) and its benefits for the oil and gas industry. It provides an overview of AWS, including its global infrastructure, rapid pace of innovation with new services, and use by enterprises. The document outlines key benefits of AWS like agility, cost savings, and security. It discusses AWS platforms and services across compute, storage, database, analytics and more. Finally, it discusses how the oil and gas industry can leverage AWS to boost operations through applications like connected digital oil fields, big data analytics, and 3D seismic visualization.
Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...Amazon Web Services
Migrating Microsoft applications to AWS can be a time-consuming process. Datapipe is a Managed Service Provider (MSP) with expertise in both AWS and Microsoft applications, offering streamlined solutions to smoothly migrate Microsoft workloads and applications to AWS.
Join us to explore how Datapipe helped FTI Consulting leverage the elastic scalability of the cloud by migrating and managing over 100TB of Microsoft workload-based data on AWS. Additionally, learn how Datapipe Managed Services for AWS can help you focus developer time on creating new solutions, not maintaining workloads.
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformWSO2
At its core, the challenge of managing Human Resources data is an integration challenge: estimates range from 2-3 HR systems in use at a typical SMB, up to a few dozen systems implemented amongst enterprise HR departments, and these systems seldom integrate seamlessly between themselves. Providing a multi-tenant, cloud-native solution to integrate these hundreds of HR-related systems, normalize their disparate data models and then render that consolidated information for stakeholder decision making has been a substantial undertaking, but one significantly eased by leveraging Ballerina. In this session, we’ll cover:
The overall software architecture for VHR’s Cloud Data Platform
Critical decision points leading to adoption of Ballerina for the CDP
Ballerina’s role in multiple evolutionary steps to the current architecture
Roadmap for the CDP architecture and plans for Ballerina
WSO2’s partnership in bringing continual success for the CD
Building Modern Streaming Analytics with Confluent on AWSconfluent
This document discusses building modern streaming analytics architectures with Confluent on AWS. It outlines challenges with traditional analytics approaches, and how a modern data strategy using real-time streaming analytics addresses those challenges. It describes how to build seamless streaming architectures using services like Amazon Kinesis and Confluent Kafka, integrated through AWS and Confluent. Examples of real-time analytics use cases are also provided.
AWS re:Invent 2016: State of the Union: Containers (CON316)Amazon Web Services
Join us to learn about the latest developments from Amazon ECS and the container ecosystem. Deepak Singh, General Manager of AWS Container Services, discusses the evolution of containers on AWS and shares our vision for continued innovation in this space. You also hear about how other companies are using the AWS container platform to innovate and build new businesses.
Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018Amazon Web Services
This document discusses how AWS can help customers optimize their Microsoft investments and realize value. It provides an overview of why customers choose AWS for their Microsoft workloads and how AWS supports a wide range of Windows and .NET workloads. The document also discusses strategies for migrating Microsoft workloads to AWS, optimizing Microsoft licensing costs, and leveraging tools and partners to accelerate migration.
MLOps vs LLMOps (by workflows and use cases) - 2024-05-21Alessandra Bilardi
MLOps @ localhost 2024
A pragmatic approach to manage ML systems by workflows and use cases.
https://www.grusp.org/conferenze_/mlops-127-0-0-1-21-maggio-2024/
The document outlines an agenda for a Getting Started on AWS event, including sessions on AWS history, infrastructure, security, databases, elasticity and management tools, with breaks for asking AWS experts questions, and thanking the sponsor of the event. The agenda runs from 8:00 am to 4:30 pm and covers topics like AWS services, architecture, and best practices through presentations and hands-on learning opportunities.
AWS Data Pipeline Tutorial | AWS Tutorial For Beginners | AWS Certification T...Edureka!
This document discusses AWS Data Pipeline, which is a service that helps reliably process and move data between AWS services and on-premises sources at specified intervals. It describes how AWS Data Pipeline can be used to build a data pipeline that collects data from different sources, performs analytics using EMR, and generates weekly reports. The key components of AWS Data Pipeline include the pipeline definition, pipeline, task runner, schedules, and pipeline components like instances and attempts. It also provides an example of using AWS Data Pipeline to import and export data from DynamoDB.
A review of the highlights of AWS re:Invent 2017. There was simply too many announcement to include them all. This is my cheat sheet to the top ones for me (@marknca)
- AWS was asked to attend technical due diligence engagements for two companies to evaluate cost optimization and migration opportunities.
- For the first company, AWS projected $100K per month in cost savings from optimization and 11% lower costs from migrating applications to containers on AWS.
- For the second company, AWS estimated a 39% cost savings over 3 years from migrating applications to AWS, with average annual savings of $1.6M.
- After both deals closed, AWS collaborated on plans to realize identified savings and growth opportunities within 100 days.
In this presentation from the recent AWS Oil & Gas event in Aberdeen we introduce the AWS cloud, its benefits and some of the organisations that are using AWS today.
We also cover some specific use-case and case-studies in the oil and gas sector.
The document outlines an agenda for a Getting Started on AWS workshop. It includes:
- A registration period from 8:00-9:00am followed by a welcome session.
- Technical sessions on AWS history, infrastructure, security, databases, and tools from 9:15am-4:00pm with breaks scheduled throughout for asking experts questions.
- Closing remarks from 3:55-4:00pm and a final question period from 4:00-4:30pm. Experts will be available during all breaks and the lunch period to answer questions.
The document provides details on the agenda timing, topics to be covered, and opportunities to engage with AWS experts throughout the event
This document outlines an agenda for a webinar on building secure, event-driven microservices with Confluent Cloud on AWS. The agenda includes presentations on building modern streaming analytics with Confluent on AWS, event streaming made easy with Confluent, and a lab on building end-to-end streaming data pipelines with Confluent Cloud. The hosts for the webinar are Ahmed Zamzam from Confluent and Nuno Barreto from AWS.
Build real-time streaming data pipelines to AWS with Confluentconfluent
Traditional data pipelines often face scalability issues and challenges related to cost, their monolithic design, and reliance on batch data processing. They also typically operate under the premise that all data needs to be stored in a single centralized data source before it's put to practical use. Confluent Cloud on Amazon Web Services (AWS) provides a fully managed cloud-native platform that helps you simplify the way you build real-time data flows using streaming data pipelines and Apache Kafka.
Customers who run SAP on AWS have lowered costs, improved performance, resilience, security, and agility. Application modernization can start with SAP at the core – but it can also start with machine learning, internet of things, big data and analytics. In this session, AWS is presenting and demonstrating use cases for modernizing IT systems that incorporates SAP. Customer Larsen & Toubro Infotech (LTI) shares their innovation agenda and journey to the cloud with AWS.
Harpreet Singh, SAP Solution Architect, Amazon Web Services
Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora Datavail
Learn and leverage database migration best practices from moving off commercial Oracle databases to Amazon RDS or Aurora. We’ll cover common pitfalls, issues, the biggest differences between the engines, migration best practices, and how some of our customers have completed these migrations.
Now that you have assembled the delivery team, it's time to gain insights from the methodology and the various tools that AWS uses to help customers migrate their Data Centres to AWS. This session highlights some of the key native AWS tools and services that organisations are using to migrate their DCs into the Cloud.
Speaker: Shane Baldacchino, Solutions Architect, Amazon Web Services
The document discusses Amazon Web Services (AWS) and its benefits for the oil and gas industry. It provides an overview of AWS, including its global infrastructure, rapid pace of innovation with new services, and use by enterprises. The document outlines key benefits of AWS like agility, cost savings, and security. It discusses AWS platforms and services across compute, storage, database, analytics and more. Finally, it discusses how the oil and gas industry can leverage AWS to boost operations through applications like connected digital oil fields, big data analytics, and 3D seismic visualization.
Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...Amazon Web Services
Migrating Microsoft applications to AWS can be a time-consuming process. Datapipe is a Managed Service Provider (MSP) with expertise in both AWS and Microsoft applications, offering streamlined solutions to smoothly migrate Microsoft workloads and applications to AWS.
Join us to explore how Datapipe helped FTI Consulting leverage the elastic scalability of the cloud by migrating and managing over 100TB of Microsoft workload-based data on AWS. Additionally, learn how Datapipe Managed Services for AWS can help you focus developer time on creating new solutions, not maintaining workloads.
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformWSO2
At its core, the challenge of managing Human Resources data is an integration challenge: estimates range from 2-3 HR systems in use at a typical SMB, up to a few dozen systems implemented amongst enterprise HR departments, and these systems seldom integrate seamlessly between themselves. Providing a multi-tenant, cloud-native solution to integrate these hundreds of HR-related systems, normalize their disparate data models and then render that consolidated information for stakeholder decision making has been a substantial undertaking, but one significantly eased by leveraging Ballerina. In this session, we’ll cover:
The overall software architecture for VHR’s Cloud Data Platform
Critical decision points leading to adoption of Ballerina for the CDP
Ballerina’s role in multiple evolutionary steps to the current architecture
Roadmap for the CDP architecture and plans for Ballerina
WSO2’s partnership in bringing continual success for the CD
Building Modern Streaming Analytics with Confluent on AWSconfluent
This document discusses building modern streaming analytics architectures with Confluent on AWS. It outlines challenges with traditional analytics approaches, and how a modern data strategy using real-time streaming analytics addresses those challenges. It describes how to build seamless streaming architectures using services like Amazon Kinesis and Confluent Kafka, integrated through AWS and Confluent. Examples of real-time analytics use cases are also provided.
AWS re:Invent 2016: State of the Union: Containers (CON316)Amazon Web Services
Join us to learn about the latest developments from Amazon ECS and the container ecosystem. Deepak Singh, General Manager of AWS Container Services, discusses the evolution of containers on AWS and shares our vision for continued innovation in this space. You also hear about how other companies are using the AWS container platform to innovate and build new businesses.
Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018Amazon Web Services
This document discusses how AWS can help customers optimize their Microsoft investments and realize value. It provides an overview of why customers choose AWS for their Microsoft workloads and how AWS supports a wide range of Windows and .NET workloads. The document also discusses strategies for migrating Microsoft workloads to AWS, optimizing Microsoft licensing costs, and leveraging tools and partners to accelerate migration.
MLOps vs LLMOps (by workflows and use cases) - 2024-05-21Alessandra Bilardi
MLOps @ localhost 2024
A pragmatic approach to manage ML systems by workflows and use cases.
https://www.grusp.org/conferenze_/mlops-127-0-0-1-21-maggio-2024/
How to move your ML system from local to production - 2024-03-15Alessandra Bilardi
Incontro DevOps Italia 2024
When a Cloud Engineer has to do a review the code of its colleague Data Scientist for production environment, it is always important to understand where it is best to put the focus. Often, the best approach is to promote the resources awareness to be used and to find a framework to split the work together.
https://2024.incontrodevops.it/talks_speakers/
This document summarizes a PyData Venice meetup on February 29, 2024. The meetup will be held in-person and streamed live at 7:00 PM featuring talks from Fabio Dal Forno on Kaggle competitions and Alessandra Bilardi on an overview of the Kaggle platform. Kaggle is a platform for machine learning competitions and projects with over 17 million users worldwide and datasets, models, and discussions. The document outlines Kaggle's history and growth, resources available on the platform, and how it can be useful for learning feature engineering and algorithm tuning through real projects.
From your laptop to all resource that you need - 2023-12-09Alessandra Bilardi
PyData Impact Scholars - PyData Global 2023
Imagine you are processing your data and your ML system from your laptop and there are not enough resources, but by adding a few lines of code you can access all the resources you need. So, from your Jupyter notebook you can orchestrate tests of your code and then you can run the same code in the cloud with … a flag and little else.
https://github.com/bilardi/pydata-global/tree/master/2023
PyDataVE #13
Which open source libraries can compete with Pandas, PySpark with some activities as apply, groupby and sum ?
https://www.meetup.com/pydata-venice/events/296507635/
This document provides an introduction to key concepts in artificial intelligence including object detection using Python. It discusses input datasets, model training, and output predictions. As an example, it describes how object detection works by taking an image as input, training a model, and outputting a prediction of the classified object and its position in the image. It also references providing a demonstration of basic Python and Coolab for object detection.
This document discusses data transformation in AWS. It covers the importance of data transformation in terms of quantity, quality, and noise/compatibility. It then describes common transformation methods like extraction, parsing, cleaning, and enrichment. Several AWS services for data transformation are presented, including AWS Glue, AWS DataBrew, AWS Data Pipeline, Amazon SageMaker Data Wrangler, and notebooks. These services are compared based on difficulty, execution times, and costs.
Automation: from local test to production deploy - 2020-11-05Alessandra Bilardi
CloudConf 2020 in Streaming
Talk about a sample of Automation Solution from local to production
https://2020.cloudconf.it/
https://github.com/bilardi/aws-saving/
https://github.com/bilardi/aws-simple-pipeline/
This document provides an overview of solving a Rubik's cube, including its history, features, notation, and algorithms. It describes how Ernő Rubik invented the Rubik's cube in 1974 and discusses its notation system using letters and prime symbols to denote clockwise and counterclockwise turns of different sides. The document outlines the levels approach to solving a Rubik's cube and includes some edge and corner algorithms like the sexy move to solve each level.
AWS Summit 2018 Milano
Our part of AWS presentation: Managed Relational Databases
https://www.slideshare.net/AmazonWebServices/database-relazionali-gestiti
End-to-end pipeline agility - Berlin Buzzwords 2024Lars Albertsson
We describe how we achieve high change agility in data engineering by eliminating the fear of breaking downstream data pipelines through end-to-end pipeline testing, and by using schema metaprogramming to safely eliminate boilerplate involved in changes that affect whole pipelines.
A quick poll on agility in changing pipelines from end to end indicated a huge span in capabilities. For the question "How long time does it take for all downstream pipelines to be adapted to an upstream change," the median response was 6 months, but some respondents could do it in less than a day. When quantitative data engineering differences between the best and worst are measured, the span is often 100x-1000x, sometimes even more.
A long time ago, we suffered at Spotify from fear of changing pipelines due to not knowing what the impact might be downstream. We made plans for a technical solution to test pipelines end-to-end to mitigate that fear, but the effort failed for cultural reasons. We eventually solved this challenge, but in a different context. In this presentation we will describe how we test full pipelines effectively by manipulating workflow orchestration, which enables us to make changes in pipelines without fear of breaking downstream.
Making schema changes that affect many jobs also involves a lot of toil and boilerplate. Using schema-on-read mitigates some of it, but has drawbacks since it makes it more difficult to detect errors early. We will describe how we have rejected this tradeoff by applying schema metaprogramming, eliminating boilerplate but keeping the protection of static typing, thereby further improving agility to quickly modify data pipelines without fear.
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeWalaa Eldin Moustafa
Dynamic policy enforcement is becoming an increasingly important topic in today’s world where data privacy and compliance is a top priority for companies, individuals, and regulators alike. In these slides, we discuss how LinkedIn implements a powerful dynamic policy enforcement engine, called ViewShift, and integrates it within its data lake. We show the query engine architecture and how catalog implementations can automatically route table resolutions to compliance-enforcing SQL views. Such views have a set of very interesting properties: (1) They are auto-generated from declarative data annotations. (2) They respect user-level consent and preferences (3) They are context-aware, encoding a different set of transformations for different use cases (4) They are portable; while the SQL logic is only implemented in one SQL dialect, it is accessible in all engines.
#SQL #Views #Privacy #Compliance #DataLake
Learn SQL from basic queries to Advance queriesmanishkhaire30
Dive into the world of data analysis with our comprehensive guide on mastering SQL! This presentation offers a practical approach to learning SQL, focusing on real-world applications and hands-on practice. Whether you're a beginner or looking to sharpen your skills, this guide provides the tools you need to extract, analyze, and interpret data effectively.
Key Highlights:
Foundations of SQL: Understand the basics of SQL, including data retrieval, filtering, and aggregation.
Advanced Queries: Learn to craft complex queries to uncover deep insights from your data.
Data Trends and Patterns: Discover how to identify and interpret trends and patterns in your datasets.
Practical Examples: Follow step-by-step examples to apply SQL techniques in real-world scenarios.
Actionable Insights: Gain the skills to derive actionable insights that drive informed decision-making.
Join us on this journey to enhance your data analysis capabilities and unlock the full potential of SQL. Perfect for data enthusiasts, analysts, and anyone eager to harness the power of data!
#DataAnalysis #SQL #LearningSQL #DataInsights #DataScience #Analytics
Build applications with generative AI on Google CloudMárton Kodok
We will explore Vertex AI - Model Garden powered experiences, we are going to learn more about the integration of these generative AI APIs. We are going to see in action what the Gemini family of generative models are for developers to build and deploy AI-driven applications. Vertex AI includes a suite of foundation models, these are referred to as the PaLM and Gemini family of generative ai models, and they come in different versions. We are going to cover how to use via API to: - execute prompts in text and chat - cover multimodal use cases with image prompts. - finetune and distill to improve knowledge domains - run function calls with foundation models to optimize them for specific tasks. At the end of the session, developers will understand how to innovate with generative AI and develop apps using the generative ai industry trends.
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...Social Samosa
The Modern Marketing Reckoner (MMR) is a comprehensive resource packed with POVs from 60+ industry leaders on how AI is transforming the 4 key pillars of marketing – product, place, price and promotions.
Analysis insight about a Flyball dog competition team's performanceroli9797
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
5. Forecasting in AWS - AWS User Group Padova - 2024-02-01
What is the forecasting ?
6. Forecasting in AWS - AWS User Group Padova - 2024-02-01
The importance of the forecasting
1
2 Supply chain planning
3
Resource planning
4 Operational planning
Retail demand planning
7. Forecasting in AWS - AWS User Group Padova - 2024-02-01
Time Series data
● Target Time Series (TTS)
○ timestamp
○ item_id (articolo, SKU)
○ [ dimensione per cliente ]
○ target_value (valore)
○ [ Festività, Meteo ]
● Related Time Series (RTS)
○ timestamp
○ item_id (articolo, SKU)
○ [ dimensione per cliente ]
○ holiday (valore 0 oppure 1)
○ maximum [ Meteo, Stock, .. ]
8. Forecasting in AWS - AWS User Group Padova - 2024-02-01
Categorical data
● Anagrafica articolo
○ item_id (articolo, SKU)
○ category (caratteri o numeri)
○ [ fino a 10 colonne totali ]
● Geolocalizzazione
○ distribuzione per
■ articolo
■ articolo / cliente
○ da cosa si può partire
■ CAP
■ Nazione [ Festività ]
■ longitudine [ Meteo ]
■ latitudine [ Meteo ]
Distribution
Nominal or Ordinal Categorical Variables
9. Forecasting in AWS - AWS User Group Padova - 2024-02-01
Which data can we use to do forecasting ?
1
2 related timeseries (RTS)
3
categorical data (IM)
4 geolocalization (GEO)
target timeseries (TTS)
10. Forecasting in AWS - AWS User Group Padova - 2024-02-01
Example ● target timeseries (TTS)
● related timeseries (RTS)
● categorical data (IM)
● geolocalization (GEO)
reference
11. Forecasting in AWS - AWS User Group Padova - 2024-02-01
Example ● target timeseries (TTS)
● related timeseries (RTS)
● categorical data (IM)
● geolocalization (GEO)
reference
12. Forecasting in AWS - AWS User Group Padova - 2024-02-01
Example ● target timeseries (TTS)
● related timeseries (RTS)
● categorical data (IM)
● geolocalization (GEO)
reference
13. Forecasting in AWS - AWS User Group Padova - 2024-02-01
Example ● target timeseries (TTS)
● related timeseries (RTS)
● categorical data (IM)
● geolocalization (GEO)
reference
28. Forecasting in AWS - AWS User Group Padova - 2024-02-01
Evaluation metrics
Metrics Description Notes Limits
MAPE Mean Absolute Percentage Error ok < 50%; good <
20%; very good < 10%
for dataset without 0
MASE Mean Absolute Scaled Error training / testing values
comparison
it is unique for each
dataset
RMSE Root Mean Square Error RMSE / actual
average value < 10%
it is most sensitive to
outliers
30. Forecasting in AWS - AWS User Group Padova - 2024-02-01
Comparison of AWS services for the forecasting
Services Difficulty Execution times Costs RMSE
SM Canvas 🏖🏖🏖 10 mins $ 2.02 280
Forecast 🏖📚🔧 20 mins + 48 mins $ 11.28 57
SM JumpStart 🤓🔧🔧 5 mins + 28 mins $ 0.41 265
SM Built-in algos 🤓🔧🔧 3 mins + 10 mins $ 0.22 251