SlideShare ist ein Scribd-Unternehmen logo
1 von 23
Downloaden Sie, um offline zu lesen
Data Science
the art of foul play

Sergey Shelpuk
SoftServe, Inc.
sshel@softserveinc.com
September, 2013
“Your goal should not be to buy players, it
should be to buy wins. In order to buy
wins you should buy runs” (c)
More data is
available for
companies

Storage
technologies
allow storing
and operating it

Advanced
analytics could
be applied to
this new data to
achieve
competitive
advantage
Data Scientist: The Sexiest
Job of the 21st Century
For Today’s Graduate,
Just One Word: Statistics

Hal Varian
chief economist at Google

“I keep saying that the sexy job in the
next 10 years will be statisticians,
and I’m not kidding.”

Data Scientist: The Hottest Job
You Haven't Heard Of
Top 10 Strategic Technology Trends for 2013
• Mobile Device Battles
• Mobile Applications and
HTML5
• Personal Cloud
• Enterprise App Stores
• The Internet of Things

• Hybrid IT and Cloud
Computing
• Strategic Big Data
• Actionable Analytics
• In Memory Computing
• Integrated Ecosystems
McKinsey Global Institute projects
approximately 140,000 to 190,000 unfilled
positions of data analytics experts in the U.S. by
2018 and a shortage of 1.5 million managers
and analysts who have the ability to understand
and make decisions using big data.
Business
Tasks

• Define prospective
customers
• Define traffic jams in
the city
• Recommend
restaurants and
menus
• Adjust UI to the
particular user
• Classify body part on
X-Ray image

• Define market
niche
• Define
influencers in
the social
networks
• Define similar
customers or
projects in
portfolio
• Define informal
groups in the
organization

• Define fraud bank
transaction
• Define network
intrusion attempts
• Provide automatic
aircraft engine
testing
• Provide automatic
IT infrastructure
monitoring
• Provide clinical test
analysis

• Define the best
price for the goods
or services to
maximize profits
• Define best
working schedule
for the store
• Define best
amount of
production
• Define best
business rules

Model Family

Classification

Clustering

Anomaly
Detection

Optimization

• Naïve Bayes
• Logistic regression
• Support Vector
Machines
• Neural Networks

• K-Means
• K nearest
neighbor
• Self-organized
maps
• Mixture of
Gaussians

Algorithms

• Mixture of
Gaussians
• Self-learning
anomaly detection

•
•
•
•
•

Gradient descent
Simplex method
Newton’s method
Normal equations
Genetic algorithms
Cross Industry Standard Process
for Data Mining
SoftServe Data Science Group
Knowledge Model
Business

Logic

Technology

Level

Level

Level

• Basics of Business
Analysis
• Basics of Economics
• Basics of Product
Management
• Basics of
Organizational
Behavior

• Statistics/Probability
• Machine Learning
• Data Mining
• Artificial Intelligence

• Matlab/Octave
•R
• SQL
• Parallel Computing
Deep Learning Neural Networks
Feature extractor

Task: recognize a motorcycle

Learning
algorithm
The concept of Autoencoder
The concept of Autoencoder

…
…

…

…
…

© Andrew Y. Ng
Large scale deep learning networks

See more: Building high-level features using large scale unsupervised learning
Deep learning neural networks
Pre-trained as Autoencoder

Typical classification
neural network
Few results
Images

Video

Text/NLP

© Andrew Y. Ng
Deep Learning in SoftServe
Phase 1 results
(old-fashion anomaly detection)

Phase 2 prototype
(deep learning approach)
Useful Resources
• Introduction to Statistics
• Introduction to Artificial Intelligence

• Machine Learning
• Probabilistic Graphical Models
• Statistics One
Thank you!

Weitere ähnliche Inhalte

Was ist angesagt?

Big Data Analytics and a Chartered Accountant
Big Data Analytics and a Chartered AccountantBig Data Analytics and a Chartered Accountant
Big Data Analytics and a Chartered AccountantBharath Rao
 
Walt Disney presentation at the Chief Analytics Officer Forum East Coast USA ...
Walt Disney presentation at the Chief Analytics Officer Forum East Coast USA ...Walt Disney presentation at the Chief Analytics Officer Forum East Coast USA ...
Walt Disney presentation at the Chief Analytics Officer Forum East Coast USA ...Chief Analytics Officer Forum
 
Haroldo Gali: Google
Haroldo Gali: GoogleHaroldo Gali: Google
Haroldo Gali: GoogleStrider
 
World of data @ praxis 2013 v2
World of data   @ praxis 2013  v2World of data   @ praxis 2013  v2
World of data @ praxis 2013 v2Prithwis Mukerjee
 
How to make business using AI and Big Data?
How to make business using AI and Big Data?How to make business using AI and Big Data?
How to make business using AI and Big Data?Ivano Digital
 
Make data simple in the cognitive era
Make data simple in the cognitive eraMake data simple in the cognitive era
Make data simple in the cognitive eraIBM Analytics
 
NUS-ISS Learning Day 2018- Start with Data Governance
NUS-ISS Learning Day 2018- Start with Data GovernanceNUS-ISS Learning Day 2018- Start with Data Governance
NUS-ISS Learning Day 2018- Start with Data GovernanceNUS-ISS
 
Using text analytics to manage mobile qual to manage mobile Qual Data - Civicom
Using text analytics to manage mobile qual to manage mobile Qual Data - CivicomUsing text analytics to manage mobile qual to manage mobile Qual Data - Civicom
Using text analytics to manage mobile qual to manage mobile Qual Data - CivicomMerlien Institute
 
Benchmarking Digital Readiness: Moving at the Speed of the Market
Benchmarking Digital Readiness: Moving at the Speed of the MarketBenchmarking Digital Readiness: Moving at the Speed of the Market
Benchmarking Digital Readiness: Moving at the Speed of the MarketApigee | Google Cloud
 
Big Data LDN 2018: DATA SCIENCE AT ING
Big Data LDN 2018: DATA SCIENCE AT INGBig Data LDN 2018: DATA SCIENCE AT ING
Big Data LDN 2018: DATA SCIENCE AT INGMatt Stubbs
 
AI Strategy & Advance Analytics
AI Strategy & Advance AnalyticsAI Strategy & Advance Analytics
AI Strategy & Advance Analyticssrosen18
 
Bde presentation dv
Bde presentation dvBde presentation dv
Bde presentation dvBigDataExpo
 
Top 10 Analytics Trends 2016
Top 10 Analytics Trends 2016Top 10 Analytics Trends 2016
Top 10 Analytics Trends 2016Niranjan Krishnan
 
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...Spark Summit
 
Big Data LDN 2018: SHAPING AN AI-DRIVEN FUTURE WITH AUGMENTED INTELLIGENCE FO...
Big Data LDN 2018: SHAPING AN AI-DRIVEN FUTURE WITH AUGMENTED INTELLIGENCE FO...Big Data LDN 2018: SHAPING AN AI-DRIVEN FUTURE WITH AUGMENTED INTELLIGENCE FO...
Big Data LDN 2018: SHAPING AN AI-DRIVEN FUTURE WITH AUGMENTED INTELLIGENCE FO...Matt Stubbs
 
Cisco niels vd berg
Cisco niels vd bergCisco niels vd berg
Cisco niels vd bergBigDataExpo
 
NUS-ISS Learning Day 2018- Business agility for business leaders
NUS-ISS Learning Day 2018- Business agility for business leadersNUS-ISS Learning Day 2018- Business agility for business leaders
NUS-ISS Learning Day 2018- Business agility for business leadersNUS-ISS
 
Every angle jacques adriaansen
Every angle   jacques adriaansenEvery angle   jacques adriaansen
Every angle jacques adriaansenBigDataExpo
 

Was ist angesagt? (20)

Big Data Analytics and a Chartered Accountant
Big Data Analytics and a Chartered AccountantBig Data Analytics and a Chartered Accountant
Big Data Analytics and a Chartered Accountant
 
Walt Disney presentation at the Chief Analytics Officer Forum East Coast USA ...
Walt Disney presentation at the Chief Analytics Officer Forum East Coast USA ...Walt Disney presentation at the Chief Analytics Officer Forum East Coast USA ...
Walt Disney presentation at the Chief Analytics Officer Forum East Coast USA ...
 
eBay at The Chief Analytics Officer Forum, Europe
eBay at The Chief Analytics Officer Forum, EuropeeBay at The Chief Analytics Officer Forum, Europe
eBay at The Chief Analytics Officer Forum, Europe
 
Haroldo Gali: Google
Haroldo Gali: GoogleHaroldo Gali: Google
Haroldo Gali: Google
 
World of data @ praxis 2013 v2
World of data   @ praxis 2013  v2World of data   @ praxis 2013  v2
World of data @ praxis 2013 v2
 
How to make business using AI and Big Data?
How to make business using AI and Big Data?How to make business using AI and Big Data?
How to make business using AI and Big Data?
 
Make data simple in the cognitive era
Make data simple in the cognitive eraMake data simple in the cognitive era
Make data simple in the cognitive era
 
NUS-ISS Learning Day 2018- Start with Data Governance
NUS-ISS Learning Day 2018- Start with Data GovernanceNUS-ISS Learning Day 2018- Start with Data Governance
NUS-ISS Learning Day 2018- Start with Data Governance
 
Using text analytics to manage mobile qual to manage mobile Qual Data - Civicom
Using text analytics to manage mobile qual to manage mobile Qual Data - CivicomUsing text analytics to manage mobile qual to manage mobile Qual Data - Civicom
Using text analytics to manage mobile qual to manage mobile Qual Data - Civicom
 
Benchmarking Digital Readiness: Moving at the Speed of the Market
Benchmarking Digital Readiness: Moving at the Speed of the MarketBenchmarking Digital Readiness: Moving at the Speed of the Market
Benchmarking Digital Readiness: Moving at the Speed of the Market
 
Big Data LDN 2018: DATA SCIENCE AT ING
Big Data LDN 2018: DATA SCIENCE AT INGBig Data LDN 2018: DATA SCIENCE AT ING
Big Data LDN 2018: DATA SCIENCE AT ING
 
AI Strategy & Advance Analytics
AI Strategy & Advance AnalyticsAI Strategy & Advance Analytics
AI Strategy & Advance Analytics
 
Bde presentation dv
Bde presentation dvBde presentation dv
Bde presentation dv
 
Top 10 Analytics Trends 2016
Top 10 Analytics Trends 2016Top 10 Analytics Trends 2016
Top 10 Analytics Trends 2016
 
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
 
Big Data LDN 2018: SHAPING AN AI-DRIVEN FUTURE WITH AUGMENTED INTELLIGENCE FO...
Big Data LDN 2018: SHAPING AN AI-DRIVEN FUTURE WITH AUGMENTED INTELLIGENCE FO...Big Data LDN 2018: SHAPING AN AI-DRIVEN FUTURE WITH AUGMENTED INTELLIGENCE FO...
Big Data LDN 2018: SHAPING AN AI-DRIVEN FUTURE WITH AUGMENTED INTELLIGENCE FO...
 
Cisco niels vd berg
Cisco niels vd bergCisco niels vd berg
Cisco niels vd berg
 
Careers in analytics
Careers in analyticsCareers in analytics
Careers in analytics
 
NUS-ISS Learning Day 2018- Business agility for business leaders
NUS-ISS Learning Day 2018- Business agility for business leadersNUS-ISS Learning Day 2018- Business agility for business leaders
NUS-ISS Learning Day 2018- Business agility for business leaders
 
Every angle jacques adriaansen
Every angle   jacques adriaansenEvery angle   jacques adriaansen
Every angle jacques adriaansen
 

Ähnlich wie Data Science: The Art of Foul Play by Serhiy Shelpuk

Deteo. Data science, Big Data expertise
Deteo. Data science, Big Data expertise Deteo. Data science, Big Data expertise
Deteo. Data science, Big Data expertise deteo
 
Building Predictive Analytics on Big Data Platforms
Building Predictive Analytics on Big Data PlatformsBuilding Predictive Analytics on Big Data Platforms
Building Predictive Analytics on Big Data PlatformsOlha Hrytsay
 
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...DATAVERSITY
 
A Journey Through The Far Side Of Data Science
A Journey Through The Far Side Of Data ScienceA Journey Through The Far Side Of Data Science
A Journey Through The Far Side Of Data Sciencetlcj97
 
Barga Galvanize Sept 2015
Barga Galvanize Sept 2015Barga Galvanize Sept 2015
Barga Galvanize Sept 2015Roger Barga
 
Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...Certus Solutions
 
In-Depth Data Analytics
In-Depth Data AnalyticsIn-Depth Data Analytics
In-Depth Data AnalyticsYASH GAIKWAD
 
Deliveinrg explainable AI
Deliveinrg explainable AIDeliveinrg explainable AI
Deliveinrg explainable AIGary Allemann
 
Big Data Meetup by Chad Richeson
Big Data Meetup by Chad RichesonBig Data Meetup by Chad Richeson
Big Data Meetup by Chad RichesonSocietyConsulting
 
Tools and techniques for predictive analytics
Tools and techniques for predictive analyticsTools and techniques for predictive analytics
Tools and techniques for predictive analyticsRohanKumarJumnani
 
Gse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedGse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedcedrinemadera
 
Modern Data Challenges require Modern Graph Technology
Modern Data Challenges require Modern Graph TechnologyModern Data Challenges require Modern Graph Technology
Modern Data Challenges require Modern Graph TechnologyNeo4j
 
Turning Big Data to Business Advantage
Turning Big Data to Business AdvantageTurning Big Data to Business Advantage
Turning Big Data to Business AdvantageTeradata Aster
 
ADV Slides: Data Curation for Artificial Intelligence Strategies
ADV Slides: Data Curation for Artificial Intelligence StrategiesADV Slides: Data Curation for Artificial Intelligence Strategies
ADV Slides: Data Curation for Artificial Intelligence StrategiesDATAVERSITY
 
How Startups can leverage big data?
How Startups can leverage big data?How Startups can leverage big data?
How Startups can leverage big data?Rackspace
 
influence of AI in IS
influence of AI in ISinfluence of AI in IS
influence of AI in ISISACA Riyadh
 
Riding and Capitalizing the Next Wave of Information Technology
Riding and Capitalizing the Next Wave of Information TechnologyRiding and Capitalizing the Next Wave of Information Technology
Riding and Capitalizing the Next Wave of Information TechnologyGoutama Bachtiar
 

Ähnlich wie Data Science: The Art of Foul Play by Serhiy Shelpuk (20)

Deteo. Data science, Big Data expertise
Deteo. Data science, Big Data expertise Deteo. Data science, Big Data expertise
Deteo. Data science, Big Data expertise
 
Building Predictive Analytics on Big Data Platforms
Building Predictive Analytics on Big Data PlatformsBuilding Predictive Analytics on Big Data Platforms
Building Predictive Analytics on Big Data Platforms
 
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
 
A Journey Through The Far Side Of Data Science
A Journey Through The Far Side Of Data ScienceA Journey Through The Far Side Of Data Science
A Journey Through The Far Side Of Data Science
 
Trends in data analytics
Trends in data analyticsTrends in data analytics
Trends in data analytics
 
Barga Galvanize Sept 2015
Barga Galvanize Sept 2015Barga Galvanize Sept 2015
Barga Galvanize Sept 2015
 
Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...
 
In-Depth Data Analytics
In-Depth Data AnalyticsIn-Depth Data Analytics
In-Depth Data Analytics
 
Deliveinrg explainable AI
Deliveinrg explainable AIDeliveinrg explainable AI
Deliveinrg explainable AI
 
Big Data Meetup by Chad Richeson
Big Data Meetup by Chad RichesonBig Data Meetup by Chad Richeson
Big Data Meetup by Chad Richeson
 
Tools and techniques for predictive analytics
Tools and techniques for predictive analyticsTools and techniques for predictive analytics
Tools and techniques for predictive analytics
 
Gse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedGse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-shared
 
Modern Data Challenges require Modern Graph Technology
Modern Data Challenges require Modern Graph TechnologyModern Data Challenges require Modern Graph Technology
Modern Data Challenges require Modern Graph Technology
 
Turning Big Data to Business Advantage
Turning Big Data to Business AdvantageTurning Big Data to Business Advantage
Turning Big Data to Business Advantage
 
ADV Slides: Data Curation for Artificial Intelligence Strategies
ADV Slides: Data Curation for Artificial Intelligence StrategiesADV Slides: Data Curation for Artificial Intelligence Strategies
ADV Slides: Data Curation for Artificial Intelligence Strategies
 
How Startups can leverage big data?
How Startups can leverage big data?How Startups can leverage big data?
How Startups can leverage big data?
 
influence of AI in IS
influence of AI in ISinfluence of AI in IS
influence of AI in IS
 
Analytics in Online Retail
Analytics in Online RetailAnalytics in Online Retail
Analytics in Online Retail
 
Riding and Capitalizing the Next Wave of Information Technology
Riding and Capitalizing the Next Wave of Information TechnologyRiding and Capitalizing the Next Wave of Information Technology
Riding and Capitalizing the Next Wave of Information Technology
 
AI in the Enterprise at Scale
AI in the Enterprise at ScaleAI in the Enterprise at Scale
AI in the Enterprise at Scale
 

Mehr von SoftServe

Approaching Quality in Digital Era
Approaching Quality in Digital EraApproaching Quality in Digital Era
Approaching Quality in Digital EraSoftServe
 
Digital Product Security
Digital Product SecurityDigital Product Security
Digital Product SecuritySoftServe
 
Testing Tools and Tips
Testing Tools and TipsTesting Tools and Tips
Testing Tools and TipsSoftServe
 
Android Mobile Application Testing: Human Interface Guideline, Tools
Android Mobile Application Testing: Human Interface Guideline, ToolsAndroid Mobile Application Testing: Human Interface Guideline, Tools
Android Mobile Application Testing: Human Interface Guideline, ToolsSoftServe
 
Android Mobile Application Testing: Specific Functional, Performance, Device ...
Android Mobile Application Testing: Specific Functional, Performance, Device ...Android Mobile Application Testing: Specific Functional, Performance, Device ...
Android Mobile Application Testing: Specific Functional, Performance, Device ...SoftServe
 
How to Reduce Time to Market Using Microsoft DevOps Solutions
How to Reduce Time to Market Using Microsoft DevOps SolutionsHow to Reduce Time to Market Using Microsoft DevOps Solutions
How to Reduce Time to Market Using Microsoft DevOps SolutionsSoftServe
 
Containerization: The DevOps Revolution
Containerization: The DevOps Revolution Containerization: The DevOps Revolution
Containerization: The DevOps Revolution SoftServe
 
Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist SoftServe
 
Rapid Prototyping for Big Data with AWS
Rapid Prototyping for Big Data with AWS Rapid Prototyping for Big Data with AWS
Rapid Prototyping for Big Data with AWS SoftServe
 
Implementing Test Automation: What a Manager Should Know
Implementing Test Automation: What a Manager Should KnowImplementing Test Automation: What a Manager Should Know
Implementing Test Automation: What a Manager Should KnowSoftServe
 
Using AWS Lambda for Infrastructure Automation and Beyond
Using AWS Lambda for Infrastructure Automation and BeyondUsing AWS Lambda for Infrastructure Automation and Beyond
Using AWS Lambda for Infrastructure Automation and BeyondSoftServe
 
Advanced Analytics and Data Science Expertise
Advanced Analytics and Data Science ExpertiseAdvanced Analytics and Data Science Expertise
Advanced Analytics and Data Science ExpertiseSoftServe
 
Agile Big Data Analytics Development: An Architecture-Centric Approach
Agile Big Data Analytics Development: An Architecture-Centric ApproachAgile Big Data Analytics Development: An Architecture-Centric Approach
Agile Big Data Analytics Development: An Architecture-Centric ApproachSoftServe
 
Big Data as a Service: A Neo-Metropolis Model Approach for Innovation
Big Data as a Service: A Neo-Metropolis Model Approach for InnovationBig Data as a Service: A Neo-Metropolis Model Approach for Innovation
Big Data as a Service: A Neo-Metropolis Model Approach for InnovationSoftServe
 
Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...
Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...
Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...SoftServe
 
Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...
Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...
Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...SoftServe
 
Managing Requirements with Word and TFS by Max Markov
Managing Requirements with Word and TFS by Max MarkovManaging Requirements with Word and TFS by Max Markov
Managing Requirements with Word and TFS by Max MarkovSoftServe
 
How to Implement Hybrid Cloud Solutions Successfully
How to Implement Hybrid Cloud Solutions SuccessfullyHow to Implement Hybrid Cloud Solutions Successfully
How to Implement Hybrid Cloud Solutions SuccessfullySoftServe
 
Designing Big Data Systems Like a Pro
Designing Big Data Systems Like a ProDesigning Big Data Systems Like a Pro
Designing Big Data Systems Like a ProSoftServe
 
Product Management in Outsourcing by Roman Kolodchak and Roman Pavlyuk
Product Management in Outsourcing by Roman Kolodchak and Roman PavlyukProduct Management in Outsourcing by Roman Kolodchak and Roman Pavlyuk
Product Management in Outsourcing by Roman Kolodchak and Roman PavlyukSoftServe
 

Mehr von SoftServe (20)

Approaching Quality in Digital Era
Approaching Quality in Digital EraApproaching Quality in Digital Era
Approaching Quality in Digital Era
 
Digital Product Security
Digital Product SecurityDigital Product Security
Digital Product Security
 
Testing Tools and Tips
Testing Tools and TipsTesting Tools and Tips
Testing Tools and Tips
 
Android Mobile Application Testing: Human Interface Guideline, Tools
Android Mobile Application Testing: Human Interface Guideline, ToolsAndroid Mobile Application Testing: Human Interface Guideline, Tools
Android Mobile Application Testing: Human Interface Guideline, Tools
 
Android Mobile Application Testing: Specific Functional, Performance, Device ...
Android Mobile Application Testing: Specific Functional, Performance, Device ...Android Mobile Application Testing: Specific Functional, Performance, Device ...
Android Mobile Application Testing: Specific Functional, Performance, Device ...
 
How to Reduce Time to Market Using Microsoft DevOps Solutions
How to Reduce Time to Market Using Microsoft DevOps SolutionsHow to Reduce Time to Market Using Microsoft DevOps Solutions
How to Reduce Time to Market Using Microsoft DevOps Solutions
 
Containerization: The DevOps Revolution
Containerization: The DevOps Revolution Containerization: The DevOps Revolution
Containerization: The DevOps Revolution
 
Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist
 
Rapid Prototyping for Big Data with AWS
Rapid Prototyping for Big Data with AWS Rapid Prototyping for Big Data with AWS
Rapid Prototyping for Big Data with AWS
 
Implementing Test Automation: What a Manager Should Know
Implementing Test Automation: What a Manager Should KnowImplementing Test Automation: What a Manager Should Know
Implementing Test Automation: What a Manager Should Know
 
Using AWS Lambda for Infrastructure Automation and Beyond
Using AWS Lambda for Infrastructure Automation and BeyondUsing AWS Lambda for Infrastructure Automation and Beyond
Using AWS Lambda for Infrastructure Automation and Beyond
 
Advanced Analytics and Data Science Expertise
Advanced Analytics and Data Science ExpertiseAdvanced Analytics and Data Science Expertise
Advanced Analytics and Data Science Expertise
 
Agile Big Data Analytics Development: An Architecture-Centric Approach
Agile Big Data Analytics Development: An Architecture-Centric ApproachAgile Big Data Analytics Development: An Architecture-Centric Approach
Agile Big Data Analytics Development: An Architecture-Centric Approach
 
Big Data as a Service: A Neo-Metropolis Model Approach for Innovation
Big Data as a Service: A Neo-Metropolis Model Approach for InnovationBig Data as a Service: A Neo-Metropolis Model Approach for Innovation
Big Data as a Service: A Neo-Metropolis Model Approach for Innovation
 
Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...
Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...
Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...
 
Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...
Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...
Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...
 
Managing Requirements with Word and TFS by Max Markov
Managing Requirements with Word and TFS by Max MarkovManaging Requirements with Word and TFS by Max Markov
Managing Requirements with Word and TFS by Max Markov
 
How to Implement Hybrid Cloud Solutions Successfully
How to Implement Hybrid Cloud Solutions SuccessfullyHow to Implement Hybrid Cloud Solutions Successfully
How to Implement Hybrid Cloud Solutions Successfully
 
Designing Big Data Systems Like a Pro
Designing Big Data Systems Like a ProDesigning Big Data Systems Like a Pro
Designing Big Data Systems Like a Pro
 
Product Management in Outsourcing by Roman Kolodchak and Roman Pavlyuk
Product Management in Outsourcing by Roman Kolodchak and Roman PavlyukProduct Management in Outsourcing by Roman Kolodchak and Roman Pavlyuk
Product Management in Outsourcing by Roman Kolodchak and Roman Pavlyuk
 

Kürzlich hochgeladen

Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfSeasiaInfotech2
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embeddingZilliz
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 

Kürzlich hochgeladen (20)

Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdf
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embedding
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 

Data Science: The Art of Foul Play by Serhiy Shelpuk

  • 1. Data Science the art of foul play Sergey Shelpuk SoftServe, Inc. sshel@softserveinc.com September, 2013
  • 2.
  • 3. “Your goal should not be to buy players, it should be to buy wins. In order to buy wins you should buy runs” (c)
  • 4.
  • 5. More data is available for companies Storage technologies allow storing and operating it Advanced analytics could be applied to this new data to achieve competitive advantage
  • 6. Data Scientist: The Sexiest Job of the 21st Century For Today’s Graduate, Just One Word: Statistics Hal Varian chief economist at Google “I keep saying that the sexy job in the next 10 years will be statisticians, and I’m not kidding.” Data Scientist: The Hottest Job You Haven't Heard Of
  • 7. Top 10 Strategic Technology Trends for 2013 • Mobile Device Battles • Mobile Applications and HTML5 • Personal Cloud • Enterprise App Stores • The Internet of Things • Hybrid IT and Cloud Computing • Strategic Big Data • Actionable Analytics • In Memory Computing • Integrated Ecosystems
  • 8. McKinsey Global Institute projects approximately 140,000 to 190,000 unfilled positions of data analytics experts in the U.S. by 2018 and a shortage of 1.5 million managers and analysts who have the ability to understand and make decisions using big data.
  • 9.
  • 10. Business Tasks • Define prospective customers • Define traffic jams in the city • Recommend restaurants and menus • Adjust UI to the particular user • Classify body part on X-Ray image • Define market niche • Define influencers in the social networks • Define similar customers or projects in portfolio • Define informal groups in the organization • Define fraud bank transaction • Define network intrusion attempts • Provide automatic aircraft engine testing • Provide automatic IT infrastructure monitoring • Provide clinical test analysis • Define the best price for the goods or services to maximize profits • Define best working schedule for the store • Define best amount of production • Define best business rules Model Family Classification Clustering Anomaly Detection Optimization • Naïve Bayes • Logistic regression • Support Vector Machines • Neural Networks • K-Means • K nearest neighbor • Self-organized maps • Mixture of Gaussians Algorithms • Mixture of Gaussians • Self-learning anomaly detection • • • • • Gradient descent Simplex method Newton’s method Normal equations Genetic algorithms
  • 11. Cross Industry Standard Process for Data Mining
  • 12. SoftServe Data Science Group Knowledge Model Business Logic Technology Level Level Level • Basics of Business Analysis • Basics of Economics • Basics of Product Management • Basics of Organizational Behavior • Statistics/Probability • Machine Learning • Data Mining • Artificial Intelligence • Matlab/Octave •R • SQL • Parallel Computing
  • 14. Feature extractor Task: recognize a motorcycle Learning algorithm
  • 15. The concept of Autoencoder
  • 16. The concept of Autoencoder … … … … … © Andrew Y. Ng
  • 17. Large scale deep learning networks See more: Building high-level features using large scale unsupervised learning
  • 18.
  • 19. Deep learning neural networks Pre-trained as Autoencoder Typical classification neural network
  • 21. Deep Learning in SoftServe Phase 1 results (old-fashion anomaly detection) Phase 2 prototype (deep learning approach)
  • 22. Useful Resources • Introduction to Statistics • Introduction to Artificial Intelligence • Machine Learning • Probabilistic Graphical Models • Statistics One