Data science in Azure

© 2019 Visual BI Solutions, Inc. All rights reserved. www.visualbi.com
DATA SCIENCE IN AZURE
End-to-End Data Analytics using Azure Databricks
Chandler Stevens
VP - Microsoft BI & Analytics www.visualbi.com
Jiwon Jeon
Data Scientist

© 2019 Visual BI Solutions, Inc. All rights reserved. www.visualbi.com 2
This presentation outlines our general product direction and should not be relied on in making a purchase decision. This
presentation is not subject to your license agreement or any other agreement with Visual BI Solutions. Visual BI Solutions has
no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality
mentioned in this presentation.
This presentation and Visual BI Solution’s strategy and possible future developments are subject to change and may be
changed by Visual BI Solutions at any time for any reason without notice. This presentation is provided without a warranty of
any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. Visual BI Solutions assumes no responsibility for errors or omissions in this
presentation, except if such damages were caused by Visual BI Solutions intentionally or grossly negligent.
VISUAL BI SOLUTIONS
Legal Disclaimer

Lead Data ScientistVP, Microsoft BI and Analytics
Jiwon Jeon
jiwonj@visualbi.com
3
Introducing
Today’s Presenters
Chandler Stevens
chandlers@visualbi.com

Today’s Agenda
DATA SCIENCE INTRODUCTION1
DATA SCIENCE IN AZURE2
AZURE DATABRICKS & DEMO3
SUMMARY4
Q&A (via IM Chat)5

ABOUT VISUAL BI
All-in-One Partner for End-to-End BI & Analytics Needs
Visual BI Solutions is a leading All-in-One Business Intelligence (BI) enablement firm specializing in BI &
Analytics services, solutions, trainings and products. We have proven expertise in enabling BI & Analytics
for 100+ world’s leading brands. We can help you achieve competitive advantage by effectively managing
the Plan - Build - Run spectrum for BI.
Trusted by the largest companies world-wide
Trusted by
the industry
Integration and Partnership with
SAP and Microsoft is our forte
CONSULTING SERVICES
• Strategy
• Architecture Implementation
• Training
• Managed Services
• Visualization
• Cloud Migrations
ANALYTICS SOLUTIONS
• Dashboards by LOB
• Advanced Analytics
• Big Data Solutions
TRAINING
• SAP Business Objects Training
§ SAP Lumira Discovery (2 Days)
§ SAP Lumira Designer (3 Days )
§ SAP Web Intelligence (2 Days)
§ SAP Analysis for Office (2 Days)
• SAP Analytics Cloud Training
• Microsoft Power BI Training
SOFTWARE PRODUCTS
• VBI View – One Portal for All BI Content
• Product Extensions for SAP Lumira / SAP Design Studio
§ Visual BI Extensions (VBX Suite)
§ Document Management and Change Control
• Value Driver Tree (VDT) for Planning and Simulations

Data Science in Azure

Need of Data Science
Data becomes BIG, COMPLEX, and stays EVERYWHERE à “DATA SCIENCE” is a need and no longer a want
§ Technology is driving data creation
§ Size of data is a factor
§ Maximizing information is critical
§ Emergence of unstructured data

What is Data Science?
9
Study of exploring all available forms of data and employing scientific methods to extract knowledge
and derive insights for actionable decision-making
• A series of actions/studies from acquiring data, processing data, modeling and deploying for
integration
• Exploring structured and unstructured data
• Using scientific methods, algorithms and systems and assisted by data visualization, machine
learning, and big data platforms to obtain knowledge and insights to make actionable decision
To present the
right offer to the
right user based
on the right-time
decision
Business encounters faster
changes
Data-driven decisions need to be
made accurately
Users/Customers require
prompt responses

Stochastic
optimization
How can we achieve the best outcome including
variability?
Optimization How can we achieve the best outcome?
10
CompetitiveAdvantage
Degree of complexity
PRESCRIPTIVE
DESCRIPTIVE
PREDICTIVE
Query/Drill down What exactly is the problem?
Ad hoc reporting How many, how often, where?
Standard reporting What happened?
Predictive modeling What will happen next if…?
Forecasting What if these trends continue?
Simulation What could happen?
Alerts What actions are needed?
Credited to: Competing on Analytics, Davenport and Harris, 2007
Journey to Data Monetization

Data Science Process Lifecycle – CRISP-DM
11
Cross Industry Standard Process for Data
Mining
§ An open standard process model for common
approaches used by data mining experts.
CRISP-DM consists of the followings:
§ Project/Business understanding: understand
stakeholder motivations
§ Data understanding: finding data sources and
acquiring data
§ Data preparation: cleaning and transforming data
§ Modeling: understanding relationships in data
§ Evaluation: optimizing the model to achieve the
project goal
§ Deployment: delivering value from data

Data Science Process Lifecycle – MS TDSP
12
Team Data Science Process Lifecycle
§ Originally designed for data science projects
as part of intelligent applications.
§ Employed machine learning or artificial
intelligence models for predictive analytics.
§ To avoid misunderstandings between teams
and customers by using a well-defined set of
artifacts
TDSP consists of the followings:
§ Business understanding
§ Data acquisition and understanding
§ Modeling
§ Deployment
§ Customer acceptance
Credited to: Microsoft Azure Documentation

Data Science Process Lifecycle – MS TDSP
13
TDSP roles and tasks
Business
Understanding
Data Acquisition &
Understanding
Modeling Deployment Customer Acceptance
Project
Lead
DataScientist
Project
Manager
Solution
Architect
Create Template
Repository
Create
Project
Charter
Project
Charter
Provision Data
Infrastructure
Provision
Compute Assets
Data Ingest
& Explore
Data Summary
Report
Design Solution
Architecture
Solution Architecture
Diagram
Feature
Engineering
Model
Development
Model
Report
Develop Data
Pipeline
Deploy
Scoring
Process
Monitor
health &
metrics
Deploy
Pipeline
Check in
final
Artifacts
Finalize
Documentation
Project Final
Report
Decommission
Compute
Assets
Checkpoint
Project
Transition to
Production Support
Dashboard
Credited to: Microsoft Azure Documentation

Azure Data Stack
14
INGEST STORE MODEL & SERVEPREP & TRAIN
PolyBase
Azure SQL
Data
Warehouse Azure
Analysis
Services
Azure
Databricks
Azure
Cosmos
DB
Azure
Data
Factory
Azure
Data Lake
Storage
Power
BI
Predictive
Applications
Logs, files
and media
(unstructured
data)
Business/custom
apps
(structured data)
§ Azure Data Factory
§ Azure Import Export
Service
§ Azure Data
Migration Service
§ Azure Event Hub
§ Azure IoT Hub
§ Azure CLI
§ Azure SDK
§ Azure Blob Storage
§ Azure Data Lake
§ Azure SQL DB
§ Azure Data
Warehouse
§ Azure Cosmos DB
§ Azure Databricks
§ Azure HDInsight
§ Azure ML Service
§ Azure ML Studio
§ Azure Data Science VM
§ Azure Cognitive
Services
§ Azure Data Lake
Analytics
§ Azure Bot Services
§ Azure Stream Analytics
§ Power BI
§ IoT Apps
§ Azure Analysis Service
Azure
Machine
Learning
Streaming
data
(unstructured
data)
Azure
Event
Hub

Data Science Resources in Azure
15
Azure ML Studio Azure ML Service Azure HD Insight Azure Databricks
Whatitis
Drag–and–drop visual workspace
for ML
Managed cloud service for a
variety of open source big data
analytics workloads
Azure implementation of
Hadoop as a managed
service supporting a variety
of open-source analytics
engines
Spark-based analytics platform
Works
Build, test, and deploy models
using pre-built ML algorithms
Train, deploy, and manage ML
models at scale using Python
and CLI
Build, test and deploy ML
models with massive data
Build and deploy models and
data workflows
Features
• Publishes models as web
services for further use
• No programing is required
• Rich tools and packages
• Use external compute
engines including SQL Server
and Spark
• Auto-parameter tuning
• Developer tooling and
monitoring capabilities
• Orchestration via Azure
Data Factory
• Native Integration with Azure
for Security via AAD
• Single engine for Batch,
Streaming, ML and Graph
• Notebook-based
collaborative environment
• Autoscaling
Scenarios
• For quick exploration of data or
ML algorithms
• For testing the
operationalization of model with
least error
For integrated use of difference
resources for ML modeling and
deployment at scale
• When Hadoop
technologies are required
than Spark
• To stay in codebase
environment and/or ‘Lift
and Shift’ from on-prem
deployments
• When Spark and notebook
options are required
• When Auto-scaling is
required
• To build integrated and
performant data pipelines
For Advanced Analytics and Machine Learning solutions using Big Data in the Cloud
Azure Databricks
Spark-based analytics platform
Build and deploy models and
data workflows
• Native Integration with Azure
for Security via AAD
• Single engine for Batch,
Streaming, ML and Graph
• Notebook-based
collaborative environment
• Autoscaling
• When Spark and notebook
options are required
• When Auto-scaling is
required
• To build integrated and
performant data pipelines

Azure Databricks

Deep Learning
AI models using GPU-enabled
clusters with deep learning
frameworks
What is Azure Databricks?
17
Apache Spark-based unified analytics platform offering the best of Spark with collaborative notebooks and enterprise
features optimized for Azure.
Apache Spark environment
Databricks Runtime and
serverless compute model
Collaborative workspace
Shared notebook for data engineers,
data scientists and business users
One-click setup
Streamlined workflows
Autoscale & Autoterminate
Autoscaling up and down of
clusters & Autoterminating
inactive clusters
Integration w/ Azure services
Integration with Azure data services &
stores by SSO with Azure AD

Workflow in Azure Databricks
19
LAUNCH
WORKSPACE
Log in to Azure Databricks in the
Azure portal using single sign-on
with Azure AD.
OPEN
CLUSTERS
Create a new cluster, configure
and start it with one click. The
autoscaling feature makes scaling
clusters fast and easy. The
autoterminating feature shuts
down inactive clusters as desired.
Both features help reduce
resources and costs associated
with manual operations.
COLLABORATE ON
NOTEBOOKS
Create custom access settings for data
engineers, data scientists, and
business users for shared projects to
cooperate on the notebooks based on
individual access level.
SCHEDULE
JOBS
Run notebooks as jobs by choosing
from existing streaming or machine
learning libraries. Schedule jobs in
advance to run automatically, and
monitor their performance.
BUILD DATA SCIENCE
MODELS
Build, train, and deploy AI models
at scale using any data languages
among SQL, Python, Scala, and R.
EXPLORE
DATA
Using SQL, Python, Scala, and R in
notebooks to easily mount storage and
collect observations to build machine
learning models. Business users can
see data in easy-to-read live data
displays.

Demo

Workflow in Azure Databricks
21
LAUNCH
WORKSPACE
Log in to Azure Databricks in the
Azure portal using single sign-on
with Azure AD.
OPEN
CLUSTERS
Create a new cluster, configure
and start it with one click. The
autoscaling feature makes scaling
clusters fast and easy. The
autoterminating feature shuts
down inactive clusters as desired.
Both features help reduce
resources and costs associated
with manual operations.
COLLABORATE ON
NOTEBOOKS
Create custom access settings for data
engineers, data scientists, and
business users for shared projects to
cooperate on the notebooks based on
individual access level.
SCHEDULE
JOBS
Run notebooks as jobs by choosing
from existing streaming or machine
learning libraries. Schedule jobs in
advance to run automatically, and
monitor their performance.
BUILD DATA SCIENCE
MODELS
Build, train, and deploy AI models
at scale using any data languages
among SQL, Python, Scala, and R.
EXPLORE
DATA
Using SQL, Python, Scala, and R in
notebooks to easily mount storage and
collect observations to build machine
learning models. Business users can
see data in easy-to-read live data
displays.

2
INTRODUCING VISUAL BI MODERN ANALYTICS
§ Harness data sourced from online networks,
web pages, audio and video devices, social
media, logs and many other sources to uncover
insights and patterns
§ Refine and tune machine learning models to
boost prediction accuracy
§ Deliver big data solutions that can encompass
“lift and shift” and “cloud-native”
implementation models
§ Enable operational agility with enhanced
telemetry management
§ Move beyond understanding “what happened”
to “how can we achieve the best possible
outcome”
Modern
Analytics
Descriptive to
Predictive
Operations
with agility
Multi-stack
Implementatio
n
Boost ML
Accuracy
Harness
Diverse Data
Enable Insight Velocity with Big Data

§ Modern Data Science project requires handling big data to find
the hidden insights
§ Azure Databricks provides cloud-scale analytics platform with
fast and secured performance
§ Visual BI Solutions can provide end-to-end services for design,
implementation and end user enablement of a data science
solution for you
§ For more information reach us at solutions@visualbi.com
Summary

UPCOMING WEBINAR
CLICK HERE TO REGISTER NOW

www.visualbi.com
THANK YOU! gopal
@visua
lbi.com
www.visualbi.com
Chandler Stevens
chandlers@visualbi.com
VP – Microsoft BI & Analytics
Jiwon Jeon
jiwonj@visualbi.com
Data Scientist

Data science in Azure

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Data science in Azure

Ähnlich wie Data science in Azure (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Data science in Azure