A developer's introduction to big data processing with Azure Databricks

3. “More than any other factor,
customer experiences determine
whether companies thrive and
profit, or struggle and fade.”
– Forrester Research

4. Speed
79%
won’t return
to a slow website
Personalization
38%
won't call again if they
have to repeat themselves
Consistency
65%
get frustrated with
inconsistent device experiences

5. Harness the power of
Big Data analytics
apps to exceed
customer needs

6. Enhance any type of
app with Big Data
analytics

7. Introducing
Wide World Importers

8. Online shopping through
company website
E-commerce directly in the
consumer’s hand, anywhere
Wide World ImportersWide World Importers
Retail stores

9. Wide World Importers
Web e-commerce Mobile e-commerce
Wide World Importers seeks to expand
customers through an omni-channel strategy
Retail stores

10. The solutions needed
to reach more
customers and grow
the business
1. Scale with ease to reach more consumers
2. Unlock business insights from unstructured data
3. Enhance user experience with advanced analytics
4. Apply real-time analytics for instant updates
5. Infuse AI into apps to actively engage with customers

11. A Z U R E D A T A B R I C K S
Microsoft Azure

12. Optimized Databricks Runtime Engine
DATABRICKS I/O SERVERLESS
Collaborative Workspace
Cloud storage
Data warehouses
Hadoop storage
IoT / streaming data
Rest APIs
Machine learning models
BI tools
Data exports
Data warehouses
Azure Databricks
Enhance Productivity
Deploy Production Jobs & Workflows
APACHE SPARK
MULTI-STAGE PIPELINES
DATA ENGINEER
JOB SCHEDULER NOTIFICATION & LOGS
DATA SCIENTIST BUSINESS ANALYST
Build on secure & trusted cloud Scale without limits
A Z U R E D A T A B R I C K S

14. Azure Databricks
Logical Architectures

15. INGEST STORE PREP & TRAIN MODEL & SERVE
Azure Blob Storage
Logs, files and media
(unstructured)
Azure SQL Data
Warehouse
Azure Data Factory
Azure Analysis
Services
Azure Databricks
(Python, Scala, Spark SQL)
Polybase
Business/custom apps
(Structured)
Power BI
Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.

16. INGEST STORE PREP & TRAIN MODEL & SERVE
Azure Blob Storage
Logs, files and media
(unstructured)
Azure SQL Data
Warehouse
Azure Data Factory
Azure Analysis
Services
Polybase
Business/custom apps
(Structured)
Power BI
Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.
Azure Databricks
(Python, Scala, Spark SQL)
Azure Databricks
(Spark ML, Spark R, SparklyR)
Intelligent Apps
Cosmos DB

17. INGEST STORE PREP & TRAIN MODEL & SERVE
Logs, files and media
(unstructured)
Sensors and IoT
(unstructured)
HDInsight
(Kafka)
Power BIAzure Databricks
(Python, Scala, Spark SQL)
Intelligent Apps
Cosmos DBEvent Hub
IoT Hub
Azure Databricks
(Spark ML, Spark R, SparklyR)
Azure Blob Storage
Batch Data
(Apps, logs) Azure Data Factory

18. Azure Databricks at //BUILD 2018
Tuesday:
• BRK3320 The Developer Data Scientist – Creating New Analytics
Driven Applications using Apache Spark with Azure Databricks
• WRK2601 Using Databricks to Analyze Telemetry Data Stored in
Azure Blob Storage
Wednesday:
• BRK4102 ETL 2.0 - Data Engineering for developers
• BRK3314 Leveraging Azure Databricks to minimize time to insight
by combining Batch and Stream processing pipelines.
• BRK3708 Machine learning at scale