This document provides an overview of Pivotal's data science capabilities and tools. It discusses how Pivotal uses an Agile approach to data science projects, focusing on frequent interactions with customers. Pivotal's software stack is designed to enable data science work by supporting real-time, interactive, and batch operations on data through tools like Spring XD, Pivotal HD, and GemFire. Examples are provided of how Pivotal has used these tools for applications like connected cars and scalable video analytics.

1. Data Science
A Technical Overview
Dr Carsten Riggselsen
Principal Data Scientist
Pivotal

2. 2© Copyright 2015 Pivotal. All rights reserved.

3. 3© Copyright 2015 Pivotal. All rights reserved.
Pivotal in a Nutshell

4. 4© Copyright 2015 Pivotal. All rights reserved.
Platform to Solution Level
Ÿ Ready to be incorporated into scalable, platform-independent applications:
– Pivotal Cloud Foundry platform-as-a-service
– Pivotal Labs gold standard for modern software development
– World-class Data Science capability
Solution
Level
Ÿ Pivotal Big Data Suite is the only enterprise-grade software distribution that
contains all elements of the λ architecture
Ÿ Wrapped in a flexible and consumption-based commercial offer
Ÿ Truly enables enterprises to make the best decisions when it matters
Platform
Level
Platform
Component
Level
Ÿ GemFire is the leading in-memory data grid, processing 2 million events per second
Ÿ Hawq brings 100% ANSI compliant SQL, hundreds of concurrent queries and JDBC
& ODBC compliance to link to legacy databases
Ÿ Spring XD streaming workflow builds performant pipelines to consume and
consolidate data from a variety of endpoints

5. 5© Copyright 2015 Pivotal. All rights reserved.
Discover insights
Create analytics algorithms
Digital Transformation
Deploy analytic apps
and automation at scale
Store any type
and size of data

6. 6© Copyright 2015 Pivotal. All rights reserved.
Value of Data and Information
Time
Value of
Data
µs ms s hour day month year yr+
Spring XD
Pivotal HD
Pivotal GemFire
Information

7. 7© Copyright 2015 Pivotal. All rights reserved.
• Translate business problems into a
mathematical/statistical problem
• Combining appropriate Data Science
techniques
• Infer and estimate parameter of statistical
models
• Rely heavily on Pivotal’s stack and OSS
What We Do

8. 8© Copyright 2015 Pivotal. All rights reserved.
Data Science Toolkit
Pivotal Big Data SuitePivotal HD
Pivotal Greenplum
Database
P L A T F O R M
Pivotal CF®
KEY TOOLS
KEY LANGUAGES
SQL
Spring XD

9. 9© Copyright 2015 Pivotal. All rights reserved.
R E A LT I M E
DASHBOARD
Destination
Prediction

10. 10© Copyright 2015 Pivotal. All rights reserved.
Understanding Driving Behavior
Vehicle
Speed
(km/h)
Distance (m)
• Same car
• Same roads
• Different drive
styles
0
10
20
30
40
50
60
- 200 400 600 800 1,000 1,200 1,400

11. 11© Copyright 2015 Pivotal. All rights reserved.
Pivotal HD
Spring XD
Realtime Evaluation
Batch Training
Data Persistence
ODB/Bluetooth

12. 12© Copyright 2015 Pivotal. All rights reserved.
Example: Pivotal’s Connected Car (λ Arch)
Spring XD
Pivotal HD
Pivotal GemFire
Speed Layer Serving Layer
Batch Layer

13. 13© Copyright 2015 Pivotal. All rights reserved.
Video Recording

14. 14© Copyright 2015 Pivotal. All rights reserved.
Pivotal’s Scalable Video Analytics Architecture

15. 15© Copyright 2015 Pivotal. All rights reserved.
How We Do It
• Agile principles and practices
• Data Science enablement
• Collaboration is key

16. 16© Copyright 2015 Pivotal. All rights reserved.
• Agile is an iterative approach that
enables you to quickly change the
kind of analysis you are doing,
depending on what the data is telling
you.
• Frequent interactions and pairing
with the customer ensure that the
project stays on track and de-risks.
What Agile Data Science Means
Model Evaluation
Feature Review
Model Building
Feature EngineeringData Review
Operationalization
Scoping
Agile Data Science

17. 17© Copyright 2015 Pivotal. All rights reserved.
Frequent
Feedback
Removes
Risk

18. 18© Copyright 2015 Pivotal. All rights reserved.
Pivotal’s Software stack enables Data Science
Real-time, Interactive and Batch operations
Different architectures can be realized
Data Science is still a matter of thinking
Collaborate and interact
Conclusions

19. Digital Transformation Forum
Disrupt or Be Disrupted
19 OCTOBER · BMW WELT EVENT CENTRE · MUNICH