H2O World 2015 - Hank Roark Hank's iPython Notebook for this presentation can be found here: https://github.com/h2oai/h2o-world-2015-training/blob/master/tutorials/python-munging-modeling-pipelines/Munging-Modeling-Pipelines-Using-H2O-Pipelines.ipynb

1. MUNGING, MODELING,
AND PIPELINES USING PYTHON
Hank Roark

2. COMMUNITY FEEDBACK
Pythonic Interface to H2O, R interface parity
Rapid learning and iteration
Leverage existing knowledge and skills
Interface cleanly with PyData ecosystem
More Environments, esp. PySpark
Python Pipelines to Production

3. EXAMPLE FROM THE IOT
Domain: Prognostics and Health Management
Machine: Turbofan Jet Engines
Data Set: A. Saxena and K. Goebel (2008). "Turbofan
Engine Degradation Simulation Data Set", NASA Ames
Prognostics Data Repository
Predict Remaining Useful Life from Partial Life Runs
Six operating modes, two failure modes,
manufacturing variability
Training: 249 jet engines run to failure
Test: 248 jet engines

4. WHY THIS EXAMPLE?
GETTING READY FOR BRONTOBYTES

5. LOADING DATA

6. SUMMARY STATISTICS

7. FEATURE ENGINEERING
Calculate Total Cycles
For Each Unit

8. FEATURE ENGINEERING
Append To
OriginalFrame

9. FEATURE ENGINEERING
Create New
Feature of Cycles
Remaining

10. EXPLORATORY DATA ANALYSIS
Boolean
Indexing

11. EXPLORATORY DATA ANALYSIS
Sample the
data to local
memory

12. EXPLORATORY DATA ANALYSIS
Use your
favorite
visualization
tools
(Seaborn!)
Ugh,
where are
trends
over time
Time
Zero
Remaining
Useful
Life

13. MODEL BASED DATA ENRICHMENT
Sensor
measurements
appear in
clusters
Corresponding
to operating
mode!

14. MODEL BASED DATA ENRICHMENT
Use H2O k-means
to find cluster
centers

15. MODEL BASED DATA ENRICHMENT
Enrich existing data
with operating mode
membership

16. MORE FEATURE ENGINEERING
For non-constant
sensor
measurements
within an
operating mode,
Standardize each
sensor measurement
by operating mode
Based on the
training data

17. TRENDS OVER TIME!
Before
H2O Munging
Ready for
H2O Learning
Time Time

18. MODELING
Configure an
Estimator

19. MODELING
Train an Estimator

20. MODEL EVALUATION
Evaluate Performance
at a glance
in Python

21. MODEL EVALUATION
Evaluate Performance
at a glance
in H2O Flow

22. MODEL EVALUATION
Evaluate Performance
at a glance
graphically in Python

23. CROSS VALIDATION
Setup
Hyperparameter
Search Options

24. CROSS VALIDATION
Configure
full full
grid search

25. CROSS VALIDATION
Execute
grid search

26. CROSS VALIDATION
Evaluate results &
model selection

27. MORE CONTROL – SCIKIT PIPELINES
Create Pipelines
Hyperparameter Options
Cross validation strategy
Hyperparameter
Search Strategy
Fit

28. DATA PIPELINES USING H2OASSEMBLY
Typical
Data Preparation
Add some structure

29. H2OASSEMBLY TO PRODUCTION
Java
for
Production
Scoring
Python

30. MORE ENVIRONMENTS
PySparkling Water = Python + Spark + H2O
Python + Sparkling Water

31. COMMUNITY FEEDBACK
 Pythonic Interface to H2O, R interface parity
 Rapid learning and iteration
 Leverage existing knowledge and skills
 Interface cleanly with PyData ecosystem
 More Environments, esp. PySpark
 Python Pipelines to Production

32. RESULTS
H2O Python Framework:
H2OFrame & H2OEstimators
H2OAssembly for Data Prep Pipelines
Python, Jupyter Notebooks,
Pandas, Scikit-Learn Integration
PySparkling Water

33. RESOURCES
• Python booklet
• Tibshirani release
• Python documentation
• Github examples
• Jupyter Notebook of Example

34. THANK YOU