Joachim van der Herten 1st Data Science Leuven meetup

1. Meetup @ Leuven
Probabilistic machine learning for optimization and solving complex
@javdrher

2. Work to do
Europe is investing, but we are not alone
Its no longer only Google/Facebook/Amazon business

3. Computer science
Output
Data
Program

4. Machine learning
Output
Data
Program
Data
Output
Learning phase
Deployment
phase
Learn complex hidden patterns automatically, rather than implementing them

5. Empirical risk minimization
Train model through standard framework
Trading bias-variance through regularization
Bias Variance Regularized
(Crossvalidated)
Regularized + order
(Crossvalidated)

6. Point estimates
ERM is based on point estimates (one set of weights, for a fixed model)
The bias-variance trade-off pops up because we require a distinct choice.
Idea:
- consider a continuous range of potential models
- Some of them are more likely, others are not
I can live with doubt, and uncertainty. I think it's much more interesting to live not
knowing than to have answers which might be wrong.
- Richard Feynman

7. Bayesian parametric model: marginalize weights
Data:
Model:
Prior:
Posterior:
(ps, )

8. Bayesian nonparametric model: richer class of approximations
We will use the multivariate Gaussian to put a
prior directly on the function (a Gaussian process)
I can live with doubt, and uncertainty. I think it's
much more interesting to live not knowing than to
have answers which might be wrong.
- Richard Feynman

9. Gaussian Processes

10. Expensive optimization
Engineering of complex structures, relies on expensive simulations
Goal: optimise objective over a bounded domain

11. How to optimize?
Solving for gradient = 0?
- Too complex
- Gradient unavailable
Numerical optimization?
- Multi-modality
- Gradient unavailable
Meta-heuristics?
- Too many evaluations
- Nature took a long time to optimize

12. Dynamic programming
Let’s split the task:
- Decide location for next evaluation
- Data structure: probabilistic model
- Optimize acquisition function (sampling policy)
Goal: optimality

22. Multivariate example

23. Finding exoplanets
Lengthy recording process
Space telescope
(eliminate atmospheric aberration)
Lots of data, rare transitions

24. Stabilizing space telescope
Camera field can not be guaranteed(!)
Small movements cause changes in light distribution
To severe for reliable detection of earth-like plants

25. Just use deep learning

26. Not that simple
Signal processing makes mistake
There is no ground truth
What to do with a bunch of corrupted recordings?

27. Enter causality
Assuming X c.i. From Q, and X contains enough information on f(N)
But:

28. Results
Schölkopf, Bernhard, et al. "Removing systematic errors for exoplanet search via latent causes."
Proceedings of the 32nd International Conference on Machine Learning (ICML-15). 2015.
SAP represents relative flux measure

29. Results

30. Take home message
AI is coming!
… enormous business potential
… but it will require more effort (time x money) than you all think
… europe is not at the forefront
(... forget about killer robots)
Theory is not to be avoided!
… without, experiments are shots in the dark
… probabilistic models know what they don’t know
… provide some confidence
Multidisciplinary teams are a must to tackle cases
… and they’ll need time
@javdrher joachim@ml2grow.com www.ml2grow.com