Valencian Summer School 2015 Day 2 Lecture 11 The Future of Machine Learning José David Martín-Guerrero (IDAL, UV) https://bigml.com/events/valencian-summer-school-in-machine-learning-2015

1. The Future of Machine Learning
IDAL; Intelligent Data Analysis Laboratory
Universitat de València
http://idal.uv.es
José D. Martín Guerrero

2. 3
Future …
Something new every single day …

3. 4
Machine Learning
"A computer program is said to learn from
experience E with respect to some class of
tasks T and performance measure P, if its
performance at tasks in T, as measured by P,
improves with experience E”

4. 5
Machine learning project
PROBLEM
(Analysis)
Data
extraction
Data process
Feature
engineering
MODELS Validation
All is connected!; feedback is always
necessary for the success of the project

5. 6
Feature Engineering
PROBLEM
(Analysis)
Data
extraction
Data process
Feature
engineering
MODELS Validation
Curse of dimensionality
Typical problem in bioinformatics:
O(103) features & O(10) samples

6. The correct
use of inputs
is key for a
successful
ML
application
7
Feature Engineering
FEATURES
Feature
Selection
Feature
Extraction
Manifolds
Models
We can select a subset (selection); transform
(extraction) or "attack" the model directly (deep
learning).

7. 8
Models
PROBLEM
(Analysis)
Data
extraction
Data process
Feature
engineering
MODELS Validation
No free lunch!

8. 9
Models
Any machine learning model has a certain structure and we
have to choose this (for example, the architecture of a neural
network).
First we have to choose the model that we will use in a given
problem.
Parameters are obtained by search procedures usually
controlled by other parameters we have to choose.
Parameters
Search
Algorithm
Structure
MODEL

9. 10
Example: Deep Learning
Promising models without feature engineering; apparently,
they perform pretty well but …
How many layers, how
many neurons per layer,
which activation
function?Inputs
Outputs
Hidden Layers
The most widely used algorithm is the backpropagation after initialization
using RBM (Restricted Boltzmann Machines); what adaptation constant
must one use?; if we use regularization, how do we weigh that factor?; if
we use dropout (to avoid overfitting), what % must we remove?; if we inject
noise what is the best value for its energy?
Hectic tuning

10. 11
Example: Azure ML
Many elections

11. 12
Automatic Workflows
Automatic Model Selection
Automatic Tuning
Automatic Representation
Automatic Prediction Strategies
It would be very nice to have a formal apparatus that gives us some
‘optimal’ way of recognizing unusual phenomena and inventing new
classes of hypotheses that are most likely to contain the true one; but this
remains an art for the creative human mind.” E. T. Jaynes 1985
Future: Automatic

12. 13
Future: Automatic

13. 15
Future: Automatic
http://www.automaticstatistician.com/

14. Future: Just around the corner …
Reinforcement Learning
Supervised Learning Unsupervised Learning
Reinforcement Learning
• It does not need a teacher to learn a desired signal
• There is a goal (objective function) to be maximized
• The outcome is a sequence of actions rather than a
static model
• It can deal with long-term objectives, not only a
certain steps ahead in the future!!
• Similar to some stages of human learning

15. Reinforcement Learning
EXPERIENCE INTERACTION
ARTIFICIAL LEARNING
MAXIMIZATION OF A CERTAIN OBJECTIVE FUNCTION
POLICY
A priori knowledge Environment adaptation

16. Reinforcement Learning
AGENT
ENVIRONMENT
at
st+1 (after action at)
rt+1
st (before action at)
Long term reward
Action-value function
Optimal policy
st: State (at time t)
at: Action (at time t)
rt+1: Immediate reward
Discount rate: a reward received k time
steps in the future is worth only k−1
times what it would be worth if it were
received immediately
Values of the discount rate close to 1 avoids the agent to be myopic (maximization of rt+1)

17. Reinforcement Learning: Applicability
- Traditionally, RL has been theoretically studied but until
very recently, practical applications were restricted to
well-known synthetic problems and/or Robotics.
!
!
- Any dynamic problem that can be defined in a state-
space, in which certain actions can be taken, and an
objective function has to be maximized, is susceptible to
be tackled using RL.
!
!
- Some practical applications on Marketing or Medicine
(individualization of campaigns or treatments).
!
!
!

18. Reinforcement Learning:
An example (drug prescription)
States: evaluation of the state of the patient
!
Actions: possible actions that can taken by doctors wrt to drug prescription
!
Reward: the action involves a change in the state. Depending on this
resulting state, a reward can be assigned
!
The aim is to maximize the long-term reward
It is possible to know the dosage (actions) that
should be administered to maintain patients
within a given state.
!
Other factors can also be included in the
computation of the reward (e.g., expenses).

19. 21
Conclusions
Two ways have been mentioned:!
1. Automatic election of the parameters in a machine learning project
2. Reinforcement Learning
Predicting the future is too challenging to talk about it but it is so
exciting that one must talk about it
There’s plenty of room to come up with new ideas … already
present!
1. Validation in Bayesian nets
2. Quantum Machine Learning

20. The Future of Machine Learning
IDAL; Intelligent Data Analysis Laboratory
Universitat de València
http://idal.uv.es
José D. Martín Guerrero