Reinforcement Learning : A Beginners Tutorial

1. Reinforcement Learning A Beginner’s TutorialBy: Omar Enayet (Presentation Version)
2. The Problem
3. Agent-Environment Interface
4. Environment Model
5. Goals & Rewards
6. Returns
7. Credit-Assignment Problem
8. Markov Decision Process An MDP is defined by < S, A, p, r, > S - set of states of the environment A(s)– set of actions possible in state s - probability of transition from s - expected reward when executing ain s  - discount rate for expected reward Assumption: discrete timet = 0, 1, 2, . . . r r r t +2 t +3 . . . s . . . t +1 s s s t+3 t+1 t+2 t a a a a t t+1 t +2 t +3
9. Value Functions
12. Optimal Value Functions
13. Exploration-Exploitation Problem
14. Policies
15. Elementary Solution Methods
16. Dynamic Programming
17. Perfect Model
18. Bootstrapping
19. Generalized Policy Iteration
20. Efficiency of DP
21. Monte-Carlo Methods
22. Episodic Return
23. Advantages over DP <ul><li>No Model
24. Simulation OR part of Model
25. Focus on small subset of states
26. Less Harmed by violations of Markov Property</li></li></ul><li>First Visit VS Every-Visit
27. On-Policy VS Off-Policy
28. Action-value instead of State-value
29. Temporal-Difference Learning
30. Advantages of TD Learning
31. SARSA (On-Policy)
32. Q-Learning (Off-Policy)
33.
34. Actor-Critic Methods(On-Policy)
35. R-Learning (Off-Policy) >>Average Expected reward per time-step
36. Eligibility Traces
37.
38.
39. References Richard S. Sutton and Andrew G. Barto. Reinforcement Learning, Bradford Books, 1998. Richard Crouch, Peter Bennett, Stephen Bridges, Nick Piper and Robert Oates - Monte Carlo - 2003 Slides for reading with : <ul><li>Omar Enayet – Reinforcement Learning : A Beginner’s Tutorial- 2009</li>

Hinweis der Redaktion

By a model of the environment we mean anything that an agent can use to predict how the environment will respond to its actions. Given a state and an action, a model produces a prediction of the resultant next state and next reward. If the model is stochastic, then there are several possible next states and next rewards, each with some probability of occurring. Some models produce a description of all possibilities and their probabilities; these we call distribution models. Other models produce just one of the possibilities, sampled according to the probabilities; these we call sample models. For example, consider modeling the sum of a dozen dice. A distribution model would produce all possible sums and their probabilities of occurring, whereas a sample model would produce an individual sum drawn according to this probability distribution.
Credit assignment problem: How do you distribute credit for success among the many decisions that may have been involved in producing it?
One of the challenges that arise in reinforcement learning and not in other kinds of learning is the trade-off between exploration and exploitation. To obtain a lot of reward, a reinforcement learning agent must prefer actions that it has tried in the past and found to be effective in producing reward. But to discover such actions, it has to try actions that it has not selected before. The agent has to exploit what it already knows in order to obtain reward, but it also has to explore in order to make better action selections in the future. The dilemma is that neither exploration nor exploitation can be pursued exclusively without failing at the task. The agent must try a variety of actions andprogressively favor those that appear to be best. On a stochastic task, each action must be tried many times to gain a reliable estimate its expected reward. The exploration-exploitation dilemma has been intensively studied by mathematicians for many decades

Reinforcement Learning : A Beginners Tutorial

Du liest eine Vorschau.

Hier ansehen

Reinforcement Learning : A Beginners Tutorial

Weitere Verwandte Inhalte

Diashows für Sie (20)

Andere mochten auch (7)

Ähnlich wie Reinforcement Learning : A Beginners Tutorial (20)

Reinforcement Learning : A Beginners Tutorial

Hinweis der Redaktion