YouTube Link: https://youtu.be/DhdUlDIAG7Y ** Python Data Science Training: https://www.edureka.co/python ** This Edureka PPT on "What is Q Learning" will provide you with a detailed and comprehensive knowledge of Q-Learning and also the various aspects of Q-Learning. Python Tutorial Playlist: https://goo.gl/WsBpKe Blog Series: http://bit.ly/2sqmP4s Follow us to never miss an update in the future. YouTube: https://www.youtube.com/user/edurekaIN Instagram: https://www.instagram.com/edureka_learning/ Facebook: https://www.facebook.com/edurekaIN/ Twitter: https://twitter.com/edurekain LinkedIn: https://www.linkedin.com/company/edureka Castbox: https://castbox.fm/networks/505?country=in

2. www.edureka.co
The Bellman Equation
1
1
1 1 1

3. www.edureka.co
The Bellman Equation
1
1
1 A

4. www.edureka.co
The Bellman Equation
s = a particular state (room)
a = action
s′ = state to which the robot goes from s
𝜸 = discount factor
R(s, a) = a reward function which takes a state s and action a and outputs a reward value
V(s) = value of being in a particular state

5. www.edureka.co
The Bellman Equation
1
*

6. www.edureka.co
The Bellman Equation
Discount Factor = 0.9

7. www.edureka.co
The Bellman Equation
1
0.9
START 0.729 0.81

8. www.edureka.co
The Bellman Equation
0.9 1
0.81 0.9 1
0.729 0.81 0.9
0.66 0.729 0.81

9. www.edureka.co
Markov Decision Processes

10. www.edureka.co
The Bellman Equation
o

11. www.edureka.co
The Bellman Equation
↑
← o →
↓

12. www.edureka.co
The Bellman Equation
The probability of moving from room s to room s′ with action a
expectation of the situation that the robot incurs randomness

13. www.edureka.co
The Bellman Equation
↑ 0.8
0.03 ← o → 0.03
↓ 0.1

14. www.edureka.co
The Bellman Equation
↑ 0.8
0.03 ← o → 0.03
↓ 0.1

15. www.edureka.co
Transitioning to Q-Learning

16. www.edureka.co
Q-Learning
↑ 0.8 (V(s1))
0.03 (V(s4))← o → 0.03 (V(s2))
↓ 0.1 (V(s3))

17. www.edureka.co
Q-Learning
Q (s1,a1)
Q (s4,a4) o Q (s2,a2)
Q (s3,a3)

18. www.edureka.co
Q-Learning
Q (s1,a1)
Q (s4,a4) o Q (s2,a2)
Q (s3,a3)

19. www.edureka.co
Q-Learning
Q (s1,a1)
Q (s4,a4) o Q (s2,a2)
Q (s3,a3)

20. www.edureka.co
Q-Learning

21. www.edureka.co
Q-Learning
___

22. www.edureka.co
Q-Learning
___

23. www.edureka.co
Implementing Q-Learning
In Python (NumPy)

24. www.edureka.co
PROBLEM STATEMENT
L1 L2 L3
L4 L5 L6
L7 L8 L9

26. www.edureka.co