15. How to move a line
Rotate line counter-clockwise Increase slope
Rotate line clockwise Decrease slope
Translate line up Increase y-intercept
Translate line down Decrease y-intercept
21. Moving a line
Come
closer!
y-intercept = 3
y = 2x + 3
Slope = 2
y = 2.01x + 3.01
Step 1: Pick a small number.
(learning rate)
Step 2:
- Add learning rate to slope
- Add learning rate to y-intercept
0.01
+0.01 +0.01
0.010.01
22. Linear regression
algorithm
Come
closer!
Step 3: Pick a small number.
(learning rate)
0.01
Step 4: (repeat 1000 times)
-Pick random point
-Move line towards point
Step 2: Pick a large number.
(number of repetitions, or epochs)
1000
-If point above line, and to the right of the y-axis:
rotate counter-clockwise and translate up
-If point above line, and to the left of the y-axis:
rotate clockwise and translate up
-If point below line, and to the right of the y-axis:
rotate clockwise and translate down
-If point below line, and to the left of the y-axis:
rotate counter-clockwise and translate down
-If point above line, and to the right of the y-axis:
add 0.01 to slope
add 0.01 to y-intercept
-If point above line, and to the left of the y-axis:
subtract 0.01 to slope
add 0.01 to y-intercept
-If point below line, and to the right of the y-axis:
subtract 0.01 to slope
subtract 0.01 to y-intercept
-If point below line, and to the left of the y-axis:
add 0.01 to slope
subtract 0.01 to y-intercept
Step 1: Start with a random line
Come
closer!
Step 5: Enjoy your fitted line!
24. Step 3: Pick a small number.
(learning rate)
0.01
Step 4: (repeat 1000 times)
-Pick random point
-Move line towards point
Step 2: Pick a large number.
(number of repetitions, or epochs)
1000
-If point above line, and to the right of the y-axis:
rotate counter-clockwise and translate up
-If point above line, and to the left of the y-axis:
rotate clockwise and translate up
-If point below line, and to the right of the y-axis:
rotate clockwise and translate down
-If point below line, and to the left of the y-axis:
rotate counter-clockwise and translate down
-If point above line, and to the right of the y-axis:
add 0.01 to slope
add 0.01 to y-intercept
-If point above line, and to the left of the y-axis:
subtract 0.01 to slope
add 0.01 to y-intercept
-If point below line, and to the right of the y-axis:
subtract 0.01 to slope
subtract 0.01 to y-intercept
-If point below line, and to the left of the y-axis:
add 0.01 to slope
subtract 0.01 to y-intercept
Step 1: Start with a random line
Step 5: Enjoy your fitted line!
4 cases!
1 case?
Linear regression
algorithm
35. One rule to rule them all
Decrease slope,
decrease y-intercept
Increase slope,
increase y-intercept
Increase slope,
decrease y-intercept
Decrease slope,
increase y-intercept
+
+
+
-
-
-
-
+
Rule
To rotate:
- Add (vertical distance) x (horizontal
distance) to slope
To translate:
- Add (vertical distance) to y-intercept
36. Square trickCome
closer!
y = 2x + 3
y = 2.15x + 3.03
Step 1:
Pick a small number (learning rate) 0.01
Step 2:
- Add (learning rate) x (vertical distance) x (horizontal distance) to slope
- Add (learning rate) x (vertical distance) to y-intercept
+
+5
3
+ 0.01 x 3 x 5 + 0.01 x 3
0.01
5
3
0.01
3
+0.15 +0.03
37. Square trick
Come
closer!
y = 2x + 3
y = 1.8x + 2.96
Step 1:
Pick a small number (learning rate) 0.01
Step 2:
- Add (learning rate) x (vertical distance) x (horizontal distance) to slope
- Add (learning rate) x (vertical distance) to y-intercept
-
+5
-4
+ 0.01 x (-4) x 5 + 0.01 x (-4)
-4 0.010.01
-0.2 -0.04
5
-4
38. Linear regression
algorithm Step 3: Pick a small number.
(learning rate)
0.01
Step 4: (repeat 1000 times)
-Pick random point
-Move line towards point
Step 2: Pick a large number.
(number of repetitions, or epochs)
1000
Step 1: Start with a random line
Come
closer!
Step 5: Enjoy your fitted line!
- Add (learning rate)x(vertical distance) x
(horizontal distance) to slope
- Add (learning rate)x(vertical distance)
to y-intercept
40. Square error
Large square error Small square error
Minimize using calculus (gradient descent)
Same as the square trick!
Bad line Good line
41. Absolute error
Large absolute error Small absolute error
Minimize using calculus (gradient descent)
Develop an absolute trick!
Bad line Good line
42. Absolute trick
Step 1:
Pick a small number (learning rate)
Step 2:
- If point is above the line:
- Add (learning rate) x (horizontal distance) to slope
- Add (learning rate) to y-intercept
- If point is below the line:
- Subtract (learning rate) x (horizontal distance) to slope
- Subtract (learning rate) to y-intercept
Come
closer!
