This is lecure 2 note for JAIST summer school on computational motor control (Hirokazu Tanaka & Hiroyuki Kambara). Lecture video: https://www.youtube.com/watch?v=lNH1q4y1m-U

1. Computational Motor
Control Summer School
02: Optimal control for
deterministic systems
Hirokazu Tanaka
School of Information Science
Japan Institute of Science and Technology

2. Optimal control for deterministic (noiseless) systems.
In this lecture, we will learn:
• Invariant laws of body movements
• Calculus of variation
• Lagrange multiplier method
• Karush-Kuhn-Tucker condition
• Pontryagin’s minimum principle
• Boundary problem
• Minimum-Jerk model
• Minimum-torque-change model
• Kinematic vs dynamic planning in reaching movements

3. Straight paths and bell-shaped velocity in reaching.
Morasso (1981) Exp Brain Res
12
3 4 5
6
joint
angles
angular
velocity
angular
accel.
hand
speed
T1->T4 T3->T5 T2->T5 T1->T5

4. Power laws for curved movements.
Lacquaniti et al. (1983) Acta Psychologica
2
3
 
1
3
v 


: angular velocity
: h
: curvature
and speedv


or
curvature
angularvelocity

5. Fitts’ law for movement duration in rapid pointing movements.
Fitts (1954) J Exp Psychol
2
2
logf
D
t a b
W
 

6. “Main sequence” for saccadic movements
Bahill et al. (1975) Math Biosci

7. Optimality principle: how a unique trajectory is chosen.
http://en.wikipedia.org/wiki/Snell%27s_law
Snell’s law
Fermat’s principle of least time
1 1 1
2 2 2
sin 1/
sin 1/
v n
v n


 
 
 
 
Q Q
0 P P
1T ds
T s dt n s ds
v s c
        
   
Q
P
1
0T s n s ds
c
      

8. Optimality principle: how a unique trajectory is chosen.
Newton’s equation of motion
Hamilton’s principle of least action
 V
m

 

q
q
q
   
2
1
,
t
t
S dtL q q q
   21
,
2
L V q q q q
  0S q
http://en.wikipedia.org/wiki/Principle
_of_least_action

9. Mathematics: Calculus of variation.
   0
, ,
ft
xS x x L x dt 
     
   
0
0 0
0
0
, ,
, , ,
f
f f
f
f
t t
t
t t
t
t
L x dt L x dt
L
S x x S x x x x S x x
x
L
dt
x x
L d L L
dt
x dt
x x x
x
x
x
x x
x
  
 
 
 


 
  
    
    
        
   
 

 


0
d L L
dt x x
  
  
   0
0
ft t t
L L
x
x
x
x 
 
 
 
 
S: Action, L: Lagrangian
Euler-Lagrange equation
     x t x t x t  variation

10. Mathematics: Calculus of variation.
   0
, ,, ,, ,
ft
S x dtLx xx xxx x 
     0 0
0
2 3
2 3
2
0
2
, , , , ,, ,, ,
f f
f
f
t t
t
t L d L d L d L
x dt x dt x dt
L d L d L
x
x
S x x x x x xx dtL x dtL x
dt
dt x dt x
x x x x x x
x
x
x
   



   
     
 
  
 

     
       
 
      
     
   
 
  
     

 

00
f f
t t
L d L L
x x
x dt x x
 
    
  
   


 
2 3
2 4
0 0 f
L d L d L d L
t t
x dt x dt x dt x
        
          
        
2
2
000
0
f f f
t t t
L d L d L L d L L
x x x
x dt x dt x x dt x x
  
            
             
            
Lagrangian with higher derivatives
Euler-Poisson equation

11. Mathematics: Calculus of variation.
 
2 3
2 3
0 0 f
L d L d L d L
t t
x dt x dt x dt x
        
          
        
2
2
000
0
f f f
t t t
L d L d L L d L L
x x x
x dt x dt x x dt x x
  
            
             
            
 
2 3
2 3
0 0 f
L d L d L d L
t t
x dt x dt x dt x
        
          
        
0 0 0
0f f ft t t
x x x    
Euler-Poisson equation with general boundary conditions
Euler-Poisson equation with fixed boundary conditions

12. Smoothness criterion: Minimum-jerk model.
 
2 23
30
3
3MJ
0
, , , , , , ,
ftft d x d y
x x x x y y y y dt dt
dt dt
C L 
    
      
     

Flash & Hogan (1985) J Neurosci
 
2 3 3
2 3 6
6
3
0 2 2
L d L d L d L d d x
x
x dt x dt x dt x dt dt
        
            
        
6
6 6
6
0
d x d y
dt dt
 
Intuition: The observed movement trajectories are smooth, so
“smoothness” of trajectory may be selected in the brain.
: position, : velocity, : acceleration, : jerkx x x x

13. Smoothness criterion: Minimum-jerk model.
Flash & Hogan (1985) J Neurosci
6
6 6
6
0
d x d y
dt dt
 
 
 
00
f f
x x
x t x


   
   
0 0 0
0f f
x x
x t x t
 
 
Boundary conditions for a point-to-point movement:
Euler-Lagrange equation:
   
45
0 0
3
6 15 10f
f f f
t t t
x xt x x
t t t
       
              
       

Solution of minimum-jerk trajectory (5th order polynomial)

14. Smooth trajectory and bell-shaped velocity explained by the model.
Flash & Hogan (1985) J Neurosci
speed y accel x accel speed y accel x accel

15. Via-point movements also explained by the model.
Flash & Hogan (1985) J Neurosci
   3 4 5
0 1 2
2
3 4 5 10x t a a t a t a t a t a t t t     
       
   3 4 5
0 1 2 3 4
2
5 1 fx t a a t a t a t a t a t t tt     
     
     00 , 0 0 0x x x x  
  
     , 0f f ffx t x x t x t  
  
       
           
1 1 1 1 1 1
1 1 1 1 1 1
, , ,
, , .
x t x x t x x t x t
x t x t x t x t x t x t
   
     
  
  
  00x x
 1 1x t x
 f fx t x
 x t
 x t
Twelve unknown coefficients can be
determined by twelve boundary conditions.
 ia

16. Power law predicted by the minimum-jerk model.
Viviani & Flash (1995) J Exp Psychol

17. Smoothness criterion: path-constrained Minimum-jerk model.
Huh & Sejnowski (2015) PNAS
Cartesian coordinates Frenet-Serret coordinates
     ˆv v t  
   4 26
ˆ ˆ32 1v z z h z t z n
v v
h      
 
    
   
 
 
 
 
0 0
log , log
v
z h
v
  
 

 

18. Smoothness criterion: path-constrained Minimum-jerk model.
Huh & Sejnowski (2015) PNAS
   4 26
ˆ ˆ32 1v z z h z t z n
v v
h      
 
    
   
2
2
0
d
dzdz z
d
  
   
 
  
 
   
    

 
 
   
4 6 2 3 4
3
2 2 2
2 2
5 2 30 10 25 82 40
2 14 90 12
20 55 75 15
82 8 22 20 0 129
v z z h h h z z
z h h h h h
z h h z
z z h h zh z z h
      
     
   
        
       
   
  
  

 
Minimum-jerk Lagrangian in Frenet-Serret coordinates:
Derive Euler-Lagrange equation:
Or explicitly:

19. Derivation of two-thirds power law.
Huh & Sejnowski (2015) PNAS

 
 
   
4 6 2 3 4
3
2 2 2
2 2
5 2 30 10 25 82 40
2 14 90 12
20 55 75 15
82 8 22 20 0 129
v z z h h h z z
z h h h h h
z h h z
z z h h zh z z h
      
     
   
        
       
   
  
  

 
  0
a
e 
  
 
 
0
logh a
 
 

 
  0
b
v ev 
 
 
 
0
log
v
v b
v

  
Euler-Lagrange equation:
Spiral path:
or
Assume a solution in an exponential form:
or
 
    
 
4 6 2 3 4 3 2 2
0
4 6
4 6
0 5 25 82 40 90 12 55
(constant)
75a b
a b
v e a ab b b a a b a
e


  

       
 
Substituting the exponential forms into the Euler-Lagrange equations:
2
3
b a 

20. Model predicts the power law in spiral movements.
Huh & Sejnowski (2015) PNAS
  0
a
e 
  
 
2
3
0ev v




therefore, 2/3
v 

That’s what was found in experiment!

21. Scaling law with figure-dependent exponents: model prediction and
experimental confirmation.
Huh & Sejnowski (2015) PNAS
   sinh  

22. Optimization with equality constraints: Lagrange multiplier method.
Minimize a function f(x) under a constraint g(x)=0.
0
J f g

  
  
  
x x x
 0
J
g



  x
Image source: Wikipedia
     ,J f g  x x x
λ: Lagrange multiplier
Necessary condition:

23. Karush-Kuhn-Tucker (KKT) condition for constrained optimization.
Minimize a function f(x) under an inequality condition .
     ,J f g  x x x
 
 
0
0
0
0
J f g
g
g



  

  
 



x
x
x x
x
Image source: Wikipedia
  0g x
λ: Lagrange multiplier
KKT condition:

24. Optimization under dynamic constraint: Pontryagin’s minimum principle.
 ,x f x u
     0
,
ft
fJ g dt   u x x u
       
   
   
0 0
0
0
T
T T
T
, , , ( , )
, ,
, ( , )
f f
f
f
t t
t
t
f
f
f
J g dt dt
g dt
dt
    
      
     
 


x u p x x u p x f x u
x x u p f x u p
x xu p
x
x p
Minimize:
under constraint of EOMs:
     T
, , , ,g x u p x p ufu xHamiltonian

25. Optimization under dynamic constraint: Pontryagin’s minimum principle.
     
   
       0
0
T T
TT T
, , , , , ,
, , , ,
f
f
f
f
t
t
f f
T
t
t
x
J J J
dt
dt
   

    
   

    
    
        
 
       
           
    

   
 
      


x u p
x u p x u p p x
x p u
x x p
x u p x u p x u p
x x x
x u p p x
u
p x
p p x
 
T
,
0
g


  
  

 


   

p
f
x x x
x f x u
p
u
p
 
f
f
t t
t




x
p
EOMs
Terminal condition

26. Smoothness criterion: Minimum-torque change model.
 
T
1 2 1 2 1 2     x
 
 
 
2
2
2
2 2 2 2
1
1
1 2 1 11
2 1
1
1
2
2
,
,
, ,
,
, ,
f
f
u
u


   
    


                                 
x f x u
  0 0
T T1 1
2 2
f ft t
J dt dt  τu u uτ
   T T1
, , ,
2
 x u p u u p f x u
State vector
EOMs
Torque-change cost
Uno, Kawato & Suzuki (1989) Biol Cybern

27. Smoothness criterion: Minimum-torque change model.
Uno, Kawato & Suzuki (1989) Biol Cybern
 
T
T
,
0
 
 
    
 
 



 
 




 
 
 
x f x
p
f
x x
f
u
u u
u
p p
p
  00 x x
  00 p p
Initial condition for x:
Initial condition for p:
p0 must be chosen so that   .fft x x

28. Smoothness criterion: Minimum-torque change model.
Uno, Kawato & Suzuki (1989) Biol Cybern
Experiment Model

29. Smoothness criterion: Minimum-torque change model.
Uno, Kawato & Suzuki (1989) Biol Cybern
Experiment Model

30. Smoothness criterion: Minimum-torque change model.
Uno, Kawato & Suzuki (1989) Biol Cybern
Experiment Model

31. Is movement planning in extrinsic or intrinsic space?
Wolpert et al. (1993) Exp Brain Res
Experiment
Visual perturbation experiment:
MJ prediction: adapted path under visual perturbation
MTJ prediction: non-adapted path under visual perturbation.

32. Summary
• Human movements exhibit a variety of invariant features
(straight paths, power law, Fitts’ law, main sequence, …).
• Those invariant features are explained in terms of
optimality principles.
• There are mathematical methods for solving
optimization problems (calculus of variation, Lagrange
multiplier methods, Pontryagin’s minimum principle,
etc.).
• Smoothness in trajectory or joint torques is one of the
most successful criterion for reaching movements.

33. References
• Morasso, P. (1981). Spatial control of arm movements. Experimental Brain Research, 42(2), 223-227.
• Lacquaniti, F., Terzuolo, C., & Viviani, P. (1983). The law relating the kinematic and figural aspects of drawing
movements. Acta Psychologica, 54(1), 115-130.
• Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of
movement. Journal of Experimental Psychology, 47(6), 381.
• Bahill, A. T., Clark, M. R., & Stark, L. (1975). The main sequence, a tool for studying human eye movements.
Mathematical Biosciences, 24(3), 191-204.
• Flash, T., & Hogan, N. (1985). The coordination of arm movements: an experimentally confirmed mathematical
model. The journal of Neuroscience, 5(7), 1688-1703.
• Viviani, P., & Flash, T. (1995). Minimum-jerk, two-thirds power law, and isochrony: converging approaches to
movement planning. Journal of Experimental Psychology: Human Perception and Performance, 21(1), 32.
• Huh, D., & Sejnowski, T. J. (2015). Spectrum of power laws for curved hand movements. Proceedings of the
National Academy of Sciences, 112(29), E3950-E3958.
• Uno, Y., Kawato, M., & Suzuki, R. (1989). Formation and control of optimal trajectory in human multijoint arm
movement. Biological Cybernetics, 61(2), 89-101.
• Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995). Are arm trajectories planned in kinematic or dynamic
coordinates? An adaptation study. Experimental Brain Research, 103(3), 460-470.
• Flanagan, J. R., & Rao, A. K. (1995). Trajectory adaptation to a nonlinear visuomotor transformation: evidence of
motion planning in visually perceived space. Journal of neurophysiology, 74(5), 2174-2178.

34. Exercise
• Point-to-point minimum-jerk solution: For given initial
and final positions, draw a minimum-jerk trajectory (path
and velocity).
• Via-point minimum-jerk solution: Find a via-point
trajectory by determining the twelve coefficients with
given boundary conditions.
• Write a MATLAB code to solve the two-boundary
problem of the minimum-torque change model.