Details: https://koyama.xyz/project/one-hop_motion_transition/

1. Precomputed Optimal One-Hop Motion Transition
for Responsive Character Animation
Yuki Koyama & Masataka Goto
The Visual Computer (CGI )

2. Target Problem

3. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
State Machine-Based Control for Interactive Apps
State machine-based control is widely used.
Each state is associated with a single motion clip (in the basic usage).
In response to run-time user input, the state is changed.

4. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Source (src) motion Destination (dst) motion
Transition
Triggered by user input
at an arbitrary frame
Goal: Achieve High-Quality Motion Transition

5. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Direct Approach (Current Practice)
■ Direct approach:
• Begin transition immediately
after receiving the trigger
• Directly interpolate the src and
dst motions
■ Problem:
• Visual artifacts (e.g., foot skate)

6. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Requirements for a New Method
■ A new method should
• Be compatible with the state
machine-based workflow
• Run in interactive real-time
apps
• Finish the transition as
quickly as possible

7. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Requirements for a New Method
■ A new method should
• Be compatible with the state
machine-based workflow
• Run in interactive real-time
apps
• Finish the transition as
quickly as possible

8. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Requirements for a New Method
■ A new method should
• Be compatible with the state
machine-based workflow
• Run in interactive real-time
apps
• Finish the transition as
quickly as possible

9. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Requirements for a New Method
■ A new method should
• Be compatible with the state
machine-based workflow
• Run in interactive real-time
apps
• Finish the transition as
quickly as possible
Quick

10. Overview of Our Method

11. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Our Approach: “One-Hop” Motion Transition

12. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Example of Our Results

13. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Overview of This Work
■ Technical Challenges:
. How should we choose a hop
motion and arrange it?
. How can we enable this in
real-time apps?
■ Our Solutions:
. Formulating a search
problem
. Solving the searches in
precomputation

14. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Overview of This Work
■ Technical Challenges:
. How should we choose a hop
motion and arrange it?
. How can we enable this in
real-time apps?
■ Our Solutions:
. Formulating a search
problem
. Solving the searches in
precomputation

15. Related Work

16. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Related Frameworks
[Holden+ , ][Kovar+ , ]
Statistical modelsMotion graphs
[Zadziuk , ]
Motion matching
➡ These are incompatible with the state machine-based workflow

17. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Related Frameworks
[Holden+ , ][Kovar+ , ]
Statistical modelsMotion graphs
[Zadziuk , ]
Motion matching
➡ These are incompatible with the state machine-based workflow

18. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Related Frameworks
[Holden+ , ][Kovar+ , ]
Statistical modelsMotion graphs
[Zadziuk , ]
Motion matching
➡ These are incompatible with the state machine-based workflow

19. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Related Frameworks
[Holden+ , ][Kovar+ , ]
Statistical modelsMotion graphs
[Zadziuk , ]
Motion matching
➡ These are incompatible with the state machine-based workflow

20. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Related Frameworks
[Holden+ , ][Kovar+ , ]
Statistical modelsMotion graphs
[Zadziuk , ]
Motion matching
➡ These are incompatible with the state machine-based workflow

21. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
State Machine-Compatible Approach [Ikemoto+07]
[Ikemoto+ ]
■ Similar to ours:
• They insert “hop” (intermediate)
motions to achieve natural
transitions
■ Key difference:
• We consider not only naturalness
but also quickness to finish the
overall transition (explained later)

22. 1. Formulating a Search Problem

23. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Trigger
src
dst

24. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Trigger
src
dst
(Direct approach)

25. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Trigger
src
dst
Unnatural
transition
(Direct approach)

26. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Trigger
src
dst

27. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Trigger
src
hop
dst

28. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Trigger
src
dst
More natural
transitions
hop

29. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Trigger
src
dst
hop

30. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Trigger
src
dst
hop
Introducing
“stay durations”

31. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Even more natural
transitions
Trigger
src
dst
hop

32. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
One-Hop Transition Setting
Trigger
src
dst
hopOverall transition
duration

33. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Search Variables
Trigger
src
dst
hop

34. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Search Variables
Trigger
src
dst
hop
. Which motion clip to use as hop

35. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Search Variables
Trigger
src
dst
hop
. Which motion clip to use as hop
. Time offset of hop

36. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Search Variables
Trigger
src
dst
hop
. Which motion clip to use as hop
. Time offset of hop . Stay duration [src]
. Stay duration [hop]

37. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Search Objective
T⇤
t = arg min
Tt
{ S(Tt) + w R(Tt) }
<latexit sha1_base64="T4T3GkSkIxMtJugrGSYghaBpGts=">AAACPXicbZBLSwMxEMezPmt9VT16CRbBB5TdKigFoeDFg4eqrQrdumTTtAaz2SWZVcuyX8yL38GbNy8eFPHq1XTbQ30MhPwy/xky8/cjwTXY9rM1Nj4xOTWdm8nPzs0vLBaWls91GCvKGjQUobr0iWaCS9YADoJdRoqRwBfswr857OsXt0xpHso69CLWCkhX8g6nBEzKK9TrHlwlWyk+wC6we0iI6qbYDbj0EiMZTLBbcSv4bMM8Nwe8PbjusivD0xHVTb1C0S7ZWeC/4AyhiIZR8wpPbjukccAkUEG0bjp2BC0zC3AqWJp3Y80iQm9IlzUNShIw3Uqy7VO8bjJt3AmVORJwlh3tSEigdS/wTWVA4Fr/1vrJ/7RmDJ39VsJlFAOTdPBRJxYYQty3Ere5YhREzwChiptZMb0milAwhueNCc7vlf/Cebnk7JTKJ7vF6vHQjhxaRWtoAzloD1XREaqhBqLoAb2gN/RuPVqv1of1OSgds4Y9K+hHWF/fuYeqBQ==</latexit>

38. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Search Objective
Smoothness cost:
Encourage each
transition to be as
natural-looking as
possible
T⇤
t = arg min
Tt
{ S(Tt) + w R(Tt) }
<latexit sha1_base64="T4T3GkSkIxMtJugrGSYghaBpGts=">AAACPXicbZBLSwMxEMezPmt9VT16CRbBB5TdKigFoeDFg4eqrQrdumTTtAaz2SWZVcuyX8yL38GbNy8eFPHq1XTbQ30MhPwy/xky8/cjwTXY9rM1Nj4xOTWdm8nPzs0vLBaWls91GCvKGjQUobr0iWaCS9YADoJdRoqRwBfswr857OsXt0xpHso69CLWCkhX8g6nBEzKK9TrHlwlWyk+wC6we0iI6qbYDbj0EiMZTLBbcSv4bMM8Nwe8PbjusivD0xHVTb1C0S7ZWeC/4AyhiIZR8wpPbjukccAkUEG0bjp2BC0zC3AqWJp3Y80iQm9IlzUNShIw3Uqy7VO8bjJt3AmVORJwlh3tSEigdS/wTWVA4Fr/1vrJ/7RmDJ39VsJlFAOTdPBRJxYYQty3Ere5YhREzwChiptZMb0milAwhueNCc7vlf/Cebnk7JTKJ7vF6vHQjhxaRWtoAzloD1XREaqhBqLoAb2gN/RuPVqv1of1OSgds4Y9K+hHWF/fuYeqBQ==</latexit>

39. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Search Objective
Smoothness cost:
Encourage each
transition to be as
natural-looking as
possible
Responsiveness cost:
Encourage the overall
transition to finish as
quickly as possible
T⇤
t = arg min
Tt
{ S(Tt) + w R(Tt) }
<latexit sha1_base64="T4T3GkSkIxMtJugrGSYghaBpGts=">AAACPXicbZBLSwMxEMezPmt9VT16CRbBB5TdKigFoeDFg4eqrQrdumTTtAaz2SWZVcuyX8yL38GbNy8eFPHq1XTbQ30MhPwy/xky8/cjwTXY9rM1Nj4xOTWdm8nPzs0vLBaWls91GCvKGjQUobr0iWaCS9YADoJdRoqRwBfswr857OsXt0xpHso69CLWCkhX8g6nBEzKK9TrHlwlWyk+wC6we0iI6qbYDbj0EiMZTLBbcSv4bMM8Nwe8PbjusivD0xHVTb1C0S7ZWeC/4AyhiIZR8wpPbjukccAkUEG0bjp2BC0zC3AqWJp3Y80iQm9IlzUNShIw3Uqy7VO8bjJt3AmVORJwlh3tSEigdS/wTWVA4Fr/1vrJ/7RmDJ39VsJlFAOTdPBRJxYYQty3Ere5YhREzwChiptZMb0milAwhueNCc7vlf/Cebnk7JTKJ7vF6vHQjhxaRWtoAzloD1XREaqhBqLoAb2gN/RuPVqv1of1OSgds4Y9K+hHWF/fuYeqBQ==</latexit>

40. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Search Objective
Smoothness cost:
Encourage each
transition to be as
natural-looking as
possible
Responsiveness cost:
Encourage the overall
transition to finish as
quickly as possible
Weight:
Control the
trade-off
T⇤
t = arg min
Tt
{ S(Tt) + w R(Tt) }
<latexit sha1_base64="T4T3GkSkIxMtJugrGSYghaBpGts=">AAACPXicbZBLSwMxEMezPmt9VT16CRbBB5TdKigFoeDFg4eqrQrdumTTtAaz2SWZVcuyX8yL38GbNy8eFPHq1XTbQ30MhPwy/xky8/cjwTXY9rM1Nj4xOTWdm8nPzs0vLBaWls91GCvKGjQUobr0iWaCS9YADoJdRoqRwBfswr857OsXt0xpHso69CLWCkhX8g6nBEzKK9TrHlwlWyk+wC6we0iI6qbYDbj0EiMZTLBbcSv4bMM8Nwe8PbjusivD0xHVTb1C0S7ZWeC/4AyhiIZR8wpPbjukccAkUEG0bjp2BC0zC3AqWJp3Y80iQm9IlzUNShIw3Uqy7VO8bjJt3AmVORJwlh3tSEigdS/wTWVA4Fr/1vrJ/7RmDJ39VsJlFAOTdPBRJxYYQty3Ere5YhREzwChiptZMb0milAwhueNCc7vlf/Cebnk7JTKJ7vF6vHQjhxaRWtoAzloD1XREaqhBqLoAb2gN/RuPVqv1of1OSgds4Y9K+hHWF/fuYeqBQ==</latexit>

41. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Search Objective
Smoothness cost:
Encourage each
transition to be as
natural-looking as
possible
Responsiveness cost:
Encourage the overall
transition to finish as
quickly as possible
T⇤
t = arg min
Tt
{ S(Tt) + w R(Tt) }
<latexit sha1_base64="T4T3GkSkIxMtJugrGSYghaBpGts=">AAACPXicbZBLSwMxEMezPmt9VT16CRbBB5TdKigFoeDFg4eqrQrdumTTtAaz2SWZVcuyX8yL38GbNy8eFPHq1XTbQ30MhPwy/xky8/cjwTXY9rM1Nj4xOTWdm8nPzs0vLBaWls91GCvKGjQUobr0iWaCS9YADoJdRoqRwBfswr857OsXt0xpHso69CLWCkhX8g6nBEzKK9TrHlwlWyk+wC6we0iI6qbYDbj0EiMZTLBbcSv4bMM8Nwe8PbjusivD0xHVTb1C0S7ZWeC/4AyhiIZR8wpPbjukccAkUEG0bjp2BC0zC3AqWJp3Y80iQm9IlzUNShIw3Uqy7VO8bjJt3AmVORJwlh3tSEigdS/wTWVA4Fr/1vrJ/7RmDJ39VsJlFAOTdPBRJxYYQty3Ere5YhREzwChiptZMb0milAwhueNCc7vlf/Cebnk7JTKJ7vF6vHQjhxaRWtoAzloD1XREaqhBqLoAb2gN/RuPVqv1of1OSgds4Y9K+hHWF/fuYeqBQ==</latexit>
Weight:
Control the
trade-off

42. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Evaluating Smoothness of Transitions [Kovar+02]
Dissimilar
➡ High cost for transition
(potentially non natural)
Similar
➡ Low cost for transition
(expected to be natural)

43. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Evaluating Smoothness of Transitions [Kovar+02]
High cost ➡ Unnatural transition

44. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Evaluating Smoothness of Transitions [Kovar+02]
Low cost ➡ Natural transition

45. 2. Solving the Search Problem in Precomputation

46. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Precomputation
■ Solve the search problem for every possible trigger frame and
for every possible state transition
• May take a few days depending on the dataset size, but required
only once
■ Store optimal transitions in a look-up table
• In runtime, retrieve an optimal transition by using the triggered
frame as the key

47. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Precomputation
■ Solve the search problem for every possible trigger frame and
for every possible state transition
• May take a few days depending on the dataset size, but required
only once
■ Store optimal transitions in a look-up table
• In runtime, retrieve an optimal transition by using the triggered
frame as the key

48. Results

49. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Supposed Scenario: Interactive Dancing
Let a character dance using a game pad in an improvised manner

50. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Supposed Scenario: Interactive Dancing
■ Run in Unity
• It is straightforward to implement our method
using Unity’s PlayableAPI
■ Our dance motion dataset
• Around motion clips for hop candidates
• Each motion clip has around sec
■ Use frames (= . sec) for individual transitions
• I.e., src-to-hop ( . sec) and hop-to-dst ( . sec)
■ No postprocessing applied
• E.g., footskating removal (e.g., [Ikemoto+ ])

51. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Result: “Ghost Force” Disappears by a Standing-Up Motion

52. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Other Results
https://koyama.xyz/project/one-hop_motion_transition/

53. Discussions

54. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
The Direct Transition is Included in the Search Space
When the direct transition is already good, our solver is likely to select
the one-hop transition that is equivalent to the direct transition

55. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Future Work: Positional & Orientational Constraints
■ Current behavior
• Depending on the
inserted hop motion, the
position & orientation
may be changed
■ Future work
• Add constraints in the
search

56. Summary

57. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Summary
■ Optimal one-hop motion transition
• It inserts a hop motion between the
source and destination motions in
an optimal way
■ Consider both smoothness and
responsiveness
• Try to maximize both in balance
■ Can be precomputed
• Run in real-time apps

58. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Summary
■ Optimal one-hop motion transition
• It inserts a hop motion between the
source and destination motions in
an optimal way
■ Consider both smoothness and
responsiveness
• Try to maximize both in balance
■ Can be precomputed
• Run in real-time apps

59. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Summary
■ Optimal one-hop motion transition
• It inserts a hop motion between the
source and destination motions in
an optimal way
■ Consider both smoothness and
responsiveness
• Try to maximize both in balance
■ Can be precomputed
• Run in real-time apps

60. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Summary
■ Optimal one-hop motion transition
• It inserts a hop motion between the
source and destination motions in
an optimal way
■ Consider both smoothness and
responsiveness
• Try to maximize both in balance
■ Can be precomputed
• Run in real-time apps

61. The paper, videos, and slides are available at
https://koyama.xyz/project/one-hop_motion_transition/

63. Appendix

64. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Changing Weight

65. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Future Work: Optimizing Blending Durations (c.f., [Wang+08])
Trigger
src
dst
hop
Blending durations
(we assumed fixed durations)

66. Yuki Koyama and Masataka Goto | Precomputed Optimal One-Hop Motion Transition for Responsive Character Animation | The Visual Computer (CGI )
Future Work: Blend Tree
Blend Tree:
Each state can also be
associated a dynamic
mixture of motion clips
Challenge:
How can we
precompute all the
possible blends and
store the results?