Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2U7TpbH. Colman Bryant talks about what types of new input modalities are coming online and how they can be used and combined in different ways to surpass existing approaches in terms of throughput, discoverability, accessibility, and prediction with stories and examples from Magic Leap's Interaction Lab. Filmed at qconsf.com. Colman Bryant is currently Lead Designer for the Interaction Lab at Magic Leap, an internal incubation team that explores interaction paradigms and prototypes new products and use cases for Magic Leap's Mixed Reality platform.

1. CONFIDENTIAL
QCon: Future of HCI
Multi-Modal Inputs with Magic Leap
Colman Bryant - @OldKingColman

2. InfoQ.com: News & Community Site
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
magic-leap-input-design

3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon San Francisco
www.qconsf.com

4. Colman Bryant
Formal background in Computer Science, Art, and Music with a
focus on Digital Media studies
11+ years in Game Industry working across all aspects of Game
Design - Levels, Systems, AI, Inputs, Quests, Balance, Social...
Created and Lead several games and founded the Mobile
Incubator at Hi-Rez Studios
At Magic Leap for almost 2 years, writing product goals and use
cases and leading rapid prototyping efforts for the Interaction Lab.
Nerd.
Lead Designer of Interaction Lab

5. What is the Future of
HCI?

6. CONFIDENTIAL 4
Human Computer Interaction - the study of
how humans interact with computers
Part Science - mechanics of Interaction,
informed by purpose and adapted to our
physiology
Part Art - appealing to “Human” psychology
Art with Purpose = Design
What is
HCI?

7. CONFIDENTIAL 5
To have Interactions, you must have…
Input - the human (or outside world) puts
something into the system
Output - the computer responds and relays
this to the outside world (Feedback)
The Physics of HCI
I/O

8. CONFIDENTIAL 6
Computer
Human
(or
Environment)
Computer Returns
Output
Human Provides Input
Feedback Loop

9. CONFIDENTIAL 7
To understand the trends as we head to the
Future.
We should first look at where we’ve been in
the Past.
What’s the Future of
I/O?

10. CONFIDENTIAL 8
Definition
Affordance

11. CONFIDENTIAL 9
Definition
Input Modality

12. A Brief History of HCI
(and I/O)

13. CONFIDENTIAL 11

14. CONFIDENTIAL 12

15. CONFIDENTIAL 13

16. CONFIDENTIAL 14

17. CONFIDENTIAL 15

18. CONFIDENTIAL 16

19. CONFIDENTIAL 17

20. CONFIDENTIAL 18

21. CONFIDENTIAL 19

22. CONFIDENTIAL 20

23. CONFIDENTIAL 21

24. CONFIDENTIAL 22

25. CONFIDENTIAL 23

26. CONFIDENTIAL 24

27. CONFIDENTIAL 25

28. Do you see the
Trend?

29. CONFIDENTIAL
HCI Trend
Accessibility
Time
*Difficulty of
Development
(for Us)
Potential
Userbase

30. Thank You!

31. Just Kidding!

32. CONFIDENTIAL 30
Bad Design Has
Consequences

33. CONFIDENTIAL 31
Bad Design Has
Consequences

34. Usability Matters

35. Magic Leap to the rescue!

36. CONFIDENTIAL
Our Mission
We harmonize people and
technology to create a better,
more unified world.

37. CONFIDENTIAL 35
Magic Leap One
“Creator Edition”

38. CONFIDENTIAL 36

39. CONFIDENTIAL
Virtuality
37
We Are
Here!
(chillin)
Reality
Spatial
Non-Spatial

40. Digital Content that
Follows the Rules of
Reality

41. Demonstrating Concept

42. CONFIDENTIAL 40
Cool, but how do you USE it?

43. Outputs
“Fields”

44. CONFIDENTIAL
Lightfields -> “Depth Planes”

45. CONFIDENTIAL 43
Soundfields -> “Spatialized Audio”

46. Input
Modalities

47. CONFIDENTIAL 45

48. CONFIDENTIAL
Headpose

49. CONFIDENTIAL
Eye Tracking

50. CONFIDENTIAL
Control Inputs: 6DoF, Touchpad, Trigger, Bumper, Home

51. CONFIDENTIAL
Control Inputs: 3DoF + Touchpad

52. CONFIDENTIAL
Control Inputs: 6DoF

53. CONFIDENTIAL
Control Outputs: LED Halo and Haptic Motor

54. CONFIDENTIAL
Hand Tracking

55. CONFIDENTIAL
Hand Tracking

56. CONFIDENTIAL
Hand Tracking

57. CONFIDENTIAL
Hand Tracking

58. CONFIDENTIAL
Mic and Speech

59. CONFIDENTIAL
Magic Leap Mobile App

60. CONFIDENTIAL
Magic Leap Mobile App

61. CONFIDENTIAL 59
If It Can Connect...

62. Combining Input
Modalities

63. CONFIDENTIAL 61
Headpose + Hand Tracking

64. CONFIDENTIAL 62
Headpose + Phone Inputs + Object Recognition

65. CONFIDENTIAL 63
Speech + Gaze

66. CONFIDENTIAL 64
Attention, Prediction, & Intent

67. CONFIDENTIAL 65
Trends in HCI & I/O Why it Matters
How Magic Leap
Fits
Where We’re
Heading
Recap

69. CONFIDENTIAL 67
https://creator.magicleap.com/learn/guides/magickit-overview
For More Info...

70. Thank You!
cbryant@magicleap.com @OldKingColman

71. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
magic-leap-input-design