Somehow everybody only seems to be talking about the possibilities of AR, instead of the constraints and what those constraints mean for your game. One of the biggest constraints that we came across as we launched Mobile AR game titles to public and beta users, was that it's really hard to use touch on the screen as a Control when you're using the same screen as your View. This made us go back to the drawing board and completely rethink how to do controllers in Augmented Reality. We prototyped a string of crazy demos to explore this space. This talk covers those outlandish demos, and how those possibilities panned out in our games. It also features the gotchas that are unique to AR, and the sweet spot that innovates new game controllers for the era of Augmented Reality. We will share demos of the new game controller ideas we propose, and have used in our games - * Play drums with your full body in Augmented Reality * Become a vacuum and suck up the mess in your room with this face gestures + augmented reality app * Use hand gestures and real world objects to interact with virtual world for storytelling

1. No Touching!
Rethinking Game Controllers in AR
Jyoti Bishnoi
Not A Bot Studio
@methodiva

2. Games need
immersion

3. Immersion needs
suspension of disbelief

4. those moments can
make or break the illusion
Interactions pull me out

5. Interactions in AR
must go beyond 2d gestures

6. As an example
Tapping on virtual objects in 3D

7. sounds deceptively simple

8. but, can confuse in practice

9. Controllers are an indirection
unless...

10. Game steps into your world

11. What does that mean?

12. My perspective is my Control

13. Perspective
AR can use my position,
orientation, and motion

14. Perspective
It lets me paint in 3D,
physically outrun zombies,
and walk into space

15. is immersive
Perspective

16. encourages moving about
Perspective

17. brings the game world into our world
Perspective

18. Perspective
Don’t forget - Field of view is
limited near proximity triggers.

19. My object is my Control

20. Object
AR can track real world objects
with Object Recognition

21. Object
It lets me tell stories in AR
with real hand puppets, and throw
physical dice for a virtual board game.

22. demo
Object

24. Object
makes interaction more
physical and tactile

25. Objects
enables persistent imagination

26. Objects
can act as a bridge between
the virtual and real world

27. Don’t forget - Physical objects can be
logistically cumbersome, and
difficult to hold
Object

28. My face is my control

29. Face
AR can track my gaze,
eyebrows, blink, mouth,
face orientation

30. It lets me navigate a menu
with my gaze, drive a car
with frowns and smiles,
and vacuum with my mouth!
Face

31. demo
Face

32. Face
allows hands-free interaction

33. Face
allows both trigger actions
& continuous control

34. Face
Dont’t forget - Facial gestures are
hard to sustain for long periods of time

35. My body is my control

36. AR tracks my hands,
shoulders, legs, joints
Body

37. Body
it lets me add special effects
to dance moves,
and play drums in mid-air!

38. Body
demo

39. Body
makes for a visceral
full-body experience

40. Body
allows for complex gestures,
high-degrees of freedom

41. But,
Body
Don’t forget - Body pose recognition
only works with back-camera on iOS

43. I am the controller
You are the controller!

44. Questions?