Kinect is moving from a gaming device to a new way to interact with our digital world. While touch has taken a disproportional amount of our attention, we're seeing a flood of devices that use sensors to collect feedback from the world around us. And the Kinect is the mother of all sensors with a high visual resolution, audio, and advances in areas like face detection - how would our would look if we focused on gesture and sensor designs?

1. Hacks - Beyond Just Games

2. JEREMY JOHNSON

3. UX DESIGNER

4. DIofRuseECTORpex erience
r

5. GAMER

6. @jeremyjohnson
www.jeremyjohnsononline.com

7. 2009

8. http://www.avenue-films.com/project-natal-the-journey.html

9. Microsoft's motion-sensor Kinect came out of the
company's research lab in Beijing, away from the
company's day-to-day business, she notes. "There was
a steel umbrella put over it, to keep out the rest of the
organization. Then Microsoft Research threw Kinect
over the wall to the Xbox group, which created their
own steel umbrella to shield it. They added some
scaffolding, but this was an innovation that was mostly
done in-house. It can happen in a big company. Of
course, this is entirely the reverse of the way Office/
Windows development happens."
boyd argues that the best place "to create steel
walls is in businesses outside your core cash cows.
The role of a big company is to keep the cash cow
rolling, and then build the new."
http://www.fastcompany.com/magazine/162/generation-flux-danah-boyd

11. 18 Million
“Kinect, which launched slightly over a year ago, has sold an
impressive 18 million units worldwide. (At this rate, Kinect will
outsell the original Xbox by next year.)”
http://www.shacknews.com/article/71904/xbox-360-reaches-66-million-sales-kinect-at-18-million

12. GAMES

14. http://www.jeremyjohnsononline.com/2011/10/18/gamification-101-refresh-dallas-meeting/

15. “There’s a lot to learn from games. There are 30 years
of design lessons and thousands of products on the
market. Games have more concrete lessons to teach
application designers about learning, feedback, user
motivation and social systems than almost any field of
study.”
- Danc
http://www.lostgarden.com/2011/04/what-heck-happened-to-clippy-ribbon.html
http://www.officelabs.com/projects/ribbonhero2/Pages/default.aspx
http://www.lostgarden.com/2010/05/wordcamp-2010-why-we-turned-microsoft.html

16. http://www.engadget.com/2010/12/09/kinect-finally-fulfills-its-minority-report-destiny-video/

17. “Kinect Hacks”

18. http://www.kinecthacks.com/

19. http://www.youtube.com/user/KinectHacks/featured

20. “Microsoft chief
Steve Ballmer
narrowed down
the "early 2012"
launch window
for Kinect on the
Windows PC
during
Microsoft's CES
2012 keynote
today: February
1, 2012.”
http://www.joystiq.com/2012/01/09/kinect-for-windows-leers-at-february-1-launch-date/

21. http://techcrunch.com/2012/01/27/secret-windows-8-weapon-kinect-built-into-your-laptop/

22. http://www.xbox.com/en-US/Kinect/Kinect-Effect

23. UX DESIGNER

24. information architecture
interaction design
visual design
usability testing

25. desktop apps web apps RIAs
mobile apps touch apps ??

26. Interaction Choreography
by Senior Principal Design Technologist Jared Ficklin
User interaction with technology is going above the glass. You no longer need an
explicit tool or even direct manipulation to drive a user interface. With the ability of
technology, like the Microsoft Kinect, to see users’ movements in space, gestures
are being added to traditional methods in new layers of interaction. Designing for
this new layer of interaction requires new thinking about dexterity, ergonomics,
and whether someone might feel silly or offensive with certain gestures. We are
so involved in this space right now, that we’ve had to move our design
technologists’ desks to create enough room for all the hand waving design.
http://designmind.frogdesign.com/blog/frogs-2012-technology-trend-predictions.html

28. vs

29. http://www.fastcodesign.com/1665827/microsoft-unleashes-the-kinect-and-promptly-misses-the-point

31. http://www.fastcodesign.com/1665827/microsoft-unleashes-the-kinect-and-promptly-misses-the-point

34. “The innovation here is the fluidity of
experience and focus on the data, without
using traditional user interface conventions
of windows and frames. Data becomes the
visual elements and controls. Simple
gestures and transitions guide the user
deeper into content. A truly elegant and
unique experience.”
http://en.wikipedia.org/wiki/Metro_(design_language)

36. KINECT HACKS

37. http://www.thecreatorsproject.com/blog/the-board-of-awesomeness-is-a-kinect-controlled-skateboard

39. http://www.pocket-lint.com/news/43671/kinect-hack-wiimote-robot-cat

40. http://venturebeat.com/2011/11/15/microsoft-code-space-kinect/

42. http://www.kinecthacks.com/kinect-video-manipulation/

43. http://www.niemanlab.org/2011/08/mirror-mirror-the-new-york-times-wants-to-serve-you-info-as-youre-brushing-your-teeth/

44. http://mashable.com/2012/01/09/bodymetrics-augmented-reality-shopping/

45. http://www.etre.com/blog/2012/01/puppet_parade_interactive_kinect_puppets/

46. http://www.creativeapplications.net/openframeworks/interactive-wall-at-ud-openframeworks-kinect/?utm_source=feed&utm_campaign=rss-mo-more&utm_medium=rss

47. http://www.theverge.com/microsoft/2012/1/11/2700832/kinect-fusion-swivel-facecake-hands-on
http://latimesblogs.latimes.com/technology/2012/01/virtual-dressing-room-takes-the-hassle-out-of-taking-off-your-clothes.html

48. http://techcrunch.com/2012/01/16/primesense-demos-a-gesture-based-next-gen-tv-interface

49. Solar powered house saves energy with Kinect
Now, in the name of conserving energy that vision could come true with a solar
house designed to use an Xbox Kinect to switch off energy consuming devices
with simple movements.
http://dvice.com/archives/2012/01/solar-powered-h.php

50. http://dlb-network.com/2012/01/mass-effect-3-kinect-demo/

51. What now?

52. “Don’t try to figured
out how to cram
Kinect into an existing
UI paradigm, instead
design a UI paradigm
that’s from the
ground-up intended
to exploit the Kinect’s
functionality.”
http://www.teehanlax.com/labs/insights-into-kinect-ui/

53. http://www.useit.com/alertbox/kinect-gesture-ux.html

54. That there are no universal
standards for gestural interactions
yet is a problem in its own right,
because the UI cannot rely on
learned behavior. The Kinect has a
few system-wide standards,
however, which do help users.
http://www.useit.com/alertbox/kinect-gesture-ux.html

55. THE RULES Of KINECT
Explain what the player can do.
Represent what they are doing.
Make it fun to match the two.
Test your implementation.
http://www.microsoft.com/download/en/details.aspx?id=27978

56. Things to Think About...
Distance and Environment
How far do they need to stand? How far do they think they need to stand?
Environment design of the area. Eyesight, size of the UI.
Ergonomics
Is it comfortable? Age? People with disabilities? Common movements vs.
uncommon. How long will they interact?
Fashion
Do they look silly? Will someone of a certain age/race/gender use this? What’s
acceptable for an Avatar?
UI Patterns
What do they know from click and touch interfaces? Is there something more
natural? Try and unlearn, and imagine.
3D Space
Is it close or far away? What do we infer from spatial positioning? Can you get
people to interact in 3D space on a 2D screen?

57. Things to Think About...
Screen Resolution
What’s the maximum amount of objects you can fit on a screen? What’s the
minimum size of an object need to be on the screen?
Privacy
Can you tailor to age? Race? Gender? What is helpful and what is potentially
scary? Can you keep a snapshot for marketing purposes?

58. THANKS!
jeremy@23hd.com
@ jeremyjohnson
www.jeremyjohnsononline.com