We all see the cool demos of how augmented and virtual reality are going to be the future of our interaction with devices, but it often feels like there’s a significant barrier to getting started in building these types of apps. This session is here to help. If you’ve ever thought about developing for Oculus Rift S, HTC VIVE or the Windows Mixed Reality headsets, but haven’t known where to start, this is the session for you. At Azure day, I will present what I learned building a small game, set in a mysterious dungeon using Unity, the SteamVR plugin and Azure Cognitive Services. We'll explore how to import assets from the Unity store, how to implement locomotion techniques and interactions using the motion/touch controllers, including some tips for optimising performance and avoid motion sickness. Come along, there’ll be .NET code, demos and we’ll have some fun looking at how you can get started on your first VR/AR project enhanced by Azure and the intelligent cloud.

1. Getting started with Unity,
AR/VR and Azure Cognitive
Services for the .NET
developer
Davide Zordan
B: https://www.davide.dev
E: mail@davide.dev
T: @DavideZordan

2. Thanks to

3. Davide Zordan
Senior Software Engineer
AR/VR Mixed Reality Developer
Udacity VR Nanodegree graduate
MCSD, MCSA, MS, MCP
Microsoft MVP (2009-2014)

4. What is Mixed Reality?

5. The state of things as they “actually
exist” through our human senses
without any technology.
Artificially created sensory experiences of
people, environments and objects, which can
include sight, touch, hearing, and smell.
PHYSICAL REALITY VIRTUAL REALITY

6. MIXED REALITY
VIRTUAL REALITYPHYSICAL REALITY

7. PHYSICAL
REALITY
DIGITAL
REALITY
MIXED REALITY (MR)
WINDOWS MIXED REALITY HOLOGRAPHIC DEVICES (SEE-THROUGH) /
MAGIC LEAP ONE / Nreal Light / RealWare HMT-1
OCULUS RIFT S - QUEST / HTC VIVE - COSMOS / VALVE INDEX /
WINDOWS MIXED REALITY IMMERSIVE DEVICES (OPAQUE) / Varjo XR-1
AR
TODAY
VR
TODAY
Physical - Digital Reality Spectrum
*The term mixed reality was originally introduced in a 1994 paper by Paul Milgram and Fumio Kishino, "A Taxonomy of Mixed Reality Visual Displays."

8. Why Mixed Reality?

9. VR/AR as next computing platform
Estimated $80bn revenue by 2025
MR is revolutionizing many processes

10. Gaming
Video entertainment
Healthcare
Real Estate
Retail
Education
Engineering
Military

11. How can we develop for AR/VR?

12. Designing great VR experiences
Place illusion
Embodiment illusion
Plausibility illusion
Source: https://www.coursera.org/lecture/introduction-virtual-reality/introduction-to-plausibility-illusion-psi-K5PGj

14. Development tools

15. C#/.NET
C++
JavaScript
Game Engines (Unity, Unreal Engine, …)
OpenGL and Microsoft DirectX
WebVR / WebXR

16. A new way to interact

17. VR Locomotion
Eye gazing
Teleportation
Dash

18. VR interactions
Pointing
Selecting
Grabbing

19. Demo – VR locomotion and interactions

21. Adding vocal commands

22. Initialising the KeywordRecognizer

23. Shows UIs depending on voice commands

24. Integrating Azure Cognitive Services
Language Understanding (LUIS)

25. A
G
E
N
D
A

38. Demo – Speech recognition and LUIS

40. Performance tips

41. Mark not moving objects as static

42. Use baked lights

43. Avoid not performant code at runtime

44. Key takeaways:
Check performance early and often to avoid motion sickness
Test frequently using the target device
Design properly the experience for maximising the user immersion
Use cloud services to enhance the user experience (LUIS, speech, vision)

45. Summary
What is Mixed Reality
Why Mixed Reality
How to develop for AR/VR
VR Interactions
VR Locomotion
Adding vocal commands
Using Natural Language Understanding (LUIS)
Performance tips

46. “Any sufficiently advanced technology is
indistinguishable from magic”
Arthur C. Clark, Profiles of the future, 1961

47. Thanks
Davide Zordan
B: https://davide.dev
E: mail@davide.dev
T: @DavideZordan

48. References
AR / VR / Mixed Reality samples
https://github.com/davidezordan
SteamVR Unity plugin
https://github.com/ValveSoftware/steamvr_unity_plugin
Mixed Reality Toolkit
https://github.com/Microsoft/MixedRealityToolkit-Unity
Microsoft Ignite
https://8gportalvhdsf9v440s15hrt.blob.core.windows.net/ignite2017/session-presentations/BRK2404.PPTX
Goldman Sachs VR/AR report
https://www.goldmansachs.com/insights/pages/virtual-and-augmented-reality-report.html

49. References
Coursera “Introduction to Virtual Reality”
https://www.coursera.org/lecture/introduction-virtual-reality/introduction-to-plausibility-illusion-psi-K5PGj
Cognitive Services
https://azure.microsoft.com/en-us/services/cognitive-services
Language Understanding (LUIS) documentation
https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/
Oculus VR performance optimization guide
https://developer.oculus.com/documentation/pcsdk/latest/concepts/dg-performance-opt-guide
Mixed Reality Academy and Azure Services tutorial
https://docs.microsoft.com/en-us/windows/mixed-reality/tutorials