Slides from my presentations about Virtual and Augmented Reality - October 2020.

1. Getting started with Unity and
AR/VR for the .NET developer
Davide Zordan
B: https://davide.dev
E: mail@davide.dev
T: @DavideZordan

2. Davide Zordan
Senior software engineer
MCSD, MCSA, MS, MCP
Microsoft MVP (2009-2014)

3. Agenda
Introduction to Mixed Reality
VR interactions and locomotion using Unity and SteamVR
Adding vocal commands
Hand-tracking
Developing for AR

4. What is Mixed Reality?

5. The state of things as they “actually
exist” through our human senses
without any technology.
Artificially created sensory experiences of
people, environments and objects, which can
include sight, touch, hearing, and smell.
PHYSICAL REALITY VIRTUAL REALITY

6. MIXED REALITY
VIRTUAL REALITYPHYSICAL REALITY

7. PHYSICAL
REALITY
DIGITAL
REALITY
MIXED REALITY (MR)
WINDOWS MIXED REALITY HOLOGRAPHIC DEVICES (SEE-THROUGH) / MAGIC
LEAP ONE / Nreal Light / RealWare HMT-1 / LYNX-R1 / Facebook Project Aria
OCULUS RIFT S – QUEST 2 / HTC VIVE - COSMOS / VALVE INDEX /
WINDOWS MIXED REALITY IMMERSIVE DEVICES (OPAQUE) / Varjo XR-1
AR
TODAY
VR
TODAY
Physical - Digital Reality Spectrum
*The term mixed reality was originally introduced in a 1994 paper by Paul Milgram and Fumio Kishino, "A Taxonomy of Mixed Reality Visual Displays."

8. Why Mixed Reality?

9. VR/AR as next computing platform
Estimated $80bn revenue by 2025 *
MR is revolutionizing many processes
Source: https://www.goldmansachs.com/insights/pages/virtual-and-augmented-reality-report.html

10. Gaming
Video entertainment
Healthcare
Real Estate
Retail
Education
Engineering
Military

11. How can we develop for AR/VR?

12. Designing great VR experiences
Place illusion
Plausibility illusion
Embodiment illusion
Source: https://www.coursera.org/lecture/introduction-virtual-reality/introduction-to-plausibility-illusion-psi-K5PGj

14. Development tools

15. Game Engines (Unity, Unreal Engine 4)
OpenGL, Microsoft DirectX, Vulkan
WebVR / WebXR (A-Frame, Babylon.js)
C#/.NET
C++
JavaScript

16. A new way to interact

17. VR interactions
Pointing
Selecting
Grabbing

18. VR Locomotion
Eye gazing
Teleportation
Dash

19. DEMO
VR locomotion and custom interactions

21. Adding vocal commands:
• Unity KeywordRecognizer
• Natural Language Understanding

22. 1 – Using the KeywordRecognizer

23. 2 – Shows UIs depending on voice commands

32. DEMO
Speech recognition and multiple headsets
support

34. Performance tips

35. 1 - Mark not moving objects as static

36. 2 – Use baked lights

37. 3 – Avoid not performant code at runtime

38. Implementing Hand tracking

39. Image source: www.oculus.com/quest

40. Mixed Reality Toolkit for Unity
• https://aka.ms/MRTK
MRTK extensions for Oculus Quest
• https://github.com/HoloLabInc/MRTKExtensionForOculu
sQuest
• https://github.com/provencher/MRTK-Quest
https://aka.ms/MRTK
https://github.com/provencher/MRTK-Quest

41. DEMO
Hand tracking using Oculus Quest and
Mixed Reality Toolkit

43. Developing for AR

44. HoloLens 2
Source: https://news.microsoft.com/microsoft-at-mwc19/

51. Microsoft Azure mixed reality services:
• Remote rendering
• Spatial anchors
• Object anchors

52. Summary
What is Mixed Reality
Why Mixed Reality
How to develop for AR/VR
VR Interactions
VR Locomotion
Adding vocal commands
Performance tips
Hand-tracking
Developing for AR
Mixed Reality Services

53. Key takeaways:
Check performance early and often to avoid motion sickness
Test frequently using the target device
Design properly the experience for maximising the user immersion
Use cloud services to enhance the user experience (speech, vision, anchors)

54. “Any sufficiently advanced technology is
indistinguishable from magic”
Arthur C. Clark, Profiles of the future, 1961

55. Thanks
Davide Zordan
B: https://davide.dev
E: mail@davide.dev
T: @DavideZordan

56. References
AR / VR / Mixed Reality samples
https://github.com/davidezordan
SteamVR Unity plugin
https://github.com/ValveSoftware/steamvr_unity_plugin
Mixed Reality Toolkit
https://aka.ms/MRTK
Unity XR Interaction Toolkit
https://docs.unity3d.com/Packages/com.unity.xr.interaction.toolkit@0.9/manual/index.html
Unity XR Interaction Toolkit examples
https://github.com/Unity-Technologies/XR-Interaction-Toolkit-Examples
Goldman Sachs VR/AR report
https://www.goldmansachs.com/insights/pages/virtual-and-augmented-reality-report.html

57. References
Coursera “Introduction to Virtual Reality”
https://www.coursera.org/lecture/introduction-virtual-reality/introduction-to-plausibility-illusion-psi-K5PGj
Cognitive Services
https://azure.microsoft.com/en-us/services/cognitive-services
Oculus VR performance optimization guide
https://developer.oculus.com/documentation/pcsdk/latest/concepts/dg-performance-opt-guide
Mixed Reality Academy and Azure Services tutorial
https://docs.microsoft.com/en-us/windows/mixed-reality/tutorials
Unity: the most common mistakes to avoid
https://unity3d.com/how-to/unity-common-mistakes-to-avoid
Microsoft Ignite
https://8gportalvhdsf9v440s15hrt.blob.core.windows.net/ignite2017/session-presentations/BRK2404.PPTX