2. When I was in engineering school, we had software architecture classes and projects.
Without any experience, this is a very abstract topic
When I was an intern at Discreet (Autodesk, 3DS Max, Flame, Inferno, movies sfx), was very impressed with the “architect”, and wanted to be like him !
Turns out it takes a lot of experience to be a good architect, so share my experience architecting MiddleVR and show that the philosophy that you choose, consciously or not, actually
drives your architecture
4. (French railways)
“Creating Virtual Reality Games:!
The Fundamentals”
Founder & President
Sébastien ‘Cb’ Kuntz
Also board member of the national French VR association AFRV (http://www.afrv.fr)
and founded the VR Geeks association (http://www.vrgeeks.org)
5. And it seems I have been interested in head-mounted displays for longer that I remember.
6. Zone hunter
(Virtuality-1994)
I fell in love with VR on a field trip in London when I was a teenager.
The experience was far from perfect: the HMD was bulky, one display was blurry, but being able look around and shoot where I wanted incredible.
I quickly decided I wanted to make a career in VR.
7. FRENCH RAILWAYS - SNCF
• Polhemus magnetic tracker, treadmill, 5DT Glove
• C++ / OpenSceneGraph /VRPN / OpenAL
Fiacre: train the operators at maintenance of equipments on the railway tracks.
I was in charge of all the VR engine and application development
!
(sorry I can’t put SNCF videos online)
8. FRENCH RAILWAYS - SNCF
Simurat: check wagons before they leave on the tracks
9. VIRTOOLS
After 4 years I was hired by Virtools as their VR lead engineer.
Virtools was a great tool to create 3D applications.
Simple to learn thanks to the building blocks programming scheme, and many non-programmers, like designers and even farmers, created great
applications.
10. VIRTOOLSVR PACK
I was in charge of adding VR functionality's to Virtools engine, coding, writing documentation, support customers
11. I often say that we are at the prehistory of VR.
Last week was the Big bang of VR. I could test Oculus DK2, Sony Morpheus and Valve’s HMD.
The future is coming quickly.
13. Virtual reality
Presence in a virtual world
Interesting because:
- natural interactions: no interface to learn. if we want to look under a table, no button to press, just move your body.
- natural reactions & emotions : if I throw a ball at you, you will have the reflex to grab it. if you are afraid of heights, you will have this fear in VR.
15. • Build hardware
To create and maintain presence,
you need to ...
• Build software
• Build user experience
Most people forget about taking time to build the user experience. Test with users !
16. The challenges of creating a VR software
We already have great hardware, but we don’t know how to make the most of it.
17. “VR is mainly a software problem”!
!Jaron Lanier! !
Software is like a fuel for your car.
Without any fuel, you will not go very far, even if you have a Porsche.
19. Device management
Tracker 1! Driver Tracker 1
Application
Tracker 2!
!
Driver Tracker 2
The problem is that each driver gives access to similar data in a different way so you need to have specific part
of programs for similar drivers.
20. code for one driver
if you switch to another device, another driver:
21. This is a completely different code which gives exactly the same information but for another device.
23. Correct perspective
(c) Johnny Lee
https://www.youtube.com/watch?v=Jd3-eiid-Uw
!
3D applications assume that the user is in front of the screen.
If you move in front of the screen, it looks like a 2d painting
!
If the application knows where the user is, the screen becomes a window on the virtual world
(perceived depth without stereoscopy)
24. Correct perspective
In general 3D engines allow you to do that but you have to do all the maths by yourself
Necessary for every big VR wall, anytime stereoscopy, HMDs, Caves...
!
25. Stereoscopy
Even after you have computed your stereo images, you have different ways to display them.
It can be active stereo, passive stereo, autostereo etc.
26. Warping & Blending
Then if you have a curved screen, your perspective will have to be distorted.
27. Warping & Blending
Warping is about getting a correct perspective for screens that are not planar (curved)
Blending is about managing the overlapping regions between several projectors.
43. Asset store
• Buy / sell assets!
• 3D models!
• Textures!
• Shaders!
• Scripts!
• Editor plugins
44. Desktop
Unity is a great tool but it doesn’t natively support VR systems easily.
45. • Build hardware
To create and maintain presence,
you, the author, need to ...
• Build software
• Build user experience
Our goal is to minimize the time you spend on building the software so that you can focus on building the user experience.
46. In my mind, VR is not that difficult. Doesn’t have to be.
A part is that the hardware is hard to use, but a bigger part is that the software is hard to use mostly by design.
47. In 2010 sent an e-mail to Roland Blach asking for all the SEARIS papers and studied them attentively
Software Engineering and Architectures for Realtime Interactive Systems Working Group
49. VRPN
In VRPN, you first have to edit a configuration file, understandable by computer scientists only.
Then you have to understand how it works: callbacks,
terminology is also a bit obscur (analog..)
VRPN is only solving the devices management, and as we have seen, there are a lot of other issues when creating a VR app.
50. VR Juggler
VR Juggler needs to understand a lot of concept: proxy device? derive of off OpenGL::App ?
My main problem is that you have to bend your application to their design
51. VIRTOOLSVR PACK
VRPack’s configuration file is even more complex than VRPN.
Even working at Virtools, it took me quite a while to understand how it works !
Switching configuration files is also complicated.
52. Worldviz Vizard
• Based on OpenSceneGraph!
• Python
Vizard is a really nice environment but still a bit too complex to use.
54. The need for better
VR software tools
• Hardware getting cheaper!
• But engineers are not getting cheaper!
• Need to be more efficient and spend less
time creating apps!
• Democratization!
• VR developers are not all engineers anymore
55. Goals
• Simplify VR!
• Simplify creation of VR applications!
• Simplify deployment of VR applications!
• Create new VR tools!
• Adapting existing 3D applications
60. Once you have exported your application, you can simply add it to the list of Simulations in MiddleVR’s gui.
Then you can pick any configuration and simply click Run and your application will be dynamically reconfigured for this configuration
68. Simply configure your system
Configure what is happening in the real life.
Seeing things happening in real life changes a lot.
Can easily check
- axes swap, offsets
- screens are correctly placed
- if tracking is covering the area
!
Allows you to experiment.
70. Homography
A nice new feature is the homography. This allows you to grab corners of your viewports and have them match the physical corners of your cave.
71. Cluster & Multi-pipe
• Scenelock!
• Swaplock!
• Genlock!
• Multi-pipe (soon)
Regarding clustering, one unity player is running on each computer.
SceneLock: synchronise the state of all input devices and position/orientation of some specific objects.
SwapLock: implemented our own mechanism
Genlock: done in HW
78. Looks like english !
No need to understand what a sink is (opentracker), or forced to set callbacks.
79. MiddleVR
Here is the minimum code you have to write to use MiddleVR. You don’t have to modify the
structure of your application to integrate MiddleVR !
80. Class reference
Most VR systems out there can be described with these highlighted simple, atomic, concepts
82. The value of simplicity
“Perfection is achieved, not when there is
nothing more to add, but when there is
nothing left to take away”!
Antoine de St Exupery
Simplicity is a huge principle for me.
We don’t hesitate to remove code/feature that are covering way too much.
EVEN IF ALREADY WRITTEN !!
We’re a Startup, we don’t have resources, so we have to be simple
No resources is a strength
Lots of resources can actually be a curse
83. The value of simplicity
• For each feature you add!
• Test!
• Document!
• Support / maintain!
• Each feature costs !!
Each feature you add, you have to support it for the entire life of the product.
Simplicity is harder to achieve, so cost more at first, but in the long run is beneficial
84. The value of simplicity
• Huge value in simplicity !!
• User side!
• Faster to learn, less to remember!
• Developer side!
• Faster to code!
• Easier maintain!
• Less bugs!
• Business side!
• Need less coders!
• More interesting features!
• Easy download / setup > Quickly validate on user’s hardware !!
• Easier support
85. API in simple english
• Long discussions for naming of functions, classes, variables!
• Consistent api!
• SetNumButtons, SetButtonsNum, SetNbButtons > SetButtonsNb!
• French not good at english!
• Analog ? Digital ? Proxy ?!
• Use natural terms depending on device!
• Joystick->GetAxisValue()!
• Joystick->IsButtonPressed()!
• Keyboard->IsKeyPressed()!
• Why “Register” ? Why not “Add” ?!
• Why so complex, want to sound very clever ?!
• AddDevice, AddCamera, …!
• API in degrees, because more intuitive / simple to work with
86. Simplified API
• Simple but complete API!
• Like legos!
• Rotate!
• Node3D->Rotate( Quat )!
• Node3D->Rotate( Euler )!
• Node3D->Rotate( Axis, Angle)!
• Quat( quat ), Quat( Euler ), Quat( Axis, Angle)
Simplified API: only one way of doing things: less to learn, less to remember
Everything you need is in the Quat class, so it’s reusable in any part of the code
87. The value of simplicity
“Simplicity is the ultimate sophistication”!
DaVinci
Fight hard to create / maintain simplicity.
Engineers come and implemented what was in their mind, anticipate cases that would rarely happen.
We do code reviews. Often rewrite often 3 times before acceptable.
88. The value of simplicity
• Design user interface for the 80%!
• Don’t ruin the experience for the 20%!
• Simple things should be simple!
• Complex things can be complex!
• Put in an “advanced menu”!
• Don’t let beauty of design ruin user
experience!
• Like real life architects who want
their design to be beautiful but
totally unusable on a day to day!
• Example of Lego!
• Simple but complete
Tell horror network socket story :)
!
Current tools designed for geeks/engineers (examples vrpn/virtools vr)
89. Optimizations
• Engineers have a strong tendency to over-optimise!
• Complexifies code!
• Code must be above all readable!
• To avoid bugs!
• To maintain maintenability!
• If you want to optimise!
• First profile your code!
• Optimize what is *really* taking time!
• not what you *think* takes time
90. Empathy
“The ability to understand and share the
feelings of another.”!
!
Test with users
• if user don’t understand, not add more doc,!
• simplify !!
91. Hardware SDK are mostly too complicated
!
Even Oculus !!!!! Less time to implement haptics !!!
92. Drivers
• Why not just a few simple functions with
sensible defaults ?!
• Init()!
• Update()!
• GetValue()!
• Destroy()!
• And add more functions for more advanced
HW manufacturers don’t know how to write an API
93. Documentation
• User Guide!
• Class references
Of course the ideal tool would not need documentation,
I honestly like to write documentation.
It helps spot errors, inconsistencies.
If it’s hard to document, the feature must probably must be simplified instead of writing a more sophisticated doc !!
94. Goals
• Simplify deployment of VR applications!
• GUI to configure VR systems!
• Dynamic reconfiguration of application!
• Ship with standard configurations!
• Abstractions
96. Device management
Driver Tracker 1
Application
Driver Tracker 2
**
The problem is that each driver gives access to similar data in a different way
so you need to have specific part of programs for similar drivers.
97. Device abstraction
Virtual Device Application
Driver Tracker 1
Driver Tracker 2
A virtual device only stores data.
For tracker, position+orientation
Drivers write data in the virtual device.
Allow to change driver without application (PORTABILITY)
---
Replay allows you to playback a recorded working session.
It enables quality insurance because you can give the same input to your application and check that the output
is always the same.
98. Device management
Driver
Virtual Device Application
Simulation
Replay
Replay allows you to playback a recorded working session.
It enables quality insurance because you can give the same input to your application and check that the output
is always the same.
100. User
• Human-centric rather than techno-centric
Rather than accessing devices,
we could simply ask the position of the user’s head, or hand !
This allows to write scenarios like :
What does the user see ? Is the hand close to ... ?
104. • How to create portable interactions ?!
• Navigations can be portable!
• Except collisions!
• Generalise what is generalisable!
• + Custom part for engine dependent!
• Collisions!
• Highlights!
• Gizmos
Interactions
108. C++ Reflexion
• Also for configuration file!
• Cluster synchronisation!
• Accessing drivers specific information
Property is simple couple of getter/setter.
In header write naturally Set/GetParent
!
One line to add property in source code.
109. Working with multiple
3D engines
• Different coordinate systems!
• vrCoordinateSystem( Front, Up, Right )!
• +/- X,Y,Z!
• OpenGL:!
• Front= -Z!
• Up = +Y!
• Right = +X!
• DirectX:!
• Front=+Z!
• Up=+Y!
• Right=+X
110. Working with multiple
3D engines
• C++!
• Compiling!
• STL, Boost!
• Strings (Wide strings)!
• Link issues!
• Different compilers (VC2005/2008/2010/2012)!
• Different languages!
• C wrapper!
• Lua wrapper
113. Conclusion
Talked about the philosophy and architecture
You see how deeply the philosophy impacts the architecture
Simplicity at its core which has a huge value for everybody, but has to fight hard for it.