Istvan Siklossy, Sr. Director of Software Engineering, of Vuforia speaks to the state of AR today and how, where, and why it is deployed. This talk provides insights on augmented reality markets, applications, and technology.
Intro
The title of todays presentation is called „What‘s augmented, what‘s reality?“.
6+ years developing the Vuforia AR engine
Balance the huge expectations for the technology in the long term, while setting realistic expectations of what it can do today.
Talk to you about that and highlight long term potential as well as focus on what is real right now.
Show you the use cases and concrete application examples that are successful today.
4th wave So from desktop Web mobile AR/VR
From Early days: AR gimmicky
To: next major disruptive wave in computing platforms.
Apple CEO Tim Cook: Remarks about AR potential being bigger than VR
Why is the potnetial so big? Why are the big guys like google MS, Facebook, Apple investing so much into it?
First, AR has the potential to be used by everyone.
AR from an end-user experience still leaves you in the real world.
VR on the other hand teleports you into a different reality, more of a solitary experience like TV or consoles at home.
Second: AR‘s ability to be used anywhere, particularly when we have AR glasses on our head.
Disruptive to smart phones today
Think broader and realize disruptive to any device that has a screen.
Example: replace TV, replace projector
AR enormous potential: replace any device that has a screen today.
But lets come back to what is real, what can AR do today.
Vuforia 30 000 apps in the markets and 30 + more in development
We can look at these from a business/product perspective, we can create the following taxonomy for AR apps
Will show you some examples
Market and Sell their products.
Create products: Use AR in the design process
Operate: how do I use products
Service: How do I service and repair them?
Most traction to date - Many of you know these and I will be quick here
Marketers are attaching AR experiences to physical products & marketing materials
Some gimmicky
Some are very useful
„See before you buy“
AR lets you see the product you intend to buy at home before you buy it
Notable examples include:
Furniture
TV
Cars (Some in small size, some even in real size)
Very compelling, proven to work
Companion experience for physical products
In particular toys
LEGO Nexo Knights
Probably best example for what is happening with AR with toys for kids
Physical play sets of knights + shields that unlock special powers. Kids want more playset to unlock more.
Happening with other toy manufactures, Mattel, Hasbro
Motivation: Kids are excited about iPads – industry need to create connection back to phyiscal toy,
Another example: Colouring books
Other Marketing use case: Photo sharing
Star wars app lets you see Star wars characters in AR next to the movie poster and lets insert your friend, record a video and share
SnapChat are adding AR to their app. In some way AR is the ultime personalized user created content.
This was marketing and sell, moving on to create products
To see things before I build them as part of the design process
Virtual prototyping
Make change and show immediately in real scale
Today learn to use a new physical product at home or at work, user manual..
User manuals usually a low point in the UX: boring to read, not always easy to follow.
Even worse manual can contain instructions for more a family of models, sometimes need to jump to the right block
AR and IOT can solve all of these [start playing video]
See this illustration of operational instructions for a coffe machine
Information right on the object in 3D, as if you had someone next to you
IOT: Relevant instructions for your specific model or even specific instance
Most valuable use of AR in the Enterprise will be step-by-step instructions
Again get rid of every manual replacing it with 3D informationWill come back to this later: Instrucitons often exist in 3D ontop of the CAD models that make up the products and are translated to 2D for print. Now can be played back in AR directly.
What are the challenges?
To deliver on the earlier use case we need to
Recognize object / Calculate 6 DoF pose / Track at 30 or 60 fps
In some cases: Reconstruct the environment
CV is hard.
Us humans ar very good at recognizing any object
Challenge lies inappearance changing across environmental conditions (lighting, reflections, shadows)
Appearence changes over time (e.g. gets dirty)
Vuforia includes some of the most powerful CV and object recognition and tracking technologies in the Vuforia Engine.
Ultimate goal: Universal object recognition
recognize coffe machine, recognize industrial equipoment, recognize toy
Today no one technology that can do them all
Active area of computer vision research. We ourselves at Vuforia are investing hard into a generic object recognition technology.
To enable these use cases today we have recently released the VuMark
VuMark: Customizable code for AR.
Customized look and feel.
Encode data to recognize specific product
Basis and first step for universal object recognition challenge.
AR depends on 3D content, which is difficult and expensive to produce
Good news: There is a tremendous amount of 3D content that already exists in the form of CAD data
Now you can appreciate why PTC, a CAD vendor is so excited about AR. PTC‘s customers have a lot of CAD data
All this CAD data is just screaming to be published in AR where it can be used to create, operate, and service products
We’ll start to see tools to publish CAD data into AR
CAD data and tools to simply create step by step instructions are critical enablers and help drive adoption of AR.
Make it Work Everywhere
It is important for AR experiences to be able to be triggered across multiple devices and form factors (phones, tablets, VR viewers and eyewear)
Different form factors require different approaches to AR, this is a challenge for the industry
Vuforia supports many of these above and this is an area we have created an API that allows to build a single application to span those
Each device has different requirements so we work closely with OEMs to optimize the experience
E.g. HL
DEVICE LIST
Add-in – Samsung Gear VR, Cardboard, ViewMaster
Standalone – Pico Neo, Qualcomm 820VR, Wireless Rift
Tethered – HTC Vive, Oculus Rift, Play Station VR
Mono – Google Glass, Recon Jet, Vuzix M100
Bino – HoloLens, ODG, Epson
Microsoft BUILD Caterpillar Demo video
So in summary:
Why AR potential is so big
What use cases work today
The challenges
These are hard problems
Our role/mission in the ecosystem is to SOLVE these problems
Our mission is to democratize AR