Vuforia™, the leading AR platform, connects the physical world with digital experiences. In this session, David Beard will show how global brands are combining AR and VR to create breakthrough mixed reality experiences, and how these are enabled with Vuforia’s industry-leading tools.
Augmented World Expo (AWE) is back for its seventh year in our largest conference and expo featuring technologies giving us superpowers: augmented reality (AR), virtual reality (VR) and wearable tech. Join over 4,000 attendees from all over the world including a mix of CEOs, CTOs, designers, developers, creative agencies, futurists, analysts, investors, and top press in a fantastic opportunity to learn, inspire, partner, and experience first hand the most exciting industry of our times. See more at http://AugmentedWorldExpo.com
4. 4
A simple API describes what is in the camera’s FOV, and where
– Images
– Objects
– Text
– Meshes
VUFORIA
The leading AR platform
5. 5
PLATFORM COMPONENTS
Vuforia Engine Cloud ServicesTools
Supported Devices
• Phones & Tablets - Android &
iOS
• Eyewear - Samsung Gear VR,
Epson-BT 200, ODG R-6/7
• SDK support for Unity,
Xcode, Android Studio
Target Manager
• Create & Manage databases for
use on device or cloud
Vuforia Object Scanner
• Scan objects such as toys to
create Object Targets
Eyewear Calibration
Vuforia Cloud Recognition
Service
• Allows use of cloud reco
database with up to 1 million
targets
Vuforia Web Services
• RESTful APIs to manage cloud-
based target databases
6. 6
AR + VR = Mixed RealityAR on your headAR in your hand
AR experiences today
7. Mixed Reality
• Similar technology stack
• Different level of immersion
• Complementary
VR and AR
2 sides of the same coin
Hello everyone. My name is David Beard, I’m a senior product manager for the Vuforia platform and our developer evangelist.
My role is to help developers learn how to create AR/VR app using Vuforia, and on the best practices for designing mixed reality experiences.
I’d started with Vuforia over 5 years ago as an engineer and am currently on the product management team, responsible for tools workflows.
Today I’m going to be discussing how Vuforia can be used to create mixed reality apps in Unity
I’ll explain what mixed reality is and present some examples of mixed reality use cases
Then I’ll describe the features of Vuforia 5.5 that support mixed reality
But first, let me give you a brief introduction to Vuforia
Vuforia is PTC’s Mobile AR platform.
It is an SDK that enables your apps to see things in the real world and then interact with them. We’ve done this by taking very sophisticated computer vision technologies, building them ground-up for mobile platforms, and making these technologies accessible through an easy to use API and developer workflows.
…so that ANY app developer can quickly and very easily enable their apps with vision functionality.
Vuforia provides a very simple API. It tells developers what it sees and where it sees these objects precisely in the camera’s FoV…allowing you as developers to then render content accurately on or relative to the object
Today with Vuforia you can see a number of things
Images – these are every day images like magazine covers, product packaging, brochures
Objects – rigid like toys and appliances – objects with little or no articulating or moving parts
Text – we have text recognition for roman characters, such as English language words
And finally with our Smart Terrain feature ….. meshes of an unknown environment…Vuforia can look at a table and will return the table surface and the props on the table in a mesh on top of which , you can overlay content…so you can drive a virtual car on a table that avoids a vase or a stack of books…or falls of the table. With Vuforia smart Terrain, we provide apps the awareness and intelligence to interact with their immediate environment
SDKs
We offer the flexibility to program with Java, C++/Objective C or with Unity
About 85% of our developers use Unity because it provides an easy to use visual authoring workflow
Platform Review
There are 3 main components to the platform
The client side Engine
Android and iOS devices
Smart phones and tablets
Digital Eyewear
Tools to create and manage targets
The Target manager is an online tool for creating, evaluating and managing targets
You can create both device databases, which are embedded in your app, or cloud databases that are accessed using a web service
The Object scanner tool is used to create Object Targets
Eyewear calibration tool is for generating custom eye calibration profiles for users of optical see-through eyewear
Cloud services enable you to deploy targets to your apps from a web service online. This service can support up to 1 million target images and delivers the same kind of trackable data as a device database.
Cloud Recognition is on online image recognition service – you can query this service from your app
Cloud Recognition also supports the Vuforia Web Service REST API which enables you to interface with the service from your own Content Management System
We’d originally developed Vuforia for hand held devices – smart phones and tablets
Then as wearable displays became more readily available, we adapted our SDK to support optical and video see through eyewear
Optical see through eyewear has a transparent lens, where digital content is rendered into the field of view of the wearer
Examples include the Epson BT-200 and ODG R-6 and R-7
Video see-through eyewear relies on a device camera to see the world, which is presented to the user on the device screen
Examples include Gear VR and Google Cardboard
This evolution has enabled the development of a new class of AR/VR applications that support Mixed Reality experiences
Let me talk about Vuforia’s vision for mixed reality.
People often describe AR and VR as distinct technologies, and there are some key differences between pure AR and VR
But there are also significant similarities that make these complementary technologies
They share a similar technology stack for computing, rendering, applications and sensor inputs
They differ in their levels of immersion, between viewing the real world and experiencing a fully virtual world
And these difference are actually complementary – they enable a new range of use cases, and more creative freedom
This is the new Mattel View Master
You may remember this toy – it was originally a stereoscope. You would insert a disk of images and it enabled you to view them in 3D
Mattel has updated this classic toy as an AR/VR viewer. Now the disks are AR targets that launch panoramic VR experiences
View Master demonstrates how you can use AR to intuitively select and transition into VR experiences
Germbusters is a game developed by Realiteer, an independent developer
He’s developed a set of VR controllers using AR targets – in this game the target represents a bubble gun
This game shows that you can use an AR target as an interactive device in AR
Notice that this is a fully immersive world – AR is simply used to recognize and track the game target
Cordy Orly is a app that enables you to view a proposed commercial development near the Cordy Orly airport in France
You can view the plans in AR and then select a point of interest to view in VR
This is a good example of a dual modalilty experience – it starts as handheld AR and transitions of VR in a Viewer
We introduced several new features to better support VR apps with Vuforia 5.5 – which were released a few weeks ago
Stereo Rendering with Distortion Correction
Rotational device tracking
Mixed Reality Controller API
Let me give you some details on each of these
Stereo scene rendering for digital eyewear apps – automatically corrects for pin cushion distortion
High fidelity rendering with viewer and device specific lens distortion correction
Custom viewers can be defined by developers
Provides a device independent, stand alone, solution for head tracking.
You can use device tracking to track both the users head or their hand when they are holding the device
The device tracker returns the rotation of the device around a central pivot point
The pivot point is defined by a rotational model – we provide head and hand models that match typical human anthropometry ( body measurements )
We also enable you to change these models to define a custom rotational offset
The mixed reality controller is a utility API that handles AR / VR app lifecycle automatically
It enables you to transition from AR to VR and also from handheld to viewer based experiences with a single method call
This enables the AR/VR use cases I’ve shown and dual modality apps
The nice thing is that you can develop a standard AR app for handheld mobile devices and easily enabled it for VR by simply implementing this API and a single method call.
