1.
Wikitude SDK Advanced

2.
Agenda for today
● Who is Wikitude
● What has the Wikitude SDK to offer
－Architecture and APIs
－Plugins API
● Latest Additions to the Wikitude SDK – 3D Tracking

3.
3

4.
v v v
Wikitude AR at a glance.
First AR app
in the world in
2008
10,000 apps
powered by
wikitude SDK
3D tracking
technology
v
REFERENCES

5.
Wikitude’s path to success
20142013 2015
Business Model: B2C
Focus: Wikitude AR App
2016201220112009
Founder stage in
Salzburg garage:
First AR Browser
launched.
2010
VCs brought in.,
Management hired:
Martin Herdina,
Andy Gstoll
Established
Wikitude Inc.
in USA
Business Model: B2B
Focus: AR technology and tools
Achieved 25
million
downloads and
app installs
gloabally
Launch of
Wikitude SDK
for 2D image
recognition
Launch of
Wikitude Studio,
the AR CMS
Launch of
Wikitude Cloud
Recognition
service
Launch of 3D
tracking
SLAM

6.
Wikitude offices
wikitude GmbH (HQ) in Salzburg Austriawikitude Inc. in San Francisco, USA
wikitude Inc.
wikitude GmbH.

7.
AR creator and CMSCloud Recognition
Wikitude AR Products & Tools
Connect your own CMS

8.
Native API JavaScript API
Easy access to AR mobile development
Xamarin
PhoneGap
Cordova
titaniumunity
Android
iOS

9.
GEO / sensor based AR
Pioneering products and technologies

10.
Maturing products and technologies
2D Image Recognition
& Tracking

11.
Tailored SDKs for Smart Glasses
Partnership since 2014,
optimized SDK for BT-200, full
documentation
→BT-300 in pipeline
Partnership since 2016
optimized SDK for R-7
Partnership since 2014, SDK for
M100, full documentation
→M300 in pipeline

12.
Wikitude SDK Architecture
2D On-Device
Reco &
Tracking
2D Cloud Reco
3D Reco &
Tracking
Unity3D
Unity
Plugin
Plugins API
Custom CV
plugin
Custom
camera
plugins
Your
App
Your
App Cordova
Plugin
Titanium
Module
Xamarin
Component
Your
App
Cordova Titanium Xamarin
Your
App
Your
App
JavaScript APINative API
Your
App
3D
Rendering
Augmen-
tations
LBS
Plugin ManagerOpenGL ES2
Rendering
ARMv7 NEON
optimizations
ARMv8
optimizations
Camera
GPU
optimizations
OpenCL
RenderScript
Metal API
IMU
Hardware Abstraction Hardware Optimizations
C++ Layer

13.
Plugins API - Modularization
13
Description
The Plugins APIs allows to extend the Wikitude SDK by
3rd party functionality. Plugins for the Wikitude SDK have
access to the camera information, recognized images.
This is perfect for additional functionality that also
requires camera access. Plugins are written in C++ and
can communicate both with the JavaScript API and the
Native API.

14.
Plugins API in detail
14
Custom
Frame
Provider
Image
Frame
Standard
Frame
Provider
Wikitude
CV Engine
Rendering
Custom CV
Plugin
Plugins API
class Plugin { public:
…
void cameraFrameAvailable(const Frame&;
cameraFrame_);
void update(const vector<RecognizedTarget>
recognizedTargets_);
void startRender();
void endRender
…
};
1
2
3
4
5
6
7
class RecognizedTarget {
public:
const string&amp; getIdentifier() const;
const Mat4&amp; getModelViewMatrix() const;
const Mat4&amp; getProjectionMatrix() const;
const float getDistanceToCamera() const;
};

15.
Use-Cases for Plugins API
● Run own computer-vision related code in parallel
to Wikitude SDK
－OCR Engine
－Barcode Reader
－Face Recognition
● Own the camera stream and the camera rendering
－Custom effects and shaders
－Remote camera stream with local processing

16.
JavaScript Augmented Reality Experience
16
Architect Worlds consist of:
• .html file(s)
• .js file(s)
• .css file(s)
• Defines your augmented reality experience
• Use the Wikitude JS API provided within the Architect view context
<script src=“architect://architect.js"></script>
• Loaded by the Wikitude SDKs Architect view

17.
Architect world
17
Javascript execution triggers actions in the underlying C++ layer

18.
Architect view composition
18
OpenGL ES 2 view
* Renders camera stream
* Renders augmentations
Web view
* Loads main .html file
* Transparent background

19.
GeoObjects & HTML content
Menu bar
jQuery support
Geo-located POIs
Multiple Drawables
- 3 images
- 3 text labels
Web View
Augmented Reality View

20.
Support for jQuery and web APIs

21.
Image Recognition
Augment target images using JavaScript
API capabilities (Drawables, Sounds,
Animations,Videos, 3D Models)
var kitchenTracker = new AR.Tracker("Kitchen.wtc");
var overlay = new AR.VideoDrawable("howdens.mp4", 0.65);
var trackable2DObject = new AR.Trackable2DObject(
kitchenTracker, "Kitchen",
{ drawables:
{ cam: overlay
}});

22.
Types of augmentations
22
Augmentations are represented by AR.Drawable subclasses
• Images - AR.ImageResource / AR.ImageDrawable
• Videos (also with alpha channel) - AR.VideoDrawable
• 3D models - AR.Model
• Web views - AR.HTMLDrawable
• Labels - AR.Label

23.
….architecture, real estate….. ….gaming, filming industry….. ….indoor navigation, retail…..
New algorithms and concepts
3D Tracking / SLAM
Object Reco & Tracking

24.
Recognition & tracking
(core engine)
3D
One core engine with many “flavors”
Small Scale
Scenes / Spaces / Objects
- SLAM
- Point cloud
- Smaller scenes
- desktop,small rooms and
environments
Object Recognition
Edge Detection / CAD models
- pre-registeredCAD
models and displayed
silhouettes
- Focus: shapes & lines
Positional Tracking
Indoor Navigation”
- Computer vision plusIMU /
sensor fusion
- Malls, airports, buildings
- Untethered indoor gaming in
apartment,houseor anywhere
Large Scale
Scenes Spaces
- SLAM based
- Point cloud
- buildings / spaces
- indoors, outdoors

25.
3D Tracking Flavors
1. Object Recognition Edge Detection / CAD models
Object Recognition
Edge Detection / CAD models
- pre-registeredCAD
models and displayed
silhouettes
- Focus: shapes & lines

26.
Wikitude Lab – RnD Sneak Preview

27.
3D Tracking Flavors
2. Large Scale Scenes / Spaces
Large Scale
Scenes Spaces
- SLAM based
- Point cloud
- buildings / spaces
- indoors, outdoors
Watch here: https://www.youtube.com/watch?v=Zqb6Fo4uRkM

28.
Positional Tracking – Demo Video

29.
Philipp Nagele, CTO
Wikitude
See more.