1.
Best Practices for AR
Foundation
Krikey, Inc
Jhanvi Shriram, CEO
Dr. Ketaki Shriram, CTO

2.
AR
GAMES
USER
VIDEOS
LOCATION
BASED
Krikey: Where Create Meets Play
CEO
Jhanvi
Shriram
Stanford MBA
YouTube, JauntVR
CTO
Dr. Ketaki
Shriram
Stanford VR PhD
Oculus,
Google [x]

3.
Today’s Talk
ARF Features
& Advantages
Theories of
Immersion &
Presence
Applications in
Mobile AR
Gaming
AR Foundation: Features, Theory, Application

4.
Why Mobile AR? Why now?
Accessibility
Cross
Platform
Mobile
Optimized
Computer
Vision
Bridging
Physical and
Digital
AR Foundation democratizes access to premium
computer vision cross-platform

5.
Krikey + Unity
Unity controls Krikey’s entire graphics pipeline
CAMERA
ARFoundation for live computer vision in
camera
GAMES
Game mechanics (ex: touch inputs on
mobile device for AR interactions)
VIDEO
Video recording & editing cross platform
mobile

6.
AR Foundation:
Features & Theoretical
Applications

7.
AR Foundation
Plane
Detection
Camera
Translation
Customization
Raw Point
Cloud
Detection
AR Foundation brings together the best of ARKit and ARCore

8.
Immersion with AR Foundation
DOF
Rotation & translation (3 & 6)
LATENCY
Amount of lag in rendered environment
STEREOSCOPY
Render 3D objects at depth in real time
Three key technical affordances heighten user immersion

9.
Presence
Presence is the sense of ‘being there’ in immersive environments
HUMAN RESPONSE
Presence is determined by the degree to
which users respond to the
environment
PRESENCE AS
ABSENCE
How absent a user is from their physical
world surroundings
METRICS
Motion & biometrics

10.
Gorillas! By The Ellen Fund
EMPATHY
Immerse fans in the
world of gorillas in a
meaningful way
TRUTH
Design a gorilla
trek game
experience
CONNECT
Bring the
Rwandan forest
to life using ARF

11.
The Ellen Fund Gorillas! Trailer (Video)

12.
AR Foundation:
Krikey’s Customizations

13.
Dynamic Ground Plane
DETECT
Use ARF cross
platform to find the
lowest plane in a
given scene
LIVE
Grow assets from
ground; add more
as plane expands
real-time
RESULTS
Gorillas!: Users
walk in Rwandan
forest with baby
gorilla

14.
Default Ground Plane
DETECT
Used in instances of
low light / poor
tracking conditions
for ARF CV
ESTIMATE
Default ground
plane 1.5m below
phone pos /
orientation
RESULTS
Camera translation
enables realistic
exp for user

15.
Camera Translation
RESPONSIVENESS
Effectively show users responsiveness of
AR objects in a scene
360 VISIBILITY
Users can approach & view objects from
any angle as if they were real
IN PRACTICE
Users encouraged to approach virtual
habitats in interactive Wingspan game

16.
Wingspan on Google Maps
EMPATHY
Immerse players in
the world of birds in a
meaningful way
BEHAVIOR
Adapt board game
+ design a
walkable birding
experience
CONSERVATION
Translate empathy
to conservation
oriented behavior

17.
Wingspan Trailer (Video)

18.
AR Foundation:
Applications in Mobile AR

19.
Applications of ARF
Social
Presence
Eye Tracking Physical
Awareness
How to apply ARF’s unique features & theory to build compelling games

20.
What is Social Presence?
MULTIPLE
USERS
Sense of belonging inside
a given virtual experience
EYE
CONTACT
Emotional recognition of a
user’s existence within the
virtual world
PHYSICAL
AWARENESS
Acknowledgment of a
user’s position in the
experience gives them a
role in the world
EMPATHY
Feelings of belonging lead
to empathy and action
based on immersive
experiences
THEORY PRACTICE

21.
Eye Contact
DETECT
Phone position used
to move the
character’s eyes
REAL TIME
Eye contact
adjusts in real time
with phone
movement
RESULTS
Eye contact drives
emotional
connection in
virtual exp

22.
Physical Awareness
DETECT
User position in the
scene every N frames
RESPOND
Character should
respond to user
movement (ex:
move away)
RESULTS
User is an active
participant, not a
passive observer

23.
AR Foundation:
Review & Next Steps

24.
Tracking for Experience
Optimizations
BODY
User proximity to virtual
objects
EYE
User directionality
in the experience
Detect user interest
based on movement
patterns

25.
Point Cloud Data
LOW-LEVEL ACCESS
ARF Allows access to low-level point
cloud data
MANIPULATION
These points can be manipulated per
frame to generate a myriad of results (e.g,
Euclidean clustering for real time mesh
gen)
IN PRACTICE
Plugins including occlusion, motion
response, and more can be generated
from this approach.

26.
Occlusion
THEORY
Types of occlusion: people & object
RELEVANCE
AR should mimic physical world cues as
much as possible to generate high
presence & perceived realism
IN PRACTICE
Track points per camera feature (with
depth), cluster them, and generate 3D
meshes in Unity. These can be used to
occlude virtual characters behind
physical world objects

27.
Features In Review
AR Foundation can generate realistic social interactions
Accessibility
of
Mobile AR
Dynamic
Plane
Detection
Camera
Translation
Eye Contact &
Physical
Awareness

28.
WHERE CREATE MEETS PLAY

29.
Thank you!

30.
AR Foundation Live Demo
Q&A