Introduction to Augmented Reality with Unity3D, Vuforia & String Example implementations (iOS): Other Side - Pantalla Global: http://itunes.apple.com/app/other-side/id495565861

1. AR with Unity3D
Introduction to Augmented Reality
with
Unity3D
Saturday, May 26, 12

2. Intro
• Examples
• Pantalla Global (CCCB)
• Animac
• Salón de Turisme
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

3. Koffa@Animac
http://vimeo.com/37868229
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

4. What is AR?
• Virtual content overlayed onto the real
word
• 2D / 3D & audio
• Realtime / Live camera view
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

5. Detection
• Vision based
• QR codes
• Markers
• Real life objects
• Location based (GPS, Compass)
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

6. Vision Based AR
• Tracking
• Frame by frame analysis searching for
“trackables”
• Return position and orientation relative to
device
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

7. AR Apps
• Layar
• http://www.layar.com/
• Junaio
• http://www.junaio.com/
• Wikitude
• http://www.wikitude.com/
• Appunta ;-)
• http://appunta.com/
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

8. Use
• Arts & Gaming
• Advertising
• Navigation
• Tourism
• Education
• ... and a lot more
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

9. Unity3D Libraries
• String
• www.poweredbystring.com
• Vuforia (Qualcomm)
• https://developer.qualcomm.com/mobile-development/mobile-
technologies/augmented-reality
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

10. String - Tracking
• Framed Markers
• Multiple marker detection
• <= 10
• Unflexible marker creation
• black border surrounded by a white border
necessary
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

11. String - Tracking
• Any png
• Full marker has to be in camera (>85%)
• Markers not in scene
• dificult scaling
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

12. Vuforia - Tracking
• Image Target
• Frame Markers
• Simple 3D Objects (Boxes)
• More flexible than String on marker
creation
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

13. Vuforia - Tracking
• Multiple Image Targets or Frame Markers
• <= 5
• Works if only part of the marker is seen
• Markers placed into 3D Scene
• easy scaling
• Virtual Buttons
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

14. Target Management
• Online
• ImageTargets
• MultiTargets
• jpg & png
• RGB & B&W
!
Rich in detail
High in contrast
• Scaling (Unity units) No repetitive patterns
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

15. Performance
• String a little bit faster in detection and
tracking
• String is more stable
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

16. Platform support
• Vuforia
• Android & iOS
• String
• iOS
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

17. String - Setup
• Download library
• Unity3d
• iOS
• Add to Unity3D project
• Add to XCode project
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

18. Vuforia - Setup
• Download Vuforia library
• available in Assets Store
• Add to Unity3D Project
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

19. Testing
• String:
• In-Editor testing using webcam
• Vuforia:
! Unity Remote
• No image tracking nor life camera
• Device Only Testing
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

20. Licensing
• Unity / Unity iOS / Unity Android
• https://store.unity3d.com/
• String
• Expensive!
• http://www.poweredbystring.com/licensing
• Vuforia
• Free!!!
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

21. Decision ???
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

22. Vuforia
Saturday, May 26, 12

23. Requirements iOS
• Mac OS Snow Leopard & Lion
• XCode
• https://ar.qualcomm.at/qdevnet/sdk/ios
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

24. Requirements Android
• MS Windows 7 & XP
• Linux & Mac OS posssible but not
documented
• JDK + Eclipse + ADT
• Android SDK + NDK
• https://ar.qualcomm.at/qdevnet/sdk/android
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

25. Installation
• Download pagackage vuforia-unity-OS-
Version.zip and install
• base extension & samples
• Asset Store
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

26. Simple Project
• New project
• Add Vuforia
• Add Dataset (markers)
• Add ARCamera Prefab to scene
• Add ImageTarget Prefab to scene
• Select Dataset and ImageTarget
• Add 3D objects + Light
• Set Load Dataset
• Deploy
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

27. Unity Folder Structure
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

28. Hands-On
• DroidBalls
• Floating spheres in space
• Touch to dissappear
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

29. Raycasting
• Detect 3D object touched on screen
• Ray from touched screen point to
camera
• Detect object collisions
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

30. Tracking
• ITrackableEventHandler
• OnTrackableStateChanged
• DefaultTrackableEventHandler
• enable/disable rendering
• ITrackerEventHandler
• OnTrackablesUpdated
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

31. Splash Screen
• Display when no marker is detected
• Modify ITrackableEventHandler
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

32. Virtual Buttons
• Hotspots on an ImageTarget
• IVirtualButtonEventHandler
• OnVirtualButtonPressed
• OnVirtualButtonReleased
• Can be created at runtime
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

33. Problems
• Perfomance
• Image Recognition
• symmetry / similar elements
• contrast / illumination
• Real world sizing
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

34. Problems
• Fixed camera vs fixed space
• Physics!!!
• Multiple markers
• center???
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

35. Tips & Advices
• 3D Models
• < 2K polygons overall
•
•
< 200 polygons/object (draw calls)
<= 3 materials/object
! Device dependent
• Animations
• < 30 bones
• Use texture sequences
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

36. Tips & Advices
• Attach markers to cardboard
• Good lightning but avoid reflections
• Disable objects that are not in view
• Test using mouse instead of touch
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

37. Tips & Advices
• Scale correctly from the beginning
• Create simple reference objects to map to
real world
• e.g. cubes
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

38. Resources
• Unity3D
• http://forum.unity3d.com
• http://answers.unity3d.com
• Vuforia Forums
• https://ar.qualcomm.at/qdevnet/forums
• String ???
AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde
Saturday, May 26, 12

39. Thank you!
Andreas Blick @aquarioverde
Saturday, May 26, 12