This document provides an overview of osgART, an open source library that adds augmented reality capabilities to Open Scene Graph. It discusses osgART's scene graph approach, architecture, and basic tutorial for creating an osgART application. The tutorial covers loading video and tracking plugins, adding markers, attaching tracked transformations to a virtual camera, and placing 3D models in the scene. It also discusses how to track multiple markers simultaneously. Finally, it briefly introduces FLARManager, an open source library that makes building augmented reality applications with FLARToolkit easier.

1. COSC 426: Augmented Reality
Mark Billinghurst
mark.billinghurst@hitlabnz.org
August 14th 2012
Lecture 6: osgART

2. osgART
Developing Augmented Reality
Applications with osgART

3. What is a Scene Graph?
 Tree-like structure for organising a virtual world
 e.g. VRML
 Hierarchy of nodes that define:
 Groups (and Switches, Sequences etc…)
 Transformations
 Projections
 Geometry
 …
 And states and attributes that define:
 Materials and textures
 Lighting and blending
 …

4. Scene Graph Example

5. Benefits of a Scene Graph
 Performance
 Structuring data facilitates optimization
- Culling, state management, etc…
 Abstraction
 Underlying graphics pipeline is hidden
 Low-level programming (“how do I display
this?”) replaced with high-level concepts Image: sgi
(“what do I want to display?”)

6. About Open Scene Graph
 http://www.openscenegraph.org/
 Open-source scene graph implementation
 Based on OpenGL
 Object-oriented C++ following design pattern principles
 Used for simulation, games, research, and industrial projects
 Active development community
 Maintained by Robert Osfield
 ~2000 mailing list subscribers
 Documentation project: www.osgbooks.com
 Uses the OSG Public License (similar to LGPL)

7. About Open Scene Graph (2)
Pirates of the XXI Century Flightgear SCANeR
3DVRII Research Institute EOR VRlab Umeå University

8. Open Scene Graph Features
 Plugins for loading and saving
 3D: 3D Studio (.3ds), OpenFlight (.flt), Wavefront (.obj)…
 2D: .png, .jpg, .bmp, QuickTime movies
 NodeKits to extend functionality
 e.g. osgShadow
 Cross-platform support for:
 Window management (osgViewer)
 Threading (OpenThreads)

9. Open Scene Graph Architecture
Inter-operability with
other environments,
e.g. Python
Plugins read and NodeKits extend
write 2D image and core functionality,
3D model files Scene graph and exposing higher-level
rendering node types
functionality

10. Some Open Scene Graph Demos
osgviewer osgmotionblur osgparticle
osgreflect osgdistortion osgfxbrowser
 You may want to get the OSG data package:
 Via SVN: http://www.openscenegraph.org/svn/osg/OpenSceneGraph-Data/trunk

11. Learning OSG
 Check out the Quick Start Guide
 Free PDF download at http://osgbooks.com/, Physical copy $13US
 Join the mailing list: http://www.openscenegraph.org/projects/osg/wiki/
MailingLists
 Browse the website: http://www.openscenegraph.org/projects/osg
 Use the forum: http://forum.openscenegraph.org
 Study the examples
 Read the source? 

12. osgART

13. What is osgART?
 osgART adds AR to Open Scene Graph
 Further developed and enhanced by:
 Julian Looser
 Hartmut Seichter
 Raphael Grasset
 Current version 2.0, Open Source
 http://www.osgart.org

14. osgART Approach: Basic Scene Graph
Root
[ ]
0.988 -0.031 -0.145 0
-0.048 0.857 -0.512 0
Transform 0.141 0.513 0.846
10.939 29.859 -226.733
0
1
 To add Video see-through AR:
3D Object  Integrate live video
 Apply correct projection matrix
 Update tracked transformations in
realtime

15. osgART Approach: AR Scene Graph
Root
Transform
3D Object

16. osgART Approach: AR Scene Graph
Root
Virtual
Camera
Video
Transform
Layer
Video
3D Object
Geode

17. osgART Approach: AR Scene Graph
Root
Projection matrix from
tracker calibration
Orthographic Transformation matrix
projection
Virtual
updated from marker
Camera tracking in realtime
Video
Transform
Layer
Full-screen quad
with live texture
updated from
Video source Video
3D Object
Geode

18. osgART Approach: AR Scene Graph
Root
Projection matrix from
tracker calibration
Orthographic Transformation matrix
projection
Virtual
updated from marker
Camera tracking in realtime
Video
Transform
Layer
Full-screen quad
with live texture
updated from
Video source Video
3D Object
Geode

19. osgART Architecture
 Like any video see-through AR library, osgART requires video
input and tracking capabilities.
Video Source
e.g. DirectShow
AR Library
Application
Tracking Module
(libAR.lib)

20. osgART Architecture
 osgART uses a plugin architecture so that video sources and tracking
technologies can be plugged in as necessary
OpenCVVideo -
VidCapture -
CMU1394 -
PointGrey SDK -
VidereDesign -
Video Plugin
VideoWrapper -
VideoInput -
VideoSource -
Application
DSVL -
osgART
Intranel RTSP -
Tracker Plugin
ARToolKit4 -
ARToolkitPlus -
MXRToolKit -
ARLib -
bazAR (work in progress) -
ARTag (work in progress) -

21. Basic osgART Tutorial
 Develop a working osgART application from scratch.
 Use ARToolKit 2.72 library for tracking and video
capture

22. osgART Tutorial 1: Basic OSG Viewer
 Start with the standard Open Scene Graph Viewer
 We will modify this to do AR!

23. osgART Tutorial 1: Basic OSG Viewer
 The basic osgViewer…
#include <osgViewer/Viewer>
#include <osgViewer/ViewerEventHandlers>
int main(int argc, char* argv[]) {
// Create a viewer
osgViewer::Viewer viewer;
// Create a root node
osg::ref_ptr<osg::Group> root = new osg::Group;
// Attach root node to the viewer
viewer.setSceneData(root.get());
// Add relevant event handlers to the viewer
viewer.addEventHandler(new osgViewer::StatsHandler);
viewer.addEventHandler(new osgViewer::WindowSizeHandler);
viewer.addEventHandler(new osgViewer::ThreadingHandler);
viewer.addEventHandler(new osgViewer::HelpHandler);
// Run the viewer and exit the program when the viewer is closed
return viewer.run();
}

24. osgART Tutorial 2: Adding Video
 Add a video plugin
 Load, configure, start video capture…
 Add a video background
 Create, link to video, add to scene-graph

25. osgART Tutorial 2: Adding Video
 New code to load and configure a Video Plugin:
// Preload the video and tracker
int _video_id = osgART::PluginManager::getInstance()-
>load("osgart_video_artoolkit2");
// Load a video plugin.
osg::ref_ptr<osgART::Video> video =
dynamic_cast<osgART::Video*>(osgART::PluginManager::getInstance()-
>get(_video_id));
// Check if an instance of the video stream could be created
if (!video.valid()) {
// Without video an AR application can not work. Quit if none found.
osg::notify(osg::FATAL) << "Could not initialize video plugin!" <<
std::endl;
exit(-1);
}
// Open the video. This will not yet start the video stream but will
// get information about the format of the video which is essential
// for the connected tracker.
video->open();

26. osgART Tutorial 2: Adding Video
 New code to add a live video background
osg::Group* createImageBackground(osg::Image* video) {
osgART::VideoLayer* _layer = new osgART::VideoLayer();
_layer->setSize(*video);
osgART::VideoGeode* _geode = new osgART::VideoGeode(osgART::VideoGeode::USE_TEXTURE_2D, video);
addTexturedQuad(*_geode, video->s(), video->t());
_layer->addChild(_geode);
return _layer;
}
 In the main function…
osg::ref_ptr<osg::Group> videoBackground = createImageBackground(video.get());
videoBackground->getOrCreateStateSet()->setRenderBinDetails(0, "RenderBin");
root->addChild(videoBackground.get());
video->start();

27. osgART Tutorial 3: Tracking
 Add a Tracker plugin
 Load, configure, link to video
 Add a Marker to track
 Load, activate
 Tracked node
 Create, link with marker via tracking callbacks
 Print out the tracking data

28. osgART Tutorial 3: Tracking
 Load a tracking plugin and associate it with the video plugin
int _tracker_id = osgART::PluginManager::getInstance()->load("osgart_tracker_artoolkit2");
osg::ref_ptr<osgART::Tracker> tracker =
dynamic_cast<osgART::Tracker*>(osgART::PluginManager::getInstance()->get(_tracker_id));
if (!tracker.valid()) {
// Without tracker an AR application can not work. Quit if none found.
osg::notify(osg::FATAL) << "Could not initialize tracker plugin!" << std::endl;
exit(-1);
}
// get the tracker calibration object
osg::ref_ptr<osgART::Calibration> calibration = tracker->getOrCreateCalibration();
// load a calibration file
if (!calibration->load("data/camera_para.dat"))
{
// the calibration file was non-existing or couldnt be loaded
osg::notify(osg::FATAL) << "Non existing or incompatible calibration file" << std::endl;
exit(-1);
}
// set the image source for the tracker
tracker->setImage(video.get());
osgART::TrackerCallback::addOrSet(root.get(), tracker.get());
// create the virtual camera and add it to the scene
osg::ref_ptr<osg::Camera> cam = calibration->createCamera();
root->addChild(cam.get());

29. osgART Tutorial 3: Tracking
 Load a marker and activate it
 Associate it with a transformation node (via event callbacks)
 Add the transformation node to the virtual camera node
osg::ref_ptr<osgART::Marker> marker = tracker->addMarker("single;data/patt.hiro;80;0;0");
if (!marker.valid())
{
// Without marker an AR application can not work. Quit if none found.
osg::notify(osg::FATAL) << "Could not add marker!" << std::endl;
exit(-1);
}
marker->setActive(true);
osg::ref_ptr<osg::MatrixTransform> arTransform = new osg::MatrixTransform();
osgART::attachDefaultEventCallbacks(arTransform.get(), marker.get());
cam->addChild(arTransform.get());
 Add a debug callback to print out information about the tracked marker
osgART::addEventCallback(arTransform.get(), new osgART::MarkerDebugCallback(marker.get()));

30. osgART Tutorial 3: Tracking
 Tracking information is
output to console

31. osgART Tutorial 4: Adding Content
 Now put the tracking data to use!
 Add content to the tracked transform
 Basic cube code
arTransform->addChild(osgART::testCube());
arTransform->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");

32. osgART Tutorial 5: Adding 3D Model
 Open Scene Graph can load some 3D formats directly:
 e.g. Wavefront (.obj), OpenFlight (.flt), 3D Studio (.3ds), COLLADA
 Others need to be converted
 Support for some formats is much better than others
 e.g. OpenFlight good, 3ds hit and miss.
 Recommend native .osg and .ive formats
 .osg – ASCII representation of scene graph
 .ive – Binary OSG file. Can contain hold textures.
 osgExp : Exporter for 3DS Max is a good choice
 http://sourceforge.net/projects/osgmaxexp
 Otherwise .3ds files from TurboSquid can work

33. osgART Tutorial 5: Adding 3D Model
 Replace the simple cube with a 3D model
 Models are loaded using the osgDB::readNodeFile() function
std::string filename = "media/hollow_cube.osg";
arTransform->addChild(osgDB::readNodeFile(filename));
Export to .osg
3D Studio Max
osgART
 Note: Scale is important. Units are in mm.

34. osgART Tutorial 6: Multiple Markers
 Repeat the process so far to track more than
one marker simultaneously

35. osgART Tutorial 6: Multiple Markers
 Repeat the process so far to track more than one marker
 Load and activate two markers
osg::ref_ptr<osgART::Marker> markerA = tracker->addMarker("single;data/patt.hiro;80;0;0");
markerA->setActive(true);
osg::ref_ptr<osgART::Marker> markerB = tracker->addMarker("single;data/patt.kanji;80;0;0");
markerB->setActive(true);
 Create two transformations, attach callbacks, and add models
osg::ref_ptr<osg::MatrixTransform> arTransformA = new osg::MatrixTransform();
osgART::attachDefaultEventCallbacks(arTransformA.get(), markerA.get());
arTransformA->addChild(osgDB::readNodeFile("media/hitl_logo.osg"));
arTransformA->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");
cam->addChild(arTransformA.get());
osg::ref_ptr<osg::MatrixTransform> arTransformB = new osg::MatrixTransform();
osgART::attachDefaultEventCallbacks(arTransformB.get(), markerB.get());
arTransformB->addChild(osgDB::readNodeFile("media/gist_logo.osg"));
arTransformB->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");
cam->addChild(arTransformB.get());

36. osgART Tutorial 6: Multiple Markers

37. Basic osgART Tutorial: Summary
Standard OSG Viewer Addition of Video Addition of Tracking
Addition of basic 3D Addition of 3D Model Multiple Markers
graphics

38. FLARManager
http://transmote.com/flar

39. FLARManager:
 Makes building FLARToolkit apps easier
 Is open-source, with a free and commercial license
 Is designed to allow exploration of both augmented
reality and alternative controllers
 Decouples FLARToolkit from Papervision3D
 Configuration without recompilation, via xml config

40. FLARManager: features
 Gives more control over application environment
 Provides multiple input options
 Robust multiple marker management
 Supports multiple 3D frameworks
 Offers features for optimization
 Allows for customization

41. Resources

42. Websites
 Software Download
 http://artoolkit.sourceforge.net/
 ARToolKit Documentation
 http://www.hitl.washington.edu/artoolkit/
 ARToolKit Forum
 https://www.artoolworks.com/community/forum/
 ARToolworks Inc
 http://www.artoolworks.com/

43.  ARToolKit Plus
 http://studierstube.icg.tu-graz.ac.at/handheld_ar/
artoolkitplus.php
 osgART
 http://www.osgart.org/
 FLARToolKit
 http://www.libspark.org/wiki/saqoosha/FLARToolKit/
 FLARManager
 http://words.transmote.com/wp/flarmanager/

44. Books
 Interactive Environments with Open-Source
Software: 3D Walkthroughs and Augmented
Reality for Architects with Blender 2.43, DART
3.0 and ARToolKit 2.72 by Wolfgang Höhl
 A Hitchhikers Guide to Virtual Reality by Karen
McMenemy and Stuart Ferguson

45. More Information
• Mark Billinghurst
– mark.billinghurst@hitlabnz.org
• Websites
– www.hitlabnz.org