CASA workshop 3AMIGAS (supported by FOCUS K3D and GATE) Technical program presentation no. 1: presenter Chiara Catalano, IMATI, Italy http://www.cs.uu.nl/events/3amigas/ http://www.focusk3d.eu/ http://gate.gameresearch.nl

1. Computer Graphics Recipes for
Endowing 3D Shapes with
Semantics in Virtual Worlds
Michela Mortara, Chiara E. Catalano
CNR IMATI Genova, Italy

2. Outline
3D in Gaming: need of semantics
How to make semantics explicit:
Segmentation
Annotation
Applications to virtual worlds
Conclusions
3AMIGAS Workshop, Amstedam, June 16th 2009

3. 3D in Gaming
Complex and convincing virtual worlds
realistic environment and characters
realistic interactions and behaviour
3AMIGAS Workshop, Amstedam, June 16th 2009

4. What’s semantics?
Knowledge related to the geometry of the 3D
shape
Knowledge related to the application domain in
which the object is manipulated
Knowledge related to the meaning of the object
represented by the 3D model
3AMIGAS Workshop, Amstedam, June 16th 2009

5. Adding semantics to geometry
We have to extract and associate knowledge to
digital content
Shape analysis: extracts implicit info coded into
the geometry
Numerical values (e.g. volume, area, spatial
distribution, curvature)
Shape descriptors (structuring the shape)
Segmentation
3AMIGAS Workshop, Amstedam, June 16th 2009
geometry structure semantics

6. Adding semantics to geometry
We have to extract and associate knowledge to
digital content and its subparts
Shape Annotation: it documents the shape
with contextual knowledge either manually or
automatically
geometry structure semantics
3AMIGAS Workshop, Amstedam, June 16th 2009

7. Semantic-driven segmentations
They captures semantically relevant features
automatically
what is relevant in the gaming context?
IMATI segmentation methods for triangle meshes:
Fitting primitives
Reeb Graphs
Tailor
Plumber
They extract specific kind of knowledge suitable
also for virtual worlds
3AMIGAS Workshop, Amstedam, June 16th 2009

8. Fitting Primitives
Hierarchical face clustering algorithm able to recognise
planes, cylinders and spheres
3AMIGAS Workshop, Amstedam, June 16th 2009

9. Reeb Graph
The topology of a shape is coded into the Reeb
graph respect to f, which captures the evolution of the
level sets of f on the shape
3AMIGAS Workshop, Amstedam, June 16th 2009

10. Plumber (and Tailor)
It defines a shape decomposition into
connected components that are either
tubular features or blob regions
Skeletons and grasping
regions
3AMIGAS Workshop, Amstedam, June 16th 2009
Landmarks

11. Automatic identification of human body parts
Plumber Tailor
Skeleton extraction
segmentation characterisation
Tagging human Candidate
body parts skeletal joints
3AMIGAS Workshop, Amstedam, June 16th 2009

12. ShapeAnnotator: general framework
Shape
A surface mesh
Abstracted
Shape
Shape A segmented mesh
CG Tools
Segmentation Plug-ins Annotator
Domain
An OWL ontology
Instance
Instance
Expert Knowledge Base
3AMIGAS Workshop, Amstedam, June 16th 2009

13. ShapeAnnotator: multi-segmentation
Pick the interesting features from different
shape segmentations
(b) Morse-based (c) Plumber (d) fitting primitives
3AMIGAS Workshop, Amstedam, June 16th 2009

14. The ShapeAnnotator: ontology browser
3AMIGAS Workshop, Amstedam, June 16th 2009

15. The ShapeAnnotator: knowledge base
3AMIGAS Workshop, Amstedam, June 16th 2009

16. Possible scenarios for virtual worlds
DSW semantic search for reuse:
Models as they are: to have large diversity of human
characters and objects
Modifying models: to create new VHs, reuse
garments, take only interesting parts
Assistive living
Simulation of domestic environments with realistic
people (e.g. with disabilities), places and danger
conditions
3AMIGAS Workshop, Amstedam, June 16th 2009

17. Conclusions
3D with semantics would have high impact
Easy indexing, searching and retrieval
Efficient reuse and re-adaptation
Realistic interactions among virtual characters and
objects
Still a lot of work to do
Manual annotation
No shared conceptualisation
…
3AMIGAS Workshop, Amstedam, June 16th 2009