Indoor Navigation Infraestructure based on Augmented Reality Techniques, RECPAD 2012, 18th edition of the Portuguese Conference on Pattern Recognition (Coimbra - ISEC)

2. [Slide 2]
□ Introduction
□ Objectives
□ Related Works
□ Distributed Architecture
□ Multimodal Positioning System
□ Vision Based Methods
□ Results
□ Open Challenges
Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Contents
26/10/2012 Intro Objectives Related Architecture Results Future

3. [Slide 3]
□ Introduction
□ Objectives
□ Related Works
□ Distributed Architecture
□ Multimodal Positioning System
□ Vision Based Methods
□ Results
□ Open Challenges
Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Contents
26/10/2012 Intro Objectives Related Architecture Results Future

4. [Slide 4]
□ Introduction
□ Objectives
□ Related Works
□ Distributed Architecture
□ Multimodal Positioning System
□ Vision Based Methods
□ Results
□ Open Challenges
Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Contents
26/10/2012 Intro Objectives Related Architecture Results Future

5. [Slide 5]
□ Introduction
□ Objectives
□ Related Works
□ Distributed Architecture
□ Multimodal Positioning System
□ Vision Based Methods
□ Results
□ Open Challenges
Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Contents
26/10/2012 Intro Objectives Related Architecture Results Future

6. [Slide 6]
□ Introduction
□ Objectives
□ Related Works
□ Distributed Architecture
□ Multimodal Positioning System
□ Vision-Based Methods
□ Results
□ Open Challenges
Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Contents
26/10/2012 Intro Objectives Related Architecture Results Future

7. [Slide 7]
□ Introduction
□ Objectives
□ Related Works
□ Distributed Architecture
□ Multimodal Positioning System
□ Vision-Based Methods
□ Results
□ Open Challenges
Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Contents
26/10/2012 Intro Objectives Related Architecture Results Future

8. [Slide 8]
□ Introduction
□ Objectives
□ Related Works
□ Distributed Architecture
□ Multimodal Positioning System
□ Vision-Based Methods
□ Results
□ Open Challenges
Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Contents
26/10/2012 Intro Objectives Related Architecture Results Future

9. Introduction
Intro Objectives Related Architecture Results Future26/10/2012
[Slide 9]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha

10. Input Bin
Introduction
26/10/2012
[Slide 10]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

11. Introduction
26/10/2012
[Slide 11]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

12. Introduction
26/10/2012
[Slide 12]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

13. Input Bin
26/10/2012
[Slide 13]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
WhatisAugmentedReality?
 Variation of Virtual Environments
where virtual objects are
superimposed over the real world.
 Characteristics [Azuma97]:
 Real + Virtual
 Interactive
 Registered in 3D

14. Input Bin
26/10/2012
[Slide 14]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
WhatisAugmentedReality?
 Variation of Virtual Environments
where virtual objects are
superimposed over the real world.
 Characteristics [Azuma97]:
 Real + Virtual
 Interactive
 Registered in 3D

15. Input Bin
26/10/2012
[Slide 15]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
WhatisAugmentedReality?
 Variation of Virtual Environments
where virtual objects are
superimposed over the real world.
 Characteristics [Azuma97]:
 Real + Virtual
 Interactive
 Registered in 3D

16. Input Bin
26/10/2012
[Slide 16]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
WhatisAugmentedReality?
 Variation of Virtual Environments
where virtual objects are
superimposed over the real world.
 Characteristics [Azuma97]:
 Real + Virtual
 Interactive
 Registered in 3D

17. Input Bin
26/10/2012
[Slide 17]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
WhatisAugmentedReality?
 Variation of Virtual Environments
where virtual objects are
superimposed over the real world.
 Characteristics [Azuma97]:
 Real + Virtual
 Interactive
 Registered in 3D

18. Input Bin
26/10/2012
[Slide 18]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
WhatisAugmentedReality?
 Variation of Virtual Environments
where virtual objects are
superimposed over the real world.
 Characteristics [Azuma97]:
 Real + Virtual
 Interactive
 Registered in 3D

19. Input Bin
26/10/2012
[Slide 19]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
WhatisAugmentedReality?
 Variation of Virtual Environments
where virtual objects are
superimposed over the real world.
 Characteristics [Azuma97]:
 Real + Virtual
 Interactive
 Registered in 3D

20. Input Bin
26/10/2012
[Slide 20]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
WhatisAugmentedReality?
 Variation of Virtual Environments
where virtual objects are
superimposed over the real world.
 Characteristics [Azuma97]:
 Real + Virtual
 Interactive
 Registered in 3D

21. 26/10/2012
[Slide 21]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Morethanjust«Location»...

22. 26/10/2012
[Slide 22]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Morethanjust«Location»...
 How to measure Camera
Pose (6DOF): Tracking

23. 26/10/2012
[Slide 23]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Morethanjust«Location»...
 How to measure Camera
Pose (6DOF): Tracking
Identify well-known points

24. 26/10/2012
[Slide 24]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Morethanjust«Location»...
 How to measure Camera
Pose (6DOF): Tracking
Table Surface
Identify well-known points

25. 26/10/2012
[Slide 25]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Morethanjust«Location»...
 How to measure Camera
Pose (6DOF): Tracking
Table Surface
Table Surface
Identify well-known points

26. 26/10/2012
[Slide 26]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
ARMarket(JuniperResearch)
Total Mobile AR Revenues on
Mobile Devices Split by Category
MillionsofDollars

27. 26/10/2012
[Slide 27]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Morethana«TrendingTopic»
Augmented Reality Virtual Reality
Google Trends

28. 26/10/2012
[Slide 28]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Morethana«TrendingTopic»
Augmented Reality Virtual Reality
Google Trends
Search
Volume Index
(by Country)
Spain: 18
Portugal: 31
France: 12
UK: 61
Germany: 54
Netherlands: 80
USA: 46
India: 63

29. ContactLenses,ContactLenses,
TheFuture?TheFuture?
AR Contact Lenses Prototype
26/10/2012
[Slide 29]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
B. Parviz et al. (2009)
University of Washington

30. 26/10/2012
[Slide 30]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Google«ProjectGlass»
Google+: g.co/projectglass
4th April 2012

31. 26/10/2012
[Slide 31]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
AworldofLocation-BasedServices
[Video]

32. Objectives
Intro Objectives Related Architecture Results Future26/10/2012
[Slide 32]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha

33. Input Bin
Objectives
26/10/2012
[Slide 33]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Not use AR
«wish list»
Large
Databases
Softcomputing
Techniques
Scalable
Approach

34. Input Bin
Objectives
26/10/2012
[Slide 34]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
To not use AR
«wish list»
Large
Databases
Softcomputing
Techniques
Scalable
Approach

35. Input Bin
Objectives
26/10/2012
[Slide 35]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Large
Databases
Softcomputing
Techniques
Scalable
Approach
To not use AR
«wish list»

36. Input Bin
Objectives
26/10/2012
[Slide 36]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Large
Databases
Softcomputing
Techniques
Scalable
Approach
To not use AR
«wish list»

37. Input Bin
Objectives
26/10/2012
[Slide 37]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Large
Databases
Softcomputing
Techniques
Scalable
Approach
To not use AR
«wish list»

38. Input Bin
Sub-Objectives
26/10/2012
[Slide 38]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Multi-Layer Architecture
 Task Delegation for Multiple Services
 Efficient Platform
 Transparent Location
 Based on Intelligent Agents (Wooldridge):
 Autonomy
 Social ability
 Reactivity
 Proactivity

39. Input Bin
Sub-Objectives
26/10/2012
[Slide 39]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Multi-Layer Architecture
 Task Delegation for Multiple Services
 Efficient Platform
 Transparent Location
 Based on Intelligent Agents (Wooldridge):
 Autonomy
 Social ability
 Reactivity
 Proactivity

40. Input Bin
Sub-Objectives
26/10/2012
[Slide 40]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Multi-Layer Architecture
 Task Delegation for Multiple Services
 Efficient Platform
 Transparent Location
 Based on Intelligent Agents (Wooldridge):
 Autonomy
 Social ability
 Reactivity
 Proactivity

41. Input Bin
Sub-Objectives
26/10/2012
[Slide 41]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Multi-Layer Architecture
 Task Delegation for Multiple Services
 Efficient Platform
 Transparent Location
 Based on Intelligent Agents (Wooldridge):
 Autonomy
 Social ability
 Reactivity
 Proactivity

42. Input Bin
Sub-Objectives
26/10/2012
[Slide 42]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Multi-Layer Architecture
 Task Delegation for Multiple Services
 Efficient Platform
 Transparent Location
 Based on Intelligent Agents (Wooldridge):
 Autonomy
 Social ability
 Reactivity
 Proactivity

43. Input Bin
Sub-Objectives
26/10/2012
[Slide 43]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Multi-Layer Architecture
 Task Delegation for Multiple Services
 Efficient Platform
 Transparent Location
 Based on Intelligent Agents (Wooldridge):
 Autonomy
 Social ability
 Reactivity
 Proactivity

44. RelatedWorks
Intro Objectives Related Architecture Results Future26/10/2012
[Slide 44]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha

45. VisualTracking
26/10/2012
[Slide 45]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Compute camera pose using local
frame of reference.
Compute Relative Pose

46. Notonly«Visual»...6DOF
26/10/2012
[Slide 46]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 [Poner Vídeo de I. Shutterland]

47. TrackingorLocalization
26/10/2012
[Slide 47]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Relative or
Absolute...
 + Known Points
+ Precision
 «Easy» definition
of relations

48. 26/10/2012
[Slide 48]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Relative or
Absolute...
 + Known Points
+ Precision
 «Easy» definition
of relations
TrackingorLocalization

49. 26/10/2012
[Slide 49]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Global Position
Composite Matrix
 Relative or
Absolute...
 + Known Points
+ Precision
 «Simple» definition
of relations
TrackingorLocalization

50. 26/10/2012
[Slide 50]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
ExternalCameras
Obtain camera pose
with respect to fixed
video sources.

51. 26/10/2012
[Slide 51]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
VisualTrackingMethods
Fiducial Markers
Markerless Based Methods
 Natural Feature Tracking (KeyPoint)
 Model Based (IPB, PS, ...)
 On-the-fly reconstruction (SLAM)
 Hybrid Methods (+IMU...)

52. 26/10/2012
[Slide 52]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
FiducialMarkers
Binarization Components
Extraction
Vertexes and Edge
Detection
MarkersSearch
2DPosition
Perspective
PatternID
GetTransMat
3DRendering

53. Input Bin
FiducialMarkers:ExampleofUse
26/10/2012
[Slide 53]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Virtual Museum
of Computing
2007 (ESI/UCLM)
http://www.esi.uclm.es/museo/

54. FiducialMarkers
Limitations
26/10/2012
[Slide 54]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Marker Size (cm)
Detection(Dist)(cm)
Error / Marker Size (cm)
Rotation

55. FiducialMarkers
Limitations
26/10/2012
[Slide 55]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Origin of GRS
placed at the
center of the
Marker.
Result of propagating the
same error
 Be careful when using them as a
localization method... Propagation error!

56. FiducialMarkers
Limitations
26/10/2012
[Slide 56]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Origin of GRS
placed at the
center of the
Marker.
Result of propagating the
same error
 Be careful when using them as
localization method... Propagate error!

57. 26/10/2012
[Slide 57]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
VisualTrackingMethods
Fiducial Markers
Markerless-Based Methods
 Natural Feature Tracking (KeyPoint)
 Model Based (IPB, PS, ...)
 On-the-fly reconstruction (SLAM)
 Hybrid Methods (+IMU...)

58. 26/10/2012
[Slide 58]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
VisualTrackingMethods
Fiducial Markers
Markerless-Based Methods
 Natural Feature Tracking (KeyPoint)
 Model Based (IPB, PS, ...)
 On-the-fly reconstruction (SLAM)
 Hybrid Methods (+IMU...)

59. NaturalFeatureTracking
26/10/2012
[Slide 59]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Keypoint Extraction (Offline)
 Keypoint Matching
 Pose Calculation

60. 26/10/2012
[Slide 60]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
VisualTrackingMethods
Fiducial Markers
Markerless-Based Methods
 Natural Feature Tracking (KeyPoint)
 Model Based (IPB, PS, ...)
 On-the-fly reconstruction (SLAM)
 Hybrid Methods (+IMU...)

61. ModelBasedTracking
26/10/2012
[Slide 61]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Control Points Sampling
 Visible Edges Detection
 Control Points Matching
 Pose Calculation

62. 26/10/2012
[Slide 62]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
VisualTrackingMethods
Fiducial Markers
Markerless-Based Methods
 Natural Feature Tracking (KeyPoint)
 Model Based (IPB, PS, ...)
 On-the-fly reconstruction (SLAM)
 Hybrid Methods (+IMU...)

63. On-the-flycalculationTracking
26/10/2012
[Slide 63]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
PTAMM
[Castle 2010]

64. 26/10/2012
[Slide 64]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
VisualTrackingMethods
Fiducial Markers
Markerless-Based Methods
 Natural Feature Tracking (KeyPoint)
 Model Based (IPB, PS, ...)
 On-the-fly reconstruction (SLAM)
 Hybrid Methods (+IMU...)

65. 26/10/2012
[Slide 65]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
PreviousWork:HesperiaProject

66. OurProposal
Intro Objectives Related Architecture Results Future26/10/2012
[Slide 66]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha

67. Input Bin
26/10/2012
[Slide 67]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Global
Location
Architecture
 Distributed
 Middleware:
ZeroC Ice
 Location
Providers

68. OurAgentPlatform
26/10/2012
[Slide 68]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

69. Video
Stream
Lightweight
Agent
Lightweight
Agent
RFID
Agent
Audio
Stream
Sensor
Layer
Static
Agents
Mobile
Agents
Input Bin
ARLocation&RenderingService
26/10/2012
[Slide 69]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

70. Tracking
Layer
Inertial
System
Estimator
Agent
Compass
Acelerom.
Gyros.
GPS
Agent
Absolute Methods Relative Methods
Fiducial
Agent
Leds
Agent
Point T.
Agent
Op. Flow
Agent
Video
Stream
Lightweight
Agent
Lightweight
Agent
RFID
Agent
Audio
Stream
Sensor
Layer
Static
Agents
Mobile
Agents
ARLocation&RenderingService
26/10/2012
[Slide 70]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

71. Tracking
Layer
Inertial
System
Estimator
Agent
Compass
Acelerom.
Gyros.
GPS
Agent
Absolute Methods Relative Methods
Fiducial
Agent
Leds
Agent
Point T.
Agent
Op. Flow
Agent
Video
Stream
Lightweight
Agent
Lightweight
Agent
RFID
Agent
Audio
Stream
Sensor
Layer
Fusion
Layer Historical
Perception
Agent
Perception
Fusion
Agent
Static
Agents
Mobile
Agents
ARLocation&RenderingService
26/10/2012
[Slide 71]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

72. Tracking
Layer
Inertial
System
Estimator
Agent
Compass
Acelerom.
Gyros.
GPS
Agent
Absolute Methods Relative Methods
Fiducial
Agent
Leds
Agent
Point T.
Agent
Op. Flow
Agent
Video
Stream
Lightweight
Agent
Lightweight
Agent
RFID
Agent
Audio
Stream
Sensor
Layer
Fusion
Layer Historical
Perception
Agent
Perception
Fusion
Agent
3D Repr.
Manager
Agent
Phone
Agent
HMD
Agent
TabletPC
Agent
Representation
Layer
MultiAgent
Render Grid
Static
Agents
Mobile
Agents
ARLocation&RenderingService
26/10/2012
[Slide 72]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

73. Tracking
Layer
Inertial
System
Estimator
Agent
Compass
Acelerom.
Gyros.
GPS
Agent
Absolute Methods Relative Methods
Fiducial
Agent
Leds
Agent
Point T.
Agent
Op. Flow
Agent
Video
Stream
Lightweight
Agent
Lightweight
Agent
RFID
Agent
Audio
Stream
Sensor
Layer
Fusion
Layer Historical
Perception
Agent
Perception
Fusion
Agent
3D Repr.
Manager
Agent
Phone
Agent
HMD
Agent
TabletPC
Agent
Representation
Layer
Service
Layer
MultiAgent
Render Grid
3D Model
Repository
AgentFace
Identification
Agent
Interaction
Agent
Authentication
Agent
Static
Agents
Mobile
Agents
ARLocation&RenderingService
26/10/2012
[Slide 73]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

74. TrackingLayer
26/10/2012
[Slide 74]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Fiducial
Markers
(ARToolKit).
 Tool for
tuning
marker
position.

75. TrackingLayer
26/10/2012
[Slide 75]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Fiducial
Markers
(ARToolKit).
 On-the-fly
Recons-
truction
(PTAMM).
 Tool to
obtain
pose and
scale of
PTAM
Maps.

76.  Fiducial Markers (ARToolKit).
 On-the-fly Reconstruction (PTAMM).
 Visual Descriptors. QBIC.
 Lucene Indexes: ColourLayout and
Edge Histogram.
 Query: Voronoi Tree.
TrackingLayer
26/10/2012
[Slide 76]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

77. TrackingLayer:VisualDescriptors
26/10/2012
[Slide 77]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
Position: 2.5, 5.5
Rotation: 135º

78. TrackingLayer
26/10/2012
[Slide 78]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
 Fiducial Markers (ARToolKit).
 On-the-fly Reconstruction (PTAMM).
 Visual Descriptors.
 Stereo Location.
1. Features to track: Shi-Tomasi.
2. Optical-Flow: Lucas-Kanade.
3. Fundamental Matrix: LmedS.
4. Fundamental + Intrinsic =
Essential
Rotation Estimation (Rodrígues).

79. Tracking
Layer
Inertial
System
Estimator
Agent
Compass
Acelerom.
Gyros.
GPS
Agent
Absolute Methods Relative Methods
Fiducial
Agent
Leds
Agent
Point T.
Agent
Op. Flow
Agent
Video
Stream
Lightweight
Agent
Lightweight
Agent
RFID
Agent
Audio
Stream
Sensor
Layer
Fusion
Layer Historical
Perception
Agent
Perception
Fusion
Agent
3D Repr.
Manager
Agent
Phone
Agent
HMD
Agent
TabletPC
Agent
Representation
Layer
Service
Layer
MultiAgent
Render Grid
3D Model
Repository
AgentFace
Identification
Agent
Interaction
Agent
Authentication
Agent
Static
Agents
Mobile
Agents
ARLocation&RenderingService
26/10/2012
[Slide 79]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

80. FusionLayer
26/10/2012
[Slide 80]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
With Historical
Without Hist.
 Objective: Tracking Stabilization.
 Solution: Use of a historical vector.
 Issues: Precision loss.

81. FuzzyController
26/10/2012
[Slide 81]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
OpticalFlow PTAM Weight
NG NP Z PP PG NG NP Z PP PG LOW MED HIGH
Weight

82. 26/10/2012
[Slide 82]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
R1/ if(OF == NG & PT == NG) -> P = HIGH;
R2/ if(OF == NP & PT == NP) -> P = HIGH;
R3/ if(OF == Z & PT == Z) -> P = HIGH;
R4/ if(OF == PP & PT == PP) -> P = HIGH;
R5/ if(OF == PG & PT == PG) -> P = HIGH;
R6/ if((OF == PP | OF == PG) & (PT == NP | PT == NG)) -> P = LOW;
R7/ if((OF == NP | OF == NG) & (PT == PP | PT == PG)) -> P = LOW;
R8/ if((OF == NP | OF == Z) & PT == NG) -> P = MED;
R9/ if((OF == PP | OF == Z) & PT == PG) -> P = MED;
R10/ if(OF == NG & (PT == NP | PT == Z)) -> P = MED;
R11/ if(OF == PG & (PT == PP | PT == Z)) -> P = MED;
Fuzzy Controller (11 Rules):
FuzzyController

83. 26/10/2012
[Slide 83]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
FuzzyController

84. Results
Intro Objectives Related Architecture Results Future26/10/2012
[Slide 84]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha

85. Input Bin
Results
26/10/2012
[Slide 85]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future

86. FutureWorks
Intro Objectives Related Architecture Results Future26/10/2012
[Slide 86]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha

87. 26/10/2012
[Slide 87]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
OpenChallenges

88. 26/10/2012
[Slide 88]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
OpenChallenges

89. 26/10/2012
[Slide 89]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
OpenChallenges

90. 26/10/2012
[Slide 90]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
OpenChallenges

91. 26/10/2012
[Slide 91]Carlos Gonzalez Morcillo (Carlos.Gonzalez@uclm.es) – University of Castilla-La Mancha
Intro Objectives Related Architecture Results Future
OpenChallenges