Clemens Havas from Uni Salzburg

1. 27 November 2015
3D Rapid Mapping on Mobile Devices
Clemens Havas, Jakob Miksch, Nikolai Janakiev, Muhammad
Salman, Bernd Resch & Thomas Blaschke
WhereCamp 2015
1

2. Clemens Havas 27 November 2015
Processing Structure from Motion on standard
smartphones in real time
Navigate through a building by simply taking pictures
of the rooms
2
Overview :: Intention

3. Clemens Havas 27 November 2015
Smartphone takes pictures
Comparison with database
(server)
Delivers location
3
Chadly Marouane et al. “Visual Positioning Systems - An Extension to MoVIPS".
In: 5th International Conference on Indoor Positioning and Indoor Navigation (IPIN
2014) 2014, pp. 1-10.
Martin Werner, Moritz Kessel, and Chadly Marouane. “Indoor positioning using
smartphone camera". In: Indoor Positioning and Indoor Navigation (IPIN), 2011
International Conference on IEEE. 2011, pp. 1-6.
Related Work :: Indoor Positioning

4. Clemens Havas 27 November 2015
Real time
3D model generation
Processing completely on
the smartphone
4
Peter Ondruska, Pushmeet Kohli, and Shahram Izadi. “MobileFusion: Real-time
Volumetric Surface Reconstruction and Dense Tracking On Mobile Phones". In:
International Symposium on Mixed and Augmented Reality (ISMAR). Fukuoka,
Japan, 2015
Related Work :: Mobile Fusion

5. Clemens Havas 27 November 2015
5
Images
Structure from Motion
Point Cloud Processing
Visualisation
( Fingerprinting + Indoor
Positioning)
Overview :: Workflow

6. Clemens Havas 27 November 2015
6
Name Open Source Lib./Soft. Support
Theia Yes Library University Project
Point Cloud Library Yes Library Companies
LIBVISO2 (No) Library University Project
OpenMVG Yes Library University Project
Aerial Vision Group No Software University Project
aSPECT 3D No Software Company
Depth Image
Processing
Yes Library University Project
Visual SFM No Software University Project
SfM :: Finding the best Library/Software
Other criteria
Dependencies
Input/Output
Documentation
API
Features
Performance
Support

7. Clemens Havas 27 November 2015
Written in C++
Collection of tiny open source libraries
Complete SfM Pipeline
Incremental and Global Pipeline
Support by the author: Pierre Moulon
7
https://openmvg.readthedocs.org/en/latest/
SfM :: OpenMVG

8. Clemens Havas 27 November 2015
OpenMVG just for Linux and Windows
How to build OpenMVG for Android?
Usage of C++11 Standard: Not fully compatible with
Android
Many external libraries (Eigen, Ceres-Solver,
Cereal,...)
8
SfM :: Challenges

9. Clemens Havas 27 November 2015
Solved the stated challenges and succeded in
making OpenMVG usable for Android
Successfully performed parts of the SfM Pipeline
9
SfM :: Results

10. Clemens Havas 27 November 2015
Taking several different areas of a room and test the
functionalities of OpenMVG
10
Computation :: Test OpenMVG

11. Clemens Havas 27 November 2015
11
Computation :: Global SfM Pipeline
http://www.theia-
sfm.org/sfm.html#global-
sfm-pipeline

12. Clemens Havas 27 November 2015
12
Normal High
Computation :: Settings

13. Clemens Havas 27 November 2015
13
Ultra
Computation :: Settings

14. Clemens Havas 27 November 2015
Finding the best parameters for the resolution and
the number of images
Point cloud generation with a “windowing” method
with overlap to reduce duration of computation
14
Computation :: Best Parameters

15. Clemens Havas 27 November 2015
15
Computation :: Time vs Points

16. Clemens Havas 27 November 2015
16
Computation :: Different Resolutions

17. Clemens Havas 27 November 2015
One combined app (SfM, Visualisation)
Point cloud merging and reduction
Fingerprinting algorithm
Indoor Positioning
17
Conclusion :: Future Work

18. Clemens Havas 27 November 2015
Best point cloud reduction
algorithm?
Fingerprinting algorithm for
indoor navigation?
SURF better solution for SfM?
Increase algorithm performance?
18
Conclusion :: Open Questions