paper pdf download link: http://ubisec.cse.buffalo.edu/files/crowdmap.pdf

1. CrowdMap: Accurate Reconstruction of
Indoor Floor Plans from Crowdsourced
Sensor-Rich Videos
Si Chen, Muyuan Li, Kui Ren, Chunming Qiao
Department of Computer Science and Engineering
University at Buffalo – State University of New York

2. Page  2
Outline
 1. Introduction
 2. System Architecture and Design Details
 3. Implementation and Evaluation
 4. Future Work

3. Page  3
Outdoor Maps
Google Map
OpenStreetMap

4. Page  4
Indoor Maps (Floor Plans)
 Unlike outdoor environment, acquiring digital indoor floor plan information
is very challenging.
 The state-of-the-art Google Indoor Maps only have 10,000 locations
available worldwide, which is not in a position to compete with the total
number of indoor environments around the world.

5. Page  5
Indoor Maps
 The complexity of the indoor environment is the major obstacle to
achieve ubiquitous coverage.
 Existing centralized collection and on-site calibration techniques
demand professional devices and multi-party coordination, which are time
consuming, inconvenient and costly.
Google Trekker http://www.navvis.lmt.ei.tum.de/about/

6. Page  6
Crowdsourcing
http://cdn.thephonebulletin.com/wp-content/uploads/2013/01/smartphone-sensors.jpg
Recently, the wide availability of smartphones and wearable devices (e.g.
google glasses) equipped with built-in visual, acoustic and inertial
sensors makes the mobile user easier than ever to devote themselves to
contribute mobile data.

7. Page  7
Crowdsourcing
http://senda.uab.es/node/15

8. Page  8
Indoor Floor Plan Reconstruction by Crowdsourcing
There have been several studies trying to explore the possibility of using
crowdsourced inertial sensory data to generate an indoor floor plan
automatically.
Alzantot, Moustafa, and Moustafa Youssef. "Crowdinside: automatic construction of indoor
floorplans." Proceedings of the 20th International Conference on Advances in Geographic Information
Systems. ACM, 2012.

9. Page  9
Drawback of Inertial Sensor Only Methods
 However, current crowdsourcing floor plan reconstruction systems are not
able to produce accurate enough results.
– Most of existing indoor floor plan reconstruction systems heavily rely on inertial
data.
The premise of their work is that users would be
able to move across all edges and corners in
an indoor environment.
• the edge of an indoor scene is usually
blocked by furniture or other objects,
• some restricted areas in an indoor
environment are also inaccessible for most
of the users
Visual information preserve more context information for an unknown
indoor environment, such as the geometric information, color information,
lighting conditions and text information.
[1].http://www.sigmobile.org/mobicom/2014/talks/slides_6_3.pdf
[1]

10. Page  10
CrowdMap: Key Idea
 CrowdMap: An accurate indoor floor plan reconstruction system based on
sensor-rich videos.
 Key idea: leverage the spatio-temporal relationship between each
consecutive frame of the crowdsourced video.

11. Page  11
CrowdMap: Floor Plan
Basic elements of building floor plan

12. Page  12
Outline
 1. Introduction
 2. System Architecture and Design Details
 3. Implementation and Evaluation
 4. Future Work

13. Page  13
CrowdMap: System Architecture
mobile front-end
cloud back-end
i) crowdsourced data
collection module
ii) indoor path modeling
module
iii) room layout modeling
module
iv) floor plan modeling module

14. Page  14
CrowdMap: Crowdsourced Data Collection Module
We assume that users actively get involved in the data collecting tasks.
Example: a user opens our mobile application and inputs the floor
number, starts capturing the room environment by spinning his/her body
(SRS task); then, walks towards the hallway (SWS task).
The walking distance |AB| is calculated by the step counting method. In
addition, the direction change of each step ∆𝜔 is calculated by jointly
using compass, gyroscope and accelerometer.

15. Page  15
CrowdMap: Crowdsourced Data Collection Module
By using the inertial sensor data, we are able to reconstruct the
trajectory of the user when they perform the SWS task.

16. Page  16
CrowdMap: Indoor Path Modeling Module
How to aggregate multiple user trajectories and reconstruct the path of
the floor?

17. Page  17
CrowdMap: Indoor Path Modeling Module
[1].Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. "Surf: Speeded up robust features." Computer
vision–ECCV 2006. Springer Berlin Heidelberg, 2006. 404-417.
We select the state-of-the-art SURF [1] algorithm to precisely match two
candidate key-frames

18. Page  18
CrowdMap: Indoor Path Modeling Module
Correct Matching

19. Page  19
CrowdMap: Indoor Path Modeling Module
Incorrect Matching

20. Page  20
CrowdMap: Indoor Path Modeling Module
 We use multiple key-frames to determine whether the two user
trajectories can be merged.
 If there is a match between the two trajectories generated in the same
floor, there should be a common path between them in a high
probability. Hence, we use the longest common subsequence to
capture this notion.
Where Ta and Tb are the two user trajectories with length of i
and j, respectively. Parameter 𝛿 represents the maximum
length difference between two user trajectories and 𝜀 is the
distance threshold.

21. Page  21
CrowdMap: Indoor Path Modeling Module

22. Page  22
CrowdMap: Room Layout Modeling Module
We utilize crowdsourced images to create the panorama for each
place, and then use computer vision techniques to process the
panorama, and thereby, generate the room layout.

23. Page  23
CrowdMap: Room Layout Modeling Module

24. Page  24
CrowdMap: Room Layout Modeling Module

25. Page  25
CrowdMap: Floor Plan Modeling Module

26. Page  26
CrowdMap: Floor Plan Modeling Module

27. Page  27
CrowdMap: Floor Plan Modeling Module

28. Page  28
Outline
 1. Introduction
 2. System Architecture and Design Details
 3. Implementation and Evaluation
 4. Future Work

29. Page  29
CrowdMap: Implementation

30. Page  30
CrowdMap: Implementation
Screenshot for CrowdMap Mobile Frontend

31. Page  31
Microsoft Azure Platform

32. Page  32
CrowdMap: Evaluation

33. Page  33
CrowdMap: Evaluation

34. Page  34
Compare with Structure from Motion (SfM)

35. Page  35
Outline
 1. Introduction
 2. System Architecture and Design Details
 3. Implementation and Evaluation
 4. Future Work

36. Page  36
CrowdMap: Future Work
We will focus on further processing of the room panorama to extract
more context information of the room:
• object detection
• object recognition.
We also plan to further study several issues:
• user incentive mechanism
• privacy preservation mechanism.

37. Page  37