1. Pascal Kelm
Kelm@nue.tu-berlin.de
Communication Systems Group
www.nue.tu-berlin.de Technische Universität Berlin
Thursday, 04 October 2012

2. What is meant by Spatial Segmentation? 2
World map is iteratively divided into segments of
different sizes
Kelm: How Spatial Segmentation improves the Multimodal Geo-

3. Run4: only audio/visual information 3
Descriptions are pooled for each spatial segment (k-d tree) in
the different hierarchy level
Visual nearest neighbour search in lowest hierarchy
Kelm: How Spatial Segmentation improves the Multimodal Geo-

4. Visual Region Model 4
Returns the visually most similar areas, which are
represented by a mean feature vector of all training images
and videos of the respective area
Kelm: How Spatial Segmentation improves the Multimodal Geo-

5. Run4: Results 5
UG-CU 100
Th [km] TUB [%] [%]
90
1 0,1 0,1
10 0,1 0,7 80
20 0,1 0,9 70
50 0,1 1,1 Accuracy [%] 60
100 0,2 2,6
50
200 0,8 6,9 TUB
UG-CU
40
500 4,1 14,7
1000 14,8 21,2 30
2000 44,5 28,5 20
5000 81,0 29,6
10
10000 98,7 91,4
0
15000 100,0 95,7 1 10 20 50 100 200 500 1000 2000 5000 100001500020000
Margin of Error [km]
20000 100,0 100,0
Kelm: How Spatial Segmentation improves the Multimodal Geo-

6. Run1: No additional data or gazetteers 6
combines textual and visual features: translation of tags and
extracted words (NLP) from the title and the description.
Porter stemmer and stop-word elimination for each segment
and granularity in the spatial segmentation.
Visual Search for the k-nearest segments in the lowest
hierarchy
Kelm: How Spatial Segmentation improves the Multimodal Geo-

7. 7
Term-location-distribution:
Term frequency-inverse document frequency:
Kelm: How Spatial Segmentation improves the Multimodal Geo-

8. Example 8
Condence scores of the visual approach (right) restricted
to be in the most likely spatial segment determined by
the textual approach (left)
Kelm: How Spatial Segmentation improves the Multimodal Geo-

9. Run1: Results 9
Th [km] TUB [%] 100
1 13,7 90
10 32,7 80
20 36,5
70
50 39,4
100 41,8 60
Accuracy [%]
200 44,8 50
500 51,7 40
TUB
1000 62,4 30
2000 76,5
20
5000 92,3
10000 99,4 10
15000 100,0 0
1 10 20 50 100 200 500 1000 2000 5000 10000 15000 20000
20000 100,0 Margin of Error [km]
Kelm: How Spatial Segmentation improves the Multimodal Geo-

10. Run2: No additional data 10
For the highest hierarchy level the boundaries extraction using
gazetteers (GeoNames, Wikipedia and Google Maps) for the
spell checked words is added.
Kelm: How Spatial Segmentation improves the Multimodal Geo-

11. Collaborative Systems: Example 11
這是我上次去巴黎。在那裡，我得
到了我的城堡在迪斯尼樂園看。…
這是我上次去巴黎。在那裡，我得到了我的城堡在迪斯尼樂園看。
Kelm: How Spatial Segmentation improves the Multimodal Geo-

12. Collaborative Systems: Example 12
這是我上次去巴黎。在那裡，我得到了我的城堡在迪斯尼樂園看。…
Which language is it?
Chinese
This was my last trip to Paris. I visited the castle in Disneyland…
Which words gives us information? Tags?
Trip, Paris, Castle, Disneyland
Which of these nouns have got geographical information?
Paris, Disneyland
Kelm: How Spatial Segmentation improves the Multimodal Geo-

13. Geographical Ambiguity 13
Paris Disneyland
R(ci) = Rank sum
France China ci = Countries
N = Number of toponym
Canada USA
Puerto
France
Rico
… …
Kelm: How Spatial Segmentation improves the Multimodal Geo-

14. Extracted geo. items 14
kauii
hawaii
usa
00001: hawaii, kauai, usa
Kelm: How Spatial Segmentation improves the Multimodal Geo-

15. Results 15
100
90
80
70
60
Accuracy [%]
50 Run1
Run2
40 Run4
30
20
10
0
1 10 20 50 100 200 500 1000 2000 5000 10000 15000 20000
Margin of Error [km]
Kelm: How Spatial Segmentation improves the Multimodal Geo-

16. Question 16
Thanks for your attention!
Kelm: How Spatial Segmentation improves the Multimodal Geo-

17. Training Set: Weighting 17
Kelm: How Spatial Segmentation improves the Multimodal Geo-

18. Training Set: Features 18
Kelm: How Spatial Segmentation improves the Multimodal Geo-