For 2014 MSR-Bing Image Retrieval Challenge written by Win

1. Text
Image Retrieval Challenge
-Enhance relationships between query and image
Instructor：MeiChenYeh
ChenLinYu,ChiungWei Hsu
VIPLAB

2. Outline
1. Proposed method
2. Evaluation Metric
3. Experiment Result
4. Finding and Difficulty
5. Demo
6. Conclusion
7. Future work

3. Proposed Method

4. Query
Natural
Language
Processing
Tokenization
POSt
QE by WordNet QE by Wikipedia
WordNet Wikipedia
Click_count ranking Top candidates
User Clicklog from
MSR dataset

5. Apple
apple
apples
an apple
….

6. Query Processing
1. Stop word and removal
2. Tokenization
3. Stemming and Lemmatization
4. Part-of-speech Tagging
5. Wiki-suggestion (Misspelled words)
6. Expansion (wordnet and wikipeia)

7. Apple
apple
apples
an apple
….
Ranking Table
log candidate count image
apple 1890 QYQtQsx9lH
1KwA
apple 503 QJ4gfSPJYh
bw0A
… … …
apple mac 490 PvfGna70qGi
BIA

8. Click-count Ranking
MSR dataset provide real world data for user query
log.
With this, generated homemade searching table
by“Click-count”.
“Max click count rule”
Log data 1,000,000 (only 1/20)
We can make sure that candidate pictures are
most popular.

9. Apple
apple
apples
an apple
….
Ranking Table
log candidate count image
apple 1890 QYQtQsx9lH
1KwA
apple 503 QJ4gfSPJYh
bw0A
… … …
apple mac 490 PvfGna70qGi
BIA

10. Evaluation Metric

11. MSR vs DIY Method
!
!
[rel]={Excellent=3,Good=2,Bad=0}
X
✔

12. Experiment Result

13. Prepare and Work
Off-line:
NLTK to process user query log
Build Ranking table (1,000,000)
Include image(base64) to Database(800,000)
On-line:
NLTK to process query input
Query expansion by word net and wikipedia
Large-scale database query processing

14. Single unit-query
'president','frank','mars','chinese','taiwan','
dargon','crash','bird','France','Eiffel','presid
ent','tony','frank','mars','chinese','taiwan','L
ondon','Mexican','ydney',
'google','yahoo','jessica','microsoft','amazo
n','windows','apple','line','linux','android',
'world','iphone','bacteria','cat','basketball','
dog','micky','tom','jerry','christmas','table',
Test : 32 queries Acc:87.5 %

15. Compound word-query
book store, picture frame, the lost and bewildered
tourist, ice cream, cell phone, apple pie, a story as
old as time, a cool wet afternoon, many cases of
infectious disease
swimming pool, the senlie old man,pencil box , long
and winding road, tiddy bear , hot dog, jennifer
love hewitt, some cookie shaped like stars
hello kitty coloring page, kelly osbourne drinking,
micky mouse, a wet amd stinky dog
Test : 20 queries Acc:42.28 %

16. Finding and Difficulty

17. Spelling correctly can improve retrieval accuracy.
Query expansion can find more related images
!
A ambiguous query can be difficult to used.
The gap exists between users and result images,
because the word is polysemic.
The user query still has a semantic problem.
Finding

18. In a compound word query, the relationship
between previous and next word is very
important.
Query semantic is still a challenge.
Large-scale data processing is a big problem.
How to speed up search performance?
Difficulty

19. Demo

20. Conclusion

21. Enhance relationships
between query and image

22. Find relationships
between query and image

23. Future Work

24. Query
Natural
Language
Processing
Tokenization
POSt
QE by WordNet QE by Wikipedia
WordNet Wikipedia
Click_count ranking Top candidates
Named Entity Recognition
User Clicklog from
MSR dataset
Enhance

25. –ChenLin Yu, ChiungWei Hsu
“Thank you”