Presentation talks about BOSS and Content Analysis along with Dapper.

1. BOSS around the web
Souri Datta
Structured Data Extraction Team
http://www.flickr.com/photos/sumrow/1267682594/sizes/l/

2. BOSS is
Build your own
search service
http://developer.yahoo.com/search/boss/

3. Provides APIs
To our Search
database

4. TO BUILD your own
powerful
Search applications

5. BOSS allows you to
search over
Web, images,
news & Blogs

7. BOSS Pricing

8. Free for building your hacks!!

9. BOSS uses OAuth for security
Code : https://github.com/sourind/hacku/

10. Get a FREE
consumer key and
secret
http://hackyourworld.org/hacku/

11. http://developer.yahoo.com/yql/consol
e/

13. 1. Select yql query
2. Select output format
3. Copy This url

15. Finding images of “The Dark Knight
Rises”
select * from boss.search where q="The
Dark Knight Rises" and service="images"
and ck="..." and secret="..."

16. Finding “The Dark Knight Rises” in
IMDB, movies.yahoo.com
select * from boss.search where q="The Dark
Knight Rises" and
sites="imdb.com,movies.yahoo.com" and
ck="..." and secret="..."

17. Spell Check and Correction
select * from boss.search where q="The Dirk
Knight Rises" and service="spelling" and
ck="..." and secret="..."

18. Finding news on “The Dark Knight
Rises”
select * from boss.search where q="The Dark
Knight Rises" and service="news" and
ck="..." and secret="..."

19. select * from contentanalysis.analyze where
text="Sachin Tendulkar is batting very well"

20. Content Analysis from a URL
select * from contentanalysis.analyze where
url="http://www.cnn.com/"

21. Lets See it in Action!

22. Query Cheatsheet
• Find images of “The Dark Knight Rises”
• select * from boss.search where q="The Dark Knight
Rises" and service="images" and ck="..." and secret="..."
• Find reviews of “The Dark Knight Rises”
• select * from boss.search where q="reviews intitle:The
Dark Knight Rises" and service="web" and ck="..." and
secret="…"
• Search for Avatar but not the movie:
• select * from boss.search where q="Avatar -movie" and
ck="..." and secret="... "
• Search pdfs of “The Dark Knight Rises”
• select * from boss.search where q="The Dark Knight
Rises" and type="pdf" and ck="..." and secret="..."

23. Query Cheatsheet
• Find all the news of “The Dark Knight Rises”
• select * from boss.search where q="The Dark Knight
Rises" and service="news" and ck="..." and secret="..."
• Get long abstracts in the results
• select * from boss.search where q="The Dark Knight
Rises" and abstract="long" and ck="..." and secret="…"
• Retrieve 51-100 results of the query
• select * from boss.search where q="The Dark Knight
Rises" and start=51 and ck="..." and secret="... "

24. EXAMPLES

25. duckduckgo.com

28. Why extraction is difficult?
• Internet has lot of information
• Not all can be processed by machines
– Unstructured data
– E.g. DiscountedPrice and RedcudedPrice of a product
(both mean the same)
• Ultimate aim is to publish data in structured
format
• Most simple way- xml,json

29. Web Scraping
• Demo Dapper

30. More Resources
• Yahoo! BOSS: http://developer.yahoo.com/boss
• BOSS Technical Documentation:
http://developer.yahoo.com/search/boss/boss_api
_guide/
• Content Analysis :
http://developer.yahoo.com/contentanalysis/
• Oauth sample code :
https://github.com/sourind/hacku/

31. Questions??
http://www.flickr.com/photos/reem_unique/4119729692/

32. Thanks!!
• http://slideshare.net/souridatta
• https://github.com/sourind/