Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Conversational Internet - Creating a natural language interface for web pages

Presenting a paper to the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A 2013)

  • Als Erste(r) kommentieren

Conversational Internet - Creating a natural language interface for web pages

  1. 1. © 2009 IBM CorporationConversational Internet:A natural language interface for webpagesDale Lane – IBM United Kingdom14 May 2013dale.lane@uk.ibm.com
  2. 2. © 2012 IBM Corporation2Challenge
  3. 3. © 2012 IBM Corporation3Challenge
  4. 4. © 2012 IBM Corporation4Motivation
  5. 5. © 2012 IBM Corporation5Understanding the page : Identifying type
  6. 6. © 2012 IBM Corporation6Understanding the page : Identifying navigation options
  7. 7. © 2012 IBM Corporation7Understanding the page : Identifying calls-to-action
  8. 8. © 2012 IBM Corporation8Understanding the user : Retrieving information
  9. 9. © 2012 IBM Corporation9Understanding the user : Mouse actions
  10. 10. © 2012 IBM Corporation10Understanding the user : Mouse actions
  11. 11. © 2012 IBM Corporation11Understanding the user : Keyboard actions
  12. 12. © 2012 IBM Corporation12Understanding the user : Keyboard actions
  13. 13. © 2012 IBM Corporation13Understanding the user : Keyboard actions
  14. 14. © 2012 IBM Corporation14Conversational Internet Presenting an early-stage prototype being developed to explore the potential forquestion answering as an alternative approach to screen-readers for retrievinginformation from web pages Architecture and approach inspired by active area of research and development inquestion answering on a knowledge derived from a corpus of documents(Ferucci, Lally, Chu-Carroll, et al)http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717
  15. 15. © 2012 IBM Corporation15ImplementationbrowserextensionUIMALanguageWareJavaclient server
  16. 16. © 2012 IBM Corporation16Implementationnew request new requestresponsewithconv. idresponsefollow-uprequestrequestwithconv. id“what can I do?”“the options are...”“I want to do...”
  17. 17. © 2012 IBM Corporation17Implementation : Client Firefox extension Submits current state ofthe page to server foranalysis Training mode
  18. 18. © 2012 IBM Corporation18Implementation : Server Analyzing the page Processing user queries
  19. 19. © 2012 IBM Corporation19Implementation : Server : Understanding the page
  20. 20. © 2012 IBM Corporation20Implementation : Server : Understanding the pageWhat type of site is this? Machine learning classifiers Whitelists of known domains
  21. 21. © 2012 IBM Corporation21Implementation : Server : Understanding the pageWhat can we infer from markup used? Semantic tags ARIA CSS class names
  22. 22. © 2012 IBM Corporation22Implementation : Server : Understanding the pageWhat does the structure of the page layout suggest? Machine learning models for common page elements
  23. 23. © 2012 IBM Corporation23Implementation : Server : Understanding the pageWhat does the text of the page tell us? Natural Language Processing using LanguageWare to recognize common formsof call-to-action
  24. 24. © 2012 IBM Corporation24Implementation : Server : Responding to queries
  25. 25. © 2012 IBM Corporation25Implementation : Server : Responding to queriesInterpreting the query NLP rules created with LanguageWare to map to closest known command type WordNet to attempt matches using synonyms of unknown terms
  26. 26. © 2012 IBM Corporation26Implementation : Server : Responding to queriesExtracting the requested information Information requested from page extracted from serialized CAS created by firstpipeline, by retrieving sections with relevant annotations
  27. 27. © 2012 IBM Corporation27Implementation : Server : Responding to queriesPreparing a response Speech generated using Nuance NDev and streamed to client
  28. 28. © 2012 IBM Corporation28Future work / Limitations Use cases Usability testing RIA / AJAX sites
  29. 29. © 2012 IBM Corporation29 Paper submitted to W4A– http://goo.gl/3X2iv Overview presentation– http://youtu.be/uS6oquJdgbw Demonstration of the prototype– http://youtu.be/tSGyPCcO-bYDale Lanedale.lane@uk.ibm.com@dalelane

×