Weitere Ă€hnliche Inhalte Ăhnlich wie Lightning talk :IBM Content Analytics with Enterprise Search - Wolfgang Jung (20) Mehr von lucenerevolution (20) KĂŒrzlich hochgeladen (20) Lightning talk :IBM Content Analytics with Enterprise Search - Wolfgang Jung1. LIGHTNING TALKS
Powered by Lucene:
IBM Content Analytics with Enterprise Search
Wolfgang Jung
Barcelona, 19th October 2011 © 2011 IBM Corporation
2. IBM Content Analytics with Enterprise Search
Our agenda in the next 10 minutes
LIGHTNING TALKS
IBM is commited to Open Source
â Decade of contribution to the community.
Adoption of Apache Lucene to IBM Content Analytics
â The Why, What & examples.
Demonstration of IBM Content Analytics
â see the development results live.
Be enlightened !
2 © 2011 IBM Corporation
3. IBM Content Analytics with Enterprise Search
IBM is commited to Open Source
Decade of lineage and contributions to the open source community
â Apache Hadoop.
IBMâs use of BigIndex for Search is mention in Chuck Lamsâs âHadopp in Actionâ
â Apache Derby
â Apache Geronimo and Jetty
â Eclipse: Founded by IBM, PMC Board of Directors
â Apache UIMA: Unstructured Information Management Architecture.
Developed by IBM, Contributed to Apache
â Apache Jakarta: Lucene. PMC members
Significant contributions via IBM Lucene Extension Library (ILEL)
â Linux ... and more!
3 © 2011 IBM Corporation
4. IBM Content Analytics with Enterprise Search
Adoption of Apache Lucene
to IBM Content Analytics with Enterprise Search
The use of UIMA is existing since first release in 2005 of IBM OmniFind and later
IBM Content Analytics, continued into todayâs IBM Content Analytics with
Enterprise Search
http://www-01.ibm.com/software/data/content-management/analytics/uima.html
IBMâs decision for the use of Lucene
âIndex is a common technology and better to improve
âlower cost of maintenance
âadvantage in incremental indexing
âextensibility
4 © 2011 IBM Corporation
5. IBM Content Analytics with Enterprise Search
Adoption of Apache Lucene
to IBM Content Analytics with Enterprise Search
IBM is a very active contributor. Look for PMC members:
âMichael McCandless; Shai Erera; Doron Cohen
http://lucene.apache.org/who.html
IBM extended Lucene based on our needs. Two examples already
contributed to community :
âQuery Parser
âFacets
5 © 2011 IBM Corporation
6. IBM Content Analytics with Enterprise Search
Adoption of Apache Lucene
to IBM Content Analytics with Enterprise Search
On 13th December 2006, IBM and Yahoo! announced IBM OmniFind Yahoo! Edition, as
âno-cost, entry level enterprise search product developed to help eliminate financial and
technology barriers to intranet and Web search.â
http://www-03.ibm.com/press/us/en/pressrelease/20767.wss
This technology included Lucene as index technology and had full support by IBM
â 45,000+ downloads from the website http://omnifind.ibm.yahoo.net
â IBM support contracts for clients with âIBM Elite Support for OmniFind Yahoo Editionâ
â Below 15 incidents regarding index technology
Technology is seen as success for IBM
6 © 2011 IBM Corporation
7. IBM Content Analytics with Enterprise Search
Content Analytics generates new insights and aggregates key
findings gathered from large data volumes in a visualized form
Extracted Concept
Claimant: Soft Tissue Injury
Automatic
Visualizing
Person Injury Body Part Location Results of concept evaluation
are displayed to the users
Noun Verb Noun Phrase Prep Phrase
Claus sprained his ankle on the step
Analysed documents
with identified concepts
Sources of Information
Internal (ECM, Files, DBMS, etc.)
and External (Social, News, etc.)
7 © 2011 IBM Corporation
8. IBM Content Analytics with Enterprise Search
Rapid Insights from Automotive Complaints
We will be using publically available data from the National Highway Traffic Safety Agency (NHTSA)
to demonstrate how IBM Content Analytics can be used to identify problems with automobiles.
NHTSA receives various reports about malfunctions, accidents, and other issues with automobiles
from dealerships, repair facilities, and from the general public. NHTSA publishes the data at
http://www.nhtsa.gov. For this demo we have created a collection from the NHTSA âcomplaintsâ
data spanning several years ending in early 2010. We will show how this and similar data can be
analyzed to arrive at rapid insights not possible by manually reading through the complaint records.
8 © 2011 IBM Corporation
12. LIGHTNING TALKS
Powered by Lucene:
IBM Content Analytics with Enterprise Search
Wolfgang Jung
Barcelona, 19th October 2011 © 2011 IBM Corporation