Fun with Text - Managing Text Analytics

Cohan Sujay Carlos
CEO, Aiaioo Labs
Fun with Text
Managing Text Analytics

What I am going to talk about.
Text Analytics
1. Examine 3 kinds of opportunities
2. Discuss 3 text analytics problems
3. Touch upon 3 things to watch out for
and 3 things to embrace.

What if we can master “text”?
What do we get from it?
There are opportunities in every vertical:
1. Aerospace / Defense / Automotive –-- Filing of various routine documents /
Technical specification standardization / Competitive intelligence and
customer feedback management

2. Healthcare / Life sciences –-- Reporting / Storing relevant patents and
publications / Analysis of research and competitive intelligence

2. Healthcare / Life sciences –-- Reporting / Storing relevant patents and
publications / Analysis of research and competitive intelligence
3. Legal and Government –-- Legal and administrative filings / Case document
and administrative record management / Analysis of legal and
administrative documents (land records, case files)

Do you observe a pattern?
In every vertical …
Output Text / Store and Transform Text / Ingest and Analyze Text

How do we unlock
the value in “text”?
Natural Language Generation Natural Language Understanding
Natural Language Processing (aka Text Analytics)

Use Case 1:
Customer Service
Let’s say you have some text … … and a database or spreadsheet with columns
“John Chambers of Springfield, MA
reported a problem with the clutch
on his Ford Ranger purchased in
Boston, MA in 2005.”
… and you have to fill in the database fields
from the information in the text …
Reporter Location (of
Reporter)
Product

Use Case 1:
Land Records
“Property K45L234
(lot 23-24) in Wake County
of 3000 sq ft
was sold to James Fischer
on 3-30-1997 …”

Use Case 1:
Land Records
“Property K45L234
(lot 23-24) in Wake County
of 3000 sq ft
was sold to James Fischer
on 3-30-1997 …”
Title Number Lot County

Use Case 1:
M&A Transactions
“Acme Financials, a subsidiary
of Lehman Sisters, was acquired
by John Doe Corp on 5/26/2001.”

Use Case 1:
M&A Transactions
“Acme Financials, a subsidiary
of Lehman Sisters, was acquired
by John Doe Corp on 5/26/2001.”
Acquirer Acquired Date

Use Case 1: Customer Service
[ Information Extraction ]
Entities are pieces of text that could go into the fields in the database.
Identifying entities and the relations between them
Reporter Location (of
Reporter)
Product

Reporter Location Product
John
Chambers
Springfield,
MA
Ford
Ranger

Relations tell you about the connections between entities.
Relations connect the entities that belong in a row.
John
Chambers
Springfield,
MA
Ford
Ranger
Location of Reporter

Information extraction converts:
unstructured information into structured information.
John
Chambers
Springfield,
MA
Ford
Ranger

Information extraction can improve efficiencies
in processes where humans read text and copy fields into databases.
John
Chambers
Springfield,
MA
Ford
Ranger

How can text analytics methods be used
to automate entity and relation extraction?
Rule based methods Machine learning methods
Aiaioo Labs aiaioo.com

Rule-based frameworks for entity and relation extraction?
http://services.gate.ac.uk/annie/

It uses lists of first names and last names of persons, and names of
places … and matches them in the text …
How does GATE/Annie identify entities and the relations?
“John Chambers of Springfield, MA reported a problem with the clutch
on his Ford Ranger purchased in Boston, MA in 2005.”
“Jack”
“Jill”
“John”
“Chambers”
“Miller”
“Farnsworth”
“Springfield”
“Boston”
“Cambridge”
“MA”
“CA”
“MD”

Machine learning frameworks for entity and relation extraction?
https://opennlp.apache.org/
Apache OpenNLP

Machine learning frameworks need training data.
https://opennlp.apache.org/documentation/1.6.0/manual/opennlp.html

From examples such as:
It learns to recognize:
How does OpenNLP identify entities and the relations?
“John Chambers of Springfield, MA reported a problem with the clutch
on his Ford Ranger purchased in Boston, MA in 2005.”
“<START:reporter>John Archer<END> of <START:location>Maryland<END>
reported a problem with his <START:product>Figo<END>.”
“<START:reporter>Vince Chambers<END> of <START:location>Denver,
CO<END> had trouble with his <START:product>Focus<END>.”

How to choose between text analytics methods
for entity and relation extraction?
3 months to reasonably performing model
Typically higher precision
Typically less flexibility
Typically less recall
1+ years to reasonably performing model
Typically lower precision
Typically more flexibility
Typically higher recall + overall performance

5’11”
5’ 8”
Can you classify these door heights as: Short / Tall ?
5’8”
5’11” 6’2”
6’6”
5’ 2”
6’8”
6’9”
6’10”

5’11”
5’ 8”
In analytics, an analyst comes up
with a rule.
5’8”
5’11” 6’2”
6’6”
5’ 2”
6’8”
6’9”
6’10”
If door_height < 6’ then Short else Tall

5’11”
5’ 8”
In machine learning, the computer comes up with a
rule from examples.
5’8”
5’11” 6’2”
6’6”
5’ 2”
6’8”
6’9”
6’10”

How do we unlock
The first use case …
Information Extraction

How do we unlock
The second use case …
Text Categorization
Labeling text with one or more category labels

Use Case 2:
Organizing Text for Storage
Let’s say you have some text … … and you want to mark it as one of …
Report
Inquiry

Use Case 2: Organizing Text
[ Text Categorization ]
Start by collecting some samples of documents
of each of your categories
Report Inquiry
I have a problem
This complaint is about
Where can I buy a
Do you sell furniture

Train a classifierwith them.
Report Inquiry
I have a problem
This complaint is about
Where can I buy a
Do you sell furniture

Start by collecting some samples of documents
of each of your categories
Politics Sports
The United Nations
The United States and
Manchester United
Manchester and Barca

Train a classifierwith them.
Politics Sports
The United Nations
The United States and
Manchester United
Manchester and Barca

Run the classifieron a new piece of text.
The classifierwill return a label.
Politics
Nations and States

How can text analytics methods be used
to automate organization/categorization?

But rule-based methods work for classification too.
Rule-based text categorization is often used in:
Social media sentiment classification

We use lists of negative and positive words (usually adjectives)
(available in the AFINN gazetteer) … and match them in the text …
How do we use rules to identify sentiment?
“I am sad that Steve Jobs died.”
“sad”
“bad”
“evil”
“distraught”
“dead”
“died”
“thrilled”
“excited”
“amazed”
“happy”
“love”
“joy”

Can we use entity and relation extraction to do better?
“I am sad that [Steve Jobs died].”
Analysis: This person holds a positive opinion
of Steve Jobs
The –ve entity ‘sad’ is related to the –ve event ‘Steve Jobs died’.

How to choose between text analytics methods
for text categorization?
Typically higher precision
Typically less flexibility
Typically less recall
Typically lower precision
Typically more flexibility
Typically higher recall + overall performance

How do we unlock
The third use case …
Question Answering
Generating a response to an inquiry

Use Case 3:
Answering Questions
Let’s say you get a question … … and you want to answer to be one of …
“Do you ship your cars to Boston, MA?” Yes
No

Use Case 3:
Answering Questions
First you classify the question into one of 3 types… and these are…
“Do you ship your cars to Boston, MA?”
“Who is the CEO of Apple?”
“Why is the sky blue?”
Yes/No questions
Factoid questions
Non-factoid questions

Use Case 3:
Answering Questions
Look for answers in databases that you created using entity / relationship extraction
“Do you ship your cars to Boston, MA?”
“Who is the CEO of Apple?”
“Why is the sky blue?”
Product Ships To
Cars USA
CEO Firm
Tim Cook Apple

To watch out for:
Text Analytics Traps
1. Testing on Training Data
2. Using US Training Data for India
3. Treating all Data Sources as One

To embrace:
Text Analytics Tricks
1. UI Compensation for AI Inaccuracy
2. Raising Precision at the Cost of Recall
3. Domain Specific Rules

About Aiaioo Labs
AI Research Lab
1. http://aiaioo.com
2. http://aiaioo.com/publications
3. http://aiaioo.wordpress.com

THANK YOU

Fun with Text - Managing Text Analytics

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Fun with Text - Managing Text Analytics

Ähnlich wie Fun with Text - Managing Text Analytics (18)

Mehr von aiaioo

Mehr von aiaioo (8)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Fun with Text - Managing Text Analytics