The Future of AI: Going BeyondDeep Learning, Watson, and the Semantic Web

Tetherless World Constellation, RPI
The Future of AI: Going Beyond
Deep Learning, Watson, and the
Semantic Web
Jim Hendler
Tetherless World Professor of Computer, Web and Cognitive Sciences
Director, Institute for Data Exploration and Applications
Rensselaer Polytechnic Institute
http://www.cs.rpi.edu/~hendler
@jahendler (twitter)
Major talks at: http://www.slideshare.net/jahendler

Knowledge and Learning
Knowledge representation in the age of
Deep Learning, Watson, and the Semantic Web
Jim Hendler
Tetherless World Professor of Computer, Web and Cognitive Sciences
Director, Institute for Data Exploration and Applications
Rensselaer Polytechnic Institute
http://www.cs.rpi.edu/~hendler
Major talks at: http://www.slideshare.net/jahendler

Talk derives in part from a recent book

New Journal: Data Intelligence
Knowledge graph is one of the
topics we are interested in, please
consider submitting a paper!
(handout in your conference
bag)

What has happened?
• Several important AI technologies have
moved through “knees in the curve” bringing
much of the attention to AI again
–Deep Learning (eg AlphaGo, vision processing)
–Associative learning (eg Watson)
–Semantic Web (eg search and schema.org)

A) Deep Learning
“phase transition” in capabilities of neural
networks w/machine power

Trained on lots of categorized images
Imagenet: Duck Imagenet: Cat

Impressive results
Increasingly powerful techniques have yielded
incredible results in the past few years

B) Associative knowledge (text mining/QA)
© IBM, used with permission.

Impressive Results
Watson showed the power of “associative knowledge”

C) Semantic Web

“Knowledge graphs” mined from extracted
and learned data

Impressive Results
Google finds embedded metadata on >30% of its crawl – Guha, 2015

Not just Google…

Summary: AI has done some way cool stuff
• Deep Learning: neural learning from data with high
quality,
• Watson: Associative learning from data with high
quality
• Semantic Web/Knowledge Graph: Graph links
formation from extraction, clustering and learning
but, there are still problems…

Combining these technologies

still a long way to go

Many intermediate steps
(P. Norvig, WWW 4/2016, w. permission)

Why did knowledge graph need
“”Human Judgments”?
Association ≠ Correctness
P. Mika, 2014 w.permission
Michelangelo
Leonardo
Raphael
Donnatello

Knowledge Representation?
• A knowledge representation (KR) is most fundamentally a surrogate, a substitute for the thing
itself, used to enable an entity to determine consequences by thinking rather than acting,
i.e., by reasoning about the world rather than taking action in it.
• It is a set of ontological commitments, i.e., an answer to the question: In what terms should I think
about the world?
• It is a fragmentary theory of intelligent reasoning, expressed in terms of three components: (i) the
representation's fundamental conception of intelligent reasoning; (ii) the set of inferences the
representation sanctions; and (iii) the set of inferences it recommends.
• It is a medium for pragmatically efficient computation, i.e., the computational environment in which
thinking is accomplished. One contribution to this pragmatic efficiency is supplied by the guidance a
representation provides for organizing information so as to facilitate making the recommended
inferences.
• It is a medium of human expression, i.e., a language in which we say things about the
world.
R. Davis, H. Shrobe, P. Szolovits (1993)

KR: Human Expression
Cute kid story: first two words

Telling cats from ducks doesn’t need KR
!

“Saying things about the world” does
"If I was telling it to a
kid, I'd probably say
something like 'the cat
has fur and four legs
and goes meow, the
duck is a bird and it
swims and goes
quack’. "
How would you explain the difference between a
duck and a cat to a child?
Woof

Learning semantic inferencing
Bassem Makni, 2018 Phd (now at IBM Yorktown)

In noisy data
Bassem Makni, 2018 Phd (now at IBM Yorktown)

The challenge of “background knowledge”
What is the relationship
between this man and
this woman?

AI systems coming along well…
this woman?
Deep learning produced Scene Graph w/relationships
(Klawonn & Heims, 2018)

But the challenges remain
this woman?
Deep learning produced Scene Graph w/relationships
(Klawonn, 2018)
Seeing the bride adds
significant information
that cannot be easily
learned w/o background
knowledge

A major problem to deploying AI in key areas

Adding knowledge to scene graphs
Matthew Klawonn, PhD, 2019

KR: Surrogate knowledge?
Which could you sit in?
What is most likely to bite what?
Which one is most likely to become a computer
scientist someday?
…

“Surrogate” knowledge
Which could you sit in?
What is most likely to bite what?
Which one is most likely to become a computer
scientist someday?
How would they go about doing it?

KR: Recommended vs. Possible inference
Which one would you save if the house was on
fire?

Ethical AI systems need certainty
Which one would you save if the house was on
fire? Would you use a robot baby-sitter
without knowing which of the three
possibilities it would choose?

Human-Aware AI
• Context is key
– AI learning systems still perform best in well-defined
contexts (or trained situations, or where their
document set is complete, etc.)
– Humans are good at recognizing context and deciding
when extraneous factors don’t make sense
• Or add extra “inferencing” (the bride example)

The challenge
• If we want to implement KR systems on top of
neural and associative learners we have an issue
– The numbers coming out of Deep Learning and
Associative graphs are not probabilities
– They don’t necessarily ground in human-meaningful
symbols
• ”sub-symbolic” learning …
• Association by clustering …
• Errorful extraction …

The challenges
• Can we avoid throwing out the reasoning baby
with the grounding bathwater?
– Long-term planning
– Rules that need to be followed
– Human Interaction
• Even if computers don’t need to be symbolic communicators,
WE DO!!!
– Background knowledge (context) is symbolic

Human-AI interaction
• Evidence that “centaurs” win
– Human(s) and computer(s) teams currently beat either
at chess (Go centaurs underway)
– Anecdotal evidence that humans w/Watson top choices
outperform Watson or human alone at Jeopardy
– Medical study (diagnostic):
• Doctor w/computer outperformed just doctor, just computer,
two doctors

And this matters!
“There was no rule about how long we were allowed to think before we reported a strike …
but we knew that every second of procrastination took away valuable time, that the Soviet
Union’s military and political leadership needed to be informed without delay. All I had to do
was to reach for the phone; to raise the direct line to our top commanders — but I couldn’t
move. I felt like I was sitting on a hot frying pan … when people start a war, they don’t
start it with only five missiles …”
We must all strive to be like Petrov and learn to trust the
combination of AI training and human intuition.
Stanislav Petrov: The man who saved the world

The Future of AI: Going BeyondDeep Learning, Watson, and the Semantic Web

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie The Future of AI: Going BeyondDeep Learning, Watson, and the Semantic Web

Ähnlich wie The Future of AI: Going BeyondDeep Learning, Watson, and the Semantic Web (20)

Mehr von James Hendler

Mehr von James Hendler (12)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)