Slides of my talk "Code Biology and (the future of) AI" at the second international code biology conference in Jena, 16-20 June 2015

1. Code
Biology
and
(the
future
of)
Ar5ficial
Intelligence
Joachim
De
Beule

2. Recent
advances
in
AI
Deep
learning
A
dark
future
Superintelligences
more
dangerous
than
nukes
A
brighter
future
Collec5ve
intelligence

3. “A
revolu*on
in
ar*ficial
intelligence
is
currently
sweeping
through
computer
science.
The
technique
is
called
deep
learning
and
it’s
affec*ng
everything
from
facial
and
voice
to
fashion
and
economics.”

4. “In
some
sense
deep
learning
is
what
happened
when
machine
learning
hit
big
data”
“Two
kinds
of
data:
raw
data
(pictures,
music,
…)
and
symbolic
data
(text)”
“With
deep
learning,
we
can
bridge
the
gap
between
the
physical
world
and
the
world
of
compu5ng”
-­‐-­‐
Adam
Berenzweig,
founding
CTO
of
Clarifai

5. Ref:
Deep
Learning:
Intelligence
from
Big
Data,
Tue
Sep
16,
2014,
Stanford
Graduate
School
of
Business
Neural
Networks
of
the
80’s

6.
What’s
New?
ü Big
Data
• The
internet
&
Social
Media
• Metadata:
tags,
transla5ons,
…
• Mechanical
Turk
Ref:
Deep
Learning:
Intelligence
from
Big
Data,
Tue
Sep
16,
2014,
Stanford
Graduate
School
of
Business

7.
What’s
New?
ü Big
Data
ü Scale
• 80’s:
1-­‐10M
(106)
neurons/synap5c
connec5ons
• Google
Brain:
1B
(109)
(10M
video’s,
16k
computers,
3
days)
•
Adult:
100T
(1014)
•
Infant:
1Q
(1015)
Ref:
Deep
Learning:
Intelligence
from
Big
Data,
Tue
Sep
16,
2014,
Stanford
Graduate
School
of
Business

8.
What’s
New?
ü Big
Data
ü Scale
ü Algorithmic
advances
• Successive
layers
of
learning/representa5on
• Unsupervised
pre-­‐training
à
Structure
NN
(feature
detectors)
• Then
supervised
back-­‐prop
à
classify/predict
labeled
data
Ref:
Deep
Learning:
Intelligence
from
Big
Data,
Tue
Sep
16,
2014,
Stanford
Graduate
School
of
Business

9.
What’s
New?
ü Big
Data
ü Scale
ü Algorithmic
advances

13. We
have
been
able
to
reduce
the
word
error
rate
for
speech
by
over
30%
compared
to
previous
methods.
This
means
that
rather
than
having
one
word
in
4
or
5
incorrect,
now
the
error
rate
is
one
word
in
7
or
8.
While
s5ll
far
from
perfect,
this
is
the
most
drama5c
change
in
accuracy
since
the
introduc5on
of
hidden
Markov
modeling
in
1979,
and
as
we
add
more
data
to
the
training
we
believe
that
we
will
get
even
becer
results.

15. November
18,
2014

16. Asked
whether
two
unfamiliar
photos
of
faces
show
the
same
person,
a
human
being
will
get
it
right
97.53
percent
of
the
5me.
New
sodware
developed
by
researchers
at
Facebook
can
score
97.25
percent
on
the
same
challenge,
regardless
of
varia5ons
in
ligh5ng
or
whether
the
person
in
the
picture
is
directly
facing
the
camera.

17. Feb
26,
2015

18. • Isotherm
is
to
temperature
as
isobar
is
to?
(i)
atmosphere,
(ii)
wind,
(iii)
pressure,
(iv)
la*tude,
(v)
current.
• Iden*fy
two
words
(one
from
each
set
of
brackets)
that
form
a
connec*on
(analogy)
when
paired
with
the
words
in
capitals:
CHAPTER
(book,
verse,
read),
ACT
(stage,
audience,
play).
• Which
is
the
odd
one
out?
(i)
calm,
(ii)
quiet,
(iii)
relaxed,
(iv)
serene,
(v)
unruffled.
•
Which
word
is
closest
to
IRRATIONAL?
(i)
intransigent,
(ii)
irredeemable,
(iii)
unsafe,
(iv)
lost,
(v)
nonsensical.
• Which
word
is
most
opposite
to
MUSICAL?
(i)
discordant,
(ii)
loud,
(iii)
lyrical,
(iv)
verbal,
(v)
euphonious.
Ref:
arxiv.org/abs/1505.07909
:
Solving
Verbal
Comprehension
Ques5ons
in
IQ
Test
by
Knowledge-­‐
Powered
Word
Embedding

20. The
future?

21. The
future?

22. “I
am
in
the
camp
that
is
concerned
about
super
intelligence.
First
the
machines
will
do
a
lot
of
jobs
for
us
and
not
be
super
intelligent.
That
should
be
posi*ve
if
we
manage
it
well.
A
few
decades
a[er
that,
though,
the
intelligence
is
strong
enough
to
be
a
concern.
I
agree
with
Elon
Musk
and
some
others
on
this
and
don't
understand
why
some
people
are
not
concerned.”
Stephen
Hawking
(hcp://www.bbc.com/news/technology-­‐30290540)
"The
development
of
full
ar*ficial
intelligence
could
spell
the
end
of
the
human
race
[…]
It
would
take
off
on
its
own,
and
re-­‐design
itself
at
an
ever
increasing
rate
[…]
Humans,
who
are
limited
by
slow
biological
evolu*on,
couldn't
compete,
and
would
be
superseded.”

24. • Oren
Etzioni
(Computer
science,
Univ.
Washington,
CEO
of
the
Allen
Ins5t.
for
Ar5ficial
Intelligence):
“The
popular
dystopian
vision
of
AI
is
wrong
for
one
simple
reason:
it
equates
intelligence
with
autonomy.
That
is,
it
assumes
a
smart
computer
will
create
its
own
goals,
and
have
its
own
will,
and
will
use
its
faster
processing
abili*es
and
deep
databases
to
beat
humans
at
their
own
game.
It
assumes
that
with
intelligence
comes
free
will,
but
I
believe
those
two
things
are
en*rely
different”
• Michael
Licman
(AI,
Brown
Univ.,
former
program
chair
for
the
Ass.
of
the
Advancmnt
of
AI):
“There
are
indeed
concerns
about
the
near-­‐term
future
of
AI
—
algorithmic
traders
crashing
the
economy,
or
sensi*ve
power
grids
overreac*ng
to
fluctua*ons
and
shucng
down
electricity
for
large
swaths
of
the
popula*on.
[...]
These
worries
should
play
a
central
role
in
the
development
and
deployment
of
new
ideas.
But
dread
predic*ons
of
computers
suddenly
waking
up
and
turning
on
us
are
simply
not
realis*c.”
• Yann
LeCun
(Facebook’s
director
of
research,
one
of
the
world’s
top
experts
in
deep
learning):
“Some
people
have
asked
what
would
prevent
a
hypothe*cal
super-­‐intelligent
autonomous
benevolent
A.I.
to
“reprogram”
itself
and
remove
its
built-­‐in
safeguards
against
gecng
rid
of
humans.
Most
of
these
people
are
not
themselves
A.I.
researchers,
or
even
computer
scien*sts.”
• Andrew
Ng
(founded
Google’s
Google
Brain
project,
now
Chief
Scien5st
at
Baidu):
“Computers
are
becoming
more
intelligent
and
that’s
useful
as
in
self-­‐driving
cars
or
speech
recogni*on
systems
or
search
engines.
That’s
intelligence,”
he
said.
“But
sen*ence
and
consciousness
is
not
something
that
most
of
the
people
I
talk
to
think
we’re
on
the
path
to.”

25. Assump5on:
Deeper
level
neurons
are
more
“abstract”
However,
what
was
discovered:
-­‐ A
single
neuron's
feature
is
no
more
interpretable
as
a
meaningful
feature
than
a
random
set
of
neurons.
-­‐ NN’s
do
not
"unscramble"
the
data
by
mapping
features
to
individual
neurons
in
say
the
final
layer.
The
informa5on
that
the
network
extracts
is
just
as
much
distributed
across
all
of
the
neurons
as
it
is
localized
in
a
single
neuron.
-­‐ Furthermore,
Every
deep
neural
network
has
"blind
spots"
in
the
sense
that
there
are
inputs
that
are
very
close
to
correctly
classified
examples
that
are
misclassified.

30. The
Symbol
Grounding
Problem
010000110101010
011110101010100
110100101010100
1011010101111…
Jpeg
coding
01000001
01000011
01010100
ASCII
coding
CAT
Deep
NN
Harnad,
S.
(1990)

31. The
Symbol
Grounding
Problem
010000110101010
011110101010100
110100101010100
1011010101111…
Jpeg
coding
01000001
01000011
01010100
ASCII
coding
CAT
Human
coding
Human
coding
Deep
NN
Human
Qualifica5on
or
Semiosis
Harnad,
S.
(1990)

32. The
Symbol
Grounding
Problem
• Categories
(signs
and
meanings)
are
ar5facts
• The
rela5on
between
them
is
arbitrary
• They
are
realized
by
agents
performing
semiosis
Diagram
of
Self-­‐regula5on

33. The
future?

34. The
future?
“Collec*ve
intelligence
is
the
opposite
of
ar*ficial
intelligence”

36. Ø Outer
world
onto
inner
world
(human
neuronal
coding)
Ø
Inner
worlds
onto
each
other
(collec5ve
intelligence)
Ø
Collec5ve
intelligence
onto
inner
world

37. • Semiosis
(life)
=
self-­‐regula5on
(produc5on
and
consump5on
of
variety,
closure)
Self-­‐regulatory
system
(Agent)

38. • Semiosis
(life)
=
self-­‐regula5on
(produc5on
and
consump5on
of
variety,
closure)
• Tool
usage
(supplementa5on
of
variety)

39. • Semiosis
(life)
=
self-­‐regula5on
(produc5on
and
consump5on
of
variety,
closure)
• Tool
usage
(supplementa5on
of
variety)
• Extension
and
specializa5on
(constraints)
“Now,
as
the
Internet
revolu*on
unfolds,
we
are
seeing
not
merely
an
extension
of
mind
but
a
unity
of
mind
and
machine,
two
networks
coming
together
as
one.”
[Deepstuff,
May
25,
2015]

40. • Semiosis
(life)
=
self-­‐regula5on
(produc5on
and
consump5on
of
variety,
closure)
• Tool
usage
(supplementa5on
of
variety)
• Extension
and
specializa5on
(constraints)
• Coordina5on
(conven5onal
codes)
Agent
1
Agent
2

41. • Semiosis
(life)
=
self-­‐regula5on
(produc5on
and
consump5on
of
variety,
closure)
• Tool
usage
(supplementa5on
of
variety)
• Extension
and
specializa5on
(constraints)
• Coordina5on
(conven5onal
codes)
à
Metasystem
or
“Major
Transi5on”
Agent
1
Agent
2
Meta
agent

42. “In
a
sense,
deep
learning
is
what
happened
when
machine
learning
hit
big
data”
“Two
kinds
of
data:
raw
data
(pictures,
music,
…)
and
symbolic
data
(text)”
“With
deep
learning,
we
can
bridge
the
gap
between
the
physical
world
and
the
world
of
compu5ng”
-­‐-­‐
Adam
Berenzweig,
founding
CTO
of
Clarifai

43. The
Next
Major
Transi5on?
Symbolic
Collec5ve
intelligence
(deep
learning)
Physical
Collec5ve
ac5ng
(da5ng,
vo5ng,
…)
Informa5on
seeking
ac5ng
Tagging
and
training

45. Replica*on
always
involves
coding!
3,
15
or
33
numbers?
The
Symbol
Grounding
Problem
Harnad,
S.
(1990)

48. ü
A
robot
may
not
injure
a
human
being
or,
through
inac5on,
allow
a
human
being
to
come
to
harm.
ü A
robot
must
obey
the
orders
given
to
it
by
human
beings,
except
where
such
orders
would
conflict
with
the
First
Law.
ü A
robot
must
protect
its
own
existence
as
long
as
such
protec5on
does
not
conflict
with
the
First
or
Second
Laws.

52. MIT
Technology
review,
Robert
D.
Hof,
April
23,
2014

54. Professor
Geoff
Hinton,
who
was
hired
by
Google
two
years
ago
to
help
develop
intelligent
opera5ng
systems,
said
that
the
company
is
on
the
brink
of
developing
algorithms
with
the
capacity
for
logic,
natural
conversa5on
and
even
flirta5on.
“Basically,
they’ll
have
common
sense”
“Thought
vectors,
Hinton
explained,
work
at
a
higher
level
by
extrac5ng
something
closer
to
actual
meaning”