This is presentation in the nanosymposium of the Society for Neuro Science, in 2013 at San Diego

1. Brain-inspired
equivalence structure (ES)
extraction technique
for generating frames
Hiroshi Yamakawa
FUJITSU LABORATORIES LTD.
JAPAN

2. Outline
1. Human-level intelligence can explore from neocortex
learning.


Artificial intelligence (AI) lacks flexible sampling function of neocortex
Equivalence structure (ES) extraction is key for such a function
2. Use local sequences to extract equivalence structures (ESs).

Inspired by theta phase precession of hippocampal formation
3. Simple simulation of ES extraction
4. Summary

Frame generation:
Promising way to achieve artificial general intelligence (AGI)
2

3. HLDL mostly means human-level artificial intelligence (HLAI)
…because untrodden machine intelligences D
are concentrated on neocortex.
Complete intelligence
Deep learning (DL)
High-performance
unsupervised machine
learning technology
corresponding to
neocortex
Feasible intelligence (with limited resources)
Human
intelligence
Artificial
intelligence
Control theory
(Cerebellum)
D
Efficient arithmetic
operation and logic
inference
Deep
C r e a t i v i t ylearning
Human-level DL (HLDL)
Fully simulate neocortex
computing and its learning
functions.
(with help of hippocampus).
(DL)
I n t u i t P a tn t e r n
i o
r e c o g n i t i o n
Ge n e r a l
i n t e l l i g e n c e
Retrieval
from big
data
Reinforcement
learning
Emotion
(Basal ganglia)
(Amygdala)
What is the problem in achieving HLDL?

4. Deep learning lacks flexible sampling
Example: Intuitive “decision making” for chess-like game
High-level features
Precuneus
Caudate
Quick generation
of best next-move
Hippocampus
Perception of Supports intuitive decision making
board pattern - Cannot be explained by experts
Support learning
of neocortex
- Cannot be acquired by deep learning
Eye
support
learning
Visual
cortex
Convolution
Sampling
V3
Convolution
Sampling
V2
Hippocampus
MTG
/V6
Convolution
Sampling
Convolution layer：
• Well-developed for
machine learning:
Simple cell, Auto encoder
network, SOM, Boltzmann
machine, Info-MAX, Manifold
learning, ...
Sampling/Pooling layer:
• Human encode structure of
hierarchical retinotopy:
→ Complex cell, Max-pooling, ...
Chess-like
game (Wan,
V1
Convolution
Sampling
Science 2011)
4
• Supports visual invariances
→ Need flexible sampling

5. Equivalence structures for flexible sampling
Equivalence structure (ES)
…indicates portions of subspace that could be regarded as equivalent.
Variable set: x
Original
frame
A
B
C
D
E
F
G
H
Subspace
Subspace
Input sequence
X
Y
Z
A
B
C
D
E
F
1 2 3 4 5 6 7
1 2 3 4 5 6 7
Time
Time
Combined
1 2 3 4 5 6 7
frame
Invariance
Increased events enhance deductive inference.
Time: t
Invariance in basic image processing
Invariance for face recognition
Need more
flexibility for
higher-level
sampling
5

6. Outline
1. Human-level intelligence can explore from neocortex
learning.


Artificial intelligence (AI) lacks flexible sampling function of neocortex
Equivalence structure (ES) extraction is key for such a function
2. Use local sequences to extract equivalence structures (ESs).

Inspired by theta phase precession of hippocampal formation
3. Simple simulation of ES extraction
4. Summary

Frame generation:
Promising way to achieve artificial general intelligence (AGI)
6

7. Static patterns are poor for ES extraction
…this could
exist in
neocortex.
If using common static binary patterns
as similarity to compare subspaces,
Input sequence
Set of N variables
Problem:
Variation in static
patterns 2d is not
enough to categorize
thousands of
subspaces NCd(~Nd).
A
B
C
D
E
F
G
H
ESs
X
Y
Z
1 2 3 4 5 6 7
Original
frame
Time: t
Subspace of
d variables
A
B
C
Combined
frame
D
E
F
Too many
other
subspaces
Needs similarity with rich variation.
7

8. Subspaces can be compared using local sequences
Theta phase precession
Several sequential events are
packed in each phase (~5 Hz)
Inspired by information
representation in
hippocampus.
( Sato and Yamaguchi : Neural Computation 2003)
Subspace of d variables
Set of N variables
ESs
Input sequence
A
B
C
D
E
F
G
H
X
Y
Z
D
E
F
1 2 3 4 5 6 7
1 2 3 4 5 6 7
Time
Time
Combined
frame
1 2 3 4 5 6 7
Original
frame
A
B
C
Time: t
Local sequences are used to
compare subspaces.
(Skipping a detailed explanation.)
8

9. Outline
1. Human-level intelligence can explore from neocortex learning.


Artificial intelligence (AI) lacks flexible sampling function of neocortex
Equivalence structure (ES) extraction is key for such a function
2. Use local sequences to extract equivalence structures (ESs).

Inspired by theta phase precession of hippocampal formation
3. Simple simulation of ES extraction
4. Summary

Frame generation:
Promising way to achieve artificial general intelligence (AGI)
9

10. Simple simulation to validate this idea
 Input image for experiment: Dot wave sequence
A swinging dot image in sequence of one-dimensional spaces,
representing an idealized video image of natural scenes (up to 300 frames)
Ti e
m
di .I
m D
1
A
1
B
2
C
3
D
4
E
5
F
6
G
7
H
8
0
0
0
1
0
0
0
0
2
0
0
1
0
0
0
0
0
3
0
0
0
1
0
0
0
0
4
0
0
1
0
0
0
0
0
5
0
0
0
1
0
0
0
0
6
0
0
0
0
1
0
0
0
7
0
0
0
0
0
1
0
0
8
0
0
0
0
0
0
1
0
9
0
0
0
0
0
0
0
1
10
11
12
13
14
15
16
17
18
19
20
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
300
 Expected ES: A cluster of adjacent variable sets
Cluster of 12 subspaces, each of which consisting of 3 adjacent variables, is
expected to be extracted depending on spatial continuity of input sequence.
A
B
C
D
E
F
C
D
E
F
G
H
X
B
C
D
E
F
G
B
C
D
E
F
G
Y
C
D
E
F
G
H
A
B
C
D
E
F
Z
Cluster of subspaces
10
Combined frame

11. Result: Expected ES is extracted as a cluster
Subspaces
All permutation of subspaces (366 patterns)
Expected ES containing adjacent variable
set is extracted as cluster from a numbers
of local sequences clustering.
Index of local sequences
(Only non-zero elements shown)
11
Combined
frame
X
Y
Z
E
D C
B
C D
G
F
E
D
E
F
F
E
D
E
F
G
D
C B
C
D E
C
B
A
A
B C
H
G F
F
G H

12. Outline
1. Human-level intelligence can explore from neocortex
learning.


Artificial intelligence (AI) lacks flexible sampling function of neocortex
Equivalence structure (ES) extraction is key for such a function
2. Use local sequences to extract equivalence structures (ESs).

Inspired by theta phase precession of hippocampal formation
3. Simple simulation of ES extraction
4. Summary

Frame generation:
Promising way to achieve artificial general intelligence (AGI)
12

13. Summary
 Untrodden machine intelligent functions are concentrated on
neocortex, so emergence of HLDL mostly means emergence of HLAI.
 Learning of sampling layer is minimally needed to generate high-level
features for HLDL. This learning is assumed to be equivalence
structure extraction.
 Inspired by theta phase precession, I
introduced “numbers of local sequences”
for each subspace. Clustering of
subspaces by these frequencies enabled
extraction of ESs in a simple
demonstration.
 I'd like to specify the sub-region of the
hippocampal formation within theta loops
that perform ES extraction.
 Future work includes constructing a
neocortex-hippocampus model
implementing ES extraction.
13
Where is responsible sub-region
for ES extraction in theta loop of
hippocampal formation?
(Buzsaki, 2007)

14. Human-level general AI needs ability to generate frames.
General intelligence systems should be able to learn to solve
problems that were unknown at time of their creation.
Obviously, human brain can generate new frames to solve
various new problems using learning ability of neocortex.
Ⅰ
Neuron
Ⅱ
Column
Ⅲ
Ⅳ
Ⅴ
Ⅵ
Events
A
1
2
3
4
5
6
Variables
B
C
D
E
Combined
new frame
Equivalence
structure
(ES)
Values
frame
Designing HLDL by referring to neocortex is a promising approach.
14