2. WBAIçĺżľ
Vision: to create a world in which AI exists in
harmony with humanity
Values
Study: Deepen and spread our expertise
Imagine: Broaden our views through public dialogue
Build: Create AGI through open collaboration
Mission: to promote the open
development of WBA
Basic Ideas
3. The Third Whole Brain Architecture Hackathon
Letâs Create AGI
Prototypes!
Wake up,
Hippocampus!
Hackathon:
September
16th -18th, 2017
Venue: Ďcafe, Tokyo
End of
Registration:
August 8th
5. Human Intelligent Features
⢠Having values
⢠Having survival capablity
⢠Being conscious
⢠Having an Ego
⢠Versatile for various tasks
â General Intelligence
⢠Etc.
GI that WBAI aims for is one of
the intelligent characteristics that
the current AI has not realized at
the human level.
8. Advances in DL
Learning from
the Brain
Nengo
(2015ă)Symbol
Emergence in
Robotics
A Map of AGI Developing Organizations
Engineering
Realization
Neocortex
oriented
Whole Brain
oriented
9. ML
Knowledge
Design
Connectome
Two-photon
Imaging
The Brain is ready to serve for AI!
ANNAI
(Cog. Sci)
Measuring
in Neuro-
science
Screws
Pistons
Automobile
Analogy
Overall
Design
Human
Brain
Engine
MacroMesoMicro
Higher-level
Description
Language
Whole
Brain ANN
model
Whole
Brain
Cognitive
Architecture
McCulloch
& Pitts
Model
Programming
Language
Neocortical
Areas
Neurons
Local
Neural
Circuits
Electrodes
fMRI, EEG
Deep Learning
Unified
Understanding
Multi-
layered
perceptron
Perceptron
Constraining
AI Architecture
10. WBA can be the fastest path to AGI.
âto create a human-like artificial general
intelligence (AGI) by learning from the
architecture of the entire brain.â
An anatomical illustration from
Sobotta's Human Anatomy (1908)
11. Brain-Inspired Open Platform Strategyďź WBAI
Systematizing
catch-up to
reproduce
(verify)
advancing
technologies
Collaborative
development on a
brain-inspired open
platform, enabling
technological
integration
Rapid Advances in technologies around AGI
(e.g., @DeepMind)
To âharmonize AGI with human beingsâ by
democratizing AI technologies
12. Hackathon Teams & Sample Code
Neuroscientific Info.
(e.g., connectome)
Virtual Mouse
Virtual Experiment Env. (Unity)
Architecture
Description
Middleware
(BriCA Core ver.1)
ML Modules
(Refactored DQN)
ăťNeocortex: CNN
ăťBG: Q-Learning
ăťHippocampus:
Episodic replay
Virtual Task Env.
ROS-like, light-
weight, multi-
platform with
virtual time
support
Engineer
Architect
Neuroscientist
14. BriCA Language for Brain-inspired Platform Description
BriCA Language: DSL to describe the structure of
cognitive architecture
Points:
⢠Bridging neuroscientific findings (connectome) & AI
⢠Basis for collaborative development of architecture
â Combining ML Modules (ANNs, graphical models)
â Middleware independent (Brica Core, ROS, etc.)
15. Wake Up! Hippocampus
Major functions of the Hippocampus:
⢠Episodic Memory â Long-term Memory
⢠NavigationďźSelf Location
⢠At the top of the perception
17. The Aim of the Hackathon
⢠Goal:
â To make the first working WBA
⢠Sample Code:
â Deep Reinforcement Learning (DQN) refactored for
a simple WBA
â The Hippocampus module works as the replay
memory of DQN.
⢠To do in the Hackathon:
â Implement hippocampus functions such as episodic
memory and space cognition by modifying the
sample code
19. Hippocampus & the realization of AGI
⢠There have been partial implementations for
hippocampal functions (listed below) ,
but not done on a single system.
ăťEpisodic Memory ăťOne-shot Learning ăťWorking Memoryăź
ăťSpace Cognition ăťSelf Location ăťInteraction with Environment
ăťTransfer Learning ăťEfficient Learning ăťDecision Making
ăťAction Generation ăťAttention ăťPlanning ăťImagination
ăťIntuition ăťConsciousness
⢠Hackathon participants will develop brain-inspired
cognitive architecture, which could be a prototype
of AGI by implementing the Hippocampus.
20. Hippocampus: the Seat of Memory
Major functions of Hippocampus
⢠Episodic Memory
⢠Space Cognition, Navigation
âPlace Cells: fire only when the animal passes particular locations in the
environment.
âGrid Cells: fire when the animal comes to certain places located at the vertices
of triangles in the environment
âHead Direction Cells: fire according to the direction of the head
Hippocampus
Laszlo Seress
ŠCC-BY-SA
We offer Maze Tasks that cannot be
solved without Hippocampal functions
22. Provided Maze Task (Place Conditioning)
⢠Linear Maze Task
⢠The linear track has patterns on the walls and floor and
reward points indicated by blue, green, and red.
⢠The mouse starts from an end of the track and receives
a reward if it waits at a reward point for two seconds.
⢠A reward is placed at a presented green spot at the
center. The mouse receives a reward if it waits at the
spot for two seconds.
â Q: how to learn by utilizing place cells and projections to/from
the cortex? (Masaaki S et al., eNeuro 2017)
⢠Re-learning by placing rewards at other spots (blue or
red) .
â Q: What kind of cell activities (e.g., of place cells)
facilitates re-learning?
23. Evaluation Task Styles
⢠Systems developed in the hackathon will be
evaluated with tasks on an environment
simulator (Unity/LIS).
⢠While the following task styles are available,
a system with higher generality to realize
multiple tasks and functions will be evaluated
highly.
⢠Provided Tasks
âProvided by WBAI
⢠Free Tasks
âSet by participants
24. Evaluation (Judging) Policies
⢠Developed systems will be evaluated by juries with
the following criteria:
âNeuroscientific reality
⢠Realization of various cognitive functions on hippocampus
⢠Neural connectivity between hippocampus and other brain
areas
⢠Correspondence with neuronal activities of hippocampus
and other brain areas
âEngineering utility
âOriginality
⢠Systems utilizing the software provided by WBAI,
such as BriCA, WBCA, BiCAmon will get extra points.
⢠The winner will be awarded with cash prize.
25. An example of hippocampal modeling (robot navigation)
Tang et al., Neural Networks archive
Volume 87: 27-37 (2017)
Pro: Navigation with hippocampal neural network.
Con: an implementation for the specific task.
âs model
26. An example of hippocampal modeling (Place Cells)
LĹrincz A. and Szirtes G.
Neural Networks 22 (2009) 738â747
Pro: Reproducing Place Cells
Con: Neuroscientific reality of the algorithm is debatable.
âs model
ICA was used for pattern separation.
29. Check sheet
Episodic Memory Space Cognition Place Cells Hippocampal
Circuits
A STAR NG ? NG â
Lorincz NG NG â ?
RatSLAM NG â NG NG
30. The points of the Hackathon
⢠While Neocortex Module, Basal Ganglia Module,
& Hippocampus Module are offered, the
Hippocampus Module only implements
experience replay.
⢠Modules & connections among them are
specified based on connectome.
Participants are to compete for solving tasks by
implementing Hippocampus Modules.
⢠If an AI system realizes multiple cognitive
functions here, it would advance AGI
development.
32. Agenda
1. Systems Summary
2. Systems Requirement & Environment Building
3. Products & supports offered by WBAI
4. Demonstration & comments
33. 1. Systems Summary
Demand action direction from Unity Task Environment to Agent
Unity Environment
Participant PC
Agent Server
Images from Agentâs
point of view, End
signal, etc.
Next Action
HTTP Async.
Com.
? Async.
Conn.:
Virtual
Time
Scheduler
Neocortex:
Feature Extractor
Component
Hippocampus:
Experience Component
Basal Ganglia:
Q-net Component
BriCA:
Cognitive Architecture Framework
ML modules connected based on WBRA
Component Connection
Setting
34. 2. Systems Requirement & Environment
Building
Recommende
d
Environment
Development
Environment
How to run
⢠CPU: Intel i5 or later
⢠Memory: 8G or more
⢠Download standard Unity IDE
⢠Download Python-enabled IDE
⢠Start the Python Agent with a command
⢠Run with Unity IDE or with a shell script
35. 3. Products & supports offered from WBAI
Products
Supports
⢠Unity Task Environment
⢠Mazes, etc.
⢠Agent Server
Source Code
Documentation
⢠Glossary
⢠Architecture Summary
⢠Modification for the hackathon
⢠Environment Building
⢠Q&A on Slack channels
⢠Advisory team for task & technical questions
⢠Real-time response is not guaranteed.
⢠FAQ to be compiled
36. Q-Learning
To obtain Function Q to estimate the Value of Action a at State s;
Approximate optimal Q Function by updating the following
equation at each step.
Q (s, L) = Low
Q (s, R) = High
Q (s, No-op) = Low
s: State , a: Action (pressing button, etc) , r: reward (game score, etc)
Preprocessed s: (self location, ball position)
37. Feature Extraction with DNN
37
End-to-end learning from input to estimation with a multi-layer
neural network enables feature extraction for the task.
No human feature design is necessary.
(Common use of CNN in image processing)
https:
//www.slideshare.net/nlab_utokyo/
deep-learning-49182466
Feature expression obtained by learning
38. DQN (Deep Q-Network)
38
The Q-function of Q-Learning is approximated with
DNN
By end-to-end learning with DNN, the value can be
estimated directly from images.
Value
DEEP
https:
//www.slideshare.net/nlab
_utokyo/deep-learning-
49182466
39. observation
- RGB Image
277*277
- depth Image 32*32
odometry
Velocity & angular
velocity
Episode end signal
reward
- bool
- int value
Unity Design â What is sent to the Agent
40. Switching Tasks
- Tasks are designed
for Unity scenesďź
- After achieving the
task for a certain times,
the Agent moves to
the next task.
- Environment waits for
Agentâs action.
Unity Design â Task Progress
43. BriCA Core and Timing
BriCA Core:
⢠Framework for constructing brain-inspired
cognitive architecture
⢠Can connect & integrate ML modules in an
asynchronous manner.
⢠The time & timing of ML module execution can be
designed with its time scheduler.
In this hackathon, as Unity and BriCA work synchronously,
if the agent uses too much time, the system would not
work smoothly.
44. Evaluation of the neuroscientific reality
⢠Realization of various cognitive functions on hippocampus
â E.g.:
⢠The Agent must wait in reward areas for a moment.
⢠The Agent may locate itself from landmark information
without the information on wall color to obtain rewards.
⢠Neuro-connectivity between hippocampus and other brain
areas
â The proposed model should be justified in the presentation.
⢠Correspondence with neuronal activities of hippocampus
and other brain areas
â Activities of reproduced Place Cells, Grid Cells and Head Direction
Cells will be evaluated.
46. Timeline till the Hackathon
⢠07-23: 2nd Orientation
⢠08-08: Due Date for Registration
â From the Registration Form
https: //goo.gl/forms/PqAPlNoln53nyCG92
â Please make âWBAhackathon2017â folder in your team Github.
⢠08-10: Acceptance notice
⢠08-18: Sample Code to be published
⢠08-19: Sample Code orientation & presentation by
participants
⢠09-16
â Opening: 10h JST
Talk by Kenji Doya (11h)
Reception in the evening (talking on the hippocampus)
⢠09-18
â End of work: Noon
â Review, Presentation, and Awarding: till 17h
47. The Third WBA Hackathon Summary
ďŽ Date: September 16th, 17th, and 18th, 2017
ďŽ Venue: ÎŚ CafĂŠ, Tokyo, Japan
ďŽ Cost
ďŽ Admission Free
ďŽ Transportation, Lodging & Reception fees will be paid by
the participants.
Students may receive financial aids (to be explained below).
Remote sites can be set up.
48. Lodging
Participants (up to 20 persons) can stay at a hotel
reserved by WBAI for the evenings of September
15th, 16th, and 17th.
In case youâd like to stay in the hotel (for 4,290JPY
per person/night), please inform us from the
registration form.
49. Other information
⢠Computation Environment
â BYOD (Bring your own device)
⢠Making your code public
We ask you to publish the code and presentation material
you made for the hackathon for open development of
WBA under:
â WBAI Contributor Agreement
â Apache License (Version 2.0)
on Github with README.
⢠Financial aids for students
Students meeting all the following conditions will be eligible for
financial aids with regard to travel and lodging expenses up to
JPY65,000/person.
â You should fully participate in the hackathon at ÎŚ Cafe on
September 16th, 17th, and 18th.
â Your work at the hackathon with a Readme file should be
published on GitHub.