Summary of AAAI Fall Symposium session on Cognitive Assistance in Government and Public Sector Applications

1. Frank Stein (IBM), Chuck Howell (MiITRE), Jim Spohrer (IBM)
Friday November 10, 2017
11/11/2017 1
Cognitive Assistance in Government
And Public Sector Applications

2. 11/11/2017
© IBM UPWard 2016
2

3. Day 1
• Keynote: Mark Maybury (MITRE)
– AI Progress - saving lives, but bad actors growing too e.g., fake
news
• William Regli (DARPA)
– AI changing the way we think - students performing like experts
• Randy Bailey (NASA)
– AI Pilot - deskilling is becoming a problem ("push button one, it
solves everything"), partnership better with roles and
responsibilities (R2D2)
• Kevin Burns (MITRE)
– AI Ergonomics for safety - better ways of measuring cognitive
load for different types of work (science of human-machine
teaming)
• Steve Meckl, Gheorghe Tecuci (GMU)
– Cog for advanced cyber security threat detection
• Jim Whitmore (Dickinson)
– Cog for Cyber Security risk analysis
• Joshi (UMBC), Branting (MITRE), Chan (USC)
– Cog for Fed Acq. Req., Admin Adjudication, Homeless Youth
11/11/2017
© IBM UPWard 2016
3

4. Day 2
11/11/2017
© IBM UPWard 2016
4
• Keynote: Ken Forbus (Northwestern)
– How system can learn as much with 10
examples as with 10,000
• Alun Preece (Cariff)
– How people and chatbots can increase
situational awareness
• Chieko Asakawa (IBM)
– Cogs for blind
• Matt Dering, Conrad Tucker (PSU)
– Blame and responsibilities
• Jim Spohrer (IBM)
– A view on the future
• Faruqe (GWU). Skeivik( (IBM), Watkins
(GWU), Grubbs (VA), Medsker (GWU)
– Cogs for health, autism, education,
government

5. Leaderboards Framework
AI Progress on Open Leaderboards - Benchmark Roadmap
Perceive World Develop Cognition Build Relationships Fill Roles
Pattern
recognition
Video
understanding
Memory Reasoning Social
interactions
Fluent
conversation
Assistant &
Collaborator
Coach &
Mediator
Speech Actions Declarative Deduction Scripts Speech Acts Tasks Institutions
Chime Thumos SQuAD SAT ROC Story ConvAI
Images Context Episodic Induction Plans Intentions Summarizatio
n
Values
ImageNet VQA DSTC RALI General-AI
Translation Narration Dynamic Abductive Goals Cultures Debate Negotiation
WMT DeepVideo Alexa Prize ICCMA AT
Learning from Labeled Training Data and Searching (Optimization)
Learning by Watching and Reading (Education)
Learning by Doing and being Responsible (Exploration)
2015 2018 2021 2024 2027 2030 2033 2036
11/11/2017 (c) IBM 2017, Cognitive Opentech Group 5
Which experts would be really surprised if it takes less time… and which experts really surprised if it takes longer?
Approx.
Year
Human
Level ->

6. Every 20 years, compute costs are down
by 1000x
• Cost of Digital Workers
– Moore’s Law can be thought of as
lowering costs by a factor of a…
• Thousand times lower
in 20 years
• Million times lower
in 40 years
• Billion times lower
in 60 years
• Smarter Tools (Terascale)
– Terascale (2017) = $3K
– Terascale (2020) = ~$1K
• Narrow Worker (Petascale)
– Recognition (Fast)
– Petascale (2040) = ~$1K
• Broad Worker (Exascale)
– Reasoning (Slow)
– Exascale (2060) = ~$1K
611/11/2017 (c) IBM 2017, Cognitive Opentech Group
2080204020001960
$1K
$1M
$1B
$1T
206020201980
+/- 10 years
$1
Person Average
Annual Salary
(Living Income)
Super Computer
Cost
Mainframe Cost
Smartphone Cost
T
P
E
T P E
AI Progress on Open Leaderboards
Benchmark Roadmap to solve AI/IA

7. GPD/Employee
11/11/2017 (c) IBM 2017, Cognitive Opentech Group 7
(Source)
Lower compute costs translate into increasing productivity and GDP/employees for nations
Increasing productivity and GDP/employees should translate into wealthier citizens
AI Progress on Open Leaderboards
Benchmark Roadmap to solve AI/IA

8. 11/11/2017
© IBM UPWard 2016
8

9. But this stuff is still really hard…
11/11/2017
© IBM UPWard 2016
9

10. 11/11/2017
© IBM UPWard 2016
10