1. Jim Spohrer (IBM)
For Ntegrea, Stanford Park Hotel, Palo Alto, CA, Wed July 23, 2018
http://www.slideshare.net/spohrer/UIDP_20180419_V6
July 23, 2018 1
Future of AI:
Measuring Progress and Preparing
5. Future of AI
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 5
… when will
your smartphone
be able to take and
pass any online
course? And then
be your coach, so
you can pass too?
10. Questions
• What is the timeline for solving AI and IA?
• Who are the leaders driving AI progress?
• What will the biggest benefits from AI be?
• What are the biggest risks associated with AI, and
are they real?
• What other technologies may have a bigger
impact than AI?
• What are the implications for stakeholders?
• How should we prepare to get the benefits and
avoid the risks?
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 10
12. Every 20 years, compute costs are down
by 1000x
• Cost of Digital Workers
– Moore’s Law can be thought of as
lowering costs by a factor of a…
• Thousand times lower
in 20 years
• Million times lower
in 40 years
• Billion times lower
in 60 years
• Smarter Tools (Terascale)
– Terascale (2017) = $3K
– Terascale (2020) = ~$1K
• Narrow Worker (Petascale)
– Recognition (Fast)
– Petascale (2040) = ~$1K
• Broad Worker (Exascale)
– Reasoning (Slow)
– Exascale (2060) = ~$1K
12
3/24/2023 (c) IBM 2017, Cognitive Opentech Group
2080
2040
2000
1960
$1K
$1M
$1B
$1T
2060
2020
1980
+/- 10 years
$1
Person Average
Annual Salary
(Living Income)
Super Computer
Cost
Mainframe Cost
Smartphone Cost
T
P
E
T P E
AI Progress on Open Leaderboards
Benchmark Roadmap to solve AI/IA
13. GDP/Employee
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 13
(Source)
Lower compute costs translate into increasing productivity and GDP/employees for nations
Increasing productivity and GDP/employees should translate into wealthier citizens
AI Progress on Open Leaderboards
Benchmark Roadmap to solve AI/IA
14. Leaderboards Framework
AI Progress on Open Leaderboards - Benchmark Roadmap
Perceive World Develop Cognition Build Relationships Fill Roles
Pattern
recognition
Video
understanding
Memory Reasoning Social
interactions
Fluent
conversation
Assistant &
Collaborator
Coach &
Mediator
Speech Actions Declarative Deduction Scripts Speech Acts Tasks Institutions
Chime Thumos SQuAD SAT ROC Story ConvAI
Images Context Episodic Induction Plans Intentions Summarizatio
n
Values
ImageNet VQA DSTC RALI General-AI
Translation Narration Dynamic Abductive Goals Cultures Debate Negotiation
WMT DeepVideo Alexa Prize ICCMA AT
Learning from Labeled Training Data and Searching (Optimization)
Learning by Watching and Reading (Education)
Learning by Doing and being Responsible (Exploration)
2015 2018 2021 2024 2027 2030 2033 2036
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 14
Which experts would be really surprised if it takes less time… and which experts really surprised if it takes longer?
Approx.
Year
Human
Level ->
15. Leaders and Leaderboards:
Who is winning
• How to Measure Leadership?
– Publications or Patents
– Regions China vs USA vs EU vs ROW
– Companies Microsoft vs Google vs IBM
• How to Measure Progress?
– One capability or all leaderboards?
– SQuAD – Question Answering
– EFF Measuring AI Progress
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 15
16. AI Benefits
• Access to expertise
– “Insanely great” labor productivity for trusted
service providers
– Digital workers for healthcare, education, finance,
etc.
• Better choices
– ”Insanely great” collaborations with others on
what matters most
– AI for IA = Augmented Intelligence and higher
value co-creation interactions
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 16
17. AI Risks
• Job Loss
– Shorter term
bigger risk
= de-skilling
• Super-intelligence
– Shorter term
bigger risk
= bad actors
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 17
https://maliciousaireport.com/
https://www.wsj.com/articles/automation-makes-us-dumb-1416589342
18. Other Technologies: Bigger impact?
Yes.
• Augmented Reality (AR)/
Virtual Reality (VR)
– Game worlds
grow-up
• Blockchain/
Security Systems
– Trust and security
immutable
• Advanced Materials/
Energy Systems
– Manufacturing as cheap,
local recycling service
(utility fog, artificial leaf, etc.)
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 18
19. Stakeholders
• Individuals
• Families
• Businesses and
other Organizations
• Industry Groups
• Regional
Governments:
– Cities
– States
– Nations
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 19
20. “The best way to predict the future is to inspire the
next generation of students to build it better”
Digital Natives Transportation Water Manufacturing
Energy Construction ICT Retail
Finance Healthcare Education Government
21. Be Prepared
• Understand open AI code + data +
models + stacks + communities
– Leaderboards
– Ethical conduct
• Learn 3 R’s of IBM’s Cognitive
Opentech Group (COG)
– Read arXiv
– Redo with Github
– Report with Jupyter notebooks on
DSX and/or leaderboards
• Improve your team’s skills of rapidly
rebuilding from scratch
– Build your open code eminence
– Understand open innovation
– Communities + Leaderboards
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 21
1972 used
Punch cards
2016 used
IBM Watson
Open APIs to win…
22. Courses
• 2015
– “How to build a cognitive system for Q&A task.”
– 9 months to 40% question answering accuracy
– 1-2 years for 90% accuracy, which questions to reject
• 2025
– “How to use a cognitive system to be a better professional X.”
– Tools to build a student level Q&A from textbook in 1 week
• 2035
– “How to use your cognitive mediator to build a startup.”
– Tools to build faculty level Q&A for textbook in one day
– Cognitive mediator knows a person better than they know themselves
• 2055
– “How to manage your workforce of digital workers.”
– Most people have 100 digital workers.
3/24/2023 22
Take free online cognitive classes today at cognitiveclass.ai
34. Industries Transformed
Digital Natives Transportation Water Manufacturing
Energy Construction ICT Retail
Finance Healthcare Education Government
“The best way to predict the future is to inspire the next generation of students to build it better”
49. Brian Arthur - Economist
• The term “technological unemployment” is from John Maynard Keynes’s 1930 lecture,
“Economic possibilities for our grandchildren,” where he predicted that in the future, around
2030, the production problem would be solved and there would be enough for everyone, but
machines (robots, he thought) would cause “technological unemployment.” There would be
plenty to go around, but the means of getting a share in it, jobs, might be scarce. We are not
quite at 2030, but I believe we have reached the “Keynes point,” where indeed enough is
produced by the economy, both physical and virtual, for all of us. (If total US household
income of $8.495 trillion were shared by America’s 116 million households, each would earn
$73,000, enough for a decent middle-class life.) And we have reached a point where
technological unemployment is becoming a reality. The problem in this new phase we’ve
entered is not quite jobs, it is access to what’s produced. Jobs have been the main means of
access for only 200 or 300 years. Before that, farm labor, small craft workshops, voluntary
piecework, or inherited wealth provided access. Now access needs to change again. However
this happens, we have entered a different phase for the economy, a new era where production
matters less and what matters more is access to that production: distribution, in other words—
who gets what and how they get it. We have entered the distributive era.
3/24/2023 IBM #OpenTechAI 49
58. OpenTechAI on GitHub (April 29,
2018)
Code Stars (K) Forks (K)
Tencent NCNN 3 0.8
Google Kubeflow 3 0.3
Microsoft ONNX 3 0.3
Google TensorBoard 1 0.3
Apache SystemML 0.6 0.2
IBM (watson-intu) INTU (10 Repositories) 0.1 0.07
IBM FfDL 0.1 0.04
IBM ART 0.1 0.03
IBM MAX 0.005 0.017
LFDL AT&T Acumos (35 Repositories) 0.01 0
3/24/2023 IBM #OpenTechAI 58
59. Headlines
• 2017 Popular
– “AI vs People”
– “X-Y team up to invest big in AI”
• 2025 Commonplace
– “People using AI to become better at their
professions, serving others.”
– “Teenagers using AI to solve challenges,
and improve their communities.”
• 2085 Resilience
– “Teams competing to rapidly rebuild
socio-economic-technical systems (wise
service systems) from scratch”
– “U.N. Pluto-base makes major discovery
about nature of universe. U.F.P.
established.”
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 59
IEEE 2017
64. Icons of AI
Progress
• 1956: Dartmouth Conference
organized by:
– John McCarthy (Dartmouth, later
Stanford)
– Marvin Minsky (MIT)
– and two senior scientists:
• Claude Shannon (Bell Labs)
• Nathan Rochester (IBM)
• 1997: Deep Blue (IBM) - Chess
• 2011: Watson Jeopardy! (IBM)
• 2016: AlphaGo (Google DeepMinds)
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 64
65. AI to IA Timeline: Hard unsolved AI
problems
• 2012-2017 AI Pattern Recognition and
Learning from Massive Labeled Data
– Speech, image, translation, driverless, games
– Chatbots as digital assistants
• 2018 Video Understanding
• 2021 Episodic Memory
• 2022 Learning from Watching
• 2024 Commonsense Reasoning
• 2026 Learning from Reading
• 2028 Learning from Doing
• 2030 Fluent Conversation
• 2031-2039 Cognitive Collaborator and
Mediator; Intelligence Augmentation (IA)
3/24/2023 (c) IBM 2017, Cognitive Opentech Group 65
66. Jim Spohrer (IBM)
For C Mohan
Wednesday February 21, 2018
http://www.slideshare.net/spohrer/nsf-20180124-v18
3/24/2023 66
The Future of AI:
Measuring Progress and Preparing
77. In Conclusion
• Hundreds of AI Challenge Leaderboards
exist and the number is growing
• Can one system be built to do them all?
See: https://www.youtube.com/watch?v=8FpdEmySsuc
78. What types of digital cognitive
systems?
• Cognitive Build: Outthink Challenge (250K people)
– Imagine a digital cognitive system to help you do
something important in your personal or professional
lives
– Team to design it and advocate for it, and then
everyone votes
– Winners: reduce waste and human suffering, screen
for health issues and safety threats, learn life skills and
make better choices, find what you are looking for,
move around more effectively, provide emotional
support, provide IT support, learn about important
public policy goals and make better choices
• Types: Tool, Assistant, Collaborator, Coach, Mediator
3/24/2023 Understanding Cognitive Systems 78
94. What types of digital cognitive
systems?
• Cognitive Build: Outthink Challenge (250K people)
– Imagine a digital cognitive system to help you do
something important in your personal or professional
lives
– Team to design it and advocate for it, and then
everyone votes
– Winners: reduce waste and human suffering, screen
for health issues and safety threats, learn life skills and
make better choices, find what you are looking for,
move around more effectively, provide emotional
support, provide IT support, learn about important
public policy goals and make better choices
• Types: Tool, Assistant, Collaborator, Coach, Mediator
3/24/2023 Understanding Cognitive Systems 94
105. Tomorrow: Servitization
• Start with any traditional product that is sold to customers
• Make the product part of a smart/wise service system
– Instrument it (sensors) – Internet of Things/Everything
– Set-up an intelligent operation center to monitor all products’
performance across their life-cycles
– Use big data analytics to determine how to improve product
performance, efficiency, maintenance, etc.
– Offer customer the “product-performance-as-a-service” with
financing/Internet of Service
– Customer benefits from cost-savings, predictability
– Provider benefits margin-improvements, predictability
• Every product becomes a platform technology (a vehicle for
service innovation) for innovative university startups
106. Vision: MMaaRRSS
• Modular Manufacturing as a Regional
Recirculation Service System
– “I am the stuff that will be made into product X for
customer Y.”
– Stuff = Material, Energy, and Information Flows
– Minimize transport costs (for products and waste)
• The Vision: Circular Economy (~4 minutes)
109. What is a cognitive system (entity)?
3/24/2023 Understanding Cognitive Systems 109
110. What is a digital cognitive system
(entity)?
3/24/2023 Understanding Cognitive Systems 110
111. Backup slides
• Service systems - http://service-
science.info/archives/3368
3/24/2023 Understanding Cognitive Systems 111
112. Assisting individuals and organizations
to close their service innovation skills gap
and co-create wiser service systems
empowering employees, customers, citizens
with cognitive mediators
in the collaborative service economy
113. What is service science?
• IBM initiated effort to establish a
multidisciplinary field to study
service systems … with a focus on
people-centered, IT-enabled service
innovations for business and society
– based on service-dominant logic
– service = value co-creation
– IT-enabled service architectures
– service systems (socio-technical
systems for win-win value co-creation)
• IBM helped establish
– computer science (1945-present)
– service science (2005-present)
3/24/2023 Understanding Cognitive Systems 113
Service systems are dynamic configurations of
resources (people, technology, organizations,
and information) interconnected by
value propositions, internally and externally.
Examples:
- macro: cities, states, nations
- meso: hospitals, universities, businesses
- micro: households, families, individuals
Reference:
Spohrer J, Maglio P, Bailey J, Gruhl D (2007)
Steps toward a science of service systems.
IEEE Computer Society. 40(3):71-77(January).
114. What is service science?
• Now over 500 universities globally teach a more
multidisciplinary approach to service innovation,
including:
– Service management and marketing
– Service engineering and operations
– Service design and arts
– Service public policy and economics
– Service computing and informatics
• SSME + DAPP =
Service Science Management Engineering +
Design Arts Public Policy
– People, technology, organizations, information
interconnected by value propositions.
3/24/2023 Understanding Cognitive Systems 114
Reference:
IfM & IBM (2008). Succeeding through service
innovation: A service perspective for education,
research, business and government.
University of Cambridge Institute for Manufacturing,
Cambridge, UK. 2008.
115. How to get involved?
• Weekly speaker series
– Service innovation
– Service education & research
– Smart service/cognitive systems
• Discovery summits & book series
• Opportunities
– Institutional memberships
– Leadership & ambassadors
– Volunteer opportunities
– Awards & sponsored
conferences
3/24/2023 Understanding Cognitive Systems 115
ISSIP.org is a non-profit society
International Society of Service Innovation Professionals
Membership:
Over 1000 professionals and students from 40+ countries,
50+ companies and 50+ universities.
116. How to get involved?
• Journals (INFORMS,
etc.)
• Conferences (HICSS,
etc.)
• Courses (MIT, etc.)
• Funding (NSF, etc.)
• Society (ISSIP, etc.)
3/24/2023 Understanding Cognitive Systems 116
117. What are the hot topics?
• Smart Service Systems: Intelligence Augmentation
– AI + AR UX (Artificial Intelligence + Augmented Reality User
Experience)
– Smartphones (mobile, social, secure, etc.)
• Collaborative Economy: Servitization
– From assets to co-creation (e.g., Uber, AirBnB, etc.)
– From product to capability/outcome-as-a-service
– Manufacturing as a local recycling service
• Digital Transformation: Trust and Identity
– Blockchain: Don Tapscott’s TED Talk & book
– Big Data: Service Analytics & HAT (Hub of All Things)
3/24/2023 Understanding Cognitive Systems 117
126. Next Generation:
Future-Ready T-Shaped Adaptive Innovators
Many disciplines
Many sectors
Many regions/cultures
(understanding & communications)
Deep
in
one
sector
Deep
in
one
region/culture
Deep
in
one
discipline
127. By 2035, T-Shaped Makers with great
Building Blocks and Cognitive Mediators
3/24/2023 127
Empathy & Teamwork
sector
region/culture
discipline
Depth
Breadth
STEM
Liberal Arts
132. You say you want a
revolution?
October 8, 2017
https://www.slideshare.net/spohrer/revolution-20171008-v23
3/24/2023 (c) IBM 2017, Cognitive Opentech Group
132