Is a singularity near dw 121020

Is a singularity near?
A critical review of material presented at the
Singularity Summit 2012 #ss12 San Francisco 13-14 Oct

London Futurists David W. Wood
20th October 2012 Principal, Delta Wisdom Ltd
@dw2
1

http://hplusmagazine.com/

2


@dw2
3

3 theories of the future
1. The future is essentially just like the present
– With minor variations
– This ignores deep trends in technology, resource depletion…
2. The future is a reasonably easily predictable projection
of present-day trends
– This ignores constraints – and “collisions” (interactions)
– It also ignores the lag between underlying technology
improvements and the application of that technology
– “Grind is easy to predict, insight is hard to predict”
– Black Swans (mini-singularities) are hard to predict, but likely
3. The best way to plan for the medium term future is by
repeatedly planning for the near term future
– This ignores the threat of evolutionary dead-ends
– Local optima aren’t necessarily global optima
– Human societies have collapsed in the past, & may do again
4

The Singularity: Definition 1
“When humans
transcend biology”

When humans become
at least as much
computer/robot as
biological

A catchy phrase 
But misses out some of
the real punch
5

– John von Neumann (1950s)
• “The ever-accelerating
progress of technology … gives
the appearance of approaching
?
some essential singularity in
Technology
the history of the race beyond
which human affairs, as we
know them, could not
continue”
Time
2050
6

• “What is the Singularity? It’s a future period
during which the pace of technological change
will be so rapid, its impact so deep, that human
life will be irreversibly transformed”
– Ray Kurzweil ?
Technology

The unexpected
punch of
exponential
growth
Time
2050
7

– Alan Turing, 1951
• “My contention is that machines can
be constructed which will simulate
the behaviour of the human mind very
closely...”
• “…it seems probable that once
the machine thinking method had started, it would
not take long to outstrip our feeble powers”
• “There would be no question of the machines
dying, and they would be able to converse with
each other to sharpen their wits”
• At some stage therefore we should have to expect
the machines to take control”

8

• The advent of super-human general AI
• AI smarter than humans, for sufficiently many
purposes
– Including the ability to design and build new AI
• The process is no longer constrained by humans
• A trigger for recursive improvement
– A likely trigger for fast (and then even faster) recursive
improvement
• We could in a very short timespan have
super-super…-super-human general AI
• We are unlikely to be able to begin to conceive
what will happen next (Vernor Vinge)
9

• “When the first transhuman intelligence is
created and launches itself into recursive self-
improvement, a fundamental discontinuity is
likely to occur, the likes of which I can’t even
begin to predict”
– Michael Anissimov Technology

Time
2050
10

Some predictions of the Singularity
• Downside of bad Singularity is awful: Hollywood disaster
• Singularity would be much worse than Terminator movies…

11

The scale of intelligent minds:
A parochial view.
Village idiot Einstein

This section adapted from http://singularity.org/upload/mindisall-tv07.ppt

Eliezer Yudkowsky (2007) Singularity Institute for AI

The scale of intelligent minds:
A parochial view.
Village idiot Einstein

A more cosmopolitan view:
Mouse Village idiot

Chimp Einstein

AI
Mouse Village idiot

Chimp Einstein


Vernor Vinge:
The best answer to the question,
“Will computers ever be as smart as humans?”

is probably “Yes, but only briefly.”
AI
Mouse Village idiot

Chimp Einstein


Minds-in-general

Human minds


Minds-in-general

Transhuman mindspace

Human minds


Minds-in-general

Posthuman mindspace


Human minds


Bipping AIs

Minds-in-general

Posthuman mindspace

Freepy AIs

Gloopy AIs


Human minds


Friendly AI

The Singularity: Promise and Peril
– Luke Muehlhauser
– Executive Director, Singularity Institute
• Which are the most important events in history?
– Technology is the most important re-shaper of the
world (judging by social development)
– The most important technology that will ever be invented: superhuman AI
• Upside: We don’t die because physics requires us to die, we die
because we haven’t yet figured out how not to die
• Problem: Almost all the mind designs we could pick, for
superhuman AI, even if we were really careful, would steer us
somewhere we didn’t want to go
• Risk of over-maximising something that seems desirable
– hedonic pleasure, a single experience repeated indefinitely…
• Request: more funding, to enable more research!
22

Spread the wings of our uncertainty
– Stuart Armstong
– James Martin Research Fellow, FHI, Oxford
• How good are we at predicting advent of AGI?
– Review SI database of 257 predictions of AGI, 1950-2012
– “I believe that in about fifty years' time it will be possible, to programme
computers… to make them play the imitation game so well that an
average interrogator will not have more than 70 per cent chance of
making the right identification after five minutes of questioning” – Turing
• 1956 Dartmouth summer conference on AI
– Many of the participants predicted that a machine as intelligent as a
human being would exist in no more than a generation
• Is there evidence in favour of the so-called Maes-Garreau law?
– People tend to predict that AGI will happen just before they die
– 15-25 years away – not too soon, not too far (?)

23

• http://lesswrong.com/lw/e36/ai_timeline_predictions_are_we_getting_better/

24

– Stuart Armstong
– James Martin Research Fellow, FHI, Oxford
• What techniques can AI predictors use?
– Can’t use deductive logic, scientific method,
or past examples
– They can only use expert opinion!
• When are experts credible?
– Problem decomposable? Experts agree on stimuli? Feedback available?
• Grind is easy to predict, insight is hard to predict
– Moore’s Law is grind, AGI requires insight!
• Best timeline predictions: whole brain emulations (uploads)
– Very decomposed, justified grind, clear assumptions.
– Anders Sandberg ran Monte Carlo simulations – uncertainty spreads over
the entire next 100 years
• His own 80% estimate for advent of AGI is from 5 to 100 years
26

Channeling the Flood of Data
– Peter Norvig
– Research Director at Google
• Started by rating his own predictions from SS07
– 5 out of 6 predictions stood the test of time
• Convolutional Deep Belief Networks for
Scalable Unsupervised Learning
of Hierarchical Representations (2009)
– Hired team of academics into Google
– 16,000 CPUs on 1,000 servers, deep network with one billion parameters
Trained on 10 million YouTube videos, initially just one frame per video
– The system conceptualised human faces … and cats
• Same system applied in speech recognition
– Jump in performance was akin to 20 years at previous rate (according to one
of the participants)
• System increasingly being applied in language translation too
• Software and algorithmic advances will be more important than
hardware for the next 5-10 years at least
27

4 routes to greater-than-human AI
– Vernor Vinge
– Mathematician, Computer Scientist & Sci Fi writer
1. Progress with “classical AI project”
– Watson-like advances will cover more and more features
of what it’s like to be human
– Demos of robotics are progressively more impressive
– Sufficient hardware may already be around for AGI, in this decade
2. Intelligence Amplification – “ride the curve of improved cognition”
– Computers provide a neo-neo-cortex for humans
– Humans become the greater-than-human intelligence: hybrid intelligence
3. Digital Gaia – networked ensemble of the world’s embedded microprocessors
– Sensors and effectors. “Reality itself would wake up” BIGGEST RISK?
– Fundamental change in the nature of reality. It would have all the stability
that we currently associate with financial markets(!)
4. Group minds - Crowd-sourced intelligence that trumps all the human
intelligence systems of the past IDEALLY THE SOLUTION TO OTHER RISKS
– The Internet plus connected databases plus billions of humans
28

A critical review of material
presented at the #ss12
Singularity Summit 2012,
San Francisco 13-14 Oct
Technology

London Futurists
20th October 2012

David W. Wood
Principal
Delta Wisdom Ltd
@dw2 Time 2050

29

Our Viral Future
– Carl Zimmer
– Popular science writer and blogger
• Ramses V, Pharaoh of Egypt 1149-1145 BC
– “All his power and wealth couldn’t protect him from
microscopic particles” (smallpox -> yellow postules)
– Smallpox has killed billions (>5M each year in Europe…)
• Second virus has now been eradicated: Rinderpest, cow disease
– Polio now close to eradication (except in e.g. Pakistan)
– Deaths (and new infection rates) from AIDS have now crested
• Dangers from new “cross-over” viruses, spread quickly globally
– HIV itself is an example of a virus crossover, from chimp to humans
– SARS was 2002 cross-over, from Chinese horoscope bat ANALYSIS: RAPID
– SARS solved by quarantine, not by vaccination COLLABORATIVE
– Schmallenberg virus – horrendous birth defects in lambs – no solution
• “Viruses are going to be surprising us in the future, and at least some
of these surprises will be positive ones” – e.g. gene therapy advances
NEEDED: BETTER MONITORING, FASTER WAYS TO MAKE VACCINES 30

Over-complexity and over-interdependencies
Possible causes of relatively imminent
human-caused societal collapse
• “Extreme events” (“X-Events”) – John L. Casti (2012)
– Worldwide plague
– “Digital darkness”: Long-term failure of the Internet
– Failure of the electricity grid
– Electro Magnetic Pulse detonation – fries all electronics
– Collapse of food supplies, water supply, world oil supplies
– “The great unwinding”: Collapse of world financial markets
– Runaway climate change (positive feedback cycles)
– Geo-engineering catastrophe
– Exotic particles created by CERN (compare first H2 bomb)
– Ecosystem collapse precipitated by GMOs or nanoparticles
– Runaway superhuman intelligence Insufficiently rational
forward thinking
31

Rationality and the Future
Community is key
– Julia Galef
– President, Center for Applied Rationality (CFAR)
• “Why a better world tomorrow requires better
cognition today”
• Examples of irrationality
– Story of Eric Blair shooting an elephant… “solely to avoid looking a fool”
– People are more likely to believe others who have symmetric faces
– “One man’s death – that is a catastrophe. 100k dead – that is a statistic”
• “If you don’t want to be the captive of your genes, you had
better be rational” – Keith Stanovich
– Our genes don't care about strangers on other side of the world; WE do
• Three principles observed by CFAR “Harry Potter and the
– People don’t like being told they’re not rational methods of rationality”
– Teach rationality skills people care about
32

Rational thinking & decision making
– Daniel Kahneman
– 2002 Nobel Prize in Economics
• “Incorrect intuitions in the field of statistics”
– With Amos Tversky
• Examples of cognitive mistakes
– Example of training Israeli flight school instructors on +ve reinforcement
– We feel like we have free will even when we clearly don't
– We (even prof. forecasters) are misled by stories that seem coherent
– We need to be wary of the stories we tell ourselves about the Singularity
– Most major risks that unfolded were not the ones anticipated (N. Taleb)
• Individual experts are subject to the same biases as the rest of us
– Key role of community and feedback in achieving rationality
• “We tend to over-estimate the impact of technology on the
short-run, but under-estimate it on the long-run” – Roy Amara
33

A race?

Negative Positive
Singularity Singularity

34

A History of Violence
– Steven Pinker
– Professor of Psychology at Harvard University
• Six major declines of violence
– Based on his book “The better angels of our nature”
1. The pacification process (rise and expansion of states)
2. The civilising process (criminal justice nationalised)
3. The humanitarian revolution (abolition of judicial torture, use of the
death penalty for nonlethal crimes, witch-hunts, religious persecution,
duelling, blood sports, debtor’s prisons, slavery…)
4. The long peace (unprecedented decline in interstate war since 1946)
5. The new peace (post Cold War: democracy, trade, global community)
6. The rights revolution (decline in lynching, non-lethal hate crimes against
blacks, rape crimes, domestic violence, states allowing corporal punishment,
approval of spanking, animal hunting)
• Has human nature changed? Unlikely
– Human nature is complex: inclinations towards violence, and that oppose it
35

How to Create a Mind
– Ray Kurzweil
– Inventor, Author, Co-Founder of Sing Summit
• “The secret of human thought revealed”
• Confident in his predictions from ~30 years ago
about the continuing growth of computer power
• IBM Watson got its knowledge from reading 200 million pages in
Wikipedia and other encyclopaedias – in natural language
– Google self-driving cars may reach the marketplace within 5 years
– “We’re right before the storm with 3D printing” (Economist front cover: violin)
• Reverse engineering the brain: the ultimate source of the templates of
intelligence
– Human intelligence is pattern based. Computer intelligence is more logic based
• Three bridges to indefinite life (the third being the Singularity & brain uploads)
– By 2030, life expectancy will be increasing by at least one year every year
– He has written three books on health (people ought to read them…)
36

Too Much Magic?
– James Howard Kunstler
• “Wishful thinking, technology, and the fate of
the nation”
• Infamous for his “The Long Emergency” (2005)
– “Surviving the End of Oil, Climate Change, and Other
Converging Catastrophes of the Twenty-first Century”
• ‘TLE’ forecast the economic collapse of 2008-9
– Our civilisation is more inter-twined than you think
• Views Kurzweil with some interest
– Claims that Kurzweil (and many others) confuses
“energy” with “technology”
– Problems of energy capture, distribution, and storage
are deeply hard
– Especially when most sources of economic and
political power are pursuing shorter-term interests
37

Personal conclusions
1. Is a singularity near? Definitely Maybe. It might be
– There’s nothing inevitable about the outcome
– Both positive and negative singularities are feasible
– There are negative outcomes apart from just “AGI goes wrong”
– Negative outcomes more likely (without strong wise leadership)
2. The best of all times (new golden age) is potentially ahead
– But to reach it, we have to navigate potentially the worst of times
3. We can’t rely on free-markets to get us there
– Nor on traditional processes of politics
– These pursue “local sub-optima” (and do so dysfunctionally)
4. Wise leadership involves rationality and community
– We need to challenge and refine each other’s ideas
– Utilise technology to improve our rationality (wikis, Google)
– But don’t neglect “traditional” routes to wisdom (study, meditate)
5. The singularity community deserves our serious support
– Learn from best principles of marketing & communications 38

Winning the race?

Negative Positive
Singularity Singularity

+ Moral
bio-enhancement?

Turn H into H+

39

The case for moral bio-enhancement
– Prof Julian Savulescu, Oxford University
– Prof Ingmar Persson, University of Gothenburg
• Forthcoming (June 2013) book
“Unfit for the Future:
The Urgent Need for Moral Enhancement”
• http://philosophynow.org/issues/91/Moral_Enhancement
• We are facing two major threats:
– climate change – along with the attendant problems caused by increasingly
scarce natural resources
– war, using immensely powerful weapons
• Our Natural Moral Psychology – insufficient to meet modern challenges
• We can directly affect the biological or physiological bases of human
motivation (complementing, not replacing, traditional moral education)
– through drugs; or through genetic selection or engineering
– or by using external devices that affect the brain or the learning process
40

Hacking wetware: smart drugs and beyond
Sat 3 Nov, Andrew Vladimirov
“What are the most promising methods to enhance human mental and
intellectual abilities significantly beyond the so-called physiological norm?”
“Which specific brain mechanisms should be targeted, and how?”
“Which aspects of wetware hacking are likely to grow in prominence
in the not-too-distant future?”

Humanity+ UK
Supported by
Facebook group: UKH+
41

People in the ‘Golden Age of Technology’
Sat 24 Nov, Nick Price
Integrating human values into futures thinking
“This session will look at models for thinking about the future that integrate
human values (and potential changes in human values) alongside changes
in more tangible, measurable elements of the world - elements such as the
environment, science, technology, economics and society”

Humanity+ UK
Supported by
Facebook group: UKH+
42

The Fifth Conference on Artificial General Intelligence
St Anne's College, Oxford UK – December 8-11, 2012
http://agi-conference.org/

Chair: Ben Goertzel
Keynotes: David Hanson, Angelo Cangelosi,
Margaret Boden, Nick Bostrom

“AGI-12 is an official part of the
Alan Turing Year celebrations”
43


@dw2
44

Is a singularity near dw 121020

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Is a singularity near dw 121020