Halting Problem Proving Non-Computability Universal Turing Machines

1. cs1120 Fall 2011
David Evans
16 November 2011 Halting Problems Hockey Team

2. Reminders
PS7 Due (electronic) 11:59pm tonight
Exam 2 review sessions (Rice 442):
Today, 7:30pm My Thursday office
hours moved to 4-5pm
Thursday, 6:30pm
Friday’s class: Rice Hall Dedication
11am Dedication; noon-3pm Tours;
3pm Dean Kamen talk; 5pm Scavenger Hunt
Office hours from now on are in “Davis Commons”!
2

3. PS8, Part 1
Option C/W: Option J:
Clear description of
what you plan to do PS8 Part 1 posted on
including: course site.
1. Goal
2. Target audience
3. Plan Due next week Tuesday,
4. Team, how you will 22 November (11:59pm)
manage and
distribute work
3

4. Impossibility Results
Mathematics (Declarative Knowledge)
Friday/Monday
Gödel: Any powerful axiomatic system cannot be
both complete and consistent
If it is possible to express “This statement has no
proof.” in the system, it must be incomplete or
inconsistent.
Computer Science (Imperative Knowledge)
Are there (well-defined) problems that cannot be
Now
solved by any algorithm?
Alan Turing (and Alonzo Church): Yes!

5. Impossibility of Halts
Recall how Gödel showed incompleteness of PM:
Find a statement that leads to a contradiction
Gödel’s statement: “This statement has no proof.”
The HALTING Problem:
Input: a string representing a Python program.
Output: If evaluating the input program would ever finish,
output true. Otherwise, output false.
Assume halts(s) is a Python procedure that solves the HALTING
problem. Is there an input that leads to a contradiction?

6. Hmm..let’s try self-reference!
6

7. Informal Proof
def paradox():
if halts('paradox()'):
while True:
pass
What should halts('paradox() ') evaluate to?

8. Proving Non-Existence by Contradiction
1. Show X is nonsensical.
2. Show that if you have A you can make X.
3. Therefore, A must not exist.

9. Proving Non-Existence by Contradiction
1. Show X is nonsensical.
2. Show that if you have A you can make X.
3. Therefore, A must not exist.
Noncomputability Incompleteness
(Alan Turing) (Kurt Gödel)
X = paradox procedure X = “This statement has no
A = algorithm that solves proof.”
HALTING Problem A = a complete and
consistent axiomatic
system

10. Halting Problem is Non-Computable
1. paradox leads to a
contradiction.
def paradox(): 2. If we have halts, an
if halts('paradox()'): algorithm that solves the
while True: Halting Problem, we can
define paradox.
pass 3. Therefore, halts must not
exist.
Are there any other noncomputable problems?

11. Evaluates-to-3 Problem
Input: A string, s, representing a Python
program.
Output: True if s evaluates to 3;
otherwise, False.
Is “Evaluates-to-3” computable?

12. Proof by Contradiction
1. Show X is nonsensical.
2. Show that if you have A you can make X.
3. Therefore, A must not exist.
X = halts algorithm
A = Evaluates-to-3 algorithm

13. Undecidability Proof
Suppose we could define evaluates_to_3(s).
that decides it. Could we define halts(s)?

14. Undecidability Proof
def halts(s):
return evaluates_to_3(
cleanreturns(s) + ''''''
return 3
Where cleanreturns(s) is a
''''') procedure that replaces all return x
statements in s with x; return 3
The only way the program s passed to evaluates_to_3 could not evaluate
to 3, is if s doesn’t halt. (Note: assumes evaluating s cannot produce an
error, but we can use error handling to deal with this case.)

15. How convincing is our
Halting Problem proof?
1. paradox leads to a
def paradox(): contradiction.
if halts('paradox()'): 2. If we have halts, an algorithm
that solves the Halting
while True: Problem, we can define
pass paradox.
3. Therefore, halts must not exist.
“Proof” assumes Python exists and means exactly what it should!
Python is too complex to believe this (and Turing didn’t have it, of
course!): need a simple and precise model of computation.

16. Turing’s Model: Turing Machine
... # 1 0 1 1 0 1 1 1 0 1 1 0 1 1 1 # ...
Infinite Tape: Finite set of symbols, one in each square
Can read/write one square each step
Start Input: # Controller:
Input: 1 Limited (finite)
Write: #
Write: 0 number of states
Move:
Move:
1
Follow rules based
2 Input: 0
on current state
and read symbol
Write: #
Input: 1 Input: 0 Move: Halt Write one square
Write: 1 Write: 0 each step, move
Move: Move:
3
left or right or
halt, change state
16

17. Turing Machine Summary
Model Input, Output, and Scratch Memory
Infinite tape, divided into discrete squares
Each square can contain a single symbol from
a finite alphabet
Model Processing
Finite State Machine
Keep track of a finite state (in your head)
Follow transition rules:
next state = f(current state, input symbol)
17

18. Halting Problem for TM
Input: a string describing a Turing Machine, M
Output: if M would eventually Halt, output True;
otherwise, output False.
Is there a Turing Machine that solves the Halting Problem for TMs?
18

19. Turing Machine State
Infinite Tape
z z z z z z z z z z z z z z z z z z z z
TuringMachine ::= < Alphabet, Tape, FSM >
), X, L
), #, R
Alphabet ::= { Symbol* }
(, #, L
1
2:
look
for (
Tape ::= < LeftSide, Current, RightSide >
Start
(, X, R
OneSquare ::= Symbol | #
Current ::= OneSquare
#, 1, -
HALT
#, 0, -
LeftSide ::= [ Square* ]
Finite State Machine RightSide ::= [ Square* ]
Everything to left of LeftSide is #.
Everything to right of RightSide is #.
Although we typically draw TMs as pictures, we could write them down as
strings in this language (which could be converted to whole numbers)

20. Enumerating Turing Machines
Now that we’ve decided how to describe Turing
Machines, we can number them
TM-5023582376 = balancing parens
TM-57239683314 = less than function (Exam 1)
TM- = Universal TM
3523796834721038296738259873
TM- 3672349872381692309875823987609823712347823 = Angry Birds
Not the real numbers –
they would be much much
much much much bigger!

21. Halting Problem
Pythonic Halting Problem TM Halting Problem
Input: a string describing a Input: a string describing a
Python program, P Turing Machine, M
Output: if evaluating P would Output: if M would eventually
eventually finish, output Halt, output True;
True; otherwise, output otherwise, output False.
False.

22. Halting Problem “Proof”
Pythonic Halting Problem TM Halting Problem
HALTS(M) = TM that solves TM Halting
def paradox(): Problem: input tape describes TM
if halts('paradox()'): M, output tape: #1 if M
while True: halts, otherwise #0
pass
PARADOX = TM that:
1. simulates HALTS(PARADOX)
2. Then, if tape is #1, loop forever;
if tape is #0, HALT
simulates? This proof assumes we can design a TM that simulates any other TM!

23. An Any TM Simulator
Input: < Description of some TM M, w >
Output: result of running M on w
M Output
Universal Tape
Turing for running
TM-M
w Machine starting on
tape w

24. Manchester Illuminated Universal Turing Machine, #9
from http://www.verostko.com/manchester/manchester.html

25. Universal Computing Machine
2-state, 3-symbol Turing machine proved
universal by Alex Smith in 2007
Alex Smith,
University of Birmingham

26. What This Means
If you can:
Keep track of a finite state
Follow transition rules
Read and write to memory
you can simulate a universal Turing machine.
A Turing machine can simulate the world’s most
powerful supercomputer (it’ll just take a lot
longer and will run out of memory)
It is impossible to solve the Halting Problem for any
computing system that can simulate a TM

27. Alan Turing (1912-1954)
Published On Computable Numbers … (1936)
Introduced the Halting Problem 5 years after
Formal model of computation Gödel’s proof!
(now known as “Turing Machine”)
Codebreaker at Bletchley Park
Led efforts to break Enigma Cipher
Contributed to Colossus
After the war: convicted of “gross indecency”
(homosexuality, then a crime in
Britain), forced to undergo hormone
treatments, committed suicide eating
cyanide apple

28. Prime Minister’s Apology
It is no exaggeration to say that, without his outstanding
contribution, the history of World War Two could well have
been very different. He truly was one of those individuals we
can point to whose unique contribution helped to turn the
tide of war. The debt of gratitude he is owed makes it all the
more horrifying, therefore, that he was treated so
inhumanely....
So on behalf of the British government, and all those who
live freely thanks to Alan’s work I am very proud to say:
we’re sorry, you deserved so much better.
Gordon Brown, 10 September 2009

29. Charge
Before midnight tonight: submit PS7
Exam Review Sessions:
Today, 7:30pm; Thursday, 6:30pm
Friday’s Class: Rice Hall Dedication
Monday: Exam 2 out
Tuesday: PS8 Part 1 Due
29