Talk prepared for the 2nd Offtopicarium in Warsaw

1. Towards a Mathematical
Understanding of Intelligence
V. Kosoy

2. What is Intelligence?
• AI dates back at least to the 1950s
• There is no accepted definition of what
intelligence is
• Turing test doesn’t count: empirical rather
than mathematical

3. What is Intelligence?
• Most every day objects don’t have
mathematical definitions
• My opinion: intelligence must be different
• It seems natural to seek for this definition in
the domain of computation theory

4. Characteristics of Intelligence
• Pattern recognition
• Prediction
• Universal problem solving

5. Pattern Recognition
• Can be formalized as the Kolmogorov
complexity problem
• What is the shortest program producing given
data (string of bits)?
• Random string: has to be hardcoded
• 010101010…01 has compact description

6. Pattern Recognition
• Kolmogorov complexity is uncomputable due
to Berry paradox!

7. Solomonoff Induction
• Bayesian inference for a universe generated by
a random program
• This distribution favors low Kolmogorov
complexity: formalization of Occam’s razor!
• Making best guess with this prior allows
predicting any computable sequence
• This procedure is uncomputable!

8. Imperfect Prediction
• Shane Legg ‘06
• No Universal
Predictors
• Predicted
complexity vs.
predictor
complexity
• Unprovability

9. Universal Problem Solving
• Arguably we can only solve problems for
which the solution can be efficiently verified
• Corresponds to non-deterministic algorithms
• Efficiently computing non-deterministic
algorithms is possible iff P = NP

10. Legg-Hutter Intelligence
• Shane Legg, Marcus Hutter 2007
• Quantitative rather than qualitative
• Black boxes with input, output and utility
• Average utility in a random program universe
• No complexity considerations

11. Legg-Hutter Intelligence

12. Goedel Machine
• Juergen Shmidhuber 2003
• Essentially the same black box framework
• Reprograms itself using (Levin) proof search
• Metalearner: everything is reprogrammable
• Universe prior can be e.g. Solomonoff
• Limited by axiom system
• The degenerate environment problem

13. Asymptotic Computation
• Kolmogorov complexity and universal
prediction are asymptotically computable
• This is a realistic model of intelligence: we
can’t be sure we found the best model

14. P vs. NP revisited
• NP oracle allows efficient pattern recognition
• NP oracle allows efficient prediction
• NP oracle allows universal problem solving
• Maybe the problem is hard because its
solution is key to understanding the nature of
intelligence and even reality itself

15. Summary
• There is no satisfactory definition yet
• Connection between properties of intelligence
and natural concepts in computer science is
ominous
• Solomonoff induction, the works of Hutter,
Shmidhuber and Legg provide important
pieces for the puzzle
• Further progress will come from P vs. NP