SlideShare ist ein Scribd-Unternehmen logo
1 von 85
Learning Programming
(by Humans
by Machine)
Peter Norvig
Ng Koller
Norvig Thrun
MOOCs
and
Machine
Learning
For when you don’t
know what’s going on
Hal Abelson
(SIGCSE Outstanding Contribution Award, 2012)
The average under tutoring [with mastery learning]
was about two standard deviations above the
average of the control. (Benjamin Bloom, 1984)
Python llll lll
C ll
Scheme l
But: for “vocational”, not “academic”:
Javascript dominates
Analyzing Intro CS MOOC Data 101
Student Name Final Grade
Peter Norvig
Analyzing Intro CS MOOC Data 101
Student Name Final Grade
Peter Norvig Incomplete
160,000 students; 209 countries
Data Sources
• Enrollment / Dropout
• Quizzes and Exams
• Video Interactions
• Discussion Forums
• Program submissions
Syntax Error Messages Vague
if x = 1:
^
SyntaxError: invalid syntax
SyntaxError: Use “==“ for comparison,
use “=“ for assignment.
Inputs to Compiler Error Routine
Source
Code
Compiler
Error
Message
Ignored:
- Past program submission histories
- Problem Description
- Comments
- Test Cases
CFG:
Python
Non-Python
PCFG:
Python-ish
Probably Approximately Correct
Grammar and Intent
Match Programs to Corpus
S1 suggest
C2C1Corpus:
Submission:
Syntax Error Messages Vague
if spam == 42
print('Hello!')
^
SyntaxError: invalid syntax
^
IndentationError: unexpected indent
SyntaxError: Colon required,
use “if spam == 42:“
Attractive Nuisance
>>> table = [[]] * 5
>>> table
[[], [], [], [], []]
>>> def add(key, table):
table[hash(key) % len(table)].append(key)
>>> add(’x', table)
>>> table
[[’x'], [’x'], [’x'], [’x'], [’x']]
>>> table = listof(5, list)
def listof(n, kind):
"Make a list of n elements, each kind()."
return [kind() for _ in range(n)]
Faulty Student Models
• Given:
• nums = [1, 2, -3]
• max(nums) == 2
• max(nums, key=abs) == -3
• len(‘word’) == 4
• wordlist = file(‘words’).read().split()
• What is the longest word in wordlist?
• ?
• ??
• max(wordlist, key=len)
DWIM Attractive Nuisance
• def qsort(A):
if len(A) <= 1: return A
LT, EQ, GT = []
p = A[len(A)/2]
for x in A:
bin = (LT if x<p else GT if x>p else EQ)
bin.append(x)
return qsort(LT) + EQ + qsort(GT)
LT = EQ = GT = []
LT, EQ, GT = [], [], []
DWIM Attractive Nuisance
• spam = ['cat', 'dog', 'mouse']
• for i in range(spam):
print(spam[i])
• for i in len(spam):
print(spam[i])
• for i in range(len(spam)):
print(spam[i])
for x in spam:
print(x)
• for i in range(len(spam)):
spam[i] = spam[i].upper()
for x in spam:
x = x.upper()
Language Too General
y = (x + 1) (x - 1)
TypeError: unsupported operand
type(s) for +: 'object' and
'int'
SyntaxError: ‘*’ operator required.
use “(x + 1) * (x – 1)”
Multi-Language Confusion
File ”example.py”, line 1
root = new Node()
^
SyntaxError: invalid syntax
Test.java:6: cannot find symbol
symbol : class awt
location: package java
java.awt.Color(r, g, b);
^
Error: use “Node()” in Python;
use “new Color(…)” in Java
Multi-Language Confusion
int total = 0;
for (int x : A) {
total += x;
}
total = None
for x in A:
total = total + x
Why do students stay or go?
the movie in which no babies are born
`
Motivation, not Information
• Willpower
• Due dates
• Peer support (forums, outside groups)
• Faculty encouragement (email)
• Pride of accomplishment
• Authenticity
• Early adopter
Timeline (successful Coursera course)
#usersonsite
Real Couse
100K
50K
ramp up + exponential decay + due dates
synchronous, evergreen,
semi-synchronous, bus route
Learning Programming
Representing Programs
• Bag of words
• N-grams of tokens (tagged?)
• Static syntax tree
• Dynamic call graph
Representing Program Sequences
HMM
Alpha, Beta (steady progress) significantly better than Gamma (thrash, then
solution) on midterm. No significant difference on score for this assignment.
What is Learning?
• Learning results from
what the student does
and thinks and only
from what the student
does and thinks. The
teacher can advance
learning only by
influencing what the
student does to
learn. -
Herb Simon
• Problem first, then explanation (Peer)
• Make students to prediction (see it is wrong)
• (They didn’t like that.)
• Experiments with more open-ended problems
• Experiments with peer grading
• “Richard Hamming told me his
secret: First get together the problem
sets and exams that you want the
students to be able to solve. Then
write a book that will teach them how
to solve them.” -- Hal Varian (1993)
T D D
Do we trust Bloom?
0
0.5
1
1.5
2
2.5
Drill Intelligent
Tutoring
System
Human
Tutor
Expected
Actual
0
0.5
1
1.5
2
2.5
Drill Intelligent
Tutoring
System
Human
Tutor
Expected
Actual
More Mastery than Tutoring
0
0.5
1
1.5
2
2.5
Drill Intelligent
Tutoring
System
Human
Tutor
Expected
Actual
N = 33
More Mastery than Tutoring
How do we make it
one to one tutoring
and
mastery learning?
“Books will soon be obsolete in the public schools.
Scholars will be instructed through the eye.
It is possible to teach every branch of human knowledge
with the motion picture. Our school system will be
completely changed inside of ten years”- Edison, 1913
“Lectures were once useful,
but now, when all can read,
and books so numerous,
lectures are unnecessary.
If your attention fails,
and you miss part of a lecture,
it is lost; you cannot go back
as you do upon a book.”
- 1791
Many Types of Individualization
• Student control over rewind (multiplay)
• Student choice of where to go next (multipath)
• Flipped classroom (Koller, etc.)
• Peer instruction (Mazur)
• Peer feedback
• Forums and other one-to-one interactions
• Machine learning?
• Courseware Engineering?
Open-Ended Work
Peer Grade
Self Grade
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
Teacher Grade
StudentGrade
“The Impact of Self-and Peer-Grading on Student Learning”.
P. Sadler, E. Good. Educational Assessment (2006).
20 bits per year
192K bits per sec
Data Threshold?
Modern sequencing
methods (11 min)
Genomic
economics (8
min)
Personalized
medicine (13 min)
History: The
Human Genome
Project (12 min)
Modern GenomicsGenes and alleles
(10 min)
Basic Genetics
Refresher (OPTIONAL)
Mendelian
inheritance
(13 min)
What is DNA?
(12 min)
Genetic testing in
the commercial
world (11 min)
Protecting privacy
(9 minutes)
Case study: direct-
to-consumer
genetics (12 min)
Case study:
family planning
(13 min)
Commercial Genomics
(OPTIONAL)
Authored Multipath Sequence
Courseware Engineering?
One on One Tutoring; Mastery Learning
Social;
Motivation
Large-Scale Courseware Engineering
Machine Learning; Continual Improvement

Weitere ähnliche Inhalte

Ähnlich wie Peter Norvig - NYC Machine Learning 2013

9.b-CMPS 403-F20-Session 9-Intro to ML II.pdf
9.b-CMPS 403-F20-Session 9-Intro to ML II.pdf9.b-CMPS 403-F20-Session 9-Intro to ML II.pdf
9.b-CMPS 403-F20-Session 9-Intro to ML II.pdf
AmirMohamedNabilSale
 
Presentation of GetTogether on Functional Programming
Presentation of GetTogether on Functional ProgrammingPresentation of GetTogether on Functional Programming
Presentation of GetTogether on Functional Programming
Filip De Sutter
 
Artificial intelligency full_ppt_persentation_way2project_in
Artificial intelligency full_ppt_persentation_way2project_inArtificial intelligency full_ppt_persentation_way2project_in
Artificial intelligency full_ppt_persentation_way2project_in
Sumit Sharma
 
MLlecture1.ppt
MLlecture1.pptMLlecture1.ppt
MLlecture1.ppt
butest
 
MLlecture1.ppt
MLlecture1.pptMLlecture1.ppt
MLlecture1.ppt
butest
 

Ähnlich wie Peter Norvig - NYC Machine Learning 2013 (20)

James Coplien - Trygve - October 17, 2016
James Coplien - Trygve - October 17, 2016James Coplien - Trygve - October 17, 2016
James Coplien - Trygve - October 17, 2016
 
transfer.pptx
transfer.pptxtransfer.pptx
transfer.pptx
 
Four Languages From Forty Years Ago
Four Languages From Forty Years AgoFour Languages From Forty Years Ago
Four Languages From Forty Years Ago
 
Python basic
Python basicPython basic
Python basic
 
Beyond Ruby (RubyConf Argentina 2011)
Beyond Ruby (RubyConf Argentina 2011)Beyond Ruby (RubyConf Argentina 2011)
Beyond Ruby (RubyConf Argentina 2011)
 
9.b-CMPS 403-F20-Session 9-Intro to ML II.pdf
9.b-CMPS 403-F20-Session 9-Intro to ML II.pdf9.b-CMPS 403-F20-Session 9-Intro to ML II.pdf
9.b-CMPS 403-F20-Session 9-Intro to ML II.pdf
 
Anthiil Inside workshop on NLP
Anthiil Inside workshop on NLPAnthiil Inside workshop on NLP
Anthiil Inside workshop on NLP
 
Representation Learning of Text for NLP
Representation Learning of Text for NLPRepresentation Learning of Text for NLP
Representation Learning of Text for NLP
 
Presentation of GetTogether on Functional Programming
Presentation of GetTogether on Functional ProgrammingPresentation of GetTogether on Functional Programming
Presentation of GetTogether on Functional Programming
 
Beyond the Symbols: A 30-minute Overview of NLP
Beyond the Symbols: A 30-minute Overview of NLPBeyond the Symbols: A 30-minute Overview of NLP
Beyond the Symbols: A 30-minute Overview of NLP
 
NLP Bootcamp 2018 : Representation Learning of text for NLP
NLP Bootcamp 2018 : Representation Learning of text for NLPNLP Bootcamp 2018 : Representation Learning of text for NLP
NLP Bootcamp 2018 : Representation Learning of text for NLP
 
Artificial intelligency full_ppt_persentation_way2project_in
Artificial intelligency full_ppt_persentation_way2project_inArtificial intelligency full_ppt_persentation_way2project_in
Artificial intelligency full_ppt_persentation_way2project_in
 
An introduction to deep learning concepts
An introduction to deep learning conceptsAn introduction to deep learning concepts
An introduction to deep learning concepts
 
Four Languages From Forty Years Ago (NewCrafts 2019)
Four Languages From Forty Years Ago (NewCrafts 2019)Four Languages From Forty Years Ago (NewCrafts 2019)
Four Languages From Forty Years Ago (NewCrafts 2019)
 
[D2 COMMUNITY] Spark User Group - 머신러닝 인공지능 기법
[D2 COMMUNITY] Spark User Group - 머신러닝 인공지능 기법[D2 COMMUNITY] Spark User Group - 머신러닝 인공지능 기법
[D2 COMMUNITY] Spark User Group - 머신러닝 인공지능 기법
 
Learning and Modern Programming Languages
Learning and Modern Programming LanguagesLearning and Modern Programming Languages
Learning and Modern Programming Languages
 
Deep Learning for NLP: An Introduction to Neural Word Embeddings
Deep Learning for NLP: An Introduction to Neural Word EmbeddingsDeep Learning for NLP: An Introduction to Neural Word Embeddings
Deep Learning for NLP: An Introduction to Neural Word Embeddings
 
MLlecture1.ppt
MLlecture1.pptMLlecture1.ppt
MLlecture1.ppt
 
MLlecture1.ppt
MLlecture1.pptMLlecture1.ppt
MLlecture1.ppt
 
Deep Learning, an interactive introduction for NLP-ers
Deep Learning, an interactive introduction for NLP-ersDeep Learning, an interactive introduction for NLP-ers
Deep Learning, an interactive introduction for NLP-ers
 

Mehr von Michael Scovetta

Modern Kernel Pool Exploitation: Attacks and Techniques
Modern Kernel Pool Exploitation: Attacks and TechniquesModern Kernel Pool Exploitation: Attacks and Techniques
Modern Kernel Pool Exploitation: Attacks and Techniques
Michael Scovetta
 
Exploitation and State Machines
Exploitation and State MachinesExploitation and State Machines
Exploitation and State Machines
Michael Scovetta
 
Don't Give Credit: Hacking Arcade Machines
Don't Give Credit: Hacking Arcade MachinesDon't Give Credit: Hacking Arcade Machines
Don't Give Credit: Hacking Arcade Machines
Michael Scovetta
 
The Listening: Email Client Backdoor
The Listening: Email Client BackdoorThe Listening: Email Client Backdoor
The Listening: Email Client Backdoor
Michael Scovetta
 
DEFCON 18- These Aren't the Permissions You're Looking For
DEFCON 18- These Aren't the Permissions You're Looking ForDEFCON 18- These Aren't the Permissions You're Looking For
DEFCON 18- These Aren't the Permissions You're Looking For
Michael Scovetta
 
Systematic Detection of Capability Leaks in Stock Android Smartphones
Systematic Detection of Capability Leaks in Stock Android SmartphonesSystematic Detection of Capability Leaks in Stock Android Smartphones
Systematic Detection of Capability Leaks in Stock Android Smartphones
Michael Scovetta
 

Mehr von Michael Scovetta (20)

Android Attacks
Android AttacksAndroid Attacks
Android Attacks
 
Strategic Surprise
Strategic SurpriseStrategic Surprise
Strategic Surprise
 
Stackjacking
StackjackingStackjacking
Stackjacking
 
Modern Kernel Pool Exploitation: Attacks and Techniques
Modern Kernel Pool Exploitation: Attacks and TechniquesModern Kernel Pool Exploitation: Attacks and Techniques
Modern Kernel Pool Exploitation: Attacks and Techniques
 
Exploitation and State Machines
Exploitation and State MachinesExploitation and State Machines
Exploitation and State Machines
 
Don't Give Credit: Hacking Arcade Machines
Don't Give Credit: Hacking Arcade MachinesDon't Give Credit: Hacking Arcade Machines
Don't Give Credit: Hacking Arcade Machines
 
Attacking the WebKit Heap
Attacking the WebKit HeapAttacking the WebKit Heap
Attacking the WebKit Heap
 
The Listening: Email Client Backdoor
The Listening: Email Client BackdoorThe Listening: Email Client Backdoor
The Listening: Email Client Backdoor
 
Smooth CoffeeScript
Smooth CoffeeScriptSmooth CoffeeScript
Smooth CoffeeScript
 
DEFCON 18- These Aren't the Permissions You're Looking For
DEFCON 18- These Aren't the Permissions You're Looking ForDEFCON 18- These Aren't the Permissions You're Looking For
DEFCON 18- These Aren't the Permissions You're Looking For
 
Systematic Detection of Capability Leaks in Stock Android Smartphones
Systematic Detection of Capability Leaks in Stock Android SmartphonesSystematic Detection of Capability Leaks in Stock Android Smartphones
Systematic Detection of Capability Leaks in Stock Android Smartphones
 
Consumer Password Worst Practices
Consumer Password Worst PracticesConsumer Password Worst Practices
Consumer Password Worst Practices
 
HTML5 Web Security
HTML5 Web SecurityHTML5 Web Security
HTML5 Web Security
 
A collection of examples of 64 bit errors in real programs
A collection of examples of 64 bit errors in real programsA collection of examples of 64 bit errors in real programs
A collection of examples of 64 bit errors in real programs
 
If You Don't Like the Game, Hack the Playbook... (Zatko)
If You Don't Like the Game, Hack the Playbook... (Zatko)If You Don't Like the Game, Hack the Playbook... (Zatko)
If You Don't Like the Game, Hack the Playbook... (Zatko)
 
Scaling Cyberwarfare (Roelker)
Scaling Cyberwarfare (Roelker)Scaling Cyberwarfare (Roelker)
Scaling Cyberwarfare (Roelker)
 
High Assurance Systems (Fisher)
High Assurance Systems (Fisher)High Assurance Systems (Fisher)
High Assurance Systems (Fisher)
 
PROCEED and Crowd-Sourced Formal Verification
PROCEED and Crowd-Sourced Formal VerificationPROCEED and Crowd-Sourced Formal Verification
PROCEED and Crowd-Sourced Formal Verification
 
National Cyber Range (Ranka)
National Cyber Range (Ranka)National Cyber Range (Ranka)
National Cyber Range (Ranka)
 
Beyond Passwords (Guidorizzi)
Beyond Passwords (Guidorizzi)Beyond Passwords (Guidorizzi)
Beyond Passwords (Guidorizzi)
 

Kürzlich hochgeladen

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Kürzlich hochgeladen (20)

Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 

Peter Norvig - NYC Machine Learning 2013

  • 1. Learning Programming (by Humans by Machine) Peter Norvig
  • 3.
  • 4. For when you don’t know what’s going on
  • 5.
  • 6.
  • 7. Hal Abelson (SIGCSE Outstanding Contribution Award, 2012)
  • 8.
  • 9. The average under tutoring [with mastery learning] was about two standard deviations above the average of the control. (Benjamin Bloom, 1984)
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24. Python llll lll C ll Scheme l But: for “vocational”, not “academic”: Javascript dominates
  • 25. Analyzing Intro CS MOOC Data 101 Student Name Final Grade Peter Norvig
  • 26. Analyzing Intro CS MOOC Data 101 Student Name Final Grade Peter Norvig Incomplete
  • 27.
  • 29. Data Sources • Enrollment / Dropout • Quizzes and Exams • Video Interactions • Discussion Forums • Program submissions
  • 30. Syntax Error Messages Vague if x = 1: ^ SyntaxError: invalid syntax SyntaxError: Use “==“ for comparison, use “=“ for assignment.
  • 31. Inputs to Compiler Error Routine Source Code Compiler Error Message Ignored: - Past program submission histories - Problem Description - Comments - Test Cases
  • 33.
  • 35. Match Programs to Corpus S1 suggest C2C1Corpus: Submission:
  • 36. Syntax Error Messages Vague if spam == 42 print('Hello!') ^ SyntaxError: invalid syntax ^ IndentationError: unexpected indent SyntaxError: Colon required, use “if spam == 42:“
  • 37. Attractive Nuisance >>> table = [[]] * 5 >>> table [[], [], [], [], []] >>> def add(key, table): table[hash(key) % len(table)].append(key) >>> add(’x', table) >>> table [[’x'], [’x'], [’x'], [’x'], [’x']] >>> table = listof(5, list) def listof(n, kind): "Make a list of n elements, each kind()." return [kind() for _ in range(n)]
  • 38. Faulty Student Models • Given: • nums = [1, 2, -3] • max(nums) == 2 • max(nums, key=abs) == -3 • len(‘word’) == 4 • wordlist = file(‘words’).read().split() • What is the longest word in wordlist? • ? • ?? • max(wordlist, key=len)
  • 39. DWIM Attractive Nuisance • def qsort(A): if len(A) <= 1: return A LT, EQ, GT = [] p = A[len(A)/2] for x in A: bin = (LT if x<p else GT if x>p else EQ) bin.append(x) return qsort(LT) + EQ + qsort(GT) LT = EQ = GT = [] LT, EQ, GT = [], [], []
  • 40. DWIM Attractive Nuisance • spam = ['cat', 'dog', 'mouse'] • for i in range(spam): print(spam[i]) • for i in len(spam): print(spam[i]) • for i in range(len(spam)): print(spam[i]) for x in spam: print(x) • for i in range(len(spam)): spam[i] = spam[i].upper() for x in spam: x = x.upper()
  • 41. Language Too General y = (x + 1) (x - 1) TypeError: unsupported operand type(s) for +: 'object' and 'int' SyntaxError: ‘*’ operator required. use “(x + 1) * (x – 1)”
  • 42. Multi-Language Confusion File ”example.py”, line 1 root = new Node() ^ SyntaxError: invalid syntax Test.java:6: cannot find symbol symbol : class awt location: package java java.awt.Color(r, g, b); ^ Error: use “Node()” in Python; use “new Color(…)” in Java
  • 43. Multi-Language Confusion int total = 0; for (int x : A) { total += x; } total = None for x in A: total = total + x
  • 44. Why do students stay or go?
  • 45.
  • 46. the movie in which no babies are born
  • 47. `
  • 48.
  • 49. Motivation, not Information • Willpower • Due dates • Peer support (forums, outside groups) • Faculty encouragement (email) • Pride of accomplishment • Authenticity • Early adopter
  • 50.
  • 51. Timeline (successful Coursera course) #usersonsite Real Couse 100K 50K ramp up + exponential decay + due dates synchronous, evergreen, semi-synchronous, bus route
  • 53.
  • 54. Representing Programs • Bag of words • N-grams of tokens (tagged?) • Static syntax tree • Dynamic call graph
  • 56.
  • 57. Alpha, Beta (steady progress) significantly better than Gamma (thrash, then solution) on midterm. No significant difference on score for this assignment.
  • 59. • Learning results from what the student does and thinks and only from what the student does and thinks. The teacher can advance learning only by influencing what the student does to learn. - Herb Simon
  • 60. • Problem first, then explanation (Peer) • Make students to prediction (see it is wrong) • (They didn’t like that.) • Experiments with more open-ended problems • Experiments with peer grading
  • 61. • “Richard Hamming told me his secret: First get together the problem sets and exams that you want the students to be able to solve. Then write a book that will teach them how to solve them.” -- Hal Varian (1993) T D D
  • 62. Do we trust Bloom?
  • 63.
  • 64.
  • 68. How do we make it one to one tutoring and mastery learning?
  • 69.
  • 70. “Books will soon be obsolete in the public schools. Scholars will be instructed through the eye. It is possible to teach every branch of human knowledge with the motion picture. Our school system will be completely changed inside of ten years”- Edison, 1913
  • 71. “Lectures were once useful, but now, when all can read, and books so numerous, lectures are unnecessary. If your attention fails, and you miss part of a lecture, it is lost; you cannot go back as you do upon a book.” - 1791
  • 72. Many Types of Individualization • Student control over rewind (multiplay) • Student choice of where to go next (multipath) • Flipped classroom (Koller, etc.) • Peer instruction (Mazur) • Peer feedback • Forums and other one-to-one interactions • Machine learning? • Courseware Engineering?
  • 73. Open-Ended Work Peer Grade Self Grade 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Teacher Grade StudentGrade “The Impact of Self-and Peer-Grading on Student Learning”. P. Sadler, E. Good. Educational Assessment (2006).
  • 74. 20 bits per year 192K bits per sec
  • 76. Modern sequencing methods (11 min) Genomic economics (8 min) Personalized medicine (13 min) History: The Human Genome Project (12 min) Modern GenomicsGenes and alleles (10 min) Basic Genetics Refresher (OPTIONAL) Mendelian inheritance (13 min) What is DNA? (12 min) Genetic testing in the commercial world (11 min) Protecting privacy (9 minutes) Case study: direct- to-consumer genetics (12 min) Case study: family planning (13 min) Commercial Genomics (OPTIONAL) Authored Multipath Sequence
  • 77.
  • 78.
  • 80.
  • 81.
  • 82.
  • 83.
  • 84.
  • 85. One on One Tutoring; Mastery Learning Social; Motivation Large-Scale Courseware Engineering Machine Learning; Continual Improvement