SlideShare ist ein Scribd-Unternehmen logo
1 von 46
Downloaden Sie, um offline zu lesen
Testing Computer-Aided
Mnemonics and Feedback for Fast
Memorization of High-Value Secrets
1
Sauvik Das
Carnegie Mellon
University
Jason Hong
Carnegie Mellon
University
Stuart Schechter
Microsoft
Research
USEC 2016
Summary
Question: Can computer-assisted mnemonics help
people learn strong, randomly assigned passwords
quickly and with high long-term recall?
Method: Design and implementation of two computer-
assisted mnemonic training regimens and an
experimental evaluation of their effectiveness.
2
Summary
3
Outcome: Our story mnemonic, in which users weave
chunks of their random secret into sentences, works
really well (7.5 learning sessions, 84% two-week recall)
A pink _ _ _ _ is chasing a giant _ _ _ _ blowing in the _ _ _ _.cat
cat leaf wind
First off…
More stuff about passwords? Ugh. Beat a dead horse
much?
But I cannot, in good faith, call myself a “usable
security” researcher without doing a paper on
passwords. So, here it is.
4
Sometimes, we need strong secrets
5
Encrypted
drives
Password
managers
Journalists /
whistle blowers
Strong passwords
If we still need strong passwords, we should make
strong passwords easy to use.
Typical approach: Teach people strong password
generation strategies. But people aren’t random
enough.
Out-of-vogue approach: Give people randomly
assigned strong passwords. But, people can’t
remember anything…right?
6
Actually, it works…
Bonneau & Schechter (2014) conditioned lay people to
learn strong assigned secrets (56.4 bits) incrementally.
On login, had people enter assigned secret (which is
shown after a delay that gradually increases).
People could avoid delay if they enter the assigned
secret from memory.
Example: apple beetle crane dog ear flame
7
8
apple beet
User ID:
hotshot@handsome.cc
Password:
9
apple beet
User ID:
hotshot@handsome.cc
Password:
apple beetle crane dog ear flame
94% of participants
memorized all of
their secret words.
10
But…
The process was long
(required 36 sessions)
Recall fell sharply
after 2 weeks (62%
remembered)
11
Can we do better?
Ideally, the process should:
- Require fewer training sessions
- Retained for longer
12
Mnemonics?!
“Strategies to enhance the learning and recall of
information”
Most useful when memorizing lists.
Basic idea: Transform abstract, difficult to visualize
concepts into tangible, easy to visualize concepts.
13
14
Chain-type
Weave list items into
narrative chain.
an apple being eaten by a beetle
while being lifted up by a crane.
Memorize: apple, beetle, crane
Peg-type
Peg list items onto
cueing structure.
a nun throwing an apple
a shoe squashing a beetle
a tree being lifted by a crane
one is nun
two is shoe
three is tree
15
Can we use mnemonics to make learning
strong secrets quicker and longer-lasting?
Research Question:
Our contribution
1. Two computer-assisted mnemonic training schemes
designed to incrementally teach people strong secrets.
2. Experimental evaluation of these training schemes
relative to the rote baseline akin to prior work.
16
Method
17
Method Overview
Recruit participants, via MTurk, for a recurring “attention”
test over 10 days.
Assign participants, randomly, to a treatment or control
group.
Teach participants a randomly selected strong secret with
an assigned training regimen.
Compare, across training regimens, how fast participants
learn their secrets and how well they remember their
secret words 2-weeks after the fact.
18
Recruitment
Participants solicited via MTurk to take “attention test”.
“Attention test” cover story used to distract participants
from focusing on learning their secret words.
Participants had to return to do 45 attention tests over
maximum of 10 days with at least 1 hour between each.
19
Assignment
Participants were randomly assigned to one of five
experimental groups:
Story: Mnemonic treatment.
Peg-Word: Mnemonic treatment.
Feedback: Active learning treatment.
Rote: Control. Baseline from prior work.
Dropout: Control. No secret words.
20
Assignment
Participants were randomly assigned to one of five
experimental groups:
Story: Mnemonic treatment.
Peg-Word: Mnemonic treatment.
Rote: Control. Baseline from prior work.
Feedback: Active learning treatment.
Dropout: Control. No secret words.
21
Teach secret
Secret Encoding: 6 nouns randomly selected out of a
broader set of 676 options (56.4 bits).
Assigned in two, 3-word chunks.
Presented one chunk at a time — only shown second
chunk after “learning first” (entering 3x consecutively
from memory)
22
Teach secret
On each login, participants had to enter their secret
words.
If they couldn’t remember, they would eventually be
shown their words to copy.
23
apple beetle crane
apple beetle crane
Comparison
Primarily interested in two comparisons of mnemonic
vs rote baseline:
- Number of learning sessions required: Lower is
better (previous study rote method: median of 36)
- 2-week recall rate: Higher is better (previous study
rote method: 62%)
24
Experimental
Groups
25
3 Groups
Story Mnemonic: Users wrote two sentences, each
containing three of their secret words, in order.
Peg-Word Mnemonic: Users were given public “peg”
words and created sentences linking each peg to a secret.
Rote Baseline: No special training. Users just had to
manually enter words to the best of their memory.
Dimensions of Variance: mnemonic creation, chunk
rehearsal, and hint progression.
26
Rote Baseline
Mnemonic Creation: none
Chunk Rehearsal: Text-boxes with hints ultimately
rendered on top for copying.
Hint Progression: Started at 0 seconds, progresses up
to a maximum of 10 seconds with a 0.5 second
increase every subsequent attempt.
27
apple beetle crane
apple beetle crane
Story Mnemonic
Mnemonic Creation: Asked participants to write a
visual, memorable sentence stringing together all
three words.
Example: cat, leaf, wind => A pink cat is chasing a giant
leaf blowing in the wind.
Participants had to comply to proceed — our app
checked whether the story sentence was valid.
28
Story Mnemonic
Chunk Rehearsal: Three-phases of chunk rehearsal to
gradually wean participants off mnemonic assistance.
Introduced the hint well to facilitate this gradual transition. The
hint well allowed participants to enter their secret words inline.
29
A pink _ _ _ _ is chasing a giant _ _ _ _ blowing in the _ _ _ _.
Phase 1: Full assistance
cat
cat
cat leaf wind
cat leaf wind
Story Mnemonic
30
A pink _ _ _ _ is chasing a giant _ _ _ _ blowing in the _ _ _ _.
Phase 2: Reduced assistance
cat
cat
Phase 3: No assistance
cat
Story Mnemonic
Phase Transition: Start in full-assistance, then reduced-
assistance, then no-assistance. Transition after first
entering from memory in previous phase.
Hint Progression: Start at 1 second, max of 10
seconds. Increment by 0.5 seconds on each attempt.
31
Peg-Word Mnemonic
Mnemonic Creation: First, asked participants to select rhyme-based
peg words. Then, asked participants to write three visual, memorable
sentences stringing together public peg words with secret words.
Example: cat, leaf, wind
1 is {nun, bun, gun, sun}
2 is {shoe, shrew, zoo, screw}
3 is {tree, bee, key, sea}
32
Peg-Word Mnemonic
33
Public peg
word
Secret
word
Mnemonic hint sentence
nun cat The nun pulls the screeching cat off of her head.
leaf shoe The leaf lands gracefully in the worn old shoe.
wind tree The wind has no effect on the steadfast tree.
Participants had to comply to proceed — our app checked
whether the peg sentences were valid.
Peg-Word Mnemonic
Chunk Rehearsal: Same three-phases of chunk rehearsal to gradually
wean participants off mnemonic assistance. Difference: hint
sentences were shown per word instead of per chunk.
34
One is nun. The nun pulls the screeching _ _ _ _ off of her head.
cat leaf wind
cat
But hints still operated per-chunk, not per-word.
Peg-Word Mnemonic
Phase Transition: Same as story.
Hint Progression: Same as story.
35
Results
Learning Speed & Memorability
36
Some Descriptive Stats
37
351
participants
signed up.
242
participants
finished.
31
years old on
average (sd 10).
52
percent
female.
Rote Story Peg
Assigned 71 81 75
Completed 51 57 48
2-week
return
42 48 43
Learning sessions
Participants who entered their secret words, from
memory, three consecutive times had learned their
secret words.
The number of sessions prior to that 3-chain were
“learning sessions”.
Q1: How did the number of learning sessions differ
between those in our mnemonic treatments versus
those in the rote baseline?
38
Learning Sessions
39
Both story (7.5) and peg-word (9) participants required
significantly fewer learning sessions than rote (12).
2-Week Recall Rate
Q2: How did the 2-week recall rate vary between
participants in the mnemonic treatments versus the
rote baseline?
40
2-Week Recall Rate
41
Name Entropy Example
Perfect 56.4 apple beetle crane dog ear flame
Single Swap 53.8 apple crane beetle dog ear flame
Forgot One 49.6 apple ? crane dog ear flame
Relaxed Order 46.9 ear apple dog beetle flame crane
2-Week Recall Rate
42
Story
Peg-
Word
Rote
Perfect 75% 42% 60%
Single
Swap
84% 47% 65%
Forgot
One
84% 67% 74%
Relaxed
Order
86% 47% 65%
Takeaways
Story did significantly
better than Rote.
Peg-Word,
surprisingly, did worse
(but non-significant).
p<0.05
p<0.10
Results Summary
Mnemonics require fewer learning sessions and can
provide recall benefits, but only if not too complicated.
Story mnemonic did best, providing significant
improvements over rote on both reducing required
learning sessions and 2-week recall.
We can perhaps improve retention further if we also
facilitate remembering the mnemonics themselves.
43
Conclusion
44
Some use-cases warrant the use of provably strong
secrets.
Bonneau and Schechter (2014): Lay people can reliably
learn such secrets through conditioning, but it takes a
while and recall falls sharply after extended disuse.
We designed and implemented two computer-assisted
mnemonic training regimens to speed-up learning and
increase recall rates after extended disuse.
Our story mnemonic did both of those things, but our
peg-word mnemonic was surprisingly ineffective.
45
Testing Computer-Aided
Mnemonics and Feedback for Fast
Memorization of High-Value Secrets
46
Sauvik Das
Carnegie Mellon
University
Jason Hong
Carnegie Mellon
University
Stuart Schechter
Microsoft
Research
Email contact: sauvik@cmu.edu
Key-takeaway:
Our computer-assisted story mnemonic decreased the
required number of sessions required to learn and increased
the two-week recall rates of strong, randomly assigned secrets.

Weitere ähnliche Inhalte

Ähnlich wie Testing Computer-Assisted Mnemonics and Feedback for Fast Memorization of High-Value Secrets

Psychology HL - IA
Psychology HL - IAPsychology HL - IA
Psychology HL - IAMahitha Sai
 
Psy 352 week 2 quiz new
Psy 352 week 2 quiz newPsy 352 week 2 quiz new
Psy 352 week 2 quiz newsimonbaileyy
 
mapping design patterns
mapping design patternsmapping design patterns
mapping design patternsYishay Mor
 
HCI Fundamentals - Part 2 : Human memory and thinking
HCI Fundamentals - Part 2 : Human memory and thinkingHCI Fundamentals - Part 2 : Human memory and thinking
HCI Fundamentals - Part 2 : Human memory and thinkingtamizh arthanari
 
Math Teaching Strategies Presentation
Math Teaching Strategies PresentationMath Teaching Strategies Presentation
Math Teaching Strategies PresentationMethusael Cebrian
 
Oz_Chap 2_M3_Lesson Slides_Variables.pptx
Oz_Chap 2_M3_Lesson Slides_Variables.pptxOz_Chap 2_M3_Lesson Slides_Variables.pptx
Oz_Chap 2_M3_Lesson Slides_Variables.pptxALEJANDROLEONGOVEA
 
Understand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analyticsUnderstand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analyticsVitomir Kovanovic
 
Manufacturing Effective Training Strategies.ppt
Manufacturing Effective Training Strategies.pptManufacturing Effective Training Strategies.ppt
Manufacturing Effective Training Strategies.pptMadan Karki
 
Action Sequence Mining and Behavior Pattern Analysis for User Modeling
Action Sequence Mining and Behavior Pattern Analysis for User ModelingAction Sequence Mining and Behavior Pattern Analysis for User Modeling
Action Sequence Mining and Behavior Pattern Analysis for User ModelingPeter Brusilovsky
 
COURSE REDESIGN: Model-based Problem Solving in the Large Science Classroom
COURSE REDESIGN:  Model-based Problem Solving  in the Large Science ClassroomCOURSE REDESIGN:  Model-based Problem Solving  in the Large Science Classroom
COURSE REDESIGN: Model-based Problem Solving in the Large Science ClassroomSteven BREWER
 
2008 Memory Revision
2008 Memory Revision2008 Memory Revision
2008 Memory Revisionjillfitz
 
Sentiment analysis of Twitter Data
Sentiment analysis of Twitter DataSentiment analysis of Twitter Data
Sentiment analysis of Twitter DataNurendra Choudhary
 
G6 m2-a-lesson 2-t
G6 m2-a-lesson 2-tG6 m2-a-lesson 2-t
G6 m2-a-lesson 2-tmlabuski
 
LearningAG.ppt
LearningAG.pptLearningAG.ppt
LearningAG.pptbutest
 

Ähnlich wie Testing Computer-Assisted Mnemonics and Feedback for Fast Memorization of High-Value Secrets (20)

Psychology HL - IA
Psychology HL - IAPsychology HL - IA
Psychology HL - IA
 
Groupcfinalppt
GroupcfinalpptGroupcfinalppt
Groupcfinalppt
 
Group Bfinalppt
Group BfinalpptGroup Bfinalppt
Group Bfinalppt
 
Psy 352 week 2 quiz new
Psy 352 week 2 quiz newPsy 352 week 2 quiz new
Psy 352 week 2 quiz new
 
mapping design patterns
mapping design patternsmapping design patterns
mapping design patterns
 
HCI Fundamentals - Part 2 : Human memory and thinking
HCI Fundamentals - Part 2 : Human memory and thinkingHCI Fundamentals - Part 2 : Human memory and thinking
HCI Fundamentals - Part 2 : Human memory and thinking
 
Math Teaching Strategies Presentation
Math Teaching Strategies PresentationMath Teaching Strategies Presentation
Math Teaching Strategies Presentation
 
Oz_Chap 2_M3_Lesson Slides_Variables.pptx
Oz_Chap 2_M3_Lesson Slides_Variables.pptxOz_Chap 2_M3_Lesson Slides_Variables.pptx
Oz_Chap 2_M3_Lesson Slides_Variables.pptx
 
Understand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analyticsUnderstand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analytics
 
Manufacturing Effective Training Strategies.ppt
Manufacturing Effective Training Strategies.pptManufacturing Effective Training Strategies.ppt
Manufacturing Effective Training Strategies.ppt
 
Unit1 revision
Unit1 revisionUnit1 revision
Unit1 revision
 
An Experimental Study of Instructor Immediacy
An Experimental Study of Instructor Immediacy An Experimental Study of Instructor Immediacy
An Experimental Study of Instructor Immediacy
 
Action Sequence Mining and Behavior Pattern Analysis for User Modeling
Action Sequence Mining and Behavior Pattern Analysis for User ModelingAction Sequence Mining and Behavior Pattern Analysis for User Modeling
Action Sequence Mining and Behavior Pattern Analysis for User Modeling
 
COURSE REDESIGN: Model-based Problem Solving in the Large Science Classroom
COURSE REDESIGN:  Model-based Problem Solving  in the Large Science ClassroomCOURSE REDESIGN:  Model-based Problem Solving  in the Large Science Classroom
COURSE REDESIGN: Model-based Problem Solving in the Large Science Classroom
 
Media as Levers
Media as LeversMedia as Levers
Media as Levers
 
2008 Memory Revision
2008 Memory Revision2008 Memory Revision
2008 Memory Revision
 
Sentiment analysis of Twitter Data
Sentiment analysis of Twitter DataSentiment analysis of Twitter Data
Sentiment analysis of Twitter Data
 
Memory
MemoryMemory
Memory
 
G6 m2-a-lesson 2-t
G6 m2-a-lesson 2-tG6 m2-a-lesson 2-t
G6 m2-a-lesson 2-t
 
LearningAG.ppt
LearningAG.pptLearningAG.ppt
LearningAG.ppt
 

Mehr von Sauvik Das

Privacy for the People | HCIC 2023
Privacy for the People | HCIC 2023Privacy for the People | HCIC 2023
Privacy for the People | HCIC 2023Sauvik Das
 
Social Cybersecurity: Reshaping Security Through An Empirical Understanding o...
Social Cybersecurity: Reshaping Security Through An Empirical Understanding o...Social Cybersecurity: Reshaping Security Through An Empirical Understanding o...
Social Cybersecurity: Reshaping Security Through An Empirical Understanding o...Sauvik Das
 
Thumprint: Socially-Inclusive Local Group Authentication Through Shared Secre...
Thumprint: Socially-Inclusive Local Group Authentication Through Shared Secre...Thumprint: Socially-Inclusive Local Group Authentication Through Shared Secre...
Thumprint: Socially-Inclusive Local Group Authentication Through Shared Secre...Sauvik Das
 
Epistenet: Facilitating Programmatic Access & Processing of Semantically Rela...
Epistenet: Facilitating Programmatic Access & Processing of Semantically Rela...Epistenet: Facilitating Programmatic Access & Processing of Semantically Rela...
Epistenet: Facilitating Programmatic Access & Processing of Semantically Rela...Sauvik Das
 
A Market In Your Social Network: The Effect of Extrinsic Rewards on Friendsou...
A Market In Your Social Network: The Effect of Extrinsic Rewards on Friendsou...A Market In Your Social Network: The Effect of Extrinsic Rewards on Friendsou...
A Market In Your Social Network: The Effect of Extrinsic Rewards on Friendsou...Sauvik Das
 
Examining Game World Topology Personalization
Examining Game World Topology PersonalizationExamining Game World Topology Personalization
Examining Game World Topology PersonalizationSauvik Das
 
Revival Actions in a Shooter Game
Revival Actions in a Shooter GameRevival Actions in a Shooter Game
Revival Actions in a Shooter GameSauvik Das
 
Self-Censorship on Facebook
Self-Censorship on FacebookSelf-Censorship on Facebook
Self-Censorship on FacebookSauvik Das
 

Mehr von Sauvik Das (8)

Privacy for the People | HCIC 2023
Privacy for the People | HCIC 2023Privacy for the People | HCIC 2023
Privacy for the People | HCIC 2023
 
Social Cybersecurity: Reshaping Security Through An Empirical Understanding o...
Social Cybersecurity: Reshaping Security Through An Empirical Understanding o...Social Cybersecurity: Reshaping Security Through An Empirical Understanding o...
Social Cybersecurity: Reshaping Security Through An Empirical Understanding o...
 
Thumprint: Socially-Inclusive Local Group Authentication Through Shared Secre...
Thumprint: Socially-Inclusive Local Group Authentication Through Shared Secre...Thumprint: Socially-Inclusive Local Group Authentication Through Shared Secre...
Thumprint: Socially-Inclusive Local Group Authentication Through Shared Secre...
 
Epistenet: Facilitating Programmatic Access & Processing of Semantically Rela...
Epistenet: Facilitating Programmatic Access & Processing of Semantically Rela...Epistenet: Facilitating Programmatic Access & Processing of Semantically Rela...
Epistenet: Facilitating Programmatic Access & Processing of Semantically Rela...
 
A Market In Your Social Network: The Effect of Extrinsic Rewards on Friendsou...
A Market In Your Social Network: The Effect of Extrinsic Rewards on Friendsou...A Market In Your Social Network: The Effect of Extrinsic Rewards on Friendsou...
A Market In Your Social Network: The Effect of Extrinsic Rewards on Friendsou...
 
Examining Game World Topology Personalization
Examining Game World Topology PersonalizationExamining Game World Topology Personalization
Examining Game World Topology Personalization
 
Revival Actions in a Shooter Game
Revival Actions in a Shooter GameRevival Actions in a Shooter Game
Revival Actions in a Shooter Game
 
Self-Censorship on Facebook
Self-Censorship on FacebookSelf-Censorship on Facebook
Self-Censorship on Facebook
 

Kürzlich hochgeladen

Charateristics of the Angara-A5 spacecraft launched from the Vostochny Cosmod...
Charateristics of the Angara-A5 spacecraft launched from the Vostochny Cosmod...Charateristics of the Angara-A5 spacecraft launched from the Vostochny Cosmod...
Charateristics of the Angara-A5 spacecraft launched from the Vostochny Cosmod...Christina Parmionova
 
Immunoblott technique for protein detection.ppt
Immunoblott technique for protein detection.pptImmunoblott technique for protein detection.ppt
Immunoblott technique for protein detection.pptAmirRaziq1
 
KDIGO-2023-CKD-Guideline-Public-Review-Draft_5-July-2023.pdf
KDIGO-2023-CKD-Guideline-Public-Review-Draft_5-July-2023.pdfKDIGO-2023-CKD-Guideline-Public-Review-Draft_5-July-2023.pdf
KDIGO-2023-CKD-Guideline-Public-Review-Draft_5-July-2023.pdfGABYFIORELAMALPARTID1
 
Q4-Mod-1c-Quiz-Projectile-333344444.pptx
Q4-Mod-1c-Quiz-Projectile-333344444.pptxQ4-Mod-1c-Quiz-Projectile-333344444.pptx
Q4-Mod-1c-Quiz-Projectile-333344444.pptxtuking87
 
complex analysis best book for solving questions.pdf
complex analysis best book for solving questions.pdfcomplex analysis best book for solving questions.pdf
complex analysis best book for solving questions.pdfSubhamKumar3239
 
Loudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxLoudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxpriyankatabhane
 
FBI Profiling - Forensic Psychology.pptx
FBI Profiling - Forensic Psychology.pptxFBI Profiling - Forensic Psychology.pptx
FBI Profiling - Forensic Psychology.pptxPayal Shrivastava
 
How we decide powerpoint presentation.pptx
How we decide powerpoint presentation.pptxHow we decide powerpoint presentation.pptx
How we decide powerpoint presentation.pptxJosielynTars
 
bonjourmadame.tumblr.com bhaskar's girls
bonjourmadame.tumblr.com bhaskar's girlsbonjourmadame.tumblr.com bhaskar's girls
bonjourmadame.tumblr.com bhaskar's girlshansessene
 
Forensic limnology of diatoms by Sanjai.pptx
Forensic limnology of diatoms by Sanjai.pptxForensic limnology of diatoms by Sanjai.pptx
Forensic limnology of diatoms by Sanjai.pptxkumarsanjai28051
 
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptxGENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptxRitchAndruAgustin
 
DECOMPOSITION PATHWAYS of TM-alkyl complexes.pdf
DECOMPOSITION PATHWAYS of TM-alkyl complexes.pdfDECOMPOSITION PATHWAYS of TM-alkyl complexes.pdf
DECOMPOSITION PATHWAYS of TM-alkyl complexes.pdfDivyaK787011
 
Explainable AI for distinguishing future climate change scenarios
Explainable AI for distinguishing future climate change scenariosExplainable AI for distinguishing future climate change scenarios
Explainable AI for distinguishing future climate change scenariosZachary Labe
 
Gas-ExchangeS-in-Plants-and-Animals.pptx
Gas-ExchangeS-in-Plants-and-Animals.pptxGas-ExchangeS-in-Plants-and-Animals.pptx
Gas-ExchangeS-in-Plants-and-Animals.pptxGiovaniTrinidad
 
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Sérgio Sacani
 
well logging & petrophysical analysis.pptx
well logging & petrophysical analysis.pptxwell logging & petrophysical analysis.pptx
well logging & petrophysical analysis.pptxzaydmeerab121
 
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPRPirithiRaju
 

Kürzlich hochgeladen (20)

Charateristics of the Angara-A5 spacecraft launched from the Vostochny Cosmod...
Charateristics of the Angara-A5 spacecraft launched from the Vostochny Cosmod...Charateristics of the Angara-A5 spacecraft launched from the Vostochny Cosmod...
Charateristics of the Angara-A5 spacecraft launched from the Vostochny Cosmod...
 
Immunoblott technique for protein detection.ppt
Immunoblott technique for protein detection.pptImmunoblott technique for protein detection.ppt
Immunoblott technique for protein detection.ppt
 
KDIGO-2023-CKD-Guideline-Public-Review-Draft_5-July-2023.pdf
KDIGO-2023-CKD-Guideline-Public-Review-Draft_5-July-2023.pdfKDIGO-2023-CKD-Guideline-Public-Review-Draft_5-July-2023.pdf
KDIGO-2023-CKD-Guideline-Public-Review-Draft_5-July-2023.pdf
 
Q4-Mod-1c-Quiz-Projectile-333344444.pptx
Q4-Mod-1c-Quiz-Projectile-333344444.pptxQ4-Mod-1c-Quiz-Projectile-333344444.pptx
Q4-Mod-1c-Quiz-Projectile-333344444.pptx
 
complex analysis best book for solving questions.pdf
complex analysis best book for solving questions.pdfcomplex analysis best book for solving questions.pdf
complex analysis best book for solving questions.pdf
 
Loudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxLoudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptx
 
FBI Profiling - Forensic Psychology.pptx
FBI Profiling - Forensic Psychology.pptxFBI Profiling - Forensic Psychology.pptx
FBI Profiling - Forensic Psychology.pptx
 
How we decide powerpoint presentation.pptx
How we decide powerpoint presentation.pptxHow we decide powerpoint presentation.pptx
How we decide powerpoint presentation.pptx
 
Interferons.pptx.
Interferons.pptx.Interferons.pptx.
Interferons.pptx.
 
bonjourmadame.tumblr.com bhaskar's girls
bonjourmadame.tumblr.com bhaskar's girlsbonjourmadame.tumblr.com bhaskar's girls
bonjourmadame.tumblr.com bhaskar's girls
 
Forensic limnology of diatoms by Sanjai.pptx
Forensic limnology of diatoms by Sanjai.pptxForensic limnology of diatoms by Sanjai.pptx
Forensic limnology of diatoms by Sanjai.pptx
 
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptxGENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
 
DECOMPOSITION PATHWAYS of TM-alkyl complexes.pdf
DECOMPOSITION PATHWAYS of TM-alkyl complexes.pdfDECOMPOSITION PATHWAYS of TM-alkyl complexes.pdf
DECOMPOSITION PATHWAYS of TM-alkyl complexes.pdf
 
Explainable AI for distinguishing future climate change scenarios
Explainable AI for distinguishing future climate change scenariosExplainable AI for distinguishing future climate change scenarios
Explainable AI for distinguishing future climate change scenarios
 
Gas-ExchangeS-in-Plants-and-Animals.pptx
Gas-ExchangeS-in-Plants-and-Animals.pptxGas-ExchangeS-in-Plants-and-Animals.pptx
Gas-ExchangeS-in-Plants-and-Animals.pptx
 
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
 
AZOTOBACTER AS BIOFERILIZER.PPTX
AZOTOBACTER AS BIOFERILIZER.PPTXAZOTOBACTER AS BIOFERILIZER.PPTX
AZOTOBACTER AS BIOFERILIZER.PPTX
 
PLASMODIUM. PPTX
PLASMODIUM. PPTXPLASMODIUM. PPTX
PLASMODIUM. PPTX
 
well logging & petrophysical analysis.pptx
well logging & petrophysical analysis.pptxwell logging & petrophysical analysis.pptx
well logging & petrophysical analysis.pptx
 
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
 

Testing Computer-Assisted Mnemonics and Feedback for Fast Memorization of High-Value Secrets

  • 1. Testing Computer-Aided Mnemonics and Feedback for Fast Memorization of High-Value Secrets 1 Sauvik Das Carnegie Mellon University Jason Hong Carnegie Mellon University Stuart Schechter Microsoft Research USEC 2016
  • 2. Summary Question: Can computer-assisted mnemonics help people learn strong, randomly assigned passwords quickly and with high long-term recall? Method: Design and implementation of two computer- assisted mnemonic training regimens and an experimental evaluation of their effectiveness. 2
  • 3. Summary 3 Outcome: Our story mnemonic, in which users weave chunks of their random secret into sentences, works really well (7.5 learning sessions, 84% two-week recall) A pink _ _ _ _ is chasing a giant _ _ _ _ blowing in the _ _ _ _.cat cat leaf wind
  • 4. First off… More stuff about passwords? Ugh. Beat a dead horse much? But I cannot, in good faith, call myself a “usable security” researcher without doing a paper on passwords. So, here it is. 4
  • 5. Sometimes, we need strong secrets 5 Encrypted drives Password managers Journalists / whistle blowers
  • 6. Strong passwords If we still need strong passwords, we should make strong passwords easy to use. Typical approach: Teach people strong password generation strategies. But people aren’t random enough. Out-of-vogue approach: Give people randomly assigned strong passwords. But, people can’t remember anything…right? 6
  • 7. Actually, it works… Bonneau & Schechter (2014) conditioned lay people to learn strong assigned secrets (56.4 bits) incrementally. On login, had people enter assigned secret (which is shown after a delay that gradually increases). People could avoid delay if they enter the assigned secret from memory. Example: apple beetle crane dog ear flame 7
  • 10. 94% of participants memorized all of their secret words. 10
  • 11. But… The process was long (required 36 sessions) Recall fell sharply after 2 weeks (62% remembered) 11
  • 12. Can we do better? Ideally, the process should: - Require fewer training sessions - Retained for longer 12
  • 13. Mnemonics?! “Strategies to enhance the learning and recall of information” Most useful when memorizing lists. Basic idea: Transform abstract, difficult to visualize concepts into tangible, easy to visualize concepts. 13
  • 14. 14 Chain-type Weave list items into narrative chain. an apple being eaten by a beetle while being lifted up by a crane. Memorize: apple, beetle, crane Peg-type Peg list items onto cueing structure. a nun throwing an apple a shoe squashing a beetle a tree being lifted by a crane one is nun two is shoe three is tree
  • 15. 15 Can we use mnemonics to make learning strong secrets quicker and longer-lasting? Research Question:
  • 16. Our contribution 1. Two computer-assisted mnemonic training schemes designed to incrementally teach people strong secrets. 2. Experimental evaluation of these training schemes relative to the rote baseline akin to prior work. 16
  • 18. Method Overview Recruit participants, via MTurk, for a recurring “attention” test over 10 days. Assign participants, randomly, to a treatment or control group. Teach participants a randomly selected strong secret with an assigned training regimen. Compare, across training regimens, how fast participants learn their secrets and how well they remember their secret words 2-weeks after the fact. 18
  • 19. Recruitment Participants solicited via MTurk to take “attention test”. “Attention test” cover story used to distract participants from focusing on learning their secret words. Participants had to return to do 45 attention tests over maximum of 10 days with at least 1 hour between each. 19
  • 20. Assignment Participants were randomly assigned to one of five experimental groups: Story: Mnemonic treatment. Peg-Word: Mnemonic treatment. Feedback: Active learning treatment. Rote: Control. Baseline from prior work. Dropout: Control. No secret words. 20
  • 21. Assignment Participants were randomly assigned to one of five experimental groups: Story: Mnemonic treatment. Peg-Word: Mnemonic treatment. Rote: Control. Baseline from prior work. Feedback: Active learning treatment. Dropout: Control. No secret words. 21
  • 22. Teach secret Secret Encoding: 6 nouns randomly selected out of a broader set of 676 options (56.4 bits). Assigned in two, 3-word chunks. Presented one chunk at a time — only shown second chunk after “learning first” (entering 3x consecutively from memory) 22
  • 23. Teach secret On each login, participants had to enter their secret words. If they couldn’t remember, they would eventually be shown their words to copy. 23 apple beetle crane apple beetle crane
  • 24. Comparison Primarily interested in two comparisons of mnemonic vs rote baseline: - Number of learning sessions required: Lower is better (previous study rote method: median of 36) - 2-week recall rate: Higher is better (previous study rote method: 62%) 24
  • 26. 3 Groups Story Mnemonic: Users wrote two sentences, each containing three of their secret words, in order. Peg-Word Mnemonic: Users were given public “peg” words and created sentences linking each peg to a secret. Rote Baseline: No special training. Users just had to manually enter words to the best of their memory. Dimensions of Variance: mnemonic creation, chunk rehearsal, and hint progression. 26
  • 27. Rote Baseline Mnemonic Creation: none Chunk Rehearsal: Text-boxes with hints ultimately rendered on top for copying. Hint Progression: Started at 0 seconds, progresses up to a maximum of 10 seconds with a 0.5 second increase every subsequent attempt. 27 apple beetle crane apple beetle crane
  • 28. Story Mnemonic Mnemonic Creation: Asked participants to write a visual, memorable sentence stringing together all three words. Example: cat, leaf, wind => A pink cat is chasing a giant leaf blowing in the wind. Participants had to comply to proceed — our app checked whether the story sentence was valid. 28
  • 29. Story Mnemonic Chunk Rehearsal: Three-phases of chunk rehearsal to gradually wean participants off mnemonic assistance. Introduced the hint well to facilitate this gradual transition. The hint well allowed participants to enter their secret words inline. 29 A pink _ _ _ _ is chasing a giant _ _ _ _ blowing in the _ _ _ _. Phase 1: Full assistance cat cat cat leaf wind cat leaf wind
  • 30. Story Mnemonic 30 A pink _ _ _ _ is chasing a giant _ _ _ _ blowing in the _ _ _ _. Phase 2: Reduced assistance cat cat Phase 3: No assistance cat
  • 31. Story Mnemonic Phase Transition: Start in full-assistance, then reduced- assistance, then no-assistance. Transition after first entering from memory in previous phase. Hint Progression: Start at 1 second, max of 10 seconds. Increment by 0.5 seconds on each attempt. 31
  • 32. Peg-Word Mnemonic Mnemonic Creation: First, asked participants to select rhyme-based peg words. Then, asked participants to write three visual, memorable sentences stringing together public peg words with secret words. Example: cat, leaf, wind 1 is {nun, bun, gun, sun} 2 is {shoe, shrew, zoo, screw} 3 is {tree, bee, key, sea} 32
  • 33. Peg-Word Mnemonic 33 Public peg word Secret word Mnemonic hint sentence nun cat The nun pulls the screeching cat off of her head. leaf shoe The leaf lands gracefully in the worn old shoe. wind tree The wind has no effect on the steadfast tree. Participants had to comply to proceed — our app checked whether the peg sentences were valid.
  • 34. Peg-Word Mnemonic Chunk Rehearsal: Same three-phases of chunk rehearsal to gradually wean participants off mnemonic assistance. Difference: hint sentences were shown per word instead of per chunk. 34 One is nun. The nun pulls the screeching _ _ _ _ off of her head. cat leaf wind cat But hints still operated per-chunk, not per-word.
  • 35. Peg-Word Mnemonic Phase Transition: Same as story. Hint Progression: Same as story. 35
  • 36. Results Learning Speed & Memorability 36
  • 37. Some Descriptive Stats 37 351 participants signed up. 242 participants finished. 31 years old on average (sd 10). 52 percent female. Rote Story Peg Assigned 71 81 75 Completed 51 57 48 2-week return 42 48 43
  • 38. Learning sessions Participants who entered their secret words, from memory, three consecutive times had learned their secret words. The number of sessions prior to that 3-chain were “learning sessions”. Q1: How did the number of learning sessions differ between those in our mnemonic treatments versus those in the rote baseline? 38
  • 39. Learning Sessions 39 Both story (7.5) and peg-word (9) participants required significantly fewer learning sessions than rote (12).
  • 40. 2-Week Recall Rate Q2: How did the 2-week recall rate vary between participants in the mnemonic treatments versus the rote baseline? 40
  • 41. 2-Week Recall Rate 41 Name Entropy Example Perfect 56.4 apple beetle crane dog ear flame Single Swap 53.8 apple crane beetle dog ear flame Forgot One 49.6 apple ? crane dog ear flame Relaxed Order 46.9 ear apple dog beetle flame crane
  • 42. 2-Week Recall Rate 42 Story Peg- Word Rote Perfect 75% 42% 60% Single Swap 84% 47% 65% Forgot One 84% 67% 74% Relaxed Order 86% 47% 65% Takeaways Story did significantly better than Rote. Peg-Word, surprisingly, did worse (but non-significant). p<0.05 p<0.10
  • 43. Results Summary Mnemonics require fewer learning sessions and can provide recall benefits, but only if not too complicated. Story mnemonic did best, providing significant improvements over rote on both reducing required learning sessions and 2-week recall. We can perhaps improve retention further if we also facilitate remembering the mnemonics themselves. 43
  • 45. Some use-cases warrant the use of provably strong secrets. Bonneau and Schechter (2014): Lay people can reliably learn such secrets through conditioning, but it takes a while and recall falls sharply after extended disuse. We designed and implemented two computer-assisted mnemonic training regimens to speed-up learning and increase recall rates after extended disuse. Our story mnemonic did both of those things, but our peg-word mnemonic was surprisingly ineffective. 45
  • 46. Testing Computer-Aided Mnemonics and Feedback for Fast Memorization of High-Value Secrets 46 Sauvik Das Carnegie Mellon University Jason Hong Carnegie Mellon University Stuart Schechter Microsoft Research Email contact: sauvik@cmu.edu Key-takeaway: Our computer-assisted story mnemonic decreased the required number of sessions required to learn and increased the two-week recall rates of strong, randomly assigned secrets.