The document discusses mastery training in diagnostic radiology. It outlines the current situation for radiology residents, which involves daily case reporting, tutorials, lectures and study time. It also examines what occurs from a learning perspective as residents encounter new cases. The document explores how expertise develops through cumulative case experience of increasing complexity. It proposes indicators of mastery such as knowledge base, experience, recognition of unusual presentations and assessment of accuracy, completeness, speed and search strategy.

1. 6/12/12
Introduc3on
• What
do
we
do
as
diagnos3c
radiologists?
– One
of
our
key
tasks
is
to
provide
a
diagnosis,
and
jus3fy
it
àRadiology
prac3ce
–
See/detect,
recognize,
interpret
(Krupinski,
2010;
Modine
&
Mello-­‐Thomas,
2010)
Mastery
Training
in
Diagnos3c
• Why
should
we
be
interested
in
this
topic
(of
mastery
Radiology
training)?
– Competency
and
proficiency
is
not
enough,
we
should
Dr
Goh
Poh
Sun
challenge
ourselves
to
“master”
a
field
MB,BS(Melb),
FRCR,
FAMS,
MHPE(Maastricht)
Senior
Consultant
Department
of
Diagnos3c
Radiology
• How
can
we
examine
this
topic,
from
the
perspec3ve
of
a
Na3onal
University
Hospital
radiology
resident,
and
radiology
teacher?
Na3onal
University
Health
System
Singapore
– What
can
a
resident
do
on
a
day
to
day
basis?
– What
are
some
strategies
a
radiology
teacher
can
use?
– How
can
we
use
educa3onal
theory
to
improve
radiology
training
What
is
the
current
situa3on
for
a
What
occurs
from
a
learning
radiology
resident?
perspec3ve?
• Daily
case
repor3ng
with
feedback
• Encounter
new
diagnos3c
category
–
“I
have
never
– Subspecialty
based
seen
this
before”
• Encounter
more
examples
of
previously
encountered
• Several
tutorials
per
week
categories
–
“I
didn’t
know
that
it
could
look
like
this”
• Weekly
topic
based
lectures
(and
annual
courses/conf)
• Encounter
look
alike
and
confounding
cases
–
“these
2
• Weekly
study
3me
cases
really
look
similar
except
for
this
key
finding”
• Resident
topic
based
presenta3ons
and
journal
• Understand
“why”?
–
explain
the
radiological
reviews
(including
conferences
and
symposia)
appearance
• Systema3c
reading
and
case
review
to
prepare
for
• Progressively
develop
the
confidence
to
state
the
likely
fellowship
examina3ons
diagnosis
with
convic3on,
and
provide
arguments
for
and
against
a
par3cular
diagnosis
1

2. 6/12/12
How
is
exper3se
and
diagnos3c
mastery
in
radiology
developed?
• 1.
exper3se
in
radiology
is
directly
correlated
with
experience
• 2.
case
experience
should
be
cumula3ve
• 3.
case
experience
should
include
full
spectrum
of
the
presenta3on
of
any
given
diagnosis
• 4.
case
experience
should
include
material
of
increasing
complexity
• 5.
increasing
complexity
involves
cases
showing
typical,
then
less
typical,
less
easily
detectable
features,
with
less
associated
features,
or
more
confounding
associated
features,
or
more
than
one
pathology,
and
progressively
less
or
contradictory
addi3onal
Lets
dis3ll
some
ideas
from
the
informa3on
educa3onal
research
literature
•
•
6.
radiology
task
is
simple
three
step
process
-­‐
see,
recognize,
diagnose
7.
this
requires
perceptual
learning
or
pagern
recogni3on,
categorical
learning
or
the
ability
to
list
key
features,
and
diagnos3c
reasoning
or
the
ability
to
correlate
with
the
clinical
sehng
8.
experts
use
rapid
wholis3c
percep3on
linked
to
experience,
which
are
mental
schemas
à
à
à
•
or
representa3ons
of
the
key
features
of
a
wide
range
of
pathology
• 9.
experts
use
rapid
200msec
big
picture
percep3on
to
generate
the
most
likely
hypothesis,
which
then
drives
deliberate
search
for
key
features
• 10.
experts
rapidly
decide
on
whether
radiograph
or
scan
is
normal,
or
has
a
likely
abnormality,
and
then
use
visual
search
and
reasoning
to
confirm
diagnosis
• 11.
online
case
repositories
can
accelerate
the
development
of
exper3se,
by
widening
the
case
experience
of
radiologists,
beyond
that
of
individual
day
to
day
direct
experience
• 12.
the
key
is
to
use
case
material
in
a
deliberate
way
Outline
What
are
the
indicators
of
mastery?
• What
are
the
indicators
of
mastery?
• Literature
• How
can
this
be
assessed?
• Survey
and
focus
group
discussion
• How
can
we
train
for
mastery?
– Brainstorm
• What
is
the
role
of
key
images?
– Rank
and
jus3fy
• What
is
the
role
of
case
repositories?
2

3. 6/12/12
What
are
the
indicators
of
mastery?
How
can
this
be
assessed?
• knowledge
base
• Plain
films
• years
of
experience
• Cross
sec3onal
imaging
• ability
to
recognize
unusual
presenta3ons
– How
many
are
required?
• understanding
underlying
principles
– What
spectrum/complexity
of
cases
is
required?
• accuracy
(of
descrip3on)
• completeness
(of
descrip3on)
• Assessment
• speed
– Accurate/complete
selec3on
of
key
images/key
features
• efficiency
of
search
strategy
– Accurate
descrip3on
of
key
features
• effec3veness
• confidence
– Speed
and
confidence
in
screening
all
images
and
iden3fying
key
images/screening
all
areas
of
plain
film
• low
error
rates
and
iden3fying
key
features
Examples
• Lets
look
at
plain
radiographs
à
CXR
• and
• Cross
sec3onal
imaging
à
CT
scans
of
the
brain
3

4. 6/12/12
4

5. 6/12/12
How
can
we
train
for
mastery?
• Role
of
deliberate
prac3ce
(systema3c
case
review,
with
expert
guidance
and
feedback)
• Role
of
background
reading
/
reflec3on
on
day
to
day
clinical
experience
(“ad
hoc”
case
review)
à
typical
case,
varia3ons,
look
alike
and
differen3al
diagnoses
What
is
deliberate
prac3ce?
What
is
the
role
of
key
images?
• Deliberate
prac,ce
refers
to
a
consciously
planned
and
• Key
images
exemplify
most
important
findings
executed
program
of
repeated
focused
exercise
of
the
skills
required
for
expert
performance
accompanied
by
• Can
be
used
to
learn
key
features
appropriate
feedback
(Anders
Ericsson,
Krampe,
&
Tesch-­‐Romer,
1993).
• Repeated
use
can
help
accelerate
mastery
• Paired
and
mixed
prac,ce
involves
the
use
of
a
series
training
of
similar
and
contras3ng
examples
of
imaging
abnormali3es,
with
an
agempt
to
describe
and
reflect
– hgp://learningneuroradiology.blogspot.com/
on
similari3es,
and
differences
between
the
similar
2012/11/deep-­‐learning.html
and
contras3ng
examples
(Anders-­‐Ericsson,
2004;
Hatala,
Brooks
&
Norman,
2003;
Norman,
2008;
Norman,
Young
&
Brooks,
2007).
5

6. 6/12/12
How
long
does
it
take?
• Literature
– 10
to
20
years
(chess
masters,
concert
level
musicians,
top
athletes
-­‐
Anders
Ericsson,
2004)
– 10,000
hours
to
25,000
hours
(lots
and
lots
of
prac3ce)
– (for
paediatric
ankle
radiographs)
20
cases
per
diagnosis
à
threshold,
maximum
rate
of
improvement
21
to
50
cases,
and
no
upper
limit
(Pusic
et.
al.,
Academic
Medicine,
2011)
What
are
case
repositories?
Literature
Review
What
is
the
role
of
case
repositories?
Efficiency
and
effec3veness
Rumble,
1997
Compare
and
contrast
commonly
use
to
solve
individual
day
to
day
clinical
problems
• Collec3ons
of
key
images,
both
plain
film
and
Berbaum,
Franken
&
Smith,
1985
cross
sec3onal
imaging
(hard
copy
à
digital
Radiology
prac3ce
–
See/detect,
recognize
Paired
prac3ce
promotes
ability
to
recognize
Interpret
key
diagnos3c
features,
and
ability
to
elaborate
form
+/-­‐
hyperlink)
Krupinski,
2010
on
these
features
Modine
&
Mello-­‐Thomas,
2010
Markman
&
Gentner,
1993
• Use
in
the
con3nuum
of
training
Mixed
prac3ce
promotes
schema
acquisi3on
– Undergraduate
à
exemplar
case
Compare
and
Contrast
Prac3ce
and
analogous
transfer
Hatala,
Brooks
&
Norman,
2003
Glick
&
Pagerson,
1992
– Residency
and
fellowship
à
mastery
training
Norman,
Young
&
Brooks,
2007
Norman,
2008
Combina3on
of
paired
and
mixed
prac3ce
– CPD/CME
or
compare
and
contrast,
helps
develop
and
Deliberate
Prac3ce
refine
categorical
knowledge
(Anders
Ericsson,
Krampe
Namy
&
Clepper,
2010
&
Tesch-­‐Romer,
1993)
6

7. 6/12/12
Perceptual
Learning
vs
Cogni3ve
Reasoning
–
Addi3onal
Recommenda3ons
from
Literature
• In
addi3on
to
gaining
exposure
to
the
whole
spectrum
of
the
presenta3on
of
any
given
condi3on,
go
deeper
to
try
to
understand
why
• Underlying
anatomy,
pathology,
clinical
background
• This
aids
percep3on
and
memory
retrieval,
and
also
helps
problem
solving
using
first
principles
• Novices
start
using
linear
problem
solving
/
analy3cal
thinking,
experts
rely
first
on
big
picture
perceptual
reasoning
/
pagern
recogni3on
/
non-­‐analy3cal
thinking,
with
analy3cal
thinking
as
back-­‐up
– Engel,
Medical
Teacher,
2008
How
is
exper3se
and
diagnos3c
mastery
in
radiology
developed?
• 1.
exper3se
in
radiology
is
directly
correlated
with
experience
(RLOs
/
Cases
in)
Digital
Repositories
Support
•
•
2.
case
experience
should
be
cumula3ve
3.
case
experience
should
include
full
spectrum
of
the
presenta3on
of
any
given
diagnosis
the
Con3nuum
of
Medical
Educa3on
• 4.
case
experience
should
include
material
of
increasing
complexity
• 5.
increasing
complexity
involves
cases
showing
typical,
then
less
typical,
less
easily
• Con3nuum
–
Undergraduate,
Postgraduate,
detectable
features,
with
less
associated
features,
or
more
confounding
associated
features,
or
more
than
one
pathology,
and
progressively
less
or
contradictory
addi3onal
Con3nuing
Professional
Development
informa3on
• Undergraduate/Novice
Resident
–
Building
the
•
•
6.
radiology
task
is
simple
three
step
process
-­‐
see,
recognize,
diagnose
7.
this
requires
perceptual
learning
or
pagern
recogni3on,
categorical
learning
or
the
founda3on,
basic
concepts,
fundamental
ability
to
list
key
features,
and
diagnos3c
reasoning
or
the
ability
to
correlate
with
the
clinical
sehng
knowledge
à
Exemplar
Cases
• 8.
experts
use
rapid
wholis3c
percep3on
linked
to
experience,
which
are
mental
schemas
or
representa3ons
of
the
key
features
of
a
wide
range
of
pathology
• Postgraduate
–
Applying
knowledge,
building
• 9.
experts
use
rapid
200msec
big
picture
percep3on
to
generate
the
most
likely
skills,
drills
and
quizzes
à
Broaden
range
hypothesis,
which
then
drives
deliberate
search
for
key
features
• 10.
experts
rapidly
decide
on
whether
radiograph
or
scan
is
normal,
or
has
a
likely
• CPD
–
Keeping
up
to
date,
learning
new
concepts
abnormality,
and
then
use
visual
search
and
reasoning
to
confirm
diagnosis
and
skills
à
Revision,
Deliberate
Prac3ce,
• 11.
online
case
repositories
can
accelerate
the
development
of
exper3se,
by
widening
the
case
experience
of
radiologists,
beyond
that
of
individual
day
to
day
direct
experience
Working
on
Differen3a3on
and
Discrimina3on
• 12.
the
key
is
to
use
case
material
in
a
deliberate
way
7

8. 6/12/12
Typical
textbook
example(s)
à
Aim
to
gain
exposure
to
many,
many
more
(the
whole
spectrum)
How
can
we
use
these
ideas?
• Residents
–
take
advantage
of
ad
hoc
case
encounters
to
undertake
deliberate
prac3ce;
and
also
systema3cally
schedule
sessions
of
deliberate
prac3ce
–
do
not
wait
3ll
revising
for
the
fellowship
• Radiology
teachers
–
systema3cally
collect
as
many
examples
of
each
diagnos3c
category
on
an
ongoing
basis,
build
up
collec3on
of
full
spectrum
of
each
disease,
including
look-­‐alikes
and
differen3al
diagnosis
• For
both
residents
and
teachers
–
consider
reviewing
and
presen3ng
cases
in
groups
of
similar
and
contras3ng
cases,
to
improve
discrimina3on
and
diagnos3c
ability
• Plan
for
lots
and
lots
of
prac3ce
(“deliberate
prac3ce”)
Spaced
(distributed)
vs
Massed
What
do
we
do
with
case
collec3ons?
Prac3ce
Learning
is
more
effec3ve
when
spread
over
several
shorter
sessions,
over
a
long
period
3me,
compared
with
repeated
studying
over
a
short
period
of
3me
–
“spacing
effect”
• From
teacher’s
perspec3ve,
in
what
form
can
Ebbinghaus,
in
Memory:
a
contribu3on
to
experimental
psychology,
1913
they
be
collected?
• From
resident’s
perspec3ve,
how
can
I
use
this
case
material
in
an
efficient
and
effec3ve
way?
Bloom
&
Schuell,
The
Journal
of
Educa3onal
Research,
1981
hgp://www.jstor.org/discover/10.2307/27539823?
uid=3738992&uid=2&uid=4&sid=21101388570483
8

9. 6/12/12
Outline
• Personalised
learning
content
through
RLOs
Personalized
Learning
Content
Through
RLOs
• Illustra3ve
examples
of
crea3ve
use
of
RLOs
• Experience
of
posi3ve
educa3onal
aspects
for
Lets
look
from
teacher’s
perspec3ve,
then
student’s
students
and
tutors
• Problems
using
this
approach
for
students
and
tutors
Personalized
Learning
Through
RLOs
Illustra3ve
Examples
• What
are
RLOs?
• How
can
RLOs
be
used?
– Raw
asset,
e.g.,
an
image
(most
flexible,
reusable)
– Learning
asset,
e.g.,
annotated
image.
– Task
or
exercise,
e.g.,
interac3ve
case,
or
problem
solving
task.
– Learning
design
with
content
(containing
some
or
all
of
classes
1–3).
– Generic
learning
design,
e.g.,
a
generic
problem-­‐based
learning
design.
– Koppi,
A.
J.
&
Hodgson,
L.
(2001)
Universitas
21
Learning
Resource
Catalogue
using
IMS
Meta-­‐
data
and
a
New
Classifica3on
of
Learning
Objects.
EDMEDIA
2001,
World
Conference
on
Educa7onal
Mul7media,
Hypermedia
and
Telecommunica7ons,
Tampere,
Finland,
June
25-­‐30,
998–1001.
9

10. 6/12/12
Examples
of
Posi3ve
Educa3onal
Purpose
Experiences
• Collec3on
of
RLOs
• Case-­‐based
problem
solving
• Sehng
up
Repository
of
Cases
• Increasing
experience
• Use
of
Cases
• Mastery
learning
Raw
Asset
à
Learning
Asset
(annotated
image)
à
Task
or
Exercise
Raw
Asset
à
Learning
Asset
(annotated
image)
à
Task
or
Exercise
10

11. 6/12/12
Building
Up
Online
Repositories
• Over
2
years
(2010
to
2012),
2
online
repositories
set
up
containing
over
1000
case
examples
covering
CT
Lets
look
at
an
example
of
a
case
scans
of
the
brain;
and
over
1000
CXRs
(Chest
Radiographs)
repository
• Radiology
images
are
hyperlinked
in
2
categories
(both
unannotated/not
labeled;
and
annotated
with
diagnosis/searchable
online)
• Material
used
and
reused
in
different
formats/
platorms
–
including
websites,
blogs,
face
to
face
case
based
teaching
and
guided/self
directed
learning;
and
index
linked
to
other
repositories
of
similar
or
complementary
cases
Online
Case
Repository
11

12. 6/12/12
12

13. 6/12/12
Imagine
the
teaching
and
learning
possibili3es
with
1000s
of
cases
in
eBook
format
on
(internet
enabled)
tablets
/
smartphones
Imagine
if
the
cases
can
be
selected,
grouped
and
reordered
to
fit
any
learning
or
teaching
purpose
We
can
go
beyond
radiology
images
à
pathology
à
clinical
findings
13

14. 6/12/12
Advantages
of
online
case
repositories
Problems
Using
RLO
Approach
• For
Tutors
• Formalizing
System
and
Procedures
for
Regular
– Online
repository
facilitates
access
Systema3c
Collec3on
of
RLOs
– Unannotated
case
images
give
greatest
flexibility
for
• Quality
control
and
peer
review
repurposing
and
different
educa3onal
usage
– URL
linkage
for
each
case
allows
easy
incorpora3on
of
• Metadata,
tagging,
labeling
and
finding
resources,
and
case
material
on
websites,
blogs
etc.
(this
addresses
the
ensuring
pa3ent
confiden3ality/anonymity
issue
of
copyright
à
use
a
hyperlink,
similar
to
a
cita3on
• Lack
of
awareness
in
academic
culture
–
u3liza3on
and
in
tradi3onal
academic
publica3on)
sharing
• For
Residents
– Online
case
repositories
can
be
an
efficient
method
for
• Reluctance
to
make
unpublished
teaching
material
reviewing
many
cases
publicly
available
– Facilitates
personalized,
customized
self-­‐directed
or
• Lack
of
reward
systems
and
recogni3on
guided
learning
– Koppi,
Bogle
&
Bogle,
2005
14

15. 6/12/12
How
is
exper3se
and
diagnos3c
mastery
in
radiology
developed?
Take
Advantage
of
Regular
Clinical
• 1.
exper3se
in
radiology
is
directly
correlated
with
experience
Ac3vi3es
as
Source
of
RLOs
•
•
2.
case
experience
should
be
cumula3ve
3.
case
experience
should
include
full
spectrum
of
the
presenta3on
of
any
given
diagnosis
• 4.
case
experience
should
include
material
of
increasing
complexity
• Day
to
day
work
à
Systema3c
Case
collec3on
• 5.
increasing
complexity
involves
cases
showing
typical,
then
less
typical,
less
easily
detectable
features,
with
less
associated
features,
or
more
confounding
associated
(annotated
in
PACS,
or
use
of
mobile
phone
features,
or
more
than
one
pathology,
and
progressively
less
or
contradictory
addi3onal
informa3on
camera
and
online
repositories)
• 6.
radiology
task
is
simple
three
step
process
-­‐
see,
recognize,
diagnose
• Weekly
clinical
case
audits
(5%)
à
Good
• 7.
this
requires
perceptual
learning
or
pagern
recogni3on,
categorical
learning
or
the
ability
to
list
key
features,
and
diagnos3c
reasoning
or
the
ability
to
correlate
with
the
examples
of
normal
and
discrepancies
used
clinical
sehng
for
teaching
and
self
directed
learning
• 8.
experts
use
rapid
wholis3c
percep3on
linked
to
experience,
which
are
mental
schemas
or
representa3ons
of
the
key
features
of
a
wide
range
of
pathology
9.
experts
use
rapid
200msec
big
picture
percep3on
to
generate
the
most
likely
• Weekly
clinical-­‐radiology-­‐pathology
rounds
à
•
hypothesis,
which
then
drives
deliberate
search
for
key
features
case
material
reused
for
residency
self
• 10.
experts
rapidly
decide
on
whether
radiograph
or
scan
is
normal,
or
has
a
likely
abnormality,
and
then
use
visual
search
and
reasoning
to
confirm
diagnosis
directed
learning
or
assessment
• 11.
online
case
repositories
can
accelerate
the
development
of
exper3se,
by
widening
the
•
case
experience
of
radiologists,
beyond
that
of
individual
day
to
day
direct
experience
12.
the
key
is
to
use
case
material
in
a
deliberate
way
Take
Home
Points
Future
role
of
educator
(s)
• Mastery
is
the
ability
to
recognize
all
possible
presenta3ons
of
a
par3cular
condi3on,
and
differen3ate
this
from
look-­‐alike’s
and
differen3al
diagnoses
–
with
speed,
accuracy,
confidence
• Knowledge
transmission
to
knowledge
filterer,
• Start
with
common
clinical
condi3ons/presenta3ons
editor,
facilitator
• Gain
exposure
to
all
possible
presenta3ons,
and
go
deeper
to
understand
the
underlying
anatomical/pathological/clinical
• Coach,
resource
person,
systema3cally
collect
background
• This
way
will
recognize
atypical
condi3ons,
or
a
new
condi3on
(full
range)
and
link
resources
/
form
interest
• Systema3c
deliberate
prac3ce
(with
feedback)
is
the
key
à
groups
training
effect
/
lots
of
prac3ce
• Aim
for
at
least
40
to
50
examples
of
each
diagnosis
• Ini3ate
discussion,
ask
ques3ons,
give
advice,
• Plan
for
spaced
(distributed)
rather
than
massed
prac3ce
–
provide
informa3on,
suggest
resources
allocate
3me
for
systema3c
prac3ce
• Use
image
repositories
intelligently
–
digital,
hardcopy,
textbooks/journals
15

16. 6/12/12
Typical
textbook
example(s)
à
Aim
to
gain
exposure
to
many,
many
more
(the
whole
spectrum)
Thank
you
Q
and
A
16