The present study is among the first to empirically examine the learning outcomes associated with the Rosetta Stone program as a textbook in a class or instead of altogether. Although initial results of basic proficiency and fluency revealed no significant difference between groups, continued linguistic analysis of individual oral and written data has revealed differences in terms of basic lexical and morphosyntactic knowledge as well as proficiency. This session presents the analysis of individual and group data in order to make the case for why programs such as Rosetta Stone cannot replace language classes.

1. Gillian
Lord
University
of
Florida
glord@ufl.edu
Is
Rose%a
Stone
the
future
of
language
learning?

2. Why
this
study?
• Omnipresent
and
powerful
marke9ng*
makes
Rose<a
Stone
…
• En7cing
to
the
average
language
learner
• Temp7ng
to
administrators
• Appealing
to
educators?
• The
best
known
language
program
on
the
market
*“RoseBa
Stone
…
spent
$98.5
million
on
adver7sing
in
2011,
up
from
$70.5
million
in
2010,
according
to
Kantar
Media”
www.ny7mes.com/2012/06/20/business/media/roseBa-­‐stone-­‐ads-­‐emphasize-­‐fun-­‐not-­‐efficiency.html

3. A(n
important)
side
note…
• The
study
reported
here
was
conducted
with
Rose8a
Stone’s
knowledge
and
technical
support.
• They
were
not
involved
in
the
design,
data
collec>on
or
analysis.
• The
Rose8a
Stone
licenses
were
purchased
at
regular
price.

4. What
do
reviews
of
Rosetta
Stone
say?
• Lafford,
Lafford
&
Sykes
(2010)
• Evaluate
if
programs
provide
the
tools
necessary
for
effec7ve
language
learning,
based
on
features
that
research
has
shown
to
be
important
(interac7on,
relevant
contextualiza7on
of
language,
etc.)
• “…
these
products
do
not
incorporate
a
number
of
the
[necessary]
research-­‐based
insights
(e.g.,
the
need
for
culturally
authen7c,
task-­‐
based
ac7vi7es)
that
informed
SLA
scholars
might
have
given
them.”
• Santos
(2011)
• Lack
of
context
• General
inability
to
respond
to
spontaneous
student
speech
• What
Rose8a
Stone
calls
interac7on
is
“a
rather
poor
and
limited
version
of
what
one
would
encounter
in
a
real-­‐life
conversa7on”
• DeWaard
(2013)
• “Not
a
viable
replacement
of
current
instruc7on
at
the
postsecondary
level”
• Based
on
personal
experience,
professional
reac7ons
What
do
academic
reviews
of
Rosetta
Stone
say?

5. What
do
empirical
studies
show?
• Vesselinov
(2009)
–
commissioned
by
RoseBa
Stone;
RS
users
who
knew
nothing
prior
to
using
the
program
demonstrated
increased
knowledge
of
the
language
afer
a
period
of
use.
• Nielson
(2011)
–
self-­‐study
programs
in
workplace;
some
success
but
remarkable
aBri7on;
lack
of
community
(e.g.,
Rovai,
2002)
• Stevenson
&
Liu
(2010)
–
lack
of
ability
to
engage
learners
in
true
interac7on;
users
do
not
take
advantage
of
Web
2.0
tools
to
network.
• This
study
–
Phase
1
of
analysis
indicated
that
first-­‐
semester
gains
in
some
areas
are
comparable
between
RS
users
and
a
classroom
control…

6. Participants
• Par7cipants
were
University
of
Florida
students
enrolled
in
Beginning
Spanish
1
(avg.
age
=
20)
• L1
English
• No
other
L2
proficiency
(beyond
h.s.
requirement)
• No
prior
Spanish
instruc>on
• Par7cipants
belonged
to
one
of
3
environments:
• Classroom
(C):
N=4
• Rose8a
Stone
(RS):
N=4
• Classroom+Rose8a
Stone
(RS+C):
N=4
Original
popula7on
had
20-­‐25
par7cipants
in
each
of
the
three
groups.

7. Participants
Control
(Classroom)
group
(C)
• In-­‐tact
sec7on
of
Beginning
Spanish
• Followed
regular
syllabus
with
standard
materials
• Carried
out
standard
classroom
assessment
materials
• Met
with
researcher
3x
during
semester

8. Participants
RoseBa
Stone
group
(RS)
• Self-­‐selected
(required
by
IRB)
• Not
required
to
aBend
any
regular
class
• Used
Rose8a
Stone
package
(“Conversa7onal
Spanish”):
• 16-­‐week
course
designed
to
cover
material
comparable
to
a
face-­‐to-­‐face
beginning
class
• 6
units
of
Rose8a
Stone®
Version
4
TOTALe®
Spanish,
each
has
4
lessons
[Level
1,
half
of
Level
2]
• Minimum
of
6
RoseBa
StudioTM
sessions
• Minimum
of
8
hours
in
RoseBa
WorldTM
• Monitoring
of
program
access
and
7me
on
task
• Followed
predetermined
deadlines
in
progressing
through
the
material
• Met
with
researcher
3x
during
semester

9. Participants
Classroom
+
RoseBa
Stone
group
(RS+C)
• In-­‐tact
sec7on
of
Beginning
Spanish
class
• Same
instructor
as
control
group
• Used
Rose8a
Stone
materials
as
their
textbook
(including
all
features
described
for
RS
group)
• Met
with
researcher
3x
during
semester

10. Data
collected
• General
oral
and
wri<en
proficiency
and
skills
• January,
March,
May
• Par9al
CLEP
test
(30
items)
• May
• Versant
Automated
proficiency
test
• May
• Assessment
of
aYtudes
• January,
May
• Discussion
of
experiences
• January,
March,
May

11. Phase
1
Results:
CLEP
test
Average
scores
(converted
to
%)
38.65
39.17
47.50
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
100.00
Classroom
RoseBa
Stone
RS+class
p
=
0.165

12. Phase
1
Results:
Versant
test
Average
scores
(converted
to
%)
27.08
26.25
20.00
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
100.00
Classroom
RoseBa
Stone
RS
+
Class
p
=
0.615

13. PHASE
2
ANALYSIS

14. Final
interview
-­‐
Classroom
INTERVIEWER:
Cuéntame,
¿qué
te
gusta
hacer
en
tu
7empo
libre,
o
los
fines
de
semana?
SL:
Repitas,
please.
INTERVIEWER:
¿Qué
te
gusta
hacer?
SL:
Qué
te
gusta
hacer…
INTERVIEWER:
¿Te
gusta
ir
a
películas?
¿Te
gusta
escuchar
música?
SL:
Uh,
¿fin
de
semana?
INTERVIEWER:
Sí.
SL:
Uh,
sí.
En
fin
de
semana,
yo…
yo
estudio,
uh,
mucho.
INTERVIEWER:
¿Sí?
SL:
Uh,
para
mis
exámenes.
Sí.
Yo
tengo
muchos
examines
en
química
orgánica,
biología,
y
laboratorio.
Uh,
sí.
Mucho,
uh…
no,
muy
ocupado.
So,
no
películas,
no,
uh,
deportes.
INTERVIEWER:
¿Cuál
fue
la
úl7ma
película
que
viste?
SL:
Cuál
te…
INTERVIEWER:
La
úl7ma
vez,
the
last
7me,
que
viste
una
película.
SL:
Phew…
Hmm.
Let’s
see…
dos
menses.
INTERVIEWER:
Meses,
mhm.
SL:
Meses.
Ago.
¿Cómo
se
dice
“ago”?
INTERVIEWER:
Hace.
Hace
dos
meses.
SL:
Hace,
sí.
INTERVIEWER:
Wow.
SL:
Yo
no…
yo
no
veo
muchas
películas
en
Gainesville.
INTERVIEWER:
¿Qué
película
fue
esa,
hace
dos
meses?
¿Cómo
se
llamaba?
SL:
Uh,
el
pelí—la
película…
¿cómo
se
dice
“was”?
INTERVIEWER:
Era,
o
fue.
SL:
Era.
La
película
era…
INTERVIEWER:
¿No
te
acuerdas?
SL:
Yo
no…
sí.

15. Final
interview
–
Rosetta
Stone
INTERVIEWER:
Mhm,
¿y
qué
haces
en
Gainesville?
SH:
Um…
you’re
going
to
have
to
forgive
me,
my
mind’s
like
blown…
Um,
yo
estoy
estudiar.
INTERVIEWER:
¿Tú
estudias?
¿Y
qué
más?
SH:
Yo
trabajo
en
un
restaurante
de
Dragonfly.
INTERVIEWER:
Y,
¿con
mucha
frecuencia,
vas
de
compras?
SH:
Yo
no
entendí,
repe7rlo,
por
favor.
INTERVIEWER:
¿Con
mucha
frecuencia,
vas
de
compras?
“Ir
de
compras”
significa
go
shopping.
SH:
All
right,
say
that
one
more
7me,
please.
INTERVIEWER:
¿Con
mucha
frecuencia,
vas
de
compras?
SH:
Uh,
no,
uh,
no
voy
a…
what
did
you,
how
did
you
say
“to
go
shopping”?
INTERVIEWER:
Ir
de
compras.
SH:
No
voy
de
compras.
INTERVIEWER:
Y,
¿qué
vas
a
hacer
este
verano?
SH:
Este
verano,
yo
voy
a
visitar
Brazil.
INTERVIEWER:
Vas
a
visitar
Brazil,
y
¿vas
a
estudiar
en
Brazil?
SH:
No,
um,
yo
voy
a
trabajar
en
Brazil.
INTERVIEWER:
Y,
em,
¿qué
más
a
hacer
en
Brazil?
¿Vas
a
leer,
vas
a
jugar
deportes?
SH:
What
am
I
going
to
do
in
Brazil?
I
thought
I
just
answered
that.
INTERVIEWER:
¿Solo
trabajar?
SH:
I
don’t
know,
I’m
going
on
a
missions
trip,
I
don’t
know
how
to
express
that
in
Spanish,
but…
INTERVIEWER:
Pues,
buena
suerte,
muchas
gracias.

16. Fluency
analysis
• All
interviews
• 3
groups
x
4
par7cipants
=
12
par7cipants
x
3
interviews
=
36
• Transcribed
and
analyzed
for
fluency
measures
• “Fluency”
• Total
number
of
words
spoken
• Number
of
Spanish
words;
Number
of
English
words
• Number
of
dysfluencies
• Lexical
density
(number
of
unique
Spanish
words)
• Number
of
fillers/non-­‐lexical
items

17. General
observations
GROUP
Total
#
words
#
Spanish
words
#
English
words
#
Fillers
#
Clarifica9on
requests
in
Spanish
#
Clarifica9on
requests
in
English
Repe99ons/false
starts
#Unique
words
Classroom
Average
147.78
109.97
21.67
16.14
2.06
1.03
2.81
50.22
RS+C
Average
90.61
47.53
34.75
8.33
0.11
1.75
2.14
27.22
RS
Average
131.21
88.53
39.91
8.72
0.61
3.24
5.54
45.32

18. Ratio
of
L1/L2
words
0.26
0.83
0.68
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
Ra9o
of
English-­‐to-­‐Spanish
words
used,
by
group
Control
Average
RS
+
class
Average
RoseBa
Stone
Average
Classroom
Average
0
=
no
English
words
produced
1
=
1
English
word
produced
for
every
Spanish
word

19. Assistance
requests
2.06
1.03
0.11
1.75
0.61
3.24
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
#
Clarifica7on
requests
in
Spanish
#
Clarifica7on
requests
in
English
Average
#
of
clarifica9on
/
assistance
requests
by
group
Control
Average
RS
+
class
Average
RoseBa
Stone
Average

20. Assistance
requests
2.06
1.03
0.11
1.75
0.61
3.24
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
#
Clarifica7on
requests
in
Spanish
#
Clarifica7on
requests
in
English
Average
#
of
clarifica9on
/
assistance
requests
by
group
Control
Average
RS
+
class
Average
RoseBa
Stone
Average

21. Other
considerations
(1)
Language
learning
motivation
• All
learners
indicated
that
their
primary
mo7va7on
was
to
fulfill
their
language
requirement
• One
third
(4/12)
reported
that
they
chose
Spanish,
specifically,
as
a
language
that
would
be
useful
in
their
par7cular
careers
• RS
volunteers
par7cipated
out
of
curiosity
or
convenience,
but
shared
same
degree
of
language
learning
mo7va7on.

22. Motivation
–
Classroom
Group
Group
High
school
language
background
Why
Spanish?
Classroom
No
Spanish
French
3
I
am
required
to
take
a
language
for
my
major.
Classroom
No
Spanish
La7n
3
Language
requirement
and
for
myself
since
I
feel
Spanish
is
a
good
asset
as
a
physician.
Classroom
No
Spanish
Fulfill
[college
requirement].
Classroom
No
Spanish
La7n
AP
I
am
going
to
Panama
on
a
service
trip.
I
believe
formal
classes
would
help
me
gain
a
beBer
grasp
of
the
language
than
picking
it
up
on
my
own.

23. Motivation
–
RS+C
Group
Group
High
school
language
background
Why
Spanish?
Rose<a
Stone
+
Class
No
Spanish
I
am
taking
Spanish
because
I
feel
like
it
will
be
beneficial
later
on
in
life.
Rose<a
Stone
+
Class
No
Spanish
As
a
requirement
and
to
benefit
my
future
jobwise.
Rose<a
Stone
+
Class
No
Spanish
French
2
I
need
two
semesters
of
a
foreign
language
to
graduate.
Rose<a
Stone
+
Class
No
Spanish
French
4
College
requirement.

24. Motivation
–
RS
Group
Group
High
school
language
background
Why
Spanish?
Why
volunteer
for
RS?
Rose<a
Stone
No
Spanish
Foreign
language
requirement.
Heard
a
lot
of
good
things
about
RoseBa
Stone
so
decided
to
try
it.
Rose<a
Stone
No
Spanish
La7n
3
CLAS
requirement
Can
beBer
manage
my
7me
and
schedule
and
move
more
at
my
own
pace
without
dealing
with
class.
Rose<a
Stone
No
Spanish
French
2
Required
for
major.
Sounded
beneficial.
Rose<a
Stone
No
Spanish
ASL
3
Spanish
is
useful
in
my
state/needed
FL
requirement.
I
was
going
to
use
my
own
to
supplement
educa7on
anyway.

25. Other
considerations
(2)
Time
on
task
GROUP
Comple9on
Rate
Average
Score
Total
Course
Usage
(hours)
Total
Class
Time
(hours)
Classroom
96.99%
90.77%
70.00
39.00
RS+C
93.67%
98.63%
32.81
37.25
RS
97.67%
95.88%
30.69
NA

26. Other
considerations
(3)
Attitude
survey
• Few
changes
(pre-­‐post)
in
any
group
• Significant
changes
(RS+C,
C)
on
item
#3:
• “I
am
enjoying
my
Spanish-­‐learning
experience
this
semester.”
• Significant
changes
(RS)
on
item
#11:
• “Interac>ng
via
chat
or
telephone
is
comparable
to
interac>ng
face-­‐to-­‐face.”
•
Changes
(RS,
RS+C)
on
item
#19:
• “I
would
prefer
to
learn
a
language
on
my
own
>me
and
at
my
own
pace
than
in
a
group
or
classroom
seGng.
“

27. Conclusions
• Across
environments…
• Comparable
outcomes
on
some
measures
• Different
development
of
conversa7onal
skills,
discourse
strategies
• Skep7cism
towards
program
is
jus7fied
• More
research
(always!)
needed
• Larger,
more
varied
sample
size
• Different
proficiency
levels
• Broader
student
group
• More
sensi7ve/appropriate
tes7ng
measures
• Asess
cultural
awareness
and
competence

28. Thank
you.
glord@u[l.edu
• Special
thanks
to:
• UF
College
of
Liberal
Arts
and
Sciences
• UF
Humani7es
Scholarship
Enhancement
fund
• Carlos
Enrique
Ibarra
(sta7s7cs)
• Caroline
Reist,
Keegan
Storrs,
Diana
Wade
(RA)
• Laura
Bradley
(RoseBa
Stone)
QR
code
here

29. Works
Cited
Bley-­‐Vroman,
R.
(1988).
“The
fundamental
character
of
foreign
language
learning.”
In
W.
Rutherford
&
M.
Sharwood
Smith
(Eds.),
Grammar
and
second
language
teaching
(pp.
19-­‐30).
Rowley,
MA:
Newbury
House.
Bley-­‐Vroman,
R.
(2009).
“The
evolving
context
of
the
Fundamental
Difference
Hypothesis.”
Studies
in
Second
Language
Acquisi>on
31(2),
175-­‐198.
DeWaard,
L.
(2013).
“Is
RoseBa
Stone
a
viable
op7on
for
L2
learning?”
Forthcoming
in
ADFL
Bulle>n.
Godwin-­‐Jones,
R.
(2007).
“Emerging
technologies;
Tools
and
trends
in
self-­‐paced
language
instruc7on.
Language
Learning
and
Technology,”
11(2),
10-­‐17.
Retrieved
26
September
2012
from
hBp://llt.msu.edu/vol11num2/emerging/
Godwin-­‐Jones,
R.
(2009).
“Emerging
technologies:
Speech
tools
and
technologies.
Language
Learning
and
Technology,”
13(3),
4-­‐11.
Retrieved
26
September
2012
from
hBp://llt.msu.edu/vol13num3/emerging.pdf
Krashen,
S.
D.
&
Terrell,
T.
D.
(1983).
The
Natural
Approach:
Language
acquisi>on
in
the
classroom.
Hayward,
CA:
Alemany
Press.
Lafford,
B.,
Lafford,
P.
&
Sykes,
J.
(2007).
“Entre
dicho
y
hecho
…:
An
assessment
of
the
applica7on
of
research
from
second
language
acquisi7on
and
related
fields
to
the
crea7on
of
Spanish
CALL
materials
for
lexical
acquisi7on.”
CALICO
Journal,
24(3),
427-­‐529.
Nielson,
K.
B.
(2011).
“Self-­‐study
with
language
learning
sofware
in
the
workplace.”
Language
Learning
and
Technology,
15(3),
110-­‐129.
Retrieved
26
September
2012
from
hBp://llt.msu.edu/issues/october2011/nielson.pdf
Rovai,
A.
P.
(2002).
“Development
of
an
instrument
to
measure
classroom
community.”
The
Internet
and
Higher
Educa>on,
5,
197-­‐211.
Santos,
V.
(2011).
“Review
of
Rose8a
Stone
Portuguese
(Brazil)
levels
1,
2,
&
3.”CALICO
Journal,
29(1),
177-­‐194.
Stevenson,
M.
P.
&
Liu,
M.
(2010).
“Learning
a
language
with
web
2.0:
Exploring
the
use
of
social
networking
features
of
foreign
language
learning
websites.”
CALICO
Journal,
27(2),
233-­‐259
Vesselinov,
Roumen.
Measuring
the
Effec>veness
of
RoseBa
Stone.
hBp://resources.roseBastone.com/CDN/us/pdfs/Measuring_the_Effec7veness_RS-­‐5.pdf.

31. Rosetta
Stone
interface

32. Rosetta
Stone
interface
(vocabulary)

33. Rosetta
Stone
interface
(grammar)

34. Rosetta
Stone
interface
(pronunciation)

35. Rosetta
Stone
interface
(World
–
“play”)

36. Rosetta
Stone
interface
(World
–
“talk”)

37. Rosetta
Stone
interface
(World
–
“explore”)

38. Rosetta
Stone
interface
(Studio)