Rosario @icalt 2012, Rome 4-6 july

1. Modeling Critical Factors of
Quality in e-Learning
A Structural Equations Model Test
ICALT 2012 –Rome, 4-6 July
Rosário
Cação
Carlos
Lucas
de
Freitas

2. Agenda
1. Introduc+on
2. Quality
in
e-­‐learning
as
a
mul+-­‐dimensional
concept
3. A
three-­‐factor
model
of
quality
in
e-­‐learning
4. Empirical
research
• Purpose
statement
• The
par+cipants
• The
instrument
• The
measurement
model
• The
structural
model:
regression
weights
and
path
es+mates
• Model
fit
5. Conclusions
and
future
work

3. 1.
Introduc:on
• Quality
is
one
of
the
keys
to
business
success
and
compe++ve
advantage.
• There
is
no
unique
or
widely
accepted
understanding
of
what
quality
is,
and
how
to
measure
it
and
there
is
a
lack
of
consensus
about
what
quality
in
e-­‐learning
is.
We have tested, and confirmed, an existing model that
represents the perception of quality in e-learning

4. 2.
Quality
in
e-­‐learning
as
a
mul:-­‐dimensional
concept

5. 3.
A
three-­‐factor
model
of
quality
in
e-­‐learning
Cação,
R.,
&
Figueiredo,
A.
D.
d.
(2010).
Future
u+lity
as
a
key
dimension
in
e-­‐learning
quality.
Interna'onal
Journal
of
Informa'on
and
Opera'ons
Management
Educa'on,
3(4),
322-­‐336.

6. 3.
A
three-­‐factor
model
of
quality
in
e-­‐learning
Why
have
we
used
this
model?
• It
is
consistent
with
Juran's
(1951)
view
of
quality
as
fitness
for
use
• It provides a long-term approach to the concept of quality, as it
includes, not only effective uses or outcomes, but also expected uses
• It places the trainees at the center of the concept of quality and
emphasizes their role in the construction of knowledge and quality
• It is based on the opinion of real customers, not on the opinion of
experts, potential customers, or even on the researchers' opinion
• The final structure of the dimensions of quality was grounded on
statistical evidence and on data reduction techniques

7. 4.
Empirical
research
Purpose
statement
To develop a measurement model and test a structural model
made up of three constructs that affect the trainees'
perception of quality in e-learning: training process, training
attitudes, and training utility
We have used structural equation modeling (SEM) to test the
model with a new sample of data.

8. 4.
Empirical
research
The participants
Customers of a Portuguese provider of 2741 answers
asynchronous e-learning for professional 64% were women
training, with ten years of experience in the
consumer e-learning market and an
accumulated count of over 60.000 clients
from 29 countries.
The company offers around 200 short-term
courses ranging in length between 1 and 9
weeks.

9. 4.
Empirical
research
The
instrument
A 1 to 10 scale, online survey, at the end of the courses, where 10 is the
highest value.
• Global satisfaction
• Fulfillment of expectations
• Initial motivation
• Final motivation
• Fulfillment of training objectives
• The platform and its functions
• Training contents
• The trainer’s expertise
• The contribution of the forum for the learning process
• The dynamics and help of the trainer in the forum
• Competence, kindness, and promptness of the staff
• Immediate professional utility
• Future professional utility
• Global quality perception

10. 4.
Empirical
research
The
measurement
model
The
measurement
model
included
three
constructs
represen+ng
latent
variables:
• Training
process
included
beliefs
the
trainees
have
toward
the
day-­‐to
-­‐day
of
the
course
• Training
a1tudes
represented
reac+ons
and
beliefs
the
trainees
have
towards
the
training
course
• Training
u3lity
was
the
extent
to
which
the
trainees
feel
that
the
course
will
have
impact
on
their
personal
and
professional
life,
considering
both
the
short
and
the
long
term

11. 4.
Empirical
research
The
measurement
model
We
have
defined
a
recursive
model
with
the
following
hypothesized
structural
rela+onships:
H1:
Training
process
is
posi+vely
related
to
the
percep+on
of
quality
in
e-­‐
learning
H2:
Training
a>tudes
are
posi+vely
related
to
the
percep+on
of
quality
in
e-­‐
learning
H3:
Training
u'lity
is
posi+vely
related
to
the
percep+on
of
quality
in
e-­‐
learning
We
have
followed
a
two-­‐step
SEM
process,
i.e.,
we
have
tested
first
the
fit
and
the
construct
validity
of
the
measurement
model
(Hair
et
al,
1992,
pp.
717-­‐718).
The
es+ma+on
technique
used
was
the
scale-­‐free
least
squares
es'mates
because
the
measures
revealed
severe
non-­‐normality.

12. 4.
Empirical
research
The
structural
model
Standardized regression weights
Standardized paths of the
Variable Training Training Training
hypothesized model
Process Attitude Utilities
Hypo Causal Path Standardized
- Path
X6 0.786
thesis Coefficient
X7 0.903
H1 Training process → perception of 0.42
X8 0.853
quality in e-learning
X9 0.725
H2 Training attitudes → perception 0.33
X10 0.844
of quality in e-learning
X11 0.832
H3 Training utility → perception of 0.22
X1 0.928
quality in e-learning
X2 0.902
X3 0.584
X4 0.863
X5 0.899
X12 0.904
X13 0.917

13. 4.
Empirical
research

14. 4.
Empirical
research
Model
fit
CFA
Structural
model
model
Chi-­‐square
(χ2)
Chi-­‐square
29.941
30.716
Degrees
of
freedom
62
72
Absolute
fit
measures
Goodness-­‐of-­‐fit
index
(GFI)
0.996
0.996
Root
mean
square
residual
(RMR)
0.121
0.114
Incremental
fit
indices
Normed
fit
index
(NFI)
0.995
0.996
Rela+ve
fit
index
(RFI)
0.994
0.995
Parsimony
fit
indices
Parsimony
normed
fit
index
(PNFI)
0.791
0.788
Adjusted
goodness-­‐of-­‐fit
index
(AGFI)
0.994
0.995

15. 5.
Conclusions
and
future
work
• Our
research
enabled
us
to
confirm
a
model
of
quality
in
e-­‐learning
composed
by
three
factors.
• According
to
this
model,
the
percep+ons
of
quality
in
e-­‐learning
can
be
explained,
with
comfortable
goodness-­‐of-­‐fit,
by
three
factors:
the
training
process,
the
training
a>tudes,
and
the
training
u'li'es.

16. 5.
Conclusions
and
future
work
The
model
provides
e-­‐learning
companies
with
a
conceptual
framework
to
beeer
understand
what
quality
is.
• It
makes
clear
that
improving
quality
implies
working
on
two
dis+nct
and
addi+onal
areas,
besides
the
training
process;
• it
emphasizes
the
valua+on
of
the
training
outcomes,
as
well
as
the
role
of
the
trainees'
agtudes
in
the
construc+on
of
the
perceived
quality.
The
model
can
also
be
used
to
classify
and
organize
the
mul+ple
dimensions
of
quality
proposed
in
the
literature
and
to
determine
specific
courses
of
ac+on
intended
to
improve
quality
percep+ons
in
a
specific
factor
of
quality.

17. Slides available at
www.slideshare.net/rosariocacao
Rosário
Cação
mrac@dei.uc.pt