This document discusses quality in online education. It begins by defining quality in different contexts such as manufacturing and education. It then discusses the debate around quality in online learning, with some seeing it as inferior to face-to-face ("e-atheists") and others touting its benefits without evidence ("e-zealots"). It also examines definitions of quality in online education and factors like course design, teaching, and learning. Finally, it discusses expectations for completion rates in online courses and potential best practices to improve rates.

1. “Quality”
in Online Education

2. Barry Dahl dot com

4. Quality
Definition?
high grade; superiority; excellence

5. Quality
in Manufacturing
Definition?
What does ISO say? International
Organization for Standardization

6. ISO 8402:1986
• This ISO standard defines quality
as “the totality of features and
characteristics of a product or
service that bears its ability to
satisfy stated or implied needs.”

7. Another Definition
• In manufacturing, a measure of
excellence or a state of being free from
defects, deficiencies, and significant
variations, brought about by the strict
and consistent adherence to measurable
and verifiable standards to achieve
uniformity of output that satisfies
specific customer or user requirements.

8. Quality
in Education
Definition?
High standards, consistently applied,
with efficiency or “value for money”

9. Quality
in Education
Definition?
Or, what D. Green said in 1994
What is quality in higher education?
Buckingham, U.K.: SRHE and The Open University Press.

10. What is quality in higher education?
• …quality is a relative concept, ....different
interest groups or ‘stakeholders’ in higher
education have different priorities and their
focus of attention may be different. The best
that can be achieved is to define as clearly as
possible the criteria that each stakeholder
uses when judging quality, and for these
competing views to be taken into account
when assessments of quality are undertaken
(p. 17).

11. The e-Learning Battle
E-Learning Atheists
E-Learning Zealots
Images purchased & edited by Barry Dahl: rights reserved 11

12. The e-Learning Battle
E-Learning Atheists • These are the
naysayers.
• They don’t believe
that anything good
can come from online
learning.
• In fact, they’re pretty
sure that it’s the
DEVIL.
12

13. Sample Atheist Comment
Online education is fake-education.
It is not "real". Convenient? Yes.
Profitable? Yes. Popular with
students who mostly just want a
"degree"? Yes.
And it is garbage in the end.
Source: http://chronicle.com/article/How-Big-Can-E-Learning-Get-At/128809/
13

14. The e-Learning Battle
• The Zealots are those
who tout the benefits
of online learning
without having any
evidence to back it up.
• Zealots begin many of
their sentences with
“one time there was
this one student …” E-Learning Zealots
14

15. Sample Zealot Comment
And here we are, in 2011, fer cryin‘aloud,
with dodgy naysayers STILL kicking and
screaming and throwing fits in regards
to online teaching and learning. Get over it,
already – the days of having students seated
around your flowing toga in utter awe of your
knowledge and acumen are
LONG GONE. Welcome to the present.
Source: http://chronicle.com/article/How-Big-Can-E-Learning-Get-At/128809/
15

16. e-Atheists Viewpoints
1. Online students are missing the true
college experience.
2. Online students are lonely.
3. Online students can’t possibly learn
as much.
4. Online students are being ripped off.

17. e-Atheists Viewpoints
1. Online students are missing the true
college experience.
2. Online students are lonely.
3. Online students can’t possibly learn
as much.
4. Online students are being ripped off.

18. Quality
in Online Ed
Definition?

19. E-Learning Quality =
Quality Matters™
Right?

20. Through the use of rubrics and standards
related to the quality of online courses
(i.e. Quality Matters™), we are
sufficiently addressing the questions
about e-learning quality
60% 1.Real ity
Reality
40% Myth
2.Myth

21. Quality Matters is Sufficient
• Um, no, it isn’t!!
• Quality Matters
looks at the quality
of course design.
• That’s good, but it’s
only one leg holding
up the stool.

22. Quality Matters is Sufficient
Learning
• Um, no, it isn’t!!
• Quality Matters
looks at the quality
of course design.
Quality
• That’s good, but it’s Concerns
only one leg holding
up the stool. Teaching Design

23. 3 Major Components of e-Quality
Learning Level
Is High
Learning Assessment
Teaching Level Course Design
Is High Meets Standards

24. Possible Reasons?
Learning Level
Is Low
Learning Assessment
Teaching Level Course Design
Is High Meets Standards

25. Is This a Problem?
Learning Level
Is High
Learning Assessment
Teaching Level Course Design
Is High Below Standard

26. Independent Study, Perhaps?
Learning Level
Is High
Learning Assessment
Teaching Level Course Design
Is Low Meets Standards

27. Beautiful, Just Beautiful
Learning Level
Is Low
Learning Assessment
Teaching Level Course Design
Is Low Meets Standards

28. Maybe, but unlikely
Learning Level
Is Low
Learning Assessment
Teaching Level Course Design
Is High Below Standard

29. Triple Ick – Just Start Over?
Learning Level
Is Low
Learning Assessment
Teaching Level Course Design
Is Low Below Standard

30. Quality
We know it when
we see it

31. Expectations about Completion

32. Weekly News Articles

33. CCRC Reports: Washington & Virginia
http://ccrc.tc.columbia.edu/Publication.asp?UID=872

34. CCRC Reports: Washington & Virginia
Washington State community and technical colleges in the fall of 2004.
Students were tracked for nearly five years, until the spring of 2009
“students were more likely to fail or withdraw from
online courses than from face-to-face courses”
“were slightly but significantly less likely to attain an
educational award or transfer to a four-year institution”
http://ccrc.tc.columbia.edu/Publication.asp?UID=872

35. Say What?
• “slightly but significantly less likely”
• Translation … “I’m really, really sure
that there is a very small difference.”

36. What are Reasonable
Expectations for Completion
Rates for Online Learners?
And how do you know?
Is completion a
measure of quality?

37. Can’t Sleep? Read Some Comments
http://chronicle.com/article/Why-Are-So-Many-Students-Still/127584/

38. The Author Asks
• “But isn't it time that we had an honest national
conversation about online learning? With
countless studies showing success rates in
online courses of only 50 per cent—as
opposed to 70-to-75 percent for comparable
face-to-face classes— isn't it time we asked
ourselves some serious questions? Such as:
Should every course be taught online? And
should we allow every student—or any student
who wishes to—to take online courses?”

39. Let’s Find Some Data!

40. U of Central Florida
• Center for Distributed Learning
– On the average, fully online courses have
slightly lower success rates and higher
withdrawal rates than either their face-to-
face or Web-enhanced counterparts
– http://cdl.ucf.edu/research/rite/dl-impact-
evaluation/#Success

41. U. North Texas – Spring 2007
• Definitions make a difference:
• Completion: finished the course –
received an A,B,C,D,F grade (no W,I)
• Success: is defined as earning a grade
of A, B, or C for an undergraduate
course.

42. U. North Texas – Spring 2007
Undergraduates (n=7,730)
Success…
74%
Success… 80%
Completion… 93%
Completion… 95%
0% 20% 40% 60% 80% 100%

43. UIS (Springfield) Online Course Data
Completion %
Summer '07
Compare
Spring '07
To 96%
Fall '06 On Campus
Summer '06
Spring '06
Fall '05
90% 91% 92% 93% 94% 95%
http://online.uis.edu/info/completion.html

44. Bellevue CC Online Course Success
http://www.sbctc.edu/docs/data/stdt_success_retention_
in_online_courses_bcc.pdf

45. What are the
Expectations?

46. Expectations of Success Rates?
On-ground Success
25%
75%
A,B,C D,F,W

47. Expectations of Success Rates?
On-ground Success Online Success
Equal?
25% 25%
75% 75%
A,B,C D,F,W A,B,C D,F,W

48. Expectations of Success Rates?
On-ground Success Online Success
WHY?
25% 25%
75% 75%
A,B,C D,F,W A,B,C D,F,W

49. Why, indeed. For example…
• Do we expect that every math
instructor will have the same
success rates?

50. Why, indeed. For example…
• Do we expect that the
accounting program and the
art program will have the same
success rates?

51. Why, indeed. For example…
• Do we expect that the urban
community college and the
rural community college will
have the same success rates?

52. LSC – 3 yrs – Developmental Courses
Passing Grades: A,B,C
67.8%
68%
66%
60.9%
64%
58.5%
62%
60%
58%
56%
54%
52%
Online Day Night

53. LSC – 3 yrs – Developmental Courses
Passing Grades: A,B,C
67.8%
68%
66%
60.9%
64%
58.5%
62%
60%
58%
56%
54%
52%
Online Day Night

54. Demographics Differences?
Gender On-Campus Gender Online
68%
50%
50%
32%
Male Female Male Female

55. More Differences?
On-Campus Ages Online Ages
49%
35%
65%
51%
24 & ↓ 25 & ↑ 24 & ↓ 25 & ↑

56. First-time Learners?
• Each term, how
many students
are sitting in a F2F
(traditional)
classroom for the
very first time?
• Est. < 1%

57. First-time Learners?
• Each term, how • Each term, how
many students many online
are sitting in a F2F students are
(traditional) “sitting” in an
classroom for the online classroom
very first time? for the very first
time?
• Est. < .1%
• Est. >= 40%

58. Instructors?
• Do we expect that the new
online instructors will have the
same success rates as
experienced online
instructors?

59. Typical “Learning Curve”
90%
Success Rates – New Online Instructors
80%
Instructor 1
Intsructor 2
70% Instructor 3
60%
Term 1 Term 2 Term 3 Term 4 Term 5

60. Fair Comparison?
On-Ground Online
Success
Rates
= Success
Rates

61. Fair Comparison?
=

62. Fair Comparison?
Course
Withdrawal = Academic
Non-success

63. A Perspective from David Diaz
• “However, I believe that many online
students who drop a class may do so
because it is the right thing to do. In other
words, because of the requirements of
school, work, and/or family life in
general, students can benefit more from a
class if they take it when they have enough
time to apply themselves to the class work.”
http://technologysource.org/article/online_drop_rates_revisited

64. A Perspective from David Diaz
• “Thus, by dropping the class, they may be
making a mature, well-informed decision
that is consistent with a learner with
significant academic and life experience.
This explanation would be consistent with
their demographics while calling into
question the idea that these students are
academically unsuccessful or possess
inferior academic abilities.”
http://technologysource.org/article/online_drop_rates_revisited

65. What to do
About It?

66. Potential Good Practices
• To help improve completion/success
rates, consider the following:
– Required new online student orientation
– Additional support mechanisms beyond the
instructor
– Early intervention when they start falling
behind
– Formalizing your expectations for online
learners and online faculty

67. More Potential Good Practices
• To help improve completion/success
rates, consider the following:
– Require interaction - no electronic
correspondence courses
– Mid-term course evaluations
– Shorter class lengths (8 wks or less)
– Provide better “consumer” information pre-
enrollment
– Professional Development Opportunities

68. One Example
Shorter Course Lengths

69. Why Don’t Adult Students Persist?
• From the Maricopa CC System:
– 10% due to academic reasons such as
course too hard or too heavy a load.
– 65% due to life interruptions (includes the
normal things of family or job
issues, illness, etc. etc.)
Source: Ron Bleed, CIO of Maricopa CC
(retired)

70. Example of Life Interruption
PsychologyPsychology Psychology
12 Weeks
Invested,
Humanities
Humanities Humanities 12 Credits
Paid For,
No Credits
Earned,
Biology Biology Biology With Little
Incentive
To Return
History History History
4 weeks 8 weeks 12 weeks

71. Alternate Outcome
Psychology 12 Weeks
Invested,
12 Credits
Paid For,
Humanities 9 Credits
Earned,
With More
Incentive
Biology To Return
History - No
4 weeks 8 weeks 12 weeks

72. Alternate Outcome #2
Psychology 12 Weeks
Invested,
12 Credits
Paid For,
Humanities 6 Credits
Earned,
With Some
Incentive
Biology - No To Return
History - No
4 weeks 8 weeks 12 weeks

73. LSC – FY2010 – All Online Courses
26.2% 3.00 2.93
30%
20.7% 2.90
25%
20% 2.80 2.75
15% 2.70
10% 2.60
5% 2.50
0%
D,F,W GPA
8-week 16-week 8-week 16-week

74. The Role of Course Design
in Online Quality
It’s the start, not the ending

75. Course Design Rubrics Can Help
• But they only go so far.
• Did the interactions actually occur?
• Did the feedback actually happen in
a timely manner?
• The design rubric is only part of the
answer.

76. Course Design Rubrics Can Help
V. LEARNER INTERACTION
General Review Standard: The effective design of instructor-learner interaction and
meaningful learner cooperation is essential to learner motivation, intellectual commitment,
and personal development. (From LSC Course Design Rubric)
Specific Review Standards Points
V.1 The course design provides learning activities to
foster instructor-student, content-student, and if 3
appropriate, student-student interaction.
V.2 The student requirements for course interaction
are clearly articulated. 3
V.3 Clear standards are set for instructor response
and availability (turn-around time for email, grades 2
posted, etc.)

77. Do You Evaluate Teaching Quality?
• If not, why not?
• If so, what do you base the
evaluation on?
• How clear are the expectations of
the factors upon which they will be
evaluated?

78. Expectations for (of) Faculty
Links to these pages at:
http://xlents.com

79. St. Petersburg College
• Examples of posted expectations for faculty:
• Respond to email within 24 hours, 5 of the 7
days a week at the instructor’s discretion.
• Exam marked within 7 days of due date.
• Assignment marked within 7 days of due date
• Routine updates to all students every 7 days.

80. Lawrence Tech University
• Examples of posted expectations for faculty:
• Check the discussion forum daily and be sure
to post responses to student contributions at
least four times weekly.

81. Penn State
• Examples of posted expectations for faculty:
• The instructor is asked to grade and submit
to students all digitally formatted
assignments and exams within two business
days of receipt.

82. Colorado CC Online
• Examples of posted expectations for faculty:
• As mandated by our NCA
accreditation, CCCOnline courses cannot be
self-paced. Therefore, CCCOnline requires
faculty to create a Course Schedule that
provides unit, week, and specific activity
dates.
• Instructor responds individually to all student
introductory posts.

83. Lake Superior
• Examples of posted expectations for faculty:
• Course Outlines: There is only one official course
outline for each course title offered at LSC. This
means the following:
– Delivery method is not generally considered during the
curriculum approval process.
– Each class syllabus should include the official course
outcomes regardless of the delivery method employed.
– Each class syllabus should include the official course
description regardless of the delivery method employed.

84. College expectations of online faculty
• Sample of possible topics: • Example - Final Exams:
– Course design – Online faculty are expected
to give final exams (or due
– Interaction w/students dates for other types of
final projects) during the
– Online office hours final exam period as
– Feedback/response time established and posted by
the college.
– Final exams – The final exam period
– Proctored exams typically consists of four
weekdays and may or may
– Due dates/times not include a weekend.
This schedule is posted on
– Sick/Personal leave the college website and
otherwise makes the
information available to all
faculty.

85. Comprehensive
Faculty Evaluations

86. Evaluation Components
1. Self Evaluation
2. Professional Development Plan
3. Course Evaluations (by students)
4. Course Observation (by admin)

87. Classroom Observation
An Administrator’s
Right?

89. Basic Expectations – 1A

90. Basic Expectations – 1A

91. Basic Expectations – 1A

92. Learning Experiences – 2A

93. Learning Experiences – 2B

94. Activity & Interactivity – 3E

95. Where is the Value in eLearning?
• Why is SJC engaged in eLearning?
• What are the benefits to the school?
• What are the driving factors?

96. To Make a Difference

97. “Quality”
in Online Education
Discussion
Time