SLN SOLsummit 2010
http://slnsolsummit2010.edublogs.org
February 25, 2010
Phil Ice, Director of Course Design, Research & Development, American Public University System
Using the Community of Inquiry Framework to Assess the Impact of Instructional Design Strategies and New Technologies in Online Courses
This presentation will examine how the efficacy of instructional design components and new online learning technologies can be assessed with indicators of the Community of Inquiry Framework (CoI). The CoI framework has attracted considerable interest and has been used extensively to study and design online educational environments (Garrison & Arbough, 2007). The CoI explains the online learning experience as a function of three overlapping presences – social, cognitive, and teaching. The construct was validated through factor analysis by a multi-institutional team of researchers in 2007 (Swan, Richardson, Ice, Garrison, Cleaveland-Innes & Arbough, 2008), however, many questions remain as to what factors influence the effective projection of each presence. As the model is based on constructivist learning theory, the impact of well designed instruction and pedagogically based application of new technologies should impact the level and quality of interactions probed by the CoI indicators. This session will examine how quantitative and qualitative analysis of course outcomes, using the CoI survey instrument and associated rubrics can be applied to continuous quality improvement from an instructional design perspective. Participants will be provided with instruments, analysis techniques and ideas or application in their own practice.
Ähnlich wie Phil Ice's: Using the Community of Inquiry Framework to Assess the Impact of Instructional Design Strategies and New Technologies in Online Courses
Ähnlich wie Phil Ice's: Using the Community of Inquiry Framework to Assess the Impact of Instructional Design Strategies and New Technologies in Online Courses (20)
Phil Ice's: Using the Community of Inquiry Framework to Assess the Impact of Instructional Design Strategies and New Technologies in Online Courses
1. Using the CoI
to Assess
ID Strategies
and
New
Technologies
in Online
Courses
Phil Ice, Ed.D.
SLN SOL Summit
Syracuse, 2010
2. Community of Inquiry Framework
a process model of learning in online and
blended educational environments
grounded in a collaborative constructivist
view of higher education
assumes effective online learning requires
the development of a community of learners
that supports meaningful inquiry and deep
learning
4. Social Presence
the ability of participants in a community of
inquiry to project themselves socially and
emotionally -- as ‘real’ people
the degree to which participants in computer
mediated communication feel socially and
emotionally connected
5. Social Presence - Elements
affective expression (expressing emotion,
self-projection)
open communication (learning climate, risk
free expression)
group cohesion (group identity, collaboration)
6. Cognitive Presence
the extent to which learners are able to
construct and confirm meaning through
sustained reflection and discourse in a critical
community of inquiry
7. Cognitive Presence - Elements
triggering event (sense of puzzlement)
exploration (sharing information & ideas)
integration (connecting ideas)
resolution (synthesizing & applying new
ideas)
8. Teaching Presence
the design, facilitation and direction of
cognitive and social processes for the purpose
of realizing personally meaningful and
educationally worthwhile learning outcomes
9. Teaching Presence - Elements
design and organization (setting curriculum &
activities)
facilitation (shaping constructive discourse)
direct instruction (focusing & resolving
issues)
10. Community of Inquiry Survey
9 social presence items (3 affective
expression, 3 open communication, 3 group
cohesion)
12 cognitive presence items (3 triggering, 3
exploration, 3 integration, 3 resolution)
13 teaching presence items (4 design &
facilitation, 6 facilitation of discourse, 3
direct instruction)
11. CoI Survey Validation
tested in graduate courses at four institutions
in the US and Canada
principal component factor analysis
three factor model predicted by CoI
framework confirmed
Arbaugh, Cleveland-Innes, Diaz, Garrison,
Ice, Richardson, Shea & Swan - 2008
12. Community of Inquiry Survey Instrument (draft v15)
Developed by Ben Arbaugh, Marti Cleveland-Innes, Sebastian Diaz, Randy Garrison, Phil
Ice, Jennifer Richardson, Peter Shea & Karen Swan
Teaching Presence
Design & Organization
1. The instructor clearly communicated important course topi
2. The instructor clearly communicated important course goals.
3. The instructor provided clear instructions on how to participate in course learning activities.
4. The instructor clearly communicated important due dates/time frames for learning activities.
Facilitation of Discourse
5. The instructor was helpful in identifying areas of agreement and disagreement on course topics that helped me to
learn.
6. The instructor was helpful in guiding the class towards understanding course topics in a way that helped me
clarify my thinking.
7. The instructor helped to keep course participants engaged and participating in productive dialogue.
8. The instructor helped keep the course participants on task in a way that helped me to learn.
9. The instructor encouraged course participants to explore new concepts in this course.
10. Instructor actions reinforced the development of a sense of community among course participants.
Direct Instruction
11. The instructor helped to focus discussion on relevant issues in a way that helped me to learn.
12. The instructor provided feedback that helped me understand my strengths and weaknesses.
13. The instructor provided feedback in a timely fashion.
13. Social Presence
Affective Expression
14. Getting to know other course participants gave me a sense of belonging in the course.
15. I was able to form distinct impressions of some course participants.
16. Online or web-based communication is an excellent medium for social interaction.
Open communication
17. I felt comfortable conversing through the online medium.
18. I felt comfortable participating in the course discussions.
19. I felt comfortable interacting with other course participants.
Group cohesion
20. I felt comfortable disagreeing with other course participants while still maintaining a sense of trust.
21. I felt that my point of view was acknowledged by other course participants.
22. Online discussions help me to develop a sense of collaboration.
14. Cognitive Presence
Triggering Even
23. Problems posed increased my interest in course issues.
24. Course activities piqued my curiosity.
25. I felt motivated to explore content related questions.
Exploration
26. I utilized a variety of information sources to explore problems posed in this course.
27. Brainstorming and finding relevant information helped me resolve content related questions.
28. Discussing course content with my classmates was valuable in helping me appreciate different perspectives.
Integration
29. Combining new information helped me answer questions raised in course activities.
30. Learning activities helped me construct explanations/solutions.
31. Reflection on course content and discussions helped me understand fundamental concepts in this class.
Resolution
32. I can describe ways to test and apply the knowledge created in this course.
33. I have developed solutions to course problems that can be applied in practice.
34. I can apply the knowledge created in this course to my work or other non-class related activities.
16. Flavor’s of the Day
Cloud computing and virtualized applications
have tremendous promise
However – the rise to prominence is so rapid
that adequate evaluation is difficult
Longevity OR extensibility need to be
considered prior to adoption to maximize ROI
17. New Applications are Good
New applications that impact learning are
better
Higher Education often lags out of an
abundance of caution
Academics want to see learning outcomes
before they are willing to adopt
Give your faculty what they want
20. Sometimes a Little is Enough
The potential for overkill exists
Just because extremely rich apps exist
doesn’t mean they are always needed
Remember that distance learners are isolated
from the instructor and classmates physically
This may be by choice
But they still want some contact
21. Audio Feedback
Pilot Study revealed the following benefits of
providing asynchronous audio feedback using
Acrobat Pro:
THEME 1 – Ability to understand nuance.
THEME 2 – Feelings of increased involvement.
THEME 3 – Increased content retention.
THEME 4 – Instructor caring.
22. Audio Feedback & the CoI
The following slides compare the findings of
the multi-institutional CoI sample (n = 1085)
that received text-based feedback and
responses from a multi-institutional sample(n
= 1138) that received audio feedback
In the items addressed there was a
significant difference (p > .05) in responses
23. Teaching Presence
The instructor was helpful in identifying
areas of agreement and disagreement on
course topics that helped me to learn.
Summer 2007 / mean = 4.12
Audio group / mean = 4.43
The instructor encouraged course
participants to explore new concepts in this
course.
Summer 2007 / mean = 4.44
Audio group / mean = 4.58
24. Teaching Presence
The instructor provided feedback that helped
me understand my strengths and weaknesses
relative to the course’s goals and objectives.
Summer 2007 / mean = 4.28
Audio group / mean = 4.57
25. Social Presence
Online or web-based communication is an
excellent medium for social interaction.
Summer 2007 / mean = 3.90
Audio group / mean = 4.27
26. Cognitive Presence
I felt motivated to explore content related
questions.
Summer 2007 / mean = 4.31
Audio group / mean = 4.55
Reflection on course content and discussions
helped me understand fundamental concepts
in this class.
Summer 2007 / mean = 4.37
Audio group / mean = 4.49
28. SaaS Word Processors
Two online, graduate level education courses
(combined n=78) students were asked to
complete assignments (mean page count =
9.75 pages), alternating Word and Buzzword
as the creation and submission mechanism.
Buzzword is an online document editor that is
a part of Acrobat.com
29. Document Analysis
average use of non-text based resources (e.g.
hyperlinks, graphics, tables, etc.) was 5.1 for
Word submissions and 14.3 for Buzzword
based submissions.
Cognitive Presence Indicators
Exploration Integration Resolution
Word 10.8 3.3 2.3
Buzzword 11.2 6.2 5.1
30. Interview Data
Buzzword’s ease of use (as compared to
Word) cited as primary reason for inclusion of
more links, graphics, etc.
Majority (n = 12) indicated that it was easier
for them to express complex concepts using
multimedia.
31. Interview Data
The concept of a Buzzword being a personal
reflective space was common among 72% of students
over 34 years old.
As an example these students frequently developed a
document on their own and then shared it with
classmates.
82% of students 34 and younger recognized it as a
collaborative tool and began workflow projects by
sharing a common document.
Qualitative analysis revealed that these younger
students frequently (68%) likened Buzzword to a Wiki
or similar collaborative tool. In contrast, only 27% of
older students made this association.
33. Understanding Why
The CoI survey and rubrics based off of it can tell you
what is happening but not why
Other measures are needed
Grading Rubrics and Student Interviews make great
sources of data
Involve your faculty – this is data they may already
have
34. Rich Internet Application Study
Full Sail University – Web Design and Development
Program
Study conducted in Deployment of Flash Projects
Course
Implement Flash via Multiple Deployment Types
HTML-based Deployments with Flash Content
Full Flash Deployments using FlashPlayer and AIR
Runtime
Significant gains on 4 Cognitive Presence items –
positive for RIA’s over conventional HTML applications
36. Student Interview Data
11 Participants
Level of engagement
Perceived Learning
Level of thought required
Applicability to future coursework and career
37. Student Interview Data
Cursory Data Analysis
All believed Photo Viewer activity to be more
engaging
9 believed they learning more from Photo View
activity
All believed Photo View activity required more
higher order thought
9 believed Photo Viewer activity would be more
relevant to the career ambitions
38. Designing for
• Director of Course Design, Research &
Development
• American Public University System
Meaningful
• pice@apus.edu
Learning
39. Socio-Epistemological Orientations
Social – Group vs. Individual
Epistemological
Objectivist – lower order thought processes
vis-à-vis Bloom’s Taxonomy
Constructivist – higher order thought
processes vis-à-vis Bloom’s Taxonomy
Student satisfaction and perceptions of
Community may be impacted by the instructors
Socio-Epistemological orientation – as projected in
content and interactivity
(Arbaugh & Benbunnan-Fich, 2006)
40. Setting and Coding
American Public University System – fully online
institution
Eight undergraduate and eight graduate level
courses were coded for epistemological
orientations
Lower three levels of Bloom’s coded as
objectivist
Higher three levels of Bloom’s coded as
constructivist
Coding of all course activities and discussions
Majority of indicators determined
classification
41. Sample and Data Collection
CoI Survey administered for six course terms in
all sections of courses that were coded
N = 4397
Undergraduate – 2576
Graduate – 1821
Factor Analysis ran:
Overall
By level
By course
By five year age bands
By clusters – defined by school
42. Research Question
Does epistemological orientation influence factor
loading patterns?
Are other variables responsible for factor loading
patterns?
Impetus – despite validation of the CoI in 2008, a
few subsequent factor analyses have produced a
two factor solution
Anecdotal evidence – two factor solution
appeared among groups where the emphasis
was on training as opposed to true knowledge
acquistion
43. Findings I
Factor analysis of all courses combined produced
a three factor solution
Factor analysis of all undergraduate courses
combined produced a three factor solution
Factor analysis of all graduate courses combined
produced a three factor solution
Factor analysis of individual courses (n range of
221 - 405) produced a three factor solution
Factor analysis by school produced three factor
solutions
44. Findings II
Age banding 18 - 22, 23 - 27, 28 - 32, 33 - 37,
38 - 42, 43 - 47, 48 - 52, 53 - 57, 58 – 62
Undergraduate maximum age band = 43 – 47
Graduate minimum age band = 23 – 27
45. Findings III
Factor analysis by age band
18 - 22 produce a 2 factor solution regardless
of epistemological orientation or course level
23 - 37 produce 3 factor solution regardless of
epistemological orientation or course level
38 - 62 overall produce a 3 factor solution
overall
38 - 47 produce a 2 factor solution when the
epistemological orientation is objectivist
38 - 47 produce a 3 factor solution when the
epistemological orientation is constructivist
48 - 62 produce a 2 factor solution regardless
of epistemological orientation or course level
46. Observations
Students between 23 - 37 appear to find ways to
collaborate or view learning as a collaborative
process regardless of level
Students 18 - 22 appear to view teaching and
cognitive presence as the same construct
regardless of course orientation
Students 48 - 62 appear to view teaching and
cognitive presence as the same construct
Students 38 - 47 appear to be influenced by the
epistemological orientation of course materials
and activities
47. Future Research I
How does the perception of learning activities
differ between students 23 – 27 years old and their
peers
Why do students 18 - 22 not transfer native
social networking and collaboration skills to
learning
How can life skills be used to leverage learning
for students 48 - 62 years old
Why is epistemological orientation significant for
students 38 - 47 and not other age groupings
49. Thank You!
• Director of Course Design, Research &
Development
Phil Ice, Ed.D.
Director ofAmerican Public University System
• Course Design, Research & Development
American Public University System
• pice@apus.edu
pice@apus.edu