SLN SOLsummit 2010 http://slnsolsummit2010.edublogs.org February 25, 2010 Peter Shea, Senior SLN Researcher, SUNY Albany New Directions in SLN Research The SLN Research Agenda is in a state of dynamic change. Following on more than a decade of survey research chronicling the rise of online learning in higher education and responding to the expressed interests of the SLN community we are now moving to new research questions and methods. This presentation will provide an overview of lessons learned and present a variety of new research approaches that investigate how people learn online (HPLO). The new HPLO framework seeks to address the changing nature of teaching and learning in technology mediated environments through the application of quantitative content analysis, action research, design experimentation and other quantitative and qualitative methods. This session will conclude with a discussion of needs for local research projects and services that can be provided including research design, data collection, analysis, reporting, publication, and presentation opportunities.

1. New Directions for SLN
Research: 2009-2012
Peter Shea
Senior Researcher for the SUNY Learning Network
Educational Theory and Practice & CCI
University at Albany, SUNY

2.  Where we have been – some lessons learned
 Where we are going (maybe)
 New people, research questions, methods
 Publications/presentations
 Example of current topics, questions
 We need your help!

3. Where we have been with SLN
Research: Early results
 In the late 1990s the questions were more
concerned with “can we really do this?”
 Will students accept online learning?
 Will faculty accept online learning?
 Will it work for some disciplines and not others?
 Will age, gender or other demographics play a big
role?

4. Where we have been with SLN
Research: Early results
 Students loved online learning: reported high
levels of satisfaction and learning “a great deal”
 Happiest students reported highest levels of
interaction with faculty and other students
(quantity and quality)
 Happiest students more concerned with
interaction with instructors than other students.

5. More early lessons learned
 The discipline of study did not play a big role in
satisfaction or reported learning
 Appeared that students in many different
disciplines could be satisfied with online learning
 Gender played a small but significant role with
woman reporting somewhat higher satisfaction,
fewer technical difficulties, more learning…
 Looked like this thing could take off…

6. Background
Growth in complete online courses offered
3000
2500
2000
1500
1000
500
0
95-96 96-97 97-98 98-99 99-00 00-01 01-02 "02-03"

7. Background
Growth in enrollments
50000
40000
30000
20000
10000
0
95-96 96-97 97-98 98-99 99-00 00-01 01-02 "02-03"

8. Background
Growth in complete online degree programs
60
Certificates
50
Associate
40 Baccalaureate
Graduate Degree
30
20
10
0
95-96 96-97 97-98 98-99 99-00 00-01 01-02 "02-03"

12. Learning Effectiveness
 In several surveys we asked students to compare
their level of learning online to a comparable
course they had taken in the classroom
 We found that the majority of student reported that
they learned more online than in the classroom…

13. Quality of Outcomes: Results of Meta-
Analyses of OL vs. CI
Confirmed our early
results…… 13

14. The question is “why”?
 Students report that they learn more online
 Studies of learning outcomes “confirm” that they
do.
 Why is this?

15. Need a theory (or theories)
 In 2002 we began work on Community of Inquiry
Theory
 Initially used Teaching Presence
◦ Instructional Design, Facilitating Discourse, Direct
Instruction
 Students who had faculty who did teaching
presence training rated their courses better on
every component of that construct
 Also reported learning more than students whose
instructors had not taken such training

16. Led to additional research on CoI
 Large scale inter-institutional effort to validate the
CoI model
 Publication of factor analysis of CoI survey
 Development of a model that outlined the
interactions of “presences” with each other

18. Current directions in CoI Research
 Coding entire courses for instances of Teaching
Presence, Social Presence, Cognitive Presence
 Attempting to replicate research on “perceptions”
of cognitive presence at individual course level
and learner level
 Document the process of knowledge construction
 Document the outcomes of collaborative
knowledge construction

19. Some results
 Code “whole course” rather than just the threaded
discussion

21. Instructor TP by In/Out of Threaded Discussion
When we coded other artifacts outside the course, we see that Instructor B exhibited
higher levels of teaching presence than the instructor who participated in the course
discussion.

22. Instructor TP by Course Area
Instructor A utilized private folders as the primary means of communication while
Instructor B utilized email most often.
Further evidence that that the majority of instructor TP will be found outside the
traditionally coded area of threaded discussions.

23. Other “surprises”
 We coded for all the “presences”
 But CoI is a process model, so…
 We also coded for the Structure of Observed
Learning Outcomes (SOLO) taxononmy
 Students didn’t score very highly on either
 Why? Difference instructional goals? Different
instructional/learning metaphors?

24. Issues with the CoI Model
 In coding “whole courses” issues arose
 Students are trying to accomplish goals that don’t
seem to fit in the model
 Coders can’t agree how many of the student
interactions should be coded
 Evidence that there is something going on that the
model doesn’t reflect

25. Necessary Negotiations and Remaining Disagreements in Student
Debates
Course B Con 1 Course B Con 2
Remaining Remaining
Disagreeme Disagreeme
PRE POST nt Change PRE POST nt Change
DE-DE 0 5 +5 DE-DE 2 2
DE-NC 0 0 DE-NC 0 0
NC-DE 9 3 3 -6 NC-DE 1 1 1
FD-FD 0 0 FD-FD 4 11 +7
FD-NC 3 3 3 FD-NC 9 4 4
NC-FD 1 1 1 NC-FD 2 0 -2
DI-DI 1 3 +2 DI-DI 4 5 +1
DI-NC 0 0 DI-NC 1 0 -1
NC-DI 2 1 1 -1 NC-DI 2 0 -2
AS-AS 0 5 +5 AS-AS 0 2 +2
AS-NC 6 0 -6 AS-NC 5 3 3 -2
NC-AS 0 0 NC-AS 1 0 -1
NC-NC 20 21 +1 NC-NC 25 26 +1
42 42 8 56 54 8
Debates IRR – Course B
Cohen’s Kappa Holsti’s CR
Pre Post Pre Post
Con 1 -0.063 0.683 0.538 0.861
Con 2 0.338 0.763 0.700 0.885

31. Need more research…
 To understand circumstances under which online
environments result in better learning outcomes
we need more research
 More and better theoretical frameworks
 More people working on the question
 More settings for research
 More help from others

32.  Getting started
 More experienced

33.  Jason Vickers Lilia Cai-Hurteau
 Sedef Uzuner Jerry Lin
 Suzanne Hayes Prahalad Rangan
 Mary Gozza-Cohen Lenore Horowitz
 Ruchi Mehta Chun-Sau Li
 Anna Nolan Stacey McCall
 Temi Bidjerano Asil Ozdogru
 Anna Valtcheva Alexandra Pickett
 Task has been to participate/contribute to an ongoing
project of research and co-author a paper/presentation

34.  Shea, P. & Bidjerano, T. (in review). Understanding different learning
outcomes in hybrid and online environments: An empirical investigation of
the Community of Inquiry Framework. Journal of Interactive Learning
Environments.
 Shea, P., Hayes, S., Vickers, J., Uzuner, S., Gozza-Cohen, M, Mehta, R. &
Valtcheva, A., Rangan, P. (in review). A quantitative content analysis of two
online course: Processes, outcomes, learning, grades. British Journal of
Educational Technology.
 Shea, P., Hayes, S., Vickers, J., Uzuner, S., Gozza-Cohen, M, Mehta, R. &
Valtcheva, A. (2010). A reexamination of the community of inquiry
framework: Social network and quantitative content analysis. The
Internet and Higher Education, doi:10.1016/j.iheduc.2009.11.002.
 Shea, P. & Bidjerano, T. (2009). Community of inquiry as a theoretical
framework to foster "epistemic engagement" and "cognitive presence" in
online education. Computers and Education, 52 (3), 543 – 553.
 Shea, P. & Bidjerano, T. (2009). Cognitive presence: A cluster analysis.
Journal of Computing in Higher Education, 21(3), 199-217.

35.  Community of Inquiry Model (Garrison et. al.)
 Diffusion of Innovation (Rogers)
 Chronotopes (time/space) (Bahktin)
 Self Efficacy (Bandura) – Self Regulated Learning
(Zimmerman, Pintrich)
 Epistemic Engagement (Lareamendy-Joerns)
 Need others…

36.  How well does CoI framework describe/explain/predict online
learners attitudes and performance in online environments?
 What factors motivate/undermine motivation of faculty to teach
online?
 What motivates contributors to OER initiatives?
 How does LE (f2f, hybrid, online) shape quality of learning?
 Gender and online learning
 Language learning online
 K-12 online learning: stakeholder attitudes
 Interaction types: content interactions
 Self regulated learning and self efficacy
 Others – from you…

37.  Surveys
 SEM
 Data Mining
 QCA
 Text analysis
 Design experiment
 Focus group/interviews

38. A new model for research
 SLN research associates partner with campuses
to investigate questions of mutual interest
 Design a research study collaboratively
 Develop research questions
 Collect data
 Analyze data
 Report results
◦ White papers, conferences, journals

39. Services
Assistance with:
 Research design
 Data collection
 Analysis
 Reporting
 Publication
 Presentation opportunities
 Seeking additional external funding

40. Your thoughts?
 Are there questions you would like to investigate?
 Are there faculty on your campus who might like to
collaborate on online education research?
 What topics?
 What questions?
 Can we expand upon current pilot projects?