Plant propagation: Sexual and Asexual propapagation.pptx
Esai april2015
1. Design method for the evaluation and
quality assurance of an online learning
environment.
Marianne Checkley
ESAI 2015
2. Background and Context
• Personalised online learning programme
for young early school leavers aged 13-16
years.
• 52 students
• Blended centre and at-home
• 10 online teachers
• Maximising multiple data streams in
examining the online educational
experience
• Creating a narrative approach
Presentation
Focus
3. Instruments developed to evaluate, benchmark, and provide
quality-assurance processes for online learning environments
are primarily used at Third Level (Oncu & Cakir, 2011).
Research from iNacol designed to measure quality
in K12 online and blended learning environments divides
strategies into outcomes and processes (Patricks et al, 2012).
Including proficiency, individual student growth, graduation rate,
college and career readiness and closing the achievement gap.
For quality assurance of processes the following headings are
assessed under comprehensive criteria: Content, Instructional
Design, Student Assessment, Technology, Course Evaluation and
Support.
4. Developing an evaluation framework for the educational
experience on iScoil:
• Research suggests incorporating the data streams available from
a virtual learning environment (Kennedy and Soifer, 2013) with a
student centred qualitative approach to participation from all
stakeholders in order to consider outcomes in a contextualized
way (Niehaus, 2014).
• Deepwell (2007) recognises the capacity of Stake’s Countenance
approach to organise large amounts of diverse data and as such
is ideal for the analysis of online learning programmes.
Three interdependent phases Antecedent, Transaction, Outcome.
5. Antecedent
Conditions existing
prior to instruction
that may relate to
outcomes
Transaction
The process of
instruction
Outcomes
The effects of the
programme
What is the profile of
iScoil students?
How do the learners
engage with the online
platform?
What certification is
achieved?
What is the theory
underpinning curriculum
design?
What is the learning
experience?
What are the
progression routes?
What is the teaching
experience?
How does student profile
impact outcomes?
How does student profile
impact engagement?
Are there unintended
outcomes?
How is the curriculum
delivered?
Is there potential for
improvement
Table 1: Research Questions within a countenance framework
6. Figure 1: Community of Inquiry Model (Swan et al, 2009)
EDUCATIONAL
EXPERIENCE
Supporting
Discourse
COGNITIVE PRESENCE
Selecting
Content
TEACHING PRESENCE
Setting
Climate
SOCIAL PRESENCE
Features for Data Collection 1
7. Features for Data Collection 2
Figure 2: Cognitive Apprenticeship Model (Collins, Brown & Newman, 1989)
8. Antecedent
Conditions existing prior
to instruction that may
relate to outcomes
Transaction
The process of
instruction
Outcomes
The effects of the
programme
Student Profile: Gender-
Reason for Referral-
Location of Learning.
Learner engagement.
patterns on VLE
Certification achieved
according to student
profile
Design of Curriculum. The Learning Experience. Progression Routes
according to student
profile.
Learner-Teacher-Content
Interaction.
The Teaching Experience
Learner Content
Interaction.
Table 2: Features for Data Collection within a countenance framework
9. Research Question
• How do the students
engage with the
online platform?
• What is the learning
experience?
• What is the teaching
experience?
• How is the curriculum
delivered?
• How does student
profile impact on
engagement and
learning?
Feature
• Learner engagement
patterns on VLE
• The teaching
experience
• The learning
experience
• Learner/Teacher/Con
tent Interaction
• Learner/Content
Interaction
Method
• VLE Log Analysis
• Focus Group
• Semi-Structured
Interviews
• Authoring Tool Open
GLM
• Outcomes Matrix
How the Framework works out: Transaction Phase
Figure 3: Corresponding Research Questions, Features and Methods across a
countenance approach for the Transaction Phase
11. Reflective use of Authoring Tools
A relevant example is a study where multiple design and authoring tools
explore one lesson (Prieto et al., 2013). The study ultimately highlighted the
complexity involved in the learning design process as different tools and
processes were a better fit to different teachers depending on their
pedagogical aims or institutional and technological contexts.
A number of computational tools that guide the thinking of education
practitioners in pedagogical design have emerged from learning design
research over the last ten years (Masterman, Walker & Bower, 2013).
While these design and authoring tools were developed to facilitate
the development, adaptation and sharing of teachers’ pedagogical
ideas, they are also, according to Laurillard (2012) useful as tools for
reflection on practice.
13. Antecedent
Student
Profile
0 2 4 6 8 10 12 14
Anxiety
Behavioural
School Refusal
Disaffection
Illness
Female
Male
0
5
10
15
Disaffection
Behavioural
School
Refusal Anxiety
Illness
Blended Male Blended Female At-Home Female At-Home Male
Gender and Reason for Referral
Gender, Location and Reason for Referral
14. Antecedent
Curriculum
Design
RWL
4%
Produce
18%
Practice
42%
Inquire
36%
Results from reflective use of the Learning
Designer grounds the design in constructivist
learning theory with an emphasis on an
active model of learning.
However absence of discussion based
activities limit the opportunity for
interaction and social construction of
knowledge.
Figure 5: Results of Learning Designer analysis of Communications Unit 2 Writing
15. Transaction Phase
Feedback Loops
Positive Affirmation
Personalised
Teaching Teams
Constructivism By Proxy
Modular Assessment
Relevant Content
Open GLM University of Vienna
Results of thematic analysis of variety of data
16. Outcomes
0 2 4 6 8
6 Modules
5 Modules
4 Modules
3 Modules
2 Modules
1 Modules
At-Home N=19 Blended N=12
0 1 2 3 4 5 6 7
Unknown
Second Level School
VEC Second Level School
Youthreach
Solas
Community Training Centres
Blended At-Home
Certification
Progression
17. Synchronous Social Presence
Project Based Learning
Student Online Collaboration
A Model for Prevention
Conclusions
The interdependence of
sampling across phases used a
parallel and simultaneous design
(Cresswell & Plano-Clark, 2011)
that permitted triangulation of
data and ultimately
strengthened the research
process.
Research design supported a
study where data were collected,
analysed and interpreted in a way
that provided an insight into an
online learning journey from
induction to exit and also
identifying areas for
improvement.
18. References
Deepwell, F. (2007). Embedding Quality in e-Learning Implementation through Evaluation. Educational Technology &
Society, 10 (2), 34-33.
Kennedy, S. & Soifer, D. (2013). Why blended learning can’t stand still: A commitment to constant innovation is needed
to realise the potential of individualised learning. Lexington Institute.
Laurillard D. & Ljubojevic D. (2011). Evaluating learning designs through the formal representation of pedagogical
patterns. In Investigations of E-Learning Patterns: Context Factors, Problems and Solutions. (eds J.W. Kohls & C.
Kohls), pp. 86–105. IGI Global, Hershey, PA.
Masterman, L. (2013). The Challenge of Teacher’s Design Practice. In Beetham, H. and Sharpe, R. (Eds.), Rethinking
Pedagogy for a Digital Age, (64-77). New York: Routledge.
Oncu, S., & Cakir, H. (2011). Research in online learning environments: Priorities and methodologies. Computers &
Education, 57(1), 1098-1108. doi:10.1016/j.compedu.2010.12.009
Patrick, S., Edwards, D., Wicks, M. & Watson, J. (2012). Measuring Quality from Inputs to Outcomes: Creating Student
Learning Performance Metrics and Quality Assurance for Online Schools. Vienna (VA): International Association for K-
12 Online Learning.
Prieto, L., Dimitriadisa, Y., Craftb, B., Derntlc, M., Katsamanie, M., Laurillard, D., Masterman, E., Retalis, S. &
Villasclaras, E. (2013). Learning design Rashomon II: exploring one lesson through multiple tools. Research in Learning
Technology, 21.
Stake, R.E. (1967). The countenance of educational evaluation. Teachers
College Record, 68 (7), 523-540.
Swan, K., Garrison, D.R., and Richardson, J.C. (2009). A constructivist approach to online learning: The Community of
Inquiry framework. Information Technology in Higher Education: Progressive Learning Frameworks. Hershey, PA.
Hinweis der Redaktion
How does it work? Grown organically in response to needs. It works in practice how does it work in theory. What is the learning experience made up of? Examine closely through lens of COI. Dewey and Peirce model for scientific investigation Garrison/Anderson Canada to provide conceptual order and a good practice tool for use of computer mediated communication in supporting an educational experience. Constitutes three channels essential to an educational transaction – Cognitive/Social/Teaching Presence
Using Authoring tools normally used in the advance planning of courses to reflect on the design of lessons. Based on study by Prieto et al. Pedagogical Pattern Collector analyses the learning types used. Communications Module Unit 2 Writing Learning outcomes skills based … Draft at least 3 pieces of writing …. Use writing to discuss learning aims/plans/tasks. Places the lesson design and how the activities break down into a framework
Using Authoring tools normally used in the advance planning of courses to reflect on the design of lessons. Based on study by Prieto et al. Pedagogical Pattern Collector analyses the learning types used. Communications Module Unit 2 Writing Learning outcomes skills based … Draft at least 3 pieces of writing …. Use writing to discuss learning aims/plans/tasks. Places the lesson design and how the activities break down into a framework
