On October 9th 2018, SOONER PhD candidates Renée Jansen (Utrecht University) and Martine Schophuizen (Open University NL) organized a research workshop at the Teaching Lab at TU Delft. During this workshop various researchers from Dutch, Flemish and German institutions came together to discuss self-regulated learning in digital and (open) online learning environments.
SOONER Research Meeting: Self-regulated Learning in Technology Enhanced Learning Environments
1. SOONER research meeting:
SRL in technology enhanced
learning environments
Renée Jansen
Utrecht University
Martine Schophuizen
Open University of the Netherlands
TU Delft teaching lab, 9th of October 2018
2. Aim & activities
Brainstorm, discuss and share ideas on
measuring
and supporting SRL in technology enhanced
learning environments
• Defining SRL
• Educational settings
• SRL measurement 1/2
• SRL in trace data & coffee break
• SRL measurement 2/2
• SRL Interventions
• Practical issues
• Closing & drinks
3. Who is who?
• Name
• Affiliation
• Relationship between your work/research/interest
and today’s meeting 1 sentence!
4. Defining Self Regulated Learning
Students who self-regulated
their learning are
metacognitively,
motivationally, and
behaviorally active
participants in their own
learning process.
- Zimmerman (1986)
SRL is an event with four stages: (1) task definition, (2) goal
setting and planning, (3) enacting tactics and strategies, and
(4) metacognitively adapting studying techniques.
- Winne & Hadwin (1998)
Students use
strategies to
control and
regulate their
learning. Three
categories of
strategies are: (1)
cognitive learning
strategies, (2)
self-regulatory
strategies to
control cognition,
and (3) resource
management
strategies
- Pintrich (1999)
Students’ SRL
involves three
features: (1) the use of
SRL strategies, (2)
responsiveness to
self-oriented feedback
about learning
effectiveness, and (3)
interdependent
motivational
processes.
- Zimmerman (1990)
6. Educational settings
Traditional
face-2-face
Blended Fully online
What kind of students do you
commonly encounter?
Homogeneous or heterogeneous
group?
What resources are often used?
What medium is often used?
To what extent is there
collaboration?
How is structure offered?
To what extent is SRL needed?
How do you characterize ‘good
SRL’ in this environment?
7. SRL measurement
SRL can be measured with self-
reports (questionnaires,
interviews, think aloud, diary
studies), with observations, or
with automatically collected
data (trace data).
• Pick an educational context
• How do you think SRL should
be measured in that context?
8. SRL in trace data & coffee break
Trace
SRL
interpretations
Researcher
s
9. • Bottom-up approach
– First analyze trace data, then link
frequently occurring activities /
sequences to SRL
– Explorative approach
• Top-down approach
– Link activities / sequences to SRL up
front, analyze their occurrence in the
trace data
Kizilcec, Pérez-Sanagustín, & Maldonado (2017)
Analyzing SRL in trace data
10. SRL in trace data
• Look at the definitions of SRL and good SRL behavior we
defined earlier.
• Can we quantify those based on trace data?
• 3 themes:
– Blended
– Fully Online
– Native environment
Make it concrete. Problems usually arise when you try to define it exactly…
Think of sequences, frequencies etc.
11. SRL measurement
Measuring SRL with self-report
measures may have negatives,
but measuring SRL with trace
data is also not without hurdles.
• What would you consider a
best practice?
12. SRL measurement
Moving forward…
• Should we use trace data for
SRL measurement?
• Would mixed method
research be a solution?
• Which data gathering
methods to combine? Same
grain size or different grain
size?
13. SRL interventions
• Who was or is working on an intervention for SRL?
– Context/environment
– Type of intervention
– Outcome measures
– Did it work according to plan?
14. Practical issues
• What kinds of practical issues have you
encountered, what practical questions do you have?
– Platforms
– Data
– Access to data
– Data analysis tools/scripts
– Possible interventions?
15. SOONER research workshop:
SRL in technology enhanced learning environments
Thank you for joining!
Renée Jansen
Utrecht University
r.s.jansen@uu.nl
Martine Schophuizen
Open University of the Netherlands
martine.schophuizen@ou.nl
Hinweis der Redaktion
Explanation: Explain aim of the meeting (brainstorm and share ideas on measuring and supporting SRL in TEL), description of activities to come.
Explanation: Ask people to quickly introduce themselves: name, affiliation, relation of their research to today’s topic.
Aim: Create a joint definition we can work off for the afternoon; get insight into differences of opinion on what is/is not SRL.
Explanation: Numerous SRL definitions exist. While there are similarities between them, there are also differences. Show definitions on slide. How do you define SRL?
Follow-up/prompts:
What activities are part of SRL? (e.g., goal setting, time management).
Do you consider SRL to be temporal?
Is SRL a characteristic or domain-specific / dependent on context?
Is motivation part of SRL? What aspects of motivation? (e.g., self-efficacy, task value, goal orientation)
Aim: Create common ground about educational settings; get clarity on what is ‘good SRL’ in these settings.
Explanation: While a plurality of educational settings exists, ranging from completely face to face, to fully online (and also differing on numerous other aspects), we want to broadly define these. Support tool for the rest of the afternoon. Build from the more easily definable aspects to the more complicated ones.
Aim: Create common ground about educational settings; get clarity on what is ‘good SRL’ in these settings.
Explanation: While a plurality of educational settings exists, ranging from completely face to face, to fully online (and also differing on numerous other aspects), we want to broadly define these. Support tool for the rest of the afternoon. Build from the more easily definable aspects to the more complicated ones.
Aim: Discuss measuring SRL after we learned/shared more about measuring SRL with trace data.
Explanation: Measuring SRL with self-report measures may have negatives, but measuring SRL with trace data is also not without hurdles. What would you consider a best practice? How to move forward?
Follow-up/prompts:
Should we use trace data for SRL measurement?
Would mixed method research be a solution?
Which data gathering methods to combine? Same grain size or different grain size?
Aim: Discuss how to analyze SRL in trace data; familiarize those not yet familiar with this research with the topic; share insights in what can/cannot be done.
Explanation: One of the major benefits of online learning for educational research is the automatic storage of trace data (log data, clickstream data). How do we leap from trace data to SRL interpretations?
– Explanation top-down, bottom-up (REF) –
To create some common ground, and share some of the research done in this field, we have some excerpts from articles.
-- Excerpts from articles indicating SRL in trace data –
Explanation: We would like to discuss in small groups how you would extract SRL information from trace data. Log in to Coursera and enroll for a course. Write down how you would quantify good SRL behavior (e.g., sequences, frequencies, spacing of sessions).
Follow-up:
Share ideas: how would you measure SRL from trace data?
Share experiences about operationalizing SRL (easy/hard); what did you struggle with?; easy to reach agreement?
How many assumptions do you have to make?
Are there aspects defined in step 3e that we cannot measure with trace data? Which one(s)?
Aim: Discuss how to analyze SRL in trace data; familiarize those not yet familiar with this research with the topic; share insights in what can/cannot be done.
Explanation: One of the major benefits of online learning for educational research is the automatic storage of trace data (log data, clickstream data). How do we leap from trace data to SRL interpretations?
– Explanation top-down, bottom-up (REF) –
To create some common ground, and share some of the research done in this field, we have some excerpts from articles.
-- Excerpts from articles indicating SRL in trace data –
Explanation: We would like to discuss in small groups how you would extract SRL information from trace data. Log in to Coursera and enroll for a course. Write down how you would quantify good SRL behavior (e.g., sequences, frequencies, spacing of sessions).
Follow-up:
Share ideas: how would you measure SRL from trace data?
Share experiences about operationalizing SRL (easy/hard); what did you struggle with?; easy to reach agreement?
How many assumptions do you have to make?
Are there aspects defined in step 3e that we cannot measure with trace data? Which one(s)?
Aim: Discuss measuring SRL after we learned/shared more about measuring SRL with trace data.
Explanation: Measuring SRL with self-report measures may have negatives, but measuring SRL with trace data is also not without hurdles. What would you consider a best practice? How to move forward?
Follow-up/prompts:
Should we use trace data for SRL measurement?
Would mixed method research be a solution?
Which data gathering methods to combine? Same grain size or different grain size?
Aim: Discuss measuring SRL after we learned/shared more about measuring SRL with trace data.
Explanation: Measuring SRL with self-report measures may have negatives, but measuring SRL with trace data is also not without hurdles. What would you consider a best practice? How to move forward?
Follow-up/prompts:
Should we use trace data for SRL measurement?
Would mixed method research be a solution?
Which data gathering methods to combine? Same grain size or different grain size?
Aim: Brainstorm on possible SRL interventions.
Explantion: How can we stimulate SRL as we have defined it in step 3e? Anything is possible, so… try to base ourselves on own experiences and literature.
Follow-up/prompts:
What kind of study would you like to conduct?
Is someone currently developing an intervention? Do you need help/suggestions?
Split between blended learning and online learning.
How can we measure the effect of the intervention (refer back to steps 4-5-6)
Case description
Aim: Share practical information; work need-based.
Explanation: What kinds of practical issues have you encountered, what practical questions do you have? Share experiences.
Follow-up/prompts:
What platform have you worked with? (e.g., edX, Coursera, Blackboard, Moodle)
What data do you get?
How to get access to data?
Data analysis tools/scripts? platform (in)dependent?
What is possible in terms of interventions?
