uts-learning-futures-learning-analytics

LEARNING ANALYTICS
— an essential tool for learning in the future
Simon Buckingham Shum
Professor of Learning Informatics
Director, Connected Intelligence Centre
twitter @sbuckshum #UTSfuturelearn
UTS CRICOS PROVIDER CODE: 00099F uts.edu.au

3
can
we
tell
from
your
digital
profile
if
you’re
learning?

4
Every educational product now comes with a “dashboard”…

5
I was speaking at this event
http://codeactsineducation.wordpress.com

6
I asked Siri to find the conference website…
“Find code acts in
education”

7
I asked Siri to find the conference website…
“Find code acts in
education”

h,ps://twi,er.com/Wiswijzer2/status/414055472451575808
8
“Note:
check
the
huge
difference
between
knowing
and
measuring…”

Visualising “attainment” and “progress”
10

‘Joe’ may be developing curiosity, resilience, and
collaboration skills, but these are left invisible

CRITICAL: accounting tools are not neutral
“accounting tools...do not simply
aid the measurement of economic
activity, they shape the
reality they measure”
Du Gay, P. and Pryke, M. (2002) Cultural Economy: Cultural Analysis and Commercial Life. Sage, London. pp. 12-13
12

sta:s:cal
analysis
of
educa:onal
data
is
hardly
new
13

14
Institutional/Academic Data and Analytics:
current focus on summative data and trends
for senior educational leaders and policymakers
Macro:
region/state/national/international

15
Macro:

16
Macro:
Meso:
institution-wide

17
NSW DEC • CESE Datahub (+ many comparable open data initiatives)
https://data.cese.nsw.gov.au

18
UK RAISEonline data: schools live or die by this

19
Learning Analytics:
Data explosion at the micro level
enriching, and enriched by, the meso+macro levels
Macro:
Meso:
institution-wide
Micro:
individual student progress
Learning Analytics UNESCO Policy Briefing — http://iite.unesco.org/publications/3214711

Towards rapid formative feedback
outcome
intent
researchers
/
educators
/
instrucGonal
designers
administrators
/
leaders
/
policymakers
20

Faster feedback loops outcome
could enable more
rapid adaptation by
students and
teachers, and of
learning resources
and designs
intent
researchers
/
educators
/
instrucGonal
designers
administrators
/
leaders
/
policymakers
21
Towards rapid formative feedback

22
http://cdn.nmc.org/media/2014-nmc-horizon-report-k12-EN.pdf

23
http://redarchive.nmc.org/horizon-project/horizon-reports http://www.open.ac.uk/blogs/innovating

http://solaresearch.org/Policy_Strategy_Analytics.pdf http://tech.ed.gov/files/2013/02/Expanding-Evidence-Approaches.pdf 24

Ed-Tech startups and venture capital
26
http://techcrunch.com/2014/02/19/google-capital-invests-40m-in-learning-analytics-firm-renaissance-learning-at-1b-valuation
https://www.edsurge.com/n/2014-10-06-diving-into-data-analytics-tools-in-k-12

Ed-Tech startups and venture capital
“LearnCapital is a venture
capital firm focused exclusively
on funding entrepreneurs with
a vision for better and smarter
learning.”
http://learncapital.com 27

Hang on: algorithm designers in ed-tech startups
are shaping the learning experience?…
28
Candace Thille (Stanford)
“Why would universities
outsource their core
competency to ed-tech
h,p://reinventors.net/roundtables/strategies-‐for-‐organizaGonal-‐change
vendors?”

the
core
ques:on
in
slow
mo)on
29

30
can
we
tell
from
your
digital
profile
if
you’re
learning?

31
Who?
can
we
tell
from
your
digital
profile
if
you’re
learning?

32
Who?
How? With what confidence?
After what kinds of training?
can
we
tell
from
your
digital
profile
if
you’re
learning?

33
Who?
can
we
tell
from
your
digital
profile
if
you’re
learning?
Sourcing which data,
with what integrity?

34
Who?
can
we
tell
from
your
digital
profile
if
you’re
learning?
Sourcing which data,
with what integrity?
What kind of learning?
What kind of teaching?

35
what kinds of learners are
we trying to create?
– this should drive our analytics

Let’s hear from the UTS graduates of 2026
36
http://learningemergence.net/2014/04/07/learning-dispositions-authentic-inquiry-in-a-primary-school

The
Knowledge
–
Agency
Window
Open-‐
ended
Enquiry
Pre-‐scribed
curriculum
Teacher-‐directed
enquiry
Student
led-‐enquiry
Content/Expert
led
teaching
Student-‐led
revision
Teacher
Directed
Learning
Self-‐directed
Learning

We worry about disengaged low achievers like Joe
— but we need to worry about Emily too...
38 Guy Claxton: Constant change is here to stay: why schooling will always be about the future.
UK ESRC Futures Meeting, May 2011. http://www.slideshare.net/edfutures/guy-claxton-esrc-futures-may11

So our analytics depends on what space we’re in
Open-‐
ended
Enquiry
Pre-‐scribed
curriculum
Teacher-‐directed
enquiry
Student
led-‐enquiry
Authenticity
Agency
Identity
Content/Expert
led
teaching
Student-‐led
revision
Teacher
Directed
Learning
Self-‐directed
Learning

43
Analytics for the
UTS learning
model?
http://www.uts.edu.au/research-and-teaching/teaching-and-learning/learning2014/new-approaches

44
Analytics for the
UTS learning
model?

Analytics for the
UTS learning
model?

46
More efficient learning for mastering core skills
“In this study, results showed that
OLI-Statistics students [blended
learning] learned a full semester’s
worth of material in half as much
time and performed as well or
better than students learning from
traditional instruction over a full
semester.”
Lovett M, Meyer O and Thille C. (2008) The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student
learning. Journal of Interactive Media in Education 14. http://jime.open.ac.uk/article/2008-14/352

Learning analytics for this?
“We’re looking at the profiles of what it means to be
effective in the 21st century. […] Resilience
will be the defining concept. When challenged
and bent, you learn and bounce back stronger.”
“Dispositions are now at least as
important as Knowledge and Skills.
…They cannot be taught.
They can only be cultivated.”
John Seely Brown
US Dept. of Educ. http://reimaginingeducation.org conference (May 28, 2013)
Dispositions clip: http://www.c-spanvideo.org/clip/4457327
47
Whole talk: http://www.c-spanvideo.org/program/SecD

Learning analytics for this?
“It’s more than knowledge and skills. For the
innovation economy, dispositions
come into play: readiness to
collaborate; attention to
multiple perspectives; initiative;
persistence; curiosity.”
Larry Rosenstock
LearningREimagined project: http://learning-reimagined.com
Larry Rosenstock:
http://audioboo.fm/boos/1669375-50-seconds-of-larry-rosenstock-ceo-of-hightechhigh-on-how-he-would-re-imagine-learning
48

Survey-based analytics for learning dispositions
(Ruth Deakin Crick, Univ. Bristol / UTS:CIC Visiting Professor)
Deakin Crick et al (In Press). Developing Resilient Agency in
Learning: the Crick Learning for Resilient Agency Profile.
49

50
Survey-based analytics for learning dispositions
Mindful
Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Deakin Crick et al (In Press). Developing Resilient Agency in
Learning: the Crick Learning for Resilient Agency Profile.
Rapid
Visual
Feedback
to
SGmulate
Self-‐Directed
Change
A
framework
for
a
coaching
conversaGon

Mindful Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Taking responsibility for
my own learning over
time through defining my
purposes, understanding
and managing my
feelings, knowing how I
go about learning &
planning my learning
journey carefully.

Mindful Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Making connections
between what I already
know & new information
& experience. Making
meaning by linking my
story, my new learning &
my purpose.

Mindful Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Using my intuition &
imagination to generate
new ideas & knowledge.
Taking risks & playing
with ideas and artefacts
to arrive at new solutions.

Mindful Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Wanting to get beneath the
surface & find out more.
Always wondering why and how.

Mindful Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Being part of a learning community at
work, at home, in education & in my
social networks. Knowing I have
social resources to draw on when I
need them

Mindful Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Being able to work with
others, to collaborate and
co-generate new ideas
and artefacts. Being able
to listen and contribute
productively to a team.

Mindful Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Having the optimism & hope
that I can learn & achieve over
time. Having a growth mindset;
believing I can generate my
own new knowledge for what I
need to achieve

Mindful
Agency
Sense
making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
1 Dependent Openness to learning Closed
An emotional orientation of being open & ready to invest in learning,
having flexible self-belief, willing to persist & manage any self-doubt.
A necessary pre-requisite for developing resilience in learning

From self-report to behavioural analytics
for learning dispositions?
Mindful Agency
Sense making
Tagging/sharing/blogging/social
patterns reveal how you see
connections between ideas?
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Social network patterns,
teamwork effectiveness and
initiation of relationships?
Questioning, arguing and
search behaviours reveal
intrinsic curiosity and
epistemic commitments?

Analytics for the
UTS learning
model?
Relatively simple:
• How much/when library and
other resources are accessed
• Annotations on texts, images,
movies
• Video replays and rewinds
• Learner-shared resources

Shifts in epistemic commitments?
(Simon Knight, KMi Open U. UK)
Does the way you search
online reveal what you think
counts as trustworthy
knowledge?
What is it to ‘know’ when we search? http://sjgknight.com/finding-knowledge/2014/02/knowledge-in-search
Danish exams permit Net: http://sjgknight.com/finding-knowledge/2013/07/danish-use-of-internet-in-exams-epistemology-pedagogy-assessment
Epistemic networks for epistemic commitments: http://oro.open.ac.uk/39254

Shifts in epistemic commitments?
(Simon Knight, KMi Open U. UK)
Dimensions of Epistemic Belief
Certainty The degree to which knowledge is conceived as stable or changing, ranging from
absolute, to tentative and evolving
Simplicity The degree to which knowledge is conceived as compartmentalised or
interrelated, ranging from knowledge as made up of discrete and simple facts to
knowledge as complex and comprising interrelated concepts
Source The relationship between knower and known, ranging from the belief that
knowledge resides outside the self and is transmitted, to the belief that it is
constructed by the self
Justification What makes a sufficient knowledge claim, ranging from the belief in observation
or authority as sources, to the belief in the use of rules of inquiry and evaluation
of expertise
Knight, Simon; Buckingham Shum, Simon and Littleton, Karen (2014). Epistemology, assessment, pedagogy: where
learning meets analytics in the middle space. Journal of Learning Analytics (In press). http://oro.open.ac.uk/39226

64
Analytics for the
UTS learning
model?

65
Are fieldwork students competent?

In the future, how might we
track teamworking? (U. Sydney)
Automatic student tracking
from multiple digital
tabletops in classroom
Analyse the students’ activity
traces for significant
patterns
Visualise on a teacher’s
dashboard
R. Martinez, K. Yacef, J. Kay, and B. Schwendimann.
An interactive teacher’s dashboard for monitoring
multiple groups in a multi-tabletop learning
environment. Proceedings of Intelligent Tutoring
Systems, pages 482-492. Springer, 2012.

In the future, how might we track teamworking?
R. Martinez, K. Yacef, J. Kay, and B. Schwendimann.
An interactive teacher’s dashboard for monitoring
multiple groups in a multi-tabletop learning
environment. Proceedings of Intelligent Tutoring
Systems, pages 482-492. Springer, 2012.

Learning
Technology
KMi,
OU
AI
&
ArgumentaGon
Learning
DisposiGons
Learning
AnalyGcs
SemanGc
ScienGfic
Human-‐Centred
InformaGcs
Publishing
Dialogue
/
Issue
/
Argument
Mapping
Social learning
analytics
— quantifying
“professional
identity”

Visual analytics to filter a social learning network
by topic and type of social tie
Schreurs B, Teplovs C, Ferguson R, De Laat
M and Buckingham Shum S. (2013)
Visualizing Social Learning Ties by Type
and Topic: Rationale and Concept
Demonstrator. Proc. 3rd International
Conference on Learning Analytics &
Knowledge. Leuven, BE: ACM, 33-37. Open
Access Eprint: http://oro.open.ac.uk/36891

What epistemic contributions are learners making in the community?
De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. (2011). Discourse-centric learning analytics. 1st Int. Conf.
Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr). ACM: New York. Eprint: http://oro.open.ac.uk/25829
71
Rebecca is playing the
role of broker,
connecting different
peers’ contributions in
meaningful ways We now have the basis
for recommending that
you engage with
people NOT like you…

72
Analytics for the
UTS learning
model?

Real time traffic-lights for students based on predictive
model (Purdue University Signals)
Validate a statistical model from:
• ACT or SAT score
• Overall grade-point average
• LMS usage composite
• LMS assessment composite
• LMS assignment composite
• LMS calendar composite
Predicted 66%-80% of struggling
students who needed help
Campbell et al (2703 07). Academic Analytics: A New Tool for a New Era, EDUCAUSE
Review, vol. 42, no. 4 (July/August 2007): 40–57. http://bit.ly/lmxG2x

Prototyping a social learning analytics dashboard
1 2 3
5 4
Your most
recent mood
comment:
“Great, at last
I have found all
the resources
that I have
been looking
for, thanks to
Steve and
Ellen.
In your last discussion with your mentor, you decided
to work on your resilience by taking on more
learning challenges
Your ELLI Spider
shows that you have
made a start on
working on your
resilience, and that
you are also
beginning to work on
your creativity, which
you identified as
another area to work
on.
Ferguson R and Buckingham Shum S. (2012) Social Learning Analytics: Five Approaches. Proc. 2nd International Conference on Learning Analytics &
Knowledge. Vancouver, 29 Apr-2 May: ACM: New York, 23-33. DOI: http://dx.doi.org/10.1145/2330601.2330616 Eprint: http://oro.open.ac.uk/32910
74

75
Analytics for the
UTS learning
model?

Teacher’s dashboard for EnquiryBlogger

The hallmarks of educated writing/critical thinking
Relevance
Understanding & Knowledge
Structure & Organisation
Linguistic Accuracy
Illustrations
Referencing
Argumentation
78

Rhetorical functions of metadiscourse
identified by the Xerox Incremental Parser (XIP)
BACKGROUND KNOWLEDGE
“Recent studies indicate …”
“… the previously proposed …”
“… is universally accepted ... “
NOVELTY
... new insights provide direct evidence ...
... we suggest a new ... approach ...
... results define a novel role ...
OPEN QUESTION
… little is known …
… role … has been elusive
Current data is insufficient …
GENERALIZING
... emerging as a promising approach
Our understanding ... has grown
exponentially ...
... growing recognition of the importance ...
CONTRASTING IDEAS
… unorthodox view resolves …
paradoxes …
In contrast with previous
hypotheses ...
... inconsistent with past
findings ...
SIGNIFICANCE
studies ... have provided important
advances
Knowledge ... is crucial for ... understanding
valuable information ... from studies
SURPRISE
We have recently observed ... surprisingly
We have identified ... unusual
The recent discovery ... suggests intriguing roles
SUMMARIZING
The goal of this study ...
Here, we show ...
Altogether, our results ... indicate
79

XIP rhetorical parser
applied to student writing
Simsek, D., S. Buckingham Shum, Á. Sándor, A. De Liddo and R. Ferguson (2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of
Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics &
Knowledge. Leuven, BE (Apr. 8-12, 2013). http://oro.open.ac.uk/37391
80

XIP rhetorical parser
applied to student writing
CONTRAST
SUMMARY &
CONTRIBUTION
Simsek, D., S. Buckingham Shum, Á. Sándor, A. De Liddo and R. Ferguson (2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of
Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics &
Knowledge. Leuven, BE (Apr. 8-12, 2013). http://oro.open.ac.uk/37391
81

h,p://th02.deviantart.net/fs71/PRE/f/2011/119/c/0/rabbit_shadow_by_plasGcx76-‐d3f5qtd.jpg
83
Learners will leave an increasingly rich “digital shadow”
Learning Analytics use this to provide rapid formative feedback
(but this is of course an imperfect proxy for real experience)

84
Now the shadow can talk back!
Let’s design shadows that learners + teachers value

uts-learning-futures-learning-analytics

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie uts-learning-futures-learning-analytics

Ähnlich wie uts-learning-futures-learning-analytics (20)

Mehr von Simon Buckingham Shum

Mehr von Simon Buckingham Shum (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

uts-learning-futures-learning-analytics