UTSLTF2014 keynote - Learning.Analytics for Learning.Futures?

Google Docs for interactive exercises:
Learning.Analytics for
Learning.Futures?
Simon Buckingham Shum
Director, Connected Intelligence Centre
@sbuckshum #LearningAnalytics
Ruth Deakin Crick
UTS:CIC Visiting Professor & University of Bristol UK
@ruthdeakincrick
bit.ly/utsltf2014
cic.uts.edu.au

Learning.Futures @UTS
5
http://learningemergence.net/2014/04/07/learning-dispositions-authentic-inquiry-in-a-primary-school

cic.uts.edu.au
Iden%ty
Forma%on
Personal
Quali%es
Knowledge
&
Skills
Competence
6

we
should
be
talking
first
about
cic.uts.edu.au
where
we
want
to
go
then
we
can
discuss
engines,
performance
and
dashboards
7

VUCA
Volatile • Uncertain
Complex • Ambiguous

9
well-defined routes
mapped by others
routine performance
success depends on
following the rules
http://weheartit.com/entry/14357348

10
well-defined routes
mapped by others
routine performance
success depends on
following the rules
unknown territory
naturally complex
each challenge unique
success depends
on improvisation
http://awe-inspired.deviantart.com/art/White-Water-Rapids-45947012
http://weheartit.com/entry/14357348

From the known to the unknown
Unknown
Strange
Uncomfortable
What we know
Familiar
Comfortable

From the known to the unknown
Unknown
Strange
Uncomfortable
What we know
Familiar
Comfortable
“liminal space”

“Liminal Space… when you have left the tried and true
but have not yet been able to replace it with anything else.
…when you are between your old comfort zone and any possible new
answer… If you are not trained in how to hold
anxiety, how to live with ambiguity, how to
entrust and wait, you will run…
anything to flee this terrible cloud of unknowing.”
Richard Rohr O.F.M.
— on the spirituality of liminal space
Limina is the Latin word for threshold, the space betwixt and between
http://sojo.net/magazine/2002/01/grieving-sacred-space

So while we initially strive to make our students feel
comfortable […] we then must help them balance their desire for
security with the need to take risks and explore new ideas
and possibilities.
Rather than attempting to resolve the tension, a college should help
students find their place on this precarious
threshold, in the liminal space between the
familiar and strange, the old and the new.
Johansson & Felten, 2014

Our learners face this. So do we as staff.
Unknown
Strange
Uncomfortable
What we know
Familiar
Comfortable
“liminal space”

Can we equip students for
liminal space?
Do we know how to teach for
liminality?

The
Knowledge
–
Agency
Window
Open-‐
ended
Enquiry
Pre-‐scribed
curriculum
Teacher-‐directed
enquiry
Student
led-‐enquiry
Content/Expert
led
teaching
Student-‐led
revision
Teacher
Directed
Learning
Self-‐directed
Learning

Analytics depends on what space you’re in
Deakin Crick et al (2011), Learning Futures Final Report. Paul Hamlyn Foundation. http://learningemergence.net/library/reports/learning-futures-evaluation-2011
cic.uts.edu.au
Open-‐
ended
Enquiry
Pre-‐scribed
curriculum
Teacher-‐directed
enquiry
Student
led-‐enquiry
Content/Expert
led
teaching
Student-‐led
revision
Teacher
Directed
Learning
Self-‐directed
Learning
Teaching
as Design
Teaching as
Script
Authenticity
Agency
Identity

2 different worldviews (500 school teachers)
Ruth Deakin Crick, Chris Goldspink & Margot Foster (2013). Telling Identities: Learning as Script or Design? Learning Emergence Discussion Paper (June, 2013). PDF
http://learningemergence.net/events/lasi-dla-wkshp
cic.uts.edu.au
Develop democratic
relationships
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
Build a community of
learners
Negotiate learning
Challenge and
support learners
Teach students how
to learn
Foster deep
understanding
Explore the
construction of
knowledge
Communicate in
multiple modes
Apply and assess in
authentic contexts
Connect learning to
lives and aspirations
Build on learners
understanding
Promote dialogue as
a means of learning
Dynamic Design
Script

Knowledge Agency Window exercise
cic.uts.edu.au
24
bit.ly/utsltf2014

scaffolding open-ended
enquiry
cic.uts.edu.au
25

The big challenge for 21st century educators
“educational philosophy and theory
cic.uts.edu.au
face the unfamiliar and challenging
task of theorising a formative process
which is not guided from the start by
the target form designed in advance”
Zygmunt Bauman
26
Zygmunt Bauman, 2001: The Individualised Society, Cambridge, Polity Press.

Conveying
new knowledge
cic.uts.edu.au 27
Choosing
carefully
Collecting
thoroughly
Connecting
deeply
Constructing
creatively

28
Choosing and
Deciding
Observing and
Describing
Generating
Questions
Uncovering
Stories
Knowledge
Mapping
Applying in the
World
Connecting
with existing
knowledge
Assessing and
Validating
Interface with
success
criteria
Using Genuine
Interest to
Understand, Explore,
Create and Improve
BREAK-THROUGH
INNOVATION
LEARNING
and
PERFORMANCE
Dynamics
of Open-
Ended
Enquiry

Jess’s Choice
cic.uts.edu.au 29

30
What will be there in 15 years?
What was there before?
How many people have been there?
How was the gorge made?
Have any famous people been there?
Why do relationships matter?
What kind of people used to be there?

‘It’s made me not so scared to
learn other things,’ ‘It was a
tiny little project and it spiraled
into all these other things that
were connected.’

Anchored in personal interest…Starting with
experience …..
So my choice was made….. the gypsy caravan, and my
friend…. I looked at the questions generated from my initial mind-map
cic.uts.edu.au
and I began to see a way of grouping them.

Is there a thin line between the acceptance and rejection of
gypsy travellers as a minority group within a wider
community?
34
The outcome is derived from the process

35
…I was digging deep, maybe I was
experiencing a little of what we had talked
about, “theory U”. An inward bound
journey leading to an outward, letting go of
preconceived ideas and experiencing
change through the process.

cic.uts.edu.au
anchored in personal interest
(rather than didactic intent)
starts with experience
(rather than opinion or theory)
creates its own dynamic
(rather than following taught routines)
relates and unites disciplines and domains
(rather than fragmenting them)
Integrates values with learning
(rather than separating them)
is reflexive on its process
(rather than concerned only with outcome)
allows the outcome to derive from the process
(rather than pre-determine it)
36

assessing open-ended enquiry?
cic.uts.edu.au
37
bit.ly/utsltf2014

Equipping
learners to
tackle
complexity
38
Competence
in the world
Knowledge skills
& understanding
Personal
qualities, values
attitudes &
dispositions
Identity
story, desire,
motivation,
relationships,
purpose
Deakin Crick, R. (2012). Student
Engagement: Identity, Learning
Power and Enquiry - a complex
systems approach. in:
Christenson, S., Reschly, A. &
Wylie, C. (eds.) The Handbook
of Research on Student
Engagement. New York:
Springer
Practical
Propositional
Presentational
Experiential
Public
Personal

39
….if the UK wants to
produce more engineers,
it needs to redesign the
education system so that
these ENGINEERING
HABITS OF MIND
become the desired
outcomes of engineering
education…p3
Royal Academy of Engineering,
(2014) Thinking Like an Engineer,
London, RAE.

This material is based upon work supported, in whole or in part, by the U.S. Department of Defense through the
Systems Engineering Research Center (SERC) under Contracts H98230-08-D-0171 and HQ0034-13-D-0004 . The
SERC is a federally funded University Affiliated Research Center (UARC) managed by Stevens Institute of
Technology consisting of a collaborative network of over 20 universities. See www.SERCuarc.org.
cic.uts.edu.au
40
Systems Engineering Research Center (SERC)
Stevens Institute of Technology, New Jersey, USA.
.characteristics of effective engineers…

cic.uts.edu.au
1. Paradoxical Mindset
― Big Picture Thinking and Attention to
Detail
― Strategic and Tactical
― Analytic and Synthetic
― Courageous and Humble
― Methodical and Creative
2. Effective Communication
― Modes (oral and written; good
speakers and listeners)
― Audience (bridge between problem
domain and solution domain)
― Content (social, managerial, technical)
― Purpose (understanding needs,
negotiation, information brokering,
technical arbitration, driving
consensus)
41
3. Flexible Comfort Zone
― Open Minded
― Rational Risk Taking
― Multidisciplinary
― Enjoys Challenges
4. Smart Leadership
― Quick Learning and Abstraction
― Knowing when to stop
― Focused on ‘Vision’ for System
― Ability to Connect the Dots
― Patience
5. Self Starter
― Curiosity _ Passionate and Motivated
― Eager to Learn

where do learning
analytics fit?
cic.uts.edu.au
42

cic.uts.edu.au
Academic Analytics: focus on data for senior
educational leaders
Macro:
region/state/national/international
Meso:
institution-wide
43

cic.uts.edu.au
Learning Analytics:
Data explosion at the micro level
enriching, and enriched by, the meso+macro levels
Macro:
region/state/national/international
Meso:
institution-wide
Micro:
individual student progress
Learning Analytics UNESCO Policy Briefing — Simon Buckingham Shum
44

analytics for open-ended
enquiry?
cic.uts.edu.au
45

cic.uts.edu.au
dispositional
analytics
46

Survey-based analytics for learning dispositions
cic.uts.edu.au
Deakin Crick et al (In Press). Developing Resilient Agency in
Learning: the Crick Learning for Resilient Agency Profile.
47

Survey-based analytics for learning dispositions
cic.uts.edu.au
Mindful
Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Deakin Crick et al (In Press). Developing Resilient Agency in
Learning: the Crick Learning for Resilient Agency Profile.
Rapid
Visual
Feedback
to
SImulate
Self-‐Directed
Change
A
framework
for
a
coaching
conversaIon
48

Using my intuition &
imagination to generate new
ideas & knowledge. Taking
risks & playing with ideas
and artefacts to arrive at
new solutions.
cic.uts.edu.au
Taking responsibility for my own learning over time through defining my purposes, understanding and
managing my feelings, knowing how I go about learning & planning my learning journey carefully.
Mindful Agency
Sense making
Creativity
Having the optimism & hope that I can
learn & achieve over time. Having a
growth mindset; believing I can generate
my own new knowledge for what I need
to achieve
Hope and
optimism
Being able to work with others, to
collaborate and co-generate new
ideas and artefacts. Being able to
listen and contribute productively
to a team.
Collaboration
Being part of a learning community at
work, at home, in education & in my
social networks. Knowing I have social
resources to draw on when I need them
Belonging Curiosity
Making connections
between what I already
know & new information &
experience. Making
meaning by linking my
story, my new learning &
my purpose.
Wanting to get beneath the
surface & find out more.
Always wondering why and
how.
1
An emotional orientation of being open & ready to invest in learning, having flexible self-belief, willing to
persist & manage any self-doubt. A necessary pre-requisite for developing resilience in learning
Dependent Openness to learning Closed

cic.uts.edu.au
blogging +
dispositional
analytics
50

cic.uts.edu.au
Blogging for learning
Blogs offer learners opportunities to incorporate many perspectives, develop
carefully crafted contributions, reflect and make considered responses to
others (Ferguson et al., 2007). This medium provides an environment in which
people can observe, articulate and refine practices (Efimova et al., 2004).
At the same time, by making use of the comment facility, bloggers are able to
share thoughts, ideas and opinions (Du and Wagner, 2005). In order for
students to engage effectively with this emerging genre, they need to be able
to experiment and take ownership of their writing, learning to develop a blog
as a space for personal learning, reflection and interaction (Bryman and
Burgess, 1994).

cic.uts.edu.au
EnquiryBlogger (Wordpress Multisite plug-ins)

Composing and categorising a blog post
cic.uts.edu.au
Standard blog editor,
including option to embed
multimedia
Categories relating to authentic
enquiry, which are visualized by
the plugins

cic.uts.edu.au
Enquiry Spiral widget

cic.uts.edu.au
ELLI Spider widget

cic.uts.edu.au
Mood View widget

cic.uts.edu.au
Dashboard view

cic.uts.edu.au
Teacher’s dashboard for EnquiryBlogger

cic.uts.edu.au
Masters level EnquiryBloggers
Graduate School of Education, University of Bristol

cic.uts.edu.au 64
Primary School EnquiryBloggers
Bushfield School, Milton Keynes

cic.uts.edu.au
Primary School EnquiryBloggers
Bushfield School, Milton Keynes

writing analytics
cic.uts.edu.au
66

cic.uts.edu.au
Towards Academic Writing Analytics
Relevance
Understanding & Knowledge
Structure & Organisation
Linguistic Accuracy
Illustrations
Referencing
Argumentation
67

cic.uts.edu.au
Rhetorical functions of metadiscourse
identified by the Xerox Incremental Parser (XIP)
BACKGROUND KNOWLEDGE
Recent studies indicate …
… the previously proposed …
… is universally accepted ...
NOVELTY
... new insights provide direct evidence ...
... we suggest a new ... approach ...
... results define a novel role ...
OPEN QUESTION
… little is known …
… role … has been elusive
Current data is insufficient …
GENERALIZING
... emerging as a promising approach
Our understanding ... has grown
exponentially ...
... growing recognition of the importance ...
CONTRASTING IDEAS
… unorthodox view resolves …
paradoxes …
In contrast with previous
hypotheses ...
... inconsistent with past
findings ...
SIGNIFICANCE
studies ... have provided important
advances
Knowledge ... is crucial for ... understanding
valuable information ... from studies
SURPRISE
We have recently observed ... surprisingly
We have identified ... unusual
The recent discovery ... suggests intriguing roles
SUMMARIZING
The goal of this study ...
Here, we show ...
Altogether, our results ... indicate
68

cic.uts.edu.au
XIP rhetorical parser
applied to student writing
Simsek, D., S. Buckingham Shum, Á. Sándor, A. De Liddo and R. Ferguson (2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of
Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics &
Knowledge. Leuven, BE (Apr. 8-12, 2013). http://oro.open.ac.uk/37391
69

SUMMARY &
CONTRIBUTION
cic.uts.edu.au
XIP rhetorical parser
applied to student writing
CONTRAST
Simsek, D., S. Buckingham Shum, Á. Sándor, A. De Liddo and R. Ferguson (2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of
Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics &
Knowledge. Leuven, BE (Apr. 8-12, 2013). http://oro.open.ac.uk/37391
70

Buckingham Shum S.J., Uren V., Li G., Sereno B. and Mancini C. (2007). Modelling Naturalistic Argumentation in Research Literatures: Representation and
Interaction Design Issues. Int. Jnl. Intelligent Systems, Special Issue on Computational Models of Natural Argument, Eds: C. Reed and F. Grasso, 22: 17-47.
Sereno B., Buckingham Shum S. and Motta E. (2005). ClaimSpotter: an environment to support sensemaking with knowledge triples. Proceedings of the 10th
international conference on Intelligent user interfaces. San Diego, California, USA: ACM, 199-206.
cic.uts.edu.au
Detailed video analysis of user experience
71
• how users engage with a
text annotated by a
rhetorical parser
• whether users can
annotate a text
semantically

Behavioural analytics for learning dispositions?
Tagging/sharing/blogging/social
patterns reveal how you see
connections between ideas?
cic.uts.edu.au
Mindful Agency
Sense making
Creativity
Hope and
optimism
Collaboration
Belonging Curiosity
Social network patterns,
teamwork effectiveness and
initiation of relationships?
Questioning, arguing and
search behaviours reveal
intrinsic curiosity and
epistemic commitments?
72

dispositional analytics exercise
cic.uts.edu.au
73
bit.ly/utsltf2014

cic.uts.edu.au
mapping our
learning@UTS
analytics for organisational learning
74

1. Build a picture of the target problem and
‘what works’ — what’s known about solutions
cic.uts.edu.au
Coordinating
Maps
(of the system,
problem, and
strategies)
75

A “Driver Diagram” summarising what is known
about Productive Persistence in mathematics
cic.uts.edu.au
Examples of coordinating Maps to guide a
systemic improvement community
A map of the “Challenge Space”
for student mathematics
http://www.carnegiefoundation.org/elibrary/getting-ideas-action-building-networked-
improvement-communities-in-education
http://www.carnegiealphalabs.org/persistence/
76

cic.uts.edu.au
2. Coordinate to test solutions in different
contexts in UTS
Coordinating
Maps
(of the system,
problem, and
strategies)
UTS learning
cUonTtSe lxeta rning
cUoTnSte lxeta rning
context
77

cic.uts.edu.au
3. Pool the Data (quant. + qual.)
Data &
Evidence
Coordinating Hub
Maps
(of the system,
problem, and
strategies)
UTS learning
context
78

cic.uts.edu.au
An evidence hub shows who is working on
which parts of the problem
evidence-hub.net
People / Organizations / Projects / Claims / Evidence
Evidence Hub for Research by Children & Young People: http://rcyp.evidence-hub.net
79

cic.uts.edu.au
4. Analyse the data
Data &
Evidence
Hub
Human + Machine
Analytics
Coordinating
Maps
(of the system,
problem, and
strategies)
UTS learning
context
80

cic.uts.edu.au
5. interpret, narrate, debate the meaning
Data &
Evidence
Hub
Collective
Sensemaking
Coordinating
Maps
(of the system,
problem, and
strategies)
Human + Machine
Analytics
UTS learning
context
81

6. Share insights in Hub, update your maps, and
repeat…
cic.uts.edu.au
Data &
Evidence
Hub
Collective
Sensemaking
Coordinating
Maps
(of the system,
problem, and
strategies)
Human + Machine
Analytics
UTS learning
context
82

Evidence Hub: indexing stories + evidence
cic.uts.edu.au
Systems Learning & Leadership Evidence Hub: http://sysll.evidence-hub.net
A wizard guides the user
through the submission of a
structured story:
• What’s the Issue?
• What claim are you making/
addressing?
• What kind of evidence
supports/challenges this?
• Link it to papers/data
• Index it against the Hub’s
core themes
83

cic.uts.edu.au
…which generates a knowledge tree
http://learningemergence.net/2013/07/17/deed-elli-ai-ci-systemic-school-learning
Issue
Potential
Solution
Supporting
Evidence
(practitioner
story)
84

Impact Map: how much evidence is there to
support each hypothesis?
cic.uts.edu.au
http://oermap.org/hypothesis/578/hypothesis-a-performance/ 85

exercise on improvement cycle
cic.uts.edu.au
86
bit.ly/utsltf2014

cic.uts.edu.au
summary
use of analytics is shaped by
pedagogy, epistemology,
assessment regime
87

analytics for the assessment of
authentic enquiry / C21 skills
are a frontier challenge
! formative, personalised
feedback to learners
cic.uts.edu.au
88

to learn more…
cic.uts.edu.au
89

Designing holistic analytics for 21st century learners?
cic.uts.edu.au
http://reinventors.net/roundtables/new-metrics
90

LearningEmergence.net http://learningemergence.net
cic.uts.edu.au
91

ALASI 2014
Australian Learning Analytics
Summer Institute
Thurs 20th–Fri 21st Nov, 2014, Sydney
alasi2014.uts.edu.au
twitter: #alasi2014
cic.uts.edu.au
In association with Hosted by

cic.uts.edu.au
cic.uts.edu.au
CIC events coming up…
Nov 14, 10am-2pm: Assessing Learning
Dispositions using CLARA
(Ruth Deakin Crick)
Nov 17, 9.30-10.30am: Open Digital
Badges (Dan Hickey, Univ. Indiana)
Nov 20-21: Australian Learning Analytics
Summer Institute

UTSLTF2014 keynote - Learning.Analytics for Learning.Futures?

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie UTSLTF2014 keynote - Learning.Analytics for Learning.Futures?

Ähnlich wie UTSLTF2014 keynote - Learning.Analytics for Learning.Futures? (20)

Mehr von Simon Buckingham Shum

Mehr von Simon Buckingham Shum (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

UTSLTF2014 keynote - Learning.Analytics for Learning.Futures?