How to harness the big data deluge to usefully inform learning and teaching practices in higher education.

1. Learning Analytics &
Educational Research
Leveraging Big Data in Powerful Ways
Alyssa Friend Wise
Image Credit: Graham Cook via Flickr (CC BY 2.0), adapted
Associate Professor, Simon Fraser University
Educational Technology & Learning Design

2. Image Credit: Dakotilla via Flickr (CC BY 2.0)
Impassive clock!
Terrifying, sinister god,
Whose finger threatens us
and says: Remember!
…
Three thousand six hundred
times an hour, Second
whispers: Remember!
(Baudelaire, 1857)
The clock is a powerful machinery that creates
the product of “seconds” and “minutes” (Mumford, 1934)

3. Image Credit: Dakotilla via Flickr (CC BY 2.0)
The clock is a powerful machinery that creates
the product of “seconds” and “minutes”
and thus changes our relationship to time
Was a critical tool for navigation
originally in calculating ship’s
longitude, today for GPS
Is important in communication,
scheduling connections of people
within and across geographic space
Creates an awareness of allocation
that can lead to greater efficiencies
or value-based decision-making
(Mumford, 1934)
Time’s Quantification & Standardization

4. IF BEING ABLE TO TRACK
(AND KEEP TRACK OF) SOMETHING
CHANGES OUR RELATIONSHIP TO IT,
WHAT IS IT THAT BIG DATA IS
CHANGING OUR RELATIONSHIP TO
IN THE REALM OF EDUCATION?

5. Image Credit: UGA College of Agriculture and Environmental Sciences via Flickr (CC BY 2.0), adapted

6. Image Credit: UGA College of Agriculture and Environmental Sciences via Flickr (CC BY 2.0), adapted

7. BIG DATA IS CHANGING OUR
RELATIONSHIP TO
THE PROCESS OF LEARNING

8. EPHEMERAL FLOW OF EXPERIENCE
GLOBAL “PROCESS” VARIABLES

ONLY RECENTLY HAVE TRUE PROCESS DATA &
ANALYSES BECOME AVAILABLE
INNOVATION /
CHARACTERISTIC
OUTCOMES
LEARNING
PROCESS

9. EMERGING FORMS OF PROCESS DATA IN EDUCATION
 LOG-FILE DATA FROM ONLINE COURSES (INC. MOOCS)
 CLICKSTREAM DATA FROM LMSS USED IN F2F COURSES
 USAGE DATA FROM DIGITAL TEXTBOOKS AND NEW
SPECIALIZED CONTENT BROWSERS (E.G. NSTUDY)
 DATA FROM PHYSICAL SPACE (MOVEMENT, EYETRACKS…)
 SELF-CONTRIBUTED DATA (POLLING, HABITS, GOALTRACKS)

10. VISIONS OF WHAT WE CAN DO WITH THIS DATA
CONTRIBUTE TO BROAD BASED UNDERSTANDING OF LEARNING
 IDENTIFY GLOBAL PATTERNS ACROSS STUDENTS
 FIND MEANINGFUL DISTINCTIONS -> SUBSETS OF STUDENTS
INFORM US AT THE “FRONT LINES” OF HIGHER EDUCATION
 CREATE ACTIONABLE LOCAL INTELLIGENCE
 FOR INSTRUCTORS TO PRACTICE RESPONSIVE TEACHING
 FOR STUDENTS TO BECOME ACTIVE AGENTS OF THEIR
OWN LEARNING - “N = ME” (WINNE, IN PRESS)

11. L E A R N I N G A N A LY T I C S

12. LEARNING
ANALYTICS
THE COLLECTION AND
ANALYSIS OF DATA
TRACES RELATED TO
LEARNING IN ORDER
TO INFORM AND
IMPROVE THE
PROCESS AND/OR ITS
OUTCOMES (SIEMENS ET AL., 2011)

13. 0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
1
0
1
0
1
1
0
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
0
1
0
1
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
0
1
1
0
1
0
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
0
0
1
1
0
1
0
1
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
0
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
1
0
0
1
0
1
0
1
0
1
0
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
0
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
1
0
0
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
1
0
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
1
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
1
0
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
1
1
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
1
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
1
0
1
0
1
0
0
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
1
1
0
1
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
0
1
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
0
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
0
1
1
0
1
0
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
0
1
0
0
1
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
0
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
1
1
1
0
0
0
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
1
1
1
0
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
0
1
1
0
0
1
1
1
0
1
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
0
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
1
0
1
1
0
1
0
1
0
1
0
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
1
0
1
0
1
1
0
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
0
1
0
1
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
0
1
1
0
1
0
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
0
0
1
1
0
1
0
1
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
0
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
1
0
1
0
1
0
0
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
1
1
0
1
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
0
1
1
0
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
0
1
1
0
1
0
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
1
0
0
1
0
0
1
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
1
1
1
0
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
0
1
1
0
0
1
1
1
0
1
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
0
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
1
0
1
1
0
1
0
1
0
1
0
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
0
1
0
0
1
0
1
1
1
0
1
0
0
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
0
1
0
1
1
0
0
1
1
0
1
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
1
0
0
0
1
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
1
0
1
1
1
1
0
1
1
0
0
1
1
0
1
1
0
1
0
0
1
0
1
0
1
1

14. Sub UniquePostsRead()
For k = 1 To MaxUser Step 1
RowCount = Range("A1").CurrentRegion.Rows.Count
For w = 1 to MaxWeek Step 1
StartTime = Sheets("Week").Cells(w + 1, 2)
EndTime = Sheets("Week").Cells(w + 1, 3)
PostNum = 0
PostsIndex = 0
Do While Cells(i, datestamp) <= EndTime And i <= RowCount
If Cells(i, Source) = “Read" Then
If Cells(i, Message_Author) <> Val(ActiveSheet.Name)
And Cells(i, Scan) <> "X" Then
flag = 0
For j = 1 To PostsIndex Step 1
If Posts(j) = Cells(i, Message_Id) Then
flag = 1
j = PostsIndex
End If
Next j
If flag = 0 Then
PostsIndex = PostsIndex + 1
Posts(PostsIndex) = Cells(i, Message_Id)
End If
End If
End If
Sheets(“Stats").Cells(Line, 22) = PostsIndex
Next w
Next k
End Sub
PercentPostsRead =SUniquePostsRead
TotalPostNumber

16. Image Credit: Joshua Rothass via Flickr (CC BY 2.0), adapted

17. Image Credit: US Department of Education via Flickr (CC BY 2.0), adapted

18. LEARNING
ANALYTICS
USES COMPUTATIONAL
METHODS TO GENERATE
INSIGHT INTO LEARNING
PROCESSES THAT CAN BE
USED TO INFORM HUMAN-
DECISION MAKING WHILE
LEARNING EVENTS ARE
STILL IN PROCESS
(WISE, IN PREPARATION)

19. HOW DO WE HELP
LEARNING ANALYTICS BE
AN INNOVATION THAT
MAKES A REAL IMPACT
ON TEACHING AND
LEARNING ?

20. WE NEED TO DESIGN FOR
WAYS IN WHICH ANALYTICS
CAN USEFULLY
REFLECT & INFORM
THE TEACHING AND LEARNING
PRACTICES OF INSTRUCTORS
AND STUDENTS

21. PART 1: WORKING WITH
INDICATORS THAT
MEANINGFUL REFLECT
LEARNING PROCESSES

22. HOW DO WE DEVELOP
RICH INDICATORS
THAT CAN BE MEANINGFUL
TO TEACHERS AND
STUDENTS AS REFLECTIONS
OF THEIR PARTICULAR
PRACTICES OF TEACHING
AND LEARNING?

23. WHAT’S THE
LEARNING
MODEL?
Image Credit: Torley via Flickr (CC BY 2.0)

24. MORE
IS
BETTER
Image Credit: Torley via Flickr (CC BY 2.0)

25. WE CAN
DO
BETTER!
Image Credit: Torley via Flickr (CC BY 2.0)

27. ONLINE DISCUSSION
LEARNING MODEL
Externalizing one’s
ideas by contributing
posts to an online
discussion
Taking in the
externalizations of
others by accessing
existing posts
• Social constructivist perspective - online discussions as a forum
for learning through conversation
• Students learn as they articulate their ideas, are exposed to the
ideas of others, and negotiate differences in perspective
• Focus on how students contribute comments (“speak”) and
attend to other’s messages (“listen”)

28. Speaking
 Mechanism for sharing ideas
 Value in speaking that is
 Recurring, responsive , rationaled
 Distributed temporally and
conversationally
 Moderately portioned
 While “speaking” is visible, not all
qualities are salient in the system
(esp. as related to time)
 Post quality info valuable, but
complex to assess
Listening
 Attending to the ideas of others is critical,
but “invisible”
 Value in listening that is
 Broad yet Deep (to consider multiple
ideas; predicts posts’ content quality)
 Integrated (so comments are informed by
others’ views)
 Recurrent (to provide context for
discussion flow; predicts responsiveness)
 Early research suggested universally poor
behaviors, but recent work shows
students listen in very distinct ways
 E.g. Disregardful, Coverage, Focused,
Thorough
ONLINE DISCUSSION
LEARNING MODEL

29. ONLINE DISCUSSION
LEARNING MODEL ANALYTICS

30. Criteria Metric Definition
Temporal
Distribution
Percent of sessions
with posts
Number of sessions in which a student made a post,
divided by their total of number sessions
Speaking
Quantity
Number of posts
Total number of posts a student contributed to the
discussion
Average post length
Total number of words posted by a student divided by the
number of posts they made to the discussion
Listening
Breadth
Percent of posts
viewed
Number of unique posts that a student viewed divided by
the total number of posts in the discussion
Percent of posts read
Number of unique posts that a student read divided by
the total number of posts in the discussion
Listening
Recurrance
Number of reviews of
own / others’ posts
Number of times a student revisited posts that they had
made / viewed previously in the discussion
Conversational
Distribution
Posts made / viewed
throughout discussion
Dispersion or concentration of posts made / viewed by a
student in the discussion space
ONLINE DISCUSSION
LEARNING MODEL ANALYTICS

31. PART 2: USING DATA TRACES
TO INFORM OUR TEACHING
AND LEARNING ACTIVITIES

32. HOW DO WE CONSIDER AND
DESIGN FOR WAYS IN
WHICH ANALYTICS CAN PLAY
A PART IN THE LARGER
ACTIVITY PATTERNS
OF INSTRUCTORS AND
STUDENTS?

33. A M O D E L F O R T E A C H E R S
–
C O N N E C T T H E U S E O F
L E A R N I N G A N A LY T I C S
T O T H E P R A C T I C E O F
L E A R N I N G D E S I G N
(LOCKYER, HEATHCOTE, & DAWSON, 2013)

34. KEY CONCEPTUAL QUESTIONS
1. WHAT ARE THE GOALS OF THE EDUCATIONAL
ACTIVITY? [WHAT IS THE POINT?]
2. WHAT DO PRODUCTIVE & UNPRODUCTIVE
LEARNING PROCESSES TO MEET THESE GOALS
LOOK LIKE? [WHAT IS THE PROCESS?]
3. HOW CAN THE AVAILABLE ANALYTICS SERVE
AS INDICATORS OF THESE?
[WHAT IS THE PROXY?]

35. • Purpose of engaging in [online discussions]
• Expectations for a productive process of engaging in
[online discussions]
• How the learning analytics provide a proxy for [this]
LINKING LEARNING ANALYTICS
& LEARNING DESIGN
articulating one’s ideas, being exposed to the ideas of
others, negotiating differences in perspective
attending deeply to a spectrum of others’ ideas, and
contributing comments that are responsive and rationaled,
percent of posts read introduced is a metric that has clear
meaning in the context of the activity

36. Metric Student 1
(Week X)
Student 2
(Week X)
Class Average
(Week X)
Range of participation 2 days 6 days 5 days
# of sessions 8 3 11
Average session length 13 min 48 min 39 min
% of sessions with posts 35% 67% 49%
# of posts made 9 4 7
Average post length 126 words 386 words 216 words
% of posts read 42% 87% 75%
#of reviews of own posts 2 22 13
#of reviews of others’ posts 3 12 8
ONLINE DISCUSSION
LEARNING ANALYTICS

37. ONLINE DISCUSSION
LEARNING ANALYTICS

38. A M O D E L F O R S T U D E N T S
–
C O N N E C T T H E U S E O F
L E A R N I N G A N A LY T I C S
T O T H E P R A C T I C E S O F
S E L F - R E G U L AT E D
L E A R N I N G
(WISE, 2014)

39. WHY FOCUS ON STUDENTS AS USERS OF
LEARNING ANALYTICS?
ENGAGE THEM AS ACTIVE PARTNERS IN LEARNING
ABILITY TO MAKE IMMEDIATE LOCAL CHANGES
ACTIVATE METACOGNITIVE PROCESSES
EMPOWERMENT NOT ENSLAVEMENT
DEMOCRATIZE ACCESS TO DATA
ONE-TO-ONE RATIO AT ANY SCALE

40. STUDENT
TUNING MODEL
ALIGN
DESIGN FRAMEWORK

41. Integration (technological and pedagogical) made analytics a
coherent part of the learning process
Students embraced agency in setting (often recurring) personal
goals and evaluating their progress, no “big brother” issues
Individual, peer, and instructor reference frames were important for
making sense of the data; reactions were both cognitive and
emotional
Reflection on data a powerful starting place
Concrete and proximal goal-setting is harder
Change happens slowly, isn’t always intentional, requires support!
STUDENT ANALYTICS USE
INITIAL FINDINGS (!)

42. DISTINCT CHANGE PROFILES

43. HOW CAN YOU USE
ANALYTIC DATA TO USEFULLY
REFLECT & INFORM
YOUR TEACHING
AND THE
LEARNING PRACTICES OF
YOUR STUDENTS
?

44. Image Credit: enjosmith via Flickr (CC BY 2.0), adapted
Learning Analytics & Educational Research
Leveraging Big Data in Powerful Ways