How can a university that already has very high levels of student performance and retention use data from its Blackboard® learning management system to identify effective teaching practices and at risk students? Based on experience gained from a year-long pilot of Blackboard Analytics™ for Learn at Emory University, this presentation will discuss (1) several unique challenges associated with the use of Blackboard Analytics™ to monitor high performing students, (2) the value of Blackboard Analytics™ as a data warehouse against which to run custom queries and apply more sophisticated data mining techniques, and (3) several preliminary insights obtained through the application of those techniques at Emory University.
11. Founded in 1836
11 Schools
(4 Undergraduate, 7 Graduate)
14,513 Students
(7,836 Undergraduate, 6,677 Graduate & Professional)
Average Freshman Retention Rate: 95.3%
4-year Graduation Rate: 84%
94% College Success Rate (<> DFW)
US News National University Ranking #20
Fact & Figures as of Fall 2013 (http://www.emory.edu/home/about/factsfigures/)
#EmoryLA
#BbWorld14
A destination university internationally recognized as an inquiry-driven, ethically engaged,
and diverse community, whose members work collaboratively for positive transformation in
the world through courageous leadership in teaching, research, scholarship, health care,
and social action.
14. “If you invest in educational
technology for the sake of fashion,
you might just find yourself
wearing nothing at all”
14
#EmoryLA
#BbWorld14
15. An Opportunity
Learning Analytics in Support of Emory’s Strategic Vision
“Ensuring Highest Student Quality and Enhancing
the Student Experience”
“Strengthening Faculty Distinction”
15
#EmoryLA
#BbWorld14
21. Challenges
3. Unexpected consequences
21
#EmoryLA
#BbWorld14
“I believe it's important for professors to have this tool
available to them, so they can see early if we're not turning
up for class enough and intervention might be warranted. I
do know that I need to get in a habit of logging onto BB at
least twice a day. I need to make it part of my daily life”
“I liked being able to compare my activity to
others, but I don't think it changed my
behavior. It may be useful to others.”
“I think its useful if someone is concerned
about how they are doing in the class or if
they are uncertain about how much time
they should be committing to the class. I
personally did not find it useful but I can see
how someone may use it as a tool to
determine if they are meeting expectations”
“I think this system also puts undue pressure on students to
log in or click on specific items in blackboard more, without
taking into consideration factors such as work done offline
in the class or via email. This is especially true for distance
learners. I personally cut and paste many of my assignments
in Word so they are easier to view and manipulate. I also
like to keep files from each semester so I do a considerable
amount of work offline and I am doing very well in my
classes so far. When I logged in to check my activity, the data
I saw pretty much showed the opposite-it looked like I
wasn't very engaged at all.”
“If this is a tool that will be used by
Professors to grade participation I would
have likely used it more; however, if this tool
were indeed used by Professors it may cause
students to post unnecessarily.”
24. Preliminary Insights
Blackboard Use
Blackboard use at Emory University is far higher than
previously thought
24
#EmoryLA
#BbWorld14
62%
38%
Course Items Added No Course Items Added
5% 5%
0%
25%
50%
75%
100%
Course Items Added No Course Items Added
DFW Rate by Blackboard Use
Course Items Added No Course Items Added
… but it has no effect on student success
25. Preliminary Insights
Course Design
Engaged instructors make successful students
25
#EmoryLA
#BbWorld14
5.3%
6.1%
4.8% 4.8%
0%
5%
10%
Files Items
DFWRate
Items vs Files
5.3% 5.4%
1.0%
4.0%
0%
5%
10%
Course Links External URLs
DFWRate
Internal Links vs External URLs
Not Used Used
26. Preliminary Insights
At Risk Students
26
#EmoryLA
#BbWorld14
6%
4%
0%
5%
10%
15%
20%
25%
Male Female
DFW Rate by Gender
9%
4%
0%
5%
10%
15%
20%
25%
Transfer Student Non-Transfer Student
First Year DFW Rate by Transfer Status
3% 4%
7%
6%
0%
5%
10%
15%
20%
25%
Freshman Sophomore Junior Senior
DFW Rate by Class
29. Next Steps
1. Data Wrangling Approach
2. Community of Interest
3. Workshops
4. Learning Analytics Speaker Series
5. Georgia Learning Analytics Symposium
29
#EmoryLA
#BbWorld14
30. Timothy D. Harfield
Scholar in Residence (Learning Analytics)
timothy.harfield@emory.edu
@tdharfield
Please provide feedback for this session:
1. Select “Schedule” icon in BbWorld14 Mobile App
2. Find Challenges and Opportunities for Promoting
Success among the Successful using Blackboard
Analytics™
3. Click “Tap here to take a survey.”
THANKS!