This presentation looks at the assessment concepts underpinning the Mobler app and focusses on the implications of using mobile devices for assessing the students' learning in higher education. These slides were presented at the Eduhub SIG E-Assessment Meeting Nov. 26, 2015

1. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Lessons Learnt from using the Mobler App:
From Test Questions
Towards Mobile Formative Assessments
Marion R. Gruber & Christian Glahn
with support of Olga Tartakovski (IPMZ)
SIG E-Assessment 2015

2. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Our Main E-Assessment Challenge
Need for reliable and scalable assessment of
student learning
Summative assessments are not reflecting the
formative nature of learning
• Blended Learning
• Flipped Classroom

3. Faculty of Arts and Social Sciences, DLF Blended Learning Center
How can we extend our students’ learning time?

4. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Design Principles for Mobile Learning Apps
• Integration with the overall learning process
• Extend and enrich the student’s learning
experience
• Create and bridge into richer learning settings

5. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Mobler App
for iOS and Android

6. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Additional Features

7. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Supported Question Types
• Single and Multiple Choice
• Sorting Questions
• Cloze Questions
• Basic Numeric Questions
• (KPrim Questions Work in Progress)

8. Faculty of Arts and Social Sciences, DLF Blended Learning Center
System Architecture
(Glahn, 2013)

9. Faculty of Arts and Social Sciences, DLF Blended Learning Center
System Architecture
(Glahn, 2013)

10. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Data Cockpit for Lecturers

11. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Course Layout in Ilias
Course
Active
Question pool
Inactive
Question pool

12. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Why Question Pools and not Tests?
• Tests have an intrinsic testing function
• Inherent logic of questions and responses
• Result oriented
• Inner structure
• Question Pools have an organising function
• Collections of questions
• Not result oriented
• External structure

13. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Types of Possible Assessments

14. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Possible Assessment Forms
Gaps
Rhythm

15. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Possible Assessment Forms
Gaps
Rhythm

16. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Endless Opportunities for
Learning Designs!
http://de.slideshare.net/phish108/2013-mobile-blendedlearningmoblercardsen

17. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Re-purposing of existing
Learning Resources!

18. Faculty of Arts and Social Sciences, DLF Blended Learning Center
This sounds too good to be true ...

19. Faculty of Arts and Social Sciences, DLF Blended Learning Center
We gave Mobler to University Students
• Introductory lecture on „Mass Communication and Media
Research“ (PuK I&II) at the UZH IPMZ
• 410 participants, of which 125 used the app frequently
• 13,192 answered questions in the app
• Evaluated the student behaviour
• Listened to the students feedback
• Summative evaluation after the end of the pilot

20. Faculty of Arts and Social Sciences, DLF Blended Learning Center
The Learning Design
Lorem ipsum …Lorem ipsum …
Lorem ipsum …
E.1
E.1+2
E.1+2+3
E.1+2+3+4
Lorem ipsum … Lorem
ipsum
…
Lorem
ipsum
…
Lorem
ipsum
…

21. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Smart Phone Adoption
72%
26%
2%
iOS
Android
Windows
n=50

22. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Mobile Device Ownership and Use
The students use several mobile devices
• 92% Laptops
• 48% Tablets
• 64% use 3 or more devices on a regular basis
n=50 (multiple responses )

23. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Common Practices
Daily used apps and services
• 94% texting
• 74% online search
• 72% social networking
• 62% news and blogs
• 61% e-mail
• 52% photos
Weekly used apps and services
• 52% navigation
• 52% telephony
• 50% camera
• 44% videos
n=50 (multiple responses)

24. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Mobile Technologies are like
Pen and Paper to our Students
http://jeffberezny.com/2014/02/18/why-ephemeral-social-networks-are-here-to-stay/

25. Faculty of Arts and Social Sciences, DLF Blended Learning Center
How did the students use Mobler?

26. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Application Usage
Weekly app sessions during the term (n=125)

27. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Didactical Factors (6 point Likert-scale)
• No unnecessary workload was created (µ = 5.9)
• Useful addition to online and lecture hall activities
(µ = 5.5)
• The app should replace conventional online
activities (µ = 3.4)
• Would use the app only if it was compulsory
(µ = 1.5)
• More frequent use if deeper integration with the
curriculum (µ = 3.2)
n=37

28. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Usage Context
n=37

29. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Motivational factors
• Helps to learn more frequently (57%)
• Motivates them to learn more intensively (51%)
• Recommend using the app to fellow students (91%)
• Use improved app in other courses (86%)
n=37

30. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Conclusions
• The majority of today’s students learn in a multi-device ecology
• Gamification elements were completely irrelevant
• Mobile learning activities can enrich the learning experiences
beyond having access to functions on a smart phone
• Bite-sized learning on smart phones supports learners in bridging
learning into uncommon learning settings (e.g., public transport)
• The students appreciate the mobile app, but don’t consider it as a
replacement for other learning solutions
• The students value the authorized content provided for learning
• Many students wait until the last moment to initiate their learning

31. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Present and Future Steps
• Ongoing improvements based on the students’ feedback
• Broader LMS Support (i.e. Moodle support)
• Second pilot with in PuK I&II at the UZH
• First pilot in Banking at the HTW Chur
• Second round of evaluations in January 2016

32. Faculty of Arts and Social Sciences, DLF Blended Learning Center
Thank you!
Mag. Dr. Marion R. Gruber
Universität Zürich, Philosophische Fakultät, DLF
marion.gruber@phil.uzh.ch
@em3rg3 @UZHphilDLF
M.A. Olga Tartakovski
Universität Zürich, Philosophische Fakultät, IPMZ,
o.tartakovski@ipmz.uzh.ch
Dr. Christian Glahn
HTW Chur, Blended Learning Center
christian.glahn@htwchur.ch
@phish108 @htwblc