SKERG Seminar on Aug 18, 2015 titled, "Designing and Evaluating a Contextual Mobile Application to Support Situated Learning" by Dr. Abeer Ali Alnuaim, the Vice Chair of the Natural Sciences and Engineering Department at KSU. Synopsis: This research emerged from seeking to identify ways of getting Human-Computer Interaction Design students into real world environments, similar to those in which they will eventually be designing, thus maximising their ability to identify opportunities for innovation. In helping students learn how to become proficient and innovative designers and developers, it is crucial that their ‘out of the classroom’ experience of the environments in which their designs will be used, augments and extends in-class learning. The aim of this research is to investigate the process of designing a mobile learning application in a blended learning model. This app was designed to support students in a design task and to develop their independent learning and critical thinking skills, as part of their Human-Computer Interaction coursework. It explores the challenges in implementing and deploying such an app in the learning context. A number of evaluations were conducted to assess the design, usability and effectiveness of the app. Promising results show that the app has helped students in developing critical skills for designing technology. However, there were a number of concerns discovered regarding the context of use of a mobile device, including usability of interface elements and acceptability of using the app in a public place.

1. Designing and Evaluating a
Contextual Mobile Application
to Support Situated Learning
Abeer Alnuaim, PhD
Abalnuaim@ksu.edu.sa

2. Testing
Effectiveness of the App (HCI) In-Context Evaluation (DUE)
Development Methodology
Four Phases
HCI Teaching
Motivation Aim

3. HCI Teaching
Human-computer Interaction studies the way people
interact with computers in a particular context and
evaluates the extent to which these computer based
systems are, or are not, designed for successful
interaction.
Teaching interaction design is a challenging task.

4. HCI Teaching
Applying empathic design strategies when designing
aids in developing a product that pleases the user
(McDonagh and Thomas, 2010) .
Thus, immersing students into real would environments
to gather requirements could generate empathy and
thus designing a product that is related to the users’
needs.

5. Motivation & Aim
• This research emerged from seeking to identify ways
of getting Human-Computer Interaction Design
students into real world environments.
• Mobile learning applications can provide
contextual information that could help students stay
focussed on the purpose and outcome of the
activity, rather than being distracted by the
process.
• Mobile Learning is all about personalisation.
• Aim: To investigate the process of designing a
mobile learning application in a contextual mobile
learning model.

6. A contextual mobile
learning model

7. HCI’s Students’
Coursework
It involved designing a GUI for a touch-screen based kiosk
to be installed in the university’s cafeteria to offer support to
students and staff, helping them make the right meal
choices.
The assignment was structured as a group project involving
three or four students, where the initial work consisted of
requirements gathering and analysis to produce a set of
artefacts such as a PACT analysis personas and scenarios,
and a set of functional and non-functional requirements.
The assignment deliverable was an in-class presentation of
their work.

8. Development
Methodology
The development process has been identified in
phases. These Phases were derived according to the
User-Centred Design Process (UCD) in the field of HCI.
In UCD. The phases for this study were as follows:

9. Phase One: Requirements
Gathering

10. Phase One Key Findings:
• Students lose focus on the purpose of tasks when away
from classroom. They may get distracted by their
surroundings and miss out key elements.
• Some students have been found to struggle in analysing
their findings and specifically in using their findings to
develop new ideas.
• Students care about their privacy and would not easily
compromise it.
• Android smartphones are poplar among students.

11. Phase 3 &4 : Design and
Evaluation
• Designing a contextual mobile learning application
requires consideration of a number of issues. These
include students’ different learning styles and
preferences, the location’s characteristics and its
physical and psychological effect on the user, as well as
the appropriateness of the location-specific content.
• An iterative design approach was followed.

12. Detailed activities within the
iterative development process

13. Iteration 5

14. Evaluating the
effectivness
Vavoula and Sharples’s (2009) three-level framework for
evaluating mobile learning was used.

15. Participants
• There were 55 students enrolled in the HCI module,
seven females and 48 males.
• Students were allowed to form self-selecting groups of 3
or 4.
• This resulted in 17 groups; however, only 16 groups
presented their work.
• 3 groups borrowed the university’s HTC desire phones,
• 10 groups used their own phones
• While 3 groups neither borrowed or used their own.

16. HCI’s Deployment
Methodology

17. Presentation Results
Most groups did very well in their assignment.
The average class mark for this cohort for the
elements that were supported by use of the app is
77.32%, which is above the cohort average mark
for the assignment as a whole which is 66.15%.

18. Questionnaire
• 23 students filled in and returned the paper
questionnaire.
• The mean SUS score based on all the
responses was 69; this is above the average
SUS score.

19. Pedagogical Usability
Statements
N
Mean
Scores
The app helped me in my
observation
23 4.17
The app gave me hints on
what to look for
22 4.36
The app helped me
organise my ideas
22 4.18
It was helpful to have a
space for note taking
23 2.43
The app helped our group
members to share ideas
and notes
23 3.74
The Forum (Blog) within
the app was useful
18 3.00
It was useful to track my
progress through Profile
21 3.71
The app helped me
develop ideas for PACT
22 4.36

20. Findings
• overall engagement with the observational
requirements activity, as well as the quality of the
students’ insights and requirements emerging as a
consequence of their observations supported by
the use of the sLearn, indicates that the app has
had an effective impact on student learning.
• The approach used in immersing the student in real
environments, can help them view the situation
from the perspective of the user, hence helping
them to generate empathy.

21. Findings
Challenges in designing and evaluating mobile
learning in this context:
• group dynamics
• ownership of mobile devices
• willingness and motivation to engage or try
something new
• intrinsic ability
• the novelty value of the app, and the usability of
the interface versus the helpfulness of the content.

22. In-Context Evaluation
To accurately assess the use and value of this mobile
learning app for higher education students, it was
necessary to conduct in-situ evaluations in the
environment of its intended users.
Moreover, it is critical to understand what might
influence the usability and the user experience of this
app in such a busy environment.

23. Participants
• MSc students enrolled in the ‘Designing the User
Experience’ Module
• They are familiar with concepts of HCI.
• 7 out of 30 students participated in the evaluation,
five males and two females.

24. ObservationsResults
• Hesitation or confusion in general (At the
beginning).
• Confused whether to save the notes first or post
then save.
• Did not know how to remove the keyboard.
• Conscious that staff will notice that he/she is not
here for buying food.

25. Interview
Theme One: General usability
1. Instructions: more explicit instructions should be
available to students at the beginning.
2. Redesign: all participants prefer that under each
prompt there should be a text box to write their
notes and observations.
Theme Two: The Context
Personality and self-consciousness: 28.57% of the
participants did not feel comfortable looking around
at people and writing on the phone.

26. Discussions
The results of the individual evaluations were
considered as a whole, then they were grouped into
three main overlapping categories relating to:
(1) Design and GUI of the app,
(2) Usability, User experience and Students’
Perspectives, and
(3) Designing and deploying a Blended Learning
Model.

27. Guidelines for implementing a mobile application
for situated learning activities in HE
Guideline
Accessible from the learner’s mobile
device- Multi platform compatible
Suitable Contextual Content
Provide Independent Choices
Multimodal Interaction
Collaborative facility
Clear Instructions

28. What is next?

29. References
• Alnuaim, A. (2015) Designing and evaluating a
contextual mobile learning application to support
situated learning. PhD, University of the West of England.
• MCDONAGH, D. and THOMAS, J., 2010. Disability
relevant design: Empathic design strategies supporting
more effective new product design outcomes. The
Design Journal, 13(2), pp. 180-198.
• VAVOULA, G. and SHARPLES, M., 2009. Meeting the
Challenges in Evaluating Mobile Learning: A 3-Level
Evaluation Framework. International Journal of Mobile
and Blended Learning, 1(2), pp. 54-75, Copyright, IGI
Global. Reprinted with permission of the publisher.