Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Reall: rubric for the evaluation of apps in language learning
1. REALL: Rubric for the
Evaluation of Apps in
Language Learning
Elena Martín Monje (UNED), Jorge Arús Hita (UCM),
Pilar Rodríguez Arancón (UNED) & Cristina Calle Martínez (UCM)
ATLAS Research Group
Logroño, 2-3 May 2013
2. Table of contents
Introduction
Pedagogic assessment of mobile learning apps for EFL
◦ Stage 1: Analysis and categorization of EFL apps available
◦ Stage 2: Design of an evaluation rubric for the pedagogic
assessment of EFL apps
◦ Stage 3: Rubric for the evaluation of apps in language
learning (REALL)
Conclusion
References
3. Introduction
Mobile learning
Mobile-Assisted Language Learning (MALL)
SO-CALL-ME project
Stages in this research study:
Stage 1:
Analysis and
categorization
of EFL apps
available
Stage 2:
Design of
rubric for
pedagogic
assessment of
EFL apps
Stage 3:
Creation of
REALL (Rubric
for the
Evaluation of
Apps in
Language
Learning
4. Previous rubrics & categorizations
Taken from http://langwitches.org/blog/2012/03/31/ipad-apps-and-blooms-taxonomy/
6. Stage 1: Apps available
3 criteria: apps’ cognitive value, similarity/
complementarity with SO-CALL-ME
67 apps assessed
Frequent technical problems downloading/ starting apps
Vast majority for Apple devices
Most expensive: dictionaries, grammar
Many for 2-3€ or with initial free sample pack
Categories:
◦ Games
◦ App versions of printed materials (e.g. coursebooks)
◦ Apps for pronunciation/ grammar/vocabulary
7. Stage 2: Rubric for pedagogic
assessment
Quality guide for digital learning objects (Fernández-
Pampillón et al., 2011)
• Cognitive value and pedagogic
coherence (1)
• Content quality (2)
• Capacity to generate learning (3)
• Interactivity and adaptability (4)
• Motivation (5)
Pedagogical
criteria
• Format and layout (6)
• Usability (7)
• Accessibility (8)
• Visibility (9)
• Compatibility (10)
Technical
criteria
8. Rubric with descriptors and scale 1-5.
E.g. Criterion 3 in rubric for pedagogic assessment:
The 5 top EFL apps of Stage 1 assessed with this rubric:
1 2 3 4 5
3. Capacity to
generate
learning
Contents do not
help to achieve
learning goals
or autonomous
learning
Contents help
autonomous
learning but not
clearly the
achievement of
the initial
learning goals
Contents help
to achieve the
learning goals
but neither
autonomous
learning nor
relating old
knowledge to
new knowledge
Contents help
to achieve the
learning goals
but not
autonomous
learning OR not
relating old
knowledge to
new knowledge
Contents help
to achieve the
learning goals,
autonomous
learning and
relating old
knowledge to
new knowledge
9.
10. Stage 3: Rubric for linguistic
assessment
Common European Framework of Reference for Languages
(Council of Europe, 2001)
Focus on levels A2-B2
Initial rubric for oral
comprehension
Categories:
Level
Type of texts
Topics
Delivery
E.g. Delivery in rubric for linguistic assessment:
1 2 3 4 5 N/A
Delivery
Language
difficulty, clarity
and speed mix
different levels.
If adaptive,
delivery is
clearly not well
adapted
Language difficulty,
clarity and speed
rarely belong to the
same level. If
adaptive, delivery
rarely corresponds
to the right level
Language difficulty,
clarity and speed
tend to belong to
the same level. If
adaptive, delivery
more often than not
corresponds to the
right level
Language difficulty,
clarity and speed
usually belong to
the same level. If
adaptive, delivery
usually
corresponds to the
right level
Language
difficulty, clarity
and speed belong
to the same level.
If adaptive,
delivery
corresponds to
the right level
12. The most comprehensive apps assessed were Speakingpal
and Learn English Elementary Podcasts
13. Conclusions
Pedagogic & technical quality ≠ linguistic value &
adequacy for EFL teaching & learning
Importance of all three dimensions:
◦ Pedagogical
◦ Technical
◦ Linguistic
Piloting of stage 2 & stage 3 rubrics enables a future
study on a larger scale
Future theoretical framework for successful,
pedagogically & linguistically sound EFL apps
14. References
Arús-Hita, Jorge; Pilar Rodríguez-Arancón and Cristina Calle-
Martínez (in press) “A pedagogic assessment of mobile learning
applications”. In Proceedings of ICDE 2013, Mobilizing Distance
Education, UNED, Madrid.
Council of Europe (2001). Common European Framework of
Reference for Languages. Cambridge: Cambridge University
Press.
Fernández-Pampillón Cesteros, Ana; Elena Domínguez Romero
and Isabel de Armas Ranero (2011) Herramienta para la revisión
de la Calidad de Objetos de Aprendizaje Universitarios (COdA):
guía del usuario. v.1.1. Madrid: e-prints Complutense. Retrieved
from http://eprints.ucm.es/12533/ (accessed 18/04/2013).