ISELEAR Presentation at XVI International Symposium on Computers in Education (SIIE 2014), November 13, 2014, Logroño, Spain.

1. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Domain-driven competence assessment in virtual learning
enviroments. Application to planning and time management
skills
Antonio Balderas, ´Alvaro Gal´an-Pi˜nero
J. Antonio Caballero Hern´andez, Gregorio Rodr´ıguez G´omez
Juan Manuel Dodero, Manuel Palomo-Duarte
Universidad de C´adiz
November, 2014
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

2. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Content
1 Context & issue
2 ReST web service
3 Model-Driven approach
4 Case study
5 Conclusions and future research
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

3. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Table of contents
1 Context & issue
2 ReST web service
3 Model-Driven approach
4 Case study
5 Conclusions and future research
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

4. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Higher education: focus has turned from knowledge to skills
Relevance on developing subject-specific knowledge
Development of generic competences
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

5. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Learning Managment Systems (LMS) provide teachers with facilities to
manage courses
How to assess generic skills?
Assessment through grading
Difficult to link learning activities with skills in curricula
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

6. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Can assessment tools support teachers to evaluate generic skills?
This work presents two alternative answers:
1 A ReST-based web app for (manual) assessment of skills
2 An Model-driven approach for (computer-assisted) assessment of skills
Two LMS-independent computer tools were built up and adapted for
Moodle
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

7. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Table of contents
1 Context & issue
2 ReST web service
3 Model-Driven approach
4 Case study
5 Conclusions and future research
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

8. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
GesCompEval: ReST-based web service & app for learning assessment
Web interface to link skills and learning outcomes with learning
activities and vice versa
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

9. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Integrated in Moodle by the GesCompEval MD extension
Uses the EvalCOMIX web service for skill assessment
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

10. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Table of contents
1 Context & issue
2 ReST web service
3 Model-Driven approach
4 Case study
5 Conclusions and future research
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

11. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
EvalCourse design follows the MDA approach
Eclipse Modeling Framework (EMF) project based on a domain
meta-model to produce an Rich Client Platform (RCP) desktop
application
Users are teachers with or without computer programming skills
EvalCourse outcomes are in different formats
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

12. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
EvalCourse SASQL: a Domain-Specific Language (DSL) designed to
query the LMS for online learning assessment
Evidence name of the indicator:
get students
show milestones | participation | access | interaction
in assignment | workshop | forum | campus | wiki [list ids]
from course id | backup.
SASQL syntax
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

13. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Table of contents
1 Context & issue
2 ReST web service
3 Model-Driven approach
4 Case study
5 Conclusions and future research
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

14. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Case study developed in an course on Functional Programming
Computer Science Degree in the University of Cadiz during the
2013-14 academic year
Students’ assessment of planning and time management skills
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

15. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
First, EvalCOMIX+GesCompEval used to provide explicit grades for
each skill
e.g. submission time is assessed as a single attribute with 4 values to
measure time management skills
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

16. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Evidences of the skills are marked up in the activity (e.g. forum)
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

17. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Evidences and grades are reported
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

18. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Table: Grades obtained with GesCompEval
Time management Knowledge in practical situations
anon4 91.75 94
anon5 50.50 94
anon7 66.67 96
anon8 58.75 94
anon9 42.00 100
anon10 25.25 100
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

19. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Second, EvalCourse was used in the same course
1st aspect of analysis: checking if students delivered their assignments
on time, delayed, or had pending assignments
SASQL query:
Evidence milestone list:
get students
show milestones
in assignment.
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

20. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

21. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Table: List of students’ assignments
Name Ratio Milestones In-time Pending Access
anon4 100 4 4 0 218
anon5 100 4 4 0 78
anon7 50 3 2 1 135
anon8 100 4 4 0 171
anon9 75 4 3 0 149
anon10 50 4 2 0 117
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

22. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
After, query issued to find out students’ participation in (e.g. forum)
activity
SASQL query:
Evidence forum interaction:
get students
show interaction
in forum.
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

23. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Table: Students’ participation in forum activity
Name Debate starter Debate participation Total Access
anon4 1 2 3 64
anon5 3 1 4 36
anon7 1 3 4 75
anon8 0 0 0 48
anon9 0 2 3 28
anon10 3 1 4 41
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

24. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Comparison: in Gescompeval, anon4 has a higher grade, the rest
have less than 67
Table: Grading obtained by Gescompeval
Plan and manage time Knowledge in practical situations
anon4 91.75 94
anon5 50.50 94
anon7 66.67 96
anon8 58.75 94
anon9 42 100
anon10 25.25 100
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

25. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Comparison: in EvalCourse, considering only the activities
delivered on time, anon4+anon5+anon8 achieved the max for
this skill
Table: Assignments of the students
Name Ratio Milestones In-time Pending Access
anon4 100 4 4 0 218
anon5 100 4 4 0 78
anon7 50 3 2 1 135
anon8 100 4 4 0 171
anon9 75 4 3 0 149
anon10 50 4 2 0 117
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

26. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Comparison: but considering the number of tasks submitted on
time and a number of accesses above the average, only anon4
fullfills both indicators
Table: Assignments of the students
Name Ratio Milestones In-time Pending Access
anon4 100 4 4 0 218
anon5 100 4 4 0 78
anon7 50 3 2 1 135
anon8 100 4 4 0 171
anon9 75 4 3 0 149
anon10 50 4 2 0 117
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

27. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
These are just examples of (valid?) interpretations the teacher could do
with the reported indicators
The validity of applying certain indicators to assess a particular skill is
outside of scope of this work
Deeper research required, but it provides technical support to do it
As long as teachers manage to provide accurate indicators, they can
perform ’objective’ assessment of skills in massive online courses (e.g.
MOOCs) through EvalCourse
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

28. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Table of contents
1 Context & issue
2 ReST web service
3 Model-Driven approach
4 Case study
5 Conclusions and future research
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

29. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Generic skills enable succesful integration of students in authentic
learning environments
But in regular academic contexts, assessing skills is complex and
somewhat subjective
Two alternatives to perform skill assessment:
A manual skill assessment approach + GesCompEval ReST-based web
service & app)
A computer-assisted assessment approach + EvalCourse model-driven
approach
GesCompEval + EvalCourse can be applied to assess skills of students
in terms of their interaction with the LMS or other
computer-supported learning environments
EvalCourse is aimed for courses with a great number of students
and/or activities
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

30. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Ongoing and future work:
EvalChrome: a Chrome plugin that allows to assess Moodle activities
that are developed on external web pages (e.g. a wiki) and mark-up
evidences of assessments for further feedback
Connector to link EvalCourse and Mediawiki under development
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management

31. Context & issue ReST web service Model-Driven approach Case study Conclusions and future research
Thanks for your attention
GesCompEval:
https://www.assembla.com/spaces/inteweb-gescompeval
EvalCourse: https://www.assembla.com/spaces/evalcourse
Antonio Balderas et al.
Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management
Aknowledgements: