Presentation at 8th International Technology, Education and Development Conference (INTED 2014), 10th–12th March 2014, Valencia

1. Massive Open Online Courses
in Higher Education –
Performance Assessment in Open Learning
Arrangements
8th InternationalTechnology,
Education and Development Conference,
10th-12th March 2014,Valencia
Presenting author: Daniela Pscheida,TU Dresden, Germany
Contact: daniela.pscheida@tu-dresden.de
Twitter: @danielapscheida

2. cMOOCs in Higher Education
(c = connectivist)
Connectivist MOOCs are Open Online Courses based on the
online activity and thematic exchange of learners
instead of broadcasting video lectures.
That‘s why this format has high potential for higher education –
as it supports the training of important soft skills for the 21st century:
 media literacy
 experiences in personal learning
 creation of personal knowledge networks

3. Challenges of cMOOCs in
Higher Education
The implementation of a cMOOC learning scenario is
connected to certain challenges:
 curricular integration, e.g. if there are participants from different
universities and faculties
 technological and media capabilities, e.g. if there are participants
with different levels of experience
 supervision and motivation of students
 documentation and assessment of learning performance

4. PerformanceAssessment in cMOOCs
Two incompabilities between connectivist learning and
higher education:
 The context of higher education demands a complete collection
and transparent assessment of learning outcomes along
predefined learning objectives to award Credit Points – while
connectivism encourages learners to focus on their own learning
interests and needs (openness of learning process).
 The core idea of connectivism assumes intrinsically motivated
learners – while student learners are used to receive continuous
feedback and orientation (guidance) from lecturers.

5. The SOOC13
SOOC = Saxon Open Online Course
 learning innovation project by three German universities
(Dresden, Chemnitz, Siegen)
 nine-week course in summer term 2013, 242 registered participants, 122
students
 topic: „Learning 2.0 – Individual learning and knowledge management
with social media“
 course organization via central weblog (www.sooc13.de), materials and
impulses
 blog aggregator to collect course-related blog posts from participants
and tweets with hashtag #sooc13
 expert talks and live sessions via Adobe Connect

6. Tasks and requirements
The SOOC13
 regular participation in live
sessions (expert talks)
 writing and publishing at
least one blog post in each
thematic section
 e-portfolio (collection of
the results of research and
reflection tasks and
smaller experiments)

7. Performance recording and feedback
The SOOC13
 first: RSS feed aggregator
 later we additionally introduced: e-portfolio-form
 support by two student tutors
 likes and individual comments on participants‘ contributions –
to stimulate further discussions and debates
 no feedback for assessment purposes during the course run

8. Performance assessment
The SOOC13
 no exam
 assessment based on different course activities, especially the
collected e-portfolio contributions
 general criteria: spelling, grammar, language, comprehension,
compliance with scientific standards
 quality of content: understanding and correct application of
technical terms and theories, etc.
 ‚soft‘ criteria: aspects arising from self-directed learning and
connectivism, e.g. reflectivity and referring back

9. Evaluation of the SOOC
Survey conducted as online questionnaires at the beginning, in the
middle and at the end of the course and
virtual evaluation workshop after the course in the form of an online
written discussion
 Results presented here summarize
the interim and final evaluation of the sooc13 as well as
the interim evaluation of the sooc1314.

10. „Do you think you complied with the requirements of
the course?“
Evaluation Results from SOOC13
60%
23%
17%
0% 10% 20% 30% 40% 50% 60% 70%
Yes
No
I cannot say
N = 30
 more than one third of
participants were unsure
about or did not think so
 the main reason for
uncertainty regarding the
own performance was a
lack of feedback (60%)

11. „Are you satisfied with the assessment system of the
SOOC13?“
Evaluation Results from SOOC13
With 41% of participants being
uncertain about that and
another 21% saying „no“,
there‘s a good chance for
improvement.
38%
21%
41%
0% 10% 20% 30% 40% 50%
Yes
No
I cannot say
N = 29

12. Improvement of cMOOC assessment system by:
 an instrument, that allows continuous feedback for contributions
 increase the transparency in terms of the assessment criteria
... to avoid uncertainty among students
... to raise the quality of discussions
 introduce digital badges for the second course run
Lessons Learned

13. Digital badges in SOOC1314
Digital Badges
 as a formative measurement instrument
 „small“ badges for each reported contribution of the e-portfolio
form in states:
bronze
silver
gold
 each badge was combined with a brief verbal judgement
 to pass the course at least a bronze badge had to be received at
any given task

14. Evaluation Results from SOOC1314
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
The portfolio-form is easy to use and well structured.
I understand the significance of the badges.
For me the literal assessments clarify the significance of the
badges.
For me the Badges and the literal assessments are transparent
methods of evaluation.
N = 20
I fully agree
I somewhat agree
I cannot say
I rather disagree
I do not agree at all

15. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
The evaluation with badges helps me, to appreciate my own
achievements.
The volume of the literal assessment is appropriate.
The evalution method with badges and literal assessments is fair.
The literal assessment show me possibilities, where I can improve next
time.
N = 20
always
often
sometimes
rarely
never
Evaluation Results from SOOC1314

16.  Evaluation results from the SOOC1314 survey prove, that the
new assessment system with badges and verbal judgements is
much more transparent and understandable – which also led to
an increased satisfaction of participants.
Conclusion
 Further innovations at the horizon:
using methods of learning analytics to improve the overview
and objectivity of a cMOOC in higher education.