1. A Life-Cycle Model for
the Quality Evaluation
of Educational Content
A. Kameas A. Stefani

2. Higher Education challenges
 A new stage is set for the transition to the
paperless University;
 Education Institutions’ dual role:
publishers and distributors of educational
content.
 Challenge: Standardization of design
and production procedures

3. Research Challenge and Methodology:
Summary
 Research question
 how to design digital educational content
 suitable for open and distance learning
 based on formal Q&A practices.
 Methodology
 a top-down analysis of Q&A needs through the
definition of a content life-cycle model;
 a bottom-up synthesis of a standard-like hierarchy
through the definition of certification guidelines.
 Goal:
 to map needs, stakeholders, and quality attributes to
quality characteristics, sub-characteristics and metrics
of an ISO-like standard.

4. Digital Educational Content
 We refer to
‘Digital
Educational
Content’ in the
form of:

5. State of the Art (1/3)

6. State of the Art (2/3)

7. State of the Art (3/3)

8. Research challenges (1/2)
 Educational content and learning processes
are closely interconnected.
 There should exist specifications that allow:
 compatibility with the educational processes,
 to achieve the maximum out of the learning
outcomes,
 reduction of costs during content use (ideally
without compromising educational quality),
 reuse of the content.

9. Research challenges (2/2)
 There is a need for a clear grouping of
 processes (what should be done),
 content (to what) and
 stakeholders (by whom);
 Design a quality life cycle (QLC) for educational
content:
 adoption of (some) characteristics of quality
standards (formality),
 eliciting good practices through the application of
benchmarking methods (practicality),
 coverage of all types of educational content
(completeness).

10. QLC Design Methodology:
insights
 What to measure from phase to phase?
 diverse nature: educational, pedagogical and technical facets.
numerous parameters: functionality, educational suitability,
educational correctness, medium of content delivery…
 Different aspects must be assessed through different
methods/tools.
 What is the best method to asses which part?
 How to measure it?
 Quality: is neither measurable nor strictly defined (in the field
of distance education)
 model quality, quality assurance procedures, compliance and
quality-of-use.

11. QLC Development Methodology:
insights
The development of QLC requires both
 Process View: running a project with analysis,
design and development phases - PMBOK
 Data View producing a product – ISO standards.
 E.g. Product lifecycle according to PMBOK

12. Top-Down step: The Life cycle
Model

13. Bottom-up Approach: HOU
experience
 Guidelines/Best Practises derived from HOU
experience:
 Use of standards ISO9001:2008 and ELOT:1429;
 12 years of formally applying Q&A in teaching
practises and content;
 HOU as a publisher: 300 books, 500h of video-
lectures;
 Several GB of digital content (Wikis, hypertext,
ebooks) by 2015;
 Multi-step quality control based on in-breed
methods

14. Bottom-up Approach: guidelines
 Quality Control practical rules
 used by life-cycle stakeholders (authors/evaluators/QC
experts) for
 the assessment and certification
 The certification guide includes
 11 guidelines
 relating to content accuracy, expression, educational
feedback and self-assessment, completeness, scientific
accuracy, readability, motivation for further reading and
personal research, clarity of expression and up-to-dateness.

15. Bottom-up Approach: example (1/3)

16. Bottom-up Approach: example (2/3)

17. Bottom-up Approach: example (3/3)

18. Towards a new Quality Model
 Initial results have produced a dual, data and process
standard with 8 quality characteristics, 30 sub-
characteristics and 72 metrics
 Future work:
 Verification
and Validation Tool
 Validate standard

19. Thank you for your Attention!