Presentation on quality assurance of e-learning and he development of the epprobate labelt

1. Quality assurance and quality
enhancement of eLearning in the UK,
and the development of the
epprobate initiative
Harvey Mellar
Borderless Quality‧Infinite Innovation
International Conference on e-Learning Quality and Innovative Instruction
Taiwan, 31 July 2012
www.ioe.ac.uk/staff/LKLB_34.html

2. epprobate
www.epprobate.com
www.epprobate.com 2

3. Institute of Education,
University of London
www.ioe.ac.uk
www.epprobate.com 3

4. London Knowledge Lab
www.lkl.ac.uk
www.epprobate.com 4

5. University of London
International Programmes
www.londoninternational.ac.uk
www.epprobate.com 5

6. Outline
Quality and eLearning
The changing landscape of knowledge and
learning
Quality assurance in the UK
Quality assurance of eLearning
epprobate
www.epprobate.com 6

7. QUALITY AND ELEARNING
www.epprobate.com 7

8. What is quality?
 Exception: Quality as something exceptional, and
distinctive
 Perfection: Quality as a consistent or flawless
outcome
 Fitness for purpose: Quality as fulfilling a
customer's requirements, needs or desires
 Value for money: Quality as return on investment
 Transformation - the enhancement and
empowerment of students or the development of
new knowledge
Harvey, L. (1995). Editorial (The Key Issues: the quality agenda) Quality in Higher Education
www.epprobate.com 8

9. CONCERNS ABOUT THE QUALITY OF
ELEARNING
www.epprobate.com 9

10. Corporate training
American Society for Training & Development (ASTD)
What concerns does your organization have about e-learning?
“Quality of programs – 31.3%”
www.astd.org/Publications/Newsletters/ASTD-Links/ASTD-Links-Articles/2011/01/E-Learning-Trends-2011
www.epprobate.com 10

11. Higher Education
66% of faculty say online courses are inferior or
somewhat inferior to face-to-face courses
Conflicted: Faculty and Online Education, 2012 A Joint Project of The Babson Survey Research Group and Inside Higher Ed I. Elaine
Allen, Jeff Seaman, with Doug Lederman Scott Jaschik
http://www.insidehighered.com/news/survey/conflicted-faculty-and-online-education-2012
www.epprobate.com 11

12. Schools
www.carnegielearning.com
www.epprobate.com 12

13. http://www.nytimes.com/2011/10/09/technology/a-classroom-software-boom-but-mixed-results-despite-the-hype.html?pagewanted=all
www.epprobate.com 13

14. THE CHANGING LANDSCAPE OF
KNOWLEDGE AND LEARNING
www.epprobate.com 14

15. New demands and new suppliers
 Demands
– Many more students involved in higher education
– Demands for equitable access and retention
– Increasing need to address issues of student demand and
choice
– Constraints on public spending result in higher fees
 Suppliers
– Shift towards a more deregulated market
– Courses offered by international providers
– Increase in for-profit institutions in education
– Disaggregation of services
– Collaborations and partnerships
www.epprobate.com 15

16. New skills
www.p21.org/index.ph
www.epprobate.com 16

17. Using multimodal representations
http://www.mat.ucsb.edu/res_proj3.php
www.epprobate.com 17

18. Developing techno-mathematical
literacies
Hoyles, Celia (2007) Understanding the System: Techno-Mathematical Literacies in the Workplace
www.epprobate.com 18

19. Learning in the Network Society
Castells, M. (2001) The Internet galaxy: reflections on the Internet, business, and society
www.epprobate.com 19

20. Harnessing the Data Deluge
http://www.economist.com/node/15579717?Story_ID=15579717
www.epprobate.com 20

21. LEARNING IN NEW WAYS
www.epprobate.com 21

22. New (ish) technologies
M-learning
Tangibles
Serious games and
immersive worlds
Haptic
technologies
www.epprobate.com 22

23. Opening access
OERs, MOOCs and MOTS
Open Educational Resources
OER Massive Online Open Courses
Mass Online Tutoring Systems
www.epprobate.com 23

24. Learning as connecting
 Learning is a process of connecting specialized nodes
or information sources
 Learning may reside in non-human appliances
 Capacity to know is more critical than what is currently
known
 Nurturing and maintaining connections is needed to
facilitate continual learning
 Ability to see connections (built networks) between
fields, ideas, and concepts
 Currency (accurate, up-to-date knowledge) is the intent
of all connectivist learning activities
Siemens, G. (2005) Connectivism: A Learning Theory for the Digital Age
www.epprobate.com 24

25. Teaching as a design science
“Teaching is not rocket science. It is, in
fact, far more complex and demanding
work than rocket science.”
- Richard Elmore
“Teaching is changing. It is no longer simply about passing on
knowledge to the next generation. Teachers in the 21st century, in
all educational sectors, have to cope with an ever-changing cultural
and technological environment. Teaching is now a design science.
Like other design professionals - architects, engineers, town
planners, programmers – teachers have to work out creative and
evidence-based ways of improving what they do.”
- Diana Laurillard
“Every day, teachers design and test new ways of
teaching, using learning technology to help their learners.
But their discoveries remain local. By representing and
communicating their best ideas as structured
pedagogical patterns, teachers could develop this vital
professional knowledge collectively” www.epprobate.com 25
- Diana Laurillard

26. Pedagogical design patterns
• Context
– Where, when, who (all the things you can’t change)
• Problem
– We want to do A under condition B but are constrained by
C
Cookbook: ingredients,
• Solution What are we trying to
achieve / solve?
procedure, expected
outcomes
Problem Solution
Context
When, Where,
Who
www.epprobate.com 26

27. QUALITY ASSURANCE IN THE UK
www.epprobate.com 27

28. QA in UK Higher Education
Universities and colleges manage the quality and standards of
their awards by means of their own internal quality assurance
procedures
 Course validation
 Annual monitoring
 Periodic course review
 External examiners
 Peer observation
 Student feedback
– Course evaluation surveys
– Focus groups
– Student representation
http://www.qaa.ac.uk
www.epprobate.com 28

29. Quality Assurance Agency (QAA)
 Quality Code
– Standards, quality and public
information
 Institutional review
– Review team of five, including a
student
– Institutions submit a self-evaluation
document and a student written
submission
– Preliminary visit - one and a half days
+ Review visit - three to five days
– Institutional reports publicly available
on the QAA website
 Outcomes Reports
www.epprobate.com 29

30. Involvement of learners in quality
assurance
www.epprobate.com 30

31. From quality assurance to quality
enhancement
QUALITY ASSURANCE QUALITY ENHANCEMENT
 Inhibits frank reporting  Encourages and requires frank
reporting
 Promotes incremental
improvement of academic  Facilitates transformational
change
practice
 Supports and manages risk
 Discourages risk taking taking
 Retrospective approaches  Prospective approaches to
to quality management quality management
Raban, C. (2007) „Assurance Versus Enhancement: Less Is More?‟ Journal of Further and Higher Education
www.epprobate.com 31

32. QUALITY ASSURANCE OF ELEARNING
www.epprobate.com 32

33. Quality Code - Section 2
www.epprobate.com 33

34. Benchmarking
 A group of universities set up
in a benchmarking club
 Each completes an
Institutional Review
Document
 They jointly develop a set of
criteria called good practice
statements
 Each institution then scores its
performance against the good
practice statements
www.epprobate.com 34

35. A research study on quality
assurance of eLearning
 Case studies of Postgraduate courses
 For each case study
– Collect and review all quality assurance
documentation
– Interview stakeholders
 Comparative examination of data
– Map of issues not captured by the quality assurance
procedures
– Identification of aspects of the courses which impact
on the implementation of the QA procedures
www.epprobate.com 35

36. Results: Factors affecting the
application of QA procedures
ORGANISATIONAL
CONTEXT
DISTRIBUTED DISAGGREGATED
TEAMS PROCESSES
DISTANT LOCATION OPENNESS OF
OF STUDENTS COURSES TO REVIEW
www.epprobate.com 36

37. Workshop on QA/QE
procedures and eLearning
Based on existing research
Delivered in 15 UK universities, then in
Germany, Sweden, Saudi Arabia
Examined issues, challenges and possible
solutions
Dilemma, should we
– modify existing quality procedures, or
– create new quality procedures specifically for
eLearning?
www.epprobate.com 37

38. QA/QE in eLearning Special
Interest Group
www.qe-sig.net
www.epprobate.com 38

39. Toolkit
http://qaqe-sig.net/wp-content/uploads/2011/11/Toolkit_version_2011_8_11-Final-edit-DoH.pdf
www.epprobate.com 39

40. UK Quality Code for Higher
Education - Learning and Teaching
Example indicators
 An understanding of the learning process informs
learning and teaching practices, which use
evidence-informed approaches derived from the
outcomes of research, scholarship and the
evaluation of professional practices.
 Higher education providers assure themselves
that for every student both the physical and
virtual environments they provide are safe,
accessible, reliable and usable and that their use
is characterised by dignity, courtesy and respect.
http://www.qaa.ac.uk/Newsroom/Consultations/Pages/learning-teaching.aspx
www.epprobate.com 40

41. Summary: Issues for assessing
quality of eLearning
 Development of a quality culture
 Use of research, scholarship and the evaluation of
professional practices to inform learning and
teaching practices
 Ensuring the currency and accuracy of content
 Involvement of learners
 Involvement of other stakeholders
 Use of learning analytics
 Internationalisation
www.epprobate.com 41

42. The international quality label for eLearning courseware
EPPROBATE
www.epprobate.com 42

43. Objectives
Increase trust in eLearning
Deliver a quality label focusing on courseware
Facilitate a consensus building process about
eLearning quality
Establish an international network of
reviewers and partners
www.epprobate.com 43

44. The review process
 Self assessment document
– Asks for evidence
• Evaluations
• Learning analytics
 Review panel
– Pedagogic expert
– Content expert
– Learner
– Courseware producer
 Panel reviews courseware in terms of the quality grid
 Feedback to producer
www.epprobate.com 44

45. THE QUALITY GRID
http://epprobate.com/index.php/en/epprobate-quality-grid
www.epprobate.com 45

46. A. COURSE DESIGN
1. Provision of course
information, learning
objectives and
instructional guidance
2. Constructive alignment
www.epprobate.com 46

47. B. LEARNING DESIGN
3. Learner needs
4. Personalisation
5. Instructional strategies
www.epprobate.com 47

48. C. MEDIA DESIGN
6. Media integration
7. Interface
8. Interoperability and
technological standards
www.epprobate.com 48

49. D. CONTENT
9. Accuracy and values of
content
10. Intellectual property rights
11. Legal compliance
www.epprobate.com 49

50. epprobate - meeting the
challenges
 Development of a quality culture
 Use of research, scholarship and the evaluation of
professional practices to inform learning and
teaching practices
 Ensuring the currency and accuracy of content
 Involvement of learners
 Involvement of other stakeholders
 Use of learning analytics
 Internationalisation
www.epprobate.com 50

51. References
http://www.mendeley.com/groups/2338531/quality-assurance-and-quality-enhancement-of-elearning-in-the-uk/
Google short URL: goo.gl/H5lp4
www.epprobate.com 51