This document presents a framework for analyzing instructional design in online offerings. It consists of six components: objective, task, condition, physical interaction, feedback, and criteria. These components are used to evaluate whether an instructional design meets learners' needs by analyzing how objectives, tasks, conditions, interactions, feedback, and assessment criteria are implemented. Several examples applying the framework to online courses and modules are provided and discussed. The presentation concludes by considering whether the framework is useful and what its limitations might be.

1. Evaluating the Effectiveness
of Instructional Design
in Online Offerings:
An Analytic Approach
Michael Cenkner, M.Ed.
Athabasca University

2. Agenda for this session:
Presenting a framework
Quick overview of the framework
An example
Applying the framework
Analyzing some examples together
Conclusions
Usefulness?
Limitations?

3. My professional context
Digital resources - learnAlberta.ca
Online courses - Working with faculty
Web. 2.0, etc. –ID within flood of new technologies
When something seems inadequate, how to say
“why”?
A need a way to evaluate that is:
Objective
Shareable
→ Analytical approach

4. An analytic framework:
reverse-engineering the learning experience
ID analysis:
Look at implementation
↓
Identify model (inc.
learning obj’s)
↓
Evaluate whether needs
may be met
ID development:
Look at needs
↓
Develop model (inc.
learning obj’s)
↓
Implementation

5. Framework components (implicit or explicit)
(Implicit or explicit)
Objective
• What should the learner be able to do as a result of this experience?
Task
• What is the activity they are doing in order to learn?
Condition
• What is going on around and within learners (before and during)?
Physical Interaction
• What do they physically do to learn this?
Feedback
• What thing happens in the environment as a result of the learner’s
actions?
Criteria
• How is the learning being measured?

6. Notes on the analysis
Components are implicit or explicit
• Explicit – They are identified (e.g.
objectives)
• Implicit – You will need to identify
Looking for alignment,
not mere inclusion!

7. Guiding questions for components
Objective
• Are there one or more clear (explicit or implicit) objectives?
• If there is an objective, is it appropriate for the given learners?
Task
• Is there a stated or implied task? Does the task have a beginning and an end?
• Is the task an example of the kind of learning indicated in the objective?
Condition
• For the given task, do learners: Have motivation?
• Prior knowledge and/or skills?
• Adequate time?
• Necessary resources?
• Adequate orientation?
Physical Interaction
• Is the intended physical interaction (in all its steps or variations) clear?
• Are learners physically able to engage as intended and necessary?
• Are affordances available and useable as intended?
Feedback
• Is there feedback? From what source(s)?
• Assumed? Necessary? Happening?
• Timely?
• An obviously direct result of the learner’s actions?
Criteria
• Are criteria provided? If so, are they summative only or formative as well?
• If so, will the resulting framework appear reasonable, fair and motivating to the learners?
• Do criteria address the same knowledge and skills as those intended to be developed through
the intervention?

8. Objectives
“Evaluate the challenges of research in [this field] of management”
Tasks
• Facilitate discussion on weekly (assigned) topic
• Participate in discussion
Conditions
Weekly topic reflects topics related to research challenges in the field
Cohort group – have done one online course previously
Physical interaction
Participants read, write and post in Moodle discussion module.
Feedback
Student facilitator is participating at least twice.
Instructor gives weekly summary of each discussion.
Criteria
Quantity and quality of posts
Example 1, Objective 1

9. Objectives
“Demonstrate collaborative skills appropriate to … research in [this] field”
Tasks
• Facilitate discussion on weekly (assigned) topic
• Participate in discussion
Conditions
Participants are current practitioners doing a graduate certificate.
Each student doing own research proposal assignment
Physical interaction
Participants read, write and post in Moodle discussion module.
Feedback
Student facilitator is participating at least twice.
Instructor gives weekly summary of each discussion.
Criteria
Quantity and quality of posts
Example 1, Objective 2

10. Objectives
“Introduce students to the field of American literature as it has emerged and evolved in
North American academic settings.”
Tasks
1. Read texts
2. Post answer to weekly assigned question
3. Participate in discussion; comment on one other student.
Conditions
Second-year undergraduate students
Discussion is not facilitated except informally by instructor
Physical interaction
Participants read, write and post in Moodle discussion module.
Feedback
Informally from other students and/or instructor
Criteria
Quantity and quality of posts
Example 2, Objective 1

11. Objectives
“Discuss the field of American literature in terms of the present globalizing moment that
has produced challenges to the concepts of the nation-state.”
Tasks
1. Read texts
2. Post answer to weekly assigned question
3. Participate in discussion; comment on one other student.
Conditions
Second-year undergraduate students
Discussion is not facilitated except informally by instructor
Physical interaction
Participants read, write and post in Moodle discussion module.
Feedback
Informally from other students and/or instructor
Criteria
Quantity and quality of posts
Example 2, Objective 2

12. Example 3: Open Course-Ware from Carnegie-Mellon

13. Objectives
To greet someone in French
Tasks
1. Watch and listen to the dialogues
2. Identify which sentence is being said by listening
3. Order the sentences by dragging the words on-screen
Conditions
Learners are motivated.
Learners can replay dialogues at will.
Physical interaction
Use the mouse to control the play bar and links
Feedback
System indicates correct or incorrect answer.
Criteria
Sentence has been properly identified.
Sentence has been properly ordered
Example 3, Objective 1

14. Objectives
To pronounce the alphabet and basic sounds of French
Task
1. Listen to the sounds both separately and within words and sentences
Physical interaction
Participants read, write and post in Moodle discussion module.
Interaction
Use the mouse to control the play bar and links
Conditions
Learner can repeat sounds at will
Feedback
Learner can listen to self.
Criteria
Not provided.
Example 3, Objective 2

15. Objectives
To evaluate instructional design using the framework components
Task
1. Evaluate various examples
2. Discuss advantages and limitations
Conditions
Prior
• Participants are interested in and have some experience of the topic.
• Participants have been introduced to the framework.
During
• Participants can refer to components checklist
Physical interaction
Talking f2f; look at PowerPoint
Feedback
Feedback from presenter and from other participants.
Criteria
Each question in the components checklist has been answered correctly according to
the consensus of the group.
Example 4: This presentation

16. Objectives
To apply the principles of Domain X
Task
1. Read about the principles
2. Discuss the principles verbally and/or in writing
Conditions
Prior
• Participants are motivated to learn the principles
• Participants have access to the readings laying out the principles
During
• Participants can refer to readings
Physical interaction
Participants read, write and post in Moodle discussion module.
Feedback
Feedback from instructor and from other participants.
Criteria
To be determined at a future date
Generic Example: Course X

17. Conclusion(s)
Does this framework work?
Is this framework useful?
What are its limitations?
Comments/questions?