2. Background
• Prior to todays knowledge, it is normal that an initial draft would have been
put into a final production and then distributed to a specific population. We
learn that the materials at that time were not sufficient to support the
instructional effort.
3. Chapter 10 Objectives
• Describe the purposes for and various stages of formative evaluation of
instructor-developed materials, instructor-selected materials, and instructor-
presented instruction.
• Describe the instruments used in a formative evaluation.
• Develop an appropriate formative evaluation plan and construct instruments
for a set of instructional materials or an instructor presentation.
• Collect data according to a formative evaluation plan for a given set of
instructional materials or instructor presentation.
4. THE CONCEPT OF FORMATIVE
EVALUATION
The collection of data and information during the development of instruction
that can be used to improve the effectiveness of the instruction.
5. Purpose of The Concept of Formative
Evaluation
• To obtain data that can be used to revise the instruction to make it more
efficient and effective.
6. 5 questions related to decisions you made while
developing the materials are appropriate for all
materials:
• Are the materials appropriate for the type of learning outcome?
• Do the materials include adequate instruction on the subordinate skills, and
are these skills sequenced and clustered logically?
• Are the materials clear and readily understood by representative members of
the target group?
• What is the motivational value of the materials?
• Can the materials be managed efficiently in the manner they are mediated?
7. Types of data to collect include the following:
• Reactions of the subject-matter expert, whose responsibility it is to verify that the
content of the module is accurate and current.
• Reactions of a manager or supervisor who has observed the learner using the skills
in the performance context.
• Test data collected on entry skills tests, pretests, and posttests.
• Comments or notations made by learners to you or marked on the instructional
materials about difficulties encountered at particular points in the materials.
8. Continued…
• Data collected on attitude questionnaires or debriefing comments in which
learners reveal their overall reactions to the instruction and their perceptions
of where difficulties lie with the materials and the instructional procedures in
general.
• The time required for learners to complete various components of the
instruction.
9. Role of Subject-Matter, Learning, and Learner
Specialists in Formative Evaluation
• SME may be able to comment on the accuracy and currency of the
instruction.
• Learning specialist may be able to critique your instruction related
to what is known about enhancing that particular type of learning
• Learner specialist may be able to provide insights into the
appropriateness of the material for the eventual performance context.
10. THE THREE PHASES OF FORMATIVE
EVALUATION
1. One-to-One Evaluation
2. Small-Group Evaluation
3. Field Trial
11. One-to-one Evaluation
• Purpose- to identify and remove the most obvious
errors in the instruction and to obtain initial
performance indications and reactions to the
content by learners.
12. Criteria
• The three main criteria and the decisions designers make during the
evaluation are as follows:
• Clarity: Is the message, or what is being presented, clear to individual target
learners?
• Impact: What is the impact of the instruction on individual learner’s
attitudes and achievement of the objectives and goals?
• Feasibility: How feasible is the instruction given the available resources
(time/context)?
13. Selecting Learners
• One of the most critical decisions by the designer in the formative evaluation
is the selection of learners to participate in the study.
• This is not an experiment; there is no need for random selection of large
numbers of learners.
• The designer wants to select a few learners who represent the range of ability
in the group because prior learning or ability is usually one of the major
determiners of ability to learn new skills and information.
14. DATA COLLECTION
• The first category, message, relates to how clear the basic message is to the learner
determined by such factors as vocabulary, sentence complexity, and message
structures.
• The second category, links, refers to how the basic message is tailored for the
learner, including contexts, examples, analogies, illustrations, demonstrations, and so
forth.
• The third area, procedures, refers to characteristics of the instruction such as the
sequence, the size of segment presented, the transition between segments, the pace,
and the variation built into the presentation
15. Examples of questions of interest include the
following:
• How should the maturity, independence, and motivation of the learner
influence the general amount of time required to complete the instruction?
• Can learners such as this one operate or easily learn to operate any
specialized equipment required?
• Is the learner comfortable in this environment?
• Is the cost of delivering this instruction reasonable given the time
requirements?
16. PROCEDURES
• The typical procedure in a one- to- one evaluation is to explain to the learner
that a new set of instructional materials has been designed and that you
would like his or her reaction to them.
• You should say that any mistakes that learners might make are probably due
to deficiencies in the material and not theirs.
• Encourage the learners to be relaxed and to talk about the materials.
• You should have the learners not only go through the instructional materials
but also have them take the test provided with the materials.
17. You must also evaluate the utility of the
evaluation instrument, particularly
the following elements:
• The observability of each of the elements to be judged
• The clarity of the manner in which they are paraphrased
• The efficiency of the sequencing order.
18. ASSESSMENTS AND QUESTIONAIRES
• After the students in the one- to- one trials have completed the instruction,
they should review the posttest and attitude questionnaire in the same
fashion.
• After each item or step in the assessment, ask the learners why they made the
particular responses that they did.
• This will help you spot not only mistakes but also the reasons for the
mistakes, which can be quite helpful during the re-vision process.
19. CHANGE AGENT
• When it comes to a change agent, they are responsible for
collaborating with the other members on the professional community
in order to take responsibility for both school and instructional
decisions. In doing so, these instructional decisions are helping
create-student centered learning communities.
