1. Evaluation of Determining Instructional Purposes Training Program
A Proposal Submitted to Far West Laboratory by
Building Evaluation Solutions Together, Inc.
Introduction
Far West Laboratory for Educational and Research Development (FWL) issued a request for
proposal for an evaluation of its Determining Instructional Purposes (DIP) training program.
This document is a proposal from Building Evaluation Solutions Together, Inc. (BEST) of Boise,
Idaho submitted in response to the FWL request for proposal. The purpose is to present the
BEST evaluation design, development, and implementation, which will provide FWL with
comprehensive, valid and reliable data on which to base marketing decisions related to DIP
training materials.
Description of Program Being Evaluated
The program to be evaluated is the Determining Instructional Purposes (DIP) training program
for school administrators and graduate students, which is comprised of three self-contained units:
Setting Goals, Analyzing Problems and Deriving Objectives. In each unit, trainees read materials
in each of four-to-six modules, and practice skills in individual or small group activities, with
feedback provided. Many of the practice activities place trainees in planning teams in which
skills are applied to the operations of a hypothetical school district. Training time for the three
units varies, with an estimated range from 10 – 18 hours per unit, and is overseen by a
coordinator who has been part of a prior training group. Training materials are intended for use
in short-term workshops or individual sessions spread over a period of time. Materials are in
print form with unit length between 155 and 259 pages. Units are available separately ($8.95) or
as a set ($24.95) and a Coordinator Handbook ($4.50) includes instructions for all three units.
Materials have been developed but not yet disseminated and the purpose of the evaluation is to
determine the marketability of the product and value to potential purchasers.
Evaluation Method
The primary purpose of this evaluation is to provide information and recommendations regarding
dissemination of the units. To do so, the BEST evaluators will gather information about the
content of DIP materials, their use in training sessions, and their perceived effectiveness. The
BEST team of evaluators will gather information from three sources: program developers/staff, a
subject matter expert, and potential users of the training materials. Observations, interviews,
surveys, and review of documents will be used to collect data.
Program developers and staff will be asked to 1) provide background information regarding the
process of developing DIP training materials, 2) identify perceived strengths and weaknesses of
training materials, 3) communicate marketing considerations: production schedule, target
audience/region, cost comparison of digital vs. print, past practice regarding production and
promotion of new materials, and 4) share any additional information they feel is valuable.
1

2. Information from program developers and staff will be gathered through interviews with team
leaders, informal meetings with project staff, and review of documents.
For the purpose of determining accuracy and comprehensiveness of the content, DIP training
materials will be evaluated a subject matter expert. The Subject Matter Expert will be provided
with a full set of training materials and, upon review, will use a rating instrument to evaluate the
content of DIP materials in the following categories: accuracy, completeness, relevance,
timeliness, and clarity. The Subject Matter Expert will participate in a video conference with the
BEST evaluator and FWL program director.
For the purpose of determining usefulness and usability, two groups representing potential users
will pilot the material: school administrators and graduate students in educational administration
programs. Administrators will be divided into three groups, with each group participating in a 3-
4 day training session for one of the three DIP units. BEST teams, comprised of Research
Assistants and Interns and overseen by the lead evaluator, will collect observational data as
trainees work through each unit. All participants will complete an exit survey (content, context,
impressions, usability) and a random sample of trainees will participate in a focus group
discussion. This process will be repeated with groups of graduate students. Participants in the
training workshops will receive an honorarium along with a complementary set of training
materials.
Task Schedule
Begin Date: May 3, 2010
End Date: July 22, 2010
TASK Responsible # of Days
Party
Discuss Proposal and revise Evaluation Plan as needed. FWL and 1
BEST
Collect background information from FWL program director BEST 5
and staff; document review.
Revise Evaluation Plan and submit to FWL BEST 3
FWL feedback on revisions to BEST FWL 3
Final Evaluation Plan revisions BEST 1
BEST Review of DIP Training Materials BEST 10
Develop/Adapt data collection instruments BEST 5
Recruit school administrators and graduate students to take part BEST 20
in DIP training workshops.
Subject Matter Expert review of DIP Training Materials, SME 10
completion of rating instrument and report.
2

3. TASK Responsible # of Days
Party
Video conference with Subject Matter Expert FWL, BEST, 1
SME
12 K-12 Administrators participate in DIP Training sessions BEST 3-4
and complete an exit survey. FWL
coordinator
12 Graduate Students participate in DIP Training sessions and BEST 3-4
complete an exit survey. FWL
coordinator
Focus Group discussion with 5-6 trainees selected at random. BEST 1
Data Analysis by BEST Research Assistants and Interns BEST 5
Preparation of Final Report (Draft) BEST 5
FWL feedback on Final Report FWL 5
Final Report submitted to FWL BEST 2
3

4. Project Personnel
Dr. Daniel L. Stufflebeam, Distinguished University Professor at Western Michigan University,
serves as the BEST lead evaluator. With over 40 years of experience in the field, and a leader in
the development of efficient and effective evaluation models, Dr. Stufflebeam served as Director
of the Evaluation Center at University of Western Michigan from 1973 until 2002. He holds a
B.A. in Music Education, and M.S. in Counseling and Psychology, and Dr. Stufflebeam’s
doctoral and post-doctoral work is in Measurement, Statistics, and Experimental Design.
Currently on sabbatical from his duties at Western Michigan University, Dr. Stufflebeam leads
the BEST team of evaluation professionals in a grass roots campaign to provide up-and-coming
professionals in the field of evaluation with expert models and supervised field experiences in
the BEST Evaluator Mentoring Program. In his leisure time, Dr. Stufflebeam directs the Boise
Community Band.
Subject Matter Expert: Dr. Esperto Elegante is currently Chair of Curriculum, Instruction, and
Foundation Studies in the Boise State University College of Education. A former 7th grade
science teacher, principal and superintendent, Dr. Elegante’s area of expertise is professional
development in K12 education.
Research Assistants: Recent graduates of Evaluator Certification Programs undergo a rigorous
selection process and those admitted receive additional training as part of the BEST Evaluator
Mentoring Program. With oversight by BEST leadership, Research Assistants help to gather
information, collect data, and administer surveys, utilizing current technologies throughout the
process of gathering and analyzing data. Provided with a small stipend, Research Assistants gain
valuable experience with minimal cost to potential clients.
Interns: Students currently enrolled in graduate programs related to research, education, and
evaluation work under the supervision of Research Assistants and BEST evaluators. Charged
with data entry and verification of data accuracy, hands-on experience benefits interns and their
service is provided at no cost to clients.
4

5. Budget
Fees for the evaluation of the Determining Instructional Purposes Training Program
are not to exceed $50,000.
Professional Salaries (daily) # Rate Total
Dr. Daniel Stufflebeam 60 $450.00 $27,000.00
Dr. Esperto Elegante 11 $400.00 $4,400.00
3 Research Assistants 35 $50.00 $1,750.00
3 Interns 35 $0.00 $0.00
Project Administrative
Assistant 45 $120.00 $5,400.00 $38,550.00
Honorariums
Study Participants -
Administrators 12 $350.00 $4,200.00
Study Participants - Graduate
Students 12 $100.00 $1,200.00 $5,400.00
Travel
One 6-day round trip to FWL
site 6 $350.00 $2,100.00
Estimated mileage 500 $0.30 $150.00 $2,250.00
Operations
Telephone (per month) 3 $100.00 $300.00
Postage $120.00
Rent and utilities (per month) 3 1,000 $3,000.00 $3,420.00
Materials
DIP Training materials, sets of
three unites (28 sets) to be
provided, at no cost, by FWL 28 $0.00 $0.00
DIP Training materials,
Coordinators Handbook (28)
to be provided, at no cost, by
FWL 28 $0.00 $0.00
Supplies and photocoping $350.00 $350.00
TOTAL: $49,970.00