Evanouski Evaluation Proposal

EdTech 505, Spring ’10Evaluation of Far West Laboratory for Educational Research and Development Training Units for Educators A Proposal Submitted to Far West Laboratory for Education Research and DevelopmentSubmitted by: Lora Evanouski, Criteria CEO, Boise, ID Introduction In February of 2010 Far West Laboratory for Educational and Research Development (FWL) requested proposals (RFP) for evaluation of a newly released training program Determining Instructional Purposes (DIP) training program. The Criteria group has submitted a RFP to Far West Laboratory for Educational and Research Development (FWL) in response to evaluate their Determining Instructional Purposes (DIP) training program. This document is in response to their request. Dr. Snidely Whiplash is the director of research and development for FWL and Rocky Bullwinkle the liaison for FWL. Description of Program The Determining Instructional Purposes (DIP) training program is intended to train administrators and graduate students in educational administration skills related to planning effective school programs. The DIP program consists of Coordinator’s Handbook and three training units. The units consist of Unit 1-Setting Goals, Unit 2-Analyzing Problems, and unit 3-Deriving objectives. Each unit is comprised of 4-6 modules with a limited number of instructional objectives. The modules contain reading material, are taught individually or in small group activities. There is trainee role play and feedback. The units are completely self-contained. Each unit can be taught as one unit only or combined with the other units. Estimated training time of each unit is 10-18 hours. The program coordinator is the guide and facilitator of the content. As stipulated by the program developers, no outside training or prior knowledge is needed for the program coordinator. The program coordinator may read the content of the training handbook prior to the program but is not necessary. The cost is $8.95 per unit and $24.95 for the set of three units. The coordinator’s handbook has directions for the coordinating the three units at a cost of $4.50 a copy. Evaluation Method FWL has many stakeholders that need to be considered in this evaluation. The stakeholders consist of a board of directors, public shareholders, administration, and staff. The overarching goal is to provide feedback concerning the effectiveness of the implemented program (DIP). The major objectives concerned with this evaluation to aid in the decision making process of continued use of DIP are to: ,[object Object]

measure implementation of program materials by school districtsIn collecting data from various sources these three objectives can be answered with 100% certainty to the effectiveness, quality and impact of the instruction. It will answer the question of determining the continuation and implementation of the program. Because all the materials are entirely in print form, cost and program deliverability must be considered. In addition, FWL wishes to market the DIP training program contingent upon an evaluation outcome. After a meeting with the stakeholder committee from FWL, an evaluation model can be chosen. It is imperative the stakeholder committee is involved in this crucial process to ensure their needs are being satisfied. In addition any further questions, comments or feedback will allow for adjustments in the project. FWL has chosen the decision-based evaluation model based upon the desired outcomes. Dr. Snidely Whiplash and Rocky Bullwinkle from FWl along with the Criteria team of Lora Evanouski CEO, Michael Egghead SME, Gomer Pyle senior test writer and Jack Bauer senior instructional designer assisted in making the crucial decisions on which design model would be the most effective. All results will be forwarded to FWL stakeholders. Effectiveness Quantitative Method: Pre/post-tests administered to all participants.Qualitative Method: Formal interviews of all participants. To determine the module effectiveness the evaluation team will measure the outcomes of the each unit by testing the clients that use the program. This will consist of a pre-test and post-test for each unit. This will be written and administered by Criteria. Pre-tests will determine entry level knowledge and post-tests will determine knowledge gained from participating in the DIP program. By comparing the two outcomes a firm understanding of knowledge gained will be determined. This quantitative data information will be given to the stakeholders. The tests will be written by our senior test writer Gomer Pyle and conducted through a computer. Interviews will be conducted before and after testing. All interviews will be done by either a video recording or an audio recording to ensure validity and attribute proper meaning of information collected. This can be done face-to-face or via teleconference. The information ascertained will develop an attitude assessment before training and after training for effectiveness. A random sample of 15 interviews will be used from administrators, coordinators and grad students. Quality Qualitative Methods: Observation, on-site. Likert 5-point survey and interviews will be done by evaluators To determine the quality of the DIP program, Criteria will attend at least one training session of an individual module. All evaluators will have read the training handbook before attending the session. In doing so, the evaluator will be qualified to make an evaluation of the on-site training procedure. In addition to an on-site evaluation of the training procedures, the evaluators will also observe actual implementation of a unit being executed by the administrators. Criteria will administer an attitude survey at the end of the unit. A random sampling will be used. At least 15 surveys chosen from all participants will be assessed. This will include samplings from participants, administrators and program coordinators. Impact will be assessed through these survey questions. This qualitative data will be tabulated using a Likert scale devised by the senior test writer Gomer Pyle. In addition to on site observations, interviews will be done with the participants. All interviews will be video recorded or audio recorded to ensure validity. Interviews will be done via teleconferencing. The data collected from the interviews will be cataloged and graphed. This information will be presented to the stakeholders of FWL. Implementation Qualitative Method: Likert 5 point scale-pre/post surveys, observationsQuantitative Method: Checklist of objectives met/not met The implementation of the program will consist of qualitative data collected through surveys and interviews of individuals directly involved with the program. It will consist of pre and post learning surveys. Surveys will be created by Criteria and disseminated to all participants, administrators and program coordinators throughout the units. Two different surveys will be used. The pre surveys will indicate perceptions and attitudes of what may be gained from using the DIP program. The post surveys will indicate the impact the program had upon the participants. The surveys will indicate a willingness to participate in the program as well as implement the program into future sessions. All surveys will be done via the computer. The information gleaned will be used in the decision making process of proceeding with the DIP program or choosing a better alternative. In order to gain a firm grasp on proper implementation, the evaluator will observe a unit. The evaluators will video record observation session to ensure validity. The evaluator will tabulate findings using a checklist and it will be presented to the stakeholders. Each unit will have a checklist of objectives that are either met or not met within the training session. In addition, this data will provide information as to usability of the program as well as confidence in using the coordinator handbook. This checklist will provide feedback to the instructional designer, Jack Bauer, as to proper implementation of materials. Task List ,[object Object],TaskAgency ResponsibleDeadline DateMeet with FWL stakeholders and staff to discuss EEE proposal- modifications if necessaryCriteriaFebruary 15Submit data collection plan, survey instruments, assessments, and interview protocolsCriteriaMarch 15Provide feedback of data collection plan, survey instruments, assessments, and interview protocolsFWLMarch 25Revisions of data collection plan, survey instruments, assessments, and interview protocol. Submit final copyCriteriaApril 7Delivery of surveys, assessments, and interview protocols to intended groupsCriteriaMay 1Evaluators have attended minimum of one training sessionCriteriaMay 15Evaluators observe unit in real timeCriteriaJune 1Administration of surveys, assessments, and interview protocolsCriteriaJune 1Collect all data from implemented surveys, assessments, and interviewsCriteriaAug 1Compiled/summarized data to be submitted to shareholdersCriteriaOct 1Feedback from initial findings of complied dataFWLNov 1Write final report and submit to FWLCriteriaDec 1 Project Personal Lora Evanouski, CEO and co-owner and founder of Criteria. Lora will be the main party responsible for the planning and implementation of the data collection and processing methods of the evaluation. Lora is a well respected community leader. She has 12 years experience in evaluating programs for various business and educational applications. Lora has a B.S. from Indiana University in Elementary Education, an M.A. from University of Hawaii in research and design and is currently seeking her doctorate in Educational Technology from Boise State University. Lora has been published 5 times over her academic career and is considered a leading professional for her work in the use of emerging technologies. Michael Egghead, SME and co-founder of Criteria is responsible for subject matter control. Michael holds a B.S. in Instructional Design and certifications in program development and analysis. Michael worked for Intel for 18 years as a program developer before turning his attentions to his own company 9 years ago. Michael has been published 3 times in his work on content analysis. Gomer Pyle, senior test writer. Gomer has a B.S. in Anthropology and M.S. in Psychology. Gomer’s responsibilities are writing of the surveys, interview protocols and assessments. Gomer has worked as an outside contractor for various fortune 500 companies as a training developer and coordinator for 23 years. He is a sought after asset within the program development world. Gomer has designed award winning programs and evaluations for Verizon, State Farm, Target, and Dell. Gomer has written 9 published articles in various educational journals. Jack Bauer, instructional designer. Jack has a B.S. in Instructional design from Indiana University. Jack has 8 years experience writing and designing programs. Jack’s responsibilities include designing programs, assisting in evaluations, office coordinator, scheduling and day to day operations. In addition to the staff of Criteria, interns will be used from the prestigious department of Educational Technology, Boise State University located in Boise ID. Their duties will be to assist in compiling data and office functions. Budget Lora Evanouski50 days @ $300/day$17,500 Michael Egghead30 days @ $200/day$6000 Gomer Pyle30 days @ $200/day$6000 Jack Bauer50 days @ $100/day$5000 Interns40 hours @ $10/hr$400 Total Personnel$34,900 Trip and per diem Three 2-day round trip for 1 person: Boise to Chicago (includes per diem)$ 1800 Estimated miscellaneous mileage: 1000 miles @ $.30 per mile$300 Car rental$500 Total trip costs$2600 Communications Telephone and videoconferencing$800 Postage$500 Total cost$1300 Supplies and Photocopying Office supplies and photocopying$500 Estimated Total Budget Cost$39,300 ,[object Object]

Evanouski Evaluation Proposal

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (8)

Ähnlich wie Evanouski Evaluation Proposal

Ähnlich wie Evanouski Evaluation Proposal (20)

Mehr von loraevanouski

Mehr von loraevanouski (13)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Evanouski Evaluation Proposal