Course outcomes.com_hour_2_table_of_specifications

In April 2011, The CPE PhD, LLC became Course Outcomes, LLC. The training materials are referred to in this packet are owned and copyrighted by Course Outcomes, LLC and can only be used for free trainings. They may not be rebranded nor sold without express written permission from Course Outcomes. Course Outcomes and The CPE PhD © 2011 Course Outcomes, www.courseoutcomes.com

Thank you for using The CPE PhD’s training services. The content of this training is copyrighted by The CPE PhD and may not be resold, repackaged, or printed for profit. Organization Members may download and repackage materials for their organization but not for profit without express written permission from The CPE PhD. Intro Slide © 2011 Course Outcomes, www.courseoutcomes.com

Test Construction Philosophy 600 possible items from my course. My 60 item test. Suppose you wrote all possible questions for everything your course has taught a student (600 items or so). If you put all 600 of these items in a box, and pulled out 60 items, this would be a sample taken from your population of 600 items. A test is a sample of items you could have asked. © 2011 Course Outcomes, www.courseoutcomes.com

Test Construction Philosophy Most of us do not have time to write 600 items, nor would we leave our certification tests up to a random drawing, so we create a structure to test the more important elements of our course. 600 possible items from my course. My 60 item test. I don’t have time to write 600 items so I’m going to write 6 items on history of SOX, 4 items on section 2, etc. © 2011 Course Outcomes, www.courseoutcomes.com

Test Construction Philosophy We also use intuitive reasoning to tell us that if a person can answer one item, they could probably answer another item. (e.g. If a person correctly knows the square root of 81 and 36, then how likely is it that they know the square root of 25?) Exactly how many items we need to write to make a generalization that a student knows the content can be answered intuitively via your experience or verified by a psychometrician. Psychometrician is a fancy way of saying professional test writer. © 2011 Course Outcomes, www.courseoutcomes.com

The way a student judges if your test is fair is by three criteria: 1) Is your item sample a fair representation of what they should or could have learned? Is your test aligned with your instruction? That test had nothing to do with what we learned! © 2011 Course Outcomes, www.courseoutcomes.com

The way a student judges if your test is fair is by three criteria: 2) Is a score on the 60 item test generalizable to the larger pool of what the student has learned in the course? Is your test generalizable? This test score doesn’t really show all that I have learned. © 2011 Course Outcomes, www.courseoutcomes.com

The way a student judges if your test is fair is by three criteria: 3) Do the 60 items represent (in a fairly weighted fashion) all possible items you could have asked or did you over/under emphasize some points? Is your test fairly weighted? My tests only focus on my favorite parts of my course. Have you ever had a professor like this? © 2011 Course Outcomes, www.courseoutcomes.com

Purpose of this Training Construct a fair test that is aligned with your instruction. Once your generalinstructional objectives and specific learning outcomes are established you are now prepared to create what is called a Table of Specifications. © 2011 Course Outcomes, www.courseoutcomes.com

The table of specifications is a table or matrix that has three parts: A list of content areas A list of student abilities Number of Test items Both the content area and student abilities are derived from your SLOs. Table of Specifications © 2011 Course Outcomes, www.courseoutcomes.com

GIO – Students will understand the background of Sarbanes-Oxley legislation. SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation. As an Example © 2011 Course Outcomes, www.courseoutcomes.com

Now we are ready to begin planning how many items should be written. Notice that you can write several items in each cell that assess students’ ability to do something with the content. On what date was the Sarbanes-Oxley legislation passed? June 24, 2001 October 23, 2003 January 3, 2005 December 4, 2008 What event spurred the passing of the Sarbanes-Oxley legislation? Exxon Valdees Disaster Enron Scandal Other plausible but incorrect option Other plausible but incorrect option © 2011 Course Outcomes, www.courseoutcomes.com

As an example, maybe with audit committees and independent auditors students only need to identify certain guidelines but will never need to implement any procedures regarding these topics. Notice also that with the exception of the cells with zeros, no cell has less than 2 items. The numbers represent the number of test items written for each specific learning outcome. Note that some cells have zeros This would be done if you plan on having more than one assessment covering the same content. (e.g. assessments for each day as well as a post-assessment) Otherwise you run the risk of the students seeing the same item on two different exams. This will happen when content does not lend itself to a particular student ability. © 2011 Course Outcomes, www.courseoutcomes.com

When selecting items for a test it is important to ensure that the test items reflect the emphasis of your instruction. Alignment of instruction and assessment © 2011 Course Outcomes, www.courseoutcomes.com

For example if hypothetically you spend 90% of your time teaching about financial disclosures and 10% of your time teaching about internal controls, Alignment of instruction and assessment © 2011 Course Outcomes, www.courseoutcomes.com

For example, if hypothetically you spend 90% of your time teaching about audit controls and 10% of your time teaching accounting statistics, you would want this ratio to be reflected in the test questions. Alignment of instruction and assessment © 2011 Course Outcomes, www.courseoutcomes.com

At this point you can ask yourself, “Do these percentages reflect the emphasis of my instruction and course assignments?” These percentages are calculated by all of the items in the table; in this case – 24. 2/24 = 8% This means that 8% of the test covers specific dates and events. It also means that 38% of the assessment tests student’s ability to identify If not you may need to make adjustments to your - by dividing the number of items in this entire row Test Instruction and/or Specific Learning Outcomes © 2011 Course Outcomes, www.courseoutcomes.com

Your Assignment Create a table of specifications for your course which reflects: (1) a balance of instruction and assessment (i.e. 25% of my instruction focuses on achieving this SLO so my test will measure 25% on this SLO). (2) the number of items which should be written. Ensure that your table of specifications has rows for the content, columns for student abilities, and numbers in the cells representing the number of items you will write. The content and student abilities will be taken from the specific learning outcomes you prepared for the first assignment. © 2011 Course Outcomes, www.courseoutcomes.com

If you have any questions or need assistance check the discussion board or contact Dr. Daniel Winder at the courseoutcomes@gmail.com www.courseoutcomes.com © 2011 Course Outcomes, www.courseoutcomes.com

Course outcomes.com_hour_2_table_of_specifications

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Empfohlen

Empfohlen (20)

Course outcomes.com_hour_2_table_of_specifications