Test appraisal

1. Test Appraisal System.
Concepts of Appraisal &
Procedures B
UNIVERISTY OF EDUCATION, LAHORE11/4/2015

2. Topics of Discussion
• Definition
• Meaning
• Assembling the Classroom Test
• Important Test Characteristics
• Item Analysis and its benefits
• Item Analysis for Norm Referenced Test
• Item Analysis for Criterion Referenced Test
• Establishing Test Validity and Reliability
• Preparing Test Item BankUNIVERISTY OF EDUCATION, LAHORE11/4/2015

3. According to Newstrom: “It is the process of
evaluating the performance of Students,
sharing that information with them and
searching for ways to improve their
performance’’.
Definitions
UNIVERISTY OF EDUCATION, LAHORE11/4/2015

4. • Test appraisal is the step where the
Institutions management finds out how
effective it has been at conduction and
evaluation of students.
• A “TEST Performance appraisal” is a process
of evaluating a Student’s performance of a test
in terms of its placement and promotion.
Meaning

5. Objectives of TEST Appraisal
Measuring the
efficiency
Maintaining Control.
Concrete and tangible
particulars about their
work
Assessment of
performance
Mutual goals of the
students, teachers & the
organization.
Growth & development
Increase harmony &
enhance effectiveness
 Personal development
 Self Satisfaction
 Involvement in the
Work.
STUDENTS INSTITUTIONAL
According to:
Aims at:
UNIVERISTY OF EDUCATION, LAHORE11/4/2015

6. Desirable characteristics for tests:
Apart from assessment and evaluation,
we must consider the following points
while testing or going to develop a test
appraisal system:
• Validity
• Reliability
• Utility
• Practicality
• Discrimination
UNIVERISTY OF EDUCATION, LAHORE11/4/2015

7. VALIDITY
• Validity refers to the degree in which our test or other
measuring device is truly measuring what we intended
it to measure. The test question “1 + 1 = _____” is
certainly a valid basic addition question because it is
truly measuring a student’s ability to perform basic
addition.
• The subjective testing inevitably associated with testing
the productive skills reduces reliability but they are
more valid because they test integrative knowledge of
the language so they give the teacher the opportunity
to see how students really use the language.
• Teacher can have a better view of their students’
competence of language.
• So the higher reliability, the less valid or all the
opposite.
11/4/2015 UNIVERISTY OF EDUCATION, LAHORE

8. RELIABILITY
• Reliability refers to the consistency of the
scoring of the test, both between different
raters, and between the same rater on
different occasions.
• Objective testing should give perfect
reliability.
• However, faulty tests (ambiguous multiple
choice, wrong answers on an answer sheet,
etc) can reduce the reliability of even an
objective test.
11/4/2015 UNIVERISTY OF EDUCATION, LAHORE

9. UTILITY, PRACTICALITY and DISCRIMINATION
• Utility: A test which provides a lot of feedback
to assist in the planning of the rest of a course
or future courses.
• Practicality: the efficiency of the test in
physical terms. (Does it require a lot of
equipment? Does it take a lot of time to set,
administer or mark?)
• Discrimination: the ability of a test to
discriminate between stronger and weaker
students.
11/4/2015 UNIVERISTY OF EDUCATION, LAHORE

10. UNIVERISTY OF EDUCATION, LAHORE11/4/2015
TEST/ Performance Appraisal Process

11. Issues in appraisal system
Appraisal
Design
Formal and informal
Whose performance?
Who are the raters?
What problems?
How to solve?
What to evaluate?
When to evaluate?
What methods?

12. Advantages
Provide a record of
performance over a
period of time.
Provide an
opportunity for a
manager to meet &
discuss performance
Provide the employee
with feedback about
their performance
Provide an
opportunity for an
employee to discuss
issues and to clarify
expectations
Can be motivational
with the support of a
good reward and
compensation
UNIVERISTY OF EDUCATION, LAHORE11/4/2015

13. If not done appropriately, can
be a negative experience.
Subject to rater errors &
biases.
If not done right can be a
complete waste of time.
Can be stressful for all
involved
Disadvantages
UNIVERISTY OF EDUCATION, LAHORE11/4/2015

14. What is Item Analysis ?
• Item analysis is a statistical technique which
is used for selecting and rejecting the items
of the test on the basis of their difficulty
value and discriminated power.
11/4/2015 UNIVERISTY OF EDUCATION, LAHORE

15. Objectives of Item Analysis
• To select appropriate items for the final draft
• To obtain the information about the difficulty value(D.V) of
all the items
• To provide discriminatory power (D.I) to differentiate
between capable and less capable examinees for the items
• To provide modification to be made in some of the items
• To prepare the final draft properly ( easy to difficult items)
11/4/2015 UNIVERISTY OF EDUCATION, LAHORE

16. STEPS for Item Analysis
• Arrange the scores in descending order
• Separate two sub groups of the test papers
• Take 27% of the scores out of the highest scores and 27%
of the scores falling at bottom
• Count the number of right answer in highest group (R.H)
and count the no of right answer in lowest group (R.L)
• Count the non-response (N.R) examinees
11/4/2015 UNIVERISTY OF EDUCATION, LAHORE

17. Difficulty value (D.V)
• “The difficulty value of an item is defined as the
proportion or percentage of the examinees who have
answered the item correctly” (J.P. Guilford)
• The formula for difficulty value (D.V)
• D.V = (R.H + R.L)/ (N.H + N.L)
• R.H – rightly answered in highest group
• R.L - rightly answered in lowest group
• N.H – no of examinees in highest group
• N.L - no of examinees in lowest group

18. Discrimination index (D.I)
• “Index of discrimination is that ability of an item on
the basis of which the discrimination is made
between superiors and inferiors” (Blood and Budd (1972)
• The formula for discrimination index(D.I)
• D.I = (R.H - R.L)/ (N.H or N.L)
• R.H – rightly answered in highest group
• R.L - rightly answered in lowest group
• N.H – no of examinees in highest group
• N.L - no of examinees in lowest group

19. General guidelines for discriminating index (D.I)
According to Ebel ,
D.I Item Evaluation
≥0.40 Very good items
0.30 - 0.39 Reasonably good but subject to
improvement
0.20 – 0.29 Marginal items , need
improvement
<0.19 Poor items . Rejected or revised

20. General guidelines for difficulty value (D.V)
Low difficulty value index means, that item is high
difficulty one
ex: D.V=0.20 » 20% only answered correctly for that
item. So that item is too difficult
High difficulty value index means, that item is easy
one
ex: D.V=0.80 » 80% answered correctly for that item. So
that item is too easy one

21. Criteria for selection and
rejection items
Positive discrimination index only selected
Negative and zero discrimination index items are
rejected
High and low difficulty value items are rejected

22. UNIVERISTY OF EDUCATION, LAHORE11/4/2015