ED450 Fall 2012

1. By: Lauren Chadwick
ED 450
Monday September 9, 2012

2. Bellwork
O Please use a pen or
pencil and draw 2
appointment slots Once you are finished
on the plate I gave open the folded paper
you when you and fill in the blank
entered the with the type of word
classroom: written on the outside.
1 2
Put paper in box.

3. SLO’s
O Students will …
O Understand reliability in assessment.
O Describe the 3 types of reliability evidence.
O Explain the importance of standard error of
measurement.

5. Let’s have some fun!
O Everyone get up and
stand in a circle!

7. Reliability = Consistency
O Reliability is an O 2 important ways
important criteria reliability may be
useful:
used when judging
O How reliable are
standardized tests. your assessments
for you and your
students?
O Parents may ask
how reliable
standardized tests
may be.

8. 3 Types of Reliability Evidence
O Stability
O Alternate Form
O Internal Consistency

9. Stability Reliability
O Consistency of test O Example: Midterm on
results over time Tuesday
O The goal is to see O A masked man grabs your briefcase
close scores in containing the students’ tests!
O The next day you tell the students
tests administered that their tests were stolen and that
even if they are they need to re-take the test!
given on different O Because there has been no new
occasions. information taught since the last
test, you expect that the test scores
would be similar.
O So, if you were able to see that the
scores were not similar, then your
exam would be judged to have little
stability reliability.

10. Test-Retest
O To see how stable an assessment’s
results are over time, we usually test
students, then in a week or two retest with
the same form of assessment.
O It is important though that there is no
significant event taking place in between
the two assessments. Example: field
trip, movie, speaker- exposing student to test-relevant
information.

11. So, how are scores computed?
Correlation Coefficient
(r)
So…..
O Reflects the degree of
O …if the test–retest r
similarity between
for two sets of test
scores on two tests/ scores were .84 this
O Ranges from: would show that the
+1.0 to -1.0 students’ performance
on both tests were
similar.
(close to 1.0 = strong relationship)
(closer to 0 = weak relationship) O If the r was instead
.23 this would show
that the results were
not as similar.

12. Classification- Consistency
O To decide which students
Student Test 1 Test 2
would be exempted from
further assessment. Jill 84% 99%
O Example: If you decide on Jack 65% 82%
an 80% answers correct
required for
exemption, then on a test-
retest approach, the
teacher would find the %
of students who scored ≥
80% on both
assessments.

13. How to determine
Classification-Consistency
A % of students exempt
42%
B % of students not exempt
46%
C % of students different classification
12%
D % of classification – consistency ( A + B)
-----------------------------------------------------------------------
-----------------------
88%
Classification- Consistency of that test

14. Alternate-Form Reliability
O Simply put: O Usually found in
settings where a
student is testing to
O Providing an receive his/her diploma.
equivalent retest O BUT keep in mind when
you are reviewing a
for students in state-developed test
another form other which claims to have
than the first and equivalent forms, that
failed assessment. those tests assess the
same information as the
original.

15. Internal Consistency
Reliability
O Reflects the degree O Example:
of consistency of
the items on a given 20 item test on problem
test. solving:
O Homogeneous
Students who are strong
O Each of the test’s problem solvers should
get most of the 20 items
items should be correct
assessing the same
variable.

16. FYI
O Tests containing items O Tests containing items
which a students can where students can
either be right or wrong be given different
such as multiple- grades such as in
choice, the most
commonly used essay items, the most
approaches are the: common approach is:
Kuder-Richardson Cronbach’s
procedures coefficient alpha

17. Just a thought…
O The more times you taste from a pot of
soup, the more accurate will be your
estimate of what the ingredients are.
O One taste might fool you, twenty tastes
will give you a much better idea of what’s
in the pot.
O More items on a test will yield more
reliable estimates of understanding.

18. Choose one…
Stability Reliability ≠ Alternate-Form Reliability ≠ Internal Consistency Reliability
These three forms of reliability
evidence should not be used
together. Instead, keep in mind
the kind of educational decision
linked to the results of the
assessment.

19. Standard Error of
Measurement
O Supplies information of test consistency of an
individual student by estimating their score
consistency.
O Problem: Teacher see’s student results from a
pervious IQ test. They expect that the student will
perform on that exact level
O SEM helps to remind teachers that scores
earned on commercial or classroom tests are
not always exact on student’s understanding.
O Another area where SEM is used is in
reporting of student performance on state
tests.

20. 60 item Math test
O Jack answers 53
Classificatio # correct
n
out of the 60
Advanced 54 – 60
questions correctly,
Proficient 44 – 53
so what would he
Basic 37 – 43
be classified as?
Below Basic 36 and below
a proficient student.

21. Appointment Time!
Everyone find a classmate for your 1st appointment.
You have 3 minutes to decide whether the following is
stability, alternate-form, or internal consistency reliability.
Suppose you were trying to show that the items on
an educational assessment all measured the same
variable. What type of reliability evidence would
you use?
Internal consistency reliability

22. Appointment Time!
Everyone find a classmate for your 2nd appointment.
You have 3 minutes to decide whether the following is stability,
alternate-form, or internal consistency reliability.
A new nationally standardized reading test for
junior high students has been created, but the test
developers want to make sure that the three
different forms of tests are performing in essentially
the same way. What sort of reliability evidence
should they rely on?
Alternate-form reliability

23. Why do you need to know?
O You may be called on O Understanding
to explain to parents standard error of
the meaning of their measurement will
child’s test scores. be helpful when you
need to explain to
O At least be parents and
knowledgeable about students how to
the meaning of make sense of their
reliability in educational scores on
assessments. accountability tests.

24. Check your understanding..
O Do you understand assessment reliability?
O Do you know the 3 types of reliability
evidence?
O Can you explain the importance of standard
error of measurement?