This document discusses the validity and reliability of analytical tests used for screening and diagnosis. It defines key terms like sensitivity, specificity, predictive value and discusses how changing cutoff levels can impact false positives and negatives. Screening tests are used to separate populations into those with and without a disease, while considering a test's accuracy. Continuous variable tests may require an artificial cutoff versus dichotomous screening tests. The document also examines how prevalence impacts predictive value and how using multiple screening tests can improve accuracy.

1. Validity and Reliability
of
Analytical Tests
BY
DR MUHAMMAD TAUSEEF JAVED
CONSULTANT MINISTRY OF HEALTH
MAKKAH
&
IPH LAHORE

2. Analytical Tests include both:
• Screening Tests
• Diagnostic Tests

3. •Two Important Objectives
To distinguish between people in the
population who have the diseases and
those who do not
To determine how good the test is in
separating populations of people with
and without the disease in question?

4. Epidemiological
Surveillance
vs.
Screening

5. Epidemiological
Surveillance
• What is it?
• Why do it?

6. Epidemiological
Surveillance
• Definition - ongoing & systematic
collection, analysis & interpretation of data
related to health, disease & conditions
• Two types
Passive Surveillance – uses available data or
reporting from health care provider or
regional health officer
Active Surveillance – periodic field visits to
health care facilities to identify new cases
• The present approach is the survey

7. Epidemiological
Surveillance
• Why do it?
Can help discover and control the
transmission of infectious diseases
Prevention and control programs can be
planned and implemented

8. Screening
• Definition - use of quick and simple
testing procedures to identify and
separate persons:
who have a disease from those that do not
OR
who are apparently (appear to be) well, but
who may be at risk of a disease, from
those who probably don’t have the
disease.

9. Terms Related to
Screening Tests
• Validity - relates to accuracy (correctness)
• Reliability - repeatability
• Yield - the # of tests that can be done in a
time period

10. Terms Related to Screening
Tests (cont’d)
• Sensitivity - ability of a test to
identify those who have disease
• Specificity - ability of a test to
exclude those who don’t have
disease

11. Terms Related to Screening
Tests (cont’d)
• Tests with dichotomous results – tests
that give either positive or negative
results
• Tests of continuous variables – tests that
do not yield obvious “positive” or
“negative” results, but require a cutoff
level to be established as criteria for
distinguishing between “positive” and
“negative” groups

12. An important public health
consideration, particularly in screening
free-living populations, is:
How good is the test at
identifying people
with the disease
and without the disease?
In other words:
If we screen a population, what
proportion of people who have the
disease will be correctly identified?

13. POPULATION
Test Results With Disease
Without
Disease
Positive
True Positive
(TP)
False Positive
(FP)
Negative
False Negative
(FN)
True Negative
(TN)

14. Sensitivity = = X 100
True positives
True positives +
false negatives
True positives
All persons with
the disease
= TP
TP + FN

15. Specificity = = X 100
True negatives
True negatives+
false positives
True negatives
All persons
without the
disease
= TN
TN + FP

16. Percent false negatives = % of people with the disease who
were not detected by the test
FN
FN + TP X 100

17. Percent false positives = % of people without the disease
who were incorrectly labeled by the
test as having the disease
FP
FP + TN X 100

18. In the clinical setting, a more important
question is:
If the test results are positive (or
negative) in a given patient, what is
the probability that this patient has (or
does not have) the disease?
In other words:
What proportion of patients who test
positive (or negative) actually have (or do
not have) the disease in question?

19. Predictive Value
Pos. PV = X 100 = %True Positives
TP + FP
Neg. PV = X 100 = %True Negatives
TN + FN

20. Biologic Variation of
Human Populations &
Diagnostic Issues

21. Distribution of Tuberculin Reactions
Bimodal Distribution
Easy to distinguish
between exposed group
and those not exposed.

22. Distribution of
Systolic Blood Pressure
Unimodal Distribution
With continuous variables, a
cutoff level must be established
to separate the hypertensive
group. Could choose based on
statistics, but better to base on
biologic considerations.

23. Effects of
Choosing
Different Cutoff
Levels for
Diabetes
Diagnosis in
Population with
50% Prevalence
Real World
Pseudo-
Real World

24. The major issue with deciding
to set a cutoff high or low
is the problem of
false positives
and false negatives.

25. Possible Groups
with Dichotomous Test
True Disease
Status is Known,
as with dichotomous tests.

26. Grouping All Positives
and All Negatives
True Disease Status
is Unknown,
as with continuous variables.
Artificial
Cutoff

27. •Use of Multiple Screening
Tests
Sequential (Two-stage) Testing
Simultaneous Testing

28. Hypothetical Two-Stage Screening
Only Pos. Test 1
are given Test 2

29. Hypothetical Two-Stage Screening (cont.)
TEST 2 (Glucose Tolerance Test)
Sensitivity = 90%
Specificity = 90%
DIABETES
+ -
TEST
RESULTS +
315 190 505
- 35 1710 1745
350 1900 2250
Net Sensitivity = 315/500 = 63%
Net Specificity = 7600 + 1710 = 98%
9500

30. Predictive Value

31. Prevalence & Predictive Value
Positive
As prevalence increases,
positive predictive value increases.

32. Prevalence & Predictive Value
Note: Test has
95% sensitivity and
95% specificity

33. Specificity &
Predictive
Value
As specificity
increases,
positive predictive
value increases.
As sensitivity increases,
positive predictive value
also increases, but to a
much lesser extent.

34. Specificity & Predictive Value
As specificity increases,
positive predictive value increases.

35. Results
reliable but
NOT valid
Results
reliable and
valid
Reliability (Repeatability) of Tests