1. Errors in Statistical Surveys (Sampling, Non-
sampling, Derivation and Effects)
Presented
Okpe ThankGod Damion
THANKGOD COMPUTER INSTITUTE
12-Oct-13ThankGod Computer Institute (TCI)1
2. Outline of Presentation
Introduction
Types of Errors in Statistical Survey
Stages of Non Sampling Errors
Non Response Errors
Response Errors
Control of Errors in Statistical Survey
12-Oct-13ThankGod Computer Institute (TCI)2
3. The quality of survey results is a measure of the
following three items:
How relevant is the result of the survey to the
objective.
How accurate it is to the objective of the survey.
The timeliness of the result.
12-Oct-13ThankGod Computer Institute (TCI)3
4. Types of Errors
There are two major types of errors in statistical
surveys which can affect the accuracy of the
survey data:
Sampling Errors
Non-sampling Errors
We shall treat these errors separately
12-Oct-13ThankGod Computer Institute (TCI)4
5. Sampling Errors
The errors are due to the fact that data are collected only
for a sample of the target population with consequence
that the estimate derived may differ from values that
would have been obtained from complete census
The effect of this type of error can be reduced or
minimized by increasing the sample size or improvement
in the sample design efficiency.
12-Oct-13ThankGod Computer Institute (TCI)5
6. Sampling Errors Contd.
The magnitude of sampling errors depends on the
sample design and the estimation procedure used
These two can be reduced by increasing
Sample size
Improving sample design
By using:
stratification or multi-stage design
varying selection probabilities
improved estimation procedures
12-Oct-13ThankGod Computer Institute (TCI)6
7. Non- Sampling Errors
These are errors which are not due to sampling
In order words they are the residual errors i.e. all
other types of errors which are not resulting from
sampling and affecting the quality of data
collection
Some of these non-sampling errors could
seriously affect the quality of data collected
12-Oct-13ThankGod Computer Institute (TCI)7
8. Non- Sampling Errors Contd.
Non-sampling errors are of many sources and
have many methods of controlling them
The non-sampling errors are:
Coverage errors
Response errors
Processing errors
12-Oct-13ThankGod Computer Institute (TCI)8
9. Non- Sampling Errors Contd.
Coverage errors
Are the errors that occur due to the difference between
what is actually covered from what ought to have been
covered which can either be over coverage or under-
coverage as the case may be
Response errors
Occur due to the difference in the answers actually
recorded for a question and what ought to be the correct
answers or answer.
Processing errors
Are errors that set-in due to editing, coding,
punching/data keying, etc.
12-Oct-13ThankGod Computer Institute (TCI)9
10. Stages of Non – Sampling Errors
Non-sampling errors can be classified into three
stages
Survey design and Planning Stage
Data collection Stage
Data processing and Analysis Stage
We shall now consider these stages one by one and
identify the probable errors that can come up at any
of the stage
12-Oct-13ThankGod Computer Institute (TCI)10
11. Stages of Non – Sampling Errors 10
Survey Design/Planning
Under this stage the types of error that can occur are either
coverage, non-response and response errors
Coverage Errors
The objective of the sample survey is to make inferences
about a desired target population from the observation of
units confined to a sample
The selection of the units is done by a randomized
procedure in which all units of the target population are
put which we call
12-Oct-13ThankGod Computer Institute (TCI)11
12. Stages of Non – Sampling Errors
Sampling Frame Errors
A situation where any of the unit in the frame is not covered,
results in Non-coverage which in turn give rise to coverage error
This may include a situation where some units of observation
either directly or implicitly in the operational sampling frame are
excluded
Also it may be a case of over-coverage in which case some
units appear more than once in the frame in which case we say
the sampling frame is defective
Coverage errors may occur as a result of selection of EAs,
Wrong Geographic Codes, overlapping EA boundaries, Listing
Exercise, Selection of the Ultimate Sample Units, Incorrect
Application of Sampling Procedures, Incorrect application of
rules of Association, etc
12-Oct-13ThankGod Computer Institute (TCI)12
13. Stages of Non – Sampling Errors
Estimation of Coverage Errors
Let us look at how we can estimate the magnitude of
the effect of coverage errors on our survey results in
other to determine the quality of our data
Though it is not easy to estimate the quantity of non-
coverage error and it is also expensive but we shall
mention a few methods to estimate it
12-Oct-13ThankGod Computer Institute (TCI)13
14. Stages of Non – Sampling Errors
Some of the methods include re-interview e.g. conducting a
post enumeration check on sub-sample of the survey
records with some independent source, analytically, we
may use data from other sources such as prior census, or
vital records, external migration, etc which are secondary
data to develop values for the total population and compare
with corresponding survey figures
We may also compare with aggregates from administrative
records.
12-Oct-13ThankGod Computer Institute (TCI)14
15. Non-response Errors
Non-response Errors
This is a case where one can not obtain data or
information from a selected unit of observation
It is either total i.e unit non-response or partial i.e item
non-response
To measure effect of non-response, it can be by any of
the following approaches:
Measuring the response rate in case of unit non-response or
Item response rate in the case of item non-response.
12-Oct-13ThankGod Computer Institute (TCI)15
16. Non-response Errors
The measures may give an indication of response
bias and pointer to specific problems which may
call for urgent solution either by reversal or
otherwise.
In most cases the respondents will return the
instrument back to the interviewer.
12-Oct-13ThankGod Computer Institute (TCI)16
17. Non-response Errors
Non response could be due to
Failure to gain access to sample units and may be as a
result of non-accessibility to the EA.
Failure to contact the respondent (a case of proxy
respondent)
Failure to gain the cooperation of the respondent which
may be complete or partial situation
Response burden as result of the length of the
questionnaire, e.g. enough time, memory lapse, lack of
documentation or keeping of diary, interviewer level of
education, experience, age etc.
12-Oct-13ThankGod Computer Institute (TCI)17
18. Non-response Errors 17
Improving Response Rate
In other to ensure a good quality data, efforts
must be made as a matter of policy to
maximize response in surveys
To achieve this, response rates had to be
improved by either of the following or
combination of all
12-Oct-13ThankGod Computer Institute (TCI)18
19. Non-response Errors 18
Contacting respondents
Improving Sampling Frame
Reduce time lag between listing and conduction of the
interview.
Make many call-backs as situation may warrant
Gaining respondents co-operation
Intensive training of the interviewers
Close supervision
Careful choice of interviewers (and should be well
motivated)
12-Oct-13ThankGod Computer Institute (TCI)19
20. How to Deal with Non-response Errors
Ways of compensating for non-response include
the following:
Intensive follow-up during the Data Collection process of sub sample of
the non-respondents
Collection of limited information from neighbours of the households that
were away.
Substitution method may be used i.e substituting similar units or
elements but must be made within the homogenous group if it is to be
efficient.
This method requires that the substitution be made in the field so that
the selected substitutes could be interviewed directly
This method is not encouraged and it requires skill staff
12-Oct-13ThankGod Computer Institute (TCI)20
21. How to Deal with Non-response Errors 20
Estimation based methods e.g. use of adjustment factors
Imputation (Replacing missing information) from useable data from
other sources
It is mostly used when treating item non-response. Some of the
imputation methods includes deductive imputation, mean value
impartation, registration method etc
Deductive imputation is made when the missing response in a
questionnaire can be deduced with certainty based on other
information on the same records e.g. a questionnaire on fertility with
response for number of birth but fails to put the sex of the respondent
on the same record, we could easily deduce that the sex is female
12-Oct-13ThankGod Computer Institute (TCI)21
22. ThankGod Computer Institute (TCI)
Response Errors
Sources of response errors can be traced to the followings
Interviewer inadequacy
Inability of respondent to provide the desired information
Interviewer’s inadequacy can be as a result of
Failure to put the questions clearly
Influencing respondents to answer incorrectly
Mis-recording correct responses
While respondent’s inability to provide the information may be as
a result of not able to provide the desired information for some
reasons which may include
12-Oct-13ThankGod Computer Institute (TCI)22
23. Response Errors 22
Limit impose by their knowledge, e.g. age, size
of holdings
Inability to recall or report facts correctly at the
time of interview
Deliberate mis-information or withholding
information due to dignity, tribal sentiment or
prestige
12-Oct-13ThankGod Computer Institute (TCI)23
24. Response Error
Response error depends largely on:
Design of the Survey Operation
Nature and Complexity of its Content
Systems of concepts and definitions
Design and layout of questionnaire
Wording of the questions in the questionnaire
Adequacy of the training
Monitoring and supervision programmes put in place
during the data collection process
12-Oct-13ThankGod Computer Institute (TCI)24
25. Control of errors in statistical survey
Generally, errors in survey can be controlled through
either of the followings or combination of the
followings:
Adequate planning and preparatory work including
development of survey instruments
Adequate training of field personnel
Effective monitoring and supervision of field work
during the data collection phase
12-Oct-13ThankGod Computer Institute (TCI)25
26. Control of errors in statistical survey
contd.
Skim-checking of work at various stages of
the data collection
Also, on the spot check i.e. spot check of
data collected should be done during the data
collection phase
Post enumeration survey (PES) for the
evaluation of survey results should be done
and finally documentation of errors by
sources, type and magnitude should be done.
12-Oct-13ThankGod Computer Institute (TCI)26