Mba2216 week 07 08 measurement and data collection forms

Research Design :Research Design :
Measurement &Measurement &
Data Collection FormsData Collection Forms
Research Design :Research Design :
Measurement &Measurement &
Data Collection FormsData Collection Forms
MBA2216 BUSINESS RESEARCH PROJECTMBA2216 BUSINESS RESEARCH PROJECT
by
Stephen Ong
Visiting Fellow, Birmingham City
University, UK

6-2
Design in the Research ProcessDesign in the Research Process

MeasurementMeasurement
ConceptsConcepts
13–3

13–4
LEARNING OUTCOMESLEARNING OUTCOMESLEARNING OUTCOMESLEARNING OUTCOMES
1. Determine what needs to be measured to address
a research question or hypothesis
2. Distinguish levels of scale measurement
3. Know how to form an index or composite measure
4. List the three criteria for good measurement
5. Perform a basic assessment of scale reliability
and validity
After this lecture, you should be able to

6. Describe how business researchers think of
attitudes
7. Identify basic approaches to measuring attitudes
8. Discuss the use of rating scales for measuring
attitudes
9. Represent a latent construct by constructing a
summated scale
10. Summarize ways to measure attitudes with ranking
and sorting techniques
11. Discuss major issues involved in the selection of a
measurement scale
13–5
LEARNING OUTCOMESLEARNING OUTCOMES
(cont’d)(cont’d)
LEARNING OUTCOMESLEARNING OUTCOMES

12. Explain the significance of decisions about
questionnaire design and wording
13. Define alternatives for wording open-ended and
fixed-alternative questions
14. Summarize guidelines for questions that avoid
mistakes in questionnaire design
15. Describe how the proper sequence of questions
may improve a questionnaire
16. Discuss how to design a questionnaire layout
17. Describe criteria for pretesting and revising a
questionnaire and for adapting it to global markets
LEARNING OUTCOMES (cont’d)LEARNING OUTCOMES (cont’d)LEARNING OUTCOMES (cont’d)LEARNING OUTCOMES (cont’d)

11-7
FromFrom
InvestigativeInvestigative
toto
QuestionsQuestions

WHAT DO I MEASURE?WHAT DO I MEASURE?
 Before the measurement process can be defined,
researchers have to decide exactly what it is that
needs to be produced.
 The decision statement, corresponding research
questions and research hypotheses can be used to
decide what concepts need to be measured.
 Measurement is the process of describing some
property of a phenomenon of interest usually by
assigning numbers in a reliable and valid way.
 When numbers are used, the researcher must have
a rule for assigning a number to an observation in a
way that provides an accurate description.
 All measurement, particularly in the social sciences,
contains error.
13–8

Concepts
A researcher has to know what to measure before
knowing how to measure something.
A concept is a generalized idea that represents
something of meaning.
Concepts such as age, sex, education and number of
children are relatively concrete properties and present
few problems in either definition or measurement.
Concepts such as brand loyalty, corporate culture,
and so on are more abstract and are more difficult to
both define and measure.
13–9

Operational Definitions
Researchers measure concepts through a process
known as operationalization, which is a process that
involves identifying scales that correspond to variance
in the concept.
Scales provide a range of values that correspond to
different values in the concept being measured.
Scales provide correspondence rules that indicate
that a certain value on a scale corresponds to some
true value of a concept, hopefully in a truthful way.
13–10

WHAT DO I MEASURE? (cont’d)WHAT DO I MEASURE? (cont’d)
Operational Definitions (cont’d)
Variables
 Researchers use variance in concepts to
make diagnoses.
 Variables capture different concept values.
 Scales capture variance in concepts and as
such, the scales provide the researcher’s
variables.
 For practical purposes, once a research
project is underway, there is little difference
between a concept and a variable.

Operational Definitions (cont’d)
Constructs
 Sometimes a single variable cannot capture a
concept alone.
 Using multiple variables to measure one
concept can often provide a more complete
account of some concept than could any
single variable.
 A construct is a term used for concepts that
are measured with multiple variables.
 Can be very helpful in operationlizing a
concept.
13–12

EXHIBIT 13.EXHIBIT 13.33 Susceptibility to Interpersonal Influence: An Operational DefinitionSusceptibility to Interpersonal Influence: An Operational Definition

11-14
SelectSelect
measurable phenomenameasurable phenomena
Develop a set ofDevelop a set of
mapping rulesmapping rules
Apply the mapping ruleApply the mapping rule
to each phenomenonto each phenomenon

11-15
Characteristics of MeasurementCharacteristics of Measurement

11-16
Types of ScalesTypes of Scales
OrdinalOrdinal
intervalinterval
NominalNominal
RatioRatio

11-17
Levels of MeasurementLevels of Measurement
OrdinalOrdinal
intervalinterval
RatioRatio
NominalNominalNominalNominal ClassificationClassification

11-18
Nominal ScalesNominal Scales
Mutually exclusiveMutually exclusive
andand
Collectively exhaustiveCollectively exhaustive
categoriescategories
Exhibits onlyExhibits only
classificationclassification

11-19
OrdinalOrdinalOrdinalOrdinal
intervalinterval
RatioRatio
NominalNominal ClassificationClassification
OrderOrder
ClassificationClassification

11-20
Ordinal ScalesOrdinal Scales
• Characteristics ofCharacteristics of
nominal scalenominal scale
• OrderOrder
• Implies greater thanImplies greater than
or less thanor less than

11-21
OrdinalOrdinal
IntervalIntervalIntervalInterval
RatioRatio
OrderOrder
OrderOrder
ClassificationClassification DistanceDistance

11-22
Interval ScalesInterval Scales
Characteristics ofCharacteristics of
nominal and ordinalnominal and ordinal
scalesscales
Equality of interval.Equality of interval.
Equal distanceEqual distance
between numbersbetween numbers

11-23
OrdinalOrdinal
intervalinterval
RatioRatioRatioRatio
OrderOrder
OrderOrder
Natural OriginNatural Origin
OrderOrder

11-24
Ratio ScalesRatio Scales
Characteristics ofCharacteristics of
nominal, ordinal,nominal, ordinal,
interval scalesinterval scales
Absolute zeroAbsolute zero

Levels of Scale MeasurementLevels of Scale Measurement
 The level of scale measurement is important
because it determines the mathematical
comparisons that are allowed.
 The four levels of scale measurement are:

13–26
 Nominal
 Assigns a value to an object for
identification or classification purposes.
 Most elementary level of measurement.
 Ordinal
 Ranking scales allowing things to be
arranged based on how much of some
concept they possible.
 Have nominal properties.

13–27
 Interval
 Capture information about differences in
quantities of a concept.
 Have both nominal and ordinal properties.
 Ratio
 Highest form of measurement.
 Have all the properties of interval scales
with the additional attribute of
representing absolute quantities.
 Absolute zero.

EXHIBIT 13.EXHIBIT 13.44 Nominal, Ordinal, Interval, and Ratio Scales Provide DifferentNominal, Ordinal, Interval, and Ratio Scales Provide Different
InformationInformation

EXHIBIT 13.EXHIBIT 13.55 Facts About the Four Levels of ScalesFacts About the Four Levels of Scales

12-30
Measurements are RelativeMeasurements are Relative
“Any measurement must take into account
the position of the observer. There is no
such thing as measurement absolute, there
is only measurement relative.”
Jeanette Winterson
journalist and author

12-31
The Scaling ProcessThe Scaling Process

12-32
Nature of AttitudesNature of Attitudes
Cognitive
I think oatmeal is healthier
than corn flakes for breakfast.
Affective
Behavioural
I hate corn flakes.
I intend to eat more oatmeal
for breakfast.

12-33
Improving PredictabilityImproving Predictability
Reference
groups
Reference
groups
Multiple
measures
Multiple
measures
FactorsFactors
StrongStrong
Specific
Basis
DirectDirect

12-34
Measurement ScalesMeasurement Scales
“All survey questions must be
actionable if you want results.”
Frank Schmidt, senior scientist
The Gallup Organization

12-35
Selecting aSelecting a
Measurement ScaleMeasurement Scale
Research objectives Response types
Data properties
Number of
dimensions
Forced or unforced
choices
Balanced or
unbalanced
Rater errors
Number of
scale points

12-36
Response TypesResponse Types
Rating scaleRating scale
Ranking scaleRanking scale
CategorizationCategorization
SortingSorting

12-37
Number of DimensionsNumber of Dimensions
Unidimensional
Multi-dimensional

12-38
Balanced or UnbalancedBalanced or Unbalanced
Very badVery bad
BadBad
Neither good norNeither good nor
badbad
GoodGood
Very goodVery good
PoorPoor
FairFair
GoodGood
Very goodVery good
ExcellentExcellent
How good an actress is Angelina Jolie?

12-39
Forced or Unforced ChoicesForced or Unforced Choices
Very badVery bad
BadBad
Neither good nor badNeither good nor bad
GoodGood
Very goodVery good
Very badVery bad
BadBad
GoodGood
Very goodVery good
No opinionNo opinion
Don’t knowDon’t know

12-40
Number of Scale PointsNumber of Scale Points
Very badVery bad
BadBad
Neither good norNeither good nor
badbad
GoodGood
Very goodVery good
Very badVery bad
Somewhat badSomewhat bad
A little badA little bad
A little goodA little good
Somewhat goodSomewhat good
Very goodVery good

12-41
Rater ErrorsRater Errors
Error of
central tendency
Error of leniency
•Adjust strength of
descriptive adjectives
•Space intermediate
descriptive phrases
farther apart
•Provide smaller
differences
in meaning between
terms near the
ends of the scale
•Use more scale points

12-42
Primacy Effect
Recency Effect
Reverse order of
alternatives periodically
or randomly

12-43
Halo Effect
• Rate one trait
at a time
• Reveal one trait
per page
• Reverse anchors
periodically

ATTITUDES AS HYPOTHETICALATTITUDES AS HYPOTHETICAL
CONSTRUCTSCONSTRUCTS
 Attitude
 An enduring disposition to consistently
respond in a given manner to various aspects
of the world.
 Components of attitudes:
 Affective Component
 The feelings or emotions toward an object
 Cognitive Component
 Knowledge and beliefs about an object
 Behavioural Component
 Predisposition to action
 Intentions
 Behavioural expectations

Techniques for MeasuringTechniques for Measuring
AttitudesAttitudes
 Ranking
 Requiring the respondent to rank order
objects in overall performance on the
basis of a characteristic or stimulus.
 Rating
 Asking the respondent to estimate the
magnitude of a characteristic, or quality,
that an object possesses by indicating on
a scale where he or she would rate an
object.

14–46
Techniques for MeasuringTechniques for Measuring
Attitudes (cont’d)Attitudes (cont’d)
 Sorting
 Presenting the respondent with several
concepts typed on cards and requiring the
respondent to arrange the cards into a
number of piles or otherwise classify the
concepts.
 Choice
 Asking a respondent to choose one
alternative from among several
alternatives; it is assumed that the chosen
alternative is preferred over the others.

Attitude Rating ScalesAttitude Rating Scales
 Simple Attitude Scale
 Requires that an individual agree/disagree
with a statement or respond to a single
question.
 This type of self-rating scale classifies
respondents into one of two categories (e.g.,
yes or no).
 Example:
THE PRESIDENT SHOULD RUN FOR RE-ELECTION
_______ AGREE ______ DISAGREE

12-48
Simple Category ScaleSimple Category Scale
I plan to purchase a MindWriter laptop in the
12 months.
 Yes
 No

Attitude Rating Scales (cont’d)Attitude Rating Scales (cont’d)
 Category Scale
 A more sensitive measure than a simple
scale in that it can have more than two
response categories.
 Question construction is an extremely
important factor in increasing the usefulness of
these scales.
 Example:
How important were the following in your decision to visit San Diego? (check one for each item)
VERY SOMEWHAT NOT TOO
IMPORTANT IMPORTANT IMPORTANT
CLIMATE ___________ ___________ ___________
COST OF TRAVEL ___________ ___________ ___________
FAMILY ORIENTED ___________ ___________ ___________
EDUCATIONAL/HISTORICAL ASPECTS ___________ ___________ ___________
FAMILIARITY WITH AREA ___________ ___________ ___________

EXHIBIT 14.EXHIBIT 14.11 Selected Category ScalesSelected Category Scales

12-51
Multiple-Choice,Multiple-Choice,
Single-Response ScaleSingle-Response Scale
What newspaper do you read most often for financial news?
 East City Gazette
 West City Tribune
 Regional newspaper
 National newspaper
 Other (specify:_____________)

12-52
Multiple-Choice,Multiple-Choice,
Multiple-Response ScaleMultiple-Response Scale
What sources did you use when designing your new
home? Please check all that apply.
 Online planning services
 Magazines
 Independent contractor/builder
 Designer
 Architect
 Other (specify:_____________)

12-53
Likert ScaleLikert Scale
The Internet is superior to traditional libraries for
comprehensive searches.
 Strongly disagree
 Disagree
 Neither agree nor disagree
 Agree
 Strongly agree

 Likert Scale
 A popular means for measuring attitudes.
 Respondents indicate their own attitudes
by checking how strongly they agree or
disagree with statements.
 Typical response alternatives: “strongly
agree,” “agree,” “uncertain,” “disagree,” and
“strongly disagree.”
 Example:
It is more fun to play a tough, competitive tennis match than to
play an easy one.
___Strongly Agree ___Agree ___Not Sure ___Disagree ___Strongly Disagree

EXHIBIT 14.EXHIBIT 14.22 Likert Scale Items for Measuring Attitudes toward Patients’Likert Scale Items for Measuring Attitudes toward Patients’
Interaction with a Physician’s Service StaffInteraction with a Physician’s Service Staff

12-56
Semantic DifferentialSemantic Differential

14–57
 Semantic Differential
 A series of seven-point rating scales with
bipolar adjectives, such as “good” and
“bad,” anchoring the ends (or poles) of the
scale.
 A weight is assigned to each position on the
scale. Traditionally, scores are 7, 6, 5, 4, 3, 2, 1,
or +3, +2, +1, 0, -1, -2, -3.
 Example:
ExcitingExciting ___ : ___ : ___ : ___ : ___ : ___ : ___ Calm___ : ___ : ___ : ___ : ___ : ___ : ___ Calm
InterestingInteresting ___ : ___ : ___ : ___ : ___ : ___ : ___ Dull___ : ___ : ___ : ___ : ___ : ___ : ___ Dull
SimpleSimple ___ : ___ : ___ : ___ : ___ : ___ : ___ Complex___ : ___ : ___ : ___ : ___ : ___ : ___ Complex
PassivePassive ___ : ___ : ___ : ___ : ___ : ___ : ___ Active___ : ___ : ___ : ___ : ___ : ___ : ___ Active

EXHIBIT 14.EXHIBIT 14.33 Semantic Differential Scales for Measuring Attitudes TowardSemantic Differential Scales for Measuring Attitudes Toward
SupermarketsSupermarkets

12-59
Adapting SD ScalesAdapting SD Scales
Convenience of Reaching the Store from Your Location
Nearby ___: ___: ___: ___: ___: ___: ___: Distant
Short time required to reach store ___: ___: ___: ___: ___: ___: ___: Long time required to reach store
Difficult drive ___: ___: ___: ___: ___: ___: ___: Easy Drive
Difficult to find parking place ___: ___: ___: ___: ___: ___: ___: Easy to find parking place
Convenient to other stores I shop ___: ___: ___: ___: ___: ___: ___: Inconvenient to other stores I shop
Products offered
Wide selection of different
kinds of products ___: ___: ___: ___: ___: ___: ___:
Limited selection of different
kinds of products
Fully stocked ___: ___: ___: ___: ___: ___: ___: Understocked
Undependable products ___: ___: ___: ___: ___: ___: ___: Dependable products
High quality ___: ___: ___: ___: ___: ___: ___: Low quality
Numerous brands ___: ___: ___: ___: ___: ___: ___: Few brands
Unknown brands ___: ___: ___: ___: ___: ___: ___: Well-known brands

12-60
SD Scale for Analyzing ActorSD Scale for Analyzing Actor
CandidatesCandidates

12-61
Graphic of SD AnalysisGraphic of SD Analysis

Other Scale Types (cont’d)Other Scale Types (cont’d)
 Image Profile
 A graphic representation of semantic
differential data for competing brands,
products, or stores to highlight
comparisons.
 Because the data are assumed to be
interval, either the arithmetic mean or the
median will be used to compare the profile
of one product, brand, or store with that of
a competing product, brand, or store.

EXHIBIT 14.EXHIBIT 14.44 Image Profiles of Commuter Airlines versus Major AirlinesImage Profiles of Commuter Airlines versus Major Airlines

12-64
Numerical ScaleNumerical Scale

 Numerical Scales
 Scales that have numbers as response
options, rather than “semantic space” or
verbal descriptions, to identify categories
(response positions).
 In practice, researchers have found that a scale
with numerical labels for intermediate points
on the scale is as effective a measure as the
true semantic differential.
 Example:
 Now that you’ve had your automobile for about
one year, please tell us how satisfied you are
with your Ford Taurus.
Extremely Dissatisfied 1 2 3 4 5 6 7 Extremely Satisfied

12-66
Multiple Rating List ScalesMultiple Rating List Scales
“Please indicate how important or unimportant each service characteristic
is:”
IMPORTANT UNIMPORTANT
Fast, reliable repair 7 6 5 4 3 2 1
Service at my location 7 6 5 4 3 2 1
Maintenance by manufacturer 7 6 5 4 3 2 1
Knowledgeable technicians 7 6 5 4 3 2 1
Notification of upgrades 7 6 5 4 3 2 1
Service contract after warranty 7 6 5 4 3 2 1

12-67
Stapel ScalesStapel Scales

 Stapel Scale
 Uses a single adjective as a substitute for
the semantic differential when it is
difficult to create pairs of bipolar
adjectives.
 Tends to be easier to conduct and
administer than a semantic differential
scale.

EXHIBIT 14.EXHIBIT 14.55 A Stapel Scale for Measuring a Store’s ImageA Stapel Scale for Measuring a Store’s Image

12-70
Constant-Sum ScalesConstant-Sum Scales

 Constant-Sum Scale
 Respondents are asked to divide a constant sum
to indicate the relative importance of attributes.
 Respondents often sort cards, but the task may also be
a rating task (e.g., indicating brand preference).
 Example:
 Divide 100 points among each of the following
brands according to your preference for the
brand:
 Brand A _________
 Brand B _________
 Brand C _________

12-72
Graphic Rating ScalesGraphic Rating Scales

EXHIBIT 14.EXHIBIT 14.77 A Ladder ScaleA Ladder Scale

EXHIBIT 14.EXHIBIT 14.88 Graphic Rating Scale with Picture ResponseGraphic Rating Scale with Picture Response
Categories Stressing Visual CommunicationCategories Stressing Visual Communication

 Graphic Rating Scale
 A measure of attitude that allows
respondents to rate an object by choosing
any point along a graphic continuum.
 Advantage:
 Allows the researcher to choose any interval
desired for scoring purposes.
 Disadvantage:
 There are no standard answers.

EXHIBIT 14.EXHIBIT 14.66 Graphic Rating ScaleGraphic Rating Scale

EXHIBIT 14.EXHIBIT 14.99 Summary of Advantages and Disadvantages of Rating ScalesSummary of Advantages and Disadvantages of Rating Scales

12-78
Ranking ScalesRanking Scales
Paired-comparison scale
Forced ranking scale
Comparative scale

RankingRanking
 An ordinal scale may be developed by
asking respondents to rank order (from
most preferred to least preferred) a set
of objects or attributes.
 Paired comparisons
 Sorting

12-80
Paired-Comparison ScalePaired-Comparison Scale

Paired ComparisonPaired Comparison
 A measurement technique that involves
presenting the respondent with two
objects and asking the respondent to
pick the preferred object; more than
two objects may be presented, but
comparisons are made in pairs.
 Number of comparisons = [(n)(n-1)/2]
 Example:
I would like to know your overall opinion of two brands of adhesive bandages.
They are MedBand and Super-Aid. Overall, which of these two brands—
MedBand or Super-Aid—do you think is the better one? Or are both the same?
MedBand is better _____
Super-Aid is better _____
They are the same _____

12-82
Forced Ranking ScaleForced Ranking Scale

12-83
Comparative ScaleComparative Scale

SortingSorting
 Require that respondents indicate their
attitudes or beliefs by arranging items
on the basis of perceived similarity or
some other attribute.
 Example:
Here is a sheet that lists several airlines. Next to the name of each airline is a pocket. Here
are ten cards. I would like you to put these cards in the pockets next to the airlines you
would prefer to fly on your next trip. Assume that all of the airlines fly to wherever you
would choose to travel. You can put as many cards as you want next to an airline, or you
can put no cards next to an airline.
Cards
American Airlines _____
Delta Airlines _____
United Airlines _____
Southwest Airlines _____
Northwest Airlines _____

12-86
Example : MindWriter ScalingExample : MindWriter Scaling
Likert Scale
The problem that prompted service/repair was resolved
Strongly
Disagree Disagree
Neither Agree
Nor Disagree Agree
Strongly
Agree
1 2 3 4 5
Numerical Scale (MindWriter’s Favourite)
To what extent are you satisfied that the problem that prompted service/repair was resolved?
Very
Dissatisfied
Very
Satisfied
1 2 3 4 5
Hybrid Expectation Scale
Resolution of the problem that prompted service/repair.
Met Few
Expectations
Met Some
Expectations
Met Most
Expectations
Met All
Expectations
Exceeded
Expectations
1 2 3 4 5

12-87
Ideal Scalogram PatternIdeal Scalogram Pattern
Item
Participant
Score
2 4 1 3
X X X X 4
__ X X X 3
__ __ X X 2
__ __ __ X 1
__ __ __ __ 0

Measuring Behavioural IntentionMeasuring Behavioural Intention
 Behavioural Component
 The behavioural expectations (expected
future actions) of an individual toward an
attitudinal object.
 Example:
 How likely is it that you will purchase a Honda
Fit?
 I definitely will buy
 I probably will buy
 I might buy
 I probably will not buy
 I definitely will not buy

Measuring Behavioural IntentionMeasuring Behavioural Intention
 Behavioural Differential
 A rating scale instrument similar to a
semantic differential, developed to measure
the behavioural intentions of subjects toward
future actions.
 A description of the object to be judged is placed
on the top of a sheet, and the subjects indicate
their behavioural intentions toward this object on a
series of scales.
 Example:
A 25 year-old woman sales representative
Would ___ : ___ : ___ : ___ : ___ : ___ : ___ : Would Not
ask this person for advice.

Mathematical and StatisticalMathematical and Statistical
Analysis of ScalesAnalysis of Scales
 Although you can put numbers into
formulas and perform calculations
with almost any numbers, the
researcher has to know the meaning
behind the numbers before useful
conclusions can be drawn (e.g.,
averaging the numbers used to
identify school busses is
meaningless).

Analysis of Scales (cont’d)Analysis of Scales (cont’d)
 Discrete Measures
 Discrete measures are those that take on only one
of a finite number of values.
 Most often used to represent a classificatory
variable and thus do not represent intensity of
measures, only membership.
 Common discrete scales include any yes-no
response, matching, colour choice or practically all
scales that involve selecting from a small number
of categories.
 Nominal and ordinal scales are discrete measures.
 The central tendency of discrete measures is best
captured by the mode (i.e., most frequent level).

13–92
Analysis of Scales (cont’d)Analysis of Scales (cont’d)
 Continuous Measures
 Continuous measures are those assigning values
anywhere along some scale range in a place that
corresponds to the intensity of some concept.
 Ratio measures are continuous measures.
 Strictly speaking, interval scales are not necessarily
continuous.
 e.g., Likert item ranging from 1=strongly
disagree to 5=strongly agree.
 This is a discrete scale because only the
values 1, 2, 3, 4, or 5 can be assigned.

Index MeasuresIndex Measures
 Attributes
 Single characteristics or fundamental features
that pertain to an object, person, or issue.
 Index Measures
 Assign a value based on how much of the
concept being measured is associated with
an observation.
 Indexes often are formed by putting several
variables together.
 Composite Measures
 Assign a value to an observation based on a
mathematical derivation of multiple variables.

Computing Scale ValuesComputing Scale Values
 Summated Scale
 A scale created by simply summing
(adding together) the response to each
item making up the composite measure.
 Reverse Coding
 Means that the value assigned for a
response is treated oppositely from the
other items.

EXHIBIT 13.EXHIBIT 13.66 Computing a Composite ScaleComputing a Composite Scale

Three Criteria for GoodThree Criteria for Good
SensitivitySensitivitySensitivitySensitivity
ReliabilityReliabilityReliabilityReliability ValidityValidityValidityValidity
GoodGood
GoodGood

ReliabilityReliability
 Reliability
 Reliability is an indicator of a measure’s
internal consistency.
 A measure is reliable when different attempts
at measuring something converge on the
same result.
 When the measuring process provides
reproducible results, the measuring
instrument is reliable.
 Internal Consistency
 Represents a measure’s homogeneity or the
extent to which each indicator of a concept
converges on some common meaning.
 Measured by correlating scores on subsets
of items making up a scale.

Internal ConsistencyInternal Consistency
 Split-half Method
 Assessing internal consistency by
checking the results of one-half of a set of
scaled items against the results from the
other half.
 The two scale halves should correlate
highly.
 They should also produce similar scores.

Internal Consistency (cont’d)Internal Consistency (cont’d)
 Coefficient alpha (α)
 The most commonly applied estimate of a multiple
item scale’s reliability.
 Represents the average of all possible split-half
reliabilities for a construct.
 The coefficient demonstrates whether or not the
different items converge.
 Ranges in value from 0 (no consistency) to 1
(complete consistency).
 Generally, scales with a coefficient α:

Test-Retest ReliabilityTest-Retest Reliability
 Test-retest Method
 Administering the same scale or measure to
the same respondents at two separate points
in time to test for stability.
 Represents a measure’s repeatability.
 Problems:
 The pre-measure, or first measure, may
sensitize the respondents and subsequently
influence the results of the second measure.
 Time effects that produce changes in attitude
or other maturation of the subjects.

ValidityValidity
 ValidityValidity
 Good measures should be both precise
(i.e., reliable) and accurate (i.e., valid).
 Validity is the accuracy of a measure or
the extent to which a score truthfully
represents a concept.
Does a scale measure what was intended
to be measured?
 When a measure lacks validity, any
conclusions based on that measure are
also likely to be faulty.

Validity : Face, Content …Validity : Face, Content …
 Establishing Validity:
 The four basic approaches to establishing
validity are face validity, content validity,
criterion validity, and construct validity.
Face validityFace validity refers to the subjective
agreement among professionals that a
scale logically reflects the concept
being measured.
Content validityContent validity refers to the degree
that a measure covers the domain of
interest.

Validity : Criterion …Validity : Criterion …
 Criterion validityCriterion validity addresses the question:
“Does my measure correlate with measures“Does my measure correlate with measures
of similar concepts or known quantities?”of similar concepts or known quantities?”
May be classified as either concurrent
validity or predictive validity depending
on the time sequence in which the new
measurement scale and the criterion
measure are correlated.
If measures taken at the same time 􀃎
concurrent validity.
If measures taken at different times 􀃎
predictive validity.

Validity : Construct …Validity : Construct …
 Construct validityConstruct validity exists when a measure
reliably measures and truthfully represents a
unique concept and consists of several
components:
 Face and Content validityFace and Content validity
 Convergent validityConvergent validity – another way of
expressing internal consistency; highly
reliable scales contain convergent validity.
 Criterion validityCriterion validity
 Discriminant validityDiscriminant validity – represents how unique
or distinct is a measure; a scale should not
correlate too highly (i.e., above .75) with a
measure of a different construct.

EXHIBIT 13.EXHIBIT 13.77 Reliability and Validity on TargetReliability and Validity on Target

SensitivitySensitivity
 Sensitivity
 A measurement instrument’s ability to
accurately measure variability in stimuli or
responses.
 Generally increased by adding more
response points or adding scale items.

Selecting a Measurement ScaleSelecting a Measurement Scale
 Some Practical Questions:
 Is a ranking, sorting, rating, or choice technique best?
 Should a monadic or a comparative scale be used?
 What type of category labels, if any, will be used for the
rating scale?
 How many scale categories or response positions are
needed to accurately measure an attitude?
 Should a balanced or unbalanced rating scale be chosen?
 Should a scale that forces a choice among predetermined
options be used?
 Should a single measure or an index measure be used?

 Monadic Rating Scale
 Asks about a single concept in isolation.
 The respondent is not given a specific
frame of reference.
 Example:
Now that you’ve had your automobile for about 1 year, please tell us
how satisfied you are with its engine power and pickup.

Please indicate how the amount of authority in your present position
compares with the amount of authority that would be ideal for this
position.
TOO MUCH  ABOUT RIGHT  TOO LITTLE

 Comparative Rating Scale
 Asks respondents to rate a concept in
comparison with a benchmark explicitly
used as a frame of reference.
 Example:

Selecting a MeasurementSelecting a Measurement
Scale (cont’d)Scale (cont’d)
 What Type of Category Labels, If Any?
 Verbal labels for response categories help
respondents better understand the response
positions.
 The maturity and educational levels of the
respondents will influence the labeling
decision.
 How Many Scale Categories or Response
Positions?
 Five to eight points are optimal for sensitivity.
 The researcher must determine the number of
positions that is best for the specific project.

 Balanced Rating Scale
 A fixed-alternative rating scale with an
equal number of positive and negative
categories; a neutral point or point of
indifference is at the center of the scale.
 Example:

 Unbalanced Rating Scale
 A fixed-alternative rating scale that has
more response categories at one end than
the other resulting in an unequal number
of positive and negative categories.
 Example:

 Forced-choice Rating Scale
 A fixed-alternative rating scale that
requires respondents to choose one of the
fixed alternatives.
 Non-forced Choice Scale
 A fixed-alternative rating scale that
provides a “no opinion” category or that
allows respondents to indicate that they
cannot say which alternative is their
choice.

 Factors affecting the choice of using a
single measure or an index measure:
 The complexity of the issue to be
investigated.
 The number of dimensions the issue
contains.
 Whether individual attributes of the stimulus
are part of a holistic attitude or are seen as
separate items.
 The researcher’s conceptual (problem)
definition will be helpful in making this
choice.

13-115
Overall Flowchart for InstrumentOverall Flowchart for Instrument
DesignDesign

13-116
Flowchart for InstrumentFlowchart for Instrument
Design Phase 1Design Phase 1

Strategic Concerns inStrategic Concerns in
Instrument DesignInstrument Design
What type of scale is needed?
What communication approach will be used?
Should the questions be structured?
Should the questioning be disguised?

Technology AffectsTechnology Affects
Questionnaire DevelopmentQuestionnaire Development
WebSurveyor used to write an instrument.
Write questionnairesWrite questionnaires
more quicklymore quickly
Create visually drivenCreate visually driven
instrumentsinstruments
Eliminate manualEliminate manual
data entrydata entry
Save time in dataSave time in data
analysisanalysis

13-119
Disguising Study ObjectivesDisguising Study Objectives
Situations
where
disguise is
unnecessary
Situations
where
disguise is
unnecessary
Willingly shared,
Conscious-level
information
Reluctantly shared,
Conscious-level
information
Knowable,
Limited-conscious-level
information
Subconscious-level
information

Dummy Table forDummy Table for
American Eating HabitsAmerican Eating Habits
Age
Use of Convenience Foods
Always
Use
Use
Frequently
Use
Sometime
s Rarely Use Never Use
18-24
25-34
35-44
55-64
65+

13-121
Flowchart for Instrument DesignFlowchart for Instrument Design
Phase 2Phase 2

13-122
Question CategoriesQuestion Categories
and Structureand Structure
Administrative
Target
Classification

Engagement = ConvenienceEngagement = Convenience
“Participants are becoming more and
more aware of the value of their time. The
key to maintaining a quality dialogue with
them is to make it really convenient for
them to engage, whenever and wherever
they want.”
Tom Anderson
managing partner
Anderson Analytics

13-124
Question ContentQuestion Content
Should this question be asked?
Is the question of proper scope and coverage?
Can the participant adequately
answer this question as asked?
Will the participant willingly
answer this question as asked?

13-125
Question WordingQuestion Wording
CriteriaCriteria
Shared
vocabulary Single
meaning
Misleading
assumptions
Adequate
alternatives
Personalized
Biased

13-126
Response StrategyResponse Strategy
FactorsFactors
Objectives
of the study
Participant’s
level of
information
Degree to which
participants have
thought through topic
Ease and clarity with
which participant
communicates
Participant’s
motivation to
share

13-127
Free-Response StrategyFree-Response Strategy
What factors influenced your enrollment in Metro U?
____________________________________________
____________________________________________

13-128
Dichotomous ResponseDichotomous Response
StrategyStrategy
Did you attend the “A Day at College” program at Metro U?
Yes
No

13-129
Multiple Choice ResponseMultiple Choice Response
StrategyStrategy
Which one of the following factors was most influential
in your decision to attend Metro U?
Good academic standing
Specific program of study desired
Enjoyable campus life
Many friends from home
High quality of faculty

13-130
Checklist Response StrategyChecklist Response Strategy
Which of the following factors influenced
your decision to enroll in Metro U? (Check all that apply.)
 Tuition cost
 Specific program of study desired
 Parents’ preferences
 Opinion of brother or sister
 Many friends from home attend
 High quality of faculty

13-131
Rating Response StrategyRating Response Strategy
Strongly
influential
Somewhat
influential
Not at all
influential
Good academic reputation   
Enjoyable campus life   
Many friends   
High quality faculty   
Semester calendar   

13-132
RankingRanking
Please rank-order your top three factors from the following list
based on their influence in encouraging you to apply to Metro
U. Use 1 to indicate the most encouraging factor, 2 the next
most encouraging factor, etc.
_____ Opportunity to play collegiate sports
_____ Closeness to home
_____ Enjoyable campus life
_____ Good academic reputation
_____ High quality of faculty

13-133
Summary of Scale TypesSummary of Scale Types
Type Restrictions Scale
Items
Data Type
Rating Scales
Simple Category
Scale
• Needs mutually exclusive choices One or
more
Nominal
Multiple Choice
Single-Response
Scale
• Needs mutually exclusive choices
• May use exhaustive list or ‘other’
Many Nominal
Multiple Choice
Multiple-Response
Scale
(checklist)
• Needs mutually exclusive choices
• Needs exhaustive list or ‘other’
Many Nominal
Likert Scale • Needs definitive positive or
negative statements with which to
agree/disagree
One or
more
Ordinal
Likert-type Scale •Needs definitive positive or
negative statements with which to
agree/disagree
One or
more
Ordinal

13-134
Items
Data Type
Rating Scales
Numerical
Scale
•Needs concepts with standardized meanings;
•Needs number anchors of the scale or end-points
•Score is a measurement of graphical space
One or
many
Ordinal or
Interval
Multiple
Rating List
Scale
•Needs words that are opposites to anchor the
end-points on the verbal scale
Up to
10
Ordinal
Fixed Sum
Scale
•Participant needs ability to calculate total to
some fixed number, often 100.
Two or
more
Interval or Ratio

13-135
Items
Data Type
Rating Scales
Stapel
Scale
•Needs verbal labels that are operationally
defined or standard.
One or
more
Ordinal or
Interval
Graphic
Rating
Scale
•Needs visual images that can be interpreted as
positive or negative anchors
•Score is a measurement of graphical space
from one anchor.
One or
more
Ordinal
(Interval, or
Ratio)

13-136
Items
Data Type
Ranking Scales
Paired
Comparison
Scale
• Number is controlled by participant’s
stamina and interest.
Up to 10 Ordinal
Forced Ranking
Scale
• Needs mutually exclusive choices. Up to 10 Ordinal or
Interval
Comparative
Scale
• Can use verbal or graphical scale. Up to 10 Ordinal

13-137
Internet SurveyInternet Survey
Scale OptionsScale Options

13-138

13-139

13-140
Sources of QuestionsSources of Questions
Handbook ofHandbook of
Marketing ScalesMarketing Scales
The Gallup PollThe Gallup Poll
Cumulative IndexCumulative Index
Measures ofMeasures of
Personality andPersonality and
Social-Social-
PsychologicalPsychological
AttitudesAttitudes
Measures ofMeasures of
Political AttitudesPolitical Attitudes
Index to InternationalIndex to International
Public OpinionPublic Opinion
Sourcebook of HarrisSourcebook of Harris
National SurveysNational Surveys
Marketing ScalesMarketing Scales
HandbookHandbook
American SocialAmerican Social
Attitudes DataAttitudes Data
SourcebookSourcebook

13-141
Flowchart for Instrument Design Phase 3Flowchart for Instrument Design Phase 3

13-142
Guidelines forGuidelines for
Question SequencingQuestion Sequencing
Interesting topics earlyInteresting topics early
Simple topics earlySimple topics early
Sensitive questions laterSensitive questions later
Classification questions laterClassification questions later
Transition between topicsTransition between topics
Reference changes limitedReference changes limited

Illustrating the Funnel ApproachIllustrating the Funnel Approach
How do you think this country is gettingHow do you think this country is getting
along in its relations with otheralong in its relations with other
countries?countries?
How do you think we are doing in ourHow do you think we are doing in our
relations with Iran?relations with Iran?
Do you think we ought to be dealing withDo you think we ought to be dealing with
Iran differently than we are now?Iran differently than we are now?
(If yes) What should we be doing(If yes) What should we be doing
differently?differently?
Some people say we should get tougherSome people say we should get tougher
with Iran and others think we are toowith Iran and others think we are too
tough as it is; how do you feel about it?tough as it is; how do you feel about it?

13-144
Branching QuestionBranching Question

13-145
Components of QuestionnairesComponents of Questionnaires

Example :MindWriter SurveyExample :MindWriter Survey

13-147
Overcoming InstrumentOvercoming Instrument
ProblemsProblems
Build rapportBuild rapport
Redesign question processRedesign question process
Explore alternativesExplore alternatives
Use other methodsUse other methods
PretestPretest

Questionnaire Quality and Design:Questionnaire Quality and Design:
Basic ConsiderationsBasic Considerations
 Questionnaire design is one of the most
critical stages in the survey research
process.
 A questionnaire (survey) is only as good as
the questions it asks—ask a bad question,
get bad results.
 Composing a good questionnaire appears
easy, but it is usually the result of long,
painstaking work.
 The questions must meet the basic criteria of
relevance and accuracy.

Decisions in QuestionnaireDecisions in Questionnaire
DesignDesign
1. What should be asked?
2. How should questions be phrased?
3. In what sequence should the questions
be arranged?
4. What questionnaire layout will best serve
the research objectives?
5. How should the questionnaire be
pretested? Does the questionnaire need
to be revised?

What Should Be Asked?What Should Be Asked?
 Questionnaire Relevancy
 All information collected should address a
research question in helping the decision maker
in solving the current business problem.
 Questionnaire Accuracy
 Increasing the reliability and validity of
respondent information requires that:
 Questionnaires should use simple, understandable,
unbiased, unambiguous, and nonirritating words.
 Questionnaire design should facilitate recall and
motivate respondents to cooperate.
 Proper question wording and sequencing to avoid
confusion and biased answers.

Wording QuestionsWording Questions
 Open-ended Response Questions
 Pose some problem and ask respondents to answer in
their own words.
 Advantages:
 Are most beneficial in exploratory research, especially
when the range of responses is not known.
 May reveal unanticipated reactions toward the product.
 Are good first questions because they allow respondents to
warm up to the questioning process.
 Disadvantages:
 High cost of administering open-ended response questions.
 The possibility that interviewer bias will influence the
answer.
 Bias introduced by articulate individuals’ longer answers.

Wording Questions (cont’d)Wording Questions (cont’d)
 Fixed-alternative Questions
 Questions in which respondents are given
specific, limited-alternative responses and asked
to choose the one closest to their own viewpoint.
 Advantages:
 Require less interviewer skill
 Take less time to answer
 Are easier for the respondent to answer
 Provides comparability of answers
 Disadvantages:
 Lack of range in the response alternatives
 Tendency of respondents to choose convenient
alternative

Types of Fixed-AlternativeTypes of Fixed-Alternative
QuestionsQuestions
 Simple-dichotomy (dichotomous) Question
 Requires the respondent to choose one of two alternatives
(e.g., yes or no).
 Determinant-choice Question
 Requires the respondent to choose one response from
among multiple alternatives (e.g., A, B, or C).
 Frequency-determination Question
 Asks for an answer about general frequency of occurrence
(e.g., often, occasionally, or never).
 Checklist Question
 Allows the respondent to provide multiple answers to a
single question by checking off items.

Phrasing Questions for Self-Phrasing Questions for Self-
Administered,Telephone, andAdministered,Telephone, and
Personal Interview SurveysPersonal Interview Surveys
 Influences on Question Phrasing:
 The means of data collection—telephone
interview, personal interview, self-
administered questionnaire—will influence
the question format and question phrasing.
 Questions for mail, Internet, and telephone
surveys must be less complex than those used in
personal interviews.
 Questionnaires for telephone and personal
interviews should be written in a conversational
style.

EXHIBIT 15.EXHIBIT 15.11 Reducing Question Complexity by Providing Fewer Responses forReducing Question Complexity by Providing Fewer Responses for
Telephone InterviewsTelephone Interviews

Guidelines for ConstructingGuidelines for Constructing
QuestionsQuestions
 Avoid complexity: Simpler language is better.
 Avoid leading and loaded questions.
 Avoid ambiguity: Be as specific as possible.
 Avoid double-barreled items.
 Avoid making assumptions.
 Avoid burdensome questions that may tax the
respondent’s memory.
 Make certain questions generate variance.

What Is the Best QuestionWhat Is the Best Question
Sequence?Sequence?
 Order bias
 Bias caused by the influence of earlier questions
in a questionnaire or by an answer’s position in a
set of answers.
 Funnel technique
 Asking general questions before specific
questions in order to obtain unbiased responses.
 Filter question
 A question that screens out respondents who are
not qualified to answer a second question.
 Pivot question
 A filter question used to determine which version
of a second question will be asked.

EXHIBIT 15.EXHIBIT 15.22
Flow of QuestionsFlow of Questions
to Determine theto Determine the
Level of PromptingLevel of Prompting
Required toRequired to
Stimulate RecallStimulate Recall

What Is the Best Layout?What Is the Best Layout?
 Traditional Questionnaires
 Multiple-grid question
 Several similar questions arranged in a grid
format.
 The title of a questionnaire should be
phrased carefully:
 To capture the respondent’s interest, underline
the importance of the research
 Emphasize the interesting nature of the study
 Appeal to the respondent’s ego
 Emphasize the confidential nature of the study
 To not bias the respondent in the same way
that a leading question might

EXHIBIT 15.EXHIBIT 15.33 Layout of a Page from a Telephone QuestionnaireLayout of a Page from a Telephone Questionnaire

EXHIBIT 15.EXHIBIT 15.44 Telephone Questionnaire with Skip QuestionsTelephone Questionnaire with Skip Questions

EXHIBIT 15.EXHIBIT 15.55 Personal Interview QuestionnairePersonal Interview Questionnaire

EXHIBIT 15.EXHIBIT 15.66 Example of a Skip QuestionExample of a Skip Question

Internet QuestionnairesInternet Questionnaires
 Graphical User Interface (GUI) Software
 The researcher can control the background,
colours, fonts, and other features displayed
on the screen so as to create an attractive and
easy-to-use interface between the user and
the Internet survey.
 Layout Issues
 Paging layout - going from screen to screen.
 Scrolling layout – entire questionnaire
appears on one page and respondent has the
ability to scroll down.

Internet Questionnaire LayoutInternet Questionnaire Layout
 Push Button
 A small outlined area, such as a rectangle or an
arrow, that the respondent clicks on to select an
option or perform a function, such as submit.
 Status Bar
 A visual indicator that tells the respondent what
portion of the survey he or she has completed.
 Radio Button
 A circular icon, resembling a button, that
activates one response choice and deactivates
others when a respondent clicks on it.

 Drop-down Box
 A space saving device that reveals responses when they
are needed but otherwise hides them from view.
 Check Boxes
 Small graphic boxes, next to an answers, that a respondent
clicks on to choose an answer; typically, a check mark or an
X appears in the box when the respondent clicks on it.
 Open-ended Boxes
 Boxes where respondents can type in their own answers to
open-ended questions.
 Pop-up Boxes
 Boxes that appear at selected points and contain
information or instructions for respondents.

EXHIBIT 15.EXHIBIT 15.77 Question in an Online Screening Survey for Joining a ConsumerQuestion in an Online Screening Survey for Joining a Consumer
PanelPanel

15–168
EXHIBIT 15.EXHIBIT 15.88 Alternative Ways of Displaying Internet QuestionsAlternative Ways of Displaying Internet Questions

 Software That Makes Questionnaires
Interactive
 Variable piping software
 Allows variables to be inserted into an Internet
questionnaire as a respondent is completing it.
 Error trapping software
 Controls the flow of an Internet questionnaire.
 Forced answering software
 Prevents respondents from continuing with an
Internet questionnaire if they fail to answer a
question.
 Interactive help desk
 A live, real-time support feature that solves
problems or answers questions respondents may
encounter in completing the questionnaire.

Pretesting and RevisingPretesting and Revising
QuestionnairesQuestionnaires
 Pretesting Process
 Seeks to determine whether respondents
have any difficulty understanding the
questionnaire and whether there are any
ambiguous or biased questions.
 Preliminary Tabulation
 A tabulation of the results of a pretest to
help determine whether the questionnaire
will meet the objectives of the research.

Designing Questionnaires forDesigning Questionnaires for
Global MarketsGlobal Markets
 Back Translation
 Taking a questionnaire that has previously
been translated into another language and
having a second, independent translator
translate it back to the original language.
 A questionnaire developed in one country
may be difficult to translate because
equivalent language concepts do not exist or
because of differences in idiom and
vernacular.

Further ReadingFurther Reading
 COOPER, D.R. AND SCHINDLER, P.S. (2011)
BUSINESS RESEARCH METHODS, 11TH
EDN,
MCGRAW HILL
 ZIKMUND, W.G., BABIN, B.J., CARR, J.C. AND
GRIFFIN, M. (2010) BUSINESS RESEARCH
METHODS, 8TH
EDN, SOUTH-WESTERN
 SAUNDERS, M., LEWIS, P. AND THORNHILL, A.
(2012) RESEARCH METHODS FOR BUSINESS
STUDENTS, 6TH
EDN, PRENTICE HALL.
 SAUNDERS, M. AND LEWIS, P. (2012) DOING
RESEARCH IN BUSINESS & MANAGEMENT, FT
PRENTICE HALL.

Mba2216 week 07 08 measurement and data collection forms

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (6)

Similar to Mba2216 week 07 08 measurement and data collection forms

Similar to Mba2216 week 07 08 measurement and data collection forms (20)

More from Stephen Ong

More from Stephen Ong (20)

Recently uploaded

Recently uploaded (20)

Mba2216 week 07 08 measurement and data collection forms

Editor's Notes