This presentation discusses the question of whether it is possible to compare scores from two dermatology-specific outcome measures, the PsoriQoL and the QoLIAD.
Co-calibrating scores from two dermatology-specific patient reported outcome measures.
1. Co-calibrating scores from two dermatology-specific
patient reported outcome measures
James Twiss,
Rebecca Crawford, Stephen McKenna
Galen Research Ltd, Manchester, UK
Email: jtwiss@galen-research.com
2. Objectives
• To co-calibrate scores from two different
dermatology-specific outcome measures:
- Psoriasis Quality of Life Scale (PSORIQoL)
- Quality of Life in Atopic Dermatitis (QoLIAD)
3. Introduction
• Some interventions may be used for more than
one disease
• Dermatology studies frequently combine
together patients with different kinds of skin
conditions
• Generic outcome measures traditionally used in
these circumstances (e.g. SF-36/NHP)
4. Generic outcome measures
Pros Cons
Thought to be able to compare across
diseases
Less sensitive than disease-specific
measures
No need to develop separate scales
for each disease
May miss important issues
Older scales often have poor
measurement properties
Evidence of DIF by disease
5. Dermatology Life Quality Index (DLQI)
• Generic dermatology measure used in a
large number of skin conditions
• Evidence of poor fit to Rasch model
- Overall misfit
- Poor measurement range
- Disordered response thresholds
- Several items showed DIF by disease (psoriasis
vs. atopic dermatitis)
6. Aims
• To assess whether it is possible to compare
scores from two dermatology-specific
outcome measures
- Psoriasis Quality of Life Scale (PSORIQoL)
- Quality of Life in Atopic Dermatitis (QoLIAD)
7. Methods
• Analyses conducted using RUMM2030
• The two scales were analysed separately for fit
to the Rasch model
• Problems were resolved
• The two scales were co-calibrated using a
common item design
• Item pool analysed for fit to the Rasch model
• Common items analysed for DIF by disease
8. Questionnaire comparison
• PsoriQoL – psoriasis-specific QoL measure
• QoLIAD – atopic dermatitis QoL measure
• Both based on the Needs-based model of QoL
• Content for both from patient interviews
• Both developed using Rasch analysis
• 25 items in each
• Yes/no response format in each
• 5 common items
9. Common Items
• I have no self confidence
• I just want to shut myself away
• I worry that other people will not accept
me
• I am embarrassed about my appearance
• I can’t bear anyone touching me
10. Sample
Psoriasis
(n = 146)
Atopic Dermatitis
(n = 146)
Gender (%)
Male 73 (50) 73 (50)
Female 73 (50) 73 (50)
Age (Years)
Mean (SD) 44.4 (14.7) 45.5 (16.6)
Range 66 (17-83) 62 (20-82)
Duration (years)
Mean (SD) 20.9 (13.5) 28.2 (17.5)
Range 67 (2-69) 76 (0-76)
11. Fit to the Rasch model
Item-Trait
Interaction
Chi2
PSI Item-Person interaction Unidimensionality
(CI)
Items Persons
Mean SD Mean SD
PSORIQoL 0.25 0.94 -0.22 1.12 -0.36 0.82 0.045 (0.008 – 0.082)
QoLIAD 0.28 0.91 -0.25 1.12 -0.162 0.54 0.095 (0.057 - 0.133)
12. Residual correlations
• Evidence of local dependency in QoLIAD between:
Item 4: ‘I get embarrassed when I am with people I don’t know
very well’
&
Item 6: ‘I feel tense all the time’
Item 4: ‘I get embarrassed when I am with people I don’t know
very well’
&
Item 17: ‘I can’t concentrate on anything else’
• Item 4 was removed improving model fit statistics
13. Additional fit statistics
• One item in each scale removed due to a high fit
residual
- PsoriQoL item 17 (fit residual = 2.84)
‘It interferes with my close relationships’
- QoLIAD item 16 (fit residual = 3.15)
‘I find it hard to relax’
• Minimal evidence of DIF by age or gender
• Items had good logit coverage in each scale
14. Fit to the Rasch model – combined items
Item-Trait
Interaction
Chi2
PSI Item-Person interaction
Items Persons
Mean SD Mean SD
Combined 0.056 0.88 -0.28 0.98 -0.27 0.67
15. DIF analysis of common items
• One of the 5 common items showed DIF by disease (p = 0.00098)
‘I worry that people will not accept me’
16. Logit position of common items
Item Logit position
I just want to shut myself away 2.51
I can’t bear anyone touching me 2.03
I have no self confidence 1.23
I worry that other people will not accept me 0.42
I am embarrassed about my appearance -0.85
17. Final fit to the Rasch model – combined items
Item-Trait
Interaction
Chi2
PSI Item-Person interaction
Items Persons
Mean SD Mean SD
Final items 0.123 0.89 -0.271 1.01 -0.27 0.66
19. Conclusions
• Items were successfully co-calibrated
• This study moves the debate about disease-specific
vs. generic outcome measures
forward
• Over 20 disease-specific measures available
based on the needs-based model of QoL
20. References
McKenna SP, Cook SA, Whalley D, Doward LC, Richards HL, Griffiths CEM,
Van Assache D (2003). Development of the PSORIQoL, a psoriasis-specific
measure of quality of life designed for use in clinical practice and trials. Br J
Dermatol; 149: 323-331.
Whalley D, McKenna SP, Dewar AL, Erdman RA, Kohlman T, Niero M, Cook
SA, Crickx B, Herdman MJ, Frech F, van Assche D, A New Instrument for
Assessing Quality of Life in Atopic Dermatitis (QoLIAD). British Journal of
Dermatology 2004; 150: 274-83
Twiss J, Meads DM, Preston EP, Crawford SR, McKenna SP. Can we rely on
the Dermatology Life Quality Index (DLQI) as a measure of the impact of
psoriasis or atopic dermatitis? Journal of Investigative Dermatology. (2012);
132(1):76-84.
Hinweis der Redaktion
Need to add information relating to the development of each questionnaire
PSORIQoL – developed with Rasch using a clear theoretical model
DLQI – not developed with Rasch and no clear theoretical model
Measure different things
Galen Research – we develop disease specific patient reported outcome measures to measure the impact of disease from the patients perspective
These are used widely in clinical practice and in clinical trials
We have been using Rasch analysis for a number of years as a company to improve the measurement properties of our scales
This has provided us the opportunity to co-calibrate scores from the measures to allow direct comparison of scores
The question may be asked – why bother doing this at all?
We conducted a study recently to evaluate the measurement properties of the DLQI
Rasch analysis can provide new perspective on this debate
Rasch analysis can be used to co-calibrate scores from two different dermatology-specific outcome measures
This study has the potential to move forward the debate on this issue.
This study has the potential to move forward the debate on this issue.
Life gains its quality by the ability of each person to meet their needs
Each illness impacts on patients ability to meet their needs in different ways
It isn’t necessarily the symptoms or functional limitations that are important but how these impact on patients ability to meet their needs
Psoriasis sample from the Manchester Psoriasis Service at Hope Hospital
Atopic Dermatitis sample from the National Eczema society
Atopic dermatitis sample from a trial
When we applied
Which items?
4. ‘I get embarrassed when I am with people I don’t know very well’
6. ‘I feel tense all the time’
17. ‘I can’t concentrate on anything else’
As the two items that misfit only showed misfit by one of the fit statistics we decided to keep these
Items cost several thousand pounds each to develop
In addition, we wanted to keep the theoretical structure of the measure as much as possible
Item 24 showed DIF by gender in the QoLIAD
Overall fit to the Rasch model
We decided to remove this item as it was fundamental to the calibration of the items
The scale showed overall fit to the Rasch model
Two items continued to show misfit
4 items removed if we remove these items
There are still some persons who are not covered at the extremities but overall a good coverage
It is possible to be able to take the best of both
We can get specific information about each disease
We can compare across diseases using Rasch analysis