This document provides 8 tips for what not to do when selecting a patient-reported experience measure (PREM). It advises against using a PREM just because a colleague used it, or because it was developed by a colleague, without evidence of its reliability and validity. It also warns against using PREMs developed without patient input, for different patient groups, or without understanding how the information will be used. The document stresses the importance of properly developing and validating PREMs to ensure the right information is collected.

1. ByKeith Meadows
DHP Research & Consultancy Ltd

2. Use a PREM simply DESIGNED BY
A
because a colleague has COLLEGUE
used it before Assume it’s
Not sure reliable and
what to do Developedinput
patient
without
valid without
with the evidence
information
Developed for other patient groups
ASSUME IS RELEVANT TO YOUR
PATIENT GROUP
Copyright DHP Research 2012

3. 8 Things you should NOT do when selecting a PREM www.dhpresearch.com
INTRODUCTION Who are these tips for?
Measurement of the patient’s experience is a now a For the healthcare professional, clinician and researcher who relies
on information on patient satisfaction, health and or experience,
key factor in providing improved health care.
the choice and use of a well designed and validated questionnaire
The Patient Experience Network has provided a lot of information is essential.
around the measurement of the patient’s experience including the
7-step measurement process. However, what is not discussed is
How these tips will help you?
some of the do’s and don’ts when choosing or using a PREM. By applying the 8 tips described here when selecting the
appropriate PREM together with proper planning, you will have
Regretably with the increasing availability of on-line questionnaire
greater confidence in the quality of information collected.
design packages such as SurveyMonkey, this can lead to a plethora
of questionnaires that would fail the most basic questionnaire
design criteria.
Questionnaire design is a complex process to ensure you collect
the right kind of information to enable good decision-making. Also
choosing the right PREM is also something that requires some
thought.
Copyright DHP Research 2012

4. 1 8 Things you SHOULD NOT do when selecting a PREM www.dhpresearch.com
DON’T USE A WHY?
This is a common error and it should not be assumed
because a questionnaire has been used before it will be
appropriate for YOUR particular study.
PREM JUST BECAUSE The questionnaire might for example, not have been
properly developed or it might be looking for experience
A COLLEAGUE HAS that do not fit with you study objectives.
USED IT BEFORE
Copyright DHP Research. all rights reserved
4
2012

5. 2 8 Things you SHOULD NOT do when selecting a PREM www.dhpresearch.com
DON’TUSE A PREM WHY?
Developing a PREM requires considerable experience
and knowledge. If you want to collect reliable and
JUST BECAUSE IT’S
valid information the PREM needs to have been
developed according to clearly defined criteria,
patient input and using established procedures to
BEEN DEVELOPED BY ensure satisfactory reliability and validity.
A COLLEGUE
As tempting as it maybe to go for the ‘quick and
dirty’ this will not give you the information you are
looking for.
Copyright DHP Research. all rights reserved
5
2012

6. 3 8 Things you SHOULD NOT do when selecting a PREM www.dhpresearch.com
DON’T USE A WHY?
PREMS developed without patient input must be viewed
with suspicion. If we want to find out about the patient’s
PREM THAT HAS BEEN experiences then we need to ask them.
At the development stage of a PREM it’s essential that
DEVELOPED WITHOUT through such techniques as focus groups and in-depth
interviews we really do find out what these experiences
are which can then be incorporated into the
PATIENT INPUT questionnaire.
PREMs developed on the reliance of HCP input alone, are
based on a different perspective from that of the patient
and undoubtedly will lack significant content.
Copyright DHP Research. all rights reserved
6
2012

7. 4 8 Things you SHOULD NOT do when selecting a PREM www.dhpresearch.com
DON’T USE A WHY?
PPREMs developed for a specific patient population will
unlikely be appropriate if your patient group differs in
PREM THAT HAS BEEN any significant way from its intended use.
As experiences and their relevance can vary across
DEVELOPED FOR
patient groups, PREMs can be either generic or specific.
If the PREM is generic then it should have been
developed to be applicable across different patient
DIFFERENT groups etc. If on the other hand if the PREM is specific
then it should not be used if the objectives of your
POPULATION TO
survey or patient demographics etc., differ in anyway.
Examination of the content (face validity) will give an
indication, but again documented evidence of the PREMs
YOURS development should be reviewed prior to its use.
Copyright DHP Research. all rights reserved
7
2012

8. 5 8 Things you SHOULD NOT do when selecting a PREM www.dhpresearch.com
DON’T ASSUME WHY?
The temptation to use an existing PREM is of course
understandable. It saves time and costs and means you
THAT THE PREM IS can just go ahead and use it. Well no, not really. With so
many on-line questionnaire design tools available the
risk is that the questionnaire has just simply been drawn
VALID AND RELIABLE up.
So what’s the evidence you need to convince you you’ve
WITHOUT ANY got a reliable and valid PREM to use?
Here are some basic questions you must ask:
Has the development of the PREM been documented?
EVIDENCE
•
• What was the reason for developing the PREM?
• Was there patient input and if so who were they?
• How were questions developed and were they pre-
tested?
Copyright DHP Research. all rights reserved
8
2012

9. 6 8 Things you SHOULD NOT do when selecting a PREM www.dhpresearch.com
DON’T ASSUME WHY?
PPREMs can be developed to measure a range of patient
experiences from the quality of emotional support
THE PREM WILL GIVE provided through to the quality of the food.
To ensure that the selected PREM will provide the
YOU THE
relevant information you will need evidence of the
rationale behind it’s development and whether this fits
with the objectives of your own survey. A properly
INFORMATION YOU developed PREM should have its development fully
documented.
WANT Of course looking at the content will provide you with
some idea (face validity) however, this is not a
guarantee the PREM has content validity that is:
produced with patient input and is a reliable and valid
instrument.
Copyright DHP Research. all rights reserved
9
2012

10. 7 8 Things you SHOULD NOT do when selecting a PREM www.dhpresearch.com
DON’T USE THE WHY?
Because there is no “magic question” that can provide
you with an answer to a complex set of circumstances,
NET PROMOTER perceptions, intentions, and behaviours as one single
score.
SCORE (NPS)
BECAUSE YOU’VE
BEEN TOLD IT’S THE
BEST THING AVAIALBE
Copyright DHP Research. all rights reserved
10
2012

11. 8 8 Things you SHOULD NOT do when selecting a PREM www.dhpresearch.com
DON’T USE A WHY?
You must have a clearly defined objective as to what and
how you are going to use the information collected prior
PREM IF YOU ARE NOT
to the survey. You will not achieve anything if your
approach is to think that you will decide once you’ve got
the data.
SURE WHAT TO DO • You must have a clear analysis plan
• Know in advance what questions you need answers to
WITH THE • Be sure you are asking the questions you need
answering
INFORMATION
COLLECTED
Copyright DHP Research. all rights reserved
11
2012

12. About DHP Research FIND OUT
MORE
We provide a range of consultancy and support to clinical
research teams, the NHS and academia in making key
decisions when using patient reported outcome (PRO)
and experience (PRE) measures. This includes: DHP Research & Consultancy Ltd
Bloxham Mill Business Centre
 Literature reviews and evaluation of existing patient Barford Road, Bloxham
reported outcome (PRO) measures
 Qualitative interviews to derive appropriate content for PROs
Banbury
 Write academic scripts Oxfordshire
 Training in PROs and research methodologies OX15 4FF
 Development of patient reported outcomes (PRO)measures UK: +44 (0)1295 724233
Our expertise in the measurement of patient reported
www.dhpresearch.com
experience includes:
info@dhpresearch.com
 Help in choosing the right measures
 Development of PREMs using state-of-the-art methodologies
 Interpreting PREM data.
Copyright DHP Research 2012 12

13. DHP Research & Consultancy Ltd
Bloxham Mill Business Centre, Barford Road, Bloxham,
Banbury, Oxfordshire OX15 4FF
Tel: +44 (0)1272 4233
Tel: kmeadows@dhpresearch.com
www.dhpresearch.com
Copyright DHP Research 2012