To generate new, patient-centered insights into diagnostic error, we convened diverse groups in public deliberation to recommend and evaluate actions that patients and/or their advocates would be willing and able to perform to improve diagnostic quality.

1. CLEARING THE ERROR
Using Public Deliberation to Define
Patient Roles as Partners in the
Diagnostic Process
The Jefferson Center, Dr. Tina Nabatchi at
Syracuse University’s Maxwell School of
Citizenship and Public Affairs, and the Society
to Improve Diagnosis in Medicine are leading
a 2 year research initiative to assess the value
of public deliberation in developing healthcare
policy and to identify roles patients are willing
and able to perform to improve diagnostic
quality. This effort is funded by the Agency for
Healthcare Research and Quality.

2. ACTIVATING PATIENTS IN POLICY
In an effort to improve health outcomes,
providers, insurers, and others are
working to more fully engage patients
in their own healthcare. These initial
efforts suggest educating patients about
their condition(s) and involving patients
in determining the nature and course of
their treatment can improve health and
reduce spending.
Building from the insight that, with a
little assistance and education, patients
can make effective decisions about their
own care, we’re exploring the value of
incorporating patient knowledge and
diversity into the development of health
policy, regulations, and institutional
practices. We chose first to focus on the
issue of diagnostic error.
We know the relationship between the
patient and their provider is a critical
determinant in effective diagnosis.
Unfortunately, many patients feel
unable to fully assert themselves as
mutual partners in the patient-provider
relationship.
To generate new, patient-centered
insight into this problem, we convened
diverse groups in public deliberation
to recommend and evaluate actions
that patients and/or their advocates
would be willing and able to perform to
improve diagnostic quality. Participants
also identified obstacles to action that
healthcare systems and providers should
address to improve diagnosis.
This report highlights those
recommendations and assesses the value
of public deliberation in developing more
effective health policy.
IMPROVINGDIAGNOSIS
What’s diagnostic error?
A diagnostic error is a missed, delayed,
or wrong diagnosis. An error is made
in about 1 of every 10 diagnoses.
Diagnostic error is a leading cause
of malpractice claims and likely a
significant factor in adverse health
events. Diagnostic error, however, is
less responsive to system-level change
than other types of medical error, which
has held up the search for solutions.

3. PATIENT-DRIVEN POLICY
Our engagement efforts consisted of
Citizens’ Jury-style public deliberation.
20 individuals were selected to
participate for six days of deliberation
over two weekends in late 2015.
Applicants were recruited from Onondaga
County, New York. The demographic
composition of the selected panel is
included in Appendix A.
Participants met for appoximately 50
hours to hear expert testimony and
feedback (expert list is in Appendix
B), share their personal stories and
expertise, and deliberate together to
recommend patient action steps. They
identified five main strategies for
addressing diagnostic error through
patient action.
1. Present symptoms clearly and
completely
2. Assert yourself in the relationship
3. Coordinate your care
4. Ensure accurate records and tests
5. Manage your care
The group also recommended sixteen
concrete action steps to support
implementation of the main strategies.
All sixteen action steps are listed in
Appendix C.
To maximize the opportunity for patients
and their advocates to succeed using
these recommendations, participants
were asked to think about potential
obstacles to pursuing the recommended
actions and identify strategies for health
systems to begin addressing those
obstacles. The identified obstacles and
top remedies are outlined in Appendix D.
Participants recommended concrete,
sound action to address the challenge
of diagnostic error. Their input
complements the recent work of
the Institute of Medicine and other
organizations by presenting a clear
patient perspective that responds to the
diversity of patient experience.
What’s a Citizens’ Jury?
A Citizens’ Jury provides everyday
citizens the opportunity to study an
issue deeply, deliberate together with
a diverse group of their peers, and
develop solutions to challenging public
issues.
A Citizens’ Jury includes:
• A random stratified sample of a
given community
• Unbiased information provided by a
diverse array of experts
• Time to study and discuss an issue
in depth
• Recommendations produced
through deliberation and voting

4. THE RESEARCH PROCESS
To assess the value of patient deliberation in developing patient and
system-focused healthcare policy and to evaluate the efficacy of the
recommended action steps, we conducted controlled research. The
research process is described in the graphic below.
Two matched panels of 20, randomly
selected and stratified to reflect the
demographics of Onondaga County, were
convened to recommed action steps for
patients to reduce errors in diagnosis.
Each group heard educational presentations from
experts on diagnosis and diagnostic error.
GROUPA GROUPB
Group B completed
questionnaires with
their individual
recommendations
for patient action
immediately after
the presentations.
Group A
questioned experts
and deliberated
for 6 days to
develop group
recommendations
for patient action.
GROUPC
A third group of 93 diverse
participants, Group C, was
convened to assess the
feasibility and impact of Group
A’s recommendations from
the perspective(s) of patients
and healthcare consumers
who didn’t hear extensive
educational presentations.
All 3 groups were surveyed before and after their participation to
assess changes in patient activation, trust in doctors, perceptions
about diagnostic error, knowledge of diagnostic error, perceptions
of patient efficacy and patient responsibilities, and overall health
literacy. A control group was also surveyed to provide baseline data
relative to the other groups.

5. RESEARCH RESULTS
Groups A, B, and C experienced statistically significant increases in patient activation, trust in
doctors, and perceptions about the seriousness of diagnostic error.
The percentage of participants who correctly answered a question measuring their knowledge
about diagnostic error increased in all three groups.
All three groups saw changes in their perceptions about patient engagement. After participation,
each group had a statistically significant change in their perceptions about responsibility for
diagnostic error and the role of patients in the diagnostic process.
Group A, the 6-day deliberation group, also experienced statistically significant increases in
correctly answering questions about health literacy after deliberation.
For all questions, effect sizes were largest among Group A, suggesting that longer-term
engagement, coupled with deliberation, promotes greater patient knowledge, activation, and
perceptions of self-efficacy.
Moderate or Major
Improvements in
Diagnostic Quality
Easy or Very Easy
to Use in Own
Healthcare
Likely or Very
Likely to Use in
Own Healthcare
Coordinate your care
Ensure accurate records and tests
Assert yourself in the relationship
Manage your care
Present symptoms clearly and completely
0% 10% 40% 50% 60% 70% 80% 90%20% 30% 100%
Groups C also evaluated Group A’s recommendations for impact on diagnostic error, ease of
use, and likelihood of use. Strong majorities thought all recommendations would likely lead to
moderate or major improvements in diagnostic quality. Participants felt recommendations 1, 2,
and 5 would be the easiest to implement in the course of their own care. Similarly, participants
said they were most likely to use recommendations 1, 2, and 5 in the course of their own care.
GROUP C’S ASSESSMENT OF PATIENT RECOMMENDATIONS

6. CONCLUSIONS AND NEXT STEPS
The work accomplished by patients and healthcare consumers during Clearing the Error
highlights the capacity of non-experts to participate meaningfully and productively
in assessing and improving healthcare policy. The recommendations to improve
diagnostic quality are on par, and in many ways more detailed, than available
recommendations from the National Academy of Medicine and others.
Clearing the Error also provides a foundation for further research to explore the role
of patients in improving diagnostic quality and in developing health policy.
Over the next 18 months, we will explore the efficacy of patient recommendations in
clinical settings, focusing first on the top 3 recommendations as assessed by Group C.
We also hope to test the relative efficacy of recommendations generated by
patients and doctors. For example, are patients more or less likely to follow similar
recommendations if they come from other patients?
If you are interested in learning more about research to implement and evaluate
patient recommendations, or if you’d like to use these recommendations in your own
research, let us know! You can reach us at arockway@jefferson-center.org
or 651-209-7672.
If you’d like additional materials from this research not included in this report,
contact Dr. Tina Nabatchi at tnabatch@maxwell.syr.edu.

7. CLEARING THE ERROR
Clearing the Error is a collaborative effort between the Jefferson Center, the Maxwell
School of Citizenship and Public Affairs at Syracuse University, and the Society to
Improve Diagnosis in Medicine, with funding from the Agency for Healthcare Research
and Quality.
The Jefferson Center is a nonpartisan nonprofit that
engages Americans directly to solve shared challenges and
craft better policy. Our mission is to strengthen democracy
by advancing informed, citizen-developed solutions to
challenging public issues.
Contact: Andrew Rockway
arockway@jefferson-center.org
jefferson-center.org
The Maxwell School of Citizenship and Public
Affairs is Syracuse University’s home for innovative,
interdisciplinary teaching and research in the social
sciences, public policy, public administration, and
international relations.
Contact: Dr. Tina Nabatchi
tnabatch@maxwell.syr.edu
maxwell.syr.edu
Contact: Paul Epner
Paul.Epner@improvediagnosis.org
improvediagnosis.org
The Society to Improve Diagnosis in Medicine works to
improve diagnostic quality through research, education,
and advocacy. SIDM is committed to a world where
diagnosis is accurate, timely, and efficient.

8. Demographic Onondaga County % Actual Number
Gender
Female 51.9% 9
Male 48.1% 11
Ethnicity
Asian 3.1% 0
Black 11.0% 5
Hispanic or Latino 4.0% 1
Native American or Alaska Native 0.8% 0
Native Hawaiian or Pacific Islander 0.0% 0
White 81.1% 14
Household Location
Urban 87.41% 18
Rural 12.59% 2
Party Affiliation
Democrat 36.96% 7
Independent, Unaffiliated, Other 32.77% 6
Republican 30.27% 7
Annual Household Income
Less than $15,000 13.3% 3
$15,000-$24,999 10.5% 3
$25,000-$34,999 10.0% 6
$35,000-$49,999 12.7% 2
$50,000-$74,999 18.7% 4
$75,000-$99,999 12.4% 2
$100,000+ 22.5% 0
Age
18-34 31% 7
35-64 50% 9
65 & over 19% 4
Education
Did not complete High School 10.1% 2
High School or Equivalent 45% 8
Associate’s/Bachelor’s degree 30.4% 8
Graduate or professional degree 14.5% 2
Total Number of Participants 20
APPENDIX A - DELIBERATIVE PANEL DEMOGRAPHICS

9. APPENDIX B - EXPERT PRESENTERS
Viraj Bhise
Graduate Research Assistant, University of Texas Health Science Center at Houston
Chief Operating Officer, Health Connect Compusoft
Dr. Janice Colestro
Associate Director, Veterans Health Administration Office of Medical-Legal Affairs
Dr. Laurie C. Drill-Mellum
Chief Medical Officer, MMIC
Dr. John Ely
Physician and Professor of Family Medicine at University of Iowa
Paul Epner
Executive Vice President, Society to Improve Diagnosis in Medicine
Helen Haskell
President, Mothers Against Medical Error
Dr. Stephen Martin
Faculty, Harvard Medical School
Assistant Professor, University of Massachusetts Medical School
Kathryn McDonald
Executive Director, Center for Health Policy and Center for Primary Care and Outcomes
Research at Stanford University
Patricia McGaffigan
Chief Operating Officer and Senior VP of Programs, National Patient Safety Foundation
Dr. David Meyers
Emergency Physician, Sinai Hospital
Sue Sheridan
Director of Patient Engagement, Patient-Centered Outcomes Research Institute
Dr. Joan Von Feldt
Professor of Medicine, University of Pennsylvania Hospital
Peggy Zuckerman
Board Member, Society for Participatory Medicine

10. APPENDIX C - PATIENT RECOMMENDATIONS FOR ACTION
1) Present symptoms clearly and completely
• Be truthful about your symptoms and other behaviors when telling your doctor about your
history to ensure information is accurate.
• Be prepared to discuss your symptoms. For example, 8 characteristics of symptoms are
quantity, quality, aggravating factors, alleviating factors, setting, associated symptoms,
location, timing.
2) Assert yourself in the relationship
• Be clear, concise, and persistent in communicating your symptoms and concerns.
• Ask detailed questions of your doctor, including a plan to arrive at a diagnosis so the doctor
remains engaged and focused on your concerns. For example, “could these symptoms
indicate something else or an additional issue?”
• Notify your healthcare provider if your condition worsens, does or doesn’t improve, or if new
symptoms develop.
• If you’re concerned about the accuracy of the diagnosis, seek a second opinion.
3) Coordinate your care
• Find a primary care provider/family doctor so that they can better coordinate and manage
your healthcare.
• Enlist a patient advocate, as needed, to assist you in coordinating care.
• Have your primary care provider manage all your records to ensure they are accessible to
other providers.
• Seek out a health system where different doctors work together frequently, share consistent
information, and coordinate services effectively.
4) Ensure accurate records and tests
• Maintain and update your own medical record, which includes test results, doctor notes,
images, communication with providers, and other information pertinent to your medical
history.
• If you have access to your electronic medical records or a patient portal, use that. If you
don’t have access, ask for a physical copy of your records and/or any recent updates.
• If you notice a factual inaccuracy with your medical record, advocate and insist to have the
error corrected.
5) Manage your care
• Ensure communications and expectations are clear between you and your healthcare
provider.
• Throughout the relationship, follow through on your health care provider’s recommendations
regarding the course of action to reach an accurate diagnosis. For example, completing lab
tests, going to appointments with specialists, and taking medications as prescribed.
• Follow up with your healthcare provider after appointments to obtain test results to ensure
proper testing was conducted. Thus, both patient and healthcare provider are accountable.

11. APPENDIX D - SYSTEM OBSTACLES AND SOLUTIONS
Different payment systems, insurance, and/or ability to pay can limit choice
and access or influence the course of diagnosis and treatment.
• Provide patients with clear, practical information about signing up for
insurance coverage
• Develop standardized pricing for medical services
• Ombudsman/liaison/advocate to help patient navigate choice when
managing cost vs. care
Fragmented, decentralized healthcare systems inhibit effective
communication and information-sharing across systems and between
providers. This fragmentation can also reduce provider collegiality/
collaborative problem solving.
• Transition from many proprietary electronic health record systems to one
system so there is greater interoperability across providers
• Primary care provider should be responsible for coordinating
communication between all different providers
• Patients go to portal to review or verify records; if they don’t have a
computer, review notes before they leave the office
• Develop protocols for sharing information between providers
Patients don’t know how to communicate symptoms effectively.
• Develop a video, poster, or handout for appointment settings, to include
potential questions for your doctor, topics to cover, and other information
so patients can “help doctors help you”
• Have a Kiosk or iPad at clinic/office that presents interactive questions to
help patients clarify 8 characteristics of their symptoms
The complexity of the healthcare system makes it difficult for patients to help
coordinate the diagnostic process (e.g. following up with multiple doctors,
tracking personal medical record and history, making sure test results are
analyzed).
• Access to patient portal in office, with opportunity to print records, notes,
treatment plan, test results, and other information
• Create a Patient Review Board for missed diagnosis to review
implementation of practices aimed at improving diagnostic quality
• A flag/notification on electronic health record when new information is
added and when primary care provider has read it

12. APPENDIX D - SYSTEM OBSTACLES AND SOLUTIONS
Culture encourages doctors to be efficient and productive; patients have
limited time to interact with doctors face-to-face.
• Doctor has another staff person write electronic medical record and
perform other tasks to allow more time for interaction with patient
• Shift culture of 15 minute appointments to 20 minutes; 5 minutes makes a
difference
• Doctor has patient information (8 characteristics of symptoms or etc.)
before appointment begins
• Change payment structure to encourage longer visits or at least focus on
better health outcomes; payment structure reflects "success rate" - such as
better health outcomes, fewer follow-up visits, etc.
• Have provider address most important issue first
Patients don't necessarily know what their options might be for asserting
themselves.
System Focus
• Mandatory debriefing between provider and patient to ensure clarity of next
steps during visit
• Change provider reimbursement system to incentivize/allow more time
with patients
Patient Focus
• Provide a handout or verbal instructions to patients that describe patient
“rights” - to ask questions; to ask for a 2nd opinion; to receive satisfactory
answers to questions; to receive a summary of what's going on with your
care
• Be persistent, be a pain in the ass
Patients don't necessarily trust providers. Provider attitudes or behavior can
inhibit patient openness, receptivity, confidence, and/or satisfaction.
• Doctor communicates uncertainty in the diagnosis appropriately. For
example, "I don't know what's wrong with you" vs. "I'm not sure what's
wrong with you, I have a few ideas, but Tests X and Y will help me make a
more accurate diagnosis."
• Have an assistant type into electronic health record during encounter so
that doctor can focus on patient conversation
• Ongoing education for providers about interacting and communicating with
patients