New clinical evaluation requirements

1. Understanding the Increasing
Impact of Clinical Research
on Obtaining Product CE Marks
Gert Bos and Suzanne Halliday
BSI Healthcare
Medical Device Clinical Research Conference
Q1 productions, London
3 November 2009

2. 2
This presentation …
Ten changes in relation to clinical evaluation
Ten “mistakes” that will delay time to complete
Notified Body review.

3. 3
This presentation …
Sources of information:
1. 93/42/EEC – amended by 2007/47/EC September/2007
2. Harmonised European Standards
• EN ISO 14155 -1 -2 :2003
3. Guidance Documents
• MedDev 2.7.1
• MedDev 2.12 /2
• GHTF Study Group 5 N2R8
• NB-MED 2.7.1

4. 4
10 Changes in clinical evaluation
1. Article 1, 14a and15 - definitions, databank, transparency trials
2. Annex I – ER 14  6a - clinical evaluation mandatory
3. Annex X – 1.1 - Risk/Benefit based on clinical data
4. Annex X – 1.1 - Defined & Methodologically Sound Procedure
5. Annex X – 1.1a - Clinical Investigations on Implantable Devices
6. Annex X – 1.1b - Evaluation Shall be Documented
7. Annex X – 1.1c - Evaluation Must be Actively Updated
8. Annex X – 1.1d - No Clinical Data justified with Risk Management
9. Annex X – 2.3.5 - Adverse Events to All Competent Authorities
10. Annex VIII - Devices for Clinical Investigation

5. 1. “The clinical evaluation is based on a
comprehensive analysis of available pre- and post
market clinical data …”
Annex X 1.1 – “The ‘clinical evaluation’ is based on:
1.1.1. … critical evaluation of the relevant scientific literature …
1.1.2. … critical evaluation of the results of all clinical investigations …
1.1.3. … critical evaluation of the combined clinical data …”

6. 6
Search strategy
• Write search protocol
• Document search terms and databases
• Systematic reviews & peer reviewed research work
• Use people qualified in data researching

7. 2. “The clinical evaluation is based on a
comprehensive analysis of available pre- and post
market clinical data …
…relevant to the intended use of the device in
question, including clinical performance data and safety
data.”

8. 8
Relevance of data
• Look at
 Nr of patients, diagnosis, follow up
 Clinical safety and clinical performance
 Primary and secondary endpoint
 …..
 Equivalence  non-equivalence

9. 3. “Document the selection criteria to be applied to
published literature and justification for their choice.”
Annex X 1.1 b – “The clinical evaluation and its outcome shall be documented.
This documentation shall be included and/or fully referenced in the technical
documentation of the device.”

10. AHRQ Systematic Review
10
Methods Literature Review and Meta-Analysis
To address the question about the outcomes of Total Knee Arthroplasty, the authors conducted a
systematic literature review from 1990 to October 2009.
The titles and abstracts of the resulting 3,519 references were then screened, using our inclusion
criteria: primary total knee arthroplasty studies
more than 100 knees per study Inclusion Criteria
baseline data and post-op outcomes data provided
experimental or quasiexperimental study design
English language
tricompartment
All articles that appeared to meet the screening criteria were abstracted by trained abstractors. Of
the original results, 611 references either met the inclusion criteria or needed further screening of the
full article to determine if they met inclusion. Of these, 62 studies reported pre- and post-Total
Knee Arthroplasty functional data using at least one of the four established measures we relied on:
Knee Society score, Hospital for Special Surgery score, WOMAC or SF-36.
Search Strings for Total Knee Arthroplasty Outcomes:
The literature search was done using the following combination of MeSH headings, keywords,
and publication types: arthroplasty, replacement, knee OR knee prosthesis OR "knee replacement"
OR "knee implant" OR TKAR OR knee injuries OR knee joint.

11. 4. “Appraise each individual data set, in terms of its
relevance, applicability…”
Annex X 1.1 – “The evaluation of this data, hereinafter referred to as ‘clinical
evaluation’, where appropriate taking account of any relevant harmonised
standards, must follow a defined and methodologically sound procedure …”

12. Anonymous Client Submission
12
MedDev 2.7.1

13. Anonymous Client Submission
13
“Equivalence” per MedDev 2.7.1.

14. EN ISO 14155-1:2003
14
EN ISO 14155-1:2003 does not use “relevance” or
“equivalence” but does use “similar”:
• technology
• critical performance
• design
• principles of operation
• patient or study populations
• indications for use
• severity and type of disease or condition
• conditions of use
• intended use

15. GHTF N2R8
15
GHTF N2R8 uses “comparable” and “similar”:
intended use:
• clinical condition being treated
• severity and stage of disease
• site of application to/in the body
• patient population
technical characteristics:
• design / specifications
• physiochemical properties
• deployment methods
• critical performance requirements
• principles of operation
• conditions of use
biological characteristics:
• biocompatibility of materials in contact with body fluids/tissues

16. 5. “Appraise each individual data set, in terms of its
relevance, applicability…
… quality and clinical significance.”
Annex X 1.1 d – “Where demonstration of conformity with essential
requirements based on clinical data is not deemed appropriate, adequate
justification for any such exclusion has to be given based on risk
management output ...”

17. 17
Quality and significance
• Weight results of individual papers or studies with a
“contribution to demonstrating overall performance
and safety of the device.”
• “… evaluate potential sources of bias”
 i.e. results reported on the same patients, multiple times?
 i.e. results from the designing surgeon better than all
others?

18. 6. “Where a clinical investigation has been carried
out by or on behalf of a manufacturer, it is expected
that documentation relating to the design, ethical and
regulatory approvals, conduct, results and
conclusions of the investigation needed for the
clinical evaluation will be available for consideration,
as appropriate.”
Annex X 1.1 c – “The clinical evaluation and its documentation must be actively
updated with data obtained from the post-market surveillance. Where post-
market clinical follow-up as part of the post-market surveillance plan for the
device is not deemed necessary, this must be duly justified and documented.”

19. GHTF N2R8
19
Clinical Investigation Data:
• Expected to be designed, conducted and reported in accordance
with EN ISO 14155 -1 & -2.
• Documentation to be available:
− Clinical Investigation Plan (study design, selection of
participants, treatment, blinding of participants and
investigators, dependent variables, follow up, statistical
analyses).
− CIP amendments, rationale and evidence of notification of
Regulatory Authority.
− Ethics Committed opinion, approved informed consent
forms, patient information documents.
− Case report forms, monitoring and audit records.
− Regulatory Authority approvals and correspondence.
− Final report – signed and dated.

20. MedDev 2.7.1
20
“Where the clinical investigation(s) was performed outside the EU, the
manufacturer must demonstrate that the use of the device (including
clinical practice and techniques) and patient population are equivalent to
those for which the device will be used within the EU.”
• Declaration of Helsinki
•ISO 14155 / GCP
• Pharma directive clinical
trials outside EU

21. 7. “What clinical investigation documentation/data
should be used in the clinical evaluation?
Clinical Investigation Plan – amendments, rationale
and evidence of notification of Regulatory Authority”
Annex X 2.3.5 – “All serious adverse events must be fully recorded and
immediately notified to all competent authorities of the Member States in
which the clinical investigation is being performed.”

22. 22

23. 8. “Outline clearly the conclusions reached about
the safety and performance of the device from the
evaluation, with respect to the intended use of the
device…”
Yes
No No
Do medical benefits
Is overall residual risk outweigh the overall
acceptable? (7) residual risk? (7)
Yes
Yes

24. GHTF N2R8
24
Clinical Evaluation Report:
• Conclusions:
− For each proposed indication clinical evidence
demonstrates conformity with relevant ERs:
− With or without clinical study on device itself
− Device performs as intended
− Device does not pose any safety concerns to either
recipient or end-user
− Risks associated with use of device acceptable when
weighed against benefits to patient

25. 9. “Outline clearly the conclusions reached about
the safety and performance of the device from the
evaluation, with respect to the intended use of the
device…
…State whether the risks identified in the risk
management documentation have been addressed
by the clinical data.”
Annex X 1.1 d – “Adequacy of demonstration of conformity with the
essential requirements by performance evaluation, bench testing and pre-
clinical evaluation alone has to be duly substantiated.”

26. 26
Holistic view
• Risks in RMF addressed in clinical
evaluation report
• Risks from CIP included in RMF
• Residual risks in RMF and in IFU
identical
• Positive acceptance of residual risks

27. 10. “The clinical evaluation report should be signed
and dated by the evaluator(s) and accompanied
by the manufacturer’s justification of the choice of
evaluator.”
Annex X 1.1 – “As a general rule, confirmation of conformity with the
requirements concerning the characteristics and performances referred to in
ER1 and ER3, under the normal conditions of use of the device, and the
evaluation of the side-effects and of the acceptability of the benefit/risk ratio
referred to in ER 6, must be based on clinical data.”

28. GHTF N2R8
28
Clinical Evaluation Report:
• Signed & Dated by someone “suitably qualified.”
• Possess knowledge of:
− Device technology and application.
− Research methodology (design and biostatistics).
− Diagnosis and management of conditions to be treated or
diagnosed.

29. 29
Clinical Evaluation – Ten Biggest Mistakes …
1. “Google” used to identify two or three papers, that are then summarised
to demonstrate compliance.
2. Literature review includes ONLY published studies of mechanical
testing, computer modelling, animal and in vitro experimentation.
3. Many papers identified, however not all selected for inclusion, without
justification.
4. Devices described as “substantially equivalent.”
5. Equal reliance on clinical data from one randomised control trial and
one expert opinion.

30. 30
Clinical Evaluation – Ten Biggest Mistakes …
6. Clinical Investigation – without notifying a EU Competent Authority or
without any other regulatory authority approval.
7. Clinical Investigation – Agreed a protocol with Competent Authority for
45 patients followed for 52 weeks.
Want to CE mark after 20 patients followed for 25 weeks = protocol
deviation.
8. Conclusion that the device under review is “substantially equivalent” to
other devices in the published literature.
9. No conclusions that benefits outweigh risks.
10. Unsigned and undated Clinical Evaluation.

31. Literature route easier option ??
• Clinical investigations • Literature route
+ targeted + less exensive
+ precise with good design + fast
+ better overall acceptance + low risk
- Expensive - Depends on quality of literature
- Time consuming - Historical data
- Logistically cumbersome - No perfect match of parameters
- Internal know how - Lower acceptance

32. 32
Contact Us
Name: Gert Bos
Title: Head of regulatory and Clinical Affairs
Address: BSI
Kitemark House, maylands Avenue
Hemel Hempstead, HP2 4SQ, UK
Telephone: +44 (0)1442 278664
Fax: +44 (0)8450 765601
Email: Gert.Bos@bsigroup.com
Links: www.bsigroup.com/healthcare