Quality measures in molecular labs

2. Objectives
• Understand the need for a Quality System
in molecular diagnostics laboratories
• Understand the major components of a
QS
• Understand the deliverables of QS in
molecular diagnostics

4. Available Genetic Tests and Laboratories Performing
Molecular Genetic Tests
Data source: GeneTests database /
www.genetests.org

6. Practical Applications of Molecular
Diagnostics
 Detection, quantitation, genotyping of infectious agents
Viruses, bacteria, fungi, and parasites
 Detection of defective genes and variations in the genome
o Molecular genetics: Genetic disorders (neuromuscular, endocrine,
cardiovascular, etc., diagnosis of existing disease or predisposition)
o Molecular oncology: Hematological malignancies, and gene defects,
expression profiles, etc., related to cancer
o Pharmacogenetics: Identification of metabolic gene variants (slow,
average, fast metabolizers, non-responders) to optimize drug therapy
o Genomics: Uses genomic information (gene expression and gene pattern)
for disease susceptibility, diagnostic classification, prognosis, and optimal
therapy
 Identification and characterization of individuals
Paternity, forensic medicine, pathology, transplantation

7. Special considerations for
molecular diagnostics laboratories
 Clinical validity and utility of nucleic acid tests
 Providing clinical interpretation of test results
 Staffing and technical expertise
 Regulatory oversight and best practice
considerations

8. Total Laboratory Testing Process

9. Establishment of performance specifications
 Accuracy
 Precision
 Analytic sensitivity
 Analytic specificity, to include interfering substances
 Reportable range of test results
 Reference intervals or normal values (if applicable)
 For quantitative assays:
 Linearity
 Dynamic range
 Limit or Detection (LOD)
 Lowest Limit of Quantitation (LLOQ)
 For genetic tests: establish clinical usefulness and validity,
including clinical sensitivity and specificity, and positive and
negative predictive value

10. If the test is a lab-developed human
genetic test
Conduct a review of available scientific studies and
pertinent references
Define appropriate patient populations for which the
test should be performed
Select the appropriate test methodology for the
disease or condition being evaluated
Establish analytic performance specifications and
determine quality-control procedures using the
appropriate number, type, and variety of samples
Ensure that test results can be interpreted for an
individual patient or family and that the limitations of
the test are well defined and reported

11. Specimen selection for test validation
 Evaluate the adequacy of the specimens for the
prevalence of the disease and the mutations or
variants
 Specimen types may include blood, buccal swabs,
dried blood spots, fresh or frozen tissue, paraffin-
embedded tissue, or prenatal specimens
 For a multiplex genetic test, all the mutations or
variants to be detected should be included in
performance establishment
 For rare genotypes/mutations, alternative samples
acceptable
 Specimen selection for test validation

12. Documentation of data
o Completeness
o Consistency
o Accuracy
o Reconstructability
“All data generated during the conduct of a study, except those that are
generated by automated data collection systems, shall be recorded
directly, promptly, and legibly in ink. All data entries shall be dated on
the date of entry and signed or initialed by the person entering the data.
Raw data includes the worksheets, calibration information, records and
notes of original observations, and activities of a study that are
necessary for the reconstruction and evaluation of the study. May
include photographic materials, computer printouts, automated and
hand recorded datasheets.”
FDA Good Laboratory Practices

13. 100
1,000,000
100,000
10,000
1000
10
1 2 3 4 5 6
Defectspermillion
Sigma Scale of measure
Domestic airline Fatality
rate
IRS Tax advice
(Phone In)
Order write up
Doctor prescription writing
Restaurant Bills
Airline Baggage
Handling
Laboratory Errors
Plebani M. The detection and prevention of errors in laboratory
medicine.
Ann Clin Biochem. 2010 Mar;47(Pt 2):101-10.
1

14. Background
1. Global Increase in Molecular Genetic Testing Applications
1. International Strides for Improving Quality Management in Genetic Testing services
A. OECD Guidelines for Quality Assurance in Molecular Genetic Testing. Organisation for
Economic Co‐operation and Development (OECD); 2007
B. EuroGentest – Harmonizing genetic testing across Europe
C. International Organization for Standardization (ISO) 15189 (2012). Medical laboratories
requirements for quality and competence
D. Increasing adherence to ISO and other national/international quality standards (eg,
countries in Europe, Asia, Americas, other continents)
3. CLSI Standards
A. Quality management systems (QMS) guidelines (eg, GP26‐A4)
B. Molecular methods guidelines (heritable diseases and oncology, molecular
hematopathology, FISH, infectious diseases, new test implementation, etc.)
C. Before MM20: No CLSI document specifically addressed QMS implementation and
maintenance in molecular genetic testing

16. What is the
Organization for Economic Cooperation and Development?
(OECD)
• Governmental Organization - established in 1961.
• Origin traced to post-WWII Marshal Plan
• 30 member countries + cooperative arrangements with 70+
others
• Focus is to promote economic growth and financial stability
• Biotechnology and health care viewed as key sectors
• Molecular genetic testing viewed as important and evolving
• Lack of an international framework
http://www.oecd.org

17. The Journey to Guideline Development
Focus: Molecular genetic testing for medical (disease-related) conditions
2000
2005
2007

18. Variables associated with higher quality indices
1. Standard Operating Procedures exist
2. Standard operating procedures are reviewed by director
3. Laboratory test result report is issued (written or electronic)
4. Laboratory test result report is reviewed by director
5. Offering of prenatal / pre-implantation testing
6. Affiliation with a genetics unit
7. Licensing and accreditation
8. Participation in proficiency testing
9. Not primarily in a research setting
10. Maintaining data on turnaround time
11. Lab director has an MD or PhD
12. Lab director is certified
13. Lab director has formal training
14. Lab technicians have a university degree
15. Lab technicians have relevant training

19. OECD Guidelines for Quality Assurance
in Molecular Genetic Testing
 Promote minimum standards
 Facilitate mutual recognition of quality assurance
frameworks
 Strengthen international co-operation to facilitate cross-
border flow of patient specimens for medically necessary
testing
 Increase public confidence in the governance of
molecular genetic testing

20. OECD Guidelines for Quality Assurance
in Molecular Genetic Testing
Components
1. General principles and best practices
2. Quality assurance systems
3. Proficiency testing
4. Quality of result reporting
5. Education and training standards
6. Annotations
Organization:
Principles (1,2,3...): Primarily directed to governments
Best Practices (i, ii, iii): Primarily directed to laboratories and other entities

21. General principles and best practices
A.2 - Molecular genetic testing should be delivered within the
framework of healthcare.
A.i - Regulatory/professional bodies should, as appropriate, review
whether instruments available to manage a quality assurance
framework require adaptation for molecular genetic testing
A.3 - All molecular genetic testing services should be provided and
practices under a quality assurance framework
A.iii - Molecular genetic test results should be reported back to the
referring health care professional to enable counseling and healthcare
decision making

22. Quality Assurance Systems
in Molecular Genetic Testing
B.1 - Governments and regulatory bodies should recognize that
accreditation of medical laboratories is an effective procedure
for assuring quality.
B.i. - All laboratories reporting molecular genetic testing results
for clinical care purposes should be accredited or hold an
equivalent recognition. (additional clarification for role
of research laboratories)
B.9 - Governments should encourage international collaboration
for the development and validation of molecular genetic tests.
B.viii - Laboratories should cooperate (internationally) to collect,
develop, verify and make available reference materials

23. Proficiency Testing:
Monitoring the Quality of Laboratory
Performance
C.1 - The performance of laboratories offering clinical molecular genetic
tests should be measured.
C.iii - Proficiency testing schemes should be structured to assess all
phases of the laboratory process, including result reporting.
C.4 - Accreditation or equivalent recognition should be the basis for the
international recognition of proficiency testing scheme and providers.
C.v - Laboratories should participate in a proficiency testing scheme for
every test offered, when such schemes exist. When not available, they
should participate in alternate methods.

24. Quality of Result Reporting
D.1 - All laboratories should issue molecular genetic testing
results in the form of a written and/or electronic report to the
referring health professional
D.i - Reports should communicate information effectively taking
into account that the recipient may not be a specialist.
D.2 - Where reports are issued directly to patients, it should be
encouraged that laboratories recommend consultation with an
appropriate health care professional.
D.4 - The interpretation of the result should be appropriate to the
individual and clinical situation and based on objective evidence.

25. Education and Training
Standards for Laboratory Personnel
E.2 - Standards for laboratory accreditation or other equivalent
recognition should require that all molecular genetics
personnel have a combination of education, training, skills,
and experience that ensures their competence.
E.4 - Development of educational and training programmes
should be encouraged where they do not exist.
E.vi. Comparison of education and training systems between
jurisdictions should be facilitated as a means to establish
equivalence.
E.5 - Relevant government or professional authorities should
recognize medical genetics as a discipline comprising both a
clinical and a laboratory specialty.

28. Purposes of MM20
 Provide guidance for implementing and maintaining
QMS in molecular genetic testing.
 Address specific QMS challenges in technical processes
and laboratory/user interphases of molecular genetic
laboratory services.
 Provide a resource to facilitate harmonized approaches
to accreditation to international laboratory standards.

29. What’s “Special” in MM20?
 Extends international QMS standards/guidelines (eg, ISO 15189,
CLSI document GP26) into molecular genetic testing services
 Incorporates recognized best practices for molecular genetic testing
worldwide
 Follows new/extended path of workflow
 Applies quality system essentials (QSEs) to quality management
and technical processes of molecular genetic testing
 Features many tools, job aids, resources

30. Scope
 Guidance for Implementing and Maintaining QMS for
Nucleic Acid–Based Human Molecular Genetic Testing
1. Intended for heritable (including pharmacogenetic testing) and
acquired conditions (eg, molecular oncology testing)
2. Not intended to address molecular infectious disease testing,
biochemical genetic testing, cytogenetic testing, specific technical
processes of molecular cytogenetic testing, molecular testing for
nonclinical purposes, or direct‐to‐consumer laboratory services
 Concordant with Use of ISO QMS Standards and Other
CLSI Guidelines (eg, GP26, MM01, MM12, MM17,
MM19)

31. Quality Management
Challenges

32. Need for Specific Quality
Management System
Guidance
 Quality Laboratory Services
1. Most appropriate examination procedures
2. Best suited sample(s)
3. Accurate and timely results with proper interpretation
4. Accurate and timely communications
 Quality Management Challenges for Molecular Genetic Testing
1. Diverse spectrum of testing services
2. Expanding applications impacting all medical disciplines
3. New users and clients continuously faced by laboratories
4. Laboratory’s continuing need to‐
• Consider new examination methods/procedures.
• Update existing examination procedures.
• Ensure effective communications with users and clients.
 Prerequisites for Providing Testing Services
1. Planning and preparation activities to ensure QMS/readiness for introducing or providing
molecular genetic testing services
2. Validation/verification of new or updated examination procedures before receiving test
requests/testing patient samples

33. Path of workflow

35. Quality System
Essentials for Molecular
Genetic Testing
 Discuss implementation of 12 QSEs in molecular genetic testing services (Section 6
in MM20).
6.1 Organization
6.2 Personnel
6.3 Documents and Records
6.4 Advisory Services
6.5 Assessment
6.6 Management of Nonconforming Events
6.7 Information Management
6.8 Continual Improvement
6.9 Use of Referral Laboratories
6.10 Evaluation of Vendor Qualification
6.11 Laboratory Equipment
6.12 Facilities, Environment, and Safety
 Describe policies and procedures to specifically address QMS needs in providing
molecular genetic testing services.

37. Example: Quality System
Essential Assessment
 Quality Indicators (QIs) – providing examples in the
laboratory’s path of workflow
 Internal Audits
1. Horizontal – for general processes in the path of workflow
(eg, acceptance of sequencing reactions)
2. Vertical – for a specific laboratory process (eg,following a
sample from sample receipt to result reporting)
 External Assessment
1. Voluntary and mandatory
2. Accreditation vs (third‐party) certification
 Management Review

39. Example: Quality System
Essential Information
Management
 Laboratory Information Systems (LIS) Considerations
1. Direct interface/compatibility with electronic medical records (for test ordering and
reporting)
2. Patient information (eg, race/ethnicity, indication for testing, family history) collected and
directly entered with test requests
3. Accommodating all fields/elements of molecular genetic test reports
4. Monitoring examination procedures, quality control (QC) tracking, follow‐up on TAT
 Accessibility and Retrievability
 Confidentiality and Security
 Data Management
1. Maintaining databases of sequence variants (eg, sequence variants identified in the
laboratory and literature, reference sequences, disease‐specific mutation databases)
2. Monitoring/updating as variants are reclassified
3. Ensuring consistency of result interpretation

40. Path of Workflow –
Technical Processes
 Planning and Preparation
 Validation/Verification of Test Performance
 Processes for providing examination services to
laboratory users
1. Preexamination activities
2. Examination activities
3. Postexamination activities

41. Planning and Preparation
Fundamental Management Considerations for Providing Molecular Genetic Test Services
 Ensuring all QSEs are in place and adequate for new tests/test services
 Determining preparedness for all applicable requirements
 Determining needs and demands, benefits and costs
 Identifying personnel competencies, training needs, and responsibilities
 Identifying special issues
 Informed consent
a) Genetic counseling
b) Intellectual property/licensing concerns
c) Ethical issues (testing of minors, use of tested samples, confidentiality)

42. Technical Aspects to Consider
 Specific intended use (and examination method to be used), different
planning issues:
a) Diagnostic testing
b) Carrier testing
c) Presymptomatic testing
d) Prenatal diagnosis
 Documenting clinical validity and utility:
a) Indications
b) Contraindications
 Planning for validation/verification of test performance

43. Validation/Verification of
Test Performance
 Develop validation/verification procedures
1. The intended use of the test (eg, carrier testing, fetal testing,
diagnostic testing)
2. Target genes, sequences, mutations
3. Expected patient population
4. Test methods to be compared or the method of choice to be
used
5. Type(s) of samples to be used
6. Analytical performance characteristics to be determined
7. Sources of reference materials
8. How performance specifications will be analyzed
9. How test limitations should be defined (eg, rate of allele
drop‐out, interfering mutations, polymorphisms)
10. Corrective actions when problems occur

44.  Identify samples/materials for analytical validation/verification
1. Adequate number, type, and variety of samples for establishing
test performance specifications and defining limitations
2. Control materials, calibration materials, other reference materials
 Determine analytical performance characteristics; define
performance specifications and limitations.
 Document results/findings.
 Prepare operational procedures for examination of patient samples.
 Perform ongoing validation.

45. Pre-examination Activities
 Begin with the laboratory informing users about:
① The molecular genetic tests it performs
② Test selection
③ Patient preparation (if needed)
④ Sample collection, handling, transport, submission with test requisitions (eg, sample
collection/submission manual)
 Information provided by laboratory aids health care providers and other users
in:
① Considering indications for testing/recognizing the need for a genetic test
② Selecting indicated test(s)
③ Shared decision making between a health care provider and the patient/pretest
genetic counseling leading to informed consent as indicated
④ Sending test request with patient sample(s) to laboratory
 Advisory services regarding consideration/selection of genetic tests
 Test referring process in case of sample referral

48. Examination Activities
 Examination Activities
I. Selection of test procedures according to user needs/expectations
II. Sample preparation/processing (eg, nucleic acid extraction/purification)
III. Examination procedures
IV. QC procedures
V. Documentation of test results and findings
 QC Plan/Program
I. Describe how all steps of analytical examination procedures are monitored.
II. Essential elements:
 Types of controls/control materials
 Frequency and placement of controls
 Analysis and recording of QC results
 Pass/fail criteria
 Corrective and preventive actions
 Alternative control procedures when control materials are not available

50. Postexamination Activities
 Postexamination Activities
 Reviewing examination results
 Providing laboratory interpretation
 Generating examination reports
 Transmitting test reports to the test requestor or other clinical users
 Providing laboratory consultation regarding test results, result interpretation, follow‐up
examinations or services
 Archiving of examination records, reports, and tested patient samples
 Recommended Contents of Molecular Genetic Test Reports
 Procedures for Releasing/Reporting Test Results
 Who may release test results and to whom
 Maintaining confidentiality of patient/family information
 Monitoring accidental disclosure and documenting corrective actions
 Procedures for Corrected, Revised, Amended Reports

52. Ensuring Quality of
Patient Testing
 Proficiency Testing/External Quality Assessment (PT/EQA)
 Existing programs for commonly performed molecular genetic tests (eg, CAP, CF
[European] Network, EMQN, EuroGentest, UK NEQAS)
 Differences between PT/EQA samples and actual patient samples: greater benefits
from challenging more steps of the testing process
 Section 7.3.3 in MM20 and CLSI documents GP27, MM01, MM14, and MM19
 Alternative Performance Assessment
 Interlaboratory comparison and intralaboratory evaluation when sample exchange is
not Available
 As frequent as would be required by participation in formal PT/EQA
 Rigorous in review and interpretation of results
 Management of Performance Assessment Results
 Important part of the laboratory’s quality assurance plan
 Documentation of assessment, results, and corrective actions
 Written policies on review and retention of performance assessment results

53. Personnel Qualifications,
Responsibilities, Competency

54. Personnel Competency
Assessment
 Methods for Competency Assessment for Laboratory
Management
 Appropriate levels of continuing education units
 Peer‐reviewed journal articles studied
 Membership/participation in professional organizations
 Credentials and maintenance of certification activities
 Proficiency slides or samples examined
 Interlaboratory sample exchanges with interpretation
 Assessment at annual review or other time, documented with
appropriate form
 Personnel competency assessment for testing personnel
 Determining when competency assessment is needed
 Key elements of competency assessment program for testing
personnel

55. Example of a Laboratory
Quality Manual