Genetic testing evaluation part 1 2018

John Shoffner, MD
Neurology, Biochemical Genetics, Molecular Genetics
EVIDENCE BASED ASSESSMENT OF GENETIC TESTING
PART 1
THE INTERPRETATION PROBLEM
MAY 2018

INTRODUCTION
 This presentation discusses important issues related to clinical validity and clinical
utility of genetic testing
 Genetic testing is rapidly penetrating essentially all aspects of patient care
 The number of laboratories and genetic tests has risen sharply since 2015 which presents overwhelming options to
healthcare providers that are difficult to assess
 Most genetic testing has not been rigorously assessed for use in the clinic by accepted approaches
 Most healthcare providers who order testing do not have the time or skills necessary to evaluate genetic tests the
diagnostic laboratories providing them
 Optimal use in the clinic and in clinical trials requires rigorous assessment of genetic testing options by accepted
evidence based approaches
 Evidence based assessment is required to insure clinical validity, clinical utility and economic viability
JOHN SHOFFNER MD

EVIDENCE BASED MEDICINE
 DEFINITION: The conscientious, explicit, and judicious use of current best evidence in making
decisions about the care of individual patients. The practice of evidence based medicine means
integrating individual clinical expertise with the best available external clinical evidence from
systematic research.
 ORIGIN: The term "evidence based medicine” was coined at McMaster Medical School in Canada
in the 1980's to label this clinical learning strategy, which people at the school had been
developing for over a decade.
Evidence based medicine: an approach to clinical problem-solving.
BMJ 1995 Apr 29;310(6987):1122-6 PMID 7742682
JOHN SHOFFNER MD

GENETIC TESTING IMPACT OF EVIDENCE BASED MEDICINE
 2018 study of 5404 participants who received secondary education in 78 countries
 Most from UK, USA, Russia
 Conclusion
 Genetic knowledge was poor for basic genetic literacy
 Average score 65.5% of 18 questions answered correctly
 1.2% answered all questions correctly
GENETIC TESTING AVAILABILITY HAS ADVANCED FASTER
THAN PEOPLE’S GENETIC KNOWLEDGE
New literacy challenge for the twenty-first century: genetic knowledge is poor even among well educated.
J Community Genet. 2018 Mar 28 PMID 29589204
THESE RESULTS HAVE IMPORTANT IMPLICATIONS FOR PRESENTING GENETIC TEST RESULTS TO PATIENTS,
PARTICULARLY WITH DIRECT TO CONSUMER TESTING
JOHN SHOFFNER MD

GENETIC TESTING GROWTH
SCOPE OF THE PROBLEM
JOHN SHOFFNER MD

So far in 2018
~75,000 active genetic testing units (GTUs) on the
market
Given the state of regulation of genetic testing,
assessing clinical utility, clinical validity and
economic impact is impossible for most who order
genetic testing!
DATA LINK: http://www.concertgenetics.com/wp-content/uploads/2018/04/12_ConcertGenetics_CurrentLandscapeOfGeneticTesting2018.pdf
JOHN SHOFFNER MD
2018 CONCERT GENETICS DATA

 As of March 1, 2018
 801 new genetic testing panels entered the market
 Rate of ~15 genetic testing panels per week
 In this 12 month period alone, the number of new genetic testing panels entering the market
represented >8% of the total number of panels on the market (9488)
 2017-2018 was big!
 National Society of Genetic Counselors issued an endorsement of the use of multi-gene panel
tests when clinically warranted and appropriately applied
 American College of Obstetricians and Gynecologists (ACOG) issued an opinion that
expanded carrier screening panels (often quite large) are acceptable for pre-pregnancy and
prenatal carrier screening
 American Medical Association (AMA) launched education programs on the use of precision
medicine tools, notably including expanded carrier panels and somatic cancer panels
 Center for Medicare and Medicaid Services (CMS) finalized a National Coverage
Determination that covers tests using next generation sequencing (NGS) for patients with
advanced cancer
JOHN SHOFFNER MD
RATE OF GENETIC TEST GROWTH

 Whole Exome Sequencing (WES): January
2016 – March 2018
 Number of Exome Sequencing Genetic
Testing Units (GTU) grew from 72 to 125
(74% increase)
 GTU = any orderable combination of
analytes
JOHN SHOFFNER MD
2016-2018 RATE OF GENETIC TEST GROWTH
= Total clinical exome tests offered by
US based labs

 Despite this increase in exome sequencing options,
health plans do not typically pay for this testing
 The commercial health plan paid claims in any given
quarter for the 2 billing codes specific to exome
sequencing was a fraction of a percent of the
hundreds of thousands of paid claims across all
genetic testing
JOHN SHOFFNER MD
POOR COMMERCIAL HEALTH PLAN REIMBURSEMENT
Maximum was 20 claims paid!!!

 Based on claims data from tens of
millions of commercially insured
members, this chart shows the
distribution of commercial payments
across high level clinical domains
 Left bars = 2017-2018 new testing (12
months)
 Right Bars = Areas of commercial health
plan reimbursement
JOHN SHOFFNER MD
REIMBURSEMENT CORRELATIONS POORLY WITH TESTING CATEGORIES ENTERING MARKET

 The growth in active genetic tests has been enormous
 A major issue is the difficulty in understanding the complex metrics relating to clinical validity of
each test
 Without a standardized approach to genetic testing evaluation, patients and healthcare providers
cannot make an informed decision on which laboratory or test to select
 While professional organization are important in assessing genetic testing and providing
guidelines, the problem has become too large, too fast
 Rigorous FDA regulation is needed to address the many complex issues
 Those laboratories committed to innovation, to academic partnerships, and to developing
programs that are compliant with strict regulation will be sustainable
Genetic Testing has too many important and far reaching implications to ignore. Test validation,
utility determination, and appropriate use are essential to the successful integration of genetic
testing into Precision Medicine programs
JOHN SHOFFNER MD

GENETIC TESTING
JOHN SHOFFNER MD

GENETIC TESTING
OVERVIEW
ACMG-AMP CRITERIA
DATA REPRODUCIBILITY
ACMG-AMP CRITERIA: ONE SIZE DOES NOT FIT ALL
INTER-LABORATORY INTERPRETATION DISAGREEMENT
LABORATORY AND CLINICIAN INTERPRETATION DISAGREEMENT
DATABASE ERRORS
LOSS OF FUNCTION VARIANTS
DE NOVO VARIANTS
RARE VARIANTS
FALSE POSITIVES
JOHN SHOFFNER MD

GENETIC TESTING
Clinical
Validity
• Is the gene
associated
with a
disease?
Pathogenicity
• It the variant
causative?
Clinical Utility
• Is the
information
actionable?
CRITICAL QUESTIONS FOR GENE VARIANT INTERPRETATION
JOHN SHOFFNER MD

GENETIC TESTING
BENEFITS AND DRAWBACKS OF COMMON SOURCES FOR
VARIANT INTERPRETATION
Data Source Benefits Drawbacks
Publications
• Peer reviewed
• Indexed in PubMed and searchable
• Access fees
• Research often transient
• Highly variable peer review
• Data unstructured with limited clinical info
• Variants not QC’d for nomenclature
• Cases in multiple publications
Databases
• Structured and standardized data
• Variant QC for nomenclature
• Lower barrier to submission
• Small bits can be shared at a time
• Little peer review
• Sometimes fee for access (e.g. HGMD)
• Inclusion of supporting evidence or an
interpretation may vary
Clinical Data from healthcare
providers
• Useful for classifying variants
• Quality depends on who is providing the data
and in what format
JOHN SHOFFNER MD

GENETIC TESTING
 No single data source, private or public, is comprehensive
 Data sources, including databases and publications, should be used as a
source of data/evidence, taking into account possible concerns over data
quality
 Claims must always be reassessed with the total body of evidence
TAKE HOME MESSAGE ABOUT DATA SOURCES FOR
VARIANT INTERPRETATION
JOHN SHOFFNER MD

GENETIC TESTING
 An important advancement in the interpretation of genetic variants was made in
2014 by the American College of Medical Genetics and Genomics and the
Association for Molecular Pathology (ACMG-AMP)
 While not the only classification system, it is the most commonly used. A review of
all gene variant classification systems is beyond the scope of this presentation
JOHN SHOFFNER MD

GENETIC TESTING
2014 Standardization of variant interpretation were
addressed in the ACMG-AMP guidelines
 ACMG-AMP guidelines are the most commonly used for genetic variant
interpretation but are not without problems that have important clinical
implications
 This section discusses some of the difficulties associated with genetic testing
interpretation
JOHN SHOFFNER MD

GENETIC TESTING
ACMG-AMP Variant Interpretation Guidelines
 28 criteria are used to place variants into the above evidence categories
 If a variant has insufficient criteria for placement into pathogenic or benign
categories, it is called a “variant of uncertain significance” or “VUS”
Discussion of individual criteria is beyond the scope of this presentation
JOHN SHOFFNER MD

GENETIC TESTING
ACMG-AMP Variant Interpretation Issues
 Guidelines lack specific details, parameters and cutoffs making criteria subjective, thus
decreasing inter-laboratory agreement
 More details to come!
 Guidelines are suitable for Mendelian disorders with high penetrance but are NOT SUITABLE
for disorders with variable penetrance
 The 28 criteria are not applicable to variants in all genes
 More details to come!
 Although the ACMG/AMP framework represents a major step forward in our ability to classify
variants in the context of Mendelian disease, it will need continued improvement and
refinement as our understanding of these diseases develops
Variant interpretation can change over time!
JOHN SHOFFNER MD

 The idea that the same experiments always get the same results, no matter who performs them, is one of the
cornerstones of science’s claim to objective truth. If a systematic campaign of replication does not lead to the
same results, then either the original research is flawed (as the replicators claim) or the replications are (as many
of the original researchers on priming contend)
 Although it is relatively easy to track papers that have been retracted and to remove associated annotations from
databases, the number of retractions is very small because only 0.2% of published papers are retracted annually,
whereas most papers with serious flaws remain
http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble
The Economist: Trouble at the lab, October 19, 2013
GENETIC TESTING
DATA REPRODUCIBILITY IS AN IMPORTANT ISSUE
JOHN SHOFFNER MD

DATA REPRODUCIBILITY IS AN IMPORTANT ISSUE
 2012 Nature paper
Scientists from Amgen reported that they could only reproduce 6 of 53 studies considered as landmarks in the
field of cancer research
GENETIC TESTING
Drug development: Raise standards for preclinical cancer research.
Nature 2012 Mar 28;483(7391):531-3 PMID 22460880
Believe it or not: how much can we rely on published data on potential drug targets?
Nat Rev Drug Discov 2011 Aug 31;10(9):712 PMID 21892149
 2011 Nature Reviews Drug Discovery paper
Researchers at Bayer HealthCare, reported that they could successfully reproduce results in only 25% of cases
JOHN SHOFFNER MD

IS “P-VALUE ≤0.05” A RELIGION?
ASSESSMENT OF DATA IS DIFFICULT IN LARGE STUDIES
AND VERY DIFFICULT IN SMALL STUDIES
 In 2016, a meta-analysis of P-value reporting in the biomedical literature 1990-2015
 96% of 2 million biomedical papers appealed to a p-value ≤0.05 to claim significance of the results
 Few articles included confidence intervals, Bayes factors, or effect sizes and reported only isolated P-values
 Although the P-value is critical to biomedical research, care must be taken when reviewing data to be
sure that the P-value is not a cover for poor data quality, biases, and conflicts of interest
GENETIC TESTING
Evolution of Reporting P Values in the Biomedical Literature, 1990-2015.
JAMA 2016 Mar 15;315(11):1141-8 PMID 26978209
 In 1970, Ronald Fisher proposed to use 0.05 as a P-value threshold for rejecting the null hypothesis
JOHN SHOFFNER MD
Fisher RA. Statistical methods for research workers.
14. Edinburgh: Oliver and Boyd; 1970

VARIANT INTERPRETATION DEPENDS ON
THE QUALITY OF THE STUDY DATA
 Data quality and reproducibility are important considerations for genetic variant interpretation, particularly for
rare variants and variants identified to cause rare diseases
 Single studies, even those studies containing functional data, must be assessed carefully
 Variable penetrance may be an important confounding variable which may not be recognized when a variant is
rare and only small numbers of patients are available for study
GENETIC TESTING
Expert panels are the best way to approach issues of data validity
when performing variant interpretation
JOHN SHOFFNER MD

GENETIC TESTING
 National Human Genome Research Institute–funded Clinical Genome Resource (ClinGen)
 Manages and centralizes clinically relevant genomic knowledge
 The historic lack of standardization is a major contributor for interpretation differences
 Many of these differences represent misclassifications, which can have serious consequences,
especially for medically actionable variants
 ClinGen is responding to difficulties with variant classification (as discussed in this
presentation)
 ClinGen has established a rich infrastructure including disease-specific expert working groups
that have been charged with accomplishing this goal
 Progress is slow due the extent of expertise needed for each gene
Adaptation and validation of the ACMG/AMP variant classification framework for MYH7-associated inherited cardiomyopathies:
recommendations by ClinGen's Inherited Cardiomyopathy Expert Panel.
Genet Med. 2018 Mar;20(3):351-359 PMID 29300372
JOHN SHOFFNER MD

GENETIC TESTING
EXAMPLE OF VARIABLE PENETRANCE PITFALLS
 2018 Study of ACMG-AMP criteria applicability
 MYH7 is a major contributor to several cardiomyopathies (HCM, DCM, RCM). Due to their high combined prevalence
and severe health outcomes, MYH7 is one of the most frequently tested genes in a clinical setting.
 Across MYH7-associated diseases, prevalence values were compiled from the literature and the most conservative
one was selected to derive a threshold that is applicable to all (1/200). Penetrance was set deliberately low at 30%.
 Expert revisions were tested with 60 variants using a structured double review by pairs of clinical and diagnostic
laboratory experts. Final consensus rules were established via iterative discussions.
 9 of the 28 ACMG-AMP criteria were deemed NOT applicable to MYH7 variant classification
 Internal data from participating diagnostic laboratories changed the classification of 20% of the variants (n = 12),
highlighting the critical importance of data sharing.
JOHN SHOFFNER MD

GENETIC TESTING
PITFALLS OF VARIABLE PENETRANCE
 This 2018 study emphasizes the importance of expert panels in variant curation
 Although ACMG-AMP criteria are important in standardizing variant
interpretations, all 28 criteria are NOT suitable for all genes
 Many diagnostic laboratories do not share data and do not have the internal
resources for consistent variant curation
 Without regulations for QA/QC metric transparency, evaluation of individual
laboratories by healthcare providers is difficult
JOHN SHOFFNER MD

GENETIC TESTING
ACMG-AMP Variant Interpretation Issues
Inter-Laboratory Disagreement
 Note the scope of the problem among 9 laboratories interested in improving inter-laboratory
agreement
 Remember the huge growth in genetic laboratories and testing and the LACK of participation of
most laboratories in these programs! Practices in hundreds of laboratories are not represented
in the literature
 In 2016 study of 9 laboratories, inter-laboratory concordance for scoring variants using ACMG-
AMP criteria was only 34%! These criteria are NOT easy to apply consistently.
 Consensus discussions increased concordance to 71%
Reminder: Criteria are applicable to Mendelian disease with HIGH PENETRANCE,
NOT variable penetrance
Performance of ACMG-AMP Variant-Interpretation Guidelines among Nine Laboratories in the Clinical Sequencing Exploratory Research Consortium.
Am J Hum Genet. 2016 Jun 2;98(6):1067-1076 PMID 27181684
JOHN SHOFFNER MD

GENETIC TESTING
ACMG-AMP VARIANT INTERPRETATION ISSUES
INTER-LABORATORY DISAGREEMENT
 Recommendations by authors to increase consistency in usage of ACMG-AMP criteria
 It is very difficult for clinicians to understand whether recommendations are being adopted by the huge array of
commercial laboratories
AUTHOR RECOMMENDATIONS
 Develop disease-specific allele-frequency thresholds to enable lowering of the stand-alone benign criteria from a MAF of ≥5% to
values specific to each disorder
 Establish a resource of all genes to define whether Loss of Function (LOF) is a known mechanism of disease
 See LOF comments slide below
 Make recommendations for which computational algorithms are best in practice
 Better define “well-established” functional data and/or distribute a resource that lists functional assays that meet the well-established
threshold. Also define when to use reduced strength of the rule
 Develop quantitative thresholds of evidence for and against segregation of different strengths
 Promote the development of software tools that automate computable aspects of the ACMG-AMP guidelines to improve accurate use
Performance of ACMG-AMP Variant-Interpretation Guidelines among Nine Laboratories in the Clinical Sequencing Exploratory Research Consortium.
Am J Hum Genet. 2016 Jun 2;98(6):1067-1076 PMID 27181684
JOHN SHOFFNER MD

GENETIC TESTING
ACMG-AMP VARIANT INTERPRETATION ISSUES
BETWEEN LABORATORY AND CLINICIANS
 2018 study assessed frequency and nature of differences in variant classification between clinicians and genetic
testing laboratories
 688 laboratory classifications
 18% (124) differed from clinician classification
 83% (109/124) of interpretation differences had clinical impact
 Discordance varied between laboratories. (YOUR LABORATORY CHOICE MATTERS!)
Clinically impactful differences in variant interpretation between clinicians and testing laboratories: a single-center experience.
Given the complexities of variant interpretation, this is a real problem.
Who is right?
What do you tell patients?
Determining the level of certainty/uncertainty is critical
Standardized evidence based interpretation is essential
JOHN SHOFFNER MD

GENETIC TESTING
OVERVIEW
ACMG-AMP CRITERIA
DATABASE ERRORS
DE NOVO VARIANTS
RARE VARIANTS
FALSE POSITIVES

DATABASE RELIABILITY FOR VARIANT INTERPRETATION
 A large array of databases as well as the medical literature are used by
laboratories for variant curation
 Currently databases are unregulated and are prone to errors. Database
unreliability has been a problem for many years and is slowly being addressed.
 2011 assessment: ~27% of database entries potentially unreliable (common
polymorphisms or misannotated)
 In later slides we will discuss the FDA proposal for database regulation which is
badly needed
Carrier testing for severe childhood recessive diseases by next-generation sequencing.
Sci Transl Med. 2011 Jan 12;3(65):65ra4 PMID 21228398
GENETIC TESTING
JOHN SHOFFNER MD

CLINVAR IS A COMMONLY USED DATABASE
 The database includes germline and
somatic variants of any size, type or
genomic location. Interpretations are
submitted by clinical testing
laboratories, research laboratories,
locus-specific databases, OMIM,
GeneReviews, UniProt, expert
panels and practice guidelines.
ClinVar: public archive of interpretations of clinically relevant variants.
Nucleic Acids Res 2016 Jan 4;44(D1):D862-8 PMID 26582918
DATABASE LINK: https://www.ncbi.nlm.nih.gov/clinvar/
SAMPLE PAGE
GENETIC TESTING
JOHN SHOFFNER MD

GENETIC TESTING
SIGNIFICANT VARIANT MISCLASSIFICATION
IN CLINVAR
 2018 study used whole-genome sequence data from 10,495 unrelated individuals to contrast population frequency of
pathogenic variants to the expected population prevalence of the disease
 Analyses included the ACMG-recommended 59 gene-condition sets for incidental findings and 463 genes associated with
265 OrphaNet conditions
 25,505 variants were used to identify patterns of inflation (i.e., excess genetic risk and misclassification)
 Up to 11.5% of genetic disorders with inflation in pathogenic variant sets and up to 92.3% for the variant set with conflicting
interpretations
 CONCLUSION
The study shows that databases include a significant proportion of wrongly ascertained variants
Identification of Misclassified ClinVar Variants via Disease Population Prevalence.
Am J Hum Genet. 2018 Apr 5;102(4):609-619 PMID 29625023
JOHN SHOFFNER MD

 Database is private
 High cost for use
 Curators simply enter pathogenic claims for the literature that are often wrong!
 Benign variants are not entered into the database
 NOT EXPERTLY CURATED!
HUMAN GENOME MUTATION DATABASE (HGMD)
DATABASE COMMONLY USED FOR VARIANT ASSESSMENT
HTTP://WWW.HGMD.CF.AC.UK/AC/INDEX.PHP
JOHN SHOFFNER MD
GENETIC TESTING

1000 Participants
 114 Genes Selected by Expert Panel to be Medically Actionable
239 variants classified by HGMD as “Disease Causing”
 After Expert Review, only 7.5% (18/239) of these variants were classified
as disease causing
SIGNIFICANT VARIANT MISCLASSIFICATION
IN HGMD
Actionable, Pathogenic Incidental Findings in 1,000 Participants’ Exomes
Am J Hum Genet. 2013 Oct 3;93(4):631-4 PMID 24055113
JOHN SHOFFNER MD
GENETIC TESTING

 Databases commonly used for mtDNA variant assessment harbor significant errors
 A major challenge for diagnostic laboratories is assessing the pathogenicity of mtDNA
variants because a significant amount of confusion exists in their classification
 Over 200 variants are reported in the 22 mitochondrial tRNAs but less than half meet
contemporary criteria for the “Definitely Pathogenic” classification
MITOCHONDRIAL DNA (MTDNA) MISCLASSIFICATION
A comparative analysis approach to determining the pathogenicity of mitochondrial tRNAmutations.
Hum Mutat. 2011 Nov;32(11):1319-25 PMID 21882289
JOHN SHOFFNER MD
GENETIC TESTING

 Databases commonly used for
mtDNA variant assessment
harbor significant errors
 Literature search and an array of
databases were used to identify
26 mtDNA variants in 15 mtDNA
genes that were claimed to
cause MERRF
MTDNA MISCLASSIFICATION
MYOCLONIC EPILEPSY WITH RAGGED-RED FIBERS (MERRF)
MERRF Classification: Implications for Diagnosis and Clinical Trials.
Pediatr Neurol. 2018 Mar;80:8-23 PMID 29449072
JOHN SHOFFNER MD
• Compilation of mammalian mitochondrial tRNA genes http://mamittrna.u-strasbg.fr/
• dbSNP https://www.ncbi.nlm.nih.gov/projects/SNP/
• Ensembl https://www.ensembl.org/index.html
• GeneCards http://www.genecards.org/
• Human DNA Polymerase gamma Mutation Database https://tools.niehs.nih.gov/polg/
• MalaCards Human Disease Database http://www.malacards.org/
• Mitochondrial tRNA database (mitotRNAdb)
http://mttrna.bioinf.unileipzig.de/mtDataOutput/
• MITOMAP https://www.mitomap.org/MITOMAP
• MitoTool http://www.mitotool.org/
• National Center for Biotechnology Information database
https://www.ncbi.nlm.nih.gov/
• Neuromuscular Disease Center Database http://neuromuscular.wustl.edu
• OMIM https://www.omim.org/
• POLG Pathogenicity Prediction Server http://polg.bmb.msu.edu/index.php
• UCSC genome browser https://genome.ucsc.edu/
• Uppsala human mitochondrial genome database (mtDB) http://www.mtdb.igp.uu.se/
GENETIC TESTING

 Approaches for objective, systematic, and evidence-based classification of genes and variants are evolving rapidly but
have not been uniformly applied to mtDNA variants
 Unfortunately, the 2015 ACMG– AMP guidelines for variant classification as well as other classification systems used for
nuclear DNA variants are not optimal for curation of mtDNA variants, particularly mitochondrial transfer RNA (MT-tRNA)
variants
 Mitochondrial DNA classifications rely heavily on functional data from trans-mitochondrial cybrid studies, single-fiber
studies, or mutant mt-tRNA steady– state-level studies
 Classification of MT-tRNA variants is best performed with classification system proposed by Yarham et al (2011) and
modified by González-Vioque et al (2014)
JOHN SHOFFNER MD
A comparative analysis approach to determining the pathogenicity of mitochondrial tRNA mutations.
Hum Mutat. 2011 Nov;32(11):1319-25 PMID 21882289
GENETIC TESTING
The pathogenicity scoring system for mitochondrial tRNA mutations revisited.
Mol Genet Genomic Med. 2014 Mar;2(2):107-14 PMID 24689073

 After careful classification of the 26 variants associated with MERRF in peer reviewed research papers and in commonly
used databases by criteria used for MT-tRNA variants, significant reclassification was required
 DEFINITELY PATHOGENIC – 73.1% (19/26)
 PROBABLY PATHOGENIC – 3.8% (1/26)
 POSSIBLY PATHOGENIC – 11.5% (3/26)
 NEUTRAL – 15.4% (4/26)
 15.4% OF THE VARIANTS ASSOCIATED WITH ONE MITOCHONDRIAL DISEASE PHENOTYPE WERE MISCLASSIFIED
 THE MISSCLASSIFICATION FROM PATHOGENIC TO BENIGN IS CLINICALLY RELEVANT
 Regulatory oversight of genetic databases would enhance their clinical utility and decrease discrepancies in variant
interpretation between laboratories
JOHN SHOFFNER MD
GENETIC TESTING

SEE PART 2 FOR DISCUSSION OF REGULATORY CHANGES
FOR DATABASES UNDER CONSIDERATION
BY THE FDA
GENETIC TESTING
JOHN SHOFFNER MD

JOHN SHOFFNER MD
GENETIC TESTING
OVERVIEW
ACMG-AMP CRITERIA
DATABASE ERRORS
DE NOVO VARIANTS
RARE VARIANTS
FALSE POSITIVES

GENETIC TESTING
COMMENT
LOSS OF FUNCTION (LOF) VARIANTS
 LOF variants often cause disease but not necessarily.
 The role of LOF variants must be determined individually for each gene.
 Remember: A biochemical consequence does not mean that there will be a recognized clinical consequence
 ~100 Loss of Function (LOF) Variants per Individual
 ~20 LOF Variants Homozygous per individual
 Biochemically deleterious but without recognized phenotype
 EXAMPLE: c.280C>T [p.Arg94*] in MOK [formerly RAGE1 (MIM 605762; RefSeq NM_014226.1; rs34931752)
 May be deleterious in one population but not another (men versus women for example)
 EXAMPLE: FEMALE INFERTILITY: c.598A>G [p.Met200Val] in DMC1 (meiosis specific recombinase) [MIM 602721; RefSeq NM_007068.2
 Eur. J. Endocrinol. 158, 107–115, 2008; Nucleic Acids Res. 36, 4181–4190, 2008
 Phenotypes require specific environmental trigger
 EXAMPLE: STOX1 variants and preeclampsia predisposition
 Nat. Genet. 37, 514–519, 2005; Hum. Mol. Genet. 19, 2658–2667, 2010
Deleterious- and Disease-Allele Prevalence in Healthy Individuals: Insights from Current Predictions, Mutation Databases, and Population-Scale Resequencing.
Am J Hum Genet. 2012 Dec 7;91(6):1022-32. PMID 23217326
JOHN SHOFFNER MD

GENETIC TESTING
DE NOVO VARIANTS
Be careful about pathogenicity assumptions!
 ~1 per exome (average rate of 1.2 x 10-8 per nucleotide per generation)
 De novo rate differs among genes
 Knowing the frequency of de novo variant occurrence for a gene of interest is essential
 Distinguishing de novo occurrence from mosaicism can be challenging
 Differentiation between de novo occurrence and mosaicism has a major effect on
recurrence risk assessment
 Rare de novo variants that are NOT highly penetrant are difficult to assess
 Best study for penetrance assessment is an age-related longitudinal cohort study,
preferably prospective
 De novo status may support pathogenicity when
 Disease is rare
 In a gene associated with phenotype
 Non-paternity, mosaicism and sample errors excluded
JOHN SHOFFNER MD
Exome sequencing in sporadic autism spectrum disorders identifies severe de
novo mutations.
Nat Genet. 2011 Jun;43(6):585-9. PMID 21572417
Rate of de novo mutations and the importance of father's age to disease risk.
Nature. 2012 Aug 23;488(7412):471-5 PMID 22914163

GENETIC TESTING
 Most clinically significant variants are extremely rare, making interpretation
difficult
 Different laboratories may interpret the meaning of the same variant
differently
 Healthcare providers may need to explore the details of the interpretation with the
diagnostic laboratory to understand the criteria used as well as the expertise of staff
RARE VARIANTS
JOHN SHOFFNER MD

GENETIC TESTING
Most clinically significant variants are
extremely rare and require global sharing to
amass even small amounts of data
Very few variants have enough data to make
statistically robust approaches to interpretation
• Case-control studies
• Validated functional assays
ClinGen — The Clinical Genome Resource.
N Engl J Med. 2015 June 4; 372(23): 2235–2242 PMID 26014595
83% of patients have variants
that are rare or of uncertain
clinical significance
17% of patients have pathogenic or
likely pathogenic variants seen ≥10
times
JOHN SHOFFNER MD

 FDA limits type of health-related claims DTC tests can market
 Some companies provide raw genetic data to clients
 2018 study assessed outcome when consumers received this data
 40% of variants designated as “Increased Risk” were classified as “Benign” when assessed
by appropriate criteria
 Healthcare providers must exercise caution when asked by patients to comment on DTC
genetic data
GENETIC TESTING
False-positive results released by direct-to-consumer genetic tests highlight the importance of clinical confirmation testing for appropriate patient care.
Genet Med. 2018 Mar 22. doi: 10.1038/gim.2018.38 PMID 29565420
DIRECT TO CONSUMER TESTING
THE FALSE POSITIVE PROBLEM
JOHN SHOFFNER MD

 2014 study, authors demonstrate that a seemingly low error rate in next-generation sequencing can
dramatically impact the false-positive rate for rare variants
 This is due to the fact that rare variants are, by definition, seen infrequently, making it hard to
distinguish between errors and real variants
 The error rate in NGS testing cannot be generalized to all genes assessed
 Error rate is influenced by many variables including
 Characteristics of the DNA segment being analyzed (sequence specific susceptibility to sequencing errors)
 For example: A/T rich regions and homopolymers
 Mosaicism
 Pseudogenes
GENETIC TESTING
Population genetics identifies challenges in analyzing rare variants.
Genet Epidemiol. 2015 Mar;39(3):145-8. PMID 25640419
THE FALSE POSITIVE PROBLEM AND RARE VARIANTS
JOHN SHOFFNER MD

 To assess the False Positive Problem, diagnostic laboratories must carefully assess their error rate
 Numerous diagnostics companies have entered the market recently, offering low-priced tests often at the expense of
accuracy by avoiding high-priced steps such as Sanger-sequencing confirmations and microarrays for validating
deletions and duplications
 Be sure that any diagnostic laboratory claiming 100% specificity by NGS data alone shows you’re their data, preferably
in a peer reviewed publication (MANY MAKE THIS CLAIM!)
 2016 publication by AMBRY GENETICS reported results from 20,000 hereditary cancer NGS panels spanning 47
genes
 7845 nonpolymorphic variants were Sanger sequenced
 This study reports important metrics which can be used to guide assessment of diagnostic testing offered by
the large array of laboratories
 Remember: Testing metrics must be proven for each test (group of genes). Metrics of optimal sensitivity,
specificity, positive predictive value, and negative predictive value can vary.
GENETIC TESTING
Sanger Confirmation Is Required to Achieve Optimal Sensitivity and Specificity in Next-Generation Sequencing Panel Testing.
J Mol Diagn. 2016 Nov;18(6):923-932. PMID 27720647
JOHN SHOFFNER MD

 Technical factors influence the FALSE POSITIVE RATE
 Laboratories have different sequencing approaches that influence the FALSE POSITIVE RATE. Important aspects include:
 Capture chemistry (e.g. FALSE POSITIVES more problematic in primer based target enrichment than in probe-based
enrichment)
 Sequencing platform
 Analytical pipeline
 Bioinformatics and the FALSE POSITIVE RATE
 Adjusting the threshold of variant detection during the data analysis to result in a ZERO FALSE POSITIVE RATE resulted in
missing 2.2% of relevant variants
 CONCLUSION
 This comprehensive study by AMBRY GENETICS demonstrated the rigorous approach needed to optimize
their 47 gene panel
 They determined that the panel has a 1.3% FALSE POSITIVE RATE when optimized for clinical use
GENETIC TESTING
Sanger Confirmation Is Required to Achieve Optimal Sensitivity and Specificity in Next-Generation Sequencing Panel Testing.
J Mol Diagn. 2016 Nov;18(6):923-932. PMID 27720647
JOHN SHOFFNER MD

 Genetic testing has experienced enormous growth in the number of genetic tests and the number of
laboratories since 2015
 Evidence based assessment of genetic is unable to keep up leading to impactful uncertainties for
healthcare providers and patients
 Increased regulation and oversight is needed to insure that clinical validity, clinical utility and the
economic impact of genetic testing can be appropriately assessed
EVIDENCE BASED ASSESSMENT OF GENETIC TESTING
PART 1
JOHN SHOFFNER MD
SUMMARY

Genetic testing evaluation part 1 2018

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Genetic testing evaluation part 1 2018

Ähnlich wie Genetic testing evaluation part 1 2018 (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Genetic testing evaluation part 1 2018