Introduction to NGS data, processing and mapping to detect polymorphisms.

1. www.nottingham.ac.uk/adac

2. Obtaining, QC, mapping and analysis of
NGS data.
Richard Emes
Associate Professor & Reader in Bioinformatics.
School of Veterinary Medicine and Science
Director Advanced Data Analysis Centre
richard.emes@nottingham.ac.uk
www.nottingham.ac.uk/adac
@rdemes
@ADAC_UoN
2

3. What is ADAC?
The University of Nottingham Advanced Data Analysis Centre (ADAC).
• Bioinformatics and data analysis support.
Why is this important?
• Complex data underpins much current research.
• Innovative analysis can prompt new discoveries.
• Excellent research can often be stalled due to a lack of expertise in conducting data
analysis, availability or cost of inclusion of diverse specialists.
Why ADAC?
• ADAC supports high-class research by providing analysts with expertise in a range of
bioinformatics and computer science disciplines.
• Flexible support
• Consultancy, collaboration, bespoke analysis.
• Leadership from recognized experts in the fields of bioinformatics and computer
science.
• Track record of funding
• Pivotal role in collaborations funded by amongst others Zoetis, BBSRC, NERC
and Technology Strategy Board.
• ADAC is conducting transcriptome analysis for a multinational FP7 funded
project (EU Prohealth).

4. http://www.nottingham.ac.uk/adac/
Enquiries: adac@nottingham.ac.uk or richard.emes@nottingham.ac.uk
@ADAC_UoN or @rdemes
Current Areas of expertise relevant here:
• Transcriptomics (Microarray, NGS)
• Comparative genomics (eukaryotic, prokaryotic)
• Identification of biomarkers from genetic and epigenetic
datasets
• Artificial intelligence for decision support
• Machine Learning
• Integration of complex datasets
• Data Management

5. @HWI-_FC_20BTNAAXX:2:1:215:593
ACAGTGCATGACATGCATAGCAGCATAGACTAC
+HWI-_FC_20BTNAAXX:2:1:215:593
GhhhhhhhhhhhUhhEGhhhGhhhhhhhhhhhhh

6. Some common terms
• Library: collection of molecules. This is the “complexity” of what you sequence.
• Flowcell: slide where sequencing is attached to a solid platform.
• Lane: unique sequencing unit of the flowcell.
• Reads: Raw sequence of bases and imputed quality scores.
• Fragment: Original molecule being sequenced (fragment of genome/gene). Ie
PE are reads form the same fragment.
• Cluster: DNA bound to slide, local amplification of product to amplify signal to
measure fluorescence.
• Mapping: Finds where your sequence matches to a reference. Importantly
gives a probability that this is the correct location.

7. Illumina sequencing

8. Illumina sequencing

11. What coverage do I need?

12. Obtaining NGS Data
• Short Read Archive (SRA)
• European Nucleotide Archive (ENA)

13. Obtaining NGS Data
• Short Read Archive (SRA)
• Will need to convert to FastQ using sra toolkit

14. Deciphering a fastq file
@HWI-_FC_20BTNAAXX:2:1:215:593#0/1
ACAGTGCATGACATGCATAGCAGCATAGACTAC
+HWI-_FC_20BTNAAXX:2:1:215:593#0/1
GhhhhhhhhhhhUhhEGhhhGhhhhhhhhhhhhh
Header: @HWI-_FC_20BTNAAXX:2:1:215:593#0/1
HWI-_FC_20BTNAAXX instrument identifier
2 flowcell lane
1 tile number in flowcell lane
215 x - coordinate of cluster in the tile
593 y - coordinate of cluster in the tile
#0 index of multiplexed samples
/1 member of pair /1 or /2 if Paired end

15. Deciphering a fastq file
@HWI-_FC_20BTNAAXX:2:1:215:593#0/1
ACAGTGCATGACATGCATAGCAGCATAGACTAC
+HWI-_FC_20BTNAAXX:2:1:215:593#0/1
GhhhhhhhhhhhUhhEGhhhGhhhhhhhhhhhhh
Sequence: ACAGTGCATGACATGCATAGCAGCATAGACTAC
Quality Header: +HWI-_FC_20BTNAAXX:2:1:215:593
Quality: GhhhhhhhhhhhUhhEGhhhGhhhhhhhhhhhhh

16. Deciphering a fastq file
Quality: GhhhhhhhhhhhUhhEGhhhGhhhhhhhhhhhhh
Sanger encoding = ASCII table lookup – 33
Solexa encoding = ASCII table lookup – 64
G = 71 – 64 = 7
h = 104- 64 = 40

17. Deciphering a fastq file
Quality: GhhhhhhhhhhhUhhEGhhhGhhhhhhhhhhhhh

18. SNP Calling
• Genotyping: identifying variants in a single genome
(i.e. from each parent)
• SNP Calling: identifying variants between individual genomes
ACGTGCAGCATAGCA?CGACATCGACATACGC
TGCACGTCGTATCGT?GCTGTAGCTGTATGCG
****A*******
***A*****
**T******
*****T**********
******T********
ACGTGCAGCATAGCATCGACATCGACATACGC
TGCACGTCGTATCGTAGCTGTAGCTGTATGCG
Sample Genome(s)
Reads
Reference Genome

19. INDEL Calling
• INDEL: Insertion/deletion
ACGTGCAGCATAGCA???CGACATCGACATACGC
TGCACGTCGTATCGT???GCTGTAGCTGTATGCG
****ACG*******
***ACG*****
**---********
*****---**********
******---********
ACGTGCAGCATAGCACGTCGACATCGACATACGC
TGCACGTCGTATCGTGCAGCTGTAGCTGTATGCG
Sample Genome(s)
Reads
Reference Genome

20. A pipeline for SNP identification
• Quality Control
– FastQC, Fastx toolkit
• Trimming:
– Sickle, Trimgalore, Trimomatic, Cutadapt
• Mapping
– BWA, Bowtie, Stampy
• Remove Duplicates
– Picard tools, Samtools rmdup
• Call SNPs / INDELS
– Samtoools mpileup, VarScan, GATK, Many
others!

21. Galaxy: https://usegalaxy.org/
Toolbox Workflows

22. Visualize the data
• FASTQC
• Stand Alone or non-interactive
– Basic Statistics module, includes:
• Filename: The original filename of the file which was analyzed
• Encoding: Says which ASCII encoding of quality values was found in this file.
• Total Sequences: A count of the total number of sequences processed.
• Sequence Length: Provides the length of the shortest and longest sequence in
the set. If all sequences are the same length only one value is reported.

23. Visualize the data
• FASTQC: Per Base Sequence Quality:
Red line = Median quality
Yellow box = IQR
Whiskers = 10%-90%
Blue line = Mean quality
If the lower quartile for
any base is less than 10,
or if the median for any
base is less than 25.
If the lower quartile for
any base is less than 5 or
if the median for any base
is less than 20.

24. Visualize the data
• FASTQC: Per Sequence Quality Scores:
If the most frequently
observed mean quality is
below 27 - this equates
to a 0.2% error rate.
If the most frequently
observed mean quality is
below 20 - this equates
to a 1% error rate.

25. Visualize the data
• FASTQC: Per Base Sequence Content
Proportion of each base
position in a file for which
ATCG DNA bases has
been called.
If the difference between
A and T, or G and C is
greater than 10% in any
position.
If the difference between
A and T, or G and C is
greater than 20% in any
position.
Possibly adapters or
affect of trimming.

26. Visualize the data
• FASTQC: Sequence Length Distribution
Distribution of fragment
sizes in the file.
If all sequences are not
the same length.
If any of the sequences
have zero length.

27. Visualize the data
• FASTQC: Duplicate Sequences
Degree of duplication
within first 200,000
reads of file. Distribution
of duplication levels in
dataset
If non-unique sequences
make up more than 20%
of the total.
If non-unique sequences
make up more than 50%
of the total.

28. Visualize the data
• FASTQC: Overrepresented Sequences
• FASTQC: Adapter Content
Lists all of the sequences which make up more than 0.1% of the total.
If any sequence is found to represent more than 0.1% of the total.
If any sequence is found to represent more than 1% of the total.
To know if your library contains a significant amount of adapter in order to be able
to assess whether you need to adapter trim or not.
If any sequence is present in more than 5% of all reads.
If any sequence is present in more than 10% of all reads.

29. Cut adapters
-f = the type of file (in this case fastq)
-q CUTOFF, Trim low-quality ends from reads before adapter removal.
-a ADAPTER, Sequence of an adapter that was ligated to the 3' end. The adapter itself and
anything that follows is trimmed.
-m 100 minimum length of reads following adapter removal. Reads less than 100 will be
discarded
--discard-untrimmed any reads without an adapter will be discarded.
-o output file also in fastq format.
cutadapt -f fastq -q 20 -a AGATCGGAAGAG -m 100 --discard-untrimmed
-o SNP.test.trimmed.fastq SNP.test.fastq

30. Cut adapters
Galaxy: Fastx_clipper from fastx_toolkit

31. Quality filters

33. Quality filters
Fastx_toolkit
-q = Minimum quality score to keep.
-p = Minimum percent of bases in a read that must have [-q] quality.
- i input file (output of adapter trimming step)
-v verbose
-Q quality encoding
-o output file also in fastq format.
fastq_quality_filter -q 20 -p 70 -i SNP.test.trimmed.fastq -v –Q64
-o SNP.test.trimmed.QC.fastq

34. Quality filters
• Galaxy: filter by quality

35. Align to genome
The problem
• Generally a large genome (Human > 3Gb)
• Large number of short reads
The solution
• Index genome into hash of kmers or short sequences
• Use efficient aligners
– Large number of aligners available.
• Common aligners: Bowtie1/2, BWA, Stampy

36. Align to genome
Example Bowtie 2 alignment
Build index
-f fasta formatted genome file
./bowtie.index.files/chr17 output location for index files
Galaxy: select pre-built index when using bowtie or BWA
bowtie2-build -f ./genome/chr17.fa ./bowtie.index.files/chr17

37. Align to genome
Align to reference
-p number of processors
--end-to-end alignment is not local
-k 1 number of positions read is allowed to align k = 1 means all non-
uniquely mapping reads are discarded
-x path to indexed genome file to align reads to.
-U reads are unpaired (in this case
-S output in SAM format
bowtie2 -p 4 --end-to-end -k 1 -x ./bowtie.index.files/chr17
-U SNP.test.trimmed.QC.fastq -S SNP.test.trimmed.QC.fastq.sam

39. Alignment file formats
SAM - Sequence Alignment/Map format. (BAM is a binary compressed
equivalent)
– TAB-delimited text format
– header section (optional)
@HD = Header, VN[format version], SO[sorting order]
@SQ = Reference sequence line
@PG = Program, ID[program ID], CL[command line]
@HD VN:1.0 SO:unsorted
@SQ SN:chr17 LN:81195210
@PG ID:bowtie2 PN:bowtie2 VN:2.2.1 CL:"/
home/rde/tools/bowtie2-2.2.1/bowtie2-align-s --wrapper
basic-0 -p 4 --end-to-end -k 1 -x ./bowtie.index.files/
chr17 -S SNP.test.trimmed.QC.fastq.sam -U
SNP.test.trimmed.QC.fastq”

40. Alignment file formats
• SAM - Sequence Alignment/Map format.
– TAB-delimited text format
– alignment section mandatory 11 columns
– Optional fields
– For details http://samtools.github.io/hts-specs/SAMv1.pdf
– Samflags http://picard.sourceforge.net/explain-flags.html
instrument_name:100:flowcellID:1:1:32575:625 0
chr17 72387574 255 141M *
0 0
GGAAGAGCTGGGACCAGGCCCAGCAATTACCTCACCATGGTTGGGTGCAACCAAG
TGGGGCAACTCTTTGGCCAGAAAGCAAAAGTCTTTTTAGCTTCAATGTAGGCCAT
TCTGGGTCCCAGACCCACAGCTTTGGACATT
Dgga^h`hedac`hcc`c^_f`ChggeDfC`e`_h_ddgd^_`be^bcg^acchC
`^^bCdagfbgf_`^dc^^cD`dbha^Dbc`fb^d`CdaD^``dDb_edffdDDh
_chceabh`heecd_h`gb_b^Ch`Dd_^c` AS:i:0 XN:i:
0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:
141 YT:Z:UU
1 Query template
2 Bitwise flag
3 Reference Name
4 1-based leftmost position
5 Mapping Quality
6 CIGAR string
7 Reference Name of mate
8 Position of mate
9 Observed template length
10 Sequence
11 Phred-scaled Quality

41. Remove duplicates
• Duplicate reads generated as artifact in the library
generation step results in false confidence in variants.
• samtools [rmdup], Picard tools [MarkDuplicates]
samtools view -bS SNP.test.trimmed.QC.fastq.sam
-o SNP.test.trimmed.QC.fastq.bam
samtools rmdup -s SNP.test.trimmed.QC.fastq.bam
SNP.test.trimmed.QC.fastq.rmdup.bam

42. Remove duplicates

43. Call Variants
• Identify regions in alignment where sequence differs
****A*******
***A*****
**T******
*****T**********
******T********
ACGTGCAGCATAGCATCGACATCGACATACGC
Reads
Reference Genome

44. Call Variants
samtools sort SNP.test.trimmed.QC.fastq.rmdup.bam
SNP.test.trimmed.QC.fastq.rmdup.sorted
samtools index SNP.test.trimmed.QC.fastq.rmdup.sorted.bam
samtools faidx ./genome/chr17.fa
samtools mpileup -f ./genome/chr17.fa
SNP.test.trimmed.QC.fastq.rmdup.sorted.bam
| java -jar ./tool/VarScan.v2.3.6.jar mpileup2snp --output-vcf –
strand-filter 0
samtools mpileup
-f faidx indexed reference sequence file
VarScan mpileup2snp or mpileup2indel
--min-coverage Minimum read depth at a position to make a call [8]
--min-reads2 Minimum supporting reads at a position to call variants [2]
--min-avg-qual Minimum base quality at a position to count a read [15]
--min-var-freq Minimum variant allele frequency threshold [0.01]
--strand-filter Ignore variants with >90% support on one strand [1]

45. Call Variants

46. Call Variants

47. VCF Variant Call Format file
• Header text marked with ##
• Column headings marked with #
• Mandatory columns
– CHROM Chromosome
– POS Position of variant start
– ID Unique variant ID
– REF Reference Allele
– ALT Alternate non-reference alleles (comma separated)
– QUAL Phred quality score
– FILTER Filtering information
– INFO User annotation

48. Visualize: IGV

50. So Many SNPS – So What?
• Get gene
• Functional Analysis to identify key candidates
Identify
Homologues
Locate
variants
Identify
Ontologies
Pathway and
interaction
analysis
Locate SNPs
on structure
Compare to
current data

51. • Functional Analysis to identify key candidates
A step by step example (don’t do this with lots of variants!)
• Get gene
• Modify bases as shown in VCF file.
• BLASTx to identify reading frame.
• Produce mRNA, and encoded peptide sequence fasta files (provided)
• Determine variant positions in mRNA & peptide sequences

52. Compare to
current data
• dbSNP
– SNP already known?
• Repositories such as Ensembl, UCSC
– In splice variant?
– In known regions, domain etc?
• Variant effect predictor (more later)

53. • Visualize the position in genome – Ensembl/UCSC
– Add custom track using GFF/bed file
• Coding/non-coding
– Synonymous / non-synonymous
– Codon usage
• Locate in relation to known domains
– Pfam
– SMART
• Repeat regions
Locate
variants

54. • For single genes
– Search for available information PubMed, interpro
etc
• For multiple genes
– BLAST2GO
– DAVID
Identify
Ontologies

55. • Pathway analysis
– Understand the process of your gene. Does it make
biological sense?
– IPA
– DAVID
– Webgestalt
• Interaction analysis
– BioGRID
– STRING
– PSICQUIC
Pathway and
interaction
analysis

56. • Structure prediction
– BLAST of PDB
– Predict structure
• PSIPRED – 2° structure
• ITASSER – 3° structure
• Phyre – 3° structure
– Locate in 3D
• Swiss PDB viewer
Locate SNPs
on structure

57. • Predict effect of SNPs
• Suspect
• VEP
Locate SNPs
on structure
Arg
Ser

58. Identify
Homologues
Emes R.D. Inferring function from homology. in Methods in Molecular Biology 453: Humana Press 2008.

59. Links and websites 1
• Many at http://emeslab.wordpress.com/useful-links/
• SRA: http://www.ncbi.nlm.nih.gov/sra
• SRA toolkit http://eutils.ncbi.nih.gov/Traces/sra/?view=software
• ENA: http://www.ebi.ac.uk/ena/
• Galaxy: https://usegalaxy.org/
• FASTQC: www.bioinformatics.babraham.ac.uk/projects/fastqc/
• Fastx_toolkit: http://hannonlab.cshl.edu/fastx_toolkit/
• Bowtie 1: http://bowtie-bio.sourceforge.net/index.shtml
• Bowtie 2: http://bowtie-bio.sourceforge.net/bowtie2/index.shtml
• BWA: http://bio-bwa.sourceforge.net/
• Stampy: http://www.well.ox.ac.uk/project-stampy
• SAMtools: http://samtools.sourceforge.net
• PicardTools: http://picard.sourceforge.net
• VarScan: http://varscan.sourceforge.net
• IGV: https://www.broadinstitute.org/software/igv/

60. Links and websites 2
• dbSNP: http://www.ncbi.nlm.nih.gov/SNP/
• Ensembl: http://www.ensembl.org/index.html
• UCSC Genome Browser: https://genome.ucsc.edu/
• Pfam: http://pfam.xfam.org/
• SMART: http://smart.embl-heidelberg.de/
• GO: http://www.geneontology.org/
• BLAST2GO: http://www.blast2go.com/b2ghome
• IPA: http://www.ingenuity.com/products/login
• DAVID: http://david.abcc.ncifcrf.gov/
• Webgestalt: http://bioinfo.vanderbilt.edu/webgestalt/
• BioGRID: http://thebiogrid.org/
• PSICQUIC: http://www.ebi.ac.uk/Tools/webservices/psicquic/view/main.xhtml
• ITASSER: http://zhanglab.ccmb.med.umich.edu/I-TASSER/
• Phyre: http://www.sbg.bio.ic.ac.uk/phyre2/
• SuSPect: http://www.sbg.bio.ic.ac.uk/~suspect/
• PSIPRED: http://bioinf.cs.ucl.ac.uk/psipred/
• SwissPDB: http://spdbv.vital-it.ch/

61. Richard Emes
Associate Professor & Reader in Bioinformatics.
School of Veterinary Medicine and Science
Director Advanced Data Analysis Centre
richard.emes@nottingham.ac.uk
www.nottingham.ac.uk/adac
@rdemes
@ADAC_UoN
61