Efficient Alignment Algorithms for DNA Sequencing Data

149 Pages·2017·2.15 MB·English

WWeesstteerrnn UUnniivveerrssiittyy
SScchhoollaarrsshhiipp@@WWeesstteerrnn
Electronic Thesis and Dissertation Repository
1-31-2018 11:00 AM
EEffifficciieenntt AAlliiggnnmmeenntt AAllggoorriitthhmmss ffoorr DDNNAA SSeeqquueenncciinngg DDaattaa
Nilesh Vinod Khiste, The University of Western Ontario
Supervisor: Lucian Ilie, The University of Western Ontario
: NA, The University of Western Ontario
: NA, The University of Western Ontario
A thesis submitted in partial fulfillment of the requirements for the Doctor of Philosophy degree
in Computer Science
© Nilesh Vinod Khiste 2018
Follow this and additional works at: https://ir.lib.uwo.ca/etd
Part of the Bioinformatics Commons, Computational Biology Commons, Genomics Commons, and
the Other Computer Engineering Commons
RReeccoommmmeennddeedd CCiittaattiioonn
Khiste, Nilesh Vinod, "Efficient Alignment Algorithms for DNA Sequencing Data" (2018). Electronic Thesis
and Dissertation Repository. 5192.
https://ir.lib.uwo.ca/etd/5192
This Dissertation/Thesis is brought to you for free and open access by Scholarship@Western. It has been accepted
for inclusion in Electronic Thesis and Dissertation Repository by an authorized administrator of
Scholarship@Western. For more information, please contact [email protected].
Abstract
The DNA Next Generation Sequencing (NGS) technologies produce data at a low cost, en-
abling their application to many ambitious fields such as cancer research, disease control, per-
sonalized medicine etc. However, even after a decade of research, the modern aligners and
assemblers are far from providing efficient and error free genome alignments and assemblies
respectively. This is due to the inherent nature of the genome alignment and assembly prob-
lem, which involves many complexities. Many algorithms to address this problem have been
proposedovertheyears,buttherestillisahugescopeforimprovementinthisresearchspace.
Many new genome alignment algorithms are proposed over time and one of the key differ-
entiatorsamongthesealgorithmsistheefficiencyofthegenomealignmentprocess. Ipresenta
newalgorithmforefficientlyfindingMaximalExactMatches(MEMs)betweentwogenomes:
E-MEM (Efficient computation of maximal exact matches for very large genomes). Comput-
ing MEMs is one of the most time consuming step during the alignment process. E-MEM can
be used to find MEMs which are used as seeds in a genome aligner to increase its efficiency.
The E-MEM program is the most efficient algorithm as of today for computing MEMs, and it
surpassesallcompetitorsbylargemargins.
There are many genome assembly algorithms available for use, but none produces perfect
genomeassemblies. Itisimportantthatassembliesproducedbythesealgorithmsareevaluated
accurately and efficiently.This is necessary to make the right choice of the genome assembler
to be used for all the downstream research and analysis. A fast genome assembly evalua-
tor is a key factor when a new genome assembler is developed, to quickly evaluate the out-
comeofthealgorithm. IpresentafastandefficientgenomeassemblyevaluatorcalledLASER
(LargegenomeASsemblyEvaluatoR),whichisbasedonaleadinggenomeassemblyevaluator
QUAST,butsignificantlymoreefficientbothintermsofmemoryandruntime.
TheNGStechnologieslimitthepotentialofgenomeassemblyalgorithmsbecauseofshort
read lengths and nonuniform coverage. Recently, third generation sequencing technologies
have been proposed which promise very long reads and a uniform coverage. However, this
i
technology comes with its own drawback of high error rate of 10 - 15% consisting mostly of
indels. Thelongreadsequencingdataareusefulonlyaftererrorcorrectionobtainedusingself
readalignment(orreadoverlapping)techniques. Iproposeanewselfreadalignmentalgorithm
for Pacific Biosciences sequencing data: HISEA (Hierarchical SEed Aligner), which has very
high sensitivity and precision as compared to other state-of-the-art aligners. HISEA is also
integrated into Canu assembly pipeline. Canu+HISEA produces better assemblies than Canu
withitsdefaultalignerMHAP,atamuchlowercoverage.
Keywords: Bioinformatics, De novo genome assembly, E-MEM, Genome assembly eval-
uation,HISEA,HiSeq,Illumina,LASER,Longreadalignment,Maximalexactmatches,Next-
generationsequencing,PacificBiosciences,Sequencealignment
ii
It has been a very good experience coming back to academia after a long stint in industry.
However, it was a very tough decision to leave behind a set career path. I could not have done
itwithoutthesupportofmyfamily. Idedicatethisworktothem.
iii
Acknowlegements
I would like to express my sincere gratitude to everyone involved in the completion of
this dissertation. Most importantly, I would like to thank my supervisor Dr. Lucian Ilie for
introducingmetotheamazingresearchfieldofBioinformaticsandstringalgorithms. Hiscon-
tinuous encouragement helped me to explore various interesting problems in Bioinformatics
and use my research to propose efficient solutions for some of them. I really appreciate his
extraordinarypatienceinreviewingmymanuscripts,researchproposals,presentationsandthis
dissertation. His critical feedback and suggestions were always helpful in enhancing the qual-
ity of these documents. I am very grateful to my supervisor for the valuable time he spent
discussingmyideasandguidingme.
IamthankfultotheDepartmentofComputerScienceattheUniversityofWesternOntario
forawardingmewiththeWesternGraduateResearchScholarship(WGRS).Iamalsograteful
to the Ministry of Training, Colleges and Universities (Ontario) for awarding me the Ontario
GraduateScholarship(OGS)fortwoconsecutiveyears.
I am also thankful to my examiners and members of my supervisory committee Dr. Ming
Li, Dr. Kathleen Hill, Dr. Roberto Solis-Oba, Dr. Anwar Haque and Dr. Kaizhong Zhang. I
am thankful to my colleagues Dr. Mike Molnar and Yiwei Li for all the technical discussions
around various topics. I would like to thank my friends and family members for their constant
support and encouragement which kept me going and led to the completion of this research
work,whichisoneofthebiggestaccomplishmentsofmylife.
iv
Contents
Abstract i
Dedication iii
Acknowlegements iv
ListofFigures x
ListofTables xiii
ListofAppendices xv
1 Introduction 1
1.1 DNAsequencing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.1 Sangermethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.2 Nextgenerationsequencing . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.3 Thirdgenerationsequencing . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 MaximalExactMatches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3 AssemblyEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.4 SequenceAlignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 MaximalExactMatches: E-MEM 14
2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.1 BasicNotionsandDefinitions . . . . . . . . . . . . . . . . . . . . . . 14
Suffixtreeandsuffixarray . . . . . . . . . . . . . . . . . . . . . . . . 15
v
FM-index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
LCPinterval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.2 MUMmer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.3 Vmatch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
ComputationofMEMsusingEnhancedSuffixArray . . . . . . . . . . 20
2.1.4 SparseMEM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
ComputationofMEMsusingSparseSuffixArray . . . . . . . . . . . . 22
ParallelizationtechniqueinsparseMEM . . . . . . . . . . . . . . . . . 23
2.1.5 EssaMEM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.1.6 BackwardMEM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
ComputingParentIntervals . . . . . . . . . . . . . . . . . . . . . . . . 26
ComputationofMEMsusingtheFM-index . . . . . . . . . . . . . . . 27
CompressedSuffixArrayimplementation . . . . . . . . . . . . . . . . 28
2.1.7 SlaMEM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
SampledLCPArray . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
SampledSmallerValues . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.1.8 Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2 E-MEMalgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.3 SequenceStorage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.4 Efficientk-merStorage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.5 HashTableandhashingfunction . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.6 Searchingquery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.7 HandlingredundantMEMmatches . . . . . . . . . . . . . . . . . . . . . . . . 37
2.8 Dealingwithambiguousbases(N) . . . . . . . . . . . . . . . . . . . . . . . . 38
2.9 Splitparameter-memoryreduction . . . . . . . . . . . . . . . . . . . . . . . 39
2.10 VerylargenumberofMEMs . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.11 Outputformats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
vi
2.12 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.12.1 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.12.2 HumanvsMouse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
MinimumMEMlength100 . . . . . . . . . . . . . . . . . . . . . . . . 42
MinimumMEMlength300 . . . . . . . . . . . . . . . . . . . . . . . . 45
2.12.3 HumanvsChimp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
MinimumMEMlength100 . . . . . . . . . . . . . . . . . . . . . . . . 47
MinimumMEMlength300 . . . . . . . . . . . . . . . . . . . . . . . . 49
2.12.4 TriticumaestivumvsTriticumdurum . . . . . . . . . . . . . . . . . . 51
MinimumMEMlength100 . . . . . . . . . . . . . . . . . . . . . . . . 51
MinimumMEMlength300 . . . . . . . . . . . . . . . . . . . . . . . . 52
2.13 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3 AssemblyEvaluation: LASER 55
3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.2 QUASTIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2.1 Contigsizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.2.2 Misassembliesandstructuralvariations . . . . . . . . . . . . . . . . . 59
3.2.3 Genomerepresentation . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.2.4 NAxandNGAx . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2.5 Visualizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Cumulativelength . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Nxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
NAxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
NGxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
NGAxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
GCcontentplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.3 LASERImprovements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
vii
3.3.1 E-MEMintegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.3.2 Coderemodeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.3.3 NUCmerchanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4 GenomeAlignment: HISEA 69
4.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.1.1 BLASR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.1.2 DALIGNER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.1.3 GraphMap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.1.4 MHAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.1.5 Minimap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.2 HISEAIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.3 HISEAalgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.3.1 Storingreadsandhashingthereferenceset . . . . . . . . . . . . . . . 78
4.3.2 Searchingthequeryset . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.3.3 Filteringandclustering . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.3.4 Computingandextendingalignments . . . . . . . . . . . . . . . . . . 83
4.4 Alignmentevaluationmethod . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.4.1 ComputeDynamicProgrammingAlignment . . . . . . . . . . . . . . . 87
4.4.2 Sensitivitycomputation . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.4.3 Specificitycomputation . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.4.4 Precisioncomputation . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.4.5 F scorecomputation . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
1
4.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.5.1 Alignmentresults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Standalonecomparison . . . . . . . . . . . . . . . . . . . . . . . . . . 92
viii
Sensitivity-adeepdive . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Sensitivityvsoverlapsize . . . . . . . . . . . . . . . . . . . . . . . . 97
MHAPsketchsizeandMinimapminimizers . . . . . . . . . . . . . . . 98
4.5.2 Assemblyresults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
SensitivityofHISEAandMHAP-assemblypipeline . . . . . . . . . . 101
4.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5 ConclusionsandFutureResearch 116
5.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.2 Futureresearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Bibliography 119
A E-MEMResultsForMEMComputation 126
A.1 HumanvsMouse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
A.1.1 MinimumMEMlength100 . . . . . . . . . . . . . . . . . . . . . . . . 127
A.1.2 MinimumMEMlength300 . . . . . . . . . . . . . . . . . . . . . . . . 128
A.2 HumanvsChimp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
A.2.1 MinimumMEMlength100 . . . . . . . . . . . . . . . . . . . . . . . . 129
A.2.2 MinimumMEMlength300 . . . . . . . . . . . . . . . . . . . . . . . . 130
A.3 TriticumaestivumvsTriticumdurum . . . . . . . . . . . . . . . . . . . . . . . 131
A.3.1 MinimumMEMlength100 . . . . . . . . . . . . . . . . . . . . . . . . 131
A.3.2 MinimumMEMlength300 . . . . . . . . . . . . . . . . . . . . . . . . 131
CurriculumVitae 132
ix