Multi-versioned Data Storage and Iterative Processing in a Parallel Array Database Engine

138 Pages·2014·12.53 MB·English

(cid:13)c Copyright2014
EmadSoroush

Multi-versioned Data Storage and Iterative Processing in a Parallel Array
Database Engine
EmadSoroush
Adissertation
submittedinpartialfulfillmentofthe
requirementsforthedegreeof
DoctorofPhilosophy
UniversityofWashington
2014
ReadingCommittee:
MagdalenaBalazinska,Chair
DanSuciu
BillHowe
ProgramAuthorizedtoOfferDegree:
ComputerScienceandEngineering
UniversityofWashington
Abstract
Multi-versionedDataStorageandIterativeProcessinginaParallelArrayDatabaseEngine
EmadSoroush
ChairoftheSupervisoryCommittee:
AssociateProfessorMagdalenaBalazinska
ComputerScienceandEngineering
Scientists today are able to generate data at an unprecedented scale and rate. For example
the Sloan Digital Sky Survey (SDSS) generates 200GB of data containing millions of objects on
each night on its routine operation. The large hadron collider is producing even more data today
whichisapproximately30PBannually. TheLargeSynopticSurveyTelescope(LSST)alsowillbe
producing approximately 30TB of data per night in a few years. Also, in many fields of science,
multidimensional arrays rather than flat tables are standard data types because data values are
associated with coordinates in space and time. For example, images in astronomy are 2D arrays
ofpixelintensities. Climateandoceanmodelsusearraysormeshestodescribe3Dregionsofthe
atmosphere and oceans. As a result, scientists need powerful tools to help them manage massive
arrays.
Thisthesisfocusesonvariouschallengesinbuildingparallelarraydatamanagementsystems
thatfacilitatemassive-scaledataanalyticsoverarrays.
Thefirstchallengewithbuildinganarraydataprocessingsystemissimplyhowtostorearrayson
disk. Thekeyquestionishowtopartitionarraysintosmallerfragmentscalledchunksthatformthe
unitofIO,processing,anddatadistributionacrossmachinesinacluster. Weexplorethisquestion
in ArrayStore, a new read-only storage manager for parallel array processing. In ArrayStore, we
studytheimpactofdifferentchunkingstrategiesonqueryprocessingperformanceforawiderange
ofoperations,includingbinaryoperatorsanduser-definedfunctions. ArrayStorealsoproposestwo
newtechniquesthatenableoperatorstoaccessdatafromadjacentarrayfragmentsduringparallel
processing.
Thesecondchallengethatweexploreinbuildingarraysystemsistheabilitytocreate,archive,
andexploredifferentversionsofthearraydata. WeaddressthisquestioninTimeArr,anewappend-
onlystoragemanagerforanarraydatabase. Itskeycontributionistoefficientlystoreandretrieve
versions of an entire array or some sub-array. To achieve high performance, TimeArr relies on
several techniques including virtual tiles, bitmask compression of changes, variable-length delta
representations,andskiplinks.
Thethirdchallengethatwetackleinbuildingparallelarrayenginesishowtoprovideefficient
iterativecomputationonmulti-dimensionalscientificarrays. Wepresentthedesign,implementation,
andevaluationofArrayLoop,anextensionofSciDBwithnativesupportforarrayiterations. Inthe
contextofArrayLoop,wedevelopamodelforiterativeprocessinginaparallelarrayengine. Wethen
presentthreeoptimizationstoimprovetheperformanceofthesetypesofcomputations: incremental
processing,mini-iterationoverlapprocessing,andmulti-resolutionprocessing.
Finally,asmotivationforourworkandalsotohelppushourtechnologybackintothehandsof
scienceusers,wehavebuilttheAscotDBsystem. AscotDBisanew,extensibledataanalysissystem
fortheinteractiveanalysisofdatafromastronomicalsurveys. AscotDBprovidesacompellingand
powerfulenvironmentfortheexploration,analysis,visualization,andsharingoflargearraydatasets.
TABLEOFCONTENTS
Page
ListofFigures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
ListofTables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Chapter1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 ScientificApplicationRequirements . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 SciDB:Open-sourceDBMSwithInherentSupportforArrayProcessing . . . . . . 6
1.3 ThesisOutlineandContributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Chapter2: ArrayStore: StorageManagerforComplex,ParallelArrayProcessing . . . 12
2.1 Requirements,Challenges,andContributions . . . . . . . . . . . . . . . . . . . . 12
2.2 ProblemStatement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 ArrayStoreStorageManager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 ArrayStoreAccessMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Chapter3: TimeArr: StorageManagerwithEfficientSupportforVersioning . . . . . . 40
3.1 Challenges,andContributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.2 TimeArrOverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.3 VersionStorageandRetrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.4 ApproximateQueries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Chapter4: ArrayLoop: SciDBwithSupportforIterativeComputation . . . . . . . . . 70
i
4.1 Requirements,Challenges,andContributions . . . . . . . . . . . . . . . . . . . . 70
4.2 MotivatingApplications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.3 IterativeArray-ProcessingModel . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.4 IncrementalIterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.5 IterativeOverlapProcessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.6 Multi-ResolutionOptimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.7 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Chapter5: AscotDB:DataAnalysisandExplorationPlatformforAstronomers . . . . 99
5.1 AscotDBOverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.2 AscotDBFront-endandUserInteractions . . . . . . . . . . . . . . . . . . . . . . 101
5.3 AscotDBMiddlewarePythonSupportandSciDBbackend . . . . . . . . . . . . . 103
5.4 AscotDBLessonsLearnedandFutureDirections . . . . . . . . . . . . . . . . . . 105
Chapter6: RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6.1 RelatedWorkonArrayProcessingandSystems . . . . . . . . . . . . . . . . . . . 107
6.2 RelatedWorkonArrayStorage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.3 RelatedWorkonArrayDataVersioning . . . . . . . . . . . . . . . . . . . . . . . 111
6.4 RelatedWorkonIterativeProcessing . . . . . . . . . . . . . . . . . . . . . . . . . 112
Chapter7: EvaluationDatasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Chapter8: ConclusionandFutureDirections . . . . . . . . . . . . . . . . . . . . . . 116
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
ii
LISTOFFIGURES
FigureNumber Page
1.1 SciDBarraydatamodelexample. . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Amotivatingexamplefromastronomydomain. . . . . . . . . . . . . . . . . . . . 11
2.1 Chunkrepresentationsinarrays. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Differentchunkingstrategies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Per-chunkdatadistributiondifferencesinREGorIREGstrategies. . . . . . . . . . 15
2.4 Multi-layeroverlapexampleincanopyclustering. . . . . . . . . . . . . . . . . . . 25
2.5 Arraydicingqueryon3Dand6Ddatasets. . . . . . . . . . . . . . . . . . . . . . . 31
2.6 Joinqueryon3Dand6Darrays. . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.7 ParallelselectionwithdifferentpartitioningstrategiesonREGchunks. . . . . . . . 34
2.8 ParallelsubsamplewithREGorIREGchunksdistributedusingrangepartitioning. 35
2.9 CanopyClusteringalgorithmwithorwithoutoverlaponthe3Ddataset. . . . . . . 37
2.10 Volume-densityapplicationon3Ddatasetwithandwithoutoverlap. . . . . . . . . 38
3.1 Illustrationofachainofbackwarddeltaversionsfora3x3array. . . . . . . . . . . 41
3.2 The4x4x4arrayA1isdividedintoeight2x2x2chunks. . . . . . . . . . . . . . . . 43
3.3 Representationofasinglearraychunkwithmultipleversions. . . . . . . . . . . . 48
3.4 TileDeltaLayout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.5 InternalstructureofaTileDeltaVinTimeArr. . . . . . . . . . . . . . . . . . . . . 49
3.6 Skiplinks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.7 Validv.s. Invalidstatesofskiplinks. . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.8 CumDiffandLocDiffexamples. . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.9 Timetocreate100versionsofatwo-dimensionalarraywithnormallydistributed
updates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.10 TimetofetcheachversionintheGFSdataset. . . . . . . . . . . . . . . . . . . . . 62
3.11 Timetofetchtheoriginalversionwheretheregionwindowchangesfromonetileto
thewholechunk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
iii
3.12 Timetofetcheachversioninasingle-chunkarraywith60versions. . . . . . . . . 65
3.13 CumulativequeryruntimeofworkloadsQ-NORMandQ-UNIFORM. . . . . . . . . . . 66
3.14 Timetofetcheachversionfrom1to50onatwo-dimensionalarraywithuniformly
distributedupdates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.15 Approximatehistoryquery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.1 Illustrativecomparisonofonesingleimageanditscorrespondingco-added image. 74
4.2 terativearrayAanditsstateateachiterationfor“iterativesourcedetection”application. 75
4.3 IterativearrayAanditsstateafterthreeminorsteps. . . . . . . . . . . . . . . . . 78
4.4 Twoexamplesofwindowassignmentfunctions. . . . . . . . . . . . . . . . . . . . 80
4.5 CumulativeruntimeoftheSigmaClipapplicationonlsstimageswithandwithout
incrementaliterativeprocessingoptimization. . . . . . . . . . . . . . . . . . . . . 86
4.6 Snapshotsfromthefirst3iterationsoftheSigmaClipapplicationwithincremental
optimizationontheLSSTdataset. . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.7 merge()operatorexampleinSigmaClipapplication. . . . . . . . . . . . . . . 87
4.8 Theschematicpictureforminiiterationoptimization. . . . . . . . . . . . . . . . . 90
4.9 Controlflowdiagramformini-iteration-basedprocessinginArrayLoop. . . . . . . 91
4.10 Illustrationofthemulti-resolutionoptimizationfortheSourceDetectapplication. 93
4.11 SigmaClipapplication: incrementalstrategyv.s. non-incremental. Constantk = 3
inallthealgorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.12 SourceDetectapplication: Iterativeoverlapprocessingwithmini-iterationopti-
mization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.13 SourceDetectapplication: Multi-resolutionOptimization. . . . . . . . . . . . 98
5.1 AscotDBarchitecture: SciDBasback-end,pythonmiddleware,AscotandIPython
asfront-ends. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.2 TwomodesofinteractionwithAscotDB:visualinterfaceandIPythoninterface. . . 102
5.3 “Analysis”and“Exploration”phases . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.4 The user interacts with AscotDB by alternating between exploration and analysis
phases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
iv
LISTOFTABLES
TableNumber Page
2.1 AccessMethodAPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.2 Namingconventionusedinexperiments. . . . . . . . . . . . . . . . . . . . . . . . 29
2.3 Paralleljoin(shufflingphase)fordifferenttypesofchunkpartitioningstrategies. . 36
2.4 “Localjoinphase”withregularchunkspartitionedacross8nodes. . . . . . . . . . 36
3.1 TimeArrVersionedArrayAPI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2 GFSdataset: Skiplinksoverheadatversioninsertiontime. . . . . . . . . . . . . . 66
3.3 Averagetimetocreateoneversionafterappending50versionsonatwo-dimensionalarray
withuniformlydistributedupdates. . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
v