Recent Advances in Algorithmic Differentiation

355 Pages·2012·6.969 MB·English

by  Bruce Christianson (auth.), Shaun Forth, Paul Hovland, Eric Phipps, Jean Utke, Andrea Walther (eds.)

Lecture Notes
in Computational Science 87
and Engineering
Editors:
TimothyJ. Barth
MichaelGriebel
DavidE. Keyes
RistoM. Nieminen
DirkRoose
TamarSchlick
Forfurthervolumes:
http://www.springer.com/series/3527
•
Shaun Forth Paul Hovland Eric Phipps
(cid:2) (cid:2)
Jean Utke Andrea Walther
(cid:2)
Editors
Recent Advances
in Algorithmic
Differentiation
123
Editors
ShaunForth JeanUtke
AppliedMathematics MathematicsandComputerScience
andScientificComputing Division
CranfieldUniversity ArgonneNationalLaboratory
Shrivenham Argonne
Swindon Illinois
UnitedKingdom USA
PaulHovland AndreaWalther
MathematicsandComputerScience DepartmentofMathematics
Division UniversityofPaderborn
ArgonneNationalLaboratory Paderborn
Argonne Germany
Illinois
USA
EricPhipps
SandiaNationalLaboratory
Albuquerque
NewMexico
USA
ISSN1439-7358
ISBN978-3-642-30022-6 ISBN978-3-642-30023-3(eBook)
DOI10.1007/978-3-642-30023-3
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2012942187
MathematicsSubjectClassification(2010):65D25,90C30,90C31,90C56,65F50,68N20,41A58,
65Y20
(cid:2)c Springer-VerlagBerlinHeidelberg2012
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
The Sixth International Conference on Automatic Differentiation (AD2012) held
July 23–27, 2012, in Fort Collins, Colorado (USA), continued this quadrennial
conferenceseries.Whilethefundamentalideaofdifferentiatingnumericalprograms
is easy to explain, the practical implementation of this idea for many nontrivial
numerical computations is not. Our community has long been aware of the
discrepancybetweentheaspirationofanautomaticprocesssuggestedbythename
automatic differentiation and the reality of its practical use, which often requires
substantialeffortfromthe user. New algorithmsandmethodsimplementedin dif-
ferentiationtoolsimprovetheirusabilityandreducetheneedforuserintervention.
On the other hand, the demands to compute derivatives for numerical models
on parallel hardware, using a wide variety of libraries and having components
implementedindifferentprogramminglanguages,posenewchallenges,particularly
for the efficiency of the derivativecomputation.These challenges, as well as new
applications,havebeendrivingresearchforthepastfouryearsandwillcontinueto
doso.Despiteretainingautomaticdifferentiationintheconferencename,theeditors
purposely switched to algorithmic differentiation (AD) in the proceedings title.
Thus,theconferenceproceedingsfollowsomewhatbelatedlythemoreappropriate
namingchosenbyAndreasGriewankforthefirsteditionofhisseminalmonograph
coveringoursubjectarea.ThisnamebetterreflectstherealityofADusageandthe
researchresultspresentedinthepaperscollectedhere.
The31contributedpaperscovertheapplicationofADtomanyareasofscience
and engineering as well as aspects of AD theory and its implementation in tools.
Forallpapersthereferees,selectedfromtheprogramcommitteeandthewiderAD
community, as well as the editors have emphasized accessibility of the presented
ideasalsotonon-ADexperts.
IntheADtoolsarenanewimplementationsareintroducedcovering,forexample,
Java and graphical modeling environments, or join the set of existing tools for
Fortran.NewdevelopmentsinADalgorithmstarget:efficientderivativesformatrix-
operation,detection and exploitationof sparsity, partialseparability,the treatment
ofnonsmoothfunctions,andotherhigh-levelmathematicalaspectsofthenumerical
computationstobedifferentiated.
v
vi Preface
ApplicationsstemfromtheEarthsciences,nuclearengineering,fluiddynamics,
and chemistry, to name just a few. In many cases the applications in a given area
of science or engineering share characteristics that require specific approaches
to enable AD capabilities or provide an opportunity for efficiency gains in the
derivative computation. The description of these characteristics and of the tech-
niques for successfully using AD should make the proceedings a valuable source
ofinformationforusersofADtools.
Theimageonthebookcovershowsthe high-harmonicemissionspectrumofa
semiconductorquantumdotfordifferentexcitationconditions.Tofavorspecificfre-
quenciesonehastofindanappropriateinputpulsewithinalargeparameterspace.
Thiswasaccomplishedbycombiningagradient-basedoptimizationalgorithmwith
AD.ThedataplotswereprovidedbyMatthiasReichelt.
Algorithmicdifferentiationdrawsonmanyaspectsof appliedmathematicsand
computer science and ultimately is useful only when users in the science and
engineering communities become aware of its capabilities. Furthering collabora-
tions outside the core AD community, the AD2012 program committee invited
leading experts from diverse disciplines as keynote speakers. We are grateful to
LorenzBiegler(CarnegieMellonUniversity,USA),LucaCapriotti(CreditSuisse,
USA),DonEstep(ColoradoStateUniversity,USA),AndreasGriewank(Humboldt
University,Germany),MaryHall(UniversityofUtah,USA),BarbaraKaltenbacher
(University of Klagenfurt, Austria), Markus Pu¨schel (ETH Zurich, Switzerland),
andBertSpeelpenning(MathPartners,USA)foracceptingtheinvitations.
We want to thank SIAM and the NNSA and ASCR programs of the US
DepartmentofEnergyfortheirfinancialsupportofAD2012.
Albuquerque,Chicago ShaunForth
Paderborn,Shrivenham PaulHovland
April2012 EricPhipps
JeanUtke
AndreaWalther
Preface vii
ProgramCommitteeAD2012
BradBell,UniversityofWashington(USA)
MartinBerz,MichiganStateUniversity(USA)
ChristianBischof,TUDarmstadt(Germany)
MartinBu¨cker,RWTHAachen(Germany)
BruceChristianson,UniversityofHertfordshire(UK)
DavidGay,AMPLOptimizationInc.(USA)
AndreasGriewank,HumboldtUniversityBerlin(Germany)
LaurentHascoe¨t,INRIA(France)
PatrickHeimbach,MassachusettsInstituteofTechnology(USA)
KoichiKubota,ChuoUniversity(Japan)
KyokoMakino,MichiganStateUniversity(USA)
Jens-DominikMu¨ller,QueenMaryUniversityofLondon(UK)
UweNaumann,RWTHAachen(Germany)
BoyanaNorris,ArgonneNationalLaboratory(USA)
TrondSteihaug,UniversityofBergen(Norway)
•
Contents
ALeibnizNotationforAutomaticDifferentiation............................ 1
BruceChristianson
SparseJacobianConstructionforMappedGridVisco-Resistive
Magnetohydrodynamics......................................................... 11
DanielR.ReynoldsandRaviSamtaney
Combining Automatic Differentiation Methods for
High-DimensionalNonlinearModels.......................................... 23
JamesA.Reed,JeanUtke,andHanyS.Abdel-Khalik
ApplicationofAutomaticDifferentiationtoanIncompressible
URANSSolver.................................................................... 35
EmreO¨zkaya,AnilNemili,andNicolasR.Gauger
ApplyingAutomaticDifferentiationtotheCommunityLandModel...... 47
Azamat Mametjanov,Boyana Norris, Xiaoyan Zeng, Beth
Drewniak,JeanUtke,MihaiAnitescu,andPaulHovland
UsingAutomaticDifferentiationtoStudytheSensitivityofa
CropModel....................................................................... 59
ClaireLauvernet,LaurentHascoe¨t,Franc¸ois-XavierLeDimet,and
Fre´de´ricBaret
EfficientAutomaticDifferentiationofMatrixFunctions.................... 71
PederA.Olsen,StevenJ.Rennie,andVaibhavaGoel
Native Handling of Message-Passing Communicationin
Data-FlowAnalysis .............................................................. 83
Vale´riePascualandLaurentHascoe¨t
IncreasingMemory Localityby ExecutingSeveralModel
InstancesSimultaneously........................................................ 93
RalfGieringandMichaelVoßbeck
ix
x Contents
AdjointModeComputationofSubgradientsforMcCormick
Relaxations........................................................................ 103
MarkusBeckers,ViktorMosenkis,andUweNaumann
EvaluatinganElementoftheClarkeGeneralizedJacobianof
aPiecewiseDifferentiableFunction............................................ 115
KamilA.KhanandPaulI.Barton
The ImpactofDynamicDataReshapingonAdjointCode
GenerationforWeakly-TypedLanguagesSuchasMatlab.................. 127
JohannesWillkomm,ChristianH.Bischof,andH.MartinBu¨cker
OntheEfficientComputationofSparsityPatternsforHessians........... 139
AndreaWalther
ExploitingSparsityinAutomaticDifferentiationonMulticore
Architectures ..................................................................... 151
BenjaminLetschert,KshitijKulshreshtha,AndreaWalther,Duc
Nguyen,AssefawGebremedhin,andAlexPothen
AutomaticDifferentiationThroughtheUseofHyper-Dual
NumbersforSecondDerivatives ............................................... 163
JeffreyA.FikeandJuanJ.Alonso
ConnectionsBetweenPowerSeriesMethodsandAutomatic
Differentiation.................................................................... 175
DavidC.Carothers,StephenK.Lucas,G.EdgarParker,JosephD.
Rudmin,JamesS.Sochacki,RogerJ.Thelwell,AnthonyTongen,
andPaulG.Warne
HierarchicalAlgorithmicDifferentiationACaseStudy..................... 187
JohannesLotz,UweNaumann,andJo¨rnUngermann
StoringVersusRecomputationonMultipleDAGs ........................... 197
HeatherCole-Mullen,AndrewLyons,andJeanUtke
UsingDirectedEdgeSeparatorstoIncreaseEfficiencyinthe
DeterminationofJacobianMatricesviaAutomaticDifferentiation ....... 209
ThomasF.Coleman,XinXiong,andWeiXu
AnIntegerProgrammingApproachtoOptimalDerivative
Accumulation..................................................................... 221
JieqiuChen,PaulHovland,ToddMunson,andJeanUtke
TheRelativeCostofFunctionandDerivativeEvaluationsin
theCUTErTestSet .............................................................. 233
TorstenBosseandAndreasGriewank
JavaAutomaticDifferentiationToolUsingVirtualOperator
Overloading....................................................................... 241
PhuongPham-QuangandBenoitDelinchant