Bayesian Logical Data Analysis for the Physical Sciences. A Comparative Approach with Mathematica Support

488 Pages·2005·4.984 MB·English

by  P.C. Gregory

This page intentionally left blank
Bayesian Logical Data Analysis for the Physical
Sciences
A Comparative Approach with MathematicaTM Support
Increasingly,researchersinmanybranchesofsciencearecomingintocontactwith
BayesianstatisticsorBayesianprobabilitytheory.Byencompassingbothinductive
anddeductivelogic,Bayesiananalysiscanimprovemodelparameterestimatesby
manyordersofmagnitude.Itprovidesasimpleandunifiedapproachtoalldata
analysisproblems,allowingtheexperimentertoassignprobabilitiestocompeting
hypothesesofinterest,onthebasisofthecurrentstateofknowledge.
This book provides a clear exposition of the underlying concepts with large
numbersofworkedexamplesandproblemsets.Thebookalsodiscussesnumerical
techniquesforimplementingtheBayesiancalculations,includinganintroduction
toMarkovchainMonteCarlointegrationandlinearandnonlinearleast-squares
analysis seen from a Bayesian perspective. In addition, background material is
providedinappendicesandsupportingMathematicanotebooksareavailablefrom
www.cambridge.org/052184150X, providing an easy learning route for upper-
undergraduate, graduate students, or any serious researcher in physical sciences
orengineering.
PHIL GREGORY is Professor Emeritus at the Department of Physics and
AstronomyattheUniversityofBritishColumbia.

Bayesian Logical Data Analysis for the Physical
Sciences
A Comparative Approach with MathematicaTM Support
P.C.Gregory
DepartmentofPhysicsandAstronomy,UniversityofBritishColumbia
  
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
The Edinburgh Building, Cambridge  , UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridg e.org /9780521841504
© Cambridge University Press 2005
This book is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format 2005
- ---- eBook (NetLibrary)
- --- eBook (NetLibrary)
- ---- hardback
- --- hardback
Cambridge University Press has no responsibility for the persistence or accuracy of
s for external or third-party internet websites referred to in this book, and does not
guaranteethatanycontentonsuchwebsitesis,orwillremain,accurateorappropriate.
Disclaimer of warranty
We make no warranties, express or implied, that the programs contained in this volume
are free of error, or are consistent with any particular standard of merchantability, or
that they will meet your requirements for any particular application. They should not
be relied on for solving a problem whose incorrect solution could result in injury to a
personorlossofproperty.Ifyoudousetheprogramsinsuchamanner,itisatyourown
risk. The authors and publisher disclaim all liability for direct or consequential
damages resulting from your use of the programs.
Contents
Preface pagexiii
Softwaresupportxv
Acknowledgements xvii
1Roleofprobabilitytheoryinscience1
1.1Scientificinference1
1.2Inferencerequiresaprobabilitytheory2
1.2.1Thetworulesformanipulatingprobabilities4
1.3UsualformofBayes’theorem5
1.3.1Discretehypothesisspace5
1.3.2Continuoushypothesisspace6
1.3.3Bayes’theorem–modelofthelearningprocess7
1.3.4ExampleoftheuseofBayes’theorem8
1.4Probabilityandfrequency10
1.4.1Example:incorporatingfrequencyinformation11
1.5Marginalization12
1.6Thetwobasicproblemsinstatisticalinference15
1.7AdvantagesoftheBayesianapproach16
1.8Problems17
2Probabilitytheoryasextendedlogic21
2.1Overview21
2.2Fundamentalsoflogic21
2.2.1Logicalpropositions21
2.2.2Compoundpropositions22
2.2.3TruthtablesandBooleanalgebra22
2.2.4Deductiveinference24
2.2.5Inductiveorplausibleinference25
2.3Briefhistory25
2.4Anadequatesetofoperations26
2.4.1Examinationofalogicfunction27
2.5Operationsforplausibleinference29
v
vi Contents
2.5.1ThedesiderataofBayesianprobabilitytheory30
2.5.2Developmentoftheproductrule30
2.5.3Developmentofsumrule34
2.5.4Qualitativepropertiesofproduct
andsumrules36
2.6Uniquenessoftheproductandsumrules37
2.7Summary39
2.8Problems39
3Thehow-toofBayesianinference41
3.1Overview41
3.2Basics41
3.3Parameterestimation43
3.4Nuisanceparameters45
3.5ModelcomparisonandOccam’srazor45
3.6Samplespectrallineproblem50
3.6.1Backgroundinformation50
3.7Oddsratio52
3.7.1Choiceofprior pðTjM ;IÞ 53
1
3.7.2Calculationof pðDjM ;T;IÞ 55
1
3.7.3Calculationof pðDjM ;IÞ 58
2
3.7.4Odds,uniformprior58
3.7.5Odds,Jeffreysprior58
3.8Parameterestimationproblem59
3.8.1Sensitivityofoddsto T 59
max
3.9Lessons61
3.10Ignorancepriors63
3.11Systematicerrors65
3.11.1Systematicerrorexample66
3.12Problems69
4Assigningprobabilities72
4.1Introduction72
4.2Binomialdistribution72
4.2.1Bernoulli’slawoflargenumbers75
4.2.2Thegambler’scoinproblem75
4.2.3Bayesiananalysisofanopinionpoll77
4.3Multinomialdistribution79
4.4Canyoureallyanswerthatquestion?80
4.5Logicalversuscausalconnections82
4.6Exchangeabledistributions83
4.7Poissondistribution85
Contents vii
4.7.1Bayesianandfrequentistcomparison87
4.8Constructinglikelihoodfunctions89
4.8.1Deterministicmodel90
4.8.2Probabilisticmodel91
4.9Summary93
4.10Problems94
5Frequentiststatisticalinference96
5.1Overview96
5.2Theconceptofarandomvariable96
5.3Samplingtheory97
5.4Probabilitydistributions98
5.5Descriptivepropertiesofdistributions100
5.5.1Relativelineshapemeasuresfordistributions101
5.5.2Standardrandomvariable102
5.5.3Othermeasuresofcentraltendencyanddispersion103
5.5.4Medianbaselinesubtraction104
5.6Momentgeneratingfunctions105
5.7Somediscreteprobabilitydistributions107
5.7.1Binomialdistribution107
5.7.2ThePoissondistribution109
5.7.3Negativebinomialdistribution112
5.8Continuousprobabilitydistributions113
5.8.1Normaldistribution113
5.8.2Uniformdistribution116
5.8.3Gammadistribution116
5.8.4Betadistribution117
5.8.5Negativeexponentialdistribution118
5.9CentralLimitTheorem119
5.10BayesiandemonstrationoftheCentralLimitTheorem120
5.11Distributionofthesamplemean124
5.11.1Signalaveragingexample125
5.12Transformationofarandomvariable125
5.13Randomandpseudo-randomnumbers127
5.13.1Pseudo-randomnumbergenerators131
5.13.2Testsforrandomness132
5.14Summary136
5.15Problems137
6Whatisastatistic?139
6.1Introduction139
6.2The (cid:1)2 distribution141
viii Contents
6.3Samplevariance S2 143
6.4TheStudent’s tdistribution147
6.5 Fdistribution(F-test)150
6.6Confidenceintervals152
6.6.1Variance (cid:2)2 known152
6.6.2Confidenceintervalsfor (cid:3),unknownvariance156
6.6.3Confidenceintervals:differenceoftwomeans158
6.6.4Confidenceintervalsfor (cid:2)2 159
6.6.5Confidenceintervals:ratiooftwovariances159
6.7Summary160
6.8Problems161
7Frequentisthypothesistesting162
7.1Overview162
7.2Basicidea162
7.2.1Hypothesistestingwiththe (cid:1)2 statistic163
7.2.2Hypothesistestonthedifferenceoftwomeans167
7.2.3One-sidedandtwo-sidedhypothesistests170
7.3Aretwodistributionsthesame?172
7.3.1Pearson (cid:1)2 goodness-of-fittest173
7.3.2Comparisonoftwo-binneddatasets177
7.4Problemwithfrequentisthypothesistesting177
7.4.1Bayesianresolutiontooptionalstoppingproblem179
7.5Problems181
8Maximumentropyprobabilities184
8.1Overview184
8.2Themaximumentropyprinciple185
8.3Shannon’stheorem186
8.4AlternativejustificationofMaxEnt187
8.5GeneralizingMaxEnt190
8.5.1Incorporatingaprior190
8.5.2Continuousprobabilitydistributions191
8.6HowtoapplytheMaxEntprinciple191
8.6.1Lagrangemultipliersofvariational
calculus191
8.7MaxEntdistributions192
8.7.1Generalproperties192
8.7.2Uniformdistribution194
8.7.3Exponentialdistribution195
8.7.4NormalandtruncatedGaussiandistributions197
8.7.5MultivariateGaussiandistribution202