Methods of Multivariate Analysis

738 Pages·2002·3.69 MB·English

by  Alvin C. Rencher

Methods of Multivariate Analysis
Second Edition

Methods of Multivariate Analysis
Second Edition
ALVINC.RENCHER
BrighamYoungUniversity
AJOHNWILEY&SONS,INC.PUBLICATION
∞
Thisbookisprintedonacid-freepaper.
Copyright(cid:1)c 2002byJohnWiley&Sons,Inc.Allrightsreserved.
PublishedsimultaneouslyinCanada.
Nopartofthispublicationmaybereproduced,storedinaretrievalsystemortransmittedinanyform
orbyanymeans,electronic,mechanical,photocopying,recording,scanningorotherwise,exceptas
permittedunderSections107or108ofthe1976UnitedStatesCopyrightAct,withouteithertheprior
writtenpermissionofthePublisher,orauthorizationthroughpaymentoftheappropriateper-copyfeeto
theCopyrightClearanceCenter,222RosewoodDrive,Danvers,MA01923,[phone],fax(978)
750-4744.RequeststothePublisherforpermissionshouldbeaddressedtothePermissionsDepartment,
JohnWiley&Sons,Inc.,605ThirdAvenue,NewYork,NY10158-0012,[phone],fax(212)
850-6008.E-Mail:[email protected].
Fororderingandcustomerservice,call1-800-CALL-WILEY.
LibraryofCongressCataloging-in-PublicationData
Rencher,AlvinC.,1934–
Methodsofmultivariateanalysis/AlvinC.Rencher.—2nded.
p. cm.—(Wileyseriesinprobabilityandmathematicalstatistics)
“AWiley-Intersciencepublication.”
Includesbibliographicalreferencesandindex.
ISBN0-471-41889-7(cloth)
1. Multivariateanalysis. I. Title. II. Series.
QA278.R452001
519.5(cid:2)35—dc21 2001046735
PrintedintheUnitedStatesofAmerica
10 9 8 7 6 5 4 3 2 1
Contents
1. Introduction 1
1.1 WhyMultivariateAnalysis?, 1
1.2 Prerequisites, 3
1.3 Objectives, 3
1.4 BasicTypesofDataandAnalysis, 3
2. MatrixAlgebra 5
2.1 Introduction, 5
2.2 NotationandBasicDefinitions, 5
2.2.1 Matrices,Vectors,andScalars, 5
2.2.2 EqualityofVectorsandMatrices, 7
2.2.3 TransposeandSymmetricMatrices, 7
2.2.4 SpecialMatrices, 8
2.3 Operations, 9
2.3.1 SummationandProductNotation, 9
2.3.2 AdditionofMatricesandVectors, 10
2.3.3 MultiplicationofMatricesandVectors, 11
2.4 PartitionedMatrices, 20
2.5 Rank, 22
2.6 Inverse, 23
2.7 PositiveDefiniteMatrices, 25
2.8 Determinants, 26
2.9 Trace, 30
2.10 OrthogonalVectorsandMatrices, 31
2.11 EigenvaluesandEigenvectors, 32
2.11.1 Definition, 32
2.11.2 I+AandI−A, 33
2.11.3 tr(A)and|A|, 34
2.11.4 PositiveDefiniteandSemidefiniteMatrices, 34
2.11.5 TheProductAB, 35
2.11.6 SymmetricMatrix, 35
v
vi CONTENTS
2.11.7 SpectralDecomposition, 35
2.11.8 SquareRootMatrix, 36
2.11.9 SquareMatricesandInverseMatrices, 36
2.11.10 SingularValueDecomposition, 36
3. CharacterizingandDisplayingMultivariateData 43
3.1 MeanandVarianceofaUnivariateRandomVariable, 43
3.2 CovarianceandCorrelationofBivariateRandomVariables, 45
3.2.1 Covariance, 45
3.2.2 Correlation, 49
3.3 ScatterPlotsofBivariateSamples, 50
3.4 GraphicalDisplaysforMultivariateSamples, 52
3.5 MeanVectors, 53
3.6 CovarianceMatrices, 57
3.7 CorrelationMatrices, 60
3.8 MeanVectorsandCovarianceMatricesforSubsetsof
Variables, 62
3.8.1 TwoSubsets, 62
3.8.2 ThreeorMoreSubsets, 64
3.9 LinearCombinationsofVariables, 66
3.9.1 SampleProperties, 66
3.9.2 PopulationProperties, 72
3.10 MeasuresofOverallVariability, 73
3.11 EstimationofMissingValues, 74
3.12 DistancebetweenVectors, 76
4. TheMultivariateNormalDistribution 82
4.1 MultivariateNormalDensityFunction, 82
4.1.1 UnivariateNormalDensity, 82
4.1.2 MultivariateNormalDensity, 83
4.1.3 GeneralizedPopulationVariance, 83
4.1.4 DiversityofApplicationsoftheMultivariateNormal, 85
4.2 PropertiesofMultivariateNormalRandomVariables, 85
4.3 EstimationintheMultivariateNormal, 90
4.3.1 MaximumLikelihoodEstimation, 90
4.3.2 DistributionofyandS, 91
4.4 AssessingMultivariateNormality, 92
4.4.1 InvestigatingUnivariateNormality, 92
4.4.2 InvestigatingMultivariateNormality, 96
CONTENTS vii
4.5 Outliers, 99
4.5.1 OutliersinUnivariateSamples, 100
4.5.2 OutliersinMultivariateSamples, 101
5. TestsonOneorTwoMeanVectors 112
5.1 MultivariateversusUnivariateTests, 112
5.2 Testson(cid:1)with(cid:2)Known, 113
5.2.1 ReviewofUnivariateTestfor H :µ=µ
0 0
withσ Known, 113
5.2.2 MultivariateTestfor H :(cid:1)=(cid:1) with(cid:2)Known, 114
0 0
5.3 Testson(cid:1)When(cid:2)IsUnknown, 117
5.3.1 ReviewofUnivariatet-Testfor H :µ=µ withσ
0 0
Unknown, 117
5.3.2 Hotelling’sT2-Testfor H :(cid:1)=(cid:1) with(cid:2)Unknown, 117
0 0
5.4 ComparingTwoMeanVectors, 121
5.4.1 ReviewofUnivariateTwo-Samplet-Test, 121
5.4.2 MultivariateTwo-SampleT2-Test, 122
5.4.3 LikelihoodRatioTests, 126
5.5 TestsonIndividualVariablesConditionalonRejectionof H by
0
theT2-Test, 126
5.6 ComputationofT2, 130
5.6.1 ObtainingT2fromaMANOVAProgram, 130
5.6.2 ObtainingT2fromMultipleRegression, 130
5.7 PairedObservationsTest, 132
5.7.1 UnivariateCase, 132
5.7.2 MultivariateCase, 134
5.8 TestforAdditionalInformation, 136
5.9 ProfileAnalysis, 139
5.9.1 One-SampleProfileAnalysis, 139
5.9.2 Two-SampleProfileAnalysis, 141
6. MultivariateAnalysisofVariance 156
6.1 One-WayModels, 156
6.1.1 UnivariateOne-WayAnalysisofVariance(ANOVA), 156
6.1.2 MultivariateOne-WayAnalysisofVarianceModel
(MANOVA), 158
6.1.3 Wilks’TestStatistic, 161
6.1.4 Roy’sTest, 164
6.1.5 PillaiandLawley–HotellingTests, 166
viii CONTENTS
6.1.6 UnbalancedOne-WayMANOVA, 168
6.1.7 SummaryoftheFourTestsandRelationshiptoT2, 168
6.1.8 MeasuresofMultivariateAssociation, 173
6.2 ComparisonoftheFourManovaTestStatistics, 176
6.3 Contrasts, 178
6.3.1 UnivariateContrasts, 178
6.3.2 MultivariateContrasts, 180
6.4 TestsonIndividualVariablesFollowingRejectionof H bythe
0
OverallMANOVATest, 183
6.5 Two-WayClassification, 186
6.5.1 ReviewofUnivariateTwo-WayANOVA, 186
6.5.2 MultivariateTwo-WayMANOVA, 188
6.6 OtherModels, 195
6.6.1 HigherOrderFixedEffects, 195
6.6.2 MixedModels, 196
6.7 CheckingontheAssumptions, 198
6.8 ProfileAnalysis, 199
6.9 RepeatedMeasuresDesigns, 204
6.9.1 Multivariatevs.UnivariateApproach, 204
6.9.2 One-SampleRepeatedMeasuresModel, 208
6.9.3 k-SampleRepeatedMeasuresModel, 211
6.9.4 ComputationofRepeatedMeasuresTests, 212
6.9.5 RepeatedMeasureswithTwoWithin-Subjects
FactorsandOneBetween-SubjectsFactor, 213
6.9.6 RepeatedMeasureswithTwoWithin-Subjects
FactorsandTwoBetween-SubjectsFactors, 219
6.9.7 AdditionalTopics, 221
6.10 GrowthCurves, 221
6.10.1 GrowthCurveforOneSample, 221
6.10.2 GrowthCurvesforSeveralSamples, 229
6.10.3 AdditionalTopics, 230
6.11 TestsonaSubvector, 231
6.11.1 TestforAdditionalInformation, 231
6.11.2 StepwiseSelectionofVariables, 233
7. TestsonCovarianceMatrices 248
7.1 Introduction, 248
7.2 TestingaSpecifiedPatternfor(cid:2), 248
7.2.1 Testing H : (cid:2)=(cid:2) , 248
0 0
CONTENTS ix
7.2.2 TestingSphericity, 250
7.2.3 Testing H : (cid:2)=σ2[(1−ρ)I+ρJ], 252
0
7.3 TestsComparingCovarianceMatrices, 254
7.3.1 UnivariateTestsofEqualityofVariances, 254
7.3.2 MultivariateTestsofEqualityofCovarianceMatrices, 255
7.4 TestsofIndependence, 259
7.4.1 IndependenceofTwoSubvectors, 259
7.4.2 IndependenceofSeveralSubvectors, 261
7.4.3 TestforIndependenceofAllVariables, 265
8. DiscriminantAnalysis:DescriptionofGroupSeparation 270
8.1 Introduction, 270
8.2 TheDiscriminantFunctionforTwoGroups, 271
8.3 RelationshipbetweenTwo-GroupDiscriminantAnalysisand
MultipleRegression, 275
8.4 DiscriminantAnalysisforSeveralGroups, 277
8.4.1 DiscriminantFunctions, 277
8.4.2 AMeasureofAssociationforDiscriminantFunctions, 282
8.5 StandardizedDiscriminantFunctions, 282
8.6 TestsofSignificance, 284
8.6.1 TestsfortheTwo-GroupCase, 284
8.6.2 TestsfortheSeveral-GroupCase, 285
8.7 InterpretationofDiscriminantFunctions, 288
8.7.1 StandardizedCoefficients, 289
8.7.2 Partial F-Values, 290
8.7.3 CorrelationsbetweenVariablesandDiscriminant
Functions, 291
8.7.4 Rotation, 291
8.8 ScatterPlots, 291
8.9 StepwiseSelectionofVariables, 293
9. ClassificationAnalysis:AllocationofObservationstoGroups 299
9.1 Introduction, 299
9.2 ClassificationintoTwoGroups, 300
9.3 ClassificationintoSeveralGroups, 304
9.3.1 EqualPopulationCovarianceMatrices:Linear
ClassificationFunctions, 304
9.3.2 UnequalPopulationCovarianceMatrices:Quadratic
ClassificationFunctions, 306