An Intermediate Course in Probability

319 Pages·2009·4.16 MB·English

by  Allan Gut (auth.)

Statistics Texts in Statistics

Series Editors
G. Casella
S. Fienberg
I. Olkin

For other titles published in this series, go to
www.springer.com/series/417
Allan Gut
An Intermediate Course

in Probability

Second Edition
Allan Gut
Department of Mathematics
Uppsala University
SE-751 06 Uppsala
Sweden
[email protected]
Series Editors:
George Casella Stephen Fienberg Ingram Olkin
Department of St atistics Department of Statistics Department of Statistics
University of Florida Carnegie Mellon University Stanford University
Gainesville, FL 32611-8545 Pittsburgh, PA 15213-3890 Stanford, CA 94305
USA USA USA

ISSN 1431 -875X
ISBN 978-1-4419-0161-3 e-ISBN 978-1-4 419-0162-0
DOI 10.1007/978-1-4419-0162-0
Springer Dordrecht Heidelberg London New York

Library of Congress Control Number: 2009927493

© Springer Science+Business Media, LLC 2009
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York,
NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in
connection with any form of information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they
are not identified as such, is not to be taken as an expression of opinion as to whether or not they are
subject to proprietary rights.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)
Preface to the First Edition
The purpose of this book is to provide the reader with a solid background
and understanding of the basic results and methods in probability theory
before entering into more advanced courses (in probability and/or statistics).
The presentation is fairly thorough and detailed with many solved examples.
Severalexamplesaresolvedwithdifferentmethodsinordertoillustratetheir
different levels of sophistication, their pros, and their cons. The motivation
forthisstyleofexpositionisthatexperiencehasprovedthatthehardpartin
courses of this kind usually is the application of the results and methods; to
know how, when, and where to apply what; and then, technically, to solve a
givenproblemonceoneknowshowtoproceed.Exercisesarespreadoutalong
the way, and every chapter ends with a large selection of problems.
Chapters 1 through 6 focus on some central areas of what might be called
pure probability theory: multivariate random variables, conditioning, trans-
forms,ordervariables,themultivariatenormaldistribution,andconvergence.
A final chapter is devoted to the Poisson process because of its fundamental
role in the theory of stochastic processes, but also because it provides an ex-
cellentapplicationoftheresultsandmethodsacquiredearlierinthebook.As
an extra bonus, several facts about this process, which are frequently more
or less taken for granted, are thereby properly verified. The book concludes
with three appendixes: In the first we provide some suggestions for further
reading and in the second we provide a list of abbreviations and useful facts
concerningsomestandarddistributions.Thethirdappendixcontainsanswers
to the problems given at the end of each chapter.
The level of the book is between the first undergraduate course in prob-
ability and the first graduate course. In particular, no knowledge of measure
theoryisassumed.Theprerequisites(beyondafirstcourseinprobability)are
basic analysis and some linear algebra.
Chapter 5 is, essentially, a revision of a handout by professor Carl-Gustav
Esseen. I am most grateful to him for allowing me to include the material in
the book.
vi
The readability of a book is not only a function of its content and how
(well) the material is presented; very important are layout, fonts, and other
aesthetical aspects. My heartfelt thanks to Anders Vretblad for his ideas,
views, and suggestions, for his design and creation of the allan.sty file, and
for his otherwise most generous help.
I am also very grateful to Svante Janson for providing me with various
index-making devices and to Lennart Norell for creating Figure 3.6.1.1 Ola
H¨ossjer and Pontus Andersson have gone through the manuscript with great
care at different stages in a search for misprints, slips, and other obscurities;
I thank them so much for every one of their discoveries as well as for many
otherremarks(unfortunately,Iamresponsibleforpossibleremaininginadver-
tencies).Ialsowishtothankmystudentsfromasecondcourseinprobability
theoryinUppsalaandJanOhlinandhisstudentsfromasimilarcourseatthe
StockholmUniversityforsendingmealistofcorrectionsonanearlierversion
of this book.
Finally, I wish to thank Svante Janson and Dietrich von Rosen for sev-
eral helpful suggestions and moral support, and Martin Gilchrist of Springer-
Verlag for the care and understanding he has shown me and my manuscript.
Uppsala Allan Gut
May 1995
1 Figure 3.7.1 in this, second, edition.
Preface to the Second Edition
The first edition of this book appeared in 1995. Some misprints and (minor)
inadvertencieshavebeencollectedovertheyears,inpartbymyself,inpartby
students and colleagues around the world. I was therefore very happy when
I received an email from John Kimmel at Springer-Verlag asking whether I
would be interested in an updated second edition of the book.
And here it is!
In addition to the cleaning up and some polishing, I have added some
remarks and clarifications here and there, and a few sections have moved to
new places.
Moreimportant,thiseditionfeaturesanewchapter,whichprovidesanin-
troductory outlook into further areas and topics, such as stable distributions
and domains of attraction, extreme value theory and records, and, finally,
an introduction to a most central tool in probability theory and the theory
of stochastic processes, namely the theory of martingales. This chapter is
included mainly as an appetizer to the more advanced theory, for which sug-
gestedfurtherreading is giveninAppendixA.IwishtothankSvanteJanson
for a careful reading of the chapter and for several remarks and suggestions.
I conclude the preface of this second edition by extending my heartfelt
thanks to John Kimmel for his constant support and encouragement—for
always being there—over many years.
Uppsala Allan Gut
April 2009

Contents
Preface to the First Edition ................................... v
Preface to the Second Edition ................................. vii
Notation and Symbols ......................................... xiii
Introduction................................................... 1
1 Models ................................................ 1
2 The Probability Space................................... 2
3 Independence and Conditional Probabilities ................ 4
4 Random Variables ...................................... 5
5 Expectation, Variance, and Moments ...................... 7
6 Joint Distributions and Independence ..................... 8
7 Sums of Random Variables, Covariance, Correlation......... 9
8 Limit Theorems ........................................ 10
9 Stochastic Processes..................................... 11
10 The Contents of the Book................................ 11
1 Multivariate Random Variables .......................... 15
1 Introduction ........................................... 15
2 Functions of Random Variables ........................... 19
2.1 The Transformation Theorem........................ 20
2.2 Many-to-One ...................................... 23
3 Problems .............................................. 24
2 Conditioning ............................................. 31
1 Conditional Distributions ................................ 31
2 Conditional Expectation and Conditional Variance .......... 33
3 Distributions with Random Parameters ................... 38
4 The Bayesian Approach ................................. 43
5 Regression and Prediction................................ 46
x Contents
6 Problems .............................................. 50
3 Transforms............................................... 57
1 Introduction ........................................... 57
2 The Probability Generating Function...................... 59
3 The Moment Generating Function ........................ 63
4 The Characteristic Function.............................. 70
5 Distributions with Random Parameters.................... 77
6 Sums of a Random Number of Random Variables ........... 79
7 Branching Processes..................................... 85
8 Problems .............................................. 91
4 Order Statistics ..........................................101
1 One-Dimensional Results ................................101
2 The Joint Distribution of the Extremes ...................105
3 The Joint Distribution of the Order Statistic ...............109
4 Problems ..............................................113
5 The Multivariate Normal Distribution ...................117
1 Preliminaries from Linear Algebra ........................117
2 The Covariance Matrix ..................................119
3 A First Definition.......................................120
4 The Characteristic Function: Another Definition ............123
5 The Density: A Third Definition ..........................125
6 Conditional Distributions ................................127
7 Independence ..........................................130
8 Linear Transformations..................................131
9 Quadratic Forms and Cochran’s Theorem..................136
10 Problems ..............................................140
6 Convergence .............................................147
1 Definitions .............................................147
2 Uniqueness.............................................150
3 Relations Between the Convergence Concepts...............152
4 Convergence via Transforms..............................158
5 The Law of Large Numbers and the Central Limit Theorem..161
6 Convergence of Sums of Sequences of Random Variables .....165
7 The Galton–Watson Process Revisited.....................173
8 Problems ..............................................176
7 An Outlook on Further Topics ...........................187
1 Extensions of the Main Limit Theorems ...................188
1.1 The Law of Large Numbers: The Non-i-i.d. Case .......188
1.2 The Central Limit Theorem: The Non-i-i.d. Case .......190
1.3 Sums of Dependent Random Variables ................190