Statistical Information and Likelihood: A Collection of Critical Essays by Dr. D. Basu

385 Pages·1988·14.391 MB·English

by  J. K. Ghosh (auth.), J. K. Ghosh (eds.)

Lecture Notes in Statistics
Vol. 1: R.A. Fisher: An Appreciation. Edited by S.E. Fienberg and D.V. Hinkley. XI, 208 pages,
1980.
Vol. 2: Mathematical Statistics and Probability Theory. Proceedings 1978. Edited by W. Klonecki,
A. Kozek, and J. Rosinski. XXIV, 373 pages, 1980.
Vol. 3: B. D. Spencer. Benefit-Cost Analysis of Data Used to Allocate Funds. VIII, 296 pages, 1980.
Vol. 4: E. A. van Doorn, Stochastic Monotonicity and Queueing Applications of Birth-Death Proces
ses. VI, 118 pages, 1981.
Vol. 5: T. Rolski, Stationary Random Processes Associated with Point Processes. VI, 139 pages,
1981.
Vol. 6: S. S. Gupta and D.-Y. Huang, Multiple Statistical Decision Theory: Recent Developments. VIII,
104 pages, 1981.
Vol. 7: M. Akahira and K. Takeuchi, Asymptotic Efficiency of Statistical Estimators. VIII, 242 pages,
1981.
Vol. 8: The First Pannonian Symposium on Mathematical Statistics. Edited by P. Revesz, L. Schmet
terer, and V. M. Zolotarev. VI, 308 pages, 1981.
Vol. 9: B. J0rgensen, Statistical Properties of the Generalized Inverse Gaussian Distribution. VI, 188
pages, 1981.
Vol. 10: A. A. Mcintosh, Fitting Linear Models: An Application on Conjugate Gradient Algorithms. VI,
200 pages, 1982.
Vol. 11: D. F. Nicholls and B. G. Quinn, Random Coefficient Autoregressive Models: An Introduction.
V, 154 pages, 1982.
Vol. 12: M. Jacobsen, Statistical Analysis of Counting Processes. VII, 226 pages, 1982.
Vol. 13: J. Pfanzagl (with the assistance of W. Wefelmeyer), Contributions to a General Asymptotic
Statistical Theory. VII, 315 pages, 1982.
Vol. 14: GUM 82: Proceedings of the International Conference on Generalised Linear Models.
Edited by R. Gilchrist. V, 188 pages, 1982.
Vol. 15: K. R. W. Brewer and M. Hanif, Sampling with Unequal Probabilities. IX, 164 pages, 1983.
Vol. 16: Specifying Statistical Models: From Parametric to Non-Parametric, Using Bayesian or Non
Bayesian Approaches. Edited by J. P. Florens, M. Mouchart, J. P. Raoult, L. Simar, and A. F. M.
Smith. XI, 204 pages, 1983.
Vol. 17: I. V. Basawa and D. J. Scott, Asymptotic Optimal Inference for Non-Ergodic Models. IX,
170 pages, 1983.
Vol. 18: W. Britton, Conjugate Duality and the Exponential Fourier Spectrum. V, 226 pages, 1983.
Vol. 19: L. Fernholz, von Mises Calculus For Statistical Functionals. VIII, 124 pages, 1983.
Vol. 20: Mathematical Learning Models - Theory and Algorithms: Proceedings of a Conference.
Edited by U. Herkenrath, D. Kalin, W. Vogel. XIV, 226 pages, 1983.
Vol. 21: H. Tong, Threshold Models in Non-linear Time Series Analysis. X, 323 pages, 1983.
Vol. 22: S. Johansen, Functional Relations, Random Coefficients and Nonlinear Regression with
Application to Kinetic Data. VIII, 126 pages. 1984.
Vol. 23: D. G. Saphire, Estimation of Victimization Prevalence Using Data from the National Crime
Survey. V, 165 pages. 1984.
Vol. 24: T. S. Rao, M. M. Gabr, An Introduction to Bispectral Analysis and Bilinear Time Series
Models. VIII, 280 pages, 1984.
Vol. 25: Time Series Analysis of Irregularly Observed Data. Proceedings, 1983. Edited by E. Parzen.
VII, 363 pages, 1984.
Lecture Notes in
Statistics
Edited by J. Berger, S. Fienberg,
J. Gani, and K. Krickeberg
45
J.K. Ghosh
Editor
Statistical Information
and Likelihood
A Collection of Critical Essays
by Dr. D. 8asu
Springer-Verlag
New York Berlin Heidelberg London Paris Tokyo
1.K. Ghosh
Indian Statistical Institute
Calcutta 700 035
India
Mathematics Subject Classification (1980): 62AlO
Library of Congress Cataloging-in-Publication Data
Ghosh,1.K
Statistical information and likelihOod
(Lecture notes in statistics; 45)
Bibliography: p.
1. Estimation theory. 2. Basu, D. (Dev) I. Ghosh,
1.K II. Title. III. Series: Lecture notes in
statistics (Springer-Verlag) ; v. 45.
QA276.8.B393 1988 519.5'44 88-4931
© 1988 by Springer-Verlag New York Inc.
All rights reserved. This work may not be translated or copied in whole or in part without the written permission
of the publisher (Springer-Verlag, 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in
connection with reviews or scholarly analysis. Use in connection with any form of information storage and
retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or here
after developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc. in this publication, even if the former are
not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Mer
chandise Marks Act, may accordingly be used freely by anyone.
Camera-ready text prepared by the editor.
98765 4 3 2 1
ISBN -13:978-0-387-96751-6 e-ISBN-13: 978-1-4612-3894-2
DOl: 10.1007/978-1-4612-3894-2
DEDICATED
To the Fond and ColoUlful
Memories of
SIR RONALD AYLMER FISHER (1890-1962)
and
PROFESSOR JERZY NEYMAN (1894-1981)
FOREWORD
It is an honor to be asked to write a foreword to this book, for I believe
that it and other books to follow will eventually lead to a dramatic change in the
current statistics curriculum in our universities.
I spent the 1975-76 academic year at Florida State University in Tallahassee.
My purpose was to complete a book on Statistical Reliability Theory with Frank
Proschan. At the time, I was working on total time on test processes. At the same
time, I started attending lectures by Dev Basu on statistical inference. It was
Lehmann's hypothesis testing course and Lehmann's book was the text. However,
I noticed something strange - Basu never opened the book. He was obviously not
following it. Instead, he was giving a very elegant, measure theoretic treatment
of the concepts of sufficiency, ancillarity, and invariance. He was interested in
the concept of information - what it meant. - how it fitted in with contemporary
statistics. As he looked at the fundamental ideas, the logic behind their use seemed
to evaporate. I was shocked. I didn't like priors. I didn't like Bayesian statistics.
But after the smoke had cleared, that was all that was left.
Basu loves counterexamples. He is like an art critic in the field of statistical
inference. He would find a counterexample to the Bayesian approach if he could.
So far, he has failed in this respect.
In 1979, Basu wrote the following : "It is about 12 years now that I finally
came to the sad conclusion that most of the statistical methods that I had learned
from pioneers like Karl Pearson, Ronald Fisher and Jerzy Neyman and survey practi
tioners like Morris Hanson, P. C. Mahalanobis and Frank Yates are logically unten
able." I believe he is right. Read Basu.
Richard Barlow
PREFACE
These essays on foundations of Statistical Inference have been brought together
in a single volume to honour Dr Basu, as he is affectionately known to his colleagues
in the Indian Statistical Institute. The essays are among the most significant con
tributions of our time to questions of foundation. The sequence in which they have
been arranged, in consultation with Dr. Basu, makes it possible to read them (at
least Parts I and II) as a single contemporary discourse on the likelihood principle,
the paradoxes that attend its violation and the radical deviation from classical statis
tical practices that its adoption would entail. One may also read them, with the
aid of the author's notes, as a record of a personal quest.
If one accepts the likelihood principle, one is led almost inevitably to a Baye
sian pO:3ition. Basu is a Bayesian too. However, he holds this view in a tentative
rather than dogmatic way, even though his rejection of almost all classical statistics
is complete. This tentative Bayesian view appears in several chapters but specially
ill chapters VII an::! XVII where it is evident in the position that he adopts regarding
elimination of nuisance parameters or partial likelihood. It also appears in his reluc
tance, expressed in many private discussions, to put a prior on a large or infinite
dimensional parameter space.
The relation of Part III to the earlier parts needs to be made clear. The first
three chapters are a rounding off of some of the earlier discussions. Three, namely
chapters XXII, XXIII and XXIV present a few well-known results of Basu which appear
directly or indirectly in many of the examples in Part I. Chapters XX and XXI are
on an entirely different aspect of foundations. They examine the impact of choice
of loss functions on the notion of efficient estimates in small and large samples.
Chapter XX contains a beauti ful construction which shows the existence of a best
unbiased estimate depends crucially on convexity of the loss function. Chapter XXI
contains the germ of a seminal idea which took its final shape in the hands of Baha
duro Both are forerunners of concern about robustness under varying loss functions.
The following biographical details came out in course of an interview with
him in October 1987.
Basu had his first course of Statistics from Professor M.C. Chakrabarty in
the honours programme in Mathematics at Dacca University in what is now Bangla
desh. This was around 1945. He got his master's degree in Mathematics also from
Dacca and taught there from 1947 to 1948 when he moved to Calcutta. This was
the time of independence, and partition of India when the whole subcontinent was
in turmoil. Basu spent some time in an insurance company, trying to become an actu
ary. Realising that this was not his vocation, he returned to Dacca as a Senior
Preface
Lecturer in 1950. After a few weeks he came to Calcutta again and joined the
Indian Statistical Institute as a research scholar under Professor C. R. Rao. After
submitting his thesis in 1953 he went to Berkeley as a Fullbright scholar. He came
back a cO'Tlplete Neyman-Pearsonian. His first doubts began to form when he learnt
about ancillaries and conditional inference fro'Tl Fisher during Fisher's visit to the
Institute in 1955. Fro'Tl then onwards he began examining very critically both the
Neyman-Pearsonian and the Fisherian framework, his examination eventually forcing
him to a Bayesian point of view, via the likelihood route, which had been opened
up but then abandon9d by Barnard and BirnbauTl. His final conversion, if one may
call it so, took place in a rather amusing W3Y in 1968. Though not a Bayesian yet,
he had been invited by Professor H. K. Nandi to be the first speaker in a Bayesian
symposium at the Indian Science Congress held in Benaras in January, 1968. Basu
says his lecture convinced none but himself; in course of preparina for that lecture
he became a Bayesian. The polemical papers on foundation began to appear after
1968.
The papers are being reproduced more or less as they were originally published,
except for the fact that occasionally a long paper has been broken up into more than
one chapter and the reference style has been made uniform. A few inconsistencies in
spellings and references remain. References to earlier chapters are sometimes in
the form of references to the corresponding papers or parts thereof. Moreover, all
references to contemporary or past events remain as they originally appeared.
Bringing out this volume would not have been possible without the help and
co-operation of many people. Professor Barlow was kind enough to write a foreword.
Dr. Basu has provided a summing up and author's notes at my request. I would like
to thank all my colleagues in the Institute who spent a lot of time looking through
the manuscript but a special mention must be made of Tapas Samanta and Sumitra
Purkayastha who put in an enormous amount of work. My thanks are due to Subhas
Dutta for typing the manuscript and to Prandhan Nandy for taking care of all the
correspondence related to the publication of the monograph and much else.
Indian Statistical Institute J. K. Ghosh
Calcutta 700 035
November, 1987
viii
ACKNOWLEDGEMENT
I am grateful to
Professor G. A. Barnard
Professor O. Barndorff-Nielsen
Professor D. R. Cox
Professor A. P. Dempster
Professor A. W. F. Edwards
FJro fessor V. P. Godambe
Professor David V. Hinkley
Professor J.D. Kalbfleisch
Professor Oscar Kempthorne
Professor J. C. Koop
Professor David A. Lane
Pro fessor S. L. Lauri tzen
Professor J. N. K. Rao
Professor R. Royall
Professor D. B. Rubin
for their permission to include their discussion on Dr. Basu's papers and to
Academic Press
American Statistical Association
Concordia University
Hall, Rinehart a!ld Winston
Insti tute 0 f Mathematical Statistics
Interna tiona I Statistical Insti tute
John Wiley & Sons Limited
North Holland Publishing Company
Statistical Publishing Society
University of California Press
Uni versi t y 0 f North Carolina
University of Sao Paulo
for glvmg permission to reproduce material published by them. The exact source is
indicated in the Reference List at the end of the volume.
J. K. G.
CONTENTS
Foreword
Editor's Preface
Acknowledgement
Author's Summing Up
PART I
INFORMA nON AND LIKELIHOOD
Chapter I
Recovery of Ancillary Information
O. Notes
1. Introduction 3
2. The Sample Size Analogy 9
3. A Logical Difficulty 13
4. Conceptual Statistical Experiments 16
Chapter II
Statistical Information and Likelihood Part I : Principles 20
O. Notes 20
1. Statistical Information 21
2. Basic Definitions and Relations 23
3. Some Principles of Inference 27
4. Information as a Function 31
5. Fisher Information 33
6. The Likelihood Principle 35
Chapter III
Statistical Information and Likelihood Part II : Methods 43
43
O. Notes
Contents
1. Non-Bayesian Likelihood Methods 45
2. Likelihood: Point Function or a Measure? 48
3. Maximum Likelihood 55
Chapter IV
Statistical Information and Likelihood Part III : Paradoxes 60
O. Notes 60
1. A Fallacy of Five Terms 61
2. The Stopping Rule Paradox 64
3. The Stein Paradox 70
Chapter V
Statistical Information and Likelihood : Discussions 78
O. Notes 78
1. Discussions 79
2. Barnard-Basu Correspondence 88
Chapter VI
Partial Sufficiency 98
O. Notes 98
1. Introduction 99
2. Specific Sufficient Statistics 101
3. Partial Sufficiency 102
4. H-sufficiency 104
5. Invariantly Sufficient Statistics 108
6. Final Remarks 111
Chapter VII
Elimination of Nuisance Parameters 114
O. Notes 114
1. The Elimination Problem and Methods 115
2. Marginalization and Conditioning 117
xii