The Improbability Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day

314 Pages·2014·0.97 MB·English

by  David J. Hand

The author and publisher have provided this e-book to you for your personal use
only. You may not make this e-book publicly available in any way. Copyright
infringement is against the law. If you believe the copy of this e-book you
are reading infringes on the author’s copyright, please notify the publisher
at: us.macmillanusa.com/piracy.
For Shelley
The really unusual day would be one where nothing unusual happens.
—Persi Diaconis1
Contents
Title Page
Copyright Notice
Dedication
Epigraph
Preface
1. The Mystery
2. A Capricious Universe
3. What Is Chance?
4. The Law of Inevitability
5. The Law of Truly Large Numbers
6. The Law of Selection
7. The Law of the Probability Lever
8. The Law of Near Enough
9. The Human Mind
10. Life, the Universe, and Everything
11. How to Use the Improbability Principle
Epilogue
Appendix A: Mind-Numbingly Large and Mind-Bogglingly Small
Appendix B: Rules of Chance
Notes
Index
Also by David J. Hand
A Note About the Author
Copyright
Preface
This book is about extraordinarily improbable events. It’s about why incredibly
unlikely things happen. But more: it’s about why they just keep on happening,
time after time after time.
At first glance, this seems like a contradiction. How can events be incredibly
unlikely and yet keep on happening? Unlikely surely means rare.
The fact that it isn’t a contradiction is suggested by many real-life examples:
people winning lotteries multiple times, lightning repeatedly striking the same
unfortunate man, extreme financial crashes occurring again and again, and so on.
But it certainly needs explaining.
The universe has laws which describe the way it works. Newton’s laws of
motion tell us how dropped objects fall and why the moon orbits the earth. They
explain why a car seat presses into your back when you accelerate, and why the
ground hits you so hard when you trip and fall. Other laws of nature show us
how stars are born and how stars die, where humanity comes from, and perhaps
where it’s going.
The same applies to exceedingly unlikely events. The Improbability Principle
is my name for a set of laws of chance which, together, tell us that we should
expect the unexpected, and why.
The laws composing the principle arise at several levels. Some refer to
fundamental aspects of the way the universe is constructed—as fundamental as
the basic, abstract truth that two plus two equals four. Others hinge on deep
properties of what we mean by probability. And yet others arise at the level of
human psychology: the brain is not a simple recording device. In the right
circumstances, any one of the laws is sufficient to lead to an instance of the
principle, but it’s when they come together and work in unison that the force of
the principle becomes really striking. And the inconceivably unlikely happens.
Books like this one are built on research, conversations, and discussions with
many people over many years—far too many to acknowledge properly. But
some were especially helpful in seeing the ideas through the final stages to
becoming a book. My friends and colleagues Mike Crowe, Kate Land, Niall
Adams, Nick Heard, and Christoforos Anagnostopoulos kindly gave me
comments on various drafts. My agent, Peter Tallack, and my editor, Amanda
Moon, played critical roles in the journey from rough draft to finished product.
Coincidentally (or perhaps not, since coincidences are one manifestation of the
Improbability Principle), while the book was still at the conceptual stage, David
Harding, founder of Winton Capital Management, approached me about a role in
his company. The deep inferential challenges I encountered there encouraged me
to think more deeply about rare events. And finally, I am most grateful to my
wife, Shelley, for tolerating, yet again, my spiritual absences as the book
gradually took shape, as well as for making invaluable comments on its content.
3
WHAT IS CHANCE?
All life is chance.

—Dale Carnegie
In 1986 Bill Shaw survived a train crash in Lockington in East Yorkshire, England, that
killed nine people. Although train crashes attract a lot of media attention, fortunately
they’re quite rare: in 2001 there were about 0.1 fatalities per billion passenger miles in the
UK, meaning that rail travel is an exceptionally safe mode of transport. Since train
crashes are so rare, the chance of a husband and wife both being involved in different
train crashes must be incredibly small. And yet this is just what happened to the Shaws:
fifteen years after her husband, at Great Heck near Selby, Bill’s wife Ginny also survived
a deadly train crash. This one killed ten people. Both accidents were caused by vehicles
on the tracks. “I couldn’t believe what she was telling me,” said Bill, thinking back to
when he was woken by the 7:00 a.m. phone call from his wife. “It looks like someone
wanted her to experience what we had been through … Uncannily, then it turned out to
have been caused by a van straddling the lines. This is exactly the same situation in which
Ginny found herself. It’s got to be a freak coincidence, it’s just totally unbelievable … It
seems for some very odd reason the family have all been in the wrong place at the wrong
time.”
Anyone experiencing such an unfortunate coincidence as the Shaws did
would naturally seek an explanation, some linking factor. Is there something
about this coincidence, or coincidences in general, that helps us understand why
it happened?
There are various definitions of the word “coincidence.” The statisticians
Persi Diaconis and Fred Mosteller defined a coincidence as “a surprising
concurrence of events, perceived as meaningfully related, with no apparent
causal connection.”1 My Concise Oxford Dictionary defines it as “a remarkable
concurrence of events or circumstances without apparent causal connection.”
The Wikipedia definition is more elaborate: “a collection of two or more events
or conditions, closely related by time, space, form, or other associations which
appear unlikely to bear a relationship as either cause to effect or effects of a
shared cause, within the observer or observers’ understanding of what cause can
produce what effects.”
The first definition includes the fact that there must be an element of surprise.
Even if, just as I reach the end of a chapter in the book I’m reading, it starts to
rain, I’m not going to sit up and say, “Wow, what a coincidence.” And there
must be more than one event involved: while a single unusual event is one thing,
two or more occurring close together is something else. If the leg of my chair
breaks just as a peal of thunder rings out, I might wonder if it was simply
chance. Many people noted the coincidence in 2013 when, only hours after Pope
Benedict XVI announced that he would be resigning, St. Peter’s Basilica in
Rome was struck by lightning.
The definition also says that the events must be seen as meaningfully related,
despite having no obvious causal connection. Two entirely disconnected events,
even surprising ones, which had no apparent link at all would not arouse
comment. You’d probably not see as related the fact that a roulette ball fell on
number 7 at 9:00 p.m. in a casino, and you broke the heel of your shoe as you
stepped out of a taxi on the way home from work three days later. Why should
they be? An infinite number of events happen all around us all the time—life is
just a series of events—so for a coincidence, something has to single out these
particular events and link them in a meaningful way. The link could just be one
of time—my chair leg and the peal of thunder—but it mustn’t be an obvious
causal one: if you stamped your foot at the casino when the 7 came up, and
broke your heel, you’d hardly regard it as a coincidence; simple causality
explains it. On September 11, 2001, the National Reconnaissance Office was
planning a simulation in which a malfunctioning private jet would crash into the
agency’s headquarters in Chantilly, Virginia, about four miles from
Washington’s Dulles International Airport. At 8:10 a.m. on that day, about an
hour before the simulation was scheduled to start, American Airlines Flight 77
took off from Dulles. An hour and a half later its hijackers flew it into the
Pentagon. The reality and the simulation just look too similar for there to be no
meaningful relationship, and yet there is no causal connection between them.2
We’ve already seen that countless explanations have been advanced for
surprising co-occurrences of events. Many of these cite powers and causes