How Not to Be Wrong : The Power of Mathematical Thinking

judging a decade’s worth of mutual funds by the ones that still exist at the end of the ten years is like judging our pilots’ evasive maneuvers by counting the bullet holes in the planes that come back.

there’s no reason at all to expect the planes to have an equal likelihood of survival no matter where they get hit.

“What assumptions are you making? And are they justified?”

Here’s a rule of thumb that makes sense to me: if the magnitude of a disaster is so great that it feels right to talk about “survivors,” then it makes sense to measure the death toll as a proportion of total population.

So how are we supposed to rank atrocities, if not by absolute numbers and not by proportion? Some comparisons are clear. The Rwanda genocide was worse than 9/11 and 9/11 was worse than Columbine and Columbine was worse than one person getting killed in a drunk-driving accident. Others, separated by vast differences in time and space, are harder to compare. Was the Thirty Years’ War really more deadly than World War I? How does the horrifyingly rapid Rwanda genocide stack up against the long, brutal war between Iran and Iraq?

Most mathematicians would say that, in the end, the disasters and atrocities of history form what we call a partially ordered set. That’s a fancy way of saying that some pairs of disasters can be meaningfully compared, and others cannot. This isn’t because we don’t have accurate enough death counts, or firm enough opinions as to the relative merits of being annihilated by a bomb versus dying of war-induced famine. It’s because the question of whether one war was worse than another is fundamentally unlike the question of whether one number is bigger than another. The latter question always has an answer. The former does not. And if you want to imagine what it means for twenty-six people to be killed by terrorist bombings, imagine twenty-six people killed by terrorist bombings—not halfway across the world, but in your own city. That computation is mathematically and morally unimpeachable, and no calculator is required

De Moivre’s observation is the same one that underlies the computation of the standard error in a political poll. If you want to make the error bar half as big, you need to survey four times as many people. And if you want to know how impressed to be by a good run of heads, you can ask how many square roots away from 50% it is. The square root of 100 is 10. So when I got 60 heads in 100 tries, that was exactly one square root away from 50-50. The square root of 1,000 is about 31; so when I got 538 heads in 1,000 tries, I did something even more surprising, even though I got only 53.8% heads in the latter case and 60% heads in the former.

Common sense suggests that, at this point, tails must be slightly more likely, in order to correct the existing imbalance.

But common sense says much more insistently that the coin can’t remember what happened the first ten times I flipped it!

I won’t keep you in suspense—the second common sense is right. The law of averages is not very well named, because laws should be true, and this one is false. Coins have no memory. So the next coin you flip has a 50-50 chance of coming up heads, the same as any other

That’s how the Law of Large Numbers works: not by balancing out what’s already happened, but by diluting what’s already happened with new data, until the past is so proportionally negligible that it can safely be forgotten

Scoring by raw number of heads gives the Big team an insuperable advantage; but using percentages slants the game just as badly in favor of the Smalls. The smaller the number of coins—what we’d call in statistics the sample size—the greater the variation in the proportion of heads.

And it’s the same, too, for brain cancer. Small states have small sample sizes—they are thin reeds whipped around by the winds of chance, while the big states are grand old oaks that barely bend. Measuring the absolute number of brain cancer deaths is biased toward the big states; but measuring the highest rates—or the lowest ones!—puts the smallest states in the lead. That’s how South Dakota can have one of the highest rates of brain cancer death while North Dakota claims one of the lowest. It’s not because Mount Rushmore or Wall Drug is somehow toxic to the brain; it’s because smaller populations are inherently more variable.