Ordinary claims require ordinary evidence
A corollary to extraordinary claims require extraordinary evidence is “ordinary claims require only ordinary evidence”noteAttributed to Gwern Branwen, who hasn’t popularized this claim nearly as much as Carl Sagan popularized the extraordinary version.
This corollary is probably more important in practice, as, in our day-to-day lives, we encounter more ordinary claims than extraordinary claims, but may be more tempted to reject many of those ordinary claims by demanding extraordinary evidence (a bias known as “motivated skepticism”).
Example: A failing new employee
One contributed example:
A few years back, a senior person at my workplace told me that a new employee wasn’t getting his work done on time, and that she’d had to micromanage him to get any work out of him at all. This was an unpleasant fact for a number of reasons; I’d liked the guy, and I’d advocated for hiring him to our Board of Directors just a few weeks earlier (which is why the senior manager was talking to me). I could have demanded more evidence, I could have demanded that we give him more time to work out, I could have demanded a videotape and signed affidavits… but a new employee not working out, just isn’t that improbable. Could I have named the exact prior odds of an employee not working out, could I have said how much more likely I was to hear that exact report of a long-term-bad employee than a long-term-good employee? No, but ‘somebody hires the wrong person’ happens all the time, and I’d seen it happen before. It wasn’t an extraordinary claim, and I wasn’t licensed to ask for extraordinary evidence. To put numbers on it, I thought the proportion of bad to good employees was on the order of 1 : 4 at the time, and the likelihood ratio for the manager’s report seemed more like 10 : 1.
Or to put it another way: The rule is ‘extraordinary claims require extraordinary evidence’, not ‘inconvenient but ordinary claims require extraordinary evidence’.
In everyday life, we consider many more ordinary claims than extraordinary claims — including many ordinary claims whose inconvenience might tempt us to dismiss their adequate but ordinary evidence. In those cases, it’s important to remember that ordinary claims only require ordinary evidence.
Parents:
- Bayesian update
Bayesian updating: the ideal way to change probabilistic beliefs based on evidence.
I feel like this page, in particular the example, is suggesting that “if the probability of X is > 50%, act as though X were true.” This is clearly (to me) untrue, and/or unwise. For instance, suppose you have a box, and a priori there is a 90% chance it contains $10, and a 10% chance it contains a letterbomb. The odds are clearly in favor of $10, but I’d question the sanity of anybody who tried to open the box (without precautions, extra checks, etc.). AFAICT, even when X is probably true, cost/benefit can change what the best course of action is. It seems like this deserves at least a mention.
Hmm… is this a corollary so much as a converse or an addendum? It would be a corollary (by being the contrapositive) if the statement were “Only extraordinary claims require extraordinary evidence”.