Arguments Against Bayesian Epistemology
Showing only #4320 and its comments.
See full discussionLog in or sign up to participate in this discussion.
With an account, you can revise, criticize, and comment on ideas.The Popper-Miller Theorem
In 1983, Karl Popper and David Miller published a paper in Nature titled "A proof of the impossibility of inductive probability" that used Bayesian math to prove something uncomfortable: the part of a theory that goes beyond the evidence never gets supported by that evidence. It actually gets negative support. In their words: "probabilistic support in the sense of the calculus of probability can never be inductive support." (Popper & Miller, 1983)
They expanded on this in a second paper: "although evidence may raise the probability of a hypothesis above the value it achieves on background knowledge alone, every such increase in probability has to be attributed entirely to the deductive connections that exist between the hypothesis and the evidence." (Popper & Miller, 1987)
Here's what that means concretely. Say your theory is "swans are white because the genes controlling feather pigmentation in the swan lineage produce only white melanin." This is an explanation: it tells you why swans are white, not just that they are. It also predicts that the next swan you see will be white.
You see a white swan. Your overall confidence in the theory goes up. But Popper and Miller split the theory into two pieces:
- The deductive piece: "this particular swan I'm looking at is white." The evidence directly confirmed the theory's prediction for this case.
- The inductive piece: "and the reason it's white is a genetic mechanism that applies to all swans, including the ones I haven't looked at." This is the actual explanation — the part that would represent learning something new about the world.
They proved mathematically that piece 2 — the explanation, the part that matters — always receives zero or negative support from the evidence. The only work the evidence ever does is confirm the prediction it directly touched. It never reaches the explanation behind it.
The Math
What follows is a simplified sketch of the proof. For the full formal treatment, see the original paper.
Step 1: Define "support."
The support that evidence e gives to hypothesis h is defined as the change in probability:
s(h|e) = p(h|e) − p(h)
If this number is positive, the evidence raised the probability of the theory. Bayesians call this "confirmation."
Step 2: Decompose the hypothesis.
Popper and Miller split h into two components:
The deductive component: (h ∨ e), meaning "h or e." This is the part of h that is logically connected to the evidence. If e is true, then (h ∨ e) is automatically true, so evidence trivially supports it.
The inductive component: (h ∨ ¬e), meaning "h or not-e." This is the part of h that goes beyond the evidence — the part that would still need to be true even if the evidence hadn't occurred.
The hypothesis h is logically equivalent to the conjunction of these two components: h ⟺ (h ∨ e) ∧ (h ∨ ¬e).
Step 3: Calculate the support for each component.
Using standard probability rules, the support for the deductive component is:
s(h ∨ e | e) = 1 − p(h ∨ e)
This is always ≥ 0, since p(h ∨ e) ≤ 1. The evidence always supports the deductive part. No surprise, the evidence is logically contained in it.
The support for the inductive component is:
s(h ∨ ¬e | e) = −(1 − p(e))(1 − p(h|e))
Both (1 − p(e)) and (1 − p(h|e)) are ≥ 0 (assuming we're not dealing with certainties), so their product is ≥ 0, and the negative sign means the whole expression is always ≤ 0.
Step 4: The result.
The total support decomposes as:
s(h|e) = s(h ∨ e | e) + s(h ∨ ¬e | e)
The first term (deductive) is always non-negative. The second term (inductive) is always non-positive. The evidence never positively supports the part of the theory that goes beyond the evidence. Whatever "boost" h gets from e is entirely accounted for by the deductive connection between them. The inductive component — the explanation, the mechanism, the part that would represent genuine learning about the unobserved — is always counter-supported.
Implication
The implication is devastating for Bayesian epistemology: the entire framework of "updating beliefs with evidence" is an illusion. The number goes up, but the going-up is entirely accounted for by deduction. There is no induction hiding inside Bayes' theorem. The Bayesians' own math proves it.
David Deutsch, who has been working with colleague Matjaž Leonardis on a more accessible presentation of the theorem (Deutsch on X/Twitter, 2020), puts it this way: "There's a deductive part of the theory whose credence goes up. But the instances never imply Ahe theory. So you want to ask: 'The part of the theory that's not implied logically by the evidence – why does our credence for that go up?' Well, unfortunately it goes down." (Joseph Walker Podcast, Ep. 139, "Against Bayesianism")
Criticism 1: The Decomposition is Arbitrary
The Popper-Miller theorem works by splitting any prediction h into two pieces and then showing the evidence always hurts one of them. The entire argument rises or falls on whether that split is the right one. This is the most common objection in the literature.
Say your prediction is "it will rain tomorrow" and your evidence is "the barometer is falling." They split the prediction into:
- "Rain OR barometer falling": the part that overlaps with the evidence
- "Rain OR barometer NOT falling": the part that "goes beyond" the evidence
The evidence trivially supports the first part. But it hurts the second: you now know the barometer IS falling, which kills the "barometer not falling" escape route, so the whole thing narrows to just "rain", a harder path than before. Popper and Miller call this second part the "inductive content," show it always gets negative support, and declare induction impossible.
But this is not the only way to carve up "it will rain." You could split it into
- "rain AND barometer falling" OR
- "rain AND barometer NOT falling"
And now the evidence clearly boosts the first piece. Or you could not split it at all and just ask: does a falling barometer raise the probability of rain? Yes. That's inductive support, no decomposition needed. Only Popper and Miller's particular carving guarantees the "beyond" part always gets hurt.
So why this split? Their rule: the part that "goes beyond" the evidence must share no nontrivial logical consequences with it. The "beyond" part and the evidence must have absolutely nothing in common*. The only proposition satisfying this is (h ∨ ¬e), which forces the decomposition and makes the theorem work.
Philosopher Charles Chihara argued this rule is way too strict. Consider:
- Prediction: "All metals expand when heated"
- Evidence: "This rod is copper"
Together these yield: "This copper rod will expand when heated." Neither alone tells you that. It clearly goes beyond the evidence. But under Popper and Miller's rule it doesn't count, because it shares a consequence with the evidence (both mention this copper rod). Chihara's alternative: k "goes beyond" e if e does not logically entail k.
Under this looser definition, the negative support result disappears. He published this with Donald Gillies, who had earlier defended the theorem but agreed the decomposition question needed revisiting. (Chihara & Gillies, 1990, PDF)
Ellery Eells made a related point: look at "rain OR barometer NOT falling": it welds your weather prediction to the negation of your barometric reading. That's not a clean extraction of "the part about rain that has nothing to do with barometers." It's a Frankenstein proposition the algebra created. Eells argued this assumption has been "almost uniformly rejected" in the literature. (Eells, 1988, PDF)
Placeholder Criticism: The Decomposition is NOT Arbitrary
Deutsch argues the decomposition is not arbitrary: it follows necessarily from the probability calculus itself. He and Leonardis have been working on a paper to make this clearer, noting that "Popper and Miller's two papers on this are very condensed and mathematical and use special terminology they created," which has made the result difficult for others to evaluate fairly. The difficulty of presentation has been mistaken for a flaw in the argument. (Joseph Walker Podcast, Ep. 139)
Deutsch never actually explains why the decomposition is necessary. Therefore this criticism is a placeholder and to be updated once someone finds out his reasoning.
The Popper-Miller theorem works by splitting any prediction h into two pieces…
I wonder if your revision from hypothesis to revision was a bit sweeping.
Do they really argue predictions can be split into two pieces? That doesn’t sound right. But I could see hypotheses being split in two.