Proof of Bayes' rule: Probability form

Let \(\mathbf H\) be a variable in \(\mathbb P\) for the true hypothesis, and let \(H_k\) be the possible values of \(\mathbf H,\) such that \(H_k\) is mutually exclusive and exhaustive. Then, Bayes’ theorem states:

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e\mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)},$$

with a proof that runs as follows. By the definition of conditional probability,

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e \wedge H_i)}{\mathbb P(e)} = \dfrac{\mathbb P(e \mid H_i) \cdot \mathbb P(H_i)}{\mathbb P(e)}$$

By the law of marginal probability:

$$\mathbb P(e) = \sum_{k} \mathbb P(e \wedge H_k)$$

By the definition of conditional probability again:

$$\mathbb P(e \wedge H_k) = \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)$$


Note that this proof of Bayes’ rule is less general than the proof of the odds form of Bayes’ rule.


Using the Diseasitis example problem, this proof runs as follows:

$$\begin{array}{c} \mathbb P({sick}\mid {positive}) = \dfrac{\mathbb P({positive} \wedge {sick})}{\mathbb P({positive})} \\[0.3em] = \dfrac{\mathbb P({positive} \wedge {sick})}{\mathbb P({positive} \wedge {sick}) + \mathbb P({positive} \wedge \neg {sick})} \\[0.3em] = \dfrac{\mathbb P({positive}\mid {sick}) \cdot \mathbb P({sick})}{(\mathbb P({positive}\mid {sick}) \cdot \mathbb P({sick})) + (\mathbb P({positive}\mid \neg {sick}) \cdot \mathbb P(\neg {sick}))} \end{array} $$


$$3/7 = \dfrac{0.18}{0.42} = \dfrac{0.18}{0.18 + 0.24} = \dfrac{90\% \* 20\%}{(90\% \* 20\%) + (30\% \* 80\%)}$$

Using red for sick, blue for healthy, and + signs for positive test results, the proof above can be visually depicted as follows:

bayes theorem probability

todo: if we replace the other Venn diagram for the proof of Bayes’ rule, we should probably update this one too.