Proof of Bayes' rule: Probability form

Let \(\mathbf H\) be a vari­able in \(\mathbb P\) for the true hy­poth­e­sis, and let \(H_k\) be the pos­si­ble val­ues of \(\mathbf H,\) such that \(H_k\) is mu­tu­ally ex­clu­sive and ex­haus­tive. Then, Bayes’ the­o­rem states:

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e\mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)},$$

with a proof that runs as fol­lows. By the defi­ni­tion of con­di­tional prob­a­bil­ity,

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e \wedge H_i)}{\mathbb P(e)} = \dfrac{\mathbb P(e \mid H_i) \cdot \mathbb P(H_i)}{\mathbb P(e)}$$

By the law of marginal prob­a­bil­ity:

$$\mathbb P(e) = \sum_{k} \mathbb P(e \wedge H_k)$$

By the defi­ni­tion of con­di­tional prob­a­bil­ity again:

$$\mathbb P(e \wedge H_k) = \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)$$

Done.

Note that this proof of Bayes’ rule is less gen­eral than the proof of the odds form of Bayes’ rule.

Example

Us­ing the Dise­a­sitis ex­am­ple prob­lem, this proof runs as fol­lows:

$$\begin{array}{c} \mathbb P({sick}\mid {positive}) = \dfrac{\mathbb P({positive} \wedge {sick})}{\mathbb P({positive})} \\[0.3em] = \dfrac{\mathbb P({positive} \wedge {sick})}{\mathbb P({positive} \wedge {sick}) + \mathbb P({positive} \wedge \neg {sick})} \\[0.3em] = \dfrac{\mathbb P({positive}\mid {sick}) \cdot \mathbb P({sick})}{(\mathbb P({positive}\mid {sick}) \cdot \mathbb P({sick})) + (\mathbb P({positive}\mid \neg {sick}) \cdot \mathbb P(\neg {sick}))} \end{array} $$

Numer­i­cally:

$$3/7 = \dfrac{0.18}{0.42} = \dfrac{0.18}{0.18 + 0.24} = \dfrac{90\% \* 20\%}{(90\% \* 20\%) + (30\% \* 80\%)}$$

Us­ing red for sick, blue for healthy, and + signs for pos­i­tive test re­sults, the proof above can be vi­su­ally de­picted as fol­lows:

bayes theorem probability

todo: if we re­place the other Venn di­a­gram for the proof of Bayes’ rule, we should prob­a­bly up­date this one too.

Parents: