Bayes' rule: Proportional form

If \(H_i\) and \(H_j\) are hy­pothe­ses and \(e\) is a piece of ev­i­dence, Bayes’ rule states:

$$\dfrac{\mathbb P(H_i)}{\mathbb P(H_j)} \times \dfrac{\mathbb P(e\mid H_i)}{\mathbb P(e\mid H_j)} = \dfrac{\mathbb P(H_i\mid e)}{\mathbb P(H_j\mid e)}$$

if-af­ter(Fre­quency di­a­grams: A first look at Bayes): In the Dise­a­sitis prob­lem, we use this form of Bayes’ rule to jus­tify calcu­lat­ing the pos­te­rior odds of sick­ness via the calcu­la­tion \((1 : 4) \times (3 : 1) = (3 : 4).\)

!if-af­ter(Fre­quency di­a­grams: A first look at Bayes): In the Dise­a­sitis prob­lem, 20% of the pa­tients in a screen­ing pop­u­la­tion have Dise­a­sitis, 90% of sick pa­tients will turn a chem­i­cal strip black, and 30% of healthy pa­tients will turn a chem­i­cal strip black. We can use the form of Bayes’ rule above to jus­tify solv­ing this prob­lem via the calcu­la­tion \((1 : 4) \times (3 : 1) = (3 : 4).\)

If in­stead of treat­ing the ra­tios as odds, we ac­tu­ally calcu­late out the num­bers for each term of the equa­tion, we in­stead get the calcu­la­tion \(\frac{1}{4} \times \frac{3}{1} = \frac{3}{4},\) or \(0.25 \times 3 = 0.75.\)

If we try to di­rectly in­ter­pret this, it says: “If a pa­tient starts out 0.25 times as likely to be sick as healthy, and we see a test re­sult that is 3 times as likely to oc­cur if the pa­tient is sick as if the pa­tient is healthy, we con­clude the pa­tient is 0.75 times as likely to be sick as healthy.”

This is valid rea­son­ing, and we call it the pro­por­tional form of Bayes’ rule. To get the prob­a­bil­ity back out, we rea­son that if there’s 0.75 sick pa­tients to ev­ery 1 healthy pa­tient in a bag, the bag com­prises 0.75/​(0.75 + 1) = 37 = 43% sick pa­tients.

Spotlight visualization

One way of look­ing at this re­sult is that, since odds ra­tios are equiv­a­lent un­der mul­ti­pli­ca­tion by a pos­i­tive con­stant, we can fix the right side of the odds ra­tio as equal­ing 1 and ask about what’s on the left side. This is what we do when see­ing the calcu­la­tion as \((0.25 : 1) \cdot (3 : 1) = (0.75 : 1),\) the form sug­gested by the the­o­rem proved above.

We could vi­su­al­ize Bayes’ rule as a pair of spotlights with differ­ent start­ing in­ten­si­ties, that go through lenses that am­plify or re­duce each in­com­ing unit of light by a fixed mul­ti­plier. In the Dise­a­sitis case, if we fix the right-side blue beam as hav­ing a start­ing in­ten­sity of 1 and a mul­ti­ply­ing lens of 1, and we fix the left-side beam of hav­ing a start­ing in­ten­sity of 0.25 and a mul­ti­ply­ing lens of 3x, then the re­sult gives us a vi­su­al­iza­tion of the calcu­la­tion pre­scribed by Bayes’ rule:

bayes lights

Note the similar­ity to a wa­ter­fall di­a­gram. The main thing the spotlight vi­su­al­iza­tion adds is that we can imag­ine vary­ing the ab­solute in­ten­si­ties of the lights and lenses, while pre­serv­ing their rel­a­tive in­ten­si­ties, in such a way as to make the right-side beams and lenses equal 1.

draw the pre-pro­por­tional, odds form of the spotlight vi­su­al­iza­tion.

todo: add ex­am­ple prob­lem in pro­por­tional/​spotlight form

Use­ful­ness in in­for­mal argument

The pro­por­tional form of Bayes’ rule is per­haps the fastest way of de­scribing Bayesian rea­son­ing that sounds like it ought to be true. If you were hav­ing a fic­tional char­ac­ter sud­denly give a Bayesian ar­gu­ment in the mid­dle of a story be­ing read by many peo­ple who’d never heard of Bayes’ rule, you might have them say:

“Sup­pose the Dark Mark is cer­tain to con­tinue while the Dark Lord’s sen­tience lives on, but a pri­ori we’d only have guessed a twenty per­cent chance of the Dark Mark con­tin­u­ing to ex­ist af­ter the Dark Lord dies. Then the ob­ser­va­tion, “The Dark Mark has not faded” is five times as likely to oc­cur in wor­lds where the Dark Lord is al­ive as in wor­lds where the Dark Lord is dead. Is that re­ally com­men­su­rate with the prior im­prob­a­bil­ity of im­mor­tal­ity? Let’s say the prior odds were a hun­dred-to-one against the Dark Lord sur­viv­ing. If a hy­poth­e­sis is a hun­dred times as likely to be false ver­sus true, and then you see ev­i­dence five times more likely if the hy­poth­e­sis is true ver­sus false, you should up­date to be­liev­ing the hy­poth­e­sis is twenty times as likely to be false as true.”

Similarly, if you were a doc­tor try­ing to ex­plain the mean­ing of a pos­i­tive test re­sult to a pa­tient, you might say: “If we haven’t seen any test re­sults, pa­tients like you are a thou­sand times as likely to be healthy as sick. This test is only a hun­dred times as likely to be pos­i­tive for sick as for healthy pa­tients. So now we think you’re ten times as likely to be healthy as sick, which is still a pretty good chance!”

Vi­sual di­a­grams and spe­cial no­ta­tion for odds and rel­a­tive like­li­hoods might make Bayes’ rule more in­tu­itive, but the pro­por­tional form is prob­a­bly the most valid-sound­ing thing that is quan­ti­ta­tively cor­rect that you can say in three sen­tences.

write a from-scratch Stan­dalone In­tro of the pro­por­tional form of Bayes’ rule in par­tic­u­lar, us­ing the Dise­a­sitis ex­am­ple and go­ing from fre­quency di­a­gram to wa­ter­fall to spotlight, with no proofs, just to jus­tify the pro­por­tional form. add to Main a state­ment that if you can phrase things in pro­por­tional form, there ex­ists a Stan­dalone In­tro that jus­tifies it quickly.

Parents:

  • Bayes' rule

    Bayes’ rule is the core the­o­rem of prob­a­bil­ity the­ory say­ing how to re­vise our be­liefs when we make a new ob­ser­va­tion.