# Bayes' rule: Guide

Bayes’ rule or Bayes’ the­o­rem is the law of prob­a­bil­ity gov­ern­ing the strength of ev­i­dence—the rule say­ing how much to re­vise our prob­a­bil­ities (change our minds) when we learn a new fact or ob­serve new ev­i­dence.

You may want to learn about Bayes’ rule if you are:

• A pro­fes­sional who uses statis­tics, such as a sci­en­tist or doc­tor;

• A com­puter pro­gram­mer work­ing in ma­chine learn­ing;

• A hu­man be­ing.

As Philip Tet­lock found when study­ing “su­perfore­cast­ers”, peo­ple who were es­pe­cially good at pre­dict­ing fu­ture events:

The su­perfore­cast­ers are a nu­mer­ate bunch: many know about Bayes’ the­o­rem and could de­ploy it if they felt it was worth the trou­ble. But they rarely crunch the num­bers so ex­plic­itly. What mat­ters far more to the su­perfore­cast­ers than Bayes’ the­o­rem is Bayes’ core in­sight of grad­u­ally get­ting closer to the truth by con­stantly up­dat­ing in pro­por­tion to the weight of the ev­i­dence.

— Philip Tet­lock and Dan Gard­ner, Superforecasting

## Learn­ing Bayes’ rule

This guide to Bayes’ rule uses Ar­bital’s tech­nol­ogy to al­low for mul­ti­ple fla­vors of in­tro­duc­tion. They vary by tech­ni­cal level, speed, and top­ics cov­ered. After you pick your path, re­mem­ber that you can still switch be­tween pages, in par­tic­u­lar by us­ing the “Say what?” and “Go faster” but­tons.

Which case fits you best? a: I want to have a ba­sic the­o­ret­i­cal and prac­ti­cal un­der­stand­ing of the Bayes’ rule. -wants: [62d,Bayes’ Rule and its im­pli­ca­tions b: I can eas­ily read alge­bra and don’t mind the ex­pla­na­tion mov­ing at a fast pace. Just give me the ba­sics, quick! wants: Bayes’ Rule and its differ­ent forms -wants: Bayes’ Rule and its im­pli­ca­tions c: I want the ba­sics, but I’m also in­ter­ested in read­ing more about the the­o­ret­i­cal im­pli­ca­tions and the rea­sons why Bayes’ rule is con­sid­ered so im­por­tant. wants: Bayes’ Rule and its im­pli­ca­tions -wants: Bayes’ Rule and its differ­ent forms d: I’d like to read ev­ery­thing! I want to have a deep the­o­ret­i­cal and prac­ti­cal un­der­stand­ing of the Bayes’ rule. wants: Bayes’ Rule and its differ­ent forms,Bayes’ Rule and its im­pli­ca­tions ]

wants-req­ui­site(Bayes’ Rule and its differ­ent forms): %%wants-req­ui­site(Bayes’ Rule and its im­pli­ca­tions): %box: Your path will go over all forms of Baye’s Rule, along with de­vel­op­ing deep ap­pre­ci­a­tion for its sci­en­tific use­ful­ness. Your path will con­tain 12 pages:

• Fre­quency di­a­grams: A first look at Bayes

• Water­fall di­a­grams and rel­a­tive odds

• In­tro­duc­tion to Bayes’ rule: Odds form

• Bayes’ rule: Pro­por­tional form

• Ex­traor­di­nary claims re­quire ex­traor­di­nary evidence

• Or­di­nary claims re­quire or­di­nary evidence

• Bayes’ rule: Log-odds form

• Shift to­wards the hy­poth­e­sis of least surprise

• Bayes’ rule: Vec­tor form

• Belief re­vi­sion as prob­a­bil­ity elimination

• Bayes’ rule: Prob­a­bil­ity form

• Bayesian view of sci­en­tific virtues %

%start-path(Com­pre­hen­sive guide to Bayes’ Rule)% %% %%!wants-req­ui­site(Bayes’ Rule and its im­pli­ca­tions): %box: No time to waste! Let’s plunge di­rectly into a sin­gle-page ab­bre­vi­ated in­tro­duc­tion to Bayes’ rule. % %% <div>

!wants-req­ui­site(Bayes’ Rule and its differ­ent forms): %%wants-req­ui­site(Bayes’ Rule and its im­pli­ca­tions): %box: Your path will teach you the ba­sic odds form of Bayes’ rule at a rea­son­able pace and then delve into the deep mys­ter­ies of the Bayes’ Rule! Your path will con­tain 8 pages:

• Fre­quency di­a­grams: A first look at Bayes

• Water­fall di­a­grams and rel­a­tive odds

• In­tro­duc­tion to Bayes’ rule: Odds form

• Belief re­vi­sion as prob­a­bil­ity elimination

• Ex­traor­di­nary claims re­quire ex­traor­di­nary evidence

• Or­di­nary claims re­quire or­di­nary evidence

• Shift to­wards the hy­poth­e­sis of least surprise

• Bayesian view of sci­en­tific virtues %

%start-path(Bayes’ Rule and its im­pli­ca­tions)% %%

%%!wants-req­ui­site(Bayes’ Rule and its im­pli­ca­tions): %box: Your path will teach you the ba­sic odds form of Bayes’ rule at a rea­son­able pace. It will con­tain 3 pages:

• Fre­quency di­a­grams: A first look at Bayes

• Water­fall di­a­grams and rel­a­tive odds

• In­tro­duc­tion to Bayes’ rule: Odds form %

%start-path(In­tro­duc­tion to Bayes’ Rule odds form)% %% <div>

Children:

Parents:

• Bayes' rule

Bayes’ rule is the core the­o­rem of prob­a­bil­ity the­ory say­ing how to re­vise our be­liefs when we make a new ob­ser­va­tion.

• Bayesian reasoning

A prob­a­bil­ity-the­ory-based view of the world; a co­her­ent way of chang­ing prob­a­bil­is­tic be­liefs based on ev­i­dence.

• Joe made a good point about the way this is phrased not sort­ing peo­ple quite right:

joe AM
“bad at math” = out of Ar­bital’s range

eric_bruy­lant AM
cur­rently, yes the bad at math we’re talk­ing about is sig­nifi­cantly a psy­cholog­i­cal aver­sion, not lack of background

joe AM
I’d say one of the things you might want to do

is to … oh

eric_bruy­lant AM
and we can’t do ther­apy yet

joe AM
in that case, I think it’s some­what poorly worded

be­cause some peo­ple who are not psy­cholog­i­cally averse might still con­sider them­selves “bad at math”

just be­cause they never re­ally put any effort into it

like, they can’t mul­ti­ply two-digit num­bers, but they’d whip out a calcu­la­tor if they had to

any­way: I’d say one of the things you might want to do is to have a list of prob­lems that those peo­ple should be able to un­der­stand the full mean­ing of, al­though not nec­es­sar­ily solve

eric_bruy­lant AM
hm, yea. I kinda agree, though I’m not sure how to get all the peo­ple with an aversion

joe AM
I’d say more, “I don’t like math.”

eric_bruy­lant AM
since many of them won’t re­al­ize the is­sue is an aver­sion rather than them be­ing bad at math

that seems like an im­prove­ment to me

I’ll put a mark on the page about it

joe AM
and I’d re­word math 0 to “I don’t ​hate​ math, but I’m not par­tic­u­larly good at it.” (ed­ited)

since Math 0 is sup­posed to rep­re­sent “not very skil­led”

eric_bruy­lant AM
seems good

joe AM
so they ​are​ “bad at math”, just not bad enough to have a pho­bia around it

• Note: I’m not cer­tain about the al­ter­nate word­ing, and meant to sug­gest changes to the math 0 or math 1 pages rather than di­rectly here. I may also be miss­ing some­thing, so am let­ting Nate or EY check/​rewrite rather than ap­prov­ing.

• What about call­ing this page the “tu­to­rial” rather than “guide”? Tu­to­ri­als are more likely to be in­ter­ac­tive. And both the main and ex­plore tabs feel more like what I would ex­pect a “guide” to be than this page.

Guided walk-through or guided path would also work.

• I’m very con­fused why you need two links to the same page (and one of them is blue).

• In case a new user is con­fused by hov­er­ing a green link and see­ing the popup sud­denly poof in; in that case, the blue link gives them a sim­ple way to “just click” some­thing with no un­ex­pected be­hav­ior.

• a.) As Neil Tyson De­grasse ex­presses, sci­ence is true re­gard­less of be­lief:

1.) I no longer sub­scribe to the con­cept of be­lief.

2.) By defi­ni­tion and re­search, be­lief is a con­cept that es­pe­cially per­mits ig­no­rance of ev­i­dence. (See google defi­ni­tion of be­lief…)

3.) Such a model, while per­mit­ting ev­i­dence based thoughts, oth­er­wise largely per­mits ig­no­rance of ev­i­dence!

4.) In­stead, I con­tact sci­en­tific think­ing, some­thing which has long per­mit­ted mankind to make mis­takes, but how­ever, largely fa­cil­i­tat­ing keen­ness of ev­i­dence, con­trary to the con­cept of be­lief!