Bayes' rule: Guide

Bayes’ rule or Bayes’ the­o­rem is the law of prob­a­bil­ity gov­ern­ing the strength of ev­i­dence—the rule say­ing how much to re­vise our prob­a­bil­ities (change our minds) when we learn a new fact or ob­serve new ev­i­dence.

You may want to learn about Bayes’ rule if you are:

  • A pro­fes­sional who uses statis­tics, such as a sci­en­tist or doc­tor;

  • A com­puter pro­gram­mer work­ing in ma­chine learn­ing;

  • A hu­man be­ing.

As Philip Tet­lock found when study­ing “su­perfore­cast­ers”, peo­ple who were es­pe­cially good at pre­dict­ing fu­ture events:

The su­perfore­cast­ers are a nu­mer­ate bunch: many know about Bayes’ the­o­rem and could de­ploy it if they felt it was worth the trou­ble. But they rarely crunch the num­bers so ex­plic­itly. What mat­ters far more to the su­perfore­cast­ers than Bayes’ the­o­rem is Bayes’ core in­sight of grad­u­ally get­ting closer to the truth by con­stantly up­dat­ing in pro­por­tion to the weight of the ev­i­dence.

— Philip Tet­lock and Dan Gard­ner, Superforecasting

Learn­ing Bayes’ rule

This guide to Bayes’ rule uses Ar­bital’s tech­nol­ogy to al­low for mul­ti­ple fla­vors of in­tro­duc­tion. They vary by tech­ni­cal level, speed, and top­ics cov­ered. After you pick your path, re­mem­ber that you can still switch be­tween pages, in par­tic­u­lar by us­ing the “Say what?” and “Go faster” but­tons.

Which case fits you best? a: I want to have a ba­sic the­o­ret­i­cal and prac­ti­cal un­der­stand­ing of the Bayes’ rule. -wants: [62d,Bayes’ Rule and its im­pli­ca­tions b: I can eas­ily read alge­bra and don’t mind the ex­pla­na­tion mov­ing at a fast pace. Just give me the ba­sics, quick! wants: Bayes’ Rule and its differ­ent forms -wants: Bayes’ Rule and its im­pli­ca­tions c: I want the ba­sics, but I’m also in­ter­ested in read­ing more about the the­o­ret­i­cal im­pli­ca­tions and the rea­sons why Bayes’ rule is con­sid­ered so im­por­tant. wants: Bayes’ Rule and its im­pli­ca­tions -wants: Bayes’ Rule and its differ­ent forms d: I’d like to read ev­ery­thing! I want to have a deep the­o­ret­i­cal and prac­ti­cal un­der­stand­ing of the Bayes’ rule. wants: Bayes’ Rule and its differ­ent forms,Bayes’ Rule and its im­pli­ca­tions ]

wants-req­ui­site(Bayes’ Rule and its differ­ent forms): %%wants-req­ui­site(Bayes’ Rule and its im­pli­ca­tions): %box: Your path will go over all forms of Baye’s Rule, along with de­vel­op­ing deep ap­pre­ci­a­tion for its sci­en­tific use­ful­ness. Your path will con­tain 12 pages:

  • Fre­quency di­a­grams: A first look at Bayes

  • Water­fall di­a­grams and rel­a­tive odds

  • In­tro­duc­tion to Bayes’ rule: Odds form

  • Bayes’ rule: Pro­por­tional form

  • Ex­traor­di­nary claims re­quire ex­traor­di­nary evidence

  • Or­di­nary claims re­quire or­di­nary evidence

  • Bayes’ rule: Log-odds form

  • Shift to­wards the hy­poth­e­sis of least surprise

  • Bayes’ rule: Vec­tor form

  • Belief re­vi­sion as prob­a­bil­ity elimination

  • Bayes’ rule: Prob­a­bil­ity form

  • Bayesian view of sci­en­tific virtues %

%start-path(Com­pre­hen­sive guide to Bayes’ Rule)% %% %%!wants-req­ui­site(Bayes’ Rule and its im­pli­ca­tions): %box: No time to waste! Let’s plunge di­rectly into a sin­gle-page ab­bre­vi­ated in­tro­duc­tion to Bayes’ rule. % %% <div>

!wants-req­ui­site(Bayes’ Rule and its differ­ent forms): %%wants-req­ui­site(Bayes’ Rule and its im­pli­ca­tions): %box: Your path will teach you the ba­sic odds form of Bayes’ rule at a rea­son­able pace and then delve into the deep mys­ter­ies of the Bayes’ Rule! Your path will con­tain 8 pages:

  • Fre­quency di­a­grams: A first look at Bayes

  • Water­fall di­a­grams and rel­a­tive odds

  • In­tro­duc­tion to Bayes’ rule: Odds form

  • Belief re­vi­sion as prob­a­bil­ity elimination

  • Ex­traor­di­nary claims re­quire ex­traor­di­nary evidence

  • Or­di­nary claims re­quire or­di­nary evidence

  • Shift to­wards the hy­poth­e­sis of least surprise

  • Bayesian view of sci­en­tific virtues %

%start-path(Bayes’ Rule and its im­pli­ca­tions)% %%

%%!wants-req­ui­site(Bayes’ Rule and its im­pli­ca­tions): %box: Your path will teach you the ba­sic odds form of Bayes’ rule at a rea­son­able pace. It will con­tain 3 pages:

  • Fre­quency di­a­grams: A first look at Bayes

  • Water­fall di­a­grams and rel­a­tive odds

  • In­tro­duc­tion to Bayes’ rule: Odds form %

%start-path(In­tro­duc­tion to Bayes’ Rule odds form)% %% <div>

Children:

Parents:

  • Bayes' rule

    Bayes’ rule is the core the­o­rem of prob­a­bil­ity the­ory say­ing how to re­vise our be­liefs when we make a new ob­ser­va­tion.

    • Bayesian reasoning

      A prob­a­bil­ity-the­ory-based view of the world; a co­her­ent way of chang­ing prob­a­bil­is­tic be­liefs based on ev­i­dence.