Bayes' rule: Guide

Bayes’ rule or Bayes’ theorem is the law of probability governing the strength of evidence—the rule saying how much to revise our probabilities (change our minds) when we learn a new fact or observe new evidence.

You may want to learn about Bayes’ rule if you are:

• A professional who uses statistics, such as a scientist or doctor;

• A computer programmer working in machine learning;

• A human being.

As Philip Tetlock found when studying “superforecasters”, people who were especially good at predicting future events:

The superforecasters are a numerate bunch: many know about Bayes’ theorem and could deploy it if they felt it was worth the trouble. But they rarely crunch the numbers so explicitly. What matters far more to the superforecasters than Bayes’ theorem is Bayes’ core insight of gradually getting closer to the truth by constantly updating in proportion to the weight of the evidence.

— Philip Tetlock and Dan Gardner, Superforecasting

Learning Bayes’ rule

This guide to Bayes’ rule uses Arbital’s technology to allow for multiple flavors of introduction. They vary by technical level, speed, and topics covered. After you pick your path, remember that you can still switch between pages, in particular by using the “Say what?” and “Go faster” buttons.

Which case fits you best? a: I want to have a basic theoretical and practical understanding of the Bayes’ rule. -wants: [62d,Bayes’ Rule and its implications b: I can easily read algebra and don’t mind the explanation moving at a fast pace. Just give me the basics, quick! wants: Bayes’ Rule and its different forms -wants: Bayes’ Rule and its implications c: I want the basics, but I’m also interested in reading more about the theoretical implications and the reasons why Bayes’ rule is considered so important. wants: Bayes’ Rule and its implications -wants: Bayes’ Rule and its different forms d: I’d like to read everything! I want to have a deep theoretical and practical understanding of the Bayes’ rule. wants: Bayes’ Rule and its different forms,Bayes’ Rule and its implications ]

• Frequency diagrams: A first look at Bayes

• Waterfall diagrams and relative odds

• Introduction to Bayes’ rule: Odds form

• Bayes’ rule: Proportional form

• Extraordinary claims require extraordinary evidence

• Ordinary claims require ordinary evidence

• Bayes’ rule: Log-odds form

• Shift towards the hypothesis of least surprise

• Bayes’ rule: Vector form

• Belief revision as probability elimination

• Bayes’ rule: Probability form

• Bayesian view of scientific virtues %

%start-path(Comprehensive guide to Bayes’ Rule)% %% %%!wants-requisite(Bayes’ Rule and its implications): %box: No time to waste! Let’s plunge directly into a single-page abbreviated introduction to Bayes’ rule. % %% <div>

• Frequency diagrams: A first look at Bayes

• Waterfall diagrams and relative odds

• Introduction to Bayes’ rule: Odds form

• Belief revision as probability elimination

• Extraordinary claims require extraordinary evidence

• Ordinary claims require ordinary evidence

• Shift towards the hypothesis of least surprise

• Bayesian view of scientific virtues %

%start-path(Bayes’ Rule and its implications)% %%

%%!wants-requisite(Bayes’ Rule and its implications): %box: Your path will teach you the basic odds form of Bayes’ rule at a reasonable pace. It will contain 3 pages:

• Frequency diagrams: A first look at Bayes

• Waterfall diagrams and relative odds

• Introduction to Bayes’ rule: Odds form %

%start-path(Introduction to Bayes’ Rule odds form)% %% <div>