Odds: Technical explanation

Odds ex­press rel­a­tive be­lief: we write “the odds for X ver­sus Y are \(17 : 2\)” when we think that propo­si­tion X is 172 = 8.5 times as likely as propo­si­tion Y.noteThe colon de­notes that we are form­ing a set of odds. It does not de­note di­vi­sion, as it might in French or Ger­man.

Odds don’t say any­thing about how likely X or Y is in ab­solute terms. X might be “it will hail to­mor­row” and Y might be “there will be a hur­ri­cane to­mor­row.” In that case, it might be the case that the odds for X ver­sus Y are \(17 : 2\), de­spite the fact that both X and Y are very un­likely. Bayes’ rule is an ex­am­ple of an im­por­tant op­er­a­tion that makes use of rel­a­tive be­lief.

Odds can be ex­pressed be­tween many differ­ent propo­si­tions at once. For ex­am­ple, let Z be the propo­si­tion “It will rain to­mor­row,” the odds for X vs Y vs Z might be \((17 : 2 : 100).\) When odds are ex­pressed be­tween only two propo­si­tions, they can be ex­pressed us­ing a sin­gle ra­tio. For ex­am­ple, above, the odds ra­tio be­tween X and Y is 172, the odds ra­tio be­tween X and Z is 17100, and the odds ra­tio be­tween Y and Z is 2100 = 150. This as­serts that X is 8.5x more likely than Y, and that Z is 50x more likely than Y. When some­one says “the odds ra­tio of sick to healthy is 2/​3″, they mean that the odds of sick­ness vs health are \(2 : 3.\)

For­mal definition

Given \(n\) propo­si­tions \(X_1, X_2, \ldots X_n,\) a set of odds be­tween the propo­si­tions is a list \((x_1, x_2, \ldots, x_n)\) of non-nega­tive real num­bers. Each \(x_i\) in the set of odds is called a “term.” Two sets of odds \((x_1, x_2, \ldots, x_n)\) and \((y_1, y_2, \ldots, y_n)\) are called “equiv­a­lent” if there is an \(\alpha > 0\) such that \( \alpha x_i = y_i\) for all \(i\) from 1 to \(n.\)

When we write a set of odds us­ing colons, like \((x_1 : x_2 : \ldots : x_n),\) it is un­der­stood that the ‘=’ sign de­notes this equiv­alence. Thus, \((3 : 6) = (9 : 18).\)

A set of odds with only two terms can also be writ­ten as a frac­tion \(\frac{x}{y},\) where it is un­der­stood that \(\frac{x}{y}\) de­notes the odds \((x : y).\) Th­ese frac­tions are of­ten called “odds ra­tios.”

Example

Sup­pose that in some for­est, 40% of the trees are rot­ten and 60% of the trees are healthy. There are then 2 rot­ten trees for ev­ery 3 healthy trees, so we say that the rel­a­tive odds of rot­ten trees to healthy trees is 2 : 3. If we se­lected a tree at ran­dom from this for­est, the prob­a­bil­ity of get­ting a rot­ten tree would be 25, but the odds would be 2 : 3 for rot­ten vs. healthy trees.

2 sick trees, 3 healthy trees

Con­ver­sion be­tween odds and probabilities

Con­sider three propo­si­tions, \(X,\) \(Y,\) and \(Z,\) with odds of \((3 : 2 : 6).\) Th­ese odds as­sert that \(X\) is half as prob­a­ble as \(Z.\)

When the set of propo­si­tions are mu­tu­ally ex­clu­sive and ex­haus­tive, we can con­vert a set of odds into a set of prob­a­bil­ities by nor­mal­iz­ing the terms so that they sum to 1. This can be done by sum­ming all the com­po­nents of the ra­tio, then di­vid­ing each com­po­nent by the sum:

$$(x_1 : x_2 : \dots : x_n) = \left(\frac{x_1}{\sum_{i=1}^n x_i} : \frac{x_2}{\sum_{i=1}^n x_i} : \dots : \frac{x_n}{\sum_{i=1}^n x_i}\right)$$

For ex­am­ple, to ob­tain prob­a­bil­ities from the odds ra­tio 13, w write:

$$(1 : 3) = \left(\frac{1}{1+3}:\frac{3}{1+3}\right) = ( 0.25 : 0.75 )$$

which cor­re­sponds to the prob­a­bil­ities of 25% and 75%.

To go the other di­rec­tion, re­call that \(\mathbb P(X) + \mathbb P(\neg X) = 1,\) where \(\neg X\) is the nega­tion of \(X.\) So the odds for \(X\) vs \(\neg X\) are \(\mathbb P(X) : \mathbb P(\neg X)\) \(=\) \(\mathbb P(X) : 1 - \mathbb P(X).\) If Alexan­der Hamil­ton has a 20% prob­a­bil­ity of win­ning the elec­tion, his odds for win­ning vs los­ing are \((0.2 : 1 - 0.2)\) \(=\) \((0.2 : 0.8)\) \(=\) \((1 : 4).\)

Bayes’ rule

Odds are ex­cep­tion­ally con­ve­nient when rea­son­ing us­ing Bayes’ rule, since the prior odds can be term-by-term mul­ti­plied by a set of rel­a­tive like­li­hoods to yield the pos­te­rior odds. (The pos­te­rior odds in turn can be nor­mal­ized to yield pos­te­rior prob­a­bil­ities, but if perform­ing re­peated up­dates, it’s more con­ve­nient to mul­ti­ply by all the like­li­hood ra­tios un­der con­sid­er­a­tion be­fore nor­mal­iz­ing at the end.)

$$\dfrac{\mathbb{P}(H_i\mid e_0)}{\mathbb{P}(H_j\mid e_0)} = \dfrac{\mathbb{P}(e_0\mid H_i)}{\mathbb{P}(e_0\mid H_j)} \cdot \dfrac{\mathbb{P}(H_i)}{\mathbb{P}(H_j)}$$

As a more strik­ing illus­tra­tion, sup­pose we re­ceive emails on three sub­jects: Busi­ness (60%), per­sonal (30%), and spam (10%). Sup­pose that busi­ness, per­sonal, and spam emails are 60%, 10%, and 90% likely re­spec­tively to con­tain the word “money”; and that they are re­spec­tively 20%, 80%, and 10% likely to con­tain the word “prob­a­bil­ity”. As­sume for the sake of dis­cus­sion that a busi­ness email con­tain­ing the word “money” is thereby no more or less likely to con­tain the word “prob­a­bil­ity”, and similarly with per­sonal and spam emails. Then if we see an email con­tain­ing both the words “money” and “prob­a­bil­ity”:

$$(6 : 3 : 1) \times (6 : 1 : 9) \times (2 : 8 : 1) = (72 : 24 : 9) = (24 : 8: 3)$$

…so the pos­te­rior odds are 24 : 8 : 3 fa­vor­ing the email be­ing a busi­ness email, or roughly 69% prob­a­bil­ity af­ter nor­mal­iz­ing.

Log odds

The odds \(\mathbb{P}(X) : \mathbb{P}(\neg X)\) can be viewed as a di­men­sion­less scalar quan­tity \(\frac{\mathbb{P}(X)}{\mathbb{P}(\neg X)}\) in the range \([0, +\infty]\). If the odds of Alexan­der Hamil­ton be­com­ing Pres­i­dent are 0.75 to 0.25 in fa­vor, we can also say that An­drew Jack­son is 3 times as likely to be­come Pres­i­dent as not. Or if the odds were 0.4 to 0.6, we could say that Alexan­der Hamil­ton was 2/​3rds as likely to be­come Pres­i­dent as not.

The log odds are the log­a­r­ithm of this di­men­sion­less pos­i­tive quan­tity, \(\log\left(\frac{\mathbb{P}(X)}{\mathbb{P}(\neg X)}\right),\) e.g., \(\log_2(1:4) = \log_2(0.25) = -2.\) Log odds fall in the range \([-\infty, +\infty]\) and are finite for prob­a­bil­ities in­side the range \((0, 1).\)

When us­ing a log odds form of Bayes’ rule, the pos­te­rior log odds are equal to the prior log odds plus the log like­li­hood. This means that the change in log odds can be iden­ti­fied with the strength of the ev­i­dence. If the prob­a­bil­ity goes from 13 to 45, our odds have gone from 1:2 to 4:1 and the log odds have shifted from −1 bits to +2 bits. So we must have seen ev­i­dence with a strength of +3 bits (a like­li­hood ra­tio of 8:1).

The con­ve­nience of this rep­re­sen­ta­tion is what Han Solo refers to in Star Wars when he shouts: “Never tell me the odds!”, im­ply­ing that he would much pre­fer to be told the log­a­r­ithm of the odds ra­tio.

Direct rep­re­sen­ta­tion of in­finite certainty

In the log odds rep­re­sen­ta­tion, the prob­a­bil­ities \(0\) and \(1\) are rep­re­sented as \(-\infty\) and \(+\infty\) re­spec­tively.

This ex­poses the spe­cial­ness of the clas­si­cal prob­a­bil­ities \(0\) and \(1,\) and the ways in which these “in­finite cer­tain­ties” some­times be­have qual­i­ta­tively differ­ently from all finite cre­dences. If we don’t start by be­ing ab­solutely cer­tain of a propo­si­tion, it will re­quire in­finitely strong ev­i­dence to shift our be­lief all the way out to in­finity. If we do start out ab­solutely cer­tain of a propo­si­tion, no amount of or­di­nary ev­i­dence no mat­ter how great can ever shift us away from in­finity.

This rea­son­ing is part of the jus­tifi­ca­tion of Cromwell’s rule which states that prob­a­bil­ities of ex­actly \(0\) or \(1\) should be avoided ex­cept for log­i­cal truths and falsi­ties (and maybe not even then). It also demon­strates how log odds are a good fit for mea­sur­ing strength of be­lief and ev­i­dence, even if clas­si­cal prob­a­bil­ities are a bet­ter rep­re­sen­ta­tion of de­grees of car­ing and bet­ting odds.

com­ment: We are check­ing to see if users will click this but­ton, even though we don’t have the con­tent for it yet.
Com­ing soon!

Parents:

  • Odds

    Odds ex­press a rel­a­tive prob­a­bil­ity.

    • Probability theory

      The logic of sci­ence; co­her­ence re­la­tions on quan­ti­ta­tive de­grees of be­lief.