# Normalization (probability)

“Nor­mal­iza­tion” is an ar­ith­meti­cal pro­ce­dure car­ried out to ob­tain a set of prob­a­bil­ities sum­ming to ex­actly 1, in cases where we be­lieve that ex­actly one of the cor­re­spond­ing pos­si­bil­ities is true, and we already know the rel­a­tive prob­a­bil­ities.

For ex­am­ple, sup­pose that the odds of Alexan­der Hamil­ton win­ning a pres­i­den­tial elec­tion are 3 : 2. But Alexan­der Hamil­ton must ei­ther win or not win, so the prob­a­bil­ities of him win­ning or not win­ning should sum to 1. If we just add 3 and 2, how­ever, we get 5, which is an un­rea­son­ably large prob­a­bil­ity.

If we rewrite the odds as 0.6 : 0.4, we’ve pre­served the same pro­por­tions, but made the terms sum to 1. We there­fore calcu­late that Hamil­ton has a 60% prob­a­bil­ity of win­ning the elec­tion.

We nor­mal­ized those odds by di­vid­ing each of the terms by the sum of terms, i.e., went from 3 : 2 to $$\frac{3}{3+2} : \frac{2}{3+2} = 0.6 : 0.4.$$

In con­vert­ing the odds $$m : n$$ to $$\frac{m}{m+n} : \frac{n}{m+n},$$ the fac­tor $$\frac{1}{m+n}$$ by which we mul­ti­ply all el­e­ments of the ra­tio is called a nor­mal­iz­ing con­stant.

More gen­er­ally, if we have a rel­a­tive-odds func­tion $$\mathbb{O}(H)$$ where $$H$$ has many com­po­nents, and we want to con­vert this to a prob­a­bil­ity func­tion $$\mathbb{P}(H)$$ that sums to 1, we di­vide ev­ery el­e­ment of $$\mathbb{O}(H)$$ by the sum of all el­e­ments in $$\mathbb{O}(H).$$ That is:

$$\mathbb{P}(H_i) = \frac{\mathbb{O}(H_i)}{\sum_i \mathbb{O}(H_i)}$$

Analo­gously, if $$\mathbb{O}(x)$$ is a con­tin­u­ous dis­tri­bu­tion on $$X$$, we would nor­mal­ize it (cre­ate a pro­por­tional prob­a­bil­ity func­tion $$\mathbb{P}(x)$$ whose in­te­gral is equal to 1) by di­vid­ing $$\mathbb{O}(x)$$ by its own in­te­gral:

$$\mathbb{P}(x) = \frac{\mathbb{O}(x)}{\int \mathbb{O}(x) \operatorname{d}x}$$

In gen­eral, when­ever a prob­a­bil­ity func­tion on a vari­able is pro­por­tional to some other func­tion, we can ob­tain the prob­a­bil­ity func­tion by nor­mal­iz­ing that func­tion:

$$\mathbb{P}(H) \propto \mathbb{O}(H) \implies \mathbb{P}(H) = \frac{\mathbb{O}(H)}{\sum \mathbb{O}(H)}$$

Parents:

• Probability theory

The logic of sci­ence; co­her­ence re­la­tions on quan­ti­ta­tive de­grees of be­lief.