Properties of the logarithm

With a solid in­ter­pre­ta­tion of log­a­r­ithms un­der our belt, we are now in a po­si­tion to look at the ba­sic prop­er­ties of the log­a­r­ithm and un­der­stand what they are say­ing. The defin­ing char­ac­ter­is­tic of log­a­r­ithm func­tions is that they are real val­ued func­tions \(f\) such that

Prop­erty 1: Mul­ti­ply­ing in­puts adds out­puts.

$$ f(x \cdot y) = f(x) + f(y) \tag{1} $$
for all \(x, y \in\) \(\mathbb R^+\). This says that when­ever the in­put grows (or shrinks) by a fac­tor of \(y\), the out­put goes up (or down) by only a fixed amount, which de­pends on \(y\). In fact, equa­tion (1) alone tells us quite a bit about the be­hav­ior of \(f,\) and from it, we can al­most guaran­tee that \(f\) is a log­a­r­ithm func­tion. First, let’s see how far we can get us­ing equa­tion (1) all by it­self:

Prop­erty 2: 1 is mapped to 0.

$$ f(1) = 0. \tag{2} $$

This says that the amount the out­put changes if the in­put grows by a fac­tor of 1 is zero — i.e., the out­put does not change if the in­put changes by a fac­tor of 1. This is ob­vi­ous, as “the in­put changed by a fac­tor of 1” means “the in­put did not change.”

Ex­er­cise: Prove (2) from (1).

$$f(x) = f(x \cdot 1) = f(x) + f(1),\text{ so }f(1) = 0.$$

Prop­erty 3: Re­cip­ro­cat­ing the in­put negates the out­put.

$$ f(x) = -f\left(\frac{1}{x}\right). \tag{3} $$

This says that the way that grow­ing the in­put by a fac­tor of \(x\) changes the out­put is ex­actly the op­po­site from the way that shrink­ing the in­put by a fac­tor of \(x\) changes the out­put. In terms of the “com­mu­ni­ca­tion cost” in­ter­pre­ta­tion, if dou­bling (or tripling, or \(n\)-times-ing) the pos­si­bil­ity space in­creases costs by \(c\), then halv­ing (or third­ing, or \(n\)-parts-ing) the space de­creases costs by \(c.\)

Ex­er­cise: Prove (3) from (2) and (1).

$$x \cdot \frac{1}{x} = 1,\text{ so }f(1) = f\left(x \cdot \frac{1}{x}\right) = f(x) + f\left(\frac{1}{x}\right).$$

$$f(1)=0,\text{ so }f(x)\text{ and }f\left(\frac{1}{x}\right)\text{ must be opposites.}$$

Prop­erty 4: Di­vid­ing in­puts sub­tracts out­puts.

$$ f\left(\frac{x}{y}\right) = f(x) - f(y). \tag{4} $$

This fol­lows im­me­di­ately from (1) and (3).

Ex­er­cise: Give an in­ter­pre­ta­tion of (4).

There are at least two good in­ter­pre­ta­tions:

  1. \(f\left(x \cdot \frac{1}{y}\right) = f(x) - f(y),\) i.e., shrink­ing the in­put by a fac­tor of \(y\) is the op­po­site of grow­ing the in­put by a fac­tor of \(y.\)

  2. \(f\left(z \cdot \frac{x}{y}\right) = f(z) + f(x) - f(y),\) i.e., grow­ing the in­put by a fac­tor of \(\frac{x}{y}\) af­fects the out­put just like grow­ing the in­put by \(x\) and then shrink­ing it by \(y.\)

Try trans­lat­ing these into the com­mu­ni­ca­tion cost in­ter­pre­ta­tion if it is not clear why they’re true. <div><div>

Prop­erty 5: Ex­po­nen­ti­at­ing the in­put mul­ti­plies the out­put.

$$ f\left(x^n\right) = n \cdot f(x). \tag{5} $$

This says that mul­ti­ply­ing the in­put by \(x\), \(n\) times in­curs \(n\) iden­ti­cal changes to the out­put. In terms of the com­mu­ni­ca­tion cost metaphor, this is say­ing that you can em­u­late an \(x^n\) digit us­ing \(n\) differ­ent \(x\)-digits.

Ex­er­cise: Prove (5).

This is easy to prove when \(n \in \mathbb N:\)
$$f\left(x^n\right) = f(\underbrace{x \cdot x \cdot \ldots x}_{n\text{ times}}) = \underbrace{f(x) + f(x) + \ldots f(x)}_{n\text{ times}} = n \cdot f(x).$$

For \(n \in \mathbb Q,\) this is a bit more difficult; we leave it as an ex­er­cise to the reader. Hint: Use the proof of (6) be­low, for \(n \in \mathbb N,\) to boot­strap up to the case where \(n \in \mathbb Q.\)

For \(n \in \mathbb R,\) this is ac­tu­ally not prov­able from (1) alone; we need an ad­di­tional as­sump­tion (such as con­ti­nu­ity) on \(f\). <div><div>

Prop­erty 5 is ac­tu­ally false, in full gen­er­al­ity — it’s pos­si­ble to cre­ate a func­tion \(f\) that obeys (1), and obeys (5) for \(n \in \mathbb Q,\) but which ex­hibits patholog­i­cal be­hav­ior on ir­ra­tional num­bers. For more on this, see patholog­i­cal near-log­a­r­ithms.

This is the first place that prop­erty (1) fails us: 5 is true for \(n \in \mathbb Q,\) but if we want to guaran­tee that it’s true for \(n \in \mathbb R,\) we need \(f\) to be con­tin­u­ous, i.e. we need to en­sure that if \(f\) fol­lows 5 on the ra­tio­nals it’s not al­lowed to do any­thing in­sane on ir­ra­tional num­bers only.

Prop­erty 6: Root­ing the in­put di­vides the out­put.

$$ f(\sqrt[n]{x}) = \frac{f(x)}{n}. \tag{6} $$

This says that, to change the out­put one \(n\)th as much as you would if you mul­ti­plied the in­put by \(x\), mul­ti­ply the in­put by the \(n\)th root of \(x\). See Frac­tional digits for a phys­i­cal in­ter­pre­ta­tion of this fact.

Ex­er­cise: Prove (6).

$$(\sqrt[n]{x})^n = x,\text{ so }f\left((\sqrt[n]{x})^n\right)\text{ has to equal }f(x).$$

$$f\left((\sqrt[n]{x})^n\right) = n \cdot f(\sqrt[n]{x}),\text{ so }f(\sqrt[n]{x}) = \frac{f(x)}{n}.$$

As with (5), (6) is always true if \(n \in \mathbb Q,\) but not nec­es­sar­ily always true if \(n \in \mathbb R.\) To prove (6) in full gen­er­al­ity, we ad­di­tion­ally re­quire that \(f\) be con­tin­u­ous.

Prop­erty 7: The func­tion is ei­ther triv­ial, or sends some in­put to 1.

$$ \text{Either $f$ sends all inputs to $0$, or there exists a $b \neq 1$ such that $f(b)=1.$}\tag{7} $$

This says that ei­ther \(f\) is very bor­ing (and does noth­ing re­gard­less of its in­puts), or there is some par­tic­u­lar fac­tor \(b\) such that when the in­put changes by a fac­tor of \(b\), the out­put changes by ex­actly \(1\). In the com­mu­ni­ca­tion cost in­ter­pre­ta­tion, this says that if you’re mea­sur­ing com­mu­ni­ca­tion costs, you’ve got to pick some unit (such as \(b\)-digits) with which to mea­sure.

Ex­er­cise: Prove (7).

Sup­pose \(f\) does not send all in­puts to \(0\), and let \(x\) be an in­put that \(f\) sends to some \(y \neq 0.\) Then \(f(\sqrt[y]{x}) = \frac{f(x)}{y} = 1.\)%%note: You may be won­der­ing, “what if \(y\) is nega­tive, or a frac­tion?” If so, see Strange roots. Short ver­sion: \(\sqrt[-3/4]{x}\) is perfectly well-defined.%%

\(b\) is \(\sqrt[y]{x}.\) We know that \(b \neq 1\) be­cause \(f(b) = 1\) whereas, by (2), \(f(1) = 0\). <div><div>

Prop­erty 8: If the func­tion is con­tin­u­ous, it is ei­ther triv­ial or a log­a­r­ithm.

$$ \text{If $f(b)=1$ then } f(b^x) = x. \tag{8} $$

This prop­erty fol­lows im­me­di­ately from (5). Thus, (8) is always true if \(x\) is a ra­tio­nal, and if \(f\) is con­tin­u­ous then it’s also true when \(x\) is ir­ra­tional.

Prop­erty (8) states that if \(f\) is non-triv­ial, then it in­verts ex­po­nen­tials with base \(b.\) In other words, \(f\) counts the num­ber of \(b\)-fac­tors in \(x\). In other words, \(f\) counts how many times you need to mul­ti­ply \(1\) by \(b\) to get \(x\). In other words, \(f = \log_b\)!

Many texts take (8) to be the defin­ing char­ac­ter­is­tic of the log­a­r­ithm. As we just demon­strated, one can also define log­a­r­ithms by (1) as con­tin­u­ous non-triv­ial func­tions whose out­puts grow by a con­stant (that de­pends on \(y\)) when­ever their in­puts grow by a fac­tor of \(y\). All other prop­er­ties of the log­a­r­ithm fol­low from that.

If you want to re­move the “con­tin­u­ous” qual­ifier, you’re still fine as long as you stick to ra­tio­nal in­puts. If you want to re­move the “non-triv­ial” qual­ifier, you can in­ter­pret the func­tion \(z\) that sends ev­ery­thing to zero as \(\log_\infty\). Allow­ing \(\log_\infty\) and re­strict­ing our­selves to ra­tio­nal in­puts, ev­ery func­tion \(f\) that satis­fies equa­tion (1) is iso­mor­phic to a log­a­r­ithm func­tion.

In other words, if you find a func­tion whose out­put changes by a con­stant (that de­pends on \(y\)) when­ever its in­put grows by a fac­tor of \(y\), there is ba­si­cally only one way it can be­have. Fur­ther­more, that func­tion only has one de­gree of free­dom — the choice of \(b\) such that \(f(b)=1.\) As we will see next, even that de­gree of free­dom is rather paltry: All log­a­r­ithm func­tions be­have in es­sen­tially the same way. As such, if we find any \(f\) such that \(f(x \cdot y) = f(x) + f(y)\) (or any phys­i­cal pro­cess well-mod­eled by such an \(f\)), then we im­me­di­ately know quite a bit about how \(f\) be­haves.