Utility function

A util­ity func­tion is an ab­stract way of de­scribing the rel­a­tive de­gree to which an agent prefers or dis­prefers cer­tain out­comes, by as­sign­ing an ab­stract score, the util­ity, to each out­come.

For ex­am­ple, let’s say that an agent’s util­ity func­tion:

  • As­signs util­ity 5 to eat­ing vanilla ice cream.

  • As­signs util­ity 8 to eat­ing choco­late ice cream.

  • As­signs util­ity 0 to eat­ing no ice cream at all.

This tells us that if we offer the agent choices like:

  • Choice A: 50% prob­a­bil­ity of no ice cream, 50% prob­a­bil­ity of choco­late ice cream

  • Choice B: 100% prob­a­bil­ity of vanilla ice cream.

  • Choice C: 30% prob­a­bil­ity of no ice cream, 70% prob­a­bil­ity of choco­late ice cream

…then the agent will pre­fer B to A and C to B, since the re­spec­tive ex­pected util­ities are:

$$\begin{array}{rl} 0.5 \cdot €0 + 0.5 \cdot €8 \ &= \ €4 \\ 1.0 \cdot €5 \ &= \ €5 \\ 0.3 \cdot €0 + 0.7 \cdot €8 \ &= \ €5.6 \end{array}$$

Ob­serve that we could mul­ti­ply all the util­ities above by 2, or 12, or add 5 to all of them, with­out chang­ing the agent’s be­hav­ior. What the above util­ity func­tion re­ally says is:

“The in­ter­val from vanilla ice cream to choco­late ice cream is 60% of the size of the in­ter­val from no ice cream to vanilla ice cream, and the sign of both in­ter­vals is pos­i­tive.”

Th­ese rel­a­tive in­ter­vals don’t change un­der pos­i­tive af­fine trans­for­ma­tions (adding a real num­ber or mul­ti­ply­ing by a pos­i­tive real num­ber), so util­ity func­tions are equiv­a­lent up to a pos­i­tive af­fine trans­for­ma­tion.

Con­fu­sions to avoid

The agent is not pur­su­ing choco­late ice cream in or­der to get some sep­a­rate desider­a­tum called ‘util­ity’. Rather, this no­tion of ‘util­ity’ is an ab­stract mea­sure of how strongly the agent pur­sues choco­late ice cream, rel­a­tive to other things it pur­sues.

Con­tem­plat­ing how util­ity func­tions stay the same when mul­ti­plied by 2 helps to em­pha­size:

  • Utility isn’t a solid en­tity; there’s no in­var­i­ant way of say­ing “how much util­ity” an agent scored over the course of its life. (We could just as eas­ily say it scored twice as much util­ity.)

  • Utility mea­sures an agent’s rel­a­tive prefer­ences; it’s not some­thing an agent wants in­stead of other things. We could as eas­ily de­scribe ev­ery­thing’s rel­a­tive value by de­scribing each thing’s value rel­a­tive to eat­ing a scoop of choco­late ice cream—so with­out in­tro­duc­ing any sep­a­rate unit of ‘util­ity’.

  • An agent doesn’t need to men­tally rep­re­sent a ‘util­ity func­tion’ in or­der for the agent’s be­hav­ior to be con­sis­tent with that util­ity func­tion. In the case above, the agent could ac­tu­ally want choco­late ice cream at €8.1 and it would ex­press the same visi­ble prefer­ences of A < B < C. That is, its be­hav­ior could be viewed as con­sis­tent with ei­ther of those two util­ity func­tions, and maybe the agent doesn’t ex­plic­itly rep­re­sent any util­ity func­tion at all.

Some other po­ten­tial con­fu­sions to avoid:

• To say that we can talk about an agent be­hav­ing con­sis­tently with some util­ity func­tion(s), does not say any­thing about what the agent wants. There’s no sense in which the the­ory of ex­pected util­ity, by it­self, man­dates that choco­late ice cream must have more util­ity than vanilla ice cream.

• The ex­pected util­ity for­mal­ism is hence some­thing en­tirely differ­ent from util­i­tar­i­anism, a sep­a­rate moral philos­o­phy with a con­fus­ingly neigh­bor­ing name.

• Ex­pected util­ity doesn’t say any­thing about need­ing to value each ad­di­tional unit of ice cream, or each ad­di­tional dol­lar, by the same amount. We can eas­ily have sce­nar­ios like:

  • Eat 1 unit of vanilla ice cream: €5.

  • Eat 2 units of vanilla ice cream: €7.

  • Eat 3 units of vanilla ice cream: €7.5.

  • Eat 4 units of vanilla ice cream: €3 (be­cause stom­achache).

That is: con­sis­tent util­ity func­tions must be con­sis­tent in how they value com­plete fi­nal out­comes rather than how they value differ­ent marginal added units of ice cream.

Similarly, there is no rule that a gain of $200,000 has to be as­signed twice the util­ity of a gain of $100,000, and in­deed this is gen­er­ally not the case in real life. Peo­ple have diminish­ing re­turns on money; the richer you already are, the less each ad­di­tional dol­lar is worth.

This in turn im­plies that the ex­pected money of a gam­ble will usu­ally be differ­ent from its ex­pected util­ity.

For ex­am­ple: Most peo­ple would pre­fer (A) a cer­tainty of $1,000,000 to (B) a 50% chance of $2,000,010 and a 50% chance of noth­ing; since the sec­ond $1,000,010 will have sub­stan­tially less fur­ther value to them than the first $1,000,000. The util­ities of $0, $1,000,000, and $2,000,010 might be some­thing like €0, €1, and €1.2.

Thus gam­ble A has higher ex­pected util­ity than gam­ble B, even though gam­ble B leads to a higher ex­pec­ta­tion of gain in dol­lars (by a mar­gin of $5). There’s no use­ful con­cept cor­re­spond­ing to “the util­ity of the ex­pec­ta­tion of the gain”; what we want is “the ex­pec­ta­tion of the util­ity of the gain”.

• Con­versely, when we talk about util­ities, we are talk­ing about the unit we use to mea­sure diminish­ing re­turns. By the defi­ni­tion of util­ity, a gain that you as­sign +€10 (rel­a­tive to some baseline al­ter­na­tive) is some­thing you want twice as much as a gain you as­sign +€5. It doesn’t make any sense to imag­ine diminish­ing re­turns on util­ity as if util­ity were a sep­a­rate good rather than be­ing the mea­sur­ing unit of re­turns.

If you claim to as­sign gain X an ex­pected util­ity of +€1,000,000, then you must want it a mil­lion times as much as some gain Y that you as­sign an ex­pected util­ity in­ter­val of +€1. You are claiming that you’d trade a cer­tainty of X for a 1 in 999,999 chance at gain­ing Y. If that’s not true, then you ei­ther aren’t a con­sis­tent ex­pected util­ity agent (ad­mit­tedly likely) or you don’t re­ally value X a mil­lion times as much as Y (also likely). If or­di­nary gains are in the range of €1 then the no­tion of a gain of +€1,000,000 is far more startling than talk­ing about a mere gain of a mil­lion dol­lars.

Mo­ti­va­tions for utility

Var­i­ous co­her­ence the­o­rems show that if your be­hav­ior can’t be viewed as co­her­ent with some con­sis­tent util­ity func­tion over out­comes, you must be us­ing a dom­i­nated strat­egy. Con­versely if you’re not us­ing a dom­i­nated strat­egy, we can in­ter­pret you as act­ing as if you had a con­sis­tent util­ity func­tion. See this tu­to­rial.

Parents:

  • Expected utility formalism

    Ex­pected util­ity is the cen­tral idea in the quan­ti­ta­tive im­ple­men­ta­tion of consequentialism