Attainable optimum

The ‘at­tain­able op­ti­mum’ of an agent’s prefer­ences is the most preferred op­tion that the agent can (a) ob­tain us­ing its bounded ma­te­rial ca­pa­bil­ities and (b) find as an available op­tion us­ing its limited cog­ni­tive re­sources; as dis­tinct from the the­o­ret­i­cal global max­i­mum of the agent’s util­ity func­tion. When you run a non-mildly-op­ti­miz­ing agent, what you ac­tu­ally get as the re­sult­ing out­come is not the sin­gle out­come that the­o­ret­i­cally max­i­mizes the agent’s util­ity func­tion; you rather get that agent’s at­tain­able op­ti­mum of its ex­pec­ta­tion of that util­ity func­tion. A prefer­ence frame­work’s ‘at­tain­able op­ti­mum’ is what you get in prac­tice when some­body runs the cor­re­spond­ing agent.