Ideal target
The ‘ideal target’ of a meta-utility function \(\Delta U\) which behaves as if a ground-level utility function \(U\) is taking on different values in different possible worlds, is the value of \(U\) in the actual world; or the expected value of \(U\) after updating on all possible accessible evidence. If chocolate has €8 utility in worlds where the sky is blue, and €5 utility in worlds where the sky is not blue, then in the AI’s ‘ideal target’ utility function, the utility of chocolate is €8.
Parents:
- Moral uncertainty
A meta-utility function in which the utility function as usually considered, takes on different values in different possible worlds, potentially distinguishable by evidence.