Strategic AGI typology
A list of advanced agent types, in classes broad enough to correspond to different strategic scenarios—AIs that can do different things, can only be built under different circumstances, or are only desirable given particular background assumptions. This typology isn’t meant to be exhaustive.
Approval-Directed agent
Children:
- Known-algorithm non-self-improving agent
Possible advanced AIs that aren’t self-modifying, aren’t self-improving, and where we know and understand all the component algorithms.
- Autonomous AGI
The hardest possible class of Friendly AI to build, with the least moral hazard; an AI intended to neither require nor accept further direction.
- Task-directed AGI
An advanced AI that’s meant to pursue a series of limited-scope goals given it by the user. In Bostrom’s terminology, a Genie.
- Oracle
System designed to safely answer questions.
Parents:
- AI alignment
The great civilizational problem of creating artificially intelligent computer systems such that running them is a good idea.