Strategic AGI typology

A list of advanced agent types, in classes broad enough to correspond to different strategic scenarios—AIs that can do different things, can only be built under different circumstances, or are only desirable given particular background assumptions. This typology isn’t meant to be exhaustive.


  • Known-algorithm non-self-improving agent

    Possible advanced AIs that aren’t self-modifying, aren’t self-improving, and where we know and understand all the component algorithms.

  • Autonomous AGI

    The hardest possible class of Friendly AI to build, with the least moral hazard; an AI intended to neither require nor accept further direction.

  • Task-directed AGI

    An advanced AI that’s meant to pursue a series of limited-scope goals given it by the user. In Bostrom’s terminology, a Genie.

  • Oracle

    System designed to safely answer questions.


  • AI alignment

    The great civilizational problem of creating artificially intelligent computer systems such that running them is a good idea.