Sufficiently advanced Artificial Intelligence
A ‘sufficiently advanced Artificial Intelligence’ is a cognitive agent or cognitive algorithm with capabilities great enough that we need to think about it in a qualitatively different way from robotic cars; a machine intelligence smart enough that we need to start doing AI alignment theory to it.
For example, we probably don’t need to worry about an AI that tries to prevent us from pressing its off-switch, until the AI knows that (a) it has an off-switch and (b) pressing the off-switch will prevent the AI from achieving its other goals.
In turn, this knowledge is a special case of Big-picture strategic awareness, which might follow from learning many facts about many domains via Artificial General Intelligence.
The page on advanced agent properties starts to list out some of the different ways that an AI could be ‘smart enough’ in this sense, along with the particular problems that might be encountered with an AI that smart.
Parents:
- Advanced agent properties
How smart does a machine intelligence need to be, for its niceness to become an issue? “Advanced” is a broad term to cover cognitive abilities such that we’d need to start considering AI alignment.