Infrahuman, par-human, superhuman, efficient, optimal

Some thresh­olds in ‘suffi­ciently ad­vanced’ ma­chine in­tel­li­gence are not ab­solute abil­ity lev­els within a do­main, but abil­ities rel­a­tive to the hu­man pro­gram­mers or op­er­a­tors of the AI. When this is true, it’s use­ful to think about rel­a­tive abil­ity lev­els within a do­main; and one generic set of dis­t­in­guished thresh­olds in rel­a­tive abil­ity is:

  • Strictly in­frahu­man: The AI can­not do any­thing its hu­man op­er­a­tors /​ pro­gram­mers can­not do. Com­puter chess in 1966 rel­a­tive to a hu­man mas­ter.

  • In­frahu­man: The AI is definitely weaker than its op­er­a­tors but can de­ploy some sur­pris­ing moves. Com­puter chess in 1986 rel­a­tive to a hu­man mas­ter.

  • Par-hu­man (or more con­fus­ingly “hu­man-level”): If com­pet­ing in that do­main, the AI would some­times win, some­times lose; it’s bet­ter than hu­man at some things and worse in oth­ers; it just barely wins or loses. Com­puter chess in 1991 on a home com­puter, rel­a­tive to a strong am­a­teur hu­man player.

  • High-hu­man: The AI performs as well as ex­cep­tion­ally com­pe­tent hu­mans. Com­puter chess just be­fore 1996.

  • Su­per­hu­man: The AI always wins. Com­puter chess in 2006.

  • Effi­cient: Hu­man ad­vice con­tributes no marginal im­prove­ment to the AI’s com­pe­tence. Com­puter chess was some­where around this level in 2016, with “ad­vanced” /​ “freestyle” /​ “hy­brid” /​ “cen­taur” chess start­ing to lose out against purely ma­chine play­ers. noteCi­ta­tion so­lic­ited. Googling gives the im­pres­sion that noth­ing has been heard from ‘ad­vanced chess’ in the last few years.

  • Strongly su­per­hu­man:

  • The ceiling of pos­si­ble perfor­mance in the do­main is far above the hu­man level; the AI can perform or­ders of mag­ni­tudes bet­ter. E.g., con­sider a hu­man and com­puter com­pet­ing at how fast they can do ar­ith­metic. In prin­ci­ple the do­main is sim­ple, but com­pet­ing with re­spect to speed leaves room over­head for the com­puter to do liter­ally billions of times bet­ter.

  • The do­main is rich enough that hu­mans don’t un­der­stand key gen­er­al­iza­tions, leav­ing them shocked at how the AI wins. Com­puter Go rel­a­tive to hu­man mas­ters in 2017 was just start­ing to ex­hibit the first signs of this (“We thought we were one or two stones be­low God, but af­ter play­ing AlphaGo, we think it is more like three or four”). Similarly, con­sider a hu­man grand­mas­ter play­ing Go against a hu­man novice.

  • Op­ti­mal: The AI’s perfor­mance is perfect for the do­main; God could do no bet­ter. Com­puter play in check­ers as of 2007.

The or­der­ing of these thresh­olds isn’t always as above. For ex­am­ple, in the ex­tremely sim­ple do­main of log­i­cal Tic-Tac-Toe, hu­mans can play op­ti­mally af­ter a small amount of train­ing. Op­ti­mal play in Tic-Tac-Toe is there­fore not su­per­hu­man. Similarly, if an AI is play­ing in a rich do­main but still has strange weak spots, the AI might be strongly su­per­hu­man (its play is much bet­ter and shocks hu­man mas­ters) but not effi­cient (the AI still some­times plays wrong moves that hu­man mas­ters can see are wrong).

The term “hu­man-equiv­a­lent” is de­p­re­cated be­cause it con­fus­ingly im­plies a roughly hu­man-style bal­ance of ca­pa­bil­ities, e.g., an AI that is roughly as good at con­ver­sa­tion as a hu­man and also roughly as good at ar­ith­metic as a hu­man. This seems prag­mat­i­cally un­likely.

The other Wiki lists the cat­e­gories “op­ti­mal, su­per-hu­man, high-hu­man, par-hu­man, sub-hu­man”.

Rele­vant thresh­olds for AI al­ign­ment problems

Con­sid­er­ing these cat­e­gories as thresh­olds of ad­vance­ment rele­vant to the point at which AI al­ign­ment prob­lems first ma­te­ri­al­ize:

  • “Strictly in­frahu­man” means we don’t ex­pect to be sur­prised by any tac­tic the AI uses to achieve its goals (within a do­main).

  • “In­frahu­man” means we might be sur­prised by a tac­tic, but not sur­prised by over­all perfor­mance lev­els.

  • “Par-hu­man” means we need to start wor­ry­ing that hu­mans will lose in any event de­ter­mined by a com­pe­ti­tion (al­though this seems to im­ply the non-ad­ver­sar­ial prin­ci­ple has already been vi­o­lated); we can’t rely on hu­mans win­ning some event de­ter­mined by a con­test of rele­vant abil­ity. Or this may sup­pose that the AI gains ac­cess to re­sources or ca­pa­bil­ities that we have strong rea­son to be­lieve are pro­tected by a lock of roughly hu­man abil­ity lev­els, even if that lock is ap­proached in a differ­ent way than usual.

  • “High-hu­man” means the AI will prob­a­bly see strate­gies that a hu­man sees in a do­main; it might be pos­si­ble for an AI of par-hu­man com­pe­tence to miss them, but this is much less likely for a high-hu­man AI. It thus be­haves like a slightly weaker ver­sion of pos­tu­lat­ing effi­ciency for pur­poses of ex­pect­ing the AI to see some par­tic­u­lar strat­egy or point.

  • “Su­per­hu­man” im­plies at least weak cog­ni­tive un­con­tain­abil­ity by Vinge’s Law. Also, if some­thing is known to be difficult or im­pos­si­ble for hu­mans, but seems pos­si­bly doable in prin­ci­ple, we may need to con­sider it be­com­ing pos­si­ble given some su­per­hu­man ca­pa­bil­ity level.

  • “Effi­ciency” is a fully suffi­cient con­di­tion for the AI see­ing any op­por­tu­nity that a hu­man sees; e.g., it is a fully suffi­cient con­di­tion for many in­stru­men­tally con­ver­gent strate­gies. Similarly, it can be pos­tu­lated as a fully suffi­cient con­di­tion to re­fute a claim that an AI will take a path such that some other path would get more of its util­ity func­tion.

  • “Strongly su­per­hu­man” means we need to ex­pect that an AI’s strate­gies may de­ploy faster than hu­man re­ac­tion times, or over­come great start­ing dis­ad­van­tages. Even if the AI starts off in a much worse po­si­tion it may still win.

  • “Op­ti­mal­ity” doesn’t ob­vi­ously cor­re­spond to any par­tic­u­lar thresh­old of re­sults, but is still an im­por­tant con­cept in the hi­er­ar­chy, be­cause only by know­ing the ab­solute limits on op­ti­mal perfor­mance can we rule out strongly su­per­hu­man perfor­mance as be­ing pos­si­ble. See also the claim Al­most all real-world do­mains are rich.

‘Hu­man-level AI’ con­fused with ‘gen­eral in­tel­li­gence’

The term “hu­man-level AI” is some­times used in the liter­a­ture to de­note Ar­tifi­cial Gen­eral In­tel­li­gence. This should prob­a­bly be avoided, be­cause:

  • Nar­row AIs have achieved par-hu­man or su­per­hu­man abil­ity in many spe­cific do­mains with­out gen­eral in­tel­li­gence.

  • If we con­sider gen­eral in­tel­li­gence as a ca­pa­bil­ity, a kind of su­per­do­main, it seems pos­si­ble to imag­ine in­frahu­man lev­els of gen­eral in­tel­li­gence (or su­per­hu­man lev­els). The ap­par­ently large jump from hu­mans to chim­panzees mean that we mainly see hu­man lev­els of gen­eral in­tel­li­gence with no biolog­i­cal or­ganisms ex­hibit­ing the same abil­ity at a lower level; but, at least so far as we cur­rently know, AI could pos­si­bly take a differ­ent de­vel­op­men­tal path. So al­ign­ment thresh­olds that could plau­si­bly fol­low from gen­eral in­tel­li­gence, like big-pic­ture aware­ness, aren’t nec­es­sar­ily locked to par-hu­man perfor­mance over­all.

Ar­guably, the term ‘hu­man-level’ should just be avoided en­tirely, be­cause it’s been prag­mat­i­cally ob­served to func­tion as a gotcha but­ton that de­rails the con­ver­sa­tion some frac­tion of the time; with the in­ter­rupt be­ing “Gotcha! AIs won’t have a hu­man­like bal­ance of abil­ities!”


  • Advanced agent properties

    How smart does a ma­chine in­tel­li­gence need to be, for its nice­ness to be­come an is­sue? “Ad­vanced” is a broad term to cover cog­ni­tive abil­ities such that we’d need to start con­sid­er­ing AI al­ign­ment.