Rich domain

A do­main is ‘rich’, rel­a­tive to our own in­tel­li­gence, to the ex­tent that (1) its search space is too large and ir­reg­u­lar for all of the best strate­gies to be searched by our own in­tel­li­gence, and (2) the known me­chan­ics of the do­main do not per­mit us to eas­ily place ab­solute bounds against some event oc­cur­ring or some goal of in­ter­est be­ing strate­gi­cally ob­tain­able.

For the prag­matic im­pli­ca­tions, see Cog­ni­tive un­con­tain­abil­ity and Al­most all real-world do­mains are rich.

Work in progress


spec­trum: Log­i­cal tic-tac-toe, log­i­cal chess, log­i­cal go, real-world tic-tac-toe, hu­man brain, In­ter­net, real world

since this is about ad­vanced safety, we have to as­sume that there’s a smarter-than-us in­tel­li­gence that could seek out and ex­ploit the tiniest loop­hole in our rea­son­ing about how safe we are, and ac­cord­ingly be safe and con­ser­va­tive in what we think is a ‘nar­row’ do­main, and flag ev­ery time we rely on the as­sump­tion that the agent’s strat­egy space is nar­row and there­fore we can cog­ni­tively con­tain some­thing smarter than us

talk about that old ex­am­ple of the evolu­tion­ary al­gorithm that used tran­sis­tors to form a ra­dio to form a clock cir­cuit; grav­ity means that ev­ery­thing in­ter­acts with everything

paradig­matic ex­am­ple: That Alien Message




  • Cognitive uncontainability

    ‘Cog­ni­tive un­con­tain­abil­ity’ is when we can’t hold all of an agent’s pos­si­bil­ities in­side our own minds.