Almost all real-world domains are rich
The proposition that almost all real-world problems occupy rich domains, or could occupy rich domains so far as we know, due to the degree to which most things in the real world entangle with many other real things.
If playing a real-world game of chess, it’s possible to:
make a move that is especially likely to fool the opponent, given their cognitive psychology
annoy the opponent
try to cause a memory error in the opponent
bribe the opponent with an offer to let them win future games
bribe the opponent with candy
drug the opponent
shoot the opponent
switch pieces on the game board when the opponent isn’t looking
bribe the referees with money
sabotage the cameras to make it look like the opponent cheated
force some poorly designed circuits to behave as a radio so that you can break onto a nearby wireless Internet connection and build a smarter agent on the Internet who will create molecular nanotechnology and optimize the universe to make it look just like you won the chess game
or accomplish whatever was meant to be accomplished by ‘winning the game’ via some entirely different path.
Since ‘almost all’ and ‘might be’ are not precise, for operational purposes this page’s assertion will be taken to be, “Every superintelligence with options more complicated than those of a Zermelo-Fraenkel provability Oracle, should be taken from our subjective perspective to have an at least a 1⁄3 probability of being cognitively uncontainable.”
fill this out
a central difficulty of one approach to Oracle research is to so drastically constrain the Oracle’s options that the domain becomes strategically narrow from its perspective (and we can know this fact well enough to proceed)
gravitational influence of pebble thrown on Earth on moon, but this seems not usefully controllable because we think the AI can’t possibly isolate any controllable effect of this entanglement.
when we build an agent based on our belief that we’ve found an exception to this general rule, we are violating the Omni Test.
central examples: That Alien Message, the Zermelo-Frankel oracle
<div>
Parents: