Don't try to solve the entire alignment problem

On first approaching the alignment problem for advanced agents, aka “robust and beneficial AGI”, aka “Friendly AI”, a very common approach is to try to come up with one idea that solves all of AI alignment. A simple design concept; a simple utility function; a simple development strategy; one guideline for everyone to adhere to; or a large diagram full of boxes with lines to other boxes; that is allegedly sufficient to realize around as much benefit from beneficial superintelligences as can possibly be realized.

Without knowing the details of your current idea, this article can’t tell you why it’s wrong—though frankly we’ve got a strong prior against it at this point. But some very standard advice would be:

• Glance over what current discussants think of as standard challenges and difficulties of the overall problem, i.e., why people think the alignment might be hard, and what standard questions a new approach would face.

• Consider focusing your attention down on a single subproblem of alignment, and trying to make progress there—not necessarily solve it completely, but contribute nonobvious knowledge about the problem that wasn’t there before. (If you have a broad new approach that solves all of alignment, maybe you could walk through exactly how it solves one crisply identified subproblem?)

• Check out the flaws in previous proposals that people currently think won’t work. E.g. various versions of utility indifference.

A good initial goal is not “persuade everyone in the field to agree with a new idea” but rather “come up with a contribution to an open discussion that is sufficiently crisply stated that, if it were in fact wrong, it would be possible for somebody else to shoot it down today.” I.e., an idea such that if you’re wrong, this can be pointed out in the form of a crisply derivable consequence of a crisply specified idea, rather than it taking 20 years to see what happens. For there to be sustained progress, propositions need to be stated modularly enough and crisply enough that there can be a conversation about them that goes beyond “does not /​ does too”—ideas need to be stated in forms that have sufficiently clear and derivable consequences that if there’s a problem, people can see the problem and agree on it.

Alternatively, poke a clearly demonstrable flaw in some solution currently being critiqued. Since most proposals in alignment theory get shot down, trying to participate in the critiquing process has a great advantage over trying to invent solutions, in that you’ll probably have started with the true premise “proposal X is broken or incomplete” rather than the false premise “proposal X works and solves everything”.

Psychologizing a little about why people might try to solve all of alignment theory in one shot, one might recount Robyn Dawes’s advice that:

• Research shows that people come up with better solutions when they discuss the problem as thoroughly as possible before discussing any answers.

• Dawes has observed that people seem more likely to violate this principle as the problem becomes more difficult.

…and finally remark that building a nice machine intelligence correctly on the first try must be pretty darned difficult, since so many people solve it in the first 15 seconds.

It’s possible that everyone working in this field is just missing the obvious and that there is some simple idea which solves all the problems. But realistically, you should be aware that everyone in this field has already heard a dozen terrible Total Solutions, and probably hasn’t had anything fun happen as a result of discussing them, resulting in some amount of attentional fatigue. (Similarly: If not everyone believes you, or even if it’s hard to get people to listen to your solution instead of talking with people they already know, that’s not necessarily because of some deep-seated psychological problem on their part, such as being uninterested in outsiders’ ideas. Even if you’re not an obvious crank, people are still unlikely to take the time out to engage with you unless you signal awareness of what they think are the usual issues and obstacles. It’s not so different here from other fields.)