Show me what you've broken

See AI safety mind­set. If you want to demon­strate com­pe­tence at com­puter se­cu­rity, cryp­tog­ra­phy, or AI al­ign­ment the­ory, you should first think in terms of ex­pos­ing tech­ni­cally demon­stra­ble flaws in ex­ist­ing solu­tions, rather than solv­ing en­tire prob­lems your­self. Rele­vant Bruce Sch­neier quotes: “Good en­g­ineer­ing in­volves think­ing about how things can be made to work; the se­cu­rity mind­set in­volves think­ing about how things can be made to fail” and “Any­one can in­vent a se­cu­rity sys­tem that he him­self can­not break. Show me what you’ve bro­ken to demon­strate that your as­ser­tion of the sys­tem’s se­cu­rity means some­thing.”

Parents:

  • AI safety mindset

    Ask­ing how AI de­signs could go wrong, in­stead of imag­in­ing them go­ing right.