Who Masters These Technologies In Some Ways Will Be Master Of The World

Vox  |   In an economic race with enormous winner-takes-all stakes, a company is primarily thinking about whether to deploy their system before a competitor. Slowing down for safety checks risks that someone else will get there first. In geopolitical AI arms race scenarios, the fear is that China will get to AI before the US and have an incredibly powerful weapon — and that, in anticipation of that, the US may push its own unready systems into widespread deployment.

Even if alignment is a very solvable problem, trying to do complex technical work on incredibly powerful systems while everyone is in a rush to beat a competitor is a recipe for failure.

Some actors working on artificial general intelligence, or AGI, have planned significantly to avoid this dangerous trap: OpenAI, for instance, has terms in its charter specifically aimed at preventing an AI race once systems are powerful enough: “We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.”

I am generally optimistic about human nature. No one actively wants to deploy a system that will kill us all, so if we can get good enough visibility into the problem of alignment, then it’ll be clear to engineers why they need a solution. But eager declarations that the race is on make me nervous.

Another great part of human nature is that we are often incredibly competitive — and while that competition can lead to great advancements, it can also lead to great destruction. It’s the Cold War that drove the space race, but it was also WWII that drove the creation of the atomic bomb. If winner-takes-all competition is the attitude we bring to one of the most powerful technologies in human history, I don’t think humanity is going to win out.