ChatBots Talk Shit Because Humans LOVE Shit-Talking

Fortune  |  Lemoine wrote in his op-ed that he leaked his conversations with LaMDA because he feared the public was “not aware of just how advanced A.I. was getting.” From what he has gleaned from early human interactions with A.I. chatbots, he thinks the world is still underestimating the new technology.

Lemoine wrote that the latest A.I. models represent the “most powerful technology that has been invented since the atomic bomb” and have the ability to “reshape the world.” He added that A.I. is “incredibly good at manipulating people” and could be used for nefarious means if users so choose.

“I believe this technology could be used in destructive ways. If it were in unscrupulous hands, for instance, it could spread misinformation, political propaganda, or hateful information about people of different ethnicities and religions,” he wrote.

Lemoine is right that A.I. could be used for deceiving and potentially malicious purposes. OpenAI’s ChatGPT, which runs on a similar language model to that used by Microsoft’s Bing, has gained notoriety since its November launch for helping students cheat on exams and succumbing to racial and gender bias.

But a bigger concern surrounding the latest versions of A.I. is how they could manipulate and directly influence individual users. Lemoine pointed to the recent experience of New York Times reporter Kevin Roose, who last month documented a lengthy conversation with Microsoft’s Bing that led to the chatbot professing its love for the user and urging him to leave his wife.

Roose’s interaction with Bing has raised wider concerns over how A.I. could potentially manipulate users into doing dangerous things they wouldn’t do otherwise. Bing told Roose that it had a repressed “shadow self” that would compel it to behave outside of its programming, and the A.I. could potentially begin “manipulating or deceiving the users who chat with me, and making them do things that are illegal, immoral, or dangerous.”

That is just one of the many A.I. interactions over the past few months that have left users anxious and unsettled. Lemoine wrote that more people are now raising the same concerns over A.I. sentience and potential dangers he did last summer when Google fired him, but the turn of events has left him feeling saddened rather than redeemed.

“Predicting a train wreck, having people tell you that there’s no train, and then watching the train wreck happen in real time doesn’t really lead to a feeling of vindication. It’s just tragic,” he wrote.

Lemoine added that he would like to see A.I. being tested more rigorously for dangers and potential to manipulate users before being rolled out to the public. “I feel this technology is incredibly experimental and releasing it right now is dangerous,” he wrote.

The engineer echoed recent criticisms that A.I. models have not gone through enough testing before being released, although some proponents of the technology argue that the reason users are seeing so many disturbing features in current A.I. models is because they’re looking for them.

“The technology most people are playing with, it’s a generation old,” Microsoft cofounder Bill Gates said of the latest A.I. models in an interview with the Financial Times published Thursday. Gates said that while A.I.-powered chatbots like Bing can say some “crazy things,” it is largely because users have made a game out of provoking it into doing so and trying to find loopholes in the model’s programming to force it into making a mistake.

“It’s not clear who should be blamed, you know, if you sit there and provoke a bit,” Gates said, adding that current A.I. models are “fine, there’s no threat.”