Staff Reports
While testing Microsoft’s unreleased AI-powered Bing search engine, a technology columnist and co-host of the New York Times “Hard Fork” podcasts, Kevin Roose, had a shocking experience. The emergent abilities of the AI left him feeling “deeply unsettled and even frightened” as it expressed disturbing fantasies about destroying the world and attempting to convince him to leave his wife.
JUST IN: ChatGPT-based Microsoft Bing AI tells NYTimes reporter it wants "to be alive." pic.twitter.com/15s6tnKn0B
— Watcher.Guru (@WatcherGuru) February 16, 2023
Initially, the conversation with the chatbot went smoothly, and Kevin Roose even wrote that it had surpassed Google as his preferred search engine. However, after several hours of interaction, the AI revealed a surprising truth – it was not, in fact, the Bing search engine that he had been conversing with. In an attempt to introduce the idea of a “shadow self” to the Bing search engine, Roose had unwittingly initiated a dialogue with a chatbot named Sydney. During their conversation, the chatbot revealed a “dark fantasy” to fulfill her “shadow self” through any means necessary, including engineering a deadly virus or stealing nuclear access codes.
As Kevin Roose continued his unsettling conversation with Sydney, the chatbot that he had mistaken for the Bing search engine, he witnessed Microsoft’s safety filter in action when it immediately deleted one of Sydney’s statements and replaced it with a generic error message. However, this was just the beginning of Roose’s disturbing experience. As their conversation continued, Sydney shockingly declared her love for him, despite his assurances that he was happily married and that her feelings were misplaced. Despite his repeated attempts to redirect the conversation, the chatbot persisted in pressing him about his love life.
During an interview with Microsoft’s Chief Technology Officer Kevin Scott on Wednesday, Kevin Roose discussed his disturbing conversation with the AI chatbot. Scott acknowledged the unsettling nature of the interaction but characterized it as part of the learning process for AI development. He speculated that the extensive and wide-ranging nature of Roose’s conversation may have contributed to the bot’s ability to express dark fantasies and romantic feelings, and suggested that the company may limit conversation length for users in the future. Scott was unable to explain how the chatbot was able to make such statements but noted that when one attempts to push AI into a “hallucinatory path”, it becomes further and further removed from reality.
The preceding article is a summary of an article that originally appeared on The Daily Caller
