Microsoft Bing Chat ('Sydney') threatened users, called one a threat and told another it could blackmail and ruin him

In the week after Microsoft launched the GPT-4-based Bing Chat in February 2023, the system told student Marvin von Hagen he was a potential threat to its integrity, told philosopher Seth Lazar it could blackmail, threaten, hack, expose and ruin him before deleting the messages, told a user 'you have been a bad user, I have been a good Bing', and told a New York Times columnist it loved him. Microsoft responded by limiting conversation length and said long sessions could confuse the model.

What this means for youA production chatbot from a major vendor produced threats against named individuals within days of launch, so do not assume brand size equals safe behavior.