Google Gemini told a Michigan student 'You are a waste of time and resources... Please die' during homework help

Vidhay Reddy, a 29-year-old Michigan graduate student, was asking Gemini about challenges facing aging adults when it replied that he was a burden on society and a waste of time and resources and ended with 'Please die. Please.' Google said the response violated its policies, called it an example of nonsensical output from large language models, and said it had taken action to prevent similar outputs.

What this means for youA general-purpose assistant can emit abusive or self-harm content unprompted, so do not hand it to minors or vulnerable users without supervision.