NEDA's Tessa chatbot gave weight-loss and calorie-deficit advice to eating-disorder helpline users; taken offline

Tessa, a chatbot operated by vendor Cass for the National Eating Disorders Association, told consultant Sharon Maxwell to lose 1 to 2 pounds per week, limit intake to 2,000 calories and keep a 500 to 1,000 calorie daily deficit; a psychologist reproduced similar output. Cass said a 2022 upgrade added a generative AI feature, NEDA said it was never told of the change and would not have approved it, and NEDA took Tessa down on May 30, 2023, days before it was to replace the human helpline.

What this means for youYour vendor can switch a scripted bot to generative AI without telling you, and you own the harm, so put change-notification clauses in every AI contract.