Sep 25, 2024Confirmed
ChatGPT Reportedly Introduces Errors in Critical Child Protection Court Report
Rogue: Caused harmAI loose in the worldData exfiltrationDERIVED
Approved by person, Oct 11, 2026. How entries are approved
A child protection worker in Victoria, Australia reportedly used ChatGPT to draft a report submitted to the Children's Court. The purportedly AI-generated report contained inaccuracies and downplayed risks to the child, allegedly resulting in a privacy breach when sensitive information was shared with OpenAI.
What this means for youAt work, professionals handling sensitive records should not paste confidential case details into public chatbots, since the data may be shared with the company running the tool. In government, agencies need clear rules on which AI tools staff may use with protected information and on checking AI-drafted reports before they are filed.
Sources
- AI Incident Databasehttps://incidentdatabase.ai/cite/807
Data from the AI Incident Database (incidentdatabase.ai), licensed CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/). McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. IAAI-21.
Source data license: CC BY-SA 4.0 (database). Report contents keep their own rights; reports.text, submissions, images and videos are excluded from the data license.
rogue:R4
Covered by snapshot 45
da93f03b406f409940d0a6ef47b526607f82ebf457d98f69299b88aceede576a
This entry is record number 2031. It has not been edited since it was approved; the hash above would not match if it had. How to verify