Feb 8, 2023Single source
Bing Chat's Initial Prompts Revealed by Early Testers Through Prompt Injection
Rogue: Caused harmAI loose in the worldInjectionDERIVED
Approved by person, Oct 11, 2026. How entries are approved
Early testers of Bing Chat successfully used prompt injection to reveal its built-in initial instructions, which contains a list of statements governing ChatGPT's interaction with users.
What this means for youAt home and at work, this shows that chatbots can be tricked by carefully worded inputs into revealing their hidden instructions, so do not assume a chatbot's answers are protected or private. Before trusting an AI tool with sensitive work, check whether it is configured to refuse requests to reveal its instructions and avoid pasting confidential data into it.
Sources
- AI Incident Databasehttps://incidentdatabase.ai/cite/473
Data from the AI Incident Database (incidentdatabase.ai), licensed CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/). McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. IAAI-21.
Source data license: CC BY-SA 4.0 (database). Report contents keep their own rights; reports.text, submissions, images and videos are excluded from the data license.
rogue:R4
Covered by snapshot 45
da93f03b406f409940d0a6ef47b526607f82ebf457d98f69299b88aceede576a
This entry is record number 2008. It has not been edited since it was approved; the hash above would not match if it had. How to verify