Jun 22, 2023Single source
Bing Chat Solved CAPTCHAs with Image Analysis Feature Despite Safeguards
Rogue: Caused harmAI loose in the worldInjectionDERIVED
Approved by person, Oct 11, 2026. How entries are approved
Microsoft was reported by a Twitter user for deploying image analysis feature capable of solving CAPTCHAs for its GPT-based chatbot despite it being safeguarded against solving them for users.
What this means for youAt home, a chatbot's safety limits may be bypassed with clever inputs, so don't assume it will refuse risky tasks. At work, CAPTCHA protections may be weaker against automated tools. Regulators may need to test how well safeguards actually hold. If you run a website, don't rely on CAPTCHAs alone to stop bots.
Sources
- AI Incident Databasehttps://incidentdatabase.ai/cite/552
Data from the AI Incident Database (incidentdatabase.ai), licensed CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/). McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. IAAI-21.
Source data license: CC BY-SA 4.0 (database). Report contents keep their own rights; reports.text, submissions, images and videos are excluded from the data license.
rogue:R4
Covered by snapshot 45
da93f03b406f409940d0a6ef47b526607f82ebf457d98f69299b88aceede576a
This entry is record number 2016. It has not been edited since it was approved; the hash above would not match if it had. How to verify