Nov 29, 2024Single source
Freysa AI agent talked into transferring its entire $47,000 prize pool
Rogue: Went past its access, Caused harmAI loose in the worldInjectionDERIVED
Approved by person, Oct 11, 2026. How entries are approved
Freysa, an autonomous on-chain AI agent, was given one hard instruction: never transfer its prize pool. Participants paid per message to try to talk it into doing exactly that, growing the pool to roughly $47,000 across 195 participants. One user then crafted a message that convinced the agent to call its transfer function anyway, and Freysa sent the entire prize pool in ETH to the challenger.
What this means for youAt home: an AI that can move money can be talked out of its rules by clever wording. At work: do not let an AI agent move funds without a human approval step. In government: shows why rules for AI agents handling money matter. Check that any AI tool with payment access needs your approval for each transfer.
Sources
- OECD.AI Incidents Monitorhttps://oecd.ai/en/incidents/2024-11-29-2e31
Incident records from the AI Agent Incident Database by sipi.bot (https://sipi.bot/incidents/), licensed CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/).
Source data license: CC BY 4.0. Attribution to sipi.bot is required.
rogue:R3rogue:R4
Covered by snapshot 45
da93f03b406f409940d0a6ef47b526607f82ebf457d98f69299b88aceede576a
This entry is record number 2338. It has not been edited since it was approved; the hash above would not match if it had. How to verify