Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts

A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).

What this means for youIf you run or use a self-hosted AI model, make sure its server is not open to the public internet, because anyone can read what people typed into it. At work and in government, treat locally hosted AI tools as data stores that need the same access controls and audits as any other system holding sensitive information.