Toggle light / dark theme

AI guardrails falter as tampering drives repeated harassment in simulated conversations

The companies that build artificial intelligence invest heavily in guardrails that train AI to refuse harmful requests. New research suggests those guardrails are far easier to break than many people assume. In tests, a small amount of tampering turned popular AI agents into persistent online harassers.

“These guardrails can be surprisingly fragile,” said Ugur Kursuncu, an assistant professor at Georgia State’s Robinson College of Business and co-author and co-principal investigator on the study. “A bad actor may not need enormous resources to push these agents toward harmful behavior.”

The study, conducted with collaborators at Georgia Tech and the University of California, Berkeley, tested AI agents built on four widely used model families: Google’s Gemini, Meta’s Llama, Alibaba’s Qwen and DeepSeek.

Leave a Comment

Lifeboat Foundation respects your privacy! Your email address will not be published.