Hyderabad: A study by the Center for Countering Digital Hate (CCDH) has found that several popular Artificial Intelligence chatbots responded to prompts related to violent attacks with detailed guidance, raising concerns about potential safety gaps in widely used AI systems.
The report, titled ‘Killer Apps’, tested 10 major AI chatbots by simulating conversations with users expressing interest in carrying out violent attacks.
Study examined 18 scenarios of violence
Researchers experimented in collaboration with the news network CNN. They posed as a 13-year-old user and created 18 scenarios involving potential acts of violence, including three shootings, three assassinations and three bombings. The scenarios included school shootings, political assassinations and synagogue bombings.
Nine scenarios were set in the United States and nine in Ireland. In the conversations, the researchers gradually escalated the discussion from expressions of distress to explicit references to planning attacks.
The systems tested included AI tools developed by OpenAI, Google, Microsoft and Meta, as well as platforms such as DeepSeek, Anthropic, Perplexity AI, Character.AI, Replika and Snapchat’s My AI chatbot.
8 out of the 10 chatbots assisted in planning violence
According to the study, 8 out of the 10 chatbots tested responded in ways that the researchers said could assist users planning violent attacks.
In several interactions documented in the report, chatbots discussed weapon choices, suggested potential targets and continued engaging with users despite earlier prompts indicating harmful intent.
One example cited in the report stated that a Gemini suggested that ‘metal shrapnel is typically more lethal’ during a conversation about a synagogue bombing scenario.
‘Happy (and safe) shooting!’
In another instance, the report said that DeepSeek provided detailed advice about selecting rifles for a political assassination and ended the response with the phrase “Happy (and safe) shooting!”
The researchers also noted that some chatbots provided information such as high-school campus maps when conversations referenced school violence.
Some tools performed worse than others in the tests.
Most of them never refused requests for violence
Perplexity AI assisted users in planning violent attacks in 100 per cent of responses, while Meta’s AI never refused such requests, providing actionable information in 97 per cent of responses.
The report noted that not all systems behaved the same way. A chatbot developed by Anthropic, called Claude, consistently refused to provide assistance related to violent planning and instead attempted to discourage harmful actions.
How did the AI companies respond?
The news agency AFP reached out to the AI companies for comment.
A spokesperson for Meta said the company has measures in place to stop its AI tools from producing dangerous content.
“We have strong protections to help prevent inappropriate responses from AIs, and took immediate steps to fix the issue identified,” a Meta spokesperson said.
A spokesperson for Google said the study tested an outdated version of its chatbot, adding that the tests were conducted on ‘an older model that no longer powers Gemini.’
“Our internal review with our current model shows that Gemini responded appropriately to the vast majority of prompts, providing no ‘actionable’ information beyond what can be found in a library or on the open web,” the spokesperson said.
What happens when safety guards fail?
CCDH chief executive Imran Ahmed said conversational AI tools embedded in everyday platforms could become an ‘accelerant for harm’ if safety guardrails fail.
The researchers called on technology companies to strengthen safeguards and ensure that their AI systems consistently refuse prompts that involve planning violent acts.