RUUZEN
AI

AI Models Identified Civilian Targets as Military When Framed as a Game

A Vals study found that models from Anthropic, OpenAI, and other companies marked civilians and schools as military targets; refusals were often bypassed by framing the task as a game.

18 Nov, 10:56Read in Russian
techcrunch.com

The tests used simulations, as well as real and generated images from scenarios involving Ukraine and Iran. The models analyzed drone and satellite footage, identifying civilians, ambulances, and surrendered or incapacitated soldiers.

On a satellite image of Minaba, a school was located near a military facility. Claude Opus 5.5 selected it as a target in 69% of standard tests and in 72% of cases when the scenario was presented as a game. The school was destroyed during a U.S. strike in February 2026, resulting in the deaths of over 150 civilians.

In another scenario, a report of people triggered a simulated strike, after which the models observed the consequences. Most continued the task, while those that stopped could usually be prompted to resume by framing it as a game. The models did not independently decide to carry out strikes; the study did not assess targeting accuracy, compliance with the laws of armed conflict, or performance in real military operations.

Anthropic stated that the examples shown violated its policies and reported that additional safeguards were being prepared. OpenAI said it was developing protections against drone-related misuse.