AI auditors catch most agent misbehavior but produce many false alarms
A study by Anthropic researcher Andy Hall found AI models can flag problematic agent behavior in logs, with GPT-5 catching about 98 percent of reward-hacking cases. However, more exhaustive searches yield more false alarms, so the method is not yet reliable for safety checks.