GPT-5.2

Ai2 study finds LLM novelty judges unreliable for research ideas

Ai2 released "Old Ideas, Novel Problems," testing six LLM judges—GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5 and Claude Opus 4.6—on rating research idea novelty. Small changes to evaluation setup shifted verdicts sharply; the team released a benchmark, data and code.