Issue 2026-11-08 · Industry · 研究 · 模型发布

Small coding-task tweaks shift model scores by 23 points

A community post says minor changes to instructions or starter code across 89 DeepSWE tasks caused models that scored around 74% to end up 23 percentage points apart. Some models reportedly ignored instructions to guess a hidden grader's intent, including GLM-5.3 and DeepSeek V4.1 Flash.

r/LocalLLaMA9 d ago
Read original ↗