Issue 2026-09-21 · Industry · 研究 · 安全
Decontamination reports can't fix benchmark contamination, evaluator must own the test
A write-up argues lab-run decontamination reports cannot fix benchmark contamination: labs check themselves, corpora contain copyrighted works that can't be disclosed, and paraphrase, forum, GitHub, or synthetic-derived data evade matching. It proposes evaluators own the test, submissions never see labels, evaluation runs offline and is rebuilt from a named commit, and where possible test data is generated after submissions freeze, with only reproduced results counting.
Read original ↗- Moonshot's open-weight model tops OpenAI on financial research benchmark
- Report: closed models dominate price-performance frontier as open share hits low
- Australian universities expand ChatGPT access to build AI-native graduates
- Franklin Templeton taps OpenAI for investment research
- Cult.fit uses Codex to boost operational efficiency