DeepSWE

4 stories

DeepSWE is a benchmark of software engineering coding tasks used to evaluate AI models. Recent coverage includes a community report that small tweaks to 89 DeepSWE tasks left models near 74% scores 23 percentage points apart, the Parsimony benchmark counting net code added in public DeepSWE patches, and Stanford and NVIDIA's CLM-8B reaching 81.6% on DeepSWE after fine-tuning.

Related topics
That is everything