ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Code

T. Hua, H. Hua, Violet Xiang, B. Klieger, S. Truong, W. Liang, F.-Y. Sun, N. Haber.

NeurIPS 2025 Datasets & Benchmarks Track · Spotlight

An execution harness for 212 research-code challenges from 20 recent ML papers. Across 32 evaluated models, the strongest reached 33.0% on the HARD split, and 43 tasks remained unsolved by every model.