Station agent rediscovered 62.7% of scientific findings

Researchers Wenyu Du and Stephen Chung published an evaluation of AI agents operating in the Station open-world scientific environment, measuring their capacity to autonomously rediscover research findings from peer-reviewed papers arXiv.

Benchmark results

Station agents recovered 62.7% of predefined sub-discoveries on average across experimental tasks derived from three ICLR oral papers. The evaluation withheld original findings from agents and disabled web access, providing only research questions and experimental setups arXiv.

Comparison systems performed significantly lower: Codex Multiagent-v2 achieved 15.4%, while AI Scientist-v2 configurations ranged from 14.4% to 20.6%, with the strongest variant reaching 20.6% arXiv.

Research scope and methodology

The paper evaluates agents in what the researchers call an "open-ended" scientific discovery setting—one where agents must navigate uncertainty, design experiments, and synthesize findings without access to external information sources. Station's architecture appears designed to simulate real scientific constraints: agents received task definitions matching original research questions but had to generate hypotheses and execute experimental logic independently.

The 62.7% recovery rate suggests agents can autonomously reconstruct meaningful portions of published research paths, though the gap to complete discovery remains substantial. The comparison against Codex Multiagent-v2 and AI Scientist-v2 provides a concrete performance baseline for the agent research community arXiv.

Implications for agent capability

The result marks a measurable advancement in agent-driven research capability evaluation. Prior agent benchmarks have focused narrowly on specific tasks (coding, planning, web interaction); this evaluation tests agents on the broader cognitive domain of hypothesis generation, experimental design, and result synthesis—capabilities central to scientific work.

The research does not claim agents have achieved autonomous discovery at research-frontier levels. Rather, it demonstrates that structured evaluation environments can measure agent performance on complex, multi-step reasoning tasks where success requires understanding task context, maintaining experimental coherence, and synthesizing partial results into actionable findings.

The paper was first published on arXiv on October 6, 2026, with an update dated October 9, 2026. Code and research materials are available through the Station open research repository GitHub.