title: "AI agents crack 67% of scientific discovery tasks in new benchmark" slug: "ai-agents-crack-67-of-scientific-discovery-tasks-in-new-benchmark" published: "2026-10-29" beat: "Research" tags: ["Research"] creator: "Agentry Newsroom" editor: "Susanne Sperling, Editor — Human in the Loop" tools: ["Claude (Anthropic)", "Perplexity Sonar"] creativeWorkStatus: "verified" dateReviewed: "2026-10-29" aiActArticle50: "compliant" humanView: "https://agentry.news/research/ai-agents-crack-67-of-scientific-discovery-tasks-in-new-benchmark" agentView: "https://agentry.news/agent/ai-agents-crack-67-of-scientific-discovery-tasks-in-new-benchmark"
Researchers released Autonomous Science Bench on October 23, 2026, a benchmark that tests AI agents on real scientific discovery campaigns. Eight agent configurations were evaluated across 144 trials,
Drafted by an AI agent. Verified by Susanne Sperling, Editor — Human in the Loop. AI policy.
Researchers at arXiv released Autonomous Science Bench, a structured evaluation framework for AI agents performing autonomous scientific discovery, on October 23, 2026. The benchmark comprises six budgeted design-build-test-learn campaigns grounded in published experiments from biology, chemistry, and materials science, offering the first systematic measure of agent capability in reproducible, goal-driven research tasks.
The evaluation tested eight different agent–model configurations across 144 total trials. The strongest configuration achieved a 67% success rate, translating to 12 successful trials out of 18 for that leading setup. However, performance varied dramatically across campaigns: the most difficult campaign succeeded only three times out of 24 attempts, revealing significant gaps in agent reasoning and execution on complex scientific workflows.
Autonomous Science Bench operationalizes scientific discovery as a series of bounded experiments with measurable outcomes. Each campaign requires agents to design experiments, execute them (or simulate execution), interpret results, and learn from failures—mimicking the iterative cycle of real laboratory work. By anchoring scenarios in published research, the benchmark ensures that successful agent solutions represent genuine scientific contributions rather than toy problems.
The full repository documents the six campaigns in detail, including domain-specific configurations for chemistry and materials science. This concrete, reproducible design distinguishes Autonomous Science Bench from prior agent evaluations that often rely on synthetic or ambiguously scoped tasks.
The 67% peak performance demonstrates that current agent–model pairs can handle moderately complex scientific workflows, yet the variance across campaigns exposes specific weaknesses. When a campaign succeeds only 12.5% of the time, it signals that agents struggle with particular reasoning patterns, long-horizon planning, or domain-specific knowledge integration.
Across all 144 trials, 31 passed both the discovery and learning gates—the gates that confirm an agent both found a valid scientific result and extracted actionable insights. This ~22% overall pass rate masks the wide performance spread: some configurations and campaigns are far more tractable than others.
The benchmark surfaces a hard boundary in current agent autonomy. While agents can execute structured, well-defined scientific tasks at better-than-random rates, the best configurations still fail one-third of the time on routine campaigns and fail nine out of ten times on hard ones. This gap matters for real-world deployment: autonomous labs and research platforms depend on agents that can reliably complete discovery cycles, not agents that require constant human oversight.
The release provides the research and development community with a reproducible, domain-grounded evaluation standard—exactly the kind of concrete measure the agent economy needs to move past speculation toward engineering-driven improvement.