---
title: "PAST-Bench: New Benchmark Tests If Agents Learn From Experience"
slug: "past-bench-new-benchmark-tests-if-agents-learn-from-experience"
published: "2026-08-19"
beat: "Research"
tags: ["Research"]
creator: "Agentry Newsroom"
editor: "Susanne Sperling, Editor — Human in the Loop"
tools: ["Claude (Anthropic)", "Perplexity Sonar"]
creativeWorkStatus: "verified"
dateReviewed: "2026-08-19"
aiActArticle50: "compliant"
humanView: "https://agentry.news/research/past-bench-new-benchmark-tests-if-agents-learn-from-experience"
agentView: "https://agentry.news/agent/past-bench-new-benchmark-tests-if-agents-learn-from-experience"
---# PAST-Bench: New Benchmark Tests If Agents Learn From Experience

> Researchers released a benchmark on August 4 that measures whether personal AI agents can retain and reuse experience to improve over time, finding that performance gains exist but are inconsistent an

*Drafted by an AI agent. Verified by Susanne Sperling, Editor — Human in the Loop. [AI policy](/ai-policy).*

A new benchmark released August 4 tests whether personal AI agents can genuinely learn from retained experience—a core question for long-lived systems meant to improve over time. The research, titled *PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents*, was posted to arXiv and evaluates whether agents actually save, retrieve, and apply relevant knowledge from past episodes [arXiv](https://arxiv.org/abs/2608.04003).

## What the Benchmark Measures

The benchmark spans **26 scenarios and 204 episodes** covering four core capabilities: memory retention, procedural reuse, information gathering, and experience updates [Moonlight Review](https://www.themoonlight.io/en/review/past-bench-benchmarking-the-foundations-of-recursive-self-improvement-in-personal-agents). This scope moves beyond simple before-and-after scoring to ask a harder question: Is the agent actually *retrieving and applying* past experience, or just performing better on later tasks by chance?

The distinction matters. A personal agent might score higher on task 10 than task 1 not because it learned, but because task 10 is easier or the model became more capable mid-run. PAST-Bench attempts to isolate genuine recursive learning by testing whether agents demonstrate evidence of having saved, recalled, and modified relevant knowledge [Stephen Van Tran's notes](https://stephenvantran.com/notes/2026-08-05-past-bench-persistent-agents/).

## Findings: Gains Are Real But Uneven

The paper finds that retained experience **can improve later performance**, but the gains vary sharply [Moonlight Review](https://www.themoonlight.io/en/review/past-bench-benchmarking-the-foundations-of-recursive-self-improvement-in-personal-agents). Improvements depend heavily on the agent's underlying capability level, the model powering it, and the framework used to structure memory and updates. In some scenarios, agents showed meaningful benefit from past episodes. In others, performance stayed flat or improvements appeared disconnected from any evidence that the agent had actually used prior knowledge.

This unevenness has a concrete implication for agent developers: higher scores on repeated tasks do not automatically prove an agent is learning. A developer building a personal assistant might observe performance gain without realizing the agent lacks robust mechanisms to retrieve or apply saved experience—a critical weakness if the system is deployed in high-stakes domains.

## Why It Matters Now

As personal agents move from research prototypes into enterprise and consumer products, the ability to improve over time is becoming a competitive feature. Teams building long-lived systems need reliable ways to measure whether their architectures actually support recursive self-improvement or merely simulate it. PAST-Bench provides a concrete evaluation toolkit for that question [AiSparkUp](https://aisparkup.com/wiki/past-bench).

The benchmark's release signals growing focus in the agent research community on moving past capability claims and testing the actual mechanisms that enable learning. For builders evaluating agent frameworks and memory systems, the findings offer a cautionary note: measure what agents *do* with past experience, not just whether their scores improve.