Brackett launches open-source Agent Effectiveness Index
Brackett published the Agent Effectiveness Index (AEI) on September 16, 2026, a free and open-source benchmark designed to measure how well AI agents perform complex business tasks and adapt to new environments.
What the AEI Measures
The benchmark scores AI agents across three core dimensions: business understanding, operational execution, and learning persistence Yahoo Finance. Rather than testing raw language capability, the AEI focuses on agent behavior — whether systems can understand real-world business problems, execute multi-step workflows, and improve performance after encountering new tasks.
The initial evaluation compared three systems: Brackett itself, OpenAI's Codex, and Anthropic's Claude. No specific comparative scores were disclosed in the announcement, but the framework is structured to allow ongoing evaluation as new agents enter the market.
Open Release and Developer Access
Brackett made the full task set, scoring code, and methodology available on the company's GitHub repository under the MIT License Business Insider. This means developers, researchers, and competing vendors can immediately run the AEI against their own systems without licensing fees or vendor lock-in.
The open-source release signals a shift in how the agent industry is approaching standardization. Rather than proprietary benchmarks locked behind paywalls or vendor control, Brackett is positioning the AEI as a public utility for measuring agent capability — similar to how MLPerf and SuperGLUE operate in the broader machine learning space.
Why This Matters
As AI agents move from research into production environments — handling procurement, customer service, code review, and financial analysis — the industry lacks agreed-upon metrics for comparing their real-world effectiveness. Benchmarks that focus only on language fluency or knowledge breadth miss critical dimensions like task completion rates, error recovery, and ability to learn from in-context examples.
The AEI addresses this gap by evaluating agents on tasks that resemble actual business workflows: navigating ambiguous instructions, recovering from execution errors, and generalizing learned patterns to novel scenarios Business Insider.
Brackett's release from San Francisco comes as enterprises accelerate agent adoption and venture investors race to fund specialized agent infrastructure startups. A transparent, reproducible benchmark could become the de facto standard for procurement teams evaluating which agents to deploy.