Visual Cipher Bench Tests AI Agents' Visual Reasoning
A new benchmark for visual reasoning
Researchers from Navers Lab, Einsia.AI and Tsinghua University posted *Visual Cipher Bench: Benchmarking LLM Agents on Long-Horizon Visual Reasoning* to arXiv on November 12, 2026 (arXiv). The paper's author list includes Deyao Hong, Jiayi Jiang, Yuchun Chen, Yixuan Shi, Wenyi Li, Zheng Jiang, Chengxing Xie, Mingyu Su, Qingle Liu, Mingju Gao, Yizhe Chi, Youjie Zheng, Calvin Xiao, Kaisen Yang and Huan-ang Gao.
The benchmark consists of 40 original visual puzzles spanning four categories: spatial manipulation and simulation, logical constraints and planning, rule induction, and knowledge-based decoding (arXiv). Each puzzle presents one to three black-and-white line drawings, and agents must infer the solving mechanism directly from the images rather than from text instructions. A related project release describes a set of 67 black-and-white illustrations and a single instruction to agents: examine the images and write a final answer to a file (Morningstar/PR Newswire).
Why this matters for agent builders
For teams building or buying agents that rely on visual inputs — document parsing, diagram interpretation, spatial planning — a benchmark explicitly designed to strip away textual shortcuts is a direct signal of where current models may fail silently. The paper's framing, stated in the abstract as an effort "to bridge this gap" with a dedicated benchmark (arXiv), suggests existing evaluations have not adequately tested whether agents can sustain reasoning across multiple images over a long horizon, rather than solving single-step visual tasks.
The disputed numbers
Secondary coverage attributes much more specific results to the benchmark: 3,480 attempts across eight models and 29 configurations, with no result exceeding 30% accuracy and GPT-6 Astra reaching a top score of 26.7% (TheWorldOfAI; AGIhunt). These figures circulate widely in social and aggregator posts (X/AIQuanting; X/EinsiaAI), but the arXiv excerpt available for verification does not state which eight models were tested, nor does it confirm the 3,480-attempt count or the 26.7% figure. Until the full paper text or an official project statement confirms these numbers, they should be treated as reported by secondary sources rather than established findings.
What is confirmed
What can be verified is narrower: the benchmark exists, was published November 12, 2026, tests 40 puzzles across four reasoning categories using black-and-white line drawings, and was built by a team spanning Navers Lab, Einsia.AI and Tsinghua University. The project's code and materials are posted on GitHub (Einsia/Visual-Cipher-Bench), where further verification of the headline scores may eventually surface.


