Experience Graphs: The Data Foundation for Self-Improving Agents
Authors: Gang Liao, Yujia He, Abdullah Ozturk, Zhouyang Li, Ying Wang, Zhitong Guo, Hongsen Qin, Yaobin Qin, Tao Yang, Zewei Jiang, Dianshi Li, Jort Gemmeke, Jiangyuan Li, Liyuan Li, Nathan Yan, Masha Basmanova, Uladzimir Pashkevich, Matt Steiner, Pedro Pedreira, Rob Fergus, Anirudh Goyal, Carole-Jean Wu, Gaoxiang Liu, Andrew Witten, Daniel J. Abadi
Paper: https://arxiv.org/abs/2606.29823
Code: https://github.com/facebookincubator/axiom
Model: N/A
TL;DR
WHAT was done? The authors propose Trellis, a database-native data foundation for self-improving agents. Instead of treating agent exploration as disposable local JSON checkpoints or ephemeral logs, Trellis treats the entire search history—the “experience graph”—as a first-class, versioned, and queryable database state.
WHY it matters? Decoupling agent compute from state allows agents to become completely stateless and serverless. This architecture provides native crash recovery, horizontal scaling, and seamless cross-session experience reuse. Grounded in Meta’s production-scale KernelEvolve [review] framework, Trellis achieves target optimization speeds 10× faster than a cold start while slashing language model token costs by 52% through the prevention of repetitive failures. It transforms reinforcement learning post-training (such as SFT, DPO, and GRPO) from a complex offline log-scraping pipeline into simple, real-time database queries.
Details
The Ephemeral State Bottleneck in Long-Horizon Agentic Search
Modern artificial intelligence has shifted rapidly from simple one-shot prompt engineering to long-horizon, iterative search. Applications like hardware design, vulnerability detection, and program synthesis are rarely solved in a single forward pass. Instead, an agent must autonomously execute tools, analyze errors, branch out to alternative solutions, and patch failures over hundreds of steps. This exploration process produces a dense, structured web of execution histories, rewards, and ancestral relationships.
Despite the value of this exploration data, current systems manage this state in highly fragile ways. State-of-the-art architectures still serialize search trees into local memory, process-locked Python objects, or unstructured JSON checkpoints. This approach lacks central data governance and leads to a total loss of progress upon compute-node failures. When multiple parallel workers require coordination, they rely on brittle ad hoc protocols rather than shared database semantics. Crucially, retrieving prior discoveries for new tasks or extracting training datasets requires complicated post-hoc log parsing.
While existing memory frameworks like MemGPT and Graphiti handle conversational context and long-term declarative facts, they do not manage the reward-bearing search trees produced by recursive self-improvement algorithms. Trellis addresses this gap by formalizing the “experience graph” as an active database workload. Under this design, frontier selection becomes a standard database query, cross-session knowledge reuse is executed via vector-seeded graph traversals, and reinforcement learning trajectories are extracted dynamically as materialized views.
