Shengtao Zhang← All publications

arXiv preprint · 2026

MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs

Junwei Liao, Haoting Shi, Ruiwen Zhou, Jiaqian Wang, Shengtao Zhang, Wei Zhang, Ying Wen, Zhiyu Li, Feiyu Xiong, Bo Tang, Weinan Zhang, Muning Wen

AgentsRLMemory

Abstract

MemQ

MemQ models memory evolution as a structured credit-assignment problem. A provenance DAG records which retrieved memories contributed to each newly created memory, while TD-style eligibility traces propagate feedback backward through those dependencies. This makes retrieval utility reflect both immediate usefulness and downstream influence, improving self-evolving memory agents across multi-step interaction, code, embodied reasoning, and expert QA tasks.

Credit assignment for agent memory, propagated through the provenance chains that make later memories possible.