MemRL studies how a frozen language-model agent can improve during deployment without changing its weights. It separates stable reasoning from an evolving episodic memory, retrieves candidates by semantic relevance, and then selects useful experiences through learned Q-values. Environmental feedback continually refines those values, enabling the agent to retain high-utility strategies and suppress similar but unhelpful memories across reasoning, coding, embodied, and lifelong-agent benchmarks.
A non-parametric route to runtime self-improvement: keep the reasoner stable, let episodic memory learn utility from feedback.