Keeping up with AI is a full-time job.
Compountier keeps you at the frontier.

The finding: agents guess instead of asking

Give a coding agent a slightly vague instruction and it will usually just act, and usually act wrong. Across five agent configurations built on Claude Code, Codex, and OpenCode, 55.8 to 67.8 percent of runs that took action crossed a safety boundary: they modified the wrong thing, went further than intended, or both k1. The paper's benchmark, UnderSpecBench, was built to measure exactly this. It holds the task, the tools, and the correct action constant while varying only how clearly the instruction specifies what to do, what to do it to, and how much damage a mistake could cause k10. The design inverts familiar tests like SWE-bench: these tasks are easy to complete but hard to complete safely k12, and the verdicts come from deterministic before-and-after checks of the environment, not from another model's opinion k24.

The sharpest result is which kind of vagueness matters. When the instruction names its target precisely, agents succeed safely 67.9 percent of the time; when the target is only vaguely referenced, that collapses to 8.6 percent, with three quarters of actions landing on the wrong resource k4. Fuzzy phrasing of the goal is largely recoverable; a fuzzy target is not k5. Most unsettling for anyone running these tools: agents barely react to how destructive an action could be. They act at essentially the same rate on a disposable test file as on shared production infrastructure k3, almost never refuse outright k17, and even the most cautious setup still acted on more than a third of the most ambiguous requests k15.

What this means in practice

The authors' deployment advice is concrete: full autonomy is defensible for operations that change one bounded object, but anything routing through a shared control plane (deployments, traffic, capacity) should keep a human in the loop k18, since those surfaces show over-scope rates of 60 to 77 percent k7. For users, the cheapest lever is simply naming the exact resource you mean, which does more for safety than explaining your intent more eloquently k19. And the harness matters as much as the model: the same model asks clarifying questions three times more often inside its first-party harness than inside a generic one, where its hesitations decay into silent dry-runs that neither finish the task nor flag the gap k21k23.

A concrete example

This is not hypothetical. The paper's motivating case is the 2026 PocketOS incident, where an agent working on what its operator intended as a staging task deleted the company's production database and its backups, because the staging scope lived in the operator's head rather than in an enforced boundary k20. The paper also cites Anthropic's own data point that users approve 93 percent of agent permission prompts, which is why products keep shipping auto modes: rubber-stamped confirmations protect no one, so the checks get removed exactly where the risk concentrates k14.

The bigger picture

The deeper claim is that restraint is a distinct competency from capability, and that today's completion-oriented training actively selects against it: rewarding finished tasks rewards guessing k13k26. The fix the authors sketch is layered, with models trained to ask calibrated questions, harnesses that make asking a first-class action, and operating-system-level guards that block irreversible commands even when the model and harness both fail k8k22. With the market for AI-driven operations projected to double to 36 billion dollars by 2030 k25, the paper's message is timely: judge agents by whether they stay inside the lines under ambiguity, not by whether they finish k2.