When AI agents book, ban, trade and pay — who's actually accountable

The most interesting AI story this week isn't about a new model beating a benchmark. It's about an AI agent getting a person banned from a restaurant reservation platform — and then another AI agent talking the platform into reversing the ban. Brian Distelburger used an autonomous agent to book a table at 4 Charles in New York, Resy flagged the behavior as suspicious (correctly, in a sense — it wasn't human), and locked him out. The unban apparently required its own bit of agentic maneuvering. I find this genuinely funny, but also a preview of something structural: platforms built around implicit assumptions of human behavior are going to keep colliding with agents that technically follow the rules while breaking every unwritten one.

That collision gets a lot more serious once money is involved, not just dinner reservations. There's a sharp piece making the rounds asking a question I think will define agent adoption for the next few years: if you give an AI agent a budget and tell it to shop within constraints, and it does something you didn't quite intend, who proves what you actually approved? This isn't hypothetical anymore. Binance just launched Agent OS, letting AI agents execute real crypto trades through ChatGPT, Claude, and Codex. Crypto is famously unforgiving about mistakes — there's no customer service line to reverse a bad trade the way Resy reversed a ban. Combine autonomous multistep reasoning with irreversible financial transactions and you get a legal gray zone that current consumer protection frameworks weren't built for. Intent verification, which sounds like a dry compliance term, is about to become one of the most contested issues in AI agent design.

By the way, this is exactly the kind of pressure Miles Brundage, formerly of OpenAI, is pointing at when he says AI companies need real guardrails now rather than later. His argument isn't that safety teams are always right — it's that employees raising concerns deserve to be taken seriously before the incident happens, not analyzed afterward in a postmortem. Agent commerce is a perfect test case: the failure modes are becoming visible in real time, from Resy's blunt instrument of a ban to whatever quietly goes wrong the first time an Agent OS trade executes on bad reasoning. Separately, the Existential Risk Alliance is hiring research managers for AI safety and biosecurity work — a reminder that the infrastructure for taking these questions seriously is still being built, staffed one role at a time, even as deployment races ahead of it.

There's a useful contrast here with robotics, where Unitree's own CEO is publicly tempering expectations, suggesting a genuine humanoid robot breakthrough could be a decade out. It's a rare moment of an industry leader pumping the brakes rather than the hype. Software agents don't have that luxury of patience — they're already loose in production, booking tables and, soon, trading real money. The physical world imposes limits that force honesty about timelines. Digital agents don't have that friction yet, and I suspect we'll learn what "too fast" looks like the hard way before we learn it the careful way.