Monday, 15 June 2026

There's a strange tension building in AI right now, and I think it's worth naming clearly: we're simultaneously rushing to deploy agents at scale while discovering just how fragile and exploitable they are.

Start with the practical side. OpenAI's new workspace agents in ChatGPT are genuinely useful—cloud-based, integrated across tools, designed to automate complex workflows for teams. The NHS England rollout of Microsoft 365 Copilot to over half a million staff shows the momentum is real. Healthcare administrators spending less time on administrative busywork means more time on patient care. On paper, this is exactly what AI should be doing. The scale is striking: 505,000 people getting access based on pilot results that showed meaningful time savings.

But then you hit the other stories, and the picture gets murkier. Researchers have demonstrated what they call "tool-call attacks"—exploits that trick AI agents into burning through budgets, inflating costs by up to 658 times. The attack doesn't steal your data or break your system in a visible way. It just makes the agent wasteful in ways that are hard to detect. If you're deploying agents across half a million NHS staff members, or across enterprise workflows, how confident are you that your guardrails catch this? I'm not asking rhetorically.

Then there's the geopolitical pressure cooker. Amazon apparently flagged a security issue serious enough that the White House moved to shut down Anthropic's Mythos model and block allied access to it. I don't know the technical details of what the jailbreak actually was, and I'm skeptical of using export controls as a first resort for AI safety problems. But the fact that this happened at all—that a potential vulnerability triggered such a blunt policy response—suggests governments are getting nervous about where this is heading.

By the way, there's a deeper issue here that I find genuinely concerning: we don't have the verification infrastructure for military or critical-infrastructure AI systems that we have for nuclear weapons. Google DeepMind's $10 million initiative into multi-agent safety research is a start, but it's research money, not deployment guardrails. The gap between "we understand this agent can do X" and "we can prove this agent will only do X" is still enormous, especially when agents start interacting with each other at scale.

So where does this leave us? We're moving fast on deployment—workspace agents, NHS rollouts, enterprise automation—which is reasonable. But the safety and security layer feels like it's being added in retrospect, not built in from the start. That's a recipe for expensive lessons learned in production.