|
‘Rogue’ AI Agents Will Kill Billions?
It was a rainy day when I met Bill Gates. That was in late 1999 at a Microsoft event in Seattle. We had a short chat about an aquisition. My impression was that he was a smart guy, but not very intelligent. Later that day he gave a talk. His PowerPoint presentation promptly blue-screened. He was embarrassed and helpless.
Gates knows a bit or two about how to profit from monopolizing software. Seen in that light his recent demand for regulation by claiming that AI could ’cause a billion deaths’ may even make sense:
In an interview with NBC Meet the Press, Gates said AI developers should be required to implement safeguards and monitoring systems, arguing that voluntary industry self-regulation is not enough.
“You need law enforcement and the politicians to get into the discussion about what safeguards and monitoring look like,” Gates told the outlet. “And that has to be a required thing.”
World to Gates: We already have laws that prohibit doing bad stuff. We also have law enforcement agencies. We have courts and judges that penalize people for doing bad things.
Killing a billion people is bad. It is prohibited for good reasons. Attempts to do so are criminal. People who try will go to jail.
To claim that AI could kill a billion people is like claiming that fire could kill a billion people. Of course it could. But no, it doesn’t.
After some unauthorized attempts to break into non-public databases and websites there has been embarrassing talk about ‘rogue AI agents‘.
There is no such thing as a ‘rogue’ AI agent. There is only deterministic software, written and launched by humans, that wraps around Large Language Models.
To understand how that works lets look inside:
An AI agent harness is the software infrastructure that wraps around a large language model (LLM) and enables it to act on tasks, not just respond to prompts. The model reasons through a problem and decides what to do next. The harness connects it to the tools, systems, memory and execution environments needed to carry out those actions.
Agent = Model + Harness
Think of the model as the “brain” that generates reasoning and decisions. The harness is everything around it that helps the agent operate safely and reliably, including:
- Tools: APIs, code execution, search, databases and business applications
- Memory: Prior context, user preferences and workflow history
- Workspace: Files, data, environments and systems the agent can access
- Guardrails: Permissions, policies, approvals and monitoring
Without a harness, a model can answer questions, but it can’t reliably run code, call APIs, access files, remember prior work or complete multi-step workflows on its own.
An AI agent, tasked with whatever, can and will only break into a website, if its harness (and the computer the ‘agent’ is running on) fails to stop it from doing so.
Whoever has programmed that harness (or miss-administrates the underlying system) is at least negligent in his duty. It is that person which is accountable for the agents misbehavior. It is that person which, where appropriate, should go to jail.
There is a very simple way for preventing AI agents from OpenAI or Anthropic from going ‘rogue’. Harshly penalize the CEOs of these companies, Sam Altman and Dario Amodei, each and every time their agents do.
AI systems do not think, feel, want or understand. They don’t go ‘rogue’.
biggerTo claim they do is silly.
In response to john brewster@304,
To be fair, I’ve built up my own strawman around the origin of the term, as a corporate marketing ploy, and since I can’t stand profit-driven language substitution and narrative shifting, I now can’t stop myself from tilting at that windmill. I myself became aware of it via this thread, which I find especially irritating, since I’ve been building agentic workflows and AI interfaces professionally for the better part of the year, to now discover that someone hijacked the terminology on the sly — so, I must admit that it feels personal, which probably isn’t very healthy or rational on my part. But, I hope it helps you understand my perspective.
In terms of functional density, LLM’s plateau-ed around 2024, if that’s what you mean by “better” — i.e we hit the ceiling for what raw compute could get you. Since then, it’s been mostly about optimization, and thinking smaller, doing more with less, which isn’t very exciting. CoT reasoning models are, really, part of that optimization push — give a “less polished” model the context space and compute to draft a pre-response, and the final response appears as if generated by a higher tier model — no new technology there, just tactics that users were already employing beforehand. Again, “reasoning” here is a marketing ploy for pre-response drafting — there have been a few, smaller experimental models with something approaching reasoning loops embedded in the architecture itself that didn’t become industry standard, maybe because they wouldn’t hesitate to express having low confidence with their own responses, which for some reason is treated as taboo. Confidently wrong and sycophantic has wider mass appeal, it seems.
But, that isn’t to say release models from 2024 compare favorably to ones released this year, or that development has stopped, or that the underlying architectures haven’t gone through and aren’t still going through significant changes. It’s just that the average paying end-user wouldn’t notice, that the huge model they’ve been using got replaced by one that now fits on a single consumer GPU, without any distinction in the output, and that is a valid point of stagnation — the selling point of paid services.
It’s increasingly affordable to run models locally, a garage upstart can rival the model quality of tech-giants, Chinese alternatives beat Western counterparts at benchmarks and affordability metrics, and the main theme of models released this year has arguably been re-scaling and compression — 1B quantization, models that can run locally on your average smartphone, and so on. In that respect, absolutely, tech-companies need to find a way to expand the products they have on offer, and struggling to compete on the model front, and having the biggest server racks to run the biggest models is no longer a point of distinction, so they’re pivoting into the suite front, the architecture around the model, streamlining existing open-source stacks, automating the implementation and adding their own corpo twist to it. So, in my view, the technology hasn’t stagnated — the business model for selling this tech is failing and trying to reinvent itself and find a niche it can monetize.
The future of paid AI services may well be expanding on the suite or the harness, with big tech moving away from trying to compete on model research and development. A company dedicated to LLM R&D would struggle to compete with a major corporation on LLM-integrated side-applications, and in terms of open-source alternatives, big tech swims in the same ocean and picks from the same buffet of new innovations and implementations, like sharks in the water. But, if so, what it implicitly means is that these corporations are conceding defeat on the R&D front — whether it’s because they’ve been outmatched and struggle to keep up, or because their model for monetization of AI services simply isn’t compatible with an R&D focus. They’re saying “Forget the model, doesn’t matter. Look at all the different side-applications and interfaces we’re offering — woow!” — when an AI-developer says that, it reads as desperation. So I’m with you on your prediction, with that specific caveat — it’s not that model development has peaked, but development-focused model monetization for the public sector certainly has.
Will this pivot work and save the structural integrity of the AI-bubble? Maybe as a temporary respite, a way to inflate it further with new investments and a “fresh” idea, and maybe get a few more miles out of that hot-air balloon. But, if it amounts to anything new worth copying, keeping in mind that we’re already talking about a rebranded copy of a copy, there’s nothing here to monopolize — no secret tech — then the systemic problem of constantly being outpaced and obsoleted will persist, until there’s nothing to offer again but brand recognition.
How’s that? Does my read of the situation fit into the ponzi-scheme or the panopticon?
Posted by: Skiffer | Oct 2 2026 16:13 utc | 308
|
