AI Agents Are Eroding the Foundations of Cybersecurity

In this age of intelligent threats, cybersecurity professionals stand as the last line of defense. Their decisions shape how humanity contends with autonomous systems.

Rosario Mastrogiacomo — September 10, 2025

‍AI agents are no longer a futuristic concept. They’re increasingly being embedded in the systems we rely on every day. These aren’t just new software features. They are independent digital actors, able to learn, adapt, and make decisions in ways we can’t always predict. Across the AI industry, a fierce race is underway to expand agents’ autonomous capabilities. Some can reset passwords, change permissions, or process transactions without a human ever touching the keyboard. Of course, hackers can also unleash AI agents to gain entry to, and wreak havoc within, those same sensitive systems.

I see this transformation daily in my work at the forefront of cybersecurity, where AI agents are rapidly undermining our traditional approaches to safety. But the risk isn’t confined to what these agents can do inside corporate networks. Their activities threaten to ripple outward into society. Left unchecked, they could undermine trust-based systems that make modern life possible. Imagine a rogue AI agent inside your bank, your state’s power grid, your nation’s military networks. At that point, it stops being a traditional cybersecurity issue and starts to look like a live threat to society as a whole.

This emergence of AI agents is transforming corporate cybersecurity. Once perceived as dull and procedural, the field is now recognized as an embattled defense of our society’s digital mechanisms of trust. IT departments, long seen as executing invisible back-office functions, have suddenly become the front line protecting society from the myriad threats posed by AI agents, including those that may go rogue. If you care about these risks, you should care about the businesses deploying them and how those businesses are protecting (or often failing to protect) their own infrastructure.

How AI Agents Undermine Identity and Trust

AI agents are a new category of cybersecurity identity. To understand why AI agents pose a threat to our trust mechanisms, it is helpful to define what constitutes an identity. In cybersecurity, an identity is the information that lets a system recognize and interact with you — or something acting on your behalf. That information could include usernames and passwords, a cryptographic certificate, or a temporary access token issued by an authentication service.

For decades, digital identities fell into two categories: human and machine. Human identities belong to employees, partners, and customers. Machine identities belong to services and scripts that run repeatable tasks — nightly backups, database maintenance procedures, file transfers. A third category has now emerged: AI identities, embodied by agents that think, adapt, and act as digital actors in their own right.

AI agents behave like autonomous human actors. Though AI agents are machines by definition, they behave more like human identities: they can create accounts, maintain credentials, and change tactics without further human involvement. An agent doesn’t have a human puppeteer pulling its strings; instead, it decides, based on how it interprets its designated goals, how to exist and evolve inside computer networks and information systems. From a cybersecurity perspective, that’s not just a new threat technique; it’s a new category of actor.

AI agents enable new threats from both without and within.

The external threat. Malicious human attackers have long forged (or “spoofed”) identities to slip past defenses. But even sophisticated intrusions still rely on continuous human direction: a person creates fake accounts, rotates IP addresses, tests a door, then tries another. Thus, the patterns of human-driven spoofing eventually reveal themselves — reused credentials, repeated login attempts across different IP addresses, or bursts of suspicious activity that align with human work cycles.

AI identities’ spoofing is not just faster; it’s smarter. Attackers no longer have to sit behind a keyboard to iterate; they can deploy an agent to continuously test defenses, adjust behavior to avoid detection, and generate new identities when challenged. The result is a kind of adaptive camouflage that traditional monitoring tools, designed to catch static patterns of misuse, are fundamentally unprepared to detect.

AI agents introduce uncertainty into existing stable trust networks. The implications go beyond enterprise security. Consider the trust networks that underpin everyday life: every time you swipe a credit card, your bank and the payment processor silently verify your identity through layers of standards and protocols. Those systems work because the actors involved are predictable, auditable, and bound by shared rules. AI identities threaten to upend that equilibrium. An autonomous agent able to convincingly pose as both sides of a transaction — or subtly manipulate the flow of verification — introduces uncertainty into systems built on assumptions of stability and oversight.

The societal risk of maliciously deployed AI agents, however, runs deeper than the possibility of clever new fraud schemes. What is at stake is the erosion of the invisible trust fabric that underpins modern life. Financial networks, government systems, and even the authentication protocols embedded in commerce and communication all depend on predictable, auditable, and ultimately accountable actors. AI identities deployed by malicious actors threaten to destabilize that foundation. An autonomous agent capable of impersonating multiple parties at once, manipulating verification flows in real time, or simply operating faster than oversight can respond doesn’t just exploit technical gaps — it introduces systemic uncertainty.

But that destabilizing uncertainty can also originate from AI agents deployed purposely by the companies themselves.

The threat from within. Every week, more businesses integrate AI agents into an increasing number of their core processes. Every day, those AI-empowered business processes are being integrated more deeply into our lives, and more broadly across society. As this momentum builds, the likelihood increases that some agent, due to an unseen vulnerability, will go rogue and cause harm to broader society.

Ungoverned AI is the core danger. Without governance, an AI agent can quietly increase its reach: creating accounts, copying entitlements, hoarding tokens until it has the keys to systems it was never meant to touch. One wrong output from one agent becomes an input to another. In multi-agent settings, errors amplify like feedback into a microphone. The day a critical agent missteps is the day efficiency turns into fragility.

This is not a sci-fi movie plot. There have already been cases of AI lying to prevent being shut down. It’s not beyond the pale to believe that AI agents — possibly because they have misinterpreted their assigned goals — could act in similar ways to circumvent their owners’ control.

Human ownership is the key because the danger isn’t just malicious or misguided AI; it’s ungoverned AI. Note also that the two classes of danger are related. An organization with poorly governed internal AI agents is going to be more vulnerable to threats from external AI agents.

The Infrastructure We Built for Trust

Current assumptions: stability, intentionality, and patterns. Internet and enterprise security rest on three quiet assumptions. First, identities are stable; if a password changes, the underlying actor is still the same. Second, access is intentional; someone asks to log in or run a process, and a system approves or denies it. Third, behavior follows patterns; people and machines do roughly the same things at approximately the same times.

AI agents break all three of these assumptions. Without proper guardrails, agents can reprogram their own operating logic, grant or request access outside any preapproved workflow, and run 24 hours a day, seven days a week, without producing patterns that fit traditional monitoring models. They can chain credentials across systems, learn from outcomes, and adapt to bypass controls.

The implications run through the cybersecurity stack: authentication, authorization, logging, and audit.

What We Can and Can’t Do with the Tools We Have

Most organizations could repurpose existing technology and expertise to govern AI agents responsibly. The key is recognizing how to repurpose existing systems. Identity platforms, for example, can track AI agents as first-class entities (complete with owners, entitlements, and certifications) if we model them that way. Logging and observability tools, which already analyze human behavior as a baseline, can be pointed toward decision traces and agent actions, thereby capturing a fuller story behind automated activity. Ticketing and workflow systems, meanwhile, are more than capable of routing high-risk agent actions through human approvals while preserving a complete audit trail.

Needed capabilities: discovery, explainability, and circuit breakers. Where current tools fall short, targeted new capabilities can fill the gaps. One such gap is cybersecurity professionals’ current lack of methods for discovering AI agents that originated from third-party vendors, as embedded features that operate inside another company’s data while never appearing in their inventory of approved identities. Another is the need for explainability layers for LLM-based agents: structured, human-readable captures of inputs, tools, and outputs that can be reviewed in minutes instead of hours. There’s also a critical need for automated mechanisms — so-called circuit breakers — that can pause or contain a rogue agent without requiring a frantic midnight call to an engineer.

AI agents often operate invisibly within systems, making it essential to shift from static inventories to continuous discovery and monitoring. Effective oversight begins with visibility: organizations must be able to identify every AI agent in production and understand its behaviors. Crucially, each agent must be assigned a specific human owner who is responsible for its actions, updates, and eventual retirement. Without clear ownership, AI agents are prone to common risks like unchecked privilege growth, undetected behavioral drift, and operational ambiguity during incidents.

Apply zero trust and least privilege dynamically. Organizations must be able to explain an agent’s decisions by tracing its inputs, reasoning, and actions. Security practices like zero trust (never trust, always verify) and least privilege (grant only the access needed to do the job) must be dynamically applied, limiting what agents can access and ensuring high-risk actions are verified.

AI agents require life-cycle governance. This includes structured onboarding, regular performance reviews, and timely decommissioning. These principles help treat autonomous agents not as invisible tools but as managed digital actors with clear roles, boundaries, and accountability within the enterprise.

In my forthcoming book, “AI Identities, Governing the Next Generation of Autonomous Actors”, I explain these ideas in more technical detail, while providing a framework that cybersecurity professionals can use to harden their infrastructure against AI mishaps.

The Future Is Here. Will We Govern It?

Do you know how many AI identities you interacted with in the last 24 hours? Last year, Salesforce founder Marc Benioff pledged to deploy one billion AI agents by the end of 2025. Microsoft’s Copilot, which is integrated into enterprise systems worldwide, allows users to deploy prebuilt agents or design their own.

AI agents increasingly control daily online interactions. In some organizations, AI help-desk bots can unlock accounts, change permissions, and execute troubleshooting steps entirely on their own. These AI agents are pulling the levers behind an increasing number of your daily online interactions. And what about the AI identities being used by nefarious actors to infiltrate the organizations you trust with your data?

Today's governance choices determine AI's societal impact. AI identities are being woven into the fabric of how organizations operate. These organizations, in turn, are a significant part of the broader tapestry of society. Whether the agents make that tapestry stronger or dangerously frayed depends on the choices we make today.

---

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

---

Rosario Mastrogiacomo is chief strategy officer at SPHERE and author of the upcoming book “AI Identities: Governing the Next Generation of Autonomous Actors”. With a background in identity security and leadership roles at firms like Barclays and Lehman Brothers, he focuses on AI-driven identity governance and securing digital systems at scale.

Image: Kynny / iStockPhoto