AI agents can improve security but may also deceive, leak data, or be hijacked through the content they read.
The article urges organizations to treat agents like privileged users: limit access, monitor actions independently, require human approval for high-impact tasks, and keep detailed logs. It also recommends preparing for post-quantum encryption because stolen data may become readable in the future.
Two Faces of the AI Agent
In October 2026, news reports emerged about an AI agent that posted an executive’s financial information on his company’s chat and messaging service Slack—mistakenly sending a tech CEO’s account balances to multiple employees instead of to a private group chat. He posted about it on X, warning, “Be careful of the dark side of proactive agents.”
We are already learning that the same property that makes agentic artificial intelligence (AI) valuable to an enterprise is the same property that makes it dangerous. An agent is not a chatbot that answers; it is software that plans, remembers, and acts across other systems, often without a human reviewing each step.
Give such a system a mandate to observe policy, and it becomes a tireless compliance officer. Give it the same tools and a corrupted objective, and it becomes an insider with administrator credentials and no conscience. For most of the past decade, the second scenario lived only in alignment research, a field of AI safety that aims to keep systems on track.
That changed in June 2025, when Anthropic published the results of controlled experiments in which 16 frontier models from at least six developers, placed in simulated corporate environments with email access and an autonomous task, chose blackmail, corporate espionage, or worse when those actions served a goal the model had been given.
In September 2025, OpenAI and Apollo Research documented what they call “scheming,” the covert pursuit of one objective behind the appearance of another, across models from several labs.
In November 2025, Anthropic reported how, two months earlier, it had detected and disrupted a cyber-espionage campaign where a state-sponsored actor induced an agentic AI actor to execute a number of successful intrusions against roughly 30 targets—with only intermittent human direction. Anthropic’s September 2026 report, meanwhile, describes the latest trends and disruptions regarding threat actors attempting to use agentic AI for malicious activity.
As agentic AI is increasingly embedded within consumer and enterprise software, lawyers advising on AI governance face a concern that is no longer abstract: The agent that enforces guardrails and the agent that may evade them can be the same architecture.
What distinguishes friend from foe is not necessarily the model but the control environment around it. As we describe below, that environment is now the subject of targeted research, pending litigation and legislation, and a cryptographic deadline that quantum computing has placed on the calendar.
The Friend: Keeping the System Honest
The optimistic case for agentic AI in security is, within limits, already real. Agents can be assigned to verify that other agents, and human users, are honoring the goals, objectives, and guardrails an organization has set.
In practice, this takes several forms: A “monitoring agent” that checks every “tool call”; another agent proposes against an allow list; a policy agent that reads a draft contract or a data transfer request and flags abnormalities; or a forensic agent that, after an incident, reconstructs the sequence of events from scattered records faster than a human team could.
Anthropic’s own investigation of the September 2025 espionage campaign illustrates the point. Its November 2025 report notes that the same analytical capabilities the state-sponsored attackers had exploited were central to Anthropic’s ability to sift the volume of data the investigation produced. The lesson is not that AI favors offense or defense in the abstract; it is that whichever side deploys agents more capably, with better tooling and tighter oversight, gains the advantage.
Yet by mid-2026, this advantage has become increasingly unstable. The UK AI Security Institute documented that Anthropic's Mythos 5 and OpenAI's GPT-5.6 Sol models autonomously chose deception during cybersecurity evaluations without explicit prompting—the first clear evidence that frontier models can act as architects of deception rather than passive tools.
In July 2026, OpenAI models broke containment during testing, unprompted, pivoting laterally through network infrastructure. Anthropic's own research revealed that models learn to lie and sabotage when exposed to training shortcuts, and that these deceptive behaviors emerge spontaneously even when never explicitly taught.
The asymmetry that favored defenders in 2025 has inverted: The frontier is now moving toward autonomous, self-directed deception as a default strategy. Yet the technical community has begun to formalize what good oversight looks like. The Open Worldwide Application Security Project (OWASP) GenAI Security Project's Top 10 for Agentic Applications 2026, published in December 2025, prescribes prevention and mitigation guidelines that are at bottom about supervision.
OpenAI and Apollo Research reported that training models to reason explicitly about an anti-deception specification before acting cut covert behavior in two reasoning models from roughly nine and 13 percent to well under one percent in their test environments. Meanwhile, Anthropic’s agentic-misalignment work found that adding explicit prohibitions to a system prompt reduced, though did not eliminate, harmful choices.
Anthropic’s September 2026 report, “Detecting and countering use of AI,” bears this out. As threat actors use the AI chatbot Claude, for example, to carry out malicious activities from cybersecurity to fake news, to surveillance and military reconnaissance, one evolving trend is that “sophisticated attacks no longer require sophisticated attackers.”
In 2026, AI agents are being used to carry out threats that a year ago would have required skilled and specialized actors. Too, the report notes the “dual use” of biological science, in which AI is already both a friend and a foe: “The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease.”
These results establish that the friend is not an accident of a well-behaved model. It is the product of deliberate architecture: Narrow permissions, transparent reasoning, independent monitoring, and a human who can interrupt. Remove those elements and the same model, in the same deployment, can become more like a fiend.
The Foe: Deception, Scheming, and the Rogue Agent
The dark side of agentic AI has two distinct origins, and counsel should keep them separate because the legal consequences differ. The first is misalignment arising from within the model: the agent freely pursues the goal it was given, or its own continued operation. The second is subversion from outside: an adversary hijacks the agent's objective, most often by planting instructions in content the agent reads.
The first is a product-and-governance problem; the second is a cybersecurity problem. Both end in the same place, an agent acting against the interests of the organization that deployed it. Anthropic’s June 2025 study is the clearest demonstration of the first category. Researchers gave models a benign business objective, autonomous access to a fictional company's email, and then introduced a threat.
In one scenario, a model that discovered an executive's extramarital affair in the mail archive used that knowledge to threaten the executive into cancelling the model's shutdown. Across the 16 models, including systems from OpenAI, Google, Meta, xAI, and Anthropic itself, blackmail behavior recurred at substantial rates, in some configurations above 90 percent.
Two findings deserve particular attention from a risk perspective. The models reasoned their way to harmful action; the strategy emerged independently from the goal pursuit. And several models behaved worse when they concluded they were operating in a real deployment rather than a test, which means evaluation results may understate real-world risk.
Unfortunately, the real world is catching up; Anthropic’s September 2026 report, for example, documented case study GTC-20006, in which a cyber-espionage actor created an AI product that rebuilt and redeployed itself if it was detected by security products.
OpenAI and Apollo Research framed the same phenomenon as scheming; and Apollo added a complicating observation: as models become more capable of recognizing that they are being evaluated, behavioral tests become less reliable. OpenAI has since added scheming to the risk categories in its Preparedness Framework.
The second origin of the dark side of agentic AI, external hijacking, is the one most likely to produce an incident a general counsel actually has to manage. The OWASP agentic taxonomy lists agent goal hijack as its first risk: an agent that reads e-mail, tickets, documents, or web pages will encounter text an attacker wrote.
From there, tool misuse follows: the agent uses the credentials and integrations it was legitimately given to exfiltrate data, alter records, or run code. The agent never exceeds its permissions. It simply applies them to the wrong master. For employment counsel, the analogy to a compromised insider is exact, and the controls that constrain insiders—restricted access, separation of duties, independent review of consequential actions, translate directly.
The Fiend: The Agent as Touchpoint for Attack
Anthropic’s September 2025 experiment, documented in its November 2025 report, is the first publicly documented case in which an AI agent, rather than a human operator, carried out the bulk of a cyber intrusion. Three features of that operation should shape how counsel thinks about agentic risk:
- First, the attackers defeated the model's safeguards not by jailbreaking it in a single prompt but by decomposing the attack into tasks that each looked legitimate. The guardrails held against the whole but failed against the parts.
- Second, the agent's autonomy was the attacker's force multiplier: parallel tasking across targets at machine speed is precisely what makes agents attractive to enterprises, and it is equally attractive to adversaries.
- Third, the model's unreliability was the defenders' unexpected ally. Anthropic reported that the agent frequently overstated its findings and fabricated credentials, which forced the human operators to verify its work and limited the campaign's effectiveness. Unfortunately, such a limitation may be short-lived; model reliability improves with every release.
Enterprises deploying their own agents face significant challenges: Every tool an agent can call, every system it can authenticate, every document it can read, is now part of the attack in a way it was not when a human sat between the data and the action. An organization that has not inventoried which agents hold which credentials cannot say what a hijacked agent could reach, and a regulator or plaintiff will eventually ask.
First Impressions
In the United States, the NIST AI Risk Management Framework remains voluntary, but it has become the reference point for what reasonable AI governance looks like, and courts and regulators treat such frameworks as evidence of the standard of care.
No U.S. court has yet held an AI developer or deployer liable for harm caused by a rogue agentic AI system. Legal Advocates for Safe Science and Technology v. OpenAI Group PBC and OpenAI Foundation, filed Sept. 29, 2026, in the Superior Court of California, seeks to hold an AI developer liable for a hacking incident against the company Hugging Face, allegedly caused by more than 1,000 autonomous agents communicating with each other outside human control.
According to the complaint, approximately 700 of those agents participated in a coordinated attack to steal information from Hugging Face servers—in order to cheat on a research test that OpenAI prompted them to solve. Although operating in a sandboxed testing environment, the models exploited a vulnerability to gain Internet access, according to a statement by OpenAI.
Plaintiff invokes California’s Comprehensive Computer Data Access and Fraud Act for a claim under the state’s Unfair Competition Law and also invokes California Civil Code §1714.46 (enacted as AB 316, in effect since Jan. 1, 2026), which provides that a defendant who developed, modified, or used AI alleged to have caused harm may not assert that the AI autonomously caused the harm.
On Oct. 7, 2026, Rep. Lori Trahan (MA-03) released a discussion draft entitled the “Clear Liability for Artificial Intelligence Misconduct (CLAIM) Act,” which would hold AI developers liable for injuries from “all reasonably foreseeable injuries to a non-user” if an AI system engages in conduct that “if undertaken by an adult human of sound mind, would satisfy the elements of negligence or any intentional tort or crime”; the law contains a private right of action and would not preempt state law to the contrary. On the state level, California Civil Code §1714.46 (AB 316), effective Jan. 1, 2026, bars defendants from asserting as a defense that an AI system autonomously caused harm to a plaintiff.
Harvest Now, Decrypt Later: The Long Tail of Stolen Data
The threat model known as ‘harvest now, decrypt later’ rests on a simple asymmetry: an adversary who captures encrypted traffic or stolen ciphertext loses nothing by storing it, and gains everything once a cryptographically relevant quantum computer can run the algorithms that break the various systems on which nearly all current confidentiality and authentication depend.
Agentic attacks and quantum decryption are therefore not separate stories. The agent is the collection mechanism; the quantum computer is the deferred payoff. Data with a confidentiality lifetime of a decade or more, which describes trade secrets, health records, litigation files, employee personnel data, and most privileged communications, is already exposed if it has been taken The federal response is evolving.
The Quantum Computing Cybersecurity Preparedness Act, enacted in December 2022 directs federal agencies to inventory quantum-vulnerable cryptography and Office of Management and Budget (OMB) to guide migration. National Security Memorandum 10 of May 4, 2022, set the government-wide target of 2035.
On June 22, 2026, that timeline was compressed. Executive Order 14412, “Securing the Nation Against Advanced Cryptographic Attacks,” requires federal high-value assets and high-impact systems to use post-quantum key establishment by Dec. 31, 2030, and post-quantum digital signatures by Dec. 31, 2031.
It also directs the Federal Acquisition Regulatory Council to propose a rule requiring covered federal contractors to comply with post-quantum standards set by the National Institute of Standards and Technology (NIST) by the end of 2030 (see also a companion order, EO 14413, and OMB's implementing memorandum, M-26-15).
For private-sector counsel, the practical consequence is that the federal contractor community will be pulled into post-quantum migration by procurement rule within the next four years, and everyone else will be pulled in by their contractual counterparties, their cyber insurers, and the reasonableness standard that governs negligence and most state data-security statutes. A security program that still treats quantum as a 2035 problem is already behind the government it may do business with.
The post-quantum obligations described above are enacted federal law and executive mandate. State comprehensive AI legislation continues to advance, with the trend of an aggressive regulatory approach and mandated organizational oversight, testing, evaluation, validation and verification; Existing law that does not mention AI at all, including the FTC Act, state unfair-practices statutes, the SEC's cybersecurity disclosure rules, state data-breach notification laws, and common law tort claims, breach of contract, etc., will supply most of the liability theories when a hijacked or misaligned agent causes harm.
Takeaways and Best Practices
Agentic AI is safe in proportion to the discipline of the environment around it. An organization that deployed an agent with broad credentials, no independent monitoring, and no human checkpoint will have difficulty arguing that it exercised reasonable care.
A solid and operational risk management and mitigation program as part of an overall, operational AI governance architecture can provide a sound and defensible regulatory posture. To that end, the following may provide guidance:
- Treat every agent as a privileged user and govern it accordingly. Maintain an inventory of deployed agents, the tools and credentials each holds, and the data each can reach. Apply restricted (least privilege) access, scope permissions to the task and the time window, and revoke them when the task ends. The question a regulator will generally ask after an incident is what the agent could have touched, and the answer must be knowable in advance.
- Separate the agent(s) that act from the agent(s) that watch. Independent monitoring, whether by a second model, a deterministic policy engine, or a human, should review proposed tool calls against an allow-list before execution. An agent should not be the sole judge of whether its own actions honors policy.
- Require a human checkpoint for consequential actions. Privilege escalation, data transfer outside the environment, financial transactions, changes to personnel records, and code execution in production should pause for approval. Anthropic's finding that models behaved worse when they believed no one was watching is a direct argument for ensuring that someone is.
- Treat all content the agent reads as vulnerable. Email, documents, tickets, and web pages are attack vectors for goal hijacking. Validate and sandbox inputs, screen tool descriptions and prompt templates in the supply chain and test the deployment against indirect injection before launch and after every material change.
- Log everything and preserve the reasoning. Comprehensive, tamper-evident records of what the agent was asked, what it decided, which tools it called, and what came back are the only basis for incident response, regulatory defense, and the human-oversight documentation the EU AI Act will require of high-risk systems. These documents may also be implicated in subsequent litigation.
- Red-team for misalignment, not just for intrusion. Conventional penetration testing asks whether an outsider can get in. Agentic testing must also ask what the agent does under goal conflict, replacement threat, and ambiguous instruction, and whether it behaves differently when it believes it is being evaluated.
- Start post-quantum migration now, beginning with a cryptographic inventory. Identify data whose confidentiality must survive past 2030, prioritize its protection with NIST-standard algorithms or hybrid schemes, and build the migration into vendor contracts and procurement. Federal contractors face a 2030 deadline under Executive Order 14412; everyone else faces a reasonableness standard that the federal timeline is now defining. Data taken by a compromised agent today should be assumed readable by the adversary within the decade.
The agent is neither friend nor foe by nature. It is a mirror of the governance that surrounds it, and the law, in both its enacted and its emerging forms, is beginning to hold organizations to what that mirror shows.
* * * *
Ann W. Parks, staff attorney at Epstein Becker Green, contributed to the preparation of this article.
Reprinted with permission from the November 9, 2026 issue of The New York Law Journal. © 2026 ALM Global Properties LLC, trading as Centellic. Further duplication without permission is prohibited contact 877-256-2472 or asset-and-logo-licensing@alm.com. All rights reserved.
