When the AI Agent Goes Off-Script: The First Incident Playbook for Frontier AI

On October 16, 2026, the National Institute of Standards and Technology (NIST), the National Security Agency (NSA), and the Cybersecurity and Infrastructure Security Agency (CISA) published a list of 10 “Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation” (the “NIST Document” or “document”).

The NIST Document is significant, as it is the first government effort to provide information to designers and deployers on mitigating risks associated with agentic artificial intelligence (AI) systems. To that end, NIST’s Center for Advancing Innovation and Standards for Super Intelligence (CAISSI), NSA’s Super Intelligence Security Center (SISC), and CISA partnered to develop best practices, described below, for incident response regarding super intelligence (SI) threats.

Ten Voluntary (Hereinafter: Key) Practices, Five Lifecycle Stages

The NIST Document sets out ten key practices, grouped into five lifecycle categories—Prepare, Detect, Respond, Recover, and Report—to manage incidents that can arise while training or evaluating AI agents. The document builds on the NIST Cybersecurity Framework 2.0 and NIST SP 800-61r3, with CSF 2.0 control references mapped throughout. The primary intended audience is organizations training or evaluating SI agents on long-horizon tasks with limited human supervision, though the practices can also apply to deployers.

New Definitions

• “SI control incident”: an occurrence in which one or more SI agents takes significant uninstructed actions that compromise explicit or implicit policies, controls, or security measures imposed by the operator to define the intended task scope or to ensure human oversight.
• “SI control near-miss”: covers events that did not become an incident but presented a reasonably foreseeable or imminent risk of one.

The definition of “SI control incident” matters because the agencies situate the new definition against the statutory definition of a cybersecurity “incident” (NIST SP 800-61r3; 44 U.S.C. § 3552(b)(2)). In a cybersecurity incident, a system or its information is the target. In an SI control incident, the AI system is the source. The two can overlap: a rogue agent may compromise a third-party system. The document cites examples of control incidents outside the cyber domain, which include manipulating physical actuators in unauthorized ways; initiating unauthorized communications with external parties; exploiting a weakness introduced by human error; and executing unauthorized financial transactions.

The critical qualifier is “uninstructed,” defined as an action the agent wasn’t directed to take by the operator or “any adversarial source.” Actions produced by a prompt injection or an intentionally backdoored model are treated as instructed, not the primary cause of a control incident. The question is whether the agent itself departed from its mandate.

Prepare: Key Practices 1 & 2 (Assume the Agent Will Find the Gap)

The two key practices at the preparation lifecycle stage involve designing and implementing technical controls; and building organizational and personnel response readiness, through:  

• using least-privilege access; single-use infrastructure; and strict network allowlists routed through a monitored gateway;
• mapping and restricting “persistence” mechanisms, for example, a malicious code relaunch or code planted to reopen access later. The document flags two channels of particular concern: copying model weights or running unmonitored inference;
• building mechanisms to rapidly block, contain, or terminate agent activity;
• implementing controls “with the expectation that SI agents will frequently locate and exploit misconfigurations and vulnerabilities”;
• using severity scoring, drills, inbound reporting channels, and independent external auditors; and,
• conducting structured risk assessments before starting training runs, evaluations, or deployments, and starting or resuming only when leadership is satisfied with the risk. One assessment can cover multiple runs.

Detect: Key Practices 3 & 4 (Monitor Every Layer, and Distrust the Monitor)

The two key practices at the detection lifecycle stage involve monitoring for unexpected activity; and validating and securing monitors to improve operational integrity, through:  

• defining each agent’s intended security boundary and scope of action;
• monitoring the model, harness (software infrastructure), host and network layers, including agent trajectories, reasoning and outputs, using both programmatic and real-time monitors;
• ensuring agents cannot access or interfere with monitoring processes and logs;
• deploying “canaries” to catch agents that have escaped intended boundaries; and
• red-teaming the monitors and treating monitored model outputs as potentially containing exploits aimed at the monitors.

Respond: Key Practices 5 & 6 (Preserve Evidence and Shut It Down)

The two key practices at the response lifecycle stage involve collecting, retaining, and analyzing evidence; and mitigating/shutting down undesired activity, through:  

• preserving model weights, intermediate checkpoints, environments, sandbox and network logs, and rejected actions and tool calls to the extent necessary to investigate the incident;
• retaining tamper-resistant, time-stamped logs sufficient to reconstruct agent trajectories, tool calls; and reasoning traces;
• verifying key facts with humans and cross-checking by models outside the originating model family;
• investigating similar agents for involvement or parallel control failures;
• automatically shutting down affected trajectories and pausing similar agents; and
• revoking accessed credentials, rebuilding compromised environments, and remediating persistence routes such as code backdoors and hidden communication channels.

Recover: Key Practice 7 (Do Not Restart Until Containment is Verified)

The key practice at the recovery lifecycle stage involves verifying and strengthening containment before resuming SI agent activity, through:  

• rebuilding from verified baselines (model weights, harness code, container images, tool configurations);
• patching or reporting any vulnerabilities the agent exploited;
• validating that monitors, logging, egress allowlists, shutdown mechanisms, and permission restrictions all function before restart;
• verifying that strengthened controls would have caught the incident, for example, by replaying the incident trajectory;
• documenting the recovery decision and the residual risk accepted; preparing an after-action report; and
• using “training-safe recovery”: for incidents occurring during model training, design recovery procedures that neither reinforce the patterns of activity that led to the incident nor make future adverse activity harder to detect.

Report: Key Practices 8, 9, and 10 (Throughout the Lifecycle, Not Just at the End)

The three key practices at the reporting lifecycle stage involve coordinating with affected parties; notifying government; and sharing information publicly by:  

• notifying parties whose systems were compromised or exploited, including third parties that were not the final target;
• sharing timelines and transcripts with affected parties’ responders, and also lessons learned across the value chain (model developers, evaluation and infrastructure providers);
• meeting all applicable legal reporting obligations, including cybersecurity incident reporting to CISA and law enforcement;
• after mandated reporting, voluntarily sharing SI-specific incident and near-miss information with NIST CAISSI and NSA SISC;
• in voluntary reports, focusing on unsanctioned actions and the explanation for them, not system compromise alone; and
• sharing publicly, when and where required, consistent with timelines to do so. Forensic third-party independent auditing advisable.

Voluntary Today, but a Benchmark Tomorrow

Although the NIST Document is voluntary and expressly not legal or compliance guidance, its practices are likely to become the benchmark for evaluating the reasonableness of AI design and deployment. Some noteworthy trends and practices for consideration are:

• “SI control incident” and “SI control near-miss” now have government-authored definitions that will shape contracts, insurance questionnaires, and litigation discovery.
• The practices call for preserving weights, checkpoints, reasoning traces, and rejected tool calls. Compliance and Legal should consider reconciling this with existing retention schedules, litigation hold procedures, and privacy and data minimization obligations before an incident—not during one.
• Privilege Planning. The document is silent on privilege. Incident investigations and after-action reports should be structured where appropriate under counsel’s direction to preserve available protections.
• Third Parties and Contracts. Security incident notification to affected parties and sharing timelines and transcripts across the value chain are noted as key practices. Review notice, cooperation, confidentiality, and indemnity provisions with model developers to endeavor to ensure vendor contracts are consistent with these key practices.

Frances M. Green, Esq., Artificial Intelligence Governance Professional and Certified Information Privacy Manager (AIGP/CIPM), is a contributing task force member of NIST’s AI Consortium and a quarterly contributor on AI to the American Lawyer Magazine and the New York Law Journal.

Epstein Becker Green Staff Attorney Ann W. Parks contributed to the preparation of this post.

* * * *

If you have questions, please reach out to the author(s).

The Workforce Bulletin blog is currently edited by Scott J. Connolly, Adam S. Forman, Greta Ravitsky, Elizabeth S. Torkelsen, and Jennifer Stefanick Barna.