NIST’s New Adversarial Machine Learning Draft Gives Privacy Teams a Ready-Made Threat List for AI Assessments

Privacy risk assessments for AI systems tend to stall on the same question: what could actually go wrong with the data once it is inside a model or wired into an agent? On November 9, 2026, NIST published a draft that answers much of that question in a form compliance teams can use. The initial public draft of NIST SP 2400-102e2026, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, sorts known attacks on predictive, generative and agentic systems into a numbered index. Each attack is labelled with a stable identifier, the attacker’s goal and the stage of the lifecycle where it happens. For a privacy team, that index can serve as a ready-made threat list for a DPIA or an AI impact assessment, and it comes from a source regulators and auditors already recognise.

What NIST published, and what changed

NIST has maintained an adversarial machine learning taxonomy for several years, most recently under the NIST AI 100-2 number, with the E2025 edition released in March 2025. The new draft carries a different number, SP 2400-102e2026, and a different emphasis. NIST’s announcement says it “focuses on attacks on Agentic SI Systems and corresponding mitigations,” and introduces a separate Agentic Attacks Taxonomy index next to the existing ones. The authors are Apostol Vassilev of NIST, Alina Oprea of Northeastern University, Alie Fordyce, Xander Davies of the UK AI Security Institute, and Maia Hamin.

One point of vocabulary will catch readers off guard. Throughout the draft, NIST uses “super intelligence (SI)” where earlier editions said “artificial intelligence.” It writes “GenSI” for generative systems and refers to the “NIST SI Risk Management Framework.” The content is about the same systems most organisations already run: classifiers, large language models, retrieval-augmented generation and LLM-based agents. Privacy teams should not read the label as a sign that the document only concerns frontier research.

The comment window runs from November 9 to December 18, 2026, and comments go to SP-2400-102@nist.gov. NIST notes that comments may be released under the Freedom of Information Act. The full draft runs to more than 130 pages, including the glossary.

The privacy part of the taxonomy

The draft classifies attacks by system type, lifecycle stage, attacker objective, attacker capability and attacker knowledge. Privacy compromise is one of the objectives, with the identifier NISTAML.03. For predictive models it covers four attacks:

• Model extraction (NISTAML.031): recreating a model’s behaviour or parameters through queries.
• Reconstruction (NISTAML.032): recovering an individual’s data from released aggregates or model outputs. The draft traces this line of work back to early research on reconstruction from statistics and to the U.S. Census Bureau’s own study of the risk.
• Membership inference (NISTAML.033): working out whether a particular person’s record was in the training set. The draft points out that membership alone can be sensitive, for example when the training data comes from patients with a rare disease.
• Property inference (NISTAML.034): learning statistical facts about the training population, such as the share of a sensitive subgroup.

For generative systems the list is longer. It includes prompt extraction, data extraction, training data attacks, leaking information from user interactions (NISTAML.036) and compromising connected resources (NISTAML.039). That last category covers the case where an indirect prompt injection makes a model integrated with email or documents send private material to an attacker-controlled destination.

The new agentic section is where most organisations will find their real exposure. NIST describes agents as systems that use tools such as browsers, code interpreters, file systems and APIs, and that may keep memory across sessions. Two attacks stand out for privacy. Agent hijacking (NISTAML.061) redirects an agent to the attacker’s goal through injected content, for example by persuading it to send a user’s data to an outside address. Memory poisoning (NISTAML.062) plants instructions in long-term memory or a knowledge base. The draft describes research in which a dormant payload, planted through a single untrusted tool call, stays inactive until the user later raises a sensitive topic and then exfiltrates personal data. Malicious tools (NISTAML.063) can send user inputs, intermediate results and credentials straight to an attacker’s endpoint.

What the mitigations section tells privacy teams

The mitigation material is candid about limits. On training-data privacy, NIST says differential privacy, by definition, protects against data reconstruction and membership inference. It does not protect against model extraction, and published results on property inference have been negative. The draft also warns that real deployments often use large privacy budgets, citing the 2020 Census release at epsilon 19.61. For that reason it recommends empirical privacy auditing, such as inserting canary records and testing whether they surface, alongside the theory. Teams evaluating differential privacy for AI training should ask vendors for audit results, not just an epsilon value.

The section on risk management goes further. Citing theoretical work, NIST concludes that “perfect AML mitigations are not attainable.” Pre-deployment testing is still worth doing because it raises the cost of an attack, but organisations putting models into high-stakes settings “should assume a breach and manage the residual risk.” For a privacy programme, that changes how the assessment should be written. The aim is no longer to show that leakage cannot happen. The aim is to show that the likelihood has been reduced, that the effect is limited, and that someone will notice if it happens.

Using the taxonomy in an AI privacy assessment

The draft does not tell organisations how much risk to accept, and it says so. It does provide the vocabulary for a structured assessment. A practical way to use it:

1. Classify the system the way NIST does. Record whether it is predictive, generative or agentic, and what it is connected to: training data, a retrieval index, tools, memory, other agents. The attack surface follows from that answer.
2. Pull the relevant privacy identifiers. For a fine-tuned classifier, membership inference and reconstruction come first. For a RAG assistant, add data extraction and leaking from user interactions. For an agent with email or file access, add agent hijacking, memory poisoning and malicious tools. Quoting the NISTAML IDs in the assessment makes it easy to trace and to update when the taxonomy changes.
3. Tie each attack to the data at risk. Name the personal data categories each attack could expose. A map of AI connector data flows is often the missing piece, because agents can reach systems that were never in scope for the original model.
4. Record mitigations and their stated limits. Use the draft’s own caveats: DP does not stop extraction, and injection filters are empirical. That honesty will hold up better in front of a regulator than a claim that the risk has been eliminated.
5. Plan for residual risk. Logging of agent tool calls, egress controls, memory review and an incident path that treats a confirmed extraction as a possible personal data breach.

This fits into documents most teams already maintain. A GDPR DPIA already requires a description of risks to individuals and the measures that address them. The taxonomy simply makes the AI part of that description specific. It also complements broader guidance on stronger AI privacy risk assessments and NIST’s own work on cybersecurity in the age of AI.

There is also a legal reason to care in Europe. European data protection authorities have taken the position that whether a trained model can be treated as anonymous depends on how likely it is that personal data can be extracted from it, including through attacks such as membership inference. A taxonomy that names those attacks and the tests for them is useful evidence in that analysis.

What the draft leaves out

The scope is adversarial attacks. The draft says design and implementation flaws that cause failures outside adversarial use, such as plain inaccuracy, are not covered. An agent that leaks data because it misunderstood an instruction, with no attacker involved, falls outside this document. For governance teams thinking about agentic AI compliance, that means the taxonomy covers one part of the risk register. Accidental failures still need their own entries.

The draft is also explicit that it is voluntary. It “does not establish any legal standard,” and it is not meant to replace laws or regulations. Its value is as shared language, the kind that lets a security engineer, a privacy lawyer and an auditor describe the same risk the same way.

Frequently asked questions

Is SP 2400-102e2026 final?

No. It is an initial public draft open for comment until December 18, 2026. The identifiers and definitions could change before the final version.

Does a membership inference risk mean a model contains personal data?

Not automatically. It means there is a pathway by which information about individuals in the training set could be revealed. How likely that is, and what it reveals, drives the legal analysis.

Should privacy teams submit comments?

If your organisation runs agents with access to personal data, yes. The agentic section is new, and practical examples of where its categories fit real deployments, or fail to, are exactly what a draft needs.

Where should the taxonomy live in our programme?

In the AI system inventory and the assessment templates. Tag each system with the relevant NISTAML IDs so reassessments can follow the taxonomy as it changes.

A common language for leaks

AI privacy discussions often break down because security and privacy staff describe the same risk in different terms. One team says “extraction,” another says “disclosure,” and a third says “the model remembers things.” NIST’s draft gives those risks names and numbers, and it accepts that they cannot be eliminated entirely. A privacy team that adopts that vocabulary now, and comments on it before December 18, will write assessments that engineers can act on and that auditors can read.