Microsoft and NVIDIA Publish Open-Weight AI Risk Guide for Enterprises

Microsoft and NVIDIA have published joint guidance for enterprises using open-weight AI models, with a shared risk taxonomy and an approach to evaluating safeguards. Announced on October 16, 2026, Risk Management for Open-Weight Models in Enterprise is the first in a planned series. It draws on established ISO standards and the NIST AI Risk Management Framework.

The companies argue that access to a model’s weights does not, by itself, determine the risk of using it. The relevant assessment includes the intended use, deployment and surrounding controls. That distinction matters for privacy teams deciding whether a locally operated model can safely handle customer records, employee information or confidential business material.

Running a model inside the organization can reduce the need to send prompts to an external model provider. It does not establish that everyone who can query the system should receive everything it can produce. Nor does it explain what the application retains, which documents it retrieves or whether another service receives its logs.

The guidance offers a starting point for examining those questions. Enterprises still need an assessment tied to the actual system and evidence supporting their decision to deploy it.

Source materials reviewed October 22, 2026.

The first release focuses on the model, with more layers to follow

Microsoft describes the initial publication as focused primarily on model-layer risks. Later installments are intended to address the runtime, harness and application layers in more detail. The harness is the software connecting model outputs to tools and their execution. A model evaluation should therefore not be presented as a complete assessment of an agent’s permissions, connected tools or operational behavior.

The joint repository contains the paper and sections for the taxonomy, evaluations and mitigations. Microsoft and NVIDIA co-maintain it. Evaluation resources map tests to selected risks, while mitigation resources are organized by system layer. The repository is intended to develop over time, so organizations should record the version of any resource they use.

For a business reviewing an AI purchase, separate the upstream model assessment from the downstream deployment decision. A developer can provide useful test results without knowing which customer records an application will expose or which actions it will execute. Those questions require information from the people building and operating the deployment.

For illustration, an internal drafting service might return text for an employee to review. Another application could use the same model to retrieve support tickets and send responses to customers. Even if their underlying weights match, their data access and consequences differ. Approval for the drafting service should not automatically extend to the customer-facing application.

Open weights and open source do not mean the same thing

Weights are the learned parameters used by a model to produce outputs. Access to them can allow an enterprise to run and modify the model, subject to its terms. That access does not necessarily include the training data, training code or complete information about how the model was developed.

The Open Source Initiative’s Open Source AI Definition 1.0 goes beyond weight availability. It addresses freedoms to use, study, modify and share, alongside requirements for data information, code and parameters. Procurement records should preserve the actual license and available documentation rather than rely on a product’s informal “open” label.

Model access and hosting are also separate decisions. An open-weight model can be offered through a managed platform, while self-hosting can place more responsibility for safeguards on the enterprise. The deployment contract and configuration determine who operates a given control.

That leaves several questions for a buyer. Who maintains the serving software? Which party updates the safety filters? Who investigates a suspected disclosure? Can the organization roll back a change or suspend access? Answers should name responsible teams and services rather than assume that downloading the model transferred a complete managed service with it.

A common taxonomy gives different teams something specific to review

The initial taxonomy has five broad categories: Cybersecurity, Data Protection, Content, Human–AI Interaction and Bias. It supplies terminology rather than a deployment checklist or an organization’s acceptance criteria. The following examples show how Captain Compliance would translate those categories into practical questions; they are not quotations from the guide.

Assigning a category should lead to a decision about the system. If the data-protection concern is access across customers, the application owner needs to investigate access restrictions and retrieval behavior. A general content-safety score does not answer that question.

Some findings need several owners. A retrieval permission may be a security responsibility, while the business determines who should receive the information and the privacy team evaluates the proposed use. Keep those responsibilities connected to the same finding so one team’s completed task does not conceal another team’s unresolved work.

Privacy risk includes training data, live inputs and inferred information

The paper’s data-protection taxonomy distinguishes disclosure of training data from disclosure of personal information supplied during use. It also covers sensitive inferences, re-identification and failures of isolation between tenants. These concerns require different evidence.

An enterprise evaluating training-data disclosure should examine what is known about the model’s provenance and the possibility of recovering personal or confidential information through queries. Removing customer names from a new prompt does not answer whether the model can reproduce information retained from earlier training.

For information supplied during use, map the request and response path. A customer record may enter through a prompt, a retrieved document or a tool response. It may then appear in an answer or operational log. Restricting access at the original database is insufficient if the application later passes an unrestricted copy into a shared context.

Consider an illustrative customer-service application using a locally hosted model. The model service stays within the enterprise network, but a support dashboard sends full conversations to a separate monitoring provider. The hosting decision has reduced one potential transfer while the monitoring configuration creates another. The privacy review needs both facts.

Derived information deserves attention too. A system can infer something sensitive from information that looked ordinary when collected. The review should examine what the output says about a person, the evidence supporting it and who may use the inference. Merely removing direct identifiers does not establish that the result cannot be linked back to someone.

This is a useful place to connect technical testing with legal review. In its December 2024 opinion on AI models, the European Data Protection Board said anonymity requires a case-specific assessment. It addressed both identification of people represented in training data and extraction of personal data through queries. An assertion that a model is anonymous needs supporting evidence.

The EDPB also examined lawful bases and the consequences of unlawfully processed training data. A model’s availability for download does not establish permission to use every underlying record, and a good benchmark result does not supply a legal basis for a new processing purpose. Those issues belong in the deployment review alongside security and performance.

A benchmark score does not establish the likelihood of harm in production

One useful distinction in the paper is between a model’s propensity to exhibit behavior under test and the likelihood of an event in a particular deployment. Exposure and operating conditions affect the latter. The result of a test describes the tested conditions, not every future request.

For example, suppose an enterprise creates an adversarial test set designed to elicit confidential information and observes disclosures in several cases. That finding identifies a problem worth investigating. It does not, by itself, establish the percentage of ordinary users who will experience a disclosure. Equally, an absence of disclosures in a small test set does not establish that leakage is impossible.

NVIDIA’s accompanying explanation discusses garak, its vulnerability-probing toolkit, and BBQ, a question-answering bias benchmark. It emphasizes interpreting findings within the evaluation’s coverage and limitations. These tools provide evidence for specific questions; they do not produce a universal approval to deploy.

The BBQ project, for example, describes question sets relevant to U.S. English-speaking contexts. A multinational service would need to investigate its own languages and populations. Passing a test designed for one context should not be treated as evidence that every customer receives comparable service.

Before testing, define the behavior that would block or restrict deployment. For a service extracting invoice details, an invented account number may be unacceptable even if most fields are correct. For a draft-writing aid, an error caught in a reliable review process may have a different consequence. The acceptance rule should reflect the proposed workflow and applicable obligations.

Retain enough context to reproduce the result. Record the model version, input set, scoring method and deployed controls, with safeguards for sensitive evaluation material. Review disputed findings and identify what the test did not cover. A dashboard’s aggregate score is much less useful when no one can explain which version it assessed.

Testing should also examine whether a mitigation breaks legitimate use. A filter can reduce harmful output while rejecting valid customer requests. Review both outcomes before accepting it as an effective control for the intended service.

Model changes can invalidate the evidence behind an approval

The paper warns that fine-tuning, quantization and other modifications can affect safeguards even where task performance remains useful. Its recommendation is to evaluate safety separately from capability. A successful accuracy test should not stand in for testing the behavior that a prior mitigation addressed.

For an enterprise, that means approval should identify the configuration covered. A team optimizing a model to fit available hardware should check whether existing evidence remains applicable. The same question arises when a different adapter, serving configuration or content filter enters the deployment.

Record the source of the acquired model and how the deployed version differs. Where available, signatures can help establish artifact integrity; they do not prove the accuracy of a model’s outputs or the lawfulness of every training source. A gap in provenance should remain visible even where behavioral tests look favorable.

The change process should state which tests must run again and who can approve the result. A change limited to presentation may warrant a different review from adding a new document source or enabling a tool. The aim is to assess the changed behavior without treating every release as either automatically safe or an entirely new project.

NVIDIA’s explanation also calls for keeping findings, mitigation and retest results together, with an owner for remaining gaps. When evidence does not support the intended use, options include narrowing the deployment, selecting a different model or postponing release. Recording the decision is part of the work.

The guide supports risk management; it does not certify compliance

The NIST AI Risk Management Framework is intended for voluntary use. ISO/IEC 23894 provides guidance on integrating AI risk management into organizational activity. References to these resources do not turn the Microsoft–NVIDIA paper into a regulator’s approval or establish that a deployment meets every applicable law.

A useful assessment keeps legal requirements and internal acceptance criteria distinguishable. The organization may decide that a particular residual error rate is tolerable for an internal task. That decision does not override a restriction on disclosing personal data or a contractual prohibition on using customer information for training.

The paper is also a joint industry resource. Both publishers sell AI infrastructure and services. Its method can be assessed on its substance without treating a vendor’s evaluation tool as mandatory or assuming that a branded safeguard fits every architecture. Choose controls that address the identified failure and verify them in the system where they will operate.

Operationally, the most useful record links a risk scenario to a control, an owner and evidence. If the concern is an answer exposing another customer’s record, the record should explain the restriction that prevents that outcome and the tests supporting it. A completed questionnaire with no connection to the implementation leaves the central question unanswered.

How Captain Compliance helps put the review into practice

Captain Compliance’s AI Governance software supports inventories of systems, models, agents and use cases, with assessments, approval workflows and control evidence. For an open-weight deployment, that record can identify the approved purpose, responsible owner and review conditions. The organization can preserve the evidence behind its approval rather than rely on a model name appearing on an approved-tools list.

Assessment workflows help teams collect information from the people responsible for privacy, engineering, security and procurement. Findings can lead to assigned mitigation and a documented review decision. A proposed model modification or new integration can then be assessed against the existing record instead of starting with an unrelated questionnaire.

Data mapping supplies context about source systems, processing purposes, recipients and retention. In a retrieval-based application, it helps document which collections feed the service and where outputs or logs may go. The map needs validation by the teams operating those systems; it should not be treated as automatic discovery of every information flow.

Technical safeguards still need implementation in the relevant model service, application and infrastructure. Governance software can assign the requirement and preserve test evidence. Access controls and output restrictions must actually enforce the approved boundary, with runtime capabilities depending on the supported integrations.

A practical first project is one deployment the organization understands well. Define its permitted use, investigate the relevant failure scenarios and assemble the evidence needed for an approval decision. The resulting record should make clear what was tested, which gaps remain and what change would require another review.

Book a Captain Compliance demo to discuss AI inventories, privacy assessments and the evidence supporting your model deployments.