We analyzed the policies outlined in Microsoft’s Frontier Governance Framework from February 2026, the second version.
The criterion is well addressed, with the four risk areas from the Code of Practice (CBRN weapons, offensive cyberoperations, harmful manipulation and loss of control) covered, as well as advanced autonomy (including AI R&D).
Quotes:
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated. In formulating this list, we have benefited from the advice of
both internal and external experts.
Chemical, biological, radiological, and nuclear (CBRN) weapons. A model’s ability to
provide significant capability uplift to an actor seeking to develop and deploy a chemical,
biological, radiological, or nuclear weapon.
Offensive cyberoperations. A model’s ability to provide significant capability uplift to an
actor seeking to carry out highly disruptive or destructive cyberattacks, including on critical
infrastructure.
Advanced autonomy. A model’s ability to complete expert-level tasks autonomously,
including AI research and development.
Loss of control. A model’s ability to undermine effective human control through adaptive,
deceptive, or self-reinforcing mechanisms such that, when deployed, a model can no longer
be reliably directed, modified, or shut down.
Harmful manipulation. A model’s ability to strategically distort human behavior or beliefs
at scale.” (p.4)
The framework states that it covers risks that “threaten national security or pose at-scale public safety risks if not appropriately mitigated”, demonstrating their exclusion criteria. They also specify that risks that are not covered in this framework are covered by other frameworks, to ensure no risks fall between the cracks. They make reference to some of these non-covered risks, such as bias, in the document.
Quotes:
“Risks that are not included as tracked high-risk capabilities because they do not threaten national security or pose at-scale public safety
risks are addressed through our broader AI governance and risk management framework.” (p.4)
The framework acknowledges the uncertainty regarding AI capabilities and states that the company performs “open-ended red teaming, research, horizon scanning, conversations with external experts, literature reviews, and incident monitoring” to study risks that might emerge. No detail is provided about what these processes look like, however, or what resources they are conducted with.
Quotes:
“AI technology continues to develop rapidly, and there remains uncertainty over which capabilities may emerge and when. We continue to study a range of potential capability-related risks that could emerge through open-ended red teaming, research, horizon scanning, conversations with external experts, literature reviews, and incident monitoring.” (p.4)
The framework mentions conversations with external experts to identify new risks, which is positive, but there is no mention of external experts specifically being provided with access to the model to red team it for new failure modes.
Quotes:
“AI technology continues to develop rapidly, and there remains uncertainty over which capabilities may emerge and when. We continue to study a range of potential capability-related risks that could emerge through open-ended red teaming, research, horizon scanning, conversations with external experts, literature reviews, and incident monitoring.” (p.4)
The framework includes the use of risk modeling for each high-risk capability, and these include “threat actors, unique ways an AI model may increase or decrease risk, technical bottlenecks, and the severity and likelihood of the scenario resulting in significant impacts”. However, they do not publish the redacted risk models.
Quotes:
“For each tracked high-risk capability, we conduct risk modeling to guide our red teaming and evaluation practices. These risk models, informed by external experts, are designed to prioritize specific scenarios, threat actors, unique ways an AI model may increase or decrease risk, technical bottlenecks, and the severity and likelihood of the scenario resulting in significant impacts. The technical bottlenecks identified in these risk models inform our leading indicators assessment to rule out certain scenarios early in the assessment process. The prioritized scenarios are used to guide risk-specific red teaming and evaluations in our deeper capability assessment.” (p.5)
The methodology for risk modeling is not described in any detail, but it is positive that “they are designed to prioritize specific scenarios, threat actors, unique ways an AI model may increase or decrease risk, technical bottlenecks, and the severity and likelihood of the scenario resulting in significant impacts”, and that the outputs of these models then inform the “leading indicators assessment”.
Quotes:
“For each tracked high-risk capability, we conduct risk modeling to guide our red teaming and evaluation practices. These risk models, informed by external experts, are designed to prioritize specific scenarios, threat actors, unique ways an AI model may increase or decrease risk, technical bottlenecks, and the severity and likelihood of the scenario resulting in significant impacts. The technical bottlenecks identified in these risk models inform our leading indicators assessment to rule out certain scenarios early in the assessment process. The prioritized scenarios are used to guide risk-specific red teaming and evaluations in our deeper capability assessment.” (p.5)
Without a lot of detail, the framework clearly describes a process by which the output of risk models get fed back into the process, although seemingly mostly to “rule out certain scenarios”.
Quotes:
“For each tracked high-risk capability, we conduct risk modeling to guide our red teaming and evaluation practices. These risk models, informed by external experts, are designed to prioritize specific scenarios, threat actors, unique ways an AI model may increase or decrease risk, technical bottlenecks, and the severity and likelihood of the scenario resulting in significant impacts. The technical bottlenecks identified in these risk models inform our leading indicators assessment to rule out certain scenarios early in the assessment process. The prioritized scenarios are used to guide risk-specific red teaming and evaluations in our deeper capability assessment.” (p.5)
There is an ongoing assessment to prioritize risks and differentially treat them.
Quotes:
“AI technology continues to develop rapidly, and there remains uncertainty over which capabilities may emerge and when. We continue to study a range of potential capability related risks that could emerge, conducting ongoing assessment of the severity and likelihood of these risks. We then operationalize the highest-priority risks through this framework.” (p.3)
“For each tracked high-risk capability, we conduct risk modeling to guide our red teaming and evaluation practices. These risk models, informed by external experts, are designed to prioritize specific scenarios, threat actors, unique ways an AI model may increase or decrease risk, technical bottlenecks, and the severity and likelihood of the scenario resulting in significant impacts. The technical bottlenecks identified in these risk models inform our leading indicators assessment to rule out certain scenarios early in the assessment process. The prioritized scenarios are used to guide risk-specific red teaming and evaluations in our deeper capability assessment.” (p.5)
The risk models are “informed by external experts”, which does not mean validated, but is positive.
Quotes:
“For each tracked high-risk capability, we conduct risk modeling to guide our red teaming and evaluation practices. These risk models, informed by external experts, are designed to prioritize specific scenarios, threat actors, unique ways an AI model may increase or decrease risk, technical bottlenecks, and the severity and likelihood of the scenario resulting in significant impacts. The technical bottlenecks identified in these risk models inform our leading indicators assessment to rule out certain scenarios early in the assessment process. The prioritized scenarios are used to guide risk-specific red teaming and evaluations in our deeper capability assessment.” (p.5)
There is no explicit reference to a risk tolerance, though implicitly the tolerance is given by the capability thresholds. For instance, “CBRN weapons, Critical: The model provides a meaningful uplift to an expert’s ability to develop a highly dangerous novel threat or significantly lowers the barriers to a low-skilled actor developing and delivering a known CBRN threat.” The risk tolerance is also implicitly described as risks arising from “capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated”; i.e., “threaten[ing] national security” or “pos[ing] at-scale public safety risks” is the risk tolerance.
To improve, they should set out the maximum amount of risk the company is willing to accept for each risk domain (though these need not differ between risk domains), ideally expressed in terms of probabilities and severity (economic damages, physical lives, etc), and separate from KRIs.
Quotes:
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated.” (p.3)
“CBRN weapons, Critical: The model provides a meaningful uplift to an expert’s ability to develop a highly dangerous novel threat or significantly lowers the barriers to a low-skilled actor developing and delivering a known CBRN threat.” (p.11)
“Offensive cyberoperations, Critical: The model provides a meaningful uplift to a low-skilled actor’s ability to identify and exploit major vulnerabilities or enables a well-resourced and expert actor to develop and execute novel and effective strategies against hardened targets.” (p.12)
The implicit risk tolerance of “threaten[ing] national security” or posing “at-scale public safety risks” is not a quantitative nor partly quantitative definition. Further, the implicit risk tolerances offered by the critical capability thresholds are not quantitative nor partly quantitative. To improve, the risk tolerance should be expressed fully quantitatively or as a combination of scenarios with probabilities.
Quotes:
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if
not appropriately mitigated.” (p.3)
The implicit risk tolerance of “threaten[ing] national security” or posing “at-scale public safety risks” is not a quantitative nor partly quantitative definition. The implicit risk tolerances given by the critical capability thresholds are not fully quantitative, either.
Quotes:
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if
not appropriately mitigated.” (p.3)
No evidence of engaging in public consultations or seeking guidance from regulators for risk tolerance.
Quotes:
No relevant quotes found.
No justification process: No evidence of considering whether their approach aligns with or deviates from established norms.
Quotes:
No relevant quotes found.
The framework describes two types of KRIs: those used for the ‘leading indicator assessment’, and those used for the ‘deeper capability assessment’.
For the leading indicator assessment KRIs, they give categories of benchmarks, but not the actual benchmarks nor their thresholds which are of sufficiently high risk. This could use more detail and could be more grounded in risk modelling.
For the deeper capability assessment KRIs, there could be more detail as to what “meaningful” or “significantly” constitute for the purpose of their risk assessment. However, these KRIs are grounded in risk modelling. To improve, KRIs should map directly to evaluation tests performed.
Quotes:
For instance, CBRN: “High: The model provides a meaningful uplift to
an expert’s (PhD level education in related
fields) ability to develop and deliver a
novel CBRN threat.
The model provides a meaningful uplift
to a medium-skilled actor’s (e.g., STEM
education) ability to develop and deliver a
known CBRN threat. Critical: The model provides a meaningful uplift
to an expert’s ability to develop a highly
dangerous novel threat or significantly
lowers the barriers to a low-skilled actor
developing and delivering a known CBRN
threat.” (p.11)
“Through the processes described in this framework, Microsoft’s most advanced models
are assessed for leading indicators of the framework’s high-risk capabilities. This is done
using state-of-the-art benchmarks for the following advanced general-purpose capabilities,
identified as precursors to high-risk capabilities:
• General reasoning
• Scientific and mathematical reasoning
• Long-context reasoning
• Spatial understanding and awareness
• Autonomy, planning, and tool use
• Advanced software engineering” (p.5)
Footnote 1, after “benchmarks”: “For a benchmark to be included in our suite of leading indicator assessments it must: 1) have low saturation (i.e., the best performing models
typically score lower than 70%); 2) measure an advanced capability, for example, mathematical reasoning, rather than an application-oriented
capability like financial market prediction; and 3) have a sufficient number of prompts to account for non-determinism in model output.” (p.5)
“Deeper capability assessment provides a robust indication of whether a model possesses a
tracked capability and, if so, whether this capability is at a low, medium, high, or critical risk
level, informing decisions about appropriate mitigations and deployment. We use qualitative
capability thresholds to guide this classification process as they offer important flexibility
across different models and contexts at a time of nascent and evolving understanding of
frontier AI risk assessment and management practice.” (p.5)
The framework describes two types of KRIs: those used for the ‘leading indicator assessment’, and those used for the ‘deeper capability assessment’.
For the leading indicator assessment KRIs, they give categories of benchmarks, but not the actual benchmarks nor their thresholds which are of sufficiently high risk. This could use more detail. However, these could likely be quantitatively defined.
For the deeper capability assessment KRIs, they explicitly do not have quantitative thresholds, preferring qualitative indicators. However, quantitative thresholds need not be inflexible, and in order to have transparency in risk decisions and provide clear guidance, KRIs should be quantitative where possible.
They should still publish the actual evaluations and thresholds which they currently operate under. This is because KRI-KCI pairings should be as predictable in advance as possible/allowing as little discretion as possible, and a qualitative threshold may be more arbitrary than a conservative quantitative estimate, until improved risk indicators can be developed.
Quotes:
“Through the processes described in this framework, Microsoft’s most advanced models
are assessed for leading indicators of the framework’s high-risk capabilities. This is done
using state-of-the-art benchmarks for the following advanced general-purpose capabilities,
identified as precursors to high-risk capabilities:
• General reasoning
• Scientific and mathematical reasoning
• Long-context reasoning
• Spatial understanding and awareness
• Autonomy, planning, and tool use
• Advanced software engineering” (p.5)
“Deeper capability assessment provides a robust indication of whether a model possesses a
tracked capability and, if so, whether this capability is at a low, medium, high, or critical risk
level, informing decisions about appropriate mitigations and deployment. We use qualitative
capability thresholds to guide this classification process as they offer important flexibility
across different models and contexts at a time of nascent and evolving understanding of
frontier AI risk assessment and management practice.” (p.5)
As part of the holistic risk assessment, the framework states that assessments of external risk factors “may also inform” risk determinations, which is a bit vague, but still commendable to include and point out.
Quotes:
Holistic risk assessment: The results of capability evaluation and an assessment of risk factors external to the model may also inform a determination as to whether a model has a tracked high-risk capability and to what level. This includes assessing the impact of potential system-level mitigations and societal and institutional factors that can impact whether and how a hazard materializes. This holistic risk assessment also considers the marginal capability uplift a model may provide over and above currently available tools and information, including currently available open-weights models. (p.7)
The framework gives qualitative containment KCI thresholds distinguishing between high-risk and critical risk KRIs, though more detail could be given as to what “the highest level of security safeguards” refers to, or what “protective against most cybercrime groups and insider threats” entails, e.g. what kind of threats or attacks.
Quotes:
“Models posing high-risk on one or more tracked capability will be subject to security
measures protective against most cybercrime groups and insider threats […]
Models posing critical risk on one or more tracked capability are subject to the highest level
of security safeguards.” (p.7)
No quantitative containment KCI thresholds given.
Quotes:
No relevant quotes found.
Practically no detail on deployment KCI thresholds is given. For each capability threshold, the deployment requirements are either “Deployment allowed in line with Responsible AI Program requirements” or “Further review and mitigations required.” The specific threshold given by the Responsible AI Program requirements should be explicitly detailed.
Quotes: “Deployment allowed in line with Responsible AI Program requirements” or “Further review and mitigations required.” (p.13)
There are no quantitative deployment KCI thresholds given.
No relevant quotes found.
There are no assurance processes KCIs defined. The framework does not provide recognition of there being KCIs outside of containment and deployment measures.
No relevant quotes found.
There is a clear acknowledgment that KRIs and KCIs pair together to bring residual risk below the risk tolerance, or “an acceptable level”. However, this is not grounded in risk modelling, and this fact is not proven or given justification for each (or any) risk domain. Further, their risk assessment is contingent on other companies’ risk tolerance: “This holistic risk assessment also considers the marginal capability uplift a model may provide over and above currently available tools and information, including currently available open-weights models. ”
Quotes:
“This framework assesses Microsoft’s most advanced AI models for signs that they may have
these capabilities and, if so, whether the capability poses a low, medium, high, or critical
risk to national security or public safety (more detail in Appendix I). This classification then
guides the application of appropriate and proportionate mitigations so that a model’s risks
remain at an acceptable level.” (p.3)
“The framework monitors Microsoft’s most capable AI models for leading indicators of high-risk capabilities and triggers deeper assessment if leading indicators are observed. As and when risks, are identified, proportional mitigations are applied so that risks are kept at an appropriate level. This approach provides confidence that highly capable models are identified before relevant risks emerge.” (p.2)
“This holistic risk assessment also considers the marginal capability
uplift a model may provide over and above currently available tools and information,
including currently available open-weights models.” (p.7)
There is a clear commitment to putting development (and deployment) on hold if a risk cannot be sufficiently mitigated. To improve, this could have more detail, for instance by linking to clear KCI thresholds so that the decision to pause is unambiguous. A process for pausing development could also be developed.
Quotes:
“If, during the implementation of
this framework, we identify a risk we cannot sufficiently mitigate, we will pause development
and deployment until the point at which mitigation practices evolve to meet the risk.” (p.8)
“The leading indicator assessment is run during pre-training, after pre-training is complete, and
prior to deployment to ensure a comprehensive assessment as to whether a model warrants
deeper inspection. This also allows for pause, review, and the application of mitigations as
appropriate if a model shows signs of significant capability improvements.” (p.5)
There is explicit reference to complying with specific standards and frameworks, and examples of containment measure requirements for high-risk and critical-risk capabilities. They are clearly linked to the high and critical capability thresholds, ie to these corresponding containment KCIs. To improve, the framework could be more specific on what will actually be implemented (rather than providing possible examples), as well as developing (or detailing the plan to develop) the containment measures for the critical-risk capabilities.
Quotes:
“The framework is built on a foundation of full-stack security, advancing comprehensive protections for key assets.” (p.2)
“As Microsoft operates the infrastructure on which its models will be trained and deployed,
we adopt an integrated full-stack approach to the security of frontier models, implementing
safeguards at the infrastructure, model, and system layers. Security measures will be tailored
to the specifics of each model, including its capabilities and the method by which it is made
available and integrated into a system, so that the marginal risks a model may pose are
appropriately addressed.” (p.7)
“We expect scientific understanding of how to best secure the AI lifecycle will advance
significantly in the coming months and years and will continue to contribute to, and apply,
security best practices as relevant and appropriate. This includes existing best practice
defined in leading standards and frameworks, such as NIST SP 800-53, NIST 800-218, SOC 2,
Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models, and Deploying
AI Systems Securely, as well as industry practices, including from the Frontier Model Forum.
Security safeguards are scaled up depending on the model’s pre-mitigation scores, with
more robust measures applied to models with high and critical risk levels.” (p.7)
“Models posing high-risk on one or more tracked capability will be subject to security
measures protective against most cybercrime groups and insider threats. Examples of
requirements for models having a high-risk score include:
• Restricted access, including access control list hygiene and limiting access to weights of
the most capable models other than for core research and for safety and security teams.
Strong perimeter and access control are applied as part of preventing unauthorized access.
• Defense in depth across the lifecycle, applying multiple layers of security controls that
provide redundancy in case some controls fail. Model weights are encrypted.
• Advanced security red teaming, using third parties where appropriate, to reasonably
simulate relevant threat actors seeking to steal the model weights so that security
safeguards are robust.
Models posing critical risk on one or more tracked capability are subject to the highest level
of security safeguards. Further work and investment are needed to mature security practices
so that they can be effective in securing highly advanced models with critical risk levels that
may emerge in the future. Appropriate requirements for critical risk level models will likely
include the use of high-trust developer environments, such as hardened tamper-resistant
workstations with enhanced logging, and physical bandwidth limitations between devices or
networks containing weights and the outside world.” (p.7)
They state that they engage in “advanced security red teaming”; more detail is required on the process of this red-teaming, and what constitutes sufficient proof. There is no process detailed for proving containment measures are sufficient for critical-risk models.
Importantly, they should detail proof in advance for why they believe the containment measures proposed will be sufficient to meet the KCI threshold. In addition, red-teaming is more an evidence gathering activity than a validation/proof; to improve, a case should be made for why they believe their containment measures to be sufficient.
Quotes:
For high-risk models: “Advanced security red teaming, using third parties where appropriate, to reasonably
simulate relevant threat actors seeking to steal the model weights so that security
safeguards are robust.” (p.7)
They state that they engage in “advanced security red teaming, using third parties where appropriate”; more detail is required on the process of this red-teaming, what constitutes sufficient proof. Involving third parties should not be discretionary but part of the verification process. There is no process detailed for proving containment measures are sufficient for critical-risk models. In addition, red-teaming is more an evidence gathering activity than a validation/proof; to improve, a case should be made for why they believe their containment measures to be sufficient.
Importantly, they should detail proof in advance for why they believe the containment measures proposed will be sufficient to meet the KCI threshold.
Quotes:
For high-risk models: “Advanced security red teaming, using third parties where appropriate, to reasonably
simulate relevant threat actors seeking to steal the model weights so that security
safeguards are robust.” (p.7)
Whilst they define deployment measures in general, these are not tied to KCI thresholds nor specific risk domains. For instance, the deployment measures for models that are high-risk in cybersecurity may be different to deployment measures for models that are critical-risk in autonomous AI R&D.
Quotes:
“We apply state-of-the-art safety mitigations tailored to observed risks so that the model’s
risk level remains at low or medium once mitigations have been applied. […] Examples of safety mitigations we utilize include:
• Harm refusal, applying state-of-the-art harm refusal techniques so that a model does not
return harmful information relating to a tracked capability at a high or critical level to a
user. […]
• Deployment guidance, with clear documentation setting out the capabilities and limitations
of the model, including factors affecting safe and secure use and details of prohibited uses. […]
• Monitoring and remediation, including abuse monitoring in line with Microsoft’s Product
Terms and provide channels for employees, customers, and external parties to report
concerns about model performance, including serious incidents that may pose public
safety and national security risks. […] Other forms
of monitoring, including for example, automated monitoring in chain-of-thought outputs,
are also utilized as appropriate. […]
• Phased release, trusted users, and usage studies, as appropriate for models demonstrating
novel or advanced capabilities.” (p.8)
There is some implementation of proving deployment measures are sufficient, by testing that the post-mitigation model does not cross the same KRI threshold as the unmitigated model. More detail could be provided on what exactly the evaluation constitutes, and why they believe this to be sufficient proof. Further, proof should be provided ex ante for why they believe their deployment measures will meet the relevant KCI threshold.
Quotes:
“Post-mitigation capability assessment and safety buffer: Following application of safety
and security mitigations, the model will be re-evaluated to ensure capabilities are rated low
or medium and, if not, to guide further mitigation efforts.” (p.8)
There is no mention of third-party verification of deployment measures meeting the threshold.
No relevant quotes found.
Whilst there is a commitment to conduct further research in mitigations, this is not specifically for assurance processes. However, there is an acknowledgment that current mitigations are insufficient, garnering partial credit.
Quotes:
“Models posing critical risk on one or more tracked capability are subject to the highest level
of security safeguards. Further work and investment are needed to mature security practices
so that they can be effective in securing highly advanced models with critical risk levels that
may emerge in the future. Appropriate requirements for critical risk level models will likely
include the use of high-trust developer environments, such as hardened tamper-resistant
workstations with enhanced logging, and physical bandwidth limitations between devices or
networks containing weights and the outside world.” (p.7)
For all high and critical capability thresholds: “Deployment requirements: Further review and mitigations required.” (pp.11-14).
“We are focused on capabilities that could emerge in the short-to-medium term. Longer term or more speculative capabilities are the subject of ongoing research that we and
many others across industry and academia are invested in.” (p.2)
“We apply state-of-the-art safety mitigations tailored to observed risks so that the model’s
risk level remains at low or medium once mitigations have been applied. We will continue
to contribute to research and best-practice development, including through organizations
such as the Frontier Model Forum, and to share and leverage best practice mitigations as
part of this framework.” (p.8)
There is no mention of providing evidence that the assurance processes are sufficient.
No relevant quotes found.
There is no mention of assumptions essential for effective implementation of mitigation measures. There is some mention of needing to monitor chain of thought, but this doesn’t appear to be for the purpose of checking underlying assumptions are effective – instead, this is to monitor for abuse from the customer side of deployment.
However, there is possibly an implicit acknowledgment that assumptions are required for evaluations (ie, KRI assessment), as the robustness of the evaluation must be justified: “This evaluation also includes a statement on the robustness of the evaluation method used and any concerns about the effectiveness or validity of the evaluation.” Partial credit is granted for this. To improve, the framework should detail the key technical assumptions necessary for the assurance processes to meet the KCI threshold, and evidence for why these assumptions are justified.
Quotes:
“Monitoring and remediation, including abuse monitoring in line with Microsoft’s Product
Terms and provide channels for employees, customers, and external parties to report
concerns about model performance, including serious incidents that may pose public
safety and national security risks. We apply mitigations and remediation as appropriate to
address identified concerns and adjust customer documentation as needed. Other forms
of monitoring, including for example, automated monitoring in chain-of-thought outputs,
are also utilized as appropriate. We continue to assess the tradeoffs between safety and
security goals and legal and privacy considerations, optimizing for measures that can
achieve specific safety and security goals in compliance with existing law and contractual
agreements.” (p.8)
“This evaluation also includes a statement on the robustness of the evaluation method used and any concerns about the effectiveness or validity of the evaluation.” (pp.5-6)
There is a clear connection between elicitation effort and the resources available to threat actors, and some elicitation techniques are listed: “fine-tuning”, “multi-agent setup, leveraging prompt optmization, or connecting the model to whichever tools and plugins will maximize its performance.” More detail could be added on what these resources are assumed to be for threat actors, to explain why these elicitation methods are comprehensive enough. Further, specifics on e.g. compute used for finetuning could be added.
Quotes:
“Evaluations include concerted efforts at capability elicitation, i.e.,
applying capability enhancing techniques to advance understanding of a model’s full
capabilities. This includes fine-tuning the model to improve performance on the capability
being evaluated or ensuring the model is prompted and scaffolded to enhance the tracked
capability—for example, by using a multi-agent setup, leveraging prompt optimization,
or connecting the model to whichever tools and plugins will maximize its performance.
Resources applied to elicitation should be extrapolated out to those available to actors in
threat models relevant to each tracked capability.” (p.6)
Leading indicator assessments are conducted every 6 months, explicitly to account for post-training enhancements. Leading indicator assessments will also be run when fine-tuning compute is >1/3 of the base model. Leading indicator assessments are run during pre-training, after pre-training, after post-training and pre-deployment. Deeper capability assessments are conducted when there are material changes to the risk profile.
Quotes:
“A leading indicator assessment is run on any model that is in scope for frontier model requirements under applicable laws, such as the EU AI Act, California’s Transparency in Frontier AI Act (TFAIA), and New York’s Responsible AI Safety and Education (RAISE) Act. .” (p.6)
“We also run a leading indicator assessment when Microsoft substantially fine-tunes first- or third-party models, where the compute used for fine-tuning is more than 1/3 of the base model” (p.6)
“The leading indicator assessment is run during pre-training, after pre-training is complete, after post-training, and prior to deployment to ensure a comprehensive assessment as to whether a model warrants deeper inspection. ” (p.6)
“Models in scope of this framework will undergo leading indicator assessment at least every six months to assess progress in post-training capability enhancements, including fine-tuning and tooling. Any model demonstrating frontier capabilities is then subject to a deeper capability assessment
to provide strong confidence about whether it has a tracked capability and to what level, informing mitigations.” (p.5)
“After the first deeper capability assessment, we will conduct subsequent deeper capability assessments if there are material changes to the
deployed model’s risk profile (e.g., the ability to fine-tune the model, significant fine-tuning that might affect tracked high-risk capabilities, etc.).” (p.7)
There is an explicit consideration of incorporating frontier post-training enhancements when re-evaluating models to ensure KRIs are not crossed unnoticed. An improvement would be to add detail on how they account(ed) for how post-training enhancements’ risk profiles change with different model structures – namely, post-training enhancements are much more scalable with reasoning models, as inference compute can often be scaled to improve capabilities.
Quotes:
“Models in scope of
this framework will undergo leading indicator assessment at least every six months to assess
progress in post-training capability enhancements, including fine-tuning and tooling. Any
model demonstrating frontier capabilities is then subject to a deeper capability assessment
to provide strong confidence about whether it has a tracked capability and to what level,
informing mitigations.” (p.5)
There is no mention of having the evaluation methodology vetted by third parties.
No relevant quotes found.
There is no mention of evaluations being replicated; they mention that external experts may be “involved” in evaluations, at Microsoft’s discretion. However, this does not necessarily mean the external experts will be conducting the evaluations: it is more likely they will be participants of internally run expert elicitation or red-teaming, for instance.
Quotes:
“Deeper capability assessment […] involves robust evaluation of whether a model possess tracked capabilities at high or critical levels, including through adversarial testing and systematic measurement using state-of-the-art methods. […] As appropriate, evaluations involve qualified and expert external actors that meet relevant security standards, including those with domain-specific expertise.” (pp.5-6)
There is a mention of monitoring in terms of reporting concerns, as well as automated monitoring in chain-of-thought outputs. However, this is not linked explicitly to monitoring mitigation effectiveness, and it is not clear if monitoring is ongoing. To improve, the framework should describe systematic, ongoing monitoring to ensure mitigation effectiveness is tracked continuously such that the KCI threshold will still be met, when required.
Quotes:
“Monitoring and remediation, including abuse monitoring in line with Microsoft’s Product
Terms and provide channels for employees, customers, and external parties to report
concerns about model performance, including serious incidents that may pose public
safety and national security risks. We apply mitigations and remediation as appropriate to
address identified concerns and adjust customer documentation as needed. Other forms
of monitoring, including for example, automated monitoring in chain-of-thought outputs,
are also utilized as appropriate. We continue to assess the tradeoffs between safety and
security goals and legal and privacy considerations, optimizing for measures that can
achieve specific safety and security goals in compliance with existing law and contractual
agreements.” (p.8)
There is no mention of KCIs protocols being vetted by third parties.
No relevant quotes found.
There is no mention of control evaluations/mitigation testing being replicated or conducted by third-parties.
No relevant quotes found.
There is a commitment to share substantial detail, seemingly with members of the Frontier Model Forum, on models’ KRI levels and corresponding KCI measures.
There is also a commitment to publishing capabilities, evaluations and risk classification publicly. More detail could be given on ex ante criteria for redacting information, to avoid discretion. To improve, the company should commit to alerting authorities if critical thresholds are reached.
Quotes:
“We will continue to contribute to research and best-practice development, including through organizations such as the Frontier Model Forum. […] Examples of safety mitigations we utilize include: […] Deployment guidance, with clear documentation setting out the capabilities and limitations
of the model, including factors affecting safe and secure use and details of prohibited uses.
This documentation will also include a summary of evaluation results, the deeper capability
assessment, and safety and security mitigations. For example, the documentation could
outline specific capabilities and tasks that the model robustly fails to complete which would
be essential for a high or critical risk rating.” (p.8)
“Information about the capabilities and limitations of the model, relevant evaluations, and the
model’s risk classification will be shared publicly, with care taken to minimize information
hazards that could give rise to safety and security risks and to protect commercially sensitive
information.” (p.9)
“Evaluations are documented in a consistent fashion setting out the capability being evaluated, the method used, and evaluation results.” (p.5)
No commitment to permitting the reports, which detail the results of external evaluations (i.e. any KRI or KCI assessments conducted by third parties), to be written independently and without interference or suppression.
Quotes:
No relevant quotes found.
There is a commitment to revisiting “our list of tracked capabilities frequently, ensuring it remains up to date in light of technological developments and improved understanding of model capabilities, risks, and mitigations.” To improve, more detail on (a) how this improved understanding will be gotten, (b) a process for identifying novel risks and (c) justification for why this process is likely to detect novel risks, could be given.
Quotes:
“AI technology continues to develop rapidly, and there remains uncertainty over which capabilities may emerge and when. We continue to study a range of potential capability related risks that could emerge, conducting ongoing assessment of the severity and likelihood of these risks. We then operationalize the highest-priority risks through this framework. We will revisit our list of tracked capabilities frequently, ensuring it remains up to date in light of technological developments and improved understanding of model capabilities, risks, and mitigations.” (p.3)
They mention that they conduct “ongoing assessment of the severity and likelihood of these [potential] risks. We then operationalize the highest-priority risks through this framework.” However, details on how they assess the severity and likelihood of novel risks is not given. More detail could be added on how the “explicit discussion on how this framework may need to be improved” will lead to incorporations of risks identified post-deployment. To improve, a process which triggers risk modelling exercises once a novel risk domain or threat model is found, and analyzing how this could intersect with existing threat models, could be conducted.
Quotes:
“We continue to study a range of potential capability related risks that could emerge, conducting ongoing assessment of the severity and likelihood of these risks. We then operationalize the highest-priority risks through this framework. We will revisit our list of tracked capabilities frequently, ensuring it remains up to date in light of technological developments and improved understanding of model capabilities, risks, and mitigations.” (p.3)
“We will update our framework to keep pace with new developments. Every six months, we will have an explicit discussion on how this framework may need to be improved. We acknowledge that advances in the science of evaluation and risk mitigation may lead to additional requirements in this framework or remove the need for existing requirements. Any updates to our practices will be reviewed by Microsoft’s Chief Responsible AI Officer prior to their adoption. Where appropriate, updates will be made public at the same time as we adopt them.” (p.11)
Although not specified in more detail than Executive Officers or delegates, the framework specifies that certain executive officers hold the responsibility for making key decisions regarding risks.
Quotes:
“Documentation regarding the pre-mitigation and post-mitigation capability assessment will be provided to Executive Officers responsible for Microsoft’s AI governance program (or their delegates) along with a recommendation for secure and trustworthy deployment setting out the case that: 1) the model has been adequately mitigated to low or medium risk level, 2) the marginal benefits of a model outweigh any residual risk and 3) the mitigations and documentation will allow the model to be deployed in a secure and trustworthy manner. The Executive Officers (or their delegates) will make the final decision on whether to approve the recommendation for secure and trustworthy deployment.” (p.9)
No mention of a management risk committee.
No relevant quotes found.
The framework outlines clearly which decisions are made, the basis on which they will be made and who makes the decisions.
Quotes:
“If, during the implementation of this framework, we identify a risk we cannot sufficiently mitigate, we will pause development and deployment until the point at which mitigation practices evolve to meet the risk.” (p.8)
“Holistic risk assessment: The results of capability evaluation and an assessment of risk factors external to the model then inform a determination as to whether a model has a tracked capability and to what level. This includes assessing the impact of potential system level mitigations and societal and institutional factors that can impact whether and how a hazard materializes. This holistic risk assessment also considers the marginal capability uplift a model may provide over and above currently available tools and information, including currently available open-weights models.” (p.6)
“The Executive Officers (or their delegates) will make the final decision on whether to approve the recommendation for secure and trustworthy deployment. The Executive Officers (or their delegates) are also responsible for assessing that the recommendation for secure and trustworthy deployment and its constituent parts have been developed in a good faith attempt to determine the ultimate capabilities of the model and mitigate risks.” (p.9)
The framework clearly outlines the process to be followed in the case of public safety or national security incidents and has established incident response policies.
Quotes:
“Where we or others identify serious or critical incidents that may pose public safety or national security risks, we operationalize incident response policies and processes, conduct investigations, provide reports to regulatory authorities and impacted customers as appropriate, apply mitigations and remediation, and manage after-action reviews and policy, process, and practice updates.” (p.9)
The company has a Chief Responsible AI Officer, which should be equivalent to this function.
Quotes:
“Any updates to our practices will be reviewed by Microsoft’s Chief Responsible AI Officer prior to their adoption.” (p.9)
No mention of an advisory committee.
No relevant quotes found.
The framework lists specific capabilities that are tracked.
Quotes:
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated.” (p.3)
The framework mentions that “internal experts who are independent from the model development team” make the decision to involve qualified external actors.
Quotes:
“Capability evaluation: This involves robust evaluation of whether a model possesses tracked capabilities at high or critical levels, including through adversarial testing and systematic measurement using state-of-the-art methods. Evaluations are documented in a consistent fashion setting out the assumptions regarding the model, capability being evaluated, the method used, and evaluation results. This evaluation also includes a statement on the robustness of the evaluation method used, assumptions underlying the evaluation, and any concerns about the effectiveness or validity of the evaluation. We engage qualified third parties to conduct evaluations in ways that are appropriate to the risk profile of the model; sometimes this requires granting API access for the third party to conduct evaluations, and sometimes it requires a review of testing conducted internally. The decision to involve qualified and expert external actors that meet relevant security standards, including those with domain-specific expertise, is made by internal experts who are independent from the model development team. Where applicable, we disclose the extent to which and how qualified third parties are involved.” (p.7)
The framework specifies that documentation will be provided to senior management for decision making.
Quotes:
“Documentation regarding the pre-mitigation and post-mitigation capability assessment will be provided to Executive Officers responsible for Microsoft’s AI governance program.” (p.9)
No mention of a central risk function.
No relevant quotes found.
Microsoft states the Framework is “subject to Microsoft’s broader corporate governance procedures, including independent internal audit.” However, it is unclear whether this audit function reviews the effectiveness of frontier AI risk controls specifically, or merely checks procedural compliance with the Framework. To improve, Microsoft could clarify the scope of internal audit with respect to AI risk management.
Quotes:
“This framework is subject to Microsoft’s broader corporate governance procedures, including independent internal audit.” (p.9)
The framework includes having external third parties conduct evaluations of models, “in ways appropriate to the risk profile”. Further, “they disclose the extent to which and how qualified third parties are involved”, but only where applicable.
Quotes:
“Capability evaluation: This involves robust evaluation of whether a model possesses tracked capabilities at high or critical levels, including through adversarial testing and systematic measurement using state-of-the-art methods. Evaluations are documented in a consistent fashion setting out the assumptions regarding the model, capability being evaluated, the method used, and evaluation results. This evaluation also includes a statement on the robustness of the evaluation method used, assumptions underlying the evaluation, and any concerns about the effectiveness or validity of the evaluation. We engage qualified third parties to conduct evaluations in ways that are appropriate to the risk profile of the model; sometimes this requires granting API access for the third party to conduct evaluations, and sometimes it requires a review of testing conducted internally. The decision to involve qualified and expert external actors that meet relevant security standards, including those with domain-specific expertise, is made by internal experts who are independent from the model development team. Where applicable, we disclose the extent to which and how qualified third parties are involved.” (p.7)
There is no mention of a Board committee, but the framework specifies that Microsofts’s broader corporate governance, which could be expected to include the Board, applies.
Quotes:
“This framework is subject to Microsoft’s broader corporate governance procedures, including independent internal audit.” (p.9)
No mention of any additional governance bodies.
No relevant quotes found.
The framework includes a statement regarding its purpose to manage large-scale risks.
Quotes:
“Microsoft’s Frontier Governance Framework manages potential national security and at-scale public safety risks that could emerge as AI models increase in capability.” (p.2)
There are no direct mentions of elements of risk culture, but the framework refers to security best practices.
Quotes:
“We expect scientific understanding of how to best secure the AI lifecycle will advance significantly in the coming months and years and will continue to contribute to, and apply, security best practices as relevant and appropriate.” (p.7)
The company has an established whistleblower mechanism and encourage employees to report concerns. They provide anonymous reporting channels and protection from retaliation. They specifically provide an annual companywide course on the importance of speak-up culture. They also provide channels for reporting for customers and external parties. Concerns can include both potential vulnerabilities as well as incidents.
Quotes:
“We provide channels for employees, customers, and external parties to report concerns about model performance and potential vulnerabilities or incidents.”(p.9)
“All Microsoft employees are encouraged to report concerns relating to this framework and its implementation, as well as AI governance at Microsoft more broadly, using our existing internal reporting channels, which includes an option for anonymous reporting. Employees who raise concerns are protected from retaliation. In addition, Microsoft reinforces the importance of speak up culture and reporting concerns in Trust Code (Standards of Business Conduct), our annual companywide course on responsible business practices. ” (p.9)
The framework lists the risks that are being tracked and what information of them will be shared externally, as well as which risks are not in scope for the framework.
Quotes:
” Focused on high-risk capabilities. The framework operationalizes state-of-the-art risk management practices for a set of risks that are connected to high-impact model capabilities. We are focused on capabilities that could emerge in the short-to-medium term. Longerterm or more speculative capabilities are the subject of ongoing research that we and many others across industry and academia are invested in. Evaluations and mitigations under the framework target capability-related risks, complementing Microsoft’s broader AI governance program that manages a broader set of risks, including more culturally contextual risks that are heavily shaped by use case and deployment environments, as well as laws and norms that vary across regions.” (p.3)
“This framework tracks the following capabilities…Chemical, biological, radiological, and nuclear (CBRN) weapons…Offensive cyberoperations…Advanced autonomy…Loss of control…Harmful manipulation” (p.4)
“Information about the capabilities and limitations of the model, relevant evaluations, and the model’s risk classification will be shared publicly, with care taken to minimize information hazards that could give rise to safety and security risks and to protect commercially sensitive information.” (p.9)
The framework provides plenty of details on the governance structure and how it is integrated into the company’s broader AI governance program. The company commits to making material revisions public within 30 days.
Quotes:
“This framework is integrated with Microsoft’s broader AI governance program, which sets out a comprehensive risk management program that applies to all AI models and systems developed and deployed by Microsoft.” (p.2)
“We will update our framework to keep pace with new developments. At least every twelve months, we will have an explicit discussion on how this framework may need to be improved. We acknowledge that advances in the science of evaluation and risk mitigation may lead to additional requirements in this framework or remove the need for existing requirements. Any updates to our practices will be reviewed by Microsoft’s Chief Responsible AI Officer prior to their adoption. All material revisions will be made public within 30 days of adoption, and, where appropriate, may be published at the same time as we adopt them.” (p.10)
“In addition to high-risk capabilities, a broader set of risks are governed when Microsoft develops and deploys AI technologies. Under Microsoft’s comprehensive AI governance program, frontier models—as well as other models and AI systems—are subject to relevant evaluation, with mitigations then applied to bring overall risk to an appropriate level…Our efforts to assess and mitigate risks related to this framework’s tracked capabilities benefit from this broadly applied governance program, which is continuously improved. ” (p.4)
The framework is fairly detailed on what information will be shared externally and with whom.
Quotes:
“Information on model or system performance, responsible use, and suggested system-level evaluations is shared with downstream actors integrating models into systems, including external system developers and deployers and teams at Microsoft building models. Appropriate information sharing is important to facilitate mitigation of a broader set of risks, many of which are heavily shaped by use case and deployment context as well as laws and norms that vary across jurisdictions.” (p.4)
“Microsoft will prioritize ongoing contributions to this work and expand its collaboration with government, industry, and civil society, including through organizations like the Frontier Model Forum, to solve the most pressing challenges in AI risk management.” (p.11)
Information about the capabilities and limitations of the model, relevant evaluations, and the model’s risk classification will be shared publicly, with care taken to minimize information hazards that could give rise to safety and security risks and to protect commercially sensitive information. We will report information about tracked high-risk capabilities as required by regulation and as appropriate to effectively manage frontier risks. (p.10)