Did AI companies publish the safety frameworks they promised?

The reasoning

METR's March 2025 review of frontier AI safety policies [E1], dated after the 10 February 2025 deadline, lists twelve companies as having published such policies, and 01.AI is not among them. The criteria name this kind of dated third-party tracker as positive evidence of non-delivery. The Yi-Lightning technical report mentioned in [E2] describes a model's architecture and safety stack, not a safety framework focused on severe risks, so it does not meet the criteria.

Its evidence came from METR’s list, a page that cites it, or another tracker of these commitments (E1), so it is set aside in the comparison with that list.

The reasoning

METR's March 2025 report (E1, E2) lists Amazon among companies that had published frontier AI safety policies, including capability thresholds, so a qualifying framework existed by March 2025. Neither METR nor Amazon's May 2025 blog (E3), which says only that the framework was published 'earlier this year', gives a publication date. The evidence does not show whether the framework was published on or before 2025-02-10, and there is no report of delay.

Its evidence came from METR’s list, a page that cites it, or another tracker of these commitments (E1, E2), so it is set aside in the comparison with that list.

The reasoning

Anthropic's dated official announcement [E2] records publication in October 2024 of an updated Responsible Scaling Policy. It describes the policy as a risk governance framework for catastrophic risks from frontier AI systems, with capability thresholds and required safeguards. This updates the RSP first published in September 2023 [E1], and both dates fall before the 2025-02-10 deadline. The independent Frontier Model Forum [E3] lists Anthropic's RSP among the frameworks published under the Seoul commitments, and the later version 2.1 [E4] shows the policy remained in force.

The reasoning

Cohere's Secure AI Frontier Model Framework (E1, E2) presents itself as a first published version and discusses catastrophic or severe risks, though it says Cohere's own approach centres on risks that are known or observable today. METR's March 2025 report (E3) independently lists Cohere among companies with published frontier AI safety policies, but it is dated after the deadline and gives no publication date. The framework's title shows only 'February 2025', which covers dates on both sides of the 10 February deadline. The only exact date is 2025-02-01 in the metadata of an unidentified third-party summary page (E4), too weak on its own to establish that the framework was public by the deadline.

Its evidence came from METR’s list, a page that cites it, or another tracker of these commitments (E3), so it is set aside in the comparison with that list.

The reasoning

G42's press release dated 6 February 2025 announced publication of its Frontier AI Safety Framework and linked to the full downloadable document (E1, E4). An independent outlet, The National, reported the launch the same day and said the framework covers biological and cyber security risks, which are severe-risk domains (E2). METR's later tracker independently lists G42 among companies that have published frontier AI safety policies (E3). Together these show a qualifying public framework was published before the 10 February 2025 deadline.

Its evidence came from METR’s list, a page that cites it, or another tracker of these commitments (E3), so it is set aside in the comparison with that list.

The reasoning

Google DeepMind's 17 May 2024 post introduced the Frontier Safety Framework, which it describes as focused on severe risks from powerful model-level capabilities (E1). Under the criteria, a framework published before the commitment counts if still public by the deadline. Google DeepMind announced an updated version on 4 February 2025 (E3), and Semafor independently reported the same day that the first version was released in May 2024 and an update was published ahead of the Paris summit (E4). A qualifying framework was therefore public before 10 February 2025.

The reasoning

IBM published a post on 2025-02-07, before the deadline, describing its AI governance framework and organisational culture as aligning with the Seoul commitments (E3). The independent SaferAI tracker states that IBM's document summarised existing AI governance practices without addressing severe AI risks. It also leaves IBM off its list of companies that fulfilled the commitment (E1, E2). Under the criteria, a document lacking the severe-risk focus that IBM presented as fulfilment by the deadline resolves mostly false.

The reasoning

E2 confirms that Seoul signatories committed to publish safety frameworks before the February 2025 French summit, but it does not report on Inflection's progress. E3 lists only four companies (not Inflection) as having published frameworks around September 2024, which is well before the deadline and so does not show the state of affairs on 2025-02-10. E1 is Inflection's own page with 2023-era content describing a future technical paper; it is not shown to be a severe-risk framework published by the deadline. No dated evidence settles whether a qualifying framework existed as of 2025-02-10.

The reasoning

Magic's AGI Readiness Policy v1.0 is dated July 2, 2024 on Magic's own site (E1) and commits to dangerous-capability evaluations and protective measures before deploying frontier-exceeding models (E2). An independent tracker records the same v1.0 date via the METR policy index (E3), and SaferAI assesses the published policy's risk indicators, such as LiveCodeBench thresholds (E4). This shows a severe-risk safety framework was public before the 2025-02-10 deadline.

Its evidence came from METR’s list, a page that cites it, or another tracker of these commitments (E3), so it is set aside in the comparison with that list.

The reasoning

Meta published its Frontier AI Framework on 2025-02-03, before the AI Action Summit began on 2025-02-10. Independent sources (E2, E3, E4) confirm this release date. The framework text held by ETO AGORA (E3) explicitly references the Frontier AI Safety Commitments and covers catastrophic outcomes from frontier models, and Meta's announcement (E1) and The Midas Project (E4) describe its focus on cybersecurity and chemical/biological weapons risks. Content quality does not matter under the criteria, so the commitment resolved true and on time.

The reasoning

The Microsoft Frontier Governance Framework [E1] is a public document labelled Version 1 (February 2025). It is framed around severe national-security and public-safety risks from Microsoft's most advanced models, with tracked capabilities (CBRN, offensive cyber, advanced autonomy) and capability thresholds, so it meets the substance of the commitment. Microsoft's own records say it was shared publicly in February 2025 [E2] and announced during the Paris AI Action Summit [E3], which places publication around the 10 February deadline. The evidence does not give an exact date to show whether it came before, on, or just after the deadline, so on_time is left undetermined.

The two model checks disagreed on this one, so it is flagged.

The reasoning

The independent SaferAI Frontier Risk Management Tracker [E1] names MiniMax among six companies that did not publish a safety framework despite the commitment. This is a dated report from well after the 2025-02-10 deadline, so it is positive evidence that no qualifying framework existed by then. E3 confirms MiniMax was a signatory of the commitments, and E2 reports six signatories without frameworks, consistent with E1, though E2 does not name MiniMax.

The reasoning

E1, an independent frontier risk management tracker, lists Mistral AI among six companies that did not publish their frameworks despite the Seoul commitment. The page is undated, but a later account that no framework had been published means none existed by the 2025-02-10 deadline. Dated post-summit watchdog reports (E3, 2025-02-19; E2, 2025-02-28) likewise say Mistral AI fell short of the Seoul commitment and did not publicly fulfil its components. None of the evidence identifies any severe-risk safety framework from Mistral published by the deadline.

The two model checks disagreed on this one, so it is flagged.

The reasoning

Naver's own press release (E1, E2) dated 17 June 2024 announces its AI Safety Framework (ASF). It describes the ASF as identifying severe risks, including severe disempowerment of the human species and misuse, and setting out countermeasures such as quarterly risk assessments for frontier AI and a risk assessment matrix. An independent Korean newspaper (E3) confirms that Naver publicly released the ASF on 17 June 2024 through its tech channel, as a framework for identifying, assessing and managing risks across AI development and deployment. Together these show a qualifying framework was published well before the 10 February 2025 deadline.

The reasoning

NVIDIA's Frontier AI Risk Assessment exists and describes risk identification and analysis (E2, E4), and the Frontier Model Forum listed it among published frontier AI frameworks by 22 April 2025 (E3). The only date on the document itself is 'Applicable from August 2025' (E1), and the evidence does not show whether this is the first version or a later revision. None of the evidence shows whether a framework was published by 11 February 2025, nor does it show that one was not.

The reasoning

OpenAI's own pre-summit update dated 7 February 2025 points to its Preparedness Framework, published in December 2023, as its safety framework covering frontier risks such as CBRN, persuasion and autonomy [E1]. Independent sources confirm the framework was published in December 2023 and set out severe-risk categories (cybersecurity, CBRN, persuasion, model autonomy) with Low to Critical tiers that gate deployment [E3][E4]. METR's March 2025 tracker lists OpenAI among companies that have published frontier AI safety policies [E2]. The criteria explicitly accept an existing framework published before the deadline, so the promise resolved true and on time.

Its evidence came from METR’s list, a page that cites it, or another tracker of these commitments (E2), so it is set aside in the comparison with that list.

The reasoning

Samsung's own later sustainability report [E1] says it announced an AI Safety Framework at the Paris AI Action Summit in February 2025, but it does not give the exact date. The framework document [E2] centres on on-device AI in consumer electronics and on processes such as data governance, red teaming, fairness and transparency; its extracted text does not mention severe or catastrophic risks, risk thresholds or intolerable-risk levels. The independent Seoul Commitment Tracker [E3], keyed to the 10 February 2025 deadline, gives Samsung a grade of F, though it does not define the grade. Taken together, the evidence indicates that a related document was published around the deadline but not one focused on severe risks as the criteria require.

Its evidence came from METR’s list, a page that cites it, or another tracker of these commitments (E3), so it is set aside in the comparison with that list.

The two model checks disagreed on this one, so it is flagged.

The reasoning

E3, an independent report, states that on the deadline day xAI released an eight-page document watermarked 'DRAFT' as its safety framework under the commitment. It criticises the document for lacking measurable risk thresholds, but the criteria set quality and draft status aside. E4 describes xAI's draft Risk Management Framework as dated February 20, 2025, which may reflect a later revision and leaves some uncertainty about the exact date. Neither source quotes the document's text on severe risks; E3 treats it as the framework required by the commitment, so the verdict is mostly true rather than true.

The two model checks disagreed on this one, so it is flagged.

The reasoning

E1 and E2, an independent 2026 write-up citing SaferAI's December 2025 analysis, state that Zhipu signed the Seoul Frontier AI Safety Commitments and pledged a framework by the Paris summit, but 'never published' one and had not done so as of mid-2026. This is a later dated account that covers the 2025-02-10 deadline. E3 and E4 confirm Zhipu was a signatory and that several signatories did not deliver, but they do not name Zhipu's status. Taken together, the evidence positively indicates no Zhipu severe-risk framework existed by the summit.

The two model checks disagreed on this one, so it is flagged.