Generative AI: product safety standards
- Document
- 22 January 2025
- Event
- 22 January 2025
- Retrieved
- 16 September 2026
The classroom note
A school leader drafting a request for proposals for a generative AI product now has a checklist to hand to bidders that did not exist in 2023. On 22 January 2025, England's Department for Education published Generative AI: product safety standards, addressed explicitly to edtech developers and suppliers to schools and colleges rather than to schools themselves, though the department says schools and colleges may also find them helpful in assessing which AI products are safe for use in education. The document was updated again on 19 January 2026 to add standards on cognitive development, emotional and social development, mental health, and manipulation, so a description of its contents reflects the guidance as retrieved on 16 September 2026, not a fixed 2025 text.
What the evidence says
The standards are specific enough to be checked against a real product rather than aspirational. On filtering, they require moderation adjusted based on different levels of risk, age, appropriateness and the user's needs, detection across multiple languages, images, common misspellings and abbreviations, and full content moderation regardless of the device used, including bring your own device. On monitoring, products should flag harmful searches, give real-time user notification in age-appropriate language when harmful or inappropriate content has been blocked, and track metrics including the rate of requests for cognitive offloading and a user's level of personal and emotional engagement, reporting concerns to a school's Designated Safeguarding Lead. On data, the standards prohibit using personal data for any commercial purposes, including further model training and fine-tuning, without confirmation of appropriate lawful basis, and bar using learner or teacher-created work to train models without consent. A related department page, updated 12 August 2025, directs schools to these standards as part of broader safety guidance.
The implementation question
The standards are guidance, not statutory regulation; nothing in the published document compels a vendor to comply or a school to reject a non-compliant product. Their practical mechanism is procurement leverage: a school can put these clauses into a contract or a request for proposals and ask a vendor to attest against each one, converting a voluntary standard into a purchasing requirement the school itself chooses to enforce. That shifts the verification burden onto the school, which must actually test claims like contextual filtering accuracy or data-training restrictions rather than accept a vendor's word.
What holds and what fails
The standards hold as a detailed, checkable procurement reference, precise enough that a school can ask a vendor to demonstrate compliance clause by clause rather than accept a general safety claim. They fail to the extent no independent body verifies vendor compliance before a product reaches a classroom; the standards describe what a safe product should do, not a certification a school can look up. Treating the standards' existence as proof any given product meets them would be an editorial error the document itself does not invite.
- Has the vendor been asked to attest against each standard individually, or only to a general claim of safety?
- Does the product's data-training practice match the standard's restriction on commercial use of personal data?
- Which version of the standards, January 2025 or the January 2026 update, did the school's procurement process actually check against?
A detailed standard is only as useful as the verification a school is willing and able to do itself; the department has supplied the checklist, not the audit.
Sources & reading trail
States the filtering, monitoring and data-use requirements for generative AI products in schools, and confirms the January 2025 publication and January 2026 update dates.
Source published: 22 January 2025 · Retrieved: 16 September 2026
Department's cross-referenced policy page directing schools to the product safety standards, last updated 12 August 2025.
Source published: Not established · Retrieved: 16 September 2026
Departments, studies and vendor documents establish the record; the implementation reading and the boundary are School AI Atlas editorial analysis. This retrospective draft does not imply the site published on the event date.