UK OSA Minimum Standards of Accuracy

Why It Matters

Technology notices are the mechanism in the Online Safety Act for compelling proactive scanning, including of privately communicated CSEA content. These standards define the accuracy, fairness and robustness a classifier must demonstrate before it can be mandated, and are the first UK statutory benchmark for evaluating content-detection models.

Who Must Comply

Safety Provisions

• Technology must score at least 80 out of 100 on Objective 1.1 (performance metrics: evaluated against the false positive rate and other appropriate metrics for its intended deployment environment) and at least 40 out of 100 on every other objective
• Technology must score at least 60 out of 100 on each of the four principles and at least 60 out of 100 overall
• Dataset quality: development, training and test data must be comprehensive, representative of the harm being detected and diverse enough to test generalisation to unseen data
• Reproducible performance across the environments for which the technology is designed
• Secondary validation: outputs evaluated against expert human judgement where possible, particularly in complex or nuanced cases
• Fairness: identification and mitigation of unfair bias across different groups of people
• Robustness against adversarial attacks such as evasion attacks, injection attacks and input perturbations
• Maintainability: ability to modify or update the technology to preserve accuracy against new threats
• Evidence is graded 5 points (robust and comprehensive), 1 point (notable limitations) or 0 points (missing, misleading or inaccurate) per question set by Ofcom

Compliance & Enforcement

Primary Source