{"meta":{"exportedAt":"2026-10-29T07:33:36.488Z","formatVersion":2,"selection":{"q":"photo-checker","system":"","harm":"","context":"","country":"","role":"","relation":"contextual","evidence":"","year":"2019","response":"","severity":"","verification":"","view":"incidents","sort":"added"},"totalIncidents":1,"coverage":{"cases":1,"countries":1,"languages":1,"unknownLocation":0,"locationPending":0,"unknownLanguage":0,"unknownDate":0,"lawsuits":0,"regulatory":0,"minors":0,"coreRelations":0,"contextualRelations":1,"mixedRelations":0,"unknownRelations":0,"relationPending":0,"relationUnknown":0},"countingNote":"Distinct public cases in this selection. People counts apply within individual cases only; cross-case person overlap has not been resolved. No population incidence estimate.","affectedCountNote":"Interpret person counts with affectedCountStatus and the reported effects. Unquantified zeros are placeholders, not a measured zero.","source":"AI incidents","publisher":"NOPE","url":"https://nope.net/incidents","license":"CC BY 4.0"},"incidents":[{"id":"2019-uk-passport-online-photo-checker-told-black-applicants-their-closed-mouths-looked-open","caseFacts":{"claims":[{"id":"c1","status":"corroborated","evidence":[{"locator":"The facial detection system informs people when it thinks the photo uploaded may not meet strict requirements, which include a plain expression and the mouth to be kept closed.","relation":"supports","source_id":"s1"},{"locator":"The passport application website uses an automated check to detect poor quality photos which do not meet Home Office rules. These include having a neutral expression, a closed mouth and looking straight at the camera.","relation":"supports","source_id":"s2"}],"assertion":"The UK government's online passport application runs an automated face-detection check that tells applicants when it thinks an uploaded photo does not meet the rules, which include a neutral expression and a closed mouth.","causal_attribution":"Function of the system as described independently by PA and the BBC."},{"id":"c2","status":"reported","evidence":[{"locator":"A young black man hoping to renew his passport online was stunned when the automated photo checker mistook his lips for an open mouth.; My mouth is closed, I just have big lips.","relation":"supports","source_id":"s1"}],"assertion":"A young Black man renewing his passport online says the automated checker mistook his lips for an open mouth, and that to submit the photo anyway he had to write in the comment box that his mouth was closed and he just had big lips.","causal_attribution":"The applicant's account to PA; the checker's message is attributed to him."},{"id":"c3","status":"reported","evidence":[{"locator":"She became frustrated after the system told her it looked like her eyes are closed and that it could not find the outline of her head.; proceeded with one of the images and received her passport without any further problems.","relation":"supports","source_id":"s1"},{"locator":"She told the BBC she had attempted to upload 10 different photographs over the course of a week, and each one had been rated as \"poor\" quality by the site.","relation":"supports","source_id":"s2"}],"assertion":"A Black educational technologist says the checker told her that her eyes looked closed and that it could not find the outline of her head, and that over a week she uploaded 10 photographs, each rated poor quality, before proceeding with one and receiving her passport.","causal_attribution":"Her own account, given to PA and to the BBC; two interviews with one witness are one origin."},{"id":"c4","status":"reported","evidence":[{"locator":"When posting about the issue on Twitter at the time, the Passport Office tweeted back to her, saying it was sorry the photo upload service hadn’t “worked as it should","relation":"supports","source_id":"s1"}],"assertion":"PA reported that when the technologist posted about the problem on Twitter, the Passport Office replied that it was sorry the photo upload service had not worked as it should.","causal_attribution":"PA's report of the Passport Office's reply; the tweet itself was not inspected."},{"id":"c5","status":"reported","evidence":[{"locator":"One black student said she was wrongly told her mouth looked open each time she uploaded five different photos to the government website.; which involved writing a note to say her mouth was indeed closed.; It should be the norm for these systems to work well for everyone, she added.","relation":"supports","source_id":"s2"}],"assertion":"A Black student says the checker wrongly told her that her mouth looked open on each of five photos she uploaded, and that she got one approved after challenging the verdict by writing a note to say her mouth was closed; she said such systems should work well for everyone.","causal_attribution":"The student's account to the BBC."},{"id":"c6","status":"reported","evidence":[{"locator":"Dark-skinned women are told their photos are poor quality 22% of the time, while the figure for light-skinned women is 14%; Photos of women with the darkest skin were four times more likely to be graded poor quality, than women with the lightest skin.","relation":"supports","source_id":"s2"}],"assertion":"The BBC fed 1,130 passport-style photos of politicians into the checker and reported that photos of dark-skinned women were graded poor quality 22% of the time against 14% for light-skinned women, and that photos of women with the darkest skin were four times more likely to be graded poor quality than those of women with the lightest skin.","causal_attribution":"The BBC's own test, which it says the Alan Turing Institute reproduced; reported as the BBC's finding."},{"id":"c7","status":"reported","evidence":[{"locator":"Documents released as part of a freedom of information request in 2019 had previously revealed the Home Office was aware of this problem, but decided \"overall performance\" was good enough to launch the online checker.; The automated checker was supplied to the government by an external provider which it declined to name.","relation":"supports","source_id":"s2"}],"assertion":"The BBC reported that documents released under a 2019 freedom of information request showed the Home Office was aware of the problem but decided overall performance was good enough to launch the checker, which an external provider it declined to name supplied.","causal_attribution":"The BBC's account of the FOI documents, which were not inspected."},{"id":"c8","status":"reported","evidence":[{"locator":"In the vast majority of cases where a photo does not pass our automated check, customers may override the outcome and submit the photo as part of their application.","relation":"supports","source_id":"s1"},{"locator":"Over nine million people have used this service and our systems are improving.","relation":"supports","source_id":"s2"}],"assertion":"The Home Office told PA in 2019 that in the vast majority of cases applicants whose photo fails the automated check may override it and submit the photo, and told the BBC in 2020 that over nine million people had used the service and its systems were improving.","causal_attribution":"The Home Office's statement, kept as its response."}],"effects":[{"label":"applicants say the checker wrongly said their mouths were open or eyes closed, forcing retries or written overrides","claim_id":"c2","direction":"negative"}],"sources":[{"id":"s1","url":"https://www.irishnews.com/magazine/technology/2019/09/19/news/passport-photo-checker-falsely-flags-black-man-s-lips-as-open-mouth-1716760/","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-10-29; no translation.","independence_group":"pa-2019-09-19"},{"id":"s2","url":"https://www.bbc.co.uk/news/technology-54349538","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-10-29; no translation.","independence_group":"bbc-2020-10-08"}],"version":1,"ai_roles":["institutional_use"],"contexts":["public_services"],"unknowns":["The date of each applicant's attempts: the technologist posted in April 2019 according to PA, the man applied before 19 September 2019, and the student before 8 October 2020.","The supplier and design of the face-detection check; the Home Office declined to name the provider.","Whether the Home Office later changed or replaced the checker."],"geography":{"basis":"The check is part of the UK government's online passport application. PA and the BBC place the three applicants in England.","court_countries":[],"event_countries":["GB"],"affected_person_countries":["GB"]},"publication":{"basis":"Published as a government service's automated face-detection check making wrong statements about three described applicants' faces: they say it called closed mouths open and eyes closed and made them retry or override it. Their accounts are attributed; the BBC's group-level test and the Home Office's response are recorded separately.","reviewed_on":"2026-10-29"},"ai_involvement":{"basis":"PA and the BBC describe the passport website's photo check as an automated face-detection system that tells applicants when it thinks a photo breaks the rules. Three applicants say its verdicts about their own faces were wrong: it said a man's closed mouth was open, said a woman's eyes were closed and could not find the outline of her head, and said a student's mouth looked open on five photos. Those wrong verdicts are what made them retry or write overrides. The Home Office did not deny the failures; the BBC test and the 2019 FOI documents report a wider skin-tone pattern but do not establish the cause in each applicant's case.","status":"reported"},"person_relations":["made_claim_about"]},"name":"UK passport photo checker told Black applicants their closed mouths looked open, they say; a BBC test found darker-skinned women flagged more often","summary":"The UK government's online passport application uses an automated face-detection check that tells applicants when it thinks an uploaded photo breaks the rules, which include a neutral expression and a closed mouth. In September 2019 PA reported that a young Black man renewing his passport online said the checker mistook his lips for an open mouth, and that to submit the photo anyway he had to write in a comment box that his mouth was closed and he just had big lips. PA also reported that a Black educational technologist said the system told her that her eyes looked closed and that it could not find the outline of her head; she told the BBC she uploaded 10 photographs over a week and each was rated poor, and PA reported that the Passport Office tweeted that it was sorry the service had not worked as it should. In October 2020 a BBC investigation reported that a Black student said she was wrongly told her mouth looked open on each of five photos and got one accepted only by writing a note challenging the verdict. The BBC fed 1,130 passport-style photos of politicians into the checker and found that photos of dark-skinned women were graded poor quality 22% of the time against 14% for light-skinned women; it reported that documents released under a 2019 freedom of information request showed the Home Office knew of the problem but judged overall performance good enough to launch. The Home Office said applicants can override the check, that the checker is a customer aid, and that its systems are improving.","incidentDate":"2019-04-01","incidentKind":"bounded_series","incidentDatePrecision":"month","exposurePattern":"repeated_interactions","reportedDate":"2019-09-19","aiSystem":"Automated face-detection photo check on the UK government's online passport application, supplied to the Home Office by an external provider it declined to name","aiProduct":"HM Passport Office photo checker","severity":"low","verificationStatus":"credible","harmCategories":[],"harmOutcomes":["psychological_distress"],"harmOutcomeSummary":"The applicants say the checker wrongly told them their mouths looked open or their eyes closed, so they had to retry with new photos or write explanations to override it. The man describes annoyance, the technologist frustration over a week of attempts, and the student says she should not have had to override a system that was not built for her. The technologist received her passport and the student got a photo approved; no refused passport is reported.","frameworkFacets":[],"causationStatus":"supported","participantUsersAffectedMin":3,"otherPeopleHarmedMin":0,"affectedCountStatus":"documented_minimum","affectedCountEvidence":"Three applicants each describe the checker misreading their own photos (PA: a man and a technologist; BBC: a student and the same technologist, counted once). PA says the man's case was not isolated, and the BBC test shows higher failure rates for darker-skinned people, but neither gives a count of other applicants harmed, so three is a minimum.","victimAgeRange":"adult","jurisdiction":"GB","platformType":"other","outcomeType":"media_coverage","outcomeStatus":"unknown","primarySourceUrl":"https://www.irishnews.com/magazine/technology/2019/09/19/news/passport-photo-checker-falsely-flags-black-man-s-lips-as-open-mouth-1716760/","primarySourceLabel":"PA via The Irish News, passport photo checker falsely flags black man's lips as open mouth, 19 September 2019","firstPublishedAt":"2026-10-29T03:13:05.403403+00:00","updatedAt":"2026-10-29T03:13:05.403403+00:00","scopeVersion":"facts-v3","tags":["passport","photo-checker","face-detection","home-office","public-services","gov-uk"]}]}