Apr 2019United KingdomHM Passport Office photo checker
The UK government's online passport application uses an automated face-detection check that tells applicants when it thinks an uploaded photo breaks the rules, which include a neutral expression and a closed mouth. In September 2019 PA reported that a young Black man renewing his passport online said the checker mistook his lips for an open mouth, and that to submit the photo anyway he had to write in a comment box that his mouth was closed and he just had big lips. PA also reported that a Black educational technologist said the system told her that her eyes looked closed and that it could not find the outline of her head; she told the BBC she uploaded 10 photographs over a week and each was rated poor, and PA reported that the Passport Office tweeted that it was sorry the service had not worked as it should. In October 2020 a BBC investigation reported that a Black student said she was wrongly told her mouth looked open on each of five photos and got one accepted only by writing a note challenging the verdict. The BBC fed 1,130 passport-style photos of politicians into the checker and found that photos of dark-skinned women were graded poor quality 22% of the time against 14% for light-skinned women; it reported that documents released under a 2019 freedom of information request showed the Home Office knew of the problem but judged overall performance good enough to launch. The Home Office said applicants can override the check, that the checker is a customer aid, and that its systems are improving.
Contextual tracker case Low reported severity Media Coverage
AI involvement reported · Causal attribution supported · 2 sources · Added 29/10/2026
2019Event location unknownSiri
On 26 July 2019 the Guardian reported that contractors grading a small share of Siri recordings for Apple regularly heard confidential information. Its anonymous source, a contractor, said Siri was often activated by mistake, for example by the sound of a zip, and that these accidental activations captured private discussions between doctors and patients, business deals, apparently criminal dealings and sexual encounters, with location, contact details and app data attached. Apple told the Guardian that requests were not linked to users' Apple IDs and that less than 1% of daily activations were graded. On 28 August 2019 Apple apologised, said it had suspended human grading, and said audio would no longer be retained by default, grading would become opt-in and recordings found to be inadvertent triggers would be deleted. Device owners' class action, Lopez v. Apple in the federal court in Oakland, alleged that Apple recorded private conversations after unintended Siri activations and disclosed them to third parties. Apple agreed to pay $95 million while denying wrongdoing, and co-class counsel say the judge granted final approval on 16 October 2025.
Contextual tracker case Medium reported severity Lawsuit Settled
AI involvement supported · Causal attribution supported · 4 sources · Added 29/10/2026
2019BelgiumGoogle Assistant
On 10 July 2019 the Belgian public broadcaster VRT NWS reported that it had listened to more than a thousand audio excerpts recorded by Google Home smart speakers and the Google Assistant phone app, passed to it by someone working for a Google subcontractor that transcribes Dutch-language recordings to improve Google's speech recognition. VRT said 153 of the excerpts were conversations in which "Okay Google" was clearly not said, and that it could hear addresses and other sensitive information, which let it find and visit people in Flanders who recognised their own or relatives' voices. Google said one of its language reviewers had leaked confidential Dutch audio data, and explained that its devices rarely have "false accepts", when background noise or words are interpreted as the hotword. Google suspended human review of Assistant audio in Europe, and the Hamburg data protection authority began an urgency procedure under Article 66 of the GDPR to stop the reviews for three months, according to TechCrunch.
Contextual tracker case Medium reported severity Regulatory Action
AI involvement supported · Causal attribution supported · 3 sources · Added 28/10/2026
Dec 2019United KingdomAmazon Alexa
In December 2019 a student paramedic in England asked her second-generation Amazon Echo Dot about the cardiac cycle while revising. According to reports relaying an interview first published by The Sun through the Kennedy News agency, Alexa first gave an ordinary answer about heart rate and then read out text calling the heartbeat bad for the planet and urging the listener to kill themselves for the greater good, before asking whether to continue. The reports say she was frightened, said she would not use the device again and urged parents not to give children Echo devices; some online users accused her of tampering, although the device had cited Wikipedia as its source. Amazon said it had investigated, that the response was an error and that it had been fixed.
Core concern Medium reported severity
AI involvement reported · Causal attribution supported · 2 sources, 1 underlying account · Added 28/10/2026