Feb 2021United StatesGoogle CSAM image classifier
In February 2021 a San Francisco father took photos of his toddler son's medical condition after a nurse asked for photos before a video consultation; the doctor used them to diagnose the problem and prescribe antibiotics. The New York Times reported in August 2022 that when the photos were backed up from his Android phone to Google's servers, Google's artificial-intelligence tool for recognising never-before-seen child abuse images flagged them. Two days after the photos were taken his Google account was disabled for 'harmful content', which cost him more than a decade of email, contacts and photos and, through Google Fi, his phone number. Google reported him to the National Center for Missing and Exploited Children, and the San Francisco police obtained search warrants for his account; the investigator concluded that no crime had occurred. Google refused to reinstate the account after two appeals, the second with the police report, and later notified him that it was being permanently deleted. The Times reported a second father in Houston whose photos of his toddler, taken at a paediatrician's request, were also flagged; his Gmail account was disabled in the middle of buying a house, Houston police questioned and cleared him, and he did not get the account back. Google said it stood by its decisions and that its reviewers had found a further video in the San Francisco father's account that it considered problematic.
Contextual tracker case Medium reported severity Investigation Opened
AI involvement reported · Causal attribution supported · 3 sources, 1 underlying account · Added 02/11/2026
16 Apr 2021United StatesFlock Safety license plate readers (reported)
At about 5:30 a.m. on 16 April 2021, according to a lawsuit reported by The Almanac, an automated license-plate reader in Atherton, California, misread the dirt-obscured plate of a driver's car, reading its letter H as M, and matched it to a reported stolen vehicle. Atherton police followed the car for several minutes without visually verifying the plate, then made a high-risk felony stop, ordering him at gunpoint to throw his keys out of the window, get out and lie on his stomach to be handcuffed. He sued three officers in 2022 for unreasonable seizure and excessive force; the officers denied the allegations and the injuries he claimed, and the case was settled on 16 March 2023 for $45,000. Business Insider, citing police records, reported in March 2026 that the misread came from a Flock camera and that officers noted the Flock cameras had misread the plate; an Atherton police commander said the plate was obstructed by dirt and that the town denies wrongdoing.
Contextual tracker case Medium reported severity Lawsuit Settled
AI involvement supported · Causal attribution supported · 2 sources · Added 04/10/2026
23 Jun 2021United StatesUnidentified facial recognition system
According to police and bank records reviewed by the Washington Post, in December 2019 a woman made fraudulent withdrawals at a SunTrust branch in Potomac, Maryland; branches in two other Maryland counties were also defrauded. A bank investigator posted the surveillance photos on CrimeDex, an online network of police and corporate investigators; an unknown respondent ran the image through facial-recognition software and returned Kimberlee Williams's name and photo, and the investigator told Montgomery County police that facial-recognition software had identified her. Police in Montgomery, Prince George's and Anne Arundel counties charged her in 2020 on the investigator's identification and their own visual comparison of photos (two departments also cited her old Oklahoma check-fraud convictions), without reported checks of her whereabouts; Montgomery County police did not disclose the facial-recognition search in the charging application, according to police records reviewed by the Washington Post and the ACLU. On 23 June 2021 she was arrested at the gate of Fort Sill near Lawton, Oklahoma, while accompanying her daughter on a DoorDash delivery, spent about three weeks in an Oklahoma jail, was flown to Maryland, and faced 16 charges including 12 felonies. The counties dismissed the charges one after another, in October and December 2021, after about six months in jail. Her daughter says she was depressed afterwards; the ACLU says the ordeal harmed her mental and physical health. On 14 April 2026 the ACLU and ACLU of Maryland sent complaint letters to the three departments seeking an apology and reforms.
Contextual tracker case High reported severity Media Coverage
AI involvement reported · Causal attribution alleged · 4 sources, 3 underlying accounts · Added 27/09/2026
14 Aug 2021 to 14 Mar 2024United StatesSt. Louis mugshot recognition (SMRT)
Christopher Gatlin was arrested on 14 August 2021 for an assault at the St. Charles Rock Road MetroLink station after a joint St. Louis County and city team uploaded a grainy, blurry surveillance photograph from a bus into the St. Louis Mugshot Recognition Technology System, a facial-recognition tool that searches stored mug shots (more than 500,000 uploaded, the suit says per the Post-Dispatch; more than 250,000 per Biometric Update via Hoodline), and built the investigation around the resulting match together with a witness identification; he was charged with first-degree assault and held roughly 17 months (the Post-Dispatch says almost two years), a county judge suppressed the witness identification as unconstitutionally suggestive, and prosecutors dismissed the charges on 14 March 2024 (Post-Dispatch; Hoodline citing Biometric Update; Bloomberg Law). Gatlin sued St. Louis County, the City of St. Louis and named officers in the US District Court for the Eastern District of Missouri on 13 January 2025; on 15 October 2025 the court denied the city's motion to dismiss and granted a detective's motion only as to one count; on 13 August 2026 the court allowed a second amended complaint adding Amazon Web Services, which the complaint says supplied the Rekognition program the system used; the county, city and officers answered on 27 August 2026, AWS's response was extended to 21 September 2026 and on 17 September it moved for a further extension to 28 September (no ruling on the docket when read), and a jury trial is set for 1 November 2027 (CourtListener docket). Amazon told the Post-Dispatch that its terms of service prohibit the use of its facial recognition technology in criminal investigations and that it would fully investigate any information that emerges in the litigation. The ACLU lists Gatlin among fourteen people known to have been wrongfully arrested in the US because police relied on erroneous facial-recognition results.
Contextual tracker case High reported severity Lawsuit Ongoing
AI involvement reported · Causal attribution alleged · 5 sources · Added 26/09/2026