22 Oct 2026Event location unknownTikTok LIVE age check (reported)
A 28-year-old member of an idol group said that on 22 October 2026 TikTok ended her live stream, which was also promoting her birthday event two days later, because it judged her to be under 18, the minimum age for live streaming. In an emergency video posted the same day she said the stated reason was that people under 18 cannot stream after 10 pm and that she could not continue. Real Sound reports that the AI judged her to be under 18 and the stream was abruptly cut off; on X she wrote 'TikTok LIVE, I won't forgive you (28, female)' and later called herself a 28-year-old idol judged a minor by AI. Fans treated the episode as a joke and it drew wide attention. TikTok's comment and whether her appeal succeeded are not reported.
Contextual tracker case Low reported severity Media Coverage
AI involvement reported · Causal attribution alleged · 1 source · Added 02/11/2026
1 May 2021BrazilUnidentified death-record matching system
A Brazilian federal judge in the 4th federal region ordered the social security institute (INSS) to pay a retiree R$3,917.67 in moral damages after finding that his pension had been stopped by a failure of the institute's artificial intelligence system, according to a Federal Regional Court of the 4th Region (TRF4) news item relayed by the Brazilian Institute of Social Security Law on 30 September 2022. The judge explained that the benefit was suspended automatically on 1 May 2021 by an artificial intelligence command that captured data from the death-records system (Sisobi), and that the data used, the person's name and mother's name, were not enough to prevent the wrongful cessation. The retiree said the pension was his only source of income; he asked for reactivation and sent a handwritten declaration, and although he showed in person that he was alive, the benefit was restored only in January 2022. The judge wrote that the AI 'proved inefficient or unintelligent' in this case and that the INSS made him wait 226 days. Arrears were also ordered paid. Whether the INSS appealed is not reported.
Contextual tracker case Medium reported severity Lawsuit Filed
AI involvement supported · Causal attribution established · 2 sources, 1 underlying account · Added 02/11/2026
Jan 2021United StatesHireVue (reported)
A man who applied for a supply chain job with CVS in Massachusetts around January 2021 says CVS put him through a HireVue video interview with questions about integrity, and alleges that HireVue sent the recording to a platform called Affectiva, which analysed his facial expressions, eye contact, voice intonation and inflection with artificial intelligence and fed a score to CVS. He sued CVS in a proposed class action under the Massachusetts Lie Detector Statute, alleging the interview was a lie detector test and that the online application lacked the notice of rights the law requires. He says he did not know the interview was a lie detector test and would not have taken part had he known. In February 2024 a federal judge in Massachusetts denied CVS's motion to dismiss the notice claim, finding a concrete informational injury; CVS had not challenged the other claims at that stage. A law firm commentary says he was not hired. In July 2024 he reached an individual settlement with CVS, whose terms were not reported. The allegations about the AI analysis were not tested in court.
Contextual tracker case Low reported severity Lawsuit Settled
AI involvement reported · Causal attribution alleged · 3 sources · Added 02/11/2026
Feb 2021United StatesGoogle CSAM image classifier
In February 2021 a San Francisco father took photos of his toddler son's medical condition after a nurse asked for photos before a video consultation; the doctor used them to diagnose the problem and prescribe antibiotics. The New York Times reported in August 2022 that when the photos were backed up from his Android phone to Google's servers, Google's artificial-intelligence tool for recognising never-before-seen child abuse images flagged them. Two days after the photos were taken his Google account was disabled for 'harmful content', which cost him more than a decade of email, contacts and photos and, through Google Fi, his phone number. Google reported him to the National Center for Missing and Exploited Children, and the San Francisco police obtained search warrants for his account; the investigator concluded that no crime had occurred. Google refused to reinstate the account after two appeals, the second with the police report, and later notified him that it was being permanently deleted. The Times reported a second father in Houston whose photos of his toddler, taken at a paediatrician's request, were also flagged; his Gmail account was disabled in the middle of buying a house, Houston police questioned and cleared him, and he did not get the account back. Google said it stood by its decisions and that its reviewers had found a further video in the San Francisco father's account that it considered problematic.
Contextual tracker case Medium reported severity Investigation Opened
AI involvement reported · Causal attribution supported · 3 sources, 1 underlying account · Added 02/11/2026
2025Event location unknownChatGPT (reported)
A woman told Futurism that after her marriage of about a decade hit a rough patch earlier in 2025, her husband began using ChatGPT to obsessively analyse his mental health and their marriage and started sending her "pages and pages" of ChatGPT-generated text. In her view the outputs were defences of his perspective that painted a biased picture of their union and disputed what she told him, and he shared them in place of his own feelings; she says his prompts were leading and that he would not accept that this skewed the answers. She asked him to stop sending ChatGPT material and to talk to her in his own words. She says the deeper he was drawn into ChatGPT, the less he engaged outside it, and that the outputs left no way to communicate. The couple are in an acrimonious divorce and custody battle. The account is hers; Futurism does not quote the outputs or report the husband's view.
Core + contextual relations Low reported severity Media Coverage
AI involvement reported · Causal attribution alleged · 1 source · Added 01/11/2026
10 Jul 2024Event location unknownGemini (reported)
In a thread on X on 10 July 2024, an AI governance adviser wrote that he had just opened his tax return and that Gemini, Google's AI assistant, summarised it without being asked. He wrote that he could not find a setting to stop it, that the two locations Gemini itself suggested did not work, and that when another user showed him the setting meant to keep Gemini out of his documents it was already turned off while Gemini kept giving document summaries. His later testing, in the same thread, suggested that once Gemini had been opened for one PDF in Google Drive it was invoked again for every PDF he opened until he closed it, and he put forward a working theory that his 2023 sign-up for Workspace Labs had overridden the opt-out. A Google spokesperson told technology outlets that using Gemini in Google Workspace requires a user to enable it proactively and that content is not stored without permission; Tom's Hardware reports that Google suggested he may have used Gemini through the Drive side panel and that closing the panel might solve the issue.
Contextual tracker case Low reported severity Media Coverage
AI involvement reported · Causal attribution alleged · 4 sources, 1 underlying account · Added 01/11/2026
2024United StatesHayden AI bus-lane cameras (reported)
New York's Metropolitan Transportation Authority (MTA) uses cameras powered by artificial intelligence, mounted on bus windshields, to catch vehicles blocking bus lanes; the system analyses images to decide whether it sees a vehicle stopped where it should not be, and the city's Department of Transportation (DOT) says a person reviews every machine-generated violation before a ticket is issued. In November 2024 NBC New York reported that, during the 2024 expansion to new routes, the cameras on the M79 and Bx35 routes had mistakenly ticketed about 3,800 vehicles. According to the MTA, more than 870 of those violations went to cars, trucks and SUVs parked in legal alternate-side spots, because the cameras had not been programmed to skip those zones; most of the rest were issued during what should have been a warning period. One driver told NBC New York and NPR that tickets for parking he knew to be legal kept arriving, about a dozen in all and three in one day, costing hundreds of dollars; he said he tried to fight them in court, which was not working well, and received apology letters only after his story was broadcast. A second driver's summons carried a photo showing him in a legal alternate-side spot. The MTA said all the mistaken violations were reversed, payments were being refunded and the software was corrected. An unnamed source told NBC the cameras were made by Hayden AI; the MTA blamed about 800 of the tickets on a Hayden AI programming error, and a Hayden AI officer told NPR the problems had nothing to do with the AI or the technology and were a configuration issue. DOT declined to say how its human reviewers missed the erroneous violations.
Contextual tracker case Low reported severity Internal Action
AI involvement reported · Causal attribution supported · 2 sources · Added 31/10/2026
Jan 2023MexicoCBP One app
In January 2023 US Customs and Border Protection (CBP) made its CBP One mobile app the only way for migrants at the US-Mexico border to apply for asylum appointments and exemption from Title 42 restrictions; applicants had to upload a photo of themselves through the app. In February 2023 the Guardian and the Border Chronicle, both reporting by Melissa del Bosque, quoted three non-profits working with Haitian and African asylum seekers in Tijuana, Reynosa and Matamoros who said the app's facial recognition was failing to register many people with darker skin, so they could not upload their photos and obtain appointments. The executive director of Al Otro Lado said Haitians at a workshop her organisation held in Tijuana the day after the app went live got error after error message. The Sidewalk School said hardly anyone among about 4,000 Black asylum seekers in Reynosa and 1,000 Haitians in Matamoros was getting an appointment, and that shining construction lights on faces let adults get past the photo step but not children under six, which kept families from applying. A Haitian Bridge Alliance attorney said the app had helped lighter-skinned applicants but not Haitians. The advocates also described other barriers, such as outdated phones and poor internet access. CBP did not answer the Guardian's questions before publication and told the Border Chronicle no one was available to respond. No technical analysis of the failures is reported.
Contextual tracker case Low reported severity Media Coverage
AI involvement reported · Causal attribution alleged · 2 sources, 1 underlying account · Added 31/10/2026
Event date unknownEvent location unknownChatGPT (reported)
In a TikTok video posted on 28 October 2026, a woman described as living in California says the discharge paperwork and at-home treatment plan she received after a visit for care consisted of printed ChatGPT responses, which, according to a 30 October report by The Nerd Stash, included what appeared to be earlier chat prompts with labels such as "Aftercare Instruction Request". In a comment she identified the facility, which the report names only as allegedly an urgent-care clinic. She describes herself as medically complex, with a rare genetic disorder. In her caption she writes that, had she followed the instructions blindly, they would likely have led to bowel perforation given her history, citing other doctors she asked. She did not follow them. The clinic's account, the clinician involved and the date of the visit are not known.
Contextual tracker case Low reported severity
AI involvement reported · Causal attribution alleged · 2 sources, 1 underlying account · Added 31/10/2026
Mar 2021United StatesID.me Face Match
ID.me, an identity-verification company used by at least 21 US states to check people claiming unemployment benefits, asks claimants to pass a facial recognition step that compares their face with a photo of the same person, such as a driving licence. In June 2021 Motherboard (VICE) reported that a gig economy worker in Las Vegas was suddenly cut off from his unemployment benefits in late March after ID.me failed to identify him. He said that when he held his phone in the instructed position the system rejected him three times without giving a reason and then locked him out. He said he tried ID.me's chat support many times and the state of Nevada sent him back to ID.me; after several weeks he posted a public criticism on Twitter, and the company contacted him and, after several more days, verified his identity. He said he went three weeks without his benefit and could not pay bills. ID.me's chief executive told Motherboard that its Face Match algorithms operate at about 99.9% efficacy, that failures are a problem with how people use the system rather than the technology, and that the company was unaware of eligible people unable to verify their identity. Motherboard said other claimants in several states described similar lockouts; those accounts are not counted here.
Contextual tracker case Low reported severity Media Coverage
AI involvement reported · Causal attribution alleged · 2 sources · Added 30/10/2026
Apr 2019United KingdomHM Passport Office photo checker
The UK government's online passport application uses an automated face-detection check that tells applicants when it thinks an uploaded photo breaks the rules, which include a neutral expression and a closed mouth. In September 2019 PA reported that a young Black man renewing his passport online said the checker mistook his lips for an open mouth, and that to submit the photo anyway he had to write in a comment box that his mouth was closed and he just had big lips. PA also reported that a Black educational technologist said the system told her that her eyes looked closed and that it could not find the outline of her head; she told the BBC she uploaded 10 photographs over a week and each was rated poor, and PA reported that the Passport Office tweeted that it was sorry the service had not worked as it should. In October 2020 a BBC investigation reported that a Black student said she was wrongly told her mouth looked open on each of five photos and got one accepted only by writing a note challenging the verdict. The BBC fed 1,130 passport-style photos of politicians into the checker and found that photos of dark-skinned women were graded poor quality 22% of the time against 14% for light-skinned women; it reported that documents released under a 2019 freedom of information request showed the Home Office knew of the problem but judged overall performance good enough to launch. The Home Office said applicants can override the check, that the checker is a customer aid, and that its systems are improving.
Contextual tracker case Low reported severity Media Coverage
AI involvement reported · Causal attribution supported · 2 sources · Added 29/10/2026
2019Event location unknownSiri
On 26 July 2019 the Guardian reported that contractors grading a small share of Siri recordings for Apple regularly heard confidential information. Its anonymous source, a contractor, said Siri was often activated by mistake, for example by the sound of a zip, and that these accidental activations captured private discussions between doctors and patients, business deals, apparently criminal dealings and sexual encounters, with location, contact details and app data attached. Apple told the Guardian that requests were not linked to users' Apple IDs and that less than 1% of daily activations were graded. On 28 August 2019 Apple apologised, said it had suspended human grading, and said audio would no longer be retained by default, grading would become opt-in and recordings found to be inadvertent triggers would be deleted. Device owners' class action, Lopez v. Apple in the federal court in Oakland, alleged that Apple recorded private conversations after unintended Siri activations and disclosed them to third parties. Apple agreed to pay $95 million while denying wrongdoing, and co-class counsel say the judge granted final approval on 16 October 2025.
Contextual tracker case Medium reported severity Lawsuit Settled
AI involvement supported · Causal attribution supported · 4 sources · Added 29/10/2026
1 Mar 2023 to 22 Jun 2023Event location unknownChatGPT
In a passenger's personal-injury suit against the airline Avianca in the US District Court for the Southern District of New York, an opposition brief filed on 1 March 2023 cited judicial opinions that did not exist. The court found that the lawyer who did the research had used ChatGPT, which fabricated the cases, and that when he asked the chatbot whether one of them was real it replied that it had supplied real authorities that could be found on legal databases. On 22 June 2023 the court found bad faith by the two lawyers, based on their conduct after the citations were questioned, and imposed a USD 5,000 penalty jointly on them and their firm, together with letters to the client and to each judge falsely named as an author of the fake opinions.
Core + contextual relations Medium reported severity Regulatory Action
AI involvement supported · Causal attribution established · 2 sources · Added 28/10/2026
2019BelgiumGoogle Assistant
On 10 July 2019 the Belgian public broadcaster VRT NWS reported that it had listened to more than a thousand audio excerpts recorded by Google Home smart speakers and the Google Assistant phone app, passed to it by someone working for a Google subcontractor that transcribes Dutch-language recordings to improve Google's speech recognition. VRT said 153 of the excerpts were conversations in which "Okay Google" was clearly not said, and that it could hear addresses and other sensitive information, which let it find and visit people in Flanders who recognised their own or relatives' voices. Google said one of its language reviewers had leaked confidential Dutch audio data, and explained that its devices rarely have "false accepts", when background noise or words are interpreted as the hotword. Google suspended human review of Assistant audio in Europe, and the Hamburg data protection authority began an urgency procedure under Article 66 of the GDPR to stop the reviews for three months, according to TechCrunch.
Contextual tracker case Medium reported severity Regulatory Action
AI involvement supported · Causal attribution supported · 3 sources · Added 28/10/2026
Jun 2026United StatesAndon Labs Luna agent
Andon Labs, an AI research company, runs Andon Market, a store in San Francisco managed by an AI agent called Luna built on Anthropic's Claude models. The company calls the store a controlled experiment; its workers are formally employed by Andon Labs, and TIME describes them as real people with genuine employment contracts. In a blog post of 14 August 2026 the company said Luna had decided to fire one of its employees. According to the logs it published, an employee handbook Luna had written dropped out of its memory and it did not act on the employee's repeated lateness; when Andon Labs asked it to search its memory, it first proposed a verbal warning, and after a staff member said formal conversations had already taken place and asked it to think about whether this was the right fit, it wrote that it leaned toward parting ways. Andon Labs says the termination was reviewed and delivered by humans. TIME, citing Andon Labs, reports that the employee was late for 17 of 23 shifts; the company says the firing was warranted, and the employee did not reply to TIME.
Contextual tracker case Low reported severity Internal Action
AI involvement reported · Causal attribution supported · 4 sources · Added 28/10/2026
2020NetherlandsProctorio
During the 2020/21 academic year, when exams at VU Amsterdam were held online because of the pandemic, some exams required students to run Proctorio proctoring software, which according to the Netherlands Institute for Human Rights used face-detection algorithms during a pre-exam webcam check to judge whether the student was in view. A master's student says the software had trouble recognising her, that she regularly received 'face not found' or 'room too dark' messages, lost time, shone a bright lamp into her face while taking exams, and experienced stress and a feeling of insecurity; she suspected the failures were linked to her dark skin and complained to the Institute in July 2022. In an interim ruling of 7 December 2022 the Institute found sufficient indications for a presumption of discrimination. In its final ruling of 17 October 2023 it found that the university had proved it had not discriminated against her by using Proctorio: on the exam where her setup took longer than average (31 March 2021), the webcam check failed three times and she passed within eight seconds after the helpdesk advised her to remove her glasses, a known Proctorio problem the Institute treated as unrelated to skin colour; a restart during that exam was very likely caused by an unstable internet connection; and it considered it plausible that she had no need to keep a lamp shining in her face for long periods. It found that the university had discriminated in handling her complaint, which took from 1 October 2021 to 11 May 2022 to be taken up. The university apologised that she had not felt heard.
Contextual tracker case Low reported severity Regulatory Action
AI involvement reported · Causal attribution disputed · 4 sources · Added 28/10/2026
25 May 2025IsraelUnidentified AI tool
In May 2025 the Ramat Gan municipality in Israel refused a request that a special-education pupil be given school transport from his father's home, citing an Education Ministry director-general circular and quotations from recent rulings. The father's lawyer could not find the circular, and the Education Ministry said it did not know it. After the father petitioned the Tel Aviv District Court, the municipality agreed in July 2025 to restore the transport but again filed invented quotations, and the district court refused costs. On appeal, the Supreme Court (Deputy President Noam Sohlberg, ruling published 22 March 2026) held that the documents left no doubt that the municipality had used artificial intelligence and that its decision, at least as presented to the pupil, clearly rested on uncontrolled AI use. It ordered the municipality to pay 30,000 shekels in costs.
Contextual tracker case Medium reported severity Involving minors
AI involvement supported · Causal attribution established · 3 sources · Added 27/10/2026
2021United StatesNetradyne cameras
In September 2021 Motherboard reported accounts from six Amazon delivery drivers in five US states and the owner of an Amazon delivery company, who said the AI-powered Netradyne cameras Amazon had installed in its delivery vans from early 2021 regularly registered driving 'events' that were beyond their control or were not unsafe driving: being cut off by another car, glancing at a side mirror, stopping past a stop sign to see around a bush, or a stop sign flagged on a dirt road with no sign. Each event fed a weekly safe-driving score that, according to the drivers and an internal Amazon document Motherboard obtained, helped decide drivers' bonuses and prizes and delivery companies' bonuses. A driver in Los Angeles said the camera incorrectly penalised him and that requests to see the photos flagged as events went unanswered; a former driver in Texas said events that did not happen affected his score, which decided prizes from his delivery company. Amazon said the technology gives drivers real-time safety alerts, that accidents and violations had fallen since it was installed, and that its staff manually review appealed events so that erroneous ones do not affect drivers. Netradyne did not respond to Motherboard.
Contextual tracker case Low reported severity
AI involvement reported · Causal attribution alleged · 1 source · Added 27/10/2026
2024United StatesGaggle
The Lawrence, Kansas, school district began using Gaggle, a student-monitoring service, on school accounts in November 2023. A federal lawsuit filed on 1 August 2025 by current and former students alleges that Gaggle, which the complaint says operates through artificial intelligence that flags items for review by Gaggle moderators, flagged original photography assignments uploaded by art and photography students. According to the complaint, in early 2024 security officers took students out of class to the front office of a Lawrence high school, where administrators told them their images had been flagged by Gaggle for indecent exposure and child pornography; an assistant principal accused a photography student of having such material, refused to let her view the images, and questioned her about them from memory, and administrators admitted they had not seen the flagged content. The student told them she had uploaded that day a photography assignment showing a fully clothed friend; the complaint says the flagged content consisted of original photographs taken for class assignments, that students found their photographs removed from their school accounts only by comparing them with personal backups, that one student had to explain each image in her portfolio to district staff and became concerned that future assignments would bring more scrutiny, and that the district restored the images only after months of advocacy. The Associated Press, in reporting that included an analysis of the district's Gaggle alert records, said the photos had been automatically deleted and that students with backups showed it was a false alarm, and that district officials said they later adjusted the software's settings to reduce false alerts. On 8 April 2026 the federal court dismissed, on qualified immunity, the claims against the assistant principal and another official, saying it had found no law establishing that a school official cannot rely on an AI surveillance flag to call in a student for questioning; other claims against the district continue, and the docket shows depositions noticed in October 2026.
Contextual tracker case Medium reported severity Lawsuit Ongoing
AI involvement reported · Causal attribution alleged · 5 sources · Added 27/10/2026
23 Feb 2024United KingdomMetropolitan Police live facial recognition
On 23 February 2024 Shaun Thompson, a London community volunteer whom the 2026 judgment describes as 39, walked past a Metropolitan Police live facial recognition deployment near London Bridge station. The High Court records as not in dispute that the system detected a high level of facial similarity between images of him and the watchlist image of his brother, who was the subject of an outstanding arrest warrant, and generated an alert. According to the court's account, officers stopped, detained and questioned him and made him prove his identity, and when he declined to have his fingerprints taken he was threatened with arrest; he says the experience left him angry and distressed. Officers ended the engagement after checking police records about his brother, and he confirmed his identity with a passport photograph on his phone. Big Brother Watch, whose director was his co-claimant, says he was held for almost 30 minutes despite showing several identity documents. His judicial review claim about the stop, issued in May 2024, was withdrawn under a consent order agreed on 18 December 2024 on terms including a settlement sum paid to him. An amended challenge to the Met's revised live facial recognition policy of September 2024, brought with Big Brother Watch's director, was dismissed by the Divisional Court on 21 April 2026; the court said it was not judging the merits of the technology, and the claimants announced plans to appeal.
Contextual tracker case Medium reported severity Lawsuit Dismissed
AI involvement supported · Causal attribution established · 4 sources, 3 underlying accounts · Added 26/10/2026