{"meta":{"exportedAt":"2026-11-20T00:11:45.904Z","formatVersion":2,"selection":{"q":"appeal","system":"","harm":"","context":"","country":"","role":"","relation":"contextual","evidence":"","year":"2021","response":"","severity":"","verification":"","view":"incidents","sort":"added"},"totalIncidents":3,"coverage":{"cases":3,"countries":2,"languages":2,"unknownLocation":0,"locationPending":0,"unknownLanguage":0,"unknownDate":0,"lawsuits":1,"regulatory":1,"minors":0,"coreRelations":0,"contextualRelations":3,"mixedRelations":0,"unknownRelations":0,"relationPending":0,"relationUnknown":0},"countingNote":"Distinct public cases in this selection. People counts apply within individual cases only; cross-case person overlap has not been resolved. No population incidence estimate.","affectedCountNote":"Interpret person counts with affectedCountStatus and the reported effects. Unquantified zeros are placeholders, not a measured zero.","source":"AI incidents","publisher":"NOPE","url":"https://nope.net/incidents","license":"CC BY 4.0"},"incidents":[{"id":"2021-brazil-federal-judge-finds-inss-ai-death-record-match-wrongly-ended-living-retirees-pension-awards-damages","caseFacts":{"claims":[{"id":"c1","status":"reported","evidence":[{"locator":"'O magistrado concluiu que houve falha do sistema de inteligência artificial do INSS ao fazer o cruzamento de dados com o Sistema de Óbitos (Sisobi).'; 'a suspensão do benefício ocorreu automaticamente em 01/05/2021 por comando de inteligência artificial que capturou dados do Sisobi. Os dados utilizados, nome próprio e nome da mãe, não se mostraram suficientes para evitar a indevida cessação do benefício.'","relation":"supports","source_id":"s1"},{"locator":"'a suspensão do benefício ocorreu automaticamente em 01/05/2021 por comando de inteligência artificial que capturou dados do Sisobi'","relation":"supports","source_id":"s2"}],"assertion":"A federal judge concluded that the INSS's artificial intelligence system failed when cross-matching data with the death register (Sisobi), and that the retiree's benefit was suspended automatically on 1 May 2021 by an artificial intelligence command that used only his name and mother's name.","causal_attribution":"The court's finding as the TRF4 news item reports it."},{"id":"c2","status":"reported","evidence":[{"locator":"'sendo prejudicado por não ter qualquer outra fonte de renda'; 'O autor da ação chegou a enviar declaração escrita de próprio punho ao INSS e, embora tenha pessoalmente demonstrado que estava vivo, e não morto, nada foi feito até reativação do benefício em janeiro deste ano (2022).'","relation":"supports","source_id":"s1"}],"assertion":"The retiree said the pension was his only source of income, that he asked for reactivation without a decision, and that he sent a handwritten declaration; the benefit was restored only in January 2022.","causal_attribution":"His account and the court record as the news item gives them."},{"id":"c3","status":"reported","evidence":[{"locator":"'a inteligência artificial adotada pelo INSS se revelou ineficiente ou desinteligente nesta hipótese, de modo que a sua ilegitima inconsistência ocasionou prejuízos materiais à parte autora que atingiram diretamente a sua subsistência, pois se tratava de sua única fonte de rendimentos'; 'o sujeitou a espera de 226 dias'","relation":"supports","source_id":"s1"}],"assertion":"The judge wrote that the INSS's artificial intelligence proved inefficient or unintelligent in this case, that its inconsistency harmed the retiree's subsistence because the pension was his only income, and that the INSS made him wait 226 days.","causal_attribution":"The judge's words quoted in the news item."},{"id":"c4","status":"reported","evidence":[{"locator":"'no valor de R$ 3.917,67 (três mil, novecentos e dezessete reais e sessenta e sete centavos)'; 'os valores atrasados, bem como aqueles vencidos entre a sentença e a efetiva implantação do benefício (DIP) serão executados na forma de requisição de pagamento'","relation":"supports","source_id":"s1"},{"locator":"'no valor de R$ 3.917,67'","relation":"supports","source_id":"s2"}],"assertion":"The judge ordered the INSS to pay R$3,917.67 in moral damages and the arrears.","causal_attribution":"The operative part as reported."}],"effects":[{"label":"pension, his only income, stopped by a wrong automatic death-register match and restored only months later","claim_id":"c3","direction":"negative"}],"sources":[{"id":"s1","url":"https://www.ibdp.org.br/2022/09/30/justica-condena-inss-a-pagar-indenizacao-apos-cancelar-beneficio-de-segurado-que-ainda-estava-vivo/","kind":"news_article","access":"read","language":"pt","translation_note":"Read in Portuguese on 2026-11-02; translation by the research agent, an AI. Professional association's republication of a TRF4 news item ('Fonte TRF4') on the sentence; the sentence was not read.","independence_group":"trf4-news-arapongas-inss-sentence"},{"id":"s2","url":"https://wagner.adv.br/justica-condena-inss-a-pagar-indenizacao-apos-cancelar-beneficio-de-segurado-que-ainda-estava-vivo/","kind":"news_article","access":"read","language":"pt","translation_note":"Read in Portuguese on 2026-11-02; translation by the research agent, an AI. Law firm's republication of the same TRF4 news item ('Fonte: TRF 4ª Região'); undated on the page.","independence_group":"trf4-news-arapongas-inss-sentence"}],"version":1,"ai_roles":["institutional_use"],"contexts":["public_services","finance","justice"],"unknowns":["The case number, the sentence date and whether the INSS appealed.","What kind of system the 'artificial intelligence command' was beyond a name and mother's-name match, and whose death record was matched.","The retiree's age."],"geography":{"basis":"The pension was paid and stopped by Brazil's national social security institute, and a federal court in the 4th federal region of Brazil decided the case.","court_countries":["BR"],"event_countries":["BR"],"affected_person_countries":[]},"publication":{"basis":"Published on a TRF4 court news item, relayed by IBDP and a law firm, reporting a federal judge's finding that an INSS artificial intelligence match with the death register automatically stopped a living retiree's pension, his only income, for months, and the damages awarded.","reviewed_on":"2026-11-02"},"ai_involvement":{"basis":"The federal judge, as the TRF4 news item reports, found that an INSS artificial intelligence command cross-matched the retiree with the death register using only name and mother's name and automatically suspended his pension on 1 May 2021, and that this wrong match caused the loss of his only income. The technical nature of the system beyond the judge's description is not disclosed, and the sentence itself was not read.","status":"supported"},"person_relations":["made_decision_about"]},"name":"Brazilian federal judge finds an INSS AI death-record match wrongly ended a living retiree's pension, his only income, and awards damages","summary":"A Brazilian federal judge in the 4th federal region ordered the social security institute (INSS) to pay a retiree R$3,917.67 in moral damages after finding that his pension had been stopped by a failure of the institute's artificial intelligence system, according to a Federal Regional Court of the 4th Region (TRF4) news item relayed by the Brazilian Institute of Social Security Law on 30 September 2022. The judge explained that the benefit was suspended automatically on 1 May 2021 by an artificial intelligence command that captured data from the death-records system (Sisobi), and that the data used, the person's name and mother's name, were not enough to prevent the wrongful cessation. The retiree said the pension was his only source of income; he asked for reactivation and sent a handwritten declaration, and although he showed in person that he was alive, the benefit was restored only in January 2022. The judge wrote that the AI 'proved inefficient or unintelligent' in this case and that the INSS made him wait 226 days. Arrears were also ordered paid. Whether the INSS appealed is not reported.","incidentDate":"2021-05-01","incidentKind":"single_event","incidentDatePrecision":"day","exposurePattern":"unknown","reportedDate":"2022-09-30","aiSystem":"An INSS artificial intelligence routine that cross-matched benefit records with the national death register (Sisobi) using name and mother's name and stopped benefits automatically","aiProduct":"Unidentified death-record matching system","severity":"medium","verificationStatus":"credible","harmCategories":[],"harmOutcomes":["financial_loss"],"harmOutcomeSummary":"The judge found that an automatic AI match with the death register wrongly stopped the retiree's pension, his only income, and that the INSS took 226 days to restore it, which harmed his subsistence.","frameworkFacets":[],"causationStatus":"established","participantUsersAffectedMin":0,"otherPeopleHarmedMin":1,"affectedCountStatus":"exact","affectedCountEvidence":"One person: the retiree whose pension the automatic match stopped.","victimAgeRange":"unknown","jurisdiction":"BR","platformType":"other","outcomeType":"lawsuit_filed","outcomeStatus":"unknown","primarySourceUrl":"https://www.ibdp.org.br/2022/09/30/justica-condena-inss-a-pagar-indenizacao-apos-cancelar-beneficio-de-segurado-que-ainda-estava-vivo/","primarySourceLabel":"IBDP, Justiça condena INSS a pagar indenização após cancelar benefício de segurado que ainda estava vivo, 30 September 2022","firstPublishedAt":"2026-11-02T03:10:13.026551+00:00","updatedAt":"2026-11-02T03:10:13.026551+00:00","scopeVersion":"facts-v3","tags":["public-benefits","automated-decision","death-register-match","social-security","court-damages"]},{"id":"2021-us-google-ai-flagged-fathers-medical-photos-of-toddlers-as-child-abuse-accounts-disabled-police-cleared","caseFacts":{"claims":[{"id":"c1","status":"reported","evidence":[{"locator":"'It was a Friday night in February 2021.'; 'The nurse said to send photos so the doctor could review them in advance.'; 'the doctor diagnosed the issue and prescribed antibiotics'","relation":"supports","source_id":"s1"},{"locator":"'The doctor used that image to diagnose'","relation":"supports","source_id":"s3"}],"assertion":"In February 2021 a San Francisco father photographed his toddler son's medical condition after a nurse asked for photos before a video consultation, and the doctor used them to diagnose the problem and prescribe antibiotics.","causal_attribution":"Background facts from the father's account to the Times."},{"id":"c2","status":"reported","evidence":[{"locator":"'Google developed an artificially intelligent tool that could recognize never-before-seen exploitative images of children'; 'this technology flagged them'; 'A human content moderator for Google would have reviewed the photos after they were flagged by the artificial intelligence'","relation":"supports","source_id":"s1"},{"locator":"'automatically uploaded to the cloud, Google’s system identified them as CSAM'","relation":"supports","source_id":"s3"}],"assertion":"When the photos were backed up to Google's servers, Google's AI tool for recognising never-before-seen child abuse images flagged them; a human moderator reviews flagged photos before Google locks an account and reports it.","causal_attribution":"The Times attributes the flag to Google's AI tool; the human review step is described as the general process."},{"id":"c3","status":"reported","evidence":[{"locator":"'His account had been disabled because of “harmful content” that was “a severe violation of Google’s policies and might be illegal.”'; 'he had to get a new phone number with another carrier'","relation":"supports","source_id":"s1"},{"locator":"'ended up losing access to his emails, contacts, photos, and even his phone number'","relation":"supports","source_id":"s2"}],"assertion":"Two days after the photos were taken his Google account was disabled for harmful content; he lost email, contacts and photos, and his Google Fi phone number.","causal_attribution":"The account loss followed directly from Google's flag and decision."},{"id":"c4","status":"reported","evidence":[{"locator":"'copies of the search warrants served on Google and his internet service provider'; '“I determined that the incident did not meet the elements of a crime and that no crime occurred,”'","relation":"supports","source_id":"s1"},{"locator":"'the San Francisco police department opened an investigation into him'","relation":"supports","source_id":"s3"}],"assertion":"San Francisco police obtained search warrants for his Google account and internet provider, and the investigator concluded that no crime had occurred.","causal_attribution":"The Times quotes the investigator's report; the report itself was not inspected."},{"id":"c5","status":"reported","evidence":[{"locator":"'Google responded that it would not reinstate the account, with no further explanation.'; 'appealed his case to Google again, providing the police report, but to no avail'; 'his account was being permanently deleted'","relation":"supports","source_id":"s1"}],"assertion":"Google rejected his appeals, including one with the police report, and later notified him that the account was being permanently deleted.","causal_attribution":"Reported from the father's account to the Times."},{"id":"c6","status":"reported","evidence":[{"locator":"'the same scenario was playing out in Texas'; 'which made the broker suspicious until'; 'A detective from the Houston Police department called in the fall of 2021, asking him to come into the station.'; 'But he, too, was unable to get his decade-old Google account back'","relation":"supports","source_id":"s1"}],"assertion":"A Houston father whose photos of his toddler, taken at a paediatrician's request, were flagged the next day had his Gmail account disabled while buying a house; police called him in and cleared him, and he could not recover the account.","causal_attribution":"Reported by the Times from the second father's account."},{"id":"c7","status":"reported","evidence":[{"locator":"'A Google spokeswoman said the company stands by its decisions, even though law enforcement cleared the two men.'; 'reviewers had not detected a rash or redness in the photos he took'","relation":"supports","source_id":"s1"},{"locator":"'use a combination of hash matching technology and artificial intelligence to identify it and remove it from our platforms'; 'reviews flagged content for accuracy and consults with pediatricians'","relation":"supports","source_id":"s2"},{"locator":"'Google has said it will stand by its decision'","relation":"supports","source_id":"s3"}],"assertion":"Google says it uses hash matching and artificial intelligence to identify child sexual abuse material, that its child-safety team reviews flagged content and consults paediatricians, and that it stands by its decisions although law enforcement cleared both men; it told the Times its reviewers saw no rash or redness and found a further video it considered problematic.","causal_attribution":"Google's account disputes that the decisions were wrong."}],"effects":[{"label":"Google accounts disabled and police investigations after AI flagged medical photos as child abuse material","claim_id":"c3","direction":"negative"}],"sources":[{"id":"s1","url":"https://www.nytimes.com/2022/08/21/technology/google-surveillance-toddler-photo.html","kind":"news_report","access":"read","language":"en","translation_note":"Read in English by the research agent (an AI) on 2026-11-02 through the Internet Archive capture web.archive.org/web/20221231230516id_/ of the article (HTTP 200; the live page returned 403); no translation and no human reviewer.","independence_group":"nyt-aug-2022-google-flagged-fathers-report"},{"id":"s2","url":"https://www.theverge.com/2022/8/21/23315513/google-photos-csam-scanning-account-deletion-investigation","kind":"news_report","access":"read","language":"en","translation_note":"Read in English by the research agent (an AI) on 2026-11-02 (HTTP 200); no translation. Relays the Times report; the Google statement it carries was emailed to The Verge.","independence_group":"nyt-aug-2022-google-flagged-fathers-report"},{"id":"s3","url":"https://www.theguardian.com/technology/2022/aug/22/google-csam-account-blocked","kind":"news_report","access":"read","language":"en","translation_note":"Read in English by the research agent (an AI) on 2026-11-02 (HTTP 200); no translation. Relays the Times report and carries a Google spokesperson statement.","independence_group":"nyt-aug-2022-google-flagged-fathers-report"}],"version":1,"ai_roles":["institutional_use"],"contexts":["privacy","health","justice","everyday_life"],"unknowns":["Whether either account was ever reinstated or either father sued after August 2022.","What the human reviewer saw and decided in each case, beyond the general process the Times describes.","The content of the further video Google says it found, which the father does not remember."],"geography":{"basis":"The Times places the first father in San Francisco and his investigation with the San Francisco Police Department, and the second in Houston, Texas, questioned by Houston police.","court_countries":[],"event_countries":["US"],"affected_person_countries":["US"]},"publication":{"basis":"Google's AI classifier made a claim about each father's photos that informed Google's decision to disable his account and report him, after which police investigated and cleared him. The account rests on the New York Times report and Google's own statements, which dispute that the decisions were wrong.","reviewed_on":"2026-11-02"},"ai_involvement":{"basis":"The New York Times reports that Google's artificial-intelligence tool for recognising never-before-seen child abuse images flagged the fathers' medical photos when they were backed up to Google's servers. That flag led Google to disable their accounts and report them, which led to the police investigations. Google confirms it uses AI alongside hash matching and that human reviewers check flagged content, so the AI flag informed the human decision.","status":"reported"},"person_relations":["made_claim_about","informed_decision_about"]},"name":"Google's AI flagged two fathers' medical photos of their toddlers as child abuse; accounts disabled, police investigated then cleared them, NYT reports","summary":"In February 2021 a San Francisco father took photos of his toddler son's medical condition after a nurse asked for photos before a video consultation; the doctor used them to diagnose the problem and prescribe antibiotics. The New York Times reported in August 2022 that when the photos were backed up from his Android phone to Google's servers, Google's artificial-intelligence tool for recognising never-before-seen child abuse images flagged them. Two days after the photos were taken his Google account was disabled for 'harmful content', which cost him more than a decade of email, contacts and photos and, through Google Fi, his phone number. Google reported him to the National Center for Missing and Exploited Children, and the San Francisco police obtained search warrants for his account; the investigator concluded that no crime had occurred. Google refused to reinstate the account after two appeals, the second with the police report, and later notified him that it was being permanently deleted. The Times reported a second father in Houston whose photos of his toddler, taken at a paediatrician's request, were also flagged; his Gmail account was disabled in the middle of buying a house, Houston police questioned and cleared him, and he did not get the account back. Google said it stood by its decisions and that its reviewers had found a further video in the San Francisco father's account that it considered problematic.","incidentDate":"2021-02-01","incidentKind":"bounded_series","incidentDatePrecision":"month","exposurePattern":"single_interaction","reportedDate":"2022-08-21","aiSystem":"Google's machine-learning tool for recognising never-before-seen child sexual abuse images (used with hash matching), applied to photos backed up from Android phones to Google's servers, followed by human review","aiProduct":"Google CSAM image classifier","severity":"medium","verificationStatus":"credible","harmCategories":[],"harmOutcomes":["legal_harm","other_material_harm"],"harmOutcomeSummary":"The New York Times reports that Google's AI flagged two fathers' medical photos of their toddlers as child abuse material, after which Google disabled their accounts (losing years of email, photos and, for one, his phone number) and police investigated both before clearing them; Google refused to reinstate the accounts.","frameworkFacets":[],"causationStatus":"supported","participantUsersAffectedMin":2,"otherPeopleHarmedMin":0,"affectedCountStatus":"exact","affectedCountEvidence":"Two account holders: the San Francisco and Houston fathers whose accounts were disabled and who were investigated. Their sons, counted by the CyberTipline among potential new child victims, and the San Francisco father's wife are not counted as harmed people.","victimAgeRange":"adult","jurisdiction":"US","platformType":"other","outcomeType":"investigation_opened","outcomeStatus":"unknown","primarySourceUrl":"https://www.nytimes.com/2022/08/21/technology/google-surveillance-toddler-photo.html","primarySourceLabel":"The New York Times, 21 Aug 2022: Google flagged a father as a criminal over medical photos of his toddler taken for the doctor","firstPublishedAt":"2026-11-02T03:05:51.915854+00:00","updatedAt":"2026-11-02T03:05:51.915854+00:00","scopeVersion":"facts-v3","tags":["google","child-safety-classifier","account-disabled","police-investigation","medical-photos","privacy","false-positive"]},{"id":"2021-amazon-delivery-drivers-say-netradyne-ai-van-cameras-flagged-driving-events-they-did-not-commit-lowering-bonus-scores","caseFacts":{"claims":[{"id":"c1","status":"reported","evidence":[{"locator":"'In February, Amazon announced that it would install cameras made by the AI-tech startup Netradyne in its Amazon-branded delivery vans'; 'Each time the camera registers an event, footage is uploaded into a system, recorded, and affects a score drivers receive at the end of the week for safe driving.'","relation":"supports","source_id":"s1"}],"assertion":"Amazon installed AI-powered Netradyne cameras in its delivery vans during 2021; the cameras detect driving events, upload the footage and feed a weekly safe-driving score.","causal_attribution":"Describes the system and its role in the score."},{"id":"c2","status":"reported","evidence":[{"locator":"'the camera in his van started to incorrectly penalize him whenever cars cut him off'; 'the Netradyne camera in his van has also detected “events” that didn’t actually happen, but that impacted his performance score at Amazon'","relation":"supports","source_id":"s1"}],"assertion":"Drivers said the cameras registered events they did not commit or that were not unsafe: a driver in Los Angeles said it incorrectly penalised him when cars cut him off, and a former driver in Texas said it detected events that did not happen and that they affected his performance score.","causal_attribution":"The drivers' accounts attribute the penalties to the camera's detections; Amazon and Netradyne did not address these individual events."},{"id":"c3","status":"reported","evidence":[{"locator":"'A few times, we’ve been in the country on a dirt road, where there’s no stop sign, but the camera flags a stop sign.'; 'If I look into my mirrors to make sure I am safe to change lanes, it dings me for distraction because my face is turned to look into my mirror.'","relation":"supports","source_id":"s1"}],"assertion":"Other drivers described false stop-sign detections, including a stop sign flagged on a dirt road with no sign, and being flagged for distraction when checking mirrors.","causal_attribution":"Drivers' accounts of the cameras' detection errors."},{"id":"c4","status":"reported","evidence":[{"locator":"'For many Amazon drivers, these performance scores determine whether they receive weekly bonuses, prizes, and extra pay.'; 'Before I would be able to win prizes and stuff, as soon as cameras came along, it went downhill,'; 'which receive bonuses by earning “fantastic” scores on a weekly scorecard'","relation":"supports","source_id":"s1"}],"assertion":"According to drivers and an Amazon scorecard document obtained by Motherboard, the weekly scores determine bonuses, prizes and extra pay, and the former Texas driver said winning prizes went downhill once the cameras came.","causal_attribution":"Connects the detections to pay; the amount lost by any individual driver is not reported."},{"id":"c5","status":"reported","evidence":[{"locator":"'but they don’t respond. There’s no room for discussion around the possibility that maybe the camera’s data isn’t clean.'; 'made the appeal process futile.'","relation":"supports","source_id":"s1"}],"assertion":"The Los Angeles driver said requests through his delivery company to see the photos flagged as events went unanswered, and a delivery-company owner said appeals were futile.","causal_attribution":"The drivers' and owner's account of the appeal route."},{"id":"c6","status":"reported","evidence":[{"locator":"'a team of Amazon employees manually reviews all events that are appealed to ensure that erroneous events do not impact drivers or Amazon delivery companies'; 'This technology provides drivers real-time alerts to help them stay safe when they are on the road.'; 'accidents had decreased by 48 percent'","relation":"supports","source_id":"s1"}],"assertion":"Amazon said the cameras give real-time safety alerts, that accidents had fallen 48 percent since installation, and that its employees manually review all appealed events so that erroneous events do not affect drivers.","causal_attribution":"Amazon's response; its safety figures are its own claims."}],"effects":[{"label":"Drivers say false camera events lowered their weekly scores","claim_id":"c2","direction":"negative"},{"label":"Scores decide bonuses and prizes","claim_id":"c4","direction":"negative"}],"sources":[{"id":"s1","url":"https://www.vice.com/en/article/amazons-ai-cameras-are-punishing-drivers-for-mistakes-they-didnt-make/","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-10-27 (HTTP 200, full article body).","independence_group":"motherboard-netradyne-2021-09-20"}],"version":1,"ai_roles":["institutional_use"],"contexts":["work","finance"],"unknowns":["How many events recorded against these drivers were wrong, and whether any was later overturned on appeal.","How much bonus or prize value any individual driver lost.","Whether Amazon or Netradyne changed the detection thresholds after these reports."],"geography":{"basis":"The drivers worked from Amazon delivery stations in Los Angeles, Sugar Land (Texas), Mobile (Alabama), Oklahoma and Kentucky, and the vans and routes are in the United States.","court_countries":[],"event_countries":["US"],"affected_person_countries":["US"]},"publication":{"basis":"Published because the drivers describe specific detection errors by the AI cameras (cars cutting in scored as unsafe following, mirror checks scored as distraction, a stop sign flagged where none existed) and say those detections lowered the weekly scores that decide their bonuses and prizes. The accounts are attributed, Amazon's response and appeal process are recorded, and the amount lost is left unknown.","reviewed_on":"2026-10-27"},"ai_involvement":{"basis":"Motherboard describes the Netradyne cameras as AI-powered devices that detect driving events (following distance, stop-sign and signal violations, distracted driving) and upload them to a score. The drivers say the cameras' own detections were wrong: a car cutting in was recorded as unsafe following distance, a glance at a mirror as distraction, and a stop sign was flagged on a road that had none. Those detections entered the weekly scores that, according to the drivers and Amazon's internal scorecard document, decide bonuses and prizes.","status":"reported"},"person_relations":["made_claim_about"]},"name":"Amazon delivery drivers say Netradyne AI cameras in their vans flagged driving 'events' they did not commit, lowering the weekly scores that set bonuses","summary":"In September 2021 Motherboard reported accounts from six Amazon delivery drivers in five US states and the owner of an Amazon delivery company, who said the AI-powered Netradyne cameras Amazon had installed in its delivery vans from early 2021 regularly registered driving 'events' that were beyond their control or were not unsafe driving: being cut off by another car, glancing at a side mirror, stopping past a stop sign to see around a bush, or a stop sign flagged on a dirt road with no sign. Each event fed a weekly safe-driving score that, according to the drivers and an internal Amazon document Motherboard obtained, helped decide drivers' bonuses and prizes and delivery companies' bonuses. A driver in Los Angeles said the camera incorrectly penalised him and that requests to see the photos flagged as events went unanswered; a former driver in Texas said events that did not happen affected his score, which decided prizes from his delivery company. Amazon said the technology gives drivers real-time safety alerts, that accidents and violations had fallen since it was installed, and that its staff manually review appealed events so that erroneous ones do not affect drivers. Netradyne did not respond to Motherboard.","incidentDate":"2021-01-01","incidentKind":"ongoing_experience","incidentDatePrecision":"year","exposurePattern":"repeated_interactions","reportedDate":"2021-09-20","aiSystem":"Netradyne AI-powered four-lens cameras installed in Amazon-branded delivery vans, which detect driving 'events' such as following distance, stop-sign and signal violations and distracted driving, upload the footage and feed Amazon's weekly driver and delivery-company scores","aiProduct":"Netradyne cameras","severity":"low","verificationStatus":"credible","harmCategories":[],"harmOutcomes":["professional_harm","financial_loss"],"harmOutcomeSummary":"Drivers told Motherboard that the AI cameras registered events they did not commit or that were not unsafe, lowering the weekly scores that decided their bonuses and prizes; one said winning prizes 'went downhill' once the cameras came, and another described the penalties as upsetting.","frameworkFacets":[],"causationStatus":"alleged","participantUsersAffectedMin":0,"otherPeopleHarmedMin":2,"affectedCountStatus":"partial","affectedCountEvidence":"Two drivers are counted: the Los Angeles driver, who said the camera incorrectly penalised him in the data used for his weekly evaluation and bonus, and the former Sugar Land driver, who said events that did not happen affected his score and his prizes. Motherboard spoke to six drivers and a delivery-company owner who described false or unfair events more generally; the others are not counted because a personal score or pay consequence is not reported for each. The fleet size and driver numbers in the article are not counts of harmed people. The drivers are not counted as direct AI users.","victimAgeRange":"adult","jurisdiction":"US","platformType":"other","outcomeStatus":"unknown","primarySourceUrl":"https://www.vice.com/en/article/amazons-ai-cameras-are-punishing-drivers-for-mistakes-they-didnt-make/","primarySourceLabel":"Motherboard (VICE), 20 September 2021: Amazon's AI cameras are punishing drivers for mistakes they didn't make","firstPublishedAt":"2026-10-27T03:09:21.159219+00:00","updatedAt":"2026-10-27T03:09:21.159219+00:00","scopeVersion":"facts-v3","tags":["work","delivery-drivers","workplace-monitoring","in-vehicle-camera","driver-scores","pay","institutional-use"]}]}