{"meta":{"exportedAt":"2026-10-09T08:38:50.837Z","formatVersion":2,"selection":{"q":"security","system":"","harm":"","context":"","country":"","role":"","relation":"contextual","evidence":"","year":"2025","response":"","severity":"","verification":"","view":"incidents","sort":"added"},"totalIncidents":2,"coverage":{"cases":2,"countries":1,"languages":2,"unknownLocation":1,"locationPending":0,"unknownLanguage":0,"unknownDate":0,"lawsuits":0,"regulatory":0,"minors":1,"coreRelations":0,"contextualRelations":2,"mixedRelations":0,"unknownRelations":0,"relationPending":0,"relationUnknown":0},"countingNote":"Distinct public cases in this selection. People counts apply within individual cases only; cross-case person overlap has not been resolved. No population incidence estimate.","affectedCountNote":"Interpret person counts with affectedCountStatus and the reported effects. Unquantified zeros are placeholders, not a measured zero.","source":"AI incidents","publisher":"NOPE","url":"https://nope.net/incidents","license":"CC BY 4.0"},"incidents":[{"id":"2025-baltimore-county-md-school-ai-gun-alert-chip-bag-student-searched","caseFacts":{"claims":[{"id":"c1","status":"corroborated","evidence":[{"locator":"after an AI-driven security system flagged the teen's empty bag of chips as a possible firearm","relation":"supports","source_id":"s1"},{"locator":"How did it come to be that we had police officers with guns drawn approaching a kid because of a bag of Doritos?","relation":"supports","source_id":"s3"},{"locator":"Omnilert AI Gun Detection System warned school leaders at Kenwood High School that a student had a gun.","relation":"supports","source_id":"s3"},{"locator":"Police officers responded to the school, searched the individual, and quickly confirmed that they were not in possession of any weapons.","relation":"supports","source_id":"s3"}],"assertion":"An AI gun detection system (Omnilert) alerted on a Kenwood High School student in Baltimore County, the object was a bag of chips and not a weapon, and police searched the student.","causal_attribution":"The reports connect the police response to the alert. The district and the vendor say the system operated as designed and that humans review alerts, so responsibility for the police response is contested between the system, the review steps and the responding staff."},{"id":"c2","status":"reported","evidence":[{"locator":"They made me get on my knees, put my hands behind my back, and cuffed me,","relation":"supports","source_id":"s1"},{"locator":"they had a gun pointed at me","relation":"supports","source_id":"s1"},{"locator":"handcuffed and searched","relation":"supports","source_id":"s2"}],"assertion":"The student reported being made to kneel, being handcuffed and searched, and having a gun pointed at them.","causal_attribution":"First-person account given to a local television station and relayed by the CNN wire and the Guardian (one chain). CNN also states the handcuffing in its own voice without naming a source other than the student interview. Police statements inspected confirm a search only."},{"id":"c3","status":"reported","evidence":[{"locator":"reviewed and canceled the gun detection alert after confirming there was no weapon","relation":"supports","source_id":"s1"},{"locator":"The principal didn't immediately realize the alert had been canceled","relation":"supports","source_id":"s1"},{"locator":"she reported the matter to Kenwood's school resource officer, who called local police for support","relation":"supports","source_id":"s1"},{"locator":"The Department of School Safety and Security reviewed and canceled the initial alert after confirming there was no weapon.","relation":"supports","source_id":"s3"}],"assertion":"The school district security department reviewed and canceled the alert after confirming there was no weapon. A district spokesperson said the principal did not immediately realize the alert had been canceled. The principal reported the matter to the school resource officer, who called police.","causal_attribution":"Account of the review sequence comes from the principal letter and district spokespeople. It attributes the police response to a communication gap after human review and does not exonerate the alert."},{"id":"c4","status":"reported","evidence":[{"locator":"Baltimore County officials say they want a review of the process that led to Monday's police response.","relation":"supports","source_id":"s1"},{"locator":"Baltimore County councilmembers are calling on school officials and police to review an A.I. gun detection system","relation":"supports","source_id":"s3"},{"locator":"Our counselors will provide direct support to the students who were involved in this incident","relation":"supports","source_id":"s2"}],"assertion":"County councilmembers called for a review of the AI system, and the school said counselors would support the students involved.","causal_attribution":"Institutional responses reported by the outlets. The outcome of any review is not established in the inspected sources."}],"effects":[{"label":"Student searched by armed police after a false AI weapon alert","claim_id":"c1","direction":"negative"}],"sources":[{"id":"s1","url":"https://abc7.com/post/student-handcuffed-doritos-bag-mistaken-gun-schools-ai-security-system-baltimore-county-maryland/18073796/","kind":"news_report","access":"read","language":"en","translation_note":"","independence_group":"wbal-relay"},{"id":"s2","url":"https://www.theguardian.com/us-news/2025/oct/24/baltimore-student-ai-gun-detection-system-doritos","kind":"news_report","access":"read","language":"en","translation_note":"","independence_group":"wbal-relay"},{"id":"s3","url":"https://www.cbsnews.com/baltimore/news/false-alarm-gun-detection-kenwood-maryland-artificial-intelligence-review/","kind":"news_report","access":"read","language":"en","translation_note":"","independence_group":"wjz-cbs-baltimore"}],"version":1,"ai_roles":["institutional_use"],"contexts":["education","public_services"],"unknowns":["Event date is derived from the reports' \"Monday\" references relative to their publication dates (23 to 26 October 2025). The inspected bodies do not print the calendar date of the event.","The handcuffing and gun-pointing account is one first-person account relayed by two outlets, and CNN also states the handcuffing in its own voice without another named source. Police statements inspected confirm only a search after a report of a suspicious person with a weapon.","The outcome of the county review is not reported in the inspected sources.","The student's exact age is not stated. Sources call the student a teen and a child.","Vendor and district statements say the system worked as designed and that humans review alerts. Whether the review step or the alert caused the police dispatch is not settled."],"geography":{"basis":"The reports place the event at a high school in Baltimore County, Maryland, with the county police department and county council responding. Country is stated by place names in the inspected reports and is not inferred from outlet or vendor headquarters.","court_countries":[],"event_countries":["US"],"affected_person_countries":["US"]},"publication":{"basis":"Three news reports read in full (one via Internet Archive capture). The AI alert, the absence of a weapon and the search are separately reported. The handcuffing account is attributed to the student. The student's name is omitted because the student is a minor.","reviewed_on":"2026-09-29"},"ai_involvement":{"basis":"Two separately reported chains say the AI alert flagged the object: the CNN wire with a vendor statement (its student account is WBAL-derived, as is the Guardian, a WBAL relay), and local CBS reporting with vendor and superintendent statements. Both also report human review steps between the alert and the police response.","status":"supported"},"person_relations":["made_claim_about"]},"name":"Baltimore County, Maryland: school AI gun alert on a bag of chips leads to police search of a student","summary":"On a Monday evening in October 2025 (20 October by the reports' weekday references) an AI gun detection system (Omnilert) used at Kenwood High School alerted on a student holding an empty bag of chips. Police responded and searched the student and found no weapon. The student told a local television station that officers made them kneel, handcuffed them and pointed a gun at them. The school district says its security department reviewed and canceled the alert after confirming there was no weapon, and a district spokesperson said the principal did not immediately realize the alert had been canceled. County councilmembers called for a review. No outcome of a review is established in the sources.","incidentDate":"2025-10-20","incidentKind":"single_event","incidentDatePrecision":"day","exposurePattern":"single_interaction","reportedDate":"2025-10-23","aiSystem":"Omnilert AI Gun Detection System (video analytics on school security cameras)","aiProduct":"Omnilert gun detection","aiCompany":"Omnilert","severity":"medium","verificationStatus":"credible","harmCategories":[],"harmOutcomes":["loss_of_liberty","psychological_distress"],"harmOutcomeSummary":"A student was searched by police after the AI alert and reported being handcuffed at gunpoint and fearing for their life (student account relayed by CNN and the Guardian). The school letter confirms a search and offers counseling.","frameworkFacets":[],"causationStatus":"supported","participantUsersAffectedMin":0,"otherPeopleHarmedMin":1,"affectedCountStatus":"documented_minimum","affectedCountEvidence":"One student who was searched. Other students who witnessed the event are mentioned in the school letter but are not counted without a reported harm to each.","victimAgeRange":"minor","platformType":"other","primarySourceUrl":"https://www.cbsnews.com/baltimore/news/false-alarm-gun-detection-kenwood-maryland-artificial-intelligence-review/","primarySourceLabel":"CBS Baltimore (WJZ): A.I. gun detection false alarm at school has Baltimore County leaders calling for review, 23 October 2025","firstPublishedAt":"2026-09-29T12:42:19.255568+00:00","updatedAt":"2026-09-30T01:16:56.487109+00:00","scopeVersion":"facts-v3","tags":["historical-2025"]},{"id":"2025-br-meu-inss-rural-worker-retirement-claim-rejected","caseFacts":{"claims":[{"id":"c1","status":"reported","evidence":[{"locator":"instantly turned down because the system identified","relation":"supports","source_id":"s1"},{"locator":"teve seu pedido negado de forma automática devido a um erro de identificação, sendo registrada como homem no sistema","relation":"supports","source_id":"s2"}],"assertion":"In a retirement claim filed through the Meu INSS app, the request was rejected because the system identified the claimant as a man.","causal_attribution":"Rest of World reports the rejection as caused by the system's identification of the claimant as a man. The source is the claimant's account as reported by the journalist. The Portuguese rewrite depends on the same article. No decision record was seen."},{"id":"c2","status":"reported","evidence":[{"locator":"in February was approved in March","relation":"supports","source_id":"s1"},{"locator":"went straight to INSS directors, who identified and corrected the mistake in the app.","relation":"supports","source_id":"s1"}],"assertion":"The rejected retirement claim was filed in February and approved in March after INSS directors corrected the mistake in the app.","causal_attribution":"The article ties the approval to the claimant's contact at an agricultural workers' confederation, and states that the case went straight to INSS directors. It does not say the contact routed it."},{"id":"c3","status":"reported","evidence":[{"locator":"had led to numerous rejections, with few options for recourse","relation":"supports","source_id":"s1"},{"locator":"proving everything, and [the benefit] still gets denied. It’s a humiliation,","relation":"supports","source_id":"s1"}],"assertion":"Rest of World reports that minor errors in the claimant's claims filed through the app led to numerous rejections, and quotes the claimant saying that despite having all the documents proving a health condition the benefit is still denied.","causal_attribution":"The article attributes the earlier rejections to minor errors in claims filed through the app. Dates of these rejections and whether an AI component made them are not stated."},{"id":"c4","status":"reported","evidence":[{"locator":"Each automated decision is based on specified legal criteria, ensuring that the standards set by the social security legislation are respected,","relation":"supports","source_id":"s1"},{"locator":"Some policyholders have been misusing the program, filing multiple requests in the hopes of obtaining different results, the spokesperson said.","relation":"supports","source_id":"s1"}],"assertion":"An INSS spokesperson told Rest of World that each automated decision follows specified legal criteria, and that some policyholders file multiple requests hoping for different results.","causal_attribution":"This is the institution's stated position and does not address the claimant's specific rejection."}],"effects":[{"label":"Retirement claim rejected, approved after INSS directors corrected the error","claim_id":"c1","direction":"negative"}],"sources":[{"id":"s1","url":"https://restofworld.org/2025/brazil-ai-social-security-app-rejected/","kind":"news_report","access":"read","language":"en","translation_note":"Full body read from a Wayback capture (20260128230439id_) after the live URL returned HTTP 429. Original English.","independence_group":"restofworld-daros"},{"id":"s2","url":"https://olhardigital.com.br/2025/04/25/pro/sistema-de-ia-do-inss-causa-recusas-injustas-de-beneficios/","kind":"news_report","access":"read","language":"pt","translation_note":"Portuguese rewrite of the Rest of World article (it credits Rest of World). Read in the original Portuguese by the reviewing agent without a human translator. Adds no independent reporting.","independence_group":"restofworld-daros"}],"version":1,"ai_roles":["institutional_use"],"contexts":["public_services"],"unknowns":["Whether the wrong-sex rejection came from an AI model, a rule engine or a data-matching step is not established.","The article gives the filing month as February without a year. The year 2025 is read from the article date (24 April 2025) and its present-tense context.","Dates and causes of the claimant's earlier sick-pay rejections are not stated, so this record covers only the February 2025 retirement claim.","No INSS decision record or independent confirmation of the claimant's account was inspected. The Portuguese rewrite adds no independent reporting.","Rest of World's statement that the app rejected requests from hundreds of people is an unsourced aggregate and is not counted."],"geography":{"basis":"The claimant is described as living in a remote town in northeast Brazil. Where the claim was filed from is not stated, so no event country is recorded. No court is involved.","court_countries":[],"event_countries":[],"affected_person_countries":["BR"]},"publication":{"basis":"A named-outlet journalist interviewed the claimant and the INSS, and the article reports a concrete rejected claim, its reported cause and its correction. The article gives the filing and approval months (February, March) without a year, so 2025 is inferred from its 24 April 2025 date. The AI role is reported and unverified, so claims are attributed and the record is marked alleged.","reviewed_on":"2026-09-29"},"ai_involvement":{"basis":"Rest of World describes Meu INSS as an AI-powered app (computer vision and natural language processing over uploaded documents) and reports that the claim was rejected because the system identified the claimant as a man. The article does not say whether a model or a fixed rule produced this rejection, and the INSS describes the decisions as automated.","status":"reported"},"person_relations":["made_decision_about"]},"name":"Brazil: rural worker's retirement claim rejected through the Meu INSS app after the system reportedly recorded a wrong sex","summary":"In February (2025, inferred from the article date) a retirement claim filed through Brazil's Meu INSS social security app for a former sugarcane worker in northeast Brazil was rejected. Rest of World reported that the request was turned down because the system identified the claimant as a man. The claimant, who has chronic illnesses, is quoted saying the benefit is still denied despite complete documents, and Rest of World reports that minor errors in earlier claims through the app had led to numerous rejections. The retirement claim was approved in March after INSS directors corrected the mistake, which the claimant attributed to a contact at an agricultural workers' confederation. The INSS states that automated decisions follow specified legal criteria.","incidentDate":"2025-02-01","incidentKind":"single_event","incidentDatePrecision":"month","exposurePattern":"single_interaction","reportedDate":"2025-04-24","aiSystem":"Meu INSS app (AI-assisted automated analysis of benefit claims)","aiProduct":"Meu INSS app","aiCompany":"Dataprev (developer of the app, per Rest of World); INSS deploys it","severity":"low","verificationStatus":"credible","harmCategories":[],"harmOutcomes":["other_material_harm"],"harmOutcomeSummary":"According to Rest of World, a retirement claim was rejected through the app, reportedly because the system recorded the wrong sex, and was approved in March after INSS directors corrected the error.","frameworkFacets":[],"causationStatus":"alleged","participantUsersAffectedMin":1,"otherPeopleHarmedMin":0,"affectedCountStatus":"documented_minimum","affectedCountEvidence":"One claimant with a rejected and later corrected retirement claim. Rest of World says the app has rejected requests from hundreds of people for minor errors, which is an unsourced aggregate and is not counted.","victimAgeRange":"adult","platformType":"other","primarySourceUrl":"https://restofworld.org/2025/brazil-ai-social-security-app-rejected/","primarySourceLabel":"Rest of World, 'Brazil's AI-powered social security app is wrongly rejecting claims', 24 April 2025","firstPublishedAt":"2026-09-29T12:42:06.218216+00:00","updatedAt":"2026-09-30T01:16:57.309447+00:00","scopeVersion":"facts-v3","tags":["historical-2025"]}]}