You might of used Grok before and it’s parent company xAI is challenging a new Minnesota law that imposes substantial penalties on websites, applications and software services that allow people to create realistic sexualized images of identifiable individuals.
The lawsuit presents a difficult legal question that states will increasingly confront as generative AI becomes more powerful: How can lawmakers protect people from nonconsensual intimate imagery without imposing an unconstitutional restriction on legitimate expression or general-purpose creative tools?
xAI argues that Minnesota’s law goes far beyond preventing image-based sexual abuse. The company claims the statute is an overbroad, content-based restriction that could impose liability for consensual, private or nonsexual images protected by the First Amendment.
Minnesota officials contend that the law targets technology capable of inflicting severe harm by creating realistic intimate images of real people without their participation or consent.
A federal judge declined xAI’s emergency request to stop the statute from taking effect on 1 August 2026. The denial did not resolve the constitutional dispute, and xAI’s broader challenge remains pending.
What Minnesota’s Nudification Law Prohibits
Minnesota House File 1606 creates a statewide prohibition on certain forms of “nudification” technology.
The law defines nudification as altering or generating an image or video to depict an intimate body part that did not appear in the original image. The resulting content must be realistic enough that a reasonable person would believe the depicted body part belongs to the identifiable individual.
The statute applies to people and companies that own or control websites, applications, software, programs or other services. Covered operators may not:
- Allow users to access, download or use a service to nudify an image or video.
- Nudify an image or video on behalf of a user.
- Advertise or promote a service that performs those functions.
The law includes an exemption when the software requires substantial technological or artistic skill and judgment from the user. That exemption appears intended to distinguish automated, consumer-facing nudification tools from professional creative software requiring meaningful human involvement.
Exactly where that line falls may become an important issue in the litigation.
Penalties Can Reach $500,000 Per Violation
The Minnesota attorney general can seek civil penalties of up to $500,000 for each unlawful access, download or use of prohibited nudification technology.
The law also gives individuals depicted in prohibited images a right to bring civil lawsuits. Available remedies can include:
- Compensatory damages for mental anguish and suffering.
- Up to three times the actual damages sustained.
- Punitive damages.
- Injunctive relief.
- Attorney fees and litigation costs.
- Other relief considered appropriate by the court.
xAI argues that the potential penalties are grossly disproportionate and could reach extraordinary amounts when applied to a platform with a large user base.
If users generated 100,000 prohibited images, for example, applying the statutory maximum to each violation could theoretically create $50 billion in exposure. Whether penalties would actually be calculated or imposed that way remains uncertain.
xAI Says the Law Regulates Protected Expression
xAI filed its lawsuit against Minnesota Attorney General Keith Ellison in the U.S. District Court for the District of Minnesota.
The company argues that Minnesota has imposed a content-based restriction because liability depends on what an AI-generated image depicts. Content-based laws receive demanding constitutional scrutiny when they regulate protected expression.
xAI does not dispute that Minnesota has a legitimate interest in addressing nonconsensual intimate imagery. Its complaint instead argues that the statute reaches substantially beyond that interest.
According to xAI, the law may apply even when:
- The person depicted consented to the image.
- The content remains private and is never distributed.
- The image is artistic, satirical or otherwise expressive.
- The platform operator did not intend for its technology to be misused.
- A user bypassed safeguards established by the provider.
- The resulting image would not ordinarily be considered explicit.
The company also challenges Minnesota’s use of the term “intimate part,” arguing that the incorporated statutory definition may cover a broader range of body depictions than the term “nudification” would ordinarily suggest.
xAI maintains that the First Amendment protects visual-expression tools as well as at least some of the images those tools create.
Minnesota Focuses on the Harm to Victims
Supporters of the law argue that AI-generated intimate imagery is not a harmless form of digital experimentation.
Nudification services can transform ordinary photographs from social media, school events or family albums into realistic sexualized images. Victims may experience reputational damage, harassment, emotional distress and fear that other people will mistake the fabricated material for authentic photographs.
The legislation was championed after Minnesota lawmakers learned about cases involving people who allegedly used social media photographs to create sexualized images of women they knew.
Sen. Erin Maye Quade, one of the law’s sponsors, has compared the conduct to using technology to peer through someone’s window and capture an intimate image. The central premise is that people should not lose control of their bodily privacy merely because an AI system can generate a convincing fabrication.
Minnesota officials also argue that holding only individual users responsible may be insufficient. Users may be anonymous, located outside the state or unable to pay meaningful damages. The law therefore places responsibility on companies that make automated nudification capabilities readily available.
xAI Points to Its Existing Safeguards
xAI says its policies prohibit users from generating nude or sexualized images of people without consent.
The company maintains that it has implemented technical protections intended to prevent prohibited generations. It has also pursued legal action against users accused of circumventing those controls.
That position raises another issue likely to appear in future AI regulation: Should a provider face strict liability whenever a user defeats its safeguards, or should the law consider the provider’s knowledge, intent and reasonable compliance efforts?
A liability standard that does not recognize good-faith safety measures could encourage companies to block broad categories of legitimate content. Conversely, a weak standard could allow providers to rely on written policies while deploying ineffective technical protections.
Courts and lawmakers may need to determine what level of image filtering, identity protection, age detection, user verification and incident response constitutes reasonable prevention.
Judge Declines to Block the Law Immediately
U.S. District Judge Donovan Frank denied xAI’s request for a temporary restraining order shortly before the law took effect.
The judge noted that xAI waited nearly three months after the law was signed before filing its emergency motion. The company sought relief only days before the 1 August effective date.
That delay weakened xAI’s argument that it faced the type of immediate harm required for emergency relief.
The decision was procedural and did not determine that Minnesota’s law is constitutional. The court is expected to consider xAI’s request for a preliminary injunction after receiving fuller briefing from both sides.
A hearing is scheduled for 19 August 2026. The law remains in effect while the litigation continues unless the court issues a subsequent order restricting enforcement.
The Law Targets Technology, Not Only Distribution
One of the most significant features of the Minnesota statute is that it does not focus exclusively on publishing or distributing nonconsensual intimate imagery.
Existing laws often target the person who knowingly shares an intimate image without consent. Minnesota’s law reaches further by restricting access to the technology used to create the image.
That approach could allow the state to intervene before fabricated content is distributed. It also creates a more difficult First Amendment question because image-generation tools can have lawful and unlawful applications.
The distinction will be central to the case. Minnesota views automated nudification services as products designed around an inherently invasive function. xAI characterizes its technology as a general-purpose visual-expression tool that can be misused despite provider safeguards.
What AI Providers Should Take From the Case
Regardless of which side ultimately prevails, AI companies should expect closer scrutiny of how their products prevent the creation of nonconsensual intimate content.
Providers of image and video generation tools should consider implementing controls involving:
- Clear prohibitions against sexualizing identifiable people without consent.
- Technical detection of prompts requesting nudification or sexual alteration.
- Controls preventing users from uploading images of minors for sexualized editing.
- Identity and age verification for access to adult-content features.
- Rate limits and monitoring for repeated attempts to bypass safeguards.
- Human review of high-risk activity and user reports.
- Preservation of evidence relating to serious abuse.
- Rapid removal and escalation procedures.
- Auditing to determine whether safety controls work in practice.
- Documented procedures for responding to victims and law enforcement.
Companies should also examine whether their marketing, feature descriptions or interface designs could be interpreted as promoting prohibited uses.
A policy stating that nonconsensual content is forbidden will carry limited weight if the system routinely produces it or if users can evade controls with simple prompt variations.
A Test Case for AI Regulation and the First Amendment
The Minnesota dispute is about more than a single application or company. It represents an early test of how far states can go when regulating AI systems capable of creating harmful but potentially expressive content.
Lawmakers face legitimate pressure to protect victims before fabricated intimate images spread across the internet. AI providers have a legitimate interest in preventing states from imposing vague or sweeping liability on technologies with lawful uses.
The strongest regulatory approach may ultimately require greater precision than a complete ban or complete immunity. Laws can focus on identifiable victims, lack of consent, provider knowledge, reasonable safety measures and the severity of the resulting harm.
For now, Minnesota’s law is in effect, xAI’s constitutional challenge continues and other states will be watching closely.
The eventual outcome could shape whether future AI safety laws regulate the harmful image, the user who requested it, the provider that generated it—or all three.