San Francisco city attorney David Chiu has issued cease-and-desist letters to Apple and Google, ordering them to remove 13 artificial intelligence-powered “nudify” applications from their app stores. The apps, which have been used to generate realistic nude images of individuals without their consent, are now the subject of a legal push that could reshape how tech giants handle AI-generated sexual content. According to a report from a technology news outlet, the letters demand that both companies stop what the city attorney describes as “aiding and abetting” the sale of sexualized AI deepfakes, which is prohibited under California law.
This is not the first time that San Francisco has taken a hard line against nonconsensual intimate imagery, but it is one of the most direct actions ever aimed at the app store gatekeepers themselves. By targeting Apple and Google, the city attorney is making it clear that platform owners cannot simply rely on user reports or reactive takedowns. Instead, they are being told to take proactive steps to identify and remove apps that facilitate the creation of simulated nude images. The legal theory behind the letters is that app stores are not neutral distributors but active participants in making these tools available to millions of users.
The demand and the apps
The 13 apps named in the cease-and-desist letters are said to use generative AI to “undress” photos of real people. Some of these apps have been available on the Apple App Store and Google Play Store for months, and they often market themselves as entertainment tools or photo editing utilities. However, their core function is to create nonconsensual sexual imagery, which is illegal in many jurisdictions. San Francisco’s city attorney has asked both companies to remove the apps immediately and to take steps to prevent similar apps from appearing in the future.
A Google spokesperson said the company has already suspended the five Android apps cited in Chiu’s letter, adding that the Play Store “does not allow apps that contain sexual content.” Google’s response suggests the company is willing to act quickly when faced with legal pressure, but it also raises questions about why these apps were approved in the first place. Apple, on the other hand, has not yet issued a public statement. However, the company has previously stated that its App Store review team is committed to protecting user safety, and it has guidelines that prohibit apps with explicit sexual content or that encourage nonconsensual behavior.
The mechanics of nudify apps
“Nudify” apps have become increasingly sophisticated in recent years. Early versions of these apps were crude and easily detectable, often producing obvious artifacts that made the altered images look fake. Modern versions, however, use state-of-the-art deep learning models that can seamlessly generate nude bodies and superimpose them onto the faces of real people. The results can be so realistic that even close friends and family members may be fooled. These apps typically require only a single photo of a person, which can be taken from social media, a group photo, or even a video frame. The AI then reconstructs the person’s body and generates a nude image without their consent or knowledge.
The psychological and social damage caused by these apps is profound. Victims of AI-generated deepfake pornography often report feeling violated, humiliated, and powerless. The images can be shared widely across the internet, leading to harassment, reputational harm, and even job loss. Unlike traditional photo editing, the AI does not need to be manually guided; it can generate a realistic image in seconds. This ease of use has made nudify apps especially dangerous, as they allow anyone with a smartphone to create nonconsensual pornography.
California law and legal precedent
California has been at the forefront of the fight against nonconsensual deepfakes. In 2019, the state passed AB 602, a law that made it illegal to distribute intimate images without consent, whether they are real or computer-generated. The law also allows victims to sue the creators and distributors of such content. More recently, California lawmakers have introduced bills specifically targeting AI-generated deepfakes, including those that depict real people in sexual situations. The cease-and-desist letters from San Francisco rely on this legal framework, arguing that Apple and Google are participating in the sale of a product that violates the law.
The city attorney’s office is likely preparing to take further legal action if Apple and Google do not comply. Under California law, businesses that “aid and abet” illegal activity can be held liable for damages. By allowing nudify apps to remain on their platforms, Apple and Google could be exposed to lawsuits from victims and from the state. The letters are a warning that the city is willing to use the full extent of its authority to protect residents from AI-enabled abuse.
Platform policies under scrutiny
Both Apple and Google have long-standing policies against explicit sexual content. Apple’s App Store Review Guidelines, for example, state that apps containing “overtly sexual or pornographic material” will be rejected. Google’s Play Store similarly prohibits apps that contain or promote sexual content, including pornography and apps that can be used to create it without consent. Yet these policies have not always been effectively enforced when it comes to AI-powered nudification tools.
Some of the apps cited in the letters have been available for months, which suggests that automated review processes and manual inspections have failed to catch them. Many nudify apps hide their purpose behind vague descriptions, innocuous icons, and user agreements that claim the software is for “entertainment purposes only.” They also often operate behind paywalls, requiring users to subscribe before they can use the full range of features. This business model makes them profitable, which is why so many developers continue to produce them despite the obvious harms.
Tech platforms have increasingly been criticized for their slow response to deepfake abuse. The 2024 election season saw a surge in AI-generated fake videos and audio, prompting social media companies to adopt new labeling and removal policies. However, app stores have been slower to act, perhaps because they view themselves as infrastructure rather than content providers. The San Francisco letters could change that by forcing Apple and Google to take on a more active watchdog role.
Broader context of AI deepfakes
The demand to remove nudify apps is part of a larger global conversation about the dangers of generative AI. While AI can be used for creative and productive purposes, it also enables new forms of abuse. Deepfake pornography exploded in popularity in 2023 and 2024, with dedicated websites and messaging bots attracting millions of users. Most of the victims are women and girls, and the images often circulate on social media and adult websites. The spread of deepfake nude photos has been linked to cyberbullying, self-harm, and in some tragic cases, suicide.
Several countries have passed laws specifically addressing sexual deepfakes. The United Kingdom, for instance, made the sharing of deepfake pornography a criminal offense in early 2025. Australia has similar legislation, and the European Union is working on new AI rules that would require platforms to assess the risks of deepfake technology. In the United States, federal law is still lagging, which has led states like California, New York, and Texas to take their own action.
What’s next for app stores
The cease-and-desist letters may be just the beginning. If Apple and Google refuse to comply, San Francisco could pursue an injunction or seek financial penalties. The outcome of this case could have far-reaching implications for how app stores regulate AI-powered tools. It might also prompt other cities and states to issue similar demands, creating a patchwork of rules that tech companies must navigate.
For now, Google’s decision to suspend the five apps is a sign that at least one platform is taking the matter seriously. Apple has not yet commented, but the company’s silence is unlikely to last. The App Store has often been described as a “walled garden” where Apple maintains strict control over what users can download. That control comes with responsibility, and the city attorney is now demanding that Apple live up to it.
Developers of nudify apps may try to circumvent the takedown by re-releasing the same tools under different names or through sideloading. But the legal pressure from San Francisco makes it clear that such workarounds will be met with continued resistance. In the meantime, victims of AI-generated abuse are left waiting for meaningful protection, both from the platforms that host these tools and from the legal system that is only now beginning to catch up.
The growing availability of AI-generated nonconsensual imagery represents one of the most disturbing side effects of the generative AI boom. As the technology becomes more accessible and realistic, the potential for misuse will only increase. This latest action against Apple and Google marks a critical step toward accountability, but it is unlikely to be the last. Lawmakers, advocates, and tech companies will need to work together to develop comprehensive solutions that balance innovation with the fundamental right to personal privacy and bodily autonomy.
Source: The Verge News