The United Kingdom’s National Crime Agency (NCA) has issued a stark warning to parents: stop sharing pictures of your children on public social media platforms. The advisory comes in collaboration with the Internet Watch Foundation (IWF), a charity tasked with identifying and removing child sexual abuse material online. The reason behind this urgent appeal is the dramatic rise in AI-generated child sexual abuse imagery, which has reached alarming levels over the past year.
According to the IWF, 8,029 AI-generated images and videos of realistic child sexual abuse were identified in 2025 — a 14 percent increase from the previous year. While still images remain a significant concern, the growth in synthetic video content is particularly staggering. In 2024, only 13 confirmed AI-generated abuse videos were found. By 2025, that number had skyrocketed to 3,440. Under UK law, such material is treated as child sexual abuse imagery regardless of whether it is created by a human or artificial intelligence.
The growing threat of AI-generated abuse
The joint guidance from the NCA and IWF urges parents to review their privacy settings and restrict photo sharing to trusted groups, such as a “close friends” list. It also recommends auditing older posts for identifying details like a child’s face, school uniform, or location, and revisiting consent given to schools, clubs, and sports organizations that photograph children. “While we and policing colleagues tackle offenders, prevention remains vital,” said Tim Wright, a senior manager at the NCA.
The phenomenon of “sharenting” — a portmanteau of sharing and parenting that entered the Collins English Dictionary in 2016 — has long been a topic of concern for privacy and identity theft. However, the emergence of sophisticated AI tools capable of convincingly manipulating photos has transformed an old worry into a far more immediate and dangerous threat. These tools can scrape images from public social media accounts, school websites, and family blogs, then use generative adversarial networks (GANs) or diffusion models to produce explicit material featuring real children.
The IWF previously reported a chilling case in which a criminal gang scraped pupils’ photographs from a school website and used AI to create over 100 sexual images of the children. The data also reveals a heavily gendered dimension: in 2024, 98 percent of confirmed AI abuse imagery involving identifiable children depicted girls. This pattern aligns with broader trends in deepfake pornography, which overwhelmingly targets women and girls.
Children's awareness and fears
Children themselves are acutely aware of the threat. Research conducted by UNICEF found that a quarter of children fear their images being turned into explicit deepfakes without their consent. Campaigners have long pushed for stronger European tools to combat nonconsensual deepfake imagery, and the issue has gained traction in legislative circles across the continent. The concern is not limited to the UK; similar warnings have been issued by child safety organizations in the United States, Australia, and the European Union.
The ability to generate convincing synthetic media has exploded in recent years. Open-source AI models, such as Stable Diffusion and various deepfake frameworks, have made it possible for individuals with minimal technical skills to create realistic images and videos. While some companies have attempted to implement safety filters, these are often easily bypassed. Researchers have demonstrated that current safeguards are insufficient to prevent the misuse of AI for generating child sexual abuse material.
Regulatory responses and legal changes
The UK government has taken steps to address the issue. It has moved to ban so-called “nudification apps” — tools that digitally remove clothing from images of real people — and has amended the law to allow AI developers to test whether their systems can be abused to produce such material. The IWF, which campaigned for the ban, has described nudification apps as products with “no reason to exist.” Meanwhile, Ofcom, the UK’s communications regulator, is enforcing the Online Safety Act, which imposes a duty of care on platforms to protect children. Ofcom recently opened a child safety investigation into Telegram, following previous probes into X (formerly Twitter) and its AI chatbot Grok. Ministers are also considering a ban on social media use for under-16s.
The regulatory landscape is evolving rapidly. In the European Union, the Digital Services Act requires platforms to assess and mitigate systemic risks, including the spread of child sexual abuse material. The proposed EU AI Act also includes provisions for high-risk AI systems, which would cover deepfake generators. However, enforcement remains a challenge, especially given the global nature of the internet and the speed at which AI technology advances.
The role of platforms and tech companies
Social media companies are under increasing pressure to proactively detect and remove AI-generated abuse material. Meta, Google, and TikTok have all invested in AI-driven moderation tools, but the sheer volume of content makes it difficult to catch everything. The IWF has developed a proprietary tool called Image Hash List, which uses perceptual hashing to identify known child sexual abuse images, including those generated by AI. However, new variants are constantly emerging, requiring continuous updates.
Some experts argue that the burden should not fall solely on parents. While individual precautions are important, systemic solutions are needed. These include mandatory age verification, default privacy settings for minors, and stronger penalties for non-compliance by platforms. The NCA guidance strikes a balance, emphasizing that the goal is not to scare parents into silence but to encourage informed sharing with trusted audiences. “These are not hypothetical threats, they are real,” said IWF chief executive Kerry Smith, stressing that the aim is sharing with trusted people rather than no sharing at all.
Historical context: From sharenting to synthetic abuse
The term “sharenting” first gained popularity in the early 2010s as parents began documenting their children’s lives on social media. Initial concerns centered around identity theft, with scammers using publicly available photos to create fake profiles or commit fraud. A study by Barclays estimated that sharenting could account for up to two-thirds of identity fraud cases targeting young people by 2030. But the rise of generative AI has added a new and deeply disturbing dimension.
Unlike traditional photo manipulation, modern AI can generate photorealistic images and videos that are nearly indistinguishable from authentic content. This has lowered the barrier for abusers, who no longer need technical expertise to create explicit material. In some cases, AI-generated abuse is used to blackmail children or parents, a practice known as “sextortion.” The NCA has reported a surge in sextortion cases involving AI-generated images.
The technology behind these tools continues to improve. Latent diffusion models, introduced in 2022, allow for high-resolution image generation from text prompts. Video generation models, such as Sora and others, are still nascent but advancing rapidly. Experts warn that within a few years, AI-generated abuse videos may become indistinguishable from real footage, making detection and prosecution even more difficult.
Practical steps for parents
The joint guidance offers several actionable steps for parents. First, review the privacy settings on all social media accounts where children’s images are posted. Ensure that posts are set to “friends only” or “close friends” rather than public. Second, audit old posts for any identifying information such as school names, locations, or uniforms. Consider removing or editing such posts. Third, revisit consent given to schools, sports clubs, and other organizations that photograph children. Ask how these images are stored, who has access, and whether they are shared online.
Additionally, parents should be cautious about using cloud services that automatically back up and share photos. Many such services have default public settings. Using password-protected albums or encrypted sharing platforms can reduce risk. Finally, parents should educate their children about online safety and encourage them to speak up if they feel uncomfortable about any images being shared.
The wider deepfake reckoning — from celebrities to schoolchildren — has already forced lawmakers to move faster than they planned. The NCA guidance shifts some of that urgency to the family photo album. The advice, in the end, is less about panic than about shrinking the pool of raw material that abusers can exploit. As AI technology becomes more accessible, the responsibility to protect children increasingly rests on the collective actions of parents, platforms, and policymakers.