Across Instagram and other social media platforms, everyone has recently been busy engaging in the 80s trend, where users post AI-generated images of themselves in an 80s look and setting. However, beyond the environmental impacts of generative AI, what seems like a harmless trend has troubling implications, especially when it comes to feeding generative AI children’s pictures.
Meta’s announcement addresses CSAM only after it is uploaded to its social media platforms. Which brings us to the question of: what happens before Meta identifies and reports the CSAM content on its platforms?
Over the past few months, Meta has faced scrutiny in India over the presence of child sexual exploitation and abuse material (CSEAM/CSAM) on its platforms. A BBC Eye investigation found that advertisements were reportedly used to promote such material and direct users to Telegram channels where CSAM was allegedly being sold. Sometimes Meta AI suggestions on videos featuring children redirected users to adult websites.
On September 15, 2026, Meta agreed to report child safety-related matters directly to India’s cybercrime portal, managed by the Indian Cybercrime Coordination Centre (I4C). The company said the new arrangement would strengthen its coordination with Indian authorities in tackling online child exploitation. However, Meta’s announcement addresses CSAM only after it is uploaded to its social media platforms. Which brings us to the question of: what happens before Meta identifies and reports the CSAM content on its platforms?
The extent of CSAM online
In 2025, the Internet Watch Foundation (IWF) assessed 8,029 images and videos depicting realistic child sexual abuse. Of those, 3,443 videos were found to be AI-generated, as opposed to 2024, when IWF had identified only 13 such AI-generated videos. The year 2025 saw a 264-fold increase in AI-generated CSAM. And this data represents only the material IWF encountered and assessed; the scale of such AI-generated CSAM around the world could be much larger.
IWF found that 65 per cent of the AI-generated videos it assessed were classified as Category A — depicting penetrative sexual assault, sadism, or bestiality — the most serious category under the UK’s CSAM classification system. And 97 per cent of the AI-generated images it assessed depicted girls.
The year 2025 saw a 264-fold increase in AI-generated CSAM. And this data represents only the material IWF encountered and assessed; the scale of such AI-generated CSAM around the world could be much larger.
A 2025 report by the National Center for Missing & Exploited Children (NCMEC), a US non-profit, throws further light on the issue. In 2025, the organisation received more than 400,000 CyberTipline reports with a genAI nexus. More than 182,000 reports involved offenders possessing AI-generated CSAM or generating or attempting to generate CSAM with AI.
Since NCMEC began tracking AI-generated video content in 2023, it says more than 275 victims of AI-generated CSAM have been identified, and more than 158,000 images and videos have been categorised as AI-generated CSAM. While the IWF and NCMEC data are not directly comparable because their samples vary and the studies are conducted in different countries, the data, when read together, points to the growing child-safety concerns involving generative AI.
Social media is the last stop
Videos depicting what look like very young girls in sexualised situations with adults are often found circulating on Instagram. The videos appear to be AI-generated, with some even carrying Meta’s ‘AI Content’ label. And these videos are publicly accessible to a global audience.
In light of recent developments, some of these AI-generated CSAM videos appear to have been taken down by the platform. Because these videos appeared on Instagram, Meta is responsible for detecting, removing, and reporting them. But platform accountability does not begin and end with Meta’s platforms alone. Someone, somewhere, had to generate or modify the material using a different platform before it could be uploaded to a Meta platform.
This chain can begin much earlier. For instance, a photograph of a child, uploaded to an AI system without any malicious intent by the child’s own parents, might be used for this purpose. India’s Digital Personal Data Protection (DPDP) framework contains separate provisions under Section 9 focusing on the safety of children online. However, these provisions apply only when the child actually exists. Once a photograph containing biometric facial data enters the AI ecosystem, it can serve as a dataset for fine-tuning the AI model’s outputs. The concern, therefore, should encompass both scenarios: what happens to the original biometric data, and what the technology can generate from the child’s likeness.
An AI model can generate an entirely synthetic image of a child, and another model can modify it and turn it into a video. By the time the final file is uploaded to a social media platform, only the final output is visible, and the systems used to produce it may not be traceable.
The BBC Eye Investigation exposed how an account that had not initially searched for such material could be led towards increasingly explicit content, including CSAM, through Instagram’s recommendation system. Meta said it had removed the accounts and content identified by the BBC, but the issue did not end there. It becomes even murkier when the content being circulated is AI-generated.
An AI model can generate an entirely synthetic image of a child, and another model can modify it and turn it into a video. By the time the final file is uploaded to a social media platform, only the final output is visible, and the systems used to produce it may not be traceable.
Back to square one
Praneeth Panyam, a Hyderabad-based software engineer, explains how images and videos are generated by AI systems. He tells FII that a user’s prompt is converted into a mathematical representation that connects language with visual concepts. The image itself does not appear all at once. ‘The image starts from a pure static point,’ he says.
The same basic process can feed into another system: an image can be generated first, modified using another model, and then converted into a video by another AI system. At each stage, the output of one system can become the input for the next. The key concern for safety when it comes to AI-generated images and videos is the intention behind their creation, which is not always apparent from the final output. As Panyam puts it, ‘The intention of the user is not calculated in the image pixels.‘
Legal remedies are available once the material is out there, but greater platform accountability will ensure that AI-generated CSAM is addressed at the source, where such content is being generated at scale.
An image can therefore move from one AI platform to another and ultimately end up on social media as a video, including as AI-generated CSAM. The technical difficulty lies in reconstructing what happened before the final output appeared online. But that does not necessarily mean that this falls outside the law.
Afnan Husain, an advocate at the Bombay High Court, told FII, ‘Indian law does not make a distinction between an image captured by a camera and one generated using AI or computers when it comes to sexually explicit imagery involving a child or a figure resembling a child. In such cases, creating, saving, or sharing that image or video can attract serious criminal liability under our laws, including Section 67B of the IT Act and Section 15 of the POCSO Act.‘
Legal remedies are available once the material is out there, but greater platform accountability will ensure that AI-generated CSAM is addressed at the source, where such content is being generated at scale.
While generative AI tools often have some built-in safeguards, merely tweaking the language in the prompt can bypass the platform’s keyword-based safeguards related to sexually explicit content. Speaking to FII, Siddharth Rao, a Hyderabad-based policy researcher and journalist, says that manipulating AI systems is getting easier each day. Offering a possible solution, Rao tells FII, ‘AI models should carry an umbrella instruction to either not generate any AI-generated videos and images containing a child or a tiny human that remotely resembles a child. Or, if the output resembles a child, they should carry information showing where and when such imagery was created, alongside a clear watermark to identify the exact AI model used in generating the video.‘
While the final platform should ultimately be held accountable for preventing its further circulation, accountability should begin at the point of creation: the AI model where the content was first generated.
India’s 2026 IT Rules have already moved in this direction by mandating safeguards against prohibited synthetically generated information and, where technically feasible, requiring labelling and permanent metadata or other provenance mechanisms to identify the computer resource used to create or alter such content.
CSAM imagery and videos pass through several systems before reaching social media platforms such as Instagram. While the final platform should ultimately be held accountable for preventing its further circulation, accountability should begin at the point of creation: the AI model where the content was first generated.
Editor’s Note: Some quotes in this piece have been edited for clarity and length.
Madhuri Kankipati is an independent writer and researcher based in Khammam, Telangana. She translates between Telugu and English, with a focus on women writers. Her work focuses on policy, gender, literature, digital culture, AI, and the impact of AI in Indian publishing. Her writing has previously appeared on The Chakkar, Muse India, and Borderless Journal. She also writes about books and AI ethics.
