There is growing concern over the misuse of AI to create and circulate Child Sexual Abuse Material (IANS)
New Delhi: The Centre has strengthened India’s online child safety framework by tightening intermediary obligations under the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021, amid growing concerns over the misuse of artificial intelligence to create and circulate Child Sexual Abuse Material (CSAM).
The move comes after the government sought a detailed report from a social media intermediary over alleged advertisements linked to CSAM, while the National Commission for Protection of Child Rights (NCPCR) also issued notices to social media platforms.
To understand how such crimes can unfold, let’s look into a recent case. A 14-year-old Class IX student, “Riya” (name changed), was contacted on social media by a person posing as a fellow student. After gaining her trust over several weeks, the stranger persuaded her to share personal photographs.
The images were then allegedly manipulated using AI tools to create explicit fake content and circulated through anonymous groups, messaging platforms and links directing users to websites hosting CSAM. Despite the content being reported, copies continued to surface through mirror links and private groups, causing severe emotional distress and online bullying. The case study is fictional and intended only to illustrate the type of online risks children can face.
According to the government, the amended IT Rules require intermediaries to deploy appropriate technical measures to detect and prevent AI-generated harmful content, including CSAM, non-consensual intimate imagery and deepfakes. Social media platforms have also been directed to remove unlawful content within three hours of a valid government or court order, while complaints relating to nudity, morphed images and impersonation must be addressed within two hours. Significant social media intermediaries are also required to proactively identify previously removed child sexual abuse material and comply with mandatory reporting requirements wherever applicable.
Cyber Crime Investigator and Cyber Security Expert Rajesh Gupta said the rapid advancement of generative AI has made the CSAM challenge more complex. “AI can create highly realistic synthetic sexual content that is difficult to detect and can be misused for exploitation, harassment and grooming. Social media platforms should adopt a zero-tolerance approach towards CSAM and AI-generated sexual content. The government’s stricter IT Rules and faster takedown timelines are a step in the right direction, but proactive AI-based detection, digital fingerprinting and stronger legal provisions are essential to protect children online,” he said.
AI Governance expert Ajay Sharma said the latest amendments mark a significant step in strengthening platform accountability by bringing AI-generated or synthetically generated information under the regulatory framework. He said the revised rules require automated detection measures, shorter compliance timelines and stronger due diligence by intermediaries.
Sharma added that proactive detection, improved advertisement screening and closer coordination between platforms and law enforcement would be important to prevent harmful content from reaching users.
Experts said the Centre’s stronger regulatory framework, backed by proactive AI detection, faster takedowns and greater platform accountability, can play a key role in curbing AI-generated CSAM and strengthening online safety for children.
Also Read:
- Child Exploitation A Horrific Crime, We Fight Such Abuse On, Off Our Platforms: Meta Blog After Govt’s Notice
- India May Look At Dedicated AI Regulatory Framework, Says MeitY Secretary
- Tighter Norms: MeitY Proposes Continuous Label Visibility On Full Duration Of AI-Generated Content
- Draft IT Rules Amendments Not Linked To Recent Takedowns; IT Secretary Says Changes Do Not Expand Powers
————————————————
