So-called ‘sexts’ typed out by children in the UK have risen 183% during lockdown compared to before measures were imposed, with a 55% rise in sexts drafted during normal school hours, SafeToNet said.
The UK start-up has analysed and filtered around 70 million potentially harmful messages sent by children using their safeguarding app and found girls are sending the majority of concerning messages.
Eleven-year-old girls and 13-year-old boys attempted to send the highest proportion of sexts and messages identified as cyber-bullying, but the app stopped messages being sent from girls as young as six and boys aged nine.
Detected cyber-bullying messages fell steeply in the UK as lockdown was imposed, but as some schools have started admitting pupils again rates on Sunday evenings have started to rise, suggesting fear of the week ahead.
The number of aggressive messages detected and stopped has risen 47% since lockdown measures started to be lifted, the data shows.
Founder Richard Pursey believes it is the only app of its kind to track threats to children in real time and said safeguarding software like theirs should be on every child’s phone.
Developed in consultation with more than 2,000 children, it uses artificial intelligence and behavioural analytics to balance the child’s right to privacy with the “primal need of a parent to keep their child safe”.
The company, via the SafeToNet Foundation, is offering UK parents a million free licences which can be downloaded via a link on the Government’s online safety advice page from Thursday.
Mr Pursey, a 58-year-old father of four from Kensington, west London, told the PA news agency children are killing themselves daily due to online activity and he has been left “scarred” by viewing some of the material.
He said: “This is happening pretty much every day around the world, and I know it’s happening.
“And I don’t know these children, but I know today, some children will take their lives.
“And that’s because of things like bullying and sexting, not necessarily because of grooming, it’s just kids being nasty to kids.”
He added: “I know that well, because I can see from the statistics on my platform, the significant amount of harmful messages that we’re filtering, so we’re hopefully stopping it from happening, but of course, not every child on the planet is using our platform.”
Mr Pursey said he has seen a video of an 11-year-old girl killing herself and of a man blowing his head off with an automatic rifle circulated by UK children.
He said: “When I press the button to look at this content because I have to look at it to check my software and filter it, I’m hanging on, I’m bracing myself ready to see what I see. And I’m scarred as a result of it.
“Now you imagine a nine year old boy gets sent a link, which other boys send them as a joke – ‘Look at this, you’ll find it very funny’. It’s not funny – it’s a trick for them to see a guy have his head chopped off.”
Interventions by parents to manage screen time by taking devices off kids are “flawed” and could be causing further harm, as a child being bullied will simply receive a “tsunami of abuse” when their phone is returned, he added.
The app’s AI technology detects linguistic patterns on the device but stores no data on cloud platforms, meaning nobody ever sees what the child is typing or what they are seeing, while the user is kept anonymous.
Installed by the child, it replaces the phone’s native keyboard with SafeToNet’s safeguarding keyboard which flashes amber when a threat is detected in the their typing, educating them in the moment to help stop harmful outgoing messages
The keyboard will turn red and stop children from sending potentially dangerous messages if the risky behaviour persists or increases.
Parents are also notified that their child may be at risk, but do not see the content of any messages, encouraging dialogue with their child.
Mr Pursey added: “It’s not spying on the child and that’s crucial…it’s real time, in the moment guidance through the keyboard.”
If the software uncovers a potentially serious safeguarding concern, parents will receive a “high risk” notification.
The app also directs children who may be feeling anxious or upset to wellbeing resources to help them pause before sending something rash, and has an emotions diary.
While it currently only analyses keyboard strokes, the company will be rolling out video and image screening later this year.