Protecting Children from Online Exploitation: Can a Trained Model Detect Harmful Communication Strategies?

The growing popularity of social media raises concerns about children's online safety. Of particular concern are interactions between minors and adults with predatory intentions. Unfortunately, previous research on online sexual grooming has relied on time-intensive manual annotation by domain experts, limiting both the scale and scope of possible interventions. This work explores the possibility of detecting predatory behaviours with accuracy comparable to expert annotators using machine learning (ML). Using a dataset of 6771 chat messages sent by child sex offenders, labelled by two of the authors who are forensic psychology experts, we study how well can deep learning algorithms identify eleven known predatory behaviours. We find that the best-performing ML models are consistent but not on par with expert annotation. We therefore consider a system where an expert annotator validates the ML algorithms outputs. The combination of human decision-making and computer efficiency yields precision - but not recall - comparable to manual annotation, while taking only a fraction of the time needed by a human annotator. Our findings underscore the promise of ML as a tool for assisting researchers in this area, but also highlight the current limitations in reliably detecting online sexual exploitation using ML.