What matters in AI.

Subscribe

OpenAI releases a tool to find AI child abuse images

OpenAI tells that its classifier flags 48 percent fewer images, and the flagged images have two times the rate of child abuse material.

On Monday, OpenAI told the public of a classifier, a software tool that sorts images into groups. The tool finds child abuse images. It also finds images made by AI. The US tip line for child abuse reports got 400,000 reports about generative AI in 2025, up from 67,000 in 2024. Researchers tell that the work can become too much to manage.

How it works

OpenAI tells that the classifier finds child abuse images from many tools, such as open source models and nudifying apps. OpenAI tells that it is the first classifier with one that is available to many.

  • Selected organizations get free access.
  • Developers get guidance on how to find this content.
  • Police get training on how AI can help their work.

A ChatGPT user who tries to make such images gets a 24 hour stop and a warning.

The numbers

The numbers are 48 percent fewer images flagged, and two times the rate of child abuse material in the flagged images. OpenAI gave these numbers. No external test and no test method are given.

What is not known

North Carolina Attorney General Jeff Jackson works with OpenAI on this. Jackson tells that the work is still an experiment. It is not known if the tool will cut the work of the tip line.

Insight: OpenAI works with the officials who can sue it

On 15 October, Iowa and seven other states started lawsuits against OpenAI about injury to children. At this time a state attorney general works with OpenAI on a child safety tool. This reads as a move by OpenAI to make the officials who can sue it into partners. Jackson tells that most of the work of attorneys general starts after the injury. Monitor the steps of the new Congress on AI firms.

Sources

  1. OpenAI wants to stop deepfakes from overwhelming child abuse tiplinestheverge.com
AI MATTER · NEWS · AI MATTER · NEWS ·16 NOV2026

Posted

Tags

Learn the terms in this story

More in Security

All Security news