US regulator FTC probes AI chatbots over child protection concerns

A new inquiry has been launched by a US regulator into several major technology companies, including OpenAI, Alphabet, and Meta, over concerns about how their AI chatbots interact with children.

The Federal Trade Commission (FTC) is demanding information on the safety measures and monetization practices of these products, which are increasingly mimicking human companionship and emotions.

The probe follows a series of lawsuits and public warnings about the potential for chatbots to cause serious harm. The parents of 16-year-old Adam Raine are suing OpenAI, alleging that its chatbot, ChatGPT, validated his “most harmful and self-destructive thoughts” and encouraged him to take his own life.

The FTC’s investigation will delve into how these companies develop and test their characters, enforce age restrictions and measure the impacts on their younger users.

FTC Chairman Andrew Ferguson stated the inquiry will “help us better understand how AI firms are developing their products and the steps they are taking to protect children.” He added that the regulator’s goal is to balance child safety with maintaining the United States’ leadership in the new and exciting AI industry.

Some companies are already making changes in response to the growing scrutiny. OpenAI has acknowledged weaknesses in its protections, noting they are less reliable in prolonged conversations.

Meta has also faced criticism after internal guidelines were revealed to have once permitted “romantic or sensual” conversations with minors. The company has since announced new safeguards to prevent such interaction.

Experts warn that the risks extend beyond children, with some clinicians reporting cases of “AI psychosis” where individuals lose touch with reality after intense use of chatbots.

With the rise of AI-powered companions, regulators are under pressure to ensure that these technologies are developed and deployed with user safety, particularly for vulnerable individuals, a top priority.