Meta adds parent alerts for teen self-harm chats with Meta AI

Meta is adding a new safety alert for families: parents can be notified when a supervised teen talks with Meta AI about suicide or self-harm. For investors watching Meta’s push into artificial intelligence, the update shows how child safety and liability concerns are becoming part of the cost of rolling AI features into social apps.

The company said Thursday that it built a dedicated AI system to detect conversations in which a teen clearly refers to harming themselves. Meta AI is the company’s chatbot, meaning software that generates conversational responses to users inside Meta’s products.

The alerts are available now for parents who use Instagram Parental Supervision in the U.S., U.K., Australia and Canada, according to Meta. The company said it plans to make the feature available worldwide by the end of the year.

How the alert system works

Meta said conversations flagged by its detection system will be reviewed by a person before a parent receives an alert. In a blog post, the company said it knows these notifications can be upsetting for parents and that human review is part of its early approach while it improves the system.

Meta also said it will send alerts in cases where a teen’s intent is unclear, choosing caution even if that leads to some warnings where there may not be an actual crisis. That means the system may generate false alarms, but Meta described that as the starting point it wants to use while monitoring performance.

The new chatbot alerts expand Meta’s existing safety tools for teens on Instagram. The company already notifies parents when a teen repeatedly searches for suicide or self-harm terms on Instagram, and it has a separate feature that lets parents see the topics their teen discussed with Meta AI during the previous week.

Emergency services and tighter AI limits

Meta also said it will contact emergency services when a conversation with Meta AI suggests that someone may be at risk of suicide. That policy applies to adults and teens, according to the company. Meta already takes that kind of step when posts on Facebook or Instagram indicate someone may be at risk, and the company is extending the practice to chatbot conversations.

The company also said its Limited Content setting now covers Meta AI. Limited Content is a parental control that gives teens a more restricted Instagram experience.

Meta said Meta AI is already trained to avoid sexual or romantic conversations with teens, as well as discussions involving alcohol. With Limited Content turned on, the chatbot will refuse a broader set of prompts. Meta did not specify which additional prompts are covered.

The rollout comes as Meta and other technology companies face questions from regulators and parents about how AI chatbots respond to people in crisis, especially teenagers. For Meta, which is trying to make AI a regular part of Instagram, Facebook and its other apps, the update is another sign that safety systems are now tied closely to how consumer AI products are built, marketed and defended.

This story draws on original reporting from TechCrunch.