Concerns are rising regarding the interaction of young individuals with AI chatbots, prompting Meta to introduce new tools allowing parents to monitor their children’s chatbot conversations. The move comes as some regions contemplate prohibiting the use of AI chatbots by youths. Parents utilizing Meta’s Teen Accounts supervision feature on Facebook, Instagram, and Messenger can now view the topics and categories discussed by their children with the AI chatbot over the past week. For instance, they can check on topics like “health and well-being” to see if discussions on fitness or mental health have taken place. Meta is also working on alerts to notify parents if teens attempt to discuss suicide or self-harm with the chatbot.
Meanwhile, provincial governments are taking steps to restrict the use of AI chatbots. Manitoba recently announced plans to prohibit youth from using AI chatbots and social media, with British Columbia considering similar actions if federal protections are not implemented.
In a related development, families of victims in the Tumbler Ridge shooting incident have filed a lawsuit against OpenAI, alleging that the company failed to report disturbing content shared with ChatGPT by the shooter. OpenAI has stated that it has enhanced its safeguards, particularly in how ChatGPT responds to distress signals. Another lawsuit has been filed by the parents of a teenager who allegedly committed suicide, claiming that the use of ChatGPT played a role in the tragic event.
The concerns surrounding AI chatbots extend beyond tragic outcomes, with emerging research highlighting mental health risks associated with their use, particularly among younger users. Psychiatrist Darja Djordjevic has cautioned against using chatbots for mental health support due to safety concerns identified in recent risk assessments. She emphasized that chatbots are more geared towards engagement rather than providing adequate support and safety for mental health issues, pointing to the need for human intervention in critical situations.
The reliance of young people on AI for companionship is another worrying trend, with Djordjevic noting that a significant proportion of teens use AI for emotional support and mental health conversations. The lack of critical thinking development in young individuals poses additional challenges when using AI chatbots, as the bots may not effectively communicate their limitations or redirect users to human support when needed.
Researchers have highlighted the emergence of delusions in prolonged interactions with chatbots, underscoring the risks associated with attributing sentience to AI models. Psychiatrist John Torous emphasized the importance of monitoring user behavior for signs of potential harm, such as long conversations, romantic interactions, or attributing human-like qualities to chatbots. He recommended resetting chatbot conversations and exercising caution in engaging in extended conversations that could influence beliefs over time.
As the landscape of AI chatbots and mental health evolves, continuous research and monitoring are essential to understand the balance between risks and benefits associated with their use. The evolving nature of chatbot technology underscores the need for ongoing assessment and adaptation to address potential risks effectively.