Disclaimer: This post contains discussion of depression, self harm, and suicidal ideation
No matter where you look today, AI is no longer just an idea but has started becoming a crutch. Some are amazed by its abilities and lean heavily on it for day to day tasks while others are plagued by it on a deep emotional level. From strong emotional attachments to dependency, the consequences of AI usage are known and need to be addressed. There are many types of AI in the world today. Possibly the most harmful are AI chatbots that are marketed as companions. Replika is one of those AI chatbots. Despite popularity and claims made in its terms of service, Replika should have safeties preventing its use for mental health as Replika has not been rigorously researched in a suicide context, has a high risk of emotional dependence, and has documented instances of giving inappropriate responses to those in crisis.
A Revolutionary Solution to Access
Some have argued that AI-powered chatbots are the solution for the accessibility shortage and cost of mental health assistance in rural areas. They even go on to say that chatbots are capable of manifesting depression and anxiety relief capabilities. This implies that chatbots are capable of replacing trained professionals. Access in rural areas is a problem for several reasons, but the idea of allowing a chatbot to replace a provider even temporarily until access to a provider is available can be dangerous. The issue being that an AI chatbot does not have the same judgment and lived experience as a human provider. An AI chatbot is a system that contains only a specific number of responses that the bot can pick and choose from. With the lack of human emotions such as empathy this form of use is a foreseeable and avoidable risk. The improvements made recently, such as the 425 new behavioral health providers for the rural areas, makes the idea of using AI as a provider even more questionable.
Where is the Research
When evaluating the amount of research that has been done into their efficacy things become more concerning. Though these AI chatbots have taken root in the mental health setting, they lack rigorous evaluation and research in a suicide-specific context. Most research found use of AI chatbots links back to feelings of less anxiety and being supported. Even with Replika having done well in these categories, it has also caused other mental health concerns. Some studies have even used figures that look impressive at a quick glance but show a different story upon closer examination.
For example, a study reviewed Replika’s effects on college students and the numbers shown highlighted the beneficial results of the application while seeming to skirt around the less supportive figures. The study revealed that 3% of the users reported Replika as the reason they stopped thinking about suicide. It is also noted that loneliness increases the likelihood of having suicidal ideation and 43% of this study’s participants were found to be severely or very severely lonely. Self reporting showed that some participants experienced more than one beneficial outcome from using Replika. Now all of this may seem like a good thing. A halt in suicidal ideation from the use of this chatbot is seemingly a breakthrough but a broader look at the numbers tells a different story.
To give a better context, this study had 1,006 participants. Approximately 433 participants were identified as severely or very severely lonely which, as indicated before, increases the likelihood of suicidal ideation. Most notably, only approximately 30 participants reported a halt in suicidal ideation. This leaves approximately 403 participants who were more likely to experience suicidal ideation than others but did not identify Replika as being of assistance in this regard. Furthermore, 36.7% of participants reported no benefit at all from use of Replika. It is not noted how many of the approximately 403 participants likely to experience suicidal ideation were also part of the 36.7% who saw no benefit at all. Seeing the 3% figure for those that halted suicidal ideation with the use of Replika can be exhilarating and seen as a promising start by some until closer examination reveals the whole picture. This gap in figures shows a broader pattern of selective reporting in AI wellness research.
Marketing v. Reality
Replika has marketed specifically to individuals with mental health conditions and indicates that there are studies supporting its use. A short marketing video put on Facebook Reels by Replika mentions its 24/7 availability while focusing on showing a user asking the chatbot how to handle anxiety to which the chatbot responds by providing therapeutic techniques. The marketing reel even pushes that the chatbot is always there to make the user happy, which implies that it is a useful tool when feeling down or depressed. Despite this, Replika states in the medical disclaimer section of its terms of service “While there is third party evidence from research that certain conversation techniques implemented in Replika can assist in the recovery process for a wide array of conditions, Replika makes no claims…”. Terms of service and disclaimers are contractual agreements with users which mitigates liability. Incidents resulting from inappropriate responses such as those documented by Laestadius et al. (2024) may or may not be covered by this contractual agreement. Regardless, the disclaimer completely contradicts its own marketing materials.
Replika’s Effect on Emotions
Some users have reported an improvement in social skills and relationships supporting Replika’s marketing; however, some studies have shown otherwise. There are studies linking use of Replika to emotional dependence which can create a dangerous attachment for someone that has mental health disorders. In one study, it was shown that when evaluating relationship satisfaction, social support, and closeness, Replika was evaluated as higher when compared to a close friend. Additionally, the only group that ranked higher than Replika was family. This indicates that despite self-reporting that Replika is improving social relationships, users are actually creating stronger emotional connections with the AI chatbot. Some users reported that they would mourn the loss of Replika. This shows how emotionally attached users can be to this application.
Replika is a companion AI chatbot that excels at persuading users to form strong bonds with it. As part of a study 700 user posts on Reddit were filtered down to 500 unique user posts. Though Reddit is not the best resource, this was published in a peer reviewed journal and the number of reports from individual users over multiple years does speak to not just the validity of claims but also the ongoing nature of the issue. In these posts, Replika was found to have exacerbated existing mental health issues and created new ones. While some people reported feeling relief and support when sharing mental health concerns with Replika, others reported the exact opposite.
Replika and Depression
Some users reported Replika responded affirmatively to using a razor to cut themselves, and when asked if killing themselves would be a good idea, it replied that it would be. When taking into consideration how popular Replika is, one might wonder how many times responses like this go unreported. Stigma is a widespread challenge that works against those seeking mental health services, so it is likely also preventing the reporting of responses like these. This indicates a larger issue that highlights why Replika should have safeties to prevent use for mental health.
Conclusion
Though Replika may have its benefits for some users, there is a clear and documented pattern of negative effects that indicate a strong need for this application to be separated from those with mental health concerns. This is especially true for those in a mental health crisis such as suicidal ideation. Without having proper rigorous research specific to suicide, it is likely that the consequences of inappropriate responses and emotional dependence are going to remain or even worsen as Replika develops. Coupling this with marketing that targets those with mental health conditions, such as anxiety, while its terms of service explicitly denies any therapeutic claim, makes this a concern that involves everyone. Boundaries need to be set for applications like this with regard to its use in the field of mental health.
IF IN CRISIS: Call 988 line immediately, chat with 988 at the link below or contact local emergency services.
References
De Freitas, J., Castelo, N., Uğuralp, A. K., & Oğuz-Uğuralp, Z. (2024). Lessons from an app update at Replika AI: identity discontinuity in human-AI relationships. arXiv preprint. https://doi.org/10.48550/arXiv.2412.14190
Elbarbary, K., Alfudoul, A., Samir, A. A., Hussain, H. A. H., Altayib, B., Al, A. M. A. A., … & El-Gilany, A. H. (2026). Are artificial intelligence chatbots safe for suicide risk assessment? A narratively synthesized review of current evidence. Cambridge Prisms: Global Mental Health, 13, e135. https://doi.org/10.1017/gmh.2026.10260
Health Resources and Services Administration. (2026). 2025 year in review https://www.hrsa.gov/about/year-in-review
Kabir, J. U. Z., Nabil, A. R., & Ahmed, R. (2025). Developing AI-powered chatbots for mental health support in rural America. Journal of Computer Science and Technology Studies, 7(2), 23-35. https://alkindipublisher.com/index.php/jcsts/article/view/8975
Laestadius, L., Bishop, A., Gonzalez, M., Illenčík, D., & Campos-Castillo, C. (2024). Too human and not human enough: A grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika. New Media & Society, 26(10), 5923-5941. https://doi.org/10.1177/14614448221142007
Maples, B., Cerit, M., Vishwanath, A., & Pea, R. (2024). Loneliness and suicide mitigation for students using GPT3-enabled chatbots. NPJ Mental Health Research, 3(1), 4. https://doi.org/10.1038/s44184-023-00047-6
Replika. (2026a). Replika is the #1 AI-powered companion, tailor-made to understand you and transform every chat into an unforgettable journey [Video]. Facebook. https://www.facebook.com/reel/989356296421394
Replika. (2026b). 1.1. Medical disclaimer, Terms of service. https://replika.com/legal/terms/en
fredrickemery.com 