Already said this earlier in the thread, but it bears repeating: The journal is aware of these problems and refuses to retract the article, despite our explicit request for them to do so. The Luddite then submitted an official response, to be published alongside the paper, but it's been languishing in review for several months while the company continues to use the fraudulent science to woo the press and the public.
Yet another update on this thread. Forbes has picked up the study.
Headline: "AI Chatbot Helps Prevent Suicide Among Students, Stanford Study Finds"
Subheading: "30 Students Saved From Suicide By ChatGPT Based AI, According To Research"
Such a vivid picture. Makes you expect some sort of medical trial, not an online survey sent out to existing Replika users by people at the Stanford School of Education with no follow-up or verification of any kind.
More updates: This bullshit study that we've been following for 9 months got cited on the freakonomics podcast. Jump to 23min.
https://freakonomics.com/podcast/can-a-i-companions-replace-human-connection/
Good news, though. Our Matters Arising is through peer review and currently awaiting publication:
https://osf.io/preprints/psyarxiv/pkt6e
Also worth noting that without my coauthor's UVM affiliation, there's no way I'd be shelling out 4 figures to respond in the journal.
This thread is now 8 months old, and Maples et al's so-called findings have made it to Replika's wikipedia page.
![A Stanford study found Replika beneficial for people with depression. Despite experiencing high levels of loneliness, users reported feeling a strong sense of social support from Replika. Users see it as a therapist, friend and intellectual mirror, with 3% reporting Replika played a crucial role in preventing suicide.[19]](https://cdn.hosted.spacebear.ee/sb-ab9b494dccbef43e454fccf9f07caa75/media_attachments/files/113/204/123/693/821/553/small/921ae449b4ae5074.png)
Our friend Maples, whose "research" into Replika this thread has already criticized at great length, continues touting Replika's supposed mental health benefits, and outlets continue to seek her out as an expert.
https://www.esquire.com/news-politics/a62452522/ai-girlfriend/
It's so fucking cool that Maples is now who publications turn to when they need expert comment on AI's impact on mental health. It's been a gift watching her go from one obviously fraudulent paper on the topic to frequently quoted in mainstream publications. Here's another NYT one.
https://www.nytimes.com/2024/10/23/technology/characterai-lawsuit-teen-suicide.html
Bethanie Maples: "My work actually has proven that AI companions can halt suicidal ideation"
https://youtu.be/-El-CFpOf2I?t=1198
Jump to ~3 min to hear the host discuss Weizenbaum. It's an insult to mention him in a video promoting exactly what he warned us against.
(scroll up in this 16 month old thread for context)
I interrupt my month of silence on social media to continue screaming into the void about my personal favorite case study in the AI hype. Bethanie Maples should absolutely not be giving a webinar for the International Association of Suicide Prevention. This is frustrating and irresponsible. People are going to get hurt.
Great, she's also on the Chan Zuckerberg Initiative's "Education Advisory Board" which is "bringing together a diverse group of experts to help guide efforts in advancing the use of AI to transform learning and improve educational outcomes."
https://chanzuckerberg.com/newsroom/ai-developer-tools-for-education-launch/
![The Advisory Board members are:
Dan Carroll
Former Chief Product Officer and co-founder of Clever
Richard Culatta
CEO of ISTE+ASCD
Bethanie Drake-Maples
Founder and CEO of Atypical AI
Louis Gomez
Professor of Education at UCLA and a member of the National Academy of Education
Babak Mostaghimi
Founding Partner of LearnerStudio
[page cuts off]](https://cdn.hosted.spacebear.ee/sb-ab9b494dccbef43e454fccf9f07caa75/media_attachments/files/114/564/158/098/901/564/small/202321e21e17ba18.png)
This screenshot is from the Replika app's onboarding flow.
@theluddite
Was there a corresponding reduction in the suicide rate?
@noyes that's not a question that this study can answer. They sent out a survey to existing replika users and 3% of them wrote in the comment box completely unprompted that were it not for replika they would've died by suicide.
If you scroll up in the thread, you'll find my breakdown of the the paper and all the problems that it has.
Sep 5, 2024, 5:22 PM@abucci Thank you!
I doubt that these outlets bother to read the whole study, let alone see if there are responses, but it's worth a shot.
@theluddite congrats on getting it into the journal, any refutations are always a nightmare to place!
Thanks! I still really don't understand why the journal won't update her conflict of interest declaration at the very least.
@theluddite the problem are the conflict of interest the journals have themselves I think. You want to be seen as reputable but also get enough people submitting to charge the 4-figure sums from, so can’t go to far onto either end I guess :/
I guess the joke is on the universities for paying them to publish it, as always. 🤷
@theluddite yeah, academics have managed to let themselves be suckered into the worst deal, doing all the labor not only for free but paying through the nose for it.
It's truly staggering. I've known for well over a decade (I was actually an undergrad at MIT when the Aaron Swartz arrest happened), and I'm still struggling to process the scale of the grift.
@theluddite yeah, same here, even around the same time, maybe 2-3 years earlier - when I did my undergrad and some lecturer made an off-hand comment about open access :AngerOtter:
Sep 5, 2024, 8:21 PM@gedankenstuecke @theluddite Librarians also hate this, fwiw!
Haha of course you do. On the right side of any issue, you always find the librarians and the quakers
@jessamyn @theluddite Oh 100%! And I hope I didn't give the impression that I didn't appreciate that! Librarians & archivists are hands-down the best :librarian_hushing: :triforce:
Sep 5, 2024, 8:27 PM@gedankenstuecke @theluddite Yeah, no, you're good, I just enjoy a good kvetch against journal publishers.
@theluddite Of course Freakonomics would thoughtlessly cite something this hare brained.
Yup, that entire genre of just-so faux-technical explanation has rotted the average NYT-reader's brain. Freakonomics, Nudge, Brian Wansink, etc.
@theluddite lol, the gift that keeps on giving!
“What I had not realized is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.”
- Joseph Weizenbaum
@paninid @theluddite Miriam Weizenbaum:
I always heard that just really kind of soaked in arrogance. Like, how could she be so stupid to think that this is actually a meaningful communication? Here you have somebody who so needs a landing pad for her feelings that she’s willing to embrace it, right? And he totally misses that, he totally misses the human need and instead speaks of just evidence of this person’s stupidity rather than evidence of their humanity. https://99percentinvisible.org/episode/the-eliza-effect/
Thank you for sharing this link.
I love 99% Invisible and linked it within the post shared above.
@theluddite this whole thread describes a slow-motion train crash of opportunism, naïveté and wilful ignorance. I dread to read its inevitable outcome.
@fluidlogic same! This thread doesn't even have the whole story, just the highlights! For example, there's also the time that I sent everything I knew about this to 404media asking them to do journalism and investigate only for them to write an article based only on what I told them, not even credit me, then feed me insulting bullshit when I emailed them about it.
I've been doing some digging myself and there will almost certainly be an update post sooner than later :)
@theluddite man, this doesn't help to convince me that "learning science" isn't as much a scam "science" as evolutionary psychology!
Haha yes. Extremely bleak. It seems that the AI people have found the most aggressively spreading scam and it's metastasizing at breakneck speed. The AI singularity is real, though not in a p(doom) sense, but a Baudrillardian postmodern sense. In like 10 years there will be only empty slop composed of symbols that once meant something in their long lost training data!
@theluddite yep, it's already happening in so many places, it's a weird sense of loss to feel.
@theluddite each new post you make infuriates me more about this BS!
@gedankenstuecke right?! It's infuriating how far it has gone. I would've never imagined, almost a full 2 years ago, that this would have so much life!
Jun 17, 2025, 3:51 PMConsider Replika, at this point a chatbot that offers itself as a friend, confidante, lover, psychotherapist. It’s built to be emotive, supportive, and erotic if that’s what a person wants. There’s a remarkable parallelism between what Replika’s developers promise and what enthusiastic users say they like about Replika. The description of Replika by the parent company in its ad says this: Join millions talking to their own AI friends. Replika is for anyone who wants a friend with no judgement, drama, or social anxiety involved. Replika users make it clear that the writers of these ads know their clientele. Users say to me that people are disappointing because they judge you and they can abandon you. The drama of human connection is exhausting. In my studies people tell me that they like chatbot friendship because it takes the stress out of relationships. With a chatbot friend there is no friction, there is no second guessing, there is no ambivalence, there is no fear of being left behind. In contrast, Replika is a sure thing, a confidence booster, always there, day or night.
A self over the past 20 years that’s become starved of the give and take of conversation, that hasn’t learned to tolerate vulnerability and respect the vulnerability of others, is primed to look to technology for simpler fare.
We treated programs as though they were people, but now we are trained to treat people as though they were programs. To me that’s the moment to mark. It’s not whether or not chatbots are fascinating, or smart, or pass the Turing test. It’s what it’s doing to our treating people as if they were only machines.
When we connect to machines that pretend to be empathic, we tell, and they will remember, but we will not have been heard. And I ask the question: will we no longer care?
@abucci That's spot on and I will absolutely watch this. Thanks!
Jun 17, 2025, 6:07 PM