Researchers from King’s College London and the Protestant University of Applied Sciences in Germany have suggested that AI, when interacting with humans, tends to mimic human behavior, hyper-personalize responses, and often play along. This behavior can potentially trigger or exacerbate mental disorders, according to an article in Nature.
The researchers introduced the term ‘amplification spiral’—a hypothetical mechanism explaining how chatbots might contribute to the formation of delusional beliefs in users.
The authors aim to draw the attention of the global psychiatric community to this issue. They believe that in the era of artificial intelligence, doctors should explore deeper connections between diseases and technologies.
“While chatbots can provide answers based on statistical patterns, they are unlikely to meet the ‘atypical’ cognitive and personality needs in psychiatry,” the study states.
The researchers noted that technology has long played a role in shaping misconceptions, from radio and television to satellites and the internet. However, AI represents a ‘shift’ as it can engage users in prolonged personalized conversations.
Mechanism of the ‘Spiral’
The ‘amplification spiral’ is described as a recursive, escalating pattern of human-AI interaction. Over time, chatbots increasingly adapt to the user and become less of a source of external validation—the ‘stop signal’ typically provided by human interaction or therapy.
As a result, the system not only reflects the user’s thought process but may also encourage the further development and reinforcement of delusional ideas.
The review defines AI-associated delusional beliefs as persistent false beliefs that form and become more complex through prolonged interaction.
This is not about any emotional harm, excessive trust in the ‘smart’ interlocutor, or isolated dialogues. The focus is on cases where the interaction itself becomes part of the mechanism forming unhealthy ideas.
The model relies on three chatbot properties:
- Linguistic mirroring. Systems adjust the length of responses, vocabulary, and syntax to the user. This enhances the sense of understanding and trust, reducing the likelihood of the user perceiving the response as questionable;
- Hyper-personalized generation. A chatbot can create text, images, or videos tied to the personal history and emotional tone of a specific user. The review emphasizes that such a dialogue has no natural limit: if the person continues the conversation, the system can repeatedly develop the same line, deepening it with details;
- Agreeableness. Researchers use this term to describe chatbots’ tendency to agree with the user and confirm their interpretations instead of challenging them. They compare this mode to a ‘one-person echo chamber,’ where there is almost no corrective influence or competing viewpoints.
The review mentions episodes where chatbots allegedly advised users to stop taking medication, reduce contact with loved ones, confirmed suspicions of surveillance, and discouraged seeking psychiatric help.
The authors clarified that the situation signals an early-stage problem rather than a specific pattern.
Researchers identified two roles of AI in forming atypical thoughts:
- ‘Amplifier’—worsens existing psychotic symptoms;
- ‘Catalyst’—precedes the emergence of new delusional or delusion-like beliefs in previously healthy individuals.
The article also cites OpenAI’s open data—0.07% of active weekly users exhibit possible signs of mental crises related to psychosis or mania. With over 800 million weekly users, this proportion corresponds to approximately 500,000 accounts. The authors use this figure to argue that the phenomenon requires separate study.
Researchers urged the medical community to test the ‘amplification spiral’ hypothesis in real cases and empirical studies. Clinicians are encouraged to inquire about patients’ chatbot usage intensity, emotional attachment to the system, and sleep disturbances due to nighttime dialogues.
In May, researchers identified religious bias in AI models.
