Recent research has highlighted a significant risk associated with the use of large language models (LLMs): the inadvertent induction of false memories in users. A study published in August 2024 examined the impact of AI-powered conversational agents on witness interviews. Participants who interacted with a generative chatbot were found to develop false memories at a rate over three times higher than those who engaged with a control or survey-based method. This suggests that LLMs, when used in sensitive scenarios like police interviews, can unintentionally shape and distort recollections, potentially leading to wrongful convictions or the undermining of justice. arxiv.org
The mechanism behind this phenomenon aligns with the misinformation effect, where post-event information can alter an individual's memory of the original event. LLMs, by providing suggestive or leading information, can inadvertently incorporate inaccuracies into a user's memory. This is particularly concerning given the widespread integration of AI in various sectors, including law enforcement and healthcare. The potential for LLMs to implant false memories underscores the necessity for ethical guidelines and rigorous oversight in their deployment, ensuring that these technologies do not compromise the accuracy and reliability of human recollections. arxiv.org