Signal Sources of Truth: A Multi-Platform, Multilingual Audit of Citations in AI Mental Health Information Queries

Summary

Online health information seeking is shifting from keyword search, where users consider a ranked list of links, to conversational systems that compose a single answer and curate its citations, moving source evaluation from user to platform in ways that remain poorly characterized. The researchers audited three free consumer products (ChatGPT, Perplexity, Google AI Overview) on twenty English mental health questions under two prompt conditions, with a subset of three questions also translated into six further languages of varying resource tiers. They recorded 15,942 citations across 1,140 responses and 1,713 unique domains, then classified every citation with a nine-category organizational typology applied by a deterministic classifier validated against human coding. Citations were heavily concentrated -- the ten most-cited domains accounted for 43.6% of English citations, and government, commercial health, and academic sources were closely matched at roughly 22% each. Platforms differed little in typical citation volume but sharply in consistency and in the source types they favored, explicitly requesting sources shifted composition only modestly, and non-English queries surfaced fewer citations and were routed to language-appropriate resources at significantly lower rates. The researchers release the typology, classifier, and annotated corpus as reusable instruments for auditing generative health search.