Signal Preprint tests whether LLMs can validly judge social attraction

Summary

A preprint examines whether large language models can validly serve as judges of social attraction. LLMs are increasingly being used to carry out subjective evaluations that were traditionally performed by humans. The researchers built persona profiles from ten constructs drawn from psychological and relational theory, sorting them into three tiers, socially attractive, mixed and unattractive. Across two studies, they compared the ratings produced by LLMs against judgments made by humans. The comparison was used to assess whether LLMs can reliably reproduce this kind of subjective social evaluation.