Signal Expert Survey Examines Why AI Safety Evaluation Sidelines Human-Subjects Research

Summary

A preprint by Jessica Y Bo, Paula Akemi Aoyagui, Shalaleh Rismani, Dipto Das and colleagues notes that the safety risks of AI are becoming increasingly visible in how humans interact with AI technologies. It argues that the dominant approaches for evaluating these risks lean toward technical methods such as model benchmarks and LLM simulations, which tends to sideline empirical research involving human subjects. To examine this gap, the authors conducted an expert survey with 93 respondents followed by expert interviews. The study investigates the epistemic fit and structural barriers that keep human subjects research underused in AI safety and ethics evaluation. It highlights this despite such research's potential to capture real world interaction risks that benchmarks may miss.