Signal Towards Detecting AI-Assisted Responses in Online Surveys

Summary

The use of large language models to complete online surveys threatens the validity of survey-based research, but methods for detecting such use remain underexplored. The authors introduce ASURRE, an initial benchmark dataset for AI-assisted survey participation that captures a range of usage strategies, from full generation and revision to persona-grounded agentic completion. LLM-assisted survey responses were generated using multiple LLMs on three real-world surveys spanning different disciplines, paired with genuine human responses for comparison. The team evaluated existing machine-generated text detectors against this benchmark to assess their performance. The paper focuses on establishing the evaluation benchmark rather than proposing a finished detection tool.