Signal Preprint presents Avalon-ToM-Bench for fine-grained AI theory-of-mind evaluation

Summary

A new study proposes Avalon-ToM-Bench, a benchmark for evaluating theory-of-mind ability in AI. Theory of mind is essential for agent interaction, yet existing evaluations have relied either on static scenarios that oversimplify mental-state reasoning or on interactive setups that offer limited diagnostic insight. The benchmark operationalizes theory of mind through the asymmetric-information mechanics of the social deduction game The Resistance: Avalon. Rather than scoring whole gameplay sessions, it breaks theory of mind down into a two-by-two taxonomy separating epistemic reasoning from motivational reasoning. This finer breakdown helps diagnose more precisely how well an AI reasons about others' beliefs and intentions.