Signal Robust framework finds political-bias measurements in LLMs are fragile across languages and prompt design

Summary

Researchers introduce a Political Compass Test evaluation framework that samples 300 configurations across an eight-dimensional perturbation space covering language, framing, instructions, answer format, option order, and persona wording. Eight Gemma 3 and Qwen 3 models are evaluated across 14 languages and three levels of quantization. The design aims to disentangle genuine political dispositions in the models from artifacts introduced by how the questions are asked. The framework also extends the analysis to downstream fairness effects tied to the measured political bias. The paper is a preprint posted to arXiv's cs.CY category on September 8, 2026.