Signal Do LLMs Know Your Neighborhood? Auditing LLM Priors for Neighborhood-Level Mobility Prediction and Structural Alignment

Summary

Fine-grained human mobility trajectory data are central to urban planning, transportation, public health, and emergency response, yet they are often proprietary, restricted, and privacy-sensitive, and large language models (LLMs) offer a potential alternative whose ability to infer aggregate neighborhood-level mobility remains unclear. The researchers evaluated zero-shot LLMs on Census Block Group-level mobility prediction across four U.S. metropolitan areas, using anonymized Cuebiq data to construct point-level, trajectory-level, and temporal mobility outcomes paired with sociodemographic and built-environment predictors. They compared LLM predictions with supervised baselines and introduced a directional alignment analysis testing whether LLM-implied predictor effects agree with empirical OLS and Jonckheere-Terpstra trends. Supervised models achieved 0.580 average accuracy compared with 0.435 for the best LLM, with spatial extent outcomes showing the strongest predictability but also the largest LLM-baseline gaps. Directional analysis showed that LLMs often rely on coarse, stable predictor-level priors that remain similar across outcomes and cities, including asymmetric treatment of protected-group predictors. The authors state that LLMs can partially recover aggregate mobility patterns from urban context, but their predictions should not be treated as structurally grounded without auditing empirical alignment and potential bias.