Signal Temporal Portability of Numeric User Metadata on Twitter

Summary

Numeric user metadata in social media are often reused over time, but their reusability depends on which properties an analysis needs to preserve. The authors introduce 'temporal portability' as an analytical perspective for assessing how well features and feature-based rules defined at a source time point continue to hold when reused at a target time point. Using quarterly data drawn from Japanese-language tweets in Twitter's 1% sample stream from 2020-Q1 to 2022-Q3, covering roughly 10.1 to 11.0 million unique users per quarter, they evaluated 13 numeric user features across feature distributions, same-user relative ranks, selection rates, and selected-user membership. They found that feature distributions changed across quarters and that same-user relative ranks were less well preserved at longer quarter lags. Reusing source-quarter thresholds produced selection-rate drift, and while target-quarter recalibration nearly matched source-quarter selection rates, membership turnover among selected users persisted and increased at longer lags. The authors conclude that temporal portability should be assessed based on the specific property an analysis needs to preserve.