Signal Researchers propose detecting CSAM-generating AI models directly from their weights

Summary

A paper addresses the growing ease of using low-rank adaptation (LoRA) fine-tuning to customize open-weight image-generation models, including for producing child sexual abuse material (CSAM), noting that existing moderation approaches rely on metadata, which can be falsified, or on the generated outputs themselves, which may be illegal or unacceptable to produce for screening purposes. The authors propose a safer signal derived directly from a LoRA's weights: the top-left singular vectors of a LoRA's parameter updates, which they call u1, form a compact, inference-free fingerprint of the model's strongest learned change. Using human-subject age as a benign proxy for CSAM-related training, the researchers show that u1 can identify what a given LoRA was trained on, generalizes across different base models, and abstains on unrelated benign content. They further report that the signal remains robust under additive weight noise, rescaling, and reduced numerical precision. The findings suggest harmful LoRAs could be screened directly from their weights without relying on metadata or generating potentially harmful outputs.