Signal Manipulation-Proof Oblivious Audits against Deceptive Model Providers

Summary

Augustin Godinot, Sofiane Azogagh, Julien Ferry, and Sébastien Gambs published a paper on arXiv (cs.CY) on August 5, 2026, addressing a fundamental vulnerability in algorithmic governance audits: because audits are typically declared or easily detected, model providers can manipulate the process intentionally or inadvertently, a vulnerability particularly acute in fairness evaluations where providers can infer sensitive attributes and strategically equalize allocation rates between groups to satisfy fairness metrics. The authors introduce a novel audit protocol designed to significantly increase post-audit detectability of such manipulations by enabling auditors to query models obliviously. The approach leverages a Private Information Retrieval mechanism requiring the provider to label a large set of instances while preventing it from knowing which subset will ultimately be used for the audit. The protocol is efficient, imposes minimal overhead on the auditor, and requires no modification to the audited model, its training procedure, or inference pipeline. The authors provide theoretical guarantees showing a provider attempting to hide unfairness must falsify a significantly larger number of responses under this protocol, increasing both the difficulty and likelihood of detecting manipulation, with experimental results across representative audit scenarios confirming the approach's effectiveness and practicality.