Signal Modeling Correlated Errors in AI-Agent Committees Cuts Expected Decision Loss by 15.7%

Summary

A study models how AI "committees" — groups of AI agents voting together on a decision — behave when their errors are correlated rather than independent, challenging the assumption behind classical voting guarantees. Using 174,384 votes cast by 28 language models across four binary-screening benchmarks, the researchers estimated a model where error dependence can differ between correct and incorrect cases. Incorporating this dependence structure improved the model's fit to held-out data, raising the R² from 0.840 under an independence assumption to 0.967 with the full dependence model. Applying cost-sensitive threshold selection under the independence assumption alone reduced scaled decision loss from 60.25 to 52.50 relative to simple majority voting; modeling dependence reduced it further to 50.77. Overall, this represents a 15.73% reduction in expected loss compared to majority voting, with a 95% bootstrap confidence interval of 13.41% to 16.75%. The findings suggest committees of AI agents used for automated screening or evaluation need threshold designs that account for correlated, not just independent, error patterns.