Signal HELM is cited as an open academic framework for evaluating model quality, safety, and bias

Summary

KPMG points to HELM, the Holistic Evaluation of Language Models, as an example of new tools that help organizations assess and mitigate AI risk. HELM is an open academic framework that measures the quality, safety, and bias of large language models. AI solutions architect Noelle Russell encourages organizations to adopt model evaluation frameworks of this kind. She points out that companies such as Microsoft, Amazon, and NVIDIA have built leaderboards that visualize these evaluations into their developer tools. Russell argues that paying attention to what you build means applying rigor and discipline to every model you select.