UQLM: A Python Package for Uncertainty Quantification in Large Language Models

Dylan Bouchard; Mohit Singh Chauhan; David Skarbrevik; Ho-Kyeong Ra; Viren Bajaj; Zeya Ahmad

Hallucinations, defined as instances where Large Language Models (LLMs) generate false or misleading content, pose a significant challenge that impacts the safety and trust of downstream applications. We introduce UQLM, a Python package for LLM hallucination detection using state-of-the-art uncertainty quantification (UQ) techniques. This toolkit offers a suite of UQ-based scorers that compute response-level confidence scores ranging from 0 to 1. This library provides an off-the-shelf solution for UQ-based hallucination detection that can be easily integrated to enhance the reliability of LLM outputs.

UQLM: A Python Package for Uncertainty Quantification in Large Language Models

Abstract