LLM Sample: Part Average and Part Ideal

Abstract

As Large Language Models (LLMs) increasingly impact society, it's crucial to understand the heuristics and biases that drive them. We study the response sampling of LLMs in light of value bias—a tendency to favour high-value options in their outputs. Value bias corresponds to the shift of response from the most likely sample towards some notion of ideal value represented in the LLM. Our study identifies value bias in existing and new concepts learned in context. We demonstrate that this bias significantly impacts applications such as patient recovery times. These findings highlight the need to address value bias in LLM deployment to ensure fair and balanced AI applications.