Selective Preference Aggregation

Abstract

Many tasks in machine learning depend on preferences where we aggregate preference data -- from recommending products to improving the helpfulness of responses from a large language model. In such tasks, individuals express their preferences over a set of items as votes, ratings, or rankings. Given a dataset of ordinal preferences from a group of individuals, we aggregate them into a single ranking that summarizes the collective preferences as a group. When individuals express conflicting preferences between items, standard methods are designed to arbitrate this dissent to rank one item over another. In this work, we introduce a paradigm for selective aggregation in which we abstain rather than arbitrate dissent. Given a dataset of ordinal preferences from a group of users, we aggregate their preferences into a selective ranking -- i.e., a partial order over items where every comparison is aligned with at least $1-\dissent{}$\% of users. We develop an algorithm to construct selective rankings that achieve all possible trade-offs between comparability and disagreement.