Listwise Approach to Learning to Rank: Theory and Algorithm

Abstract

This paper aims to conduct a comprehensive study on the listwise approach to learning to rank. The listwise approach learns a ranking function by taking individual lists as instances and minimizing a loss function defined on two lists (one is predicted result and the other ground truth). Existing work on the approach mainly focused on the development of new algorithms; methods such as RankCosine and ListNet have been proposed and better performances by them have also been observed. Unfortunately, the underlying theory was not sufficiently studied as far. To amend the problem, this paper proposes conducting theoretical analysis of learning to rank algorithms through investigation on the properties of the loss functions, including consistency, soundness, continuity, differentiability, convexity, and efficiency. A sufficient condition on consistency for ranking is given, which seems to be the first such result obtained in related research. The paper then conducts analysis on three loss functions: likelihood loss, cosine loss, and cross entropy loss. The latter two were used in RankCosine and ListNet respectively. The use of likelihood loss leads to the development of a new listwise method called ListMLE, whose loss function offers better properties. Experimental results have also verified the correctness of the theoretical results obtained in the paper.