Discrete Prediction Games with Arbitrary Feedback and Loss

Abstract

We investigate the problem of predicting a sequence when the information about the previous elements (feedback) is onlypartial and possibly dependent on the predicted values. This setting can be seen as a generalization of the classical multi-armed bandit problem and accommodates as a special case a natural bandwidth allocation problem. According to the approach adopted by many authors, we give up any statistical assumption on the sequence to be predicted. We evaluate the performance against the best constant predictor (regret), as it is common in iterated game analysis. We show that for anydiscrete loss function and feedback function only one of two situations can occur: either there is a prediction strategy that achieves in T rounds a regret of at most O(T ^3/4(ln T )^1/2) or there is a sequence which cannot be predicted by any algorithm without incurring a regret of Ω(T).. We prove both sides constructively, that is when the loss and feedback functions satisfya certain condition, we present an algorithm that generates predictions with the claimed performance; otherwise we show a sequence that no algorithm can predict without incurring a linear regret with probability at least 1/2.