I recently read a lot about the n-armed bandit problem and its solution with various algorithms, for example for webscale content optimization. Some discussions were referring to 'contextual bandits', I couldn't find a clear definition what the word 'contextual' should mean here. Does anyone know what is meant by that, in contrast to 'usual' bandits?
A contextual bandit algorithm not only adapts to the user-click feedback as the algorithm progresses, it also utilizes pre-existing information about the user's (and similar users) browsing patterns to select which content to display.
So, rather than starting with no prediction (cold start) with what the user will click (traditional bandit and also traditional A/B testing), it takes other data into account (warm start) to help predict which content to display during the bandit test.