I would like to model a problem as a multi-armed bandit problem. In the data we have contextual information (user demographics, preferences, etc.) but this contextual information of each user is not varying with time or round (as we are not gathering any new data). My understanding is that now as our data is static so our “context” (state) is constant in every round. Is this assumption correct? I want to know if the contextual information is constant will it be considered as a multi-armed bandit(context-free) or contextual bandit? Thanks in advance.

  • 1
    $\begingroup$ Yes, constant state is still Contextual Bandits problem. $\endgroup$
    – koch_kir
    Oct 19 at 8:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.