1
$\begingroup$

I am working on a information retrieval model where the user enters a query and the model has to retrieve 3 most relevant FAQ pairs.I am collecting implicit feedback in terms of page clicks etc.What I don't understand is how to define states if I want to implement this with reinforcement learning?If I define each query as a state, all states will be different .Can anyone shed some light on this please!!!

$\endgroup$
1
$\begingroup$

I am not convinced that framing a document retrieval success as reinforcement learning problem will be easy to get working.

However, your core question is answerable. This problem occurs in other problems that are too large to explore all possible states. For instance, in the game of Go, the AlphaZero system cannot possibly explore all game positions. Similarly, in Atari games, many of which have also had reinforcement learning applied successfully, it is very unlikely that the agent has seen all possible states of the system - typically it will have trained on a million video frames.

The answer to this issue is to use some form of function approximation that can generalise to new unseen data. That is precisely what supervised machine learning models do, so the RL agent will use one internally, typically a linear regression or neural network, so that values and or policy learned from states it has explored will be associated with similar unseen states (for some interpretation of similar).

| improve this answer | |
$\endgroup$
  • $\begingroup$ I agree with Neil that you should better adopt a supervised learning approach rather than a RL and that you need a function approximator that will generalize in unseen states. However, if you encounter once a state and you never get a chance to experience again the same or very similar state your method will generalize poorly. Think of of it as creating a classifier with 100/10,000 data instances as earthquake events. You will get a 90% accuracy classifier but won't be able to predict the rare event. $\endgroup$ – Constantinos Oct 25 '18 at 4:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.