2
$\begingroup$

I have a functional LSTM model that works with an acceptable performance. How can I now convert this supervised model to a reinforcement learning model for improving the performane? Is there any example about how to transform a supervised model to a reinforcement learning model?

Details: I have a multi-input multi-output system (since I can not share the actual problem lets assume weather forcast as example) where I need to predict the output (e.g temperature, wind speed, etc) in real time. I have a large dataset and I trianed a supervised learning model that can do the predictions pretty well in real time.

The problem is that sometimes there is big deviation between predictions and actual values. This means that probably there was a new trend in the inputs that has never been in the data sets. For such cases, I would like to gradually enhance my model predictions.

Does it mean that I have to train again my model with the old dataset and the new data or I can simply have a RL mechansim to gradually improve the model?

$\endgroup$
  • $\begingroup$ If your model is a classifier or regression over a fixed data set, then converting it to use RL as a training process is unlikely to make any improvements. RL is a very general learning process (so is "better" if you want to consider more generic approaches to learning), but is far less efficient than supervised learning in terms of CPU and convergence (so it is "worse" if you care about accuracy on a dataset). Could you give more details about your model, and why you think RL might help for your problem, so someone answering can help you decide whether it is even worth the effort? $\endgroup$ – Neil Slater Oct 21 '19 at 20:22
  • 1
    $\begingroup$ @NeilSlater: Thank you for the help. I updated my question with more details. $\endgroup$ – Ehsan Oct 22 '19 at 13:23
1
$\begingroup$

Interesting question!

I think re-using your model in a DQN setting could be interesting. Even more so if you retrain your supervised model every now and then to update the DQN model (although in that case you'd have to figure out how to re-use what your model learned from DQN vs what it learned supervised).

I think to get you started you would have to define your context that you are using RL on. Basic elements that are needed for RL are:

  • an Agent (whoever is making a decision on what to do)
  • a State (how can you describe a snapshot of your current environment that has been influenced by an agent's action)
  • an Action (so something your Agent can actively "do" to induce another state in its environment)
  • a Reward (something you can "give" your agent to learn if he has chosen something good or bad)

Assuming you want to use something like DQN you have to define those things to be able to run the algorithm. Your current supervised model would then be the starting status of your NN that is used to choose your Agent's decisions.

The keyword for what you are looking for is "Transferlearning", which describes how to use trained models for other cases. In your case even other learning methods.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.