transform a supervised neural network to reinforcement learning?

Question

I have a functional LSTM model that works with an acceptable performance. How can I now convert this supervised model to a reinforcement learning model for improving the performane? Is there any example about how to transform a supervised model to a reinforcement learning model?

Details: I have a multi-input multi-output system (since I can not share the actual problem lets assume weather forcast as example) where I need to predict the output (e.g temperature, wind speed, etc) in real time. I have a large dataset and I trianed a supervised learning model that can do the predictions pretty well in real time.

The problem is that sometimes there is big deviation between predictions and actual values. This means that probably there was a new trend in the inputs that has never been in the data sets. For such cases, I would like to gradually enhance my model predictions.

Does it mean that I have to train again my model with the old dataset and the new data or I can simply have a RL mechansim to gradually improve the model?

Answer 1

Interesting question!

I think re-using your model in a DQN setting could be interesting. Even more so if you retrain your supervised model every now and then to update the DQN model (although in that case you'd have to figure out how to re-use what your model learned from DQN vs what it learned supervised).

I think to get you started you would have to define your context that you are using RL on. Basic elements that are needed for RL are:

• an Agent (whoever is making a decision on what to do)
• a State (how can you describe a snapshot of your current environment that has been influenced by an agent's action)
• an Action (so something your Agent can actively "do" to induce another state in its environment)
• a Reward (something you can "give" your agent to learn if he has chosen something good or bad)

Assuming you want to use something like DQN you have to define those things to be able to run the algorithm. Your current supervised model would then be the starting status of your NN that is used to choose your Agent's decisions.

The keyword for what you are looking for is "Transferlearning", which describes how to use trained models for other cases. In your case even other learning methods.