7
$\begingroup$

Can Reinforcement learning be applied for time series forecasting?

$\endgroup$
6
$\begingroup$

Yes, but in general it is not a good tool for the task, unless there is significant feedback between predictions and ongoing behaviour of the system.

To construct a reinforcement learning (RL) problem where it is worth using an RL prediction or control algorithm, then you need to identify some components:

  • An environment that be in one of many states that can be measured/observed in a sequence.

  • An agent that can observe current state and take actions in the same sequence.

  • The evolution of state in the sequence should depend on some combination of the current state and the action taken, and may also be stochastic.

  • There should be a reward signal that the RL agent can observe or measure. The value of reward should depend on the same factors as the evolution of the state, but can depend on them in a different way.

The general case of time series forecasting can be made to fit with this by treating the prediction as the action, having the state evolution depend on only the current state (plus randomness) and the reward based on state and action. This will allow RL to be applied, but causality only flows one way - from the environment into your predictive model. As such, the best you can do for rewards for instance is to use some metric about the correctness of the predictions. Consequences for good or bad predictions do not affect the original environment. Essentially you will end up wrapping some predictive model for the sequence (such as a neural network) in a RL layer which could easily be replaced by basic data set handling for a supervised learning problem.

One way you could meaningfully extend series forecasting problems into RL problems is to increase the scope of the environment to include the decisions made based on the predictions, and the state of the systems that are affected by those decisions. For instance, if you are predicting stock prices, then include your portfolio and funds in the state. Likewise the actions stop being the predictions, becoming buy and sell commands. This will not improve the price prediction component (and you are likely better off treating that as a separate problem, using more appropriate tools - e.g. LSTM), but it will frame the issue overall as a RL problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.