3
$\begingroup$

I have limited experience with machine learning, I trained a few networks, but nothing out of the ordinary. I have the following problem but I am not quite sure how to approach it and I'm hoping to get some advice here.

I have an series of time dependent data (timestamp + value) which I want to use to predict the next couple of outcomes. Usually this problem could be solved by using an LSTM. Every few minutes I get a new datapoint which I want to store and also take into consideration for future predictions.

The problem I think I have is that I'd need to train the model every few minutes with changing input size (I want to train the model with the entire history at once, I don't know if that makes sense) and also I don't want to overfit the model over time.

I'm not sure which architecture or concept is suited best for such a problem, so I hope to find some advice here. Thanks!

$\endgroup$
3
$\begingroup$

This is a form of continual learning / life-long learning, which tends to be very challenging because retraining on new data make traditional models "forget" what they had previously learned. Depending on the frequency by which you receive new data and the time it takes to train the model, it is often infeasible to retrain on all data every time you receive a new sample.

w.r.t. LSTMs, you don't need to change your input size because you can partition your time series data according to a preset and constant input size. For example, if you have 100 ordered samples, you could have the LSTM receive input of 10 samples and predict/regress the following 11th sample. So you'd have 90 partitioned samples of constant size. If you had another ordered sample (101), then you'd have 91 partitioned samples of size 10.

Continual learning is an active research field, so you can find many recent works such as this one from a couple months ago on adversarial CL.

For a review of CL in robotics, you can see this paper.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.