3
$\begingroup$

I want to predict the evolution of a 2D sequence (third dimension being time) and use the predicted sequence (along with several previously predicted sequences and some scalar values changing with time) to predict the next 2D sequence, and ultimately use that prediction to predict the following sequence in time.

Example of a 2D time-slice of the data: Data to be used

I want to train my neural network to predict a new 2D "profile" (a time-slice of a 3D profile) based on the last ~3 profiles and a time-varying scalar parameter. What neural network architecture(s) would be capable of predicting the next 2D profile in time? I suspect RNNs, but which sub-models would work for this (e.g. seq2seq, NMT, etc.)?

Help would be greatly appreciated!

$\endgroup$
0
$\begingroup$

Ahoy, all! Figured out the answer.

Firstly, this is a time-series forecasting problem. You're looking to forecast (predict the temporal-evolution) of these 2D profiles. In a classification problem (most popular application of neural networks), the target value vector can be a one-hot encoded vector, where the "1" value is the correct classification; the other values (classes) are "0". You give this to your network and it will "learn" patterns in the data that correspond to each particular class. It will do this by minimizing the cost function until the predicted class (the one closest to 1) matches the correct class, and the other predictions are very close to zero. The network is essentially getting closer and closer to the actual target values.

This EXACT idea can be applied to this forecasting problem, where you feed in a vector of target values, but this time the vector isn't one-hot encoded (or, as in other cases, a probability distribution of the classes). It is just the known (target) values at each point along the profile. Yes, RNNs are good, but MLPs can also do the job. Specifically, for RNNs, you can use LSTM units and cells. No need for seqtoseq or NMT architectures for this particular problem.

One important difference is that forecasting problems do not use softmax functions. Instead, they use linear activation functions such as ReLU, and MSE/MAD for the loss.

Setting up this type of problem in your model (in the pre-processing step) may be slightly different than in classification problems. This tutorial here is very good overall, but also has invaluable pre-processing code that you can use in many forecasting applications (particularly the series_to_supervised() function.

Hope this helps!

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.