2
$\begingroup$

Let's say we have a time series $\{{\bf x}_i\}$ of features and are trying to learn to predict a time series $\{t_i\}$ using a neural network. Our goal is to be able to predict the time series value $t$ for both tomorrow and the day after tomorrow given the features ${\bf x}$ observed today.

My initial thoughts are:

  1. If our goal is to be able to predict $t$ tomorrow given features ${\bf x}$ observed today, we would make our (training, labels) dataset $({\bf x}_i, t_{i+1})$.

  2. If our goal is to be able to predict $t$ the day after tomorrow given features ${\bf x}$ observed today, we would make our (training, labels) dataset $({\bf x}_i, t_{i+2})$.

We could of course use two separate NNs to do this with a single output node but I think it would be better to use one with 2 output nodes and train on pairs $(t_{i+1}, t_{i+2})$.

Anyhow, ideally, the model should wind up predicting the same value for $t_{i+2}$ given ${\bf x}_{i+1}$ (when day $i+2$ is tomorrow) as it does when given ${\bf x}_{i}$ (when the day $i+2$ is the day after tomorrow). This is different from trying to predict two independent values with no time translation property. Generally when data has additional structure, it can be exploited in some way.

Can this desired equivalence be built into the training or architecture of the model?

$\endgroup$
0
$\begingroup$

What you are describing is commonly called incremental learning. Bayesian methods work well with incremental learning. Each posterior estimate would be updated based on the information currently available. In this case, it would the previous model estimates and new features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.