I'm trying to restore this function:

$$ F(x) = x*sin(\alpha x)+b; \space\space \alpha,b \in (-20,20) $$

My NN model(with Keras) is:

1 layer: GRU, 9 neurons, selu activation
2 layer: GRU, 3 neurons, selu activation
1 layer: GRU, 7 neurons, selu activation
1 layer: Dense, 1 neuron, linear activation
kernel init he_normal for all layers

For the training dataset I've generated function values $f(x), x \in -5..5$ (e.g. linspace(-5,5,500) and random integer values for $\alpha$ and $b$, 250 times.

Then I've selected $f[i-1]$ and $f[i]$ (previous steps) and for this "xs" the output is $f[i+1]$. So the training dataset is like:

first row: previous previous $f$, $\alpha$, $b$
second row: previous $f$, $\alpha$, $b$

Y: current $f$ value

After training over 200 epochs, mae on validation data was 0.1851.

Now if I try to predict new data it seems normal, but when I try to predict new values using previously predicted by model points it falls down and it doesn't looks like sine function at all.

What am I doing wrong?


1 Answer 1


The problem is that your error is accumulating and diverging. In other words, a small error in the first prediction is leading to a larger error in your second prediction, which is leading to an even larger error in your third prediction, and so on.

With that said, this LSTM example seems to do well predicting multiple future values for a sine wave.

  • $\begingroup$ I was thinking about that, but the question is how to avoid this effect, maybe I need more data?) $\endgroup$
    – iamlion12
    Dec 3, 2017 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.