2
$\begingroup$

I got it that when using time series data, I have to use a RNN

plot

The highlight here is that the neurons receive their output again as an input, so they can take into account the previous step.

But what I do not understand is: Why cannot regular NNs do the same? For example, when I provide an order like

0, 1, 2, 3, 4, 5, 6, 7, ..

the NN should register that there is a pattern in it. If these numbers represent days or seconds, it is not important. So why are self-looped neurons now necessary to do so?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

To be clear, RNN are not necessary as there exist other options. However, RNN are useful since they implicitly deal with temporal relationships between inputs.

Regular neural networks are idempotent, meaning that the same input can be passed multiple times and the same output will be computed (obviously if the network is not updated in the meantime).

This idempotence means that regular neural nets cannot implicitly "see" temporal patterns. To do so, they would need to be provided with temporal information as you suggest. This is actually what transformers achieve! By providing temporal embeddings on top of the input, it always a regular neural net to learn from temporal information.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thank you! Had only a brief look at transformers but will dive into it later. I have to admit that I didn't get the concept in general why regular NNs cannot handle time series. They are idempotent, ok, so a monday will always provide the same output? Or just the single monday? Despite from that, as I already suggested, when I provide a time series just as an increasing order (time has to be converted anyway), how else does the NN treat this information if not as something like a (time) series? $\endgroup$
    – Ben
    Dec 11, 2019 at 11:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.