Deep Learning is an excellent model for classification problem such as image recognition or object detection. Can we use deep learning for regression problems - Time Series prediction ? So if it can, how can we build structure of deep learning. I mean how to building layer to extract features from time series.


Recurrent neural networks (RNNs) can work with series as input or output or both.

Even a simple one-layer RNN is effectively "deep" because it has to solve similar problems as multi-layer non-recursive networks. That is because backpropagation logic in a RNN has to account for delay between input and target, which is solved by backpropagation through time - essentially adding a layer to the network for every time step of delay between first input and last output.

RNN architecture has become more sophisticated in recent years by using "gating" techniques such as Gated Recurrent Units (GRU) or Long Short Term Memory (LSTM). These have multiple trainable params - 3 or 4 - per neuron, and the schematics are more complicated than feed-forward networks. They have been demonstrated as very effective in practice, so this extra complexity does seem to pay off.

Although you can research and implement RNNs yourself in a library like Theano or Tensor Flow, several neural network libraries already implement RNN architectures (e.g. Keras, torch-rnn)

  • $\begingroup$ You have any experiments about tuning RNN parameters? In particular, I still have stucked in choosing optimal parameter. $\endgroup$ Jun 15 '16 at 14:31
  • $\begingroup$ @TrầnĐứcNhuận: No, sorry. I am still learning about RNNs and don't have much experience yet, just what I have seen in blogs, video lectures. $\endgroup$ Jun 15 '16 at 15:16
  • $\begingroup$ @TrầnĐứcNhuận You can use random search to find an almost optimal set of hyperparameters for your model. $\endgroup$
    – ahajib
    Jul 12 '16 at 19:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.