I am looking for a paper detailing the very basics of deep learning. Ideally like the Andrew Ng course for deep learning. Do you know where I can find this ?

  • 5
    $\begingroup$ -1: Where have you looked already? Found anything? $\endgroup$
    – Spacedman
    Dec 9 '14 at 13:59
  • 4
    $\begingroup$ Why is this upvoted. It doesnt show any effort whatsoever and its a duplicate of a dupe $\endgroup$
    – runDOSrun
    Jun 9 '15 at 14:56
  • $\begingroup$ Link to duplicate is a 404 error page. $\endgroup$
    – Danijel
    Mar 19 '19 at 13:32

This link contains an amazing amount of deep learning literature. Summarizing it here(going in the order a beginner ideally should)- NOTE: All these resources mainly use python.

1) First of all, a basic knowledge of machine learning is required. I found Caltech's Learning from data to be ideal of all the machine learning courses available on the net.

Andrew Ng's Coursera course is pretty good too.

2) For Neural networks, nobody explains it better than Dr.Patrick Winston. The assignments should be tried out for better understanding. They are in python.

3) For a better understanding of Neural Networks, Michael Nielsen's course should be done(as suggested by Alexey). It is pretty basic but it works.

4) For deep neural networks, and implementing them faster on GPUs, there are multiple frameworks available, such as Theano, Caffe, Pybrain, Torch,etc. Out of these Theano provides a better low level functionality that allows its user to create custom NNs. It is a python library, so being able to use numpy,scikit-learn, matplotlib, scipy along with it is a big plus. The deep learning tutorial written by Lisa Lab should be tried out for a better understanding of theano.

5) For Convolutional Neural Networks, follow andrej karpathy's tutorial.

6) For unsupervised learning, follow here and here.

7) For an intersection of deep learning and NLP, follow Richard Socher's class.

8) For LSTMs, read Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780 and Graves, Alex. Supervised sequence labelling with recurrent neural networks. Vol. 385. Springer, 2012.

Here is LSTM's Theano code.


The subject is new so most of the wisdom is scattered in papers, but here are two recent books:

And some practical material.


Neural Networks and Deep Learning by Michael Nielsen. The book is still in progress, but it looks quite interesting and promising. And it's free! Here's the link.

There are only 5 chapters so far, and the most of them talk about usual neural networks, but it's still worth having a look.

Update: the book has been finished!


Main references:

Courses on deep learning:



Toolkit-specific tutorials:


There's also Richard Socher's recent PhD dissertation on intersection of NLP and deep learning: Recursive Deep Learning for Natural Language Processing and Computer Vision


For comprehending the derivation of Back propagation algorithm, I suggest Ryan Harris youtube video which is less daunting. You may find second video as well.


Not the answer you're looking for? Browse other questions tagged or ask your own question.