I've been thinking about the Recurrent Neural Networks (RNN) and their varieties and Convolutional Neural Networks (CNN) and their varieties.

Would these two points be fair to say:

  • Use CNNs to break a component (such as an image) into subcomponents (such as an object in an image, such as the outline of the object in the image, etc.)
  • Use RNNs to create combinations of subcomponents (image captioning, text generation, language translation, etc.)

I would appreciate if anyone wants to point out any inaccuracies in these statements. My goal here is to get a more clearer foundation on the uses of CNNs and RNNs.


A CNN will learn to recognize patterns across space. So, as you say, a CNN will learn to recognize components of an image (e.g., lines, curves, etc.) and then learn to combine these components to recognize larger structures (e.g., faces, objects, etc.).

You could say, in a very general way, that a RNN will similarly learn to recognize patterns across time. So a RNN that is trained to translate text might learn that "dog" should be translated differently if preceded by the word "hot".

The mechanism by which the two kinds of NNs represent these patterns is different, however. In the case of a CNN, you are looking for the same patterns on all the different subfields of the image. In the case of a RNN you are (in the simplest case) feeding the hidden layers from the previous step as an additional input into the next step. While the RNN builds up memory in this process, it is not looking for the same patterns over different slices of time in the same way that a CNN is looking for the same patterns over different regions of space.

I should also note that when I say "time" and "space" here, it shouldn't be taken too literally. You could run a RNN on a single image for image captioning, for instance, and the meaning of "time" would simply be the order in which different parts of the image are processed. So objects initially processed will inform the captioning of later objects processed.

| improve this answer | |
  • 3
    $\begingroup$ You can get good intuition for differences of RNN model from karpathy.github.io/assets/rnn/diags.jpeg - a much copied graphic. CNNs are along with MLPs and other non-recursive models as implementing the one-to-one model case only. $\endgroup$ – Neil Slater May 6 '16 at 17:44
  • $\begingroup$ @NeilSlater I do even know the original article of this image, but never could extract anything useful from it. Please, could you elaborate what you learned from the image? $\endgroup$ – Hi-Angel Feb 11 '17 at 17:29
  • 2
    $\begingroup$ @Hi-Angel: The image visualises possible relationships between sequences and single entities that can be mapped by a model. If you already understand the permutations well, then you might not get anything from it. The reason the image appears in the article is that it demonstrates the relative flexibility of RNNs: An RNN can be applied to all the different types of problems shown (e.g. it can be used in language translation problems which match the 4th item), whilst a feed-forward network only applies to problems matching the first image. $\endgroup$ – Neil Slater Feb 11 '17 at 17:52
  • $\begingroup$ karpathy.github.io/2015/05/21/rnn-effectiveness $\endgroup$ – Bondolin Oct 9 '17 at 15:17

Difference between CNN and RNN are as follows :


  1. CNN take a fixed size input and generate fixed-size outputs.

  2. CNN is a type of feed-forward artificial neural network - are variations of multilayer perceptrons which are designed to use minimal amounts of preprocessing.

  3. CNNs use connectivity pattern between its neurons is inspired by the organization of the animal visual cortex, whose individual neurons are arranged in such a way that they respond to overlapping regions tiling the visual field.

  4. CNNs are ideal for images and videos processing.


  1. RNN can handle arbitrary input/output lengths.

  2. RNN, unlike feedforward neural networks, can use their internal memory to process arbitrary sequences of inputs.

  3. Recurrent neural networks use time-series information (i.e. what I spoke last will impact what I will speak next.)

  4. RNNs are ideal for text and speech analysis.

| improve this answer | |

I think that the two statements are not correct since the CNN does not break the component into subcomponents but rather use shared weights on all the overlapping subcomponents (recpetive fields) to find the same pattern. It is not a divide and conquer algorithm.

In general case CNN's tend to extract local and position-invariant (independently of their position) features, and RNN's tend to find diffenret patterns across the time even if they are far.

For example in the case of applying both to natural language, CNN's are good at extracting local and position-invariant features but it does not capture long range semantic dependencies. It just consider local key-phrases.

So when the result is determined by the entire sentence or a long-range semantic dependency CNN is not effective as shown in this paper where the authors compared both architechrures on NLP taks.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.