RNN vs CNN at a high level

Question

I've been thinking about the Recurrent Neural Networks (RNN) and their varieties and Convolutional Neural Networks (CNN) and their varieties.

Would these two points be fair to say:

• Use CNNs to break a component (such as an image) into subcomponents (such as an object in an image, such as the outline of the object in the image, etc.)
• Use RNNs to create combinations of subcomponents (image captioning, text generation, language translation, etc.)

I would appreciate if anyone wants to point out any inaccuracies in these statements. My goal here is to get a more clearer foundation on the uses of CNNs and RNNs.

Answer 1

A CNN will learn to recognize patterns across space. So, as you say, a CNN will learn to recognize components of an image (e.g., lines, curves, etc.) and then learn to combine these components to recognize larger structures (e.g., faces, objects, etc.).

You could say, in a very general way, that a RNN will similarly learn to recognize patterns across time. So a RNN that is trained to translate text might learn that "dog" should be translated differently if preceded by the word "hot".

The mechanism by which the two kinds of NNs represent these patterns is different, however. In the case of a CNN, you are looking for the same patterns on all the different subfields of the image. In the case of a RNN you are (in the simplest case) feeding the hidden layers from the previous step as an additional input into the next step. While the RNN builds up memory in this process, it is not looking for the same patterns over different slices of time in the same way that a CNN is looking for the same patterns over different regions of space.

I should also note that when I say "time" and "space" here, it shouldn't be taken too literally. You could run a RNN on a single image for image captioning, for instance, and the meaning of "time" would simply be the order in which different parts of the image are processed. So objects initially processed will inform the captioning of later objects processed.

Answer 2

Difference between CNN and RNN are as follows :

CNN:

1. CNN take a fixed size input and generate fixed-size outputs.

2. CNN is a type of feed-forward artificial neural network - are variations of multilayer perceptrons which are designed to use minimal amounts of preprocessing.

3. CNNs use connectivity pattern between its neurons is inspired by the organization of the animal visual cortex, whose individual neurons are arranged in such a way that they respond to overlapping regions tiling the visual field.

4. CNNs are ideal for images and videos processing.

RNN:

1. RNN can handle arbitrary input/output lengths.

2. RNN, unlike feedforward neural networks, can use their internal memory to process arbitrary sequences of inputs.

3. Recurrent neural networks use time-series information (i.e. what I spoke last will impact what I will speak next.)

4. RNNs are ideal for text and speech analysis.

Answer 3

From a general point of view, CNN does not break the component into subcomponents but rather use shared weights on all the overlapping subcomponents (recpetive fields) to find the same pattern. It is not a divide and conquer algorithm.

In general case, CNNs tend to extract local and position-invariant (independently of their position) features, and RNNs tend to find diffenret patterns across the time even if they are far.

For example in the case of applying both to natural language, CNNs are good at extracting local and position-invariant features but it does not capture long range semantic dependencies. It just consider local key-phrases.

So when the result is determined by the entire sentence or a long-range semantic dependency CNN is not effective as shown in this paper where the authors compared both architechrures on NLP taks.

Answer 4

If I want to tell you, both are based on a same concept, and that is weight sharing. It is better to think about them in this way.

In CNNs, we try to find similar patterns throughout the input which can be image, text, or other things. This can be done by sliding a same filter that its parameters do not change while scanning the image to find similar patterns. Due to the fact that an image can have multiple patterns, we employ this idea with multiple filters.

On the other hand, In RNNs, the idea is that we want to find a similar patterns throughout the sequence. For instance, wherever you see a cat in a sentence, it is still a cat. Consequently, it should not matter where you see it. RNNs also employ the idea of weight sharing. They use a network that faces each input in the sequence separately to see whether a similar pattern is observed or not. However, there is a difference between CNNs and RNNs. The input of RNNs is a sequence, and the order matters. Consequently, at each time step, the RNN encounters the input of that time step, and it also receives some information from the past.

You can see that these intuitions are different than yours.

Answer 5

Other than the fact that both are types of neural networks, there is not too much of a similarity between them. The two statements you have given are correct. However there is more to it.

At a very generic level - CNNs are used to convert images into features. These features can then be used by the model to do whatever it wants to do. RNNs are used to featurize time-series data - for e.g. data like stock markets, speech, language etc - all situations where the order matters.

The first letter of both terms represent the key difference. 'C' is for convolutions - use an input filter of choice to scan thru' out the image (its numeric representation) for identifying patterns similar to the filter. By using 1000's of such filters and scanning the entire image again and again, and repeating this entire process layer after layer, we get higher and higher level of abstractions from the raw pixel data...ultimately we reach a point where a simple layer can now use all these higher level features to do what we want to do (say identify the object in the image)

'R' is for recurring, this means each layer has multiple neural nets within (all share the same weight). The idea of having multiple neural nets is to account for the dependency of the data on the previous time-step and somehow build a better representation of the given data. This 'better representation' can now be used by a regular layer to do what we want (like classifying a twitter sentiment into +ve or -ve).

Based on situations sometimes we can leverage both (CNN and RNN) of them in one model (say describing an image in a sentence) or even surprisingly we can interchange them - for e.g. use CNNs where RNN's should have been used..