5
$\begingroup$

I am studying the blog: Understanding Convolutional Neural Networks for NLP. It is very good blog.

One thing I can't understand clearly about this blog. As the figure Illustration of a Convolutional Neural Network (CNN) architecture for sentence classification as following:

enter image description here

I want to ask:

  1. I know the region sizes(2,3,4) is like 2-gram, 3-gram, 4-gram word, but what’s the meaning of number filters? Here is 2 filters for each region. Why in the author's code about sentence classification is the number of filters defined to 128? Could you give examples to explain the meaning of the number of filters? for example using the sentence of ‘I like this movie very much’ would be great.

2) I understand the height of region size (4) is 4, but in the figure, the height of region(2, 3) are 5 and 6 respectively, I don't know why? I think the height of region is 2 and 3.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

  1. Answering this in terms of NLP examples is quite hard, remember "All models are wrong, some models are useful." First think of this in an image classification problem context, you want to use a large number of filters to collect a large number of features out of the image, one could detect edges, the other could detect densely coloured areas, one might turn a region to b&w. Extend a similar logic to text, by using a lot of filters, in this case 128, you are trying to capture a lot of features. For an example like , " I like movies very much", a certain filter might detect that like is a positive word and not a similarity comparison, a certain filter of size 2 might detect very much and detect that it is an expression of degeree. You can go on like that, it will be hard to come up with 128 features but the idea is to get enough features. If you think the number is unreasonable and might lead to overfitting, you can reduce the number and compare your results.

  2. No, 1- maxpool means that you take the maximum value of the output vector after applying a filter to the input. So it has nothing to do with the longest word but rather choose an element from the output that express the extracted feature to the highest amount.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for your comments @Himanshu Rai! Your saying ''All models are wrong, some models are useful" is so impressive, although I don't know how to get this! I want to ask the width of all filters are the same as matrix width? Now in this case, all the width is 5. Based on your explanation, my understanding is that a filter(for example 2 filters-2-ngram). Take the sentence of ‘I like this movie very much as example, the filter-2 is like: I like, like this, this movie, movie very, very much', and all the width of fitter-2 is 5 . I don't know my understanding is right? $\endgroup$
    – tktktk0711
    Oct 23, 2017 at 5:35
  • $\begingroup$ I dont really understand your queation here. But if you are concerned with the dimension then each sample is some_length x 1. You apply the filter as a moving window, so how you did for the 2 x 1 filter. Similarly for a 3 x 1 filter it will be i like this, like this movie, this movie very, movie very much. However all the sentences are padded to equal length, and then embeddings are used . If you are wondering wether all filters result in the same dimensional outputs, then no that doesnt happen. $\endgroup$
    – Himanshu Rai
    Oct 23, 2017 at 5:52
  • $\begingroup$ @HimanshuRai adding to point 1: the first filter, say for the trigram will assign different weights to different words in the trigrams. Like one filter might give high weight to the 0th index and less to the 2nd. One of the other 128 filters will give a different weight. Over time, they will learn the right weights to predict correctly. $\endgroup$
    – aneesh joshi
    Nov 4, 2017 at 5:19
  • $\begingroup$ Exactly! That is how you will extract a decent feature out of the window. $\endgroup$
    – Himanshu Rai
    Nov 4, 2017 at 5:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.