0
$\begingroup$

I am trying to train a text classification model. For all sentence examples, I limit them up to 32 words, and if there are not exist 32 words, I am creating zero pad arrays. To convert each word to vector, I used a pre-trained word2vec model.

In the final setting, the shape of my data is :

x_train: 15000 samples and each sample has 32 vectors in which each vector size is 100. (15000, 32, 100).

y_train: 15000 binary targets (15000, 1)

So my question is that, should I apply 1D CNN on my x_train or 2D CNN? I think that I can do it in both ways, but is there the main correct approach to this kind of problem? I read some stuff about 1D CNN on text classification but also there are some examples with 2D CNN. What are the cons and pros?

$\endgroup$
2
$\begingroup$

The standard way would be to apply 1D convolution.

While technically there is nothing preventing you to implement a 2D convolution over your textual representations, they would be "less expressive" than normal 1D convolutions:

  • In a normal 1D convolution, the kernel would have depth 100 and the width you choose (e.g. 3, 5). The kernel is slid through the "time" dimension and is computed over the whole channel dimension:

enter image description here

  • In order to compute a 2D convolution over your text, the 2 dimensions would be time and channels. You could have the kernel height be 100 and therefore take the whole channel dimension, but this would be analogous to a 1D convolution, no there is no point in doing so. Therefore, for having a 2D convolution, you would have a kernel that does not cover the whole channel dimension at a time:

enter image description here

Therefore, while it is possible to have a 2D convolution over text, it would be less expressive that the 1D convolution kernel, which covers the whole channel dimension and can, therefore, express any computation a 2D convolution can

So the answer is that you should use a 1D convolution. This is what makes sense and what people do.

(Images were taken from here and here)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.