How to do give input to CNN when doing a text processing?

Question

As a signal processing engineering and being new to NLP, I am confused with giving input to CNN network.

With my knowledge of CNN, I am trying to build a classifier for ethnicity with inputs as text (last name(LN), middle name(MN), first name(FN)). I have a list of 8,000,000 samples with last, middle, first names and class information

array = [['person1_LN','person1_MN','person1_FN','Person1_class'],
         ['person1_LN','person1_MN','person1_FN','Person2_class'], 
         ....]

I want to apply conv layer (CL) followed by Pooling Layer(PL) on LN,MN,FN respectively.

Text processing example demonstrates with the sentences to convert to word embedding. I am trying to understand this code snippet

W = tf.Variable(tf.random_uniform([vocab_size, embedding_size], -1.0, 1.0), name='W')
self.embedded_chars = tf.nn.embedding_lookup(W, self.input_x)
self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars, -1)

This tutorial says training to be done using word2vec. Reading these two blogs I could understand nothing. In Either of the cases, my data is not a sentence, moreover in the second case on what things I have to train?

Will the CL operate if I directly give the input of words without word embeddings? If not any example on how to embed the words in my case to give the input to CNN?

Answer 1

So, here the question is asking how you use text in a CNN architecture.

Just like when we process images with a CNN, the text data will also be a 2D matrix, where the rows will represent the text features and the columns the sequence of characters (which make up a name) for example.

Now, of course, if we simply use one-hot encoding, we get data sparsity, which is not not particularly computational efficient, especially when using CNNs. So, preferably, we would use word embeddings, which you have described in your post. Here, word embeddings aim to collapse the high dimensionality of the input so that reduces the chance of data sparsity.

However, there is a potential problem with using word embeddings in your problem. The principal behind word embeddings is that it will assign similar word embeddings to words which can be easily replaced in similar contexts (e.g. Sam is kind & Simon is kind) and that don't affect the coherence of a sentence. As you can see, when it comes to proper nouns, we might end up with similar vectors for all names since they are very interchangeable in any context and do not affect the overall coherence of the sentence.

Therefore, although this is contradictory, I would advise using one-hot encoded sequence of characters as a start point for this problem and then when you implement this, you might come up with a more effective way to represent the names.